I’ve been having some issues removing anchors. I can add anchors with no issue. They will be there the next time I run the scene. I can also get updates when ARKit sends them. I can remove anchors, but not all the time. The method I’m using is to call removeAnchor() on the data provider.
worldTracking.removeAnchor(forID: uuid)
// Yes, I have also tried `removeAnchor(_ worldAnchor: WorldAnchor)`
This works if there are more than one anchor in a scene. When I’m down to one remaining anchor, I can remove it. It seems to succeed (does not raise an error) but the next time I run the scene the removed anchor is back. This only happens when there is only one remaining anchor.
do {
// This always run, but it doesn't seem to "save" the removal when there is only one anchor left.
try await worldTracking.removeAnchor(forID: uuid)
} catch {
// I have never seen this block fire!
print("Failed to remove world anchor \(uuid) with error: \(error).")
}
I posted a video on my website if you want to see it happening.
https://stepinto.vision/labs/lab-051-issues-with-world-tracking/
Here is the full code. Can you see if I’m doing something wrong? Is this a bug?
struct Lab051: View {
@State var session = ARKitSession()
@State var worldTracking = WorldTrackingProvider()
@State var worldAnchorEntities: [UUID: Entity] = [:]
@State var placement = Entity()
@State var subject : ModelEntity = {
let subject = ModelEntity(
mesh: .generateSphere(radius: 0.06),
materials: [SimpleMaterial(color: .stepRed, isMetallic: false)])
subject.setPosition([0, 0, 0], relativeTo: nil)
let collision = CollisionComponent(shapes: [.generateSphere(radius: 0.06)])
let input = InputTargetComponent()
subject.components.set([collision, input])
return subject
}()
var body: some View {
RealityView { content in
guard let scene = try? await Entity(named: "WorldTracking", in: realityKitContentBundle) else { return }
content.add(scene)
if let placementEntity = scene.findEntity(named: "PlacementPreview") {
placement = placementEntity
}
} update: { content in
for (_, entity) in worldAnchorEntities {
if !content.entities.contains(entity) {
content.add(entity)
}
}
}
.modifier(DragGestureImproved())
.gesture(tapGesture)
.task {
try! await setupAndRunWorldTracking()
}
}
var tapGesture: some Gesture {
TapGesture()
.targetedToAnyEntity()
.onEnded { value in
if value.entity.name == "PlacementPreview" {
// If we tapped the placement preview cube, create an anchor
Task {
let anchor = WorldAnchor(originFromAnchorTransform: value.entity.transformMatrix(relativeTo: nil))
try await worldTracking.addAnchor(anchor)
}
} else {
Task {
// Get the UUID we stored on the entity
let uuid = UUID(uuidString: value.entity.name) ?? UUID()
do {
try await worldTracking.removeAnchor(forID: uuid)
} catch {
print("Failed to remove world anchor \(uuid) with error: \(error).")
}
}
}
}
}
func setupAndRunWorldTracking() async throws {
if WorldTrackingProvider.isSupported {
do {
try await session.run([worldTracking])
for await update in worldTracking.anchorUpdates {
switch update.event {
case .added:
let subjectClone = subject.clone(recursive: true)
subjectClone.isEnabled = true
subjectClone.name = update.anchor.id.uuidString
subjectClone.transform = Transform(matrix: update.anchor.originFromAnchorTransform)
worldAnchorEntities[update.anchor.id] = subjectClone
print("🟢 Anchor added \(update.anchor.id)")
case .updated:
guard let entity = worldAnchorEntities[update.anchor.id] else {
print("No entity found to update for anchor \(update.anchor.id)")
return
}
entity.transform = Transform(matrix: update.anchor.originFromAnchorTransform)
print("🔵 Anchor updated \(update.anchor.id)")
case .removed:
worldAnchorEntities[update.anchor.id]?.removeFromParent()
worldAnchorEntities.removeValue(forKey: update.anchor.id)
print("🔴 Anchor removed \(update.anchor.id)")
if let remainingAnchors = await worldTracking.allAnchors {
print("Remaining Anchors: \(remainingAnchors.count)")
}
}
}
} catch {
print("ARKit session error \(error)")
}
}
}
}
visionOS
RSS for tagDiscuss developing for spatial computing and Apple Vision Pro.
Posts under visionOS tag
200 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
Hello,
I'm encountering a runtime crash when building my visionOS app with Xcode 16.3 for visionOS 2.5. Our existing AppStore/Testflight app is also instantly crashing on visionOS 2.5 when opened but works fine on e.g visionOS 2.4.
The app builds successfully but crashes on launch with this symbol lookup error (slightly adjusted because the forum complained regarding sensitive data):
Symbol not found: _$sSo22CLLocationCoordinate2DVSE12CoreLocationMc
Referenced from: <XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX> /private/var/containers/Bundle/Application/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/MyApp.app/MyApp.debug.dylib
Expected in: <XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX> /usr/lib/swift/libswiftCoreLocation.dylib
dyld config: DYLD_LIBRARY_PATH=/usr/lib/system/introspection DYLD_INSERT_LIBRARIES=/usr/lib/libLogRedirect.dylib:/usr/lib/libBacktraceRecording.dylib:/usr/lib/libMainThreadChecker.dylib:/System/Library/PrivateFrameworks/GPUToolsCapture.framework/GPUToolsCapture:/usr/lib/libViewDebuggerSupport.dylib
I've already implemented my own Codable conformance for CLLocationCoordinate2D:
extension CLLocationCoordinate2D: Codable {
// implementation details...
}
This worked fine on previous visionOS/Xcode versions. Has anyone encountered this issue or found a solution?
System details:
macOS version: 15.3.2
Xcode version: 16.3
visionOS target: 2.5
Thank you!
Hi everyone,
I'm developing a visionOS app that allows users to download large video files (similar to a movie download experience, with each file being around 10 GB). I've successfully implemented the core video download functionality using URLSession, and everything works as expected while the app is active.
Now, I’m looking to support background downloading. Specifically, I want users to be able to start a download and then leave the app (e.g., switch apps or return to the home screen) while the download continues in the background.
Additionally, I’d like to confirm a specific scenario:
If the user starts a download, then removes the headset (keeping the device turned on and connected to power), will the download continue in the background? Or does visionOS suspend the app or downloads in this case?
I’m considering using a background URLSessionConfiguration (as done in iOS/macOS) to enable this behavior, but I’m not sure if it behaves the same way on visionOS or if there are special limitations or best practices when handling large downloads on this platform.
Any insights or official guidance would be greatly appreciated!
Thanks!
Environment
Xcode: 16.2
VisionOS SDK 2.4
Swift 6.1
Targets: Apple Vision Pro (immersive space)
Frameworks: ARKit, RealityKit, SwiftUI
What I’m Trying to Do
I have a view-model class PlacementManager that holds two AR providers:
private var worldTracking: WorldTrackingProvider
private var planeDetection: PlaneDetectionProvider
I want to dynamically replace these providers in a setEnvironment(_:) method (so I can save/clear a JSON scene and restart ARKit).
What’s Happening
If I declare them as :
private let worldTracking = WorldTrackingProvider()
private let planeDetection = PlaneDetectionProvider()
I get compile-errors when I later do:
self.worldTracking = newWorldTracking // Cannot assign to property: 'worldTracking' is a 'let' constant
If I change them to un-initialized vars:
private var worldTracking: WorldTrackingProvider
private var planeDetection: PlaneDetectionProvider
then in my init() I get:
self used in property access 'worldTracking' before all stored properties are initialized
Code snipet
@Observable
final class PlacementManager : ObservableObject {
private var worldTracking: WorldTrackingProvider
private var planeDetection: PlaneDetectionProvider
// … other props …
@MainActor
init() {
// error: self.worldTracking used before init…
planeAnchorHandler = PlaneAnchorHandler(rootEntity: root)
persistenceManager = PersistenceManager(
worldTracking: worldTracking,
rootEntity: root
)
// …
}
@MainActor
func setEnvironment(env: Environnement) async {
let newWorldTracking = WorldTrackingProvider()
let newPlaneDetection = PlaneDetectionProvider()
try await appState!.arkitSession.run(
[ newWorldTracking, newPlaneDetection ]
)
self.worldTracking = newWorldTracking
self.planeDetection = newPlaneDetection
// …
}
}
What I’ve Tried
Giving them default values at declaration (= WorldTrackingProvider())
Initializing them at the top of init() before any use
Passing the new providers into arkitSession.run(...)
My Question
What is the recommended Swift-style pattern to declare and reassign these ARKit provider properties so that:
They’re fully initialized before use in init(), and
I can swap them out later in setEnvironment(...) without compiler errors?
Any pointers (or links to forum threads / docs) would be greatly appreciated!
Hi there
I'm using Reality Composer Pro to anchor virtual content to a .referenceobject. However by moving the referenceobject quickly, it causes tracking to stop. (I know this is a limitation so im trying to make it a feature)
IS there a way to play a USDZ animation at the last known location, after detecting that reference object is no longer being tracked? is it possible to set this up in Reality Composer pro?
Nearly everything is set up in Reality Composer pro with my immersive.scene just anchoring virtual content to the Reference object in the RCP Scene, so my immersive view just does this -
if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
content.add(immersiveContentEntity)
& this
.onAppear {
appModel.immersiveSpaceState = .open
}
.onDisappear {
appModel.immersiveSpaceState = .closed
}
I have tried Using SpatialTracking & WorldTrackProvider, but I'm still quite new to Swift and coding in general so im unsure how to implement in conjunction with my RCP scene and if this is actually the right way to do it.
Apologies for my lack of knowledge.
Topic:
Spatial Computing
SubTopic:
Reality Composer Pro
Tags:
ARKit
RealityKit
Reality Composer Pro
visionOS
I remember that Vision Pro's dwell control could previously be set to 0.1 seconds, but now it can't. Is there a way to adjust it?
After implementing the method of obtaining video streams discussed at WWDC in the program, I found that the obtained video stream does not include digital models in the digital space or related videos such as the program UI. I would like to ask how to obtain a video stream or frame that contains only the physical world?
let formats = CameraVideoFormat.supportedVideoFormats(for: .main, cameraPositions:[.left])
let cameraFrameProvider = CameraFrameProvider()
var arKitSession = ARKitSession()
var pixelBuffer: CVPixelBuffer?
var cameraAccessStatus = ARKitSession.AuthorizationStatus.notDetermined
let worldTracking = WorldTrackingProvider()
func requestWorldSensingCameraAccess() async {
let authorizationResult = await arKitSession.requestAuthorization(for: [.cameraAccess])
cameraAccessStatus = authorizationResult[.cameraAccess]!
}
func queryAuthorizationCameraAccess() async{
let authorizationResult = await arKitSession.queryAuthorization(for: [.cameraAccess])
cameraAccessStatus = authorizationResult[.cameraAccess]!
}
func monitorSessionEvents() async {
for await event in arKitSession.events {
switch event {
case .dataProviderStateChanged(_, let newState, let error):
switch newState {
case .initialized:
break
case .running:
break
case .paused:
break
case .stopped:
if let error {
print("An error occurred: \(error)")
}
@unknown default:
break
}
case .authorizationChanged(let type, let status):
print("Authorization type \(type) changed to \(status)")
default:
print("An unknown event occured \(event)")
}
}
}
@MainActor
func processWorldAnchorUpdates() async {
for await anchorUpdate in worldTracking.anchorUpdates {
switch anchorUpdate.event {
case .added:
//检查是否有持久化对象附加到此添加的锚点-
//它可能是该应用程序之前运行的一个世界锚。
//ARKit显示与此应用程序相关的所有世界锚点
//当世界跟踪提供程序启动时。
fallthrough
case .updated:
//使放置的对象的位置与其对应的对象保持同步
//世界锚点,如果未跟踪锚点,则隐藏对象。
break
case .removed:
//如果删除了相应的世界定位点,则删除已放置的对象。
break
}
}
}
func arkitRun() async{
do {
try await arKitSession.run([cameraFrameProvider,worldTracking])
} catch {
return
}
}
@MainActor
func processDeviceAnchorUpdates() async {
await run(function: self.cameraFrameUpdatesBuffer, withFrequency: 90)
}
@MainActor
func cameraFrameUpdatesBuffer() async{
guard let cameraFrameUpdates =
cameraFrameProvider.cameraFrameUpdates(for: formats[0]),let cameraFrameUpdates1 =
cameraFrameProvider.cameraFrameUpdates(for: formats[1]) else {
return
}
for await cameraFrame in cameraFrameUpdates {
guard let mainCameraSample = cameraFrame.sample(for: .left) else {
continue
}
self.pixelBuffer = mainCameraSample.pixelBuffer
}
for await cameraFrame in cameraFrameUpdates1 {
guard let mainCameraSample = cameraFrame.sample(for: .left) else {
continue
}
if self.pixelBuffer != nil {
self.pixelBuffer = mergeTwoFrames(frame1: self.pixelBuffer!, frame2: mainCameraSample.pixelBuffer, outputSize: CGSize(width: 1920, height: 1080))
}
}
}
I am developing a Unity application for the Apple Vision Pro using PolySpatial and RealityKit integration.
The goal is to create a graspable object (for example, a handheld cube) that includes a secondary camera. When the user grabs and moves the object, the secondary camera should render its view to a RenderTexture, which is displayed on a quad attached to the object, simulating a live camera screen.
In the Unity Editor, this setup works correctly. The RenderTexture updates in real time, and the quad displays the camera’s view as expected.
However, when building and running the application on the Vision Pro, the quad only displays the clear background color of the secondary camera. No scene content appears. The graspable interaction itself works fine: the object can be grabbed and moved as intended.
Steps I have taken:
Created a new layer (CameraFeed) and assigned the relevant objects to it.
Set the secondary camera’s culling mask to render only the CameraFeed layer.
Assigned the RenderTexture as the camera’s target texture.
Applied the RenderTexture to an Unlit/Texture material on a quad.
Confirmed the camera is active and correctly positioned relative to the object.
From my research, it appears that once objects are managed by RealityKit through PolySpatial (for example, made graspable), they are no longer rendered through Unity's normal camera pipeline. Only the main XR camera (managed by RealityKit) seems able to see these objects. Secondary Unity cameras cannot render RealityKit-synced content to a RenderTexture. If this is correct, it seems there is currently no way to implement a true live secondary camera feed showing graspable objects on Vision Pro using Unity PolySpatial.
My questions are:
Is there any official way to enable multiple camera rendering of RealityKit-managed objects through PolySpatial?
Are there known workarounds to simulate a live camera feed that still allows objects to be grabbed?
Has anyone found alternative design patterns or methods for this kind of interaction?
Environment: Unity 6.0 , PolySpatial 2.2.4, Apple Vision OS XR 2.2.4
Any insight or suggestions would be greatly appreciated.
Thank you.
We have successfully obtained the permissions for "Main Camera access" and "Passthrough in screen capture" from Apple. Currently, the video streams we have received are from the physical world and do not include the digital world. How can we obtain video streams from both the physical and digital worlds?
thank you!
Topic:
Spatial Computing
SubTopic:
Reality Composer Pro
Tags:
Enterprise
Swift
Reality Composer Pro
visionOS
When creating a folder in the code, it prompts that the file creation is successful, but when the folder does not exist in the "Download Container" file, do you have any permissions when creating the folder in VisionOS?
static func getFileManager() -> URL {
let documentsDirectory = FileManager.default.urls(
for: .documentDirectory,
in: .userDomainMask
).first!
return documentsDirectory.appendingPathComponent("SGKJ_LIBRARY")
}
static func createFileLibrary() {
let folderUrl = getFileManager()
let fileManager = FileManager.default
do {
try fileManager.createDirectory(
at: folderUrl,
withIntermediateDirectories: true,
attributes: nil
)
print("Folder created successfully: \(folderUrl.path)")
} catch {
print("Failed to create folder: \(error.localizedDescription)")
}
}
I'm experimenting with RealityView in the UI of an AUv3 plug-in. The plug-in UI is implemented in a UIKitViewController with a UIHostingController hosting a RealityView. When i run the standalone app on visionOS I want the background to be transparent, and the reality view content. how can i achieve that? I've tried turning off opaque in many views and and setting background colors to .clear.
I am trying to create an object in immersive space that is partially transparent (~50% opacity). I have implemented this in a few different ways including creating a model entity and setting its opacity component to 0.5, and creating a custom material with blending set to a transparent opacity of 0.5. These both work partially, as they behaved as intended for many cases, but seemingly randomly would act like occlusion material and block any other immersive content behind them, showing the real world instead.
Some notes: I am using RealityKit to render the semi-transparent object and an opaque object that is behind the semi-transparent object. I am using VisionOS 2.1, and am updating the location of the semi-transparent object often. Both objects are ModelEntities.
I would appreciate any guidance on how to implement this. Please let me know if there are any other questions.
In several visionOS apps, we readjust our scenes to the user's eye level (their heads). But, we have encountered issues whereby the WorldTrackingProvider returns bad/incorrect positions for the first x number of frames.
See below code which you can copy paste in any Immersive Space. Relaunch the space and observe the numberOfBadWorldInfos value is inconsistent.
a. what is the most reliable way to get the devices's position?
b. is this indeed a bug?
c. are we using worldInfo improperly?
d. as a workaround, in our apps we set to 10 the number of frames to let pass before using worldInfo, should we set our threshold differently?
import ARKit
import Combine
import OSLog
import SwiftUI
import RealityKit
import RealityKitContent
let SUBSYSTEM = Bundle.main.bundleIdentifier!
struct ImmersiveView: View {
let logger = Logger(subsystem: SUBSYSTEM, category: "ImmersiveView")
let session = ARKitSession()
let worldInfo = WorldTrackingProvider()
@State var sceneUpdateSubscription: EventSubscription? = nil
@State var deviceTransform: simd_float4x4? = nil
@State var numberOfBadWorldInfos = 0
@State var isBadWorldInfoLoged = false
var body: some View {
RealityView { content in
try? await session.run([worldInfo])
sceneUpdateSubscription = content.subscribe(to: SceneEvents.Update.self) { event in
guard let pose = worldInfo.queryDeviceAnchor(atTimestamp: CACurrentMediaTime()) else {
return
}
// `worldInfo` does not return correct values for the first few frames (exact number of frames is unknown)
// - known SO: https://stackoverflow.com/questions/78396187/how-to-determine-the-first-reliable-position-of-the-apple-vision-pro-device
deviceTransform = pose.originFromAnchorTransform
if deviceTransform!.columns.3.y < 1.6 {
numberOfBadWorldInfos += 1
logger.warning("\(#function) \(#line) deviceTransform.columns.3.y \(deviceTransform!.columns.3.y), numberOfBadWorldInfos \(numberOfBadWorldInfos)")
} else {
if !isBadWorldInfoLoged {
logger.info("\(#function) \(#line) deviceTransform.columns.3.y \(deviceTransform!.columns.3.y), numberOfBadWorldInfos \(numberOfBadWorldInfos)")
}
isBadWorldInfoLoged = true // stop logging.
}
}
}
}
}
Hi, I'm playing now with hand tracking. I want to get position of hand inside a system update function. I was not sure if transform I'm getting from hand attached AnchorEntity (with trackingMode: .predicted) would give same results as handAnchors(at:) from hand tracking provider, so I started to read them both and compare. For handAnchors i tried using context.scene.timebase.sourceTimebase!.sourceClock!.time.seconds and CACurrentMediaTime() as timestamp source. They seem to use exactly same clock, so that doesn't matter, but:
for some reason update handler is always called twice with same context.deltaTime, but first time the query finds 0 entities, second time it finds them all. The query is the standard EntityQuery(where: .has(MyComponent.self)) and in update (matching: Self.query, updatingSystemWhen: .rendering). Here's part of logs:
System update called, entity count: 0, dt: 0.01000458374619484, absTime: 4654.222593541
System update called, entity count: 11, dt: 0.01000458374619484, absTime: 4654.22262525
System update called, entity count: 0, dt: 0.009999999776482582, absTime: 4654.249390875
System update called, entity count: 11, dt: 0.009999999776482582, absTime: 4654.249425
accounting for the double update calling I started to calculate time delta of absolute time between calls and they're most of the time much bigger, or much smaller than advertised by system's context.deltaTime, only sometimes they kind of match, for example:
system: (dt: 0.01000458374619484)
scene : (dt: 0.021419291667371) (absTime: 4654.222628125001)
and the very next call
system: (dt: 0.010009 166784584522)
scene : (dt: 0.0013097083328830195) (absTime: 4654.223937833334)
but sometimes
system: (dt: 0.009999999776482582)
scene : (dt: 0.009 112249999816413) (absTime: 4654.351299 166668)
Shouldn't those be more or less equal, or am I missing something?
In the end it seems that getting hand position from AnchorEntity and with handAnchors(at:) gives kind of same results, but at different time points, so I'd love to understand what's the correct way to use them and why time flows differently :).
--Edit--
P.S. Had to put spaces everywhere in logs between "9" and "1", otherwise post was blocked due to "sensitive content" :D
I am adapting my custom UI Framework for visionOS, and I'm wondering if it is going to be possible to detect hover over different UI elements within my view. The UI Framework draws to a Metal layer in a UIView. I don't currently support uihovergesturerecognizer on the view but I guess this wouldn't help, since you don't get coordinates. I can imagine an unpleasant solution might be to add invisible UIControls for each of my custom controls that are drawn in my own framework.
While using Screen Mirroring in developer mode within my immersive space, I noticed an alignment issue with the computer cursor (transparent circle). When I move it toward an attachment view, the cursor remains horizontal instead of aligning with the surface of the attachment view. It shows correctly on a 2D window only wrong on attachment view.
Is this behavior a bug, or could it be caused by a missing or incorrect configuration on the attachment view?
Want help, thanks.
I am developing an application which make use of 2 ornaments anchored to a volumetric window, one used a toolbar and one to display different views.
The problem I am facing consistently is that the ornaments seems to scale up or down after moving the volume using the OS handle or starting a GroupActivity session.
This first image shows the ornaments as soon as I started the app, no dragging nor group activities:
This second images shows them as soon as I join a group activity session:
The map, which might seem smaller, has not been touched and has always the same scale.
In this last image I had just dragged the entire volume using the OS toolbar, resulting in the ornaments scaling down:
This is how the volume and the ornaments are declared:
WindowGroup(id: "CityVolume") {
let cityVM = CityViewModel(volumeSize: CityView.initialVolumeSize)
CityView(cityVM: cityVM)
.ornament(attachmentAnchor: .scene(.bottomFront)) {
HStack {
TourismChartsButton()
LandmarksListButton()
CenterMapButton()
ToggleImmersiveSpaceButton()
TrafficDataButton()
BusLinesButton()
}
.padding()
.offset(z: 10)
.rotation3DEffect(Angle(degrees: 15), axis: (x: 1.0, y: 0.0, z: 0.0))
}
.ornament(attachmentAnchor: .scene(.back)) {
ZStack {
if AppModel.Instance.tourismVM.isChartViewVisible {
TourismChartsView()
}
if AppModel.Instance.busLinesVM.isDataViewEnabled {
BusLineView()
}
}
}
.task(observeGroupActivity)
.onAppear {
appModel.cityVM = cityVM
}
}
.windowStyle(.volumetric)
.windowResizability(.contentSize)
.volumeWorldAlignment(.gravityAligned)
.defaultSize(CityView.initialVolumeSize, in: .meters)
It happens also without starting a SharePlay session, but not as frequently as during SharePlay. Experienced the same behaviour with toolbars.
Am I doing something wrong with how I created the ornaments? Am I missing something?
I am looking for a material that functions in the same way that Occlusion Material does, except that it only partially occludes whatever is behind it. One way that I have thought of doing this was to change the opacity of the entity that was covered in Occlusion Material, however this did not change anything. Please let me know if this is possible.
As far as I know, Apple hasn’t opened access to the Vision Pro camera for developers yet, so I’m trying to find possible workarounds within the current capabilities. I’m wondering if there’s any way to apply a mesh to a person in the scene in Vision Pro, or if there’s an alternative approach to roughly detect a human shape in front of the user?
Hi, I'm working with CameraFrameProvider from Enterprise API. Is it always capped at 30fps, or is there something I can switch to get more?
I assume it is capped at 30, so let me cram in additional question here :). If I'd get a developer strap and attach an external camera capable of doing >30fps, will I get the full stream, or some other limitation will kick in?