Add an AR Object To Your SwiftUI Preview Canvas With RealityKit
Daily Coding Tip 022
 |
Here’s how to make the SwiftUI preview canvas an image anchor, meaning that RealityKit will automatically detect it using the camera. When the Canvas is detected, a 3D model of a steel box will appear. This is the default object when creating a blank Augmented Reality project, so I thought I’d stick with it for the sake of expediency.
The main challenge with creating image anchors is making sure that the app knows what size those anchors will be in the real world. This creates a challenge, as I want the virtual object itself to act as part of the image anchor. Even though we can specify the size of the virtual box, that doesn’t mean it will appear that way in the preview.
This is because if we don’t move the box a certain distance ‘into’ the screen, it will be exactly where the camera is and be clipped from existence.
I used an iPhone 12 Pro for this, but you can use whatever device you have. Your preview canvas will match the device you’re using, unless you set the preview device manually. To do that, click anywhere on the preview to highlight it, and you’ll see the word ARViewContainer at the bottom of the panel. Clicking this will give you access to many settings for the preview, including which device is being used.
I went to SizeScreens to get the dimensions of my device. My iPhone is 7.15 cm wide, but the screen is only 6.5 cm wide.
I want my virtual box to have a centimeter to spare on either side, so that would make it 4.5 centimeters in size. Open the Experience.rcproject file, and click the Open in Reality Composer button in the top right. All we want to do here is click on the steel box, change its width, height and depth to 4.5 cm, and close Reality Composer. If you don’t close that window, the changes will not be saved, and you will not see evidence of them elsewhere.
Create a new project, choosing iOS as the platform and SwiftUI as the user interface. This will give you a basic AR app that, if you run it, displays a steel box on any horizontal surface. We can do something more interesting than that!
Replace the code in ARViewContainer with this:
The first function here creates a preview. This basically removes the steel box from the horizontal plane anchor it came with and anchors it to the camera instead. I don’t know exactly why I have to move the box 1.86 metres away in the Z direction in order for it to be displayed at the size I want. On my MacBook Pro, 75% zoom level makes the preview exactly the same size as my iPhone 12 Pro. This is useful, because we need to know the actual size of everything in the real world. On your screen, the zoom level required to make the preview the right size might be different.
1.86 metres was enough to give me a 1 cm gap on either side of my 4.5 cm cube, so I was happy.
You’ll notice that I use conditional compilation flags in the makeUIView function, so that a preview is returned if the target environment is a Simulator, and the actual box anchor is returned on a physical device. As Simulators don’t have cameras, and the preview canvas is a kind of Simulator, it is impossible to run the normal kind of AR app on them. Our preview will instead display the box on a white background. This will serve as our image anchor, so it’s important that the box on the screen is the size we want our virtual box to be.
I used the Measure app on my iPhone to get a vague idea of physical size but you may want to use a ruler for precision, as it seems to round to the nearest centimetre for some reason.
Use Cmd + Shift + 4 to open the Snipping Tool, and draw a rectangle tightly around the preview device, as seen below:
This will act as our image anchor. Go back to Experience.rcproject, but don’t click on the box. Clicking anywhere in empty space will select nothing, and the properties panel will display the anchor type. Change it to Image, and import the screenshot from your Desktop. Make sure you set the physical width to the width of your device, which in my case was 7.15 cm. If you find that the box seems to be halfway through the image, click on it, click on the green arrow and drag it slightly.
This will snap the box into place on top of the image.
This should be all you need to do. Run the app on your device, and point the device at your Mac screen.
You may need to adjust the brightness of your screen to make the preview more similar to the image anchor.
Create your profile
Only paid subscribers can comment on this post
Check your email
For your security, we need to re-authenticate you.
Click the link we sent to , or click here to log in.