arkit-virtual-backdrop

Superimposes your image into a 3D rendered world using Metal+ARKit.

This example calculates depth to viewer's face, and uses it to remove the background behind the viewer's head and body, replacing it with a 3D scene rendered in Metal. The viewer's image is rendered as a plane in front of the geometry using a Metal shader that passes through when the depth is beyond a threshold.

The depth distance to include behind the user's face is a work in progress, but accurately setting this value is the key to successful compositing. A value too large causes background artifacts from the camera to appear in front of the rendered scene, and a value too small removes the user's body. Currently the value is calculated based on the camera's height in relation to the face, as well as the camera's angle.

Issues to be resolved/investigated further:

Border around head and body is noisy
Depth images are not recieved consistently

You might also like...

ARKit Base Project. Place virtual objects based on WWDC example project

Comments

Cannot launch this app?

Hi, I want to test this app, but I knows little about iOS development. When I use xcode and a real iPhone 6s to test, xcode outputs following debugging information, this app is not launched successfully. Please help, Thanks!

Xcode outputs following debugging information:

2018-06-24 14:17:30.402621+0800 Backdrop Demo[5498:2198260] [DYMTLInitPlatform] platform initialization successful 2018-06-24 14:17:31.042208+0800 Backdrop Demo[5498:2198007] Metal GPU Frame Capture Enabled 2018-06-24 14:17:31.044135+0800 Backdrop Demo[5498:2198007] Metal API Validation Enabled 2018-06-24 14:17:31.682183+0800 Backdrop Demo[5498:2198007] [MC] System group container for systemgroup.com.apple.configurationprofiles path is /private/var/containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles 2018-06-24 14:17:31.682790+0800 Backdrop Demo[5498:2198007] [MC] Reading from public effective user settings. 2018-06-24 14:17:31.695632+0800 Backdrop Demo[5498:2198007] [General] No image capture device found. 2018-06-24 14:17:31.846104+0800 Backdrop Demo[5498:2198007] [Session] ARSession (0x13bd12420): Unable to run the session, configuration is not supported on this device: <ARFaceTrackingConfiguration: 0x1c028fc30 worldAlignment=Gravity lightEstimation=Enabled> 2018-06-24 14:17:31.853462+0800 Backdrop Demo[5498:2198007] [Session] Session (0x13bd12420): did fail with error: Error Domain=com.apple.arkit.error Code=100 "Unsupported configuration." UserInfo={NSLocalizedDescription=Unsupported configuration., NSLocalizedFailureReason=The provided configuration is not supported on this device.}

opened by hitzhangjie 2