Superimposes your image into a 3D world using Metal+Arkit

Overview

arkit-virtual-backdrop

Superimposes your image into a 3D rendered world using Metal+ARKit.

Screenshot of example

This example calculates depth to viewer's face, and uses it to remove the background behind the viewer's head and body, replacing it with a 3D scene rendered in Metal. The viewer's image is rendered as a plane in front of the geometry using a Metal shader that passes through when the depth is beyond a threshold.

The depth distance to include behind the user's face is a work in progress, but accurately setting this value is the key to successful compositing. A value too large causes background artifacts from the camera to appear in front of the rendered scene, and a value too small removes the user's body. Currently the value is calculated based on the camera's height in relation to the face, as well as the camera's angle.

Issues to be resolved/investigated further:

  • Border around head and body is noisy
  • Depth images are not recieved consistently
You might also like...
ARKit Base Project. Place virtual objects based on WWDC example project
ARKit Base Project. Place virtual objects based on WWDC example project

ARKit - Placing Virtual Objects in Augmented Reality Learn best practices for visual feedback, gesture interactions, and realistic rendering in AR exp

This library uses ARKit Face Tracking in order to catch user's smile.
This library uses ARKit Face Tracking in order to catch user's smile.

SmileToUnlock Make your users smile before opening the app :) Gif with the demonstration Installation Cocoapods The most preferable way to use this li

Power! Unlimited power for ARKit 2.0!
Power! Unlimited power for ARKit 2.0!

A long time ago in a galaxy, far, far away... It is a period when iPhone SE and iPhone X were destroyed from the apple store, the AR market was under

A simple application created for educational purposes for mastering ARKit
A simple application created for educational purposes for mastering ARKit

ARDrawing AR Drawing is a simple application created for educational purposes for mastering ARKit. The basis of the project is copied from the project

Trying TDD with ARKit
Trying TDD with ARKit

ARPlacer BDD Spec As a user I want to place a random object in real world. I also want to see the distance between AR object and my phone. Use Cases

Draw VR content over live camera feed with ARKit

funny-ar Exercise with ARKit: draw VR content over live camera feed: work is in

ARKit: Projecting 3D mesh to 2D coordinate
ARKit: Projecting 3D mesh to 2D coordinate

ARKitDemo ARKit: Projecting 3D mesh to 2D coordinate A simple utility to project 3D face mesh in 2D coordinate on device screen. Sources: https://deve

Furniture E-Commerce Augmented Reality(AR) app in iOS powered by ARKit
Furniture E-Commerce Augmented Reality(AR) app in iOS powered by ARKit

HomeMax-iOS Furniture E-Commerce Augmented Reality(AR) app in iOS powered by ARKit and SceneKit. Inspired by IKEA place app. Description Experience on

Reality-iOS - NFT Augmented Reality(AR) app that demonstrate application of ARImageTracking in iOS powered by ARKit 2

Reality-iOS NFT Augmented Reality(AR) app that demonstrate application of ARImag

Comments
  • Cannot launch this app?

    Cannot launch this app?

    Hi, I want to test this app, but I knows little about iOS development. When I use xcode and a real iPhone 6s to test, xcode outputs following debugging information, this app is not launched successfully. Please help, Thanks!

    Xcode outputs following debugging information:

    2018-06-24 14:17:30.402621+0800 Backdrop Demo[5498:2198260] [DYMTLInitPlatform] platform initialization successful 2018-06-24 14:17:31.042208+0800 Backdrop Demo[5498:2198007] Metal GPU Frame Capture Enabled 2018-06-24 14:17:31.044135+0800 Backdrop Demo[5498:2198007] Metal API Validation Enabled 2018-06-24 14:17:31.682183+0800 Backdrop Demo[5498:2198007] [MC] System group container for systemgroup.com.apple.configurationprofiles path is /private/var/containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles 2018-06-24 14:17:31.682790+0800 Backdrop Demo[5498:2198007] [MC] Reading from public effective user settings. 2018-06-24 14:17:31.695632+0800 Backdrop Demo[5498:2198007] [General] No image capture device found. 2018-06-24 14:17:31.846104+0800 Backdrop Demo[5498:2198007] [Session] ARSession (0x13bd12420): Unable to run the session, configuration is not supported on this device: <ARFaceTrackingConfiguration: 0x1c028fc30 worldAlignment=Gravity lightEstimation=Enabled> 2018-06-24 14:17:31.853462+0800 Backdrop Demo[5498:2198007] [Session] Session (0x13bd12420): did fail with error: Error Domain=com.apple.arkit.error Code=100 "Unsupported configuration." UserInfo={NSLocalizedDescription=Unsupported configuration., NSLocalizedFailureReason=The provided configuration is not supported on this device.}

    opened by hitzhangjie 2
Owner
Gabriel Montague
Gabriel Montague
A library that allows you to generate and update environment maps in real-time using the camera feed and ARKit's tracking capabilities.

ARKitEnvironmentMapper Example To run the example project, clone the repo, and run pod install from the Example directory first. Installation ARKitEnv

SV Hawks 91 Dec 4, 2022
PlacenoteSDK Sample app in native iOS using ARKit, written primarily in Swift

Placenote SDK for iOS Placenote SDK lets you easily build cloud-based Augmented Reality (AR) apps that pin digital content to locations in the real wo

Placenote 93 Nov 15, 2022
A library that allows you to generate and update environment maps in real-time using the camera feed and ARKit's tracking capabilities.

ARKitEnvironmentMapper Example To run the example project, clone the repo, and run pod install from the Example directory first. Installation ARKitEnv

SV Hawks 91 Dec 4, 2022
ARID - Augmented Reality app using Apple’s ARKit framework which can recognise faces of famous scientists

ARID Augmented Reality app using Apple’s ARKit framework which can recognise fac

Hemanth 0 Jan 12, 2022
ARDicee - Simple augmented reality app using SceneKit and ARKit

ARDicee Simple augmented reality app using SceneKit and ARKit Requirements Xcode

donggyu 3 Feb 4, 2022
AR Ruler - A simple iOS app made using ARKit and SceneKit

A simple iOS app made using ARKit and SceneKit.Which can try to simplify little things in your life such as measuring stuff.

Dishant Nagpal 5 Aug 31, 2022
Using ARKit and LiDAR to save depth data and export point cloud, based on WWDC20-10611 sample code

Save iOS ARFrame and Point Cloud This project improves the usability of the sample code from WWDC20 session 10611: Explore ARKit 4. Note that the samp

null 4 Dec 22, 2022
ARKit + CoreLocation: Combines the high accuracy of AR with the scale of GPS data.

ARKit: Uses camera and motion data to map out the local world as you move around. CoreLocation: Uses wifi and GPS data to determine your global locati

Andrew Hart 5.3k Dec 27, 2022
ARKit Demo Application

ARKitNavigationDemo Work in progress. In Progress Region — For one, we could render far fewer nodes. In fact, it’s a bit distracting that the entire t

Christopher Webb 296 Dec 16, 2022
An iOS Framework Capture & record ARKit videos 📹, photos 🌄, Live Photos 🎇, and GIFs 🎆.

An iOS Framework that enables developers to capture videos ?? , photos ?? , Live Photos ?? , and GIFs ?? with ARKit content.

Ahmed Bekhit 1.5k Dec 24, 2022