A library that allows you to generate and update environment maps in real-time using the camera feed and ARKit's tracking capabilities.

Related tags

Augmented Reality ARKitEnvironmentMapper
Overview

ARKitEnvironmentMapper

CI Status Version License Platform

Example

To run the example project, clone the repo, and run pod install from the Example directory first.

Installation

ARKitEnvironmentMapper is available through CocoaPods. To install it, simply add the following line to your Podfile:

pod 'ARKitEnvironmentMapper'

Usage

To initialize with a base environment map with an image asset named, for example, "room", use the following code:

let environmentMapper = ARKitEnvironmentMapper(withImageName: "room")

Note that the input image has to have a 2:1 aspect ratio.

Alternatively, you can initialize it with a height and a color:

let environmentMapper = ARKitEnvironmentMapper(withMapHeight: 512, withDefaultColor: .red)

To start the mapping process, call the startMapping() method. You should call this method a couple of seconds after running your ARSession in order not to get wrong mappings on your environment map.

To stop the mapping process, simply call stopMapping().

To update the environment map with the current feed of the camera, you can use the following code in your class implementing ARSessionDelegate:

func session(_ session: ARSession, didUpdate frame: ARFrame) {
  environmentMapper.updateMap(withFrame: frame)
}

You can call updateMap(withFrame:) in didUpdate regardless of the preferred FPS, as it is not executed every frame. Its frequency is bound by the value ARKitEnvironmentMapper.Options.updatesPerSecond and the default value is 10 updates per second.

After mapping the environment for a while you can get the current generated environment map and set it as the environment map of your ARSCNView with the following code:

sceneView.scene.lightingEnvironment.contents = environmentMapper.currentEnvironmentMap(as: .cgImage)

Note: The ideal way to set this should be to use .mtlTexture as input to bypass any image conversion and memory operation overhead. However, due to an Apple bug, setting a MTLTexture as the environment map currently has no effect. If you think this is not an Apple bug and you do have a solution, please don't hesitate to send a pull request.

Output

Below is a sample output ARKitEnvironmentMapper generates when initialized with a base environment map:

=>

Here is another sample output when initialized with a white background in the same environment:

Author

Halil Ibrahim Kayim

Software Engineer @ Surreal
Twitter Github LinkedIn

License

ARKitEnvironmentMapper is available under the MIT license. See the LICENSE file for more info.

Comments
  • The airplane model in the example is too bright and how to fix it?

    The airplane model in the example is too bright and how to fix it?

    I install the example in iphone6s, but the model is too bright. img_2077 I tried to change the code in ARKitEnvironmentMapper/Classes/ImageConverter.swift, such as changing " let ev = (1 / 2) * (maxLightIntensity - estimatedIntensity) / maxLightIntensity" to "let ev = (1 / 200) * (maxLightIntensity - estimatedIntensity) / maxLightIntensity" and the variable "experimentalExposureCorrectionEnabled" in ARKitEnvironmentMapper+Options.swift changed to be true , but nothing changed, the model is still too bright. How should I fix this problem?

    opened by ghoshaw 2
  • validateRenderPassDescriptor:644: failed assertion `Texture at colorAttachment[0] has usage (0x01) which doesn't specify MTLTextureUsageRenderTarget (0x04)'

    validateRenderPassDescriptor:644: failed assertion `Texture at colorAttachment[0] has usage (0x01) which doesn't specify MTLTextureUsageRenderTarget (0x04)'

    Hi! I change the model from the airplane to a sphere, and add the plane detect to put the sphere on the plane. But when I apply the environmentmap, the error goes " 2019-01-11 09:43:40.371207+0800 ARKitEnvironmentMapper_Example[19867:1744813] [DYMTLInitPlatform] platform initialization successful 2019-01-11 09:43:41.228950+0800 ARKitEnvironmentMapper_Example[19867:1744618] Metal GPU Frame Capture Enabled 2019-01-11 09:43:41.232703+0800 ARKitEnvironmentMapper_Example[19867:1744618] Metal API Validation Enabled 2019-01-11 09:43:41.584089+0800 ARKitEnvironmentMapper_Example[19867:1744618] [MC] System group container for systemgroup.com.apple.configurationprofiles path is /private/var/containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles 2019-01-11 09:43:41.585921+0800 ARKitEnvironmentMapper_Example[19867:1744618] [MC] Reading from public effective user settings. validateRenderPassDescriptor:644: failed assertion `Texture at colorAttachment[0] has usage (0x01) which doesn't specify MTLTextureUsageRenderTarget (0x04)' "

    And I google to find that "textureDescriptor.usage = MTLTextureUsage.renderTarget" this line should be added, but I do not know where to add it.

    opened by ghoshaw 2
  • Finished environment mapping is too bright and needs to rotate 180 degrees

    Finished environment mapping is too bright and needs to rotate 180 degrees

    Hi!

    Must start off by saying that this tool is amazing.

    Im having a problem with my finished environment mapping. As the title says, the brightness is too high, is this a problem you are having aswell? The objects are really standing out in the scene. The second issue that I am having is that the environment map needs to be rotated horizontally 180 degrees, which now seems like the alignment of the env map is that of how it was fed to the camera.

    Have you encountered these problems before?

    Cheers

    opened by joekay 2
  • Use a higher bpc coordinate conversion image for better mapping accuracy

    Use a higher bpc coordinate conversion image for better mapping accuracy

    Right now, the coordinate conversion texture is 32 bits-per-pixel, which makes coordinate conversion somewhat inaccurate. Use either a 64 bpc or 128 bpc texture to achieve higher accuracy.

    enhancement 
    opened by halileohalilei 0
  • Relax the assumption that the user's position is fixed

    Relax the assumption that the user's position is fixed

    Determine an outer radius, which encloses the environment of the user, and an inner radius in which the user can move. Handle the viewport projections accordingly.

    enhancement 
    opened by halileohalilei 2
Owner
SV Hawks
SV Hawks
GitHub
AREarthObservatory - A minimal iOS AR app that visualizes time-series changes in the global environment based on data from NASA satellites

A minimal iOS AR app that visualizes time-series changes in the global environment based on data from NASA satellites

Yasuhito Nagatomo 3 Aug 11, 2022
Draw VR content over live camera feed with ARKit

funny-ar Exercise with ARKit: draw VR content over live camera feed: work is in

pavel 3 Dec 18, 2021
This library uses ARKit Face Tracking in order to catch user's smile.

SmileToUnlock Make your users smile before opening the app :) Gif with the demonstration Installation Cocoapods The most preferable way to use this li

Ruslan Serebriakov 628 Oct 22, 2022
Augmented Reality image tracking with SwiftUI, RealityKit and ARKit 4.

ARImageTracking This is an Augmented Reality Xcode project that uses Apple's newest RealityKit framework and ARKit 4 features, to dynamically track a

Richard Qi 198 Dec 7, 2022
Caffeine tracking for everyone

Cortado Cortado is an iPhone app to help you track your caffeine consumption habits. Besides letting you manually enter caffeine data, it detects when

Emilia 169 Oct 16, 2022
IOS example app to generate point clouds in ARKit using scenedepth

Visualizing a Point Cloud Using Scene Depth Place points in the real-world using the scene's depth data to visualize the shape of the physical environ

Isak Diaz 20 Oct 31, 2022
Smart Online Shopping iOS App with Augmented Reality (AR) and simple Social Media features using SwiftUI and Google Firebase Cloud Services

Table of contents App Demo How to Run Context Content How it's written Inspiration App Demo AR.online.shopping.iOS.demo.mp4 How to Run First make sure

Ashkan Goharfar 13 Nov 1, 2022
Using ARKit and LiDAR to save depth data and export point cloud, based on WWDC20-10611 sample code

Save iOS ARFrame and Point Cloud This project improves the usability of the sample code from WWDC20 session 10611: Explore ARKit 4. Note that the samp

null 4 Dec 22, 2022
It is a music and podcast creation system that enables users to create multiple tracks using AR(Augmented Reality).

AR-Studio It is a music creation system enables users to create multiple tracks using AR(Augmented Reality). Introduction AR Studio turns your iPad, a

Ayush Singh 5 Dec 3, 2022
Cut and paste your surroundings using AR

An AR+ML prototype that allows cutting elements from your surroundings and pasting them in an image editing software.

Cyril Diagne 14.5k Dec 27, 2022
ARDicee - Simple augmented reality app using SceneKit and ARKit

ARDicee Simple augmented reality app using SceneKit and ARKit Requirements Xcode

donggyu 3 Feb 4, 2022
AR Ruler - A simple iOS app made using ARKit and SceneKit

A simple iOS app made using ARKit and SceneKit.Which can try to simplify little things in your life such as measuring stuff.

Dishant Nagpal 5 Aug 31, 2022
ARVoxelKit - Lightweight Framework for Voxel graphic using AR + SceneKit

ARVoxelKit Lightweight Framework for Voxel graphic using AR + SceneKit Requirements ARVoxelKit requires iOS 11 and devices, which support ARKit Usage

null 79 Dec 29, 2022
PlacenoteSDK Sample app in native iOS using ARKit, written primarily in Swift

Placenote SDK for iOS Placenote SDK lets you easily build cloud-based Augmented Reality (AR) apps that pin digital content to locations in the real wo

Placenote 93 Nov 15, 2022
ARID - Augmented Reality app using Apple’s ARKit framework which can recognise faces of famous scientists

ARID Augmented Reality app using Apple’s ARKit framework which can recognise fac

Hemanth 0 Jan 12, 2022
A minimal iOS AR app that displays a wave animation using RealityKit2 Geometry Modifier

AR Simple Sea A minimal iOS AR app that displays a wave animation using RealityKit2 Geometry Modifier. Xcode 13.3 Target: iOS / iPadOS 15.0+ SwiftUI,

Yasuhito Nagatomo 15 Dec 29, 2022
An iOS Framework Capture & record ARKit videos 📹, photos 🌄, Live Photos 🎇, and GIFs 🎆.

An iOS Framework that enables developers to capture videos ?? , photos ?? , Live Photos ?? , and GIFs ?? with ARKit content.

Ahmed Bekhit 1.5k Dec 24, 2022
This package enables easy, convenient fading in and fading out in RealityKit.

RKFade This package enables easy, convenient fading in and fading out in RealityKit. Be sure to call FadeSystem.register() (Not FadeSystem.registerSys

Grant Jarvis 2 Nov 7, 2022
Funny app with the fitting of sunglasses Used SwiftUI and RealityKit

Reality-Glasses-2021 on SwiftUI Funny app with the fitting of sunglasses. Used SwiftUI and RealityKit Interface and Features: Simple Intuitive Design

null 1 Nov 13, 2021