Hi,

Thanks for this great repo. I've had a play around with the samples and it seems only the front camera is working properly. The device I am using to run the program is iPhone 11 and my XCode version is 12.2.

When I try to run Real-time Depth and Real-time Depth Mask, I run into a fatal error in AVCaptureDevice+Extension line 39.

I've tried to have a look at it myself but since I'm relatively new to Swift, I've been unable to resolve the problem myself. The availableFormats is always empty which results in the guard failing. I'm not too sure why since the format field is always populated.

Am I doing something wrong?

I cloned master @106813e0860f381f1821baddf76b9cdeebfcc490, opened the .xcworkspace, and change the signing, then tried to compile, which resulting in four compilation errors:

/Users/jon/git/iOS-Depth-Sampler/iOS-Depth-Sampler/Renderer/MetalRenderer.swift:71:39: 
Value of type 'MTLDrawable' has no member 'texture'

/Users/jon/git/iOS-Depth-Sampler/iOS-Depth-Sampler/Samples/Realtime-Depth/RealtimeDepthViewController.swift:38:54:
 Value of type 'MTLDrawable' has no member 'layer'

/Users/jon/git/iOS-Depth-Sampler/iOS-Depth-Sampler/Samples/ARKit/ARKitDepthViewController.swift:34:28: 
Value of type 'ARSCNView?' has no member 'device'

/Users/jon/git/iOS-Depth-Sampler/iOS-Depth-Sampler/Samples/ARKit/ARKitDepthViewController.swift:39:54:
 Value of type 'MTLDrawable' has no member 'layer'

grayPixelDataメソッド内でindexを1/2していた箇所について、Depthの 16bitフォーマットを32bitでscanしていたためではないかと思いました。 converting(toDepthDataType: kCVPixelFormatType_DepthFloat32) メソッドを実行してz軸を逆方向にしてindexをそのまま使用できることを確認しましたので、pullrequestさせて頂きました。

I am using the ARKit to detect face with depthData.

However, I am not able to distinguish whether this face is a real world person or a picture placed in front of the device camera.

Hi there!

Thank you very much for creating this open source repo, I plan to use it for future ios machine learning projects! In the WWDC 2017 Talk, Apple discusses that the depth output is geometrically distorted to align with images produced by the camera. They mention to get precise true depth measurements, you need to correct for lens distortion. the WWDC Talk says that a reference implementation to correct for lens distortion is commented in the AVCameraCalibrationData.h file.

It would be great if you can add an example view controller that a user taps to see a recitfied depth image and enable developers to work with precise, true depth measurements. I attached the reference implementation from AVCameraCalibrationData.h for ease of reference and if anyone can add this it would be amazing!

The following reference implementation illustrates how to use the lensDistortionLookupTable, 
    inverseLensDistortionLookupTable, and lensDistortionCenter properties to find points in the 
    lens-distorted or undistorted (rectilinear, corrected) space. If you have a distorted image (such as a photo taken by a camera) and want to find a particular point in a corresponding undistorted image, you would call the sample method below using the inverseLensDistortionLookupTable. If you have an undistorted (aka distortion-corrected) image and want to find a point in the distorted image's space, you would call the sample method below using the lensDistortionLookupTable.
 
    To apply distortion correction to an image, you'd begin with an empty destination buffer and iterate through it 
    row by row, calling the sample implementation below for each point in the output image, passing the 
    lensDistortionLookupTable to find the corresponding value in the distorted image, and write it to your 
    output buffer. Please note that the "point", "opticalCenter", and "imageSize" parameters below must be
    in the same coordinate system, i.e. both at full resolution, or both scaled to a different resolution but
    with the same aspect ratio.
 
    The reference function below returns floating-point x and y values. If you wish to match the results with 
    actual pixels in a bitmap, you should either round to the nearest integer value or interpolate from surrounding
    integer positions (i.e. bilinear interpolation from the 4 surrounding pixels).
 
- (CGPoint)lensDistortionPointForPoint:(CGPoint)point
                           lookupTable:(NSData *)lookupTable
               distortionOpticalCenter:(CGPoint)opticalCenter
                             imageSize:(CGSize)imageSize
{
    // The lookup table holds the relative radial magnification for n linearly spaced radii.
    // The first position corresponds to radius = 0
    // The last position corresponds to the largest radius found in the image.
 
    // Determine the maximum radius.
    float delta_ocx_max = MAX( opticalCenter.x, imageSize.width  - opticalCenter.x );
    float delta_ocy_max = MAX( opticalCenter.y, imageSize.height - opticalCenter.y );
    float r_max = sqrtf( delta_ocx_max * delta_ocx_max + delta_ocy_max * delta_ocy_max );
 
    // Determine the vector from the optical center to the given point.
    float v_point_x = point.x - opticalCenter.x;
    float v_point_y = point.y - opticalCenter.y;
 
    // Determine the radius of the given point.
    float r_point = sqrtf( v_point_x * v_point_x + v_point_y * v_point_y );
 
    // Look up the relative radial magnification to apply in the provided lookup table
    float magnification;
    const float *lookupTableValues = lookupTable.bytes;
    NSUInteger lookupTableCount = lookupTable.length / sizeof(float);
 
    if ( r_point < r_max ) {
        // Linear interpolation
        float val   = r_point * ( lookupTableCount - 1 ) / r_max;
        int   idx   = (int)val;
        float frac  = val - idx;
 
        float mag_1 = lookupTableValues[idx];
        float mag_2 = lookupTableValues[idx + 1];
 
        magnification = ( 1.0f - frac ) * mag_1 + frac * mag_2;
    }
    else {
        magnification = lookupTableValues[lookupTableCount - 1];
    }
 
    // Apply radial magnification
    float new_v_point_x = v_point_x + magnification * v_point_x;
    float new_v_point_y = v_point_y + magnification * v_point_y;
 
    // Construct output
    return CGPointMake( opticalCenter.x + new_v_point_x, opticalCenter.y + new_v_point_y );
}

Hi I was wondering if you could add a sample that could load up a grayscale jpg image and add that as the depth map of another image that does not already have depth data?