On some devices, the rotation rates exhibit a low-level drift on one or more rotation axes. The symptom expressions are not identical, but they appear to be related to low component quality (iPhone 5c versus higher end devices) and/or rough usage (drops, etc).

Test devices Kilo and Juliet (both iPhones 5c) are affected. Lay them flat on a table or hold them absolutely still in your hand, and you'll see the camera drift up or down to the min/max y values, and in an infinite rotation in the x direction.

In an ideal scenario, we could ask users to calibrate their gyroscopes and apply a corrective factor to all inputs. Barring that, the next best thing we can try is to add a low-pass filter which ignores input less than a given threshold. In my non-scientific testing with the only affected devices at my disposal, I found that a noise threshold between 0.10 and 0.15 filtered out the noise with a minimal loss in sensitivity. Less than 0.10 and the 360 camera position starts to drift. I'm using 0.12 for now.

See also: https://forums.developer.apple.com/thread/12049

The primary goal was to add play and pause methods to NYT360VideoViewController. This required some new methods and properties in other classes to make it work. I also cleaned up some files I encountered to make the method groupings and variable names easier to follow.

Note: this PR will have minor merge conflicts with this other PR but I wanted to keep these changes separate to make them easier to review.

This PR fixes visual distortions when resizing an NYT360VideoViewController's view. The underlying cause is that SCNView does not appear to support content resizing modes like "aspect fit" or "aspect fill". To correct the distortions, we have to preserve the SCNView's aspect ratio across all resizing, and use view containment to "crop" the out-of-bounds portions of the view.

• NYT360VideoViewController no longer overrides loadView to set it's view to an instance of SCNView, but instead adds a SCNView as a subview inside viewDidLoad.
• I initialize the SCNView with an arbitrary but educated guess about the optimal size (in points). Since source videos for 360 content are shot in landscape orientation, I convert the UIScreen bounds to a landscape orientated bounds and use that to initialize the SCNView.
• In NYT360VideoViewController, I override viewDidLayoutSubviews to update the frame of its SCNView property. The aspect ratio is preserved even though the actual point values will be scaled to fill the containing view. Thus, the SCNView will always use a landscape orientation congruent with the device display, which will prevent distortion regardless of the aspect ratio of the containing view (inline in a table view cell, full screen in portrait mode, etc.)

I also improved the autorotation animations by animating the yFov (field of view) changes to the SceneKit came node, to make the FOV change during automation less jarring.

MO-7031 Calls for selectively disabling vertical panning for 360 player views embedded inline in a scroll view. This PR lays down the groundwork for supporting this feature:

• Breaks out some complex math that was implemented in NYT360CameraController.m into separate NYT360EulerAngleCalculations helper functions, both to isolate this logic and also to expose it more easily to unit tests.
• Cleans up some inline functions and constants around the calculation code that was previously in the camera controller.
• Adds an allowedPanningAxes property to NYT360CameraController which allows host applications to suppress x or y (or both) axis input from both device motion and pan gesture changes.
• Implements the suppression of x/y axis movement in the newly-created NYT360EulerAngleCalculations helper functions.
• Adds unit tests for verifying that the correct axes are suppressed when calculating new camera position and euler angles.

With these changes in place, it should be possible in a followup PR to adjust the axis option on both NYT360ViewController and on any future inline 360 video player view.

There is a bug in SceneKit wherein a paused video node will begin playing again when the application becomes active. This is caused by cascading calls to [fooNode setPaused:NO] across all nodes in a scene. To prevent the video node from unpausing along with the rest of the nodes, we must subclass SKVideoNode and override setPaused:, only unpausing the node if its nytDelegate allows it.

closes #5

This PR provides a means to pass a shared CMMotionManager from the host application to NYT360Videos. Further, it provides a protocol (and a default implementation of that protocol) which wraps a shared CMMotionManager in an API that prevents misuse of the shared manager.

Note: I'll fix the merge conflicts later this afternoon.

When attempting to run the sample project on iOS 11.3 with Xcode 9.3. The example video URL (https://vp.nyt.com/video/360/hls/video.m3u8) fails to load and the console shows:

2018-04-24 16:20:13.539335+0800 NYT360VideoExample[6285:2948864] CredStore - performQuery - Error copying matching creds.  Error=-25300, query={
    class = inet;
    "m_Limit" = "m_LimitAll";
    "r_Attributes" = 1;
    sync = syna;
}

After adjusting to a freely available video URL, the video plays fine.

In order to set the accurate size to the SKScene, the scene is created using the information obtained from the AVAssetTrack of the player: the dimensions of the video and the specified transformation (which could be CGAffineTransformIdentity). As a transform is applied the values can be negative so the final size is using absolute values. Besides that, to prevent exceptionally failures, the previous hardcoded size (1280x1280) was kept.

Hi,

Installed the 360 player via cocoapods and first things we saw was that the camera seems not to be centered correctly on the front facing side of the 360 video.

Saw a Pull request for it so we integrated the modification locally but it's not on cocoapods. Do you plan to release a new version soon?

Regards,

Kevin

https://jira.nyt.net/browse/MO-7073 https://jira.nyt.net/browse/MO-7078 https://jira.nyt.net/browse/MO-7079

• Splits allowed panning axes into two separate properties: one for device motion and another for pan gesture recognition. This is so that the updated design requirements for the newsreader can be satisfied.
• Adds a convenience method for reorienting the vertical component of the camera angle to the horizon, which is a necessary consequence of allowing vertical device motion panning while inline (per the updated newsreader design requirement).

This PR addresses several issues with compass angle calculations, and makes some small API alterations for clarity.

1. Refactors compass angle calculations so that the angle is always correct, even when the camera is pointed directly down or up. The essential change is to use the y component of the euler angles instead of world transform math, since the former requires only minimal manipulation to be useful as a compass angle. Unit tests have been added to ensure correctness. The newly-added C function also includes a reference angle parameter to lay the foundation for any long-term future changes that require a different reference angle from video-to-video.
2. Refactors how compass angle updates are triggered. The previous code only triggered in response to scene kit updates. The updated code is also triggered by pan gestures (there seemed to be cases where the former did not fire often enough on its own).
3. Renames "camera angle" properties and methods to use "compass angle" nomenclature, since that's closer to the mark.
4. Tweaks NYT360ViewControllerDelegate to more closely adhere to platform style.

When I try to play a 4K video from remote url, it does not play smoothly. It play with. stuck.Please help me. Remote Url:- http://13.127.138.63/woweapp/assets/uploads/episodes/video_1019013136.mp4

When running the example project on iOS 11.3 with Xcode 9.3, the Main Thread Checker detects that the UI API is called on a background thread instead of the main thread in updateCameraAngleForCurrentDeviceMotion() and Xcode reports -[UIApplication statusBarOrientation] must be used from main thread only on https://github.com/NYTimes/ios-360-videos/blob/develop/Sources/NYT360CameraController.m#L109.

After surrounding the code block by dispatch_async(dispatch_get_main_queue(), ^{ ... }); the warning no longer appears.

Main Thread Checker: UI API called on a background thread: -[UIApplication statusBarOrientation]
PID: 6294, TID: 2952262, Thread name: (none), Queue name: com.apple.scenekit.renderingQueue.SCNView0x10bd0c930, QoS: 33
Backtrace:
4   NYT360Video                         0x00000001047c9a48 -[NYT360CameraController updateCameraAngleForCurrentDeviceMotion] + 384
5   NYT360Video                         0x00000001047c8ea8 -[NYT360ViewController renderer:updateAtTime:] + 116
6   SceneKit                            0x0000000196c5af80 <redacted> + 444
7   SceneKit                            0x0000000196c5d018 <redacted> + 252
8   SceneKit                            0x0000000196c5d69c <redacted> + 84
9   SceneKit                            0x0000000196c5daa0 <redacted> + 728
10  SceneKit                            0x0000000196d0446c <redacted> + 512
11  SceneKit                            0x0000000196bc07dc <redacted> + 56
12  SceneKit                            0x0000000196ccd388 <redacted> + 80
13  libdispatch.dylib                   0x00000001047e9260 _dispatch_call_block_and_release + 24
14  libdispatch.dylib                   0x00000001047e9220 _dispatch_client_callout + 16
15  libdispatch.dylib                   0x00000001047f7e80 _dispatch_queue_serial_drain + 768
16  libdispatch.dylib                   0x00000001047ec730 _dispatch_queue_invoke + 328
17  libdispatch.dylib                   0x00000001047f8dd8 _dispatch_root_queue_drain_deferred_wlh + 352
18  libdispatch.dylib                   0x00000001047ffebc _dispatch_workloop_worker_thread + 676
19  libsystem_pthread.dylib             0x0000000182643e70 _pthread_wqthread + 860
20  libsystem_pthread.dylib             0x0000000182643b08 start_wqthread + 4
2018-04-24 16:27:12.053023+0800 NYT360VideoExample[6294:2952262] [reports] Main Thread Checker: UI API called on a background thread: -[UIApplication statusBarOrientation]
PID: 6294, TID: 2952262, Thread name: (none), Queue name: com.apple.scenekit.renderingQueue.SCNView0x10bd0c930, QoS: 33
Backtrace:
4   NYT360Video                         0x00000001047c9a48 -[NYT360CameraController updateCameraAngleForCurrentDeviceMotion] + 384
5   NYT360Video                         0x00000001047c8ea8 -[NYT360ViewController renderer:updateAtTime:] + 116
6   SceneKit                            0x0000000196c5af80 <redacted> + 444
7   SceneKit                            0x0000000196c5d018 <redacted> + 252
8   SceneKit                            0x0000000196c5d69c <redacted> + 84
9   SceneKit                            0x0000000196c5daa0 <redacted> + 728
10  SceneKit                            0x0000000196d0446c <redacted> + 512
11  SceneKit                            0x0000000196bc07dc <redacted> + 56
12  SceneKit                            0x0000000196ccd388 <redacted> + 80
13  libdispatch.dylib                   0x00000001047e9260 _dispatch_call_block_and_release + 24
14  libdispatch.dylib                   0x00000001047e9220 _dispatch_client_callout + 16
15  libdispatch.dylib                   0x00000001047f7e80 _dispatch_queue_serial_drain + 768
16  libdispatch.dylib                   0x00000001047ec730 _dispatch_queue_invoke + 328
17  libdispatch.dylib                   0x00000001047f8dd8 _dispatch_root_queue_drain_deferred_wlh + 352
18  libdispatch.dylib                   0x00000001047ffebc _dispatch_workloop_worker_thread + 676
19  libsystem_pthread.dylib             0x0000000182643e70 _pthread_wqthread + 860
20  libsystem_pthread.dylib             0x0000000182643b08 start_wqthread + 4

I'm using NYT360Video to play 360 content. But I'm facing problem while playing downloaded 360 content. Downloaded content is not playing and only audio is coming. We are using Apple FailPlay Streaming download technique. Does anyone gone through this kind of issue, or any suggestions ?

https://stackoverflow.com/questions/47375426/downloaded-content-is-not-playing-in-nyt360video-player

Hi,

We're playing locally stored videos. We tested with good quality video (found on VLC's website and on the web). Some are 400mb 4K video.

Compared to the experience on VLC on a Mac with the same videos, the quality of the rendering seems bad.

Tested on an iPhone 7 Plus and on an iPad Mini 2. It may have something to do with the @2x or @3x scale.

Thanks for your feedback.

Kevin

The following should be submitted to Apple as a bug radar.

Bug

Setting an SKScene as the contents of an SCNMaterialProperty leads to numerous retain cycles in internal SceneKit and SpriteKit classes.

Steps to Reproduce

1. Set up an SKScene with a single SKVideoNode child.
2. Set up an SCNNode using a primitive geometry.
3. Assign the SKScene as the contents of the SCNNode’s geometry.firstMaterial.diffuse.contents.
4. Relinquish all references to the SKScene and SCNNode.

Expected Behavior

The SKScene and the SCNNode should be deallocated.

Actual Result

The SKScene and many other instances are leaked due to numerous retain cycles.

Notes

Please run the attached sample code on iOS 10. You can see the retain cycles plainly illustrated using Xcode 8's visual memory debugger. The Leaks instrument will also show some of these leaks when running on a real device running iOS 10. I have not been able to get Instruments to report a leak on iOS 9, and intermittently on iOS 10. However, the visual memory debugger in Xcode 8 always reveals the retain cycles.

You can verify that SCNMaterialProperty.contents is the culprit by commenting out Line 63 in PlayerScene.m.

Uploads

Sample Project: https://www.dropbox.com/s/1j0ib2ojbr1xbym/Memory%20Leak.zip?dl=0

Screenshot