Running Swift automatic differentiation on iOS

Overview

Differentiation Demo

This is an example of Swift's automatic differentiation running on iOS. It is a modified version of the game from ARHeadsetKit tutorial #8, where the user knocks out cubes with their hand. Automatic differentiation computes a cube's velocity from its equations of motion, then shows the velocity in text.

Differentiation is disabled in the Swift 5.5 toolchain due to compiler instability, but can be activated by importing the Differentiation package. The following code segment shows how differentiation is used to find the vertical component of a cube's velocity:

import Differentiation

let cube = cubes[...]

@differentiable(reverse)
func getLocationY(t: Float) -> Float {
    let acceleration: SIMD3<Float> = [0, -9.8, 0]
    let a_contribution = acceleration.y / 2 * t * t

    return a_contribution + cube.velocity0.y * t + cube.location0.y
}

let dydt = gradient(at: cube.timeSinceCollision, of: getLocationY(t:))
let velocityText = String(format: "%.1f", dydt)
let messageText = "Velocity (Y): \(velocityText) m/s"

The complete implementation is located in Game/GameRendererExtensions.swift.

Rationale

The purpose of this demo is to make a case for the resurrection of Swift for TensorFlow, which relies on differentiation. Python ML libraries cannot run on iOS devices, yet real-time machine learning makes apps more intelligent. With stable automatic differentiation and a Metal GPU backend, new opportunities and flexible workflows could be unlocked for mobile app developers.

Video of App

differentiation-video.mp4

For those with a slow internet connection:

Text in augmented reality showing that Y velocity is -17.7 meters per second

If you have any insights or suggestions about this project, please comment on the Swift Forums post.

You might also like...
Ezviz IoT Home SDK iOS Demo.
Ezviz IoT Home SDK iOS Demo.

SDK Demo SDK Demo 演示了萤石 App SDK 的接入流程以及 SDK 开放的功能,因此通过参考 Demo 可以基本解决再接入过程中碰到的问题。 Demo 模块介绍: 登录模块:包含注册,登录,忘记密码功能 家庭模块:包含添加家庭,删除家庭,修改家庭信息,邀请家庭成员和移除家庭成员等

[yolov5] + [ios] + [tensorflow lite]

YOLOv5 - TensorFlow Lite Object Detection iOS Example Application iOS Versions Supported: iOS 12.0 and above. Xcode Version Required: 10.0 and above O

Resource monitor - A flutter plugin for Android and IOS to monitor CPU and RAM usage of device.

resource_monitor A flutter plugin for Android and IOS to monitor CPU and RAM usage of device. TODO Implement Android Side of this plugin. Add listener

Slot-machine - SwiftUI - iOS App Slot Machine game
Slot-machine - SwiftUI - iOS App Slot Machine game

Slot Machine SwiftUI Masterclass project iPhone, iPad and Mac (with Catalyst framework) Complex interface Extensions and custom View Modifiers Input t

Hand-gesture recognition on iOS app using CoreML
Hand-gesture recognition on iOS app using CoreML

GestureAI-CoreML-iOS Hand-gesture recognizer using CoreML Demo Screenshots This app is using RNN(Recurrent Neural network) with CoreML on iOS11. The m

Tiny YOLO for iOS implemented using CoreML but also using the new MPS graph API.
Tiny YOLO for iOS implemented using CoreML but also using the new MPS graph API.

YOLO with Core ML and MPSNNGraph This is the source code for my blog post YOLO: Core ML versus MPSNNGraph. YOLO is an object detection network. It can

Gyros-identifier - An iOS app capable of detecting whether a photo contains a gyros or not
Gyros-identifier - An iOS app capable of detecting whether a photo contains a gyros or not

gyros-identifier 🍽️ In case you enter a shady restaurant, with suspicious price

Pose Estimation on iOS with TensorFlow Lite
Pose Estimation on iOS with TensorFlow Lite

This project is Pose Estimation on iOS with TensorFlow Lite. If you are interested in iOS + Machine Learning, visit here you can see various DEMOs. 2D

Spokestack: give your iOS app a voice interface!
Spokestack: give your iOS app a voice interface!

Spokestack provides an extensible speech recognition pipeline for the iOS platform. It includes a variety of built-in speech processors for Voice Acti

Owner
Philip Turner
Spreading news that anyone can use $5 Google Cardboard to replicate $3,500 Microsoft Hololens
Philip Turner
Automatic spoken language identification (LID) using deep learning.

iLID Automatic spoken language identification (LID) using deep learning. Motivation We wanted to classify the spoken language within audio files, a pr

Thomas Werkmeister 85 Apr 3, 2022
Automatic colorization using deep neural networks. Colorful Image Colorization. In ECCV, 2016.

Colorful Image Colorization [Project Page] Richard Zhang, Phillip Isola, Alexei A. Efros. In ECCV, 2016. + automatic colorization functionality for Re

Richard Zhang 3k Dec 27, 2022
Mobile-ios-ml - SBB Mobile Machine Learning for iOS devices

ESTA library: Machine Learning for iOS This framework simplifies the integration

Swiss Federal Railways (SBB) 9 Jul 16, 2022
Artificial intelligence/machine learning data structures and Swift algorithms for future iOS development. bayes theorem, neural networks, and more AI.

Swift Brain The first neural network / machine learning library written in Swift. This is a project for AI algorithms in Swift for iOS and OS X develo

Vishal 331 Oct 14, 2022
This repo contains beginner examples to advanced in swift. Aim to create this for learning native iOS development.

iOS-learning-with-swift-22 This repo contains beginner examples to advanced in swift. Aim to create this for learning native iOS development. Oh, you

Umesh Jangid 0 Jan 9, 2022
A Swift library for creating and exporting CoreML Models in Swift

SwiftCoreMLTools A Swift Library for creating CoreML models in Swift. Work in progress This library expose a (function builder based) DSL as well as a

Jacopo Mangiavacchi 140 Dec 5, 2022
Largest list of models for Core ML (for iOS 11+)

Since iOS 11, Apple released Core ML framework to help developers integrate machine learning models into applications. The official documentation We'v

Kedan Li 5.6k Jan 3, 2023
BetterMood is an iOS app that uses Tensorflow to recognize user’s emotions

BetterMood is an iOS app that uses Tensorflow to recognize user’s emotions, convert it into categories then send via our api along with the user’s date of birth and name, to end up with a emotion analyse and horoscope prediction.

Yosri 2 Sep 30, 2021
Easily craft fast Neural Networks on iOS! Use TensorFlow models. Metal under the hood.

Bender Bender is an abstraction layer over MetalPerformanceShaders useful for working with neural networks. Contents Introduction Why did we need Bend

xmartlabs 1.7k Dec 24, 2022
Model stock prediction for iOS

Stockify Problem Investing in Stocks is great way to grow money Picking the right stocks for you can get tedious and confusing Too many things to foll

Sanchitha Dinesh 1 Mar 20, 2022