Bender
Bender is an abstraction layer over MetalPerformanceShaders useful for working with neural networks.
Contents
- Introduction
- Why did we need Bender
- Basic usage
- Requirements
- Getting involved
- Examples
- Installation
- Changelog
The documentation can be found under the Documentation folder:
- API contains the most important information to get started.
- Supported Layers explains which layers are supported and how they map to TensorFlow ops.
- Importing explains how to import models from other frameworks such as TensorFlow. You can also find information on how to enhance this functionality for custom implementations.
Introduction
Bender is an abstraction layer over MetalPerformanceShaders which is used to work with neural networks. It is of growing interest in the AI environment to execute neural networks on mobile devices even if the training process has been done previously. We want to make it easier for everyone to execute pretrained networks on iOS.
Bender allows you to easily define and run neural networks using the most common layers like Convolution, Pooling, FullyConnected and some normalizations among others. It is also flexible in the way it receives the parameters for these layers.
We also want to support loading models trained on other frameworks such as TensorFlow or Caffe2. Currently Bender includes an adapter for TensorFlow that loads a graph with variables and "translates" it to Bender layers. This feature supports a subset of TensorFlow's operations but we plan to enhance it to cover more cases.
Why did we need Bender?
At Xmartlabs we were about to start a Machine Learning project and investigated frameworks to use in iOS. We found MetalPerformanceShaders useful but not very user friendly and we saw ourselves repeating a lot of code and information. That is why we starting building a framework to handle that kind of stuff.
We also found ourselves creating scripts to translate the models we had from training with TensorFlow to iOS. This means transposing the weights to the MPSCNN format and also mapping the parameters of the different kinds of layers in TensorFlow to the parameters used by the MPSCNN kernels. TensorFlow can be compiled for iOS but currently it does not support running on GPU which we wanted to do. We also did not want to include TensorFlow's static library into our project. This is why we also started to work on an adapter that would parse a TF graph and translate it to our Bender layers.
Usage
You can define your own network in Bender using our custom operator or you can load a model exported from TensorFlow. Defining a network and loading a model can be done like this:
import MetalBender
let url = Bundle.main.url(forResource: "myModel", withExtension: "pb")! // A TensorFlow model.
let network = Network.load(url: url, inputSize: LayerSize(h: 256, w: 256, f: 3))
network.run(input: /* ... */) { output in
// ...
}
You can read more information about this in Importing.
If you want to define your network yourself you can do it like this:
let network = Network(inputSize: LayerSize(h: 256, w: 256, f: 3))
network.start
->> Convolution(convSize: ConvSize(outputChannels: 16, kernelSize: 3, stride: 2))
->> InstanceNorm()
->> Convolution(convSize: ConvSize(outputChannels: 32, kernelSize: 3, stride: 2), neuronType: .relu)
->> InstanceNorm()
->> FullyConnected(neurons: 128)
->> Neuron(type: .tanh)
->> FullyConnected(neurons: 10)
->> Softmax()
// ...
and once you're done with all your layers:
network.initialize()
To know more about this have a look at API.
Requirements
- Xcode 9
- iOS 11.0+ (but deployment target is iOS 10.0, so iOS 10 is supported)
Getting involved
- If you want to contribute please feel free to submit pull requests.
- If you have a feature request please open an issue.
- If you found a bug or need help please check older issues, FAQ and threads on StackOverflow before submitting an issue.
Before contribute check the CONTRIBUTING file for more info.
If you use Bender in your app We would love to hear about it! Drop us a line on Twitter.
Examples
Follow these steps to run the examples:
- Clone Bender repository (or download it).
- Run
carthage update --platform iOSin the downloaded folder. - Open Bender workspace and run the Example project.
There is an Image recognition example which includes a MobileNet model in Bender and one in CoreML. It is also set up to run an Inception model but you will have to download it separately as it is almost 100 MB in size. You can download it from http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz but then you have to freeze it and add it to the 'Example' Xcode project as 'inception_v3.pb'.
Installation
CocoaPods
To install Bender, simply add the following line to your Podfile:
pod 'MetalBender', '~> 0.5'
Remember that Bender compiles for iOS 10. So you must add
platform :ios, '10.0'to your Podfile
Carthage
Carthage is a simple, decentralized dependency manager for Cocoa.
To install Bender, add the following line to your Cartfile:
github "xmartlabs/Bender" ~> 0.5
Then run:
carthage update --platform iOS
Finally, drag the built .framework binaries for MetalBender, MetalPerformanceShadersProxy and SwiftProtobuf to your application's Xcode project.
Author
Change Log
This can be found in the CHANGELOG.md file.
![Can't Build on xcode 8 [Macpods]](https://avatars.githubusercontent.com/u/1128542?v=4)
87 Dec 29, 2022
776 Dec 18, 2022
2 Dec 5, 2021
500 Oct 30, 2022
3k Dec 27, 2022
74.9k Dec 29, 2022
377 Jun 29, 2022
2 Sep 30, 2021
14 Dec 12, 2022
125 Nov 28, 2022
323 Oct 20, 2022
27 Dec 29, 2022
77.1k Dec 31, 2022
169 Dec 11, 2022
14 Oct 19, 2022
40 Jan 25, 2022
183 Jan 5, 2023
1.1k Dec 22, 2022