Convert Torch7 models into Apple CoreML format

Last update: Nov 11, 2022

Overview

Convert Torch7 models into Apple CoreML format.

This tool helps convert Torch7 models into Apple CoreML format which can then be run on Apple devices.

Installation

pip install -U torch2coreml

In order to use this tool you need to have these installed:

Xcode 9
python 2.7

If you want to run tests, you need MacOS High Sierra 10.13 installed.

Dependencies

coremltools (0.6.2+)
PyTorch

How to use

Using this library you can implement converter for your own model types. An example of such a converter is located at "example/fast-neural-style/convert-fast-neural-style.py". To implement converters you should use single function "convert" from torch2coreml:

from torch2coreml import convert

This function is simple enough to be self-describing:

def convert(model,
            input_shapes,
            input_names=['input'],
            output_names=['output'],
            mode=None,
            image_input_names=[],
            preprocessing_args={},
            image_output_names=[],
            deprocessing_args={},
            class_labels=None,
            predicted_feature_name='classLabel',
            unknown_layer_converter_fn=None)

Parameters

model: Torch7 model (loaded with PyTorch) | str
A trained Torch7 model loaded in python using PyTorch or path to file with model (*.t7).

input_shapes: list of tuples Shapes of the input tensors.

mode: str ('classifier', 'regressor' or None)
Mode of the converted coreml model:
'classifier', a NeuralNetworkClassifier spec will be constructed.
'regressor', a NeuralNetworkRegressor spec will be constructed.

preprocessing_args: dict
'is_bgr', 'red_bias', 'green_bias', 'blue_bias', 'gray_bias', 'image_scale' keys with the same meaning as https://apple.github.io/coremltools/generated/coremltools.models.neural_network.html#coremltools.models.neural_network.NeuralNetworkBuilder.set_pre_processing_parameters

deprocessing_args: dict
Same as 'preprocessing_args' but for deprocessing.

class_labels: A string or list of strings.
As a string it represents the name of the file which contains the classification labels (one per line). As a list of strings it represents a list of categories that map the index of the output of a neural network to labels in a classifier.

predicted_feature_name: str
Name of the output feature for the class labels exposed in the Core ML model (applies to classifiers only). Defaults to 'classLabel'

unknown_layer_converter_fn: function with signature:
(builder, name, layer, input_names, output_names)
builder: object - instance of NeuralNetworkBuilder class
name: str - generated layer name
layer: object - PyTorch (python) object for corresponding layer
input_names: list of strings
output_names: list of strings
Returns: list of strings for layer output names
Callback function to handle unknown for torch2coreml layers

Returns

model: A coreml model.

Currently supported

Models

Only Torch7 "nn" module is supported now.

Layers

List of Torch7 layers that can be converted into their CoreML equivalent:

Sequential
ConcatTable
SpatialConvolution
ELU
ReLU
SpatialBatchNormalization
Identity
CAddTable
SpatialFullConvolution
SpatialSoftMax
SpatialMaxPooling
SpatialAveragePooling
View
Linear
Tanh
MulConstant
SpatialZeroPadding
SpatialReflectionPadding
Narrow
SpatialUpSamplingNearest
SplitTable

License

Use of this source code is governed by the MIT License that can be found in the LICENSE.txt file.

Comments

Loading Model error : torch.utils.serialization.read_lua_file.T7ReaderException

After training a model from the example and trying to convert the model to core ml

this error appears

File "convert-fast-neural-style.py", line 53, in <module>
   main()
 File "convert-fast-neural-style.py", line 39, in main
   'blue_bias': 103.939
 File "/usr/local/lib/python2.7/dist-packages/torch2coreml/_torch_converter.py", line 171, in convert
   torch_model = load_lua(model)
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 599, in load_lua
   return reader.read()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 586, in read
   return self.read_table()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 514, in wrapper
   result = fn(self, *args, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 563, in read_table
   v = self.read()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 584, in read
   return self.read_object()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 514, in wrapper
   result = fn(self, *args, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 537, in read_object
   return reader_registry[cls_name](self, version)
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 242, in read_nn_class
   attributes = reader.read()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 586, in read
   return self.read_table()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 514, in wrapper
   result = fn(self, *args, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 563, in read_table
   v = self.read()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 586, in read
   return self.read_table()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 514, in wrapper
   result = fn(self, *args, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 563, in read_table
   v = self.read()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 584, in read
   return self.read_object()
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 514, in wrapper
   result = fn(self, *args, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/torch/utils/serialization/read_lua_file.py", line 543, in read_object
   "constructor").format(cls_name))
torch.utils.serialization.read_lua_file.T7ReaderException: don't know how to deserialize Lua class nn.InstanceNormalization. If you want to ignore this error and load this object as a dict, specify unknown_classes=True in reader's constructor

opened by engahmed1190 9

TypeError: convert() got an unexpected keyword argument 'input_name'

when running setup.sh an error appears

Traceback (most recent call last):
  File "convert-fast-neural-style.py", line 54, in <module>
    main()
  File "convert-fast-neural-style.py", line 39, in main
    'blue_bias': 103.939
TypeError: convert() got an unexpected keyword argument 'input_name'
Traceback (most recent call last):
  File "convert-fast-neural-style.py", line 54, in <module>
    main()
  File "convert-fast-neural-style.py", line 39, in main
    'blue_bias': 103.939
TypeError: convert() got an unexpected keyword argument 'input_name'
Traceback (most recent call last):
  File "convert-fast-neural-style.py", line 54, in <module>
    main()
  File "convert-fast-neural-style.py", line 39, in main
    'blue_bias': 103.939
TypeError: convert() got an unexpected keyword argument 'input_name'
Traceback (most recent call last):
  File "convert-fast-neural-style.py", line 54, in <module>
    main()
  File "convert-fast-neural-style.py", line 39, in main
    'blue_bias': 103.939
TypeError: convert() got an unexpected keyword argument 'input_name'

environment contains

adium-theme-ubuntu (0.3.4) coremltools (0.6.3) decorator (4.0.6) h5py (2.7.0) ipython (2.4.1) numpy (1.13.1) pexpect (4.0.1) pip (9.0.1) protobuf (3.4.0) ptyprocess (0.5) pycurl (7.43.0) python-apt (1.1.0b1) PyYAML (3.12) setuptools (36.3.0) simplegeneric (0.8.1) six (1.10.0) torch (0.2.0.post3) torch2coreml (0.1.0) unity-lens-photos (1.0) virtualenv (15.1.0) wheel (0.29.0)

opened by engahmed1190 7

High Memory Usage in iOS

I try the example iOS code provided by this repo.

https://github.com/prisma-ai/torch2coreml/tree/master/example/fast-neural-style/ios

The coreml model is only ~10MB. However, the memory usage is very high, usually up to 200MB, which causes the app to terminated due to memory issue sometimes.

opened by guanjhensu 2

Install error: PyTorch does not currently provide packages for PyPI

I'm just getting into machine learning, and I wanted to try converting something into the coreml format. I wrote a very simple torch program which (not complete, I don't have access to the latest one right now) was something like this:

import numpy as np
import torch
from torch.autograd import Variable
model = torch.nn.Linear(1, 1)
loss_fn = torch.nn.MSELoss(size_average=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
for t in range(10000):
    x = Variable(torch.from_numpy(np.random.random((1,1)).astype(np.float32)))
    y = x * 3
    y_pred = model(x)
    loss = loss_fn(y_pred, y)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print loss.data[0]

torch.save... # some more code here

Then I installed Anaconda and PyTorch, along with coremltools, but when I run pip2 install -U torch2coreml I get the error:

Collecting torch2coreml
  Using cached torch2coreml-0.2.0-py2.7-none-any.whl
Collecting torch (from torch2coreml)
  Using cached torch-0.1.2.post1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/_5/xy3fwjvx6h9fj8my9t_lnl_c0000gn/T/pip-build-EpKgGE/torch/setup.py", line 11, in <module>
        raise RuntimeError(README)
    RuntimeError: PyTorch does not currently provide packages for PyPI (see status at https://github.com/pytorch/pytorch/issues/566).

    Please follow the instructions at http://pytorch.org/ to install with miniconda instead.


    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/_5/xy3fwjvx6h9fj8my9t_lnl_c0000gn/T/pip-build-EpKgGE/torch/

I'm probably making some silly mistake here, so any help is appreciated. Thanks!

opened by perryprog 2

unable to import mlmodel to ios

when i try to import mlmodel, for example candy.mlmodel, to xcode, I got a validation error saying:

"There was a problem in decoding this coreml document" validator error: In layer instancenormalization_5: incorrect mean size 0

opened by theheena 2

Exception: Model prediction is only supported on macOS version 10.13

Is there any way to run those models on Linux

Traceback (most recent call last):
  File "stylize-image.py", line 28, in <module>
    main()
  File "stylize-image.py", line 21, in main
    stylized_image = net.predict({'inputImage': image})['outputImage']
  File "/usr/local/lib/python2.7/dist-packages/coremltools/models/model.py", line 244, in predict
    raise Exception('Model prediction is only supported on macOS version 10.13.')
Exception: Model prediction is only supported on macOS version 10.13.

opened by engahmed1190 2

Error while trying to create coreml model

I have followed all the instructions to create a model from "http://cs.stanford.edu/people/jcjohns/fast-neural-style/models/instance_norm/mosaic.t7" I have created my own style model, but when I tried to convert to coreml, I am getting this error. I thought my model file is wrong, but I dowloaded this file "http://cs.stanford.edu/people/jcjohns/fast-neural-style/models/instance_norm/mosaic.t7" and tried with that. The result is same. No luck. Could you provide me a solution to fix that?

Traceback (most recent call last): File "convert-fast-neural-style.py", line 175, in main() File "convert-fast-neural-style.py", line 161, in main unknown_layer_converter_fn=convert_instance_norm File "/usr/local/lib/python2.7/dist-packages/torch2coreml/_torch_converter.py", line 192, in convert with torch.legacy.nn.Sequential module as root" TypeError: Model must be file path to .t7 file or pytorch loaded model with torch.legacy.nn.Sequential module as root

opened by girginsoft 1
Cannot use convert-fast-neural-style.py to convert models to CoreML

Hello, I am using the script given in the example fast-neural-style. Prepare models worked as expected but when it comes to conversion, the error occurred. Don't know how to figure this out.

input: python convert-fast-neural-style.py -input prepared_models/candy.t7 -output coreml_models/candy.mlmodel

output: 2018-01-20 12:58:11.787 python[23363:3203919] +[MLModel compileModelAtURL:error:]: unrecognized selector sent to class 0x7fff460e58e8 Traceback (most recent call last): File "convert-fast-neural-style.py", line 176, in main() File "convert-fast-neural-style.py", line 162, in main unknown_layer_converter_fn=convert_instance_norm File "/Users/vega/workspace/virtualenv/lib/python2.7/site-packages/torch2coreml/_torch_converter.py", line 294, in convert return MLModel(builder.spec) File "/Users/vega/workspace/virtualenv/lib/python2.7/site-packages/coremltools/models/model.py", line 153, in init self.proxy = _get_proxy_from_spec(filename) File "/Users/vega/workspace/virtualenv/lib/python2.7/site-packages/coremltools/models/model.py", line 77, in _get_proxy_from_spec return _MLModelProxy(filename) RuntimeError: Caught an unknown exception!

opened by wei20 1
Missing Style Transfer models in IOS test application

There are no coreml_models in styleTransfer IOS application. I tried to download those mosaic and others models . For instance i have downloaded FNS-Mosaic model and included in the code but while running that i am getting error " unexpectedly found nil while unwrapping an Optional value" in line 81 of FNS-Mosaic. Many Thanks.

opened by itsviral 1

Preprocess an image before passing to a ResNet model

I noticed that the preprocessing and the deprocessing only worked for the VGG model, and it also seems true for Apple's coremltools. In the original fast-neural-style code , the ResNet preprocess is done as

function M.resnet.preprocess(img)
  check_input(img)
  local mean = img.new(resnet_mean):view(1, 3, 1, 1):expandAs(img)
  local std = img.new(resnet_std):view(1, 3, 1, 1):expandAs(img)
  return (img - mean):cdiv(std)
end

while for VGG is done through

function M.vgg.preprocess(img)
  check_input(img)
  local mean = img.new(vgg_mean):view(1, 3, 1, 1):expandAs(img)
  local perm = torch.LongTensor{3, 2, 1}
  return img:index(2, perm):mul(255):add(-1, mean)
end

The difference is that for the ResNet model the image is in the range [0, 1] while for the VGG it is in the range [0, 255].

In your example, to use coremltools' API you defined the preprocessing and deprocessing as

    coreml_model = convert(
        model,
        [input_shape],
        input_names=['inputImage'],
        output_names=['outputImage'],
        image_input_names=['inputImage'],
        preprocessing_args={
            'is_bgr': True,
            'red_bias': -123.68,
            'green_bias': -116.779,
            'blue_bias': -103.939
        },
        image_output_names=['outputImage'],
        deprocessing_args={
            'is_bgr': True,
            'red_bias': 123.68,
            'green_bias': 116.779,
            'blue_bias': 103.939
        },
        unknown_layer_converter_fn=convert_instance_norm
    )

which is natural for the VGG model.

I wonder if there is a way to do the same thing for the ResNet model.

Thanks.

opened by jianpingliu 1

TypeError: Model must be file path to .t7 file or pytorch loaded model

This error simply shows up on executing only python convert-fast-neural-style.py -input models/candy.t7 -output ./candy.mlmodel

Please help me. I am not able to convert Torch-trained models in CoreML. Any help is highly appreciated.

opened by RahulBhalley 0
ValueError: expected 5D input (got 4D input)

Hello!,how to solve this problem,thank you very much!

Preparing models for conversion Converting models to CoreML Converting prepared_models/candy.t7 Traceback (most recent call last): File "convert-fast-neural-style.py", line 176, in main() File "convert-fast-neural-style.py", line 162, in main unknown_layer_converter_fn=convert_instance_norm File "/home/sharp/.local/lib/python2.7/site-packages/torch2coreml/_torch_converter.py", line 211, in convert input_shapes File "/home/sharp/.local/lib/python2.7/site-packages/torch2coreml/_torch_converter.py", line 67, in _infer_torch_output_shapes is_batch=True File "/home/sharp/.local/lib/python2.7/site-packages/torch2coreml/_torch_converter.py", line 30, in _forward_torch_random_input result = torch_model.forward(input_tensors[0]) File "/home/sharp/.local/lib/python2.7/site-packages/torch/legacy/nn/Module.py", line 33, in forward return self.updateOutput(input) File "/home/sharp/.local/lib/python2.7/site-packages/torch/legacy/nn/Sequential.py", line 36, in updateOutput currentOutput = module.updateOutput(currentOutput) File "convert-fast-neural-style.py", line 45, in updateOutput return self._instance_norm.forward(Variable(input)).data File "/home/sharp/.local/lib/python2.7/site-packages/torch/nn/modules/instancenorm.py", line 46, in forward self._check_input_dim(input) File "/home/sharp/.local/lib/python2.7/site-packages/torch/nn/modules/instancenorm.py", line 242, in _check_input_dim .format(input.dim())) ValueError: expected 5D input (got 4D input)

opened by usingcsharp 4
How to use Trained model in Pytorch?

Hi I have trained a model using images. Now I want to test it with a single image.How to use my model weights. Its saved as a pth file. I'm new to python and deep learning. I used to work on R. In R once the model is trained, model.predict / predict command will give us the predictions. How its done in Pytorch?

opened by Karthik-89 0

setup.sh fails when converting to CoreML.

Hi, I am trying to setup your project and setup.sh script fails with next error for every model:

Preparing models for conversion
/Users/teologov/torch/install/bin/lua: /Users/teologov/torch/install/share/lua/5.2/torch/File.lua:375: unknown object
stack traceback:
        [C]: in function 'error'
        /Users/teologov/torch/install/share/lua/5.2/torch/File.lua:375: in function 'readObject'
        /Users/teologov/torch/install/share/lua/5.2/torch/File.lua:409: in function 'load'
        prepare_model.lua:29: in function 'main'
        prepare_model.lua:46: in main chunk
        [C]: in function 'dofile'
        ...ogov/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
        [C]: in ?

I tried manually to load the model with th> torch.load('./fast-neural-style/models/instance_norm/la_muse.t7').model and it still fails:

/Users/teologov/torch/install/share/lua/5.2/torch/File.lua:375: unknown object
stack traceback:
        /Users/teologov/torch/install/share/lua/5.2/trepl/init.lua:506: in function </Users/teologov/torch/install/share/lua/5.2/trepl/init.lua:499>
        [C]: in function 'error'
        /Users/teologov/torch/install/share/lua/5.2/torch/File.lua:375: in function 'readObject'
        /Users/teologov/torch/install/share/lua/5.2/torch/File.lua:409: in function 'load'
        [string "_RESULT={torch.load('./fast-neural-style/mode..."]:1: in main chunk
        [C]: in function 'xpcall'
        /Users/teologov/torch/install/share/lua/5.2/trepl/init.lua:661: in function 'repl'
        ...ogov/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:204: in main chunk
        [C]: in ?

opened by teologov 5

Issue with CycleGAN models

i'm trying to convert one of the CycleGAN pre trained model with 'convert-fast-neural-style.py'. i got the output model but when i used it in my app, not getting the expected output. I guess the issue with arguments provided inside the 'convert-fast-neural-style.py'. is there any way i can inspect a pre trained model to find the correct arguments for the conversion.

Model Used : https://people.eecs.berkeley.edu/~taesung_park/CycleGAN/models/apple2orange_cpu.t7

opened by devajith 1
iOS Usage Issue

I downloaded the models from https://coreml.store/fns-candy but get 'Unexpectedly found nil while unwrapping an optional value' on line 81 of FNS_Candy.swift

opened by raginggoat 0
Commercial use of models

Hi! First of all, congratulations for this project.

I have a simple question about the use of it. Considering that it is a open source project and in the short tutorial it was mentioned that all models are open source as well, I'm a little confused because all Justin Johnson’s pre-trained models are marked as free for personal or research use. If the models are not open source, would you suggest any other open source models in torch7 that are compatible? Thank you in advance.

opened by wesleymds 2

Owner

Prisma Labs

GitHub

The MobileNet neural network using Apple's new CoreML framework

MobileNet with CoreML This is the MobileNet neural network architecture from the paper MobileNets: Efficient Convolutional Neural Networks for Mobile

698 Dec 4, 2022

Real-Time image recognition for iOS with Vision(CoreML) and InceptionV3

Kesan-ML-iOS Real-Time image recognition for iOS with CoreML and InceptionV3  Test Flight Description Real-Time image recognition Integrating app wit

3 Jun 26, 2022

PyTorch to CoreML: Writing custom layers with Metal

pytorch-coreml-custom-layer-example PyTorch to CoreML: Writing custom layers with Metal Convert PyTorch model cd Convert python3 -m pip install -r req

7 Sep 29, 2022

Converted CoreML Model Zoo.

CoreML-Models Converted CoreML Model Zoo. CoreML is a machine learning framework by Apple. If you are iOS developer, you can easly use machine learnin

483 Jan 5, 2023

Holistically-Nested Edge Detection (HED) using CoreML and Swift

HED-CoreML Holistically-Nested Edge Detection (HED) using CoreML and Swift This is the repo for tutorial, that contains an example application that ru

101 Dec 25, 2022

A CoreML model which classifies images of food

Food101 for CoreML Description This is the Food101 dataset implemented in Apple's new framework called CoreML. The Food101 dataset can predict foods f

133 Nov 17, 2022

Predict handwritten digits with CoreML

MNIST for CoreML (CNN) Description This is the MNIST dataset implemented in Apple's new framework CoreML. The MNIST dataset can predict handwritten (d

63 Nov 17, 2022

TextDetection-CoreML - This project is Text Detection on iOS using Vision built-in model

This project is Text Detection on iOS using Vision built-in model. If you are interested in iOS + Machine Learning, visit here yo

61 Nov 17, 2022

ImageClassification - Usage The MobileNetv2 CoreML Model with StoryBoard

Image Classification with The MobileNetV2 MobileNetV2: Inverted Residuals and Li

0 Jun 25, 2022

Gloth - Generates a CoreML Word Tagger Model trained on interactive fiction commands

Gloth Gloth: Fold dough 83 times Generates a CoreML Word Tagger Model trained on

0 Feb 12, 2022

Text Classifier App for iOS, powered by Apple Vision & CreateML framework

2 Sep 15, 2022

Apple Developer Academy, Nano Challenge_2 Project

How Old Am I Key Features • Authors • Screenshots • Skills & Tech Stack • License ?? Project Title AI 나이측정 테스트, How Old Am I Key Features 얼굴 인식 선택된 이미

2 Sep 14, 2022

Style Art library process images using COREML with a set of pre trained machine learning models and convert them to Art style.

StyleArt Style Art is a library that process images using COREML with a set of pre trained machine learning models and convert them to Art style. Prev

222 Dec 17, 2022

JSONHelper - ✌ Convert anything into anything in one operation; JSON data into class instances, hex strings into UIColor/NSColor, y/n strings to booleans, arrays and dictionaries of these; anything you can make sense of!

JSONHelper Convert anything into anything in one operation; hex strings into UIColor/NSColor, JSON strings into class instances, y/n strings to boolea

788 Jul 19, 2022

Discover, download, compile & launch different image processing & style transfer CoreML models on iOS.

⚠️ ⚠️ ⚠️ IMPORTANT: I'm no longer maintaining Awesome-ML. Awesome ML is an iOS app that is made to demonstrate different image processing CoreML model

171 Nov 8, 2022

A camera designed in Swift for easily integrating CoreML models - as well as image streaming, QR/Barcode detection, and many other features

Would you like to use a fully-functional camera in an iOS application in seconds? Would you like to do CoreML image recognition in just a few more sec

868 Dec 29, 2022

Source code related to blog post about creating custom parseable format styles in iOS 15 (and similar versions of tvOS, watchOS, and macOS). Blog post can be found at https://emptytheory.com/2021/08/14/creating-custom-parseable-format-styles-in-ios-15/

Project Title Custom Parseable Format Styles Description With iOS 15 (and the latest versions for watch, iPad, etc.), we can now convert certain value

13 Oct 28, 2022