PyTorch to CoreML: Writing custom layers with Metal

Overview

pytorch-coreml-custom-layer-example

PyTorch to CoreML: Writing custom layers with Metal

Convert PyTorch model

cd Convert
python3 -m pip install -r requirements.txt
python3 convert.py -o /path/to/output/resulting/mlmodel/file

Run the demo app

Open TorchCoreMLDemo.xcodeproj in Xcode, build and run the app.

Shader

The code of the grid sample Metal-shader is here.

More information about grid sample (warper) algorithm you can find here.

You might also like...
Photo Assessment using Core ML and Metal.
Photo Assessment using Core ML and Metal.

PhotoAssessment Photo Assessment (i.e. quality score) using Core ML and Metal. ๐Ÿ“š Article ไฝฟ็”จ Metal ๅ’Œ Core ML ่ฏ„ไปท็…ง็‰‡่ดจ้‡ Parallel Computation using MPS ๐Ÿ”ฎ

๐Ÿค— Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
๐Ÿค— Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

English | ็ฎ€ไฝ“ไธญๆ–‡ | ็น้ซ”ไธญๆ–‡ | ํ•œ๊ตญ์–ด State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow ๐Ÿค— Transformers provides thousands of pretrained models

CoreML-Face-Parsing - how to use face-parsing CoreML model in iOS
CoreML-Face-Parsing - how to use face-parsing CoreML model in iOS

CoreML-Face-Parsing The simple sample how to use face-parsing CoreML model in iO

A powerful SwiftUI Architecture that merges Redux to the functional world of Swift. While bringing powerful workflows to streamline CoreML/Metal/IPFS usage in the Apple ecosystem.
A powerful SwiftUI Architecture that merges Redux to the functional world of Swift. While bringing powerful workflows to streamline CoreML/Metal/IPFS usage in the Apple ecosystem.

GraniteUI - v0.0 - WIP A powerful SwiftUI Architecture that merges Redux event handling and state management with functional programming. While bringi

Swift Package Manager plug-in to compile Metal files that can be debugged in Xcode Metal Debugger.

MetalCompilerPlugin Swift Package Manager plug-in to compile Metal files that can be debugged in Xcode Metal Debugger. Description Swift Package Manag

A Swift library to take the power of UIView.animateWithDuration(_:, animations:...) to a whole new level - layers, springs, chain-able animations and mixing view and layer animations together!
A Swift library to take the power of UIView.animateWithDuration(_:, animations:...) to a whole new level - layers, springs, chain-able animations and mixing view and layer animations together!

ver 2.0 NB! Breaking changes in 2.0 - due to a lot of requests EasyAnimation does NOT automatically install itself when imported. You need to enable i

Store and retrieve Codable objects to various persistence layers, in a couple lines of code!
Store and retrieve Codable objects to various persistence layers, in a couple lines of code!

tl;dr You love Swift's Codable protocol and use it everywhere, who doesn't! Here is an easy and very light way to store and retrieve Codable objects t

Concept for organizing View and Data layers within SwiftUI App
Concept for organizing View and Data layers within SwiftUI App

SwiftUI MVVM Concept It is not about how to create Lists and Charts and View design. It's about possibility how to organize View and Data layers withi

YiVideoEditor is a library for rotating, cropping, adding layers (watermark) and as well as adding audio (music) to the videos.

YiVideoEditor YiVideoEditor is a library for rotating, cropping, adding layers (watermark) and as well as adding audio (music) to the videos. YiVideoE

Tool to debug layouts directly on iOS devices: inspect layers in 3D and debug each visible view attributes
Tool to debug layouts directly on iOS devices: inspect layers in 3D and debug each visible view attributes

Introduction Features Inspect layouts directly on iOS devices Inspection could be triggered only if app is running under DEBUG build configuration, so

โœ‚ Easy to use and flexible library for manually laying out views and layers for iOS and tvOS. Supports AsyncDisplayKit.

ManualLayout Table of Contents Installation Usage API Cheat Sheet Installation Carthage Add the following line to your Cartfile. github "isair/ManualL

An iOS library to route API paths to objects on client side with request, mapping, routing and auth layers

WANetworkRouting Developed and Maintained by ipodishima Founder & CTO at Wasappli Inc. Sponsored by Wisembly A routing library to fetch objects from a

React-native-image-generator - Library to generate images from layers
React-native-image-generator - Library to generate images from layers

react-native-image-generator Library for generate images from other images Insta

An awesome Swift HTTP library to rapidly create communication layers with API endpoints

An awesome Swift HTTP library to rapidly create communication layers with API endpoints

Typed key-value storage solution to store Codable types in various persistence layers with few lines of code!
Typed key-value storage solution to store Codable types in various persistence layers with few lines of code!

๐Ÿ—‚ Stores A typed key-value storage solution to store Codable types in various persistence layers like User Defaults, File System, Core Data, Keychain

A fully customisable subclass of the native UIControl which allows you to create beautiful buttons without writing any line of code.
A fully customisable subclass of the native UIControl which allows you to create beautiful buttons without writing any line of code.

A fully customisable subclass of the native UIControl which allows you to create beautiful buttons without writing any line of code. Preview You'll be

A 2048 game writing with SwiftUI.
A 2048 game writing with SwiftUI.

2048 Game (SwiftUI app) This is a simple game to demonstrate the new SwiftUI framework. Note that the game algorithm may have issues, and this is stil

A Swift-based API for reading from & writing to the Apple System Log (more commonly known somewhat inaccurately as
A Swift-based API for reading from & writing to the Apple System Log (more commonly known somewhat inaccurately as "the console")

CleanroomASL Notice: CleanroomASL is no longer supported The Apple System Log facility has been deprecated by Apple. As a result, we've deprecated Cle

A lightweight library for writing HTTP web servers with Swift

Taylor Disclaimer: Not actively working on it anymore. You can check out some alternatives Swift 2.0 required. Working with Xcode 7.1. Disclaimer: It

Comments
  • Question about `register_mil_to_nn_mapping`

    Question about `register_mil_to_nn_mapping`

    I have read your work. It's really cool! I'm also working on coremltools recently and have some confusion, hoping you can help me. According to official guide about Custom Operators, it has not mentioned to set @register_mil_to_nn_mapping. But it seems like this is essential when custom a pytorch layer. I'm little confusing about how to set it, or, how to define it?

    opened by Tengxu-Sun 0
  • Process `grid` tensor by `builder.add_transpose` makes the function always run on CPU

    Process `grid` tensor by `builder.add_transpose` makes the function always run on CPU

    Hi, @dneprDroid I'm trying to implement a 5-D input tensor version of custom grid_sample based on your work. Now i meet two problems hoping you can help me. First, you said "we should transpose the grid to [n, 2, w, h] shape (for encoding it to a coreml 2-channel texture)". But i found this process may lead to the function always run on CPU in my implementation. The GPU function encode works without this process. I read your code but haven't find a solution. Have you met this problem ever?

    Second, since i want implement a 5-D input tensor version of custom grid_sample, how id<MTLTexture> in encode work at this time? I found the data storage is different with ordinary buffer, and i'm not sure how to read data in id<MTLTexture> so that the data can be calculated in later computation process. Thank you!

    opened by Tengxu-Sun 3
Owner
Aleksandr Ovechko
Aleksandr Ovechko
Converted CoreML Model Zoo.

CoreML-Models Converted CoreML Model Zoo. CoreML is a machine learning framework by Apple. If you are iOS developer, you can easly use machine learnin

MLBoy 483 Jan 5, 2023
Convert Torch7 models into Apple CoreML format

Convert Torch7 models into Apple CoreML format. Short tutorial This tool helps convert Torch7 models into Apple CoreML format which can then be run on

Prisma Labs 377 Nov 11, 2022
Holistically-Nested Edge Detection (HED) using CoreML and Swift

HED-CoreML Holistically-Nested Edge Detection (HED) using CoreML and Swift This is the repo for tutorial, that contains an example application that ru

Andrey Volodin 101 Dec 25, 2022
A CoreML models that detects nudity in a picture

Nudity Detection for CoreML Description This is the OpenNSFW dataset implemented in Apple's new framework called CoreML. The OpenNSFW dataset can pred

Philipp Gabriel 99 Oct 13, 2022
A CoreML model which classifies images of food

Food101 for CoreML Description This is the Food101 dataset implemented in Apple's new framework called CoreML. The Food101 dataset can predict foods f

Philipp Gabriel 133 Nov 17, 2022
Predict handwritten digits with CoreML

MNIST for CoreML (CNN) Description This is the MNIST dataset implemented in Apple's new framework CoreML. The MNIST dataset can predict handwritten (d

Philipp Gabriel 63 Nov 17, 2022
The MobileNet neural network using Apple's new CoreML framework

MobileNet with CoreML This is the MobileNet neural network architecture from the paper MobileNets: Efficient Convolutional Neural Networks for Mobile

Matthijs Hollemans 698 Dec 4, 2022
TextDetection-CoreML - This project is Text Detection on iOS using Vision built-in model

This project is Text Detection on iOS using Vision built-in model. If you are interested in iOS + Machine Learning, visit here yo

tucan9389 61 Nov 17, 2022
ImageClassification - Usage The MobileNetv2 CoreML Model with StoryBoard

Image Classification with The MobileNetV2 MobileNetV2: Inverted Residuals and Li

Hamit SEYREK 0 Jun 25, 2022
Gloth - Generates a CoreML Word Tagger Model trained on interactive fiction commands

Gloth Gloth: Fold dough 83 times Generates a CoreML Word Tagger Model trained on

Chris Sessions 0 Feb 12, 2022