I'm trying to make Flutter based Audio Unit Extensions possible.

Overview

Flutter AUv3 Audio Unit Error Demo

Motivation

We are the developers of Audanika, a professional MIDI Controller app written in Flutter. Many of our users require us to support Audio Unit Extensions (AUV3) with Audanika.

https://www.youtube.com/watch?v=dV8p6LVJ3ms

Goal

We want to be the first Flutter based app, supporting Audio Unit Extensions.

Problems

Until now we were not able to show a Flutter view controller within an Audio Unit Extensions

Solution

We need support from the Flutter team, to help us to make FlutterViewController work as part of an Audio Unit Extension. Here are the steps to reproduce the issue:

1. Check out and prepare the repository

git clone [email protected]:gatzsche/flutter-audio-unit-extension-experiments.git
cd flutter-audio-unit-extension-experiments/
cd flutter_auv3_unit/
flutter pub get

2. Build and run the plugin container app

  • Open flutter_auv3_unit/ios/Runner.xcworkspace/ with Xcode.

  • Select target Runner

    INL010485.png

  • Run Runner on a chosen simulator

    INL010486.png

3. Build and run the plugin host app

  • Open AUV3Host/AUv3Host/AUv3Host.xcodeproj/ with Xcode

  • Select iOSAUv3Host as target

    INL010487.png

  • Run iOSAUv3Host on the same simulator the plugin container app was executed.

    INL010488.png

4. Debug the audio unit using the plugin host app

  • Switch back to flutter_auv3_unit/ios/Runner.xcworkspace/ Xcode project

  • Select audio unit target FlutterAUV3

    INL010489.png

  • Run the plugin on the same simulator as before

    INL010490.png

  • Select AUv3Host as the app to run

    INL010491.png

  • Within AUv3Host: select the Instruments tab

    INL010492.png

  • Select Flutter Example Audio Unit

    INL010493.png

  • Wait three seconds

  • Click on View

    INL010494.png

5. Here is the bug

  • Normally the Flutter default app should be shown
  • But you see only the red background of the FlutterViewController

6. Hints

  • The commandline output shows that flutter is executed and also the widget tree is built. Probably the rendering does not work properly.

    INL010497.png

You might also like...
AudioPlayer is a simple class for playing audio in iOS, macOS and tvOS apps.
AudioPlayer is a simple class for playing audio in iOS, macOS and tvOS apps.

AudioPlayer AudioPlayer is a simple class for playing audio in iOS, macOS and tvOS apps.

Beethoven is an audio processing Swift library
Beethoven is an audio processing Swift library

Beethoven is an audio processing Swift library that provides an easy-to-use interface to solve an age-old problem of pitch detection of musical signals.

FDWaveformView is an easy way to display an audio waveform in your app
FDWaveformView is an easy way to display an audio waveform in your app

FDWaveformView is an easy way to display an audio waveform in your app. It is a nice visualization to show a playing audio file or to select a position in a file.

Swift audio synthesis, processing, & analysis platform for iOS, macOS and tvOS
Swift audio synthesis, processing, & analysis platform for iOS, macOS and tvOS

AudioKit AudioKit is an audio synthesis, processing, and analysis platform for iOS, macOS (including Catalyst), and tvOS. Installation To add AudioKit

Voice Memos is an audio recorder App for iPhone and iPad that covers some of the new technologies and APIs introduced in iOS 8 written in Swift.
Voice Memos is an audio recorder App for iPhone and iPad that covers some of the new technologies and APIs introduced in iOS 8 written in Swift.

VoiceMemos Voice Memos is a voice recorder App for iPhone and iPad that covers some of the new technologies and APIs introduced in iOS 8 written in Sw

YiVideoEditor is a library for rotating, cropping, adding layers (watermark) and as well as adding audio (music) to the videos.

YiVideoEditor YiVideoEditor is a library for rotating, cropping, adding layers (watermark) and as well as adding audio (music) to the videos. YiVideoE

App for adding and listening audio files
App for adding and listening audio files

SomeSa SomeSa (самса) – приложение, позволяющее загружать и воспроизводить произвольные аудиофайлы. Протестировано на форматах файлов .wav и .mp3, раз

Painless high-performance audio on iOS and Mac OS X

An analgesic for high-performance audio on iOS and OSX. Really fast audio in iOS and Mac OS X using Audio Units is hard, and will leave you scarred an

Audio Filters on iOS and OSX

Audio Filters on iOS and OSX Implement high quality audio filters with just a few lines of code and Novocaine, or your own audio library of choice. NV

Owner
Gabriel Gatzsche
Inventor of SoundPrism. CEO & Founder of Grace Cloud
Gabriel Gatzsche
Proof of concept app for trying to integrate passkeys and WebAuthn into Vapor

Vapor Passkey Demo Proof of concept app for trying to integrate passkeys and WebAuthn into Vapor Usage Clone the project, then in Terminal run swift r

Tim Condon 70 Dec 20, 2022
The Amazing Audio Engine is a sophisticated framework for iOS audio applications, built so you don't have to.

Important Notice: The Amazing Audio Engine has been retired. See the announcement here The Amazing Audio Engine The Amazing Audio Engine is a sophisti

null 523 Nov 12, 2022
AudiosPlugin is a Godot iOS Audio Plugin that resolves the audio recording issue in iOS for Godot Engine.

This plugin solves the Godot game engine audio recording and playback issue in iOS devices. Please open the Audios Plugin XCode Project and compile the project. You can also use the libaudios_plugin.a binary in your project.

null 3 Dec 22, 2022
KeyAudioManager - A swift package to make it a lot easier to play audio in your app

KeyAudioManager A swift package to make it a lot easier to play audio in your ap

Pedro Esli 3 Apr 28, 2022
Extensions and classes in Swift that make it easy to get an iOS device reading and processing MIDI data

MorkAndMIDI A really thin Swift layer on top of CoreMIDI that opens a virtual MIDI destination and port and connects to any MIDI endpoints that appear

Brad Howes 11 Nov 5, 2022
SwiftAudioPlayer - Swift-based audio player with AVAudioEngine as its base

SwiftAudioPlayer Swift-based audio player with AVAudioEngine as its base. Allows for: streaming online audio, playing local file, changing audio speed

null 417 Jan 7, 2023
11t is an iOS and Android app for connecting to Mastodon, written in Flutter.

README 11t is an iOS and Android app for connecting to Mastodon, written in Flutter. I connect to Mastodon on mastodon.social, but everyone can start

Jeroen Smeets 88 Dec 23, 2022
'Minimalistic Push' is a minimalistic push-up application now built with Flutter.

Minimalistic Push Minimalistic Push is one of the simplest push-up trackers out there. You can track your push-ups in the training mode and see an ove

Jonas Poxleitner 10 Dec 29, 2022
AudioKit is an audio synthesis, processing, and analysis platform for iOS, macOS, and tvOS.

AudioKit is an audio synthesis, processing, and analysis platform for iOS, macOS (including Catalyst), and tvOS. Installation To add AudioKit

AudioKit 9.5k Dec 31, 2022
AudioPlayer is syntax and feature sugar over AVPlayer. It plays your audio files (local & remote).

AudioPlayer AudioPlayer is a wrapper around AVPlayer. It also offers cool features such as: Quality control based on number of interruption (buffering

Kevin Delannoy 676 Dec 25, 2022