From @kenwheeler Original Ticket: AudioKit/AudioKit#2727

Describe the bug I'm witnessing two issues with the Sequencer.

1. When I have two tracks, and add a note at 0.0 for each:

conductor.sequencer.tracks[selected].sequence.add(noteNumber: 60, position: Double((index)) * 0.25, duration: 1.0)

I'm seeing that the first note of the second track isn't played. All subsequent notes play alright.

2. When adding notes programmatically using the same API above, _while the sequencer is playing_, I'm hearing offset note timings. If I stop the sequencer and restart it, the timing is correct.

To Reproduce Steps to reproduce the behavior:

1. Add two notes programmatically using the above API at the 0.0 position and play with looping enabled

2. For the other issue, use the same API as above to add notes programmatically while the sequencer is playing

Expected behavior I would expect a) two tracks, each with a note at 0.0 position to be able to play simultaneously. and b) I would expect notes added when the sequencer is playing to be time accurate in the current play context.

Screenshots If applicable, add screenshots to help explain your problem.

Details (please complete the following information):

* Type: [e.g. iPhone6, iPad, Mac] iPhone 13 pro

* OS: [e.g. iOS13.1] iOS 15.4.1

* AudioKit Version [e.g. 5.0] 5.3.3,
  **Other Context**
  I'm using samplers.

p.s. Thanks so much, this project rules

I've recently switched from the AppleSequencer to the Sequencer in order to build a metronome app (I was having the long-standing first measure bug with it).

Strangely, I was seeing the CPU usage going up a lot after the switch. After isolating various part of the code, it seems that using the CallbackInstrument was the cause of all this.

Is there something going on there? Here's a capture of Time Profiler, it seems that multiple threads are spawned.

Here's the sample code I'm using if you want to test on your side.

Using AudioKit 5.3.0.

import AudioKit
import AudioKitEX
import UIKit

class ViewController: UIViewController {

    let engine = AudioEngine()
    let sequencer = Sequencer()
    var callbackTrack: SequencerTrack?

    override func viewDidLoad() {
        super.viewDidLoad()

        let callbacksInstrument = CallbackInstrument(midiCallback: { status, note, _ in
            print(status, note)
        })
        callbackTrack = sequencer.addTrack(for: callbacksInstrument)
        
        engine.output = callbacksInstrument
        try? engine.start()
    }

}

macOS Version(s) Used to Build

macOS 12 Monterey

Xcode Version(s)

Xcode 13.4.1

Description

If noteOn events are called while loading AudioKit's AppleSampler() it can occasionally crash on line 121 of DSPBase.mm. Here is a test project to illustrate the crash: https://github.com/NickCulbertson/TestRenderBlock

This is the current AUInternalRenderBlock:

AUInternalRenderBlock DSPBase::internalRenderBlock()
{
    return ^AUAudioUnitStatus(
        AudioUnitRenderActionFlags *actionFlags,
        const AudioTimeStamp       *timestamp,
        AUAudioFrameCount           frameCount,
        NSInteger                   outputBusNumber,
        AudioBufferList            *outputData,
        const AURenderEvent        *realtimeEventListHead,
        AURenderPullInputBlock __unsafe_unretained pullInputBlock)
    {

        assert( (outputBusNumber == 0) && "We don't yet support multiple output busses" );
        if (pullInputBlock) {
            if (bCanProcessInPlace && inputBufferLists.size() == 1) {
                // pull input directly to output buffer
                inputBufferLists[0] = outputData;
                AudioUnitRenderActionFlags inputFlags = 0;
                pullInputBlock(&inputFlags, timestamp, frameCount, 0, inputBufferLists[0]);
            }
            else {
                // pull input to internal buffer
                for (size_t i = 0; i < inputBufferLists.size(); i++) {
                    inputBufferLists[i] = internalBufferLists[i];
                    
                    UInt32 byteSize = frameCount * sizeof(float);
                    for (UInt32 ch = 0; ch < inputBufferLists[i]->mNumberBuffers; ch++) {
                        inputBufferLists[i]->mBuffers[ch].mDataByteSize = byteSize;
                    }
                    
                    AudioUnitRenderActionFlags inputFlags = 0;
                    pullInputBlock(&inputFlags, timestamp, frameCount, i, inputBufferLists[i]);  //CRASH HERE
                }
            }
        }
        
        outputBufferList = outputData;
        
        processWithEvents(timestamp, frameCount, realtimeEventListHead);
        
        return noErr;
    };
}

Just a thought, but this is Apple's generated render block in DSPKernel when making an AUv3. It might be useful for comparison:

// MARK: -  AUAudioUnit (AUAudioUnitImplementation)

// Subclassers must provide a AUInternalRenderBlock (via a getter) to implement rendering.
- (AUInternalRenderBlock)internalRenderBlock {
    /*
     Capture in locals to avoid ObjC member lookups. If "self" is captured in
     render, we're doing it wrong.
     */
    // Specify captured objects are mutable.
    __block OverdriveSynthDSPKernel *state = &_kernel;
    __block BufferedInputBus *input = &_inputBus;

    return ^AUAudioUnitStatus(AudioUnitRenderActionFlags 				*actionFlags,
                              const AudioTimeStamp       				*timestamp,
                              AVAudioFrameCount           				frameCount,
                              NSInteger                   				outputBusNumber,
                              AudioBufferList            				*outputData,
                              const AURenderEvent        				*realtimeEventListHead,
                              AURenderPullInputBlock __unsafe_unretained pullInputBlock) {

        AudioUnitRenderActionFlags pullFlags = 0;

        if (frameCount > state->maximumFramesToRender()) {
            return kAudioUnitErr_TooManyFramesToProcess;
        }

        AUAudioUnitStatus err = input->pullInput(&pullFlags, timestamp, frameCount, 0, pullInputBlock);

        if (err != noErr) { return err; }

        AudioBufferList *inAudioBufferList = input->mutableAudioBufferList;

        /*
         Important:
         If the caller passed non-null output pointers (outputData->mBuffers[x].mData), use those.

         If the caller passed null output buffer pointers, process in memory owned by the Audio Unit
         and modify the (outputData->mBuffers[x].mData) pointers to point to this owned memory.
         The Audio Unit is responsible for preserving the validity of this memory until the next call to render,
         or deallocateRenderResources is called.

         If your algorithm cannot process in-place, you will need to preallocate an output buffer
         and use it here.

         See the description of the canProcessInPlace property.
         */

        // If passed null output buffer pointers, process in-place in the input buffer.
        AudioBufferList *outAudioBufferList = outputData;
        if (outAudioBufferList->mBuffers[0].mData == nullptr) {
            for (UInt32 i = 0; i < outAudioBufferList->mNumberBuffers; ++i) {
                outAudioBufferList->mBuffers[i].mData = inAudioBufferList->mBuffers[i].mData;
            }
        }

        state->setBuffers(inAudioBufferList, outAudioBufferList);
        state->processWithEvents(timestamp, frameCount, realtimeEventListHead, nil /* MIDIOutEventBlock */);

        return noErr;
    };
}

Issue opened by @Truba Opening here because this is an AudioKitEX issue - may be related to #10 Original Ticket: AudioKit/AudioKit#2536

Moves some funcs to sequencerTrack directly, marks the old ones as deprecated, and makes some other sequence track editing from within the sequencer more consistent.

REPLACE ALL OF THIS TEXT WITH A DESCRIPTION OF YOUR CODE CHANGES

Try to make sure your changes conform to the Swift API Design Guidelines. If you have 40 minutes you'd like to have well-spent, watch Modern Swift API Design from WWDC 2019.

Changes:

1. Add totalDuration getter in Sequence.swift
2. Update totalDuration every time a new note event is added in Sequence.swift
3. Add property observer to sequence in SequencerTrack.swift
4. Add warnings and change the track length if the totalDuration exceeds the bounds of the track
5. Add testNoteBounds() with passing MD5
6. Update Package.resolved to current AK
7. Add totalDuration to NoteEventSequence initializer
8. Change testAdd() to use new initializer

This should help with #10 and #12

This reverts the changes made here: https://github.com/AudioKit/AudioKit/pull/2632.

The CallbackInstrument is now using NSTimer back again, but with a run loop depending on the thread the instrument was instantiated.

This could fix https://github.com/AudioKit/AudioKitEX/issues/5.

Is your feature request related to a problem? Please describe.

An AudioUnit will often need to know the maximum buffer size that will be requested for rendering. AudioKit doesn't provide this.

Describe the solution you'd like

maxFramesToRender should be passed to DSPBase::init.

Describe alternatives you've considered

This information could possibly be grabbed from the size of the output buffers but that is inelegant.

Are there examples for this somewhere in the code?

Describe the bug Note that this is more of a "heads up" issue; I don't know if they will actually apply once Swift 5.7 and Xcode 14 exit beta. It may necessitate a maintainer of AudioKitEX filing a Radar issue.

When building any application that has a dependency on AudioKitEX with the upcoming Swift 5.7 or nightly Swift toolchains in Xcode 14.0 beta, the build will fail when attempting to build the CAudioKitEX component with the following error in DSPBase.h:133:

This issue is present in both the release and main branch versions of AudioKitEX.

To Reproduce Steps to reproduce the behavior:

1. Install Xcode 14 beta.
2. Install either the trunk snapshot or the Swift 5.7 development snapshot.
3. Open any project that uses AudioKitEX, such as the AudioKit Cookbook.
4. Select either one of the prerelease toolchains from the Xcode > Toolchains menu.
5. Attempt to build the project using any scheme and target combination.
6. The build will fail with the error from the screenshot.

Author's note: My project doesn't use STKAudioKit, but another error was raised when I was recreating the issue with the Cookbook app. It was in Skini.cpp:142, and the error was Missing '#include <stdlib.h>'; 'atof' must be declared before it is used.

Expected behavior The build should have completed and my application/Cookbook should have launched.

Screenshots Error screenshot is in description of bug.

Details (please complete the following information):

• Type: 2021 MacBook Pro w/ M1 Pro CPU
• OS: macOS Monterey 12.4
• Xcode: 14.0 beta
• Swift: Nightly and 5.7 development toolchains
• AudioKit Version: Latest release and main branch

Additional context I'm happy to provide any additional context, if necessary.