Audio Filters on iOS and OSX

First of all, thanks for putting this code up on the Internet, I'm pretty sure it helped a lot of people in making their first steps into iOS/OSX audio, like it did for me. However, I decided to switch to The Amazing Audio Engine and my own implementation of filters (like NVDSP did with Novocaine) and I thought I'd share why.

First reason is thread safety: neither Novocaine nor NVDSP are thread safe as far as I can tell. Because audio processing happens in a separate high-priority thread, while e.g. filter parameters can be changed from the main thread, you need to design your classes with thread safety in mind. For example, when a user changes one of the peaking filter parameters and while you are calculating your 5 coefficients, the audio thread may use them for filtering the audio. These race conditions may not be so rare as you might think, especially with smaller buffer sizes on OS X, but they won't always result in audible glitches. In this particular case, if you want to avoid locking, you need to have two copies of the coefficients and also have an atomic flag that indicates (using OSAtomicX()) whether they were copied to the real-time thread or not. As one example of what could be done.

But then comes the problem of efficiency and again, neither Novocaine nor NVDSP were designed with efficiency in mind. For example, you do dynamic allocation of buffers in each cycle and it's just a horrendously bad thing to do. Let alone that Objective-C messaging is scattered everywhere in real-time code. On top of that, if you want to implement a multi-band EQ you could avoid copying vDSP_deq22() buffers for each of the filters by swapping them. I've noticed a dozen of other things that could've been optimized.

TAAE by contrast is made with these things in mind. The code that uses TAAE may be a little uglier, but at least all real-time code is pure C, no dynamic allocation and no I/O happening there, and the documentation stresses that user code that's executed in the real-time thread should adhere to the same principles. TAAE can even give you warnings in Debug mode that your real-time routine is taking too long to execute.

Anyway, like I said I'm still grateful to those who wrote and contributed to Novocaine and NVDSP but once you consider efficiency and thread safety (you don't want to drain your users' phone battery and you don't want them to hear occasional glitches, right?) you realize that these two libraries fall short. Fixing these things though might require a serious effort, if not a complete revamp. In any case unfortunately I won't be contributing to these repositories but I'm willing to help to anyone who'd consider redoing them.

Thanks again, and have a nice coding!

Hi is there a way we can load an audio in memory using Novocaine and then applying filters to it and once we are done we can save the modified sound (including filters) by making a new file?

currently all I could find is to apply NVDSP filters on the currently playing audio stream. Is there any way to save this modified audio stream? If not then are you planing to support this feature in future? If yes, when can we be able to have that feature?

I also can't figure out how to feed it another song. It plays one song, then stops.

• (void)updateWithMediaItemURL:(NSURL *) itemURL {

  _fileReader = [[AudioFileReader alloc] initWithAudioFileURL:itemURL samplingRate:audioManager.samplingRate numChannels:audioManager.numOutputChannels];

    __block float lastTimeLeft = - 1.0f;
    float timeLeft = self.fileReader.duration - [self.fileReader getCurrentTime];
  
    if ( timeLeft > 1.1) { //&& timeLeft != lastTimeLeft
        lastTimeLeft = timeLeft;
    } else {
        [self pause];
  
        if (!self.fileReader.playing) {
            audioManager.outputBlock = nil;
            [self.mediaDelegate nextSongPlay];
        }
    }
    NSLog(@"%f", timeLeft);
  
    [audioManager setOutputBlock:^(float *data, UInt32 numFrames, UInt32 numChannels)   {
    [self.fileReader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels];
  
    PEQ.G = PEQ_centerFrequency;
    [PEQ filterData:data numFrames:numFrames numChannels:numChannels];
    [CDT counterClipping:data numFrames:numFrames numChannels:numChannels];
  

  }]; }

Is there any privilege to apply band reject filter? or Is it possible to apply two filters at once? I tried it but with no success. I think it should have band reject filter too.

I wrote a recorder in remote IO Unit.I want to calculate the sound level of the samples in the render callback. I found NVDSP project have an util method to getDBLevel:

- (float) getdBLevel:(float *)data numFrames:(UInt32)numFrames numChannels:(UInt32)numChannels;

So, I try to pass the AudioBuffer as parameters of this method:

static OSStatus performRender(void *inRefCon,
                          AudioUnitRenderActionFlags    *ioActionFlags,
                          const AudioTimeStamp      *inTimeStamp,
                          UInt32                        inBusNumber,
                          UInt32                        inNumberFrames,
                          AudioBufferList           *ioData){

AudioBuffer buf = ioData->mBuffers[0];
float db = [[[NVSoundLevelMeter alloc] init] getdBLevel:buf.mData numFrames:inNumberFrames numChannels:buf.mNumberChannels];

but this is not work as expect, it always return -50db.

Any idea?

In method filterContiguousData (line 81) when you copy the results back:

memcpy(data, tOutputBuffer, numFrames * sizeof(float));

I think it should be

memcpy(data, tOutputBuffer + 2, numFrames * sizeof(float));

I haven't tested this yet, but if it is a bug it may go unnoticed in audio.

Many other examples on the Net do this. See e.g. http://vboehm.net/2015/04/using-vdsp_deq22-correctly/

Hello, I'm using NVDSP with my application and I have a little problem with it when the app go to background, unlike your exemple, my application keep the audio playing when the app go to background, and occasionally the audio played in an interrupted manner. I see the same problem when I run your exemple, The audio Is automatically played, I tap the home button and Just before the app go to background the audio play with interruption. Any Idea why this is happening please ?

Hi! I am trying to apply an EQ on a iOS AVPlayer. I know that it is not the best way to go, but i don't have room to choose the player used here.

From what i found here: [https://chritto.wordpress.com/2013/01/07/processing-avplayers-audio-with-mtaudioprocessingtap/] it is possible to extract raw audio data from AVPLayer using MTAudioProcessingTap. Using this, i end up with a "process" method which is the way to treat raw data from AVPlayer. In the example below this "process" method apply vDSP_vsmul on the audio flux (it's a simplified example from the link attached).

If i am right, i should be able to use your Filters here instead of vDSP_vsmul. My goal is to make a 10(ish)-band EQ (with your PeakingEQ which seems nice)... Could you point me on how to achieve this inside that "process" function ? I am a bit lost with those vDSP / Filters stuffs... but i feel i am not far from it: The processing in your library is vDSP based too, so the works done by your library could seat here..

// Function called by MTAudioProcessingTap allowing treatment of audio flux..
void process(
    MTAudioProcessingTapRef tap, 
    CMItemCount numberFrames,
    MTAudioProcessingTapFlags flags, 
    AudioBufferList *bufferListInOut,
    CMItemCount *numberFramesOut, 
    MTAudioProcessingTapFlags *flagsOut)
{
    // Error detection
    OSStatus err = MTAudioProcessingTapGetSourceAudio(tap, numberFrames, bufferListInOut, flagsOut, NULL, numberFramesOut);
    if (err) NSLog(@"Error from GetSourceAudio: %ld", err);

   // Arbitrary treatment given as exemple
    float scalar = 5;
    vDSP_vsmul(bufferListInOut->mBuffers[0].mData, 1, &scalar, bufferListInOut->mBuffers[0].mData, 1, bufferListInOut->mBuffers[0].mDataByteSize / sizeof(float));
    vDSP_vsmul(bufferListInOut->mBuffers[1].mData, 1, &scalar, bufferListInOut->mBuffers[1].mData, 1, bufferListInOut->mBuffers[1].mDataByteSize / sizeof(float));
}

Thank you !

Best Regards,

Thomas

Hello Dear,

I am trying to build 10 band equaliser. Well i copy pasted Equaliser.mm's code in "viewWillAppear". And added 9 more Sliders in xib file. and changed IBAction code too.

(void)HPFSliderChanged:(UISlider *)sender {
 PEQ[sender.tag - 1].centerFrequency = sender.value;
 NSLog(@"%f",sender.value);
 }

I wanna know am i doing right ? and the what will be range of the Sliders? Like in HPF slider range is 2k to 8k Thanks

I'm not really familiar with audio filters, so I looked at your example and in it was what seemed to be exactly what I'm looking for, it had all the right frequencies (and the comment at the top said it was a 10-band equalizer).

But I tried it out and something just doesn't sound right about it..... the implementation in this one sounds weird compared to applications like mpv ~~and winamp~~ (nevermind, winamp sounds just as weird). Is that just how a PeakingEQFilter works? Should I be using another filter along with the PeakingEQFilters? And why did you set the Q-factor to 2?

Here's mpv's implementation of it's equalizer: https://github.com/mpv-player/mpv/blob/master/audio/filter/af_equalizer.c

Hi, I'm working on iOS application which is written in Swift 5. I'm looking to create A weight bandpass filter. My query is that currently, I am receiving audio samples through a microphone using tap. How can I use a High bandpass filter on an audio stream coming from the microphone to evaluate dB value?

Hey there! I'm trying to use your bandpass filter to filter an array of values (I'm trying to do heart beat detection). I'm new to signal processing and I was wondering if you could explain how to get the coefficients within your code. I see they are being called in this example on stackoverflow: https://stackoverflow.com/questions/10604690/noise-distortion-after-doing-filters-with-vdsp-deq22-biquad-iir-filter/10608609#10608609. I'm trying to create a bandpass filter between 0.667 Hz and 4.167 Hz. I have a sample rate of 30 frames per second. If you have any tips or can confirm if I could use your framework to do this that would be helpful! Thank you!

// import Novocaine.h and NVDSP.h #import "NVDSP/Filter/NVBandpassFilter.h" NVBandpassFilter *BPF = [[NVBandpassFilter alloc] initWithSamplingRate:audioManager.samplingRate]; BPF.centerFrequency = 2500.0f; BPF.Q = 0.9f; [BPF filterData:data numFrames:numFrames numChannels:numChannels];

We need to comment out the following in the RingBuffer, or we get errors.

#ifdef __cplusplus

#endif

Does anyone know why this is the case? We can't init the RingBuffer with the error.

Any idea on how complicated it would be to add support for digital room correction filter. http://en.wikipedia.org/wiki/Digital_room_correction

Also how about a decay filter?

How shall i proceed with Noise filtering from the Audio using NVDSP. Please show me some way to achieve that. Also i want to recognise a sound pattern in the recorder audio and remove it from that audio file.