iOS application for finding formants in spoken sounds

William Entriken

Last update: Dec 25, 2022

Related tags

Audio swift language ios application app language-learning speech-recognition speech-processing speech-therapy mature

Overview

FORMANT PLOTTER

Buy on the App Store: https://itunes.apple.com/us/app/formant-analyzer/id799183655?mt=8&uo=4&at=11l6hc&ct=fnd

This is an iOS project to analyze formants. The user speaks and the formant is plotted on the screen immediately. It is designed for speaking a single vowel syllable. It will try to isolate the vowel sound from any surrounding consonants if it can.

Formant Research

Other related tools and formant information

Praat: http://www.fon.hum.uva.nl/praat/
WaveSurfer: https://sourceforge.net/projects/wavesurfer/
Perry R. Cook, "Identification of control parameters in an articulatory vocal tract model, with applications to the synthesis of singing", 1990, Ph.D Dissertation, CCRMA https://ccrma.stanford.edu/~kglee/m220c/formant.html

Vowel formant chart:

vowel		F1	F2	F3
ee	male	270	2290	3010
	female	310	2790	3310
	child	370	3200	3730
e	male	530	1840	2480
	female	610	2330	2990
	child	690	2610	3570
ae	male	660	1720	2410
	female	850	2050	2850
	child	1030	2320	3320
ah	male	730	1090	2440
	female	590	1220	2810
	child	680	1370	3170
oo	male	300	870	2240
	female	370	950	2670
	child	430	1170	3260

Speech Acoustics Made Easy http://web.archive.org/web/20120914101638/http://www.cochlear.com/files/assets/speech_acoustics_made_easy.pdf
English vowel word reference http://www.fonetiks.org/engsou2am.html
Chinese vowel diagram http://en.wikipedia.org/wiki/Chinese_vowel_diagram
Wiki page http://en.wikipedia.org/wiki/Formant
The National Center for Voice and Speech http://www.ncvs.org/ncvs/tutorials/voiceprod/tutorial/filter.html
Linguistics 110 Berkeley http://linguistics.berkeley.edu/~kjohnson/ling110/Homework_assignments/HW7_PlotVowels/PlotYourVowels.pdf
MATLAB Speech Signal Analysis http://www.phon.ucl.ac.uk/courses/spsci/matlab/lect10.html
MATLAB Formant Tracker example http://www.mathworks.com/matlabcentral/fileexchange/8959-formant-tracker
Formant Java example http://chronos.ece.miami.edu/~dasp/SeniorProject/Presentation/416Presentation.pdf
Digital Bubble Bath - Don H. Johnson Great discussion on formant analysis and practical implementation http://www.clear.rice.edu/elec431/projects96/digitalbb/formants.html

The Formant Plotter

The program starts in green state. When the user starts talking (i.e. RMS goes above 0dBm for at least 0.1 seconds), the program goes into listening state and records the sound. When the user stops talking (i.e. RMS goes below 0dBm for at least 0.1 seconds), the program returns to ready state and draws graphs.

Graph drawing is done as follows: The recorded sound is truncated to remove the first and last 10% of the data. Then perform a Fast Fourier Transform (FFT) with autocorrelation. The result is plotted linear from 0 - 4000 Hz on the X axis and from -60 to 0 dB log scale on the Y axis.

The second graph is drawn as follows: An image is placed on the background for the chart (you create an image to start with) and two dots are plotted on the chart, representing the highest and lowest sample value from the recording. That's it.

The correct algorithm which takes the FFT results which were plotted above and creates the vowel plot is discussed in Formant Research above.

Some potential next steps include:

Use autocorrelation to increase trimming accuracy
Windowing on the truncated sound buffer so that edge samples have an attenuated effect
Root polishing. The code has been written but commented out (please see PlotView.m). If we can test and refine this part, we will have better estimates of roots of LPC polynomials, and formant frequencies. We may not want VERY accurate estimates of formant frequencies and may not need root polishing.
Elimination of weak roots (far away from unit circle). They do not produce a peak in H(w) and should be ignored. I hope that if we reduce order of LPC, we may not see such weak roots. This should be investigated after reduction of LPC filter order.

Comments

program issue on iphone and ioad mini

i have just paid for and diownloaded this program but i cannot use it as i keep getting a retrying message and red button in the top right hand corner - i have allowed access to the microphobne. is this a known fault or am i doing something wrong?

thank you

opened by bermanrm 32
Build Failed in Xcode 7.2.1

Hello,

I am grateful that I could find this library. In this project, I have trouble with error. See below screen capture.

Could you please tell me how to do?

opened by tatsuya-yokoyama 1
Research: review experts' published materials

We cite several experts research in this work.

Delete the copy of their work from this project and cite it instead as a hyperlink.

Make a README.md in the research folder to discuss their existing work and algorithms.

Then reach out to them to let them know we are citing them.

opened by fulldecent 0
Research: separate and document WAV conversion

WAV files are the primary input. They are converted to RAW files with a specific encoding in /Sounds/ and /Raw_Buffer/ by using generate_raw_buffer.m.

Move all these processes into a single folder and document with an README.md

opened by fulldecent 0
Research: separate and document vowel trimming
In SpeechAnalyzer.m there are several simple methods related to finding the relevant vowel sound in the utterance:

- (NSRange)vowelRange is the main method and it depends on the following two

- (NSRange)strongSignalRange

- (NSRange)truncateRangeTails:(NSRange)range

This is already implemented in off_line_tail_clipping WE2012-02-09.m

To prove this out, we must:

Rerun the Matlab program

Document how to run the Matlab program and expected outputs (make a README.md)

Rename the files so people can find them

Create a XCTest in iOS to perform the same action using the app

Set the XCTest to expect the same values as output from Matlab

Cross fingers and run it
opened by fulldecent 0
Add scope document

Hello Greg! Thanks for sharing. I'm glad to see your interest here and further research into this project.

As for the tooling, I believe Formant Analyzer does what you are describing. It works best of course if there is good quality audio coming in (decent microphone, pointed in correct direction, with decent gain, without excessive amplification). Also, word choice can help. It is calibrated on expecting monosyllabic utterances. "Hiiii" will work well, "Helloooooooo" will not. You can see some of this because there are four screens in the app, and basically they are all the debugging steps I use to review how it works.

I originally made the app to help my wife with accent reduction (born in China, living in USA). And it mostly would a home for speech pathologists (adults that could speak but now have less function, children with difficulty learning).

For your own work, please do publish the vowel chart you have created. Include F1/F2, how you measured it, demographics of the speakers (age, gender, language, region, type of speaking (casual, singing), anything else relevant, how you measured F1/F2). If you can also upload sample audio files this will be ideal because researchers use this and I can add them as test files for the project.

The algorithms we are using are based on research we found on MathWorld. These are all implemented originally in Matlab and then they are ported over to iOS. And the results are compared on sample files to validate the implementation correctness. I believe Formant Analyzer is better than Pratt for the specific case of unattended monosyllabic plotting of L1/L2. But if there are some things we can learn from Pratt I'm happy to check it out.

Also I know our vowel plots are an item to improve. I don't see good primary sources on this. And there seems to be many regionally-relevant publications on this (for Chinese speakers, European, etc.) So I'm not sure what to do there. But either way, the L1/L2 readout will be the same. Only the relative placement on the vowel chart will change.

opened by fulldecent 0
Create a way to donate audio samples using a web form
On the plot form, add "donate audio sample". It uploads to a server (where?) and then opens a google form to add information to that sample:

Speaker demographics

Speaker accent / disabilities

Intended word

Intended vowel sound (make a huge dropdown)

Contact information
opened by fulldecent 0
Switch App Store app to free app
Use this approach https://stackoverflow.com/a/19391432/300224

And have an in-app upgrade for some specific feature.

Choose that feature

Implement IAP

Make rest of app free
opened by fulldecent 1

Releases(1.2.0)

1.2.0(Nov 17, 2015)
New formant plot with full size on ALL iOS devices

Back / forward buttons in the help viewer

Faster processing

For anyone researching speech topics or interested in the project. I just want to let you know this is an open source project and you are welcome to visit our GitHub page (from Help link inside the app). All help and feedback is much appreciated!
Source code(tar.gz)
Source code(zip)

Owner

William Entriken

EN/中文 [email protected]

GitHub https://fulldecent.github.io/formant-analyzer/

Soundable is a tiny library that uses AVFoundation to manage the playing of sounds in iOS applications in a simple and easy way

Soundable is a tiny library that uses AVFoundation to manage the playing of sounds in iOS applications in a simple and easy way. You can play

89 Nov 21, 2022

SwiftySound is a simple library that lets you deal with Swift sounds easily

SwiftySound Overview SwiftySound is a simple library that lets you deal with Swift sounds easily. Static methods Sound.play(file: "dog.wav") Sound.pla

1.1k Dec 17, 2022

SoundManager - A simple framework to load and play sounds in your app.

3 Jan 5, 2022

The easiest way to prepare, play, and remove sounds in your Swift app!

Chirp The easiest way to prepare, play, and remove sounds in your Swift app! ##Installation ###CocoaPods Installation Chirp is available on CocoaPods.

309 Dec 17, 2022

Analyser BPM in Swift for your music/sounds/records, whatever..

BPM-Analyser Analyser BPM in Swift for your music/sounds/records, whatever.. Powered with Superpowered Preview: How To: Copy theese files to your proj

71 Dec 24, 2022

Soundable allows you to play sounds, single and in sequence, in a very easy way

Overview Soundable is a tiny library that uses AVFoundation to manage the playing of sounds in iOS applications in a simple and easy way. You can play

89 Nov 21, 2022

The application display album artwork with detailed information from the iTunes API.

Music-Info The application display album artwork with detailed information from the iTunes API. UIKit, CoreData, Foundation Navigation (TabBarControll

0 Nov 5, 2021

SimplePlayer - A simple application to fetch songs from iTunes Library by searching song's artist

SimplePlayer is a simple application to fetch songs from iTunes Library by searching song's artist.

0 Jan 30, 2022

Detect-a-Beacon - An educational application using Apple iBeacon technology to detect beacons

Detect-a-Beacon An educational application using Apple iBeacon technology to det

1 Aug 9, 2022

'Minimalistic Push' is a minimalistic push-up application now built with Flutter.

Minimalistic Push Minimalistic Push is one of the simplest push-up trackers out there. You can track your push-ups in the training mode and see an ove

10 Dec 29, 2022

Nobetci Eczacim is open-source App Store Application which users can get pharmacy information and location easily and safely.

Pharmacy-On-Duty Nobetci Eczacim Project Description This project designed for make it easier for people to search for a pharmacy on duty. App Store A

5 Sep 19, 2022

iOS application for finding formants in spoken sounds

Related tags

Overview

FORMANT PLOTTER

Formant Research

The Formant Plotter

Comments

program issue on iphone and ioad mini

Build Failed in Xcode 7.2.1

Research: review experts' published materials

Research: separate and document WAV conversion

Research: separate and document vowel trimming

Add scope document

Create a way to donate audio samples using a web form

Switch App Store app to free app

Releases(1.2.0)

1.2.0(Nov 17, 2015)

Owner

William Entriken

Soundable is a tiny library that uses AVFoundation to manage the playing of sounds in iOS applications in a simple and easy way

SwiftySound is a simple library that lets you deal with Swift sounds easily

SoundManager - A simple framework to load and play sounds in your app.

The easiest way to prepare, play, and remove sounds in your Swift app!

Analyser BPM in Swift for your music/sounds/records, whatever..

Soundable allows you to play sounds, single and in sequence, in a very easy way

The application display album artwork with detailed information from the iTunes API.

SimplePlayer - A simple application to fetch songs from iTunes Library by searching song's artist

Detect-a-Beacon - An educational application using Apple iBeacon technology to detect beacons

'Minimalistic Push' is a minimalistic push-up application now built with Flutter.

Nobetci Eczacim is open-source App Store Application which users can get pharmacy information and location easily and safely.

AudiosPlugin is a Godot iOS Audio Plugin that resolves the audio recording issue in iOS for Godot Engine.

AudioKit is an audio synthesis, processing, and analysis platform for iOS, macOS, and tvOS.

AudioPlayer is a simple class for playing audio in iOS, macOS and tvOS apps.

Swift audio synthesis, processing, & analysis platform for iOS, macOS and tvOS

AudioKit Synth One: Open-Source iOS Synthesizer App

Classical music front-end for Apple Music: iOS app

:musical_keyboard: A simple iOS synthesiser powered by Pure Data. Based on the Korg Monotron Delay.

Music Player for iOS which looks & feels like classic player