A native iOS UI widget for adding Spokestack to any iOS app.

Overview

Spokestack Tray iOS

A native iOS library for adding Spokestack to any iOS app.

Installation

Edit Podfile

CocoaPods is a dependency manager for Cocoa projects. For usage and installation instructions, visit their website. To integrate Spokestack into your Xcode project using CocoaPods, specify it in your Podfile:

pod 'SpokestackTray-iOS'

Edit Info.plist

Add the following to your Info.plist to enable permissions. Also ensure your iOS deployment target is set to 13.0.

Without these your app will crash

  <key>NSMicrophoneUsageDescription</key>
  <string>For making voice requests</string>
  <key>NSSpeechRecognitionUsageDescription</key>
  <string>For understanding your voice requests</string>

Set the AudioSession category

For most apps, this should be put in AppDelegate.application

        do {
            let session = AVAudioSession.sharedInstance()
            try? session.setCategory(.playAndRecord, options: [.defaultToSpeaker, .allowAirPlay, .allowBluetoothA2DP, .allowBluetooth])
            try? session.setActive(true, options: [])
        }

Usage

import SpokestackTray
import Spokestack

    override func viewDidLoad() {
        
        super.viewDidLoad()
        self.view.backgroundColor = .green
        
        /// Child view controller
        
        self.addHostingController()

        let configuration: TrayConfiguration = TrayConfiguration()
        
        /// When the tray is opened for the first time this is the synthesized
        /// greeting that will be "said" to the user

        configuration.greeting = """
        Welcome! This example uses models for Minecraft. Try saying, \"How do I make a castle?\"
        """
        
        /// When the tray is listening or processing speech there is a animated gradient that
        /// sits on top of the tray. The default values are red, white and blue

        configuration.gradientColors = [
            "#61fae9".spstk_color,
            "#2F5BEA".spstk_color,
            UIColor.systemRed
        ]

        /// Apart of the initialization of the tray is to download the nlu and wakeword models.
        /// These are the default Spokestack models, but you can replace with your own

        configuration.nluModelURLs = [
            NLUModelURLMetaDataKey: "https://d3dmqd7cy685il.cloudfront.net/nlu/production/shared/XtASJqxkO6UwefOzia-he2gnIMcBnR2UCF-VyaIy-OI/nlu.tflite",
            NLUModelURLNLUKey: "https://d3dmqd7cy685il.cloudfront.net/nlu/production/shared/XtASJqxkO6UwefOzia-he2gnIMcBnR2UCF-VyaIy-OI/vocab.txt",
            NLUModelURLVocabKey: "https://d3dmqd7cy685il.cloudfront.net/nlu/production/shared/XtASJqxkO6UwefOzia-he2gnIMcBnR2UCF-VyaIy-OI/metadata.json"
        ]
        configuration.wakewordModelURLs = [
            WakeWordModelDetectKey: "https://d3dmqd7cy685il.cloudfront.net/model/wake/spokestack/detect.tflite",
            WakeWordModelEncodeKey: "https://d3dmqd7cy685il.cloudfront.net/model/wake/spokestack/encode.tflite",
            WakeWordModelFilterKey: "https://d3dmqd7cy685il.cloudfront.net/model/wake/spokestack/filter.tflite"
        ]

        configuration.cliendId = "YOUR_CLIENT_ID"
        configuration.clientSecret = "YOUR_CLIENT_SECRET"
        
        /// The handleIntent callback is how the SpeechController and the TrayViewModel know if 
        /// NLUResult should be processed and what text should be added to the tableView.

        let greeting: IntentResult = IntentResult(node: InterntResultNode.greeting.rawValue, prompt: configuration.greeting)
        var lastNode: IntentResult = greeting

        configuration.handleIntent = {intent, slots, utterance in

            switch intent {
                case IntentResultAmazonType.repeat.rawValue:
                    return lastNode
                case IntentResultAmazonType.yes.rawValue:
                    lastNode = IntentResult(node: InterntResultNode.search.rawValue, prompt: "I heard you say yes! What would you like to make?")
                case IntentResultAmazonType.no.rawValue:
                    lastNode = IntentResult(node: InterntResultNode.exit.rawValue, prompt: "I heard you say no. Goodbye")
                case IntentResultAmazonType.stop.rawValue,
                     IntentResultAmazonType.cancel.rawValue,
                     IntentResultAmazonType.fallback.rawValue:
                    lastNode = IntentResult(node: InterntResultNode.exit.rawValue, prompt: "Goodbye!")
                case IntentResultAmazonType.recipe.rawValue:
                    
                    if let whatToMakeSlot: Dictionary<String, Slot> = slots,
                       let slot: Slot = whatToMakeSlot["Item"],
                       let item: String = slot.value as? String {
                    
                        lastNode = IntentResult(node: InterntResultNode.recipe.rawValue,
                                                prompt: """
                                                If I were a real app, I would show a screen now on how to make a \(item). Want to continue?
                                                """
                                    )
                    }
                    
                case IntentResultAmazonType.help.rawValue:
                    lastNode = greeting
                default:
                    lastNode = greeting
            }
            
            return lastNode
        }
        
        /// Which NLUNodes should trigger the tray to close automatically

        configuration.exitNodes = [
            InterntResultNode.exit.rawValue
        ]
        
        /// Callback when the tray is opened. The call back is called _after_ the animation has finished
        
        configuration.onOpen = {
            LogController.shared.log("isOpen")
        }
        
        /// Callback when the tray is closed. The call back is called _after_ the animation has finished
        
        configuration.onClose = {
            LogController.shared.log("onClose")
        }
        
        /// Callback when a `TrayListenerType` has occured
        
        configuration.onEvent = {event in
            LogController.shared.log("onEvent \(event)")
        }
        
        let tray: SpokestackTrayViewController = SpokestackTrayViewController(self, configuration: configuration)
        tray.addToParentView()
        tray.listen()
    }

Contributing

See the contributing guide to learn how to contribute to the repository and the development workflow.

Documentation

Spokestack Functions

These public methods from the SpokestackTray library

listen

Tell Spokestack to start listening.

This starts the speech pipeline by first checking to make sure that the NLU / Wakeword models have been downloaded and the necessary permissions have been granted before activating the pipeline.

stopListening

Tell Spokestack to stop listening

addToParentView

Adds the SpokestackTrayViewController to the parent view controller's view

removeFromParentView

Stops the speechpipeline and removes the SpokestackTrayViewController from the parent view controller's view

You might also like...
Alfred Workflow to be used with Wooshy, that brings any macOS windows to the foreground

Wooshy: Window to the Foreground! Switch apps with Alfred. Switch app windows with Wooshy: Window to the Foreground! ScreenFlow.mp4 Why Aren't you ann

iOS's Stocks App clone written in React Native for demo purpose (available both iOS and Android).
iOS's Stocks App clone written in React Native for demo purpose (available both iOS and Android).

FinanceReactNative iOS's Stocks App clone written in React Native for demo purpose (available both iOS and Android). Data is pulled from Yahoo Finance

๐Ÿ€ iOS and Android NBA app created with React Native
๐Ÿ€ iOS and Android NBA app created with React Native

Swish An iOS and Android NBA app created with React Native. If you would like to request a feature, find a bug, have a question, or would like to leav

Yummies is my first attempt at building a native iOS app using Swift and SwiftUI

Yummies is my first attempt at building a native iOS app using Swift and SwiftUI. A recipe browser where you can pin your favorite ones. Powered by Edamam Recipe Search API.

A native video wallpaper app for macOS

WallpapaerZ This is an wallpaper software that runs on macOS. It can bring video wallpaper to the desktop The project is coded in storyboard with swif

Demo app to demonstrate native blur capabilities
Demo app to demonstrate native blur capabilities

Blurrable Demo app to demonstrate native blur capabilities. You can customize th

FlutterNativeDragAndDrop - A package that allows you to add native drag and drop support into your flutter app
FlutterNativeDragAndDrop - A package that allows you to add native drag and drop support into your flutter app

native_drag_n_drop A package that allows you to add native drag and drop support

The source code to How to build a news app with react native ๐Ÿ“ฐ
The source code to How to build a news app with react native ๐Ÿ“ฐ

Royal News ยท The source code to How to build a news app with react native article on NimreyCode, medium, and dev.to. Requirements: Android Studio or X

Comments
  • initial code PR

    initial code PR

    I'm still working on README and code documentation. I'm also working on, and will be submitting more PR's, regarding customizations over the next 5 days, as well as, any bugs fixes that come up. The podspec does pass linting, but i haven't published it yet. Do you want to wait until it is at least tag'd as 0.1.0 or update the README to reference the master branch until it is taged?

    opened by kwylez 1
Releases(0.0.5)
Owner
Spokestack
Voice development platform that enables customized voice navigation for mobile and browser applications
Spokestack
Static Native Template and Dynamic Styling without any other app release

FileManager Project Students and Freshers, Good opportunity for you to learn and contribute in this project. Here you would learn how you can change t

Naveen Chauhan 3 Nov 30, 2021
XCSnippetsApp - macOS application to explore code snippets from the Swift and iOS community, view and edit the snippets before adding them conveniently to Xcode

XCSnippetsApp macOS application to explore code snippets from the Swift and iOS community, view and edit the snippets before adding them conveniently

Marco Eidinger 119 Dec 27, 2022
Provides some Apple Wallet functionality, like adding passes, removing passes and checking passises for existing.

react-native-wallet-manager Provides some Apple Wallet's functionality, like adding passes, removing passes and checking passises for existing. Instal

dev.family 50 Nov 12, 2022
React Native library that implements PayPal Checkout flow using purely native code (swift).

react-native-paypal-swift React Native library that implements PayPal Checkout flow using purely native code (swift). Installation npm install react-n

Tibb 6 Nov 28, 2022
Vahesaroyan-react-native-bubble-select - React native bubble picker

@vahesaroyan/react-native-bubble-select React native bubble picker Installation

Vahe Saroyan 0 Jan 30, 2022
WobbleView is an implementation of a recently popular wobble effect for any view in your app

WobbleView is an implementation of a recently popular wobble effect for any view in your app. It can be used to easily add dynamics to user interactions and transitions.

intent 2.2k Nov 29, 2022
React Native Todo List example app which uses Redux for managing app state

react-native-redux-todo-list A sample todo list app developed by using React Native and Redux. How to run the app Install react-native If you don't ha

null 43 Oct 11, 2022
An IPFS client/api Swift Package, with the ability to add and pin any data on iOS/iPadOS/macOS

An IPFS client/api Swift Package, with the ability to add and pin any data on iOS/iPadOS/macOS. Originally bundled with GraniteUI, pulled out for independant use by any party.

Kala 4 Dec 8, 2022
Use this template as a starting point for any Swift 5 module that you want other people to include in their projects

Swift 5 Module Template Use this template as a starting point for any Swift 5 mo

James Knipe 0 Dec 28, 2021
Calculate your tip and split the bill between any number of people

Calculate your tip and split the bill between any number of people

Tirupati Balan 6 Feb 9, 2022