Syntax sugar of OpenTok iOS SDK with Audio/Video communication including screen sharing

Overview

Accelerator Core iOS

Build Status Version Status license MIT Platform PRs Welcome

Tokbox is now known as Vonage

The Accelerator Core is a solution to integrate audio/video communication to any iOS applications via OpenTok platform. Accelerator TextChat and Accelerator Annotation have been deprecated and are now part of Accelerator Core.

Accelerator Core helps you with:

Core:

  • one to one audio/video call
  • multiparty audio/video call
  • one to one screen sharing
  • multiparty screen sharing
  • UI components for handling audio/video enable&disable

TextChat:

  • one to one text chat

Annotation:

  • one to one audio/video call with annotations

Configure, build and run the sample apps

The Accelerator Core workspace contains 3 sample apps, one for the Core, TextChat and Annotation components.

  1. Get values for API Key, Session ID, and Token. See Obtaining OpenTok Credentials for important information.

  2. Install CocoaPods as described in CocoaPods Getting Started.

  3. In Terminal, cd to your project directory and type pod install.

  4. Reopen your project in Xcode using the new *.xcworkspace file.

  5. Replace the following empty strings with the corresponding API Key, Session ID, and Token values:

    - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
        // Override point for customization after application launch.
        sharedSession = [[OTAcceleratorSession alloc] initWithOpenTokApiKey:@"apikey" sessionId:@"sessionid" token:@"token"];
        return YES;
    }
  6. Use Xcode to build and run the app on an iOS simulator or device.

  7. For testing audio/video communication, we include a simple web app to make it easier: Browser-Demo. Simply open it and replace the corresponding API Key, Session ID, and Token values. Then save and load it to the browser. For multiparty, you can achieve by opening up multiple tabs.

  8. You might want to run on other platforms:

Accelerator Core Javascript

Accelerator Core Android

Core Sample Code

Each communicator instance will take the OpenTok session from OTOneToOneCommunicatorDataSource, so this applies to each communicator instance:

  • Passing the session

    - (OTAcceleratorSession *)sessionOfOTOneToOneCommunicator:(OTOneToOneCommunicator *)oneToOneCommunicator {
        return <#OTAcceleratorSession#>;
    }
  • One-to-One

    self.communicator = [[OTOneToOneCommunicator alloc] init];
    self.communicator.dataSource = self;
    [self.communicator connectWithHandler:^(OTCommunicationSignal signal, NSError *error) {
        if (signal == OTPublisherCreated && !error) {
            weakSelf.communicator.publisherView.frame = CGRectMake(0, 0, 100, 100);
            [self.publisherView addSubview:weakSelf.communicator.publisherView];
        }
        else if (signal == OTSubscriberReady && !error) {
            weakSelf.communicator.subscriberView.frame = CGRectMake(0, 0, 100, 100);
            [self.subscriberView addSubview:weakSelf.communicator.subscriberView];
        }
    }];
  • Multiparty

    self.communicator = [[OTMultiPartyCommunicator alloc] init];
    self.communicator.dataSource = self;
    [self.communicator connectWithHandler:^(OTCommunicationSignal signal, OTMultiPartyRemote *subscriber, NSError *error) {
        if (signal == OTPublisherCreated && !error) {
            weakSelf.communicator.publisherView.frame = CGRectMake(0, 0, 100, 100);
            [self.publisherView addSubview:weakSelf.communicator.publisherView];
        }
        else if (signal == OTSubscriberReady && !error) {
            subscriber.subscriberView.frame = <#your desired frame for this remote subscriberview#>;
            // your logic to handle multiple remote subscriberview(s)
        }
    }];
  • Screen Sharing

    Use - (instancetype)initWithView: or - (instancetype)initWithView:name: like so

    self.screenSharer = [[OTOneToOneCommunicator alloc] initWithView:[UIApplication sharedApplication].keyWindow.rootViewController.view];

    or

    self.screenSharer = [[OTMultiPartyCommunicator alloc] initWithView:[UIApplication sharedApplication].keyWindow.rootViewController.view];
  • Pre-defined UI

    Enable audio&video control

    self.communicator.publisherView.controlView.isVerticalAlignment = YES;
    self.communicator.publisherView.controlView.frame = CGRectMake(10, 10, CGRectGetWidth(self.publisherView.frame) * 0.1, CGRectGetHeight(self.publisherView.frame) * 0.3);

    Handle video enable/disable control

    // default
    // enable handleAudioVideo property, the publisherView will be covered by a silhouette automatically.
    self.communicator.publisherView.handleAudioVideo = YES;
    
    // disable handleAudioVideo property, the publisherView will do nothing for enabling/disabling publishVideo.
    self.communicator.publisherView.handleAudioVideo = NO;

Core Ready-to-Use Components

  • One-to-One communication

ready-in-use-2 ready-in-use-3

OTOneToOneCommunicationController *vc = [OTOneToOneCommunicationController oneToOneCommunicationControllerWithSession:<#OTAcceleratorSession#>];
[self.navigationController pushViewController:vc animated:YES];

TextChat Sample Code

  • Passing the session

    - (OTAcceleratorSession *)sessionOfOTOneToOneCommunicator:(OTOneToOneCommunicator *)oneToOneCommunicator {
        return <#OTAcceleratorSession#>;
    }
  • Start signaling text chat data

    // we assume self owns a table tableView
    [self.textChat connectWithHandler:^(OTTextChatConnectionEventSignal signal, OTConnection *connection, NSError *error) {
        if (signal == OTTextChatConnectionEventSignalDidConnect) {
            NSLog(@"Text Chat starts");
        }
        else if (signal == OTTextChatConnectionEventSignalDidDisconnect) {
            NSLog(@"Text Chat stops");
        }
    } messageHandler:^(OTTextChatMessageEventSignal signal, OTTextMessage *message, NSError *error) {
        if (signal == OTTextChatMessageEventSignalDidSendMessage || signal == OTTextChatMessageEventSignalDidReceiveMessage) {
            if (!error) {
                [weakSelf.textMessages addObject:message];
                [weakSelf.tableView reloadData];
            }
        }
    }];
  • Stop signaling text chat data

    [self.textchat disconnect];

JSON Requirements for Text Chat Signaling

The JSON used when using the OpenTok signaling API with the OpenTok Text Chat component describes the information used when submitting a chat message. This information includes the date, chat message text, sender alias, and sender ID. The JSON is formatted as shown in this example:

// var type = "text-chat"
{
    "sentOn" : 1462396461923.305,
    "text" : "Hi",
    "sender" : {
        "alias" : "Tokboxer",
        "id" : "16FEB40D-C09B-4491-A983-44677B7EBB3E"
    }
}

This formatted JSON is converted to a string, which is submitted to the OpenTok signaling API. For more information, see:

For testing text chat, we include a simple web app to make it easier: Browser-Demo-TextChat. Simply open it and replace the corresponding API Key, Session ID, and Token values. Then save and load it to the browser.

Annotation sample code

The OTAnnotationScrollView class is the backbone of the annotation features in this Sample.

self.annotationView = [[OTAnnotationScrollView alloc] init];
self.annotationView.frame = <# desired frame #>;
[self.annotationView initializeToolbarView];
self.annotationView.toolbarView.frame = <# desired frame #>;

Class design

The following classes represent the software design for the OpenTok Annotations Accelerator.

Class Description
OTAnnotator The core component for enabling remote annotation across devices and platforms.
OTAnnotationScrollView Provides essentials components for annotating on either the entire screen or a specified portion of the screen.
OTAnnotationToolbarView A convenient annotation toolbar that is optionally available for your development. As an alternative, you can create your own toolbar using OTAnnotationScrollView.
OTFullScreenAnnotationViewController A convenient view controller enables you to annotate the whole screen immediately.

3rd Party Libraries

Sample Apps that uses the Core

The following sample apps use Accelerator Core:

The accelerator and sample app access the OpenTok session through the Accelerator Session Pack layer, which allows them to share a single OpenTok session:

architecture

On the Android and iOS mobile platforms, when you try to set a listener (Android) or delegate (iOS), it is not normally possible to set multiple listeners or delegates for the same event. For example, on these mobile platforms you can only set one OpenTok signal listener. The Common Accelerator Session Pack, however, allows you to set up several listeners for the same event.

Obtaining OpenTok Credentials

To use OpenTok's framework you need a Session ID, a Token, and an API Key. You can get these values at the OpenTok Developer Dashboard . For production deployment, you must generate the Session ID and Token values using one of the OpenTok Server SDKs.

Development and Contributing

Interested in contributing? We ❤️ pull requests! See the Contribution guidelines.

Getting Help

We love to hear from you so if you have questions, comments or find a bug in the project, let us know! You can either:

Further Reading

Comments
  • Can't install with cocoapods using frameworks

    Can't install with cocoapods using frameworks

    New issue checklist

    General information

    • Library version(s): 1.0.4
    • iOS/Android/Browser version(s): IOS 10
    • Devices/Simulators/Machine affected: N/A
    • Reproducible in the demo project? (Yes/No): N/A
    • Related issues:

    Bug report

    My Podfile uses use_frameworks! since I'm using Swift. When trying to install OTAcceleratorCore, I get an error that there are static binaries:

    [!] The 'Pods-Project' target has transitive dependencies that include static binaries: (/Users/me/code/project/Pods/OTKAnalytics/OTKAnalytics.framework and /Users/me/code/project/Pods/OpenTok/OpenTok.framework)

    Expected behavior

    Should I be able to install using frameworks? Not 100% sure honestly.

    Actual behavior

    pod install errors out

    Steps to reproduce

    Example Podfile:

    target 'TestProj' do
      use_frameworks!
    
      pod 'OpenTok', '~> 2.10.0'
      pod 'OTAcceleratorCore', '~> 1.0.4'
    
    end
    

    Crash log? Screenshots? Videos? Sample project?

    $ pod install
    Analyzing dependencies
    Downloading dependencies
    Installing OTAcceleratorCore (1.0.4)
    Installing OTKAnalytics (2.0.0)
    Installing OpenTok (2.10.1)
    [!] The 'Pods-TestProj' target has transitive dependencies that include static binaries: (/Users/me/code/test-proj/Pods/OTKAnalytics/OTKAnalytics.framework and /Users/me/code/test-proj/Pods/OpenTok/OpenTok.framework)
    

    Question or Feature Request

    Should I be able to install with frameworks? If I comment one use_frameworks! then I can't install other pods that I'm using.

    question 
    opened by seanadkinson 13
  • App crashes when someone else disconnects from video call

    App crashes when someone else disconnects from video call

    New issue checklist

    General information

    • Library version(s): 1.1.2 Release (#33)
    • iOS/Android/Browser version(s): All
    • Devices/Simulators/Machine affected: All
    • Reproducible in the demo project? (Yes):
    • Related issues:

    Bug report

    Expected behavior

    App should disconnect without crashing

    ...

    Actual behavior

    App crashes when someone in the video call hangs up

    ...

    Steps to reproduce

    Make a video conference with at least two devices. Hang up on one of them, the iOS apps of remaining participants will crash.

    ...

    Crash log? Screenshots? Videos? Sample project?

    screen shot 2018-01-10 at 16 26 49 bug 
    opened by emil622 4
  • Consolidate logging

    Consolidate logging

    Pull request checklist

    • [ ] All tests pass. Demo project builds and runs.
    • [ ] I have resolved any merge conflicts. Confirmation: ____

    This fixes issue #___.

    What's in this pull request?

    ...

    enhancement 
    opened by marinaserranomontes 3
  • Improve implementation section for awesome-ios

    Improve implementation section for awesome-ios

    New issue checklist

    Question or Feature Request

    The only problem I found was the README.md, which lacks an Example Section I created this iOS Open source Readme Template so you can take a look on how to easily create an Example Section The only implementation example I found was on how to run the example app, but not actual example code was shown on the README

    What are your thoughts? 😄

    enhancement 
    opened by lfarah 3
  • merge accelerators

    merge accelerators

    Pull request checklist

    • [x] All tests pass. Demo project builds and runs.
    • [x] I have resolved any merge conflicts.

    TODO: Update README TODO: Update travis.yml See https://github.com/opentok/accelerator-core-ios/pull/66

    What's in this pull request?

    This PR brings in the TextChat and Annotation Accelerators into the Core project. It is quite a big PR but you can review commit by commit. The first two commits are mainly copying in the text chat and annotation projects. Below is a screenshot of the project structure. The sample projects and test targets have been copied over with the source files for the accelerators under the core folder.

    image

    opened by abdulajet 2
  • Update AcceleratorCore to use OpenTok v2.16.5

    Update AcceleratorCore to use OpenTok v2.16.5

    Pull request checklist

    • [ ] All tests pass. Demo project builds and runs.
    • [ ] I have resolved any merge conflicts. Confirmation: ____

    This fixes issue #___.

    What's in this pull request?

    ...

    opened by jtiet 2
  • update to 2.16.0

    update to 2.16.0

    Pull request checklist

    • [x] All tests pass. Demo project builds and runs.
    • [x] I have resolved any merge conflicts. Confirmation: ____

    This fixes issue #___.

    What's in this pull request?

    Updates library to 2.15.0

    ...

    opened by msach22 2
  • Fix Badges position

    Fix Badges position

    Pull request checklist

    • [ ] All tests pass. Demo project builds and runs.
    • [ ] I have resolved any merge conflicts. Confirmation: ____

    This fixes issue #___.

    What's in this pull request?

    Fix Badges position Add GitHub Release Badge

    opened by PanoramicRum 2
  • Fix a crash on un-subscribing.

    Fix a crash on un-subscribing.

    Pull request checklist

    • [x] All tests pass. Demo project builds and runs.
    • [x] I have resolved any merge conflicts. Confirmation: ____

    This fixes issue #___.

    What's in this pull request?

    ...

    opened by Lucashuang0802 1
  • Update iOS SDK

    Update iOS SDK

    Pull request checklist

    • [ ] All tests pass. Demo project builds and runs.
    • [ ] I have resolved any merge conflicts. Confirmation: ____

    This fixes issue #___.

    What's in this pull request?

    ...

    opened by marinaserranomontes 1
  • Constraint a version to avoid API changes

    Constraint a version to avoid API changes

    Pull request checklist

    • [ ] All tests pass. Demo project builds and runs.
    • [ ] I have resolved any merge conflicts. Confirmation: ____

    This fixes issue #___.

    What's in this pull request?

    ...

    opened by Lucashuang0802 1
Releases(4.0.0)
A framework for streaming audio between Apple devices using AirPlay.

Airstream An iOS / macOS framework for streaming audio between Apple devices using AirPlay. You can use Airstream to start an AirPlay server in your i

Qasim Iqbal 374 Oct 26, 2022
Camera and Microphone streaming library via RTMP, HLS for iOS, macOS, tvOS.

HaishinKit Camera and Microphone streaming library via RTMP, HLS for iOS, macOS, tvOS. Issuesの言語は、日本語が分かる方は日本語でお願いします! Sponsored with ?? by Enterprise

shogo4405 2.4k Jan 2, 2023
A fast and extensible gapless AudioPlayer/AudioStreamer for OSX and iOS (iPhone, iPad)

StreamingKit StreamingKit (formally Audjustable) is an audio playback and streaming library for iOS and Mac OSX. StreamingKit uses CoreAudio to decomp

Thong Nguyen 2.3k Dec 21, 2022
LaiFeng IOS Live Kit,H264 and AAC Hard coding,support GPUImage Beauty, rtmp transmission,weak network lost frame,Dynamic switching rate

LFLiveKit LFLiveKit is a opensource RTMP streaming SDK for iOS. Features Background recording Support horizontal vertical recording Support Beauty Fac

null 4.3k Jan 6, 2023
AudioPlayer is syntax and feature sugar over AVPlayer. It plays your audio files (local & remote).

AudioPlayer AudioPlayer is a wrapper around AVPlayer. It also offers cool features such as: Quality control based on number of interruption (buffering

Kevin Delannoy 676 Dec 25, 2022
Nextcloud Talk is a fully on-premises audio/video and chat communication service

Nextcloud Talk iOS app Video & audio calls and chat through Nextcloud on iOS Nextcloud Talk is a fully on-premises audio/video and chat communication

Nextcloud 112 Dec 26, 2022
An iOS and macOS audio visualization framework built upon Core Audio useful for anyone doing real-time, low-latency audio processing and visualizations.

A simple, intuitive audio framework for iOS and OSX. Deprecated EZAudio has recently been deprecated in favor of AudioKit. However, since some people

Syed Haris Ali 4.9k Jan 2, 2023
Photo-Sharing-App - Photo Sharing App With Swift

Photo Sharing App You can register and log in to this application and share your

Yağız Savran 2 Jun 14, 2022
OpenTok Text Chat Accelerator Pack enables text messages between mobile or browser-based devices.

Text Chat Accelerator Pack iOS This project is deprecated The OpenTok Text Chat Accelerator Pack for iOS project is now a part of Accelerator Core iOS

OpenTok 13 Jan 26, 2022
Swift's Sugar. Heavily inspired on Objc Sugar

Swift's Sugar. Heavily inspired on Objc Sugar

Rui Peres 154 Jun 29, 2022
📦 KeyPath dynamicMemberLookup based syntax sugar for Swift.

DuctTape ?? KeyPath dynamicMemberLookup based syntax sugar for Swift. let label: UILabel = UILabel().ductTape .numberOfLines(0) .textColor(.re

Taiki Suzuki 171 Nov 4, 2022
High-quality Interactive Audio/Video iOS SDK

腾讯云实时音视频 TRTC SDK 产品介绍 腾讯实时音视频(Tencent Real-Time Communication,TRTC),将腾讯多年来在网络与音视频技术上的深度积累,以多人音视频通话和低延时互动直播两大场景化方案,通过腾讯云服务向开发者开放,致力于帮助开发者快速搭建低成本、低延时、高

LiteAVSDK 8 Nov 3, 2022
High-quality Interactive Audio/Video Unity SDK

简体中文 | English TRTC Unity SDK Overview Leveraging Tencent's many years of experience in network and audio/video technologies, Tencent Real-Time Commun

LiteAVSDK 8 Dec 23, 2022
iOS ReplayKit Screen Share Broadcast Extension Frame Sharing

An iOS app that demonstrates how we can share CMSampleBuffer frames from SampleHandler of Broadcast Extension to the host app and how to pass callbacks to and fro from host app to SampleHandler and vice versa.

IOTric 5 Oct 15, 2022
This "Calculator" application is a simple one screen design of calculator screen i have made this single screen design application just to practice AutoLayout concepts.

Calculator Layout This "Calculator" application is a simple one screen design of calculator screen i have made this single screen design application j

Chetan Parate 1 Oct 29, 2021
Provides a custom presentation modifier that provides more options including full screen presentations. (iOS)

Presentation Also available as a part of my SwiftUI+ Collection – just add it to Xcode 13+ Provides a custom presentation modifier that provides more

SwiftUI+ 15 Dec 3, 2022
AudiosPlugin is a Godot iOS Audio Plugin that resolves the audio recording issue in iOS for Godot Engine.

This plugin solves the Godot game engine audio recording and playback issue in iOS devices. Please open the Audios Plugin XCode Project and compile the project. You can also use the libaudios_plugin.a binary in your project.

null 3 Dec 22, 2022
The Amazing Audio Engine is a sophisticated framework for iOS audio applications, built so you don't have to.

Important Notice: The Amazing Audio Engine has been retired. See the announcement here The Amazing Audio Engine The Amazing Audio Engine is a sophisti

null 523 Nov 12, 2022
Audio player demo based on Swift and SwiftUI, which can play local or network audio.

SwiftAudioDemo Audio player demo based on Swift and SwiftUI, which can play local or network audio. In this demo, I have made a radio player to play n

Jensen Zhang 6 Mar 13, 2022
​ This framework allows developers to quickly manipulate audio and video splicing operations.

MTrack This framework allows developers to quickly manipulate audio and video splicing operations.We welcome your feedback in issues and pull requests

null 1 Nov 15, 2021