---
title: Using Vonage’s OpenTok Linux SDK to Publish Custom Audio and Video – WebRTC.ventures
description: When applications require advanced functionality, such as integrating video from devices that are not easily or even impossible to acquire in a browser environment, the OpenTok Linux SDK (part of the Vonage Video API) provides C++ APIs that developers can use to integrate with devices at the native level.
image: https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Using-Vonages-OpenTok-Linux-SDK-to-Publish-Custom-Audio-and-Video.png
---

[Skip to content](https://blog.webrtc.ventures/2023/10/using-vonages-opentok-linux-sdk-to-publish-custom-audio-and-video#main-content)

![WebRTC.ventures](https://blog.webrtc.ventures/hubfs/logo-main-light.svg)

- [WebRTC Services](https://webrtc.ventures/webrtc-development-services/)
  
  Show submenu for WebRTC Services 
  
    - [Assess](https://webrtc.ventures/services/assess/)
    - [Build](https://webrtc.ventures/services/build/)
    - [Integrate](https://webrtc.ventures/services/integrate/)
    - [Test](https://webrtc.ventures/services/test/)
    - [Manage](https://webrtc.ventures/services/manage/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- Our Partners
  
  Show submenu for Our Partners 
  
    - [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/)
- Stories of Success
  
  Show submenu for Stories of Success 
  
    - [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/)
    - [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/)
    - [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/)
    - [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/)
    - [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/)
    - [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/)
    - [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/)
- WebRTC Resources
  
  Show submenu for WebRTC Resources 
  
    - [WebRTC Live](https://webrtc.ventures/webrtc-live/)
    - [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz)
    - [WebRTC.ventures Blog](https://webrtc.ventures/blog/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
    - [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/)
- [About Us](https://webrtc.ventures/our-team/)
  
  Show submenu for About Us 
  
    - [Blog](https://webrtc.ventures/blog/)
    - [Our Team](https://webrtc.ventures/our-team/)
    - [Jobs](https://webrtc.ventures/jobs/)

Open main navigation

Close main navigation

- [WebRTC Services](https://webrtc.ventures/webrtc-development-services/)
  
  Show submenu for WebRTC Services 
  
    - [Assess](https://webrtc.ventures/services/assess/)
    - [Build](https://webrtc.ventures/services/build/)
    - [Integrate](https://webrtc.ventures/services/integrate/)
    - [Test](https://webrtc.ventures/services/test/)
    - [Manage](https://webrtc.ventures/services/manage/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- Our Partners
  
  Show submenu for Our Partners 
  
    - [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/)
- Stories of Success
  
  Show submenu for Stories of Success 
  
    - [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/)
    - [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/)
    - [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/)
    - [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/)
    - [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/)
    - [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/)
    - [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/)
- WebRTC Resources
  
  Show submenu for WebRTC Resources 
  
    - [WebRTC Live](https://webrtc.ventures/webrtc-live/)
    - [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz)
    - [WebRTC.ventures Blog](https://webrtc.ventures/blog/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
    - [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/)
- [About Us](https://webrtc.ventures/our-team/)
  
  Show submenu for About Us 
  
    - [Blog](https://webrtc.ventures/blog/)
    - [Our Team](https://webrtc.ventures/our-team/)
    - [Jobs](https://webrtc.ventures/jobs/)
- [Contact us](https://webrtc.ventures/contact/)

[Contact us](https://webrtc.ventures/contact/)

 Oct 13, 2023 12:00:00 AM

# Using Vonage’s OpenTok Linux SDK to Publish Custom Audio and Video

[Justin Williams](https://blog.webrtc.ventures/author/justin-williams)

Share: [linkedin-in icon](http://www.linkedin.com/shareArticle?mini=true&url=https://blog.webrtc.ventures/2023/10/using-vonages-opentok-linux-sdk-to-publish-custom-audio-and-video) [X Twitter icon](https://twitter.com/intent/tweet?url=https://blog.webrtc.ventures/2023/10/using-vonages-opentok-linux-sdk-to-publish-custom-audio-and-video) [envelope icon](mailto:?body=https://blog.webrtc.ventures/2023/10/using-vonages-opentok-linux-sdk-to-publish-custom-audio-and-video)

Many CPaaS providers, like the [Vonage Video API](https://www.vonage.com/communications-apis/video/), have JavaScript libraries that allow application developers to build WebRTC video applications. These integrate with CPaaS infrastructure and leverage the WebRTC APIs in the browser. For typical applications, this behavior works well.

Some applications may require more advanced functionality, such as integrating video from devices that are not easily or impossible to acquire in a browser environment. This means that such libraries may be unable to be used with these types of devices. This is where native SDKs, such as [OpenTok Linux SDK](https://tokbox.com/developer/sdks/linux/) (part of the Vonage Video API), can be used.

The OpenTok Linux SDK provides C++ APIs that application developers can use to integrate with devices at the native level. Native libraries do require specific builds for the platform (OS + CPU) they are running in. The OpenTok Linux SDK requires a Linux environment, such as the Debian 12 (amd64) whose OS and architecture are fully supported. Other Linux platforms and CPU architectures are supported as well. Information about system requirements can be found [in Vonage’s documentation](https://tokbox.com/developer/sdks/linux/#system-requirements).

Examples of this can be: integrating multiple cameras in a Raspberry Pi, or transcoding video from any encoding software, like RTMP from OBS for instance. 

Another common use case would be creating a native SDK integration to provide application libraries for both iOS and Android applications, allowing just one place for the video calling integration logic to exist.

The OpenTok developers have provided [a Github repo](https://github.com/opentok/opentok-linux-sdk-samples) with a few different examples of how the OpenTok Linux SDK can be used. The examples there are very useful and provided the insight used for this article.

This article will look at a C++ application that connects to an OpenTok session and publish randomly generated audio and video signals to the session. The source code for [this example application](https://github.com/agilityfeat/opentok-encoder-demo/tree/v1) is available at Github.

# Building and Running OpenTok Linux SDK

Since the SDK is built for Debian, we use a [Docker](https://www.docker.com/) container to build and run the source code.

Note: This project used the [CLion IDE by JetBrains](https://www.jetbrains.com/clion/) as the development environment. The CLion IDE has [integration with Docker to use containers as a toolchain](https://www.jetbrains.com/help/clion/clion-toolchains-in-docker.html#create-docker-toolchain) to build and run. This also integrates well with CMake. Other IDEs may offer similar integrations with tutorials on how to achieve this.  This is not required as Docker can be used directly to shell into the container to build and run.

For this article, we will take a look at the Dockerfile used to create the build environment, the CMakeLists file used to configure the build and some source code to look at how to integrate the APIs. 

The code will connect to an OpenTok Session and publish generated audio and video.

## Development Dockerfile

The Development Dockerfile, is used to create the container image supporting the OpenTok Linux SDK. This Dockerfile is responsible for tasks like downloading dependencies and development tools to set up the build and runtime environment for the source code. The full docker file can be found [here](https://github.com/agilityfeat/opentok-encoder-demo/blob/v1/Dockerfile.cpp-env). This section will look at the key components for the Docker image.

### Base Image

For the base image, we use ubuntu:20.04. This offers a familiar development environment and builds on the supported Debian 12 OS.

```
FROM ubuntu:20.04
```

### OpenTok SDK Dependency

The Dockerfile is made to let the libopentok library version be configurable with a default value.

```
ARG LIBOPENTOK_VERSION=2.26.0
```

The file also installs useful build/development tools and sets the compiler to GCC 10 so the project can be compiled with C++ 20.

```
# OpenTok Linux SDK

RUN wget https://tokbox.com/downloads/libopentok_linux_llvm_x86_64-$LIBOPENTOK_VERSION && \
    tar xvf libopentok_linux_llvm_x86_64-$LIBOPENTOK_VERSION && \
    mv libopentok_linux_llvm_x86_64 /usr/local/src/

ENV LIBOPENTOK_PATH /usr/local/src/libopentok_linux_llvm_x86_64

# link libopentok.so into default location
RUN ln -s /usr/local/src/libopentok_linux_llvm_x86_64/lib/libopentok.so /lib/libopentok.so
```

The OpenTok SDK is installed as per the documentation [here](https://tokbox.com/developer/sdks/linux/#using-the-sdk), and sets the `LIBOPENTOK_PATH` variable to link the shared object file into the default location for the build process.

The rest of the Dockerfile includes dependencies needed for [GStreamer](https://gstreamer.freedesktop.org/) which will be used later.

### Build & Run Image

To build the image run the following command:

```
docker build -t opentok_encoder_builder:latest -f Dockerfile.cpp-env .
```

To run the image as a container:

```
docker run -it --rm --name=opentok_encoder_builder \
 --mount type=bind,source=${PWD},target=/src \
 opentok_encoder_builder:latest \
 bash
```

This command should be run in the directory that contains the source code. The `docker` command will mount this directory to the container under the */src* path. The command will open up a bash shell to the container.

## CMake Project Setup

The project configures the `CMakeLists.txt` file to compile with C++ 20, adds some extra dependencies, and configures libopentok (and any other dependencies). The CMakeLists.txt file can be found [in the provided source code](https://github.com/agilityfeat/opentok-encoder-demo/blob/v1/CMakeLists.txt). This section highlights some key parts of the build file.

### Environment Variables

[Dotenv](https://github.com/laserpants/dotenv-cpp) is used to load environment variables from a file named “.env”. Make sure to create this file and set the required fields (this is also outlined in the [project’s README.md](https://github.com/agilityfeat/opentok-encoder-demo/blob/v1/README.md)):

- **API\_KEY**: OpenTok Project API Key
- **SESSION\_ID**: Session ID for the OpenTok Session
- **TOKEN**: Token for OpenTok publisher

### Project Dependencies

Let’s look at the CMake configuration for Dotenv integration. The CMakeLists file uses `FetchContent` to get the source code directly from dotenv’s github repository.

```
# dotenv

FetchContent_Declare(
        dotenv
        GIT_REPOSITORY https://github.com/laserpants/dotenv-cpp.git
        GIT_TAG master
)
FetchContent_MakeAvailable(dotenv)
```

The CMakeLists file will also copy over the local *.env* file into the CMake build output directory.

```
# Copy .env file for local config
configure_file(${CMAKE_SOURCE_DIR}/.env ${CMAKE_CURRENT_BINARY_DIR}/.env)
```

The following shows how libopentok is configured for the project. The `$LIBOPENTOK_PATH` variable is used with `find_path` to check for the header and binary files and sets them to CMake variables. The created variables are used to include and link to the project.

```
# OpenTok

if (DEFINED ENV{LIBOPENTOK_PATH})
    message(STATUS "Opentok Path $ENV{LIBOPENTOK_PATH}")
    find_path(LIBOPENTOK_HEADER opentok.h PATHS $ENV{LIBOPENTOK_PATH}/include NO_DEFAULT_PATH)
    find_library(LIBOPENTOK_LIBRARIES libopentok NAMES libopentok.so PATHS $ENV{LIBOPENTOK_PATH}/lib NO_DEFAULT_PATH)

    message(STATUS "Opentok header $ENV{LIBOPENTOK_HEADER}")
    message(STATUS "Opentok libs $ENV{LIBOPENTOK_LIBRARIES}")
endif ()
if (NOT LIBOPENTOK_LIBRARIES AND NOT LIBOPENTOK_HEADER)
    pkg_search_module(LIBOPENTOK REQUIRED libopentok)
else ()
    set(LIBOPENTOK_LIBRARY_DIRS $ENV{LIBOPENTOK_PATH}/lib)
    set(LIBOPENTOK_INCLUDE_DIRS $ENV{LIBOPENTOK_PATH}/include)
endif ()

include_directories(${CMAKE_SOURCE_DIR}/src ${LIBOPENTOK_INCLUDE_DIRS})
link_directories(${LIBOPENTOK_LIBRARY_DIRS})

...

target_link_libraries(opentok_encoder
        PRIVATE
        ${LIBOPENTOK_LIBRARIES}
        fmt::fmt
        dotenv
)
```

# Using the SDK Libraries

This section will look at using the main APIs to connect to and publish to an OpenTok session. The project’s *main.cpp* file is available at [the provided source code](https://github.com/agilityfeat/opentok-encoder-demo/blob/v1/src/main.cpp). Ensure at this point you are able to build the project in the docker container. Also ensure that any IDE being used can get the build symbols for the project.

## Project Structure

The following is an [object model diagram](https://en.wikipedia.org/wiki/Object_diagram) that depicts the integration of the application with the libopentok library.

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/image2-Apr-03-2024-06-45-39-1232-PM.png?width=606&height=419&name=image2-Apr-03-2024-06-45-39-1232-PM.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/image2-Apr-03-2024-06-45-39-1232-PM.png?width=606&height=419&name=image2-Apr-03-2024-06-45-39-1232-PM.png)](https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/image2-Apr-03-2024-06-45-39-1232-PM.png)

A few wrapper classes are implemented to make use of the APIs from libopentok. The following sections outline how the classes make use of the APIs to publish generated audio and video.

## Initialization and Connection

The `OpenTokClient` class is responsible for initializing libopentok and managing the session. Here, the API key, OpenTok session ID, and publisher token are saved to be used later. The constructor calls `otc_init`to initialize the library. Many of the libopentok APIs return an `otc_status` which can be compared against `OTC_SUCCESS` to check for errors. If the libopentok fails to initialize then a run time error is thrown to stop program execution. The following shows the constructor for the `OpenTokClient`:

```
// src/main.cpp

OpenTokClient(
        std::string apiKey,
        std::string sessionId,
        std::string token)
        // Save variables for session connection and publishing
        : apiKey(std::move(apiKey)),
          sessionId(std::move(sessionId)),
          token(std::move(token)) {
    // Initialize libopentok
    if (otc_init(nullptr) != OTC_SUCCESS) {
        throw std::runtime_error("Could not init opentok library");
    }
}
```

With the library initialized, we will now look at managing the connection to the OpenTok session and publishing.

The following shows the `startPublishing` method which shows the main components of libopentok:

- Session initialization
- Publisher initialization
- Session connection

```
// src/main.cpp OpenTokClient

/**
 * Initializes the OpenTok Session and a publisher
 * Connects to the Session on initialization success
 */
bool startPublishing() {
    logger.debug(__FUNCTION__);
    if (!initializeSession()) {
        logger.error("{}: unable to initialize session", __FUNCTION__);
        return false;
    }
    if (!initializePublisher()) {
        logger.error("{}: unable to initialize publisher", __FUNCTION__);
        return false;
    }
    if (!connectSession()) {
        logger.error("{}: unable to connect session", __FUNCTION__);
        return false;
    }
    return true;
}
```

### Session Initialization

The *initializeSession* function creates an `otc_session` using the `otc_session_new` API. This requires a `otc_session_callbacks` struct which contains function pointers to callback functions which are invoked for different lifecycle events for the session. This also takes in a `user_data` field which as a `void *`. This is the convention used by libopentok to provide references back into the application code.  
The callback functions are set to static methods on the `OpenTokClient` class. By setting the `user_data`field to `this`, the static callback methods can get access to the `OpenTokClass` instance members, facilitating the integration of libopentok and the application code. We will see an example of this in the Session Connection section.

```
// src/main.cpp OpenTokClient

bool initializeSession() {

    ...

    // create callbacks with function pointers to class functions
    struct otc_session_callbacks sessionCallbacks{
        .on_connected = &on_session_connected,
        .on_disconnected = &on_session_disconnected,
        .on_error = &on_session_error,
        .user_data = this
    };
    // Create session with API Key, Session ID, and callbacks
    session = otc_session_new(
            apiKey.c_str(),
            sessionId.c_str(),
            &sessionCallbacks
    );

    ...
}
```

### Audio Publisher

The `OpenTokAudioPublisher` initializes audio settings for the published audio streams by setting the callback functions for `otc_audio_device_callbacks` (note that without doing this, libopentok will attempt to use the default audio device on the system, the implementation for this application shows how a custom audio device can be configured).

The `initialize` method for the `OpenTokAudioPublisher` calls `otc_set_audio_device` with the `otc_audio_device_callbacks` to integrate the custom audio:

```
// src/main.cpp OpenTokAudioPublisher

bool initialize() {

    ...

    // Create callbacks with function pointers to class functions
    struct otc_audio_device_callbacks audioDeviceCallbacks = {
        .destroy_capturer = &audio_device_destroy_capturer,
        .start_capturer = &audio_device_start_capturer,
        .get_capture_settings = &audio_device_get_capture_settings,
        .user_data = this,
    };
    // Set callbacks to configure custom audio device
    if (otc_set_audio_device(&audioDeviceCallbacks) != OTC_SUCCESS) {
        logger.error("{}: Error setting audio device", __FUNCTION__);
        return false;
    }

    ...

}
```

The `audio_device_start_capturer` callback is invoked when libopentok is ready for the audio device to start capturing audio:

```
// src/main.cpp OpenTokAudioPublisher

static otc_bool audio_device_start_capturer(
        const otc_audio_device *audio_device,
        void *user_data
) {
    // Get pointer to this by casting user_data
    auto _this = static_cast<OpenTokAudioPublisher *>(user_data);
    if (_this == nullptr) {
        return OTC_FALSE;
    }

    _this->logger.debug(__FUNCTION__);
    // Set exit thread flag to false before we create the thread
    _this->exitAudioCapturerThread = false;
    
    // Create worker thread with the start function
    if (otk_thread_create(
            &(_this->audioCapturerThread),
            &capturer_thread_start_function,
            user_data
    ) != 0) {
        return OTC_FALSE;
    }

    return OTC_TRUE;
}
```

This is where using a worker thread to create audio data is created.

The `opentok-linux-sdk-samples` repo shows some example utility classes, such as [the otk\_thread.h class](https://github.com/opentok/opentok-linux-sdk-samples/blob/main/common/src/otk_thread.h), that are used to create threads to run with libopentok and publish audio and video. These utility classes are used in this application as well.

The following code is also from the opentok-linux-sdk-examples repo. Once the buffer with the waveform is created, it is sent to be published by OpenTok using `otc_audio_device_write_capture_data`. This runs on a loop until the `exitAudioCapturerThread` member on `OpenTokAudioPublisher` is set.

Using `otk_thread_create`, a thread is created which will run the `capturer_thread_start_function`. In this function, audio is generated by creating a wave in a buffer:

```
// src/main.cpp OpenTokAudioPublisher

static otk_thread_func_return_type capturer_thread_start_function(void *arg) {
    ...
    // Create a buffer and a time variable for audio data generation
    int16_t samples[480];
    static double time = 0;
    // Set isPublishing to true now that audio will start being generated
    _this->isPublishing_ = true;
    // Poll on the exit flag
    while (!_this->exitAudioCapturerThread.load()) {
        // Generate audio waves into buffer
        for (int i = 0; i < 480; i++) {
            double val = (INT16_MAX * 0.75) * cos(2.0 * M_PI * 4.0 * time / 10.0);
            samples[i] = (int16_t) val;
            time += 10.0 / 480.0;
        }
        // Write audio buffer to publisher
        otc_audio_device_write_capture_data(samples, 480);
        // Sleep to achieve desired audio rate
        usleep(10 * 1000);
    }
    ...
}
```

### Video Publisher

Similarly to the `OpenTokAudioPublisher`, the `OpenTokVideoPublisher` creates a struct with callbacks to initialize the video publishing. The difference for this instance is that `otc_publisher_new` is used to create a `otc_publisher`, which is from libopentok and used for the video integration:

```
// src/main.cpp OpenTokVideoPublisher

bool initialize() {
    // Create callbacks with function pointers to class functions
    // for the custom video device
    struct otc_video_capturer_callbacks videoCapturerCallbacks = {
            .init = &video_capturer_init,
            .destroy = &video_capturer_destroy,
            .start = &video_capturer_start,
            .get_capture_settings = &get_video_capturer_capture_settings,
            .user_data = this
    };
    // Create callbacks with function pointers to class functions
    // for publisher callbacks
    struct otc_publisher_callbacks publisherCallbacks = {
            .on_stream_created = &on_publisher_stream_created,
            .on_stream_destroyed = &on_publisher_stream_destroyed,
            .on_error = &on_publisher_error,
            .user_data = this
    };
    // Create publisher with callback objects
    publisher = otc_publisher_new("opentok-encoder-demo", &videoCapturerCallbacks, &publisherCallbacks);
    if (publisher == nullptr) {
        logger.error("OpenTokPublisher: Could not create otc publisher");
        return false;
    }
    return true;
}
```

Similar to audio, `video_capturer_start` is called when libopentok is ready for the video capture to be started. Here, a thread is created that runs the `capturer_thread_start_function`:

```
// src/main.cpp OpenTokVideoPublisher

static otc_bool video_capturer_start(const otc_video_capturer *capturer, void *user_data) {

    ...
    // Create video publisher worker thread
    if (otk_thread_create(&(_this->videoCapturerThread), &capturer_thread_start_function, _this) != 0) {
        _this->logger.error("Error creating otk thread");
        return OTC_FALSE;
    }

    ...
}
```

In `capturer_thread_start_function`, video frames are generated by using a buffer and randomly choosing a color by performing a logical and with 0xFF to produce a grey hex color, with ARGB color scheme (again from the opentok-linux-sdk-examples repo):

```
// src/main.cpp OpenTokVideoPublisher

/**
 * This function executes on the otk thread created during video capture startup
 * On the otk thread, the program will run a loop to generate rendered frames. The frames are provided to the
 * OpenTok video publisher. The loop will sleep for a duration to achieve the desired frame rate.
 */
static otk_thread_func_return_type capturer_thread_start_function(void *user_data) {

    ...
    // Create a buffer based on the width and height of the video
    // each pixel will use 4 bytes
    auto buffer = (uint8_t *) malloc(
        sizeof(uint8_t) 
        * OpenTokVideoPublisher::width 
        * OpenTokVideoPublisher::height * 4);
    // Check the exit flag
    while (!_this->exitVideoCapturerThread.load()) {
        // Generate video frame data into buffer
        memset(buffer,
            // Randomly select a hue of gray to be written to the frame
            generate_random_integer() & 0xFF,
            OpenTokVideoPublisher::width 
            * OpenTokVideoPublisher::height * 4);
        // Create a video frame for the OpenTok publisher from the buffer
        auto otcFrame = otc_video_frame_new(
            OTC_VIDEO_FRAME_FORMAT_ARGB32,
            OpenTokVideoPublisher::width,
            OpenTokVideoPublisher::height,
            buffer);
        // Write the video frame to the OpenTok video publisher
        if (otc_video_capturer_provide_frame(
                _this->videoCapturer, 0, otcFrame
             ) != OTC_SUCCESS) {
            _this->logger.error("Unable to provide frame");
        }
        if (otcFrame != nullptr) {
            otc_video_frame_delete(otcFrame);
        }
        // Sleep to achieve desired frame rate
        usleep(1000 / OpenTokVideoPublisher::fps * 1000);
    }
    
    ...
}
```

The loop sleeps for enough time to achieve the desired frame rate.

### Session Connection

Now that the session and the publishers are initialized, the session can be connected. Back in `OpenTokClient#startPublishing`, *connectSession* is called next.

All `connectSession` does is invoke `otc_session_connect`:

```
// src/main.cpp OpenTokClient

bool connectSession() {
    if (session == nullptr) {
        logger.error("{}: Could not create opentok session", __FUNCTION__);
        return false;
    }
    // Establish connection to OpenTok session
    if (otc_session_connect(session, token.c_str()) != OTC_SUCCESS) {
        logger.error("{}: could not connect session", __FUNCTION__);
        return false;
    }
    return true;
}
```

This starts the connection process for libopentok to the OpenTok session. From here the callbacks set up in `OpenTokClient#initializeSession` are invoked.

Upon successful session connection, the `on_session_connected` callback function is invoked:

```
// src/main.cpp OpenTokClient

static void on_session_connected(otc_session *session, void *user_data) {
    auto _this = static_cast<OpenTokClient *>(user_data);
    _this->logger.debug(__FUNCTION__);
    // Set session connection status flag
    _this->isConnected_ = true;

    if (session == nullptr) {
        _this->logger.error("{}: session is null", __FUNCTION__);
        return;
    }
    if (_this->videoPublisher == nullptr) {
        _this->logger.error("{}: publisher is null",
        __FUNCTION__);
        return;
    }
    // Start audio and video publishing to session
    if (!_this->videoPublisher->publishToSession(session)) {
        _this->logger.error("{}: could not publish to session",
        __FUNCTION__);
        return;
    }

    _this->logger.debug("{}: session successfully connected",
        __FUNCTION__);
}
```

This chains into a call to the `OpenTokVideoPublisher` to start publishing video to the session. `OpenTokVideoPublisher#publishToSession` then simply calls `otc_session_publish`:

```
// src/main.cpp OpenTokVideoPublisher
bool publishToSession(otc_session *session) {
    if (!publisher) {
        logger.error("{}: publisher is null", __FUNCTION__);
        return false;
    }
    // Publish video to session
    if (otc_session_publish(session, publisher) != OTC_SUCCESS) {
        logger.error("{}: could not publish to session", __FUNCTION__);
        return false;
    }
    return true;
}
```

From here, the publisher callbacks will be invoked and the worker threads will start producing video frames and audio data to libopentok to be published. Subscribe to the session in an application that can join the video call to see the video stream coming from our custom encoder:

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/image1.gif?width=500&height=500&name=image1.gif) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/image1.gif?width=500&height=500&name=image1.gif)](https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/image1.gif)

# Summary

This article shows the key components to integrate a C++ application with libopentok. From here, it would be good to explore the details of sending audio and video to be published. The APIs libopentok provide are at a very low level, where raw bits for audio and video are sent to the APIs. 

This is where the raw or uncompressed video format details are very important, and libopentok provides different APIs to generate Video Frames from different formats, eg:

- **otc\_video\_frame\_new\_I420**: Creates a new video frame with I420 format.
- **otc\_video\_frame\_new\_MJPEG**: Creates a new video frame with MJPEG format.
- **otc\_video\_frame\_new\_from\_planes**: Creates a new video frame with a given format from its planes.

These allow for different devices or video/audio sources to be used without having to transcode too much, providing a more optimal integration.

*It’s no exaggeration to say that no one can claim longer-standing experience with the Vonage Video API than our team at WebRTC.ventures. **Leverage our expertise as the [original Vonage Video partner](https://webrtc.ventures/partners/vonage/) by [contacting us today](https://webrtc.ventures/contact/)!***

[CPaaS](https://blog.webrtc.ventures/tag/cpaas), [Managed Services](https://blog.webrtc.ventures/tag/managed-services), [Events](https://blog.webrtc.ventures/tag/events), [Video Conferencing](https://blog.webrtc.ventures/tag/video-conferencing), [Real Time Weekly](https://blog.webrtc.ventures/tag/real-time-weekly), [WebRTC Architecture](https://blog.webrtc.ventures/tag/webrtc-architecture), [Broadcasting](https://blog.webrtc.ventures/tag/broadcasting), [Medical](https://blog.webrtc.ventures/tag/medical), [WebRTC Live](https://blog.webrtc.ventures/tag/webrtc-live), [Showcase](https://blog.webrtc.ventures/tag/showcase), [Thoughts](https://blog.webrtc.ventures/tag/thoughts), [Vonage](https://blog.webrtc.ventures/tag/vonage), [Daily](https://blog.webrtc.ventures/tag/daily), [Jobs](https://blog.webrtc.ventures/tag/jobs), [Contact Centers](https://blog.webrtc.ventures/tag/contact-centers), [Telehealth](https://blog.webrtc.ventures/tag/telehealth), [Education](https://blog.webrtc.ventures/tag/education), [Technical](https://blog.webrtc.ventures/tag/technical), [General](https://blog.webrtc.ventures/tag/general), [Story of success](https://blog.webrtc.ventures/tag/story-of-success), [The WebRTC.ventures Blog](https://blog.webrtc.ventures/tag/the-webrtc-ventures-blog), [Testing](https://blog.webrtc.ventures/tag/testing), [Gaming](https://blog.webrtc.ventures/tag/gaming), [DevOps](https://blog.webrtc.ventures/tag/devops), [Symbl.ai](https://blog.webrtc.ventures/tag/symbl-ai)

## Related posts

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Default-B.jpg?height=200&name=Default-B.jpg)](https://blog.webrtc.ventures/2022/12/hamza-vonage-attention-detection)

## [Hamza blogs on Vonage about Attention Detection with the Vonage Video API](https://blog.webrtc.ventures/2022/12/hamza-vonage-attention-detection)

![Picture of Jen Oppenheimer](https://blog.webrtc.ventures/hs-fs/hubfs/Our-Team-Jen.jpg?width=50&name=Our-Team-Jen.jpg) [Jen Oppenheimer](https://blog.webrtc.ventures/author/jen-oppenheimer) 

 Dec 1, 2022 12:00:00 AM

Hamza Nasir is a WebRTC Developer on the WebRTC.ventures team currently working with Vonage on...

[Read more](https://blog.webrtc.ventures/2022/12/hamza-vonage-attention-detection)

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Enhancing-Customer-Service-Experiences-with-Vonage-and-Symbl.jpg?height=200&name=Enhancing-Customer-Service-Experiences-with-Vonage-and-Symbl.jpg)](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai)

## [Enhancing Customer Service Experiences with Vonage and Symbl.ai](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai)

[Fahad Mahmood](https://blog.webrtc.ventures/author/fahad-mahmood) 

 Feb 24, 2023 12:00:00 AM

The combination of WebRTC and Artificial Intelligence (AI) is enhancing customer service activities...

[Read more](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai)

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/The-Vonage-AI-Studio-A-Sample-Integration-with-SimplyDoc.png?height=200&name=The-Vonage-AI-Studio-A-Sample-Integration-with-SimplyDoc.png)](https://blog.webrtc.ventures/2022/04/the-vonage-ai-studio-telehealth-integration)

## [The Vonage AI Studio: A Sample Integration with SimplyDoc](https://blog.webrtc.ventures/2022/04/the-vonage-ai-studio-telehealth-integration)

[ArinSime](https://blog.webrtc.ventures/author/arinsime) 

 Apr 14, 2022 12:00:00 AM

Users today expect seamless, omnichannel communication across their devices. This is the promise of...

[Read more](https://blog.webrtc.ventures/2022/04/the-vonage-ai-studio-telehealth-integration)

### WebRTC.Ventures Services

...

- Assess
- Build
- Integrate
- Test
- Deploy & Manage

### Contact Us

...

**Video should be an opportunity, not a headache.**   
 We’re here to build, integrate, assess and optimize, test, and even deploy and manage your live video application. 

[Contact us](https://blog.webrtc.ventures/2023/10/using-vonages-opentok-linux-sdk-to-publish-custom-audio-and-video#contact)

### WebRTC Live

...

 View [past episodes](https://blog.webrtc.ventures/webrtc-live/) of our webinar interview series about the latest webRTC use cases and technical updates. Join our mailing list to learn about upcoming episodes! 

![WebRTC Live](https://blog.webrtc.ventures/hs-fs/hubfs/Template%20Images/widged-bannet.webp?height=90&name=widged-bannet.webp)

![logo-main-light](https://blog.webrtc.ventures/hubfs/logo-main-light.svg "logo-main-light")

We’re one of the few agencies in the world dedicated to WebRTC development. This dedication and experience is why so many people trust us to help bring live video application dreams to life.

 

### Let's get started!

###### **[Contact us today](https://webrtc.ventures/contact/)**

**[info@webrtc.ventures](mailto:info@webrtc.ventures)**

<https://www.linkedin.com/company/webrtc-ventures> <https://www.youtube.com/@WebrtcVentures>

### Join our mailing list

We don't e-mail often (a few times a month max), and it will be worth it when we do! Of course, we respect your privacy and will not share your contact information.

© 2023 WebRTC.ventures, an [AgilityFeat](https://agilityfeat.com/) company / [Privacy Policy](https://webrtc.ventures/privacy-policy//)

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Justin Williams",
    "url" : "https://blog.webrtc.ventures/author/justin-williams"
  },
  "dateModified" : "2024-04-03T18:50:36.808Z",
  "datePublished" : "2023-10-13T04:00:00.000Z",
  "headline" : "Using Vonage’s OpenTok Linux SDK to Publish Custom Audio and Video – WebRTC.ventures",
  "image" : [ "https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Using-Vonages-OpenTok-Linux-SDK-to-Publish-Custom-Audio-and-Video.png" ],
  "mainEntityOfPage" : {
    "@id" : "https://blog.webrtc.ventures/2023/10/using-vonages-opentok-linux-sdk-to-publish-custom-audio-and-video",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://blog.webrtc.ventures/hubfs/logo-main-light.svg"
    }
  }
}
```