--- title: Enhancing Customer Service Experiences with Vonage and Symbl.ai – WebRTC.ventures description: Fahad demonstrates building a call center app that enhances customer experience by combining the Vonage Video APIs with Symbl.ai to gather real-time insights like sentiment analysis, question insights, and transcripts. image: https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Enhancing-Customer-Service-Experiences-with-Vonage-and-Symbl.jpg --- [Skip to content](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai#main-content) ![WebRTC.ventures](https://blog.webrtc.ventures/hubfs/logo-main-light.svg) - [WebRTC Services](https://webrtc.ventures/webrtc-development-services/) Show submenu for WebRTC Services - [Assess](https://webrtc.ventures/services/assess/) - [Build](https://webrtc.ventures/services/build/) - [Integrate](https://webrtc.ventures/services/integrate/) - [Test](https://webrtc.ventures/services/test/) - [Manage](https://webrtc.ventures/services/manage/) - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/) - Our Partners Show submenu for Our Partners - [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/) - Stories of Success Show submenu for Stories of Success - [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/) - [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/) - [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/) - [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/) - [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/) - [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/) - [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/) - WebRTC Resources Show submenu for WebRTC Resources - [WebRTC Live](https://webrtc.ventures/webrtc-live/) - [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz) - [WebRTC.ventures Blog](https://webrtc.ventures/blog/) - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/) - [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/) - [About Us](https://webrtc.ventures/our-team/) Show submenu for About Us - [Blog](https://webrtc.ventures/blog/) - [Our Team](https://webrtc.ventures/our-team/) - [Jobs](https://webrtc.ventures/jobs/) Open main navigation Close main navigation - [WebRTC Services](https://webrtc.ventures/webrtc-development-services/) Show submenu for WebRTC Services - [Assess](https://webrtc.ventures/services/assess/) - [Build](https://webrtc.ventures/services/build/) - [Integrate](https://webrtc.ventures/services/integrate/) - [Test](https://webrtc.ventures/services/test/) - [Manage](https://webrtc.ventures/services/manage/) - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/) - Our Partners Show submenu for Our Partners - [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/) - Stories of Success Show submenu for Stories of Success - [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/) - [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/) - [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/) - [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/) - [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/) - [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/) - [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/) - WebRTC Resources Show submenu for WebRTC Resources - [WebRTC Live](https://webrtc.ventures/webrtc-live/) - [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz) - [WebRTC.ventures Blog](https://webrtc.ventures/blog/) - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/) - [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/) - [About Us](https://webrtc.ventures/our-team/) Show submenu for About Us - [Blog](https://webrtc.ventures/blog/) - [Our Team](https://webrtc.ventures/our-team/) - [Jobs](https://webrtc.ventures/jobs/) - [Contact us](https://webrtc.ventures/contact/) [Contact us](https://webrtc.ventures/contact/) Feb 24, 2023 12:00:00 AM # Enhancing Customer Service Experiences with Vonage and Symbl.ai [Fahad Mahmood](https://blog.webrtc.ventures/author/fahad-mahmood) Share: [linkedin-in icon](http://www.linkedin.com/shareArticle?mini=true&url=https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai) [X Twitter icon](https://twitter.com/intent/tweet?url=https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai) [envelope icon](mailto:?body=https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai) The combination of WebRTC and Artificial Intelligence (AI) is enhancing customer service activities in contact centers across a wide range of industries. Hello, everyone. This is Fahad from [WebRTC.ventures](https://webrtc.ventures/) and I am [back again](https://webrtc.ventures/2022/11/best-disruptive-digital-hack-at-tadhack-global-22/) to share my exciting WebRTC adventures! This time, I took my [Vonage](http://vonage.com/) ship to the [Symbl.ai](http://symbl.ai/)universe to explore conversational intelligence. I developed a Customer Representative Application to prove how real time conversational insights can improve customer experience for airlines. Let’s take a glimpse of this journey together! ## The Mission My goal was to first connect customers with an airline agent through the [Vonage Video APIs](https://www.vonage.com/communications-apis/video/). Then, to process the audio stream of both the customer and the airline-agent using [Symbl.ai](http://symbl.ai/)to gather real-time insights like[sentiment analysis](https://docs.symbl.ai/docs/sentiment-analysis), [question insights](https://docs.symbl.ai/docs/questions), [transcripts](https://docs.symbl.ai/docs/formatted-transcript), etc. Finally, agents would see their call report screen after the call ended and send details (e.g. booking information) via message to the customer through [Vonage SMS APIs](https://www.vonage.com/communications-apis/sms/)**. ** I took my [Vonage](http://vonage.com/) ship from the [Next.JS](http://nextjs.org/) station and set my final destination to the [Symbl.ai](http://symbl.ai/) universe. The application architecture looked like this: [![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Symbl-Vonage_architecture-1024x958.png?width=512&height=479&name=Symbl-Vonage_architecture-1024x958.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Symbl-Vonage_architecture-1024x958.png?width=512&height=479&name=Symbl-Vonage_architecture-1024x958.png)](https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Symbl-Vonage_architecture.png)               Architecture Diagram: Vonage and Symbl.ai Airline Service Application ## Implementing Call Room Using Vonage Using Vonage SDKs for video communication on the web is a fun and seamless experience. The [quick start tutorial](https://tokbox.com/developer/tutorials/web/basic-video-chat/) helps you to quickly spin up your web application. You use Vonage’s server SDK to generate sessions and access tokens that clients will use to access the Vonage infrastructure based on your application requirements. This is shown in the code example below: ``` import opentok from '@libs/opentok'; // Generate Session Id opentok.createSession(sessionOptions, (error, session) => { console.log(session.sessionId) // Generated Session Id onSuccessCallback(sessionId); }); // Generate Session Access Token const token = opentok.generateToken(sessionId); ``` The next step is to initialize and connect with the session using `sessionId` and access token on the client side. Once done, we can **subscribe** or **publish** streams and listen to different session events as well. ``` // Initialize Client Session using sessionId const sessionId = await fetchSessionId(); const session = OT.initializeSession(ACCESS_TOKEN, sessionId); // Subscribe to Remote Stream session.on('streamCreated', event => { session.subscribe(event.stream, 'subscriber', OPTIONS); }); // Connect Session and Publish Local MediaStream const stream = getLocalMediaStream(); const publisher = OT.initPublisher('publisher', PUBLISH_OPTIONS); session.connect(ACCESS_TOKEN, (error) => { if(!errror) { session.publish(publisher); } }) ``` I quickly created two room screens. One for the customer view and a second for the airline agent to connect with each other.  The customer view shows their own published video and airline agent’s subscribed video. The agent screen has some advanced options, as well. This will make their work more productive and thus improve the customer experience. We will talk more about this later. Here’s a quick peek of what the both screens initially looked: [![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) ![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png)](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) Customer Screen [![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) ![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png)](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) Initial Agent Screen Now my Vonage spaceship is ready and fully fueled to explore the Symbl.ai universe! ## Exploring the Symbl.ai Playground Once I created an account on [Symbl.ai](http://symbl.ai/), I headed over to the playground to experiment with different conversational intelligence features. First, I played around with a conversation example from the [Streaming API](https://docs.symbl.ai/docs/streaming-api), as this is what I needed for my application. There is a sample script that you can read from and record your own audio to see how it actually works. I noticed, after trying a few times, that it takes a little bit of time before you actually get conversation results. If you are speaking in a natural flow and you stop for a second, Symbl.ai understands that you may have finished your sentence. That is when you start seeing results.  Once I knew what I needed for my application and how I could improve the experience for my customers in a real-time conversation, it was time to read the [docs](https://docs.symbl.ai/) and see how this can be implemented on my web application. [![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) ![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png)](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) Symbl.ai Playground Streaming API ## Understanding Symbl.ai Symbl.ai works in three steps. First, you have to generate an authentication **token** for accessing the Symbl.ai APIs.  Once that’s done, you can **start processing the audio for conversational intelligence**. This can be accomplished in multiple ways. 1. Async APIs: They can be used for pre-recorded audio / text files. 2. Streaming APIs: They provide conversation intelligence for real-time audio using websockets. 3. Telephony API: They process audio from telephony SIP or PSTN interfaces. Once the Audio API starts processing, you can **use conversational intelligence APIs to fetch insights**. ## Implementing Symbl.ai I chose WebSDK in order to use the Streaming API. As from the [docs example](https://docs.symbl.ai/docs/getting-live-transcripts-and-conversation-intelligence), things like creating a websocket connection and fetching and sending audio streams were handled by the SDK. I only needed to call Symbl.ai’s createConnection and startProcessing SDK methods. ``` const connection = await symbl.createConnection(); await connection.startProcessing({ insightTypes: ["question", "action_item"], config: { encoding: "OPUS" }, speaker: { userId: "agnet@airwebrtc.com", name: "Fahad" } }); ``` Once I had implemented WebSDK, I noticed that I wasn’t getting live sentiment results with each message processed. Luckily, I found a sentiment analysis [tutorial](https://docs.symbl.ai/docs/live-sentiment-analysis) that connects using the websocket URL without the SDK and fetched sentiment analysis results. ``` const symblSocket = new WebSocket(SOCKET_URL); const processSymblStream = () => { // fetch Stream Buffer const streamBuffer = fetchStreamBuffer(); symblSocket.send(symblSocketBuffer); } symblSocket.onOpen = () => { // end local audio stream through socket processSymblStream(); symblSocket.send(REQUEST_MESSAGE); } symblSocket.onMessage = (event) => { displayTranscriptAndSentiments(event); } ``` For this, I followed a similar approach. Once both the user and the airline agent connect, I would send their streams to [Symbl.ai](http://symbl.ai/) using my custom websocket connection. The customer didn’t have to do much apart from sending the stream and closing the connection when needed. On the other hand, agents also listened to different websocket message events in order to fetch and display live sentiment results, question insights, etc. With question insights displayed, agents could remember all the questions their customers have asked them and are able to promptly reply to make sure the customer has the best experience.  The chat transcript screen will show the live transcript to improve the conversation and will also display sentiments. It shows three different sentiments: neutral, negative, and positive. Based on the sentiment, an agent can take appropriate steps to improve the customer experience. Some advancements had been made since the tutorial was created. I didn’t have to manually call the get message API on each message received from the web socket or by setting up an interval. Instead, I could just listen to the message data containing **message\_response** as the type through the websocket. In fact, it was much faster than setting up an interval or calling the API on each message.  My agent screen was now ready! I was able to create a solution using AI that will allow customer service agents to provide the best experience, in my case for airline customers. [![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) ![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png)](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) Agent Chat Screen  ## Agent Call Summary / Action Insights The only thing left was to create a call summary / action screen for the agent. Once a call is finished, a full call report can be seen and appropriate action can be taken by the agent. Using Symbl.ai, I displayed customer question insights and chat transcript similar to the previous agent screen. The API responses were different from the websocket responses to fetch transcript with sentiments. I had to format the data in order to display them correctly. Another important section was send SMS section to send different messages to the customer. [Vonage SMS API](https://www.vonage.com/communications-apis/sms/) helped me to quickly implement this feature. I created a [NextJS](https://nextjs.org/) serverless API to send SMS to the customers and connected it with the frontend. ``` import {Vonage} from '@vonage/server-sdk'; const vonage = new Vonage({apiKey, apiSecret}); await vonage.sms.send({to, from, message}) ``` This is how the final call end screen looked for the agent: [![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) ![](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png)](https://static.hsstatic.net/BlogImporterAssetsUI/ex/missing-image.png) Agent Summary Screen ## Conclusion The future is full of WebRTC and Artificial Intelligence (AI)! I hope you enjoyed learning how to create this application. ***Please*** [***reach out***](https://webrtc.ventures/contact/) ***to our experienced team at WebRTC.ventures if you want to empower your contact center’s customer service with WebRTC and AI***. [CPaaS](https://blog.webrtc.ventures/tag/cpaas), [Managed Services](https://blog.webrtc.ventures/tag/managed-services), [Events](https://blog.webrtc.ventures/tag/events), [Video Conferencing](https://blog.webrtc.ventures/tag/video-conferencing), [Real Time Weekly](https://blog.webrtc.ventures/tag/real-time-weekly), [WebRTC Architecture](https://blog.webrtc.ventures/tag/webrtc-architecture), [Broadcasting](https://blog.webrtc.ventures/tag/broadcasting), [Medical](https://blog.webrtc.ventures/tag/medical), [WebRTC Live](https://blog.webrtc.ventures/tag/webrtc-live), [Showcase](https://blog.webrtc.ventures/tag/showcase), [Thoughts](https://blog.webrtc.ventures/tag/thoughts), [Vonage](https://blog.webrtc.ventures/tag/vonage), [Daily](https://blog.webrtc.ventures/tag/daily), [Jobs](https://blog.webrtc.ventures/tag/jobs), [Contact Centers](https://blog.webrtc.ventures/tag/contact-centers), [Telehealth](https://blog.webrtc.ventures/tag/telehealth), [Education](https://blog.webrtc.ventures/tag/education), [Technical](https://blog.webrtc.ventures/tag/technical), [General](https://blog.webrtc.ventures/tag/general), [Story of success](https://blog.webrtc.ventures/tag/story-of-success), [The WebRTC.ventures Blog](https://blog.webrtc.ventures/tag/the-webrtc-ventures-blog), [Testing](https://blog.webrtc.ventures/tag/testing), [Gaming](https://blog.webrtc.ventures/tag/gaming), [DevOps](https://blog.webrtc.ventures/tag/devops), [Symbl.ai](https://blog.webrtc.ventures/tag/symbl-ai) ## Related posts [![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Building-an-AI-TravelAssist-with-Symbl_ai-Nebula-LLM-and-Amazon-Connect.png?height=200&name=Building-an-AI-TravelAssist-with-Symbl_ai-Nebula-LLM-and-Amazon-Connect.png)](https://blog.webrtc.ventures/2024/02/building-ai-agent-assist-with-symbl-ai-nebula-llm-and-amazon-connect) ## [Building AI Agent Assist with Symbl.ai Nebula LLM and Amazon Connect](https://blog.webrtc.ventures/2024/02/building-ai-agent-assist-with-symbl-ai-nebula-llm-and-amazon-connect) [Samson Nkrumah](https://blog.webrtc.ventures/author/samson-nkrumah) Feb 19, 2024 12:00:00 AM In my last post, Building an AI Travel Agent with Symbl.ai’s Nebula LLM and Amazon Connect, I spoke... [Read more](https://blog.webrtc.ventures/2024/02/building-ai-agent-assist-with-symbl-ai-nebula-llm-and-amazon-connect) [![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/WebRTC_ventures-is-a-Symbl_ai-Product-Partner.png?height=200&name=WebRTC_ventures-is-a-Symbl_ai-Product-Partner.png)](https://blog.webrtc.ventures/2024/01/webrtc-ventures-is-a-symbl-ai-product-partner) ## [WebRTC.ventures is a Symbl.ai Product Partner](https://blog.webrtc.ventures/2024/01/webrtc-ventures-is-a-symbl-ai-product-partner) ![Picture of Jen Oppenheimer](https://blog.webrtc.ventures/hs-fs/hubfs/Our-Team-Jen.jpg?width=50&name=Our-Team-Jen.jpg) [Jen Oppenheimer](https://blog.webrtc.ventures/author/jen-oppenheimer) Jan 19, 2024 12:00:00 AM WebRTC.ventures is excited to announce our official status as a trusted Symbl.ai Product Partner.... [Read more](https://blog.webrtc.ventures/2024/01/webrtc-ventures-is-a-symbl-ai-product-partner) [![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Voice-Transcription-with-Symbl_ai-and-the-Vonage-Video-API.png?height=200&name=Voice-Transcription-with-Symbl_ai-and-the-Vonage-Video-API.png)](https://blog.webrtc.ventures/2021/10/voice-transcription-with-symbl-ai-and-the-vonage-video-api) ## [Voice Transcription with Symbl.ai and the Vonage Video API](https://blog.webrtc.ventures/2021/10/voice-transcription-with-symbl-ai-and-the-vonage-video-api) [Hamza Nasir](https://blog.webrtc.ventures/author/hamza-nasir) Oct 25, 2021 12:00:00 AM Voice transcription, speech-to-text, and live closed captioning are in popular demand in today’s... [Read more](https://blog.webrtc.ventures/2021/10/voice-transcription-with-symbl-ai-and-the-vonage-video-api) ### WebRTC.Ventures Services ... - Assess - Build - Integrate - Test - Deploy & Manage ### Contact Us ... **Video should be an opportunity, not a headache.** We’re here to build, integrate, assess and optimize, test, and even deploy and manage your live video application. [Contact us](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai#contact) ### WebRTC Live ... View [past episodes](https://blog.webrtc.ventures/webrtc-live/) of our webinar interview series about the latest webRTC use cases and technical updates. Join our mailing list to learn about upcoming episodes! ![WebRTC Live](https://blog.webrtc.ventures/hs-fs/hubfs/Template%20Images/widged-bannet.webp?height=90&name=widged-bannet.webp) ![logo-main-light](https://blog.webrtc.ventures/hubfs/logo-main-light.svg "logo-main-light") We’re one of the few agencies in the world dedicated to WebRTC development. This dedication and experience is why so many people trust us to help bring live video application dreams to life.   ### Let's get started! ###### **[Contact us today](https://webrtc.ventures/contact/)** **[info@webrtc.ventures](mailto:info@webrtc.ventures)** ### Join our mailing list We don't e-mail often (a few times a month max), and it will be worth it when we do! Of course, we respect your privacy and will not share your contact information. © 2023 WebRTC.ventures, an [AgilityFeat](https://agilityfeat.com/) company / [Privacy Policy](https://webrtc.ventures/privacy-policy//) ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Fahad Mahmood", "url" : "https://blog.webrtc.ventures/author/fahad-mahmood" }, "dateModified" : "2024-04-03T18:39:30.665Z", "datePublished" : "2023-02-24T05:00:00.000Z", "headline" : "Enhancing Customer Service Experiences with Vonage and Symbl.ai – WebRTC.ventures", "image" : [ "https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Enhancing-Customer-Service-Experiences-with-Vonage-and-Symbl.jpg" ], "mainEntityOfPage" : { "@id" : "https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai", "@type" : "WebPage" }, "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://blog.webrtc.ventures/hubfs/logo-main-light.svg" } } } ```