---
title: Building an AI Travel Agent with Symbl.ai’s Nebula LLM and Amazon Connect – WebRTC.ventures
description: Generative AI is a powerful tool for call centers. Samson guides you through building an AI Travel Agent (aka voicebot) with the integration of Symbl.ai’s Nebula Large Language Model (LLM), Symbl.ai’s Streaming API for real-time transcription, and Amazon Connect’s robust communication platform.
image: https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Building-an-AI-Travel-Agent-with-Symbl_ai-Nebula-LLM-and-Amazon-Connect.png
---
[Skip to content](https://blog.webrtc.ventures/2024/02/building-an-ai-travel-agent-with-symbl-ais-nebula-llm-and-amazon-connect#main-content)
- [WebRTC Services](https://webrtc.ventures/webrtc-development-services/)
Show submenu for WebRTC Services
- [Assess](https://webrtc.ventures/services/assess/)
- [Build](https://webrtc.ventures/services/build/)
- [Integrate](https://webrtc.ventures/services/integrate/)
- [Test](https://webrtc.ventures/services/test/)
- [Manage](https://webrtc.ventures/services/manage/)
- [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- Our Partners
Show submenu for Our Partners
- [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/)
- Stories of Success
Show submenu for Stories of Success
- [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/)
- [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/)
- [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/)
- [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/)
- [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/)
- [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/)
- [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/)
- WebRTC Resources
Show submenu for WebRTC Resources
- [WebRTC Live](https://webrtc.ventures/webrtc-live/)
- [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz)
- [WebRTC.ventures Blog](https://webrtc.ventures/blog/)
- [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/)
- [About Us](https://webrtc.ventures/our-team/)
Show submenu for About Us
- [Blog](https://webrtc.ventures/blog/)
- [Our Team](https://webrtc.ventures/our-team/)
- [Jobs](https://webrtc.ventures/jobs/)
Open main navigation
Close main navigation
- [WebRTC Services](https://webrtc.ventures/webrtc-development-services/)
Show submenu for WebRTC Services
- [Assess](https://webrtc.ventures/services/assess/)
- [Build](https://webrtc.ventures/services/build/)
- [Integrate](https://webrtc.ventures/services/integrate/)
- [Test](https://webrtc.ventures/services/test/)
- [Manage](https://webrtc.ventures/services/manage/)
- [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- Our Partners
Show submenu for Our Partners
- [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/)
- Stories of Success
Show submenu for Stories of Success
- [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/)
- [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/)
- [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/)
- [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/)
- [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/)
- [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/)
- [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/)
- WebRTC Resources
Show submenu for WebRTC Resources
- [WebRTC Live](https://webrtc.ventures/webrtc-live/)
- [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz)
- [WebRTC.ventures Blog](https://webrtc.ventures/blog/)
- [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/)
- [About Us](https://webrtc.ventures/our-team/)
Show submenu for About Us
- [Blog](https://webrtc.ventures/blog/)
- [Our Team](https://webrtc.ventures/our-team/)
- [Jobs](https://webrtc.ventures/jobs/)
- [Contact us](https://webrtc.ventures/contact/)
[Contact us](https://webrtc.ventures/contact/)
Feb 1, 2024 12:00:00 AM
# Building an AI Travel Agent with Symbl.ai’s Nebula LLM and Amazon Connect
[Samson Nkrumah](https://blog.webrtc.ventures/author/samson-nkrumah)
Share: [linkedin-in icon](http://www.linkedin.com/shareArticle?mini=true&url=https://blog.webrtc.ventures/2024/02/building-an-ai-travel-agent-with-symbl-ais-nebula-llm-and-amazon-connect) [X Twitter icon](https://twitter.com/intent/tweet?url=https://blog.webrtc.ventures/2024/02/building-an-ai-travel-agent-with-symbl-ais-nebula-llm-and-amazon-connect) [envelope icon](mailto:?body=https://blog.webrtc.ventures/2024/02/building-an-ai-travel-agent-with-symbl-ais-nebula-llm-and-amazon-connect)
Generative AI is a powerful tool for call centers. It can be integrated as a standalone AI Sales Agent or as a co-pilot providing Agent Assist to a human sales rep, among other uses. Generative AI is particularly well-suited for industries like travel because it can pull from public information about destinations, climates, and the like in order to provide highly personalized advice for would-be travelers.
In this post, we’ll guide you through building an AI Travel Agent (aka voicebot) with the integration of Symbl.ai’s [Nebula Large Language Model (LLM)](https://symbl.ai/platform/nebula/llm/) for natural human conversations, Symbl.ai’s [Streaming API](https://docs.symbl.ai/docs/streaming-api) for real-time transcription, and [Amazon Connect’s](https://aws.amazon.com/connect/) robust communication platform.
In a later post, [Integrating Agent Assist with Symbl.ai Nebula LLM and Amazon Connect](https://webrtc.ventures/2024/02/building-ai-agent-assist-with-symbl-ai-nebula-llm-and-amazon-connect/), we show an alternate use of this functionality where the Generative AI serves as a co-pilot to a human agent. Of course, the benefits of both the AI Travel Agent and the AI Travel Agent Assist transcend the travel sector to almost any field.
Let’s start with a look at the main players in this project: the Nebula Chat, Symbl’s Streaming API, and Amazon Connect, and then we will go through the steps of building our AI Travel Agent.
## Nebula LLM
Nebula, Symbl.ai’s proprietary Large Language Model, is optimized specifically for building Generative AI experiences and workflows that involve human conversations. It is capable of processing various types of conversations such as sales, contact center, recruitment, meetings, emails, chats, and more.
Nebula can perform instruction tasks such as request summaries, follow-up questions, draft emails, and issues to review. In the travel agent scenario, it can also identify and recommend resolutions to customer issues and recommend alternative travel arrangements. All this leading to:
- Enhanced Customer Engagement
- Quick Access to Information
- Reduction in Training Time
- Increased Sales Opportunities.
- Error Reduction
Nebula is ideal for scenarios involving human dialogue and supports two distinct model variations.
- [Chat Model](https://docs.symbl.ai/reference/nebula-chat) – The Nebula chat endpoint takes a list of messages comprising a chat between a human and assistant, and returns a response containing an updated list of messages, with the last message containing a response from the model.
- [Instruct Model](https://docs.symbl.ai/docs/model-api-reference) – Model API allows calling the model to perform various tasks by sending prompts and various generation parameters with instruction and conversation transcript, and returns the generated output.
In our project, we use the Chat model.
---
#### ***Interested in learning more about the power of LLMs?** Join us for the February 21, 2024 episode of WebRTC Live featuring Symbl.ai’s Dan Nordale. [Register today.](https://webrtc.ventures/webrtc-live/)*
---
## Symbl.ai’s Streaming API for Real-time Transcription
Symbl.ai provides a wide range of services and products that leverage advanced natural language processing and machine learning. These tools are adept at analyzing text and speech data to extract valuable insights and intelligence. The features offered by Symbl.ai are diverse, encompassing:
- [Streaming conversations](https://docs.symbl.ai/reference/streaming-api-reference)
- [Async conversations](https://docs.symbl.ai/docs/async-api)
- [Telephone conversations](https://docs.symbl.ai/reference/telephony-api-reference)
In this project, we used the Streaming API, which uses the WebSocket protocol to process audio and provide conversation intelligence in real time.
## Amazon Connect AI-Powered Contact Center
Amazon Connect is the [Amazon Web Services (AWS)](https://aws.amazon.com/) cloud-based contact center service that is integral to the solutions discussed in this blog. It offers a seamless, scalable, and customizable experience for call centers of all types. Amazon Connect is designed for easy setup and scalability, allowing businesses to quickly establish and adapt their contact center operations as needed, without extensive hardware or complex software. This adaptability is crucial for businesses experiencing growth or fluctuating contact volumes.
A significant advantage of Amazon Connect is its ability to seamlessly integrate with other AWS services and third-party applications, such as we are doing today with Symbl.ai’s Nebula LLM and Streaming API. This integration facilitates real-time data processing and intelligence gathering, enhancing customer service.
Amazon Connect also provides tools for customizing customer experiences, including interactive voice response systems and real-time analytics, enabling businesses to tailor interactions to specific customer needs. Moreover, it adheres to AWS’s rigorous standards for data security and compliance, ensuring secure communications and adherence to data protection regulations.
## The Complete Tech Stack
1. [Symbl.ai](http://symbl.ai)
2. [Symbl.ai Nebula LLM](https://symbl.ai/platform/nebula/llm/)
3. [Amazon Connect](https://aws.amazon.com/connect/)
4. [Amazon Kinesis Video Stream](https://aws.amazon.com/kinesis/video-streams/)
5. [**amazon-connect-stream** library](https://github.com/amazon-connect/amazon-connect-streams)
6. [connect-rtc.js library](https://aws.github.io/connect-rtc-js/)
7. [Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API)
8. [Next.js](https://nextjs.org/)
9. [Tailwind CSS](https://tailwindcss.com/)
10. [Ant Design](https://ant.design/)
11. [Nest.js](https://nestjs.com/) with [Socket.io](http://socket.io).
## Building our AI Travel Agent
### Prerequisites
You will need access to the following;
- A pair of **appId** and **secret** for Symbl.ai, which you can get from the [platform’s main page](https://platform.symbl.ai/#/home). We use these to [retrieve a temporary access token](https://docs.symbl.ai/docs/authenticate#generate-an-access-token).
- An **API Key** for Nebula LLM, which you get by [joining the beta wait list](https://info.symbl.ai/Nebula_Private_Beta.html).
- A pair of Access and Secret keys for the AWS account where Amazon Connect is configured.
Make sure to set up your Amazon Connect account by claiming your number, setting up routing profiles, agents, queues, contact flows, etc., as described in the [‘Get started with Amazon Connect’ documentation.](https://docs.aws.amazon.com/connect/latest/adminguide/amazon-connect-get-started.html)
### Architecture
Our AI Travel Agent will work like this:
1. A customer calls one of Amazon Connect’s claimed numbers for the application.
2. Amazon Connect starts streaming to Amazon Kinesis Video Stream (KVS).
3. A human agent receives the call within the web application and answers it.
4. The audio from the customer is sent to Symbl.ai which returns a transcription of it.
5. The transcription is sent to Nebula LLM with a specific prompt so it returns the desired type of response.
6. The response from the LLM is uttered by the browser’s native Text-to-Speech (TTS) engine and sent back to Amazon Connect so the user can hear it.
These steps are depicted in the diagram below:
[ ](https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/image2-Apr-03-2024-06-42-54-8482-PM.png)
Depiction of the AI Travel Agent architecture.
### Connecting and Loading Amazon Connect Contact Control Panel (CCP) on Web
We use [Next.js](https://nextjs.org/) and [Typescript](https://www.typescriptlang.org/) for the client side code, along with the [amazon-connect-streams](https://github.com/amazon-connect/amazon-connect-streams) library for loading the CCP interface to directly receive calls in our web application. We also download the latest versions of [connect-rtc-js](https://github.com/aws/connect-rtc-js/blob/master/release/connect-rtc.min.js) and [aws-sdk](https://aws.amazon.com/sdk-for-javascript/)libraries**.**
Now load the CCP within the Next.js app by following these steps:
1. Call the **connect.core.initCCP** passing the reference of a div element where to load the CCP on the DOM.
2. Configure the CCP by passing an options object with some parameters. (You can read more about these in the [Amazon Connect Streams github repo](https://github.com/amazon-connect/amazon-connect-streams/blob/master/Documentation.md).)
3. Set **softphone.allowFramedSoftphone** to be **false**. This tells the **amazon-connect-streams** library to not use its default CCP.
4. Call **connect.core.initSoftphoneManager({ allowFramedSoftphone: true })**, which tells the **amazon-connect-streams** libraryto use **connect-rtc-js** as the Softphone manager.
5. Set a call back, in case a call is received.
The **connect-rtc-js** library acts as a wrapper around **amazon-connect-streams** so that we are able to access the **RTCSession** in our code. With this we are able to get the remote stream and **RTCPeerConnection** object.
The code for these steps is shown below:
```
// 1. Call connect.core.initCCP and 2. Configure the CCP
connect.core.initCCP(containerDivRef.current, {
ccpUrl: "",
loginPopup: true,
loginPopupAutoClose: true,
loginOptions: { autoClose: true },
// 3. Prevent amazon-connect-streams from sing its default CCP
softPhone: { allowFramedSoftphone: false },
pageOptions: {
enableAudioDeviceSettings: true,
enablePhoneTypeSettings: true,
},
});
// 4. Initialize the connect-rtc-js as the softphone manager
connect.core.initSoftphoneManager({ allowFramedSoftphone: true });
// 5. Register a call back in case a call is received
connect.core.onSoftphoneSessionInit((e) => {
// returns connect-rtc-js which is softphone manager
var softphoneManager = connect.core.getSoftphoneManager();
connectionIdRef.current = e.connectionId;
setSoftphoneManager(softphoneManager);
if (softphoneManager) {
var session = softphoneManager.getSession(e.connectionId);
const remoteStream = session?._remoteAudioStream;
session._onRemoteStreamAdded((e) => {
console.log({ remoteStreamAdded: e });
});
}
});
```
### Connecting Symbl.ai Streaming API
To connect to the Symbl.ai Streaming API, we first need an [access token to authenticate the requests](https://docs.symbl.ai/docs/authenticate#generate-an-access-token).
Once the session is connected, you can now bind listeners to the `onmessage`callback to receive updates such as transcription, trackers, sentiments, action\_items, etc. from Symbl.ai.
In our case, we are only interested in the transcriptions and then we are calling Nebula LLM only when we detect a pause in speech from the user for a specific number ofmilliseconds.
This flow is depicted in the image below.
[ ](https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/image1-Apr-03-2024-06-42-55-2950-PM.png)
Depiction of the flow for sending transcripts to Nebula LLM.
### Calling Nebula LLM
Now this is the part we have been waiting for, the power of Generative AI.
As mentioned above, Nebula provides 2 models: Instruct and Chat. In this blog post we use the latter because we want to simulate a conversation, therefore we want the [Nebula Chat Model](https://docs.symbl.ai/reference/nebula-chat) to keep track of previous messages.
Also note that there are 2 different endpoints for the Chat Model.
1. **/v1/model/chat:** Returns the generated out in a single http response.
2. **/v1/model/chat/streaming**: Returns the generative response using Server-sent-events (SSE) and allows the client to start receiving output progressively as it is getting generated.
In this demo, we use the 2nd option as it allows receiving responses as they are being generated.
Once that text response is received, we just send it to the call agent which in turn will convert text to speech (TTS) and relay that to the Amazon Connect call.
## Time for a demo of our AI Travel Agent in Action!
*If you are interested in integrating an AI Agent into your communications platform, reach out to the experts at WebRTC.ventures.* [***Contact us today!***](https://webrtc.ventures/contact/)
[CPaaS](https://blog.webrtc.ventures/tag/cpaas), [Managed Services](https://blog.webrtc.ventures/tag/managed-services), [Events](https://blog.webrtc.ventures/tag/events), [Video Conferencing](https://blog.webrtc.ventures/tag/video-conferencing), [Real Time Weekly](https://blog.webrtc.ventures/tag/real-time-weekly), [WebRTC Architecture](https://blog.webrtc.ventures/tag/webrtc-architecture), [Broadcasting](https://blog.webrtc.ventures/tag/broadcasting), [Medical](https://blog.webrtc.ventures/tag/medical), [WebRTC Live](https://blog.webrtc.ventures/tag/webrtc-live), [Showcase](https://blog.webrtc.ventures/tag/showcase), [Thoughts](https://blog.webrtc.ventures/tag/thoughts), [Vonage](https://blog.webrtc.ventures/tag/vonage), [Daily](https://blog.webrtc.ventures/tag/daily), [Jobs](https://blog.webrtc.ventures/tag/jobs), [Contact Centers](https://blog.webrtc.ventures/tag/contact-centers), [Telehealth](https://blog.webrtc.ventures/tag/telehealth), [Education](https://blog.webrtc.ventures/tag/education), [Technical](https://blog.webrtc.ventures/tag/technical), [General](https://blog.webrtc.ventures/tag/general), [Story of success](https://blog.webrtc.ventures/tag/story-of-success), [The WebRTC.ventures Blog](https://blog.webrtc.ventures/tag/the-webrtc-ventures-blog), [Testing](https://blog.webrtc.ventures/tag/testing), [Gaming](https://blog.webrtc.ventures/tag/gaming), [DevOps](https://blog.webrtc.ventures/tag/devops), [Symbl.ai](https://blog.webrtc.ventures/tag/symbl-ai)
## Related posts
[](https://blog.webrtc.ventures/2024/02/building-ai-agent-assist-with-symbl-ai-nebula-llm-and-amazon-connect)
## [Building AI Agent Assist with Symbl.ai Nebula LLM and Amazon Connect](https://blog.webrtc.ventures/2024/02/building-ai-agent-assist-with-symbl-ai-nebula-llm-and-amazon-connect)
[Samson Nkrumah](https://blog.webrtc.ventures/author/samson-nkrumah)
Feb 19, 2024 12:00:00 AM
In my last post, Building an AI Travel Agent with Symbl.ai’s Nebula LLM and Amazon Connect, I spoke...
[Read more](https://blog.webrtc.ventures/2024/02/building-ai-agent-assist-with-symbl-ai-nebula-llm-and-amazon-connect)
[](https://blog.webrtc.ventures/2024/01/webrtc-ventures-is-a-symbl-ai-product-partner)
## [WebRTC.ventures is a Symbl.ai Product Partner](https://blog.webrtc.ventures/2024/01/webrtc-ventures-is-a-symbl-ai-product-partner)
 [Jen Oppenheimer](https://blog.webrtc.ventures/author/jen-oppenheimer)
Jan 19, 2024 12:00:00 AM
WebRTC.ventures is excited to announce our official status as a trusted Symbl.ai Product Partner....
[Read more](https://blog.webrtc.ventures/2024/01/webrtc-ventures-is-a-symbl-ai-product-partner)
[](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai)
## [Enhancing Customer Service Experiences with Vonage and Symbl.ai](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai)
[Fahad Mahmood](https://blog.webrtc.ventures/author/fahad-mahmood)
Feb 24, 2023 12:00:00 AM
The combination of WebRTC and Artificial Intelligence (AI) is enhancing customer service activities...
[Read more](https://blog.webrtc.ventures/2023/02/enhancing-customer-service-experiences-with-vonage-and-symbl-ai)
### WebRTC.Ventures Services
...
- Assess
- Build
- Integrate
- Test
- Deploy & Manage
### Contact Us
...
**Video should be an opportunity, not a headache.**
We’re here to build, integrate, assess and optimize, test, and even deploy and manage your live video application.
[Contact us](https://blog.webrtc.ventures/2024/02/building-an-ai-travel-agent-with-symbl-ais-nebula-llm-and-amazon-connect#contact)
### WebRTC Live
...
View [past episodes](https://blog.webrtc.ventures/webrtc-live/) of our webinar interview series about the latest webRTC use cases and technical updates. Join our mailing list to learn about upcoming episodes!
We’re one of the few agencies in the world dedicated to WebRTC development. This dedication and experience is why so many people trust us to help bring live video application dreams to life.
### Let's get started!
###### **[Contact us today](https://webrtc.ventures/contact/)**
**[info@webrtc.ventures](mailto:info@webrtc.ventures)**
### Join our mailing list
We don't e-mail often (a few times a month max), and it will be worth it when we do! Of course, we respect your privacy and will not share your contact information.
© 2023 WebRTC.ventures, an [AgilityFeat](https://agilityfeat.com/) company / [Privacy Policy](https://webrtc.ventures/privacy-policy//)
```json
{
"@context" : "https://schema.org",
"@type" : "BlogPosting",
"author" : {
"@type" : "Person",
"name" : "Samson Nkrumah",
"url" : "https://blog.webrtc.ventures/author/samson-nkrumah"
},
"dateModified" : "2024-04-03T18:51:22.763Z",
"datePublished" : "2024-02-01T05:00:00.000Z",
"headline" : "Building an AI Travel Agent with Symbl.ai’s Nebula LLM and Amazon Connect – WebRTC.ventures",
"image" : [ "https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Building-an-AI-Travel-Agent-with-Symbl_ai-Nebula-LLM-and-Amazon-Connect.png" ],
"mainEntityOfPage" : {
"@id" : "https://blog.webrtc.ventures/2024/02/building-an-ai-travel-agent-with-symbl-ais-nebula-llm-and-amazon-connect",
"@type" : "WebPage"
},
"publisher" : {
"@type" : "Organization",
"logo" : {
"@type" : "ImageObject",
"url" : "https://blog.webrtc.ventures/hubfs/logo-main-light.svg"
}
}
}
```