---
title: Raspberry Pi Video Live Streaming with AI – WebRTC.ventures
description: "At WebRTC.ventures, we have worked on several projects implementing live streaming camera applications with a Raspberry Pi. For example, we implemented a motion detection camera that allows a user to watch their camera live and to watch pre-recorded videos that were generated when motion was detected. After doing this, we wanted to play a bit more with that and the hot technology of the year: artificial intelligence (AI)."
image: https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Raspberry-Pi-Video-Live-Streaming-with-AI-2.png
---

[Skip to content](https://blog.webrtc.ventures/2019/08/raspberry-pi-video-live-streaming-with-ai#main-content)

![WebRTC.ventures](https://blog.webrtc.ventures/hubfs/logo-main-light.svg)

- [WebRTC Services](https://webrtc.ventures/webrtc-development-services/)
  
  Show submenu for WebRTC Services 
  
    - [Assess](https://webrtc.ventures/services/assess/)
    - [Build](https://webrtc.ventures/services/build/)
    - [Integrate](https://webrtc.ventures/services/integrate/)
    - [Test](https://webrtc.ventures/services/test/)
    - [Manage](https://webrtc.ventures/services/manage/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- Our Partners
  
  Show submenu for Our Partners 
  
    - [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/)
- Stories of Success
  
  Show submenu for Stories of Success 
  
    - [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/)
    - [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/)
    - [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/)
    - [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/)
    - [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/)
    - [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/)
    - [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/)
- WebRTC Resources
  
  Show submenu for WebRTC Resources 
  
    - [WebRTC Live](https://webrtc.ventures/webrtc-live/)
    - [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz)
    - [WebRTC.ventures Blog](https://webrtc.ventures/blog/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
    - [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/)
- [About Us](https://webrtc.ventures/our-team/)
  
  Show submenu for About Us 
  
    - [Blog](https://webrtc.ventures/blog/)
    - [Our Team](https://webrtc.ventures/our-team/)
    - [Jobs](https://webrtc.ventures/jobs/)

Open main navigation

Close main navigation

- [WebRTC Services](https://webrtc.ventures/webrtc-development-services/)
  
  Show submenu for WebRTC Services 
  
    - [Assess](https://webrtc.ventures/services/assess/)
    - [Build](https://webrtc.ventures/services/build/)
    - [Integrate](https://webrtc.ventures/services/integrate/)
    - [Test](https://webrtc.ventures/services/test/)
    - [Manage](https://webrtc.ventures/services/manage/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
- Our Partners
  
  Show submenu for Our Partners 
  
    - [Amazon Chime SDK](https://webrtc.ventures/partners/amazon-chime-sdk/)
- Stories of Success
  
  Show submenu for Stories of Success 
  
    - [Remote Video Interpreting Platform with Agent Routing](https://webrtc.ventures/successes/remote-video-translation-platform/)
    - [WebRTC to SIP Gateway for ASL Interpreters](https://webrtc.ventures/successes/webrtc-to-sip-gateway-for-asl-interpreters/)
    - [Audio Listening Device to Improve Surgical Outcomes](https://webrtc.ventures/successes/audio-listening-device-to-improve-surgical-outcomes/)
    - [Albania’s First Telemedicine Platform](https://webrtc.ventures/successes/albanias-first-telemedicine-platform/)
    - [Automated Video Storytelling with the Amazon Chime SDK](https://webrtc.ventures/successes/automated-video-storytelling-with-the-amazon-chime-sdk/)
    - [Large Scale Video Conferencing](https://webrtc.ventures/successes/large-scale-video-conferencing/)
    - [Broadcasting and Webcast Control Room](https://webrtc.ventures/successes/live-video-broadcasting-and-webcast-control-room/)
- WebRTC Resources
  
  Show submenu for WebRTC Resources 
  
    - [WebRTC Live](https://webrtc.ventures/webrtc-live/)
    - [WebRTC Tips](https://www.youtube.com/playlist?list=PLayYqdnyegt3QvxX-Rt6wbFIK8727Z5Oz)
    - [WebRTC.ventures Blog](https://webrtc.ventures/blog/)
    - [Migrating From Twilio Video](https://webrtc.ventures/twilio-video/)
    - [WebRTC.ventures Training Program](https://webrtc.ventures/webrtc-ventures-training-program/)
- [About Us](https://webrtc.ventures/our-team/)
  
  Show submenu for About Us 
  
    - [Blog](https://webrtc.ventures/blog/)
    - [Our Team](https://webrtc.ventures/our-team/)
    - [Jobs](https://webrtc.ventures/jobs/)
- [Contact us](https://webrtc.ventures/contact/)

[Contact us](https://webrtc.ventures/contact/)

 Aug 13, 2019 12:00:00 AM

# Raspberry Pi Video Live Streaming with AI

[Alberto Gonzalez](https://blog.webrtc.ventures/author/alberto-gonzalez)

Share: [linkedin-in icon](http://www.linkedin.com/shareArticle?mini=true&url=https://blog.webrtc.ventures/2019/08/raspberry-pi-video-live-streaming-with-ai) [X Twitter icon](https://twitter.com/intent/tweet?url=https://blog.webrtc.ventures/2019/08/raspberry-pi-video-live-streaming-with-ai) [envelope icon](mailto:?body=https://blog.webrtc.ventures/2019/08/raspberry-pi-video-live-streaming-with-ai)

At WebRTC.ventures, we have worked on several projects implementing live streaming camera applications with a Raspberry Pi. For example, we implemented a motion detection camera that allows a user to watch their camera live and to watch pre-recorded videos that were generated when motion was detected. After doing this, we wanted to play more with that and the hot technology of the year: artificial intelligence (AI). 

While there are many ways to do that, a Raspberry Pi by itself is quite limited for heavy ML video processing, especially if you want it in real time. We decided to go ahead and build a demo that handles the heavy processing on the server.

There are many options and frameworks for streaming video from a Raspberry Pi. Here is a comparison table for some of them in a **Raspberry Pi 3**:

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_35_50-AM-1024x395.png?width=539&height=207&name=Screen-Shot-2019-08-12-at-9_35_50-AM-1024x395.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_35_50-AM-1024x395.png?width=539&height=207&name=Screen-Shot-2019-08-12-at-9_35_50-AM-1024x395.png)

Using direct WebRTC from the RPI seems to be the best choice in terms of latency. However, because of direct WebRTC’s high CPU usage, we’re opting for GStreamer this time. We’ll use GStreamer to send an RTP stream to a media server that will handle the distribution to the viewers using WebRTC. This is a more generic approach that can be applied to less powerful devices or to RTP streaming cameras.

Because we love Node.js, we thought it’d be great to build this video processing service using Node.js. We used **opencv4nodejs**, which can be found [here](https://github.com/agonza1/WebRTC-Live-Streaming-with-AI).

Although you can easily play with images and save video files using OpenCV, piping the modified video stream with OpenCV is not as easy.

## Installing OpenCV with GStreamer

To integrate with GStreamer, you will need to build OpenCV manually. You can follow the OpenCV [here](https://medium.com/@galaktyk01/how-to-build-opencv-with-gstreamer-b11668fa09c) for most of the installation steps. Remember to include GSTREAMER=ON when making it:

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_38_32-AM-1024x500.png?width=437&height=213&name=Screen-Shot-2019-08-12-at-9_38_32-AM-1024x500.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_38_32-AM-1024x500.png?width=437&height=213&name=Screen-Shot-2019-08-12-at-9_38_32-AM-1024x500.png)

Once installed, we’re able to open any RTP stream with GStreamer commands within the OpenCV:

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_39_26-AM-1024x95.png?width=473&height=43&name=Screen-Shot-2019-08-12-at-9_39_26-AM-1024x95.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_39_26-AM-1024x95.png?width=473&height=43&name=Screen-Shot-2019-08-12-at-9_39_26-AM-1024x95.png)

The line above will just pass a test video. To grab a UDP stream, you’ll need:

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_40_05-AM-1024x117.png?width=475&height=54&name=Screen-Shot-2019-08-12-at-9_40_05-AM-1024x117.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_40_05-AM-1024x117.png?width=475&height=54&name=Screen-Shot-2019-08-12-at-9_40_05-AM-1024x117.png)

Now we’ll be able to play with OpenCV and apply ML image processing to the video. Here you can find a whole example.

In our case, we want to live stream to dozens of viewers. Vanilla WebRTC won’t be able to handle it, so we’re going to use [Janus WebRTC gateway](https://janus.conf.meetecho.com/) and specifically its live streaming plugin. At the end, the high-level architecture will look like something like this:

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/pasted-image-0.png?width=525&height=248&name=pasted-image-0.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/pasted-image-0.png?width=525&height=248&name=pasted-image-0.png)

## Basic colormap filter on video using OpenCV

We’ll keep getting the video frames and applying the filter to each one:

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_41_27-AM-1024x295-1.png?width=409&height=118&name=Screen-Shot-2019-08-12-at-9_41_27-AM-1024x295-1.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_41_27-AM-1024x295-1.png?width=409&height=118&name=Screen-Shot-2019-08-12-at-9_41_27-AM-1024x295-1.png)

That’s it! Now we just need to feed this to Janus.

## Video writing using GStreamer magic

It’s important to understand OpenCV and GStreamer for this part. There are some caveats that you need to know for your GStreamer pipeline. Otherwise, you may experience missing video keyframes or other encoding issues.

In my case, I was sending H264 video from Raspberry Pi and needed to add **format=I420** explicitly in the videoWriter of OpenCV. With a test source, I had to specify the format being used by explicitly passing the format (e.g., BGR). Finally, after modifying the video, you can use OpenCV VideoWriter:

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_42_25-AM-1024x151.png?width=479&height=70&name=Screen-Shot-2019-08-12-at-9_42_25-AM-1024x151.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Screen-Shot-2019-08-12-at-9_42_25-AM-1024x151.png?width=479&height=70&name=Screen-Shot-2019-08-12-at-9_42_25-AM-1024x151.png)

## Result

Although this is definitely not a heavy processing task, we were able to stream video from a Raspberry Pi, apply a filter, and stream it live to dozens of users. You can find the demo project [here](https://github.com/agonza1/WebRTC-Live-Streaming-with-AI).

![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/pasted-image-0-2-1.png?width=565&height=349&name=pasted-image-0-2-1.png) ![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/pasted-image-0-2-1.png?width=565&height=349&name=pasted-image-0-2-1.png)

In the second part of this project, we’ll go a bit deeper and do something more complex with OpenCV and TensorFlow. Stay tuned!

[CPaaS](https://blog.webrtc.ventures/tag/cpaas), [Managed Services](https://blog.webrtc.ventures/tag/managed-services), [Events](https://blog.webrtc.ventures/tag/events), [Video Conferencing](https://blog.webrtc.ventures/tag/video-conferencing), [Real Time Weekly](https://blog.webrtc.ventures/tag/real-time-weekly), [WebRTC Architecture](https://blog.webrtc.ventures/tag/webrtc-architecture), [Broadcasting](https://blog.webrtc.ventures/tag/broadcasting), [Medical](https://blog.webrtc.ventures/tag/medical), [WebRTC Live](https://blog.webrtc.ventures/tag/webrtc-live), [Showcase](https://blog.webrtc.ventures/tag/showcase), [Thoughts](https://blog.webrtc.ventures/tag/thoughts), [Vonage](https://blog.webrtc.ventures/tag/vonage), [Daily](https://blog.webrtc.ventures/tag/daily), [Jobs](https://blog.webrtc.ventures/tag/jobs), [Contact Centers](https://blog.webrtc.ventures/tag/contact-centers), [Telehealth](https://blog.webrtc.ventures/tag/telehealth), [Education](https://blog.webrtc.ventures/tag/education), [Technical](https://blog.webrtc.ventures/tag/technical), [General](https://blog.webrtc.ventures/tag/general), [Story of success](https://blog.webrtc.ventures/tag/story-of-success), [The WebRTC.ventures Blog](https://blog.webrtc.ventures/tag/the-webrtc-ventures-blog), [Testing](https://blog.webrtc.ventures/tag/testing), [Gaming](https://blog.webrtc.ventures/tag/gaming), [DevOps](https://blog.webrtc.ventures/tag/devops), [Symbl.ai](https://blog.webrtc.ventures/tag/symbl-ai)

## Related posts

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/WebRTC-Live-69.png?height=200&name=WebRTC-Live-69.png)](https://blog.webrtc.ventures/2022/07/webrtclive-roundtable3)

## [Watch WebRTC Live #69: What Have Our Own WebRTC Experts Learned Lately?](https://blog.webrtc.ventures/2022/07/webrtclive-roundtable3)

![Picture of Jen Oppenheimer](https://blog.webrtc.ventures/hs-fs/hubfs/Our-Team-Jen.jpg?width=50&name=Our-Team-Jen.jpg) [Jen Oppenheimer](https://blog.webrtc.ventures/author/jen-oppenheimer) 

 Jul 21, 2022 12:00:00 AM

The July 20, 2022 episode of WebRTC Live was the the third installment of our WebRTC.ventures...

[Read more](https://blog.webrtc.ventures/2022/07/webrtclive-roundtable3)

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/Notifications-Can-Make-or-Break-Your-WebRTC-User-Experience.png?height=200&name=Notifications-Can-Make-or-Break-Your-WebRTC-User-Experience.png)](https://blog.webrtc.ventures/2021/03/notifications-webrtc-user-experience)

## [Notifications Can Make or Break Your WebRTC User Experience](https://blog.webrtc.ventures/2021/03/notifications-webrtc-user-experience)

[Mariana Lopez](https://blog.webrtc.ventures/author/mariana-lopez) 

 Mar 10, 2021 12:00:00 AM

There are a number of roadblocks that can prevent users in a WebRTC video, audio, or chat...

[Read more](https://blog.webrtc.ventures/2021/03/notifications-webrtc-user-experience)

[![](https://blog.webrtc.ventures/hs-fs/hubfs/Imported_Blog_Media/snowfall_fence.png?height=200&name=snowfall_fence.png)](https://blog.webrtc.ventures/2016/01/live-streaming-with-webrtc)

## [Live Streaming the Snowpocalypse with WebRTC](https://blog.webrtc.ventures/2016/01/live-streaming-with-webrtc)

[ArinSime](https://blog.webrtc.ventures/author/arinsime) 

 Jan 29, 2016 12:00:00 AM

Last week we had a major snowstorm on the East Coast of the United States, where I live. Over 60...

[Read more](https://blog.webrtc.ventures/2016/01/live-streaming-with-webrtc)

### WebRTC.Ventures Services

...

- Assess
- Build
- Integrate
- Test
- Deploy & Manage

### Contact Us

...

**Video should be an opportunity, not a headache.**   
 We’re here to build, integrate, assess and optimize, test, and even deploy and manage your live video application. 

[Contact us](https://blog.webrtc.ventures/2019/08/raspberry-pi-video-live-streaming-with-ai#contact)

### WebRTC Live

...

 View [past episodes](https://blog.webrtc.ventures/webrtc-live/) of our webinar interview series about the latest webRTC use cases and technical updates. Join our mailing list to learn about upcoming episodes! 

![WebRTC Live](https://blog.webrtc.ventures/hs-fs/hubfs/Template%20Images/widged-bannet.webp?height=90&name=widged-bannet.webp)

![logo-main-light](https://blog.webrtc.ventures/hubfs/logo-main-light.svg "logo-main-light")

We’re one of the few agencies in the world dedicated to WebRTC development. This dedication and experience is why so many people trust us to help bring live video application dreams to life.

 

### Let's get started!

###### **[Contact us today](https://webrtc.ventures/contact/)**

**[info@webrtc.ventures](mailto:info@webrtc.ventures)**

<https://www.linkedin.com/company/webrtc-ventures> <https://www.youtube.com/@WebrtcVentures>

### Join our mailing list

We don't e-mail often (a few times a month max), and it will be worth it when we do! Of course, we respect your privacy and will not share your contact information.

© 2023 WebRTC.ventures, an [AgilityFeat](https://agilityfeat.com/) company / [Privacy Policy](https://webrtc.ventures/privacy-policy//)

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Alberto Gonzalez",
    "url" : "https://blog.webrtc.ventures/author/alberto-gonzalez"
  },
  "dateModified" : "2024-04-03T17:53:39.261Z",
  "datePublished" : "2019-08-13T04:00:00.000Z",
  "headline" : "Raspberry Pi Video Live Streaming with AI – WebRTC.ventures",
  "image" : [ "https://blog.webrtc.ventures/hubfs/Imported_Blog_Media/Raspberry-Pi-Video-Live-Streaming-with-AI-2.png" ],
  "mainEntityOfPage" : {
    "@id" : "https://blog.webrtc.ventures/2019/08/raspberry-pi-video-live-streaming-with-ai",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://blog.webrtc.ventures/hubfs/logo-main-light.svg"
    }
  }
}
```