Skip to main content

Introduction

This section provides an overview of video processing, including how to create custom video processors and integrating them into your application with RealtimeKit's APIs. Real-time video processing allows you to apply effects, filters, or analytics directly to a live video stream.

Our new VideoProcessor APIs provide flexible and powerful ways to manipulate video frames.

Types of Video Processors​

We provide three types of video processors:

  1. NoDropVideoProcessor: Allows custom video processing without dropping frames.
  2. ChainVideoProcessor: Chains multiple frame processors together, useful for applying multiple effects or filters to a video stream.
  3. FilterVideoProcessor: Simpler and more efficient way to apply a single effect or filter to a video stream.

Nonetheless, you can also create your own custom video processors by implementing the VideoProcessor interface.

import realtimekit.org.webrtc.VideoFrame
import realtimekit.org.webrtc.VideoProcessor
import realtimekit.org.webrtc.VideoSink

class CustomVideoProcessor : VideoProcessor {
  override fun onCapturerStarted(started: Boolean) {}

  override fun onCapturerStopped() {}

  override fun onFrameCaptured(frame: VideoFrame?) {}

  override fun setSink(sink: VideoSink?) {}
}

Integrating Your Processor​

Once you have created and configured your VideoProcessor, you need to pass it to the RealtimeKitMeetingBuilder object. This will start the processing of the video frames captured by the camera before they are sent to other participants or rendered locally.

Usage Example​

Here's how you set a video processor on a local video track.

// Assuming 'myCustomProcessor' is an instance of any VideoProcessor implementation
// (e.g., ChainVideoProcessor, FilterVideoProcessor, etc.).

val myCustomProcessor = CustomProcessor() // Your processor creation logic

// Set the video processor on the meeting builder.
val meeting = RealtimeKitMeetingBuilder
  .setVideoProcessor(processor = myCustomProcessor)
  .build(activity)

// You can also pass an EglBase to the builder
// This is useful when using FilterVideoProcessor
val eglBase = EglBase.create()
val meeting = RealtimeKitMeetingBuilder
  .setVideoProcessor(eglBase = eglBase, processor = myCustomProcessor)
  .build(activity)