Skip to main content

Virtual Background & Blur​

Installation​

To enable the virtual background feature in your application, you first need to install the "Background Transformer" package.

npm i @cloudflare/realtimekit-virtual-background

Initialize​

import RealtimeKitVideoBackgroundTransformer from '@cloudflare/realtimekit-virtual-background';

If you installed the package using a script tag, the RealtimeKitVideoBackgroundTransformer will be automatically available for use once the installation process is complete.

2.x versions of RealtimeKitVideoBackgroundTransformer use their own rendering mechanism thus require you to disable the default per frame canvas rendering.

await meeting.self.setVideoMiddlewareGlobalConfig({
  disablePerFrameCanvasRendering: true,
});

Now that we have disabled the per frame rendering, we can initialise the RealtimeKitVideoBackgroundTransformer.

const videoBackgroundTransformer = await RealtimeKitVideoBackgroundTransformer.init({
  meeting: meeting,
  segmentationConfig: { // optional
    pipeline: 'webgl2', // 'webgl2' | 'canvas2dCpu'
  },
});

To customise RealtimeKitVideoBackgroundTransformer configs, please refer to the RealtimeKit virtual background NPM package.

Add an image as a background​

To incorporate an image as a background, create a static background video middleware using the createStaticBackgroundVideoMiddleware method provided by the videoBackgroundTransformer object.

const videoMiddleware =
  await videoBackgroundTransformer.createStaticBackgroundVideoMiddleware(
    `https://assets.dyte.io/backgrounds/bg-dyte-office.jpg`,
  );
meeting.self.addVideoMiddleware(videoMiddleware);
note

Ensure that the URL of the image allows Cross-Origin Resource Sharing (CORS) to avoid canvas tainting issues. If the CORS policy is not allowed for the image, it may result in the video feed getting stuck on a frame or appearing blank.

Blur the background​

Create a background blur video middleware using the createBackgroundBlurVideoMiddleware method provided by the videoBackgroundTransformer object.

const videoMiddleware =
  await videoBackgroundTransformer.createBackgroundBlurVideoMiddleware(10);
meeting.self.addVideoMiddleware(videoMiddleware);

Pass intensity as a parameter between 1-100

Remove blur/virtual background​

Use the removeVideoMiddleware method

meeting.self.removeVideoMiddleware(videoMiddleware);

Noise Cancellation (Krisp.ai)​

note

This is a paid addon, contact support@dyte.io for access.

Installation​

Create/Append to the .npmrc file in the project root with the below contents

//registry.npmjs.org/:_authToken=<paid-token-from-dyte>
@dytesdk:registry=https://registry.npmjs.org/
#always-auth=true
npm i @dytesdk/krisp-noise-cancellation

Usage​

Initialise the middleware

const krisp = new Krisp();
await krisp.init(meeting);

Enable / disable the noise cancellation

krisp.activate()
// krisp.deactivate()