Awesome ML Kit

Awesome ML Kit gives Unity developers easy way to employ optimized real-time ML models centered around computer vision.

Documentation

Forum thread

Background matting demo video

The current set of models includes

1. High-quality background removal/background matting based on RobustVideoMatting (optimized for Unity, both horizontal and vertical (portrait) video aspect ratio supported)

2. Depth estimation by single image

Features

1. Sample scene for each model

2. Specially made visualizations with different post effects

3. Select solid color / image / video as background replacement; put Unity content between foreground & background layers

4. Different model versions for Quality / Speed tradeoff

5. Linear/Gamma color spaces supported

5. WebCamTexture support

6. Unity VideoPlayer integration

7. AVPro Live Camera integration (recommended)

Ideal for

+ Media installations

+ Apps with video streaming (e.g. can be used with Agora.io)

+ Kiosks

Performance

NVidia RTX3070

+ Background removal: 300 FPS

+ Depth estimation: 400 FPS

NVidia GTX1050Ti (mobile)

+ Background removal: 50 FPS

+ Depth estimation: 80 FPS

Note: only desktop platforms currently supported (tested on Windows DX11 and Linux Vulkan)

Unity versions: 2021/2022 and above