Volumetric Video for Broadcasting, Metaverse Production and RT3D Assets

Volumetric Video for Broadcasting, Metaverse Production and RT3D Assets Showcasing the live studio of the future and evolve the traditional broadcast pipeline to incorporate the latest content creation technologies such as volumetric, motion capture, live action and 3D render engine.

Champions: RTÉ, FOX Sports, Hochschule Luzern, Trinity College Dublin, DAZN, Connected Ideas Ltd.,Unity Technologies, ETC/University of Southern California, Production Park

Participants: Noitom, Volograms

CLICK HERE for a selection of this project’s POC Assets & Results. The team presented their final project showcase & POC Results on IBC2022's Innovation Stage, which you can see in full in here.

The Challenge Objective:

While broadcasters want to embrace new technologies and innovative solutions to increase production quality, the challenge faced is that often, these solutions don't easily fit the timeframe, budget and resources of a traditional broadcast pipeline.

These Challenge objectives focused on the creation, delivery and application of Digital Humans, exploring ways to add both photo-realistic and stylized 3D avatars to our broadcast and digital platforms.

Another key objective included efficient generation of reusable 3D assets, suitable for live broadcast as well as VR, AR extended and mixed reality experiences.

The Aim of the POC:

POC Solutions explored:
* Swapping 2D images and video elements with 3D volumetric assets in live broadcast. Exploring additional use cases for these 3D volumetric elements (metaverse, mobile devices, apps, etc.)
* Low-cost, fast turnaround solutions for volumetric video assets. Single camera capture using mobile devices (Volu app), versus multi-camera volumetric capture studios.
AI conversion of 2D videos and images to 3D volumetric assets versus bespoke 3D avatars, mocap live puppetry, etc.
* Digitally enhancing volumetric assets through the use of real time shaders. Allowing assets to be reused in different settings by enabling features such as lighting adjustments, hue, saturation and contrast to match indoor and outdoor settings for broadcast, as well as multi-platform use.
* Applying stylized rendering techniques and filters to 3D assets and volumetric content in keeping with the timeframe and budget of a traditional broadcast design department.
* Exploring editing, cleanup and advanced rendering techniques to enhance our digital humans.

The Innovation:

Innovation explored:
* Investigated the use of volumetric video in broadcast production
* Especially an affordable solution, via Participants Volu by Volograms
* Implemented real-world use case e.g. sports, replacing common 2D elements by volumetric video in Sports
* AI conversion of 2D videos and images to 3D volumetric assets versus bespoke 3D avatars, mocap live puppetry, etc.