<Group>

IBC Show | Amsterdam | 9-12 September 2022

IBC Digital | Your Place | 21 October 2021 - 13 March 2022

Hero Subpage - All Attend & Media Pages (Red)

No content here.

Smart Remote Production for Real-Time Animation

Smart Remote Production for Real-Time Animation

2-featuredimage

IBC Accelerators 2021 Smart Remote Production for Real-Time Animation Final Showcase Highlight


The Challenge Objective: 

This novel workflow, led by RTÉ and EBU, and Co-Championed by RAI, VRT, YLE ETC and Digital Domain, with support from Nvidia, was accomplished through remote collaboration using a Cloud Based Software Bridge and off-the-shelf hardware. It also tested the feasibility of using vocal performance and body posture to drive 3D avatars from remotely connected locations, all the while using minimal equipment.   

Characters were integrated into their environments by staging and shooting the content in a real-time render engine (Unreal), and Nvidia Omniverse using collaborative virtual production techniques and AI-based software.   

The goal of this unique POC was to create the most effective, low cost “Script to 3D Character Pipeline” imaginable. And with this, IBC’s “Smart Remote Production for Real Time Animation” is now... ‘ALIVE!’ 

The Aim of the POC: 

The original aim was to explore the following approaches to low cost animated remote production by creating a proof of concept in a cloud-based workflow:  

Pose Estimation Software:  

  • Test how usable this approach is for animating the bodies of characters  

  • Can it successfully track fingers for quality animated performances?  

Speech To Facial Animation Software:  

  • Look to achieve compelling facial performances from this software  

  • Do we need to test alternatives? (ie: Live Link Face for Unreal Engine) 

Omniverse:  

  • Can we easily connect our “Pose Estimation” workflow with our “Speech to Facial” animation workflow into a cohesive performance using Omniverse?  

Unreal Engine:  

  • Can these characters be successfully imported from Omniverse and integrated into their environments? How complex is it to implement this workflow for multiple characters performing in the same set?  

Cloud Based Storage / Digital Asset Manager (DAM)  

  • Integration of external cloud-based DAM & storage 

2021 Champions:

2021 Participants:

7-featuredimage

The Innovation & Findings: 

  • Using AI-based, markerless pose estimation software  

  • Speech to facial animation software  

  • Real-time render engine to generate animated content in remote workflows 

Final POC findings:  

  1. GALAXY DANCE: Short Animated 3D Films (30-90 second clips)  

  2. “Making Of” Presentation: How we accomplished this new workflow, including a short teaser to present what we want to do in the future phases.  

  3. Findings: What went well and what needs to be improved, with results to be continued in 2022 via EBU’s umbrella.