ACCELERATOR PROJECT

5G MOTION CAPTURE FOR LIVE PERFORMANCE AND ANIMATION

FINAL SHOWCASE SESSION

Watch the 5G Motion Capture for Live Performance and Animation project's Final Showcase Session in the hero video above, and also via IBC365.

View on IBC365

The Challenge:

This ambitious challenge combined two previous Accelerator Award-winning teams under one umbrella. After a hugely successful project last year that saw private 5G networks being deployed around the world and used to support several different outdoor broadcast production workflows, their new objective was to harness the capabilities of 5G indoors for media and entertainment, with a focus on two distinct creative workstreams. These workstreams enabled cutting-edge motion capture and low-latency connectivity for live creative output, specifically targeting the performing arts and scripted 3D animation in children's hospitals.

The goal of the first workstream was to support the creation of immersive audience experiences for live or near-live performances, both for those physically present and those engaging remotely in different venues. To do this, they used innovative software-defined radio to demonstrate ultra-low latency private 5G networks and used them to carry motion capture data and low-latency wireless video feeds.

The second workstream, focusing on Live 3D Animation, demonstrated a sustainable, accessible, and inclusive way to entertain children in need. It showcased how IP holders could bring animated content to hospital wards worldwide without relying on costly render farms or outsourcing production. This workstream also emphasized creating new formats of 'live' children's content with meaningful two-way interaction between the children and animated characters in fantastical virtual worlds.

Kickstart Day Pitch:

Champions: Vodafone Group, Curing Kids Cancer, Outernet Global, Production Park, University of Strathclyde, Neutral Wireless, Goldsmiths University

Participants: D&B Solutions, Noitom, AMD

Workstream 1: 5G & Live Performing Arts

1. Experimented with building new low-latency audio and video 'illusions' that could be transmitted from any location to any destination over suitable backhaul, whether terrestrial or celestial public internet.

2. Used a Neutral Wireless private 5G network to transmit motion capture and video data, overcoming WiFi congestion at events in crowded auditoriums and increasing the range from performer to receiver.

3. Innovated and explored novel ways to interact with live multi-venue performances using motion capture and VR projection techniques. Achieved ~45ms from local movement to rendered avatars displayed in the remote venue, and ~100ms for video.

4. Considered the challenge of synchronizing audio and video – exploit factors like the speed of sound and latency of video compression codecs to achieve audiovisual synchronization at a particular distance from a projection screen.

5. Designed highly mobile and quickly deployable setups to overcome the prohibitive costs of cable runs, thus easing health and safety requirements, enabling the opening of venues that were previously challenging to cable, and allowing directors complete creative freedom within those spaces.

6. Prioritized environmental sustainability by building upon the 'low' carbon approach from the "5G in the Middle of Nowhere" project in 2022.

Workstream 2: 5G & Live 3D Animation

1. Utilized AniSign and motion capture technologies to enable the real-time performance of animated characters.

2. Leveraged global connectivity to facilitate interactions between performers and audiences regardless of location.

3. Presented scripted and improvised storytelling through animated characters in a detailed world rendered in real-time.

4. Employed virtual cameras to seamlessly integrate audience-directed activities and viewpoints into the animated world.

5. Created a live performance system capable of triggering changes within the animated world based on the emotions and responses from the children. With special thanks to two of our Champions, Curing Kids Cancer and Texas Children’s Hospital.