AR, VR, XR and MR create new realms for production


LA VEGAS—Over the past 18 months, the boom in virtual production has upended the way filmmakers, broadcasters, mobile video makers and many more conduct their creative business. These technologies enabling virtual production include augmented reality (AR), virtual reality (VR), extended reality (XR) and mixed reality (MR).

FOOTBALL’S BIGGEST STAGE: Augmented reality was used on one of sports’ biggest stages when the Super Bowl LVI halftime show created a 360-degree augmented reality experience for fans to view on their mobile devices. For the event, the organizers used Insta360 Pro 2 cameras from Insta360. This 360-degree camera with six F2.4 fisheye lenses features nine-axis gyroscopes and FlowState stabilization that provides stabilization against moving scenes.

The Pro 2 cameras were positioned around the stage and the footage was streamed over the Verizon 5G network to a show app at halftime. In the app, fans could choose one of eight different cameras around the pitch and rotate around the stage and pitch to get a 360-degree view, as if they were there.

Insta360 also helped deliver the world’s first 8K live stream of an NFL game when it covered the NFL Pro Bowl in Miami in February.

NAB Daily Show

(Image credit: NAB Show Daily)

AUGMENTED REALITY JUDO STATS: Augmented reality played an important role in the Judo Grand Prix 2022. Held in Almada, Portugal in February, the International Judo Federation (IJF) and wTVision joined forces to revamp the use of space, data and broadcast design at this annual three-day event. Organized for the first time in Tokyo in 1956, the event brings together more than 300 competitors from 41 countries.

This year, real-time data and broadcast graphics were created through wTVision’s JudoStats CG alongside the R3 Space Engine rendering system. Augmented reality graphics – created by JudoStats, R3 Space Engine and a tracking system from Stype – incorporated real-time stats, player information and performance data to create an interactive environment.

NAB Daily Show

(Image credit: NAB Show Daily)

VIRTUAL STUDIO AT WIMBLEDON: BBC Sport’s coverage of the annual Wimbledon Championships in 2021 has received an upgrade with virtual reality graphics technology from Brainstorm. The BBC, in consultation with British graphics company MOOV and Brainstorm, has created a virtual set studio. The studio used augmented reality graphics using Brainstorm’s InfinitySet XF/AR/VR studio app and integrated with the Epic Games Unreal Engine creation system.

Focused on the goal of telling better stories through improved data and graphics, green screen was used to create a virtual Wimbledon studio. The technology was also used to provide new and alternate camera angles, infographics and Wimbledon branding. During the event, five cameras were used in the studio, each equipped with a Brainstorm InfinitySet system with Unreal Engine. Advanced compositing features have been used to create realistic real-time 3D scenes with elements that look and behave like those in the real world.

“We were able to create a virtual space that enabled dynamic coverage of the tournament, which seamlessly combined real and virtual footage in a way that dramatically enhanced the story and audience engagement,” said Nev Appleton, director and co-founder of MOOV.

NAB Daily Show

(Image credit: NAB Show Daily)

THE GREEN SCREEN COMES TO THE CONFERENCE ROOM: Virtual reality brings the conference room to life. The University of the Netherlands worked with Zero Density to create a huge virtual lecture hall using the company’s Reality Engine real-time broadcast composition system.

Using virtual space, scientists and professors have access to interactive storytelling techniques and advanced visualization methods – with realistic real-time reflections and refractions of physical objects and people inside the green screen. Teachers can explain complex topics on a large virtual screen or conduct physical experiments with immersive real-time graphics.

The space uses Grass Valley LDX 86N 4K cameras along with Mo-Sys camera tracking technology, while the green screen conference space itself runs on dual reality engines.

NAB Daily Show

(Image credit: NAB Show Daily)

LONDON WELCOMES VIRTUAL PRODUCTION: When Garden Studios in London finalized its virtual production phase in 2021, the result was a 4,800 square foot space that could serve as a cost-effective filming option with unique creative opportunities. Garden Studios’ new virtual production studio allows filmmakers to shoot virtual effects in real time on set using virtual graphics displayed on an LED volume (an enclosed space where motion capture and compositing can take place) to create photorealistic backgrounds.

Among the technologies in place are the VP Pro XR server and StarTracker camera and lens tracking systems from Mo-Sys Engineering. Features of the VP Pro XR are an Unreal Engine editing interface and a feature known as Cinematic XR Focus which allows a filmmaker to focus between talent or objects in the physical studio and virtual objects positioned in the LED volume.

According to the company, this means that an LED volume can be used as more than just a backdrop, but can instead provide better interactivity between real and virtual elements. Jillian Sanders, virtual production coordinator for Garden Studios, said the Mo-Sys team came to the studio for a slew of tests, including displaying digital tracking markers on the studio’s LED ceiling.

“We were also able to help with the testing and development of their new VP Pro XR,” she said. “This exciting new tool enables features such as digital scenery extensions, the ability to focus beyond the LED wall into the digital world, and real-time rendering.”

Copyright NAB Show Daily


About Author

Comments are closed.