On-set virtual production (OSVP) is an entertainment technology for television and

film production Filmmaking or film production is the process by which a Film, motion picture is produced. Filmmaking involves a number of complex and discrete stages, beginning with an initial story, idea, or commission. Production then continues through screen ...

in which LED panels are used as a backdrop for a set, on which video or

computer-generated imagery Computer-generated imagery (CGI) is a specific-technology or application of computer graphics for creating or improving images in Digital art, art, Publishing, printed media, Training simulation, simulators, videos and video games. These images ...

can be displayed in real-time. The use of OSVP became widespread after its use in the first season of '' The Mandalorian'' (2019), which used

Unreal Engine Unreal Engine (UE) is a 3D computer graphics game engine developed by Epic Games, first showcased in the 1998 first-person shooter video game '' Unreal''. Initially developed for PC first-person shooters, it has since been used in a variety of ...

, developed by

Epic Games Epic Games, Inc. is an American Video game developer, video game and software development, software developer and video game publisher, publisher based in Cary, North Carolina. The company was founded by Tim Sweeney (game developer), Tim Sween ...

History

Australian film director Clayton Jacobson first had the idea of improving the green screen that was then in use when filming a TV ad for detergent in 2003. Watching his son playing videogames and seeing the 3D technology used in them gave him the idea. Eventually, in 2016, Jacobson and his son made one of the prototypes for a virtual production stage in their shed, using a set of LED screens. However, he could not get anyone to take an interest in developing the technology further, so gave up on it. Other filmmakers had also caught on to the idea though, and in 2018 an Australian

cinematographer The cinematographer or director of photography (sometimes shortened to DP or DOP) is the person responsible for the recording of a film, television production, music video or other live-action piece. The cinematographer is the chief of the camera ...

, Greig Fraser, used the technology to film the ''

Star Wars ''Star Wars'' is an American epic film, epic space opera media franchise created by George Lucas, which began with the Star Wars (film), eponymous 1977 film and Cultural impact of Star Wars, quickly became a worldwide popular culture, pop cu ...

'' franchise spin-off series, ''The Mandalorian'' (released 2019). Instead of using the green screen during the filming stage, the team combined

post-production Post-production, also known simply as post, is part of the process of filmmaking, video production, audio production, and photography. Post-production includes all stages of production occurring after principal photography or recording indivi ...

with the production stage of the series. They installed huge LED walls linked to powerful computers that ran

gaming software (used for '' Fortnite'', among others). They called this soundstage "the volume", a term already used to refer to a stage where

visual effects Visual effects (sometimes abbreviated as VFX) is the process by which imagery is created or manipulated outside the context of a live-action shot in filmmaking and video production. The integration of live-action footage and other live-action fo ...

techniques take place. Since its inventive use in ''The Mandalorian'', which used ILM's

StageCraft Stagecraft is a technical aspect of theatrical, film, and video production. It includes constructing and rigging scenery; hanging and focusing of lighting; design and procurement of costumes; make-up; stage management; audio engineering; ...

, the technology has become increasingly popular. Miles Perkins, industry manager of film and TV for

and maker of the

, estimated that there were around 300 stages by October 2022, increased from only three in 2019. Most of these were built during or after the

COVID-19 pandemic The COVID-19 pandemic (also known as the coronavirus pandemic and COVID pandemic), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), began with an disease outbreak, outbreak of COVID-19 in Wuhan, China, in December ...

, when lockdowns meant that

production studio A production company, production house or production studio is a studio that creates works in the fields of performing arts, new media art, film, television, radio, comics, interactive arts, video games, websites, music, and video. These groups ...

s had to find ways to produce films without traveling to other locations. In March 2023, the world's largest virtual production stage was opened at the

Docklands Studios Melbourne Docklands Studios Melbourne is a major film and television production complex located in Melbourne’s redeveloped Docklands precinct. The site is approximately from Melbourne’s Central Business District. The complex opened in 2004 and its ...

, in the city of

Melbourne Melbourne ( , ; Boonwurrung language, Boonwurrung/ or ) is the List of Australian capital cities, capital and List of cities in Australia by population, most populous city of the States and territories of Australia, Australian state of Victori ...

, Australia.

Terminology

On-set virtual production (OSVP) is also known as virtual production (VP), immersive virtual production (IVP), In-Camera Visual Effects (ICVFX), or The Volume.

Technology

With careful adjustment and calibration, an OSVP set can be made to closely approximate the appearance of a real set or outdoor location. OSVP can be viewed as an application of

extended reality Extended reality (XR) is both an umbrella term to refer to and interpolate between augmented reality (AR), mixed reality (MR), and virtual reality (VR), as well as to extrapolate (extend) beyond these, e.g. allowing us to see sound waves, rad ...

. OSVP contrasts with virtual studio technology, in which a green screen backdrop surrounds the set, and the virtual surroundings are composited into the green screen plate downstream from the camera, in that in OSVP the virtual world surrounding the set is visible to the camera, actors, and crew, and objects on set are illuminated by light from the LED screen, creating realistic interactive lighting effects, and that the virtual background and foreground are captured directly in camera, complete with natural subtle cues like lens distortion,

depth of field The depth of field (DOF) is the distance between the nearest and the farthest objects that are in acceptably sharp focus (optics), focus in an image captured with a camera. See also the closely related depth of focus. Factors affecting depth ...

effects, bokeh and lens flare. This makes it a far more natural experience that more closely approximates

location shooting Location shooting is the shooting of a film or television production in a real-world setting rather than a sound stage or backlot. The location may be interior or exterior. When filmmaking professionals refer to shooting "on location", they are ...

, making the film-making process faster and more intuitive than can be achieved on a virtual set. To render parallax depth cues correctly from the viewpoint of a moving camera, the system requires the use of

match moving In visual effects, match moving is a technique that allows the insertion of 2D elements, other live action elements or CG computer graphics into live-action footage with correct position, scale, orientation, and motion relative to the photograph ...

of the background imagery based on data from low-latency real-time

motion capture Motion capture (sometimes referred as mocap or mo-cap, for short) is the process of recording high-resolution motion (physics), movement of objects or people into a computer system. It is used in Military science, military, entertainment, sports ...

technology to track the camera, such systems as Stype and Mosys which track camera position within the volume and equate this position within the Unreal scene - so that any movement is reflected within the scene as well . Industry organizations including

SMPTE The Society of Motion Picture and Television Engineers (SMPTE) (, rarely ), founded by Charles Francis Jenkins in 1916 as the Society of Motion Picture Engineers or SMPE, is a global professional association of engineers, technologists, and e ...

, the

Academy of Motion Picture Arts and Sciences The Academy of Motion Picture Arts and Sciences (AMPAS, often pronounced ; also known as simply the Academy or the Motion Picture Academy) is a professional honorary organization in Beverly Hills, California, U.S., with the stated goal of adva ...

, and the American Society of Cinematographers have started initiatives to support the development of OSVP.

Examples

Stages that use OSVP include the various StageCraft stages, Pixomondo's Toronto-based LED stage, which has a long-term lease from CBS, ZeroSpace's Volume in New York City, or Lux Machina various stages. In Japan, the LED wall and virtual production were used by

Toei Company , simply known as Toei Company or Toei, is a Japanese entertainment company. Headquartered in Ginza, Chūō, Tokyo, it is involved in film and television production, distribution, video game development, publishing, and ownership of 34 movi ...

for its

Super Sentai The is a Japanese superhero team media franchise consisting of television series and films produced by Toei Company and Bandai, and aired by TV Asahi. The shows are of the '' tokusatsu'' genre, featuring live action characters and colorfu ...

shows '' Avataro Sentai Donbrothers''VFXアナトミーインカメラVFXによる特撮をTVシリーズで実現！　『暴太郎戦隊ドンブラザーズ』
/ref> and '' Ohsama Sentai King-Ohger'', with the latter also being produced in collaboration with

Sony is a Japanese multinational conglomerate (company), conglomerate headquartered at Sony City in Minato, Tokyo, Japan. The Sony Group encompasses various businesses, including Sony Corporation (electronics), Sony Semiconductor Solutions (i ...

PCL Inc. 新番組「王様戦隊キングオージャー」の背景CGがハイクオリティーと話題に　グリーンバックを使わない最新撮影技術とは？
/ref> the largest OSVP is at Docklands Studios Melbourne in Australia.

Productions using the technologies

Television series

* '' The Mandalorian'' (2019–present) * '' The Book of Boba Fett'' (2021) * '' Star Trek: Discovery'' (2021–2024) * '' Star Trek: Strange New Worlds'' (2022–present) * '' How I Met Your Father'' (2022–2023) * '' Our Flag Means Death'' (2022–present) * ''

Obi-Wan Kenobi Obi-Wan "Ben" Kenobi () is a character in the ''Star Wars'' franchise. In the original film trilogy, he is a Jedi Master who trains Luke Skywalker in the ways of the Force. In the prequel trilogy, he mentors Luke's father, Anakin Skywalke ...

'' (2022) * '' The Old Man'' (2022) * '' Andor'' (2022) * '' The Lord of the Rings: The Rings of Power'' (2022-present) * '' House of the Dragon'' (2022–present) * ''

1899 Events January * January 1 ** Spanish rule formally ends in Cuba with the cession of Spanish sovereignty to the U.S., concluding 400 years of the Spanish Empire in the Americas.''The American Monthly Review of Reviews'' (February 1899), p ...

'' (2022) * '' Avataro Sentai Donbrothers'' (2022–2023) * '' Ohsama Sentai King-Ohger'' (2023–2024) * ''

Silo A silo () is a structure for storing Bulk material handling, bulk materials. Silos are commonly used for bulk storage of grain, coal, cement, carbon black, woodchips, food products and sawdust. Three types of silos are in widespread use toda ...

'' (2023) * '' The Muppets Mayhem'' (2023) * '' Ahsoka'' (2023) * ''

One Piece ''One Piece'' (stylized in all caps) is a Japanese manga series written and illustrated by Eiichiro Oda. It follows the adventures of Monkey D. Luffy and his crew, the Straw Hat Pirates, as he explores the Grand Line in search of the myt ...

'' (2023-present) * ''

YuYu Hakusho is a Japanese manga series written and illustrated by Yoshihiro Togashi. It tells the story of Yusuke Urameshi, a teenage delinquent who is struck and killed by a car while saving a child's life. After several tests presented to him ...

'' (2023) * '' Percy Jackson and the Olympians'' (2023-present) * '' Masters of the Air'' (2024) * '' Fallout'' (2024-present) * ''

Doctor Who ''Doctor Who'' is a British science fiction television series broadcast by the BBC since 1963. The series, created by Sydney Newman, C. E. Webber and Donald Wilson (writer and producer), Donald Wilson, depicts the adventures of an extraterre ...

'' series 14 (2024) * '' The Acolyte'' (2024) * '' Those About to Die'' (2024) * ''

Time Bandits ''Time Bandits'' is a 1981 British fantasy adventure film co-written, produced, and directed by Terry Gilliam. It stars David Rappaport, Sean Connery, John Cleese, Shelley Duvall, Ralph Richardson, Katherine Helmond, Ian Holm, Michael ...

'' (2024) * '' Star Wars: Skeleton Crew'' (2024-2025) * '' The Eternaut'' (2025)

Feature films

* ''

Avatar Avatar (, ; ) is a concept within Hinduism that in Sanskrit literally means . It signifies the material appearance or incarnation of a powerful deity, or spirit on Earth. The relative verb to "alight, to make one's appearance" is sometimes u ...

'' (2009) * '' Oblivion'' (2013) * ''

The Jungle Book ''The Jungle Book'' is an 1894 collection of stories by the English author Rudyard Kipling. Most of the characters are animals such as Shere Khan the tiger and Baloo the bear, though a principal character is the boy or "man-cub" Mowgli, who ...

'' (2016) * '' Rogue One: A Star Wars Story'' (2016) * ''Ghost in the Shell'' (2017) * '' Solo: A Star Wars Story'' (2018) * '' First Man'' (2018) * '' Alita: Battle Angel'' (2019) * ''

The Lion King ''The Lion King'' is a 1994 American animated musical coming-of-age drama film directed by Roger Allers and Rob Minkoff, produced by Don Hahn, and written by Irene Mecchi, Jonathan Roberts, and Linda Woolverton. Produced by Walt Disney ...

'' (2019) * '' The Irishman'' (2019) * '' Star Wars: The Rise of Skywalker'' (2019) * '' The One and Only Ivan'' (2020) * '' The Midnight Sky'' (2020) * '' In vacanza su Marte'' (2020) * '' Venom: Let There Be Carnage'' (2021) * ''Red Notice'' (2021) * '' Don't Look Up'' (2021) * ''

Death on the Nile ''Death on the Nile'' is a work of detective fiction by British writer Agatha Christie, published in the UK by the Collins Crime Club on 1 November 1937 and in the US by Dodd, Mead and Company the following year. The UK edition retailed at ...

'' (2022) * '' The Batman'' (2022) * '' Top Gun: Maverick'' (2022) * '' Thor: Love and Thunder'' (2022) * '' The Gray Man'' (2022) * '' Bullet Train'' (2022) * ''

Pinocchio Pinocchio ( , ) is a fictional character and the protagonist of the children's novel, ''The Adventures of Pinocchio'' (1883) by Italian writer Carlo Collodi of Florence, Tuscany. Pinocchio was carved by a poor man named Geppetto in a Tuscan vil ...

'' (2022) * ''

Divinity Divinity (from Latin ) refers to the quality, presence, or nature of that which is divine—a term that, before the rise of monotheism, evoked a broad and dynamic field of sacred power. In the ancient world, divinity was not limited to a single ...

'' (2022) * '' Black Adam'' (2022) * '' Devotion'' (2022) * '' The Fabelmans'' (2022) * '' Avatar: The Way of Water'' (2022) * '' Ant-Man and the Wasp: Quantumania'' (2023) * ''

Shazam! Fury of the Gods ''Shazam! Fury of the Gods'' is a 2023 American superhero film based on the DC character Shazam. Produced by New Line Cinema, DC Films, and the Safran Company, and distributed by Warner Bros. Pictures, it is the sequel to '' Shazam!'' (201 ...

'' (2023) * ''

The Flash The Flash is the name of several superheroes appearing in American comic books published by DC Comics. Created by writer Gardner Fox and artist Harry Lampert, the original Flash first appeared in ''Flash Comics'' #1 (cover date, cover-dated Jan ...

'' (2023) * ''

Barbie Barbie is a fashion doll created by American businesswoman Ruth Handler, manufactured by American toy and entertainment company Mattel and introduced on March 9, 1959. The toy was based on the German Bild Lilli doll, Bild Lilli doll which Hand ...

'' (2023) *'' The Creator'' (2023) * '' The Killer ''(2023) * '' The Marvels'' (2023) * '' Candy Cane Lane'' (2023) * '' Poor Things'' (2023) * '' Society of the Snow'' (2023) * '' Aquaman and the Lost Kingdom'' (2023) * '' No Way Up'' (2024) * '' Dune: Part Two'' (2024) * '' Godzilla x Kong: The New Empire'' (2024) * '' Daddio'' (2024) * '' Transformers One'' (2024) * ''

Megalopolis A megalopolis () or a supercity, also called a megaregion, is a group of metropolitan areas which are perceived as a continuous urban area through common systems of transport, economy, resources, ecology, and so on. They are integrated enough ...

'' (2024) * '' Joker: Folie à Deux'' (2024) * '' Venom: The Last Dance'' (2024) * '' Here'' (2024) * '' Mufasa: The Lion King'' (2024) * ''

Snow White "Snow White" is a German fairy tale, first written down in the early 19th century. The Brothers Grimm published it in 1812 in the first edition of their collection ''Grimms' Fairy Tales'', numbered as Tale 53. The original title was ''Sneewittch ...

'' (2025) * '' Final Destination Bloodlines'' (2025) * '' Tron: Ares'' (2025) * ''

Mercy Mercy (Middle English, from Anglo-French , from Medieval Latin , "price paid, wages", from Latin , "merchandise") is benevolence, forgiveness, and kindness in a variety of ethical, religious, social, and legal contexts. In the social and le ...

'' (2026) * ''Campbeltown '69'' (TBD)

Short films

* ''The Vandal'' (2021)

Notes

References

{{Mixed reality Cinematic techniques Visual effects Mixed reality *