On-set virtual production (OSVP) is an entertainment technology for television and

film production Filmmaking or film production is the process by which a Film, motion picture is produced. Filmmaking involves a number of complex and discrete stages, beginning with an initial story, idea, or commission. Production then continues through screen ...

in which LED panels are used as a backdrop for a set, on which video or

computer-generated imagery Computer-generated imagery (CGI) is a specific-technology or application of computer graphics for creating or improving images in Digital art, art, Publishing, printed media, Training simulation, simulators, videos and video games. These images ...

can be displayed in real-time. The use of OSVP became widespread after its use in the first season of ''

The Mandalorian ''The Mandalorian'' is an American space Western television series created by Jon Favreau for the streaming service Disney+. It is the first live-action series in the ''Star Wars'' franchise and begins five years after the events of the fil ...

'' (2019), which used

Unreal Engine Unreal Engine (UE) is a 3D computer graphics game engine developed by Epic Games, first showcased in the 1998 first-person shooter video game '' Unreal''. Initially developed for PC first-person shooters, it has since been used in a variety of ...

, developed by

Epic Games Epic Games, Inc. is an American Video game developer, video game and software development, software developer and video game publisher, publisher based in Cary, North Carolina. The company was founded by Tim Sweeney (game developer), Tim Sween ...

History

Australian film director Clayton Jacobson first had the idea of improving the

green screen Chroma key compositing, or chroma keying, is a visual-effects and post-production technique for compositing (layering) two or more images or video streams together based on colour hues ( chroma range). The technique has been used in many fie ...

that was then in use when filming a TV ad for detergent in 2003. Watching his son playing

videogame A video game or computer game is an electronic game that involves interaction with a user interface or input device (such as a joystick, controller, keyboard, or motion sensing device) to generate visual feedback from a display device, most ...

s and seeing the 3D technology used in them gave him the idea. Eventually, in 2016, Jacobson and his son made one of the prototypes for a virtual production stage in their shed, using a set of LED screens. However, he could not get anyone to take an interest in developing the technology further, so gave up on it. Other filmmakers had also caught on to the idea though, and in 2018 an Australian

cinematographer The cinematographer or director of photography (sometimes shortened to DP or DOP) is the person responsible for the recording of a film, television production, music video or other live-action piece. The cinematographer is the chief of the camera ...

, Greig Fraser, used the technology to film the ''

Star Wars ''Star Wars'' is an American epic film, epic space opera media franchise created by George Lucas, which began with the Star Wars (film), eponymous 1977 film and Cultural impact of Star Wars, quickly became a worldwide popular culture, pop cu ...

'' franchise

spin-off Spin-off, Spin Off, Spin-Off, or Spinoff may refer to: Entertainment and media *Spinoff (media), a media work derived from an existing work *''The Spinoff'', a New Zealand current affairs magazine * ''Spin Off'' (Canadian game show), a 2013 Canad ...

series, ''The Mandalorian'' (released 2019). Instead of using the green screen during the filming stage, the team combined

post-production Post-production, also known simply as post, is part of the process of filmmaking, video production, audio production, and photography. Post-production includes all stages of production occurring after principal photography or recording indivi ...

with the production stage of the series. They installed huge LED walls linked to powerful computers that ran

gaming software (used for ''

Fortnite ''Fortnite'' is an online video game and game platform developed by Epic Games and released in 2017. It is available in seven distinct game mode versions that otherwise share the same general gameplay and game engine: '' Fortnite Battle Roy ...

'', among others). They called this soundstage "the volume", a term already used to refer to a stage where

visual effects Visual effects (sometimes abbreviated as VFX) is the process by which imagery is created or manipulated outside the context of a live-action shot in filmmaking and video production. The integration of live-action footage and other live-action fo ...

techniques take place. Since its inventive use in ''The Mandalorian'', which used ILM's

StageCraft Stagecraft is a technical aspect of theatrical, film, and video production. It includes constructing and rigging scenery; hanging and focusing of lighting; design and procurement of costumes; make-up; stage management; audio engineering; ...

, the technology has become increasingly popular. Miles Perkins, industry manager of film and TV for

and maker of the

, estimated that there were around 300 stages by October 2022, increased from only three in 2019. Most of these were built during or after the

COVID-19 pandemic The COVID-19 pandemic (also known as the coronavirus pandemic and COVID pandemic), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), began with an disease outbreak, outbreak of COVID-19 in Wuhan, China, in December ...

, when lockdowns meant that

production studio A production company, production house or production studio is a studio that creates works in the fields of performing arts, new media art, film, television, radio, comics, interactive arts, video games, websites, music, and video. These groups ...

s had to find ways to produce films without traveling to other locations. In March 2023, the world's largest virtual production stage was opened at the

Docklands Studios Melbourne Docklands Studios Melbourne is a major film and television production complex located in Melbourne’s redeveloped Docklands precinct. The site is approximately from Melbourne’s Central Business District. The complex opened in 2004 and its ...

, in the city of

Melbourne Melbourne ( , ; Boonwurrung language, Boonwurrung/ or ) is the List of Australian capital cities, capital and List of cities in Australia by population, most populous city of the States and territories of Australia, Australian state of Victori ...

, Australia.

Terminology

On-set virtual production (OSVP) is also known as virtual production (VP), immersive virtual production (IVP), In-Camera Visual Effects (ICVFX), or The Volume.

Technology

With careful adjustment and calibration, an OSVP set can be made to closely approximate the appearance of a real set or outdoor location. OSVP can be viewed as an application of

extended reality Extended reality (XR) is both an umbrella term to refer to and interpolate between augmented reality (AR), mixed reality (MR), and virtual reality (VR), as well as to extrapolate (extend) beyond these, e.g. allowing us to see sound waves, rad ...

. OSVP contrasts with virtual studio technology, in which a

backdrop surrounds the set, and the virtual surroundings are composited into the green screen plate downstream from the camera, in that in OSVP the virtual world surrounding the set is visible to the camera, actors, and crew, and objects on set are illuminated by light from the LED screen, creating realistic interactive lighting effects, and that the virtual background and foreground are captured directly in camera, complete with natural subtle cues like lens distortion,

depth of field The depth of field (DOF) is the distance between the nearest and the farthest objects that are in acceptably sharp focus (optics), focus in an image captured with a camera. See also the closely related depth of focus. Factors affecting depth ...

effects,

bokeh In photography, bokeh ( or ; ) is the aesthetic quality of the blur produced in out-of-focus parts of an image, whether foreground or background or both. It is created by using a wide aperture lens. Some photographers incorrectly restr ...

and

lens flare A lens flare happens when light is scattered, or ''flared'', in a lens system, often in response to a bright light, producing a sometimes undesirable artifact in the image. This happens through light scattered by the imaging mechanism itself, ...

. This makes it a far more natural experience that more closely approximates

location shooting Location shooting is the shooting of a film or television production in a real-world setting rather than a sound stage or backlot. The location may be interior or exterior. When filmmaking professionals refer to shooting "on location", they are ...

, making the film-making process faster and more intuitive than can be achieved on a virtual set. To render parallax depth cues correctly from the viewpoint of a moving camera, the system requires the use of

match moving In visual effects, match moving is a technique that allows the insertion of 2D elements, other live action elements or CG computer graphics into live-action footage with correct position, scale, orientation, and motion relative to the photograph ...

of the background imagery based on data from low-latency real-time

motion capture Motion capture (sometimes referred as mocap or mo-cap, for short) is the process of recording high-resolution motion (physics), movement of objects or people into a computer system. It is used in Military science, military, entertainment, sports ...

technology to track the camera, such systems as Stype and Mosys which track camera position within the volume and equate this position within the Unreal scene - so that any movement is reflected within the scene as well . Industry organizations including

SMPTE The Society of Motion Picture and Television Engineers (SMPTE) (, rarely ), founded by Charles Francis Jenkins in 1916 as the Society of Motion Picture Engineers or SMPE, is a global professional association of engineers, technologists, and e ...

, the

Academy of Motion Picture Arts and Sciences The Academy of Motion Picture Arts and Sciences (AMPAS, often pronounced ; also known as simply the Academy or the Motion Picture Academy) is a professional honorary organization in Beverly Hills, California, U.S., with the stated goal of adva ...

, and the

American Society of Cinematographers The American Society of Cinematographers (ASC), founded in Hollywood in 1919, is a cultural, educational, and professional organization that is neither a labor union nor a guild. The society was organized to advance the science and art of cinem ...

have started initiatives to support the development of OSVP.

Examples

Stages that use OSVP include the various StageCraft stages, Pixomondo's Toronto-based LED stage, which has a long-term lease from

CBS CBS Broadcasting Inc., commonly shortened to CBS (an abbreviation of its original name, Columbia Broadcasting System), is an American commercial broadcast television and radio network serving as the flagship property of the CBS Entertainme ...

, ZeroSpace's Volume in New York City, or Lux Machina various stages. In Japan, the LED wall and virtual production were used by

Toei Company , simply known as Toei Company or Toei, is a Japanese entertainment company. Headquartered in Ginza, Chūō, Tokyo, it is involved in film and television production, distribution, video game development, publishing, and ownership of 34 movi ...

for its

Super Sentai The is a Japanese superhero team media franchise consisting of television series and films produced by Toei Company and Bandai, and aired by TV Asahi. The shows are of the '' tokusatsu'' genre, featuring live action characters and colorfu ...

shows '' Avataro Sentai Donbrothers''VFXアナトミーインカメラVFXによる特撮をTVシリーズで実現！　『暴太郎戦隊ドンブラザーズ』
/ref> and '' Ohsama Sentai King-Ohger'', with the latter also being produced in collaboration with

Sony is a Japanese multinational conglomerate (company), conglomerate headquartered at Sony City in Minato, Tokyo, Japan. The Sony Group encompasses various businesses, including Sony Corporation (electronics), Sony Semiconductor Solutions (i ...

PCL Inc. 新番組「王様戦隊キングオージャー」の背景CGがハイクオリティーと話題に　グリーンバックを使わない最新撮影技術とは？
/ref> the largest OSVP is at Docklands Studios Melbourne in Australia.

Productions using the technologies

Television series

* ''

'' (2019–present) * '' The Book of Boba Fett'' (2021) * '' Star Trek: Discovery'' (2021–2024) * '' Star Trek: Strange New Worlds'' (2022–present) * '' How I Met Your Father'' (2022–2023) * '' Our Flag Means Death'' (2022–present) * ''

Obi-Wan Kenobi Obi-Wan "Ben" Kenobi () is a character in the ''Star Wars'' franchise. In the original film trilogy, he is a Jedi Master who trains Luke Skywalker in the ways of the Force. In the prequel trilogy, he mentors Luke's father, Anakin Skywalke ...

'' (2022) * '' The Old Man'' (2022) * '' Andor'' (2022) * '' The Lord of the Rings: The Rings of Power'' (2022-present) * ''

House of the Dragon ''House of the Dragon'' is an American fantasy drama television series created by George R. R. Martin and Ryan Condal for HBO. A prequel to ''Game of Thrones'' (2011–2019), it is the second television series in Martin's ''A Song of Ice and F ...

'' (2022–present) * ''

1899 Events January * January 1 ** Spanish rule formally ends in Cuba with the cession of Spanish sovereignty to the U.S., concluding 400 years of the Spanish Empire in the Americas.''The American Monthly Review of Reviews'' (February 1899), p ...

'' (2022) * '' Avataro Sentai Donbrothers'' (2022–2023) * '' Ohsama Sentai King-Ohger'' (2023–2024) * ''

Silo A silo () is a structure for storing Bulk material handling, bulk materials. Silos are commonly used for bulk storage of grain, coal, cement, carbon black, woodchips, food products and sawdust. Three types of silos are in widespread use toda ...

'' (2023) * '' The Muppets Mayhem'' (2023) * '' Ahsoka'' (2023) * ''

One Piece ''One Piece'' (stylized in all caps) is a Japanese manga series written and illustrated by Eiichiro Oda. It follows the adventures of Monkey D. Luffy and his crew, the Straw Hat Pirates, as he explores the Grand Line in search of the myt ...

'' (2023-present) * ''

YuYu Hakusho is a Japanese manga series written and illustrated by Yoshihiro Togashi. It tells the story of Yusuke Urameshi, a teenage delinquent who is struck and killed by a car while saving a child's life. After several tests presented to him ...

'' (2023) * '' Percy Jackson and the Olympians'' (2023-present) * ''

Masters of the Air ''Masters of the Air'' is a 2024 American war drama miniseries created by John Shiban and John Orloff for Apple TV+. It is based on the 2007 book of the same name by Donald L. Miller and follows the actions of the 100th Bomb Group, a Boeing ...

'' (2024) * ''

Fallout Nuclear fallout is residual radioactive material that is created by the reactions producing a nuclear explosion. It is initially present in the radioactive cloud created by the explosion, and "falls out" of the cloud as it is moved by the ...

'' (2024-present) * ''

Doctor Who ''Doctor Who'' is a British science fiction television series broadcast by the BBC since 1963. The series, created by Sydney Newman, C. E. Webber and Donald Wilson (writer and producer), Donald Wilson, depicts the adventures of an extraterre ...

'' series 14 (2024) * '' The Acolyte'' (2024) * '' Those About to Die'' (2024) * ''

Time Bandits ''Time Bandits'' is a 1981 British fantasy adventure film co-written, produced, and directed by Terry Gilliam. It stars David Rappaport, Sean Connery, John Cleese, Shelley Duvall, Ralph Richardson, Katherine Helmond, Ian Holm, Michael ...

'' (2024) * '' Star Wars: Skeleton Crew'' (2024-2025) * '' The Eternaut'' (2025)

Feature films

* ''

Avatar Avatar (, ; ) is a concept within Hinduism that in Sanskrit literally means . It signifies the material appearance or incarnation of a powerful deity, or spirit on Earth. The relative verb to "alight, to make one's appearance" is sometimes u ...

'' (2009) * '' Oblivion'' (2013) * ''

The Jungle Book ''The Jungle Book'' is an 1894 collection of stories by the English author Rudyard Kipling. Most of the characters are animals such as Shere Khan the tiger and Baloo the bear, though a principal character is the boy or "man-cub" Mowgli, who ...

'' (2016) * '' Rogue One: A Star Wars Story'' (2016) * ''Ghost in the Shell'' (2017) * '' Solo: A Star Wars Story'' (2018) * '' First Man'' (2018) * '' Alita: Battle Angel'' (2019) * ''

The Lion King ''The Lion King'' is a 1994 American animated musical coming-of-age drama film directed by Roger Allers and Rob Minkoff, produced by Don Hahn, and written by Irene Mecchi, Jonathan Roberts, and Linda Woolverton. Produced by Walt Disney ...

'' (2019) * '' The Irishman'' (2019) * '' Star Wars: The Rise of Skywalker'' (2019) * '' The One and Only Ivan'' (2020) * '' The Midnight Sky'' (2020) * '' In vacanza su Marte'' (2020) * '' Venom: Let There Be Carnage'' (2021) * ''Red Notice'' (2021) * ''

Don't Look Up ''Don't Look Up'' is a 2021 American political satire black comedy film written, co-produced, and directed by Adam McKay from a story he co-wrote with David Sirota. It stars an ensemble cast featuring Leonardo DiCaprio, Jennifer Lawrence, R ...

'' (2021) * ''

Death on the Nile ''Death on the Nile'' is a work of detective fiction by British writer Agatha Christie, published in the UK by the Collins Crime Club on 1 November 1937 and in the US by Dodd, Mead and Company the following year. The UK edition retailed at ...

'' (2022) * '' The Batman'' (2022) * '' Top Gun: Maverick'' (2022) * '' Thor: Love and Thunder'' (2022) * '' The Gray Man'' (2022) * '' Bullet Train'' (2022) * ''

Pinocchio Pinocchio ( , ) is a fictional character and the protagonist of the children's novel, ''The Adventures of Pinocchio'' (1883) by Italian writer Carlo Collodi of Florence, Tuscany. Pinocchio was carved by a poor man named Geppetto in a Tuscan vil ...

'' (2022) * ''

Divinity Divinity (from Latin ) refers to the quality, presence, or nature of that which is divine—a term that, before the rise of monotheism, evoked a broad and dynamic field of sacred power. In the ancient world, divinity was not limited to a single ...

'' (2022) * ''

Black Adam Black Adam (Teth-Adam) is a fictional character appearing in American comic books published by DC Comics. He was created by Otto Binder and C. C. Beck, and first appeared in the debut issue of Fawcett Comics' '' The Marvel Family'' comic book ...

'' (2022) * '' Devotion'' (2022) * ''

The Fabelmans ''The Fabelmans'' is a 2022 American coming-of-age drama film directed and co-produced by Steven Spielberg, who co-wrote the screenplay with Tony Kushner. Loosely based on Spielberg's early life and beginnings as a filmmaker, the semi-autobiog ...

'' (2022) * '' Avatar: The Way of Water'' (2022) * '' Ant-Man and the Wasp: Quantumania'' (2023) * ''

Shazam! Fury of the Gods ''Shazam! Fury of the Gods'' is a 2023 American superhero film based on the DC character Shazam. Produced by New Line Cinema, DC Films, and the Safran Company, and distributed by Warner Bros. Pictures, it is the sequel to '' Shazam!'' (201 ...

'' (2023) * ''

The Flash The Flash is the name of several superheroes appearing in American comic books published by DC Comics. Created by writer Gardner Fox and artist Harry Lampert, the original Flash first appeared in ''Flash Comics'' #1 (cover date, cover-dated Jan ...

'' (2023) * ''

Barbie Barbie is a fashion doll created by American businesswoman Ruth Handler, manufactured by American toy and entertainment company Mattel and introduced on March 9, 1959. The toy was based on the German Bild Lilli doll, Bild Lilli doll which Hand ...

'' (2023) *'' The Creator'' (2023) * '' The Killer ''(2023) * '' The Marvels'' (2023) * '' Candy Cane Lane'' (2023) * '' Poor Things'' (2023) * '' Society of the Snow'' (2023) * '' Aquaman and the Lost Kingdom'' (2023) * '' No Way Up'' (2024) * '' Dune: Part Two'' (2024) * '' Godzilla x Kong: The New Empire'' (2024) * '' Daddio'' (2024) * ''

Transformers One ''Transformers One'' is a 2024 American animated science fiction action film based on Hasbro's ''Transformers'' toy line. It was directed by Josh Cooley from a screenplay by Eric Pearson, Andrew Barrer and Gabriel Ferrari, the latter two of ...

'' (2024) * ''

Megalopolis A megalopolis () or a supercity, also called a megaregion, is a group of metropolitan areas which are perceived as a continuous urban area through common systems of transport, economy, resources, ecology, and so on. They are integrated enough ...

'' (2024) * '' Joker: Folie à Deux'' (2024) * '' Venom: The Last Dance'' (2024) * ''

Here Here may refer to: Music * ''Here'' (Adrian Belew album), 1994 * ''Here'' (Alicia Keys album), 2016 * ''Here'' (Cal Tjader album), 1979 * ''Here'' (Edward Sharpe album), 2012 * ''Here'' (Idina Menzel album), 2004 * ''Here'' (Merzbow album), ...

'' (2024) * '' Mufasa: The Lion King'' (2024) * ''

Snow White "Snow White" is a German fairy tale, first written down in the early 19th century. The Brothers Grimm published it in 1812 in the first edition of their collection ''Grimms' Fairy Tales'', numbered as Tale 53. The original title was ''Sneewittch ...

'' (2025) * '' Final Destination Bloodlines'' (2025) * '' Tron: Ares'' (2025) * ''

Mercy Mercy (Middle English, from Anglo-French , from Medieval Latin , "price paid, wages", from Latin , "merchandise") is benevolence, forgiveness, and kindness in a variety of ethical, religious, social, and legal contexts. In the social and le ...

'' (2026) * ''Campbeltown '69'' (TBD)

Short films

* ''The Vandal'' (2021)

Notes

References

{{Mixed reality Cinematic techniques Visual effects Mixed reality *