Free viewpoint television (FTV) is a system for viewing natural video, allowing the user to interactively control the viewpoint and generate new views of a dynamic scene from any 3D position. The equivalent system for computer-simulated video is known as

virtual reality Virtual reality (VR) is a simulated experience that employs pose tracking and 3D near-eye displays to give the user an immersive feel of a virtual world. Applications of virtual reality include entertainment (particularly video games), e ...

. With FTV, the focus of attention can be controlled by the viewers rather than a director, meaning that each viewer may be observing a unique viewpoint. It remains to be seen how FTV will affect television watching as a group activity.

History

Systems for rendering arbitrary views of natural scenes have been well known in the

computer vision Computer vision is an Interdisciplinarity, interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate t ...

community for a long time but only in recent years has the speed and quality reached levels that are suitable for serious consideration as an end-user system. Professor Masayuki Tanimoto from

Nagoya University , abbreviated to or NU, is a Japanese national research university located in Chikusa-ku, Nagoya. It was the seventh Imperial University in Japan, one of the first five Designated National University and selected as a Top Type university of ...

(Japan) has done much to promote the use of the term "free viewpoint television" and has published many papers on the

ray space representation Ray may refer to: Fish * Ray (fish), any cartilaginous fish of the superorder Batoidea * Ray (fish fin anatomy), a bony or horny spine on a fin Science and mathematics * Ray (geometry), half of a line proceeding from an initial point * Ray (g ...

, although other techniques can be, and are used for FTV.

QuickTime VR QuickTime VR (also known as QTVR) is an image file format developed by Apple Inc. for QuickTime, and discontinued along with QuickTime 7. It allows the creation and viewing of VR photography, photographically captured panoramas, and the viewing ...

might be considered a predecessor to FTV.

Capture and display

In order to acquire the views necessary to allow a high-quality rendering of the scene from any angle, several cameras are placed around the scene; either in a studio environment or an outdoor venue, such as a sporting arena for example. The output Multiview Video (MVV) must then be packaged suitably so that the data may be compressed and also so that the users' viewing device may easily access the relevant views to interpolate new views. It is not enough to simply place cameras around the scene to be captured. The geometry of the camera set up must be measured by a process known in computer vision as "camera calibration." Manual alignment would be too cumbersome so typically a "best effort" alignment is performed prior to capturing a test pattern that is used to generate calibration parameters. Restricted free viewpoint television views for large environments can be captured from a single location camera system mounted on a moving platform. Depth data must also be captured, which is necessary to generate the free viewpoint. The

Google Street View Google Street View is a technology featured in Google Maps and Google Earth that provides interactive panoramas from positions along many streets in the world. It was launched in 2007 in several cities in the United States, and has since exp ...

capture system is an example with limited functionality. The first full commercial implementation,

iFlex Oracle Financial Services Software Limited (OFSS) is a subsidiary of Oracle Corporation. It is a retail banking, corporate banking, and insurance technology solutions provider for the banking industry. It also provides risk and compliance manage ...

, was delivered in 2009 by

Real Time Race Real Time Race Limited is a company based at Daresbury Laboratory in North West England. The company was Founded in 2003 by Christopher Leigh and Stuart Scott-Goldstone. In 2008, Real Time Race worked with Malaysian owned Lotus Cars to test a v ...

. Multiview video capture varies from partial (usually about 30 degrees) to complete (360 degrees) coverage of the scene. Therefore, it is possible to output stereoscopic views suitable for viewing with a 3D display or other 3D methods. Systems with more physical cameras can capture images with more coverage of the viewable scene, however, it is likely that certain regions will always be occluded from any viewpoint. A larger number of cameras should make it possible to obtain high quality output because less interpolation is needed. More cameras mean that efficient coding of the Multiview Video is required. This may not be such a big disadvantage as there are representations that can remove the redundancy in MVV; such as inter view coding using

MPEG-4 MPEG-4 is a group of international standards for the compression of digital audio and visual data, multimedia systems, and file storage formats. It was originally introduced in late 1998 as a group of audio and video coding formats and related ...

or Multiview Video Coding, the ray space representation, geometry videos, etc. In terms of hardware, the user requires a viewing device that can decode MVV and synthesize new viewpoints, and a 2D or 3D display.

Standardization

The

Moving Picture Experts Group The Moving Picture Experts Group (MPEG) is an alliance of working groups established jointly by ISO and IEC that sets standards for media coding, including compression coding of audio, video, graphics, and genomic data; and transmission an ...

(MPEG) has normalized Annex H of

MPEG-4 AVC Advanced Video Coding (AVC), also referred to as H.264 or MPEG-4 Part 10, is a video compression standard based on block-oriented, motion-compensated coding. It is by far the most commonly used format for the recording, compression, and distr ...

in March 2009 called Multiview Video Coding after the work of a group called '3DAV' (3D Audio and Visual) headed by Aljoscha Smolic at the

Heinrich-Hertz Institute The Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI, also known as Fraunhofer HHI or Fraunhofer Heinrich Hertz Institute, is an organization of the Fraunhofer Society based in Berlin. The institute engages in applied re ...

References

Bibliography

External links

Canon announced development of a Free Viewpoint TV system
on 21 September 2017, to be showcased at Inter BEE 2017.
iview
is a British DTI project between

BBC #REDIRECT BBC Here i going to introduce about the best teacher of my life b BALAJI sir. He is the precious gift that I got befor 2yrs . How has helped and thought all the concept and made my success in the 10th board exam. ...

, Snell & Wilcox and

University of Surrey The University of Surrey is a public research university in Guildford, Surrey, England. The university received its royal charter in 1966, along with a number of other institutions following recommendations in the Robbins Report. The institu ...

to develop an FTV system.
Eye Vision
is a system developed by Professor

Takeo Kanade is a Japanese computer scientist and one of the world's foremost researchers in computer vision. He is U.A. and Helen Whitaker Professor at Carnegie Mellon University. He has approximately 300 peer-reviewed academic publications and holds aroun ...

at CMU for CBS's coverage of

Super Bowl XXXV Super Bowl XXXV was an American football game between the American Football Conference (AFC) champion Baltimore Ravens and the National Football Conference (NFC) champion New York Giants to decide the National Football League (NFL) champion fo ...