16:1 compression. Foveated image with fixation point at Stephen F. Austin statue. Foveated imaging is a

digital image processing Digital image processing is the use of a digital computer to process digital images through an algorithm. As a subcategory or field of digital signal processing, digital image processing has many advantages over analog image processing. It allo ...

technique in which the

image resolution Image resolution is the level of detail of an image. The term applies to digital images, film images, and other types of images. "Higher resolution" means more image detail. Image resolution can be measured in various ways. Resolution quantifies ...

, or amount of detail, varies across the

image An image or picture is a visual representation. An image can be Two-dimensional space, two-dimensional, such as a drawing, painting, or photograph, or Three-dimensional space, three-dimensional, such as a carving or sculpture. Images may be di ...

according to one or more "fixation points". A fixation point indicates the highest resolution region of the image and corresponds to the center of the

eye An eye is a sensory organ that allows an organism to perceive visual information. It detects light and converts it into electro-chemical impulses in neurons (neurones). It is part of an organism's visual system. In higher organisms, the ey ...

retina The retina (; or retinas) is the innermost, photosensitivity, light-sensitive layer of tissue (biology), tissue of the eye of most vertebrates and some Mollusca, molluscs. The optics of the eye create a focus (optics), focused two-dimensional ...

, the fovea. The location of a fixation point may be specified in many ways. For example, when viewing an image on a

computer monitor A computer monitor is an output device that displays information in pictorial or textual form. A discrete monitor comprises a electronic visual display, visual display, support electronics, power supply, Housing (engineering), housing, electri ...

, one may specify a fixation using a

pointing device A pointing device is a human interface device that allows a User (computing)#End-user, user to input Three-dimensional space, spatial (i.e., continuous and multi-dimensional) data to a computer. Graphical user interfaces (GUI) and Computer- ...

, like a computer mouse.

Eye tracker Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research ...

s which precisely measure the eye's position and movement are also commonly used to determine fixation points in perception experiments. When the display is manipulated with the use of an eye tracker, this is known as a gaze contingent display. Fixations may also be determined automatically using

computer algorithm In mathematics and computer science, an algorithm () is a finite sequence of Rigour#Mathematics, mathematically rigorous instructions, typically used to solve a class of specific Computational problem, problems or to perform a computation. Algo ...

s. Some common applications of foveated imaging include imaging sensor hardware and image compression. For descriptions of these and other applications, see the list below. Miniaturized foveated imaging systems can be realized by high-resolution 3D printing of multi-lens objectives directly on a

CMOS Complementary metal–oxide–semiconductor (CMOS, pronounced "sea-moss ", , ) is a type of MOSFET, metal–oxide–semiconductor field-effect transistor (MOSFET) semiconductor device fabrication, fabrication process that uses complementary an ...

(Complementary metal-oxide-semiconductor) chip. Foveated imaging is also commonly referred to as space variant imaging or gaze contingent imaging.

Applications

Foveated imaging for progressive transmission

Compression

Contrast sensitivity Contrast is the difference in luminance or color that makes an object (or its representation in an image or display) visible against a background of different luminance or color. The human visual system is more sensitive to contrast than to absolu ...

falls off dramatically as one moves from the center of the retina to the periphery. In lossy image compression, one may take advantage of this fact in order to compactly encode images. If one knows the viewer's approximate point of gaze, one may reduce the amount of information contained in the image as the distance from the point of gaze increases. Because the fall-off in the eye's resolution is dramatic, the potential reduction in display information can be substantial. Also, foveation encoding may be applied to the image before other types of image compression are applied and therefore can result in a multiplicative reduction.

Foveated sensors

Foveated sensors are multiresolution hardware devices that allow image data to be collected with higher resolution concentrated at a fixation point. An advantage to using foveated sensor hardware is that the image collection and encoding can occur much faster than in a system that post-processes a high resolution image in software.

Simulation

Foveated imaging has been used to simulate visual fields with arbitrary spatial resolution. For example, one may present video containing a blurred region representing a

scotoma A scotoma is an area of partial alteration in the field of vision consisting of a partially diminished or entirely degenerated visual acuity that is surrounded by a field of normal – or relatively well-preserved – vision. Every normal mamm ...

. By using an eye-tracker and holding the blurred region fixed relative to the viewer's gaze, the viewer will have a visual experience similar to that of a person with an actual scotoma.

Video gaming

Foveated rendering Foveated rendering is a rendering technique which uses an eye tracker integrated with a virtual reality headset to reduce the rendering workload by greatly reducing the image quality in the peripheral vision (outside of the zone gazed by the fov ...

is a technique which uses an

eye tracker Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research ...

integrated with a

virtual reality headset A virtual reality headset (or VR headset) is a Head-mounted display, head-mounted device that uses 3D near-eye displays and positional tracking to provide a virtual reality environment for the user. VR headsets are widely used with Virtual reali ...

to reduce the rendering workload by greatly reducing the image quality in the

peripheral vision Peripheral vision, or ''indirect vision'', is vision as it occurs outside the point of fixation, i.e. away from the center of gaze or, when viewed at large angles, in (or out of) the "corner of one's eye". The vast majority of the area in the ...

(outside of the zone gazed by the fovea). At the CES 2016, SensoMotoric Instruments (SMI) demoed a new 250 Hz eye tracking system and a working foveated rendering solution. It resulted from a partnership with camera sensor manufacturer

Omnivision OmniVision Technologies Inc. is an American subsidiary of Chinese semiconductor device and mixed-signal integrated circuit design house Will Semiconductor. The company designs and develops digital imaging products for use in mobile phones, lap ...

who provided the camera hardware for the new system. The

Apple Vision Pro The Apple Vision Pro is a mixed reality, mixed-reality headset developed by Apple Inc., Apple. It was announced on June 5, 2023, at Apple's Worldwide Developers Conference (WWDC) and was released first in the US, then in global territories thr ...

mixed reality headset features dynamic foveated rendering provided by its

visionOS visionOS is a mixed reality operating system derived primarily from iPadOS and its core frameworks (including UIKit, SwiftUI, ARKit and RealityKit), and MR-specific frameworks for foveated rendering and real-time interaction. It was develope ...

operating system.

Quality assessment

Foveated imaging may be useful in providing a subjective image quality measure.Z. Wang, A. C. Bovik, L. Lu and J. Kouloheris, "Foveated wavelet image quality index," SPIE's 46th Annual Meeting, Proc. SPIE, Application of digital image processing XXIV, vol. 4472, July-Aug. 2001. Traditional image quality measures, such as

peak signal-to-noise ratio Peak signal-to-noise ratio (PSNR) is an engineering term for the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. Because many signals have a very wide dynamic ...

, are typically performed on fixed resolution images and do not take into account some aspects of the human visual system, like the change in spatial resolution across the retina. A foveated quality index may therefore more accurately determine image quality as perceived by humans.

Image database retrieval

In databases that contain very high resolution images, such as a

satellite image Satellite images (also Earth observation imagery, spaceborne photography, or simply satellite photo) are images of Earth collected by imaging satellites operated by governments and businesses around the world. Satellite imaging companies sell i ...

database, it may be desirable to interactively retrieve images in order to reduce retrieval time. Foveated imaging allows one to scan low resolution images and retrieve only high resolution portions as they are needed. This is sometimes called progressive transmission.

Example images

Image:Foveated_tombstones.png, Image with foveated luminance channel Image:FoveatedLandscape.png, Foveated image with fixation selected using an entropy minimization algorithm image:Texas_state_cemetery_foveated2.png, 18:1 compression. Foveated image with fixation point on tombstone.

References

{{Reflist, 2

External links

* http://live.ece.utexas.edu * http://svi.cps.utexas.edu * http://webvision.med.utah.edu/ Computer vision, Machine Vision, Robotics