''Reference frames'' are frames of a

compressed video In information theory, data compression, source coding, or bit-rate reduction is the process of encoding information using fewer bits than the original representation. Any particular compression is either Lossy compression, lossy or Lossless com ...

that are used to define future frames. As such, they are only used in inter-frame compression techniques. In older video encoding standards, such as

MPEG-2 MPEG-2 (a.k.a. H.222/H.262 as was defined by the ITU) is a standard for "the generic coding of moving pictures and associated audio information". It describes a combination of lossy video compression and lossy audio data compression methods, w ...

, only one reference frame – the previous frame – was used for P-frames. Two reference frames (one past and one future) were used for

B-frame In the field of video compression a video frame is compressed using different algorithms with different advantages and disadvantages, centered mainly around amount of data compression. These different algorithms for video frames are called pi ...

Multiple reference frames

Some modern video encoding standards, such as H.264/AVC, allow the use of multiple reference frames. This allows the video encoder to choose among more than one previously decoded frame on which to base each

macroblock The macroblock is a processing unit in image and video compression formats based on linear block transforms, typically the discrete cosine transform (DCT). A macroblock typically consists of 16×16 samples, and is further subdivided into transform ...

in the next frame. While the best frame for this purpose is usually the previous frame, the extra reference frames can improve compression efficiency and/or

video quality Video quality is a characteristic of a video passed through a video transmission or processing system that describes perceived video degradation (typically, compared to the original video). Video processing systems may introduce some amount of dist ...

. Note that different reference frames can be chosen for different macroblocks in the same frame. The maximum number of concurrent reference frames supported by H.264 is 16. Different reference frames can be chosen for each 8x8 partition of a macroblock. Another video format that supports multiple reference frames is

Snow Snow comprises individual ice crystals that grow while suspended in the atmosphere An atmosphere () is a layer of gas or layers of gases that envelop a planet, and is held in place by the gravity of the planetary body. A planet ...

, which can handle up to eight. The

Theora Theora is a free lossy video compression format. It is developed by the Xiph.Org Foundation and distributed without licensing fees alongside their other free and open media projects, including the Vorbis audio format and the Ogg container ...

codec provides a limited form of multiple reference frames, allowing references to both the preceding frame and the most recent

intra frame Intra-frame coding is a data compression technique used within a video frame, enabling smaller file sizes and lower bitrates, with little or no loss in quality. Since neighboring pixels within an image are often very similar, rather than storing ...

Disadvantages

Encoding

Multiple reference frames can considerably increase encoding time because many of the decisions, such as

motion estimation Motion estimation is the process of determining ''motion vectors'' that describe the transformation from one 2D image to another; usually from adjacent frames in a video sequence. It is an ill-posed problem as the motion is in three dimensions b ...

, that are ordinarily carried out only on one reference frame have to be repeated on all of the reference frames.

Heuristics A heuristic (; ), or heuristic technique, is any approach to problem solving or self-discovery that employs a practical method that is not guaranteed to be optimal, perfect, or rational, but is nevertheless sufficient for reaching an immediate, ...

can be used to reduce this speed cost at the cost of quality. Very high numbers of reference frames are rarely useful in terms of quality for

live-action Live action (or live-action) is a form of cinematography or videography that uses photography instead of animation. Some works combine live-action with animation to create a live-action animated film. Live-action is used to define film, video ga ...

material because frames from farther back in time generally have less and less correlation with the current frame. This is not as true for

animated Animation is a method by which still figures are manipulated to appear as moving images. In traditional animation, images are drawn or painted by hand on transparent celluloid sheets to be photographed and exhibited on film. Today, most ani ...

sources, where repetitive motion can make high numbers of reference frames more useful.

Decoding

When decoding, reference frames must be stored in memory until they are no longer needed for further decoding. This can considerably raise the

memory Memory is the faculty of the mind by which data or information is encoded, stored, and retrieved when needed. It is the retention of information over time for the purpose of influencing future action. If past events could not be remembered ...

usage of the decoder for videos with large numbers of reference frames. The use of several reference frames also decreases

locality of reference In computer science, locality of reference, also known as the principle of locality, is the tendency of a processor to access the same set of memory locations repetitively over a short period of time. There are two basic types of reference localit ...

, which might cause a speed impact. Stand-alone players that can play AVC/MKV files from recorded DVDs or USB sticks may not be able to handle the maximum possible 16 reference frames.

References

*{{cite book , last = E. G. Richardson , first = Iain , title = H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia , publisher = John Wiley & Sons Ltd. , date = 2003 , location = Chichester

Multiple reference frames

Disadvantages

Encoding

Decoding

References

See also