Automatic image annotation (also known as automatic image tagging or linguistic indexing) is the process by which a computer system automatically assigns

metadata Metadata (or metainformation) is "data that provides information about other data", but not the content of the data itself, such as the text of a message or the image itself. There are many distinct types of metadata, including: * Descriptive ...

in the form of captioning or keywords to a

digital image A digital image is an image composed of picture elements, also known as pixels, each with '' finite'', '' discrete quantities'' of numeric representation for its intensity or gray level that is an output from its two-dimensional functions f ...

. This application of

computer vision Computer vision tasks include methods for image sensor, acquiring, Image processing, processing, Image analysis, analyzing, and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical ...

techniques is used in

image retrieval An image retrieval system is a computer system used for browsing, searching and retrieving images from a large database of digital images. Most traditional and common methods of image retrieval utilize some method of adding metadata such as captio ...

systems to organize and locate images of interest from a

database In computing, a database is an organized collection of data or a type of data store based on the use of a database management system (DBMS), the software that interacts with end users, applications, and the database itself to capture and a ...

. This method can be regarded as a type of multi-class

image classification Computer vision tasks include methods for acquiring, processing, analyzing, and understanding digital images, and extraction of high-dimensional data from the real world in order to produce numerical or symbolic information, e.g. in the form o ...

with a very large number of classes - as large as the vocabulary size. Typically,

image analysis Image analysis or imagery analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques. Image analysis tasks can be as simple as reading barcode, bar coded tags or a ...

in the form of extracted

feature vector In machine learning and pattern recognition, a feature is an individual measurable property or characteristic of a data set. Choosing informative, discriminating, and independent features is crucial to produce effective algorithms for pattern re ...

s and the training annotation words are used by

machine learning Machine learning (ML) is a field of study in artificial intelligence concerned with the development and study of Computational statistics, statistical algorithms that can learn from data and generalise to unseen data, and thus perform Task ( ...

techniques to attempt to automatically apply annotations to new images. The first methods learned the correlations between image features and training annotations. Subsequently, techniques were developed using

machine translation Machine translation is use of computational techniques to translate text or speech from one language to another, including the contextual, idiomatic and pragmatic nuances of both languages. Early approaches were mostly rule-based or statisti ...

to to attempt to translate the textual vocabulary into the 'visual vocabulary,' represented by clustered regions known as ''blobs.'' Subsequent work has included classification approaches, relevance models, and other related methods. The advantages of automatic image annotation versus

content-based image retrieval Content-based image retrieval, also known as query by image content ( QBIC) and content-based visual information retrieval (CBVIR), is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching ...

(CBIR) are that queries can be more naturally specified by the user. At present, Content-Based Image Retrieval (CBIR) generally requires users to search by image concepts such as color and

texture Texture may refer to: Science and technology * Image texture, the spatial arrangement of color or intensities in an image * Surface texture, the smoothness, roughness, or bumpiness of the surface of an object * Texture (roads), road surface c ...

or by finding example queries. However, certain image features in example images may override the concept that the user is truly focusing on. Traditional methods of image retrieval, such as those used by libraries, have relied on manually annotated images, which is expensive and time-consuming, especially given the large and constantly growing image databases in existence.

References

* * *

* Word co-occurrence model : * Annotation as machine translation : * Statistical models : : * Automatic linguistic indexing of pictures : : * Hierarchical Aspect Cluster Model : * Latent Dirichlet Allocation model : * Supervised multiclass labeling : * Texture similarity : * Support Vector Machines : * Ensemble of Decision Trees and Random Subwindows : * Maximum Entropy : * Relevance models : * Relevance models using continuous probability density functions : * Coherent Language Model : * Inference networks : * Multiple Bernoulli distribution : * Multiple design alternatives : * Image captioning : * Natural scene annotation : * Relevant low-level global filters : * Global image features and nonparametric density estimation : * Video semantics : : * Image Annotation Refinement : : : : : * Automatic Image Annotation by Ensemble of Visual Descriptors : * A New Baseline for Image Annotation : Simultaneous Image Classification and Annotation : * TagProp: Discriminative Metric Learning in Nearest Neighbor Models for Image Auto-Annotation : * Image Annotation Using Metric Learning in Semantic Neighbourhoods : * Automatic Image Annotation Using Deep Learning Representations : *Holistic Image Annotation using Salient Regions and Background Image Information : * Medical Image Annotation using bayesian networks and active learning : {{Computer vision Applications of artificial intelligence Applications of computer vision

See also

References

Further reading