Automatic image annotation (also known as automatic image tagging or linguistic indexing) is the process by which a computer system automatically assigns metadata in the form of captioning or keywords to a

digital image A digital image is an image composed of picture elements, also known as ''pixels'', each with '' finite'', '' discrete quantities'' of numeric representation for its intensity or gray level that is an output from its two-dimensional functions f ...

. This application of

computer vision Computer vision is an Interdisciplinarity, interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate t ...

techniques is used in

image retrieval An image retrieval system is a computer system used for browsing, searching and retrieving images from a large database of digital images. Most traditional and common methods of image retrieval utilize some method of adding metadata such as capti ...

systems to organize and locate images of interest from a

database In computing, a database is an organized collection of data stored and accessed electronically. Small databases can be stored on a file system, while large databases are hosted on computer clusters or cloud storage. The design of databases spa ...

. This method can be regarded as a type of multi-class

image classification Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human ...

with a very large number of classes - as large as the vocabulary size. Typically,

image analysis Image analysis or imagery analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques. Image analysis tasks can be as simple as reading bar coded tags or as sophi ...

in the form of extracted

feature vector In machine learning and pattern recognition, a feature is an individual measurable property or characteristic of a phenomenon. Choosing informative, discriminating and independent features is a crucial element of effective algorithms in pattern ...

s and the training annotation words are used by

machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine ...

techniques to attempt to automatically apply annotations to new images. The first methods learned the correlations between image features and training annotations, then techniques were developed using

machine translation Machine translation, sometimes referred to by the abbreviation MT (not to be confused with computer-aided translation, machine-aided human translation or interactive translation), is a sub-field of computational linguistics that investigates t ...

to try to translate the textual vocabulary with the 'visual vocabulary', or clustered regions known as ''blobs''. Work following these efforts have included classification approaches, relevance models and so on. The advantages of automatic image annotation versus content-based image retrieval (CBIR) are that queries can be more naturally specified by the user. CBIR generally (at present) requires users to search by image concepts such as color and

texture Texture may refer to: Science and technology * Surface texture, the texture means smoothness, roughness, or bumpiness of the surface of an object * Texture (roads), road surface characteristics with waves shorter than road roughness * Texture ...

, or finding example queries. Certain image features in example images may override the concept that the user is really focusing on. The traditional methods of image retrieval such as those used by libraries have relied on manually annotated images, which is expensive and time-consuming, especially given the large and constantly growing image databases in existence.

References

* * *

* Word co-occurrence model : * Annotation as machine translation : * Statistical models : : * Automatic linguistic indexing of pictures : : * Hierarchical Aspect Cluster Model : * Latent Dirichlet Allocation model : * Supervised multiclass labeling : * Texture similarity : * Support Vector Machines : * Ensemble of Decision Trees and Random Subwindows : * Maximum Entropy : * Relevance models : * Relevance models using continuous probability density functions : * Coherent Language Model : * Inference networks : * Multiple Bernoulli distribution : * Multiple design alternatives : * Image captioning : * Natural scene annotation : * Relevant low-level global filters : * Global image features and nonparametric density estimation : * Video semantics : : * Image Annotation Refinement : : : : : * Automatic Image Annotation by Ensemble of Visual Descriptors : * A New Baseline for Image Annotation : Simultaneous Image Classification and Annotation : * TagProp: Discriminative Metric Learning in Nearest Neighbor Models for Image Auto-Annotation : * Image Annotation Using Metric Learning in Semantic Neighbourhoods : * Automatic Image Annotation Using Deep Learning Representations : * Medical Image Annotation using bayesian networks and active learning :{{cite conference , author1 = N. B. Marvasti, author2= E. Yörük and B. Acar, name-list-style=amp, url=https://www.researchgate.net/publication/320935564, title = Computer-Aided Medical Image Annotation: Preliminary Results With Liver Lesions in CT, book-title= IEEE Journal of Biomedical and Health Informatics , year = 2018 Applications of artificial intelligence Applications of computer vision

See also

References

Further reading