Multimedia search enables information

search Searching may refer to: Music * "Searchin', Searchin", a 1957 song originally performed by The Coasters * Searching (China Black song), "Searching" (China Black song), a 1991 song by China Black * Searchin' (CeCe Peniston song), "Searchin" (C ...

using queries in multiple data types including text and other

multimedia Multimedia is a form of communication that uses a combination of different content forms, such as Text (literary theory), writing, Sound, audio, images, animations, or video, into a single presentation. T ...

formats. Multimedia search can be implemented through multimodal search interfaces, i.e., interfaces that allow to submit search queries not only as textual requests, but also through other media. We can distinguish two methodologies in multimedia search: *Metadata search: the search is made on the layers of

metadata Metadata (or metainformation) is "data that provides information about other data", but not the content of the data itself, such as the text of a message or the image itself. There are many distinct types of metadata, including: * Descriptive ...

. *

Query by example Query by Example (QBE) is a database query language for relational databases. History Query by Example was devised by Moshé M. Zloof at IBM Research during the mid-1970s, in parallel to the development of SQL, and influenced by the work on rela ...

: The interaction consists in submitting a piece of information (e.g., a video, an image, or a piece of audio) for the purpose of finding similar multimedia items.

Metadata search

Search is made using the layers in metadata which contain information of the content of a multimedia file. Metadata search is easier, faster and effective because instead of working with complex material, such as an audio, a video or an image, it searches using text. There are three processes which should be done in this method: * Summarization of media content (

feature extraction Feature may refer to: Computing * Feature recognition, could be a hole, pocket, or notch * Feature (computer vision), could be an edge, corner or blob * Feature (machine learning), in statistics: individual measurable properties of the phenome ...

). The result of feature extraction is a description. * Filtering of media descriptions (for example, elimination of Redundancy) * Categorization of media descriptions into classes.

Query by example

query by example Query by Example (QBE) is a database query language for relational databases. History Query by Example was devised by Moshé M. Zloof at IBM Research during the mid-1970s, in parallel to the development of SQL, and influenced by the work on rela ...

, the element used to search is a

content (image, audio, video). In other words, the query is a media. Often, it's used audiovisual indexing. It will be necessary to choose the criteria we are going to use for creating metadata. The process of search can be divided in three parts: *Generate descriptors for the media which we are going to use as query and the descriptors for the media in our

database In computing, a database is an organized collection of data or a type of data store based on the use of a database management system (DBMS), the software that interacts with end users, applications, and the database itself to capture and a ...

. *Compare descriptors of the query and our database’s media. *List the media sorted by maximum coincidence.

Multimedia search engine

There are two big search families, in function of the content: * Visual search engine * Audio search engine

Visual search engine

Inside this family we can distinguish two topics: image search and video search * Image search: Although usually it's used simple metadata search, increasingly is being used indexing methods for making the results of users queries more accurate using

. For example, QR codes. * Video search: Videos can be searched for simple metadata or by complex metadata generated by indexing. The audio contained in the videos is usually scanned by audio search engines.

Audio search engine

There are different methods of audio searching: * Voice search engine: Allows the user to search using speech instead of text. It uses algorithms of

speech recognition Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers. It is also ...

. An example of this technology is Google Voice Search. *Music search engine: Although most of applications which searches music works on simple metadata (artist, name of track, album…) . There are some programs of music recognition, for example Shazam"Shazam Launches Resonate TV Sales Platform"
''Billboard''. Retrieved 2024-04-17. or SoundHound.

References