HOME

TheInfoList



OR:

Sonification is the use of non-speech
audio Audio most commonly refers to sound, as it is transmitted in signal form. It may also refer to: Sound *Audio signal, an electrical representation of sound *Audio frequency, a frequency in the audio spectrum *Digital audio, representation of sound ...
to convey
information Information is an Abstraction, abstract concept that refers to something which has the power Communication, to inform. At the most fundamental level, it pertains to the Interpretation (philosophy), interpretation (perhaps Interpretation (log ...
or perceptualize data. Auditory perception has advantages in temporal, spatial, amplitude, and frequency resolution that open possibilities as an alternative or complement to visualization techniques. For example, the rate of clicking of a
Geiger counter A Geiger counter (, ; also known as a Geiger–Müller counter or G-M counter) is an electronic instrument for detecting and measuring ionizing radiation with the use of a Geiger–Müller tube. It is widely used in applications such as radiat ...
conveys the level of radiation in the immediate vicinity of the device. Though many experiments with
data sonification Data sonification is the presentation of data as sound using sonification. It is the auditory equivalent of the more established practice of data visualization. Process The usual process for data sonification is directing digital media of a dat ...
have been explored in forums such as the International Community for Auditory Display (ICAD), sonification faces many challenges to widespread use for presenting and analyzing data. For example, studies show it is difficult, but essential, to provide adequate context for interpreting sonifications of data. Many sonification attempts are coded from scratch due to the lack of flexible tooling for sonification research and data exploration.


History

The
Geiger counter A Geiger counter (, ; also known as a Geiger–Müller counter or G-M counter) is an electronic instrument for detecting and measuring ionizing radiation with the use of a Geiger–Müller tube. It is widely used in applications such as radiat ...
, invented in 1908, is one of the earliest and most successful applications of sonification. A Geiger counter has a tube of low-pressure gas; each particle detected produces a pulse of current when it ionizes the gas, producing an audio click. The original version was only capable of detecting alpha particles. In 1928, Geiger and Walther Müller (a PhD student of Geiger) improved the counter so that it could detect more types of ionizing radiation. In 1913, Dr. Edmund Fournier d'Albe of
University of Birmingham The University of Birmingham (informally Birmingham University) is a Public university, public research university in Birmingham, England. It received its royal charter in 1900 as a successor to Queen's College, Birmingham (founded in 1825 as ...
invented the optophone, which used
selenium Selenium is a chemical element; it has symbol (chemistry), symbol Se and atomic number 34. It has various physical appearances, including a brick-red powder, a vitreous black solid, and a grey metallic-looking form. It seldom occurs in this elem ...
photosensors to detect black print and convert it into an audible output. A blind reader could hold a book up to the device and hold an apparatus to the area she wanted to read. The optophone played a set group of notes: . Each note corresponded with a position on the optophone's reading area, and that note was silenced if black ink was sensed. Thus, the missing notes indicated the positions where black ink was on the page and could be used to read. Pollack and Ficks published the first perceptual experiments on the transmission of information via auditory display in 1954. They experimented with combining sound dimensions such as timing, frequency, loudness, duration, and spatialization and found that they could get subjects to register changes in multiple dimensions at once. These experiments did not get into much more detail than that, since each dimension had only two possible values. In 1970,
Nonesuch Records Nonesuch Records is an American record company and label owned by Warner Music Group, distributed by Warner Records (formerly Warner Bros. Records), and based in New York City. Founded by Jac Holzman in 1964 as a budget classical label, Nonesuch ...
released a new electronic music composition by the American composer Charles Dodge, "The Earth's Magnetic Field." It was produced at the
Columbia-Princeton Electronic Music Center The Computer Music Center (CMC) at Columbia University is the oldest center for electronic and computer music research in the United States. It was founded in the 1950s as the Columbia-Princeton Electronic Music Center. Location The CMC is h ...
. As the title suggests, the composition's electronic sounds were synthesized from data from the earth's magnetic field. As such, it may well be the first sonification of scientific data for artistic, rather than scientific, purposes. John M. Chambers,
Max Mathews Max Vernon Mathews (November 13, 1926 – April 21, 2011) was an American pioneer of computer music. Biography Max Vernon Mathews was born in Columbus, Nebraska, to two science schoolteachers. His father in particular taught physics, chemistry ...
, and F.R. Moore at
Bell Laboratories Nokia Bell Labs, commonly referred to as ''Bell Labs'', is an American industrial research and development company owned by Finnish technology company Nokia. With headquarters located in Murray Hill, New Jersey, the company operates several lab ...
did the earliest work on auditory graphing in their "Auditory Data Inspection" technical memorandum in 1974. They augmented a
scatterplot A scatter plot, also called a scatterplot, scatter graph, scatter chart, scattergram, or scatter diagram, is a type of plot or mathematical diagram using Cartesian coordinates to display values for typically two variables for a set of dat ...
using sounds that varied along frequency, spectral content, and amplitude modulation dimensions to use in classification. They did not do any formal assessment of the effectiveness of these experiments. In 1976, philosopher of technology, Don Ihde, wrote, "Just as science seems to produce an infinite set of visual images for virtually all of its phenomena--atoms to galaxies are familiar to us from coffee table books to science magazines; so 'musics,' too, could be produced from the same data that produces visualizations." This appears to be one of the earliest references to sonification as a creative practice. In early 1982 Sara Bly of the University of California, Davis, released two publications - with examples - of her work on the use of computer-generated sound to present data. At the time, the field of scientific visualization was gaining momentum. Among other things, her studies and the accompanying examples compared the properties between visual and aural presentation, demonstrating that "Sound offers and enhancement and an alternative to graphic tools." Her work provides early experiment-based data to help inform matching appropriate data representation to type and purpose. Also in the 1980s,
pulse oximeter Pulse oximetry is a noninvasive method for monitoring blood oxygen saturation. Peripheral oxygen saturation (SpO2) readings are typically within 2% accuracy (within 4% accuracy in 95% of cases) of the more accurate (and invasive) reading of art ...
s came into widespread use. Pulse oximeters can sonify oxygen concentration of blood by emitting higher pitches for higher concentrations. However, in practice this particular feature of pulse oximeters may not be widely utilized by medical professionals because of the risk of too many audio stimuli in medical environments. In 1990, the
National Center for Supercomputing Applications The National Center for Supercomputing Applications (NCSA) is a unit of the University of Illinois Urbana-Champaign, and provides high-performance computing resources to researchers in the United States. NCSA is currently led by Professor Bill ...
began generating scientific data sonifications and visualizations from the same source data and a paper describing this work was presented at the June 1991
SPIE SPIE (formerly the Society of Photographic Instrumentation Engineers, later the Society of Photo-Optical Instrumentation Engineers) is an international not-for-profit professional society for optics and photonics technology, founded in 1955. It ...
Conference on Extracting Meaning from Complex Data. Included in the supporting information for the paper was a video, winner of the 1991 Nicograph Multimedia Grand Prize, comprising several data visualizations paired with their corresponding data sonifications. In 1992, the International Community for Auditory Display (ICAD) was founded by
Gregory Kramer Gregory Paul Kramer (born 14 October 1952, in Los Angeles, California), is an American composer, researcher, inventor, meditation teacher and author. In 1975 he co-founded Electronic Musicmobile, a pioneer synthesizer ensemble later renamed Electro ...
as a forum for research on auditory display which includes data sonification. ICAD has since become a home for researchers from many different disciplines interested in the use of sound to convey information through its conference and peer-reviewed proceedings. In 2020, the composer
Simon Gray Simon James Holliday Gray (21 October 1936 – 7 August 2008) was an English playwright and memoirist who also had a career as a Academia, university lecturer in English literature at Queen Mary, University of London, for 20 years. While teach ...
recording as The Winterval Conspiracy produced a sonification of the
SARS-Cov-2 Severe acute respiratory syndrome coronavirus 2 (SARS‑CoV‑2) is a strain of coronavirus that causes COVID-19, the respiratory illness responsible for the COVID-19 pandemic. The virus previously had the Novel coronavirus, provisional nam ...
virus responsible for the
Covid-19 pandemic The COVID-19 pandemic (also known as the coronavirus pandemic and COVID pandemic), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), began with an disease outbreak, outbreak of COVID-19 in Wuhan, China, in December ...
in the form of a musical
fugue In classical music, a fugue (, from Latin ''fuga'', meaning "flight" or "escape""Fugue, ''n''." ''The Concise Oxford English Dictionary'', eleventh edition, revised, ed. Catherine Soanes and Angus Stevenson (Oxford and New York: Oxford Universit ...
. In May 2022, NASA reported the sonification (converting astronomical data associated with pressure waves into
sound In physics, sound is a vibration that propagates as an acoustic wave through a transmission medium such as a gas, liquid or solid. In human physiology and psychology, sound is the ''reception'' of such waves and their ''perception'' by the br ...
) of the black hole at the center of the Perseus galaxy cluster. In 2024, Adhyâropa Records released ''The Volcano Listening Project'' by Leif Karlstrom, which merges geophysics research and computer music synthesis with acoustic instrumental and vocal performances by Billy Contreras,
Todd Sickafoose Todd Sickafoose is an American jazz and rock musician, composer, and producer/engineer from San Francisco, California. He is best known for playing acoustic bass and keyboards with Ani DiFranco but has also led his own group, Todd Sickafoose's ...
, and other acoustic musicians.


Some existing applications and projects

* Auditory thermometer * Biodiversity * Clocks, e.g., with an audible tick every second, and with special chimes every 15 minutes * Cluster analysis of high dimensional data * Cockpit auditory displays *
DNA Deoxyribonucleic acid (; DNA) is a polymer composed of two polynucleotide chains that coil around each other to form a double helix. The polymer carries genetic instructions for the development, functioning, growth and reproduction of al ...
* Financial market monitoring *
Geiger counter A Geiger counter (, ; also known as a Geiger–Müller counter or G-M counter) is an electronic instrument for detecting and measuring ionizing radiation with the use of a Geiger–Müller tube. It is widely used in applications such as radiat ...
*
Gravitational wave Gravitational waves are oscillations of the gravitational field that Wave propagation, travel through space at the speed of light; they are generated by the relative motion of gravity, gravitating masses. They were proposed by Oliver Heaviside i ...
s at
LIGO The Laser Interferometer Gravitational-Wave Observatory (LIGO) is a large-scale physics experiment and observatory designed to detect cosmic gravitational waves and to develop gravitational-wave observations as an astronomical tool. Prior to LIG ...
* Image sonification for the visually impaired * Interactive sonification * Medical and surgical auditory displays * Multimodal (combined sense) displays to minimize visual overload and fatigue * Navigation * Ocean science * Protein folding dynamics * Pulse oximetry in operating rooms and intensive care *
Sonar Sonar (sound navigation and ranging or sonic navigation and ranging) is a technique that uses sound propagation (usually underwater, as in submarine navigation) to navigate, measure distances ( ranging), communicate with or detect objects o ...
*
Space physics Space physics, also known as space plasma physics, is the study of naturally occurring plasmas within Earth's upper atmosphere and the rest of the Solar System. It includes the topics of aeronomy, aurorae, planetary ionospheres and magnetospheres, ...
* Storm and weather sonification * Synaesthesia * Tiltification based on psychoacoustic sonification *
Variometer In aviation, a variometer – also known as a rate of climb and descent indicator (RCDI), rate-of-climb indicator, vertical speed indicator (VSI), or vertical velocity indicator (VVI) – is one of the flight instruments in an aircraft used to in ...
(rate-of-climb indicator) in a
glider (sailplane) A glider or sailplane is a type of glider aircraft used in the leisure activity and sport of gliding (also called soaring). This unpowered aircraft can use naturally occurring currents of rising air in the atmosphere to gain altitude. Sailplan ...
beeps with a variable pitch corresponding to the meter reading * Vehicle speed alarm *
Volcanic activity Volcanism, vulcanism, volcanicity, or volcanic activity is the phenomenon where solids, liquids, gases, and their mixtures erupt to the surface of a solid-surface astronomical body such as a planet or a moon. It is caused by the presence of a he ...
detection * Video sonification


Sonification techniques

Many different components can be altered to change the user's perception of the sound, and in turn, their perception of the underlying information being portrayed. Often, an increase or decrease in some level in this information is indicated by an increase or decrease in pitch,
amplitude The amplitude of a periodic variable is a measure of its change in a single period (such as time or spatial period). The amplitude of a non-periodic signal is its magnitude compared with a reference value. There are various definitions of am ...
or
tempo In musical terminology, tempo (Italian for 'time'; plural 'tempos', or from the Italian plural), measured in beats per minute, is the speed or pace of a given musical composition, composition, and is often also an indication of the composition ...
, but could also be indicated by varying other less commonly used components. For example, a stock market price could be portrayed by rising pitch as the stock price rose, and lowering pitch as it fell. To allow the user to determine that more than one stock was being portrayed, different timbres or brightnesses might be used for the different stocks, or they may be played to the user from different points in space, for example, through different sides of their headphones. Many studies have been undertaken to try to find the best techniques for various types of information to be presented, and as yet, no conclusive set of techniques to be used has been formulated. As the area of sonification is still considered to be in its infancy, current studies are working towards determining the best set of sound components to vary in different situations. Several different techniques for auditory rendering of data can be categorized: * Acoustic sonification * Audification * Model-based sonification * Parameter mapping * Stream-based sonification An alternative approach to traditional sonification is "sonification by replacement", for example Pulsed Melodic Affective Processing (PMAP). In PMAP rather than sonifying a data stream, the computational protocol is musical data itself, for example MIDI. The data stream represents a non-musical state: in PMAP an affective state. Calculations can then be done directly on the musical data, and the results can be listened to with the minimum of translation.


See also

* * *


References


External links

{{Commons category, Sonification
International Community for Auditory Display


(1997) provides an introduction to the status of the field and current research agendas.
The Sonification Handbook
an Open Access book that gives a comprehensive introductory presentation of the key research areas in sonification and auditory display.
Using Sound to Extract Meaning from Complex Data
C. Scaletti and A. Craig, 1991
Supporting video

Auditory Information Design
PhD Thesis by Stephen Barrass 1998, User Centred Approach to Designing Sonifications.
Mozzi : interactive sensor sonification on Arduino microprocessor.

Preliminary report on design rationale, syntax, and semantics of LSL: A specification language for program auralization, D. Boardman and AP Mathur, 1993.

A specification language for program auralization, D. Boardman, V. Khandelwal, and AP Mathur, 1994.



SonEnvir general sonification environment

Sonification.de
provides information about Sonification and Auditory Display, links to interesting event and related projects
Sonification for Exploratory Data Analysis
PhD Thesis by Thomas Hermann 2002, developing Model Based Sonification.
Sonification of Mobile and Wireless Communications

Interactive Sonification
a hub to news and upcoming events in the field of interactive sonification



— an open source sonification framework which makes possible to hear how any existing
Java Java is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea (a part of Pacific Ocean) to the north. With a population of 156.9 million people (including Madura) in mid 2024, proje ...
program "sounds like", by assigning instruments and pitches to code statements (if, for, etc.) and playing them as they are executed at runtime. In this way the flowing of execution is played as a flow of music and its rhythm changes depending on user interaction.
LYCAY
a Java library for sonification of Java source code * Sonification of a
Cantor set In mathematics, the Cantor set is a set of points lying on a single line segment that has a number of unintuitive properties. It was discovered in 1874 by Henry John Stephen Smith and mentioned by German mathematician Georg Cantor in 1883. Throu ...
br>

Sonification Sandbox v.3.0
a Java program to convert datasets to sounds, GT Sonification Lab, School of Psychology,
Georgia Institute of Technology The Georgia Institute of Technology (commonly referred to as Georgia Tech, GT, and simply Tech or the Institute) is a public university, public research university and Institute of technology (United States), institute of technology in Atlanta, ...
.
Program Sonification using Java
an online chapter (with code) explaining how to implement sonification using speech synthesis, MIDI note generation, and audio clips.
Biosonification
a collection of generative plant and fungi sonifications.

Live Sonification of Ocean Swell Multimodal interaction Display technology Auditory displays Sound Acoustics