Josef "Sepp" Hochreiter (born 14 February 1967) is a German

computer scientist A computer scientist is a person who is trained in the academic study of computer science. Computer scientists typically work on the theoretical side of computation, as opposed to the hardware side on which computer engineers mainly focus ( ...

. Since 2018 he has led the Institute for Machine Learning at the Johannes Kepler University of

Linz Linz ( , ; cs, Linec) is the capital of Upper Austria and third-largest city in Austria. In the north of the country, it is on the Danube south of the Czech border. In 2018, the population was 204,846. In 2009, it was a European Capital ...

after having led the Institute of Bioinformatics from 2006 to 2018. In 2017 he became the head of the Linz Institute of Technology (LIT) AI Lab. Hochreiter is also a founding director of the Institute of Advanced Research in Artificial Intelligence (IARAI). Previously, he was at the

Technical University An institute of technology (also referred to as: technological university, technical university, university of technology, technological educational institute, technical college, polytechnic university or just polytechnic) is an institution of te ...

Berlin Berlin is Capital of Germany, the capital and largest city of Germany, both by area and List of cities in Germany by population, by population. Its more than 3.85 million inhabitants make it the European Union's List of cities in the European U ...

, at the

University of Colorado The University of Colorado (CU) is a system of public universities in Colorado. It consists of four institutions: University of Colorado Boulder, University of Colorado Colorado Springs, University of Colorado Denver, and the University o ...

Boulder In geology, a boulder (or rarely bowlder) is a rock fragment with size greater than in diameter. Smaller pieces are called cobbles and pebbles. While a boulder may be small enough to move or roll manually, others are extremely massive. In ...

, and at the

Munich Munich ( ; german: München ; bar, Minga ) is the capital and most populous city of the German state of Bavaria. With a population of 1,558,395 inhabitants as of 31 July 2020, it is the third-largest city in Germany, after Berlin and Ha ...

. He is a chair of the Critical Assessment of Massive Data Analysis (CAMDA) conference. Hochreiter has made contributions in the fields of

machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine ...

, deep learning and

bioinformatics Bioinformatics () is an interdisciplinary field that develops methods and software tools for understanding biological data, in particular when the data sets are large and complex. As an interdisciplinary field of science, bioinformatics combin ...

, most notably the development of the

long short-term memory Long short-term memory (LSTM) is an artificial neural network used in the fields of artificial intelligence and deep learning. Unlike standard feedforward neural networks, LSTM has feedback connections. Such a recurrent neural network (RNN) ca ...

(LSTM) neural network architecture, but also in

meta-learning Meta-learning is a branch of metacognition concerned with learning about one's own learning and learning processes. The term comes from the meta prefix's modern meaning of an abstract recursion, or "X about X", similar to its use in metaknowled ...

reinforcement learning Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine ...

and biclustering with application to

data.

Scientific career

Long short-term memory (LSTM)

Hochreiter developed the

(LSTM) neural network architecture in his diploma thesis in 1991 leading to the main publication in 1997. LSTM overcomes the problem that recurrent neural networks (RNNs) forget information over time ( vanishing or exploding gradient). In 2007, Hochreiter and others successfully applied LSTM with an optimized architecture to very fast protein homology detection without requiring a

sequence alignment In bioinformatics, a sequence alignment is a way of arranging the sequences of DNA, RNA, or protein to identify regions of similarity that may be a consequence of functional, structural, or evolutionary relationships between the sequences. Ali ...

. LSTM networks have also been also used in

Google Voice Google Voice is a telephone service that provides a U.S. phone number to Google Account customers in the U.S. and Google Workspace (G Suite by October 2020) customers in Canada, Denmark, France, the Netherlands, Portugal, Spain, Sweden, Switz ...

for transcription and search, and in the Google Allo chat app for generating response suggestion with low latency.

Other machine learning contributions

Beyond LSTM, Hochreiter has developed "Flat Minimum Search" to increase the

generalization A generalization is a form of abstraction whereby common properties of specific instances are formulated as general concepts or claims. Generalizations posit the existence of a domain or set of elements, as well as one or more common character ...

of neural networks and introduced rectified factor networks (RFNs) for sparse coding which have been applied in bioinformatics and genetics. Hochreiter introduced modern

Hopfield networks A Hopfield network (or Ising model of a neural network or Ising–Lenz–Little model) is a form of recurrent artificial neural network and a type of spin glass system popularised by John Hopfield in 1982 as described earlier by Little in 1974 ...

with continuous states and applied them to the task of immune repertoire classification. Hochreiter worked with Jürgen Schmidhuber in the field of

on actor-critic systems that learn by "backpropagation through a model". Hochreiter has been involved in the development of

factor analysis Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in six observed ...

methods with application to bioinformatics, including FABIA for biclustering, HapFABIA for detecting short segments of identity by descent and FARMS for preprocessing and summarizing high-density

oligonucleotide Oligonucleotides are short DNA or RNA molecules, oligomers, that have a wide range of applications in genetic testing, research, and forensics. Commonly made in the laboratory by solid-phase chemical synthesis, these small bits of nucleic acids ...

DNA microarrays A DNA microarray (also commonly known as DNA chip or biochip) is a collection of microscopic DNA spots attached to a solid surface. Scientists use DNA microarrays to measure the expression levels of large numbers of genes simultaneously or to ...

to analyze

RNA Ribonucleic acid (RNA) is a polymeric molecule essential in various biological roles in coding, decoding, regulation and expression of genes. RNA and deoxyribonucleic acid ( DNA) are nucleic acids. Along with lipids, proteins, and carbohydra ...

gene expression Gene expression is the process by which information from a gene is used in the synthesis of a functional gene product that enables it to produce end products, protein or non-coding RNA, and ultimately affect a phenotype, as the final effect. ...

. In 2006, Hochreiter and others proposed an extension of the

support vector machine In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratories ...

(SVM), the "Potential Support Vector Machine" (PSVM), which can be applied to non-square kernel matrices and can be used with kernels that are not positive definite. Hochreiter and his collaborators have applied PSVM to

feature selection In machine learning and statistics, feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables, predictors) for use in model construc ...

, including gene selection for microarray data.

Awards

Hochreiter was awarded the IEEE CIS Neural Networks Pioneer Prize in 2021 for his work on LSTM.

References

External links

Home Page Sepp Hochreiter
{{DEFAULTSORT:Hochreiter, Sepp 1967 births Living people German bioinformaticians Biostatisticians Computational biology Artificial intelligence researchers Machine learning researchers Technical University of Munich faculty Academics of Johannes Kepler University Linz