Josef "Sepp" Hochreiter (born 14 February 1967) is a German

computer scientist A computer scientist is a person who is trained in the academic study of computer science. Computer scientists typically work on the theoretical side of computation, as opposed to the hardware side on which computer engineers mainly focus (a ...

. Since 2018 he has led the Institute for Machine Learning at the Johannes Kepler University of

Linz Linz ( , ; cs, Linec) is the capital of Upper Austria and third-largest city in Austria. In the north of the country, it is on the Danube south of the Czech border. In 2018, the population was 204,846. In 2009, it was a European Capital of ...

after having led the Institute of Bioinformatics from 2006 to 2018. In 2017 he became the head of the Linz Institute of Technology (LIT) AI Lab. Hochreiter is also a founding director of the Institute of Advanced Research in Artificial Intelligence (IARAI). Previously, he was at the

Technical University An institute of technology (also referred to as: technological university, technical university, university of technology, technological educational institute, technical college, polytechnic university or just polytechnic) is an institution of te ...

of Berlin, at the

University of Colorado The University of Colorado (CU) is a system of public universities in Colorado. It consists of four institutions: University of Colorado Boulder, University of Colorado Colorado Springs, University of Colorado Denver, and the University of C ...

Boulder In geology, a boulder (or rarely bowlder) is a rock fragment with size greater than in diameter. Smaller pieces are called cobbles and pebbles. While a boulder may be small enough to move or roll manually, others are extremely massive. In c ...

, and at the

Munich Munich ( ; german: München ; bar, Minga ) is the capital and most populous city of the States of Germany, German state of Bavaria. With a population of 1,558,395 inhabitants as of 31 July 2020, it is the List of cities in Germany by popu ...

. He is a chair of the Critical Assessment of Massive Data Analysis (CAMDA) conference. Hochreiter has made contributions in the fields of

machine learning Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machin ...

deep learning Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised. D ...

and

bioinformatics Bioinformatics () is an interdisciplinary field that develops methods and software tools for understanding biological data, in particular when the data sets are large and complex. As an interdisciplinary field of science, bioinformatics combine ...

, most notably the development of the

long short-term memory Long short-term memory (LSTM) is an artificial neural network used in the fields of artificial intelligence and deep learning. Unlike standard feedforward neural networks, LSTM has feedback connections. Such a recurrent neural network (RNN) c ...

(LSTM) neural network architecture, but also in meta-learning,

reinforcement learning Reinforcement learning (RL) is an area of machine learning concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward. Reinforcement learning is one of three basic machine ...

and

biclustering Biclustering, block clustering, Co-clustering or two-mode clustering is a data mining technique which allows simultaneous clustering of the rows and columns of a matrix. The term was first introduced by Boris Mirkin to name a technique introduce ...

with application to

data.

Scientific career

Long short-term memory (LSTM)

Hochreiter developed the

(LSTM) neural network architecture in his diploma thesis in 1991 leading to the main publication in 1997. LSTM overcomes the problem that

recurrent neural networks A recurrent neural network (RNN) is a class of artificial neural networks where connections between nodes can create a cycle, allowing output from some nodes to affect subsequent input to the same nodes. This allows it to exhibit temporal dynamic ...

(RNNs) forget information over time ( vanishing or exploding gradient). In 2007, Hochreiter and others successfully applied LSTM with an optimized architecture to very fast protein homology detection without requiring a

sequence alignment In bioinformatics, a sequence alignment is a way of arranging the sequences of DNA, RNA, or protein to identify regions of similarity that may be a consequence of functional, structural, or evolutionary relationships between the sequences. Ali ...

. LSTM networks have also been also used in

Google Voice Google Voice is a telephone service that provides a U.S. phone number to Google Account customers in the U.S. and Google Workspace (G Suite by October 2020) customers in Canada, Denmark, France, the Netherlands, Portugal, Spain, Sweden, Swit ...

for transcription and search, and in the

Google Allo Google Allo was an instant messaging mobile app by Google for the Android and iOS mobile operating systems, with a web client available on Google Chrome, Mozilla Firefox, and Opera. It closed on March 12, 2019. The app used phone numbers as id ...

chat app for generating response suggestion with low latency.

Other machine learning contributions

Beyond LSTM, Hochreiter has developed "Flat Minimum Search" to increase the

generalization A generalization is a form of abstraction whereby common properties of specific instances are formulated as general concepts or claims. Generalizations posit the existence of a domain or set of elements, as well as one or more common character ...

of neural networks and introduced rectified factor networks (RFNs) for sparse coding which have been applied in bioinformatics and genetics. Hochreiter introduced modern Hopfield networks with continuous states and applied them to the task of immune repertoire classification. Hochreiter worked with Jürgen Schmidhuber in the field of

on actor-critic systems that learn by "backpropagation through a model". Hochreiter has been involved in the development of

factor analysis Factor analysis is a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. For example, it is possible that variations in six observed ...

methods with application to bioinformatics, including FABIA for

, HapFABIA for detecting short segments of

identity by descent A DNA segment is identical by state (IBS) in two or more individuals if they have identical nucleotide sequences in this segment. An IBS segment is identical by descent (IBD) in two or more individuals if they have inherited it from a common a ...

and FARMS for preprocessing and summarizing high-density

oligonucleotide Oligonucleotides are short DNA or RNA molecules, oligomers, that have a wide range of applications in genetic testing, research, and forensics. Commonly made in the laboratory by solid-phase chemical synthesis, these small bits of nucleic acids c ...

DNA microarrays A DNA microarray (also commonly known as DNA chip or biochip) is a collection of microscopic DNA spots attached to a solid surface. Scientists use DNA microarrays to measure the expression levels of large numbers of genes simultaneously or to g ...

to analyze

RNA Ribonucleic acid (RNA) is a polymeric molecule essential in various biological roles in coding, decoding, regulation and expression of genes. RNA and deoxyribonucleic acid ( DNA) are nucleic acids. Along with lipids, proteins, and carbohyd ...

gene expression Gene expression is the process by which information from a gene is used in the synthesis of a functional gene product that enables it to produce end products, protein or non-coding RNA, and ultimately affect a phenotype, as the final effect. The ...

. In 2006, Hochreiter and others proposed an extension of the

support vector machine In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data for classification and regression analysis. Developed at AT&T Bell Laboratorie ...

(SVM), the "Potential Support Vector Machine" (PSVM), which can be applied to non-square kernel matrices and can be used with kernels that are not positive definite. Hochreiter and his collaborators have applied PSVM to

feature selection In machine learning and statistics, feature selection, also known as variable selection, attribute selection or variable subset selection, is the process of selecting a subset of relevant features (variables, predictors) for use in model construc ...

, including gene selection for microarray data.

Awards

Hochreiter was awarded the IEEE CIS Neural Networks Pioneer Prize in 2021 for his work on LSTM.

References

External links

Home Page Sepp Hochreiter
{{DEFAULTSORT:Hochreiter, Sepp 1967 births Living people German bioinformaticians Biostatisticians Computational biology Artificial intelligence researchers Machine learning researchers Technical University of Munich faculty Academics of Johannes Kepler University Linz