Eliezer Shlomo Yudkowsky (born September 11, 1979) is an American

decision theory Decision theory (or the theory of choice; not to be confused with choice theory) is a branch of applied probability theory concerned with the theory of making decisions based on assigning probabilities to various factors and assigning numerical ...

and

artificial intelligence Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machines, as opposed to intelligence displayed by animals and humans. Example tasks in which this is done include speech ...

(AI) researcher and writer, best known for popularizing the idea of

friendly artificial intelligence Friendly artificial intelligence (also friendly AI or FAI) refers to hypothetical artificial general intelligence (AGI) that would have a positive (benign) effect on humanity or at least align with human interests or contribute to foster the impro ...

. He is a co-founder and research fellow at the

Machine Intelligence Research Institute The Machine Intelligence Research Institute (MIRI), formerly the Singularity Institute for Artificial Intelligence (SIAI), is a non-profit research institute focused since 2005 on identifying and managing potential existential risks from artif ...

(MIRI), a private research nonprofit based in

Berkeley, California Berkeley ( ) is a city on the eastern shore of San Francisco Bay in northern Alameda County, California, United States. It is named after the 18th-century Irish bishop and philosopher George Berkeley. It borders the cities of Oakland and E ...

. His work on the prospect of a runaway

intelligence explosion The technological singularity—or simply the singularity—is a hypothetical future point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization. According to the ...

was an influence on

Nick Bostrom Nick Bostrom ( ; sv, Niklas Boström ; born 10 March 1973) is a Swedish-born philosopher at the University of Oxford known for his work on existential risk, the anthropic principle, human enhancement ethics, superintelligence risks, and the ...

's '' Superintelligence: Paths, Dangers, Strategies''.

Work in artificial intelligence safety

Goal learning and incentives in software systems

Yudkowsky's views on the safety challenges posed by future generations of AI systems are discussed in the undergraduate textbook in AI, Stuart Russell and

Peter Norvig Peter Norvig (born December 14, 1956) is an American computer scientist and Distinguished Education Fellow at the Stanford Institute for Human-Centered AI. He previously served as a director of research and search quality at Google. Norvig is t ...

's '' Artificial Intelligence: A Modern Approach''. Noting the difficulty of formally specifying general-purpose goals by hand, Russell and Norvig cite Yudkowsky's proposal that autonomous and adaptive systems be designed to learn correct behavior over time: In response to the instrumental convergence concern, where autonomous decision-making systems with poorly designed goals would have default incentives to mistreat humans, Yudkowsky and other MIRI researchers have recommended that work be done to specify software agents that converge on safe default behaviors even when their goals are misspecified.

Capabilities forecasting

In the

scenario hypothesized by I. J. Good, recursively self-improving AI systems quickly transition from subhuman general intelligence to

superintelligent A superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. "Superintelligence" may also refer to a property of problem-solving systems (e.g., superintelligent language ...

's 2014 book '' Superintelligence: Paths, Dangers, Strategies'' sketches out Good's argument in detail, while citing writing by Yudkowsky on the risk that anthropomorphizing advanced AI systems will cause people to misunderstand the nature of an intelligence explosion. "AI might make an ''apparently'' sharp jump in intelligence purely as the result of anthropomorphism, the human tendency to think of 'village idiot' and 'Einstein' as the extreme ends of the intelligence scale, instead of nearly indistinguishable points on the scale of minds-in-general." In '' Artificial Intelligence: A Modern Approach'', authors Stuart Russell and

raise the objection that there are known limits to intelligent problem-solving from

computational complexity theory In theoretical computer science and mathematics, computational complexity theory focuses on classifying computational problems according to their resource usage, and relating these classes to each other. A computational problem is a task solved ...

; if there are strong limits on how efficiently algorithms can solve various computer science tasks, then intelligence explosion may not be possible.

Rationality writing

Between 2006 and 2009, Yudkowsky and

Robin Hanson Robin Dale Hanson (born August 28, 1959) is an associate professor of economics at George Mason University and a research associate at the Future of Humanity Institute of Oxford University. He is known for his work on idea futures and markets, a ...

were the principal contributors to ''Overcoming Bias'', a cognitive and social science blog sponsored by the

Future of Humanity Institute The Future of Humanity Institute (FHI) is an interdisciplinary research centre at the University of Oxford investigating big-picture questions about humanity and its prospects. It was founded in 2005 as part of the Faculty of Philosophy and t ...

of Oxford University. In February 2009, Yudkowsky founded '' LessWrong'', a "community blog devoted to refining the art of human rationality". ''Overcoming Bias'' has since functioned as Hanson's personal blog. Over 300 blogposts by Yudkowsky on philosophy and science (originally written on ''LessWrong'' and ''Overcoming Bias'') were released as an ebook entitled ''Rationality: From AI to Zombies'' by the

(MIRI) in 2015. MIRI has also published ''Inadequate Equilibria'', Yudkowsky's 2017 ebook on the subject of societal inefficiencies. Yudkowsky has also written several works of fiction. His fanfiction novel, ''

Harry Potter and the Methods of Rationality ''Harry Potter and the Methods of Rationality'' (''HPMOR'') is a ''Harry Potter'' fan fiction by Eliezer Yudkowsky. It adapts the story of ''Harry Potter'' to explain complex concepts in cognitive science, philosophy, and the scientific method. ...

'', uses plot elements from J. K. Rowling's ''

Harry Potter ''Harry Potter'' is a series of seven fantasy novels written by British author J. K. Rowling. The novels chronicle the lives of a young wizard, Harry Potter, and his friends Hermione Granger and Ron Weasley, all of whom are students at ...

'' series to illustrate topics in science. ''

The New Yorker ''The New Yorker'' is an American weekly magazine featuring journalism, commentary, criticism, essays, fiction, satire, cartoons, and poetry. Founded as a weekly in 1925, the magazine is published 47 times annually, with five of these issues ...

'' described ''Harry Potter and the Methods of Rationality'' as a retelling of Rowling's original "in an attempt to explain Harry's wizardry through the scientific method".

Personal life

Yudkowsky is an

autodidact Autodidacticism (also autodidactism) or self-education (also self-learning and self-teaching) is education without the guidance of masters (such as teachers and professors) or institutions (such as schools). Generally, autodidacts are individu ...

and did not attend high school or college. He was raised as a Modern Orthodox Jew.

Academic publications

* * * * * * * *

References

External links

*
''Rationality: From AI to Zombies''
(entire book online) {{DEFAULTSORT:Yudkowsky, Eliezer 1979 births Living people American male bloggers American bloggers Artificial intelligence researchers Artificial intelligence ethicists Life extensionists Fan fiction writers American transhumanists People from Berkeley, California American libertarians Singularitarians Cryonicists Epistemologists People associated with effective altruism Bayesian statisticians Consequentialists People from Chicago American people of Jewish descent