Eliezer Yudkowsky
   HOME

TheInfoList



OR:

Eliezer S. Yudkowsky ( ; born September 11, 1979) is an American artificial intelligence researcher and writer on
decision theory Decision theory or the theory of rational choice is a branch of probability theory, probability, economics, and analytic philosophy that uses expected utility and probabilities, probability to model how individuals would behave Rationality, ratio ...
and
ethics Ethics is the philosophy, philosophical study of Morality, moral phenomena. Also called moral philosophy, it investigates Normativity, normative questions about what people ought to do or which behavior is morally right. Its main branches inclu ...
, best known for popularizing ideas related to
friendly artificial intelligence Friendly artificial intelligence (friendly AI or FAI) is hypothetical artificial general intelligence (AGI) that would have a positive (benign) effect on humanity or at least align with human interests such as fostering the improvement of the hu ...
. He is the founder of and a research fellow at the
Machine Intelligence Research Institute The Machine Intelligence Research Institute (MIRI), formerly the Singularity Institute for Artificial Intelligence (SIAI), is a non-profit research institute focused since 2005 on identifying and managing potential existential risks from artifi ...
(MIRI), a private research nonprofit based in
Berkeley, California Berkeley ( ) is a city on the eastern shore of San Francisco Bay in northern Alameda County, California, United States. It is named after the 18th-century Anglo-Irish bishop and philosopher George Berkeley. It borders the cities of Oakland, Cali ...
. His work on the prospect of a runaway
intelligence explosion The technological singularity—or simply the singularity—is a hypothetical point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable consequences for human civilization. According to the ...
influenced philosopher
Nick Bostrom Nick Bostrom ( ; ; born 10 March 1973) is a Philosophy, philosopher known for his work on existential risk, the anthropic principle, human enhancement ethics, whole brain emulation, Existential risk from artificial general intelligence, superin ...
's 2014 book '' Superintelligence: Paths, Dangers, Strategies''.


Work in artificial intelligence safety


Goal learning and incentives in software systems

Yudkowsky's views on the safety challenges future generations of AI systems pose are discussed in Stuart Russell's and
Peter Norvig Peter Norvig (born 14 December 1956) is an American computer scientist and Distinguished Education Fellow at the Stanford Institute for Human-Centered AI. He previously served as a director of research and search quality at Google. Norvig is th ...
's undergraduate textbook '' Artificial Intelligence: A Modern Approach''. Noting the difficulty of formally specifying general-purpose goals by hand, Russell and Norvig cite Yudkowsky's proposal that autonomous and adaptive systems be designed to learn correct behavior over time: In response to the instrumental convergence concern, that autonomous decision-making systems with poorly designed goals would have default incentives to mistreat humans, Yudkowsky and other MIRI researchers have recommended that work be done to specify software agents that converge on safe default behaviors even when their goals are misspecified.


Capabilities forecasting

In the
intelligence explosion The technological singularity—or simply the singularity—is a hypothetical point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable consequences for human civilization. According to the ...
scenario hypothesized by I. J. Good, recursively self-improving AI systems quickly transition from subhuman general intelligence to superintelligent.
Nick Bostrom Nick Bostrom ( ; ; born 10 March 1973) is a Philosophy, philosopher known for his work on existential risk, the anthropic principle, human enhancement ethics, whole brain emulation, Existential risk from artificial general intelligence, superin ...
's 2014 book '' Superintelligence: Paths, Dangers, Strategies'' sketches out Good's argument in detail, while citing Yudkowsky on the risk that anthropomorphizing advanced AI systems will cause people to misunderstand the nature of an intelligence explosion. "AI might make an ''apparently'' sharp jump in intelligence purely as the result of anthropomorphism, the human tendency to think of 'village idiot' and 'Einstein' as the extreme ends of the intelligence scale, instead of nearly indistinguishable points on the scale of minds-in-general." In ''Artificial Intelligence: A Modern Approach'', Russell and Norvig raise the objection that there are known limits to intelligent problem-solving from
computational complexity theory In theoretical computer science and mathematics, computational complexity theory focuses on classifying computational problems according to their resource usage, and explores the relationships between these classifications. A computational problem ...
; if there are strong limits on how efficiently algorithms can solve various tasks, an intelligence explosion may not be possible.


''Time'' op-ed

In a 2023 op-ed for ''Time'' magazine, Yudkowsky discussed the risk of artificial intelligence and advocated for international agreements to limit it, including a total halt on the development of AI. He suggested that participating countries should be willing to take military action, such as "destroy nga rogue datacenter by airstrike", to enforce such a moratorium. The article helped introduce the debate about AI alignment to the mainstream, leading a reporter to ask President
Joe Biden Joseph Robinette Biden Jr. (born November 20, 1942) is an American politician who was the 46th president of the United States from 2021 to 2025. A member of the Democratic Party (United States), Democratic Party, he served as the 47th vice p ...
a question about AI safety at a press briefing.


''If Anyone Builds It, Everyone Dies''

Together with Nate Soares, Yudkowsky wrote ''If Anyone Builds It, Everyone Dies'', which is being published by
Little, Brown and Company Little, Brown and Company is an American publishing company founded in 1837 by Charles Coffin Little and James Brown in Boston. For close to two centuries, it has published fiction and nonfiction by American authors. Early lists featured Emil ...
on September 16, 2025.


Rationality writing

Between 2006 and 2009, Yudkowsky and Robin Hanson were the principal contributors to ''Overcoming Bias'', a cognitive and social science blog sponsored by the
Future of Humanity Institute The Future of Humanity Institute (FHI) was an interdisciplinary research centre at the University of Oxford investigating big-picture questions about humanity and its prospects. It was founded in 2005 as part of the Faculty of Philosophy and t ...
of Oxford University. In February 2009, Yudkowsky founded '' LessWrong'', a "community blog devoted to refining the art of human rationality". ''Overcoming Bias'' has since functioned as Hanson's personal blog. Over 300 blog posts by Yudkowsky on philosophy and science (originally written on ''LessWrong'' and ''Overcoming Bias'') were released as an ebook, ''Rationality: From AI to Zombies'', by MIRI in 2015. MIRI has also published ''Inadequate Equilibria'', Yudkowsky's 2017 ebook on societal inefficiencies. Yudkowsky has also written several works of fiction. His fanfiction novel '' Harry Potter and the Methods of Rationality'' uses plot elements from J. K. Rowling's ''
Harry Potter ''Harry Potter'' is a series of seven Fantasy literature, fantasy novels written by British author J. K. Rowling. The novels chronicle the lives of a young Magician (fantasy), wizard, Harry Potter (character), Harry Potter, and his friends ...
'' series to illustrate topics in science and rationality. ''
The New Yorker ''The New Yorker'' is an American magazine featuring journalism, commentary, criticism, essays, fiction, satire, cartoons, and poetry. It was founded on February 21, 1925, by Harold Ross and his wife Jane Grant, a reporter for ''The New York T ...
'' described ''Harry Potter and the Methods of Rationality'' as a retelling of Rowling's original "in an attempt to explain Harry's wizardry through the scientific method".


Personal life

Yudkowsky is an autodidact and did not attend high school or college. He is Jewish and was raised as a Modern Orthodox Jew, but is now secular.


Academic publications

* * * * * * * *


See also

* AI box *
Friendly artificial intelligence Friendly artificial intelligence (friendly AI or FAI) is hypothetical artificial general intelligence (AGI) that would have a positive (benign) effect on humanity or at least align with human interests such as fostering the improvement of the hu ...
* Open Letter on Artificial Intelligence


Notes


References


External links

*
''Rationality: From AI to Zombies''
(entire book online)
@ESYudkowsky
– Yudkowsky on X {{DEFAULTSORT:Yudkowsky, Eliezer 1979 births Living people American male bloggers American bloggers American founders American artificial intelligence researchers Life extensionists Fan fiction writers American transhumanists Writers from Berkeley, California American libertarians Singularitarians Cryonicists American epistemologists People associated with effective altruism Consequentialists American people of Jewish descent Rationalists Former Orthodox Jews AI safety scientists American secular Jews