|
Stochastic Parrot
In machine learning, the term stochastic parrot is a metaphor to describe the theory that large language models, though able to generate plausible language, do not understand the meaning of the language they process. The term was coined by Emily M. Bender in the 2021 artificial intelligence research paper "''On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜''" by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell. Origin and definition The term was first used in the paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell (using the pseudonym "Shmargaret Shmitchell"). They argued that large language models (LLMs) present dangers such as environmental and financial costs, inscrutability leading to unknown dangerous biases, and potential for deception, and that they can't understand the concepts underlying what they learn. Gebru was asked to re ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Machine Learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine learning algorithms build a model based on sample data, known as training data, in order to make predictions or decisions without being explicitly programmed to do so. Machine learning algorithms are used in a wide variety of applications, such as in medicine, email filtering, speech recognition, agriculture, and computer vision, where it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks.Hu, J.; Niu, H.; Carrasco, J.; Lennox, B.; Arvin, F.,Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning IEEE Transactions on Vehicular Technology, 2020. A subset of machine learning is closely related to computational statistics, which focuses on making pred ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Sam Altman
Samuel H. Altman ( ; born April 22, 1985) is an American entrepreneur, investor, programmer, and blogger. He is the CEO of OpenAI and the former president of Y Combinator. Early life and education Altman grew up in St. Louis, Missouri; his mother is a dermatologist. He received his first computer at the age of 8. He was raised Jewish. He attended John Burroughs School for high school and studied computer science at Stanford University until dropping out in 2005. In 2017, he received an honorary degree from the University of Waterloo. Altman is gay, and has been out since his youth. Career Loopt In 2005, at age 19, Altman co-founded and became CEO of Loopt, a location-based social networking mobile application. After raising more than $30M in venture capital, Loopt was shut down in 2012 after failing to get traction. It was acquired by the Green Dot Corporation for $43.4 million. Y Combinator Altman began as a part-time partner at Y Combinator in 2011. In February 2 ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Criticism Of Artificial Neural Networks
In machine learning, a neural network (also artificial neural network or neural net, abbreviated ANN or NN) is a model inspired by the structure and function of biological neural networks in animal brains. An ANN consists of connected units or nodes called ''artificial neurons'', which loosely model the neurons in a brain. These are connected by ''edges'', which model the synapses in a brain. Each artificial neuron receives signals from connected neurons, then processes them and sends a signal to other connected neurons. The "signal" is a real number, and the output of each neuron is computed by some non-linear function of the sum of its inputs, called the '' activation function''. The strength of the signal at each connection is determined by a ''weight'', which adjusts during the learning process. Typically, neurons are aggregated into layers. Different layers may perform different transformations on their inputs. Signals travel from the first layer (the ''input layer'') to ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Chinese Room
The Chinese room argument holds that a digital computer executing a program cannot have a " mind," "understanding" or "consciousness," regardless of how intelligently or human-like the program may make the computer behave. The argument was presented by philosopher John Searle in his paper, "Minds, Brains, and Programs", published in '' Behavioral and Brain Sciences'' in 1980. Similar arguments were presented by Gottfried Leibniz (1714), Anatoly Dneprov (1961), Lawrence Davis (1974) and Ned Block (1978). Searle's version has been widely discussed in the years since. The centerpiece of Searle's argument is a thought experiment known as the ''Chinese room''. The argument is directed against the philosophical positions of functionalism and computationalism, which hold that the mind may be viewed as an information-processing system operating on formal symbols, and that simulation of a given mental state is sufficient for its presence. Specifically, the argument is intended to ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
1 The Road
''1 the Road'' is an experimental novel composed by artificial intelligence (AI). Emulating Jack Kerouac's '' On the Road'', Ross Goodwin drove from New York to New Orleans in March 2017 with an AI in a laptop hooked up to various sensors, whose output the AI turned into words that were printed on rolls of receipt paper. The novel was published in 2018 by Jean Boîte Éditions. Goodwin left the text unedited. Although he felt the prose was "choppy", and contained typographical errors, he wanted to present the machine-generated text verbatim, for future study. The story begins: "It was nine seventeen in the morning, and the house was heavy". Concept and execution Emulating Jack Kerouac's novel '' On the Road'', Ross Goodwin traveled from New York to New Orleans in March 2017 with the AI, in the form of a long short-term memory recurrent neural network. Three sensors provided real-world input: a surveillance camera mounted on the trunk was trained on the passing scenery, a micro ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
BERT (language Model)
Bidirectional Encoder Representations from Transformers (BERT) is a transformer-based machine learning technique for natural language processing (NLP) pre-training developed by Google. BERT was created and published in 2018 by Jacob Devlin and his colleagues from Google. In 2019, Google announced that it had begun leveraging BERT in its search engine, and by late 2020 it was using BERT in almost every English-language query. A 2020 literature survey concluded that "in a little over a year, BERT has become a ubiquitous baseline in NLP experiments", counting over 150 research publications analyzing and improving the model. The original English-language BERT has two models: (1) the BERTBASE: 12 encoders with 12 bidirectional self-attention heads, and (2) the BERTLARGE: 24 encoders with 16 bidirectional self-attention heads. Both models are pre-trained from unlabeled data extracted from the BooksCorpus with 800M words and English Wikipedia with 2,500M words. Architecture BERT is ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Grokking (machine Learning)
In machine learning, grokking, or delayed generalization, is a transition to generalization that occurs many training iterations after the interpolation threshold, after many iterations of seemingly little progress, as opposed to the usual process where generalization occurs slowly and progressively once the interpolation threshold has been reached. The term derives from the word ''grok'' coined by Robert Heinlein in his novel ''Stranger in a Strange Land''. Grokking can be understood as a phase transition In chemistry, thermodynamics, and other related fields, a phase transition (or phase change) is the physical process of transition between one state of a medium and another. Commonly the term is used to refer to changes among the basic states ... during the training process. While grokking has been thought of as largely a phenomenon of relatively shallow models, grokking has been observed in deep neural networks and non-neural models and is the subject of active research ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Karel (programming Language)
Karel is an educational programming language for beginners, created by Richard E. Pattis in his book ''Karel The Robot: A Gentle Introduction to the Art of Programming''. Pattis used the language in his courses at Stanford University, California. The language is named after Karel Čapek, a Czech writer who introduced the word ''robot'' in his play R.U.R. Principles A program in Karel is used to control a simple robot named Karel that lives in an environment consisting of a grid of streets (left-right) and avenues (up-down). Karel understands five basic instructions: move (Karel moves by one square in the direction he is facing), turnLeft (Karel turns 90 ° left), putBeeper (Karel puts a beeper on the square he is standing at), pickBeeper (Karel lifts a beeper off the square he is standing at), and turnoff (Karel switches himself off, the program ends). Karel can also perform boolean queries about his immediate environment, asking whether there is a beeper where he is st ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Reversi
Reversi is a strategy board game for two players, played on an 8×8 uncheckered board. It was invented in 1883. Othello, a variant with a fixed initial setup of the board, was patented in 1971. Basics There are sixty-four identical game pieces called ''disks'', which are light on one side and dark on the other. Players take turns placing disks on the board with their assigned color facing up. During a play, any disks of the opponent's color that are in a straight line and bounded by the disk just placed and another disk of the current player's color are turned over to the current player's color. The objective of the game is to have the majority of disks turned to display one's color when the last playable empty square is filled. History Original version Englishmen Lewis Waterman and John W. Mollett both claim to have invented the game of Reversi in 1883, each denouncing the other as a fraud. The game gained considerable popularity in England at the end of the 19th century ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Transformer (deep Learning Architecture)
A transformer is a deep learning architecture developed by researchers at Google and based on the multi-head attention mechanism, proposed in a 2017 paper " Attention Is All You Need". Text is converted to numerical representations called tokens, and each token is converted into a vector via looking up from a word embedding table. At each layer, each token is then contextualized within the scope of the context window with other (unmasked) tokens via a parallel multi-head attention mechanism allowing the signal for key tokens to be amplified and less important tokens to be diminished. Transformers have the advantage of having no recurrent units, and therefore require less training time than earlier recurrent neural architectures (RNNs) such as long short-term memory (LSTM). Later variations have been widely adopted for training large language models (LLM) on large (language) datasets, such as the Wikipedia corpus and Common Crawl. Transformers were first developed as an impr ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Reverse Engineering
Reverse engineering (also known as backwards engineering or back engineering) is a process or method through which one attempts to understand through deductive reasoning how a previously made device, process, system, or piece of software accomplishes a task with very little (if any) insight into exactly how it does so. It is essentially the process of opening up or dissecting a system to see how it works, in order to duplicate or enhance it. Depending on the system under consideration and the technologies employed, the knowledge gained during reverse engineering can help with repurposing obsolete objects, doing security analysis, or learning how something works. Although the process is specific to the object on which it is being performed, all reverse engineering processes consist of three basic steps: Information extraction, Modeling, and Review. Information extraction refers to the practice of gathering all relevant information for performing the operation. Modeling refers to th ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Benchmark (computing)
In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it. The term ''benchmark'' is also commonly utilized for the purposes of elaborately designed benchmarking programs themselves. Benchmarking is usually associated with assessing performance characteristics of computer hardware, for example, the floating point operation performance of a CPU, but there are circumstances when the technique is also applicable to software. Software benchmarks are, for example, run against compilers or database management systems (DBMS). Benchmarks provide a method of comparing the performance of various subsystems across different chip/system architectures. Purpose As computer architecture advanced, it became more difficult to compare the performance of various computer systems simply by looking at their specificatio ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
