Computer Go is the field of
artificial intelligence
Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machine
A machine is a physical system using Power (physics), power to apply Force, forces and control Motion, moveme ...
(AI) dedicated to creating a
computer program
A computer program is a sequence or set of instructions in a programming language for a computer to execute. Computer programs are one component of software, which also includes documentation and other intangible components.
A computer progra ...
that plays the traditional
board game
Board games are tabletop games that typically use . These pieces are moved or placed on a pre-marked board (playing surface) and often include elements of table, card, role-playing, and miniatures games as well.
Many board games feature a ...
Go. The field is sharply divided into two eras. Before 2015, the programs of the era were weak. The best efforts of the 1980s and 1990s produced only AIs that could be defeated by beginners, and AIs of the early 2000s were intermediate level at best. Professionals could defeat these programs even given handicaps of 10+ stones in favor of the AI. Many of the algorithms such as
alpha-beta minimax that performed well as AIs for
checkers
Checkers (American English), also known as draughts (; British English), is a group of strategy board games for two players which involve diagonal moves of uniform game pieces and mandatory captures by jumping over opponent pieces. Checkers ...
and
chess
Chess is a board game for two players, called White and Black, each controlling an army of chess pieces in their color, with the objective to checkmate the opponent's king. It is sometimes called international chess or Western chess to dist ...
fell apart on Go's 19x19 board, as there were too many branching possibilities to consider. Creation of a human professional quality program with the techniques and hardware of the time was out of reach. Some AI researchers speculated that the problem was unsolvable without creation of
human-like AI.
The application of
Monte Carlo tree search
In computer science, Monte Carlo tree search (MCTS) is a heuristic search algorithm for some kinds of decision processes, most notably those employed in software that plays board games. In that context MCTS is used to solve the game tree.
MCT ...
to Go algorithms provided a notable improvement in the late
2000s decade
File:2000s decade montage3.png, From top left, clockwise: The World Trade Center on fire and the Statue of Liberty during the 9/11 attacks in 2001; the euro enters into European currency in 2002; a statue of Saddam Hussein being toppled during t ...
, with programs finally able to achieve a
low-dan level: that of an advanced amateur. High-dan amateurs and professionals could still exploit these programs' weaknesses and win consistently, but computer performance had advanced past the intermediate (single-digit ''kyu'') level. The tantalizing unmet goal of defeating the best human players without a handicap, long thought unreachable, brought a burst of renewed interest. The key insight proved to be an application of
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
and
deep learning.
DeepMind
DeepMind Technologies is a British artificial intelligence subsidiary of Alphabet Inc. and research laboratory founded in 2010. DeepMind was acquired by Google in 2014 and became a wholly owned subsidiary of Alphabet Inc, after Google's restru ...
, a
Google
Google LLC () is an American Multinational corporation, multinational technology company focusing on Search Engine, search engine technology, online advertising, cloud computing, software, computer software, quantum computing, e-commerce, ar ...
acquisition dedicated to AI research, produced
AlphaGo
AlphaGo is a computer program that plays the board game Go. It was developed by DeepMind Technologies a subsidiary of Google (now Alphabet Inc.). Subsequent versions of AlphaGo became increasingly powerful, including a version that competed u ...
in 2015 and announced it to the world in 2016.
AlphaGo defeated Lee Sedol, a 9 dan professional, in a no-handicap match in 2016, then
defeated Ke Jie in 2017, who at the time continuously held the world No. 1 ranking for two years. Just as
checkers had fallen to machines in 1995 and
chess in 1997, computer programs finally conquered humanity's greatest Go champions in 2016–2017. DeepMind did not release AlphaGo for public use, but various programs have been built since based on the journal articles DeepMind released describing AlphaGo and its variants.
Overview and history
Professional Go players see the game as requiring intuition, creative and strategic thinking. It has long been considered a difficult challenge in the field of
artificial intelligence
Artificial intelligence (AI) is intelligence—perceiving, synthesizing, and inferring information—demonstrated by machine
A machine is a physical system using Power (physics), power to apply Force, forces and control Motion, moveme ...
(AI) and is considerably more difficult to solve than
chess
Chess is a board game for two players, called White and Black, each controlling an army of chess pieces in their color, with the objective to checkmate the opponent's king. It is sometimes called international chess or Western chess to dist ...
. Many in the field considered Go to require more elements that mimic human thought than chess. Mathematician
I. J. Good wrote in 1965:
Prior to 2015, the best Go programs only managed to reach
amateur dan level.
On the small 9×9 board, the computer fared better, and some programs managed to win a fraction of their 9×9 games against professional players. Prior to AlphaGo, some researchers had claimed that computers would never defeat top humans at Go.
Early decades
The first Go program was written by
Albert Lindsey Zobrist in 1968 as part of his thesis on
pattern recognition
Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphic ...
. It introduced an
influence function to estimate territory and
Zobrist hashing to detect
ko.
In April 1981, Jonathan K Millen published an article in ''
Byte
The byte is a unit of digital information that most commonly consists of eight bits. Historically, the byte was the number of bits used to encode a single character of text in a computer and for this reason it is the smallest addressable unit ...
'' discussing Wally, a Go program with a 15x15 board that fit within the
KIM-1
The KIM-1, short for ''Keyboard Input Monitor'', is a small 6502-based single-board computer developed and produced by MOS Technology, Inc. and launched in 1976. It was very successful in that period, due to its low price (thanks to the inexp ...
microcomputer's 1K RAM.
Bruce F. Webster published an article in the magazine in November 1984 discussing a Go program he had written for the
Apple Macintosh
The Mac (known as Macintosh until 1999) is a family of personal computers designed and marketed by Apple Inc. Macs are known for their ease of use and minimalist designs, and are popular among students, creative professionals, and software ...
, including the
MacFORTH source.
Programs for Go were weak; a 1983 article estimated that they were at best equivalent to 20 ''kyu'', the rating of a naive novice player, and often restricted themselves to smaller boards. AIs who played on the
Internet Go Server (IGS) on 19x19 size boards had around 20–15 ''kyu'' strength in 2003, after substantial improvements in hardware.
In 1998, very strong players were able to beat computer programs while giving handicaps of 25–30 stones, an enormous handicap that few human players would ever take. There was a case in the 1994 World Computer Go Championship where the winning program, Go Intellect, lost all three games against the youth players while receiving a 15-stone handicap. In general, players who understood and exploited a program's weaknesses could win even through large handicaps.
2007–2014: Monte Carlo tree search
In 2006 (with an article published in 2007),
Rémi Coulom produced a new algorithm he called
Monte Carlo tree search
In computer science, Monte Carlo tree search (MCTS) is a heuristic search algorithm for some kinds of decision processes, most notably those employed in software that plays board games. In that context MCTS is used to solve the game tree.
MCT ...
. In it, a game tree is created as usual of potential futures that branch with every move. However, computers "score" a terminal leaf of the tree by repeated random playouts (similar to
Monte Carlo
Monte Carlo (; ; french: Monte-Carlo , or colloquially ''Monte-Carl'' ; lij, Munte Carlu ; ) is officially an administrative area of the Principality of Monaco, specifically the ward of Monte Carlo/Spélugues, where the Monte Carlo Casino i ...
strategies for other problems). The advantage is that such random playouts can be done very quickly. The intuitive objection - that random playouts do not correspond to the actual worth of a position - turned out not to be as fatal to the procedure as expected; the "tree search" side of the algorithm corrected well enough for finding reasonable future game trees to explore. Programs based on this method such as MoGo and Fuego saw better performance than classic AIs from earlier. The best programs could do especially well on the small 9x9 board, which had fewer possibilities to explore. In 2009, the first such programs appeared which could reach and hold low
dan-level ranks on the
KGS Go Server on the 19x19 board.
In 2010, at the 2010 European Go Congress in Finland, MogoTW played 19x19 Go against
Catalin Taranu (5p). MogoTW received a seven-stone handicap and won.
In 2011,
Zen
Zen ( zh, t=禪, p=Chán; ja, text= 禅, translit=zen; ko, text=선, translit=Seon; vi, text=Thiền) is a school of Mahayana Buddhism that originated in China during the Tang dynasty, known as the Chan School (''Chánzong'' 禪宗), and ...
reached 5 dan on the server KGS, playing games of 15 seconds per move. The account which reached that rank uses a cluster version of Zen running on a 26-core machine.
In 2012, Zen beat
Takemiya Masaki (9p) by 11 points at five stones handicap, followed by a 20-point win at four stones handicap.
In 2013,
Crazy Stone beat
Yoshio Ishida (9p) in a 19×19 game at four stones handicap.
The 2014 Codecentric Go Challenge, a best-of-five match in an even 19x19 game, was played between Crazy Stone and Franz-Jozef Dickhut (6d). No stronger player had ever before agreed to play a serious competition against a go program on even terms. Franz-Jozef Dickhut won, though Crazy Stone won the first match by 1.5 points.
2015 onwards: The deep learning era
In October 2015,
Google DeepMind program
AlphaGo
AlphaGo is a computer program that plays the board game Go. It was developed by DeepMind Technologies a subsidiary of Google (now Alphabet Inc.). Subsequent versions of AlphaGo became increasingly powerful, including a version that competed u ...
beat
Fan Hui, the European Go champion, five times out of five in tournament conditions.
In March 2016, AlphaGo beat
Lee Sedol
Lee Sedol ( ko, 이세돌; born 2 March 1983), or Lee Se-dol, is a former South Korean professional Go player of 9 dan rank. As of February 2016, he ranked second in international titles (18), behind only Lee Chang-ho (21). He is the f ...
in the first three of five matches.
This was the first time that a
9-dan master had played a professional game against a computer without handicap. Lee won the fourth match, describing his win as "invaluable".
AlphaGo won the final match two days later.
In May 2017, AlphaGo beat
Ke Jie, who at the time was ranked top in the world, in a
three-game match during the
Future of Go Summit.
In October 2017, DeepMind revealed a new version of AlphaGo, trained only through self play, that had surpassed all previous versions, beating the Ke Jie version in 89 out of 100 games.
After the basic principles of AlphaGo were published in the journal ''Nature'', other teams have been able to produce high-level programs. By 2017, both
Zen
Zen ( zh, t=禪, p=Chán; ja, text= 禅, translit=zen; ko, text=선, translit=Seon; vi, text=Thiền) is a school of Mahayana Buddhism that originated in China during the Tang dynasty, known as the Chan School (''Chánzong'' 禪宗), and ...
and
Tencent
Tencent Holdings Ltd. () is a Chinese multinational technology and entertainment conglomerate and holding company headquartered in Shenzhen. It is one of the highest grossing multimedia companies in the world based on revenue. It is also the wo ...
's project
Fine Art
In European academic traditions, fine art is developed primarily for aesthetics or creative expression, distinguishing it from decorative art or applied art, which also has to serve some practical function, such as pottery or most metalwor ...
were capable of defeating very high-level professionals some of the time. The open source
Leela Zero engine was created as well.
Challenges for strategy and performance for classic AIs
For a long time, it was a widely held opinion that computer Go posed a problem fundamentally different from
computer chess
Computer chess includes both hardware (dedicated computers) and software capable of playing chess. Computer chess provides opportunities for players to practice even in the absence of human opponents, and also provides opportunities for analysi ...
. Many considered a strong Go-playing program something that could be achieved only in the far future, as a result of fundamental advances in general artificial intelligence technology. Those who thought the problem feasible believed that domain knowledge would be required to be effective against human experts. Therefore, a large part of the computer Go development effort was during these times focused on ways of representing human-like expert knowledge and combining this with local search to answer questions of a tactical nature. The result of this were programs that handled many specific situations well but which had very pronounced weaknesses in their overall handling of the game. Also, these classical programs gained almost nothing from increases in available computing power. Progress in the field was generally slow.
;Size of board
The large board (19×19, 361 intersections) is often noted as one of the primary reasons why a strong program is hard to create. The large board size prevents an
alpha-beta searcher from achieving deep look-ahead without significant search extensions or
pruning
Pruning is a horticultural, arboricultural, and silvicultural practice involving the selective removal of certain parts of a plant, such as branches, buds, or roots.
The practice entails the ''targeted'' removal of diseased, damaged, dead ...
heuristics.
In 2002, a computer program called MIGOS (MIni GO Solver) completely solved the game of Go for the 5×5 board. Black wins, taking the whole board.
;Number of move options
Continuing the comparison to chess, Go moves are not as limited by the rules of the game. For the first move in chess, the player has twenty choices. Go players begin with a choice of 55 distinct legal moves, accounting for symmetry. This number rises quickly as symmetry is broken, and soon almost all of the 361 points of the board must be evaluated.
;Evaluation function
One of the most basic tasks in a game is to assess a board position: which side is favored, and by how much? In chess, many future positions in a tree are direct wins for one side, and boards have a reasonable heuristic for evaluation in simple material counting, as well as certain positional factors such as pawn structure. A future where one side has lost their queen for no benefit clearly favors the other side. These types of positional evaluation rules cannot efficiently be applied to Go. The value of a Go position depends on a complex analysis to determine whether or not the group is alive, which stones can be connected to one another, and heuristics around the extent to which a strong position has influence, or the extent to which a weak position can be attacked. A stone placed might not have immediate influence, but after many moves could become highly important in retrospect as other areas of the board take shape.
Poor evaluation of board states will cause the AI to work toward positions it incorrectly believes favor it, but actually do not.
;Life and death
One of the main concerns for a Go player is which groups of stones can be kept alive and which can be captured. This general class of problems is known as
life and death. Knowledge-based AI systems sometimes attempted to understand the life and death status of groups on the board. The most direct approach is to perform a
tree search
In computer science, tree traversal (also known as tree search and walking the tree) is a form of graph traversal and refers to the process of visiting (e.g. retrieving, updating, or deleting) each node in a tree data structure, exactly once. S ...
on the moves which potentially affect the stones in question, and then to record the status of the stones at the end of the main line of play. However, within time and memory constraints, it is not generally possible to determine with complete accuracy which moves could affect the 'life' of a group of stones. This implies that some
heuristic
A heuristic (; ), or heuristic technique, is any approach to problem solving or self-discovery that employs a practical method that is not guaranteed to be optimal, perfect, or rational, but is nevertheless sufficient for reaching an immediat ...
must be applied to select which moves to consider. The net effect is that for any given program, there is a trade-off between playing speed and life and death reading abilities.
With
Benson's algorithm, it is possible to determine the chains which are unconditionally alive and therefore would not need to be checked in the future for safety.
State representation
An issue that all Go programs must tackle is how to represent the current state of the game. The most direct way of representing a board is as a one- or two-dimensional array, where elements in the array represent points on the board, and can take on a value corresponding to a white stone, a black stone, or an empty intersection. Additional data is needed to store how many stones have been captured, whose turn it is, and which intersections are illegal due to the
Ko rule. In general, machine learning programs stop there at this simplest form and let the organic AIs come to their own understanding of the meaning of the board, likely simply using Monte Carlo playouts to "score" a board as good or bad for a player. "Classic" AI programs that attempted to directly model a human's strategy might go further, however, such as layering on data such as stones believed to be dead, stones that are unconditionally alive, stones in a ''seki'' state of mutual life, and so forth in their representation of the state of the game.
System design
Historically,
symbolic artificial intelligence
In artificial intelligence, symbolic artificial intelligence is the term for the collection of all methods in artificial intelligence research that are based on high-level symbolic (human-readable) representations of problems, logic and search. Sy ...
techniques have been used to approach the problem of Go AI.
Neural networks
A neural network is a network or circuit of biological neurons, or, in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus, a neural network is either a biological neural network, made up of biological ...
began to be tried as an alternative approach in the 2000s decade, as they required immense computing power that was expensive-to-impossible to reach in earlier decades. These approaches attempt to mitigate the problems of the game of Go having a high
branching factor and numerous other difficulties.
The only choice a program needs to make is where to place its next stone. However, this decision is made difficult by the wide range of impacts a single stone can have across the entire board, and the complex interactions various stones' groups can have with each other. Various architectures have arisen for handling this problem. Popular techniques and design philosophies include:
* some form of
tree search
In computer science, tree traversal (also known as tree search and walking the tree) is a form of graph traversal and refers to the process of visiting (e.g. retrieving, updating, or deleting) each node in a tree data structure, exactly once. S ...
,
* the application of
Monte Carlo method
Monte Carlo methods, or Monte Carlo experiments, are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. The underlying concept is to use randomness to solve problems that might be deter ...
s,
* the application of
pattern matching
In computer science, pattern matching is the act of checking a given sequence of tokens for the presence of the constituents of some pattern. In contrast to pattern recognition, the match usually has to be exact: "either it will or will not be ...
,
* the creation of
knowledge-based systems, and
* the use of
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
.
Minimax tree search
One
traditional AI technique for creating game playing software is to use a
minimax
Minimax (sometimes MinMax, MM or saddle point) is a decision rule used in artificial intelligence, decision theory, game theory, statistics, and philosophy for ''mini''mizing the possible loss for a worst case (''max''imum loss) scenario. Whe ...
tree search
In computer science, tree traversal (also known as tree search and walking the tree) is a form of graph traversal and refers to the process of visiting (e.g. retrieving, updating, or deleting) each node in a tree data structure, exactly once. S ...
. This involves playing out all hypothetical moves on the board up to a certain point, then using an
evaluation function
An evaluation function, also known as a heuristic evaluation function or static evaluation function, is a function used by game-playing computer programs to estimate the value or goodness of a position (usually at a leaf or terminal node) in a g ...
to estimate the value of that position for the current player. The move which leads to the best hypothetical board is selected, and the process is repeated each turn. While tree searches have been very effective in
computer chess
Computer chess includes both hardware (dedicated computers) and software capable of playing chess. Computer chess provides opportunities for players to practice even in the absence of human opponents, and also provides opportunities for analysi ...
, they have seen less success in Computer Go programs. This is partly because it has traditionally been difficult to create an effective evaluation function for a Go board, and partly because the large number of possible moves each side can make each leads to a high
branching factor. This makes this technique very computationally expensive. Because of this, many programs which use search trees extensively can only play on the smaller 9×9 board, rather than full 19×19 ones.
There are several techniques, which can greatly improve the performance of search trees in terms of both speed and memory. Pruning techniques such as
alpha–beta pruning
Alpha–beta pruning is a search algorithm that seeks to decrease the number of nodes that are evaluated by the minimax algorithm in its search tree. It is an adversarial search algorithm used commonly for machine playing of two-player games ...
,
Principal Variation Search
Principal variation search (sometimes equated with the practically identical NegaScout) is a negamax algorithm that can be faster than alpha-beta pruning. Like alpha-beta pruning, NegaScout is a directional search algorithm for computing the mini ...
, and
MTD(f)
MTD(f) is an alpha-beta game tree search algorithm modified to use ‘zero-window’ initial search bounds, and memory (usually a transposition table) to reuse intermediate search results. MTD(f) is a shortened form of MTD(n,f) which stands for M ...
can reduce the effective branching factor without loss of strength. In tactical areas such as life and death, Go is particularly amenable to caching techniques such as
transposition table
{{no footnotes, date=November 2017
A transposition table is a cache of previously seen positions, and associated evaluations, in a game tree generated by a computer game playing program. If a position recurs via a different sequence of moves, the ...
s. These can reduce the amount of repeated effort, especially when combined with an
iterative deepening
In computer science, iterative deepening search or more specifically iterative deepening depth-first search (IDS or IDDFS) is a state space/graph search strategy in which a depth-limited version of depth-first search is run repeatedly with in ...
approach. In order to quickly store a full-sized Go board in a transposition table, a
hashing technique for mathematically summarizing is generally necessary.
Zobrist hashing is very popular in Go programs because it has low collision rates, and can be iteratively updated at each move with just two
XOR
Exclusive or or exclusive disjunction is a logical operation that is true if and only if its arguments differ (one is true, the other is false).
It is symbolized by the prefix operator J and by the infix operators XOR ( or ), EOR, EXOR, , , ...
s, rather than being calculated from scratch. Even using these performance-enhancing techniques, full tree searches on a full-sized board are still prohibitively slow. Searches can be sped up by using large amounts of domain specific pruning techniques, such as not considering moves where your opponent is already strong, and selective extensions like always considering moves next to groups of stones which are
about to be captured. However, both of these options introduce a significant risk of not considering a vital move which would have changed the course of the game.
Results of computer competitions show that pattern matching techniques for choosing a handful of appropriate moves combined with fast localized tactical searches (explained above) were once sufficient to produce a competitive program. For example,
GNU Go was competitive until 2008.
Knowledge-based systems
Human novices often learn from the game records of old games played by master players. AI work in the 1990s often involved attempting to "teach" the AI human-style heuristics of Go knowledge. In 1996, Tim Klinger and David Mechner acknowledged the beginner-level strength of the best AIs and argued that "it is our belief that with better tools for representing and maintaining Go knowledge, it will be possible to develop stronger Go programs."
[Klinger, Tim and Mechner, David. ]
An Architecture for Computer Go
' (1996) They proposed two ways: recognizing common configurations of stones and their positions and concentrating on local battles. In 2001, one paper concluded that "Go programs are still lacking in both quality and quantity of knowledge," and that fixing this would improve Go AI performance.
In theory, the use of expert knowledge would improve Go software. Hundreds of guidelines and rules of thumb for strong play have been formulated by both high-level amateurs and professionals. The programmer's task is to take these
heuristics
A heuristic (; ), or heuristic technique, is any approach to problem solving or self-discovery that employs a practical method that is not guaranteed to be optimal, perfect, or rational, but is nevertheless sufficient for reaching an immediate, ...
, formalize them into computer code, and utilize
pattern matching
In computer science, pattern matching is the act of checking a given sequence of tokens for the presence of the constituents of some pattern. In contrast to pattern recognition, the match usually has to be exact: "either it will or will not be ...
and
pattern recognition
Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphic ...
algorithms to recognize when these rules apply. It is also important to be able to "score" these heuristics so that when they offer conflicting advice, the system has ways to determine which heuristic is more important and applicable to the situation. Most of the relatively successful results come from programmers' individual skills at Go and their personal conjectures about Go, but not from formal mathematical assertions; they are trying to make the computer mimic the way they play Go. Competitive programs around 2001 could contain 50–100 modules that dealt with different aspects and strategies of the game, such as joseki.
Some examples of programs which have relied heavily on expert knowledge are Handtalk (later known as Goemate), The Many Faces of Go, Go Intellect, and Go++, each of which has at some point been considered the world's best Go program. However, these methods ultimately had diminishing returns, and never really advanced past an intermediate level at best on a full-sized board. One particular problem was overall game strategy. Even if an expert system recognizes a pattern and knows how to play a local skirmish, it may miss a looming deeper strategic problem in the future. The result is a program whose strength is less than the sum of its parts; while moves may be good on an individual tactical basis, the program can be tricked and maneuvered into ceding too much in exchange, and find itself in an overall losing position. As the 2001 survey put it, "just one bad move can ruin a good game. Program performance over a full game can be much lower than master level."
Monte-Carlo methods
One major alternative to using hand-coded knowledge and searches is the use of
Monte Carlo method
Monte Carlo methods, or Monte Carlo experiments, are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. The underlying concept is to use randomness to solve problems that might be deter ...
s. This is done by generating a list of potential moves, and for each move playing out thousands of games at random on the resulting board. The move which leads to the best set of random games for the current player is chosen as the best move. The advantage of this technique is that it requires very little domain knowledge or expert input, the trade-off being increased memory and processor requirements. However, because the moves used for evaluation are generated at random it is possible that a move which would be excellent except for one specific opponent response would be mistakenly evaluated as a good move. The result of this are programs which are strong in an overall strategic sense, but are imperfect tactically. This problem can be mitigated by adding some domain knowledge in the move generation and a greater level of search depth on top of the random evolution. Some programs which use Monte-Carlo techniques are Fuego,
The Many Faces of Go v12,
Leela,
MoGo,
Crazy Stone, MyGoFriend,
and Zen.
In 2006, a new search technique, ''upper confidence bounds applied to trees'' (UCT), was developed and applied to many 9x9 Monte-Carlo Go programs with excellent results. UCT uses the results of the ''play outs'' collected so far to guide the search along the more successful lines of play, while still allowing alternative lines to be explored. The UCT technique along with many other optimizations for playing on the larger 19x19 board has led MoGo to become one of the strongest research programs. Successful early applications of UCT methods to 19x19 Go include MoGo, Crazy Stone, and Mango. MoGo won the 2007
Computer Olympiad and won one (out of three) blitz game against Guo Juan, 5th Dan Pro, in the much less complex 9x9 Go. The Many Faces of Go won the 2008
Computer Olympiad after adding UCT search to its traditional knowledge-based engine.
Monte-Carlo based Go engines have a reputation of being much more willing to play ''tenuki'', moves elsewhere on the board, rather than continue a local fight than human players. This was often perceived as a weakness early in these program's existence. That said, this tendency has persisted in AlphaGo's playstyle with dominant results, so this may be more of a "quirk" than a "weakness."
Machine learning
The skill level of knowledge-based systems is closely linked to the knowledge of their programmers and associated domain experts. This limitation has made it difficult to program truly strong AIs. A different path is to use
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
techniques. In these, the only thing that the programmers need to program are the rules and simple scoring algorithms of how to analyze the worth of a position. The software will then automatically generates its own sense of patterns, heuristics, and strategies, in theory.
This is generally done by allowing a
neural network
A neural network is a network or neural circuit, circuit of biological neurons, or, in a modern sense, an artificial neural network, composed of artificial neurons or nodes. Thus, a neural network is either a biological neural network, made up ...
or
genetic algorithm
In computer science and operations research, a genetic algorithm (GA) is a metaheuristic inspired by the process of natural selection that belongs to the larger class of evolutionary algorithms (EA). Genetic algorithms are commonly used to gen ...
to either review a large database of professional games, or play many games against itself or other people or programs. These algorithms are then able to utilize this data as a means of improving their performance. Machine learning techniques can also be used in a less ambitious context to tune specific parameters of programs that rely mainly on other techniques. For example,
Crazy Stone learns move generation patterns from several hundred sample games, using a generalization of the
Elo rating system
The Elo rating system is a method for calculating the relative skill levels of players in zero-sum games such as chess. It is named after its creator Arpad Elo, a Hungarian-American physics professor.
The Elo system was invented as an improved ...
.
The most famous example of this approach is AlphaGo, which proved far more effective than previous AIs. In its first version, it had one layer that analyzed millions of existing positions to determine likely moves to prioritize as worthy of further analysis, and another layer that tried to optimize its own winning chances using the suggested likely moves from the first layer. AlphaGo used Monte Carlo tree search to score the resulting positions. A later version of AlphaGo, AlphaGoZero, eschewed learning from existing Go games, and instead learnt only from playing itself repeatedly. Other earlier programs using neural nets include NeuroGo and WinHonte.
Computer Go and other fields
Computer Go research results are being applied to other similar fields such as
cognitive science,
pattern recognition
Pattern recognition is the automated recognition of patterns and regularities in data. It has applications in statistical data analysis, signal processing, image analysis, information retrieval, bioinformatics, data compression, computer graphic ...
and
machine learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence.
Machine ...
.
[Muhammad, Mohsin]
''Thinking games''
Artificial Intelligence 134 (2002): p150 Combinatorial Game Theory
Combinatorial game theory is a branch of mathematics and theoretical computer science that typically studies sequential games with perfect information. Study has been largely confined to two-player games that have a ''position'' that the playe ...
, a branch of
applied mathematics
Applied mathematics is the application of mathematical methods by different fields such as physics, engineering, medicine, biology, finance, business, computer science, and industry. Thus, applied mathematics is a combination of mathemat ...
, is a topic relevant to computer Go.
John H. Conway suggested applying
surreal number
In mathematics, the surreal number system is a totally ordered proper class containing the real numbers as well as infinite and infinitesimal numbers, respectively larger or smaller in absolute value than any positive real number. The surrea ...
s to analysis of the endgame in Go. This idea has been further developed by
Elwyn R. Berlekamp and
David Wolfe in their book ''Mathematical Go''. Go endgames have been proven to be
PSPACE-hard if the absolute best move must be calculated on an arbitrary mostly filled board. Certain complicated situations such as Triple Ko, Quadruple Ko, Molasses Ko, and Moonshine Life make this problem difficult. (In practice, strong Monte Carlo algorithms can still handle normal Go endgame situations well enough, and the most complicated classes of life-and-death endgame problems are unlikely to come up in a high-level game.)
Various difficult combinatorial problems (any
NP-hard
In computational complexity theory, NP-hardness ( non-deterministic polynomial-time hardness) is the defining property of a class of problems that are informally "at least as hard as the hardest problems in NP". A simple example of an NP-hard pr ...
problem) can be converted to Go-like problems on a sufficiently large board; however, the same is true for other abstract board games, including
chess
Chess is a board game for two players, called White and Black, each controlling an army of chess pieces in their color, with the objective to checkmate the opponent's king. It is sometimes called international chess or Western chess to dist ...
and
minesweeper
A minesweeper is a small warship designed to remove or detonate naval mines. Using various mechanisms intended to counter the threat posed by naval mines, minesweepers keep waterways clear for safe shipping.
History
The earliest known usage of ...
, when suitably generalized to a board of arbitrary size.
NP-complete
In computational complexity theory, a problem is NP-complete when:
# it is a problem for which the correctness of each solution can be verified quickly (namely, in polynomial time) and a brute-force search algorithm can find a solution by tryin ...
problems do not tend in their general case to be easier for unaided humans than for suitably programmed computers: unaided humans are much worse than computers at solving, for example, instances of the
subset sum problem The subset sum problem (SSP) is a decision problem in computer science. In its most general formulation, there is a multiset S of integers and a target-sum T, and the question is to decide whether any subset of the integers sum to precisely T''.'' ...
.
[On page 11: "Crasmaru shows that it is NP-complete to determine the status of certain restricted forms of life-and-death problems in Go." (See the following reference.) ]
List of Go-playing computer programs
*
AlphaGo
AlphaGo is a computer program that plays the board game Go. It was developed by DeepMind Technologies a subsidiary of Google (now Alphabet Inc.). Subsequent versions of AlphaGo became increasingly powerful, including a version that competed u ...
, the first computer program to win in even matches against a 9-dan human Go player
* BaduGI, a program by Jooyoung Lee
*
Crazy Stone, by
Rémi Coulom (sold as Saikyo no Igo in Japan)
*
Darkforest
Darkforest is a computer go program developed by Meta Platforms, based on deep learning techniques using a convolutional neural network. Its updated version Darkfores2 combines the techniques of its predecessor with Monte Carlo tree search. The MC ...
, by
Facebook
Facebook is an online social media and social networking service owned by American company Meta Platforms. Founded in 2004 by Mark Zuckerberg with fellow Harvard College students and roommates Eduardo Saverin, Andrew McCollum, Dustin ...
*
Fine Art
In European academic traditions, fine art is developed primarily for aesthetics or creative expression, distinguishing it from decorative art or applied art, which also has to serve some practical function, such as pottery or most metalwor ...
, by
Tencent
Tencent Holdings Ltd. () is a Chinese multinational technology and entertainment conglomerate and holding company headquartered in Shenzhen. It is one of the highest grossing multimedia companies in the world based on revenue. It is also the wo ...
* Fuego, an
open source
Open source is source code that is made freely available for possible modification and redistribution. Products include permission to use the source code, design documents, or content of the product. The open-source model is a decentralized sof ...
Monte Carlo program
* Goban, a Macintosh Go program by Sen:te (requires free Goban Extensions)
*
GNU Go, an open source classical Go program
*
Leela, the first Monte Carlo program for sale to the public
*
Leela Zero, a reimplementation of the system described in the AlphaGo Zero paper
* The Many Faces of Go, by David Fotland (sold as AI Igo in Japan)
* MyGoFriend, a program by Frank Karger
* MoGo by Sylvain Gelly; parallel version by many people.
* Pachi, an open source Monte Carlo program by Petr Baudiš
* Smart Go, by Anders Kierulf, inventor of the
Smart Game Format
* Steenvreter, by Erik van der Werf
*
Zen
Zen ( zh, t=禪, p=Chán; ja, text= 禅, translit=zen; ko, text=선, translit=Seon; vi, text=Thiền) is a school of Mahayana Buddhism that originated in China during the Tang dynasty, known as the Chan School (''Chánzong'' 禪宗), and ...
, by Yoji Ojima aka Yamato (sold as Tencho no Igo in Japan); parallel version by Hideki Kato.
AlphaGo
AlphaGo
AlphaGo is a computer program that plays the board game Go. It was developed by DeepMind Technologies a subsidiary of Google (now Alphabet Inc.). Subsequent versions of AlphaGo became increasingly powerful, including a version that competed u ...
, developed by
Google DeepMind, made a significant advance by beating a professional human player in October 2015, using techniques that combined
deep learning and
Monte Carlo tree search
In computer science, Monte Carlo tree search (MCTS) is a heuristic search algorithm for some kinds of decision processes, most notably those employed in software that plays board games. In that context MCTS is used to solve the game tree.
MCT ...
.
AlphaGo was significantly more powerful than previous Go programs, and was the first to beat a 9 dan human professional in a game without handicaps on a full-sized board. Work on Go AI since has largely consisted of emulating the techniques used to build AlphaGo, which proved so much stronger than everything else.
Competitions among computer Go programs
Several annual competitions take place between Go computer programs, including Go events at the
Computer Olympiad. Regular, less formal, competitions between programs used to occur on the KGS Go Server (monthly) and the Computer Go Server (continuous).
Many programs are available that allow computer Go engines to play against each other; they almost always communicate via the Go Text Protocol (GTP).
History
The first computer Go competition was sponsored by
Acornsoft
Acornsoft was the software arm of Acorn Computers, and a major publisher of software for the BBC Micro and Acorn Electron. As well as games, it also produced a large number of educational titles, extra computer languages and business and u ...
, and the first regular ones by
USENIX. They ran from 1984 to 1988. These competitions introduced Nemesis, the first competitive Go program from
Bruce Wilcox, and G2.5 by David Fotland, which would later evolve into Cosmos and The Many Faces of Go.
One of the early drivers of computer Go research was the Ing Prize, a relatively large money award sponsored by Taiwanese banker
Ing Chang-ki
Ing Chang-ki (; 23 October 1916 – 27 August 1997) was a Chinese industrialist, Go player, and Go promoter. He was the founder of the Ing Cup. He is also known for promoting the Ing rules of Go. He also promoted one of the first digital ...
, offered annually between 1985 and 2000 at the World Computer Go Congress (or Ing Cup). The winner of this tournament was allowed to challenge young players at a handicap in a short match. If the computer won the match, the prize was awarded and a new prize announced: a larger prize for beating the players at a lesser handicap. The series of Ing prizes was set to expire either 1) in the year 2000 or 2) when a program could beat a 1-dan professional at no handicap for 40,000,000
NT dollars. The last winner was Handtalk in 1997, claiming 250,000 NT dollars for winning an 11-stone handicap match against three 11–13 year old amateur 2–6 dans. At the time the prize expired in 2000, the unclaimed prize was 400,000 NT dollars for winning a nine-stone handicap match.
Many other large regional Go tournaments ("congresses") had an attached computer Go event. The European Go Congress has sponsored a computer tournament since 1987, and the USENIX event evolved into the US/North American Computer Go Championship, held annually from 1988–2000 at the US Go Congress.
Japan started sponsoring computer Go competitions in 1995. The FOST Cup was held annually from 1995 to 1999 in Tokyo. That tournament was supplanted by the Gifu Challenge, which was held annually from 2003 to 2006 in Ogaki, Gifu. The
Computer Go UEC Cup has been held annually since 2007.
Scoring formalization in computer-computer games
When two computers play a game of Go against each other, the ideal is to treat the game in a manner identical to two humans playing while avoiding any intervention from actual humans. However, this can be difficult during end game scoring. The main problem is that Go playing software, which usually communicates using the standardized
Go Text Protocol (GTP), will not always agree with respect to the alive or dead status of stones.
While there is no general way for two different programs to "talk it out" and resolve the conflict, this problem is avoided for the most part by using
Chinese,
Tromp-Taylor, or
American Go Association
The American Go Association (AGA) was founded in 1935, to promote the board game of Go in the United States.
Founded by chess master Edward Lasker and some friends at Chumley's restaurant in New York City, the AGA is one of the oldest Western Go ...
(AGA) rules in which continued play (without penalty) is required until there is no more disagreement on the status of any stones on the board. In practice, such as on the KGS Go Server, the server can mediate a dispute by sending a special GTP command to the two client programs indicating they should continue placing stones until there is no question about the status of any particular group (all dead stones have been captured). The CGOS Go Server usually sees programs resign before a game has even reached the scoring phase, but nevertheless supports a modified version of Tromp-Taylor rules requiring a full play out.
These rule sets mean that a program which was in a winning position at the end of the game under Japanese rules (when both players have passed) could theoretically lose because of poor play in the resolution phase, but this is very unlikely and considered a normal part of the game under all of the area rule sets.
The main drawback to the above system is that some
rule sets (such as the traditional Japanese rules) penalize the players for making these extra moves, precluding the use of additional playout for two computers. Nevertheless, most modern Go Programs support Japanese rules against humans.
Historically, another method for resolving this problem was to have an expert human judge the final board. However, this introduces subjectivity into the results and the risk that the expert would miss something the program saw.
See also
*
Computer chess
Computer chess includes both hardware (dedicated computers) and software capable of playing chess. Computer chess provides opportunities for players to practice even in the absence of human opponents, and also provides opportunities for analysi ...
*
Computer Othello
*
Computer shogi
*
Go Text Protocol
References
Further reading
Co-Evolving a Go-Playing Neural Network written by Alex Lubberts & Risto Miikkulainen, 2001
* ''Computer Game Playing: Theory and Practice'', edited by M.A. Brauner (The Ellis Horwood Series in Artificial Intelligence), Halstead Press, 1983. A collection of computer Go articles. The American Go Journal, vol. 18, No 4. page 6.
SSN 0148-0243A Machine-Learning Approach to Computer Go Jeffrey Bagdis, 2007.
Minimalism in Ubiquitous Interface DesignWren, C. and Reynolds, C. (2004) Personal and Ubiquitous Computing, 8(5), pages 370–374
Video of computer Go vision system in operationshows interaction and users exploring
Joseki and
Fuseki
''Fuseki'' (Japanese: ; ) is the whole board opening in the game of Go.
Characteristics Less systematic
Since each move is typically isolated and unforced (i.e. not a sente move), patterns for play on the whole board have seen much less sy ...
.
Monte-Carlo Go presented by Markus Enzenberger, Computer Go Seminar, University of Alberta, April 2004
Monte-Carlo Go written by B. Bouzy and B. Helmstetter from Scientific Literature Digital Library
Static analysis of life and death in the game of Go written by Ken Chen & Zhixing Chen, 20 February 1999
article describing the techniques underlying Mogo
External links
All systems Goby David A. Mechner (1998), discusses the game where professional Go player
Janice Kim won a game against program
Handtalk after giving a 25-stone handicap.
Computer Goan
Computer Go Programmingpages a
Sensei's LibraryComputer Go bibliographyComputer Go mailing list* Published articles about computer Go o
Ideospheregives current estimate of whether a Go program will be best player in the world
Information on the Go Text Protocolcommonly used for interfacing Go playing engines with graphical clients and internet servers
* The Computer Go Room on th
K Go Server(KGS) for online discussion and running "bots"
an article about two computer Go games played in 1999, one with two computers players, and the other a 29-stone handicap human-computer game
What A Way to Godescribes work at Microsoft Research on building a computer Go player.
Cracking Goby Feng-hsiung Hsu, ''IEEE Spectrum'' magazine (October 2007) – Why it should be possible to build a Go machine stronger than any human player
computer-go-dataset, SGF datasets of 1,645,958 games
{{Go (game)
Game artificial intelligence
Electronic games