Mastering the game of Go with deep neural networks and tree search

### Subjects

* [Computational science](/subjects/computational-science)
* [Computer science](/subjects/computer-science)
* [Reward](/subjects/reward)

Abstract
--------

The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

[Access through your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature16961)

[Buy or subscribe](#access-options)

This is a preview of subscription content, [access via your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature16961)

 if (window.dataLayer) {
 window.dataLayer.push({
 content: { article: { relevantArticlesCount: 3 }}
 })
 }

Relevant articles
-----------------

Open Access articles citing this article.

* ###
[Molecule generation using transformers and policy gradient reinforcement learning](https://doi.org/10.1038/s41598-023-35648-w)

	+ Eyal Mazuz
	+ , Guy Shtar
	+ … Lior Rokach
*Scientific Reports*
Open Access
31 May 2023
* ###
[A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients](https://doi.org/10.1186/s12911-023-02175-7)

	+ Tianlai Lin
	+ , Xinjue Zhang
	+ … Junhui Gao
*BMC Medical Informatics and Decision Making*
Open Access
04 May 2023
* ###
[Efficient evolution of human antibodies from general protein language models](https://doi.org/10.1038/s41587-023-01763-2)

	+ Brian L. Hie
	+ , Varun R. Shanker
	+ … Peter S. Kim
*Nature Biotechnology*
Open Access
24 April 2023

Access options
--------------

[Access through your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature16961)

[Access through your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature16961)

[Change institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature16961)

[Buy or subscribe](#access-options)

/\* style specs start \*/
style{display:none!important}.LiveAreaSection-193358632 \*{align-content:stretch;align-items:stretch;align-self:auto;animation-delay:0s;animation-direction:normal;animation-duration:0s;animation-fill-mode:none;animation-iteration-count:1;animation-name:none;animation-play-state:running;animation-timing-function:ease;azimuth:center;backface-visibility:visible;background-attachment:scroll;background-blend-mode:normal;background-clip:borderBox;background-color:transparent;background-image:none;background-origin:paddingBox;background-position:0 0;background-repeat:repeat;background-size:auto auto;block-size:auto;border-block-end-color:currentcolor;border-block-end-style:none;border-block-end-width:medium;border-block-start-color:currentcolor;border-block-start-style:none;border-block-start-width:medium;border-bottom-color:currentcolor;border-bottom-left-radius:0;border-bottom-right-radius:0;border-bottom-style:none;border-bottom-width:medium;border-collapse:separate;border-image-outset:0s;border-image-repeat:stretch;border-image-slice:100%;border-image-source:none;border-image-width:1;border-inline-end-color:currentcolor;border-inline-end-style:none;border-inline-end-width:medium;border-inline-start-color:currentcolor;border-inline-start-style:none;border-inline-start-width:medium;border-left-color:currentcolor;border-left-style:none;border-left-width:medium;border-right-color:currentcolor;border-right-style:none;border-right-width:medium;border-spacing:0;border-top-color:currentcolor;border-top-left-radius:0;border-top-right-radius:0;border-top-style:none;border-top-width:medium;bottom:auto;box-decoration-break:slice;box-shadow:none;box-sizing:border-box;break-after:auto;break-before:auto;break-inside:auto;caption-side:top;caret-color:auto;clear:none;clip:auto;clip-path:none;color:initial;column-count:auto;column-fill:balance;column-gap:normal;column-rule-color:currentcolor;column-rule-style:none;column-rule-width:medium;column-span:none;column-width:auto;content:normal;counter-increment:none;counter-reset:none;cursor:auto;display:inline;empty-cells:show;filter:none;flex-basis:auto;flex-direction:row;flex-grow:0;flex-shrink:1;flex-wrap:nowrap;float:none;font-family:initial;font-feature-settings:normal;font-kerning:auto;font-language-override:normal;font-size:medium;font-size-adjust:none;font-stretch:normal;font-style:normal;font-synthesis:weight style;font-variant:normal;font-variant-alternates:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-position:normal;font-weight:400;grid-auto-columns:auto;grid-auto-flow:row;grid-auto-rows:auto;grid-column-end:auto;grid-column-gap:0;grid-column-start:auto;grid-row-end:auto;grid-row-gap:0;grid-row-start:auto;grid-template-areas:none;grid-template-columns:none;grid-template-rows:none;height:auto;hyphens:manual;image-orientation:0deg;image-rendering:auto;image-resolution:1dppx;ime-mode:auto;inline-size:auto;isolation:auto;justify-content:flexStart;left:auto;letter-spacing:normal;line-break:auto;line-height:normal;list-style-image:none;list-style-position:outside;list-style-type:disc;margin-block-end:0;margin-block-start:0;margin-bottom:0;margin-inline-end:0;margin-inline-start:0;margin-left:0;margin-right:0;margin-top:0;mask-clip:borderBox;mask-composite:add;mask-image:none;mask-mode:matchSource;mask-origin:borderBox;mask-position:0 0;mask-repeat:repeat;mask-size:auto;mask-type:luminance;max-height:none;max-width:none;min-block-size:0;min-height:0;min-inline-size:0;min-width:0;mix-blend-mode:normal;object-fit:fill;object-position:50% 50%;offset-block-end:auto;offset-block-start:auto;offset-inline-end:auto;offset-inline-start:auto;opacity:1;order:0;orphans:2;outline-color:initial;outline-offset:0;outline-style:none;outline-width:medium;overflow:visible;overflow-wrap:normal;overflow-x:visible;overflow-y:visible;padding-block-end:0;padding-block-start:0;padding-bottom:0;padding-inline-end:0;padding-inline-start:0;padding-left:0;padding-right:0;padding-top:0;page-break-after:auto;page-break-before:auto;page-break-inside:auto;perspective:none;perspective-origin:50% 50%;pointer-events:auto;position:static;quotes:initial;resize:none;right:auto;ruby-align:spaceAround;ruby-merge:separate;ruby-position:over;scroll-behavior:auto;scroll-snap-coordinate:none;scroll-snap-destination:0 0;scroll-snap-points-x:none;scroll-snap-points-y:none;scroll-snap-type:none;shape-image-threshold:0;shape-margin:0;shape-outside:none;tab-size:8;table-layout:auto;text-align:initial;text-align-last:auto;text-combine-upright:none;text-decoration-color:currentcolor;text-decoration-line:none;text-decoration-style:solid;text-emphasis-color:currentcolor;text-emphasis-position:over right;text-emphasis-style:none;text-indent:0;text-justify:auto;text-orientation:mixed;text-overflow:clip;text-rendering:auto;text-shadow:none;text-transform:none;text-underline-position:auto;top:auto;touch-action:auto;transform:none;transform-box:borderBox;transform-origin:50% 50%0;transform-style:flat;transition-delay:0s;transition-duration:0s;transition-property:all;transition-timing-function:ease;vertical-align:baseline;visibility:visible;white-space:normal;widows:2;width:auto;will-change:auto;word-break:normal;word-spacing:normal;word-wrap:normal;writing-mode:horizontalTb;z-index:auto;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;appearance:none;margin:0}.LiveAreaSection-193358632{width:100%}.LiveAreaSection-193358632 .login-option-buybox{display:block;width:100%;font-size:17px;line-height:30px;color:#222;padding-top:30px;font-family:Harding,Palatino,serif}.LiveAreaSection-193358632 .additional-access-options{display:block;font-weight:700;font-size:17px;line-height:30px;color:#222;font-family:Harding,Palatino,serif}.LiveAreaSection-193358632 .additional-login>li:not(:first-child)::before{transform:translateY(-50%);content:"";height:1rem;position:absolute;top:50%;left:0;border-left:2px solid #999}.LiveAreaSection-193358632 .additional-login>li:not(:first-child){padding-left:10px}.LiveAreaSection-193358632 .additional-login>li{display:inline-block;position:relative;vertical-align:middle;padding-right:10px}.BuyBoxSection-683559780{display:flex;flex-wrap:wrap;flex:1;flex-direction:row-reverse;margin:-30px -15px 0}.BuyBoxSection-683559780 .box-inner{width:100%;height:100%}.BuyBoxSection-683559780 .readcube-buybox{background-color:#f3f3f3;flex-shrink:1;flex-grow:1;flex-basis:255px;background-clip:content-box;padding:0 15px;margin-top:30px}.BuyBoxSection-683559780 .subscribe-buybox{background-color:#f3f3f3;flex-shrink:1;flex-grow:4;flex-basis:300px;background-clip:content-box;padding:0 15px;margin-top:30px}.BuyBoxSection-683559780 .subscribe-buybox-nature-plus{background-color:#f3f3f3;flex-shrink:1;flex-grow:4;flex-basis:100%;background-clip:content-box;padding:0 15px;margin-top:30px}.BuyBoxSection-683559780 .title-readcube,.BuyBoxSection-683559780 .title-buybox{display:block;margin:0;margin-right:10%;margin-left:10%;font-size:24px;line-height:32px;color:#222;padding-top:30px;text-align:center;font-family:Harding,Palatino,serif}.BuyBoxSection-683559780 .title-asia-buybox{display:block;margin:0;margin-right:5%;margin-left:5%;font-size:24px;line-height:32px;color:#222;padding-top:30px;text-align:center;font-family:Harding,Palatino,serif}.BuyBoxSection-683559780 .asia-link{color:#069;cursor:pointer;text-decoration:none;font-size:1.05em;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:1.05em6}.BuyBoxSection-683559780 .access-readcube{display:block;margin:0;margin-right:10%;margin-left:10%;font-size:14px;color:#222;padding-top:10px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .access-asia-buybox{display:block;margin:0;margin-right:5%;margin-left:5%;font-size:14px;color:#222;padding-top:10px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .access-buybox{display:block;margin:0;margin-right:10%;margin-left:10%;font-size:14px;color:#222;opacity:.8px;padding-top:10px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .price-buybox{display:block;font-size:30px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;padding-top:30px;text-align:center}.BuyBoxSection-683559780 .price-buybox-to{display:block;font-size:30px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;text-align:center}.BuyBoxSection-683559780 .price-info-text{font-size:16px;padding-right:10px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .price-value{font-size:30px;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .price-per-period{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .price-from{font-size:14px;padding-right:10px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .issue-buybox{display:block;font-size:13px;text-align:center;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:19px}.BuyBoxSection-683559780 .no-price-buybox{display:block;font-size:13px;line-height:18px;text-align:center;padding-right:10%;padding-left:10%;padding-bottom:20px;padding-top:30px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .vat-buybox{display:block;margin-top:5px;margin-right:20%;margin-left:20%;font-size:11px;color:#222;padding-top:10px;padding-bottom:15px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:17px}.BuyBoxSection-683559780 .tax-buybox{display:block;width:100%;color:#222;padding:20px 16px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:NaNpx}.BuyBoxSection-683559780 .button-container{display:flex;padding-right:20px;padding-left:20px;justify-content:center}.BuyBoxSection-683559780 .button-container>\*{flex:1px}.BuyBoxSection-683559780 .button-container>a:hover,.Button-1078489254:hover,.Button-2496381730:hover{text-decoration:none}.BuyBoxSection-683559780 .readcube-button{background:#fff;margin-top:30px}.BuyBoxSection-683559780 .button-asia{background:#069;border:1px solid #069;border-radius:0;cursor:pointer;display:block;padding:9px;outline:0;text-align:center;text-decoration:none;min-width:80px;margin-top:75px}.BuyBoxSection-683559780 .button-label-asia,.ButtonLabel-3296148077,.ButtonLabel-1651148777{display:block;color:#fff;font-size:17px;line-height:20px;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;text-align:center;text-decoration:none;cursor:pointer}.Button-1078489254,.Button-2496381730{background:#069;border:1px solid #069;border-radius:0;cursor:pointer;display:block;padding:9px;outline:0;text-align:center;text-decoration:none;min-width:80px;max-width:320px;margin-top:10px}.Button-1078489254 .readcube-label,.Button-2496381730 .readcube-label{color:#069}
/\* style specs end \*/Subscribe to this journal

Receive 51 print issues and online access

$199.00 per year

only $3.90 per issue

[Learn more](/nature/subscribe)Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

[Learn more](//www.nature.com/articles/nature16961.epdf?no_publisher_access=1&r3_referer=nature)Prices may be subject to local taxes which are calculated during checkout

### Additional access options:

* [Log in](https://idp.nature.com/authorize/natureuser?client_id=grover&redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature16961)
* [Learn about institutional subscriptions](https://www.springernature.com/gp/librarians/licensing/license-options)
* [Read our FAQs](https://support.nature.com/en/support/home)
* [Contact customer support](https://www.springernature.com/gp/contact)

**Figure 1: Neural network training pipeline and architecture.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_Fig1_HTML.jpg)**Figure 2: Strength and accuracy of policy and value networks.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_Fig2_HTML.jpg)**Figure 3: Monte Carlo tree search in AlphaGo.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_Fig3_HTML.jpg)**Figure 4: Tournament evaluation of AlphaGo.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_Fig4_HTML.jpg)**Figure 5: How AlphaGo (black, to play) selected its move in an informal game against Fan Hui.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_Fig5_HTML.jpg)**Figure 6: Games from the match between AlphaGo and the European champion, Fan Hui.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_Fig6_HTML.jpg)

References
----------

1. Allis, L. V. *Searching for Solutions in Games and Artificial Intelligence.* PhD thesis, Univ. Limburg, Maastricht, The Netherlands (1994)
2. van den Herik, H., Uiterwijk, J. W. & van Rijswijck, J. Games solved: now and in the future. *Artif. Intell.* **134**, 277–311 (2002)

[Article](https://doi.org/10.1016%2FS0004-3702%2801%2900152-7)
 [MATH](http://www.emis.de/MATH-item?0982.68143)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Games%20solved%3A%20now%20and%20in%20the%20future&journal=Artif.%20Intell.&doi=10.1016%2FS0004-3702%2801%2900152-7&volume=134&pages=277-311&publication_year=2002&author=van%20den%20Herik%2CH&author=Uiterwijk%2CJW&author=van%20Rijswijck%2CJ)
3. Schaeffer, J. The games computers (and people) play. *Advances in Computers* **52**, 189–266 (2000)

[Article](https://doi.org/10.1016%2FS0065-2458%2800%2980019-4)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20games%20computers%20%28and%20people%29%20play&journal=Advances%20in%20Computers&doi=10.1016%2FS0065-2458%2800%2980019-4&volume=52&pages=189-266&publication_year=2000&author=Schaeffer%2CJ)
4. Campbell, M., Hoane, A. & Hsu, F. Deep Blue. *Artif. Intell.* **134**, 57–83 (2002)

[Article](https://doi.org/10.1016%2FS0004-3702%2801%2900129-1)
 [MATH](http://www.emis.de/MATH-item?0982.68122)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep%20Blue&journal=Artif.%20Intell.&doi=10.1016%2FS0004-3702%2801%2900129-1&volume=134&pages=57-83&publication_year=2002&author=Campbell%2CM&author=Hoane%2CA&author=Hsu%2CF)
5. Schaeffer, J. et al. A world championship caliber checkers program. *Artif. Intell.* **53**, 273–289 (1992)

[Article](https://doi.org/10.1016%2F0004-3702%2892%2990074-8)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=A%20world%20championship%20caliber%20checkers%20program&journal=Artif.%20Intell.&doi=10.1016%2F0004-3702%2892%2990074-8&volume=53&pages=273-289&publication_year=1992&author=Schaeffer%2CJ)
6. Buro, M. From simple features to sophisticated evaluation functions. In *1st International Conference on Computers and Games*, 126–145 (1999)
7. Müller, M. Computer Go. *Artif. Intell.* **134**, 145–179 (2002)

[Article](https://doi.org/10.1016%2FS0004-3702%2801%2900121-7)
 [MATH](http://www.emis.de/MATH-item?0982.68120)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Computer%20Go&journal=Artif.%20Intell.&doi=10.1016%2FS0004-3702%2801%2900121-7&volume=134&pages=145-179&publication_year=2002&author=M%C3%BCller%2CM)
8. Tesauro, G. & Galperin, G. On-line policy improvement using Monte-Carlo search. In *Advances in Neural Information Processing*, 1068–1074 (1996)
9. Sheppard, B. World-championship-caliber Scrabble. *Artif. Intell.* **134**, 241–275 (2002)

[Article](https://doi.org/10.1016%2FS0004-3702%2801%2900166-7)
 [MATH](http://www.emis.de/MATH-item?0982.68126)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=World-championship-caliber%20Scrabble&journal=Artif.%20Intell.&doi=10.1016%2FS0004-3702%2801%2900166-7&volume=134&pages=241-275&publication_year=2002&author=Sheppard%2CB)
10. Bouzy, B. & Helmstetter, B. Monte-Carlo Go developments. In *10th International Conference on Advances in Computer Games*, 159–174 (2003)
11. Coulom, R. Efficient selectivity and backup operators in Monte-Carlo tree search. In *5th International Conference on Computers and Games*, 72–83 (2006)
12. Kocsis, L. & Szepesvári, C. Bandit based Monte-Carlo planning. In *15th European Conference on Machine Learning*, 282–293 (2006)
13. Coulom, R. Computing Elo ratings of move patterns in the game of Go. *ICGA J.* **30**, 198–208 (2007)

[Article](https://doi.org/10.3233%2FICG-2007-30403)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Computing%20Elo%20ratings%20of%20move%20patterns%20in%20the%20game%20of%20Go&journal=ICGA%20J.&doi=10.3233%2FICG-2007-30403&volume=30&pages=198-208&publication_year=2007&author=Coulom%2CR)
14. Baudiš, P. & Gailly, J.-L. Pachi: State of the art open source Go program. In *Advances in Computer Games*, 24–38 (Springer, 2012)
15. Müller, M., Enzenberger, M., Arneson, B. & Segal, R. Fuego – an open-source framework for board games and Go engine based on Monte-Carlo tree search. *IEEE Trans. Comput. Intell. AI in Games* **2**, 259–270 (2010)

[Article](https://doi.org/10.1109%2FTCIAIG.2010.2083662)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Fuego%20%E2%80%93%20an%20open-source%20framework%20for%20board%20games%20and%20Go%20engine%20based%20on%20Monte-Carlo%20tree%20search&journal=IEEE%20Trans.%20Comput.%20Intell.%20AI%20in%20Games&doi=10.1109%2FTCIAIG.2010.2083662&volume=2&pages=259-270&publication_year=2010&author=M%C3%BCller%2CM&author=Enzenberger%2CM&author=Arneson%2CB&author=Segal%2CR)
16. Gelly, S. & Silver, D. Combining online and offline learning in UCT. In *17th International Conference on Machine Learning*, 273–280 (2007)
17. Krizhevsky, A., Sutskever, I. & Hinton, G. ImageNet classification with deep convolutional neural networks. In *Advances in Neural Information Processing Systems*, 1097–1105 (2012)
18. Lawrence, S., Giles, C. L., Tsoi, A. C. & Back, A. D. Face recognition: a convolutional neural-network approach. *IEEE Trans. Neural Netw.* **8**, 98–113 (1997)

[Article](https://doi.org/10.1109%2F72.554195)
 [CAS](/articles/cas-redirect/1:STN:280:DC%2BD1c%2FpvVamug%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=18255614)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Face%20recognition%3A%20a%20convolutional%20neural-network%20approach&journal=IEEE%20Trans.%20Neural%20Netw.&doi=10.1109%2F72.554195&volume=8&pages=98-113&publication_year=1997&author=Lawrence%2CS&author=Giles%2CCL&author=Tsoi%2CAC&author=Back%2CAD)
19. Mnih, V. et al. Human-level control through deep reinforcement learning. *Nature* **518**, 529–533 (2015)

[Article](https://doi.org/10.1038%2Fnature14236)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2015Natur.518..529M)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2MXjsVagur0%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25719670)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Human-level%20control%20through%20deep%20reinforcement%20learning&journal=Nature&doi=10.1038%2Fnature14236&volume=518&pages=529-533&publication_year=2015&author=Mnih%2CV)
20. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. *Nature* **521**, 436–444 (2015)

[Article](https://doi.org/10.1038%2Fnature14539)
 [ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2015Natur.521..436L)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2MXht1WlurzP)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=26017442)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep%20learning&journal=Nature&doi=10.1038%2Fnature14539&volume=521&pages=436-444&publication_year=2015&author=LeCun%2CY&author=Bengio%2CY&author=Hinton%2CG)
21. Stern, D., Herbrich, R. & Graepel, T. Bayesian pattern ranking for move prediction in the game of Go. In *International Conference of Machine Learning*, 873–880 (2006)
22. Sutskever, I. & Nair, V. Mimicking Go experts with convolutional neural networks. In *International Conference on Artificial Neural Networks*, 101–110 (2008)
23. Maddison, C. J., Huang, A., Sutskever, I. & Silver, D. Move evaluation in Go using deep convolutional neural networks. *3rd International Conference on Learning Representations* (2015)
24. Clark, C. & Storkey, A. J. Training deep convolutional neural networks to play go. In *32nd International Conference on Machine Learning*, 1766–1774 (2015)
25. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. *Mach. Learn.* **8**, 229–256 (1992)

[MATH](http://www.emis.de/MATH-item?0772.68076)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Simple%20statistical%20gradient-following%20algorithms%20for%20connectionist%20reinforcement%20learning&journal=Mach.%20Learn.&volume=8&pages=229-256&publication_year=1992&author=Williams%2CRJ)
26. Sutton, R., McAllester, D., Singh, S. & Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. In *Advances in Neural Information Processing Systems*, 1057–1063 (2000)
27. Sutton, R. & Barto, A. *Reinforcement Learning: an Introduction* (MIT Press, 1998)
28. Schraudolph, N. N., Dayan, P. & Sejnowski, T. J. Temporal difference learning of position evaluation in the game of Go. *Adv. Neural Inf. Process. Syst.* **6**, 817–824 (1994)

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Temporal%20difference%20learning%20of%20position%20evaluation%20in%20the%20game%20of%20Go&journal=Adv.%20Neural%20Inf.%20Process.%20Syst.&volume=6&pages=817-824&publication_year=1994&author=Schraudolph%2CNN&author=Dayan%2CP&author=Sejnowski%2CTJ)
29. Enzenberger, M. Evaluation in Go by a neural network using soft segmentation. In *10th Advances in Computer Games Conference*, 97–108 (2003). 267
30. Silver, D., Sutton, R. & Müller, M. Temporal-difference search in computer Go. *Mach. Learn.* **87**, 183–219 (2012)

[Article](https://doi.org/10.1007%2Fs10994-012-5280-0)
 [MathSciNet](http://www.ams.org/mathscinet-getitem?mr=2914011)
 [MATH](http://www.emis.de/MATH-item?1238.91044)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Temporal-difference%20search%20in%20computer%20Go&journal=Mach.%20Learn.&doi=10.1007%2Fs10994-012-5280-0&volume=87&pages=183-219&publication_year=2012&author=Silver%2CD&author=Sutton%2CR&author=M%C3%BCller%2CM)
31. Levinovitz, A. The mystery of Go, the ancient game that computers still can’t win. *Wired Magazine* (2014)
32. Mechner, D. All Systems Go. *The Sciences* **38**, 32–37 (1998)

[Article](https://doi.org/10.1002%2Fj.2326-1951.1998.tb03356.x)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=All%20Systems%20Go&journal=The%20Sciences&doi=10.1002%2Fj.2326-1951.1998.tb03356.x&volume=38&pages=32-37&publication_year=1998&author=Mechner%2CD)
33. Mandziuk, J. Computational intelligence in mind games. In *Challenges for Computational Intelligence*, 407–442 (2007)
34. Berliner, H. A chronology of computer chess and its literature. *Artif. Intell.* **10**, 201–214 (1978)

[Article](https://doi.org/10.1016%2FS0004-3702%2878%2980012-5)
 [MATH](http://www.emis.de/MATH-item?0377.68054)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=A%20chronology%20of%20computer%20chess%20and%20its%20literature&journal=Artif.%20Intell.&doi=10.1016%2FS0004-3702%2878%2980012-5&volume=10&pages=201-214&publication_year=1978&author=Berliner%2CH)
35. Browne, C. et al. A survey of Monte-Carlo tree search methods. *IEEE Trans. Comput. Intell. AI in Games* **4**, 1–43 (2012)

[Article](https://doi.org/10.1109%2FTCIAIG.2012.2186810)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=A%20survey%20of%20Monte-Carlo%20tree%20search%20methods&journal=IEEE%20Trans.%20Comput.%20Intell.%20AI%20in%20Games&doi=10.1109%2FTCIAIG.2012.2186810&volume=4&pages=1-43&publication_year=2012&author=Browne%2CC)
36. Gelly, S. et al. The grand challenge of computer Go: Monte Carlo tree search and extensions. *Commun. ACM* **55**, 106–113 (2012)

[Article](https://doi.org/10.1145%2F2093548.2093574)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20grand%20challenge%20of%20computer%20Go%3A%20Monte%20Carlo%20tree%20search%20and%20extensions&journal=Commun.%20ACM&doi=10.1145%2F2093548.2093574&volume=55&pages=106-113&publication_year=2012&author=Gelly%2CS)
37. Coulom, R. Whole-history rating: A Bayesian rating system for players of time-varying strength. In *International Conference on Computers and Games*, 113–124 (2008)
38. KGS. Rating system math. <http://www.gokgs.com/help/rmath.html>
39. Littman, M. L. Markov games as a framework for multi-agent reinforcement learning. In *11th International Conference on Machine Learning*, 157–163 (1994)
40. Knuth, D. E. & Moore, R. W. An analysis of alpha-beta pruning. *Artif. Intell.* **6**, 293–326 (1975)

[Article](https://doi.org/10.1016%2F0004-3702%2875%2990019-3)
 [MathSciNet](http://www.ams.org/mathscinet-getitem?mr=388881)
 [MATH](http://www.emis.de/MATH-item?0358.68143)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=An%20analysis%20of%20alpha-beta%20pruning&journal=Artif.%20Intell.&doi=10.1016%2F0004-3702%2875%2990019-3&volume=6&pages=293-326&publication_year=1975&author=Knuth%2CDE&author=Moore%2CRW)
41. Sutton, R. Learning to predict by the method of temporal differences. *Mach. Learn.* **3**, 9–44 (1988)

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20to%20predict%20by%20the%20method%20of%20temporal%20differences&journal=Mach.%20Learn.&volume=3&pages=9-44&publication_year=1988&author=Sutton%2CR)
42. Baxter, J., Tridgell, A. & Weaver, L. Learning to play chess using temporal differences. *Mach. Learn.* **40**, 243–263 (2000)

[Article](https://doi.org/10.1023%2FA%3A1007634325138)
 [MATH](http://www.emis.de/MATH-item?0962.68508)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20to%20play%20chess%20using%20temporal%20differences&journal=Mach.%20Learn.&doi=10.1023%2FA%3A1007634325138&volume=40&pages=243-263&publication_year=2000&author=Baxter%2CJ&author=Tridgell%2CA&author=Weaver%2CL)
43. Veness, J., Silver, D., Blair, A. & Uther, W. Bootstrapping from game tree search. In *Advances in Neural Information Processing Systems* (2009)
44. Samuel, A. L. Some studies in machine learning using the game of checkers II - recent progress. *IBM J. Res. Develop.* **11**, 601–617 (1967)

[Article](https://doi.org/10.1147%2Frd.116.0601)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Some%20studies%20in%20machine%20learning%20using%20the%20game%20of%20checkers%20II%20-%20recent%20progress&journal=IBM%20J.%20Res.%20Develop.&doi=10.1147%2Frd.116.0601&volume=11&pages=601-617&publication_year=1967&author=Samuel%2CAL)
45. Schaeffer, J., Hlynka, M. & Jussila, V. Temporal difference learning applied to a high-performance game-playing program. In *17th International Joint Conference on Artificial Intelligence*, 529–534 (2001)
46. Tesauro, G. TD-gammon, a self-teaching backgammon program, achieves master-level play. *Neural Comput.* **6**, 215–219 (1994)

[Article](https://doi.org/10.1162%2Fneco.1994.6.2.215)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=TD-gammon%2C%20a%20self-teaching%20backgammon%20program%2C%20achieves%20master-level%20play&journal=Neural%20Comput.&doi=10.1162%2Fneco.1994.6.2.215&volume=6&pages=215-219&publication_year=1994&author=Tesauro%2CG)
47. Dahl, F. Honte, a Go-playing program using neural nets. In *Machines that learn to play games*, 205–223 (Nova Science, 1999)
48. Rosin, C. D. Multi-armed bandits with episode context. *Ann. Math. Artif. Intell.* **61**, 203–230 (2011)

[Article](https://doi.org/10.1007%2Fs10472-011-9258-6)
 [MathSciNet](http://www.ams.org/mathscinet-getitem?mr=2865077)
 [MATH](http://www.emis.de/MATH-item?1234.68171)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Multi-armed%20bandits%20with%20episode%20context&journal=Ann.%20Math.%20Artif.%20Intell.&doi=10.1007%2Fs10472-011-9258-6&volume=61&pages=203-230&publication_year=2011&author=Rosin%2CCD)
49. Lanctot, M., Winands, M. H. M., Pepels, T. & Sturtevant, N. R. Monte Carlo tree search with heuristic evaluations using implicit minimax backups. In *IEEE Conference on Computational Intelligence and Games*, 1–8 (2014)
50. Gelly, S., Wang, Y., Munos, R. & Teytaud, O. Modification of UCT with patterns in Monte-Carlo Go. *Tech. Rep.* **6062**, INRIA (2006)

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Modification%20of%20UCT%20with%20patterns%20in%20Monte-Carlo%20Go&journal=Tech.%20Rep.&volume=6062&publication_year=2006&author=Gelly%2CS&author=Wang%2CY&author=Munos%2CR&author=Teytaud%2CO)
51. Silver, D. & Tesauro, G. Monte-Carlo simulation balancing. In *26th International Conference on Machine Learning*, **119** (2009)
52. Huang, S.-C., Coulom, R. & Lin, S.-S. Monte-Carlo simulation balancing in practice. In *7th International Conference on Computers and Games*, 81–92 (Springer-Verlag, 2011)
53. Baier, H. & Drake, P. D. The power of forgetting: improving the last-good-reply policy in Monte Carlo Go. *IEEE Trans. Comput. Intell. AI in Games* **2**, 303–309 (2010)

[Article](https://doi.org/10.1109%2FTCIAIG.2010.2100396)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20power%20of%20forgetting%3A%20improving%20the%20last-good-reply%20policy%20in%20Monte%20Carlo%20Go&journal=IEEE%20Trans.%20Comput.%20Intell.%20AI%20in%20Games&doi=10.1109%2FTCIAIG.2010.2100396&volume=2&pages=303-309&publication_year=2010&author=Baier%2CH&author=Drake%2CPD)
54. Huang, S. & Müller, M. Investigating the limits of Monte-Carlo tree search methods in computer Go. In *8th International Conference on Computers and Games*, 39–48 (2013)
55. Segal, R. B. On the scalability of parallel UCT. *Computers and Games* **6515**, 36–47 (2011)

[Article](https://doi.org/10.1007%2F978-3-642-17928-0_4)
 [MathSciNet](http://www.ams.org/mathscinet-getitem?mr=2803987)
 [MATH](http://www.emis.de/MATH-item?1316.68206)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=On%20the%20scalability%20of%20parallel%20UCT&journal=Computers%20and%20Games&doi=10.1007%2F978-3-642-17928-0_4&volume=6515&pages=36-47&publication_year=2011&author=Segal%2CRB)
56. Enzenberger, M. & Müller, M. A lock-free multithreaded Monte-Carlo tree search algorithm. In *12th Advances in Computer Games Conference*, 14–20 (2009)
57. Huang, S.-C., Coulom, R. & Lin, S.-S. Time management for Monte-Carlo tree search applied to the game of Go. In *International Conference on Technologies and Applications of Artificial Intelligence*, 462–466 (2010)
58. Gelly, S. & Silver, D. Monte-Carlo tree search and rapid action value estimation in computer Go. *Artif. Intell.* **175**, 1856–1875 (2011)

[Article](https://doi.org/10.1016%2Fj.artint.2011.03.007)
 [MathSciNet](http://www.ams.org/mathscinet-getitem?mr=2847683)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Monte-Carlo%20tree%20search%20and%20rapid%20action%20value%20estimation%20in%20computer%20Go&journal=Artif.%20Intell.&doi=10.1016%2Fj.artint.2011.03.007&volume=175&pages=1856-1875&publication_year=2011&author=Gelly%2CS&author=Silver%2CD)
59. Baudiš, P. Balancing MCTS by dynamically adjusting the komi value. *ICGA J.* **34**, 131 (2011)

[Article](https://doi.org/10.3233%2FICG-2011-34302)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Balancing%20MCTS%20by%20dynamically%20adjusting%20the%20komi%20value&journal=ICGA%20J.&doi=10.3233%2FICG-2011-34302&volume=34&publication_year=2011&author=Baudi%C5%A1%2CP)
60. Baier, H. & Winands, M. H. Active opening book application for Monte-Carlo tree search in 19×19 Go. In *Benelux Conference on Artificial Intelligence*, 3–10 (2011)
61. Dean, J. et al. Large scale distributed deep networks. In *Advances in Neural Information Processing Systems*, 1223–1231 (2012)
62. Go ratings. <http://www.goratings.org>

[Download references](https://citation-needed.springer.com/v2/references/10.1038/nature16961?format=refman&flavour=references)

Acknowledgements
----------------

We thank Fan Hui for agreeing to play against AlphaGo; T. Manning for refereeing the match; R. Munos and T. Schaul for helpful discussions and advice; A. Cain and M. Cant for work on the visuals; P. Dayan, G. Wayne, D. Kumaran, D. Purves, H. van Hasselt, A. Barreto and G. Ostrovski for reviewing the paper; and the rest of the DeepMind team for their support, ideas and encouragement.

Author information
------------------

Author notes1. David Silver and Aja Huang: These authors contributed equally to this work.

### Authors and Affiliations

1. Google DeepMind, 5 New Street Square, London, EC4A 3TW, UK

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, Nal Kalchbrenner, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel & Demis Hassabis
2. Google, 1600 Amphitheatre Parkway, Mountain View, California, 94043, USA

John Nham & Ilya Sutskever

Authors1. David Silver[View author publications](/search?author=David%20Silver)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=David%20Silver) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22David%20Silver%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
2. Aja Huang[View author publications](/search?author=Aja%20Huang)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Aja%20Huang) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Aja%20Huang%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
3. Chris J. Maddison[View author publications](/search?author=Chris%20J.%20Maddison)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Chris%20J.%20Maddison) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Chris%20J.%20Maddison%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
4. Arthur Guez[View author publications](/search?author=Arthur%20Guez)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Arthur%20Guez) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Arthur%20Guez%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
5. Laurent Sifre[View author publications](/search?author=Laurent%20Sifre)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Laurent%20Sifre) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Laurent%20Sifre%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
6. George van den Driessche[View author publications](/search?author=George%20van%20den%20Driessche)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=George%20van%20den%20Driessche) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22George%20van%20den%20Driessche%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
7. Julian Schrittwieser[View author publications](/search?author=Julian%20Schrittwieser)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Julian%20Schrittwieser) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Julian%20Schrittwieser%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
8. Ioannis Antonoglou[View author publications](/search?author=Ioannis%20Antonoglou)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Ioannis%20Antonoglou) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Ioannis%20Antonoglou%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
9. Veda Panneershelvam[View author publications](/search?author=Veda%20Panneershelvam)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Veda%20Panneershelvam) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Veda%20Panneershelvam%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
10. Marc Lanctot[View author publications](/search?author=Marc%20Lanctot)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Marc%20Lanctot) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Marc%20Lanctot%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
11. Sander Dieleman[View author publications](/search?author=Sander%20Dieleman)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Sander%20Dieleman) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Sander%20Dieleman%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
12. Dominik Grewe[View author publications](/search?author=Dominik%20Grewe)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Dominik%20Grewe) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Dominik%20Grewe%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
13. John Nham[View author publications](/search?author=John%20Nham)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=John%20Nham) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22John%20Nham%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
14. Nal Kalchbrenner[View author publications](/search?author=Nal%20Kalchbrenner)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Nal%20Kalchbrenner) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Nal%20Kalchbrenner%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
15. Ilya Sutskever[View author publications](/search?author=Ilya%20Sutskever)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Ilya%20Sutskever) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Ilya%20Sutskever%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
16. Timothy Lillicrap[View author publications](/search?author=Timothy%20Lillicrap)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Timothy%20Lillicrap) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Timothy%20Lillicrap%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
17. Madeleine Leach[View author publications](/search?author=Madeleine%20Leach)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Madeleine%20Leach) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Madeleine%20Leach%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
18. Koray Kavukcuoglu[View author publications](/search?author=Koray%20Kavukcuoglu)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Koray%20Kavukcuoglu) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Koray%20Kavukcuoglu%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
19. Thore Graepel[View author publications](/search?author=Thore%20Graepel)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Thore%20Graepel) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Thore%20Graepel%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
20. Demis Hassabis[View author publications](/search?author=Demis%20Hassabis)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Demis%20Hassabis) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Demis%20Hassabis%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
### Contributions

A.H., G.v.d.D., J.S., I.A., M.La., A.G., T.G. and D.S. designed and implemented the search in AlphaGo. C.J.M., A.G., L.S., A.H., I.A., V.P., S.D., D.G., N.K., I.S., K.K. and D.S. designed and trained the neural networks in AlphaGo. J.S., J.N., A.H. and D.S. designed and implemented the evaluation framework for AlphaGo. D.S., M.Le., T.L., T.G., K.K. and D.H. managed and advised on the project. D.S., T.G., A.G. and D.H. wrote the paper.

### Corresponding authors

Correspondence to
 [David Silver](mailto:davidsilver@google.com) or [Demis Hassabis](mailto:demishassabis@google.com).

Ethics declarations
-------------------

### Competing interests

The authors declare no competing financial interests.

Extended data figures and tables
--------------------------------

**Extended Data Table 1 Details of match between AlphaGo and Fan Hui**[Full size table](/articles/nature16961/tables/1)**Extended Data Table 2 Input features for neural networks**[Full size table](/articles/nature16961/tables/2)**Extended Data Table 3 Supervised learning results for the policy network**[Full size table](/articles/nature16961/tables/3)**Extended Data Table 4 Input features for rollout and tree policy**[Full size table](/articles/nature16961/tables/4)**Extended Data Table 5 Parameters used by AlphaGo**[Full size table](/articles/nature16961/tables/5)**Extended Data Table 6 Results of a tournament between different Go programs**[Full size table](/articles/nature16961/tables/6)**Extended Data Table 7 Results of a tournament between different variants of AlphaGo**[Full size table](/articles/nature16961/tables/7)**Extended Data Table 8 Results of a tournament between AlphaGo and distributed AlphaGo, testing scalability with hardware**[Full size table](/articles/nature16961/tables/8)**Extended Data Table 9 Cross-table of win rates in per cent between programs**[Full size table](/articles/nature16961/tables/9)**Extended Data Table 10 Cross-table of win rates in per cent between programs in the single-machine scalability study**[Full size table](/articles/nature16961/tables/10)**Extended Data Table 11 Cross-table of win rates in per cent between programs in the distributed scalability study**[Full size table](/articles/nature16961/tables/11)Related audio
-------------

### [Hear from the makers of the AI that mastered Go - and the professional player it beat.](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM111_ESM.mp3)

Supplementary information
-------------------------

### [Supplementary Information](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM110_ESM.zip)

This zipped file contains game records for the 5 formal match games played between AlphaGo and Fan Hui. (ZIP 3 kb)

PowerPoint slides
-----------------

### [PowerPoint slide for Fig. 1](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM104_ESM.ppt)

### [PowerPoint slide for Fig. 2](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM105_ESM.ppt)

### [PowerPoint slide for Fig. 3](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM106_ESM.ppt)

### [PowerPoint slide for Fig. 4](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM107_ESM.ppt)

### [PowerPoint slide for Fig. 5](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM108_ESM.ppt)

### [PowerPoint slide for Fig. 6](https://static-content.springer.com/esm/art%3A10.1038%2Fnature16961/MediaObjects/41586_2016_BFnature16961_MOESM109_ESM.ppt)

Rights and permissions
----------------------

[Reprints and Permissions](https://s100.copyright.com/AppDispatchServlet?title=Mastering%20the%20game%20of%20Go%20with%20deep%20neural%20networks%20and%20tree%20search&author=David%20Silver%20et%20al&contentID=10.1038%2Fnature16961&copyright=Springer%20Nature%20Limited&publication=0028-0836&publicationDate=2016-01-27&publisherName=SpringerNature&orderBeanReset=true)

This article is cited by
------------------------

* ###
[Artificial intelligence-based traffic flow prediction: a comprehensive review](https://doi.org/10.1186/s43067-023-00081-6)

	+ Sayed A. Sayed
	+ Yasser Abdel-Hamid
	+ Hesham Ahmed Hefny*Journal of Electrical Systems and Information Technology* (2023)
* ###
[A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients](https://doi.org/10.1186/s12911-023-02175-7)

	+ Tianlai Lin
	+ Xinjue Zhang
	+ Junhui Gao*BMC Medical Informatics and Decision Making* (2023)
* ###
[Continuous improvement of self-driving cars using dynamic confidence-aware reinforcement learning](https://doi.org/10.1038/s42256-023-00610-y)

	+ Zhong Cao
	+ Kun Jiang
	+ Diange Yang*Nature Machine Intelligence* (2023)
* ###
[BacterAI maps microbial metabolism without prior knowledge](https://doi.org/10.1038/s41564-023-01376-0)

	+ Adam C. Dama
	+ Kevin S. Kim
	+ Paul A. Jensen*Nature Microbiology* (2023)
* ###
[Quantum architecture search via truly proximal policy optimization](https://doi.org/10.1038/s41598-023-32349-2)

	+ Xianchao Zhu
	+ Xiaokai Hou*Scientific Reports* (2023)

Comments
--------

By submitting a comment you agree to abide by our [Terms](/info/tandc.html) and [Community Guidelines](/info/community-guidelines.html). If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

David Silver
Aja Huang
Chris J. Maddison
Arthur Guez
Laurent Sifre
George van den Driessche
Julian Schrittwieser
Ioannis Antonoglou
Veda Panneershelvam
Marc Lanctot
Sander Dieleman
Dominik Grewe
John Nham
Nal Kalchbrenner
Ilya Sutskever
Timothy Lillicrap
Madeleine Leach
Koray Kavukcuoglu
Thore Graepel
Demis Hassabis