Deep Learning

### Subjects

* [Computer science](/subjects/computer-science)
* [Mathematics and computing](/subjects/mathematics-and-computing)

Abstract
--------

Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech.

[Access through your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature14539)

[Buy or subscribe](#access-options)

This is a preview of subscription content, [access via your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature14539)

 if (window.dataLayer) {
 window.dataLayer.push({
 content: { article: { relevantArticlesCount: 3 }}
 })
 }

Relevant articles
-----------------

Open Access articles citing this article.

* ###
[Ensemble of deep learning and machine learning approach for classification of handwritten Hindi numerals](https://doi.org/10.1186/s44147-023-00252-2)

	+ Danveer Rajpal
	+ & Akhil Ranjan Garg
*Journal of Engineering and Applied Science*
Open Access
20 July 2023
* ###
[Reduced-order modeling for stochastic large-scale and time-dependent flow problems using deep spatial and temporal convolutional autoencoders](https://doi.org/10.1186/s40323-023-00244-0)

	+ Azzedine Abdedou
	+ & Azzeddine Soulaimani
*Advanced Modeling and Simulation in Engineering Sciences*
Open Access
19 May 2023
* ###
[A comparison of machine learning methods for ozone pollution prediction](https://doi.org/10.1186/s40537-023-00748-x)

	+ Qilong Pan
	+ , Fouzi Harrou
	+ & Ying Sun
*Journal of Big Data*
Open Access
15 May 2023

Access options
--------------

[Access through your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature14539)

[Access through your institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature14539)

[Change institution](https://wayf.springernature.com?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature14539)

[Buy or subscribe](#access-options)

/\* style specs start \*/
style{display:none!important}.LiveAreaSection-193358632 \*{align-content:stretch;align-items:stretch;align-self:auto;animation-delay:0s;animation-direction:normal;animation-duration:0s;animation-fill-mode:none;animation-iteration-count:1;animation-name:none;animation-play-state:running;animation-timing-function:ease;azimuth:center;backface-visibility:visible;background-attachment:scroll;background-blend-mode:normal;background-clip:borderBox;background-color:transparent;background-image:none;background-origin:paddingBox;background-position:0 0;background-repeat:repeat;background-size:auto auto;block-size:auto;border-block-end-color:currentcolor;border-block-end-style:none;border-block-end-width:medium;border-block-start-color:currentcolor;border-block-start-style:none;border-block-start-width:medium;border-bottom-color:currentcolor;border-bottom-left-radius:0;border-bottom-right-radius:0;border-bottom-style:none;border-bottom-width:medium;border-collapse:separate;border-image-outset:0s;border-image-repeat:stretch;border-image-slice:100%;border-image-source:none;border-image-width:1;border-inline-end-color:currentcolor;border-inline-end-style:none;border-inline-end-width:medium;border-inline-start-color:currentcolor;border-inline-start-style:none;border-inline-start-width:medium;border-left-color:currentcolor;border-left-style:none;border-left-width:medium;border-right-color:currentcolor;border-right-style:none;border-right-width:medium;border-spacing:0;border-top-color:currentcolor;border-top-left-radius:0;border-top-right-radius:0;border-top-style:none;border-top-width:medium;bottom:auto;box-decoration-break:slice;box-shadow:none;box-sizing:border-box;break-after:auto;break-before:auto;break-inside:auto;caption-side:top;caret-color:auto;clear:none;clip:auto;clip-path:none;color:initial;column-count:auto;column-fill:balance;column-gap:normal;column-rule-color:currentcolor;column-rule-style:none;column-rule-width:medium;column-span:none;column-width:auto;content:normal;counter-increment:none;counter-reset:none;cursor:auto;display:inline;empty-cells:show;filter:none;flex-basis:auto;flex-direction:row;flex-grow:0;flex-shrink:1;flex-wrap:nowrap;float:none;font-family:initial;font-feature-settings:normal;font-kerning:auto;font-language-override:normal;font-size:medium;font-size-adjust:none;font-stretch:normal;font-style:normal;font-synthesis:weight style;font-variant:normal;font-variant-alternates:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-position:normal;font-weight:400;grid-auto-columns:auto;grid-auto-flow:row;grid-auto-rows:auto;grid-column-end:auto;grid-column-gap:0;grid-column-start:auto;grid-row-end:auto;grid-row-gap:0;grid-row-start:auto;grid-template-areas:none;grid-template-columns:none;grid-template-rows:none;height:auto;hyphens:manual;image-orientation:0deg;image-rendering:auto;image-resolution:1dppx;ime-mode:auto;inline-size:auto;isolation:auto;justify-content:flexStart;left:auto;letter-spacing:normal;line-break:auto;line-height:normal;list-style-image:none;list-style-position:outside;list-style-type:disc;margin-block-end:0;margin-block-start:0;margin-bottom:0;margin-inline-end:0;margin-inline-start:0;margin-left:0;margin-right:0;margin-top:0;mask-clip:borderBox;mask-composite:add;mask-image:none;mask-mode:matchSource;mask-origin:borderBox;mask-position:0 0;mask-repeat:repeat;mask-size:auto;mask-type:luminance;max-height:none;max-width:none;min-block-size:0;min-height:0;min-inline-size:0;min-width:0;mix-blend-mode:normal;object-fit:fill;object-position:50% 50%;offset-block-end:auto;offset-block-start:auto;offset-inline-end:auto;offset-inline-start:auto;opacity:1;order:0;orphans:2;outline-color:initial;outline-offset:0;outline-style:none;outline-width:medium;overflow:visible;overflow-wrap:normal;overflow-x:visible;overflow-y:visible;padding-block-end:0;padding-block-start:0;padding-bottom:0;padding-inline-end:0;padding-inline-start:0;padding-left:0;padding-right:0;padding-top:0;page-break-after:auto;page-break-before:auto;page-break-inside:auto;perspective:none;perspective-origin:50% 50%;pointer-events:auto;position:static;quotes:initial;resize:none;right:auto;ruby-align:spaceAround;ruby-merge:separate;ruby-position:over;scroll-behavior:auto;scroll-snap-coordinate:none;scroll-snap-destination:0 0;scroll-snap-points-x:none;scroll-snap-points-y:none;scroll-snap-type:none;shape-image-threshold:0;shape-margin:0;shape-outside:none;tab-size:8;table-layout:auto;text-align:initial;text-align-last:auto;text-combine-upright:none;text-decoration-color:currentcolor;text-decoration-line:none;text-decoration-style:solid;text-emphasis-color:currentcolor;text-emphasis-position:over right;text-emphasis-style:none;text-indent:0;text-justify:auto;text-orientation:mixed;text-overflow:clip;text-rendering:auto;text-shadow:none;text-transform:none;text-underline-position:auto;top:auto;touch-action:auto;transform:none;transform-box:borderBox;transform-origin:50% 50%0;transform-style:flat;transition-delay:0s;transition-duration:0s;transition-property:all;transition-timing-function:ease;vertical-align:baseline;visibility:visible;white-space:normal;widows:2;width:auto;will-change:auto;word-break:normal;word-spacing:normal;word-wrap:normal;writing-mode:horizontalTb;z-index:auto;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;appearance:none;margin:0}.LiveAreaSection-193358632{width:100%}.LiveAreaSection-193358632 .login-option-buybox{display:block;width:100%;font-size:17px;line-height:30px;color:#222;padding-top:30px;font-family:Harding,Palatino,serif}.LiveAreaSection-193358632 .additional-access-options{display:block;font-weight:700;font-size:17px;line-height:30px;color:#222;font-family:Harding,Palatino,serif}.LiveAreaSection-193358632 .additional-login>li:not(:first-child)::before{transform:translateY(-50%);content:"";height:1rem;position:absolute;top:50%;left:0;border-left:2px solid #999}.LiveAreaSection-193358632 .additional-login>li:not(:first-child){padding-left:10px}.LiveAreaSection-193358632 .additional-login>li{display:inline-block;position:relative;vertical-align:middle;padding-right:10px}.BuyBoxSection-683559780{display:flex;flex-wrap:wrap;flex:1;flex-direction:row-reverse;margin:-30px -15px 0}.BuyBoxSection-683559780 .box-inner{width:100%;height:100%}.BuyBoxSection-683559780 .readcube-buybox{background-color:#f3f3f3;flex-shrink:1;flex-grow:1;flex-basis:255px;background-clip:content-box;padding:0 15px;margin-top:30px}.BuyBoxSection-683559780 .subscribe-buybox{background-color:#f3f3f3;flex-shrink:1;flex-grow:4;flex-basis:300px;background-clip:content-box;padding:0 15px;margin-top:30px}.BuyBoxSection-683559780 .subscribe-buybox-nature-plus{background-color:#f3f3f3;flex-shrink:1;flex-grow:4;flex-basis:100%;background-clip:content-box;padding:0 15px;margin-top:30px}.BuyBoxSection-683559780 .title-readcube,.BuyBoxSection-683559780 .title-buybox{display:block;margin:0;margin-right:10%;margin-left:10%;font-size:24px;line-height:32px;color:#222;padding-top:30px;text-align:center;font-family:Harding,Palatino,serif}.BuyBoxSection-683559780 .title-asia-buybox{display:block;margin:0;margin-right:5%;margin-left:5%;font-size:24px;line-height:32px;color:#222;padding-top:30px;text-align:center;font-family:Harding,Palatino,serif}.BuyBoxSection-683559780 .asia-link{color:#069;cursor:pointer;text-decoration:none;font-size:1.05em;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:1.05em6}.BuyBoxSection-683559780 .access-readcube{display:block;margin:0;margin-right:10%;margin-left:10%;font-size:14px;color:#222;padding-top:10px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .access-asia-buybox{display:block;margin:0;margin-right:5%;margin-left:5%;font-size:14px;color:#222;padding-top:10px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .access-buybox{display:block;margin:0;margin-right:10%;margin-left:10%;font-size:14px;color:#222;opacity:.8px;padding-top:10px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .price-buybox{display:block;font-size:30px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;padding-top:30px;text-align:center}.BuyBoxSection-683559780 .price-buybox-to{display:block;font-size:30px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;text-align:center}.BuyBoxSection-683559780 .price-info-text{font-size:16px;padding-right:10px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .price-value{font-size:30px;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .price-per-period{font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .price-from{font-size:14px;padding-right:10px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:20px}.BuyBoxSection-683559780 .issue-buybox{display:block;font-size:13px;text-align:center;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:19px}.BuyBoxSection-683559780 .no-price-buybox{display:block;font-size:13px;line-height:18px;text-align:center;padding-right:10%;padding-left:10%;padding-bottom:20px;padding-top:30px;color:#222;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif}.BuyBoxSection-683559780 .vat-buybox{display:block;margin-top:5px;margin-right:20%;margin-left:20%;font-size:11px;color:#222;padding-top:10px;padding-bottom:15px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:17px}.BuyBoxSection-683559780 .tax-buybox{display:block;width:100%;color:#222;padding:20px 16px;text-align:center;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;line-height:NaNpx}.BuyBoxSection-683559780 .button-container{display:flex;padding-right:20px;padding-left:20px;justify-content:center}.BuyBoxSection-683559780 .button-container>\*{flex:1px}.BuyBoxSection-683559780 .button-container>a:hover,.Button-1078489254:hover,.Button-2496381730:hover{text-decoration:none}.BuyBoxSection-683559780 .readcube-button{background:#fff;margin-top:30px}.BuyBoxSection-683559780 .button-asia{background:#069;border:1px solid #069;border-radius:0;cursor:pointer;display:block;padding:9px;outline:0;text-align:center;text-decoration:none;min-width:80px;margin-top:75px}.BuyBoxSection-683559780 .button-label-asia,.ButtonLabel-3296148077,.ButtonLabel-1651148777{display:block;color:#fff;font-size:17px;line-height:20px;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;text-align:center;text-decoration:none;cursor:pointer}.Button-1078489254,.Button-2496381730{background:#069;border:1px solid #069;border-radius:0;cursor:pointer;display:block;padding:9px;outline:0;text-align:center;text-decoration:none;min-width:80px;max-width:320px;margin-top:10px}.Button-1078489254 .readcube-label,.Button-2496381730 .readcube-label{color:#069}
/\* style specs end \*/Subscribe to this journal

Receive 51 print issues and online access

$199.00 per year

only $3.90 per issue

[Learn more](/nature/subscribe)Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

[Learn more](//www.nature.com/articles/nature14539.epdf?no_publisher_access=1&r3_referer=nature)Prices may be subject to local taxes which are calculated during checkout

### Additional access options:

* [Log in](https://idp.nature.com/authorize/natureuser?client_id=grover&redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fnature14539)
* [Learn about institutional subscriptions](https://www.springernature.com/gp/librarians/licensing/license-options)
* [Read our FAQs](https://support.nature.com/en/support/home)
* [Contact customer support](https://www.springernature.com/gp/contact)

**Figure 1: Multilayer neural networks and backpropagation.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature14539/MediaObjects/41586_2015_BFnature14539_Fig1_HTML.jpg)**Figure 2: Inside a convolutional network.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature14539/MediaObjects/41586_2015_BFnature14539_Fig2_HTML.jpg)**Figure 3: From image to text.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature14539/MediaObjects/41586_2015_BFnature14539_Fig3_HTML.jpg)**Figure 4: Visualizing the learned word vectors.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature14539/MediaObjects/41586_2015_BFnature14539_Fig4_HTML.jpg)**Figure 5: A recurrent neural network and the unfolding in time of the computation involved in its forward computation.**![](//media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fnature14539/MediaObjects/41586_2015_BFnature14539_Fig5_HTML.jpg)

References
----------

1. Krizhevsky, A., Sutskever, I. & Hinton, G. ImageNet classification with deep convolutional neural networks. In *Proc. Advances in Neural Information Processing Systems 25* 1090–1098 (2012). **This report was a breakthrough that used convolutional nets to almost halve the error rate for object recognition, and precipitated the rapid adoption of deep learning by the computer vision community.**

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2025&pages=1090-1098&publication_year=2012&author=Krizhevsky%2CA&author=Sutskever%2CI&author=Hinton%2CG)
2. Farabet, C., Couprie, C., Najman, L. & LeCun, Y. Learning hierarchical features for scene labeling. *IEEE Trans. Pattern Anal. Mach. Intell.* **35**, 1915–1929 (2013).

[PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=23787344)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20hierarchical%20features%20for%20scene%20labeling&journal=IEEE%20Trans.%20Pattern%20Anal.%20Mach.%20Intell.&volume=35&pages=1915-1929&publication_year=2013&author=Farabet%2CC&author=Couprie%2CC&author=Najman%2CL&author=LeCun%2CY)
3. Tompson, J., Jain, A., LeCun, Y. & Bregler, C. Joint training of a convolutional network and a graphical model for human pose estimation. In *Proc. Advances in Neural Information Processing Systems 27* 1799–1807 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2027&pages=1799-1807&publication_year=2014&author=Tompson%2CJ&author=Jain%2CA&author=LeCun%2CY&author=Bregler%2CC)
4. Szegedy, C. et al. Going deeper with convolutions. Preprint at <http://arxiv.org/abs/1409.4842> (2014).
5. Mikolov, T., Deoras, A., Povey, D., Burget, L. & Cernocky, J. Strategies for training large scale neural network language models. In *Proc. Automatic Speech Recognition and Understanding* 196–201 (2011).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Automatic%20Speech%20Recognition%20and%20Understanding&pages=196-201&publication_year=2011&author=Mikolov%2CT&author=Deoras%2CA&author=Povey%2CD&author=Burget%2CL&author=Cernocky%2CJ)
6. Hinton, G. et al. Deep neural networks for acoustic modeling in speech recognition. *IEEE Signal Processing Magazine* **29**, 82–97 (2012). **This joint paper from the major speech recognition laboratories, summarizing the breakthrough achieved with deep learning on the task of phonetic classification for automatic speech recognition, was the first major industrial application of deep learning.**

[ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2012ISPM...29...82H)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep%20neural%20networks%20for%20acoustic%20modeling%20in%20speech%20recognition&journal=IEEE%20Signal%20Processing%20Magazine&volume=29&pages=82-97&publication_year=2012&author=Hinton%2CG)
7. Sainath, T., Mohamed, A.-R., Kingsbury, B. & Ramabhadran, B. Deep convolutional neural networks for LVCSR. In *Proc. Acoustics, Speech and Signal Processing* 8614–8618 (2013).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Acoustics%2C%20Speech%20and%20Signal%20Processing&pages=8614-8618&publication_year=2013&author=Sainath%2CT&author=Mohamed%2CA-R&author=Kingsbury%2CB&author=Ramabhadran%2CB)
8. Ma, J., Sheridan, R. P., Liaw, A., Dahl, G. E. & Svetnik, V. Deep neural nets as a method for quantitative structure-activity relationships. *J. Chem. Inf. Model.* **55**, 263–274 (2015).

[CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2MXhvFGns70%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25635324)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep%20neural%20nets%20as%20a%20method%20for%20quantitative%20structure-activity%20relationships&journal=J.%20Chem.%20Inf.%20Model.&volume=55&pages=263-274&publication_year=2015&author=Ma%2CJ&author=Sheridan%2CRP&author=Liaw%2CA&author=Dahl%2CGE&author=Svetnik%2CV)
9. Ciodaro, T., Deva, D., de Seixas, J. & Damazio, D. Online particle detection with neural networks based on topological calorimetry information. *J. Phys. Conf. Series* **368**, 012030 (2012).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Online%20particle%20detection%20with%20neural%20networks%20based%20on%20topological%20calorimetry%20information&journal=J.%20Phys.%20Conf.%20Series&volume=368&publication_year=2012&author=Ciodaro%2CT&author=Deva%2CD&author=de%20Seixas%2CJ&author=Damazio%2CD)
10. Kaggle. Higgs boson machine learning challenge. *Kaggle* <https://www.kaggle.com/c/higgs-boson> (2014).
11. Helmstaedter, M. et al. Connectomic reconstruction of the inner plexiform layer in the mouse retina. *Nature* **500**, 168–174 (2013).

[ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2013Natur.500..168H)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC3sXht1emtrnJ)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=23925239)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Connectomic%20reconstruction%20of%20the%20inner%20plexiform%20layer%20in%20the%20mouse%20retina&journal=Nature&volume=500&pages=168-174&publication_year=2013&author=Helmstaedter%2CM)
12. Leung, M. K., Xiong, H. Y., Lee, L. J. & Frey, B. J. Deep learning of the tissue-regulated splicing code. *Bioinformatics* **30**, i121–i129 (2014).

[CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2cXpvFCqsLg%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=24931975)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4058935)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep%20learning%20of%20the%20tissue-regulated%20splicing%20code&journal=Bioinformatics&volume=30&pages=i121-i129&publication_year=2014&author=Leung%2CMK&author=Xiong%2CHY&author=Lee%2CLJ&author=Frey%2CBJ)
13. Xiong, H. Y. et al. The human splicing code reveals new insights into the genetic determinants of disease. *Science* **347**, 6218 (2015).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20human%20splicing%20code%20reveals%20new%20insights%20into%20the%20genetic%20determinants%20of%20disease&journal=Science&volume=347&publication_year=2015&author=Xiong%2CHY)
14. Collobert, R., et al. Natural language processing (almost) from scratch. *J. Mach. Learn. Res.* **12**, 2493–2537 (2011).

[MATH](http://www.emis.de/MATH-item?1280.68161)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Natural%20language%20processing%20%28almost%29%20from%20scratch&journal=J.%20Mach.%20Learn.%20Res.&volume=12&pages=2493-2537&publication_year=2011&author=Collobert%2CR)
15. Bordes, A., Chopra, S. & Weston, J. Question answering with subgraph embeddings. In *Proc. Empirical Methods in Natural Language Processing* <http://arxiv.org/abs/1406.3676v3> (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Empirical%20Methods%20in%20Natural%20Language%20Processing&publication_year=2014&author=Bordes%2CA&author=Chopra%2CS&author=Weston%2CJ)
16. Jean, S., Cho, K., Memisevic, R. & Bengio, Y. On using very large target vocabulary for neural machine translation. In *Proc. ACL-IJCNLP* <http://arxiv.org/abs/1412.2007> (2015).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20ACL-IJCNLP&publication_year=2015&author=Jean%2CS&author=Cho%2CK&author=Memisevic%2CR&author=Bengio%2CY)
17. Sutskever, I. Vinyals, O. & Le. Q. V. Sequence to sequence learning with neural networks. In *Proc. Advances in Neural Information Processing Systems 27* 3104–3112 (2014). **This paper showed state-of-the-art machine translation results with the architecture introduced in ref. 72, with a recurrent network trained to read a sentence in one language, produce a semantic representation of its meaning, and generate a translation in another language.**

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2027&pages=3104-3112&publication_year=2014&author=Sutskever%2CI&author=Vinyals%2CO&author=Le%2CQV)
18. Bottou, L. & Bousquet, O. The tradeoffs of large scale learning. In *Proc. Advances in Neural Information Processing Systems 20* 161–168 (2007).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2020&pages=161-168&publication_year=2007&author=Bottou%2CL&author=Bousquet%2CO)
19. Duda, R. O. & Hart, P. E. *Pattern Classification and Scene Analysis* (Wiley, 1973).

[MATH](http://www.emis.de/MATH-item?0277.68056)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Pattern%20Classification%20and%20Scene%20Analysis&publication_year=1973&author=Duda%2CRO&author=Hart%2CPE)
20. Schölkopf, B. & Smola, A. *Learning with Kernels* (MIT Press, 2002).

[MATH](http://www.emis.de/MATH-item?1019.68094)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20with%20Kernels&publication_year=2002&author=Sch%C3%B6lkopf%2CB&author=Smola%2CA)
21. Bengio, Y., Delalleau, O. & Le Roux, N. The curse of highly variable functions for local kernel machines. In *Proc. Advances in Neural Information Processing Systems 18* 107–114 (2005).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2018&pages=107-114&publication_year=2005&author=Bengio%2CY&author=Delalleau%2CO&author=Le%20Roux%2CN)
22. Selfridge, O. G. Pandemonium: a paradigm for learning in mechanisation of thought processes. In *Proc. Symposium on Mechanisation of Thought Processes* 513–526 (1958).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Symposium%20on%20Mechanisation%20of%20Thought%20Processes&pages=513-526&publication_year=1958&author=Selfridge%2COG)
23. Rosenblatt, F. *The Perceptron — A Perceiving and Recognizing Automaton*. Tech. Rep. 85-460-1 (Cornell Aeronautical Laboratory, 1957).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20Perceptron%20%E2%80%94%20A%20Perceiving%20and%20Recognizing%20Automaton&publication_year=1957&author=Rosenblatt%2CF)
24. Werbos, P. *Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences*. PhD thesis, Harvard Univ. (1974).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Beyond%20Regression%3A%20New%20Tools%20for%20Prediction%20and%20Analysis%20in%20the%20Behavioral%20Sciences&publication_year=1974&author=Werbos%2CP)
25. Parker, D. B. *Learning Logic* Report TR–47 (MIT Press, 1985).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20Logic&publication_year=1985&author=Parker%2CDB)
26. LeCun, Y. Une procédure d'apprentissage pour Réseau à seuil assymétrique in *Cognitiva 85: a la Frontière de l'Intelligence Artificielle, des Sciences de la Connaissance et des Neurosciences* [in French] 599–604 (1985).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Cognitiva%2085%3A%20a%20la%20Fronti%C3%A8re%20de%20l%27Intelligence%20Artificielle%2C%20des%20Sciences%20de%20la%20Connaissance%20et%20des%20Neurosciences&pages=599-604&publication_year=1985&author=LeCun%2CY)
27. Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. *Nature* **323**, 533–536 (1986).

[ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=1986Natur.323..533R)
 [MATH](http://www.emis.de/MATH-item?1369.68284)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20representations%20by%20back-propagating%20errors&journal=Nature&volume=323&pages=533-536&publication_year=1986&author=Rumelhart%2CDE&author=Hinton%2CGE&author=Williams%2CRJ)
28. Glorot, X., Bordes, A. & Bengio. Y. Deep sparse rectifier neural networks. In *Proc. 14th International Conference on Artificial Intelligence and Statistics* 315–323 (2011). **This paper showed that supervised training of very deep neural networks is much faster if the hidden layers are composed of ReLU.**

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%2014th%20International%20Conference%20on%20Artificial%20Intelligence%20and%20Statistics&pages=315-323&publication_year=2011&author=Glorot%2CX&author=Bordes%2CA&author=Bengio%2CY)
29. Dauphin, Y. et al. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In *Proc. Advances in Neural Information Processing Systems 27* 2933–2941 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2027&pages=2933-2941&publication_year=2014&author=Dauphin%2CY)
30. Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B. & LeCun, Y. The loss surface of multilayer networks. In *Proc. Conference on AI and Statistics* <http://arxiv.org/abs/1412.0233> (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Conference%20on%20AI%20and%20Statistics&publication_year=2014&author=Choromanska%2CA&author=Henaff%2CM&author=Mathieu%2CM&author=Arous%2CGB&author=LeCun%2CY)
31. Hinton, G. E. What kind of graphical model is the brain? In *Proc. 19th International Joint Conference on Artificial intelligence* 1765–1775 (2005).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%2019th%20International%20Joint%20Conference%20on%20Artificial%20intelligence&pages=1765-1775&publication_year=2005&author=Hinton%2CGE)
32. Hinton, G. E., Osindero, S. & Teh, Y.-W. A fast learning algorithm for deep belief nets. *Neural Comp.* **18**, 1527–1554 (2006). **This paper introduced a novel and effective way of training very deep neural networks by pre-training one hidden layer at a time using the unsupervised learning procedure for restricted Boltzmann machines.**

[MathSciNet](http://www.ams.org/mathscinet-getitem?mr=2224485)
 [MATH](http://www.emis.de/MATH-item?1106.68094)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=A%20fast%20learning%20algorithm%20for%20deep%20belief%20nets&journal=Neural%20Comp.&volume=18&pages=1527-1554&publication_year=2006&author=Hinton%2CGE&author=Osindero%2CS&author=Teh%2CY-W)
33. Bengio, Y., Lamblin, P., Popovici, D. & Larochelle, H. Greedy layer-wise training of deep networks. In *Proc*. *Advances in Neural Information Processing Systems 19* 153–160 (2006). **This report demonstrated that the unsupervised pre-training method introduced in ref. 32 significantly improves performance on test data and generalizes the method to other unsupervised representation-learning techniques, such as auto-encoders.**
34. Ranzato, M., Poultney, C., Chopra, S. & LeCun, Y. Efficient learning of sparse representations with an energy-based model. In *Proc. Advances in Neural Information Processing Systems 19* 1137–1144 (2006).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2019&pages=1137-1144&publication_year=2006&author=Ranzato%2CM&author=Poultney%2CC&author=Chopra%2CS&author=LeCun%2CY)
35. Hinton, G. E. & Salakhutdinov, R. Reducing the dimensionality of data with neural networks. *Science* **313**, 504–507 (2006).

[ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2006Sci...313..504H)
 [MathSciNet](http://www.ams.org/mathscinet-getitem?mr=2242509)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BD28Xnt1KntrY%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=16873662)
 [MATH](http://www.emis.de/MATH-item?1226.68083)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Reducing%20the%20dimensionality%20of%20data%20with%20neural%20networks&journal=Science&volume=313&pages=504-507&publication_year=2006&author=Hinton%2CGE&author=Salakhutdinov%2CR)
36. Sermanet, P., Kavukcuoglu, K., Chintala, S. & LeCun, Y. Pedestrian detection with unsupervised multi-stage feature learning. In *Proc. International Conference on Computer Vision and Pattern Recognition* <http://arxiv.org/abs/1212.0142> (2013).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Computer%20Vision%20and%20Pattern%20Recognition&publication_year=2013&author=Sermanet%2CP&author=Kavukcuoglu%2CK&author=Chintala%2CS&author=LeCun%2CY)
37. Raina, R., Madhavan, A. & Ng, A. Y. Large-scale deep unsupervised learning using graphics processors. In *Proc. 26th Annual International Conference on Machine Learning* 873–880 (2009).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%2026th%20Annual%20International%20Conference%20on%20Machine%20Learning&pages=873-880&publication_year=2009&author=Raina%2CR&author=Madhavan%2CA&author=Ng%2CAY)
38. Mohamed, A.-R., Dahl, G. E. & Hinton, G. Acoustic modeling using deep belief networks. *IEEE Trans. Audio Speech Lang. Process.* **20**, 14–22 (2012).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Acoustic%20modeling%20using%20deep%20belief%20networks&journal=IEEE%20Trans.%20Audio%20Speech%20Lang.%20Process.&volume=20&pages=14-22&publication_year=2012&author=Mohamed%2CA-R&author=Dahl%2CGE&author=Hinton%2CG)
39. Dahl, G. E., Yu, D., Deng, L. & Acero, A. Context-dependent pre-trained deep neural networks for large vocabulary speech recognition. *IEEE Trans. Audio Speech Lang. Process.* **20**, 33–42 (2012).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Context-dependent%20pre-trained%20deep%20neural%20networks%20for%20large%20vocabulary%20speech%20recognition&journal=IEEE%20Trans.%20Audio%20Speech%20Lang.%20Process.&volume=20&pages=33-42&publication_year=2012&author=Dahl%2CGE&author=Yu%2CD&author=Deng%2CL&author=Acero%2CA)
40. Bengio, Y., Courville, A. & Vincent, P. Representation learning: a review and new perspectives. *IEEE Trans. Pattern Anal. Machine Intell.* **35**, 1798–1828 (2013).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Representation%20learning%3A%20a%20review%20and%20new%20perspectives&journal=IEEE%20Trans.%20Pattern%20Anal.%20Machine%20Intell.&volume=35&pages=1798-1828&publication_year=2013&author=Bengio%2CY&author=Courville%2CA&author=Vincent%2CP)
41. LeCun, Y. et al. Handwritten digit recognition with a back-propagation network. In *Proc. Advances in Neural Information Processing Systems* 396–404 (1990). **This is the first paper on convolutional networks trained by backpropagation for the task of classifying low-resolution images of handwritten digits.**

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems&pages=396-404&publication_year=1990&author=LeCun%2CY)
42. LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. *Proc. IEEE* **86**, 2278–2324 (1998). **This overview paper on the principles of end-to-end training of modular systems such as deep neural networks using gradient-based optimization showed how neural networks (and in particular convolutional nets) can be combined with search or inference mechanisms to model complex outputs that are interdependent, such as sequences of characters associated with the content of a document.**

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Gradient-based%20learning%20applied%20to%20document%20recognition&journal=Proc.%20IEEE&volume=86&pages=2278-2324&publication_year=1998&author=LeCun%2CY&author=Bottou%2CL&author=Bengio%2CY&author=Haffner%2CP)
43. Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction, and functional architecture in the cat's visual cortex. *J. Physiol.* **160**, 106–154 (1962).

[CAS](/articles/cas-redirect/1:STN:280:DyaF38%2FltFSisA%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=14449617)
 [PubMed Central](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1359523)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Receptive%20fields%2C%20binocular%20interaction%2C%20and%20functional%20architecture%20in%20the%20cat%27s%20visual%20cortex&journal=J.%20Physiol.&volume=160&pages=106-154&publication_year=1962&author=Hubel%2CDH&author=Wiesel%2CTN)
44. Felleman, D. J. & Essen, D. C. V. Distributed hierarchical processing in the primate cerebral cortex. *Cereb. Cortex* **1**, 1–47 (1991).

[CAS](/articles/cas-redirect/1:STN:280:DyaK38zltlGmsg%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=1822724)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Distributed%20hierarchical%20processing%20in%20the%20primate%20cerebral%20cortex&journal=Cereb.%20Cortex&volume=1&pages=1-47&publication_year=1991&author=Felleman%2CDJ&author=Essen%2CDCV)
45. Cadieu, C. F. et al. Deep neural networks rival the representation of primate it cortex for core visual object recognition. *PLoS Comp. Biol.* **10**, e1003963 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Deep%20neural%20networks%20rival%20the%20representation%20of%20primate%20it%20cortex%20for%20core%20visual%20object%20recognition&journal=PLoS%20Comp.%20Biol.&volume=10&publication_year=2014&author=Cadieu%2CCF)
46. Fukushima, K. & Miyake, S. Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position. *Pattern Recognition* **15**, 455–469 (1982).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Neocognitron%3A%20a%20new%20algorithm%20for%20pattern%20recognition%20tolerant%20of%20deformations%20and%20shifts%20in%20position&journal=Pattern%20Recognition&volume=15&pages=455-469&publication_year=1982&author=Fukushima%2CK&author=Miyake%2CS)
47. Waibel, A., Hanazawa, T., Hinton, G. E., Shikano, K. & Lang, K. Phoneme recognition using time-delay neural networks. *IEEE Trans. Acoustics Speech Signal Process.* **37**, 328–339 (1989).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Phoneme%20recognition%20using%20time-delay%20neural%20networks&journal=IEEE%20Trans.%20Acoustics%20Speech%20Signal%20Process.&volume=37&pages=328-339&publication_year=1989&author=Waibel%2CA&author=Hanazawa%2CT&author=Hinton%2CGE&author=Shikano%2CK&author=Lang%2CK)
48. Bottou, L., Fogelman-Soulié, F., Blanchet, P. & Lienard, J. Experiments with time delay networks and dynamic time warping for speaker independent isolated digit recognition. In *Proc. EuroSpeech 89* 537–540 (1989).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20EuroSpeech%2089&pages=537-540&publication_year=1989&author=Bottou%2CL&author=Fogelman-Souli%C3%A9%2CF&author=Blanchet%2CP&author=Lienard%2CJ)
49. Simard, D., Steinkraus, P. Y. & Platt, J. C. Best practices for convolutional neural networks. In *Proc. Document Analysis and Recognition* 958–963 (2003).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Document%20Analysis%20and%20Recognition&pages=958-963&publication_year=2003&author=Simard%2CD&author=Steinkraus%2CPY&author=Platt%2CJC)
50. Vaillant, R., Monrocq, C. & LeCun, Y. Original approach for the localisation of objects in images. In *Proc. Vision, Image, and Signal Processing* **141**, 245–250 (1994).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Vision%2C%20Image%2C%20and%20Signal%20Processing&pages=245-250&publication_year=1994&author=Vaillant%2CR&author=Monrocq%2CC&author=LeCun%2CY)
51. Nowlan, S. & Platt, J. in *Neural Information Processing Systems* 901–908 (1995).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Neural%20Information%20Processing%20Systems&pages=901-908&publication_year=1995&author=Nowlan%2CS&author=Platt%2CJ)
52. Lawrence, S., Giles, C. L., Tsoi, A. C. & Back, A. D. Face recognition: a convolutional neural-network approach. *IEEE Trans. Neural Networks* **8**, 98–113 (1997).

[CAS](/articles/cas-redirect/1:STN:280:DC%2BD1c%2FpvVamug%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=18255614)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Face%20recognition%3A%20a%20convolutional%20neural-network%20approach&journal=IEEE%20Trans.%20Neural%20Networks&volume=8&pages=98-113&publication_year=1997&author=Lawrence%2CS&author=Giles%2CCL&author=Tsoi%2CAC&author=Back%2CAD)
53. Ciresan, D., Meier, U. Masci, J. & Schmidhuber, J. Multi-column deep neural network for traffic sign classification. *Neural Networks* **32**, 333–338 (2012).

[PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=22386783)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Multi-column%20deep%20neural%20network%20for%20traffic%20sign%20classification&journal=Neural%20Networks&volume=32&pages=333-338&publication_year=2012&author=Ciresan%2CD&author=Meier%2CU&author=Masci%2CJ&author=Schmidhuber%2CJ)
54. Ning, F. et al. Toward automatic phenotyping of developing embryos from videos. *IEEE Trans. Image Process.* **14**, 1360–1371 (2005).

[ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2005ITIP...14.1360N)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=16190471)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Toward%20automatic%20phenotyping%20of%20developing%20embryos%20from%20videos&journal=IEEE%20Trans.%20Image%20Process.&volume=14&pages=1360-1371&publication_year=2005&author=Ning%2CF)
55. Turaga, S. C. et al. Convolutional networks can learn to generate affinity graphs for image segmentation. *Neural Comput.* **22**, 511–538 (2010).

[PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=19922289)
 [MATH](http://www.emis.de/MATH-item?1183.92055)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Convolutional%20networks%20can%20learn%20to%20generate%20affinity%20graphs%20for%20image%20segmentation&journal=Neural%20Comput.&volume=22&pages=511-538&publication_year=2010&author=Turaga%2CSC)
56. Garcia, C. & Delakis, M. Convolutional face finder: a neural architecture for fast and robust face detection. *IEEE Trans. Pattern Anal. Machine Intell.* **26**, 1408–1423 (2004).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Convolutional%20face%20finder%3A%20a%20neural%20architecture%20for%20fast%20and%20robust%20face%20detection&journal=IEEE%20Trans.%20Pattern%20Anal.%20Machine%20Intell.&volume=26&pages=1408-1423&publication_year=2004&author=Garcia%2CC&author=Delakis%2CM)
57. Osadchy, M., LeCun, Y. & Miller, M. Synergistic face detection and pose estimation with energy-based models. *J. Mach. Learn. Res.* **8**, 1197–1215 (2007).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Synergistic%20face%20detection%20and%20pose%20estimation%20with%20energy-based%20models&journal=J.%20Mach.%20Learn.%20Res.&volume=8&pages=1197-1215&publication_year=2007&author=Osadchy%2CM&author=LeCun%2CY&author=Miller%2CM)
58. Tompson, J., Goroshin, R. R., Jain, A., LeCun, Y. Y. & Bregler, C. C. Efficient object localization using convolutional networks. In *Proc. Conference on Computer Vision and Pattern Recognition* <http://arxiv.org/abs/1411.4280> (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Conference%20on%20Computer%20Vision%20and%20Pattern%20Recognition&publication_year=2014&author=Tompson%2CJ&author=Goroshin%2CRR&author=Jain%2CA&author=LeCun%2CYY&author=Bregler%2CCC)
59. Taigman, Y., Yang, M., Ranzato, M. & Wolf, L. Deepface: closing the gap to human-level performance in face verification. In *Proc. Conference on Computer Vision and Pattern Recognition* 1701–1708 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Conference%20on%20Computer%20Vision%20and%20Pattern%20Recognition&pages=1701-1708&publication_year=2014&author=Taigman%2CY&author=Yang%2CM&author=Ranzato%2CM&author=Wolf%2CL)
60. Hadsell, R. et al. Learning long-range vision for autonomous off-road driving. *J. Field Robot.* **26**, 120–144 (2009).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20long-range%20vision%20for%20autonomous%20off-road%20driving&journal=J.%20Field%20Robot.&volume=26&pages=120-144&publication_year=2009&author=Hadsell%2CR)
61. Farabet, C., Couprie, C., Najman, L. & LeCun, Y. Scene parsing with multiscale feature learning, purity trees, and optimal covers. In *Proc. International Conference on Machine Learning* <http://arxiv.org/abs/1202.2160> (2012).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Machine%20Learning&publication_year=2012&author=Farabet%2CC&author=Couprie%2CC&author=Najman%2CL&author=LeCun%2CY)
62. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. *J. Machine Learning Res.* **15**, 1929–1958 (2014).

[MathSciNet](http://www.ams.org/mathscinet-getitem?mr=3231592)
 [MATH](http://www.emis.de/MATH-item?1318.68153)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Dropout%3A%20a%20simple%20way%20to%20prevent%20neural%20networks%20from%20overfitting&journal=J.%20Machine%20Learning%20Res.&volume=15&pages=1929-1958&publication_year=2014&author=Srivastava%2CN&author=Hinton%2CG&author=Krizhevsky%2CA&author=Sutskever%2CI&author=Salakhutdinov%2CR)
63. Sermanet, P. et al. Overfeat: integrated recognition, localization and detection using convolutional networks. In *Proc. International Conference on Learning Representations* <http://arxiv.org/abs/1312.6229> (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Learning%20Representations&publication_year=2014&author=Sermanet%2CP)
64. Girshick, R., Donahue, J., Darrell, T. & Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In *Proc. Conference on Computer Vision and Pattern Recognition* 580–587 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Conference%20on%20Computer%20Vision%20and%20Pattern%20Recognition&pages=580-587&publication_year=2014&author=Girshick%2CR&author=Donahue%2CJ&author=Darrell%2CT&author=Malik%2CJ)
65. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. In *Proc. International Conference on Learning Representations* <http://arxiv.org/abs/1409.1556> (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Learning%20Representations&publication_year=2014&author=Simonyan%2CK&author=Zisserman%2CA)
66. Boser, B., Sackinger, E., Bromley, J., LeCun, Y. & Jackel, L. An analog neural network processor with programmable topology. *J. Solid State Circuits* **26**, 2017–2025 (1991).

[ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=1991IJSSC..26.2017B)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=An%20analog%20neural%20network%20processor%20with%20programmable%20topology&journal=J.%20Solid%20State%20Circuits&volume=26&pages=2017-2025&publication_year=1991&author=Boser%2CB&author=Sackinger%2CE&author=Bromley%2CJ&author=LeCun%2CY&author=Jackel%2CL)
67. Farabet, C. et al. Large-scale FPGA-based convolutional networks. In *Scaling up Machine Learning: Parallel and Distributed Approaches* (eds Bekkerman, R., Bilenko, M. & Langford, J.) 399–419 (Cambridge Univ. Press, 2011).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Scaling%20up%20Machine%20Learning%3A%20Parallel%20and%20Distributed%20Approaches&pages=399-419&publication_year=2011&author=Farabet%2CC)
68. Bengio, Y. *Learning Deep Architectures for AI* (Now, 2009).

[MATH](http://www.emis.de/MATH-item?1192.68503)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20Deep%20Architectures%20for%20AI&publication_year=2009&author=Bengio%2CY)
69. Montufar, G. & Morton, J. When does a mixture of products contain a product of mixtures? *J. Discrete Math.* **29**, 321–347 (2014).

[MathSciNet](http://www.ams.org/mathscinet-getitem?mr=3310972)
 [MATH](http://www.emis.de/MATH-item?1328.68170)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=When%20does%20a%20mixture%20of%20products%20contain%20a%20product%20of%20mixtures%3F&journal=J.%20Discrete%20Math.&volume=29&pages=321-347&publication_year=2014&author=Montufar%2CG&author=Morton%2CJ)
70. Montufar, G. F., Pascanu, R., Cho, K. & Bengio, Y. On the number of linear regions of deep neural networks. In *Proc. Advances in Neural Information Processing Systems 27* 2924–2932 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2027&pages=2924-2932&publication_year=2014&author=Montufar%2CGF&author=Pascanu%2CR&author=Cho%2CK&author=Bengio%2CY)
71. Bengio, Y., Ducharme, R. & Vincent, P. A neural probabilistic language model. In *Proc. Advances in Neural Information Processing Systems 13* 932–938 (2001). **This paper introduced neural language models, which learn to convert a word symbol into a word vector or word embedding composed of learned semantic features in order to predict the next word in a sequence.**

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2013&pages=932-938&publication_year=2001&author=Bengio%2CY&author=Ducharme%2CR&author=Vincent%2CP)
72. Cho, K. et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In *Proc. Conference on Empirical Methods in Natural Language Processing* 1724–1734 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Conference%20on%20Empirical%20Methods%20in%20Natural%20Language%20Processing&pages=1724-1734&publication_year=2014&author=Cho%2CK)
73. Schwenk, H. Continuous space language models. *Computer Speech Lang.* **21**, 492–518 (2007).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Continuous%20space%20language%20models&journal=Computer%20Speech%20Lang.&volume=21&pages=492-518&publication_year=2007&author=Schwenk%2CH)
74. Socher, R., Lin, C. C-Y., Manning, C. & Ng, A. Y. Parsing natural scenes and natural language with recursive neural networks. In *Proc. International Conference on Machine Learning* 129–136 (2011).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Machine%20Learning&pages=129-136&publication_year=2011&author=Socher%2CR&author=Lin%2CCC-Y&author=Manning%2CC&author=Ng%2CAY)
75. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. & Dean, J. Distributed representations of words and phrases and their compositionality. In *Proc. Advances in Neural Information Processing Systems 26* 3111–3119 (2013).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2026&pages=3111-3119&publication_year=2013&author=Mikolov%2CT&author=Sutskever%2CI&author=Chen%2CK&author=Corrado%2CG&author=Dean%2CJ)
76. Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In *Proc. International Conference on Learning Representations* <http://arxiv.org/abs/1409.0473> (2015).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Learning%20Representations&publication_year=2015&author=Bahdanau%2CD&author=Cho%2CK&author=Bengio%2CY)
77. Hochreiter, S. Untersuchungen zu dynamischen neuronalen Netzen [in German] Diploma thesis, T.U. Münich (1991).
78. Bengio, Y., Simard, P. & Frasconi, P. Learning long-term dependencies with gradient descent is difficult. *IEEE Trans. Neural Networks* **5**, 157–166 (1994).

[CAS](/articles/cas-redirect/1:STN:280:DC%2BD1c7gvFansQ%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=18267787)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Learning%20long-term%20dependencies%20with%20gradient%20descent%20is%20difficult&journal=IEEE%20Trans.%20Neural%20Networks&volume=5&pages=157-166&publication_year=1994&author=Bengio%2CY&author=Simard%2CP&author=Frasconi%2CP)
79. Hochreiter, S. & Schmidhuber, J. Long short-term memory. *Neural Comput.* **9**, 1735–1780 (1997). **This paper introduced LSTM recurrent networks, which have become a crucial ingredient in recent advances with recurrent networks because they are good at learning long-range dependencies.**

[CAS](/articles/cas-redirect/1:STN:280:DyaK1c%2FhvVahsQ%3D%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=9377276)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Long%20short-term%20memory&journal=Neural%20Comput.&volume=9&pages=1735-1780&publication_year=1997&author=Hochreiter%2CS&author=Schmidhuber%2CJ)
80. ElHihi, S. & Bengio, Y. Hierarchical recurrent neural networks for long-term dependencies. In *Proc. Advances in Neural Information Processing Systems 8* <http://papers.nips.cc/paper/1102-hierarchical-recurrent-neural-networks-for-long-term-dependencies> (1995).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%208&publication_year=1995&author=ElHihi%2CS&author=Bengio%2CY)
81. Sutskever, I. *Training Recurrent Neural Networks*. PhD thesis, Univ. Toronto (2012).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Training%20Recurrent%20Neural%20Networks&publication_year=2012&author=Sutskever%2CI)
82. Pascanu, R., Mikolov, T. & Bengio, Y. On the difficulty of training recurrent neural networks. In *Proc. 30th International Conference on Machine Learning* 1310–1318 (2013).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%2030th%20International%20Conference%20on%20Machine%20Learning&pages=1310-1318&publication_year=2013&author=Pascanu%2CR&author=Mikolov%2CT&author=Bengio%2CY)
83. Sutskever, I., Martens, J. & Hinton, G. E. Generating text with recurrent neural networks. In *Proc. 28th International Conference on Machine Learning* 1017–1024 (2011).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%2028th%20International%20Conference%20on%20Machine%20Learning&pages=1017-1024&publication_year=2011&author=Sutskever%2CI&author=Martens%2CJ&author=Hinton%2CGE)
84. Lakoff, G. & Johnson, M. *Metaphors We Live By* (Univ. Chicago Press, 2008).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Metaphors%20We%20Live%20By&publication_year=2008&author=Lakoff%2CG&author=Johnson%2CM)
85. Rogers, T. T. & McClelland, J. L. *Semantic Cognition: A Parallel Distributed Processing Approach* (MIT Press, 2004).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Semantic%20Cognition%3A%20A%20Parallel%20Distributed%20Processing%20Approach&publication_year=2004&author=Rogers%2CTT&author=McClelland%2CJL)
86. Xu, K. et al. Show, attend and tell: Neural image caption generation with visual attention. In *Proc. International Conference on Learning Representations* <http://arxiv.org/abs/1502.03044> (2015).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Learning%20Representations&publication_year=2015&author=Xu%2CK)
87. Graves, A., Mohamed, A.-R. & Hinton, G. Speech recognition with deep recurrent neural networks. In *Proc. International Conference on Acoustics, Speech and Signal Processing* 6645–6649 (2013).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Acoustics%2C%20Speech%20and%20Signal%20Processing&pages=6645-6649&publication_year=2013&author=Graves%2CA&author=Mohamed%2CA-R&author=Hinton%2CG)
88. Graves, A., Wayne, G. & Danihelka, I. Neural Turing machines. <http://arxiv.org/abs/1410.5401> (2014).
89. Weston, J. Chopra, S. & Bordes, A. Memory networks. <http://arxiv.org/abs/1410.3916> (2014).
90. Weston, J., Bordes, A., Chopra, S. & Mikolov, T. Towards AI-complete question answering: a set of prerequisite toy tasks. <http://arxiv.org/abs/1502.05698> (2015).
91. Hinton, G. E., Dayan, P., Frey, B. J. & Neal, R. M. The wake-sleep algorithm for unsupervised neural networks. *Science* **268**, 1558–1161 (1995).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=The%20wake-sleep%20algorithm%20for%20unsupervised%20neural%20networks&journal=Science&volume=268&pages=1558-1161&publication_year=1995&author=Hinton%2CGE&author=Dayan%2CP&author=Frey%2CBJ&author=Neal%2CRM)
92. Salakhutdinov, R. & Hinton, G. Deep Boltzmann machines. In *Proc. International Conference on Artificial Intelligence and Statistics* 448–455 (2009).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Artificial%20Intelligence%20and%20Statistics&pages=448-455&publication_year=2009&author=Salakhutdinov%2CR&author=Hinton%2CG)
93. Vincent, P., Larochelle, H., Bengio, Y. & Manzagol, P.-A. Extracting and composing robust features with denoising autoencoders. In *Proc. 25th International Conference on Machine Learning* 1096–1103 (2008).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%2025th%20International%20Conference%20on%20Machine%20Learning&pages=1096-1103&publication_year=2008&author=Vincent%2CP&author=Larochelle%2CH&author=Bengio%2CY&author=Manzagol%2CP-A)
94. Kavukcuoglu, K. et al. Learning convolutional feature hierarchies for visual recognition. In *Proc. Advances in Neural Information Processing Systems 23* 1090–1098 (2010).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2023&pages=1090-1098&publication_year=2010&author=Kavukcuoglu%2CK)
95. Gregor, K. & LeCun, Y. Learning fast approximations of sparse coding. In *Proc. International Conference on Machine Learning* 399–406 (2010).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Machine%20Learning&pages=399-406&publication_year=2010&author=Gregor%2CK&author=LeCun%2CY)
96. Ranzato, M., Mnih, V., Susskind, J. M. & Hinton, G. E. Modeling natural images using gated MRFs. *IEEE Trans. Pattern Anal. Machine Intell.* **35**, 2206–2222 (2013).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Modeling%20natural%20images%20using%20gated%20MRFs&journal=IEEE%20Trans.%20Pattern%20Anal.%20Machine%20Intell.&volume=35&pages=2206-2222&publication_year=2013&author=Ranzato%2CM&author=Mnih%2CV&author=Susskind%2CJM&author=Hinton%2CGE)
97. Bengio, Y., Thibodeau-Laufer, E., Alain, G. & Yosinski, J. Deep generative stochastic networks trainable by backprop. In *Proc. 31st International Conference on Machine Learning* 226–234 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%2031st%20International%20Conference%20on%20Machine%20Learning&pages=226-234&publication_year=2014&author=Bengio%2CY&author=Thibodeau-Laufer%2CE&author=Alain%2CG&author=Yosinski%2CJ)
98. Kingma, D., Rezende, D., Mohamed, S. & Welling, M. Semi-supervised learning with deep generative models. In *Proc. Advances in Neural Information Processing Systems 27* 3581–3589 (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20Advances%20in%20Neural%20Information%20Processing%20Systems%2027&pages=3581-3589&publication_year=2014&author=Kingma%2CD&author=Rezende%2CD&author=Mohamed%2CS&author=Welling%2CM)
99. Ba, J., Mnih, V. & Kavukcuoglu, K. Multiple object recognition with visual attention. In *Proc. International Conference on Learning Representations* <http://arxiv.org/abs/1412.7755> (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Learning%20Representations&publication_year=2014&author=Ba%2CJ&author=Mnih%2CV&author=Kavukcuoglu%2CK)
100. Mnih, V. et al. Human-level control through deep reinforcement learning. *Nature* **518**, 529–533 (2015).

[ADS](http://adsabs.harvard.edu/cgi-bin/nph-data_query?link_type=ABSTRACT&bibcode=2015Natur.518..529M)
 [CAS](/articles/cas-redirect/1:CAS:528:DC%2BC2MXjsVagur0%3D)
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Abstract&list_uids=25719670)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Human-level%20control%20through%20deep%20reinforcement%20learning&journal=Nature&volume=518&pages=529-533&publication_year=2015&author=Mnih%2CV)
101. Bottou, L. From machine learning to machine reasoning. *Mach. Learn.* **94**, 133–149 (2014).

[MathSciNet](http://www.ams.org/mathscinet-getitem?mr=3149132)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=From%20machine%20learning%20to%20machine%20reasoning&journal=Mach.%20Learn.&volume=94&pages=133-149&publication_year=2014&author=Bottou%2CL)
102. Vinyals, O., Toshev, A., Bengio, S. & Erhan, D. Show and tell: a neural image caption generator. In *Proc. International Conference on Machine Learning* <http://arxiv.org/abs/1502.03044> (2014).

[Google Scholar](http://scholar.google.com/scholar_lookup?&title=Proc.%20International%20Conference%20on%20Machine%20Learning&publication_year=2014&author=Vinyals%2CO&author=Toshev%2CA&author=Bengio%2CS&author=Erhan%2CD)
103. van der Maaten, L. & Hinton, G. E. Visualizing data using t-SNE. *J. Mach. Learn.Research* **9**, 2579–2605 (2008).

[MATH](http://www.emis.de/MATH-item?1225.68219)
 [Google Scholar](http://scholar.google.com/scholar_lookup?&title=Visualizing%20data%20using%20t-SNE&journal=J.%20Mach.%20Learn.Research&volume=9&pages=2579-2605&publication_year=2008&author=van%20der%20Maaten%2CL&author=Hinton%2CGE)

[Download references](https://citation-needed.springer.com/v2/references/10.1038/nature14539?format=refman&flavour=references)

Acknowledgements
----------------

The authors would like to thank the Natural Sciences and Engineering Research Council of Canada, the Canadian Institute For Advanced Research (CIFAR), the National Science Foundation and Office of Naval Research for support. Y.L. and Y.B. are CIFAR fellows.

Author information
------------------

### Authors and Affiliations

1. Facebook AI Research, 770 Broadway, New York, 10003, New York, USA

Yann LeCun
2. New York University, 715 Broadway, New York, 10003, New York, USA

Yann LeCun
3. Department of Computer Science and Operations Research Université de Montréal, Pavillon André-Aisenstadt, PO Box 6128 Centre-Ville STN, Montréal, H3C 3J7, Quebec, Canada

Yoshua Bengio
4. Google, 1600 Amphitheatre Parkway, Mountain View, 94043, California, USA

Geoffrey Hinton
5. Department of Computer Science, University of Toronto, 6 King's College Road, Toronto, M5S 3G4, Ontario, Canada

Geoffrey Hinton

Authors1. Yann LeCun[View author publications](/search?author=Yann%20LeCun)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Yann%20LeCun) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Yann%20LeCun%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
2. Yoshua Bengio[View author publications](/search?author=Yoshua%20Bengio)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Yoshua%20Bengio) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Yoshua%20Bengio%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
3. Geoffrey Hinton[View author publications](/search?author=Geoffrey%20Hinton)You can also search for this author in
 [PubMed](http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=search&term=Geoffrey%20Hinton) [Google Scholar](http://scholar.google.co.uk/scholar?as_q=&num=10&btnG=Search+Scholar&as_epq=&as_oq=&as_eq=&as_occt=any&as_sauthors=%22Geoffrey%20Hinton%22&as_publication=&as_ylo=&as_yhi=&as_allsubj=all&hl=en)
### Corresponding author

Correspondence to
 [Yann LeCun](mailto:yann@cs.nyu.edu).

Ethics declarations
-------------------

### Competing interests

The authors declare no competing financial interests.

Additional information
----------------------

Reprints and permissions information is available at [www.nature.com/reprints](http://www.nature.com/reprints).

Rights and permissions
----------------------

[Reprints and Permissions](https://s100.copyright.com/AppDispatchServlet?title=Deep%20learning&author=Yann%20LeCun%20et%20al&contentID=10.1038%2Fnature14539&copyright=Springer%20Nature%20Limited&publication=0028-0836&publicationDate=2015-05-27&publisherName=SpringerNature&orderBeanReset=true)

This article is cited by
------------------------

* ###
[A deep learning model for the classification of atrial fibrillation in critically ill patients](https://doi.org/10.1186/s40635-022-00490-3)

	+ Brian Chen
	+ David M. Maslove
	+ Stephanie Sibley*Intensive Care Medicine Experimental* (2023)
* ###
[Quantitative prediction of radiographic progression in patients with axial spondyloarthritis using neural network model in a real-world setting](https://doi.org/10.1186/s13075-023-03050-6)

	+ In-Woon Baek
	+ Seung Min Jung
	+ Ki-Jo Kim*Arthritis Research & Therapy* (2023)
* ###
[A comparison of machine learning methods for ozone pollution prediction](https://doi.org/10.1186/s40537-023-00748-x)

	+ Qilong Pan
	+ Fouzi Harrou
	+ Ying Sun*Journal of Big Data* (2023)
* ###
[Prospects and applications of on-chip lasers](https://doi.org/10.1186/s43593-022-00027-x)

	+ Zhican Zhou
	+ Xiangpeng Ou
	+ John E. Bowers*eLight* (2023)
* ###
[Ensemble of deep learning and machine learning approach for classification of handwritten Hindi numerals](https://doi.org/10.1186/s44147-023-00252-2)

	+ Danveer Rajpal
	+ Akhil Ranjan Garg*Journal of Engineering and Applied Science* (2023)

Comments
--------

By submitting a comment you agree to abide by our [Terms](/info/tandc.html) and [Community Guidelines](/info/community-guidelines.html). If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Yann LeCun
Yoshua Bengio
Geoffrey Hinton