Neocortical plasticity: an unsupervised cake but no free lunch
Extracted source text · arXiv 1911.08584
\documentclass{article}
% if you need to pass options to natbib, use, e.g.:
% \PassOptionsToPackage{numbers, compress}{natbib}
% before loading neurips_2019
% ready for submission
%\usepackage[final]{neurips_2019}
\usepackage[preprint]{neurips_2019}
\usepackage{graphicx}
% to compile a preprint version, e.g., for submission to arXiv, add add the
% [preprint] option:
% \usepackage[preprint]{neurips_2019}
% to compile a camera-ready version, add the [final] option, e.g.:
% \usepackage[final]{neurips_2019}
% to avoid loading the natbib package, add option nonatbib:
% \usepackage[nonatbib]{neurips_2019}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url} % simple URL typesetting
\usepackage{booktabs} % professional-quality tables
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
\title{Neocortical plasticity: an unsupervised cake but no free lunch}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to LaTeX to determine where to break the
% lines. Using \AND forces a line break at that point. So, if LaTeX puts 3 of 4
% authors names on the first line, and the last on the second line, try using
% \AND instead of \And before the third author name.
\author{%
Eilif B.~Muller*\\
\AND
Philippe Beaudoin\\
Element AI \\
6650 Saint-Urbain \#500\\
Montreal, QC H2S 3G9 \\
Canada \\
\\
\\
\texttt{*Correspondence to: eilif.muller@elementai.com} \\
% examples of more authors
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \AND
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
}
\begin{document}
\maketitle
\begin{abstract}
%Keywords: local learning, neocortex, dendrites, implicit objectives, generalization
The fields of artificial intelligence and neuroscience have a long history of fertile bi-directional interactions. On the one hand, important inspiration for the development of artificial intelligence systems has come from the study of natural systems of intelligence, the mammalian neocortex in particular. On the other, important inspiration for models and theories of the brain have emerged from artificial intelligence research. A central question at the intersection of these two areas is concerned with the processes by which neocortex learns, and the extent to which they are analogous to the back-propagation training algorithm of deep networks. Matching the data efficiency, transfer and generalization properties of neocortical learning remains an area of active research in the field of deep learning. Recent advances in our understanding of neuronal, synaptic and dendritic physiology of the neocortex suggest new approaches for unsupervised representation learning, perhaps through a new class of objective functions, which could act alongside or in lieu of back-propagation. Such local learning rules have implicit rather than explicit objectives with respect to the training data, facilitating domain adaptation and generalization. Incorporating them into deep networks for representation learning could better leverage unlabelled datasets to offer significant improvements in data efficiency of downstream supervised readout learning, and reduce susceptibility to adversarial perturbations, at the cost of a more restricted domain of applicability.
\end{abstract}
\section*{Unsupervised neocortex}
The neocortex is the canonically 6-layered sheet of cells forming the grey matter surface of the mammalian cerebrum. It is composed of a densely interconnected network of sub-regions responsible for learning sensory processing, speech and language, motor planning and many of the higher cognitive processes associated with rational thought. The human neocortex contains an estimated 100 trillion synapses, the points of communication between neurons which undergo persistent changes in strength and topology as a function of signals local to the synapse and a complex biochemical program \citep{holtmaat2009experience}. These processes, broadly known as synaptic plasticity, are thought to be the basis of learning and memory in the brain.
An important task of synaptic plasticity in sensory neocortical areas is to learn disentangled invariant representations \citep{dicarlo2012does}.
For example, the ventral stream of primate visual cortex, the collection of areas responsible for visual object recognition, computes hierarchically organized representations much like state-of-the art convolutional neural networks (CNNs) optimized for the task \citep{yamins2014performance}.
While there are impressive similarities in the learned representations between the ventral stream and CNNs, there are important differences in \emph{how} those representations are learned. While CNNs are trained in a supervised manner using a gradient descent optimization algorithm with an explicit global objective on large labelled datasets, the ventral stream learns from a much larger dataset (visual experience) but with only very sparse labelling. The latter property of cortical learning is attractive to emulate in CNNs, and more broadly across deep learning models. Attractive, not only because of the ability to make use of unlabelled data during learning, but also because it will impart the models with superior generalization and transfer properties, as discussed below.
\section*{The monkey's paw effect: the problem with specifying what without specifying how}
A well known and often encountered pitfall of numerical optimization algorithms for high dimensional problems, such as evolutionary algorithms, simulated annealing and also gradient descent, is that they regularly yield solutions matching \emph{what} your objective specifies to the letter, but far from \emph{how} you intended \citep{lehman2018surprising}.
The short story ``The Monkey's Paw'' by W. W. Jacobs provides a compelling metaphor. In that story, the new owner of a magical mummified monkey's paw of Indian origin is granted three wishes. The owner first wishes for \$200, and his wish is eventually granted to the penny, but with the grave side effect that it is granted through a goodwill payment from his son's employer in response to his untimely death in a terrible machinery accident \citep{jacobs1910monkey}.
The Monkey's Paw effect is also applicable to gradient descent-based optimization of deep neural nets. The relative data-hungriness of current supervised learning strategies, and the use of data augmentation to improve generalization reflect the precarious position we are in of needing to micromanage the learning processes.
Adversarial examples \citep{moosavi2016deepfool} are evidence that the monkey's paw effect none-the-less persists. It is temping to continue with the current paradigm and re-inject adversarial examples back into the learning data stream. Extrapolating, this goes in the direction of specifying the negative space of the objective, all those things the optimization should not do to solve the problem, which is potentially infinite, and rather risky in production environments like self-driving cars.
Adversarial examples represent an opportunity to address the issue in a more fundamental way \citep{yamins2016using}.
It has been argued by \citet{bengio2012deep} that if we could design deep learning systems with the explicit objective of ``disentangling the underlying factors of variation'' in an unsupervised manner, then there is much to be gained for generalization and transfer.
Such an approach offers a promising solution to the Monkey's Paw effect, as there is an explicit objective of learning good representations, from which generalization and transfer follow by definition.\footnote{For some input spaces, such as white noise, a good representation may be undefined.} One small challenge remains: how to express the objective of learning good representations? If we restrict ourselves to the subset of all possible inputs for which the neocortex learns good representations, the local processes of synaptic plasticity may provide valuable clues.
\section*{Neocortical plasticity}
The neocognitron model \citep{fukushima1980neocognitron}, the original CNN architecture, learned visual features through self-organization using local rules. Since its conception, our understanding of the neocortex and its neurons and synapses has progressed considerably.
Recent insights into the local plasticity rules for learning in the neocortex offer new inspiration for deep representation learning paradigms that learn ``disentangled representations'' from large unlabelled datasets in an unsupervised manner. A selection of recent insights into the systems of plasticity of the neocortex is shown in Fig.~\ref{fig:selection}. A new dendrite-centric view of synaptic plasticity is emerging with the discovery of the NMDA spike, a non-linear mechanism hypothesized to associate co-activated synapses through potentiation or structural changes driven by the resulting calcium currents \citep{schiller2000nmda,graupner2010mechanisms,holtmaat2009experience} (Fig.~\ref{fig:selection}A-B). Such associations, in the form of co-coding clusters of synapses, have recently been experimentally observed using optical techniques \citep{wilson2016orientation} (Fig.~\ref{fig:selection}C). Moreover neurons in the neocortex are known to form small cliques of all-to-all connected neurons which drive co-coding \citep{reimann2017cliques}, a process that would be self-reinforced through dendritic clustering by NMDA spikes (Fig.~\ref{fig:selection}D). Martinotti neurons, which are activated by such cliques of pyramidal neurons, and subsequently inhibit pyramidal dendrites \citep{silberberg2007disynaptic} provide well-timed inhibition to block further NMDA spikes \citep{doron2017timed}, and put a limit on the maximal pyramidal clique size, but also suppress activation of competing cliques (e.g. Winner-take-all (WTA) dynamics). Together, such plasticity mechanisms appear to form basic building blocks for representation learning in the feed-forward pathway of the neocortex using local learning rules. While long known competitive strategies for unsupervised representation learning indeed rely on WTA dynamics \citep{fukushima1980neocognitron, rumelhart1985feature}, deep learning approaches incorporating these increasingly apparent dendritic dimensions of learning processes have yet to be proposed \citep{poirazi2001impact, kastellakis2015synaptic}.
\begin{figure}[!t]
\centering
\includegraphics[scale=0.35]{puzzle_pieces.png}
\caption{\textbf{A selection of recent insights into the dendritic mechanisms of plasticity of the neocortex.} (\textbf{A}) Concurrent activation of >~10 nearby synapses in pyramidal neuron dendrites (red) triggers NMDA plateau potentials in dendrites (left). (\textbf{B}) Calcium drives synaptic plasticity. Synapses are bi-stable, and can be added or removed in the weak state (above). NMDA plateau potentials drive potentiation of synapses through their associated large calcium currents. (source: \citet{graupner2010mechanisms}) (\textbf{C}): Clusters of co-coding synapses are captured through these mechanisms. (\textbf{D}) Co-coding neurons form small cliques, reinforced through cluster capture. These cliques activate Martinotti cells which block further capture, implementing opposing competition. (\textbf{E}) Neocortical areas are organized in a hierarchy with top-down input arriving in layer 1 (the top-most layer) at the apical tufts of pyramidal dendrites, and at layer 6 and lower layer 5. (\textbf{F}). Temporal association of top-down and bottom-up drives cliques and plasticity.}
\label{fig:selection}
\end{figure}
Unlike CNNs, the neocortex also has a prominent feedback pathway down the hierarchy, whereby top-down input from upper layers innervate the apical tufts of pyramidal cells in layer 1 of a given cortical region \citep{felleman1991distributed}. Associations between top-down and feed-forward (bottom-up) activation are known to trigger dendritic calcium spikes and dendritic bursting \citep{larkum1999new}, which again specifically activates the WTA dynamics of the Martinotti neurons \citep{murayama2009dendritic}, but disinhibitory VIP neurons can also modulate their impact \citep{karnani2016cooperative}. These feed-back pathways have been proposed to implement \emph{predictive coding} \citep{rao1999predictive}, and error back-propagation for supervised learning algorithms \citep{guerguiev2017towards, sacramento2018dendritic}. While their importance for rapid object recognition has been recently demonstrated, their computational role remained inconclusive \citep{kar2019evidence}.
\section*{Cake but no free lunch}
With the demonstrated applicability of supervised learning for a broad range of problems and data distributions, and an ever expanding toolbox of optimized software libraries, it is unlikely that supervised learning, back-propagation and gradient descent will be dethroned as the work horses of AI for many years to come.
Nonetheless, as applications of deep networks are moving into regions where sparse data, generalization and transfer are increasingly
important, unsupervised approaches designed with the explicit goal of learning good representations from mere observation may find an important place in the AI ecosystem.
Quoting Yann LeCun\footnote{\url{https://medium.com/syncedreview/yann-lecun-cake-analogy-2-0-a361da560dae}}
\begin{quote}
“If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning.”
\end{quote}
A promising strategy would be to assume learning with sparse labels, overcoming adversarial examples, transfer learning, and few-shot learning together as the success criteria for the further development of the powerful unsupervised approaches we seek.
Recent advances in our understanding of the processes of neocortical plasticity may well offer useful inspiration, but let's close with some words of moderation. Biology's solutions also show us there will be no free lunch, i.e. neocortical unsupervised learning algorithms will be less general than supervised learning by gradient descent. Neocortex relies on structure at specific spatial and temporal scales in its input streams to learn representations. Evolution has had millions of years to configure the sensory organs to provide signals to the neocortex in ways that it can make sense of them, and that serve the animal's ecological niche. We should not expect, for example, cortical unsupervised learning algorithms to cluster frozen white noise images. A neocortical solution requires a neocortical problem (e.g. from the so-called ``Brain set'' \citep{richards2019framework}), so if we are to successfully take inspiration from it, we must also work within its limitations.
%\subsubsection*{Acknowledgments}
\subsubsection*{Acknowledgments}
Thanks to Giuseppe Chindemi, Perouz Taslakian, Pau Rodriguez, Isabeau Prémont-Schwarz, Hector Palacios Verdes, Pierre-André Noël, Nicolas Chapados, Blake Richards, and Guillaume Lajoie for helpful discussions.
%\section*{References}
%\small
\bibliographystyle{apalike}
\bibliography{refs}
\end{document}
\begin{thebibliography}{}
\bibitem[Bengio, 2012]{bengio2012deep}
Bengio, Y. (2012).
\newblock Deep learning of representations for unsupervised and transfer
learning.
\newblock In {\em Proceedings of ICML workshop on unsupervised and transfer
learning}, pages 17--36.
\bibitem[DiCarlo et~al., 2012]{dicarlo2012does}
DiCarlo, J.~J., Zoccolan, D., and Rust, N.~C. (2012).
\newblock How does the brain solve visual object recognition?
\newblock {\em Neuron}, 73(3):415--434.
\bibitem[Doron et~al., 2017]{doron2017timed}
Doron, M., Chindemi, G., Muller, E., Markram, H., and Segev, I. (2017).
\newblock Timed synaptic inhibition shapes nmda spikes, influencing local
dendritic processing and global i/o properties of cortical neurons.
\newblock {\em Cell reports}, 21(6):1550--1561.
\bibitem[Felleman and Van, 1991]{felleman1991distributed}
Felleman, D.~J. and Van, D.~E. (1991).
\newblock Distributed hierarchical processing in the primate cerebral cortex.
\newblock {\em Cerebral cortex (New York, NY: 1991)}, 1(1):1--47.
\bibitem[Fukushima, 1980]{fukushima1980neocognitron}
Fukushima, K. (1980).
\newblock Neocognitron: A self-organizing neural network model for a mechanism
of pattern recognition unaffected by shift in position.
\newblock {\em Biological cybernetics}, 36(4):193--202.
\bibitem[Graupner and Brunel, 2010]{graupner2010mechanisms}
Graupner, M. and Brunel, N. (2010).
\newblock Mechanisms of induction and maintenance of spike-timing dependent
plasticity in biophysical synapse models.
\newblock {\em Frontiers in computational neuroscience}, 4:136.
\bibitem[Guerguiev et~al., 2017]{guerguiev2017towards}
Guerguiev, J., Lillicrap, T.~P., and Richards, B.~A. (2017).
\newblock Towards deep learning with segregated dendrites.
\newblock {\em ELife}, 6:e22901.
\bibitem[Holtmaat and Svoboda, 2009]{holtmaat2009experience}
Holtmaat, A. and Svoboda, K. (2009).
\newblock Experience-dependent structural synaptic plasticity in the mammalian
brain.
\newblock {\em Nature Reviews Neuroscience}, 10(9):647.
\bibitem[Jacobs and Parker, 1910]{jacobs1910monkey}
Jacobs, W.~W. and Parker, L.~N. (1910).
\newblock {\em The monkey's paw: A Story in Three Scenes}.
\newblock Samuel French, Inc.
\bibitem[Kar et~al., 2019]{kar2019evidence}
Kar, K., Kubilius, J., Schmidt, K., Issa, E.~B., and DiCarlo, J.~J. (2019).
\newblock Evidence that recurrent circuits are critical to the ventral
stream’s execution of core object recognition behavior.
\newblock {\em Nature neuroscience}, 22(6):974.
\bibitem[Karnani et~al., 2016]{karnani2016cooperative}
Karnani, M.~M., Jackson, J., Ayzenshtat, I., Tucciarone, J., Manoocheri, K.,
Snider, W.~G., and Yuste, R. (2016).
\newblock Cooperative subnetworks of molecularly similar interneurons in mouse
neocortex.
\newblock {\em Neuron}, 90(1):86--100.
\bibitem[Kastellakis et~al., 2015]{kastellakis2015synaptic}
Kastellakis, G., Cai, D.~J., Mednick, S.~C., Silva, A.~J., and Poirazi, P.
(2015).
\newblock Synaptic clustering within dendrites: an emerging theory of memory
formation.
\newblock {\em Progress in neurobiology}, 126:19--35.
\bibitem[Larkum et~al., 1999]{larkum1999new}
Larkum, M.~E., Zhu, J.~J., and Sakmann, B. (1999).
\newblock A new cellular mechanism for coupling inputs arriving at different
cortical layers.
\newblock {\em Nature}, 398(6725):338.
\bibitem[Lehman et~al., 2018]{lehman2018surprising}
Lehman, J., Clune, J., Misevic, D., Adami, C., Altenberg, L., Beaulieu, J.,
Bentley, P.~J., Bernard, S., Beslon, G., Bryson, D.~M., et~al. (2018).
\newblock The surprising creativity of digital evolution: A collection of
anecdotes from the evolutionary computation and artificial life research
communities.
\newblock {\em arXiv preprint arXiv:1803.03453}.
\bibitem[Moosavi-Dezfooli et~al., 2016]{moosavi2016deepfool}
Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. (2016).
\newblock Deepfool: a simple and accurate method to fool deep neural networks.
\newblock In {\em Proceedings of the IEEE conference on computer vision and
pattern recognition}, pages 2574--2582.
\bibitem[Murayama et~al., 2009]{murayama2009dendritic}
Murayama, M., P{\'e}rez-Garci, E., Nevian, T., Bock, T., Senn, W., and Larkum,
M.~E. (2009).
\newblock Dendritic encoding of sensory stimuli controlled by deep cortical
interneurons.
\newblock {\em Nature}, 457(7233):1137.
\bibitem[Poirazi and Mel, 2001]{poirazi2001impact}
Poirazi, P. and Mel, B.~W. (2001).
\newblock Impact of active dendrites and structural plasticity on the memory
capacity of neural tissue.
\newblock {\em Neuron}, 29(3):779--796.
\bibitem[Rao and Ballard, 1999]{rao1999predictive}
Rao, R.~P. and Ballard, D.~H. (1999).
\newblock Predictive coding in the visual cortex: a functional interpretation
of some extra-classical receptive-field effects.
\newblock {\em Nature neuroscience}, 2(1):79.
\bibitem[Reimann et~al., 2017]{reimann2017cliques}
Reimann, M.~W., Nolte, M., Scolamiero, M., Turner, K., Perin, R., Chindemi, G.,
D{\l}otko, P., Levi, R., Hess, K., and Markram, H. (2017).
\newblock Cliques of neurons bound into cavities provide a missing link between
structure and function.
\newblock {\em Frontiers in computational neuroscience}, 11:48.
\bibitem[Richards et~al., 2019]{richards2019framework}
Richards, B.~A., Lillicrap, T.~P., Beaudoin, P., Bengio, Y., Bogacz, R.,
Christensen, A., Clopath, C., Ponte~Costa, R., De~Berker, A., Ganguli, S.,
Gillon, C.~J., Hafner, D., Kepecs, A., Kriegeskorte, N., Latham, P., Lindsay,
G.~W., Miller, K.~D., Naud, R., Pack, C.~C., Poirazi, P., Roelfsema, P.,
Sacramento, J., Saxe, A., Scellier, B., Schapiro, A.~C., Senn, W., Wayne, G.,
Yamins, D., Zenke, F., Zylberberg, J., Therien, D., and Kording, K.~P.
(2019).
\newblock A deep learning framework for neuroscience.
\newblock {\em Nature Neuroscience}, 22:1761–1770.
\bibitem[Rumelhart and Zipser, 1985]{rumelhart1985feature}
Rumelhart, D.~E. and Zipser, D. (1985).
\newblock Feature discovery by competitive learning.
\newblock {\em Cognitive science}, 9(1):75--112.
\bibitem[Sacramento et~al., 2018]{sacramento2018dendritic}
Sacramento, J., Costa, R.~P., Bengio, Y., and Senn, W. (2018).
\newblock Dendritic cortical microcircuits approximate the backpropagation
algorithm.
\newblock In {\em Advances in Neural Information Processing Systems}, pages
8721--8732.
\bibitem[Schiller et~al., 2000]{schiller2000nmda}
Schiller, J., Major, G., Koester, H.~J., and Schiller, Y. (2000).
\newblock Nmda spikes in basal dendrites of cortical pyramidal neurons.
\newblock {\em Nature}, 404(6775):285.
\bibitem[Silberberg and Markram, 2007]{silberberg2007disynaptic}
Silberberg, G. and Markram, H. (2007).
\newblock Disynaptic inhibition between neocortical pyramidal cells mediated by
martinotti cells.
\newblock {\em Neuron}, 53(5):735--746.
\bibitem[Wilson et~al., 2016]{wilson2016orientation}
Wilson, D.~E., Whitney, D.~E., Scholl, B., and Fitzpatrick, D. (2016).
\newblock Orientation selectivity and the functional clustering of synaptic
inputs in primary visual cortex.
\newblock {\em Nature neuroscience}, 19(8):1003.
\bibitem[Yamins and DiCarlo, 2016]{yamins2016using}
Yamins, D.~L. and DiCarlo, J.~J. (2016).
\newblock Using goal-driven deep learning models to understand sensory cortex.
\newblock {\em Nature neuroscience}, 19(3):356.
\bibitem[Yamins et~al., 2014]{yamins2014performance}
Yamins, D.~L., Hong, H., Cadieu, C.~F., Solomon, E.~A., Seibert, D., and
DiCarlo, J.~J. (2014).
\newblock Performance-optimized hierarchical models predict neural responses in
higher visual cortex.
\newblock {\em Proceedings of the National Academy of Sciences},
111(23):8619--8624.
\end{thebibliography}