Sackler Colloquia banner

Brain Produces Mind by Modeling

Organized by Richard Shiffrin, Danielle Bassett, Sophie Deneve, Nikolaus Kriegeskorte and Josh Tenenbaum

The connection of brain and mind remains one of the great mysteries. An intriguing possibility is that the brain produces the mind by forming a model of the entire environment – including the body, the physical environment, other agents, and the social environment. It uses this model to learn, decide, attend, remember, perceive and produce action. The model develops as the brain matures, rapidly during infancy and more slowly later. It has structural components that remain stable over long times. It has labile elements that change at multiple time scales, adapting to the current environment and goals. The mind’s formation through modeling of the world might be likened to the way scientists build models: through a combination of experiment (interaction with the world) and theory (thought).

It has long been known that perception and memory are inferences constructed from prior knowledge. A simple example of perceptual inference is the way we imagine seeing the entire forward field of view when we actually see clearly only a small foveally defined region. A striking example of memory inference are demonstrations by Beth Loftus and colleagues that we can form vivid and compelling memories of events that never happened. The brain also models ourselves, possibly instantiated as consciousness, and other agents, as postulated in the ‘theory of mind’ (raised by Premack and Woodruff in 1978 about non-human primates). The theme occurs in what has been called the ‘computational theory of mind’. It is found in a variety of attempts to model learning and behavior based on processes such as prediction error and surprise, represented in neural net modeling in cognitive science, and in artificial intelligence and machine learning as represented by error driven learning, and reinforcement learning, and implemented by inference algorithms using MCMC and belief propagation. A number of efforts have tried to bridge the gap from brain to mind by building models capable of feats of cognition whose component computations might plausibly be implemented in neural circuitry. The theme of brain as model builder is also central to the Bayesian approach in cognitive science, where judgments and decisions are explained by probabilistic inference, combining prior knowledge and current evidence. It is worth noting that the theme of ‘brain as scientist’ could be extended to ‘mind as scientist’, surely true to the extent that brain produces mind. Yet a long history of research demonstrating the occasional irrationality of human decision making places important limits on such an extension.

This Sackler Colloquium brought together these various threads and approaches in neuroscience, cognitive science, and psychological science by focusing on computational modeling that attempts to bridge the gaps between these fields. It presents a coherent view of the brain as a model builder, using the model to maximize survival and utility in a complex world. The fields relevant for this colloquium encompass most of cognitive science, cognitive neuroscience, and psychology. However, attempts to frame the connections between these in terms of computational model building are just arriving in the literature in recent years, and a Sackler Colloquium at the present time allows us to present a strong cross-section of critical research, computational modeling, and thinking.  


- A limited number of videos are available for public viewing.  More videos may be added as permission is received.


Wednesday, May 1, 2019

Distinctive Voices Lecture

How the brain invents the mind, Rebecca Saxe, Massachusetts Institute of Technology
Introduction by Richard Shiffin, Indiana University Bloomington 

Thursday, May 2, 2019

Session I. 
Chair: Jeff Zacks, Washington University at St Louis

Connecting mind and brain, Rich Shiffrin, Indiana University Bloomington

Imagery-based AI, Mathilee Kunda, Vanderbilt University

Letting mental models emerge: a bottom-up approach to understanding the top-down component of visual inference, Niko Kriegeskorte, Columbia University

Theory of visual search, Wilson Geisler, University of Texas, Theory of visual search

Session II. 
Chair: Marlene Cohen, University of Pittsburgh 

Inferring what you think from what you do, Xaq Pitkow, Rice University and Baylor College of Medicine

Hierarchical reinforcement learning supports generalization, Ann Collins, University of California, Berkeley

Brain rhythms and the encoding of linguistic structure, David Poeppel, New York University

Friday, May 3, 2019

Session III. 
Chair: Nikolaus Kriegeskorte, Columbia University

Reverse engineering common sense in the human mind and brain, Josh Tenenbaum, Massachusetts Institute of Technology

The Past in the Present: Involuntary Memory Retrieval Affects Online Event Representations, Jeff Zacks, Washington University at St Louis

Linking attentional changes in neuronal responses to perception, Marlene Cohen, University of Pittsburgh

Network architectures supporting learnability, Danielle Bassett, University of Pennsylvania

When to see the forest and when the trees: Towards a concrete understanding of abstraction, Ann Hermundstad, Janelia Research Campus

Session IV.
Chair: Josh Tenenbaum, Massachusetts Institute of Technology

Informing cognitive abstractions with neurophysiology, Brandon Turner, Ohio State University

Building knowledge by integrating memories across time, Alison Preston, University of Texas

Content channeling along the ventral stream, Talia Konkle, Harvard University

Computational modeling of human face perception, Angela Yu, University of California, San Diego

The National Academy of Sciences gratefully acknowledges the generous support for this colloquium from the following organizations:

American Psychological Association

The National Sciences Foundation

U.S. Army Research Office


Powered by Blackbaud
nonprofit software