Dr Melissa Sharpe, Princeton : "The contribution of dopamine to the model-based world"

Speaker:

Dr Melissa Sharpe,

Princeton Neuroscience Institute & National Institute on Drug Abuse, Princeton, New Jersey 

Title: "The contribution of dopamine to the model-based world"

Abstract: 

The finding that dopamine neurons signal errors in prediction was revolutionary for behavioral neuroscience. This is because reinforcement-learning theorists had argued for decades before that prediction errors were the driving force in learning- facilitating associative links between causally-related events. Such learning is now termed “model based”- referring to the development of a rich associative model of the world that can be adapted in the absence of experience and drive novel inferences about the structure of our environment. Yet when this signal was found in the brain it was argued to be synonymous with the model-free prediction error described in Sutton and Barto’s (1981) temporal-difference reinforcement learning (TDRL) model (Schultz, Dayan, and Montague, 1997). This theory has been hugely influential on the field of neuroscience. However, this model-free reinforcement learning algorithm does not endow an agent with causal knowledge of the structure of their environment. Rather, model-free TDRL assigns scalar cached values to stimuli- essentially allowing reward-paired stimuli to gain the scalar value inherent in the reward they predict without causal knowledge of their consequence. If true, this would severely restrict how dopamine can contribute to learning. In this talk, I will present data using optogenetics in rodents showing that the dopamine prediction error does in fact promote the development of rich associations between stimuli to contribute to a model-based representation of the world. I will then show how this model-based signal is utilized at the circuit level. Specifically, I will discuss how the dopamine prediction error interacts with GABAergic projection neurons in the lateral hypothalamus to contribute to the development of model-based associations. Finally, I will present some unexpected data which show that this circuit drives some forms of model-based associations but not others, prompting further research into the neural circuits which interact with the dopamine prediction error to drive cognitive representations of our world.