Read by QxMD icon Read

Nathaniel Daw

Samuel J Gershman, Nathaniel D Daw
Wereview the psychology and neuroscience of reinforcement learning (RL), which has experienced significant progress in the past two decades, enabled by the comprehensive experimental study of simple learning and decisionmaking tasks. However, one challenge in the study of RL is computational: The simplicity of these tasks ignores important aspects of reinforcement learning in the real world: (a) State spaces are high-dimensional, continuous, and partially observable; this implies that (b) data are relatively sparse and, indeed, precisely the same situation may never be encountered twice; furthermore, (c) rewards depend on the long-term consequences of actions in ways that violate the classical assumptions that make RL tractable...
September 2, 2016: Annual Review of Psychology
Claire M Gillan, Nathaniel D Daw
Psychiatry is in need of a major overhaul. In order to improve the precision with which we can treat, classify, and research mental health problems, we need bigger datasets than ever before. Web-based data collection provides a novel solution.
July 6, 2016: Neuron
Adam J Culbreth, Andrew Westbrook, Nathaniel D Daw, Matthew Botvinick, Deanna M Barch
Individuals with schizophrenia have a diminished ability to use reward history to adaptively guide behavior. However, tasks traditionally used to assess such deficits often rely on multiple cognitive and neural processes, leaving etiology unresolved. In the current study, we adopted recent computational formalisms of reinforcement learning to distinguish between model-based and model-free decision-making in hopes of specifying mechanisms associated with reinforcement-learning dysfunction in schizophrenia. Under this framework, decision-making is model-free to the extent that it relies solely on prior reward history, and model-based if it relies on prospective information such as motivational state, future consequences, and the likelihood of obtaining various outcomes...
August 2016: Journal of Abnormal Psychology
Lauren Y Atlas, Bradley B Doll, Jian Li, Nathaniel D Daw, Elizabeth A Phelps
Socially-conveyed rules and instructions strongly shape expectations and emotions. Yet most neuroscientific studies of learning consider reinforcement history alone, irrespective of knowledge acquired through other means. We examined fear conditioning and reversal in humans to test whether instructed knowledge modulates the neural mechanisms of feedback-driven learning. One group was informed about contingencies and reversals. A second group learned only from reinforcement. We combined quantitative models with functional magnetic resonance imaging and found that instructions induced dissociations in the neural systems of aversive learning...
2016: ELife
Nathan F Parker, Courtney M Cameron, Joshua P Taliaferro, Junuk Lee, Jung Yoon Choi, Thomas J Davidson, Nathaniel D Daw, Ilana B Witten
Dopaminergic (DA) neurons in the midbrain provide rich topographic innervation of the striatum and are central to learning and to generating actions. Despite the importance of this DA innervation, it remains unclear whether and how DA neurons are specialized on the basis of the location of their striatal target. Thus, we sought to compare the function of subpopulations of DA neurons that target distinct striatal subregions in the context of an instrumental reversal learning task. We identified key differences in the encoding of reward and choice in dopamine terminals in dorsal versus ventral striatum: DA terminals in ventral striatum responded more strongly to reward consumption and reward-predicting cues, whereas DA terminals in dorsomedial striatum responded more strongly to contralateral choices...
June 2016: Nature Neuroscience
Jenna M Reinen, Jared X Van Snellenberg, Guillermo Horga, Anissa Abi-Dargham, Nathaniel D Daw, Daphna Shohamy
BACKGROUND: Recent findings demonstrate that patients with schizophrenia are worse at learning to predict rewards than losses, suggesting that motivational context modulates learning in this disease. However, these findings derive from studies in patients treated with antipsychotic medications, D2 receptor antagonists that may interfere with the neural systems that underlie motivation and learning. Thus, it remains unknown how motivational context affects learning in schizophrenia, separate from the effects of medication...
November 2016: Schizophrenia Bulletin
Bradley B Doll, Nathaniel D Daw
Evidence increasingly suggests that dopaminergic neurons play a more sophisticated role in predicting rewards than previously thought.
2016: ELife
Johannes H Decker, A Ross Otto, Nathaniel D Daw, Catherine A Hartley
Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment...
June 2016: Psychological Science
Claire M Gillan, Michal Kosinski, Robert Whelan, Elizabeth A Phelps, Nathaniel D Daw
Prominent theories suggest that compulsive behaviors, characteristic of obsessive-compulsive disorder and addiction, are driven by shared deficits in goal-directed control, which confers vulnerability for developing rigid habits. However, recent studies have shown that deficient goal-directed control accompanies several disorders, including those without an obvious compulsive element. Reasoning that this lack of clinical specificity might reflect broader issues with psychiatric diagnostic categories, we investigated whether a dimensional approach would better delineate the clinical manifestations of goal-directed deficits...
2016: ELife
Bradley B Doll, Kevin G Bath, Nathaniel D Daw, Michael J Frank
UNLABELLED: Considerable evidence suggests that multiple learning systems can drive behavior. Choice can proceed reflexively from previous actions and their associated outcomes, as captured by "model-free" learning algorithms, or flexibly from prospective consideration of outcomes that might occur, as captured by "model-based" learning algorithms. However, differential contributions of dopamine to these systems are poorly understood. Dopamine is widely thought to support model-free learning by modulating plasticity in striatum...
January 27, 2016: Journal of Neuroscience: the Official Journal of the Society for Neuroscience
Yan T Wong, Margaret M Fabiszak, Yevgeny Novikov, Nathaniel D Daw, Bijan Pesaran
Selecting and planning actions recruits neurons across many areas of the brain, but how ensembles of neurons work together to make decisions is unknown. Temporally coherent neural activity may provide a mechanism by which neurons coordinate their activity to make decisions. If so, neurons that are part of coherent ensembles may predict movement choices before other ensembles of neurons. We recorded neuronal activity in the lateral and medial banks of the intraparietal sulcus (IPS) of the posterior parietal cortex while monkeys made choices about where to look and reach...
February 2016: Nature Neuroscience
Madeleine E Sharp, Karin Foerde, Nathaniel D Daw, Daphna Shohamy
Patients with loss of dopamine due to Parkinson's disease are impaired at learning from reward. However, it remains unknown precisely which aspect of learning is impaired. In particular, learning from reward, or reinforcement learning, can be driven by two distinct computational processes. One involves habitual stamping-in of stimulus-response associations, hypothesized to arise computationally from 'model-free' learning. The other, 'model-based' learning, involves learning a model of the world that is believed to support goal-directed behaviour...
February 2016: Brain: a Journal of Neurology
Laurel S Morris, Prantik Kundu, Nicholas Dowell, Daisy J Mechelmans, Pauline Favre, Michael A Irvine, Trevor W Robbins, Nathaniel Daw, Edward T Bullmore, Neil A Harrison, Valerie Voon
Discrete yet overlapping frontal-striatal circuits mediate broadly dissociable cognitive and behavioural processes. Using a recently developed multi-echo resting-state functional MRI (magnetic resonance imaging) sequence with greatly enhanced signal compared to noise ratios, we map frontal cortical functional projections to the striatum and striatal projections through the direct and indirect basal ganglia circuit. We demonstrate distinct limbic (ventromedial prefrontal regions, ventral striatum - VS, ventral tegmental area - VTA), motor (supplementary motor areas - SMAs, putamen, substantia nigra) and cognitive (lateral prefrontal and caudate) functional connectivity...
January 2016: Cortex; a Journal Devoted to the Study of the Nervous System and Behavior
Nathaniel D Daw
No abstract text is available yet for this article.
November 10, 2015: Proceedings of the National Academy of Sciences of the United States of America
Y-Lan Boureau, Peter Sokol-Hessner, Nathaniel D Daw
Many different situations related to self control involve competition between two routes to decisions: default and frugal versus more resource-intensive. Examples include habits versus deliberative decisions, fatigue versus cognitive effort, and Pavlovian versus instrumental decision making. We propose that these situations are linked by a strikingly similar core dilemma, pitting the opportunity costs of monopolizing shared resources such as executive functions for some time, against the possibility of obtaining a better outcome...
November 2015: Trends in Cognitive Sciences
Joseph E Dunsmoor, Yael Niv, Nathaniel Daw, Elizabeth A Phelps
Extinction serves as the leading theoretical framework and experimental model to describe how learned behaviors diminish through absence of anticipated reinforcement. In the past decade, extinction has moved beyond the realm of associative learning theory and behavioral experimentation in animals and has become a topic of considerable interest in the neuroscience of learning, memory, and emotion. Here, we review research and theories of extinction, both as a learning process and as a behavioral technique, and consider whether traditional understandings warrant a re-examination...
October 7, 2015: Neuron
Hang Zhang, Nathaniel D Daw, Laurence T Maloney
In many laboratory visuo-motor decision tasks, subjects compensate for their own visuo-motor error, earning close to the maximum reward possible. To do so, they must combine information about the distribution of possible error with values associated with different movement outcomes. The optimal solution is a potentially difficult computation that presupposes knowledge of the probability density function (pdf) of visuo-motor error associated with each possible planned movement. It is unclear how the brain represents such pdfs or computes with them...
August 2015: Nature Neuroscience
Sara M Constantino, Nathaniel D Daw
Although most decision research concerns choice between simultaneously presented options, in many situations options are encountered serially, and the decision is whether to exploit an option or search for a better one. Such problems have a rich history in animal foraging, but we know little about the psychological processes involved. In particular, it is unknown whether learning in these problems is supported by the well-studied neurocomputational mechanisms involved in more conventional tasks. We investigated how humans learn in a foraging task, which requires deciding whether to harvest a depleting resource or switch to a replenished one...
December 2015: Cognitive, Affective & Behavioral Neuroscience
Claire M Gillan, A Ross Otto, Elizabeth A Phelps, Nathaniel D Daw
Studies in humans and rodents have suggested that behavior can at times be "goal-directed"-that is, planned, and purposeful-and at times "habitual"-that is, inflexible and automatically evoked by stimuli. This distinction is central to conceptions of pathological compulsion, as in drug abuse and obsessive-compulsive disorder. Evidence for the distinction has primarily come from outcome devaluation studies, in which the sensitivity of a previously learned behavior to motivational change is used to assay the dominance of habits versus goal-directed actions...
September 2015: Cognitive, Affective & Behavioral Neuroscience
Bradley B Doll, Katherine D Duncan, Dylan A Simon, Daphna Shohamy, Nathaniel D Daw
Decisions may arise via 'model-free' repetition of previously reinforced actions or by 'model-based' evaluation, which is widely thought to follow from prospective anticipation of action consequences using a learned map or model. While choices and neural correlates of decision variables sometimes reflect knowledge of their consequences, it remains unclear whether this actually arises from prospective evaluation. Using functional magnetic resonance imaging and a sequential reward-learning task in which paths contained decodable object categories, we found that humans' model-based choices were associated with neural signatures of future paths observed at decision time, suggesting a prospective mechanism for choice...
May 2015: Nature Neuroscience
Fetch more papers »
Fetching more papers... Fetching...
Read by QxMD. Sign in or create an account to discover new knowledge that matter to you.
Remove bar
Read by QxMD icon Read

Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"