Read by QxMD icon Read

reward prediction error

Ravi B Sojitra, Itamar Lerner, Jessica R Petok, Mark A Gluck
Probabilistic reinforcement learning declines in healthy cognitive aging. While some findings suggest impairments are especially conspicuous in learning from rewards, resembling deficits in Parkinson's disease, others also show impairments in learning from punishments. To reconcile these findings, we tested 252 adults from 3 age groups on a probabilistic reinforcement learning task, analyzed trial-by-trial performance with a Q-reinforcement learning model, and correlated both fitted model parameters and behavior to polymorphisms in dopamine-related genes...
April 19, 2018: Neurobiology of Aging
Hanna Keren, Gang Chen, Brenda Benson, Monique Ernst, Ellen Leibenluft, Nathan A Fox, Daniel S Pine, Argyris Stringaris
Reward Prediction Errors (RPEs), defined as the difference between the expected and received outcomes, are integral to reinforcement learning models and play an important role in development and psychopathology. In humans, RPE encoding can be estimated using fMRI recordings, however, a basic measurement property of RPE signals, their test-retest reliability across different time scales, remains an open question. In this paper, we examine the 3-month and 3-year reliability of RPE encoding in youth (mean age at baseline = 10...
May 16, 2018: NeuroImage
Katherine J Pultorak, Scott A Schelp, Dominic P Isaacs, Gregory Krzystyniak, Erik B Oleson
While an extensive literature supports the notion that mesocorticolimbic dopamine plays a role in negative reinforcement, recent evidence suggests that dopamine exclusively encodes the value of positive reinforcement. In the present study, we employed a behavioral economics approach to investigate whether dopamine plays a role in the valuation of negative reinforcement. Using rats as subjects, we first applied fast-scan cyclic voltammetry (FSCV) to determine that dopamine concentration decreases with the number of lever presses required to avoid electrical footshock (i...
March 2018: ENeuro
Joshua D Berke
Dopamine is a critical modulator of both learning and motivation. This presents a problem: how can target cells know whether increased dopamine is a signal to learn or to move? It is often presumed that motivation involves slow ('tonic') dopamine changes, while fast ('phasic') dopamine fluctuations convey reward prediction errors for learning. Yet recent studies have shown that dopamine conveys motivational value and promotes movement even on subsecond timescales. Here I describe an alternative account of how dopamine regulates ongoing behavior...
May 14, 2018: Nature Neuroscience
Benedicte M Babayan, Naoshige Uchida, Samuel J Gershman
Learning to predict future outcomes is critical for driving appropriate behaviors. Reinforcement learning (RL) models have successfully accounted for such learning, relying on reward prediction errors (RPEs) signaled by midbrain dopamine neurons. It has been proposed that when sensory data provide only ambiguous information about which state an animal is in, it can predict reward based on a set of probabilities assigned to hypothetical states (called the belief state). Here we examine how dopamine RPEs and subsequent learning are regulated under state uncertainty...
May 14, 2018: Nature Communications
Amanda R Arulpragasam, Jessica A Cooper, Makiah R Nuutinen, Michael T Treadway
We are presented with choices each day about how to invest our effort to achieve our goals. Critically, these decisions must frequently be made under conditions of incomplete information, where either the effort required or possible reward to be gained is uncertain. Such choices therefore require the development of potential value estimates to guide effortful goal-directed behavior. To date, however, the neural mechanisms for this expectation process are unknown. Here, we used computational fMRI during an effort-based decision-making task where trial-wise information about effort costs and reward magnitudes was presented separately over time, thereby allowing us to model distinct effort/reward computations as choice-relevant information unfolded...
May 14, 2018: Proceedings of the National Academy of Sciences of the United States of America
Thomas D Sambrook, Ben Hardwick, Andy J Wills, Jeremy Goslin
Learning theorists posit two reinforcement learning systems: model-free and model-based. Model-based learning incorporates knowledge about structure and contingencies in the world to assign candidate actions with an expected value. Model-free learning is ignorant of the world's structure; instead, actions hold a value based on prior reinforcement, with this value updated by expectancy violation in the form of a reward prediction error. Because they use such different learning mechanisms, it has been previously assumed that model-based and model-free learning are computationally dissociated in the brain...
May 11, 2018: NeuroImage
K C Horvath, E K Miller-Cushon
Weaned dairy calves are commonly exposed to changing physical and social environments, and ability to adapt to novel management is likely to have performance and welfare implications. We characterized how behavioral responses of weaned heifer calves develop over time after introduction to a social group. Previously individually reared Holstein heifer calves (n = 15; 60 ± 5 d of age; mean ± standard deviation) were introduced in weekly cohorts (5 ± 3 new calves/wk) to an existing group on pasture (8 ± 2 calves/group)...
May 9, 2018: Journal of Dairy Science
A Ross Otto, Nathaniel D Daw
A spate of recent work demonstrates that humans seek to avoid the expenditure of cognitive effort, much like physical effort or economic resources. Less is clear, however, about the circumstances dictating how and when people decide to expend cognitive effort. Here we adopt a popular theory of opportunity costs and response vigor and to elucidate this question. This account, grounded in Reinforcement Learning, formalizes a trade-off between two costs: the harder work assumed necessary to emit faster actions and the opportunity cost inherent in acting more slowly (i...
May 8, 2018: Neuropsychologia
Anna O Ermakova, Franziska Knolle, Azucena Justicia, Edward T Bullmore, Peter B Jones, Trevor W Robbins, Paul C Fletcher, Graham K Murray
Ongoing research suggests preliminary, though not entirely consistent, evidence of neural abnormalities in signalling prediction errors in schizophrenia. Supporting theories suggest mechanistic links between the disruption of these processes and the generation of psychotic symptoms. However, it is unknown at what stage in the pathogenesis of psychosis these impairments in prediction-error signalling develop. One major confound in prior studies is the use of medicated patients with strongly varying disease durations...
April 5, 2018: Neuropsychopharmacology: Official Publication of the American College of Neuropsychopharmacology
Dennis Hernaus, James M Gold, James A Waltz, Michael J Frank
BACKGROUND: While many have emphasized impaired reward prediction error signaling in schizophrenia, multiple studies suggest that some decision-making deficits may arise from overreliance on stimulus-response systems together with a compromised ability to represent expected value. Guided by computational frameworks, we formulated and tested two scenarios in which maladaptive representations of expected value should be most evident, thereby delineating conditions that may evoke decision-making impairments in schizophrenia...
April 3, 2018: Biological Psychiatry: Cognitive Neuroscience and Neuroimaging
Matthew A Albrecht, James A Waltz, James F Cavanagh, Michael J Frank, James M Gold
People with schizophrenia (PSZ) often fail to pursue rewarding activities despite largely intact in-the-moment hedonic experiences. Deficits in effort-based decision making in PSZ may be related to enhanced effects of cost or reduced reward, i.e., through the amplification of negative prediction errors or by dampened positive prediction errors (here, positive and negative prediction errors refer to outcomes that are better or worse than expected respectively). We administered a modified Simon task to people with schizophrenia (PSZ; N=46) and healthy controls (N=32)...
April 27, 2018: Neuropsychologia
Eran Eldar, Charlotte Roth, Peter Dayan, Raymond J Dolan
Our mood often fluctuates without warning. Recent accounts propose that these fluctuations might be preceded by changes in how we process reward. According to this view, the degree to which reward improves our mood reflects not only characteristics of the reward itself (e.g., its magnitude) but also how receptive to reward we happen to be. Differences in receptivity to reward have been suggested to play an important role in the emergence of mood episodes in psychiatric disorders [1-16]. However, despite substantial theory, the relationship between reward processing and daily fluctuations of mood has yet to be tested directly...
April 20, 2018: Current Biology: CB
Alex Pine, Noa Sadeh, Aya Ben-Yakov, Yadin Dudai, Avi Mendelsohn
Discrepancies between expectations and outcomes, or prediction errors, are central to trial-and-error learning based on reward and punishment, and their neurobiological basis is well characterized. It is not known, however, whether the same principles apply to declarative memory systems, such as those supporting semantic learning. Here, we demonstrate with fMRI that the brain parametrically encodes the degree to which new factual information violates expectations based on prior knowledge and beliefs-most prominently in the ventral striatum, and cortical regions supporting declarative memory encoding...
April 26, 2018: Nature Communications
Feng-Kuei Chiang, Joni D Wallis
Reinforcement learning models have proven highly effective for understanding learning in both artificial and biological systems. However, these models have difficulty in scaling up to the complexity of real-life environments. One solution is to incorporate the hierarchical structure of behavior. In hierarchical reinforcement learning, primitive actions are chunked together into more temporally abstract actions, called "options," that are reinforced by attaining a subgoal. These subgoals are capable of generating pseudoreward prediction errors, which are distinct from reward prediction errors that are associated with the final goal of the behavior...
April 25, 2018: Journal of Cognitive Neuroscience
James D Howard, Thorsten Kahnt
There is general consensus that dopaminergic midbrain neurons signal reward prediction errors, computed as the difference between expected and received reward value. However, recent work in rodents shows that these neurons also respond to errors related to inferred value and sensory features, indicating an expanded role for dopamine beyond learning cached values. Here we utilize a transreinforcer reversal learning task and functional magnetic resonance imaging (fMRI) to test whether prediction error signals in the human midbrain are evoked when the expected identity of an appetitive food odor reward is violated, while leaving value matched...
April 23, 2018: Nature Communications
Changquan Long, Qian Sun, Shiwei Jia, Peng Li, Antao Chen
People are strongly motivated to pursue social equality during social interactions. Previous studies have shown that outcome equality influences the neural activities of monetary feedback processing in socioeconomic games; however, it remains unclear whether perception of opportunity equality affects outcome evaluation even when outcomes are maintained equal. The current study investigated the electrophysiological activities of outcome evaluation in different instructed opportunity equality conditions with event-related potentials (ERPs)...
2018: Frontiers in Human Neuroscience
Maja Brydevall, Daniel Bennett, Carsten Murawski, Stefan Bode
In a dynamic world, accurate beliefs about the environment are vital for survival, and individuals should therefore regularly seek out new information with which to update their beliefs. This aspect of behaviour is not well captured by standard theories of decision making, and the neural mechanisms of information seeking remain unclear. One recent theory posits that valuation of information results from representation of informative stimuli within canonical neural reward-processing circuits, even if that information lacks instrumental use...
April 17, 2018: Scientific Reports
Clara Kwon Starkweather, Samuel J Gershman, Naoshige Uchida
Animals make predictions based on currently available information. In natural settings, sensory cues may not reveal complete information, requiring the animal to infer the "hidden state" of the environment. The brain structures important in hidden state inference remain unknown. A previous study showed that midbrain dopamine neurons exhibit distinct response patterns depending on whether reward is delivered in 100% (task 1) or 90% of trials (task 2) in a classical conditioning task. Here we found that inactivation of the medial prefrontal cortex (mPFC) affected dopaminergic signaling in task 2, in which the hidden state must be inferred ("will reward come or not?"), but not in task 1, where the state was known with certainty...
April 9, 2018: Neuron
Darius E Parvin, Samuel D McDougle, Jordan A Taylor, Richard B Ivry
Failures to obtain reward can occur from errors in action selection or action execution. Recently, we observed marked differences in choice behavior when the failure to obtain a reward was attributed to errors in action execution compared to errors in action selection (McDougle et al. , 2016). Specifically, participants appeared to solve this credit assignment problem by discounting outcomes in which the absence of reward was attributed to errors in action execution. Building on recent evidence indicating relatively direct communication between the cerebellum and basal ganglia, we hypothesized that cerebellar-dependent sensory-prediction errors (SPEs), a signal indicating execution failure, could attenuate value updating within a basal-ganglia dependent reinforcement learning system...
April 12, 2018: Journal of Neuroscience: the Official Journal of the Society for Neuroscience
Fetch more papers »
Fetching more papers... Fetching...
Read by QxMD. Sign in or create an account to discover new knowledge that matter to you.
Remove bar
Read by QxMD icon Read

Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"