reward prediction error

BACKGROUND: Securing a professional baseball career is a formidable task, and only a unique few can overcome the obstacles necessary to become a Major League player in the Korea Baseball Organization (KBO). When achieving a spot in a KBO Major League team, a player's technical aspect may be influenced by their initial neuropsychological status. HYPOTHESIS: Personality and neurocognitive functions influence long-term pro-baseball league success. STUDY DESIGN: Cohort observational study...

Research in reinforcement learning indicates that animals respond differently to positive and negative reward prediction errors, which can be calculated by assuming learning rate bias. Many studies have shown that humans and other animals have learning rate bias during learning, but it is unclear whether and how the bias changes throughout the entire learning process. Here, we recorded the behavior data and the local field potentials (LFPs) in the striatum of five pigeons performing a probabilistic learning task...

Animals make predictions to guide their behavior and update those predictions through experience. Transient increases in dopamine (DA) are thought to be critical signals for updating predictions. However, it is unclear how this mechanism handles a wide range of behavioral timescales-from seconds or less (for example, if singing a song) to potentially hours or more (for example, if hunting for food). Here we report that DA transients in distinct rat striatal subregions convey prediction errors based on distinct time horizons...

BACKGROUND: To adapt to threats in the environment, animals must predict them and engage in defensive behavior. While the representation of a prediction error signal for reward has been linked to dopamine, a neuromodulatory prediction error for aversive learning has not been identified. METHODS: Here, we measured and manipulated norepinephrine release during threat learning using optogenetics and a novel fluorescent norepinephrine sensor. RESULTS: We found that norepinephrine response to conditioned stimuli reflects aversive memory strength...

BACKGROUND: Depression is characterized by abnormalities in emotional processing, but the specific drivers of such emotional abnormalities are unknown. Computational work indicates that both surprising outcomes (prediction errors; PEs) and outcomes (values) themselves drive emotional responses, but neither has been consistently linked to affective disturbances in depression. As a result, the computational mechanisms driving emotional abnormalities in depression remain unknown. METHODS: Here, in 687 individuals, one-third of whom qualify as depressed via a standard self-report measure (the PHQ-9), we use high-stakes, naturalistic events - the reveal of midterm exam grades - to test whether individuals with heightened depression display a specific reduction in emotional response to positive PEs...

BACKGROUND: The nature of cognitive flexibility deficits in obsessive-compulsive disorder (OCD), which historically have been tested with probabilistic reversal learning tasks, remains elusive. Here, a novel deterministic reversal task and inclusion of unmedicated patients in the study sample illuminated the role of fixed versus uncertain rules/contingencies and of serotonergic medication. Additionally, our understanding of probabilistic reversal was enhanced through theoretical computational modeling of cognitive flexibility in OCD...

Striatal dopamine drives associative learning by acting as a teaching signal. Much work has focused on simple learning paradigms, including Pavlovian and instrumental learning. However, higher cognition requires that animals generate internal concepts of their environment, where sensory stimuli, actions and outcomes become flexibly associated. Here, we performed fiber photometry dopamine measurements across the striatum of male mice as they learned cue-action-outcome associations based on implicit and changing task rules...

The tendency of all humans to experience loneliness at some point in their lives implies that it serves an adaptive function. Building on biological theories of herding in animals, according to which collective movement emerges from local interactions that are based on principles of attraction, repulsion, and alignment, we propose an approach that synthesizes these principles with theories of loneliness in humans. We present here the herding model of loneliness, that extends these principles into the psychological domain...

RATIONALE: The ability to monitor the consequences of our actions for others is imperative for flexible and adaptive behavior, and allows us to act in a (pro)social manner. Yet, little is known about the neurochemical mechanisms underlying alterations in (pro)social performance monitoring. OBJECTIVE: The aim of this functional magnetic resonance imaging (fMRI) study was to improve our understanding of the role of dopamine and oxytocin and their potential overlap in the neural mechanisms underlying performance monitoring for own versus others' outcomes...

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), but the mechanisms underlying RPE computation, particularly the contributions of different neurotransmitters, remain poorly understood. Here, we used a genetically encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons in mice. We found that glutamate inputs exhibit virtually all of the characteristics of RPE rather than conveying a specific component of RPE computation, such as reward or expectation...

The widespread adoption of deep learning to build models that capture the dynamics of neural populations is typically based on "black-box" approaches that lack an interpretable link between neural activity and function. Here, we propose to apply algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single neuron activity through a generative model...

Dopamine is implicated in adaptive behavior through reward prediction error (RPE) signals that update value estimates. There is also accumulating evidence that animals in structured environments can use inference processes to facilitate behavioral flexibility. However, it is unclear how these two accounts of reward-guided decision-making should be integrated. Using a two-step task for mice, we show that dopamine reports RPEs using value information inferred from task structure knowledge, alongside information about reward rate and movement...

BACKGROUND: Child eating behaviors (CEBs) and parental feeding practices (PFPs) play critical roles in childhood obesity. However, the bidirectional relationships between CEBs and PFPs remain equivocal. This longitudinal study aimed to explore their bidirectional relationships. METHODS: A convenience sample of 870 parents with preschoolers was recruited in this longitudinal study (Shanghai, China). Three non-responsive feeding practices (NFPs), three responsive feeding practices (RFPs), five CEBs, and covariates were collected using validated questionnaires at baseline and the 6-month follow-up...

We propose an Information bottleneck (IB) for Goal representation learning (InfoGoal), a self-supervised method for generalizable goal-conditioned reinforcement learning (RL). Goal-conditioned RL learns a policy from reward signals to predict actions for reaching desired goals. However, the policy would overfit the task-irrelevant information contained in the goal and may be falsely or ineffectively generalized to reach other goals. A goal representation containing sufficient task-relevant information and minimum task-irrelevant information is guaranteed to reduce generalization errors...

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors...

The dopaminergic system is firmly implicated in reversal learning but human measurements of dopamine release as a correlate of reversal learning success are lacking. Dopamine release and hemodynamic brain activity in response to unexpected changes in action-outcome probabilities are here explored using simultaneous dynamic [11C]Raclopride PET-fMRI and computational modelling of behavior. When participants encounter reversed reward probabilities during a card guessing game, dopamine release is observed in associative striatum...

Cognitive flexibility can enhance the ability to adjust to changing environments. Here, we use learning simulations to investigate the possible advantages of flexible learning in volatile (changing) environments. We compare two established learning mechanisms, one with constant learning rates and one with rates that adjust to volatility. We study an ecologically relevant case of volatility, based on observations of developing cleaner fish Labroides dimidiatus that experience a transition from a simpler to a more complex foraging environment...

First-degree relatives of patients with bipolar disorder are at heightened risk of mood episodes, which may be attributed to the existence of endophenotypes i.e., heritable (neuro)biological changes present in patients and their unaffected relatives (UR). In this longitudinal MRI study, we aim to investigate the trajectories of aberrant reward-related functional changes identified in UR vs healthy controls (HC). Sixty-eight UR and 65 HC of similar age and gender distribution underwent MRI at baseline while performing a card guessing task...

This article applies a two-process "neural autopilot" model to field data. The autopilot model hypothesizes that habitual choice occurs when the reward from a behavior has low numerical "doubt" (i.e., reward prediction errors are small). The model toggles between repeating a previous choice (habit) when doubt is low and making a goal-directed choice when doubt is high. The model has ingredients established in animal learning and cognitive neuroscience and is simple enough to make nonobvious predictions. In two empirical applications, we fit the model to field data on purchases of canned tuna and posting on the Chinese social media site Weibo...

The Free Energy Principle (FEP) is a normative computational framework for iterative reduction of prediction error and uncertainty through perception-intervention cycles that has been presented as a potential unifying theory of all brain functions (Friston, 2006). Any theory hoping to unify the brain sciences must be able to explain the mechanisms of decision-making, an important cognitive faculty, without the addition of independent, irreducible notions. This challenge has been accepted by several proponents of the FEP (Friston, 2010; Gershman, 2019)...