Reinforcement learning with cognitive maps
Cognitively structured recurrent states let reinforcement-learning agents accumulate evidence, form map-like internal codes, and generalize with far fewer parameters.
Path integration keeps track of position by continuously combining self-motion cues into an internal estimate of where the body is relative to the start of a path. Because entorhinal circuits that support this computation are affected early in Alzheimer's disease, path integration offers a promising behavioral window onto subtle preclinical changes.
In collaboration with Dr. Ehren Newman from the Psychological and Brain Sciences department at IU and Dr. Thomas Wolbers and Dr. Vladislava Segen from the German Center for Neurodegenerative Diseases (part of CRCNS collaboration), we investigated whether path integration is already disrupted in subjective cognitive decline (SCD), a condition in which people report worsening cognition despite performing within the normal range on standard neuropsychological tests. Participants with SCD and healthy older adults completed a self-guided virtual-reality navigation task designed to minimize compensatory strategies such as landmark use.
The main behavioral result clearly indicated that the SCD group made larger path-integration errors than controls. That difference was specific to tracking position through space rather than a broad failure to follow the task. Angular-integration performance was relatively preserved, movement dynamics were comparable across groups, and both groups showed similar adaptation over the course of the experiment.
Modeling error sources
To move beyond a single error score, we fit a hierarchical Bayesian model that decomposes path-integration behavior into interpretable latent components. The key state variable is the participant's internal estimate of location, \( \hat{\mathbf{x}}(\ell) \), as a function of traveled distance \( \ell \). Instead of assuming that all error comes from one generic source, the model allows systematic distortions of self-motion input, decay of the internal estimate, drift, and noise accumulation over the course of the path.
This means that the same final response can arise from very different mechanisms. A participant could underestimate velocity, gradually lose the integrated estimate of where the start lies, accumulate noise as movement continues, or add uncertainty only when converting the internal estimate into an overt response. The model separates those possibilities and estimates them jointly at the individual and group levels.
State update
\[ \frac{d\hat{\mathbf{x}}(\ell)}{d\ell} = -\beta \hat{\mathbf{x}}(\ell) + \alpha \mathbf{v}(\ell) + \mathbf{b} + \sigma_{0}\,\boldsymbol{\xi}(\ell) \]
- \( \beta \) is memory leak: larger values make the internal homing-vector estimate decay faster as the path unfolds.
- \( \alpha \) is velocity gain: it captures under- or over-weighting of self-motion input.
- \( \mathbf{b} \) is an additive bias term that shifts the estimated position in a consistent direction.
- \( \sigma_0 \) is accumulating noise that builds up over traveled distance.
- \( \sigma_r \) is Weber-like reporting noise applied when the internal estimate is converted into the final distance response.
The modeling analysis indicated that the most reliable separation between control and SCD participants is memory leak, meaning that the internal estimate of the starting location decays more rapidly in SCD participants as they move through space.
The parameter comparison below makes that point in two complementary ways. The top row shows the participant-level parameter estimates, while the bottom row shows the group-level posterior difference distributions. Memory leak shows the clearest positive shift for SCD relative to controls; reporting noise trends upward as well, but much less decisively.
