The Influence of Markov Decision Process Structure on the Possible Strategic Use of Working Memory and Episodic Memory

OpenBU

Show simple item record

dc.contributor.author Zilli, Eric A. en_US
dc.contributor.author Hasselmo, Michael E. en_US
dc.date.accessioned 2012-01-12T17:47:34Z
dc.date.available 2012-01-12T17:47:34Z
dc.date.issued 2008-7-23 en_US
dc.identifier.citation Zilli, Eric A., Michael E. Hasselmo. "The Influence of Markov Decision Process Structure on the Possible Strategic Use of Working Memory and Episodic Memory" PLoS ONE 3(7): e2756. (2008) en_US
dc.identifier.issn 1932-6203 en_US
dc.identifier.uri http://hdl.handle.net/2144/3445
dc.description.abstract Researchers use a variety of behavioral tasks to analyze the effect of biological manipulations on memory function. This research will benefit from a systematic mathematical method for analyzing memory demands in behavioral tasks. In the framework of reinforcement learning theory, these tasks can be mathematically described as partially-observable Markov decision processes. While a wealth of evidence collected over the past 15 years relates the basal ganglia to the reinforcement learning framework, only recently has much attention been paid to including psychological concepts such as working memory or episodic memory in these models. This paper presents an analysis that provides a quantitative description of memory states sufficient for correct choices at specific decision points. Using information from the mathematical structure of the task descriptions, we derive measures that indicate whether working memory (for one or more cues) or episodic memory can provide strategically useful information to an agent. In particular, the analysis determines which observed states must be maintained in or retrieved from memory to perform these specific tasks. We demonstrate the analysis on three simplified tasks as well as eight more complex memory tasks drawn from the animal and human literature (two alternation tasks, two sequence disambiguation tasks, two non-matching tasks, the 2-back task, and the 1-2-AX task). The results of these analyses agree with results from quantitative simulations of the task reported in previous publications and provide simple indications of the memory demands of the tasks which can require far less computation than a full simulation of the task. This may provide a basis for a quantitative behavioral stoichiometry of memory tasks. en_US
dc.description.sponsorship Silvio O. Conte Center (NIMH MH71702, NIMH MH60013, NIMH MH61492, NSF SLC SBE 0354378); National Institute on Drug Abuse (DA16454) en_US
dc.language.iso en en_US
dc.publisher Public Library of Science en_US
dc.title The Influence of Markov Decision Process Structure on the Possible Strategic Use of Working Memory and Episodic Memory en_US
dc.type article en_US
dc.identifier.doi 10.1371/journal.pone.0002756 en_US
dc.identifier.pubmedid 18648498 en_US
dc.identifier.pmcid 2447173 en_US

Files in this item

This item appears in the following Collection(s)

Show simple item record

Search OpenBU


Advanced Search

Browse

Deposit Materials