Episodic Memory

Episodic memory in artificial intelligence is a store of an agent's specific past experiences, each kept with its context of what happened, where, and when, that the agent searches later to inform a current decision. The term comes from psychology, where Endel Tulving distinguished in 1972 between episodic memory, the recollection of personal events, and semantic memory, general knowledge of facts. In AI, episodic memory takes the form of an external store that an agent writes to while it runs: reinforcement learning agents keep past transitions and returns, AI agents built on language models keep records of earlier conversations and actions, and robots and wearable assistants keep logs of what their cameras saw.

Episodic, Semantic, and Procedural Memory#

A trained neural network stores general knowledge in its model weights, which change only during training, so remembering a particular event from yesterday requires a separate store that can be written at run time. The Cognitive Architectures for Language Agents (CoALA) framework, proposed by Theodore Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas Griffiths in 2023, maps the psychological categories onto large language model (LLM) agents. Working memory is the current prompt, episodic memory holds the agent's past experiences, semantic memory holds facts about the world, and procedural memory holds skills in the form of model weights and agent code.

An agent with episodic memory reads from and writes to it on every step:

Episodic Memory in Reinforcement Learning#

Experience replay, introduced by Long-Ji Lin in 1992, is the simplest episodic store: the agent saves its transitions and trains on random samples of them, and the DQN agent that learned to play Atari games in 2015 kept the most recent 1 million transitions. Episodic control uses the store to act directly. Model-Free Episodic Control, by Charles Blundell and colleagues at DeepMind in 2016, recorded the best return ever obtained after each state and action and estimated the value of new states from their nearest neighbors in memory, which let it learn faster than DQN early in training. DeepMind's Never Give Up agent (2020) rewards itself for reaching states far from everything stored in the current episode's memory, an exploration bonus that helped its successor, Agent57, become the first deep reinforcement learning agent to beat the human benchmark on all 57 Atari games.

Episodic Memory in LLM Agents#

A language model remembers only what fits in its context window, so long-running agents write their experiences to an external store and retrieve the relevant ones into the prompt. In Generative Agents, presented at UIST 2023, Joon Sung Park and colleagues at Stanford and Google gave 25 simulated characters a memory stream of natural-language observations. Each memory is scored by recency, an exponential decay with a factor of 0.995 per game hour since it was last retrieved; importance, an integer from 1 to 10 that the language model assigns; and relevance, the cosine similarity between the memory's embedding and the query's. The three scores are min-max normalized and added with equal weights.

This example applies that scoring to a home robot's memories, using the all-MiniLM-L6-v2 sentence embedding model:

import numpy as np
from sentence_transformers import SentenceTransformer

# Episodic memories of a home robot: what happened, hours since last recalled, importance from 1 to 10
memories = [
    ("Saw the car keys on the kitchen counter", 30, 5),
    ("Charged the battery at the docking station", 2, 2),
    ("Watered the plants in the living room", 5, 3),
    ("Saw the car keys in the coat pocket in the hallway", 4, 5),
    ("The user said the back door sticks when it rains", 50, 8),
]
texts, hours, importance = zip(*memories)
model = SentenceTransformer("all-MiniLM-L6-v2")
query = model.encode("Where are my car keys?", normalize_embeddings=True)
relevance = model.encode(list(texts), normalize_embeddings=True) @ query  # cosine similarity to the query
recency = 0.995 ** np.array(hours)  # exponential decay per hour

# Generative Agents retrieval: min-max normalize each signal to 0-1, then add them with equal weights
signals = [recency, np.array(importance, dtype=float), relevance]
score = sum((s - s.min()) / (s.max() - s.min()) for s in signals)
for i in np.argsort(-score)[:3]:
    print(f"score {score[i]:.2f}, relevance {relevance[i]:.2f}: {texts[i]}")

The hallway sighting ranks first with a score of 2.44, ahead of the kitchen counter sighting at 1.89, although the older memory is slightly more similar to the question (0.71 against 0.70). Recency separates the two, so the robot reports where it saw the keys most recently. Reflexion, by Noah Shinn and colleagues in 2023, stores an agent's own written reflections on its failed attempts in an episodic buffer and reached 91% pass@1 on the HumanEval coding benchmark, against 80% for GPT-4.

Episodic Memory in Video and Embodied Agents#

For embodied AI and wearable cameras, episodic memory means answering questions about what a camera saw earlier. The Ego4D dataset, released by Meta and university partners in 2021 with 3,670 hours of first-person video from 931 camera wearers in 74 locations across 9 countries, includes an episodic memory benchmark built around questions such as "Where did I put my keys?" Its tasks locate the video moment that answers a natural-language question, find the last appearance of an object shown in a reference image, and find every instance of an activity. In the Ultralytics package, VisualAISearch embeds every image in a folder with CLIP and ranks the images against a text query by cosine similarity, which is the relevance step of such a memory for frames saved from a robot's camera.

Episodic Memory vs. Related Terms#

• Memory bank: Stores vectors for a model's own computation, such as contrastive negatives, past video frames, or normal image patches. Episodic memory stores an agent's experiences to guide its decisions.
• Retrieval-augmented generation (RAG): Retrieves passages from external documents to ground a model's answers. Episodic memory applies the same retrieval machinery to the agent's own history.
• Vector database: The storage and similarity-search layer that often holds episodic memories for production agents.
• Continual learning: Learns new tasks without catastrophic forgetting of old ones, often by replaying stored examples. Gradient Episodic Memory (GEM), by David Lopez-Paz and Marc'Aurelio Ranzato in 2017, keeps a small set of examples from each earlier task and constrains every update so that the loss on them does not increase.