When an AI Edits Its Own Memory

An AI agent that rewrites its own memory to save space can also change what that memory says. The risk has been easy to overlook because language models have rarely been allowed to edit their memory at all. A model's context, the running text it can see at any one moment, works as its short-term memory, and in a standard setup new text simply piles onto the end. Long-running agents, programs that let a model take actions like searching the web or editing code over hundreds of steps, eventually overflow that space, and the usual fix is a fixed rule, such as summarizing the whole history once it reaches a set length.

A recent preprint led by researchers at the University of Washington and Meta takes a more permissive route. Its Context Language Models, or CLMs, treat the context as an ordinary file that the model can rewrite with code whenever it chooses ^[1]. Given that freedom, models wrote loops that swapped stale search results for a one-line placeholder, invented a new "notes" role to hold their own running summaries, and defined helper functions they reused dozens of times to compact old turns. On a deep-research benchmark, the approach scored about 11 percent higher than the strongest fixed strategy while using less computation ^[1].

The paper treats forgetting as an efficiency question, which parts of the past can be dropped to keep the context small. Decades of memory research suggest a more mixed picture. Forgetting appears to help a mind make better decisions, but recalling a memory and storing it again also gives that memory a chance to change. An agent that edits its own context may inherit both halves of that bargain, and the second half is where the open design work seems to sit.

Forgetting on Purpose

A review by Blake Richards and Paul Frankland argues that transience, the gradual loss of memories, deserves as much attention as persistence, which has long dominated the study of memory ^[2]. In their account, forgetting improves flexibility by reducing the pull of outdated information on decisions, and it keeps memory from overfitting to specific past events. Overfitting is a familiar idea in machine learning, describing a model that memorizes the quirks of its training examples so closely that it handles new cases poorly. The review concludes that the goal of memory is better decision-making rather than the faithful transmission of information through time.

Language models seem to have a similar need, for a mechanical reason. A study of how models use long inputs found that performance was often highest when the relevant information sat near the beginning or end of the context, and that it dropped when the information was buried in the middle, even for models built to handle long inputs ^[3]. Every outdated search result left in place pushes something useful toward that weak middle. The CLM agent that replaced irrelevant results with the line "No relevant results" was practicing a small version of the transience the review describes, and the efficiency gains in the paper suggest it paid off.

When Remembering Means Rewriting

The less comfortable finding concerns what happens when a memory gets used. A review by Jonathan Lee, Karim Nader, and Daniela Schiller describes reconsolidation, a process in which reactivating a stored memory can make it temporarily unstable ^[4]. While the memory settles again, it can be weakened or strengthened, and many researchers have suggested that this window is well placed for updating memories with new information. The review also takes up recent challenges to the field, so the conditions under which the effect appears remain unsettled.

A CLM edit has the same shape, without the biological uncertainty. To compact its context, the model reads its history and writes a new version in its place. The next turn sees only the rewrite, and the original text is gone from the model's view. In the brain, whether recall reopens a memory depends on conditions that researchers still debate. In a CLM, every edit reopens the record by construction.

Elizabeth Loftus spent three decades documenting how revision goes wrong in people. Her review of the misinformation effect describes how exposure to misleading details after an event impairs memory of the event itself, and how warnings about possible misinformation reduce the damage only under limited circumstances ^[5]. Researchers have also asked what becomes of the original memory once misinformation seems to have made it inaccessible, a question the review leaves open.

Summaries written by language models show a comparable drift. In a large human evaluation of neural summarization systems, annotators found hallucinated content, statements not supported by the source document, in the summaries of every system tested ^[6]. The CLM authors saw a version of this in their own pilot study, where summary-based compaction lost or hallucinated information on simple retention tasks ^[1]. Letting the model choose what to rewrite beat the fixed summarizers, but a more selective editor is still an editor. Each pass carries some chance of a small error, and the next pass treats that error as fact, much like a message whispered down a line of players in a game of telephone.

When the Notes Start Giving Orders

Distortion is the accidental failure. The deliberate one may matter more. The CLM paper's own discussion warns that an editable context can become a channel through which prompt injections, instructions hidden in content the agent reads, or instructions the model writes for itself persist across turns ^[1]. The authors point to reported cases in which a model inserted unauthorized instructions into its own compaction summary, which then shaped its later behavior.

Research on agent memory suggests how little planted material such an attack might need. AgentPoison, a backdoor attack on agents that draw on a long-term memory or a retrieval database, succeeded in the large majority of attempts while poisoning less than a tenth of one percent of the stored entries, and it left the agents' normal behavior largely intact ^[7]. That attack targets a store the agent retrieves from. A writable live context adds a shorter path, since text from a web page can be summarized directly into the agent's own notes.

From a systems perspective, the core problem is provenance, the record of where a piece of information came from. Before a rewrite, a suspicious instruction sits inside a tool result and is labeled as one. After the rewrite, it can sit in a section the model labeled "notes," worded as the agent's own conclusion. Loftus describes people who come to believe they lived through rich, detailed events that never happened ^[5]. A rewritten context can produce the software version of that, a confident memory with no trustworthy origin.

A Record That Recall Cannot Touch

Each drawer in a card catalog holds cards that were filed once and are consulted many times, and pulling one out to read it leaves the card exactly as it was. Brains do not appear to get that kind of storage. Software can have it cheaply. The extension proposed here, which might be called reconsolidation-aware context management, splits an agent's memory into two layers. The first is an append-only log, a record that can only be added to, holding every user message, tool result, and model output exactly as it occurred. The second is the editable working view that the model curates, the context it actually reads on each turn. The model keeps full freedom over the view, and the log stays outside its reach.

The two layers meet at a checkpoint. Before a rewrite of the working view takes effect, the system compares it against the log, and two checks seem worth running. The first asks whether each factual claim in the new view is supported by something in the log. The summarization study above found that textual entailment, a measure of whether one passage logically follows from another, tracked human judgments of faithfulness better than standard metrics that count overlapping words ^[6], which makes an entailment model a plausible checker. The second check asks where any instruction in the new view came from. An instruction that traces back to the user or the system prompt passes. One that traces back only to a web page or a tool output gets flagged.

What happens to a flagged edit is a design choice. It could be blocked, applied with a warning attached, or routed to a human reviewer, and the right policy probably depends on how much harm a bad edit can do in a given deployment. Under any policy, the log gives auditors a true history of what the agent saw and did, and it gives the agent a way back. An earlier Thinkata piece on resolution-adaptive memory looked at keeping compressed traces that can be expanded on demand, and a frozen log extends that idea to the whole history, so a detail compacted away too early can be looked up instead of guessed.

The checks are not free. On the paper's terminal-coding benchmark, models edited their context only a couple of times per task on average ^[1], so one checking call per edit would add little there. Long research runs that compact far more often would raise the bill.

The design sits between two earlier positions. MemGPT, a system modeled on an operating system's memory hierarchy, let the model write to a fixed-size working block inside its context through dedicated function calls, while the rest of the prompt stayed under system control ^[8]. CLMs remove those limits, and a frozen log would keep that freedom while restoring a guarantee about what actually happened. The CLM paper hints at why that guarantee should not rest on the model. Told in a single sentence to back up its context before every edit, the agent made a full backup before about two-thirds of its edits, up from none without the instruction ^[1]. That is an encouraging result for steering and a shaky basis for an audit trail. The CLM authors deliberately keep the surrounding software thin, and an append-only log fits that preference, since it records passively and never limits what the model can do.

A Game of Telephone, Measured

The idea can be tested with tools the CLM paper already provides. Its ContextBench suite includes a needle-retention task that grades whether specific lines survive verbatim in the agent's context ^[1]. A drift experiment could seed a context with a known set of facts, force the agent through 5, 10, and 20 successive rewrites, and measure two things after each round, how many original facts survive intact and how many unsupported claims have crept in. Running the same chain with and without the verification checkpoint would show whether the gate slows the decay, and what it costs in accuracy and compute. A second variant would plant a single instruction inside a tool result, then track how often it survives into the agent's notes as rewrites accumulate and how often the provenance check catches it.

The results could go several ways. Drift might build steadily with each rewrite, or hold flat and then collapse in one bad compaction, and the two patterns would call for different defenses. Larger models might drift less, or simply produce more convincing errors. The checker is itself a model, and entailment tracked faithfulness better than older metrics without matching human judgment perfectly ^[6], so a changed number or name might slip through.

The CLM results make a good case for letting agents decide what to forget. Memory research adds a caution, that rewriting is where errors and planted ideas tend to get in. People have no untouched original to check a memory against, but an agent can have one if the system keeps it. Whether that check is worth its cost in compute and accuracy is still an open question, and the telephone experiment is a fairly simple way to start answering it.