Saturday, November 21, 2026

Adaptive Memory Management for LLM AI agents

Recommendable! This seems to be an interesting paper and approach!

"... Many agents save past experiences as text in an external store, and most search and update such memories using settings the agent’s developer chose by hand, such as the number of memories to retrieve or the schedule for removing them. Researchers developed a method that learns such settings per task and carries what it learns into similar tasks.

What's new: ... developed Memory as a Controlled Process (MEMCON). Their approach wraps around an existing memory backend and calls its existing store, retrieve, and forget functions, learning from experience.

Key insight: The best way to use a memory store changes as an agent gains experience.
(a) Early in a stream of tasks, the memory store holds few memories of solved tasks, so the similarities between them may be loose. Searching for memories that are similar to the task at hand may mislead a model more than it helps. Rather than ranking memories for similarity, the agent is better served by searching for memories of how it handled tasks of the same type.
(b) Later, a reworded query may reveal something new.
(c) As the memory store grows, it’s helpful to remove memories, because they compete for a limited number of retrieval slots.
If a task ends with a signal of success or failure, the system that manages the memory can score the decisions involved. The system can sort tasks into types (say, calculating recommendations or comparing stock prices), and the average score per type can tell the system which setting works best for a particular kind of task.

How it works: The authors used three harnesses (implemented in libraries including Lobster) with three models (GPT-4.1-mini, Claude Sonnet-4, and DeepSeek-V3.2) to execute tasks on the web and in simulated environments.
They wrapped existing memory backends (such as G-memory) with a small controller that kept a table of scores, one for each pairing of situation and a memory operation (such as retrieve or remove), and updated the scores after every task based on whether the agent had succeeded.

Each time the agent was about to fetch from memory, the controller first extracted a compact state representation. It combined what was happening in the task, such as the kind of goal (as specified in the benchmark, like comparing stock prices) and whether the agent was repeating an action, with the state of the memory store, such as roughly how much it held (five possible bins) and whether a successful run from an earlier similar task existed.
It picked an operation from the choices including searching memory, searching again with a reworded query, retrieving the successful run from a similar task, merging duplicate entries in the memory store, removing an entry, and skipping the search. It favored the one with the best score so far (the authors had hand-picked sensible default numbers for first time pairings) with an increase toward operations it had tried least often. The controller ran the chosen operation and added whatever came back to the model’s prompt.

When a task ended, the controller scored how well it went (based on the outcome and how many steps the agent took to achieve it). It considered the memory choices it had made during that task and updated its table of scores. Choices made near the end, when the outcome was almost settled, resulted in the greatest increases or decreases. Choices made early got less, since later choices had room to correct them.

Results: The authors compared MEMCON to other memory-management approaches using six benchmarks: three in which an agent acts step by step in a simulated environment, such as ALFWorld (household chores), PDDL (planning puzzles based on PlanBench), and ScienceWorld (science experiments); and three in which it browses the web and uses tools to answer a single question, such as TriviaQA (trivia), WebWalkerQA (navigating websites), and GAIA (assistant tasks that call for tools and documents). MEMCON consistently achieved higher success rates while reducing token consumption. ..."

From the abstract:
"Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks. Yet nearly all existing approaches, from graph-structured memories to reflective insight stores, access memory through fixed, hand-designed heuristics.
We argue that this static view of memory is a core bottleneck for agentic learning because optimal memory behavior is fundamentally context-dependent.
The early stages of the tasks, benefit from minimal retrieval because memory is sparse; recurring goal types benefit from plan reuse rather than generic nearest-neighbor lookup;
stuck agents benefit from re-retrieval with alternative queries; and 
across long task streams, the memory store itself must be consolidated and pruned to remain useful.
We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process and learns an online policy that adaptively decides when, what, and how much to retrieve, when to inject a distilled plan, and when to consolidate or forget.
MemCon is backend-agnostic: it wraps any existing memory implementation, learns from task-by-task binary feedback with no pretraining and no additional LLM calls, and uses a lightweight tabular contextual bandit with UCB [Upper Confidence Bound] exploration that converges within tens of tasks.
Across 6 benchmarks, 3 agent frameworks, and 3 LLM backbones, MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5--20%."

Jensen Huang’s Nvidia legacy, Claude Fable 5.2 and Anthropic’s IPO, World Labs Releases Atlas to Developers

Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents (preprint, open access; first published in July and last updated in October 2026 for the third time; cited by 12 as of 11/21/2026 according to Google)






No comments: