How Much Memory Does Your Agent Actually Need?

Details

ALTK-Evolve is IBM Research's framework for equipping LLM agents with self-distilled memory—lessons extracted from past task trajectories, injected back at inference time without weight updates or human annotation. The central question this post addresses is not whether to give agents memory, but how much. Testing across eight models spanning 30B dense to frontier proprietary systems, the researchers found three distinct response patterns. Strong models with room to grow—DeepSeek-V3.2 (671B MoE) being the clearest example—gained +9.5 percentage points in task completion when given their complete, self-mined guideline set, including rare edge-case lessons. Weaker or mid-tier models drowned under the same volume; gpt-oss-120b gained +16.1pp using selective per-task retrieval plus a compact high-confidence core, while the full guideline set gained less and consumed ~50% more tokens. A third group of already-saturated models showed no measurable improvement regardless of approach. Practically, this means teams building production agent pipelines cannot apply a one-size-fits-all memory strategy. A guideline budget tuned for a frontier model will hurt a smaller one. The paper also notes that prompt caching makes even large guideline sets economical at scale. No code or model weights are released; the focus is empirical findings across real task benchmarks.