Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction
| Source: BAIR Blog (Berkeley AI Research)
Tags: LLM agents, context window, belief states, BAIR, context compaction, long-horizon AI
Berkeley AI Research introduces ABBEL, a framework that replaces recursive context summarization with supervised belief states for long-horizon LLM agents — closing the performance gap that persists in production systems like Cursor's composer 2.5 even after RL fine-tuning.
Details
Context compaction — summarizing prior interaction history so it fits in the LLM context window — is already deployed in production coding assistants. Cursor's composer 2.5 and Grandcode (which beat all human competitors in online coding competitions) both rely on it. But a persistent performance gap remains: summary-based policies improve with RL fine-tuning but never close the gap to full-context policies, and Cursor still advises users to avoid compaction mid-task. ABBEL (Autoencoder-Based Belief ELicitation, inferred from the abstract) addresses this by isolating and supervising the information content of summaries as natural-language belief states. Instead of treating summarization as a side effect of task completion, ABBEL explicitly trains models to maintain belief states that accurately encode what has been learned from prior interaction — separating the encoding step from the action step. The approach draws on autoencoder principles: the model encodes prior belief, action, and observation into an updated belief state, and belief grading provides explicit supervision over the information content of that state. This is in contrast to standard summarization, where the model receives no direct signal about whether its summary captures the right information. The BAIR blog post frames this as a research preview. Technical details beyond the abstract are not available in the current source, but the production relevance is high — any team using context compaction in multi-turn agents faces exactly the problem ABBEL targets.