How AI agent memory actually works: history, compaction, recall

An agent has no memory of its own. A runtime stores state and re-injects it on each model call, through three layers: conversation history, compaction, and long-term recall.

Why does context run out?

Every step resends the conversation, and tool results dominate it. Without intervention a long run hits the context window or burns its token budget.

How does compaction work?

In Cortex, each step of agent.Runner first blanks superseded tool results, keeping the 8 most recent, then compacts if needed. Compaction only runs when ContextWindow is set.

The threshold starts at 0.75 of the window. It drops to 0.6 when under 25% of the steps remain, and by 0.05 at 3 or more tool errors or 2 or more repeats. SummaryCompactor keeps the first message and the 6 latest. A failed compaction does not stop the run.

How is recall different?

Recall survives the conversation. The recall.Store interface exposes Remember, Recall and Forget, scoped by subject and agent, with a recall_memory tool for the model. A nil store means no memory at all.

Where do conversations live between turns?

host.History is in memory by default. With NATS, connectors/cluster stores it in a KV bucket, capped at 768 KiB per conversation with the oldest exchanges dropped.

What about self-improving agents?

Hermes Agent from Nous Research advertises agent-curated memory and autonomous skill creation. That suits a personal agent. On a multi-tenant platform, automatic learning raises harder questions: who owns a memory, who can read it, and how it is erased.

What should you verify?

1. Is memory isolated per tenant?
2. Can a single memory be deleted?
3. Is memory injection treated as untrusted content?
4. Does compaction keep what the task needs?