Stateful vs. stateless agents
Memory is the most consequential — and most over-applied — decision in agent design. We help you choose deliberately, store state where it belongs, and govern it from day one.
- Pure-function steps stay stateless
- Durable state lives outside the model
- Run, session, and long-term memory split
- Retention & lineage by design
It's a storage decision, not a model decision
Whether an agent is stateful has almost nothing to do with which model you pick.
A stateless agent treats every invocation as a pure function: same input, same behavior, no memory of what came before. A stateful agent carries context forward — across the steps of one task, across a user's session, or across weeks of accumulated history.
The trap is defaulting to stateful because it feels more capable. State buys you continuity and personalization, but it costs you horizontal scalability, complicates retries, and turns a clean function into something you have to back up, migrate, expire, and audit. The right question is never 'how much memory can we give it' — it's 'what is the smallest amount of state this task actually requires, and where should it live.'
Not all state is the same
Most architectures fail by collapsing these into one bucket. We keep them separate, each with its own store and lifecycle.
Run state
The intermediate results, tool outputs, and progress of a single task. Lives in the orchestrator so a crashed step can resume instead of restarting.
Session memory
Context for the current conversation or job — what the user said, what's been decided. Scoped, short-lived, and cleared when the session ends.
Long-term memory
Durable facts and preferences that persist across sessions. The most powerful and the most governed — every write is attributable and expirable.
Stateless workers
The reasoning steps themselves. Disposable, idempotent, and identical run to run — so they scale out and retry without surprises.
Durable orchestration
A workflow engine holds run state outside any process, surviving deploys and crashes so long-running agents are replayable, not fragile.
Retention controls
Expiry, tenant isolation, and the right to forget. State you keep is state you must defend, so we make its lifecycle explicit.
Stateless vs. stateful, head to head
The tradeoffs are real and they pull in opposite directions. The skill is putting each property where it earns its keep.
| Stateless agent | Stateful agent | |
|---|---|---|
| Scaling | Horizontal — spin up N copies | Bounded by the state store and locking |
| Retries | Safe and idempotent by default | Must reconcile partial writes |
| Continuity | None — each run starts cold | Carries context across steps & sessions |
| Failure mode | Lose a request, retry it | Resume from durable checkpoint |
| Governance | Minimal — nothing to retain | Retention, isolation, right to forget |
| Best for | Classify, extract, draft, transform | Multi-step jobs, ongoing assistants |
Stateful spine, stateless limbs
In practice the strongest pattern is a hybrid. A durable orchestrator owns the run's state — its checkpoints, its place in the workflow, its accumulated results — while the steps that do the actual reasoning stay stateless and interchangeable.
That split gives you the best of both: long-running, replayable, crash-safe workflows on the outside, and cheap, horizontally scalable, easily-tested workers on the inside. When a step fails, the engine replays it against the same durable state; when load spikes, you add workers without touching the state layer.
- Workflow engine for run-scoped state
- Idempotent, stateless reasoning steps
- Replay on failure, scale on demand
Our decision path
We make the stateful/stateless call per agent, not per project — and revisit it as workflows grow.
Map the task
Is the output a pure function of the input, or does it depend on prior steps and history? That answer sets the default.
Size the state
We scope the minimum state required and assign it to a tier — run, session, or long-term — each with its own store.
Externalize it
State leaves the process for a durable store. The agent stays disposable so it scales and survives deploys.
Govern it
We attach retention, isolation, and lineage so every persisted fact is attributable, expirable, and auditable.
Frequently asked questions
Doesn't every agent need memory to be useful?
No. A surprising share of production work — classify this ticket, extract these fields, draft this reply — is a pure function of its input. Those agents should be stateless: each run starts clean, scales horizontally, and is trivially retryable. Reserve state for work that genuinely spans steps or sessions.
Where does the state actually live?
Never in the LLM and never in process memory. We externalize it to a durable store — a workflow engine like Temporal for run-scoped state, plus Postgres or Redis for session and long-term memory. The agent process stays disposable; the state outlives any single invocation, deploy, or crash.
How is a stateful agent governed differently?
State is a liability as well as an asset. Stateful agents need retention and expiry policies, tenant isolation, the right to forget, and lineage that ties every memory write back to the run that produced it. We bake those controls into the architecture rather than bolting them on later.
Can a single workflow mix both?
Almost always, and it should. The orchestrator holds the durable run state while the worker steps that do the reasoning stay stateless and replaceable. That hybrid gives you reliability and replay without paying the memory tax on every node.
Designing an agent and stuck on memory?
Bring the workflow. In one session we'll map its state tiers, pick stateful or stateless per step, and sketch the store and governance to back it.