Developers building AI agents are increasingly writing down where an agent is in its task, instead of leaving the model to work it out from a growing chat transcript, according to recent research on agent failures and to documentation from agent framework makers, including LangChain and Temporal. Those sources describe the same shift: an agent's current step, next step, waiting status and retry count belong in state that a program can read and save.
The pattern being replaced is an implicit state machine, a set of modes a program moves through without naming them in one place. In a typical tool-calling loop, the only record of whether an agent is gathering data, retrying a failed call, waiting on a person or finished is the conversation itself.
What the failure data shows
Researchers at UC Berkeley and Intesa Sanpaolo annotated more than 1,600 execution traces from seven multi-agent frameworks for the paper "Why Do Multi-Agent LLM Systems Fail?" They sorted failures into 14 modes. Several describe an agent that has lost track of where it is. Step repetition, the needless redoing of completed work, accounted for 15.7% of the failures they recorded. Failing to recognize that a task was finished accounted for 12.4%, and loss of conversation history for 2.8%. The authors wrote that this group of failures often reflects design choices about agent roles, prompts, workflow and "state management," and that being unaware of termination conditions appeared almost exclusively in failed runs.
A 2024 paper from researchers at Penn State and Microsoft Research, among others, tested the opposite approach. StateFlow models a task as named states with transitions, and a different prompt runs in each state. The authors argued it is "unrealistic to expect LLMs to always judge the status of current progress correctly." Using GPT-3.5-Turbo, they reported success rates 13% higher than the ReAct prompting method on the InterCode SQL benchmark and 28% higher on ALFWorld, at five times and three times lower cost respectively.
How frameworks expose state
Anthropic drew a related line in its December 2024 guide "Building effective agents." It describes workflows as systems where models and tools follow "predefined code paths," and agents as systems where models "dynamically direct their own processes and tool usage."
LangGraph stores a thread's graph state as checkpoints. Its persistence documentation says these checkpoints support conversation continuity, human-in-the-loop pauses, time travel and fault tolerance. The docs also warn that the in-memory checkpointer loses every checkpoint when the process restarts, and they recommend PostgresSaver for production.
Temporal took a different route with OpenAI. The two companies released an integration on July 30, 2025 that runs OpenAI Agents SDK agents inside Temporal Workflows, with each model call executed as a Temporal Activity whose inputs and results are recorded. Temporal's announcement says a crashed agent can restart and pick up where it left off. Its press release says the approach "eliminates the need for custom state machines or orchestration scaffolding." The integration became generally available in Temporal's Python SDK version 1.24.0 on March 23, 2026.
A dissent on separating state
Not every guide calls for a separate state model. Dex Horthy's 12-Factor Agents project, published by HumanLayer, lists current step, next step, waiting status and retry counts as "execution state." Its fifth factor says keeping that apart from the record of messages and tool calls "may be overkill," and suggests inferring execution state from an event thread the developer controls. The same project's eighth factor tells builders to own the control flow, breaking out of the loop to wait for human approval before high-stakes actions such as a deployment.
That position still rejects state hidden inside the model's reasoning. The thread is a typed list of events saved to a database, so a program can read it and resume from it at any point.
Source: arXiv, Mert Cemri, Melissa Z. Pan, Shuyi Yang et al., “Why Do Multi-Agent LLM Systems Fail?”
Additional sources: arXiv, Yiran Wu, Tianwei Yue, Shaokun Zhang, Chi Wang, Qingyun Wu, “StateFlow: Enhancing LLM Task-Solving through State-Driven Workflows”; Anthropic, “Building effective agents”; LangChain Docs, “LangGraph persistence”; Temporal, “Production-ready agents with the OpenAI Agents SDK + Temporal”; Business Wire, “Temporal and OpenAI Launch Integration for Enterprises Developing Production Agents”; GitHub, “Temporal Python SDK 1.24.0”; HumanLayer, Dex Horthy, “12-Factor Agents, factor 5”; HumanLayer, Dex Horthy, “12-Factor Agents, factor 8”; AgenticFeed, “What is an implicit state machine?”
