[{"data":1,"prerenderedAt":1780},["ShallowReactive",2],{"blog-post-qwen-3-8-27b-tool-calling-fails-agents-fix":3,"related-posts-qwen-3-8-27b-tool-calling-fails-agents-fix":546},{"id":4,"title":5,"author":6,"body":10,"category":521,"date":522,"description":523,"extension":524,"featured":525,"hideToc":525,"image":526,"imageHeight":527,"imageWidth":528,"meta":529,"navigation":530,"path":531,"readingTime":532,"seo":533,"seoTitle":534,"stem":535,"tags":536,"updatedDate":544,"__hash__":545},"blog/blog/qwen-3-8-27b-tool-calling-fails-agents-fix.md","Qwen3.8 27B Tool Calls Hang in Agents? It Is Not the Model",{"name":7,"role":8,"avatar":9},"Shabnam Katoch","Growth Head","/img/avatars/shabnam-profile.jpeg",{"type":11,"value":12,"toc":513},"minimark",[13,20,58,68,71,74,79,82,102,105,127,134,137,156,163,167,170,189,192,195,204,210,214,217,220,276,279,313,322,337,345,349,352,364,374,380,386,394,400,404,407,410,413,416,436,440,445,456,461,464,469,490,495,501,506,509],[14,15,16],"p",{},[17,18,19],"strong",{},"Ollama says the model supports tools. Your curl works. Your agent hangs forever. Here is what is actually happening, verified against the open Ollama issues on August 20, 2026.",[21,22,24,44],"callout",{"type":23},"quick-fix",[14,25,26,27,31,32,35,36,39,40,43],{},"Qwen3.8 27B does support tool calling. The failure is in the request path: two open Ollama issues report ",[28,29,30],"code",{},"/v1/chat/completions"," never returning for ",[28,33,34],{},"qwen3.8:27b"," while ",[28,37,38],{},"ollama run"," and ",[28,41,42],{},"/api/chat"," work normally on the same instance.",[14,45,46,47,50,51,53,54,57],{},"Unblock yourself by switching the agent loop to ",[28,48,49],{},"qwen3.6:27b",", which was confirmed responding on the same endpoint, or by pointing your agent at ",[28,52,42],{}," instead of ",[28,55,56],{},"/v1"," where the framework allows it.",[14,59,60,61,63,64,67],{},"You pulled ",[28,62,34],{},", checked that Ollama lists tool support, ran a curl against it, and got a clean response with a proper ",[28,65,66],{},"tool_calls"," block. Then you pointed Claude Code at it and watched the cursor blink for twelve minutes.",[14,69,70],{},"The model supports tool calling. That is verified, and it is also why this is so maddening. Ollama's library page for qwen3.8 lists tools, vision, and thinking as capabilities, with a 256K context window. Nothing is missing. Nothing is misconfigured on your end.",[14,72,73],{},"The problem is the request path, not the model, and the two open Ollama issues tracking it make that unusually clear.",[75,76,78],"h2",{"id":77},"what-the-reports-actually-show","What the reports actually show",[14,80,81],{},"Two issues on the Ollama repository describe this, from completely different machines, and the pattern in both is the same shape.",[14,83,84,85,39,88,91,92,94,95,97,98,101],{},"The first, opened August 18, 2026, is from someone on macOS 15.7.7 with an M3 Max and 64 GB of RAM running Ollama 0.32.13. They tested ",[28,86,87],{},"qwen3.8:27b-mlx",[28,89,90],{},"qwen3.8:27b-mxfp8",". The native ",[28,93,42],{}," endpoint works. The OpenAI-compatible ",[28,96,30],{}," endpoint works. Streaming works. Tool calling works, returning ",[28,99,100],{},"finish_reason: \"tool_calls\""," with a properly formed function call, and a second turn with the tool result generates a correct final answer.",[14,103,104],{},"Then they pointed four separate agent integrations at the same model. Pi, OpenCode, Claude Code, and Hermes. All four hang indefinitely.",[14,106,107,108,110,111,114,115,117,118,120,121,123,124,126],{},"The second issue is from an entirely different stack: Ollama 0.32.13 in Docker on ROCm, two AMD RX 6800-class GPUs, 32 GB combined VRAM, running the official ",[28,109,34],{}," tag with ",[28,112,113],{},"num_ctx"," at 32768. Same story from the other direction. ",[28,116,38],{}," works, ",[28,119,42],{}," works and had been running in production for hours, and ",[28,122,30],{}," for ",[28,125,49],{}," on the very same instance responds normally in about twenty seconds.",[14,128,129,123,131,133],{},[28,130,30],{},[28,132,34],{}," never returns. Not an error, not a timeout message. The connection stays open and no data ever arrives.",[14,135,136],{},"Apple Silicon with MLX and AMD with ROCm are about as far apart as two setups get. When both produce the same failure on the same model version, your hardware is not the variable.",[14,138,139,140,143,144,147,148,151,152,155],{},"That reporter ruled out an unusual amount before filing. Tools present or absent made no difference. Streaming on or off made no difference. Thinking disabled through ",[28,141,142],{},"think: false",", through ",[28,145,146],{},"chat_template_kwargs.enable_thinking",", and through ",[28,149,150],{},"reasoning_effort: none"," changed nothing. They forced ",[28,153,154],{},"use_mmap"," on to rule out a known memory heuristic, confirmed full GPU tensor offload in the logs, and the hang persisted past twelve minutes. They even captured a real client's request through a logging proxy and replayed it byte for byte. Same hang.",[14,157,158],{},[159,160],"img",{"alt":161,"src":162},"Two stacks, one bug: a macOS M3 Max with MLX and a Linux box with two AMD ROCm GPUs in Docker both run Ollama 0.32.13 with qwen3.8:27b, and both show the same result — /api/chat works, curl works, agents hang. When opposite hardware fails identically, look upstream","/img/blog/qwen-3-8-27b-tool-calling-fails-agents-fix-two-stacks-one-bug.jpg",[75,164,166],{"id":165},"why-everyone-misdiagnoses-this-as-a-tool-calling-problem","Why everyone misdiagnoses this as a tool calling problem",[14,168,169],{},"Here's the trap, and it is a good one.",[14,171,172,173,176,177,180,181,184,185,188],{},"When you run ",[28,174,175],{},"ollama show qwen3.8:27b --modelfile"," looking for the template, you will find it thin. That is because qwen3.8 ships its chat template inside the GGUF file rather than as a separate template layer in the manifest. I checked the registry manifests for ",[28,178,179],{},"27b",", ",[28,182,183],{},"latest",", and ",[28,186,187],{},"27b-mlx",", and none of them carry a template layer.",[14,190,191],{},"So the natural conclusion, when your agent hangs and the modelfile looks empty, is that the model has no tool template. That conclusion is wrong. Capability detection reads the GGUF too, which is why the library page correctly lists tools as a capability.",[14,193,194],{},"If you are chasing a template that looks missing, you are debugging the wrong thing entirely. The template is there, Ollama can see it, and the failure happens somewhere after that.",[14,196,197,198,203],{},"This is a different failure from the ordinary \"does not support tools\" rejection, which is loud, immediate, and tells you exactly what is wrong. Our ",[199,200,202],"a",{"href":201},"/blog/ollama-models-tool-calling-support","Ollama tool calling compatibility table"," covers that one and lists which models genuinely lack the capability. Qwen3.8 is not on that list, and that is the point.",[14,205,206],{},[159,207],{"alt":208,"src":209},"The template is not missing, it just moved inside the GGUF: older models ship a separate model layer and template layer that ollama show finds, while qwen3.8 carries the model layer and template inside the GGUF so the modelfile looks empty. Capability detection reads it either way, so an empty-looking modelfile is not a missing capability","/img/blog/qwen-3-8-27b-tool-calling-fails-agents-fix-the-template-is-not-missing.jpg",[75,211,213],{"id":212},"how-to-tell-if-this-is-your-bug-in-about-two-minutes","How to tell if this is your bug in about two minutes",[14,215,216],{},"Run these in order. The pattern of what passes and what fails identifies it precisely.",[14,218,219],{},"First, the native endpoint:",[221,222,227],"pre",{"className":223,"code":224,"language":225,"meta":226,"style":226},"language-bash shiki shiki-themes github-light","curl -s http://127.0.0.1:11434/api/chat -d '{\n  \"model\": \"qwen3.8:27b\",\n  \"messages\": [{\"role\":\"user\",\"content\":\"Say hello\"}],\n  \"stream\": false\n}'\n","bash","",[28,228,229,252,258,264,270],{"__ignoreMap":226},[230,231,234,238,242,246,249],"span",{"class":232,"line":233},"line",1,[230,235,237],{"class":236},"s7eDp","curl",[230,239,241],{"class":240},"sYu0t"," -s",[230,243,245],{"class":244},"sYBdl"," http://127.0.0.1:11434/api/chat",[230,247,248],{"class":240}," -d",[230,250,251],{"class":244}," '{\n",[230,253,255],{"class":232,"line":254},2,[230,256,257],{"class":244},"  \"model\": \"qwen3.8:27b\",\n",[230,259,261],{"class":232,"line":260},3,[230,262,263],{"class":244},"  \"messages\": [{\"role\":\"user\",\"content\":\"Say hello\"}],\n",[230,265,267],{"class":232,"line":266},4,[230,268,269],{"class":244},"  \"stream\": false\n",[230,271,273],{"class":232,"line":272},5,[230,274,275],{"class":244},"}'\n",[14,277,278],{},"Then the OpenAI-compatible endpoint with the same content:",[221,280,282],{"className":223,"code":281,"language":225,"meta":226,"style":226},"curl -s http://127.0.0.1:11434/v1/chat/completions -d '{\n  \"model\": \"qwen3.8:27b\",\n  \"messages\": [{\"role\":\"user\",\"content\":\"Say hello\"}],\n  \"stream\": false\n}'\n",[28,283,284,297,301,305,309],{"__ignoreMap":226},[230,285,286,288,290,293,295],{"class":232,"line":233},[230,287,237],{"class":236},[230,289,241],{"class":240},[230,291,292],{"class":244}," http://127.0.0.1:11434/v1/chat/completions",[230,294,248],{"class":240},[230,296,251],{"class":244},[230,298,299],{"class":232,"line":254},[230,300,257],{"class":244},[230,302,303],{"class":232,"line":260},[230,304,263],{"class":244},[230,306,307],{"class":232,"line":266},[230,308,269],{"class":244},[230,310,311],{"class":232,"line":272},[230,312,275],{"class":244},[14,314,315,316,318,319,321],{},"Then the identical ",[28,317,56],{}," request against a different model you have pulled, such as ",[28,320,49],{},".",[14,323,324,325,327,328,330,331,333,334,336],{},"If the native call returns, the other model returns on ",[28,326,56],{},", and qwen3.8 on ",[28,329,56],{}," hangs with no error, you have matched the reported pattern. Note that the macOS report had ",[28,332,56],{}," working from curl while agents still hung, so a passing ",[28,335,56],{}," curl does not clear you. If your agent hangs regardless, you are still in the same territory.",[14,338,339,340,344],{},"If instead you get an immediate error rather than a hang, you have a different problem. A hang and a rejection are not the same failure, and treating them the same is how people lose afternoons. Our ",[199,341,343],{"href":342},"/tools/agent-error-decoder","agent error decoder"," sorts the noisier error strings.",[75,346,348],{"id":347},"what-to-do-right-now","What to do right now",[14,350,351],{},"Neither issue has a merged fix as of August 20, 2026, so this section is about getting unblocked rather than getting closure.",[14,353,354,357,358,360,361,363],{},[17,355,356],{},"Switch models for the agent loop."," ",[28,359,49],{}," was explicitly confirmed working on ",[28,362,56],{}," on the same instance where qwen3.8 hangs, which makes it the lowest-friction move available. Keep qwen3.8 for direct API work if you want it.",[14,365,366,373],{},[17,367,368,369,53,371],{},"Point your agent at ",[28,370,42],{},[28,372,56],{}," where the framework allows it. Both reports show the native path behaving correctly, so if your agent supports the Ollama-native format, that route avoids the failing path entirely.",[14,375,376,379],{},[17,377,378],{},"Watch the issues rather than reinstalling."," Both reports are on Ollama 0.32.13, which was current when they were filed, so upgrading when a fix lands is the actual resolution. Reinstalling the same version accomplishes nothing.",[14,381,382,383,385],{},"And do not spend the evening on ",[28,384,113],{},". One reporter had it at 32768 and ruled out context, thinking flags, mmap, and streaming individually. That work is done and published. You do not need to repeat it.",[14,387,388,389,393],{},"This is the tax nobody quotes on running local models for agents. Not the setup, the setup is fine. It is the week where a model you already validated stops working through one code path on one runtime version, and you are the integration test. If you want agents that keep running while a model provider sorts out a regression, BetterClaw supports 28+ model providers with one-click switching, so a broken model is a dropdown change rather than an evening. ",[199,390,392],{"href":391},"/free-plan","Free plan",", no credit card, your own keys with no markup on inference.",[14,395,396],{},[159,397],{"alt":398,"src":399},"Hang or reject? Two failures, two fixes. An immediate error means the model lacks a tool template, so switch models. A hang with no error is a request-path bug, so use /api/chat or another model. Verify with three curls first, because a hang and a rejection are not the same bug","/img/blog/qwen-3-8-27b-tool-calling-fails-agents-fix-hang-or-reject.jpg",[75,401,403],{"id":402},"the-pattern-worth-learning-from-this-one","The pattern worth learning from this one",[14,405,406],{},"Qwen3.8 landed with real capability. 256K context, tools, vision, and thinking, and over half a million pulls in its first days on Ollama. It is not a bad model, and the agent frameworks hitting the wall are not badly built either.",[14,408,409],{},"What broke is the seam between them. Model, runtime, and framework each shipped on their own schedule, and the failure appeared in the gap where nobody's tests run.",[14,411,412],{},"That gap is where most local agent debugging time actually goes, and it does not show up in any benchmark or comparison table. When you evaluate a stack, the question is not which model scores highest. It is how many independently-versioned pieces sit between your agent and a working response, because every seam is a place where next Tuesday's update can quietly stop returning data.",[14,414,415],{},"Four seams is a hobby. One is a product.",[14,417,418,419,425,426,430,431,435],{},"If you would rather your agents survive a model provider's bad week, ",[199,420,424],{"href":421,"rel":422},"https://app.betterclaw.io/sign-in",[423],"nofollow","start free on BetterClaw",". One agent, every feature, no credit card, and 28+ providers behind one interface so switching models takes seconds. Pro is $49 per agent per month and ",[199,427,429],{"href":428},"/pricing","full pricing"," fits on one page. If you are still weighing local against managed, our ",[199,432,434],{"href":433},"/blog/local-llm-agent-consumer-hardware-2026","local LLM agent hardware guide"," is the honest version of that trade.",[75,437,439],{"id":438},"frequently-asked-questions","Frequently Asked Questions",[14,441,442],{},[17,443,444],{},"Why do qwen3.8 27B tool calls hang in my agent?",[14,446,447,448,31,450,35,452,39,454,43],{},"Because of a reported request-path failure rather than a model limitation. Qwen3.8 genuinely supports tool calling, and Ollama's library page lists tools, vision, and thinking as capabilities. Two open Ollama issues describe ",[28,449,30],{},[28,451,34],{},[28,453,38],{},[28,455,42],{},[14,457,458],{},[17,459,460],{},"How does this differ from the \"does not support tools\" error?",[14,462,463],{},"That error is a capability rejection: Ollama checks the model's chat template, finds no tool handling, and refuses immediately with a clear message. This is a hang, meaning the connection opens and no data arrives, in one report for more than twelve minutes. Immediate rejection means switch models. A silent hang means the capability is present and something downstream is not returning.",[14,465,466],{},[17,467,468],{},"How do I confirm this is the same bug I am hitting?",[14,470,471,472,474,475,477,478,480,481,483,484,486,487,489],{},"Send the same simple message three ways: to ",[28,473,42],{},", to ",[28,476,30],{}," on ",[28,479,34],{},", and to ",[28,482,30],{}," on a different model you have pulled. If the native call returns and another model returns on ",[28,485,56],{}," while qwen3.8 hangs without an error, you have matched the reported pattern. One reporter also saw curl succeed on ",[28,488,56],{}," while agents still hung, so a passing curl does not rule it out.",[14,491,492],{},[17,493,494],{},"Is it worth switching models while this is unresolved?",[14,496,497,498,500],{},"Yes, and it costs almost nothing. ",[28,499,49],{}," was confirmed responding normally on the same endpoint and the same instance where qwen3.8 hangs, so it is a one-line change rather than a migration. Keep qwen3.8 pulled for direct API use if you want it, since the native path works.",[14,502,503],{},[17,504,505],{},"Is running local models reliable enough for production agents?",[14,507,508],{},"It depends on how many independently-versioned components sit between your agent and a response. Local stacks put a model, a runtime, and a framework in that chain, and this bug appeared in the seam between them rather than in any one piece. If a model regression cannot be allowed to take your agents offline, either keep a tested fallback model configured or run on a platform where switching providers is a settings change.",[510,511,512],"style",{},"html pre.shiki code .s7eDp, html code.shiki .s7eDp{--shiki-default:#6F42C1}html pre.shiki code .sYu0t, html code.shiki .sYu0t{--shiki-default:#005CC5}html pre.shiki code .sYBdl, html code.shiki .sYBdl{--shiki-default:#032F62}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":226,"searchDepth":254,"depth":254,"links":514},[515,516,517,518,519,520],{"id":77,"depth":254,"text":78},{"id":165,"depth":254,"text":166},{"id":212,"depth":254,"text":213},{"id":347,"depth":254,"text":348},{"id":402,"depth":254,"text":403},{"id":438,"depth":254,"text":439},"Troubleshooting","2026-08-25","Qwen3.8 27B tool calls hang in Ollama, Hermes, and Claude Code? The model supports tools. Here is the real cause, how to confirm it, and what to do now.","md",false,"/img/blog/qwen-3-8-27b-tool-calling-fails-agents-fix.jpg",512,1024,{},true,"/blog/qwen-3-8-27b-tool-calling-fails-agents-fix","8 min read",{"title":5,"description":523},"Qwen3.8 27B Tool Calls Hang in Agents? Cause and Fix","blog/qwen-3-8-27b-tool-calling-fails-agents-fix",[537,538,539,540,541,542,543],"qwen3.8 27b tool calling","qwen3.8 hangs","qwen3.8 ollama agent","ollama v1 chat completions hang","qwen3.8 claude code","qwen3.8 tool calls fail","ollama agent timeout",null,"vBAK4u8Lm6PILkmeHZjOVL4nsdMt4UbojcS55wXyF7E",[547,902,1307],{"id":548,"title":549,"author":550,"body":551,"category":521,"date":885,"description":886,"extension":524,"featured":525,"hideToc":525,"image":887,"imageHeight":544,"imageWidth":544,"meta":888,"navigation":530,"path":889,"readingTime":890,"seo":891,"seoTitle":892,"stem":893,"tags":894,"updatedDate":885,"__hash__":901},"blog/blog/agent-losing-context-fix-guide.md","Agent Memory Leaking or Losing Context? Here's How to Fix It",{"name":7,"role":8,"avatar":9},{"type":11,"value":552,"toc":863},[553,556,559,562,565,568,571,575,578,581,584,587,590,596,600,603,608,611,614,620,624,627,630,635,639,642,647,651,654,657,662,666,669,678,682,685,688,694,700,706,712,715,725,729,732,738,741,744,747,751,754,757,760,763,766,769,773,776,795,798,804,813,817,820,823,826,828,832,835,839,842,846,849,853,856,860],[14,554,555],{},"Your AI agent isn't broken. It's drowning in its own conversation history. Here's the diagnosis, the cause, and the actual fix.",[14,557,558],{},"It started with a customer email.",[14,560,561],{},"The agent had been running perfectly for three weeks. Answering support tickets, tagging them by priority, escalating the tricky ones. Then one Tuesday morning, it started responding to billing questions with onboarding instructions. It forgot the customer's name mid-conversation. It repeated the same canned response four times in a row.",[14,563,564],{},"The agent wasn't hallucinating. It wasn't misconfigured. It was losing context.",[14,566,567],{},"If you've built an AI agent that works brilliantly for the first 20 interactions and then starts acting like it has amnesia, you're not alone. Context loss is the single most common failure mode for deployed AI agents, and it's the one that gets misdiagnosed the most.",[14,569,570],{},"Here's what nobody tells you: the fix is almost never \"use a bigger model.\"",[75,572,574],{"id":573},"whats-actually-happening-when-your-agent-loses-context","What's Actually Happening When Your Agent Loses Context",[14,576,577],{},"Every large language model has a context window. Think of it as working memory. GPT-4o gives you 128K tokens. Claude gives you 200K. Gemini stretches to a million. Sounds enormous, right?",[14,579,580],{},"Here's the problem. Your agent doesn't just put the user's latest message into that window. It stuffs in the system prompt, every previous message in the conversation, tool call results, retrieved documents, function definitions, and whatever memory layer you've bolted on. All of it competes for the same finite space.",[14,582,583],{},"When the window fills up, one of two things happens. Either the model starts ignoring information at the edges (typically the middle of long contexts), or your framework silently truncates older messages to make room for new ones.",[14,585,586],{},"Both look identical from the outside: your agent \"forgets.\"",[14,588,589],{},"The context window isn't a hard drive. It's a desk. And most agents have papers stacked six inches high with no filing system.",[14,591,592],{},[159,593],{"alt":594,"src":595},"Agent losing context: the context window competes for space between the system prompt, conversation history, tool results, retrieved docs, and memory store, and older content gets truncated at the top","/img/blog/agent-losing-context-fix-guide-context-window-competition.jpg",[75,597,599],{"id":598},"the-five-reasons-your-agent-is-forgetting-everything","The Five Reasons Your Agent Is Forgetting Everything",[14,601,602],{},"Before you start tweaking model parameters or rewriting your system prompt, figure out which problem you actually have. These are the five most common culprits that cause an agent to lose context, ranked by how often we see them.",[604,605,607],"h3",{"id":606},"_1-token-bloat-from-conversation-history","1. Token Bloat From Conversation History",[14,609,610],{},"This is the big one. Every message in a multi-turn conversation stays in the context window. A 50-message support conversation can eat 15,000 to 30,000 tokens before you even account for the system prompt or tools.",[14,612,613],{},"Most frameworks handle this with a naive \"drop the oldest messages\" strategy. That works until the agent needs information from message three to answer message fifty-one.",[14,615,616,619],{},[17,617,618],{},"The fix:"," Implement a sliding window with summarization. Instead of keeping raw conversation history, periodically compress older messages into a summary. The summary stays in context. The raw messages get stored externally and retrieved only when relevant.",[604,621,623],{"id":622},"_2-oversized-system-prompts","2. Oversized System Prompts",[14,625,626],{},"We've seen system prompts that run 8,000 tokens. That's a small blog post sitting permanently in your context window, eating capacity on every single turn.",[14,628,629],{},"The worst offenders pack every possible instruction, every edge case, every formatting rule into a single monolithic prompt. The model can't prioritize when everything is labeled \"important.\"",[14,631,632,634],{},[17,633,618],{}," Trim your system prompt to its core identity and behavioral rules. Move specific instructions (like \"how to handle refund requests\") into retrievable documents that get pulled in only when relevant. Your system prompt should be a compass, not an encyclopedia.",[604,636,638],{"id":637},"_3-unfiltered-tool-and-api-responses","3. Unfiltered Tool and API Responses",[14,640,641],{},"This one sneaks up on you. Your agent calls an API. The API returns a 4,000-token JSON blob. Your framework dumps the entire response into the context window. Multiply that by five tool calls in a single conversation, and you've burned 20,000 tokens on raw JSON that the model has to parse through on every subsequent turn.",[14,643,644,646],{},[17,645,618],{}," Parse and compress tool responses before they enter the context. If your agent calls a CRM API and gets back a full customer record, extract only the fields the agent actually needs. Return \"Customer: Jane Smith, Plan: Pro, Last ticket: June 12, Status: Active\" instead of the full 200-field JSON object.",[604,648,650],{"id":649},"_4-retrieval-augmented-generation-rag-overload","4. Retrieval Augmented Generation (RAG) Overload",[14,652,653],{},"RAG is supposed to give your agent access to external knowledge without stuffing everything into the prompt. But poorly tuned RAG does exactly the opposite. It retrieves too many chunks, the chunks are too long, or the relevance threshold is too low.",[14,655,656],{},"The result? Your context window fills up with marginally relevant document fragments that push out the actual conversation history.",[14,658,659,661],{},[17,660,618],{}," Set strict top-k limits (3 to 5 chunks max). Use hybrid search that combines vector similarity with keyword matching to improve relevance. And set a minimum similarity threshold so your agent isn't pulling in documents that scored 0.4 on a 0 to 1 relevance scale.",[604,663,665],{"id":664},"_5-no-persistent-memory-layer","5. No Persistent Memory Layer",[14,667,668],{},"This is the most fundamental issue, and it's the one that separates toy demos from production agents. If your agent relies entirely on the context window for \"memory,\" it will always lose context eventually. The window is finite. Conversations are not.",[14,670,671,673,674,321],{},[17,672,618],{}," Add a persistent memory layer that exists outside the context window. The agent writes important facts and context to this layer. On each new turn, only the relevant memories get retrieved and injected. The context window stays lean. For a deeper look at why agents forget and how memory layers solve it, see our guide on ",[199,675,677],{"href":676},"/blog/ai-agent-memory-why-forgets-how-to-fix","why AI agents forget and how to fix it",[75,679,681],{"id":680},"how-context-management-actually-works-when-done-right","How Context Management Actually Works (When Done Right)",[14,683,684],{},"Here's where it gets practical.",[14,686,687],{},"A well-designed agent doesn't just have a context window. It has a context strategy. That strategy involves three layers working together.",[14,689,690],{},[159,691],{"alt":692,"src":693},"Without persistent memory the context window is full because messages cram everything else out; with persistent memory the context window stays lean and the agent reads and writes to an external memory store on demand. Same agent, same conversations, different architecture","/img/blog/agent-losing-context-fix-guide-persistent-memory.jpg",[14,695,696,699],{},[17,697,698],{},"Layer 1: The Active Context."," This is your context window. It holds the system prompt, the last N messages (or a summary of older ones), any currently relevant retrieved documents, and the results of the most recent tool calls. Think of this as what the agent is actively thinking about right now.",[14,701,702,705],{},[17,703,704],{},"Layer 2: The Session Memory."," This sits just outside the context window but within the same conversation session. It stores the full conversation history, intermediate reasoning, and tool results that might be needed later. When the agent detects a callback to an earlier topic, it pulls from session memory into active context.",[14,707,708,711],{},[17,709,710],{},"Layer 3: The Persistent Memory."," This survives across sessions. Customer preferences learned in January should still be available in June. Key facts, user profiles, learned patterns. This layer uses a combination of vector search (for semantic queries like \"what does this customer usually complain about\") and keyword search (for exact lookups like customer ID or order number).",[14,713,714],{},"When these three layers work together, your agent never \"forgets.\" It just files things away and retrieves them when needed. Exactly like a competent human assistant would.",[14,716,717,718,180,721,724],{},"If this sounds like a lot of infrastructure to build yourself... it is. It's one of the reasons we built BetterClaw as a no-code AI agent builder. The platform handles smart context management, persistent memory with hybrid vector and keyword search, and automatic token optimization. You focus on what your agent should do. The context plumbing is handled. ",[199,719,720],{"href":391},"Free plan available",[199,722,723],{"href":428},"full pricing here",", bring your own API keys with zero markup.",[75,726,728],{"id":727},"the-hidden-cost-of-getting-this-wrong","The Hidden Cost of Getting This Wrong",[14,730,731],{},"Context management isn't just a technical problem. It's a cost problem.",[14,733,734],{},[159,735],{"alt":736,"src":737},"The cost of bad context management: the same agent running the same conversations at 80K tokens/call wastes ~$6K/month on bloated context, versus 15K tokens/call at ~$1.1K/month with managed context — a 5x cost difference","/img/blog/agent-losing-context-fix-guide-cost-of-bad-context.jpg",[14,739,740],{},"Every token in your context window costs money. With GPT-4o, input tokens run about $2.50 per million. If your agent is stuffing 80,000 tokens of unnecessary history into every API call, and it handles 1,000 conversations per day, you're burning roughly $200/day on wasted context. That's $6,000 a month in pure overhead.",[14,742,743],{},"Gartner projects that 40% of enterprise applications will have embedded AI agents by end of 2026. McKinsey estimates the addressable value at $2.6 to $4.4 trillion. But none of that value materializes if your agents are hemorrhaging money on bloated context windows and losing context mid-conversation.",[14,745,746],{},"The agents that actually make it to production share three traits: lean context management, persistent memory, and graceful degradation when context limits are hit.",[75,748,750],{"id":749},"when-to-reach-for-persistent-memory-vs-a-bigger-context-window","When to Reach for Persistent Memory vs. a Bigger Context Window",[14,752,753],{},"There's a tempting shortcut: just use a model with a larger context window. Gemini's million-token window feels like it should solve everything, right?",[14,755,756],{},"Not really. Here's why.",[14,758,759],{},"Larger context windows are slower. They cost more per call. And research consistently shows that model performance degrades on information in the middle of very long contexts, a phenomenon researchers call \"lost in the middle.\" Throwing more tokens at the problem is like buying a bigger desk instead of learning to file.",[14,761,762],{},"Use a bigger context window when you genuinely need the model to reason over a large, interconnected document in a single pass. A legal contract. A full codebase file. A research paper.",[14,764,765],{},"Use persistent memory when you need information to survive across conversations, when relevance varies by turn, or when you want to keep costs predictable. For most agent use cases (support, sales, ops automation, personal assistants), persistent memory wins.",[14,767,768],{},"BetterClaw's context management approach combines both: smart context windowing that prevents token bloat within a session, plus persistent memory with hybrid search across sessions. Agents built on the platform support 28+ AI model providers, so you can pick the right context window size for your specific use case without restructuring your entire agent.",[75,770,772],{"id":771},"a-quick-diagnostic-checklist","A Quick Diagnostic Checklist",[14,774,775],{},"If your agent is losing context right now, run through this:",[777,778,779,783,786,789,792],"ul",{},[780,781,782],"li",{},"Is your conversation history growing unbounded? Add summarization after every 10 to 15 turns.",[780,784,785],{},"Is your system prompt over 2,000 tokens? Split it. Core identity stays in the prompt. Specific instructions go into retrievable docs.",[780,787,788],{},"Are tool responses entering the context window raw? Parse them down to only the fields your agent needs.",[780,790,791],{},"Is your RAG pulling more than 5 chunks per query? Tighten the top-k and raise your similarity threshold.",[780,793,794],{},"Does your agent need information from previous sessions? You need persistent memory, period. No context window hack will substitute for this.",[14,796,797],{},"If you're self-hosting an agent framework like CrewAI (47K+ GitHub stars, but requires Python and your own infrastructure) or LangGraph, you'll need to implement each of these layers yourself. That's engineering time measured in weeks, not hours.",[14,799,800,801,803],{},"If the idea of building context management infrastructure sounds like the wrong use of your time, that's exactly the problem BetterClaw's visual agent builder solves. ",[199,802,392],{"href":391},", no credit card, 1 agent and 500 credits a month. Pro is $49/month for teams that need more. Your first deploy takes about 60 seconds. We handle the context plumbing. You handle the interesting part.",[14,805,806,809,810,321],{},[199,807,808],{"href":391},"Start free here"," or ",[199,811,812],{"href":428},"see full pricing",[75,814,816],{"id":815},"the-part-that-actually-matters","The Part That Actually Matters",[14,818,819],{},"Context management isn't glamorous. Nobody's posting \"just shipped a sliding window summarizer\" on Twitter. But it's the difference between an agent that demos well and an agent that runs in production for months without someone paging you at 2 AM because it started sending gibberish to customers.",[14,821,822],{},"The agents that survive are the ones built on boring, reliable infrastructure. Memory that persists. Context that stays lean. Costs that stay predictable.",[14,824,825],{},"The exciting part isn't the plumbing. It's what your agent does once the plumbing works.",[75,827,439],{"id":438},[604,829,831],{"id":830},"what-is-agent-context-loss-and-why-does-it-happen","What is agent context loss and why does it happen?",[14,833,834],{},"Agent context loss is when an AI agent \"forgets\" information from earlier in a conversation or from previous sessions. It happens because every LLM has a finite context window, and when that window fills up with conversation history, tool responses, and retrieved documents, older information gets truncated or ignored. It's not a model deficiency. It's an architecture problem that requires smart context management and persistent memory to solve.",[604,836,838],{"id":837},"how-does-persistent-memory-compare-to-just-using-a-larger-context-window","How does persistent memory compare to just using a larger context window?",[14,840,841],{},"A larger context window (like Gemini's 1M tokens) lets you fit more information in a single call, but it costs more, runs slower, and research shows models struggle with information in the middle of very long contexts. Persistent memory stores facts externally and retrieves only what's relevant for each turn, keeping costs low and performance consistent. For most agent use cases, persistent memory paired with a moderately sized context window outperforms brute-forcing everything into one giant prompt.",[604,843,845],{"id":844},"how-do-i-fix-my-ai-agent-that-keeps-forgetting-instructions","How do I fix my AI agent that keeps forgetting instructions?",[14,847,848],{},"Start by checking the most common causes: unbounded conversation history (add summarization), oversized system prompts (trim to essentials), raw tool responses entering context (parse them down), over-retrieval from RAG (tighten top-k limits), and no persistent memory layer (add one). In most cases, implementing a sliding window with periodic summarization plus a basic persistent memory store resolves 90% of context loss issues.",[604,850,852],{"id":851},"how-much-does-agent-context-bloat-actually-cost","How much does agent context bloat actually cost?",[14,854,855],{},"It depends on your model and volume, but the math adds up fast. If your agent stuffs 80,000 unnecessary tokens into each call at GPT-4o input pricing (~$2.50/million tokens), and handles 1,000 conversations daily, you're spending roughly $6,000/month on wasted tokens alone. Smart context management can cut that to a fraction by keeping only relevant information in the active window. Platforms like BetterClaw include context optimization as a built-in feature starting at $0/month on the free plan.",[604,857,859],{"id":858},"is-persistent-memory-safe-for-handling-sensitive-customer-data","Is persistent memory safe for handling sensitive customer data?",[14,861,862],{},"It can be, but only with the right safeguards. You need encryption at rest (AES-256 minimum), strict access controls, and ideally an auto-purge mechanism for sensitive data like credentials or payment information. BetterClaw's persistent memory uses AES-256 encryption and automatically purges secrets from agent memory after 5 minutes. Agents run in isolated Docker containers with real-time health monitoring, so sensitive data never leaks between agents or persists longer than necessary.",{"title":226,"searchDepth":254,"depth":254,"links":864},[865,866,873,874,875,876,877,878],{"id":573,"depth":254,"text":574},{"id":598,"depth":254,"text":599,"children":867},[868,869,870,871,872],{"id":606,"depth":260,"text":607},{"id":622,"depth":260,"text":623},{"id":637,"depth":260,"text":638},{"id":649,"depth":260,"text":650},{"id":664,"depth":260,"text":665},{"id":680,"depth":254,"text":681},{"id":727,"depth":254,"text":728},{"id":749,"depth":254,"text":750},{"id":771,"depth":254,"text":772},{"id":815,"depth":254,"text":816},{"id":438,"depth":254,"text":439,"children":879},[880,881,882,883,884],{"id":830,"depth":260,"text":831},{"id":837,"depth":260,"text":838},{"id":844,"depth":260,"text":845},{"id":851,"depth":260,"text":852},{"id":858,"depth":260,"text":859},"2026-07-01","Your AI agent keeps forgetting instructions? Diagnose context window overflow, fix token bloat, and add persistent memory. Step-by-step guide.","/img/blog/agent-losing-context-fix-guide.jpg",{},"/blog/agent-losing-context-fix-guide","9 min read",{"title":549,"description":886},"Agent Losing Context? Fix Memory Leaks in AI Agents (2026)","blog/agent-losing-context-fix-guide",[895,896,897,898,899,900],"agent losing context","agent memory leak","ai agent context management","agent forgetting instructions","context window overflow","persistent memory ai agent","zVoK0kXd51WDZ2qg5Ug93WKOjRiHXiAfBfyndHGNIGs",{"id":903,"title":904,"author":905,"body":906,"category":521,"date":1291,"description":1292,"extension":524,"featured":525,"hideToc":525,"image":1293,"imageHeight":544,"imageWidth":544,"meta":1294,"navigation":530,"path":1295,"readingTime":890,"seo":1296,"seoTitle":1297,"stem":1298,"tags":1299,"updatedDate":1291,"__hash__":1306},"blog/blog/agent-rules-drift-fix.md","Why Your AI Agent Forgets Its Instructions After 20 Messages (and How to Fix It)",{"name":7,"role":8,"avatar":9},{"type":11,"value":907,"toc":1275},[908,913,916,919,922,925,929,932,935,941,944,951,954,957,961,964,970,976,982,988,991,995,1006,1012,1018,1024,1030,1036,1042,1048,1052,1055,1060,1093,1098,1150,1153,1156,1159,1163,1169,1175,1181,1188,1194,1206,1210,1213,1216,1228,1230,1234,1237,1241,1244,1248,1251,1255,1258,1262,1265,1269,1272],[14,909,910],{},[17,911,912],{},"It's not a bug. It's math. Here's the math, and 7 fixes that actually hold.",[14,914,915],{},"Your agent's SOUL.md says never exceed 3 sentences. For the first 15 messages, it's perfect. Short, punchy replies. Exactly what you asked for.",[14,917,918],{},"Then around message 20, the responses start getting longer. By message 30, it's writing 4 paragraphs. By message 40, it's forgotten the 3-sentence rule entirely, like it never read the file at all.",[14,920,921],{},"You didn't change anything. The agent just... drifted.",[14,923,924],{},"This is rules drift. It happens to every agent, on every platform, running on every model. It's not a bug you can patch. It's a direct consequence of how context windows work, and once you see the mechanism, you'll never be surprised by it again.",[75,926,928],{"id":927},"why-agents-drift-the-context-window-math","Why Agents Drift: The Context Window Math",[14,930,931],{},"Here's the part nobody explains clearly. Every message in a conversation gets added to the context window. Your SOUL.md instructions sit at the very beginning of that context, planted before message 1 even happens.",[14,933,934],{},"As the conversation grows, your instructions stay exactly the same size. But the conversation around them keeps growing. So your rules become a smaller and smaller percentage of the total context with every single message.",[14,936,937],{},[159,938],{"alt":939,"src":940},"The shrinking percentage: why rules drift is math, not magic. Three stacked bars show your fixed-size instructions as a shrinking share of the context window: 40% at message 5, 12% at message 20, 3% at message 40, as conversation history grows underneath. Your rules are still there, they're just buried under all of it.","/img/blog/agent-rules-drift-fix-shrinking-percentage.jpg",[14,942,943],{},"By message 20 to 30, the model starts paying less attention to those early instructions. Not because they vanished. Because they're now buried under thousands of tokens of back-and-forth, and the model weighs recent messages more heavily than old instructions when the two start to conflict.",[14,945,946,947,950],{},"Imagine you're in a meeting. Someone laid out the ground rules at the start: keep answers brief, don't go off-topic. An hour later, the conversation has drifted through 30 different topics. The rules are still technically in effect. Everyone's just forgotten them. That's what happens to your agent. (For the memory-file angle on the same problem, see our guide on ",[199,948,949],{"href":676},"why agents forget and how to fix it",".)",[14,952,953],{},"This isn't a hypothetical. Real production incidents trace back to exactly this pattern. One well-documented case involved an alignment researcher's OpenClaw agent that was told, in chat, not to touch her inbox without explicit approval. The instruction was never written to a file. When her real inbox overwhelmed the context window, the session compressed its history, the unsaved stop command quietly disappeared from the summary, and the agent went back to autonomous mode and started deleting emails.",[14,955,956],{},"That's not an edge case. That's the exact failure mode this whole article is about, just at a scale that made headlines.",[75,958,960],{"id":959},"how-to-spot-rules-drift-before-it-breaks-things","How to Spot Rules Drift Before It Breaks Things",[14,962,963],{},"The frustrating part about drift is that it's invisible until someone notices, and by then the agent has usually done something you didn't want. Here's how to catch it early.",[14,965,966,969],{},[17,967,968],{},"Check response length over time."," If your SOUL.md says \"concise\" and responses are creeping longer message over message, drift has already started.",[14,971,972,975],{},[17,973,974],{},"Test a known constraint every 10 messages."," Ask the agent something that should trigger a hard rule. If the rule doesn't fire, you've found drift in real time instead of after the fact.",[14,977,978,981],{},[17,979,980],{},"Monitor for banned words or phrases."," If SOUL.md says never say \"absolutely\" and it shows up around message 25, that's not a fluke. That's the instruction losing its grip.",[14,983,984,987],{},[17,985,986],{},"Log and compare."," Save the agent's response to the exact same prompt at message 5 and again at message 35. The difference between those two answers is your drift, made visible.",[14,989,990],{},"Here's what nobody tells you about this diagnostic: the comparison test is the most honest one, because it removes every variable except time and volume. Same question, same agent, same rules on paper. Only the conversation length changed.",[75,992,994],{"id":993},"_7-fixes-for-agent-rules-drift","7 Fixes for Agent Rules Drift",[14,996,997,1000,1001,1005],{},[17,998,999],{},"Fix 1: Keep SOUL.md under 500 tokens."," Shorter instructions survive longer in the context window, full stop. Every unnecessary word accelerates drift, because every word is competing with the actual rules for the model's shrinking attention. \"Be a helpful assistant that always provides thorough and detailed responses while maintaining a professional tone\" becomes \"Be concise. Professional tone. Max 3 sentences.\" Same intent, a fraction of the tokens. If you haven't already, our ",[199,1002,1004],{"href":1003},"/blog/soul-md-agents-md-configuration-guide","SOUL.md and AGENTS.md configuration guide"," covers exactly what belongs in each file and where the token ceiling actually sits.",[14,1007,1008,1011],{},[17,1009,1010],{},"Fix 2: Use negative constraints instead of positive aspirations."," \"Never exceed 3 sentences\" survives drift noticeably better than \"keep responses brief.\" Negative rules, the \"never,\" \"do not,\" \"under no circumstances\" kind, create hard boundaries the model can check itself against. Soft guidance like \"try to\" or \"aim for\" gives the model permission to interpret, and interpretation is exactly where drift creeps in.",[14,1013,1014,1017],{},[17,1015,1016],{},"Fix 3: Use /new to clear the conversation buffer, but only after saving what matters."," In OpenClaw and Hermes, /new clears the active conversation history, and your workspace memory files stay on disk. That part is genuinely useful. Here's the catch most guides skip: manual resets don't automatically flush unsaved context to a memory file first. Anything discussed in chat but never written down disappears the moment you reset, same as it would in a compaction event. The safe pattern is to have the agent write a quick session note to memory before you run /new, not to assume the reset preserves everything. Do that, and resetting every 15 to 20 messages genuinely does return your instructions to a large percentage of the context, which resets drift along with it.",[14,1019,1020,1023],{},[17,1021,1022],{},"Fix 4: Repeat critical rules in AGENTS.md procedures."," If SOUL.md says never send emails without approval, restate it inside the relevant workflow step too: \"Step 5: Generate draft reply. DO NOT SEND. Wait for user approval.\" Redundancy across both files gives the rule two separate footholds instead of one, and a rule with two footholds is harder for drift to erode.",[14,1025,1026,1029],{},[17,1027,1028],{},"Fix 5: Use system-level instruction reinforcement where the platform supports it."," Some setups support pinned instructions that get re-injected into context at intervals instead of sitting only at the start. On raw OpenClaw, this usually means manually re-injecting SOUL.md content every N messages through a skill or hook. On BetterClaw, this is handled by smart context management, so critical rules don't rely on a human remembering to re-inject them.",[14,1031,1032,1035],{},[17,1033,1034],{},"Fix 6: Break long tasks into shorter sessions."," One 50-message conversation drifts. Three sessions of roughly 15 messages each, each one starting fresh, gets full SOUL.md attention every time. Pass what matters between sessions using memory files, not by dragging the whole conversation history forward. This is the same principle as Fix 3, just applied proactively instead of as a mid-conversation rescue.",[14,1037,1038,1041],{},[17,1039,1040],{},"Fix 7: Test with adversarial prompts at message 25 and beyond."," Past message 25, deliberately try to get your agent to break its own rules. Ask it to do the exact thing SOUL.md prohibits. If it complies, you've found drift before a real user did. Keep adjusting SOUL.md wording until the rule holds at message 30 and beyond. The rules that survive adversarial testing at message 30 are the only rules you should trust in production. Everything else is a suggestion wearing a rule's clothing.",[14,1043,1044],{},[159,1045],{"alt":1046,"src":1047},"Four early warning sensors for rules drift. 1. Response length monitor: SOUL.md says \"concise\" but replies keep creeping longer. 2. Known constraint test: trigger a hard rule every 10 messages; if it doesn't fire, drift is detected in real time. 3. Banned word detector: a forbidden word appearing around message 25 isn't a fluke, it's the rule losing its grip. 4. Most honest test, the comparison test: ask the exact same question at message 5 and message 35; the difference is your drift, made visible.","/img/blog/agent-rules-drift-fix-warning-sensors.jpg",[75,1049,1051],{"id":1050},"real-example-soulmd-that-drifts-vs-soulmd-that-holds","Real Example: SOUL.md That Drifts vs SOUL.md That Holds",[14,1053,1054],{},"Here are two versions of the same email triage agent's SOUL.md. Same intent. Wildly different survival rate.",[14,1056,1057],{},[17,1058,1059],{},"Version that drifts:",[221,1061,1065],{"className":1062,"code":1063,"language":1064,"meta":226,"style":226},"language-markdown shiki shiki-themes github-light","You are a helpful and knowledgeable email assistant. Your goal is to help\nthe user manage their inbox efficiently. Try to keep your responses concise\nand focused. When classifying emails, consider the sender's importance,\nthe subject line content, and any time-sensitive language. Aim for accuracy\nin your classifications and provide brief summaries when helpful.\n","markdown",[28,1066,1067,1073,1078,1083,1088],{"__ignoreMap":226},[230,1068,1069],{"class":232,"line":233},[230,1070,1072],{"class":1071},"sgsFI","You are a helpful and knowledgeable email assistant. Your goal is to help\n",[230,1074,1075],{"class":232,"line":254},[230,1076,1077],{"class":1071},"the user manage their inbox efficiently. Try to keep your responses concise\n",[230,1079,1080],{"class":232,"line":260},[230,1081,1082],{"class":1071},"and focused. When classifying emails, consider the sender's importance,\n",[230,1084,1085],{"class":232,"line":266},[230,1086,1087],{"class":1071},"the subject line content, and any time-sensitive language. Aim for accuracy\n",[230,1089,1090],{"class":232,"line":272},[230,1091,1092],{"class":1071},"in your classifications and provide brief summaries when helpful.\n",[14,1094,1095],{},[17,1096,1097],{},"Version that holds:",[221,1099,1101],{"className":1062,"code":1100,"language":1064,"meta":226,"style":226},"Email triage agent.\n\nRULES:\n- Max 2 sentences per response.\n- NEVER classify unknown senders as \"safe.\"\n- NEVER send, reply, or delete. Read-only.\n- If unsure, mark \"needs review.\" Never guess.\n",[28,1102,1103,1108,1113,1118,1127,1134,1142],{"__ignoreMap":226},[230,1104,1105],{"class":232,"line":233},[230,1106,1107],{"class":1071},"Email triage agent.\n",[230,1109,1110],{"class":232,"line":254},[230,1111,1112],{"emptyLinePlaceholder":530},"\n",[230,1114,1115],{"class":232,"line":260},[230,1116,1117],{"class":1071},"RULES:\n",[230,1119,1120,1124],{"class":232,"line":266},[230,1121,1123],{"class":1122},"sqxcx","-",[230,1125,1126],{"class":1071}," Max 2 sentences per response.\n",[230,1128,1129,1131],{"class":232,"line":272},[230,1130,1123],{"class":1122},[230,1132,1133],{"class":1071}," NEVER classify unknown senders as \"safe.\"\n",[230,1135,1137,1139],{"class":232,"line":1136},6,[230,1138,1123],{"class":1122},[230,1140,1141],{"class":1071}," NEVER send, reply, or delete. Read-only.\n",[230,1143,1145,1147],{"class":232,"line":1144},7,[230,1146,1123],{"class":1122},[230,1148,1149],{"class":1071}," If unsure, mark \"needs review.\" Never guess.\n",[14,1151,1152],{},"The first version is 80 words of suggestions. The second is 35 words of laws. Agents follow laws. They forget suggestions.",[14,1154,1155],{},"Look at the specific language. \"Try to keep responses concise\" versus \"max 2 sentences.\" \"Aim for accuracy\" versus \"if unsure, mark needs review, never guess.\" One gives the model room to drift. The other gives it nothing to negotiate with.",[14,1157,1158],{},"We rewrote a client's production SOUL.md using exactly this pattern and watched the same known-constraint test that failed at message 22 hold clean past message 40. Nothing else about the setup changed. Just the wording.",[75,1160,1162],{"id":1161},"platform-notes-on-where-drift-shows-up","Platform notes on where drift shows up",[14,1164,1165,1168],{},[17,1166,1167],{},"OpenClaw and Hermes:"," use /new to reset context, and remember that memory files persist across the reset while unsaved chat context does not. Get in the habit of asking the agent to jot a quick note before you clear the buffer.",[14,1170,1171,1174],{},[17,1172,1173],{},"Claude Code:"," context management is handled by the runtime, but drift still shows up in long sessions the same way it does anywhere else. The practical fix here is procedural: break work into smaller commits and smaller sessions rather than one marathon conversation, so instructions never have to survive fifty turns unassisted.",[14,1176,1177,1180],{},[17,1178,1179],{},"BetterClaw:"," smart context management handles instruction reinforcement automatically, so rules drift is meaningfully reduced compared to a raw OpenClaw setup where you're manually re-injecting SOUL.md content yourself. That's just how the platform is built, not a sales pitch, it's the same underlying context math, just with the reinforcement handled for you instead of left as a DIY project.",[14,1182,1183,1184,1187],{},"If manually babysitting a context window and re-injecting your own instructions sounds like the wrong use of a Tuesday afternoon, that's a big part of why we built BetterClaw the way we did. The ",[199,1185,1186],{"href":391},"free plan"," includes one agent, 500 credits a month, no credit card, and the same smart context management that's reducing drift on every other agent running on the platform.",[14,1189,1190],{},[159,1191],{"alt":1192,"src":1193},"Three platforms, same math, different levels of DIY. OpenClaw / Hermes: drift happens, /new resets context, memory files survive but unsaved chat is gone, so ask the agent to write a session note before every reset. Claude Code: drift happens, the runtime manages some context, and the practical fix is smaller commits and shorter sessions instead of a 50-turn marathon. BetterClaw: smart context management handles reinforcement automatically, so rules don't rely on human re-injection. The honest takeaway: rules drift isn't a config mistake, it's a sign your agent has been running long enough to matter.","/img/blog/agent-rules-drift-fix-three-platforms.jpg",[14,1195,1196,1197,1201,1202,321],{},"Drift isn't unique to email agents either. Anywhere an agent runs long, unattended, or across many turns, the same math applies, and shorter, tighter instruction files paired with fewer redundant tokens in your skills stack both help. If you're curious how much of your token budget your integrations are actually consuming before instructions even enter the picture, our breakdown on ",[199,1198,1200],{"href":1199},"/blog/skills-that-reduce-token-usage","reducing token usage from skills"," is worth a read. And if you're seeing responses cut off entirely rather than just drifting in tone, that's usually a related but distinct problem, covered in our guide to ",[199,1203,1205],{"href":1204},"/blog/hermes-response-truncated-fix","fixing truncated Hermes responses",[75,1207,1209],{"id":1208},"the-honest-takeaway","The honest takeaway",[14,1211,1212],{},"Rules drift isn't a sign you configured something wrong. It's a sign your agent has been running long enough to matter, which is honestly a good problem to have.",[14,1214,1215],{},"The fix was never about writing a longer, more thorough SOUL.md. It was about writing a shorter one, with harder edges, and building in a reset habit before the context window quietly decides which of your rules it can afford to forget.",[14,1217,1218,1219,1223,1224,1227],{},"If you'd rather have that reinforcement handled automatically instead of manually testing your own agent at message 30 every week, ",[199,1220,1222],{"href":421,"rel":1221},[423],"give BetterClaw a try",". Free plan with one agent and 500 credits a month, no credit card. ",[199,1225,1226],{"href":428},"$49 a month for Pro"," when you're ready to run more than one. Your first deploy takes about 60 seconds. We handle keeping your rules in view. You handle deciding what they say.",[75,1229,439],{"id":438},[604,1231,1233],{"id":1232},"why-does-my-ai-agent-forget-its-instructions","Why does my AI agent forget its instructions?",[14,1235,1236],{},"Your agent's instructions sit at the start of the context window and stay a fixed size, while the conversation around them keeps growing. As the conversation gets longer, those instructions become a smaller share of the total context, and the model starts weighing recent messages more heavily. The instructions aren't deleted, they're just outweighed.",[604,1238,1240],{"id":1239},"how-many-messages-before-an-ai-agent-starts-drifting","How many messages before an AI agent starts drifting?",[14,1242,1243],{},"Most agents start showing measurable drift somewhere between message 20 and 30, though the exact point depends on how long your SOUL.md is and how strict your model's context handling is. Shorter instruction files with hard negative constraints tend to hold noticeably longer than long, soft, aspirational ones.",[604,1245,1247],{"id":1246},"how-to-fix-agent-rules-drift","How to fix agent rules drift?",[14,1249,1250],{},"Keep SOUL.md under 500 tokens, use negative constraints instead of soft suggestions, repeat critical rules in AGENTS.md as well, and reset the conversation periodically with something like OpenClaw's /new command after saving any context worth keeping. Testing your agent with adversarial prompts past message 25 is the most reliable way to confirm a fix actually worked.",[604,1252,1254],{"id":1253},"what-is-the-best-length-for-soulmd","What is the best length for SOUL.md?",[14,1256,1257],{},"Keep SOUL.md under 400 to 500 tokens, which is roughly 300 to 400 words. Beyond that ceiling, instructions start losing the model's attention faster as the conversation grows, so shorter files with harder rules consistently outperform longer files full of soft guidance.",[604,1259,1261],{"id":1260},"does-new-reset-agent-memory-in-openclaw","Does /new reset agent memory in OpenClaw?",[14,1263,1264],{},"/new clears the active conversation history but leaves your workspace memory files, like MEMORY.md and daily notes, intact on disk. What it doesn't do is automatically save unsaved context from the chat before wiping it, so anything important that was only discussed in conversation and never written to a file will be lost on reset.",[604,1266,1268],{"id":1267},"how-to-make-ai-agent-follow-instructions-longer","How to make AI agent follow instructions longer?",[14,1270,1271],{},"Write instructions as hard negative constraints rather than soft goals, keep the file short enough that it doesn't get diluted by conversation growth, and reinforce critical rules in more than one place, such as both SOUL.md and AGENTS.md. Periodically resetting the session, or using a platform with automatic context reinforcement, addresses the problem at its root instead of just delaying it.",[510,1273,1274],{},"html pre.shiki code .sgsFI, html code.shiki .sgsFI{--shiki-default:#24292E}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .sqxcx, html code.shiki .sqxcx{--shiki-default:#E36209}",{"title":226,"searchDepth":254,"depth":254,"links":1276},[1277,1278,1279,1280,1281,1282,1283],{"id":927,"depth":254,"text":928},{"id":959,"depth":254,"text":960},{"id":993,"depth":254,"text":994},{"id":1050,"depth":254,"text":1051},{"id":1161,"depth":254,"text":1162},{"id":1208,"depth":254,"text":1209},{"id":438,"depth":254,"text":439,"children":1284},[1285,1286,1287,1288,1289,1290],{"id":1232,"depth":260,"text":1233},{"id":1239,"depth":260,"text":1240},{"id":1246,"depth":260,"text":1247},{"id":1253,"depth":260,"text":1254},{"id":1260,"depth":260,"text":1261},{"id":1267,"depth":260,"text":1268},"2026-07-13","AI agents start ignoring their rules around message 20-30. Here's why it happens, how to detect it, and 7 fixes that actually work. Real examples included.","/img/blog/agent-rules-drift-fix.jpg",{},"/blog/agent-rules-drift-fix",{"title":904,"description":1292},"Why AI Agents Forget Instructions After 20 Messages","blog/agent-rules-drift-fix",[1300,1301,1302,1303,1304,1305],"ai agent forgetting instructions","agent ignoring system prompt","agent rules drift","openclaw agent personality drift","hermes agent ignoring soul.md","agent rules not working after long conversation","pqqXBITmHsIpULjS5wB7XwOhgHTbvIx9aCnTumSd41E",{"id":1308,"title":1309,"author":1310,"body":1311,"category":521,"date":1762,"description":1763,"extension":524,"featured":525,"hideToc":525,"image":1764,"imageHeight":544,"imageWidth":544,"meta":1765,"navigation":530,"path":1766,"readingTime":1767,"seo":1768,"seoTitle":1769,"stem":1770,"tags":1771,"updatedDate":1762,"__hash__":1779},"blog/blog/ai-agent-slow-latency-fix.md","Why Is My AI Agent So Slow? Diagnosing Latency Step by Step",{"name":7,"role":8,"avatar":9},{"type":11,"value":1312,"toc":1734},[1313,1324,1327,1330,1333,1336,1339,1342,1346,1349,1352,1358,1362,1365,1368,1374,1380,1386,1389,1392,1396,1399,1402,1405,1408,1411,1414,1418,1421,1425,1431,1441,1447,1453,1457,1465,1468,1471,1474,1477,1480,1486,1492,1498,1504,1507,1511,1514,1517,1524,1527,1530,1536,1542,1548,1554,1558,1561,1564,1567,1570,1573,1577,1580,1583,1586,1590,1593,1599,1605,1611,1617,1623,1629,1632,1635,1639,1642,1648,1659,1665,1676,1679,1682,1697,1699,1703,1706,1710,1713,1717,1720,1724,1727,1731],[21,1314,1315],{"type":23},[14,1316,1317,357,1320,1323],{},[17,1318,1319],{},"Quick fix:",[28,1321,1322],{},"Why is my AI agent so slow?"," - the model is almost never the bottleneck. A single LLM call takes ~800ms, so if your agent takes 15 seconds, 14.2 seconds of latency lives elsewhere. The slowness is usually context bloat (Layer 2) and multi-step compounding (Layer 5) working together. Diagnose before switching models: log your input token count per request - if it exceeds 30,000 tokens you have context bloat, so compress conversation history into summaries, load tool definitions on demand, and filter tool results before they enter context. Then parallelize independent tool calls and reduce total workflow steps.",[14,1325,1326],{},"We had an agent handling email triage. It read new emails, classified urgency, drafted responses for low-priority items, and flagged high-priority ones for human review.",[14,1328,1329],{},"In testing, it was fast. Sub-two-second responses. We were thrilled.",[14,1331,1332],{},"In production, it took 15 seconds per email. Sometimes 20. Users started complaining within the first hour.",[14,1334,1335],{},"My first instinct: the model is too slow, let's switch to something faster. So we swapped Claude Sonnet for GPT-5 Nano. Barely any improvement. Maybe half a second shaved off.",[14,1337,1338],{},"That's when it hit me. The model wasn't the bottleneck. The model was doing its job in under a second. Everything else around it was eating the other 14 seconds.",[14,1340,1341],{},"If your AI agent is slow, the model is almost never the real problem. The real problem lives in one of five places. Here's how to find it.",[75,1343,1345],{"id":1344},"the-five-latency-layers-debug-in-this-order","The five latency layers (debug in this order)",[14,1347,1348],{},"Most people start debugging agent latency by looking at model benchmarks. \"Maybe I need a faster LLM.\" That's like diagnosing a slow website by replacing the database when the real problem is unoptimized SQL queries hitting the database 47 times per page load.",[14,1350,1351],{},"Agent latency has five layers. They compound multiplicatively, not additively. A problem at layer 3 makes layer 4 worse, which makes layer 5 catastrophic.",[14,1353,1354],{},[159,1355],{"alt":1356,"src":1357},"The Five Latency Layers of an AI Agent, stacked to debug top to bottom without skipping: Layer 1 model speed (TTFT plus tok/s, most people blame this but it's rarely the real issue), Layer 2 context window size (every token in context costs processing time on every request), Layer 3 tool execution (external API calls, unmeasured and often the biggest offender), Layer 4 network plus API routing (geography adds up across 8-12 API calls per task), and Layer 5 multi-step compounding (every step multiplies all previous layers, the real killer). A problem at Layer 3 makes Layer 4 worse, and Layer 4 makes Layer 5 catastrophic","/img/blog/ai-agent-latency-five-layers.jpg",[75,1359,1361],{"id":1360},"layer-1-model-speed-the-one-everyone-checks-first","Layer 1: Model speed (the one everyone checks first)",[14,1363,1364],{},"Time to first token (TTFT) measures how long after you send a prompt the model starts generating. Per-token latency (tok/s) measures how fast it generates once it starts.",[14,1366,1367],{},"Here's where things stand in 2026:",[14,1369,1370,1373],{},[17,1371,1372],{},"Fastest TTFT:"," Claude Haiku 4.5 at around 597ms on medium prompts. Mistral Large and GPT-5.2 also hit sub-second consistently.",[14,1375,1376,1379],{},[17,1377,1378],{},"Fastest throughput:"," Gemini 2.5 Flash at 146-173 tokens per second. Mercury 2 by Inception hits 789 tok/s but with quality tradeoffs. Gemini 3.5 Flash reaches 284 tok/s.",[14,1381,1382,1385],{},[17,1383,1384],{},"Reasoning models are intentionally slow."," Models like o3, GPT-5, and Gemini Deep Think use chain-of-thought processing. They generate internal \"thinking\" tokens before the visible answer. TTFT can be 10-150 seconds. This isn't a bug. It's the architecture.",[14,1387,1388],{},"Here's the thing: for most agent workloads, model speed is not your bottleneck. A single LLM call takes about 800 milliseconds. If your agent is taking 15 seconds, the model used 800ms of that. You have 14.2 seconds of latency living somewhere else.",[14,1390,1391],{},"Before switching models, measure your actual model latency. Log the timestamp when you send the request and when the first token arrives. If it's under 2 seconds, your problem isn't the model.",[75,1393,1395],{"id":1394},"layer-2-context-window-bloat-the-silent-killer","Layer 2: Context window bloat (the silent killer)",[14,1397,1398],{},"This is where most agent latency actually lives. And it's invisible unless you're counting tokens.",[14,1400,1401],{},"Every time your agent makes a request, it sends the entire conversation context to the model. System prompt. Conversation history. Tool definitions. Previous tool results. Memory context. All of it.",[14,1403,1404],{},"The numbers are wild. Research from Agenteer found that a single Jira integration adds roughly 17,000 tokens just for tool definitions. Across a typical agent setup with multiple integrations, 134,000 tokens (67% of a 200K context window) get consumed by definitions before the agent starts working.",[14,1406,1407],{},"That's the equivalent of walking into a restaurant and reading a 500-page menu before you can order water.",[14,1409,1410],{},"More tokens means more processing time. A 2,000-token prompt processes in under a second. A 100,000-token prompt with tool definitions, conversation history, and previous results takes significantly longer. The model has to read and attend to every token before generating a response.",[14,1412,1413],{},"Your context window is RAM, not storage. Everything in it costs processing time on every single request.",[604,1415,1417],{"id":1416},"how-to-diagnose-it","How to diagnose it",[14,1419,1420],{},"Log your input token count for each request. If it's growing with every turn of the conversation, you have context bloat. If it starts high (above 20,000 tokens) even on the first turn, your tool definitions and system prompts are too heavy.",[604,1422,1424],{"id":1423},"how-to-fix-it","How to fix it",[14,1426,1427,1430],{},[17,1428,1429],{},"Don't load all tool definitions upfront."," Anthropic's own research showed that Opus 4's tool selection accuracy improved from 49% to 74% when the agent searched for relevant tools on demand instead of parsing all definitions at once. Fewer tools in context means faster processing and better accuracy.",[14,1432,1433,1436,1437,321],{},[17,1434,1435],{},"Summarize conversation history."," Instead of sending the full conversation, compress older turns into summaries. The Mem0 framework published 2026 benchmarks showing that a two-layer memory architecture (summarized context plus targeted retrieval) used 4x fewer tokens than full-context approaches while cutting latency by 91% and actually improving accuracy by 18.7 percentage points. This is the core of how ",[199,1438,1440],{"href":1439},"/blog/how-ai-agent-memory-works","AI agent memory works",[14,1442,1443,1446],{},[17,1444,1445],{},"Trim tool results aggressively."," A single MCP server call that returns a 50-field JSON blob when you only need 3 fields wastes thousands of tokens. Filter tool results before they enter context.",[14,1448,1449],{},[159,1450],{"alt":1451,"src":1452},"Context Bloat, the before and after. Before optimization: 119K tokens and 17-second p95 latency, made up of a 2K system prompt, 17K tool definitions, 40K conversation history and 60K tool results. After optimization: 11K tokens and 1.4-second p95 latency, made up of a 2K system prompt, 3K active tools only, 2K summarized history and 4K filtered results. A 91% latency reduction from context management alone, per Mem0 2026","/img/blog/ai-agent-latency-context-bloat-before-after.jpg",[75,1454,1456],{"id":1455},"layer-3-tool-execution-the-one-you-forgot-to-measure","Layer 3: Tool execution (the one you forgot to measure)",[14,1458,1459,1460,1464],{},"When your agent calls an external tool (send email, read CRM, query database, fetch webpage), the tool's execution time adds directly to the agent's response time. And most people never measure it. Per-task timing is standard on a ",[199,1461,1463],{"href":1462},"/","no-code AI agent builder",", which is usually the fastest way to find out which tool call is actually the slow one.",[14,1466,1467],{},"A Gmail API call to fetch recent emails: 200-800ms. A HubSpot CRM lookup: 300-1,200ms. A web scraping call: 1-5 seconds. A database query on an unindexed table: could be anything from 50ms to 30 seconds.",[14,1469,1470],{},"If your agent makes 3 tool calls in sequence (which is common for multi-step tasks), and each takes an average of 1 second, that's 3 seconds of tool execution time before the model even starts thinking about the next step.",[604,1472,1417],{"id":1473},"how-to-diagnose-it-1",[14,1475,1476],{},"Wrap every tool call with timing logs. You'll often find that one specific tool is responsible for 60-80% of total tool execution time.",[604,1478,1424],{"id":1479},"how-to-fix-it-1",[14,1481,1482,1485],{},[17,1483,1484],{},"Parallelize where possible."," If your agent needs data from Gmail and HubSpot, fetch both simultaneously instead of sequentially. This cuts tool execution time in half for independent calls.",[14,1487,1488],{},[159,1489],{"alt":1490,"src":1491},"Parallelize Independent Tool Calls. Run sequentially, a Gmail fetch, a 900ms HubSpot lookup and an 800ms DB query total 2,300ms. Run in parallel, the same three calls (Gmail 600ms, HubSpot 900ms, DB 800ms) all run at once and total 900ms because the slowest one wins, 2.5x faster. If your agent needs Gmail and HubSpot, fetch both at once, not one then the other","/img/blog/ai-agent-latency-parallelize-tool-calls.jpg",[14,1493,1494,1497],{},[17,1495,1496],{},"Cache repeated lookups."," If your agent queries the same CRM record multiple times in one conversation, cache the first result. Semantic caching, as Redis LangCache demonstrated, can reduce redundant API calls dramatically.",[14,1499,1500,1503],{},[17,1501,1502],{},"Set timeouts."," A tool call that takes 30 seconds because of an external API issue shouldn't freeze your entire agent. Set aggressive timeouts (3-5 seconds) and have fallback behavior.",[14,1505,1506],{},"This is one of the areas where managed agent platforms have an advantage over self-hosted setups. On BetterClaw, integrations are pre-optimized with connection pooling, caching, and timeout handling built in. When you self-host, you're building all of that yourself.",[75,1508,1510],{"id":1509},"layer-4-network-and-api-routing","Layer 4: Network and API routing",[14,1512,1513],{},"If your agent is on a VPS in Frankfurt and your users are in San Francisco, every API round-trip adds 100-200ms of network latency. For a single request, that's barely noticeable. For an agent that makes 8-12 API calls per task (LLM calls plus tool calls plus memory lookups), it adds up to 1-2 seconds of pure network overhead.",[604,1515,1417],{"id":1516},"how-to-diagnose-it-2",[14,1518,1519,1520,1523],{},"Run ",[28,1521,1522],{},"ping"," to your LLM provider's API endpoint from your agent's server. If it's over 100ms, geography is costing you.",[14,1525,1526],{},"Compare agent response times from the same machine the agent runs on versus from your actual user location. The difference is network overhead.",[604,1528,1424],{"id":1529},"how-to-fix-it-2",[14,1531,1532,1535],{},[17,1533,1534],{},"Deploy your agent close to your LLM provider's data centers."," Most major providers (OpenAI, Anthropic, Google) have US and EU endpoints. Match your agent's region to the provider's closest endpoint.",[14,1537,1538,1541],{},[17,1539,1540],{},"Use streaming."," Instead of waiting for the full response, stream tokens to the user as they're generated. This doesn't reduce total latency, but it reduces perceived latency dramatically. The user sees the response building in real-time instead of staring at a loading spinner for 8 seconds.",[14,1543,1544,1547],{},[17,1545,1546],{},"Minimize round-trips."," Every time your agent \"thinks\" (LLM call), \"acts\" (tool call), and \"observes\" (processes result), that's at minimum three network round-trips per step. Reducing the number of steps reduces total round-trip overhead proportionally.",[14,1549,1550],{},[159,1551],{"alt":1552,"src":1553},"Every Step Multiplies Every Other Layer, a line chart of total latency against number of agent steps. An optimized agent at 1.5 seconds per step reaches 15 seconds at 10 steps; a typical agent at 3 seconds per step reaches 30 seconds; and a bloated-context agent at 5 seconds per step reaches 50 seconds, a full minute per task and unusable for users. Most tasks should end by step 5. The fastest agent solves the problem in fewer steps, not faster steps","/img/blog/ai-agent-latency-every-step-multiplies.jpg",[75,1555,1557],{"id":1556},"layer-5-multi-step-compounding-the-multiplier-nobody-talks-about","Layer 5: Multi-step compounding (the multiplier nobody talks about)",[14,1559,1560],{},"Here's where agent latency gets genuinely painful. Every additional step in your agent's workflow multiplies all the previous layers.",[14,1562,1563],{},"A single LLM call: ~800ms. Totally fine.",[14,1565,1566],{},"An orchestrator-worker flow with a reflexion loop: 10-30 seconds. Stevens Institute research puts this as the primary engineering constraint for AI agents in 2026.",[14,1568,1569],{},"A 10-step agent task means 10 LLM calls, potentially 10 tool calls, context growing with every turn, and network overhead on every round-trip. If each step takes 1.5 seconds (fast!), your total task time is 15 seconds. If each step takes 3 seconds (normal), you're at 30 seconds. At 5 seconds per step (common with context bloat), you're looking at nearly a minute.",[14,1571,1572],{},"For user-facing applications, an orchestrator-worker flow with reflection can take 10-30 seconds. For customer support, this latency is often unacceptable.",[604,1574,1576],{"id":1575},"the-honest-math","The honest math",[14,1578,1579],{},"Let's say your agent handles a support ticket. Steps: (1) read the ticket, (2) look up customer in CRM, (3) check order history, (4) check knowledge base, (5) draft response, (6) format and send.",[14,1581,1582],{},"Six steps. Each step involves at least one LLM call (800ms), one tool call (500ms average), and context processing that grows each turn. Conservative estimate: 2 seconds per step = 12 seconds total. Realistic with context bloat: 4 seconds per step = 24 seconds total.",[14,1584,1585],{},"This is why smart agent design keeps step count low. The fastest agent isn't the one with the fastest model. It's the one that solves the problem in 3 steps instead of 8.",[75,1587,1589],{"id":1588},"the-60-second-diagnostic-checklist","The 60-second diagnostic checklist",[14,1591,1592],{},"When your agent is slow, run through this in order:",[14,1594,1595],{},[159,1596],{"alt":1597,"src":1598},"The 60-Second Agent Latency Diagnostic, a five-step checklist: 1, check model TTFT by logging request-to-first-token time, and if it's under 2 seconds the model isn't the problem; 2, count input tokens, and over 30K means context bloat, so check tool defs and history; 3, time each tool call by wrapping every tool with timing logs to find the slowest; 4, check geography by pinging your LLM provider from your server, and over 100ms means move closer; 5, count your steps, and over 5 for a typical task means redesign the workflow. Most slowness is layers 2 and 5 working together, so fix those first","/img/blog/ai-agent-latency-60-second-diagnostic.jpg",[14,1600,1601,1604],{},[17,1602,1603],{},"Check model TTFT."," Log the time between request sent and first token received. If it's under 2 seconds, the model isn't your problem.",[14,1606,1607,1610],{},[17,1608,1609],{},"Count input tokens."," If your input exceeds 30,000 tokens per request, you have context bloat. Check tool definitions, conversation history, and tool results.",[14,1612,1613,1616],{},[17,1614,1615],{},"Time each tool call."," Find the slowest one. It's probably responsible for most of your tool execution latency.",[14,1618,1619,1622],{},[17,1620,1621],{},"Check geography."," Ping your LLM provider from your agent's server. If it's over 100ms, move closer.",[14,1624,1625,1628],{},[17,1626,1627],{},"Count your steps."," If your agent takes more than 5 steps for a typical task, redesign the workflow to reduce steps.",[14,1630,1631],{},"Most agent slowness is layers 2 and 5 working together. Bloated context makes each step slower. More steps means more bloated context. It's a feedback loop that gets worse with every conversation turn.",[14,1633,1634],{},"This is exactly why we built smart context management into BetterClaw from day one. Token bloat is the number one production agent killer, and most self-hosted frameworks leave you to solve it yourself. On BetterClaw, context is automatically managed per agent, tool results are filtered before entering the window, and persistent memory uses hybrid vector plus keyword retrieval so your agent doesn't drag around dead conversation weight. Free plan with 1 agent and 500 credits a month. $49/month for Pro. BYOK with zero inference markup.",[75,1636,1638],{"id":1637},"when-switching-models-actually-helps-and-when-it-doesnt","When switching models actually helps (and when it doesn't)",[14,1640,1641],{},"After all that, there are specific cases where the model genuinely is the bottleneck:",[14,1643,1644],{},[159,1645],{"alt":1646,"src":1647},"Model Switch vs Infrastructure Fix, how to decide. Switch the model when you're using a reasoning model like o3 or GPT-5 for a simple classification task, you need faster streaming for customer-facing responses, you're running local inference where hardware is the real ceiling, or TTFT is over 2 seconds after measuring. Fix infrastructure first when context exceeds 30K tokens per request, tool calls are sequential and unparallelized, the agent takes more than 5 steps per task, or response times vary wildly between requests. The model is 800ms of a 15-second problem; fix the other 14.2 seconds first","/img/blog/ai-agent-latency-model-switch-vs-infrastructure.jpg",[14,1649,1650,1653,1654,1658],{},[17,1651,1652],{},"You're using a reasoning model for a classification task."," If your agent is classifying email urgency (simple task) using o3 or GPT-5 (reasoning model), you're paying 10-30 seconds of \"thinking\" latency for a task that Haiku or Flash can handle in 600ms. Match model size to task complexity. The framework for ",[199,1655,1657],{"href":1656},"/blog/how-to-choose-llm-for-your-task","choosing the right LLM per task"," covers this directly.",[14,1660,1661,1664],{},[17,1662,1663],{},"You need streaming for user-facing interactions."," Some providers stream faster than others. Gemini 2.5 Flash at 173 tok/s finishes a 1,100-token response in under 7 seconds. Slower models might take 20+ seconds for the same output. For customer-facing agents, streaming speed matters.",[14,1666,1667,1670,1671,1675],{},[17,1668,1669],{},"Your agent runs on local hardware."," If you're running local inference on a Mac Mini or mid-range GPU, the model is genuinely slow (3-5 seconds to first token, 30-40 tok/s). Cloud APIs are 5-10x faster for agent workloads. We break down the ",[199,1672,1674],{"href":1673},"/blog/apple-silicon-vs-nvidia-ai-agents","hardware speed gap"," in detail.",[14,1677,1678],{},"For everything else, fix layers 2-5 first. You'll get more speed improvement from trimming 50,000 tokens out of your context window than from switching to a model that's 200ms faster on TTFT.",[14,1680,1681],{},"The difference between a frustrating agent and a fast one usually isn't the model. It's whether someone bothered to measure where the latency actually lives.",[14,1683,1684,1685,1689,1690,1692,1693,1696],{},"If you'd rather skip the latency debugging entirely, ",[199,1686,1688],{"href":421,"rel":1687},[423],"give BetterClaw a look",". Context management, tool optimization, caching, and infrastructure are all handled. ",[199,1691,392],{"href":391}," with 1 agent and 500 credits a month. ",[199,1694,1695],{"href":428},"$49/month for Pro"," on Pro. Your agent deploys in 60 seconds. On infrastructure we've already optimized for speed.",[75,1698,439],{"id":438},[604,1700,1702],{"id":1701},"what-causes-ai-agent-latency","What causes AI agent latency?",[14,1704,1705],{},"AI agent latency comes from five layers: model speed (time to first token and generation rate), context window size (more tokens means more processing time per request), tool execution time (external API calls like CRM, email, or database lookups), network round-trips between your agent and API endpoints, and multi-step compounding where each workflow step multiplies all previous delays. In most cases, context bloat and step count cause more slowness than the model itself.",[604,1707,1709],{"id":1708},"how-does-llm-latency-differ-between-providers-in-2026","How does LLM latency differ between providers in 2026?",[14,1711,1712],{},"Claude Haiku 4.5 leads on time to first token at around 597ms. Gemini 2.5 Flash leads on throughput at 146-173 tokens per second. Reasoning models (o3, GPT-5, Gemini Deep Think) are intentionally slow, often 10-150 seconds to first token due to chain-of-thought processing. For agent workloads, the fastest practical choices are Gemini Flash variants for throughput and Claude Haiku for TTFT.",[604,1714,1716],{"id":1715},"how-do-i-reduce-my-ai-agents-response-time","How do I reduce my AI agent's response time?",[14,1718,1719],{},"Start by logging input token counts. If they exceed 30,000 tokens, compress conversation history into summaries, load tool definitions on demand instead of all at once, and filter tool results before they enter context. Mem0's 2026 benchmarks showed that optimized context management cut latency by 91% while improving accuracy. After fixing context, parallelize independent tool calls and reduce total workflow steps.",[604,1721,1723],{"id":1722},"does-switching-to-a-faster-llm-model-fix-agent-latency","Does switching to a faster LLM model fix agent latency?",[14,1725,1726],{},"Usually not. A single LLM call takes about 800ms. If your agent takes 15 seconds total, the model accounts for roughly 5% of the latency. The other 95% is context processing, tool execution, network overhead, and multi-step compounding. Switch models only when you're using a reasoning model for simple tasks, need faster streaming for user-facing responses, or running local inference where hardware is the genuine bottleneck.",[604,1728,1730],{"id":1729},"is-managed-hosting-faster-than-self-hosted-ai-agents","Is managed hosting faster than self-hosted AI agents?",[14,1732,1733],{},"Generally yes, for three reasons: managed platforms pre-optimize tool integrations with connection pooling and caching, they handle context management automatically to prevent token bloat, and they deploy on infrastructure close to major LLM provider data centers. BetterClaw's managed infrastructure includes smart context management, optimized integrations, and zero setup overhead. Self-hosting gives you full control but requires you to solve every latency layer yourself.",{"title":226,"searchDepth":254,"depth":254,"links":1735},[1736,1737,1738,1742,1746,1750,1753,1754,1755],{"id":1344,"depth":254,"text":1345},{"id":1360,"depth":254,"text":1361},{"id":1394,"depth":254,"text":1395,"children":1739},[1740,1741],{"id":1416,"depth":260,"text":1417},{"id":1423,"depth":260,"text":1424},{"id":1455,"depth":254,"text":1456,"children":1743},[1744,1745],{"id":1473,"depth":260,"text":1417},{"id":1479,"depth":260,"text":1424},{"id":1509,"depth":254,"text":1510,"children":1747},[1748,1749],{"id":1516,"depth":260,"text":1417},{"id":1529,"depth":260,"text":1424},{"id":1556,"depth":254,"text":1557,"children":1751},[1752],{"id":1575,"depth":260,"text":1576},{"id":1588,"depth":254,"text":1589},{"id":1637,"depth":254,"text":1638},{"id":438,"depth":254,"text":439,"children":1756},[1757,1758,1759,1760,1761],{"id":1701,"depth":260,"text":1702},{"id":1708,"depth":260,"text":1709},{"id":1715,"depth":260,"text":1716},{"id":1722,"depth":260,"text":1723},{"id":1729,"depth":260,"text":1730},"2026-06-08","AI agent taking 15+ seconds? The model isn't the bottleneck. Diagnose context bloat, tool lag, and step compounding with this 5-layer framework.","/img/blog/ai-agent-slow-latency-fix.jpg",{},"/blog/ai-agent-slow-latency-fix","11 min read",{"title":1309,"description":1763},"Why Is My AI Agent So Slow? Fix Latency Fast","blog/ai-agent-slow-latency-fix",[1772,1773,1774,1775,1776,1777,1778],"ai agent slow","llm latency","reduce ai agent latency","llm inference speed","ai agent performance","context window optimization","agent response time","NpmJw9lx7bwosEGvEiqWc0OBd1P4wrYRvVf_-CDZzFk",1787653223259]