Pi: "Response was truncated before completion."

Why it happens

Pi asks for the model's maxTokens on every request, but clamps it to the space left in the context window (context window minus the estimated conversation minus a 4,096-token safety margin). A full session or a low limit both shrink the room for the answer.

1. The model's maxTokens is too low for the job. Models you add yourself in ~/.pi/agent/models.json get 16,384 output tokens and a 128,000-token context window if you leave those fields out, and a long file write can exceed 16K.
2. The session is nearly full. Because Pi clamps the request to the remaining context, a long session leaves little output room even when maxTokens is large.
3. Thinking eats the budget. On Anthropic models Pi fits the thinking budget inside the same maxTokens, capped at maxTokens minus 1,024, so a high /thinking level leaves less room for the visible answer.
4. The provider caps output below what Pi expects. OpenRouter lists a per-model max completion size, and several free models cap it at 8,192 tokens. On Pi older than 0.75.5, Bedrock's 4,096-token default did the same (issue #4848).

The fix

1. 1 Run /model to confirm which provider and model are active, then look up that model's real output limit at the provider.
2. 2 For a model you defined, set both limits on its entry in ~/.pi/agent/models.json, for example contextWindow 64000 and maxTokens 32768. Opening /model reloads the file.
3. 3 For a built-in model, add a modelOverrides block under the provider, keyed by the model id, with the corrected maxTokens. Do not set it higher than the provider allows, or the request fails instead.
4. 4 Run /compact (optionally with instructions) or /new to start clean, so the clamp leaves more room for output.
5. 5 Lower the thinking level with /thinking if a reasoning model spends the budget before answering.
6. 6 Update Pi with pi update. Current releases compact and retry once automatically on early length stops (0.84.0) and label that recovery correctly (0.84.3).

How Pi differs from Hermes finish_reason=length

Hermes and Pi react to the same provider signal in opposite ways. Hermes sends no output cap to OpenAI-compatible servers and, when a reply stops on length, asks the model to continue up to four times before printing "response remained truncated after N continuation attempts".

Pi always sends an output cap and never auto-continues. Since 0.84.0 it does one thing: if the reply stopped below the model's configured maxTokens, Pi treats it as context pressure, compacts the session and retries once (PR #7540). If the reply used the full maxTokens, there is no automatic retry and you just see the red line.

When the provider is the one truncating

If Pi believes a model has 16,384 output tokens but the upstream provider stops at 8,192, the reply stops below Pi's limit. Pi then compacts and retries, the provider stops at 8,192 again, and you get "Truncated response recovery failed after one compact-and-retry attempt." Compaction cannot help here.

Set maxTokens to the provider's real cap so Pi stops treating it as context pressure, or switch to a paid route or a model with a larger completion limit.

Making Pi finish the job

The partial reply stays in the session, so you can type continue and the model picks up from there. If the cut-off content was a tool call, Pi does not run the half-written call, so ask for the file in smaller parts or as a series of edits instead of one large write.

Still failing?

• Check that the provider id and model id in models.json match exactly, because an unknown modelOverrides id is ignored silently.
• For Ollama or another local server, raise the server's own context size, since Pi cannot change it through the OpenAI-compatible API.
• If only one provider truncates, test the same prompt on another route to confirm the cap is upstream rather than in Pi.

Related errors

• "response truncated due to output length limit"
• "response remained truncated after N continuation attempts"