Context Management & Reliability

Progressive summarisation is lossy in a specific and dangerous way: it turns exact values into approximations. "$247.30 refunded on 14 March against order 88213" becomes "a refund was discussed", and the agent later contradicts what the customer was told.

The fix is to stop relying on the summary for those facts. Extract transactional details — amounts, dates, order numbers, statuses — into a persistent case facts block that is included in every prompt, outside the summarised history. For multi-issue sessions, keep that as its own structured layer.

Long inputs have a second problem: the lost-in-the-middle effect. Models handle the beginning and end of a long input reliably and drop findings from the middle. Put key findings summaries at the top of aggregated inputs and give the detail explicit section headers.

Tool results are the other context sink. An order lookup returning forty-plus fields when five are relevant will consume a large share of the window over a long conversation. Trim outputs to the relevant fields before they accumulate, and where subagents feed a context-constrained downstream agent, have them return structured key facts, citations and relevance scores rather than verbose content and reasoning chains. Whatever you send, send the complete conversation history each request — the API is stateless.

Three triggers justify escalation: the customer asked for a human, the policy has a gap or the request needs an exception, or the agent cannot make meaningful progress. Notice what is not on that list — difficulty by itself.

When a customer explicitly demands a human, hand over immediately rather than investigating first. When they are frustrated but the issue is squarely within the agent's capability, acknowledge the frustration and offer to resolve; escalate if they say again that they want a person.

Two popular escalation signals are unreliable. Sentiment does not correlate with case complexity — angry customers often have simple problems. Self-reported confidence is poorly calibrated, and an agent that is wrongly confident on hard cases will be wrongly confident about its confidence too.

Miscalibration usually means the decision boundary was never stated. Explicit escalation criteria with few-shot examples showing escalate-versus-resolve is the proportionate fix, before anyone builds a classifier.

Ambiguity has its own rule: when a tool returns multiple customer matches, ask for another identifier. Do not pick one heuristically.

When a subagent fails, what reaches the coordinator determines whether recovery is possible. Structured error context — failure type, the query that was attempted, any partial results, and plausible alternatives — lets the coordinator retry with a modified query, try another route, or proceed with partial coverage.

A generic status hides all of that. "Search unavailable" after exhausting internal retries tells the coordinator nothing it can act on.

Two anti-patterns are worse than a generic status. Catching a timeout and returning an empty result marked successful converts a failure into silently incomplete output. Propagating the exception to a top-level handler that kills the whole workflow throws away everything the other subagents completed.

Handle transient failures locally inside the subagent and propagate only what you could not resolve, along with what you attempted. And keep the distinction between an access failure and a valid empty result visible — they demand opposite responses. Where coverage ends up incomplete, annotate the synthesis output with which findings are well-supported and which topics have gaps.

Extended exploration sessions degrade in a recognisable way. Answers turn inconsistent, and the agent starts describing "typical patterns" instead of the specific classes it found an hour ago. That is the signal that findings have fallen out of the window.

Scratchpad files are the counter. Have the agent record key findings as it goes and reference them for later questions, so the knowledge survives a context boundary. Summarise a phase before spawning the agents for the next one and inject that summary into their initial context.

Delegation is the other lever. Spawn subagents for specific questions — find all test files, trace refund flow dependencies — so the verbose output stays in their context while the main agent keeps the high-level picture.

For long-running systems, design crash recovery deliberately: each agent exports state to a known location, and the coordinator loads a manifest on resume and injects it into agent prompts. During interactive sessions, /compact reduces context filled with verbose discovery.

An aggregate accuracy figure can be a comfortable lie. 97% overall is consistent with 99% on the common document type and 70% on the one that matters, and the average hides it. Before you reduce human review, analyse accuracy by document type and by field, and confirm performance is consistent across every segment.

For ongoing measurement, sample. Stratified random sampling of high-confidence extractions measures the error rate in the population you stopped looking at, and catches novel error patterns as documents change.

Field-level confidence scores are useful once calibrated against a labelled validation set — uncalibrated, a confidence number is just another output. Calibrate, then set the review threshold from the calibration.

Route by risk: send low-confidence extractions and those from ambiguous or self-contradictory sources to human review first, so limited reviewer capacity lands where it changes outcomes.

Attribution dies at the summarisation step. Findings get compressed, the claim-source mapping is not carried through, and the final report asserts things it can no longer support. The fix is structural: require subagents to emit claim-source mappings — source URL, document name, relevant excerpt — and require the synthesis agent to preserve and merge them rather than regenerate prose.

Conflicting statistics from credible sources are not a problem to resolve by picking one. Annotate the conflict with attribution and keep both values with their original characterisation and methodological context. The document analysis agent's job is to surface the conflict explicitly; the coordinator decides how to reconcile it before synthesis.

Many apparent contradictions are just time. Require publication or data collection dates in structured outputs and a 2019 figure stops looking like a rebuttal of a 2024 one.

Finally, structure the report so a reader can tell established findings from contested ones — and render each content type in its natural form: financial data as tables, news as prose, technical findings as structured lists. Flattening everything into one format destroys information.