Prompt Engineering & Structured Output

"Check that comments are accurate" and "flag comments only when the claimed behaviour contradicts the actual code behaviour" ask for the same thing and get very different results. The second states a test the model can apply; the first states a topic.

General exhortations do not help. "Be conservative" and "only report high-confidence findings" sound like precision controls and are not — they ask the model to filter on a self-assessment it is not good at. Specific categorical criteria are what actually move the false positive rate.

False positives are expensive beyond the noise. One category that cries wolf undermines confidence in the categories that are right, so developers start skimming everything. When a category is badly calibrated, turning it off temporarily while you fix its prompt protects trust in the rest.

For consistent severity classification, define each level with concrete code examples rather than adjectives.

When detailed instructions still produce inconsistent output, few-shot examples are the most effective next move. They are especially strong on ambiguous cases — which tool to pick for a request that could go either way, what counts as a branch-level coverage gap — because they demonstrate a judgement rather than describing one.

Two to four targeted examples do more than a dozen generic ones, and the best of them show reasoning: why this action was chosen over the plausible alternative. That is what lets the model generalise to patterns you did not enumerate, instead of matching only the cases you listed.

In extraction work, few-shot examples reduce hallucination. Documents vary — inline citations versus bibliographies, methodology sections versus details buried in prose, informal measurements — and showing correct handling of that variety fixes the empty and null extractions that structural variation causes.

They also reduce false positives, by contrasting acceptable patterns against genuine issues.

Tool use with a JSON schema is the reliable way to get schema-compliant output. Define the extraction tool with the schema as its input parameters and read the structured data out of the tool_use response. JSON syntax errors stop being a category of failure.

tool_choice decides how firmly. "auto" permits the model to answer in text instead of calling anything. "any" requires a tool call but lets the model choose which — the right setting when several extraction schemas exist and you do not know the document type yet. Forced selection, {"type": "tool", "name": "extract_metadata"}, guarantees a specific extraction runs first, with enrichment following in later turns.

A strict schema removes syntax errors and nothing else. Line items that do not sum to the stated total, a value placed in the wrong field, an invented date — all of those are perfectly valid JSON.

Schema design has real consequences for hallucination. Make a field required when the source may not contain it and you have instructed the model to invent something; nullable optional fields let it say nothing. Add an "unclear" enum value for ambiguous cases and an "other" value with a detail string for categories you cannot enumerate in advance. Put format normalisation rules in the prompt alongside the schema — the schema constrains shape, not convention.

A retry with no new information is just a second roll of the dice. A retry that includes the original document, the failed extraction, and the specific validation errors gives the model something to correct against.

Know when retrying cannot work. Format mismatches and structural output errors respond to feedback. Information that simply is not in the document — because it lives in an external file you never supplied — will not appear on the third attempt either, and each attempt costs money.

Design validation the model can participate in. Extracting calculated_total alongside stated_total turns an invisible arithmetic error into a flagged discrepancy. A conflict_detected boolean lets the model report that the source contradicts itself rather than silently picking a side.

For the feedback loop that improves the system over time, add a detected_pattern field to each finding. When developers dismiss findings, you can then analyse which code constructs generate false positives instead of guessing.

The Message Batches API costs 50% less, processes within up to 24 hours, and offers no latency SLA. Those three facts decide every question on this objective.

It fits non-blocking, latency-tolerant work: overnight technical debt reports, weekly audits, nightly test generation. It does not fit anything a person is waiting on, and a blocking pre-merge check is the canonical example — "usually faster than the limit" is not a guarantee you can put in front of developers waiting to merge.

One structural limit: batch requests do not support multi-turn tool calling. You cannot execute a tool mid-request and feed the result back.

Operationally, custom_id correlates requests with responses, which is also how you identify and resubmit only the documents that failed — chunking the ones that exceeded context limits rather than resubmitting the whole batch. When you have SLA maths to do, work backwards: a 30-hour commitment against a 24-hour worst case leaves 6 hours, so submissions every 4 hours keep you inside it. And refine the prompt on a sample before committing 100 documents, because iterative resubmission is where batch savings go to die.

A model that just generated code carries the reasoning that produced it. Asked to review its own work in the same session, it is disinclined to question decisions it just justified. Telling it to be critical does not remove the context; neither does extended thinking.

An independent instance, with no prior reasoning, catches subtle issues the author's session will not. This is the same reason human code review works.

The second architecture is multi-pass. A single pass over fourteen files dilutes attention: detailed feedback on some, superficial on others, and contradictory verdicts on identical patterns in the same PR. Split it — per-file passes for local issues, then a separate integration pass for cross-file data flow. A larger context window does not fix this, because the problem is attention quality, not capacity.

Where you need to route review effort, have the model report confidence alongside each finding so triage is calibrated.