A Patch Is a Protocol
Part 2 of 6 Harness Engineering: How Coding Agents Turn Decisions into Work
In brief
In brief
A coding agent edits a file by asking an edit tool to replace exact old text with new text, or to apply a patch. The tool should refuse duplicate or missing matches and say whether a failed batch already wrote some edits. A regression test then checks the fix.
- An edit tool should reject an ambiguous target or require an explicit broader operation, then report enough context to choose the next step.
- A batch must say whether targets match the original file or earlier edits, and whether a failed operation already wrote changes.
- Inspect the final diff and test locale isolation plus cache reuse. A well-formed patch or successful write cannot check those behaviors.
The coding agent has found the bug. The preview cache uses document.id, but render(document, locale) can produce different output for the same document. The model proposes adding locale to the key. Then the edit fails because the old text appears twice.
A coding agent's edit tool should turn that failure into a clear next step: read more context and choose the intended location. Its rules must say how old text is matched, which file version a batch uses, and whether a failed operation already changed files. The agent loop needs that result before it asks the model what to do next.
The locale-cache repair's repository has a root test runner, a preview package, and a nested locale package with its own instructions. The working tree already contains an unrelated developer change. The task is to add locale to the preview key, preserve same-locale reuse, and add regression tests.
The fix touches both the lookup and the insertion
Strip the cache down to one function. Assume document contents stay fixed and callers supply a canonical locale. Those assumptions isolate the cache-key bug without adding a document-revision or locale-normalization policy.
async function preview(document: Document, locale: string) {
const cached = cache.get(document.id);
if (cached !== undefined) return cached;
const html = await render(document, locale);
cache.set(document.id, html);
return html;
}
The lookup and insertion need the same key. Changing only get leaves entries stored under a key that lookups never use. Changing only set has the reverse problem. Either partial fix might return the right language while disabling cache hits. A test that checks only output strings would miss the extra work.
For this example, JSON.stringify([document.id, locale]) records both inputs and avoids collisions from an informal separator. If IDs and locales already have a guaranteed encoding, the fix should reuse it.
The model can explain the bug correctly and still submit the wrong edit. It might replace the first document.id, forget the insertion, or overwrite a local change by regenerating the whole file. The edit tool can catch some of those mistakes before writing.
Old text must match the current file
A search/replace request combines a change with a claim about the file: the old block is what the model expects to find when the tool runs.
The file may have changed since the agent read it. A developer can save an edit, another tool can rewrite an import, or an earlier edit can change nearby code. Checking the old text tests that expectation against the current file instead of a remembered line number.
An exact match confirms that one block exists under the tool's matching rules. The rest of the file may still have changed, and the model may still have misunderstood the function.
For our repair, the old block could include the function declaration, lookup, and nearby return. Selecting only document.id leaves the tool unable to tell which use the model means. Enough context distinguishes the target. Too much unchanged text makes the edit fail on unrelated changes.
Line numbers help navigation and diagnostics, but move when someone inserts a comment. Current contents provide a stronger check before an edit.
Parsing, matching, writing, and testing are four checks
Parsing checks the patch format. Matching checks the selected files for the expected old text. Writing applies the accepted change. Tests check the modified program's behavior.
Codex's apply_patch parser (rust-v0.144.1) checks patch structure without checking whether it applies to the filesystem. A patch sits between *** Begin Patch and *** End Patch markers. Its grammar describes adding, deleting, and updating files. Separate code applies those operations. A well-formed patch can still fail against the current source.
Each stage can fail differently. A missing patch terminator is a syntax error. A missing old block is a matching error. A denied write is an execution error. A cache that never reuses an entry is a behavior error, even if the first three checks passed.
The model needs a corrected format after a parsing error, a fresh read after unexpected file changes, and a different code change after a regression fails. “Try again” discards those differences.
Duplicate old text needs more context
Suppose the module has two caches: one for HTML previews and another for document metadata. Both call cache.get(document.id) because an earlier refactor copied the small example. The agent submits that line as its old text.
A tool requiring a unique match rejects it. The agent should read both functions, identify the preview cache, and include distinguishing context in the replacement. Adding random whitespace until one match wins would conceal the ambiguity.
Pi's v0.80.6 edit-tool schema requires each old-text target to be unique in the original file. Entries in a multi-edit request match that original file and must not overlap. The caller can read those rules before sending the request.
A tool might also offer explicit replace-all for a deliberate rename. In this example, adding locale to the metadata cache would change unrelated behavior. An ambiguous local repair should not silently become a global replacement.
“Two matches; file unchanged” tells the model what to do next. Showing only one candidate hides the unresolved choice. The error should report the ambiguity rather than choose a location for the agent.
Fuzzy matching trades fewer failures for more uncertainty
Exact matching can fail when indentation, line endings, or a typographic character differ. Fuzzy, or tolerant, matching can accept some differences and let the edit proceed.
Some differences change meaning. Ignoring Python indentation may select the wrong scope. Similar-looking variable names may occur in a function with a different purpose. A high similarity score describes the text. The tool still needs to check whether the target is safe to change.
The acceptable tolerance depends on the tool's rules and surrounding checks. Comparing normalized line endings can be predictable if writing preserves the chosen file format. Picking the “closest” function among several candidates needs a way to reject uncertainty or show the selected context for review.
An exact-match rejection is cheap in our small file: reread it and make the target unique. A large generated file or repeated patch-formatting failures might justify more tolerance. Evaluate that choice with real failure traces rather than declaring one matcher universally more accurate.
A tolerant tool should still return the resulting diff. The changed bytes show whether it touched the right cache or rewrote neighboring whitespace. A message like “found something similar” cannot show that.
A batch must say which file version it matches
The lookup and insertion can change in one block or two targeted edits. A batch must define when it matches each old block.
One approach matches every target against the file as it was at the start. The tool computes non-overlapping replacements and applies them together. The model need not predict how the first edit shifts the second target. Pi's edit tool uses this approach.
Another approach matches the second target after applying the first edit. That can express a dependency, but reordering the list changes its meaning.
Our repair can match both edits against the original file. One edit adds the shared key and changes the lookup. The other changes the insertion. If their target blocks overlap, merge them. Submitting one large replacement and another inside the same region creates a conflict the tool should report.
The tool must also say whether it validates every edit before writing. Checking all replacements first can reject a missing second target without changing the first. Writing after each match can leave a partial repair. The number of requested edits tells the caller neither policy.
Failed batches can leave partial edits
Aider's v0.86.1 edit-block implementation tracks successful and failed SEARCH/REPLACE blocks, writes successful replacements, and reports the failed blocks. Its error tells the model that successful blocks were already applied and should not be resubmitted.
Suppose the lookup edit lands but the insertion edit cannot match. Retrying the whole proposal might add a duplicate key declaration or cause another error. The agent should inspect the current file and repair only the missing portion.
The same problem crosses files. The regression test may exist while the source edit failed, or the source may be fixed while writing the test was denied. Inspect the working tree to see what remains.
Codex's patch application code records which changes a patch already applied and can return that record with an error. The caller can then recover from a partial write. That record does not make every patch application an all-or-nothing transaction.
An edit tool should promise atomic (all-or-nothing) behavior only where it can guarantee it. Computing every replacement before a single file write is one level of guarantee. Rolling back a multi-file operation after the third write fails is a different level. The tool description should say which level it covers.
Other programs can change the file mid-edit
Serializing calls through one edit tool prevents those calls from racing with each other. A developer's editor, formatter, or separate process can still write the file.
A read/modify/write tool reads the file, computes replacements, then writes it back. An external save between those steps can be overwritten even if the old block matched at the read. Queueing the tool's own calls leaves that race unresolved.
A file-version check can reject the write if the full contents changed after the read. A stronger workspace lock can exclude other writers. An interactive workflow can keep edits short and report conflicts promptly. Choose for the actual setting: a personal checkout, shared filesystem, or isolated worktree.
In our example, inspect the initial diff, edit the requested region, and inspect the final diff. Build on the developer's current changes. Resetting the file for a cleaner patch would erase unrelated work.
The diff also reveals broad rewrites. Investigate a two-line fix accompanied by hundreds of line-ending changes before treating passing tests as sufficient review.
Test locale isolation and cache reuse
Test the behavior callers observe. A test of a new makeKey helper could pass while the lookup still uses the old expression.
The following is test pseudocode. createPreviewService is a small fixture with a fresh cache and a test renderer.
it("keeps locale variants separate and reuses each variant", async () => {
const render = vi.fn(async (doc, locale) => `${doc.id}:${locale}`);
const service = createPreviewService({ render });
const document = { id: "guide-7" };
expect(await service.preview(document, "en")).toBe("guide-7:en");
expect(await service.preview(document, "de")).toBe("guide-7:de");
expect(await service.preview(document, "en")).toBe("guide-7:en");
expect(await service.preview(document, "de")).toBe("guide-7:de");
expect(render).toHaveBeenCalledTimes(2);
});
With the old document-only key, the second assertion catches cross-locale contamination. If a fix disables cache hits, the output assertions may pass while the render-count assertion fails. A separate test should cover distinct document IDs with the same locale, checking the other input to the key.
Give each case a fresh service or reset the shared cache. Otherwise an earlier test may populate the entry and change which request fails. If normalization exists, use the intended canonical input here and test normalization where it is implemented. Keep de-DE and de separate unless the actual rules equate them.
The agent must also find the correct test command. In this example, root and package scripts may select different suites. Read the root configuration, preview package, and relevant nested instructions to see how the new case runs. A green command that never collected the file cannot verify the fix.
After the focused regression passes, inspect the diff and run the affected package's required checks. Diagnose broader failures and record unrelated failures already present in the working tree. Do not make speculative repairs just to get a green result.
Edit tools should make the next step clear
Before execution, the tool description should explain matching and batch rules. After execution, the result should list changed paths and distinguish an unchanged file from partial application. The model can then choose the next read or edit from what happened.
For this repair, check three things. Both cache operations use the document-and-locale key. Regression tests check isolation and reuse. The diff keeps the developer's earlier work. The next article explains what stays in the agent's context as the conversation grows.
Read the rest of the Harness Engineering series. Series reference: Harness Engineering.
Updated
