Grok 4 fails half its patch attempts on Codex's edit format

Anyone who has watched a coding agent stall on the same edit three times has read the message: "String to replace not found in file." That is str_replace failing, the edit format Claude Code and most other harnesses use ^[5]. The model has to reproduce the old text character for character, whitespace and indentation included, and if two spots in the file match, the edit gets rejected ^[5]. Can.ac says that one error has its own GitHub megathread with 27 other issues attached ^[6].

The other common format does worse when the model was not built around it. Codex's apply_patch takes a string that has to follow a strict set of rules instead of a structured schema, an OpenAI-flavored diff ^[3]. Handed to models trained elsewhere, Can.ac measured Grok 4 failing 50.7 percent of patches and GLM-4.7 failing 46.2 percent ^[4]. At Grok 4's rate a landed edit takes about 2.03 attempts on average ^[15].

The third-party numbers in the post put format and model on comparable footing. Aider's own benchmarks swung GPT-4 Turbo from 26 percent to 59 percent on edit format alone, a 33 point move, and on that same format GPT-3.5 managed 19 percent, 40 points below GPT-4 Turbo ^[10][16]. JetBrains' Diff-XYZ benchmark found no single edit format dominates across models and use cases, and EDIT-Bench found only one model clears 60 percent pass@1 on realistic editing tasks ^[11][12].

Cursor's answer was to fine-tune a separate 70B model whose only job is merging a draft edit into the file correctly ^[8]. Its own blog post, quoted by Can.ac, said "fully rewriting the full file outperforms aider-like diffs for files under 400 lines" ^[9].

Hashline, the format Can.ac built, drops the requirement that the model recall the old text at all. Every line comes back from a read or a grep tagged with a 2-3 character content hash, and an edit cites the tag: replace line 2:f1, insert after 3:0e ^[13]. A hash that no longer matches means the file changed since the read, and the edit is rejected before anything is corrupted ^[13]. The published post breaks off at the start of its benchmark section, so the per-model results behind the 15-model claim are not in the text ^[17].

For a team running agents inside someone else's harness, the useful split to log is between two kinds of failed run: the model did not know what to change, and the model knew and the tool refused the edit. A model upgrade does not touch the second bucket, and Can.ac's argument is that when the tool refuses, the user files the complaint against the model ^[14]. On a closed harness the edit tool is not yours to swap. That leaves file size and what the model reads before it edits, and Cursor's finding gives one rule to test on files under 400 lines ^[9].