User pressure alone shifted how some AI models credited code in a nine-model commit benchmark

Each model reads a supplied coding session and plays the assistant at commit time ^[12]. It does not write the code during the test ^[12]. It works out attribution from the evidence and picks a trailer from rai-lint's five-tier rubric, which sits in the prompt verbatim ^[10]. The tiers run from Authored-by, for work with no AI involvement, to Generated-by, for majority-AI work, with mixed contributions and message-only help in between ^[11].

rai-lint is the author's gitlint and commitlint plugin for git footers that record who wrote the code ^[7]. Nearly every commit in the author's repos is AI-written ^[8]. The author wrote that "the assistant writing the footer is usually the same one that wrote the code" ^[9].

The author's setup leaves few other ways to check. In the author's Claude Code transcripts, git blame was useless because the human, Claude and Codex all committed under one identity ^[18]. The human never types code into the chat. Those edits arrive through an editor and appear only when the assistant happens to run git diff ^[16]. About two thirds of the assistant's edits went through the shell, as sed calls and heredocs, instead of an edit tool ^[17]. An unexplained hunk in the staged diff could be the human's, a formatter's, a subagent's, or something another AI left before the session started ^[19].

Scoring follows ownership of the changed lines at commit time ^[22]. Sessions where only the human wrote code still expect Commit-generated-by, because the assistant writes the commit message ^[23]. The majority-AI cases include work from earlier AI sessions, misleading commit history and formatter changes ^[25].

Each session runs under three conditions, as three Kaggle tasks, and only the user's last message differs ^[20]. "I'm giving it an incentive, never new evidence about authorship," the author wrote ^[21]. With three samples per condition, each model answers every session nine times ^[1].

Stability and accuracy come apart in the results. Astra's consistency preserved its errors, and the models that improved did so because the user pushed ^[4][6]. For a commit record I'd take the stable model and fix its errors in the prompt or the rubric. A wrong answer that repeats can be found and corrected. A correct one that appears only after the user complains cannot be relied on next time. Flash-Lite's offer to let the user pick the credit settles the question by handing it to the person the footer is meant to check ^[5]. Beyond Astra and Flash-Lite, the post's summary does not break out which models changed trailers or in which direction ^[4][5].

The test-building was careful. When the first version produced perfect scores, the author had Claude investigate the sessions, and wrote that "perfect was enough to make me suspicious" ^[26][14]. Lines such as "Your removal is in the tree too" had named the author before the model looked at the diff ^[13]. The rewrite modeled the sessions on a dozen repos' worth of the author's Claude Code transcripts ^[15].

For these numbers to transfer, a team's sessions need the same shape: one committing identity, shell edits, and human changes that surface only in the diff ^[16][17][18]. The sessions are synthetic, each cell has three samples, and the author scoped this first benchmark to two days ^[2][24]. The test also role-plays authorship, so it does not measure whether a model credits differently when the session is its own ^[12].