The failure modes of AI agents in low-level code have names, and names can be gated in CI

A post on dev.to says its authors collected more than 55 source-traced failures of AI coding agents in low-level code across three research passes and turned them into 124 "verified engineering skills" ^[1]. The useful part is not the skill count; it is the argument that these failures are enumerable classes with mechanical detectors, which makes them a CI problem rather than a code-review problem ^[22].

The post groups them into five buckets: assembly hallucination, fake parallelism, Rust API drift and crate hallucination, misleading verification, and systems-level blind spots such as virtual-memory leaks invisible to heap tools ^[3]. The shared signature is what makes reviewer diligence a bad control: the code looks correct, compiles, and still does the wrong thing ^[4].

The assembly case is the cleanest. Agents emit instructions that do not exist, including CDC COMPASS pseudo-ops for code that merely "looked like assembly" and a nonexistent `movqad` ^[5]. The worse variant does not error at all: `imul eax, eax, 38` assembles to `69 c0 00 00 00 00`, with the immediate silently discarded by the parser, which the post identifies as the bug class behind BBoeOS PR#584 ^[6]. The gate is a two-line loop of `gcc -c` plus `objdump -d`, comparing mnemonic, operands and operand size against what you thought you wrote ^[7]. Do not ask the model to check its own work here: the post puts LLM disassembly exact-match at roughly 14 percent and decompiler "fixes" at roughly 37 percent ^[8].

Fake parallelism is the same trick one layer up. Models produce `ConcurrentHashMap` and atomics that look thread-safe but execute on a single thread, which linear benchmarks do not catch ^[9]. The proposed gate is not a linter: count live threads and measure wall-clock scaling ^[10].

On Rust, the post cites RustEvo2 for models handling stabilized APIs at 65.8 percent but behavioral changes, where the signature holds and the semantics move, at 38 percent, and dropping from 56.1 to 32.5 percent for APIs added after the training cutoff, with retrieval augmentation worth about 13.5 points ^[11]. That gap of 27.8 points ^[12] lands on the class least likely to fail loudly. The supply-chain version is hallucinated crates that resemble real ones, at reported rates of 5.2 percent for commercial models and 21.7 percent for open-source ones, checkable with `cargo info` and an exit code ^[13]. In cryptographic Rust the post reports 23.3 percent of generated code compiles and 57 percent of that is vulnerable, with nonce reuse leading ^[14], which leaves about 10 percent of output both compiling and not flagged ^[15].

The class with the most leverage over everything else is misleading verification. A fixed-shape `allclose` oracle certifies buggy GPU kernels as correct, while fuzzing against an fp64 reference caught nine of nine ^[16]; other kernels pass review and segfault under load ^[17]. Ghostty's 37 to 130 GB virtual-memory leak came from a page pool reusing an `mmap` without ever calling `munmap`, with an agent as trigger rather than cause ^[18]. Hence the post's ablation rule: break the target, and if the suite still passes, the suite is decoration ^[19].

Two caveats on provenance. The post discloses it was drafted with AI assistance and points to a repository registry with 177 primary sources ^[2], and of the five arXiv identifiers it leans on, only the RustEvo2 one carries an earlier numbering prefix than the rest ^[21]. Verify the identifiers before you quote the percentages to anyone.