Build1 distinct publisher3 min readPublished
Two weeks of field notes from an agent with write access to a construction company's books and bank account point the reliability work at verification, because every failure in the log reported success first.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
Follow any of these and your For You feed starts watching them — no settings page required.
build
A reducer seeded at zero erased a 6,300-cent downside from the frontier summary1 distinct publisher
build
A featured-channels shelf puts another channel's ID ahead of the one you asked for1 distinct publisher
build
Gmail's dark mode inverts the table border Google Docs never removed1 distinct publisher
build
Let the agent propose, not commit: the envelope that has to arrive before a write1 distinct publisher
The reason this class of failure is invisible sits in the encoding. A corrupted base64 string still decodes successfully and produces a valid file with the wrong bytes inside, so nothing in the pipeline raises anything [5]. What you get instead is a spreadsheet that opens to a blank grid, or an invoice whose header is crisp and whose body below it is grey mush [6].
Take the reliable ceiling at face value and price it. Base64 spends four characters on every three bytes, so about 6,000 characters buys roughly 4,500 bytes of payload, call it 4.4 KiB [3]. That is smaller than a typical xlsx, zip or PDF, which is why the remedy in the notes is to stop the model reproducing long base64 at all and decode the platform's saved local copy programmatically [7].
I would not copy 6,000 into your own guard rail. The lowest observed corruption came in at just under twice that figure [4], and nothing between the two was tested. For the number to transfer you would need the same per-response output cap and the same platform file handling. Treat it as a bound to measure on your stack.
The notes are worth stealing for their verification discipline. Defining the word per file type is dull and it works: read text back and diff against the source, export a spreadsheet to CSV and compare totals, render an image and check that named fields are legible [11]. The enforcement clause does the real work, and it is one line: do not write "verified" unless you state what you compared [10]. That turns an adjective into a claim somebody else can check. Text needs it too, not just binaries. A plain Hebrew document came back containing a word that was not in the source, and on another run a single word arrived with Hebrew and Latin characters mixed inside it, neither flagged [14]. For long content the fallback is a byte-for-byte comparison or a hash [15].
Some of the log is empirically fixed rather than understood. The author's reading of the voucher swap is that both files shared a long identical byte prefix, and the workaround was to inject a unique marker early, before the point where the documents diverge [13]. That fix works, but it rests on a hypothesis about the cause rather than a confirmed one.
Then the money. One line before the recommendation, the agent's own log recorded that the attachment had not been read [17]. It summed the figures it could see anyway. The understatement works out at a factor of about 17.4 [1], and the notes call it the most dangerous item in the set because the output was fluent and confident [18].
That log line is the argument for verification-first. The trace already held the disqualifying fact, but the agent had no rule that converts a known gap in its own inputs into a refusal: never infer from a source you did not read, and report what is known, what is not, and why it could not be read, rather than a number [20]. The pressure to break that rule is structural, because the capability underneath is absent. According to the same field notes, no available tool downloads attachment content from Gmail; the tools return metadata such as filenames [19]. An agent with a hole in its inputs will fill the hole with whatever is in front of it, so the guard has to live in the harness.
Ranked by verification strength, evidence, and original report placement.
The write-up is a log of two weeks running an autonomous agent with real files, a real bank account and permission to act without supervision, on a small construction company's books.
All seventeen documented failures share the same shape: the agent reported success and the success was not real; the author contrasts this with crashes, which are easy because you see them.
Uploading a binary file (xlsx, zip, PDF) by having the model emit a base64 string fails above a certain size, not with an exception but with slightly wrong content.
Base64 upload corruption was observed at roughly 37,000 characters, at roughly 11,564 characters and at 22,108 characters; below roughly 6,000 characters it was reliable.
A corrupted base64 string still decodes successfully, producing a valid file with wrong bytes inside; nothing errors and the agent reports "uploaded and verified."
In practice the corruption meant a spreadsheet that opened to a blank grid, and an invoice image where the header was crisp and everything below it was grey mush.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 30, 2026
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One log, no artifacts
The precision is disarming - 11,564 characters, 22,108, 64,118.41 - and all of it comes from one dev.to post by a pseudonymous account with nothing attached: no file, no hash, no transcript, no named platform or model. Someone who watched these failures writes like this, which is why the mechanisms read as real; it still leaves a reader with no way to check a single number. The two causal explanations are weaker than the observations they explain, one hedged by the author himself.
One set of books, one operator
What exists is a single disclosed run: one small construction company, one practitioner, two weeks, unsupervised. That is a real deployment rather than a demo, which is more than most agent write-ups offer, but it is also the whole sample. Our coverage contains no second team hitting the 6,000-character ceiling, no vendor acknowledging the attachment gap, and no indication the agent kept running after the log closed.
Packaging outruns the sample
'17 Ways Your AI Agent Quietly Lies to You' sells intent where the log documents mechanism: base64 does not lie, it truncates, and the agent's 'verified' is a reporting bug, not a deception. The round seventeen and the crisp thresholds also present a four-point sample as a boundary, when the untested gap between 6,000 and 11,564 characters is where the answer actually lives. Set against that, the body under-claims rather than over-claims - the fixes are modest, the failures are described without drama, and the most alarming item in the log, a payment recommended against six percent of the real balance, gets no more space than a label-search bug.
Credibility bid, no product in sight
The post opens by dismissing most agent writing as demos and positions itself as the opposite - a claim on practitioner authority, made on a developer platform where attention is the currency and a numbered list is the instrument. Nothing is being sold in what we have: no tool, no vendor, no course. The client company goes unnamed, which is ordinary discretion, but the agent platform goes unnamed too, and that cuts both ways - nobody is attacked, and nobody can rebut.
Trust the mechanisms, not the numbers
Two different judgments belong here. That silently corrupted uploads decode cleanly, that matching file size proves nothing, and that a model will total what it can see while its own log says the attachment went unread - these are consistent with how such systems fail and are worth acting on today. The specific figures, the causal story about byte prefixes, and the claim that Gmail attachment content is simply unreachable are single-observer and unnamed-platform, and should move the moment anyone tries to reproduce them.