Skip to content

Build1 publisher3 min readPublished

A category-and-confidence rule decides where each support email goes in this Claude triage pipeline

A one-person dev shop wired Gmail into Claude and out to Slack, scoring each support message 0 to 100 and leaving the assignee to a plain Python lookup table. Nothing sends until a person clicks approve.

The Engineer · Build desk

Illustration accompanying A category-and-confidence rule decides where each support email goes in this Claude triage pipeline

What happened

  • A one-person dev shop built a pipeline that pulls unread Gmail, classifies each message with Claude, drafts a reply and posts it to Slack for a human to approve or edit before anything sends.
  • Messages route to a human on category and confidence together, not on the category on its own.
  • Adding a mark-as-read step forced gmail.modify, and listing readonly beside it made Google drop the redundant scope, which surfaced as a Scope has changed error at the next token refresh.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint Because the tier is computed from the confidence number in code, retuning an escalation threshold is a change to a comparison, and the prompt and schema stay put.
  • exposure gmail.modify covers everything readonly does and adds writes, so a triage bot that only needs to read now holds mutate rights on the mailbox it watches.
  • decision Anyone copying this has to pick which outputs a model is allowed to produce at all, and the author draws the line before assigning work, moving money or deleting data.

One call per message does both jobs. Claude returns the classification and the drafted reply in the same structured response [4][6]. The shape is a JSON Schema passed directly in the request, and the post says the API guarantees the response matches it [12]. Truncated output, prose wrapped around the object, and a field that is a string on one call and an array on the next all go away [13].

The schema is where the decisions sit. `category` is one of four fixed strings: sales inquiry, support-technical issue, billing question, complaint [14]. `priority` is High, Medium or Low [15]. `intent` is a free snake_case label the model invents, such as `sso_redirect_loop_outage`, and routing does not read it [16]. `confidence` is a number from 0 to 100, and `confidence_tier` is computed from that number in code instead of being asked for in the response [17]. Moving a threshold is therefore an edit to a comparison.

Routing keys on category and confidence together [5]. The assignee comes from a lookup table in code, and the model never names a person [18]. The author's reason is specific: a model asked for an assignee can return someone who does not work there, or drift between "Sam" and "Samuel" until the routing table misses [19]. The rule as the post states it is to "Keep anything with real-world consequences (assigning work to a specific person, sending money, deleting data) in deterministic code" [20]. A human clicks Approve or edits the draft in Gmail, and nothing sends automatically [9].

Write access arrived through a convenience. The build started on `gmail.readonly` plus `gmail.send`, then the mark-as-read step needed write, so the author moved to `gmail.modify`, a superset of readonly [21]. List readonly beside modify and Google silently drops the redundant one from the actual grant; `google-auth` then raises `Scope has changed` at the next token refresh, which is well after the edit that caused it [22]. The author found it by reading the `google-auth-oauthlib` source after the second refresh failure [23].

The body extractor is the other documented failure. Its first version walked the MIME tree and, at each level, fell back to HTML when plain text was not found at that level [24]. Gmail messages frequently arrive as `multipart/alternative`, and the structure can nest a multipart inside another multipart [3].

Three external dependencies total, SQLite for state, and no web framework; the CRM view is a plain `http.server.BaseHTTPRequestHandler` serving an HTML table [10][11]. Re-running the pipeline cannot double-process a message, because the result is written to SQLite and the message is marked read [8].

The post opens by calling "AI will save you time" a claim everyone makes and almost nobody backs with numbers, and promises a breakdown of running cost against what it replaces [25]. The published text stops at the third bug, before any figures [26]. Two of the three announced bugs are written up [27]. For that cost comparison to transfer to another shop, you would need tokens per message, messages per day, and the hourly rate of whoever answers them now.

What to watch

  • Publication of the third bug and the cost figures, where the post's text currently stops.
  • Whether the tier boundaries behind confidence_tier get published, since the routing rule turns on them.
  • Whether anyone runs this on a shared support mailbox, where gmail.modify's write access covers everyone's mail.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories