Pick the BAA, not the benchmark: where HIPAA document pipelines actually spend

Start with the arrival rate. The worked example in the post adds up to 45,000 documents a month ^[1], about 1,500 a day ^[2]. Averaged across the month that is roughly 0.017 documents a second, some 3,400 times below the 60 requests per second ceiling the author names as a production failure mode ^[3][11]. Rate-limit exhaustion in a pipeline that size is not a capacity problem. It is a question of shape: paper arrives in trolleys, and the batching layer in front of the model decides how much of the day lands on the deployment in the same minute. The author notes that ten documents per request cuts token spend while raising per-request latency and rate-limit risk ^[12], which is the same dial viewed from the other end.

The arithmetic also deflates the cold-start case a little. A 2 to 3 second cold start on Consumption Functions ^[9] is 7 to 10 percent of the 30 second window the billing team needs codes inside ^[5][7]. The 1 to 2 second warm start the author reports on Premium under burst is 3 to 7 percent ^[4]. Neither figure breaks a 30 second budget by itself, so when he writes that Premium warm starts break the claims SLA ^[10], the missing term is queue depth. That is a genuine argument for deterministic containers over elastic serverless ^[4], but it is an argument about how long work waits, not about how fast a worker boots.

Where the post is most useful is the part that has nothing to do with model quality. Azure Cognitive Search is recommended because it carries a HIPAA BAA and native hybrid search, at the price of being confined to Azure regions; Pinecone and Qdrant may cost less but need their own BAAs and can add egress ^[13]. The author would take Pinecone only where the BAA can be negotiated and residency holds, and otherwise treats compliance parity as the tiebreak ^[18]. That is a procurement shortlist, and it is drawn before anyone opens a retrieval benchmark ^[3].

The failure he says he sees most often is embedding drift: a new OpenAI embedding model moves the vector space, similarity search starts returning unrelated records, and false positives spike ^[14]. His mitigation is to version embeddings and treat the index as a contract ^[5], re-indexing incrementally behind a feature flag rather than in one pass ^[15]. The corollary is that every embedding upgrade becomes a migration with traffic shifting, on a corpus where the output JSON is itself the audit artefact and an invented medication name is an audit failure ^[17].

Two caveats on the headline numbers. The 30 to 50 percent latency reduction and the claim that the bill stays under 10 percent of raw compute ^[2] are offered as field experience, with no dataset, baseline or methodology published ^[22]. And they sit oddly beside the framing that orchestration, not the model, is the biggest cost ^[1]. One of those two sentences is measuring something the other is not.