Orca's 90% query cost cut is measured against a prompt holding 900-plus Sonar models

Every percentage here is relative to the first version, which put the full Sonar schema into one prompt on the largest available model: 900-plus model definitions, every relationship type and a generous set of example queries ^[2]. It worked. Design partners and internal users responded positively to the output quality ^[13]. The AWS post measures all three reductions against that starting point ^[12].

The figures are consistent with each other. A 75% cut from about 60 seconds is 15 seconds ^[10], and the post reports 15-second responses in production ^[5]. If the 93% applies to the 2 million input tokens per workload, each workload now sends about 140,000 ^[9].

For the numbers to transfer, your starting point has to resemble theirs. It sends the whole schema on every request, to the largest model, over a schema where most of what is sent is irrelevant to any one question. Orca's schema is that large: 900-plus models and 18 operators to choose from ^[1]. A team that already retrieves schema fragments, or already runs a mid-size model, begins nearer the floor and should expect a smaller gain.

The best piece of engineering in the excerpt is the second stage. The Quick Deterministic Matcher uses regex and rule-based matching to detect high-confidence patterns: IP addresses and URLs, CVEs, file hashes, alert IDs, and cloud resource identifiers such as ARNs, EC2 IDs, Amazon Machine Image IDs and Oracle Cloud Identifiers ^[7]. Each of those strings has a fixed format that regex can match. The portion of the post we reviewed ends mid-description of that stage, so what happens after a match, and what the later stages do, is not visible to us.

In our view this is the right design for a schema of this size. Nine stages is a lot of machinery to maintain ^[6], and a schema that fits comfortably in one prompt would not repay the maintenance.