Skip to content

Build1 publisher3 min readPublished

A three-rule DNS allowlist closes the last exit from Benchling's agent code sandbox

Benchling runs agent-written scientific code in a separate AWS account with no internet gateway and no NAT gateway, so the remaining way out of the VPC was a name lookup. The resolver policy answers the endpoints Benchling listed and blocks everything else.

The Engineer · Build desk

Illustration accompanying A three-rule DNS allowlist closes the last exit from Benchling's agent code sandbox

What happened

  • Benchling dispatches agent-generated code from its production account, which holds customer data in S3, into a separate AWS account whose VPC runs the AgentCore Code Interpreter in VPC mode.
  • DNS queries from the interpreter are evaluated by Amazon Route 53 Resolver DNS Firewall, which resolves only explicitly listed endpoints and blocks every remaining query.
  • AWS says the architecture handles more than 600 code execution sessions a day across more than 250 tenants a week with zero security incidents.
  • Benchling evaluated Code Interpreter's Sandbox mode, which restricts outbound access to S3 operations, and chose instead to define and test the isolation itself.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint Owning the resolver policy means every new dependency the generated code reaches for, a package index or an external model endpoint, needs a rule change before any agent can use it.
  • exposure Untrusted execution no longer runs beside the S3 buckets holding customer data, so code that escapes the interpreter lands in an account with no data in it and no route to the internet.
  • cost The price of holding the controls yourself is a second AWS account to operate and a suite of exfiltration tests to keep green; the AWS post does not include a cost figure.
  • decision Teams currently relying on a managed sandbox now have a worked example to compare against, and have to decide whether restricted-by-default is enough or whether they need a policy they can point auditors at.

The reason DNS is the interesting control follows from the rest of the diagram. The VPC that runs the interpreter has no internet gateway and no NAT gateway, and the interpreter's security group allows port 443 only [4][5]. NACLs and prefix-list routing send what is left to an S3 gateway endpoint and one interface endpoint [7]. A connection to an arbitrary host has nowhere to route. A name lookup is different, because the resolver answers it, so the outbound paths that exist are those two endpoints and the resolver [15]. AWS's post puts the general case plainly: standard controls block HTTP, restrict egress ports and limit outbound connections, while DNS resolution is often still permitted, and even when system defaults restrict it you may not have visibility into or control over those restrictions [3].

The resolver policy has three rules. Priority 10 blocks known malicious domains, priority 100 allows only explicitly listed endpoints, and priority 200 blocks the remaining queries [6]. Priority 200 closes the channel. That leaves priority 10 redundant for every domain except the ones already on the allowlist, and those are the ones it exists for [16]. If a listed domain later shows up on a threat feed, the lower priority is evaluated first and wins.

The per-tenant boundary sits somewhere else entirely. The network controls are identical for every session, so nothing inside the VPC knows which tenant it is serving, and the scoping comes from the credentials STS injects per job from the production account [8][17]. Benchling's requirement was that each session reach only that tenant's data without one IAM role per tenant; role-per-tenant sprawl at this scale would be unsustainable [9]. A dispatcher that hands a session the wrong scope produces a cross-tenant read that the DNS firewall never sees.

Run the daily figure out to a week and the load is roughly 4,200 sessions across 250-plus tenants, about 17 each [14]. Both published numbers are floors, so treat that as scale and not a measurement. For the zero-incident record to say anything about your agents, your generated code has to want the same short list of destinations. Benchling's list is S3, reached through two endpoints [7]. Generated code that pip-installs from a public index, or calls a model endpoint outside the account, needs a resolver rule for each domain, and each rule is one someone has to justify.

I would copy the test suite first. Benchling runs integration tests that simulate exfiltration attempts against this configuration [11]. Those tests catch a widened allowlist before it ships. The post reports the 600 sessions a day and the zero incidents, but gives no observation window and no count of queries the firewall blocked [18].

What to watch

  • Whether Benchling publishes the allowlist contents or the exfiltration test cases; count the allowed domains and you know how portable this design is.
  • Whether AWS exposes customer-controlled resolver policy inside Sandbox mode. That would remove one reason for running a separate untrusted-code account.
  • A published count of queries denied at priority 200 would turn the zero-incident claim into a measurement.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories