Deltix's iOS agent only pays off if the successful run survives as a regression test

Deltix has opened a beta for an iOS testing agent that takes plain-English tasks, drives a simulator already running on the developer's Mac, records where it succeeds or stalls, and converts successful runs into reusable regression checks ^[1][4]. The exploration is the demo; the durable artifact is the product, and Deltix's own framing is that an AI tester earns its place only when its exploratory work can be preserved as a conventional test ^[2].

The product has three modes. Task is one-off exploration, Playbook saves and repeats a successful path, and Experiment runs the same task against two builds so a team can see whether a proposed interface helps or blocks the agent ^[8]. A developer can ask the agent to "sign up and send your first message," watch it move through the interface, and keep the completed path for later builds ^[3]. The agent acts on the interface it encounters rather than a script, which is also what makes it useful for seeing how a first-time user might behave ^[6].

That sequence is the whole bet. Agentic testing adapts and finds routes nobody scripted; regression testing needs repeatable checks whose failures indicate a product change rather than a model changing its mind [c9a]. Deltix says a successful exploratory run becomes a deterministic Playbook that reuses the same interactions and targets on subsequent builds, which would cut some of the locator and assertion maintenance that XCUITest suites accumulate ^[9][10]. There is no comparative reliability data on the landing page, so the claim stands unverified ^[11]. The beta terms pull in the other direction: validation results are advisory, not guaranteed to be accurate, complete, or stable from run to run, and responsibility for false positives and false negatives sits with the developer ^[12].

The local-execution story is what makes trying this cheap. Developers install a native Mac Agent and connect a running simulator ^[4], and the app binary, source code, and signing identities stay on the machine ^[5], so there is no build upload or repository handoff to negotiate before a first run [c14a]. The boundary is narrower than it sounds. Deltix's privacy notice says screenshots, accessibility-tree captures, action descriptions, and run metadata leave the Mac and are processed through Deltix's cloud and Anthropic's Claude API for navigation, scoring, and self-healing ^[13]; findings, saved Playbooks, and run data are stored in Deltix's database ^[14]. What stays local is the material you ship; what leaves is the material you see ^[18]. Test screens can carry credentials, customer records, and internal product detail regardless of where the binary lives, and Deltix instructs users to stick to test environments and avoid production customer data, credentials, protected health information, education records, and data on children under 13 ^[15]. Developers can supply their own model key, according to Deltix, but the public materials describe no fully offline inference mode ^[16].

The access terms do not agree with each other. The landing page calls this an open beta with no invite required, while the privacy notice and beta terms, both dated May 6th, 2026, describe an invite-only beta centered on the United States and Canada ^[17].

Watch whether Playbooks still pass after layouts, labels, and timing shift, and after the underlying model changes [c11a]. And watch which document Deltix corrects.