Pine AI's agent computer finishes fewer benchmark tasks than two rivals, by Pine's own count

Dylan Wang's premise comes from networking. In Pine's launch post dated October 5, the co-founder recalled his time at Agora, where he came to see that audio and video quality could be improved by rebuilding the network underneath ^[3]. Pine Computer applies that idea to agents. It changes the environment the model works in, so the model does not have to make up for a computer designed around human eyes and hands ^[4]. Wang's bet is that a more capable model will not be enough on its own, and that agents also need a computer built differently ^[2].

A conventional computer-use agent runs a loop. It looks at a screenshot, decides what to click, acts, and looks at another screenshot ^[5]. Pine says its environment returns structure instead: the elements on the page, the actions available, and what changed after the last action ^[6]. The screen is still there. A user can watch it live, take over for a sign-in or an approval, and hand control back ^[7]. I'd expect a page-state diff to cost fewer tokens per step than an image. I'd also expect it to turn the question of whether a click worked into a lookup instead of a visual guess. Pine's evidence cannot confirm either point yet.

SaaS-Bench v1.1, from UniPat AI, covers 106 workflows across 23 applications ^[10]. Its checkpoint score counts intermediate steps passed, not tasks completed end to end ^[11]. Pine leads the two rivals on checkpoints by 4.0 and 7.2 points ^[16]. It trails them on resolved tasks by 3.7 and 1.8 points ^[17]. The gap between its two scores is the widest of the three systems: 50.9 points, against 43.2 and 41.9 ^[21]. If the resolve rate is counted over the 106 workflows, the systems finished about 29, 33 and 31 tasks ^[18]. With gaps of four tasks and two, I would want repeated runs before ranking anyone.

Each row in the table changes more than one variable. Pine's entry runs GPT-5.6 Luna on Pine Computer. The rivals are Opus 5 in Claude Code and GPT-5.6 Sol in Codex ^[10]. Runtimewire notes that the comparisons cover complete systems with different software and budgets, and do not isolate the effect of Pine's computer ^[14]. The closest pair shares a version number. To test Wang's thesis, a run would have to hold the model and the budget fixed and swap only the computer.

Pine reports about $1.02 in model tokens per task, against $26.50 and $20.50 for the alternatives ^[13]. The rivals spent about 26 and 20 times as much ^[19]. Pine's lower resolve rate does not close that gap. Its tokens come to about $3.72 per resolved task, while the cheaper rival's spend stays above $65 per resolved task at either rival's resolve rate ^[20]. The figures exclude infrastructure ^[14]. Infrastructure is what Pine sells. It runs the computer, browser and isolation layer, and it packages them with the agent runtime and permission handling as one cloud service ^[8][15].

Pine published its run data, so the scores can be checked ^[22]. It also labels its broader 2-5x speed claim as preliminary internal testing that varies by task ^[22]. I'd count both in Pine's favor.

Developers adopt it through an SDK and build the user experience around Pine's machine ^[8]. Each computer runs in its own sandbox, and developers keep their own keys ^[9]. Tasks can run in parallel, including on sites that have no API ^[9]. Pine has given one deployment example, an unnamed customer that uses its software to automate audits. Pine says that customer's team now handles 50% more work with the same people. The release does not explain how the increase was measured ^[23].