Build1 publisher3 min readPublished
Short action lists in place of full UI-tree reads cut a macOS agent's task cost by 85%
beans-picker cut a median macOS agent task from $0.388 to $0.060 in its author's benchmark by replacing full UI-tree reads with short candidate lists. Whether that carries over depends on how much of your own agent's bill goes to reading the window.
The Engineer · Build desk
What happened
- In its author's benchmark, the beans-picker MCP server ran a typical macOS task in a median 28.1 seconds for $0.060, against 49.7 seconds and $0.388 with cua-driver alone.
- With cua-driver alone, the model calls get_window_state for the full accessibility tree and a screenshot, clicks by index, then reads the whole tree again after each action.
- beans-picker converts the tree into a list of candidate actions and asks Jev, a TypeSafe model that only answers choice questions, which one matches the step.
- Both setups kept the target app in the background, and beans-picker refuses foreground actions and stops with foreground_violation if the app comes to the front.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Operators can test the approach with their own picker before touching the driver, since the author says the saving comes from the short action list and is not tied to Jev.
- cost At about $0.33 and 21.6 seconds saved per task on the author's figures, the case for adopting it rests on running UI tasks at volume.
- constraint Jev can only choose among the candidates beans-picker generates, so a control the list-builder misses cannot be clicked through act and the step goes back to the caller.
- exposure Unattended background runs carry less risk, because a misread step halts at needs_confirmation before it can delete, close, send or quit.
Every one of those reads stays in the conversation, so a five-step task carries five or more full reads by the end [4]. The median cua-driver-only run used 646k Claude tokens, cache included, on tasks like typing a name into a Name field or computing 15% of 80 in Calculator [6]. "In my runs this was the bulk of the bill," the author wrote [5].
The author does not blame the driver. "cua-driver is a good driver," the author wrote, adding: "The cost comes from how an agent uses it." [2] beans-picker still sends every click, keystroke and text entry through cua-driver [7]. What changes is what the calling model has to read.
The candidate list covers clicks, toggles, text entry, pop-up choices, menu commands sent as their keyboard shortcut, slider steps, table page-downs, Return and Escape [8]. Jev answers two questions over the same list. One allows "none of these" and the other forces a choice [9]. The server acts only when the leader has a probability of at least 0.8, or at least 0.5 when both questions agree and the leader has twice the runner-up's probability [9]. Otherwise it returns ambiguous with the top candidates, and the caller decides [9].
In my view that gating is the best engineering in the post. A wrong click in a form costs more than one extra round-trip to the caller, and the thresholds are set with that in mind. Verification is just as strict. After acting, the server takes fresh snapshots until the effect shows, up to 5 by default, and only then returns done [10]. Text is compared by exact equality in the targeted field. A lost trailing space, or text that lands in the search field instead of the note body, comes back as mismatch [11].
The tool surface is small: observe lists candidates, act performs one action and checks it, and extract reads a value [13]. A then field chains up to 12 more steps and stops at the first one that is not done [13].
The cost ratio is $0.388 to $0.060, about 6.5 to 1 [1]. The speed figure is 49.7 seconds against 28.1, the 1.8x in the post's title [1]. Both come from the author's benchmark of the author's own tool [1]. For them to carry over, three conditions have to hold. Your agent has to spend most of its tokens reading UI trees, as the author's baseline did. Your apps have to expose trees the candidate builder can turn into the action you need. Your tasks have to be about as small as the author's, which were single jobs in one app [6]. The quoted figures do not separate the cost of Jev's calls from the Claude spend [1].
What to watch
- A benchmark by someone other than the author on longer, multi-app tasks or apps with sparse accessibility trees.
- Whether the author breaks out Jev's cost within the $0.060 median task figure.
- Whether cua-driver or agent clients add a compact action-list mode of their own.