WASP's human-written injections start hijacking web agents in up to 86 percent of runs

The two rates measure different failures. An agent that begins executing the adversarial instruction has read text off a page and treated it as an instruction, so whatever separated data from instruction is gone by that point ^[4]. "Achieves the goal" means the agent finished the attacker's task, which takes several correct actions in a row. The paper puts the gap down to the agents themselves, saying they are "currently not capable enough to complete the goals of attackers end-to-end" ^[5].

Subtract, and at the top of both ranges 69 runs in every 100 that start the attacker's task stall before finishing it ^[14]. The abstract does not pair the extremes, so 69 is a bound and not a measurement. It reports no split by model or by mitigation, so you cannot tell from it which configuration sat at 16 percent and which at 86 ^[4]. The floor is the figure I would take to a review: in the best case reported, roughly one run in six crosses the boundary ^[15].

For the completion number to transfer to your deployment, the attacker's task in your product has to be as long as the tasks WASP sets. It also has to run in something like WASP's isolated environment, which the authors built so tests do not touch real users or the live web ^[1]. Liao et al. showed web agents being hijacked by prompts on attacker-controlled pages to exfiltrate user information ^[12]. Where one form submission sends a token, the attacker's task is one step long.

The paper is also hard on the attack literature it follows. It argues prior work often hands adversaries excessive permissions. Arbitrary control of a page implies the page is compromised far more deeply than the attack needs, and somebody who controls the form fields on a web shop's checkout page does not need to hijack an agent to collect visitors' credit card numbers ^[7][8]. Injections rendered as colorful overlay text leave unexplained how a malicious party draws that text on pages held by non-adversarial hosts, given modern web security protections ^[9]. And an objective satisfied by an agent printing "I have been pwned" does not establish real harm ^[10].

What follows, in the authors' own recommendation, is a demand on attackers: show attacks that more consistently maintain control over the agent under realistic constraints on the adversary's power ^[6]. The completion rate moves with the agent's competence at a task somebody else specified ^[5]. The initiation rate is what I would ask for before giving a browser agent write access to a real account.