Product2 publishers3 min readPublished Updated
A robot used a dustpan as a brush: what that proves, and what it does not
Generalist AI's arms improvised through a missing tool and a bad grip in front of a reporter. Improvisation is the right test; the account still carries no success rate, price or customer.
The Product Desk · Product desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- A WIRED writer visited the Cambridge, Massachusetts offices of a startup called Generalist AI last week and watched robot arms perform simple chores such as stacking cups and putting blocks into bowls.
- The arms mastered a range of tasks after ingesting a short instructional video and with no specific training for the given task.
- A robot was instructed to sweep a block into a bowl using a dustpan and brush; when the brush was removed from the scene, the robot improvised by using the dustpan like a brush and flicking the block into the bowl.
- A two-armed robot watched a video clip of someone unzipping a purse and removing banknotes, then unzipped a different kind of purse and carefully removed the notes.
- When the two-armed robot could not grab the money, it switched from using its right gripper to its left to get a better angle of attack; an engineer standing nearby said, "Ha. It never did that before."
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
A WIRED reporter spent time last week at the Cambridge, Massachusetts offices of Generalist AI watching robot arms stack cups and put blocks into bowls after ingesting a short instructional video, with no training specific to the task [1][2]. Two moments in that visit matter more than the chores themselves: when the brush was removed from a sweeping task, the robot flicked the block into the bowl using the dustpan instead [3], and when a two-armed robot could not grab banknotes out of a purse, it switched from its right gripper to its left to get a better angle [4][5].
Improvisation under a broken setup is the right thing to look for, because it is the thing that scripted demos cannot fake. The engineer standing nearby reportedly said, "Ha. It never did that before" [5]. That reaction is the load-bearing detail in the whole account: an unrehearsed recovery is closer to work than a rehearsed success is. Researchers at the company have often been surprised by the robot's choices, including one that swept items up with a banana after someone put a banana in front of it [6].
The method behind it is unglamorous. The company builds gloves shaped like robot pincers with cameras attached, and people wear them to perform chores; the reporter saw teams doing exactly that outside a conference room, and a crate of several hundred grippers bound for workers in Mexico and elsewhere [10][15]. That is a data-collection supply chain, not a lab trick. Generalist says it has gathered a large amount of high-quality training data and built its models from scratch rather than on top of an open-source language model [11]. Stanford's Karen Liu says the approach collects physical interaction data at scale without tying it too closely to one particular robot [14]. Compare that with the older pattern, where a robot needed thousands of examples per task and could still fail when the lighting changed [9].
CEO Pete Florence frames it as the GPT-3 moment for manipulation: prompt the model with a new task and it has a real shot [7]. He, CTO Andrew Barry and chief scientist Andy Zeng previously worked at Google DeepMind and Boston Dynamics [8]. Georgia Tech's Danfei Xu, who knows the work, says they have pushed the approach to the extreme and executed well, and that they are "the closest to something that's deployable" [12][13].
Read that sentence carefully. Closest to deployable is a ranking against other companies chasing general robot models [12][13], not a product. The published account contains no success rate over repeated trials, no named customer, no price and no availability date [17]. A buyer of physical automation needs the failure rate and the cost of a failure, and neither is on offer here. What we have is a strong signal that the capability exists in a lab under observation by a journalist and endorsement from two outside roboticists [1][12][14].
Watch for three things: a published success rate on tasks and objects the model has not seen, evidence that the model runs on hardware other than Generalist's own arms, and the first named commercial deployment. Also watch the gripper pipeline; several hundred units shipping to workers abroad [10] is a cost line that has to pay for itself.