Product1 publisher3 min readPublished
Vsim's on-robot simulator checks 20,000 possible next seconds while Freddo moves
A Cambridge start-up says it trained walking and grasping in minutes. Its co-founder and Nvidia's robotics product director both put the current limit at fine dexterity and multi-step tasks like fill and pour.
The Product Desk · Product desk
What happened
- At the Cambridge start-up Vsim, a robot called Freddo walked across the office and took a plastic bottle offered to it by a staff member.
- Vsim's developers said the walking, recognition and grasping skills took a few minutes to train and upload, and that rival systems could take days to reach the same point.
- Michelle Lu and Kier Storey, who both worked on an early version of Nvidia's Isaac Sim, set up Vsim in 2022 to build their own training environment and tools.
- Nvidia started using AI agents this year to help build the virtual environments robots train in and to validate whether training solutions work.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- constraint Jobs that chain several steps sit outside what either company claims works reliably today, so a buyer specifying work now is limited to single actions in space it controls.
- decision Anyone comparing simulation vendors has to ask for a training time on its own task and its own robot, because the minutes-against-days figure comes from the vendor's own developers.
- capability Fitting the simulator onto the robot turns replanning around people and animals into a runtime feature, which is a different thing to buy from fast training.
- precedent Once Nvidia has agents building and checking virtual worlds, generated environments become the normal way to fill a simulator, and customers train against content no person inspected.
A bottle handed over in an office is one grasp in a room the vendor controls. The task on the other side of that demo is the chain Nvidia's director of product for robotics described: take the bottle, fill it up, pour. Spencer Huang said grabbing the bottle is "not too hard" and that the trouble starts "when you start doing long-horizon tasks" [17]. Kier Storey, who co-founded Vsim, put the line in the same place. "The stuff that humans are really good at, like fine dexterity, is really hard in robots," Storey said [4].
The speed claim is the founders' own account, given to the BBC: a few minutes to train the walking, recognition and grasp and upload it, where rival systems could take days [2]. Five minutes against an eight-hour working day is about 96 times faster; against a full 24 hours it is nearly 300 [23]. The report does not name a competing system, publish a benchmark, or state a price [24].
What Vsim rebuilt was the fit between old code and new chips. Storey said the underlying algorithms for robotic simulation "hark back to the 1970s and 1980s" and "are not really brilliant fits for GPUs" [8]. Starting from scratch let the two of them write for the hardware instead of around it. "Eighteen months in and we actually have a completely functional, super high-performance simulator," Lu said [9].
Training speed and runtime behaviour are separate purchases. The simulator is efficient enough to run on the hardware Freddo carries, so the robot keeps simulating while it moves [10]. Storey said it "can look about a second, or so, ahead into the future for 20,000 different kind of combinations of things that might happen" [11]. Lu said unexpected events from humans, animals or other robots "could happen very quickly and the robot needs to be able to quickly adapt to ensure its actions remain safe and on-mission" [12].
Vsim has 10 engineers [13]. Nvidia's robotics software division has hundreds, and the company dominates the market for AI chips [14]. It sells the tooling instead of the robots: simulation systems plus a world model called Cosmos, which gives a robot an understanding of real-world physics and how its environment might change [15]. Even with those computing resources, the BBC reports, the software gives only a rudimentary understanding of the real world [16]. Huang described the scanning and building of virtual worlds as manual labour, and said of the agents now doing it: "We're just throwing agents at it... it's basically given us a huge workforce" [19].
Simulation is also not the only route to a trained robot. They can be trained by watching human or video demonstrations [20].
Two axes sort this for anyone specifying a machine. First, whether the job is a single action or a chain of steps that all have to land. Second, whether you own the space or share it with people who move without warning. The bottle handoff and Huang's caveat both sit in the same cell, one step in a controlled room, and the on-robot lookahead is what addresses the second axis rather than the first. For a job in any of the other three cells, the question to put to a vendor is how many steps its policy chains before a person has to intervene, timed on your task and your robot. Freddo's own record so far is walking across an office and taking a bottle from a person's hand [1].
What to watch
- A third-party timing of Vsim's training against Isaac Sim on the same task would test the minutes-versus-days claim.
- Whether Vsim publishes pricing, or a customer runs the on-robot simulator outside a demo environment.
- Whether Nvidia's agent-built virtual worlds ship as validated environments customers can train against.