Replaying a logged search tree cut a Dream-RSI task from 550 attempts to 317

The saving comes from one substitution. A live search pays a model call and an evaluation for every candidate it considers. A replay pays for neither, because the candidate and its score are already in the log ^[6]. Replay tests different decisions within the recorded search tree, and the system does not invent new solutions while it runs ^[7].

Calling that dreaming is generous. The researchers use the word, and the agent plays through thousands of variations before selecting one for the live search ^[8]. It is a replay buffer with a tree index.

Data comes from the live run itself. The agent records its attempts and their results as it searches, and that record is what the replay executes against ^[5].

Eight tasks across three areas, with Gemini 3.1 Pro and Gemini 3.7 Flash ^[10]. Every comparison used a baseline with the same starting conditions and a fixed search strategy ^[11]. By the researchers' own account of the alternatives, a fixed strategy cannot learn from experience, so the agent may repeatedly hit the same dead ends ^[3]. The deltas are therefore measured against the weaker of the two exploration options they describe, and a team whose search already adapts mid-run should expect less.

Two figures moved by different amounts on the statistics task. Runtime came down 656 milliseconds, about 18 percent ^[1]. The attempt count came down by 233, about 42 percent ^[2]. If search budget is the binding constraint, plan around 42 percent; if the produced program's runtime is, plan around 18. The program the system wrote ran faster than sklearn and glmnet on all six test datasets ^[12].

SimpleTES supplies the largest multiple in the writeup, roughly 162 to one ^[3]. The-decoder's account counts runs for SimpleTES and attempts for Dream-RSI.

On GPU kernels, two further tasks delivered up to 2.09 times the performance inside the same budget ^[16], and math optimization work showed comparable or better results at much lower computational cost ^[17]. In a follow-up the researchers tried the other obvious use of a search history, condensing it into instructions telling the agent where to search ^[18]. The-decoder headed that section "Explicit instructions can limit exploration", and its text breaks off mid-sentence before giving the result for the one GPU task ^[19].