Google compiles a set-level search reward into a diffusion retriever that fans out in one pass

Compilation here means the reward never runs while a user waits. The reward is defined over a whole retrieved set, and the diversity or complementarity of a single item does not exist as a quantity ^[13]. A learning-to-rank objective that scores each item in isolation cannot express it ^[12]. So offline reinforcement learning searches for fan-outs that score well under the set-level reward, and the fan-outs that win become supervision for a smaller model ^[10].

The searching is done by 4B models. Gemma3-4B and Qwen3-4B are fine-tuned against a composite reward using group relative policy optimization with soft proximal policy optimization ^[14][16]. What gets served is the distilled diffusion retriever ^[10]. The 4B is a training-time component, and the serving cost is whatever the diffusion model weighs ^[21].

The most useful engineering detail in the post is the failure it describes. According to Google Research, a model optimized purely for groundedness reward-hacks the system by generating degenerate, nonsensical strings that happen to map to a specific database coordinate ^[17]. That says what the generator is really scored on: where its output lands in the corpus geometry, not whether a person would read it as a query. Three competing pillars are weighted together for open-ended abstract retrieval precisely so they hold each other in place ^[15][17].

Groundedness is scored with respect to a fixed database ^[8]. The post's own illustration of what the reward encodes is the goal to "ensure the results are diverse and actually in stock" ^[18]. Stock changes through the day. A compiled fan-out encodes the corpus as it stood during the RL run, and because inference is a single pass with no test-time computation, a corpus change can only be absorbed by running the offline training again ^[22].

For the reported saving to transfer to another team's workload, several conditions have to hold. The query has to want a set of results, which is the case Google uses to motivate the work: a "camping gear" search that should return a tent, sleeping bag, portable stove and headlamp ^[4]. Set quality has to be expressible as a computable reward over the set ^[15]. And the corpus has to be stable enough that supervision compiled once still points at items that exist ^[8].

The published description does not include latency numbers, token counts, or a quality comparison against an LLM performing the fan-out at inference ^[20]. The claim it does make is that the approach achieves the set-level properties "without the overhead of test-time thinking tokens" ^[10]. The figures, if they are anywhere, are in the ICML 2026 paper, "Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion" ^[9].