One GPU and a bash retry queue trained 14 skills for a bipedal robot duck

At upright weight 1.0, a 4 degree forward lean cost the walking policy about 0.05 per step ^[16]. Task rewards across the library were scaled to a magnitude of roughly 10 ^[12]. So the lean cost about half a percent of the task reward. That was small enough: the converged gait leaned 2 to 4 degrees forward, and two thirds of pushes tipped it forward ^[16][1]. The upright weight was tightened to 2.0 ^[16]. At that weight the same lean still costs only about 1 percent of task reward per step ^[2].

The same tension runs through all nine reward designs behind the 14 skills ^[11]. Goal terms have to dominate the regularizers early or the policy never explores, so motion-damping terms (body_ang_vel, impact penalties) were held near zero during exploration and introduced later on a curriculum ^[14]. Progress rewards were potential-based, paying only for increments past the furthest progress so far ^[15]. Standing still earns nothing.

Roulade is the documented failure. Run 1 rewarded accumulating 2 pi of rotation as fast as possible with no contact constraint, and the policy learned a violent breakdance that was genuinely optimal under that reward ^[19]. Run 2 counted rotation only during ground contact, gated the landing reward on a roll frontier of at least 260 degrees, capped the progress pay rate at 3 rad/s, and added overspeed penalties ^[20].

StandUp needed geometry in the reward: a dense height term for broad gradient, a sharp term that jumps from 0.36 to 1.0 in the last centimeter, and an L1 term that makes sitting still net negative ^[18]. What made StandUp learnable was spawning episodes mid-roll. The whole motion was not learnable; half of it was ^[18]. BallKick splits the observations instead: the real robot has no ball perception and the operator aims, so the actor is ball-blind while the critic sees the ball ^[21].

None of that was validated on a training curve. The post's position is that a good loss curve does not mean a good gait, and the stated acceptance test was a 300-frame single-env rollout GIF per skill, converted with ffmpeg palettegen and paletteuse ^[10].

Five days across 14 skills is about 8.5 hours per skill on average ^[4]. For that to transfer you need the same shape of problem: 15 servos on a 50 Hz control loop ^[1]. It also needs a velocity baseline whose domain randomization and regularization layer already works well enough that every other skill inherits it and adds only a task layer ^[13]. You also need the VRAM. The post does not say which GPU was used ^[23]. It does say 4096 environments on rough terrain exhausted memory and that 2048 passed ^[6]. That is 50 percent of the flat-terrain count, against the roughly 70 percent the post offers as its own rule of thumb ^[7][3].