Nova Forge's serverless RL is GA. The reward function is still yours to get wrong

Amazon Nova Forge's serverless multi-turn reinforcement learning option is now generally available, aimed at teams that would rather not run the training environment themselves ^[1]. That removes a cluster from the critical path, but not the component AWS itself identifies as deciding what the model actually learns: the reward function, which in Nova Forge is a grader you write in code rather than a separately trained reward model ^[2][3].

The warning is in the vendor's own post. A subtly wrong reward, AWS writes, can quietly teach the wrong thing while every training curve looks healthy ^[4]. This is not hypothetical hedging: the post closes with pitfalls drawn from a real run in which the highest-weighted reward component silently contributed no learning signal at all ^[5].

The mechanism is worth understanding before you staff around it. Nova Forge uses Group Relative Policy Optimization, which for each conversation ranks K model rollouts and updates weights according to the normalized reward, the advantage, of the batch ^[6]. A reward signal influences learning only through the variation it creates within that group ^[7]. So a term that returns the same value for every rollout contributes nothing to the update no matter what weight you assigned it in config ^[8]. Your loss curve will not tell you this. The weight in your YAML is a statement of intent, not of influence, and the only way to tell the difference is to instrument each component separately, which is what the post recommends ^[9].

Multi-turn makes the failure surface bigger. Reinforcement fine-tuning learns from evaluation signals on the model's own outputs rather than curated examples with annotated reasoning paths ^[10], and the multi-turn variant optimises cumulative reward across a whole trajectory of tool calls, code execution and error recovery rather than grading a single response ^[11]. Every turn is a place where a proxy metric can be satisfied without the outcome being achieved. Rewards can be rule-based verifiers or an LLM-as-Judge call ^[12], and the post also covers running model-generated code safely inside the reward ^[13], which is a security review, not a hyperparameter.

The generalisation claim for RL rests on borrowed evidence. AWS reproduces a figure adapted from Chu et al., 2025, showing that after equal-compute post-training from a shared checkpoint, RL improves out-of-distribution performance across all task variants while supervised fine-tuning degrades ^[14]. That is a reason to try RFT; it says nothing about whether your reward is correct.

Note also that the serverless GA announcement is an aside in a post that uses the Bring Your Own Orchestration path throughout ^[15]. On BYOO, your reward logic runs in your environment while Nova Forge coordinates rollouts, message passing and conversation state across turns ^[16]. The prerequisites are real infrastructure: a Nova Forge subscription for the SDK and multi-turn RFT APIs ^[17], a customer-managed SageMaker HyperPod cluster on Amazon ECS ^[18], an S3 bucket for rollout data and checkpoints ^[19], and sample code from the aws-samples/sample-nova-multi-turn-rl-infra repository ^[20]. The custom environment is opt-in via use_custom_env and custom_env_id in cdk.json; the default is a built-in wordle environment ^[21], and AWS says the code throughout is illustrative ^[22].

Two things to watch. First, whether the serverless path exposes per-component reward telemetry, because without it you inherit the exact blindness the post warns about. Second, whether your own team treats reward authorship as owned engineering work with review and regression tests, or as a config file someone edits before a run.