Build1 publisher2 min readPublished
AWS combines EKS, EFA and DeepEP to boost MoE reinforcement learning throughput by 40%
AWS says running DeepEP over its EFA network on EKS gives mixture-of-experts reinforcement learning 40% more throughput. Whether that reaches another cluster depends on how much of each training step goes to expert traffic between nodes.
The Engineer · Build desk
What happened
- AWS published an architecture for reinforcement-learning post-training of MoE models on EKS with EFA networking and DeepEP, headlined as delivering 40% more throughput.
- AWS argues that as MoE models get sparser to cut inference cost, their training becomes limited more by communication than by compute.
- Expert parallelism adds dynamic all-to-all token routing across devices on top of the structured traffic of tensor, data and pipeline parallelism.
- AWS says PPO-based RLHF and GRPO differ on whether a separate critic model is needed but make the same demands on infrastructure.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint The gain is aimed at expert traffic crossing nodes, so an MoE job whose experts fit inside one instance has little of the problem DeepEP is there to solve.
- decision A team that buys a faster training step also has to resize its rollout workers, since trainers that outrun inference sit idle and the loop keeps its old pace.
- exposure Each node added to an expert-parallel job is one more link that can straggle, and in lockstep training one slow link can hold up every worker or end in an NCCL timeout.
Sparsity lowers the compute each token needs. Every token still has to travel to the devices holding the experts its router picked. As active parameters shrink, that exchange takes a larger share of each step, and AWS's post makes that case for sparser MoE designs [4].
The exchange is also hard to schedule. Gradient all-reduce in data parallelism has fixed partners and fixed message sizes. Expert parallelism's all-to-all is dynamic [5]. Who sends how much to whom changes with the router's choices on every batch.
Inside one instance, that traffic runs on high-bandwidth local links. Scale past one instance and it moves to lower-bandwidth links between nodes [8]. AWS's answer is DeepEP, run over EFA, to make the expert-parallel exchange efficient across nodes, with EKS orchestrating the job [3].
RL raises the cost of a slow exchange. Policy training runs in lockstep, and one straggling worker can stall the job or trip an NCCL timeout [7]. Rollout runs alongside it as distributed inference tuned for aggregate throughput, with time to first token and inter-token latency set aside [9]. Setting latency aside is the right target for RL generation, and the post states it cleanly. Reward models, verifiers and checkpoint updates add memory, network and orchestration load on top [11].
The 40% is in the headline [1]. Up to its section on compute, memory and bandwidth pressure, the post's text does not state the baseline, the model, the node count, or whether throughput means training tokens or completed rollout-training iterations. Taken at face value, 40% more throughput is the same work in 1/1.4 of the time. That is about 29% less wall-clock [1].
For that figure to carry to another cluster, a few things have to be true. A job's step time has to be dominated by cross-node expert all-to-all, the regime the post describes. The measurement has to cover the whole loop, because slow training steps stall inference workers and thin inference throughput idles the trainers [6]. A faster training step on its own can end up waiting on rollout. The expert-parallel degree and node count also have to be close to whatever AWS tested.
What to watch
- The post's benchmark details: the baseline communication path, model size, expert-parallel degree, node count, and which throughput the 40% measures.
- Whether AWS publishes the EKS manifests and DeepEP-on-EFA build in a form another team can rerun.
- Results from teams outside AWS running DeepEP over EFA at node counts different from the tested one.