Skip to content

Topic

Sparse top-k routing

Gating technique that scores candidate expert subnetworks for each input and runs only the highest-scoring few, together with the load-balancing terms used to stop traffic collapsing onto a small number of experts.

Current clusters