build1 distinct publisher
Speculative decoding gets scaling laws you can size a draft model against
A new paper fits log-linear laws for draft acceptance rate against pretraining tokens, draft capacity and decoding batch size, and argues with a roofline model that the tree you tune at batch 1 is the wrong tree in production.
Publishers:arxiv.org
Reality
- Evidence38
- Adoption8
- Hype gap+32
- Incentives70