model

Chinchilla

DeepMind language model whose 2022 training study established the compute-optimal ratio of training tokens to parameters now used as a planning baseline for pretraining.

Known aliases

Chinchilla scaling laws
Hoffmann et al. 2022

Relationships

No evidence-backed relationships are recorded.

Current clusters

build2 publishers

Distill or just train? The answer is a division problem: teacher compute over student count

A proposed scaling law for distillation says the winner flips on two things: whether you already own a teacher, and how many students you intend to serve.

Publishers:arxiv.orgthesequence.substack.com

Reality

Evidence68
Adoption34
Hype gap+24
Incentives44
Confidence64