benchmark

NanoGPT speedrun

A community competition to train a small GPT-style language model to a fixed target loss in as little wall-clock time as possible, where records are claimed by publishing faster training code.

Known aliases

NanoGPT speedrunning competition

Relationships

No evidence-backed relationships are recorded.

Current clusters

build1 publisher

Muon throws away a gradient's singular values and keeps its directions

The optimizer behind an October 2024 NanoGPT speed record is now part of the mainstream PyTorch stack. Its scope covers hidden matrix parameters, so the rest of the model still needs a second optimizer.

Publishers:dev.to

Reality

Evidence52
Adoption32
Hype gap+12
Incentives40
Confidence45