benchmark

AIME

American Invitational Mathematics Examination, a 15-problem competition whose questions are reused as a mathematics benchmark for language models.

Known aliases

American Invitational Mathematics Examination

Relationships

No evidence-backed relationships are recorded.

Current clusters

build1 publisher

A 1.5x per-token price still bought a 25 percent cheaper correct answer in AWS's benchmark

AWS's open harness records $0.0021 per correct AIME answer for gpt-5.6-luna after an 80 percent Bedrock price cut. The figure depends on running luna with reasoning disabled while mini runs at its defaults.

Publishers:aws.amazon.com

Reality

Evidence58
Adoption: Insufficient
Hype gap+15
Incentives80
Confidence55

build1 publisher

Slicing FrontierMath by category leaves about 13 problems behind each error bar

A position paper argues that CLT-based intervals dramatically understate uncertainty below a few hundred datapoints. The specialized benchmarks frontier teams build are already smaller than that before anyone slices them by task.

Publishers:arxiv.org

Reality

Evidence62
Adoption: Insufficient
Hype gap+10
Incentives30
Confidence58