Topic

Count-based language models

Language models that estimate next-token probabilities from observed token frequencies, such as n-gram tables and cache models, without trained parameters.

Current clusters

build1 publisher

Sixteen times more training code buys the same gain as reading the rest of the file

A re-evaluation of checkpoints trained on 500K, 2M and 8M tokens of code moved the point where a zero-parameter count table overtakes a 2.45M-parameter transformer by 6.6x in document length, and at 4,000 tokens the table still wins.

Publishers:dev.to

Reality

Evidence58
Adoption: Insufficient
Hype gap−10
Incentives55
Confidence48