Skip to content

Topic

LLM Sampling

The stage of text generation in which a decoder turns a language model's probability distribution over the vocabulary into one chosen token, using strategies such as greedy decoding, temperature scaling, top-k and nucleus sampling.

Current clusters

build1 publisher

Halving temperature turns a 4:1 token preference into 16:1

Shrijith Venkatramana's walkthrough of sampling puts the odds-ratio arithmetic behind temperature on the page. It shows how much of the difference between two runs of one prompt is settled after the model has finished computing.

Publishers:dev.to

Reality

Evidence58
Adoption
Insufficient
Hype gap+8
Incentives30
Confidence55