Skip to content

Topic

LLM token accounting

Measuring and attributing the input and output tokens a language model request consumes, including cached prefixes, as the basis for cost and capacity estimates.

Current clusters