Skip to content

Topic

LLM pretraining

Training a language model's base weights from random initialization on large token corpora, before any instruction tuning or alignment work.

Current clusters