Skip to content

other

Sophia optimizer

Second-order stochastic optimizer proposed for language model pretraining, using a clipped diagonal Hessian estimate as a preconditioner.

Known aliases

  • Sophia

Relationships

No evidence-backed relationships are recorded.

Current clusters