build1 publisher
Blocking checkpoint writes took up to 40 percent of wall time on AWS's H100 test runs
AWS's NVRx walkthrough puts synchronous checkpointing ahead of GPU faults as the source of idle time on its FSDP jobs, and pairs a background save with a restart that re-enters training without cycling the container.
Publishers:aws.amazon.com
Reality
- Evidence54
- Adoption20
- Hype gap+18
- Incentives76
- Confidence56