Distributed Training Fault Tolerance

PyTorch 2.14 moves fault tolerance out of the NCCL backend and into c10d itself

Process-group reconfiguration, one-sided RMA windows and a backend-agnostic Flight Recorder land as core distributed features, alongside native linear algebra on Apple Silicon. The recovery numbers you would plan against are yours to measure.