LLM Inference Hardware

Apple's quad-die M5 Ultra puts 512GB of inference memory under the desk

Apple says the M5 Ultra reaches 512GB of unified memory at 1.2TB/s, which settles how big a local agent can get, though the September launch arrives without prices and every AI speed figure is measured against an M1.