MoE Inference Economics

The cheap-memory pitch for High Bandwidth Flash needs a grade that has no ship date

OXMIQ's Hot Chips model gives a flash-backed rack 14 times the capacity and 58 percent of the bandwidth. Two of the three HBF grades cannot keep pace with today's HBM.