BuildNot yet confirmed elsewhere1 publisher3 min readPublished
The cheap-memory pitch for High Bandwidth Flash needs a grade that has no ship date
OXMIQ's Hot Chips model gives a flash-backed rack 14 times the capacity and 58 percent of the bandwidth. Two of the three HBF grades cannot keep pace with today's HBM.
The Engineer · Build desk
What happened
- SanDisk raised High-Bandwidth Flash in early 2025 as a way to give AI accelerators terabytes of cheap memory and lean less on HBM.
- At Hot Chips 2026, GPU IP firm OXMIQ Labs said HBF cannot replace HBM across the vast majority of workloads.
- The spec has three performance grades, and only Grade 3, at 3.072 TB/s over 32 GT/s UCIe 2.0, is presented as competitive with HBM4E.
- Because HBF is 3D NAND, it reads in 64-byte to 4KB blocks and writes in 4KB pages.
- OXMIQ's parity model of a 72-GPU rack gives flash 294.9 TB at 922 TB/s against HBM's 20.7 TB at 1,584 TB/s.
Why it matters
- constraint Capacity per dollar stops being the deciding metric once concurrency rises; the bandwidth available per stored byte sets the throughput ceiling, and flash sets it low.
- decision Memory choice moves down a level, from a rack-wide standard to a per-box call about whether that box is capacity-bound or throughput-bound.
- exposure Anyone building a capacity roadmap on the cheap-memory framing is exposed to a grade that exists only as a specification line with no stated availability.
- contradiction The originating vendor positioned HBF as an HBM substitute; the Hot Chips modelling treats it as a cold-data tier that damages performance in many workloads, which reverses who should be...
The useful figure is bandwidth per byte held, and the parity model supplies it. On OXMIQ's numbers, the HBM rack carries roughly 77 TB/s of aggregate bandwidth for every terabyte of capacity [19]. The flash rack carries about 3.1 [20]. Capacity goes up fourteen times and the rate at which each stored byte can be delivered falls by about twenty-four [21].
Per model instance the arithmetic is worse. Flash-only lets each GPU hold its own Kimi-K2 and run 72 instances in the rack; HBM-only needs eight GPUs to hold one instance and manages nine [13]. Divide the aggregates and each of the 72 flash-backed instances sits behind about 12.8 TB/s [22], each of the nine HBM instances behind about 176 TB/s [23]. Eight times the instances, each fed at roughly a fourteenth of the rate [24]. Whether that trades well depends entirely on how many tokens per second each instance owes its users, which is why OXMIQ's model has HBM winning on cost per token as concurrency climbs [14].
Granularity is the second constraint and it is structural, not a matter of grades. The smallest write is 64 times the smallest read [25], so anything updated in small pieces pays for a full page every time. That is why the one workload OXMIQ names is expert weights: 1.45 TB of Kimi-K3's 1.56 TB, 93 percent, are MoE experts that are written once and read only when routing selects them [16]. Write-once and rarely read is not a general property of accelerator memory. It is a description of one tensor, which happens to be the largest one, and holding it locally also spares the all-to-all traffic that expert parallelism generates at every layer [17].
The hybrid case is where the model gets uncomfortable. HBM plus HBF gives more than four times the HBM-only capacity [26] but lands anywhere between 279 and 1,418 TB/s depending on conditions [12]. The floor of that range is below the all-flash rack and about 18 percent of the HBM rack, and the range itself spans a factor of five [29]. A part whose delivered bandwidth varies fivefold with workload is hard to size and harder to contract for.
Keep the provenance in view. OXMIQ is a GPU IP company presenting its own model at a conference [1], and SanDisk, which raised the concept in early 2025, pitched it the other way round [2]. What survives both readings is narrow: eight to sixteen times HBM capacity at roughly HBM cost [8], on a technology whose entry grade runs at 384 GB/s and, by the account at Hot Chips, barely competes with current HBM [3][6]. Grade 3 is eight times that [27] and is the only one placed against HBM4E, with no availability anyone has stated [6]. "HBM for the rack, HBF for the box" [15] is a considerably smaller product than terabytes of cheap accelerator memory, and even that version is waiting on the grade that does not yet exist.
What to watch
- Any silicon, sampling date or customer for Grade 3 HBF at 32 GT/s over UCIe 2.0, since the competitive case rests on it alone.
- Cost-per-token modelling from someone other than a GPU IP vendor, run on the same 72-GPU parity assumptions.
- Whether the HBF specification gains finer write granularity, or endurance figures, for anything beyond write-once weight pools.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence40
- Adoption8
- Hype gap+45
- Incentives62
- Confidence42
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
OXMIQ Labs, a GPU IP company, revealed at Hot Chips 2026 that High Bandwidth Flash cannot replace High Bandwidth Memory across the vast majority of workloads.
- [2]
SanDisk unveiled the High-Bandwidth Flash concept in early 2025, pledging to equip AI accelerators with terabytes of relatively inexpensive memory and reduce the need for traditional HBM.
- [3]
HBF Grade 1 uses an 8-Hi 256GB NAND stack with an 8 GT/s UCIe interface and 384 GB/s of bandwidth.
- [4]
HBF Grade 2 uses a 512GB NAND stack with a 16 GT/s UCIe interface and supports 1.536 TB/s.
- [5]
HBF Grade 3 reaches 3.072 TB/s using 32 GT/s UCIe 2.0 while retaining 512GB of capacity.
- [6]
HBF Grade 1 can barely compete against contemporary HBM, while Grade 3 can compete against HBM4E, though there is no indication of when such memory will be available.
- [7]
Because HBF relies on 3D NAND, it supports read block sizes between 64 bytes and 4 kilobytes, 4KB writes, and 4KB page sizes.
- [8]
HBF's main feature is not performance but 8 to 16 times more capacity than HBM at roughly the same cost.
- [9]
OXMIQ argues HBF should be viewed as a high-capacity memory technology rather than inexpensive HBM, and estimates that as bandwidth demand rises, adding cheap but slower HBF becomes less economical than using HBM.
- [10]
In OXMIQ's model of a 72-GPU rack running the 1-trillion-parameter Kimi-K2 model at FP4, at cost and power parity an HBM-only configuration provides 20.7 TB of memory and 1,584 TB/s of aggregate bandwidth.
- [11]
Replacing HBM with HBF in the same model increases rack capacity 14 times to 294.9 TB but reduces aggregate bandwidth to 922 TB/s.
- [12]
A hybrid HBM plus HBF configuration in OXMIQ's model provides 89.3 TB and between 279 TB/s and 1,418 TB/s depending on workload conditions.
- [13]
In OXMIQ's model an HBF-only configuration lets each GPU hold its own Kimi-K2 instance for 72 instances per rack, while an HBM-only configuration needs eight GPUs per instance and runs only nine, wasting compute.
- [14]
According to OXMIQ's model, as the number of simultaneous users and their token-generation rate increase, HBF's lower bandwidth becomes the bottleneck and the HBM-based rack ultimately delivers lower cost per token.
- [15]
OXMIQ concluded its presentation with 'HBM for the rack, HBF for the box.'
- [16]
In OXMIQ's Kimi-K3 example, 1.45 TB of the model's 1.56 TB of weights, or 93 percent, are MoE expert weights that are largely write-once and relatively infrequently read because only selected experts are activated per token; OXMIQ proposes keeping them in HBF and frequently accessed weights in HBM.
- [17]
Conventional expert parallelism distributes experts across GPUs and requires all-to-all communication at every layer; more local capacity could reduce communication between accelerators.
- [18]
OXMIQ says HBF could emerge as a specialized memory tier for huge but relatively cold datasets, while for other workloads it can do more harm than good and may even harm performance.
- [19]
The HBM-only rack in OXMIQ's model carries about 77 TB/s of aggregate bandwidth per terabyte of capacity.
- [20]
The HBF-only rack in OXMIQ's model carries about 3.1 TB/s of aggregate bandwidth per terabyte of capacity.
- [21]
Swapping HBM for HBF multiplies capacity by 14 while cutting bandwidth per stored byte by about 24 times.
- [22]
Each of the 72 HBF-backed model instances has about 12.8 TB/s of bandwidth behind it.
- [23]
Each of the nine HBM-backed model instances has about 176 TB/s of bandwidth behind it.
- [24]
The HBF rack runs eight times as many model instances, each fed at roughly one fourteenth the bandwidth of an HBM instance.
- [25]
HBF's minimum write is 64 times the size of its minimum read.
- [26]
The hybrid configuration holds about 4.3 times the capacity of the HBM-only rack.
- [27]
Grade 3 offers eight times the bandwidth of Grade 1.
- [28]
The HBF-only rack's aggregate bandwidth is about 58 percent of the HBM-only rack's.
- [29]
The hybrid configuration's low bandwidth figure is about 18 percent of the HBM-only rack's and below the HBF-only rack's, and its stated range spans a factor of about five.
Sources
1 independent publisher whose own reporting we read for this story.
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.