Build1 distinct publisher3 min readPublished
d-Matrix says stacking compute on a co-designed DRAM die moves bits at roughly a sixth of HBM4's energy cost. It still has not said who fabricates the die.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
The 296W in the ISCA paper is the whole argument, and it is checkable. 100 TB/s is 800 terabits per second; multiply by 0.37 pJ/bit and you get 296W exactly [1]. So the disclosed interface power is the quoted efficiency running flat out, not a typical-case number with slack hidden in it. Push the same 800 Tb/s across an interface at the roughly 2.4 pJ/bit that CTO Sudeep Bhoja attributes to moving data into an HBM4 base die [2] and the data movement alone wants about 1.92kW [2], against a per-package budget of 422W [7]. Even at 0.37 pJ/bit, the vertical link is eating 70% of that budget [3].
That is why the stack is upside down. d-Matrix puts the logic die on top so a cold plate sits on the compute silicon, and the DRAM underneath doubles as the interposer, carrying PCIe and die-to-die signals down through its TSVs [8]. The bill for that arrangement is thermal: the DRAM is designed for a 105C junction, where retention falls from a standard 32ms to 4ms and refresh has to run eight times as often [9]. The fix is geometry, not cleverness in the controller. Each microbank was shrunk to 1,366 rows and about 5.33MB, so a full refresh sweep costs 1.37% of bandwidth [10].
The same pattern runs through the rest of the die. Yield is handled with 72 spare banks out of 840 per chiplet, wired into a two-level mux chain that swaps out any two failures while keeping channels symmetric [11]. Errors are caught by a [132,128] Reed-Solomon code on the logic die, two symbol errors per 128 bytes, with a CRC behind it [12]. There is no PHY, no burst structure and no sideband pins, which killed conventional data-bus inversion, so d-Matrix compares each 128-byte flit to the previous one and carries a 1-bit inversion tag next to the ECC metadata, recovering about 20% of the I/O power DBI would have saved [13].
Every one of those is a DRAM design decision. Which is the problem, because TSMC makes the N4P logic die and Alchip does the ASIC design and packaging, and neither runs a DRAM fab [14]. Three companies make leading-edge DRAM in volume, and their HBM4 lines are effectively sold out through 2026 [15]. Analysts logged DRAM contract price rises of 90% to 95% in Q1 2026 alone [16], and J.P. Morgan expects prices up more than 400% between the start of 2024 and the end of 2026 [17]. SK hynix CEO Kwak Noh-jung told Reuters in July that demand will exceed supply capacity even beyond 2030 [18]. Nvidia is reportedly testing Rubin Ultra with as little as 192GB because HBM4E may not be sourceable [19]. d-Matrix has raised about $450 million [20].
The energy figure is the least fragile thing on the table, and it is also the least transferable: it exists because the bank layout, refresh behaviour and interleaved ECC columns were designed against the compute die, and no memory vendor stocks anything resembling it [21]. That is a moat and a single point of failure in the same sentence.
Ranked by verification strength, evidence, and original report placement.
Co-founder and CTO Sudeep Bhoja put the vertical interface's energy cost at 0.37 pJ/bit against roughly 2.4 pJ/bit for moving data into an HBM4 base die, calling it "a measured number" from working silicon.
At full tilt the vertical interface burns 296W of the 422W per-package budget the ISCA paper discloses.
d-Matrix presented Raptor at Hot Chips 2026, calling it the first 3D DRAM accelerator for generative inference: a TSMC 4nm compute die bonded face-to-face at a 36-micron pitch on top of a custom DRAM die, delivering 100 TB/s of bandwidth from 32GB per card.
The accompanying ISCA 2026 paper, written with the University of British Columbia, projects around 4.7 times higher throughput per card than HBM-based designs.
Every performance figure shown at Hot Chips, including 988 tokens per second per user on the 2.8-trillion-parameter Kimi K3 model at 1M-token context, is a d-Matrix projection built on early silicon.
Bhoja did not disclose who manufactures the DRAM die, and across the Hot Chips talk, the ISCA paper and every public announcement since the Pavehawk 3DIMC test silicon came online last September, d-Matrix has never identified who fabricates its custom DRAM.
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed vendor disclosure, no independent verification
The technical record is unusually specific and internally consistent - bonding pitch, refresh behaviour, spare-bank scheme, ECC construction, and a power budget whose arithmetic reproduces the quoted 0.37 pJ/bit at full bandwidth - and it is anchored in a peer-conference talk plus an ISCA paper co-authored with a university. But every number originates with d-Matrix, one publisher carries the story, all performance claims are explicitly projections on early silicon, and the single most consequential fact (who fabricates the DRAM) is absent.
Pre-commercial: conference disclosure only
There is no deployment, customer, design win, benchmark run by a third party, price or firm ship date. Adoption evidence consists of a conference disclosure and a prior test-silicon milestone, with a CEO remark pointing to 2027.
First-of-kind framing outruns a supply chain with no named fab
The 'first 3D DRAM accelerator for generative inference' framing, the 4.7x throughput projection and the 988 tokens/sec headline sit well ahead of what has been demonstrated: early silicon, projected numbers, half of HBM4's areal density, and a 2027 volume plan resting on an unnamed DRAM fabricator in the tightest memory market in years. The gap is moderate rather than severe because the efficiency claim is presented as measured and survives an internal power-budget cross-check, and because the reporting itself surfaces the caveats rather than burying them.
Startup positioning against HBM scarcity; conference-cycle publishing
d-Matrix is a roughly $450M-funded challenger presenting first-of-kind claims at a conference while an acute HBM shortage makes an HBM-free memory story maximally attractive to customers and future investors - a strong incentive to lead with projected advantages and to stay silent on a supplier relationship. On the publishing side, the piece is time-boxed Hot Chips coverage promoted with a temporary free-access window, which rewards volume and speed during the event week.
Specific and checkable, but single-sourced and pre-product
Confidence is moderate: the technical figures are precise, attributed to named executives and a co-authored paper, and partly self-consistent, so the descriptive claims are likely accurate as reported. What cannot be relied on is any forward claim - throughput, launch timing, or volume - because only one publisher carries the story, no third party has measured the silicon, one relayed item (Nvidia's Rubin Ultra memory testing) is second-hand, and the DRAM supply arrangement is entirely unknown.
build
Micron's own numbers say the DRAM squeeze is an allocation problem, not a cycle1 distinct publisher
build
SK hynix rules hybrid bonding out of HBM4E, leaving 55 microns to do the work1 distinct publisher
invest
Tiny corp wants Etched's numbers. Jane Street led $700M at $21B without publishing any1 distinct publisher
product
Nvidia circles Rebellions because the low-power inference tier is not optional2 distinct publishers
Distinct publishers with included, body-backed reporting in this cluster.