Science1 publisherNot yet confirmed elsewhere2 min readPublished
MIT Lincoln Laboratory's AI hardware survey now compares more than 120 accelerators on peak performance and power
MIT Lincoln Laboratory's AI accelerator survey has grown from 57 accelerators in its first paper to more than 120, compared on peak performance and power. Its ratings are published ceilings, so compute planners get an independent map of the market and still have to test sustained speed themselves.
The Scientist · Science desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- Reuther said the survey began about eight years ago, after a sharp rise in accelerator announcements prompted questions from government sponsors of Lincoln Laboratory's work.
- The team sorts each accelerator by whether it comes as a chip, a card or a complete system before setting them side by side.
- New entrants are tracked through daily news and citation searches that flag technical press articles, company announcements and industry presentations.
- The latest paper modeled architectural choices, such as adding more cores per processor, to see how each would change a system.
- Six startups announced their first AI accelerators in just the past few months, and Reuther plans to keep the survey going for the foreseeable future.
Compiled by The ScientistSomething wrong?How this is made
Why it matters
- constraint Because lower precision drove part of the rise in peak numbers, two accelerators' ratings compare fairly only when both were quoted at the same precision.
- exposure Companies that keep performance and power data private are hard for the survey to place, so its coverage leans toward vendors that publish.
- decision Compute planners can narrow a shortlist from the survey's peak ratings, but the purchase still turns on measured throughput and price for their own workloads.
Reuther and four LLSC colleagues, Michael Jones, Peter Michaleas, Jeremy Kepner and Vijay Gadepally, have produced six papers in the series since 2018 [13][2]. Every figure in them comes from public sources. That is a hard rule to work under, because some companies prefer to keep their performance and power data private [7]. Reuther describes the lab's role in terms of independence [1]. "AI and the hardware it runs on are such hot topics, and it is important for Lincoln Laboratory to be an unbiased technical advisor for choosing and pursuing the right technologies," he said [1]. The survey is independent in who compiles it; its inputs are numbers that companies chose to release [7].
The most useful finding for anyone reading the charts came in 2022. That paper traced rising accelerator performance to two sources: smaller, denser transistor designs, and lower numerical precision, meaning calculations that carry fewer significant digits [10]. A peak rating taken at low precision counts cheaper operations than one taken at higher precision. Only the first of the two sources is the same work done faster [10].
The thing this doesn't tell you is what an accelerator sustains on a real model, or what it costs to buy and run. Peak performance and peak power are the survey's main metrics [3], and the MIT account does not mention prices or measured throughput. Hardware type opens a second gap. ASICs perform only specific tasks, while GPUs, FPGAs and dataflow designs can be configured for a variety of workloads [14]. A peak rating records none of that flexibility, even though the survey's stated aim is to find the best accelerators for certain needs [15].
The list also goes out of date quickly. It has more than doubled in size since the first paper [16]. "It continues to surprise me how each year another five to 10 startups get funded and announced, and then release new AI accelerators," Reuther said. "One might think that the landscape is saturated enough, but then another batch of innovative accelerators is introduced." [9]
What to watch
- Whether the next LAICS paper reports each accelerator's peak figure at a matched numerical precision, or adds measured throughput alongside peak ratings.
- Whether the six startups that recently announced first accelerators publish enough performance and power data to appear in the next edition.
- Whether vendors that now keep power and performance figures private begin releasing them, changing which accelerators the survey can plot.