Researchers aim AI data-center power cuts at server software

PUE scores the building around the servers. It shows whether cooling and power systems are wasteful. It does not account for whether the software on those servers is doing useful work ^[4]. A server running dead code at full power leaves the ratio where it was ^[4].

At an efficient hyperscale site, cooling is about 7% of demand ^[6]. Nvidia says its Blackwell power profiles save up to 15% of energy ^[14]. Servers draw around 60% of demand ^[5]. Apply a 15% cut across that whole share and it comes to about 9% of facility power ^[18]. That is more than the same site would save by removing cooling entirely ^[19]. The comparison assumes the saving reaches every server. The profiles themselves tune GPUs: compute and memory frequencies, power limits, NVLink states and cache settings ^[13].

Nvidia's numbers are consistent with each other. Under a fixed power cap, 15% less energy per GPU leaves room for about 1.18 times as many GPUs. At 97% performance each, that is about 14% more throughput ^[17]. That only holds where a site is short of power and still has space and GPUs to add. The workload also has to keep 97% of its speed at the tuned settings. Nvidia claims up to 13% ^[14], just under that bound.

Perseus, Chung's training optimiser, finds the parts of a large-model training job that have less work to do. It slows them so they finish alongside the busier parts ^[11]. A part that finishes early only waits, so slowing it costs no throughput. The saving depends on imbalance, since a job whose parts carry equal work has nothing to slow. Perseus cut training energy by up to 30% this way without changing the hardware ^[12].

The FP8 figure needs the same caution. A slightly lower-precision format has to hold up on another model and prompt mix before that saving carries over. ML.Energy measured it on one model, Alibaba's Qwen 3 235B A22B Thinking, and only on problem-solving tasks ^[10].

Chung is a PhD candidate at the University of Michigan and a researcher with the ML.Energy initiative ^[8]. He treats computing as a stack: hardware at the bottom, then systems software, algorithms and applications ^[9]. Chips are slow and costly to replace. The three upper layers are easier to change, and gains there compound ^[9]. "The answer to what seems to be the core question - 'Can software or algorithms meaningfully help with power and energy?' - is yes, 100%," Chung said ^[7].

The other measures in the report are routine operations work. They are right-sized cloud instances, cached repeat prompts, small models for routine requests, better batching and compilers, and shorter prompts with capped outputs ^[16]. Chetan Visrolia of SHI has an audit that needs no tooling. He said the most obvious waste is often legacy equipment supporting old code ^[15]. "It is always the lowest-hanging fruit for quick savings," he said. "It can easily be found in any data center, as it is the loudest rack on the floor." ^[15]

Sophie Hall, a doctoral student at ETH Zurich, has studied workload shifting across Google's data-center fleet. She argues that consumption is not necessarily the main problem ^[1]. "It's more like: when do they use it, where do they use it, and how is it interacting with the grid?" she said ^[1].

Tom's Hardware did not put a price on any of these measures. The case that software is the cheaper route rests on the hardware half of Chung's stack argument ^[9].