benchmark

OpenCompass

Open-source evaluation suite that scores language models across a battery of academic benchmarks and reports averages.

Current clusters

build2 publishers

Edge0's 35B would need 22 GB/s of SSD bandwidth without the file cache

Edge0's Apache-2.0 engine reports a 35B model peaking at 2.9 GB of RAM on a 24 GB Mac mini. Work the decode rate against the active set and most of those expert reads have to be coming from the operating system's file cache.

Publishers:dev.toruntimewire.com

Reality

Evidence38
Adoption12
Hype gap+48
Incentives55
Confidence52