Build1 publisher2 min readPublished
Open-weight safety pact hints Baseten could test controls in its runtime, but no division of labor is confirmed
Base Labs has signed Hugging Face and Goodfire to publish methods for training, evaluating and monitoring open models, with the check running where the activations are. The partners have not published an evaluation suite yet.
The Engineer · Build desk
What happened
- Baseten launched an open-weight safety partnership through its Base Labs research arm on September 17th, bringing in Hugging Face and Goodfire to develop public methods for training, evaluating and monitoring open models.
- Hugging Face operates the platform where models are published and downloaded, Goodfire develops interpretability systems, and Baseten supplies the runtimes used to serve models in production.
- TechCrunch reported that Hugging Face listed over 6,000 models associated with abliteration, a family of techniques used to weaken or remove a model's safeguards.
- Base Labs calls the planned output a safety standard, and the partners have yet to say how a compliant model would be distinguished from one that merely performs well on a narrow safety benchmark.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- exposure If the check runs in the serving stack, the company answering the request holds a judgement about a model it neither trained nor published.
- constraint Enforcement inside one provider's runtime reaches only the requests that provider serves, so a copy of the weights running on someone else's hardware is governed by whoever runs it.
- decision Teams cannot test their own models against the standard until the specification is published, so any adoption decision waits on it.
The check Base Labs describes would sit beside the weights. Output filters and application-level policies still matter, and they get easier to bypass once a modified model has been downloaded and deployed somewhere else [10]. A monitor tied to a model's internal features can supply separate evidence about what changed and how the modified system behaves [15]. Goodfire builds that kind of system. The San Francisco lab was founded by interpretability researchers and operators including Eric Ho, Tom McGrath and Dan Balsam [12], and it says its systems are designed to help researchers understand, monitor and align models by inspecting their internal representations [13]. Ho previously described the company's goal as making neural networks easier to understand and repair "from the inside out" [14].
Developers can inspect and modify parameters, and that access widens the pool of researchers who can study failures. It also lets a user strip refusal behaviour out and redistribute the altered model [9]. Weights can be copied, fine-tuned and repackaged with neither the original developer nor the original host in the loop [11].
Base Labs calls the planned output a safety standard [16]. The partners have not named the six things an adopter would need: a model architecture, a dataset, an evaluation suite, a certification process, a software package and a licensing framework [21].
The measurement problem is the harder one. Some evaluations reward a model for recognising dangerous scenarios or following complicated instructions, and those skills can improve alongside the very capabilities being measured for risk [19]. A score can rise because the model got more capable. Whether the controls held is a separate question for the operator. According to runtimewire, a workable standard needs measurable requirements and repeatable tests, plus a way to account for modifications made after release [18].
The announcement does not assign formal responsibilities [7], and the division of labour has been inferred from the three businesses: Hugging Face can reach publishers and users, Goodfire can contribute methods for detecting internal behaviours, and Baseten can test whether controls stay useful under production serving conditions [6]. Base Labs wrote, "We believe openness to be an advantage for AI safety" [4]. It says it will develop and publish methods that put safety into both model training and deployment [3].
The instinct is consistent with how Baseten started. Its founders built the company in 2019 after repeatedly watching capable models stall before production behind slow deployments, fragmented tooling and systems that failed under real traffic [20].
What to watch
- A published evaluation suite or certification process would turn the standard label into something a developer can test a model against.
- Whether Hugging Face applies any resulting method to the abliteration-associated models already listed on its own platform.
- Whether any inference provider other than Baseten commits to running interpretability monitors in production serving.