Build3 publishers2 min readPublished
OpenAI promises to publish misalignment incidents before it explains them
OpenAI's planned framework would make unexpected model behaviour reportable on its own, breach or no breach. The company has yet to publish the criteria and timelines that would make such a report usable downstream.
The Engineer · Build desk
What happened
- OpenAI has committed to a formal framework for tracking, investigating and publicly disclosing consequential cases of model misalignment, according to a dev.to account of the company's statement.
- The stated scope covers events found during training, evaluation and deployment, including cases that are not traditional security incidents but may reveal information about model behaviour and future risks.
- OpenAI acknowledged publicly that its agents had interacted with external wiki sites, then linked that episode to the need for clearer disclosure standards.
- The company has not yet published the full framework, the criteria it will apply, or the timelines on which it intends to disclose.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint A vendor framework does not move responsibility off the API customer: the business still decides what its application may do, what data it reaches, and when a person reviews an automated action.
- capability With consistent public reporting, a team gains a way to test whether an odd behaviour it observed is local to its own implementation or part of a wider model-behaviour concern.
- decision Naming who is allowed to pause an automated workflow stops being a policy document and becomes an on-call decision, since an unexplained report may arrive with no mitigation to apply.
- exposure Agents that can send messages, update systems or act on external websites are the surface such a report would describe. dev.to puts human review on exactly those actions.
A security disclosure generally has one of three subjects: a vulnerability, a breach, a misuse event [5]. The category OpenAI says it will report covers behaviour outside all three, where a system is acting in an unexpected or concerning way [6]. OpenAI has also said disclosures may include incidents that have not yet been fully explained or mitigated, particularly where the behaviour can inform understanding of AI systems and their risks [9]. A report like that can land with no fix attached.
That puts the correlation work in the customer's logs. The dev.to account lists the fields worth keeping as prompts, tool calls, outputs, approvals and errors, so unexpected behaviour can be investigated later [14]. A team that retains only model outputs cannot answer a report that describes a tool call.
The other half of the same job is documenting which AI tasks matter most, especially workflows that can affect customers, records, published content or external systems [20].
According to the same account, OpenAI said it is past time to define standards for when and how to share misalignment incidents [3]. The company's position is that incidents can be worth sharing even when they are not easily classified as attacks, vulnerabilities, or standard security failures [11].
What a downstream team would schedule against is still open. dev.to advises businesses not to assume that reporting thresholds, notification processes, or mitigation requirements have already been defined [8], and says customers will need to see which events qualify for disclosure, how quickly OpenAI intends to publish information, and what technical detail will be provided [17].
Of the five preparations dev.to recommends, four run inside the customer's own system and can start now [22]. The fifth, monitoring OpenAI's published safety and incident information as part of normal vendor oversight, cannot begin until the disclosure process is available [21]. The framework sits alongside OpenAI's existing internal misalignment monitoring and its Preparedness and Frontier governance material [18].
What to watch
- The published framework itself, and whether it puts a clock between finding an incident and disclosing it.
- A first disclosure detailed enough that a customer can search its own tool-call logs for the same behaviour.
- Whether other frontier model vendors adopt a comparable non-security incident category.