Product1 publisher3 min readPublished
OpenAI's louder rollout made its independent math advisers look like company appointees
OpenAI's louder announcement of a nine-member independent math advisory group caused widespread confusion over whether the company had appointed it. Its first job is helping release scores more results from an unreleased OpenAI model, a wave researchers already dread.
The Product Desk · Product desk
What happened
- The group announced itself on September 21 in a guest post on the blog of Terence Tao, a UCLA Fields Medalist who has criticised AI companies' conduct in mathematics.
- Member Martin Hairer said the group gets no financial, technical or other support from OpenAI and that members signed only standard confidentiality terms.
- Mathematicians who spoke to The Verge, including one member, said the rollout bore the hallmarks of OpenAI's earlier rushed moves into mathematics.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- exposure Nine members now carry reputational risk from messaging they do not control, and Hairer himself expects AI companies to try to spin the group's involvement.
- decision AGMAI has to decide whether to publish its own advice on the coming results, the step OpenAI permitted that would show independence to colleagues who doubted it.
- precedent Any other frontier lab that signs on with AGMAI inherits the same question of who announces first, with OpenAI's rollout as the example of how it goes wrong.
Most people learned from OpenAI that nine of the field's leading figures had formed an independent group to advise AI labs [2]. They met the Advisory Group on Mathematics and Artificial Intelligence through a rollout The Verge describes as much louder than the group's own [5].
OpenAI's team apparently expected readers to take the text at its word. Its announcement stressed the group's independence repeatedly and said members were free to challenge the company publicly and to publish their advice [3]. Readers judged the group by who was announcing it. Several of the mathematicians The Verge spoke to in the following days assumed OpenAI had appointed it [6].
The suspicion had a basis. AGMAI's website says the group formed after OpenAI approached some of its eventual members about an advisory board, before they struck out on their own and invited others [7]. It is still unclear which of the nine OpenAI approached first [7]. Martin Hairer, a member who teaches at Imperial College London and EPFL, told The Verge the group gets no financial, technical or other support from OpenAI. He also said members signed nothing beyond standard confidentiality terms for early access to research [8]. "We don't work for OpenAI, are not paid by them, and it's totally independent," he said [10]. He called the preceding days "very frustrating" and "intense" [9].
The group's first job is release logistics: helping coordinate the release of scores more results OpenAI says its unreleased model has produced [12]. The guest post on Terence Tao's blog [1] set the remit as advising OpenAI and other frontier labs "on the review and communication of emerging results" [2]. OpenAI added that the group "will not be responsible for advising us on how to pace our internal progress on mathematics" [4]. So the group can shape how the results are presented. How fast OpenAI produces them stays with OpenAI, and the coming wave is what researchers dread [12].
The Verge wrote that OpenAI can "make impressive breakthroughs in mathematics, then colossally screw up announcing them," and of this attempt to do better: "Somehow, it has botched that too" [14]. Mathematicians it interviewed, including one member, said the rollout bore the hallmarks of OpenAI's earlier rushed forays into the field [15]. That reporting covers the announcement, because the group's review of the coming results has not yet happened in public. Hairer wrote in his own blog post that it would be naive to think AI companies would not try to spin the group's involvement [13].
Anyone setting up an outside review body, a security board or a customer council, can place it on two axes before announcing it. One axis is who speaks first and loudest, the reviewer or the reviewed. The other is whether the remit covers what the audience fears most. A body announced by the reviewer, with the feared thing in scope, gets the benefit of the doubt. A body announced by the reviewed party, with the feared thing out of scope, is assumed captured whatever the release says. On current evidence AGMAI's rollout landed in that second corner, with OpenAI louder [5] and pacing outside the remit [4]. Moving out of it takes published advice the company would not have written, and OpenAI has said members may publish [3]. Hairer said the group has begun talks with other frontier labs, though he declined to say which [11].
What to watch
- Whether AGMAI publishes its own advice on OpenAI's coming batch of results before or alongside OpenAI's release.
- Which other frontier labs AGMAI names, after Hairer said talks had begun but declined to identify them.
- Whether AGMAI discloses which of its nine members OpenAI first approached about an advisory board.