Scientists wary of AI scoops rein in their use of commercial chatbots

The case for a pattern rests on two episodes in five weeks ^[1], and Nature's report describes only one of them in detail. In that one, Buckmaster and Alpöge had worked on the Navier-Stokes problem for a year, according to Buckmaster ^[7]. Alpöge is a mathematician at Anthropic, and the collaboration was a personal project ^[7]. Buckmaster's post and OpenAI's announcement went up one day apart ^[13]. After a year of work, a gap that short looks like cause and effect. It is also what you would see if two groups working on the same long-standing open question in fluid dynamics ^[6] finished in the same week.

OpenAI's denial has two parts, and they cover different periods. The first rests on an investigation of Buckmaster's prompts in the two months before the 8 September announcement. Those prompts, the company said, "could not have influenced the system in any way, including through training" ^[16]. Two months is about a sixth of the year the pair spent on the problem ^[14]. The second part covers the whole period: OpenAI said its researchers and agents had not seen any of Buckmaster and Alpöge's work, by any means, until the pair released it publicly ^[17]. Both statements are the company's account of its own inquiry. Nature's report does not describe any outside review of it.

The precautions researchers describe are aimed at the leak explanation. Sandra Laurentino, a reproductive epigeneticist at the University of Münster, said she no longer trusts AI systems ^[9]. She uses them only to find out why her code fails, and first changes every parameter and variable name to a generic label such as "group A has feature X" ^[9]. "I am quite careful when it comes to AI," she said, adding that the controversy has "made me even more paranoid" ^[10]. Samuel Mehr, an auditory cognitive scientist in Auckland, explained his lab's rules in terms of ownership. "I think it's incredibly risky to hand out your intellectual property to third parties when you don't know what they're going to do with it," he said ^[12].

In my view these are sensible rules for unpublished data, whatever happened in September. They address only one of the two explanations. Geoffrey Irving, chief scientist at Resolution, an AI-safety research organization in Berkeley, holds the other ^[4]. Researchers are "going to be scooped, but not because they've uploaded a manuscript", he said ^[4]. "They're going to get scooped because the AIs are very good at solving problems, and they're going to get better and better." ^[5] Some AI researchers share that expectation and some do not, Nature reported ^[3].