Michigan team has Claude and ChatGPT read birdwatcher comments for species interactions

The nonnative-plant result is the finding a land manager would act on. According to the phys.org write-up, the team turned up signs that nonnative plants are creeping into West Coast ecosystems, a spread that could otherwise go unnoticed ^[7]. The write-up does not report how many comments were read, how often either model agreed with a human reader, what the runs cost, or how many nonnative species turned up.

The two case studies differ in difficulty. For the West Coast lady butterfly, the interaction was as straightforward as the butterfly landing on a flower ^[12]. Birds come in a wider array: eating or being eaten, flocking, fighting over a spot at a feeder ^[11]. Only the butterfly case is described as involving plants ^[10], so we'd expect the nonnative finding to come from it.

Brian Weeks, a senior author, said "I was surprised, and delighted, by the wide range of natural history observations that members of the public noted," ^[3]. Humans could readily spot such interactions in comments, even when no ecological vocabulary was used, but there were millions of observations to sift ^[14].

Field data on interactions is scarce for a practical reason. Lead author Hengxing Zou said, "In many cases, species interactions require direct observation and that's really hard to come by simply because you can't keep a lot of people in the field monitoring the same patch of an ecosystem for 24 hours a day, seven days a week," ^[1]. The substitute is a community of hundreds of millions of nature lovers whose observations already exist ^[4], each uploaded with metadata such as when and where it was made ^[15]. Kai Zhu, a senior author, said, "This approach gives us something ecologists have long needed: a way to examine species interactions across broad spatial and temporal scales," ^[2].

We think the approach is sound for what it claims, which is finding candidate interactions at scale. A comment is a presence record. It exists only where and when someone chose to write one, so a missing comment is not evidence that an interaction did not happen, and counts of comments are not rates of interaction.

Agreement with a human-coded sample would show how many comments a careful human reader would catch that the models miss. The team wrote prompts for existing models ^[13], so we'd expect the cost that scales with the data to be inference on every comment.