Change.org's AI writing assistant lengthened petitions without winning them more early support

The researchers' premise was that the four countries had tracked one another on petition style and outcomes before the tool arrived ^[5]. If that held, any divergence between Oct. 2 and Dec. 15, 2023, could be put down to the assistant ^[5]. A simple before-and-after comparison on one site would mix the tool's effect with whatever else changed that autumn. Comparing trends across countries strips out anything that moved all four together. It still depends on nothing else having hit only one side of the split during those weeks.

Style came from three measures: lexical diversity, the ratio of unique words to total words; readability, from words per sentence and syllables per word; and length, counting text and title ^[6]. Across the 1.5 million petitions analysed ^[1], the effect on those features was consistent. Petitions written with the assistant's help were longer and used more complicated and varied words ^[7].

Success was set at two low bars: the share of petitions reaching one comment within 30 days, and the share reaching 10 signatures by the time the data were collected ^[9]. Neither improved on pre-AI levels, and in some cases they got worse ^[9]. The drop in comments is measured relative to the pre-AI rate ^[10].

The result sits badly with laboratory work. "There is this existing research that shows AI-generated text is persuasive, it's believable and it can be creative, sometimes more so than people writing alone," Corpus said ^[12]. Of the field result, she said: "That was a surprise for us." ^[11]

Mor Naaman, the Cornell Tech information scientist on the team ^[2], suspects that the assistant was trained in part on petitions that had succeeded on the site before, though he said he could not be sure ^[13]. He said the data showed the AI-enhanced petitions carried the markers of what had been successful in the past ^[14]. If he is right, the tool reproduced the surface features of past winners and the support did not follow. He offered two explanations. One is that readers distrust text that feels machine-written, though he noted how early the study period was. "People didn't have their AI radars as high in 2023," he said ^[16]. The explanation he considered more likely is that AI-assisted petitions tended to be less specific and detailed ^[17].

The thing this doesn't tell you is whether any petition won what it asked for. One comment and 10 signatures measure early traction ^[9]. The comparison is also between countries and periods, so it estimates what offering the tool did to the platform's output as a whole. The report does not say how many petitioners actually used it. I think the result is solid for what it covers, and narrow: one platform, one window in late 2023, a tool built in when Change.org was among the first platforms to offer one ^[15], and outcomes counted as early engagement ^[9].