Build2 publishers3 min readPublished
Anthropic's referral of a user's Claude threats to police ends in a Florida felony charge
Anthropic's human reviewers sent a Florida woman's threat from a Claude chat to police, at least the third such conversation to reach officers since August. Anyone sending user data to a hosted model should now assume flagged text is read by vendor staff and can be passed to police.
The Engineer · Build desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- The arrest report quotes a Sept. 26 message from a user identified as "Carli" saying she would "shoot up" the Lee County Sheriff's Office, and a next-day message allegedly mentioned a new gun.
- Court records list a Sept. 30 felony charge under Florida Statute 836.10, the offense of a written or electronic threat of a mass shooting or act of terrorism.
- In San Antonio, a 22-year-old man was arrested and charged with felony terroristic threat after Aug. 11 queries in an Anthropic chat about shooting at an elementary school. The FBI had tipped off the police who arrested him.
- In San Francisco, Anthropic notified police about an Aug. 14 Claude user who threatened CEO Dario Amodei and said he had bought an AR-15, but that user was not arrested or charged.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- exposure Text a product sends to Claude can be read by Anthropic staff once it trips the threat screening, and the referral that follows can start with Anthropic, with no police request needed.
- contradiction Anyone gauging disclosure risk from Anthropic's published counts will see zero emergency requests, while self-started referrals like the Lee County one sit outside that tally.
- precedent Lawsuits against OpenAI for failing to warn police give every hosted-model vendor a liability reason to send more borderline conversations to law enforcement.
The arrest report, as Tom's Hardware describes it, lays out the path from chat to detective in order [3]:
1. Anthropic monitors chats for key phrases and for content that could be deemed threatening [3]. 2. Depending on severity, a flagged conversation can be elevated to human review [3]. 3. The review team decides whether to report. In this case it did, and an LCSO intelligence detective took over from there [4].
I think the design is defensible. Phrase matching decides what a person reads, and a person makes the call that carries legal consequences [3][4]. The description comes from police paperwork, though [3]. The arrest report is not yet in the court file, and Anthropic has no comment on the case on record [5][13].
The rule behind step three is Anthropic's privacy policy, effective Sept. 10. It allows disclosure to law enforcement where the company has a good-faith belief "that disclosure is reasonably necessary to ... prevent serious harm to any person or to property" [6]. The government requests report for the second half of 2025 applies a narrower test to emergency disclosures: "danger of death or serious physical injury to a person" [7]. Property is in the first test and absent from the second [6][7]. The report lists zero emergency requests from law enforcement for the period, and it does not count referrals Anthropic makes on its own [7].
As booked, the charge reads "THREAT-TERRORISTIC-STATE OFFENSES-WRITTEN/ELEC THREAT MASS SHOOTING/TERRORISM ACT" [8]. Tom's Hardware suggests the chat may fall under the statute's language on a threat transmitted "in any manner in which it may be viewed by another person" [20]. If that reading holds, the review queue matters to the prosecution as well as the referral, because a reviewer is another person who can view the chat [3][20]. The publication also says the messages the reviewers found, including the gun mention, are likely the supporting evidence for the charge [21]. No attorney is named on the docket, and a public defender was appointed [14].
Of the three Anthropic-linked cases since August, two ended in felony charges [19]. Florida's statute already has a chatbot case that reached sentencing [17]. OpenAI reported Darren Zhou to the FBI over a March ChatGPT message about killing his ex-girlfriend, according to the Palm Beach Post [17]. Zhou pleaded guilty in August under 836.10 and was sentenced to eight years' probation [17].
Tom's Hardware describes the bind as companies being expected to respect privacy while being held responsible when they do not report threats [11]. Florida's attorney general sued OpenAI in June, citing the 2025 Florida State University shooting, and asked a judge for a temporary injunction last month [9]. British Columbia sued OpenAI last month for not warning police about the Tumbler Ridge shooter, whose ChatGPT account the suit says OpenAI had identified eight months before the attack [10].
Lee County Sheriff Carmine Marceno told WINK News that what people share with an AI companion is "never truly anonymous" [18]. The reporting does not say which Claude product the Lee County user was on, or whether API and business traffic pass through the same screening [1][3]. I would not build on the assumption that they skip it. For a product that forwards end-user text to a hosted model, I'd list the vendor's review team as a recipient in the data map and treat the Sept. 10 clause as the disclosure rule in force [6].
What to watch
- The Lee County arrest report entering the court file, so the scanning and escalation description can be checked against the document itself.
- Whether Anthropic's next government requests report starts counting referrals the company initiates on its own.
- The ruling on the Florida attorney general's temporary injunction request against OpenAI, which bears on how far a vendor's duty to report reaches.