Security1 publisher3 min readPublished
TikTok's own review says the safety fix was withheld "by design," and that phrase is now evidence
Blackburn and Blumenthal want answers by September 1 on a control group of about 10% of users, including minors, that did not get an anti-rabbit-hole algorithm change.
The Watch · Security desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- Two U.S. senators are demanding answers from TikTok following media reports that the company intentionally turned off safety features for a subset of users to test how the protections would impact engagement with the app.
- In a letter on Wednesday, Sens. Marsha Blackburn (R-TN) and Richard Blumenthal (D-CT) criticized TikTok for having 'knowingly withheld a critical safety measure for millions of American users - including children - in order to determine whether protecting users would impact its financial bottom-line.'
- The letter follows Bloomberg reporting earlier this month based on a confidential company document analyzing the account activity of Chase Nasca, a 16-year-old who killed himself in February 2022 after viewing a stream of content related to sadness and suicide.
- The document revealed Nasca was part of a control group comprising about 10 percent of the platform's user base for which changes the company made to the algorithm to prevent repeated exposure to harmful subjects were withheld.
- The company review found the teen had viewed more than 7,500 videos in the two weeks leading up to his death, with 73% having themes of sadness or personal struggles and 10% violating the company's rules around the normalization of suicide or self harm.
Compiled by The WatchSomething wrong?How this is made
Why it matters
Senators Marsha Blackburn and Richard Blumenthal have asked TikTok to answer, by September 1, why it ran what they call "experiments" that disabled safety features for some users, and why minors were in the control groups [12][13]. The reason this matters more than the usual letter-to-a-platform ritual: the withholding is documented in TikTok's own internal review, in language that describes it as intentional [6].
The letter, sent Wednesday, follows Bloomberg reporting earlier this month based on a confidential company document that analyzed the account activity of Chase Nasca, a 16-year-old who killed himself in February 2022 after viewing a stream of content related to sadness and suicide [2][3]. According to that document, Nasca was in a control group of roughly 10 percent of the platform's user base for which changes TikTok made to its algorithm to prevent repeated exposure to harmful subjects were withheld [4]. The internal review's own wording: "TikTok's filter bubble prevention strategies did not take effect on this user by design" [6].
The same review found Nasca viewed more than 7,500 videos in the two weeks before his death, with 73% carrying themes of sadness or personal struggles and 10% violating the company's rules on normalizing suicide or self harm [5]. Do the arithmetic the document invites: that is at least 5,475 videos about sadness or struggle [1], at least 750 that broke TikTou's stated rules [2], and an average of roughly 536 videos a day [3].
The senators wrote that the company "knowingly withheld a critical safety measure for millions of American users" and did so "in order to determine whether protecting users would impact its financial bottom-line" [2]. That is the characterization of two legislators, not a finding of fact. But it points at the structural problem with holdout testing a protection: the measurement only has value if the unprotected group stays unprotected long enough to generate a difference, and the difference being measured is engagement.
Asked about the document, TikTok told Bloomberg that it "continue[s] to invest significantly in Trust & Safety, including robust detection systems and dedicated enforcement teams that proactively remove content that violates our Community Guidelines" [7]. That is a statement about content removal, not about experiment design. TikTok did not immediately respond to a request for comment from Recorded Future News [8].
The legislative hook is the Kids Online Safety Act, which advanced out of the Senate Commerce Committee on August 5 and which the senators say "will address exactly the kind of conduct these disclosures raise: platforms designing products and algorithms to maximize addiction while children bear the consequences" [9][10]. It is still unlikely to pass this session, with differences remaining over whether companies would carry a legal duty to prevent foreseeable harm [11]. The timing is not incidental: the controversy lands during a closely watched trial in which four state attorneys general accuse Meta of deliberately making Facebook and Instagram addictive to minors [14].
For anyone who runs experiments on a trust and safety surface, the operational lesson is narrow and unpleasant. A holdout group is written down. The ticket, the cohort definition, the readout, and the phrase "by design" all survive in a document that a plaintiff or a committee can eventually read. Whether a protection should be measured against engagement is a policy question; whether that measurement is discoverable is not.