FDA's generative-AI device rulebook is still a draft, and the comment docket shuts October 19

The two axes matter more than the exam metaphor. FDA says the rigor of premarket testing would scale with a product's risk profile, and that the profile could turn on whether a product steers a user toward an action or provides information, and on what happens when a user relies on a faulty output ^[12]. Companies are already shipping chatbots that help patients manage mental health, diabetes and other conditions ^[24], and where any one of them lands on those two axes decides whether its evidence package is a benchmark run or a prospective clinical study with trained patient actors ^[19]. Arguing that placement costs a comment letter now. It costs a submission cycle later.

The second structural choice is the tested object. FDA would evaluate the final, user-facing device rather than foundation models or isolated subcomponents ^[8]. A base model vendor's own scores are therefore not your evidence file, and the paper's separate treatment of foundation models and agentic systems ^[7] does not change the unit being graded. Any team planning to swap or upgrade the model underneath a cleared product is holding a question the paper leaves open: how much of the competency record survives that change.

Benchmarking is meant to cover clinical knowledge, analytic capabilities, safety behavior, communication and generalizability, scored with rubrics and expert adjudication ^[10][13]. Rubric authorship is where the leverage sits, and it is not vacant: academics have already proposed requiring generative AI products to pass licensure exams, and the American Medical Association is weighing whether AI licenses are the right path ^[21]. FDA also concedes benchmark testing may not fully capture real clinical practice, which is why clinical confirmation is on the menu alongside real-world evidence collection after launch ^[14][c11b]. Count the stages and the framework asks for three separate evidence streams per device rather than one clearance event ^[20].

The paper is explicitly not draft or final guidance and does not propose policy changes ^[9], and it declines to address whether any of these approaches sit inside FDA's existing legal authorities ^[4]. Rick Abramson, who directs the Digital Health Center of Excellence, wrote on LinkedIn that FDA is putting its ideas in the public square to refine them before translating them into policy ^[16]. That is the operative word. CDRH also says its goal is a nimble approach using least burdensome principles and timely patient access ^[23], and the agency framed the whole effort as aligned with the administration's priority of using AI to speed medical products to market ^[18]. Both lines are quotable back at the agency when the guidance arrives heavier than the paper.

The consultation is the visible end of work already under way: an advisory committee met in 2024 and 2025, FDA has published several requests for information, and it recently convened health AI companies to guide its thinking ^[22]. The firms in those rooms know what they intend to file. Respondents are not required to answer every question and may submit partial responses ^[11], so the entry price for everyone else is one well-argued question, not the whole paper.