An Iowa social work researcher wants abuse-case AI audited for bias before agencies deploy it
Aislinn Conrad of the University of Iowa proposes three safeguards for agencies that use AI in child, partner and elder abuse cases. Without them, she argues, automation could remove the human judgment that decisions to disrupt families require.
The Scientist · Science desk
Families facing hardship: public assistance or housing instability data could be flagged as risk, Conrad says. Families and survivors: input, learning why flagged, ability to contest. Agencies: fairness audit before deploying, regular review. Communities: AI might help direct resources.
- exposure Families facing hardship Conrad says public assistance or housing instability data could be flagged as risk, bringing more scrutiny, claim 11
- capability Families and survivors Conrad recommends input into how AI is used, learning why they were flagged and being able to contest, claim 2
- constraint Agencies deploying AI Conrad recommends a fairness audit before deploying, then regular review for unfair outcomes, claim 3
- capability Communities in severe need AI might identify severe housing instability or caregiver stress so agencies could direct resources there, claim 9
| Who | How | Kind | Claim |
|---|---|---|---|
| Families facing hardship | Conrad says public assistance or housing instability data could be flagged as risk, bringing more scrutiny | exposure | 11 |
| Families and survivors | Conrad recommends input into how AI is used, learning why they were flagged and being able to contest | capability | 2 |
| Agencies deploying AI | Conrad recommends a fairness audit before deploying, then regular review for unfair outcomes | constraint | 3 |
| Communities in severe need | AI might identify severe housing instability or caregiver stress so agencies could direct resources there | capability | 9 |
What happened
- The paper, titled Centering Survivor Agency in AI for Violence Prevention, appears in the Journal of the Society for Social Work and Research (2026).
- Conrad wants families and survivors to have real input into how the systems are used, to learn why an algorithm flagged them, and to be able to contest the result.
- Agencies should audit a system for fairness before deploying it and then review it regularly for unfair or discriminatory outcomes.
- She also wants AI aimed at identifying needed resources such as housing and child care instead of at increasing electronic surveillance.
- In February, Iowa's Department of Health and Human Services partnered with Google to develop VISION, an AI-based system for child welfare case and data management.
Why it matters
- exposure Families on public assistance, in unstable housing, or with earlier agency contact carry the records Conrad says an algorithm could read as risk, so hardship itself could draw more scrutiny.
- constraint A right to learn why a flag was raised and to contest it rules out scoring systems whose reasons cannot be put to a family in plain terms.
- decision Agencies adopting AI have to decide whether to audit before launch and review afterward, and Conrad asks for both.
- capability The same pattern-finding could be pointed at communities with severe housing instability, caregiver stress or service gaps, so agencies could direct resources there.
Conrad's argument turns on training data. "When historical data shows that poor families and families of color have been disproportionately surveilled by child welfare systems, the algorithm learns that pattern and repeats it at scale, and does so invisibly," she said. "A 'neutral' algorithm trained on biased data is just biased automation." [12]
The question came from casework. As a social worker, Conrad performed child protective services investigations in a rural part of south-central Kansas where drug use and poverty were prevalent [7]. "I saw the link between poverty and abuse over and over, and I began wondering, 'Is it really that poor kids are just more likely to be abused? Or is it a surveillance issue because they come to the attention of social services more often?'" she said [6].
In our view, a model trained on agency records learns whom the agency noticed. If noticing tracks poverty, a model can match its own labels well and still be unfair, and the pre-deployment fairness audit Conrad recommends [3] is the step that would test for it.
Conrad grants that AI systems can be useful, especially in sorting through information to find people deemed at risk, or in helping agencies judge when to intervene [8]. Her warning is that the technology could remove the human element she considers critical to informed, impartial decisions, particularly when those decisions can lead to families being disrupted or marginalized [8].
On Iowa's VISION, Conrad says it or other AI-enhanced systems could lead to information being unintentionally misused [14]. The phys.org report of the paper describes recommendations. It reports no measured audit, sample of families or error rate showing how much bias any such system produces, and it does not say whether VISION scores families for risk.
Her closing point is about purpose. "The goal shouldn't be more monitoring of vulnerable families," Conrad said. "It should be whether the technology genuinely improves access to services and outcomes for families." [10]
What to watch
- Whether Iowa's health and human services department or Google says what VISION does with risk-related data, and whether any audit or appeal process is planned.
- Whether any agency publishes a fairness audit of a deployed child welfare or abuse-screening model, with flag rates broken out by income and race.
- Whether other states using AI in abuse cases adopt explanation and contest rights for flagged families.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence35
- Adoption
- Insufficient
- Hype gap
- Insufficient
- Incentives
- Insufficient
- Confidence45
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Aislinn Conrad, associate professor in the University of Iowa School of Social Work and a former social worker, published the study, titled Centering Survivor Agency in AI for Violence Prevention, in the Journal of the Society for Social Work and Research (2026).
- [2]
Conrad recommends that families and survivors have meaningful input into how AI systems are used, that they understand why they have been flagged by an algorithm, and that they be able to contest decisions.
- [3]
Conrad recommends that before deploying an AI system, agencies audit it for fairness and then regularly review it for unfair or discriminatory outcomes.
- [4]
Conrad recommends using AI systems to identify needed resources, such as housing, child care and services, rather than to increase electronic surveillance.
- [5]
In February, Iowa's Department of Health and Human Services partnered with Google to develop VISION, a new information system that uses AI to modernize how child welfare agencies manage cases and data.
- [6]
"I saw the link between poverty and abuse over and over, and I began wondering, 'Is it really that poor kids are just more likely to be abused? Or is it a surveillance issue because they come to the attention of social services more often?'" Conrad says.
- [7]
As a social worker, Conrad performed child protective services investigations and helped children in foster care safely return to their families, in a rural region of south-central Kansas where drug use and poverty were prevalent.
- [8]
Conrad says AI systems can be useful, especially for sifting through information to identify people considered at risk or to help agencies decide when intervention may be needed, but warns the technology could remove the human element she says is critical to informed, impartial decisions, especially those that can lead to families being disrupted or marginalized.
- [9]
AI technology might be trained to identify communities where housing instability, caregiver stress or inadequate services are severe, and agencies could then direct resources toward those needs.
- [10]
"The goal shouldn't be more monitoring of vulnerable families," Conrad says. "It should be whether the technology genuinely improves access to services and outcomes for families."
- [11]
Conrad says data such as public assistance, housing instability or previous contacts with social service agencies could be flagged as signs of risk in an algorithm, so families already facing economic or social hardship may face more scrutiny.
- [12]
"When historical data shows that poor families and families of color have been disproportionately surveilled by child welfare systems, the algorithm learns that pattern and repeats it at scale, and does so invisibly," Conrad says. "A 'neutral' algorithm trained on biased data is just biased automation."
- [13]
Social service agencies are increasingly using artificial intelligence to handle cases involving child abuse, intimate partner violence and elder abuse.
- [14]
Conrad says VISION or other AI-enhanced systems could lead to information being unintentionally misused.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- phys.orgSocial service agencies need safeguards when using AI
1 article · October 14, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
- Public-Sector AI GovernanceFollow
- AI in child welfareFollow
- Algorithmic bias and stereotypingFollow