A freedom of information request has revealed that an AI system used by the UK government for assessing benefits cases is apparently getting it wrong by a “statistically significant” amount. The admission to journalists at the Guardian emerged after a fairness analysis into universal credit claimants in February 2024. It confirmed that the very tools intended to ensure equity and efficiency may in fact be discriminating against marginalised communities.
The Department for Work and Pensions (DWP) began exploring automated decision-making systems in the 2010s as part of its efforts to digitise welfare services. While early indications suggested that automated systems were being used to assess benefit entitlement and flag cases for investigation, it was not until 2021-22 that the department’s accounts revealed the use of these technologies to detect fraud in universal credit claims.
At their core, these systems rely on algorithms to quickly sift through information about claims. The algorithm applies rules to help determine whether someone qualifies for a benefit (although the DWP says the final decision is still made by a human, more on which later).
The idea was that this approach could handle claims more quickly than a human alone, while also helping make decisions more uniform and reducing delays for claimants. By using algorithms to identify potential issues quickly, the department hoped to improve overall efficiency and target resources where they were needed.
However, as these systems have become more deeply embedded, concerns about their accuracy, fairness and the unintended harm they may cause to those already in precarious situations have steadily grown.
These worries have not remained abstract, as we’ve already seen elsewhere. As part of a three-year project examining the effects of digitalisation on food security in the UK, Sudan, and India, my team and I have explored both the opportunities and risks of digitalising food assistance.
Our research has shown that the introduction from 2013 of universal credit – a welfare reform to replace six older benefits with a single, online-managed monthly payment that is adjusted when a claimant’s income changes – has coincided with a proliferation of food assistance initiatives. These include government schemes like Healthy Start, charities like food banks and food shares, and council strategies like breakfast clubs.
Our findings highlight how technology-driven welfare systems influence access to food, offering insights into both their potential benefits and the challenges they create for vulnerable populations.
Observations and analyses emerging from our data and from other academics, international observers, and independent organisations have begun to reveal how the technological shift can contribute to policies that penalise claimants before verifying their needs. This can affect their ability to afford food and other basics.
At the heart of these emerging findings lies a disturbing pattern that has been aptly labelled a “hurt first, fix later” approach.
Rather than ensuring that vulnerable claimants are supported from the start, the system appears to be designed to identify potential fraud or irregularities before releasing full payments. This puts the onus on claimants – already struggling with rising food insecurity, housing costs, and health issues – to prove that they deserve the assistance they receive.
Under this model, people can find themselves cut off or sanctioned before anyone takes a closer, more sympathetic look at the complexities of their situation.
As the DWP admitted, human oversight to correct these algorithmic judgements is crucial, and it says the final decision on payments will still be made by a human.
But a critical problem is that the DWP is grappling with severe understaffing, a problem widely documented over the past year. Civil servants within the department have described “unbearable workloads” and “all-time low staffing levels” that unions warn are contributing to a “mental ill-health epidemic” among DWP employees.
With too few staff available to handle claims promptly and fairly, the “fix later” part of the equation becomes increasingly unfeasible.
For claimants, many in or on the verge of destitution, the results can be devastating. In early 2024, 15% of UK households were experiencing hunger, including one in five with children.
Foodbank charity the Trussell Trust has recorded a 900% increase in emergency food parcel distributions since the early 2010s, and many people reliant on universal credit have found that even recent inflation-linked payment increases are cancelled out by the end of cost-of-living supplements.
With basic needs unmet, waiting weeks or months for a human review to correct an AI-driven “error” is not simply an inconvenience – it can mean missing rent or going hungry.
Digital-first approach
Those disproportionately affected by these automated judgements are often some of the same groups facing the greatest barriers to navigating digital systems: people with disabilities, the elderly, ethnic minorities and non-native English speakers.
Many lack access to stable internet or the digital literacy required to complete forms and upload documents. If algorithms already exhibit biases against these groups, and staff are too stretched to intervene, the result is compounded harm.
Automation’s promise of efficiency increasingly rings hollow. Algorithms often misinterpret complex circumstances, such as caregiving responsibilities or fluctuating work hours, as irregularities. Overburdened caseworkers may struggle to intervene effectively, leaving errors and injustices unaddressed.
Official reviews and independent assessments have repeatedly challenged the notion that digitisation alone would resolve systemic problems. For example, the National Audit Office’s 2018 report raised serious doubts about whether the new system was delivering on its promises, while the House of Commons public accounts committee has warned that implementation struggles with universal credit have damaged claimants’ wellbeing.
Frontline evidence gathered by advocacy organisations Disability Rights UK and Inclusion London aligns with these doubts.
The Trussell Trust found that 68% of working households relying on universal credit have gone without essentials in 2024. And 48% of claimants ran out of food without the means to buy more, underscoring the system’s inability to meet basic needs.
To address these concerns, experts and organisations have argued that the government must recognise that advanced analytics and automated assessments cannot replace human discernment – particularly when decisions affect basic needs like housing, heating and food security.
Unless policymakers recognise this, the UK risks entrenching a welfare system that punishes before it understands. Digital “efficiency” is coming at the cost of fairness and trust – and will exacerbate insecurity around food and the other basic needs that the welfare system is intended to address.
Iris Lim receives funding from the ESRC. She is a Postdoctoral Researcher and UK Lead for 'Digitalising food assistance: Political economy, governance, and food security effects across the Global North-South divide'. The Principal Investigator of the project is Susanne Jaspars. The research team is based at SOAS, University of London, at the Food Studies Centre within the Department of Anthropology and Sociology.
This article was originally published on The Conversation. Read the original article.
