A freedom of information request has revealed that an AI system used by the UK government for assessing benefits cases is apparently getting it wrong by a “statistically significant” amount. The admission to journalists at the Guardian emerged after a fairness analysis into universal credit claimants in February 2024. It confirmed that the very tools intended to ensure equity and efficiency may in fact be discriminating against marginalised communities.
The Department for Work and Pensions (DWP) began exploring automated decision-making systems in the 2010s as part of its efforts to digitise welfare services. While early indications suggested that automated systems were being used to assess benefit entitlement and flag cases for investigation, it was not until 2021-22 that the department’s accounts revealed the use of these technologies to detect fraud in universal credit claims.
