Government AI that lasts: what Robodebt and the Dutch benefits scandal teach

From 2016, Australia's income compliance programme matched annual tax office income data to welfare records and averaged it across fortnights to assert that people had been overpaid, shifting the burden to them to prove otherwise. The Royal Commission that reported in 2023 found the scheme was unlawful and described it as a crude and cruel mechanism ^[1].

The Dutch tax authority's systems flagged tens of thousands of families as potential fraudsters and demanded repayments of childcare benefits, often with no clear reason given. Amnesty International's investigation found the risk classification model used dual nationality as a risk factor ^[2]. The government resigned in January 2021.

• A person reviewing individual decisions before they affected someone.
• An explanation a citizen could understand and challenge.
• Testing for discrimination against protected groups.
• A fast route to correct mistakes once they were found.

The UK government's AI Playbook sets principles for public-sector AI, including meaningful human control at the right stages, understanding the limits of the technology and working openly ^[3]. The US GAO's accountability framework for federal agencies organises oversight around governance, data, performance and monitoring ^[4].

The deployments that last start with work that helps staff and citizens, such as answering questions, triaging correspondence and pre-filling forms, and keep decisions about entitlements and penalties with accountable officials.

Found this useful?

Share the key finding on X.