The concept of a census – a count of every member of a population – is almost as ancient as civilisation. The Babylonians recorded a population count on clay tiles from 4000BC. The Egyptians collected census data from 2500BC to inform projects such as the construction of the pyramids and the reassignment of land after the flooding of the Nile.
And, as every child who’s performed the nativity knows, the Romans required everyone to return to their place of birth for a count every five years to inform tax collection from the 6th century BC.
In England, the first attempt to comprehensively count and record was the Domesday Book of 1086, which took a detailed inventory of land and property. But the modern census did not come until 1801. Since then, a census has taken place every 10 years, save for the one that would have happened during the Second World War.
The census has always provided a rich snapshot of who we are, evolving to capture new types of data such as car ownership in 1971 and ethnicity from 1991. It directly informs planning for housing, schools and hospitals, and it leaves a profoundly important time capsule for future historians: there is much we know about how Britons lived 150 years ago as a result of the laborious manual work that went into it back then. And today, in an age where surveys are used to measure everything from people’s voting intention to employment outcomes, it would be no exaggeration to say that the census is the bedrock of our modern understanding of society.
Without its accurate population data – including on important characteristics such as sex, ethnicity and social class – researchers and pollsters cannot know whether they can draw reliable conclusions relating to the whole population from their sample, because they have no idea of the extent to which it is representative of wider society.
But Conservative ministers have been threatening to scrap the census since 2010 as part of their austerity drive. The 2021 census cost £900m, less than £1.50 per person per year over the period, but enough to make it a target. In June this year, the Office for National Statistics (ONS) launched a consultation on replacing the census with aggregating existing administrative data held by various government departments and public services.
The arguments put forward for scrapping the census are not just cost-based: the modern world is swimming in data, and the ONS says that piecing together information from administrative datasets could provide more timely information than the decennial census. But its proposals have triggered alarm in the social science community; 70 leading academics have signed a letter saying the 2031 census must go ahead.
They draw attention to important gaps in administrative data, which disproportionately exclude those hard-to-reach groups least likely to be claiming their benefits, or to be registered with a GP. There will also be significant inconsistencies in the quality of administrative data collected by service providers whose chief concern is not the accuracy of national statistics. In theory, these obstacles might be surmountable. But there are serious concerns among the signatories that the ONS is not up to the complexities of this task. It was supposed to produce an alternative census based on administrative data by 2020, to allow statisticians to compare it with data from the 2021 census. It never materialised.
The academics also point to a cautionary tale that highlights how the ONS can struggle even when it comes to the traditional census. In 2021, it got itself into a mess over sex and gender identity as a result of lobbying from campaigners with a minority worldview – that sex and gender identity can be conflated – that caused it to disregard the views of expert data scientists. It did not make clear to respondents that they should answer the “sex” question with their biological sex rather than the sex they identify with, and was so obdurate that a grassroots woman’s organisation had to take it to court even to get it to change the accompanying guidance document that, if followed, would have led to inaccurate sex data being recorded.
On gender identity, it adopted an impervious, “flawed” wording of the question originally formulated by campaigners rather than statisticians, which would lead to very skewed results even if relatively small numbers of people without good English or unfamiliar with gender theory misunderstood it.
That appears to be what happened; the academic Michael Biggs has shown that census respondents who spoke English least well were 5.5 times more likely to be recorded as trans than those who spoke it as their main language; one in 45 of such respondents were recorded as trans. By far the most plausible explanation is that people without good English did not understand the question. The ONS has admitted this, and other patterns are “unexpected”, but has doubled down without further investigation to say it has confidence in its data based on comparison with another survey that uses the same flawed question.
Unreliable data matters: it could direct funding for specialist trans services to areas with high migrant populations rather than those with the highest levels of need. In its latest consultation, the ONS baldly claims that administrative data provides “full coverage” on sex.
No such thing is true when service providers define and collect data on sex in a multitude of ways as a product of campaigners lobbying them to move from collecting data on biological sex, despite the fact it is such an important demographic variable. Simply put: if you don’t accurately measure sex, you cannot accurately measure structural sexism. The lesson from this sorry tale is that the ONS needs to focus more on the bread and butter of producing accurate census statistics and less on the glittering-but-elusive promise of constructing a census from administrative data. At a minimum, we would need to see a full census in 2031 run alongside an alternative exercise to see how accurate it could be.
To do anything else would be to risk corrupting what we can learn not just from the census, but from every other survey and poll that seeks to tell us something about the whole population.
• Sonia Sodha is an Observer columnist
Do you have an opinion on the issues raised in this article? If you would like to submit a letter of up to 250 words to be considered for publication, email it to us at observer.letters@observer.co.uk
