The problem with assessing census targets now is what we don’t know, writes David Williams
Comment: Politicians probably don’t like targets. Many of them have good reason not to.
Remember the “overly ambitious” 100,000 homes under KiwiBuild, dropped in 2019?
READ MORE: * Stats NZ scurries to rescue $317m census * Census turnout low despite $37m blowout
I also think of Matt Hancock, the British health secretary, who “met” a goal of reaching 100,000 daily Covid tests by counting 26,000 kits leaving an Amazon depot on a truck just before midnight.
In May 2020, Hancock – who later quit his job because of an affair and went on to become a minor TV personality by appearing on a reality show – hailed the “unprecedented expansion in British testing capability” as an incredible achievement.
The point is, if a target’s in front of a Cabinet minister it’s fair to assume they, and their officials, will do everything to try and achieve it.
Yesterday, Statistics Minister Deborah Russell, on her feet in Parliament, declared: “We have had an excellent result from the census so far.”
(Today is the last day to fill in the census.)
Russell isn’t being accused of doing a Matt Hancock by, say, counting census forms sitting on a truck leaving a depot. And, to be fair, the census targets were set well before she became minister.
However, it’s hard to see how her statement stands up to scrutiny.
The census estimates are problematic, I think, based on what we already know – but also because of what we don’t.
Let’s start with what we know.
Russell’s office issued a statement yesterday afternoon, with the glowing headline, “Census returns show significant lift”. It’s obviously an attempt to distinguish this year’s survey from the 2018 one.
That’s a pretty low bar considering the survey five years ago was widely panned as botched, or bungled, and led to the resignation of the chief statistician, Liz Macpherson.
Labour’s line earlier this year was the National-led government underfunded the previous census. That’s why it was doubling the number of census workers pounding the pavements, and increasing the number of paper forms being delivered.
This would, Russell said in a press statement in February, ensure the 2023 census was a success.
Helpfully, Stats NZ has a webpage entitled: “Measuring the success of the 2023 census.”
The first two key performance indicators are: 90 percent or greater for the national response rate for the total population; and the national response rate for people of Māori descent.
(Another KPI, for the first release of census data to occur by March 29 next year, will be missed.)
The 2018 census cost $128 million, while this one is estimated to cost $317 million. Considering the extra money being spent, you’d hope for “excellent” results.
Yesterday’s statement from Russell said: “As at 29 June, 4,560,486 people had returned their individual forms. That indicates an estimated individual return rate of 89 percent nationally, a significant increase from 82 percent in 2018.”
The estimated returns for Māori (74 percent), and Pacific peoples (79 percent) are also higher than in 2018 “but are still not as high as we would want”, Russell said.
Is this a case of our statistics boffins wanting to have it both ways?
For months, Stats NZ has provided individual form returns as “raw data”, with the caveat: “These figures include duplicates, overseas visitor returns, and should not be used to make any decisions.”
But when the minister uses them, they’re credible?
Deputy government statistician Simon Mason admitted yesterday the census tally is still considered “raw” but there’s a “strong level of confidence in how the country has responded”.
Stats NZ is still finalising the numbers.
A national collection response rate will be available next month, and the official response rate will be confirmed late next year by a post-enumeration survey.
Russell’s statement referred to the country being on track for an “excellent coverage rate” – which describes census form responses that are supplemented by other government data, known as administrative data.
What we don’t know
Let’s turn to what we don’t know about the census, and how this could derail the narrative of success.
If you track back to 2018, a big contributor to that census’s problem was more than one-in-seven people, or about 700,000, didn’t fully complete it.
(To plug those gaps, Stats NZ increased its use of administrative data.)
This year’s census has been difficult, what with North Island operations being suspended by Cyclone Gabrielle, and collections delayed in damaged areas.
But even before that, in January, the idea of “reduced data capture” – getting people to answer the bare minimum of questions – was flagged with the census board.
As pressure mounted to get response rates up, voucher and sports ticket giveaways were used as enticements.
It’s one thing to crow about a higher estimated individual return rate, but what if fewer people completed the whole thing?
We asked Russell, the minister, how many of the individual form returns she mentioned were only partially completed.
Her office sent us to Stats NZ for an answer – which initially only wanted to provide information on background. We were told if we referenced the minister in our question we should go back to her office for a response.
Political-bureaucratic pinball, if you will.
Eventually, a response from Mason was provided but no figures: “The 2023 census tallies include all forms returned by individuals, including complete and partially completed forms.”
Italicised notes in square brackets stated since the 2018 census, Stats NZ requires two or more of the following information fields about an individual on census forms for a response to be counted: name, date of birth, and an individual’s location.
Accountability and credibility on the line
There are few bigger responsibilities for a minister of statistics than a census. As Russell pointed out in Parliament, census data is used by government agencies, councils, iwi and hapū, community groups and businesses to plan for the future.
The survey’s data helps draw electoral boundaries, and is crucial for planning new schools and hospitals.
The public would get “good information”, Russell promised yesterday. For $317 million, many will argue the data produced should be excellent.
Something else the public deserves is messages without political spin.
In August 2019, then-minister James Shaw issued a statement headlined: “Review confirms most important population data is better in 2018 Census than previous surveys.”
This was the review, remember, that was so damning it led to Liz Macpherson’s resignation.
“While the review has found failings, these are being fixed,” Shaw assured. “The focus now needs to be on ensuring a successful census in 2023.”
In March, Russell, the current minister, told media she’d “absolutely” stake her job on hitting the 90 percent target, but back-pedalled by saying her resignation would be a step too far.
Her political opponents have been reminding her of that comment this week, as collections are closed and the numbers are crunched.
Yesterday, Russell, was asked in Parliament by Act’s Damien Smith who would be accountable for the census “failing” to meet targets. Russell rejected that word, saying the census had in fact “succeeded”.
Success looks a little different from the inside.
According to Stats NZ documents, responses from Māori and Pacific peoples were so low in April they were rated “level 3” incidents, the most serious.
“It has been noted in the data received so far that we aren’t missing Māori responses in traditional regions or areas – we are missing Māori responses EVERYWHERE,” one document said.
Field staff were told to use “minimum data capture”, the most basic of census form responses, as a “final visit contingency”. Basically, to get as many people to fill in their forms.
The cost of this approach is unclear. However, in early April, Tahu Kukutai (Ngāti Tīpā), a professor of demography at University of Waikato’s Te Ngira Institute for Population Research, she worried about the concerningly low indicative returns, especially for Māori ethnicity and Māori descent.
An independent review will be held in the coming months. Russell tried to hose down expectations yesterday, saying in her statement the international context is declining trends of participation.
The review is likely to find the combined census, bolstered by alternative data sources, produces high-quality population data.
The more interesting point will be how competent Stats NZ was at lifting response rates in Māori and Pasifika communities, and people with disabilities, and rebuilding trust and confidence after 2018’s failure.
Perhaps a good yardstick will be how many times the word “success” appears in the independent review’s final report.
