Can we trust official statistics?

The Australian Bureau of Statistics (ABS) published two items last month which suggested that its last five or six years' figures on poverty and inequality are probably flawed.

The first sign that something was amiss came in an article published in the April edition of Australian Economic Indicators. This is not exactly bedtime reading, and the article went largely unnoticed at first, but it made some rather worrying admissions.

For starters, it told us that since 1994/5, people interviewed about their incomes in ABS surveys have not been asked for documentary evidence like tax returns or pay slips. The result is that "the quality of responses declined". In other words, people's reported incomes got less accurate.

It also told us that "closer analysis" of the latest batch of income distribution statistics raises "a significant quality concern." Apparently, less well-off households have been under-reporting their incomes from welfare payments. This means that the gap between lower and higher income groups has probably been exaggerated.

These difficulties might seem a bit technical, maybe even trivial. But we began to see their full significance when, a week later, the ABS published a major report called Measuring Australia's Progress.

This report identified 15 indicators of 'progress', one of which concerned poverty and income distribution. Strangely, what the report said about the pattern of income distribution in Australia did not square with what previous studies have reported.

The report says that relative income inequality has remained roughly constant since 1995: "There has been little change in the income gap between households." But this is not what other researchers who have used ABS data have been saying. According to them, the relative income gap has been widening.

It turns out that one reason why the ABS findings are different from other people's is that it has recalculated the income statistics to try to remove some of the inaccuracies it has found in them. Its new version of the data shows that many people towards the bottom end of the income distribution are better off than had been thought.

Some of the most important inaccuracies in the data cannot, however, be corrected. For example, most of the households in the bottom 10 per cent of reported incomes told interviewers that they received less money each week than the income support system guarantees them. Some even reported zero or negative incomes. It is possible that a few of these claims may be accurate (small business people, for example, might make a loss one year and have to live off savings). But most clearly are not.

Faced with this problem, the ABS has taken drastic action. It has removed from its analysis all those people in the bottom ten per cent of reported incomes on the grounds that their answers are too unreliable to be used: "The extremely low incomes…recorded for some households in this group do not accurately reflect their living standards."

At the Centre for Independent Studies we have been warning for some time that the data on the lower income bands are inaccurate, and it is gratifying to find the ABS now agrees with us. Indeed, ABS warns that using these figures will result in a "misleading impression of the economic wellbeing of the most disadvantaged households."

So, official income data produced since 1995 are having to be recalculated, and the data on the bottom 10 per cent of incomes are so inaccurate that we are being advised to ignore them altogether. But where does this leave all those reports on poverty and income inequality that have been published in recent years using precisely these statistics?

Clearly, all this research will have to be re-examined. In particular, 'evidence' suggesting that the relative income gap is widening or that poverty is worsening now looks less convincing than ever.

The lesson to be learned from this sorry tale is that Australia's research community needs to be more vigilant and sceptical, even when using the government's own statistics. All research is fallible, and even the ABS can make mistakes.