On Monday, June 16, The Age published an article by Michelle Grattan, featured on the front page in bold letters, on the results of an Age/Nielsen poll, claiming “Rudd urged by 56 per cent to act on fuel”. Similar articles appeared in other Fairfax publications
First, the poll found 56 per cent of people were dissatisfied with Rudd’s performance on petrol and 78 per cent said the Government should intervene to reduce petrol prices - the “urge” and “anger” remarks are Grattan’s own political spin. About 65 per cent said that the government should cut the fuel tax, but did not say what government services should be cut to make up the shortfall. Nor were they given such obvious choices as “invade an oil-producing country” or “introduce petrol rationing”.
At least, The Age Analysis columnist (Tony Wright) had something sensible to contribute - realistically, neither the government nor opposition can do anything about rising oil prices. He adds that this should be obvious to even the dimmest respondent (and even more obvious to a respected journalist like Michelle Grattan!)
This poll is simply a piece of political grandstanding, akin to push polling. It brings the whole market research and polling industry into disrepute and makes Michelle Grattan look suspect for taking it seriously.
You could have asked whether the government should do more in respect of working women, Aborigines, reducing poverty, problem gamblers, and so on, or any other socially sensitive issue, and you would have got the same answer!
The Market Research Code of Professional Behaviour says that “Market research (which includes opinion polls) must always be carried out objectively and in accordance with established scientific principles”. Elsewhere, the Code says that the survey company must be consulted about published findings and must take action to correct misleading statements – meaning they cannot hide behind excuses such as “we asked these stupid questions and we got these stupid results”.
Elsewhere again, the Code says that survey companies must not be associated with attempts to influence people’s opinions or behaviours on any issue (push polling or influencing voters). I believe that, in this case, the spirit of that Professional Code has not been adhered to.
Everyone would be alarmed if survey companies started asking questions in polls such as whether rapists should be castrated; whether robbers should have their arms cut off, and so on. Well, we should all be alarmed when silly polls are carried out which detract from proper political and economic debate. Next time, Ms Grattan, I would politely suggest that you tell The Age management to think again.
Regarding any opinion poll or survey-based collection of data, there are two key issues to consider:
- validity - are you really measuring what you purport to be measuring? A simple example such as recall questions - “how often do you do (or buy) X?” - provide inflated answers, because people tend to collapse time and regularise their behaviour, the inflation factor being up to 200 per cent!
Another pertinent example - people confuse “digital TV” with wide-screen TVs, digital-ready TVs, etc. such that ACMA’s estimation of digital penetration of Australian homes is inflated by about 5 percentage points.
So, a first and fundamental requirement in any research is to ask valid questions, which have meaningful interpretation, not silly things like “Should the government cut the fuel tax?”;
- statistical reliability or significance - this mostly concerns how representative are people’s answers. Given a random sample of 1,000 people, statistical theory says that you can be 95 per cent confident that if you repeat that survey (or do a census) you will get the same findings within +/- 3 per cent. Mathematically, suppose 50 per cent of respondents in a random sample survey say X, then the calculation is simply 1/96*SQRT(50 x 50 / 1,000), or 50 per cent +/- 3 points. Of course, most sample surveys also suffer other sources of error, such as non-response, non-coverage (telephone surveys), and so on. Internet surveys are especially notorious, being mostly based on respondents volunteering to participate.
My definition of statistical significance is well known in industry and among former Monash students - “Statisitical significance simply means that if you ask bullshit questions of a large random sample of people, then can you be 95 per cent confident of getting the same bullshit answers within +/- 3 points!”
There is another problem with “statistical significance” - both academic and market researchers often confuse “significance” with “importance”, and so call everyone’s attention to what is a statistically significant finding, but of absolutely minimal importance for policy development. Goodness knows how much money has been wasted in the private and public sectors by such naivety, and how much damage has been done by poor policy development, based on unimportant survey findings.
This brief survey introduction leads directly to my accusation that pressuring politicians, with silly polls and sillier journalists equals “populist terrorism”.
We have a system of representative government which - it has been said - has its problems, but is superior to any other form of government. We elect leaders to govern, and we can vote them out if we do not like their policies, as recently happened with the ascendancy of the Rudd Labour Government and demise of the Howard Coalition government.
Discuss in our Forums
See what other readers are saying about this article!
Click here to read & post comments.
14 posts so far.