Like what you've read?

On Line Opinion is the only Australian site where you get all sides of the story. We don't
charge, but we need your support. Here�s how you can help.

  • Advertise

    We have a monthly audience of 70,000 and advertising packages from $200 a month.

  • Volunteer

    We always need commissioning editors and sub-editors.

  • Contribute

    Got something to say? Submit an essay.


 The National Forum   Donate   Your Account   On Line Opinion   Forum   Blogs   Polling   About   
On Line Opinion logo ON LINE OPINION - Australia's e-journal of social and political debate

Subscribe!
Subscribe





On Line Opinion is a not-for-profit publication and relies on the generosity of its sponsors, editors and contributors. If you would like to help, contact us.
___________

Syndicate
RSS/XML


RSS 2.0

Let’s get real about evidence based policy

By Ross Farrelly - posted Tuesday, 21 October 2008


One of Kevin Rudd’s favourite mantras is that he wishes to develop “evidence based” policy. That is, he backs policies which have enough supporting evidence to suggest that they will produce their desired effect. However, it is well known in scientific circles that there is one gold standard technique for discovering such a causal relationship between cause and effect - the randomised trial.

A randomised trial starts with a hypothesis - a statement of fact which is to be tested in the trial. Stating a hypothesis forces policy makers to move from a vague statements of intent to a specific measurable outcome they wish to achieve. The hypothesis is then tested by randomly selecting two groups of people. One group receives the treatment and one group does not.

The power of the randomised trial lies in the fact that the two groups are as alike as possible in every respect except for whether or not they receive the treatment. If after the treatment has been applied, there is found to be a significant difference between the two groups, that difference can be attributed to the treatment and to the treatment alone. The researcher can conclude that the treatment caused the difference and therein lies the power of the randomised trial.

Advertisement

If policy makers want to be able to claim that their policies will work, they should subject them to randomised trials beforehand. Randomised trials present the policy maker who genuinely wants to make a difference with a powerful and irrefutable tool to put his theories to the test and to draw fact based conclusions from them.

Apart from trivial and self evident cases, the randomised trial is the one and only means of establishing a cause-and-effect relationship between one phenomenon and another and should be utilised by those who are serious about evidence based policy.

There is a growing recognition internationally that randomised trials should play an important part is assessing the efficacy of social policy. One of the most outstanding examples of randomised trails in social reform is the Progressa (later known as Oportunidades) Program in Mexico.

The aim of the program was to “close the gap” between rich and poor in Mexico in terms of nutrition and education. Funding was made available to poor rural families for education and improved nutrition. However the funding was conditional on attendance at both school and a government funded infant health clinic. The results have been encouraging and the program has been expanded into urban areas and extended to target youth up to the age of 22.

Randomised trials have only been used only twice to assess social policy in Australia.

Between 1999 and 2001 the Department of Family and Community Services conducted two randomised trials on the Job Network examining the effect of interviews and follow up contact from professional staff on workforce participation by the long-term unemployed. They found that the intervention led to a reduction in the number of hours worked but an increase in the number of hours spent in studying or training.

Advertisement

In 2002 the effectiveness of the New south Wales drug court in reducing recidivism was tested in a randomised trial. The trial showed that not only did the Drug Court reduce recidivism, it was also more cost effective when measured by cost per offence averted.

Kevin Rudd speaks often about his preference for “evidence based policy”. The government’s responses to the 2020 summit are to be built on “a strong evidence base”. A raft of new policies is being introduced by the recently elected government and many of these policies are candidates for testing by randomised trials. If Rudd is serious about evidence based policy he should insist these initiatives are subjected to scrutiny by randomised trial.

In the financial year 2005-2006 the Federal government spent about $90 billion on programs which purported to benefit Australians one way or another. None of these programs were tested by randomised trial.

There are a number of factors which make randomised trials unattractive to politicians. Making a real difference is hard and many randomised trails often show that the treatment made no difference. However, our elected representatives like to be seen as decisive, energetic and positive; especially when there is a crisis. They like to be seen as doing something about the problems and demonstrating leadership where others will not.

Randomised trials on the other hand take time to implement, require an investment of time and money, show no immediate results and are based on the premise that no one actually knows what will work. Nonetheless, if the government wants to be taken seriously when it speaks of evidence policy, it cannot ignore the randomised trial.

  1. Pages:
  2. 1
  3. All

First published in The Canberra Times on September 22, 2008. The longer version of this article can be found here.



Discuss in our Forums

See what other readers are saying about this article!

Click here to read & post comments.

8 posts so far.

Share this:
reddit this reddit thisbookmark with del.icio.us Del.icio.usdigg thisseed newsvineSeed NewsvineStumbleUpon StumbleUponsubmit to propellerkwoff it

About the Author

Ross Farrelly works for a statistical software company. His blog can be found at rossfarrelly.blogspot.com.

Other articles by this Author

All articles by Ross Farrelly

Creative Commons LicenseThis work is licensed under a Creative Commons License.

Article Tools
Comment 8 comments
Print Printable version
Subscribe Subscribe
Email Email a friend
Advertisement

About Us Search Discuss Feedback Legals Privacy