Like what you've read?

On Line Opinion is the only Australian site where you get all sides of the story. We don't
charge, but we need your support. Here�s how you can help.

  • Advertise

    We have a monthly audience of 70,000 and advertising packages from $200 a month.

  • Volunteer

    We always need commissioning editors and sub-editors.

  • Contribute

    Got something to say? Submit an essay.


 The National Forum   Donate   Your Account   On Line Opinion   Forum   Blogs   Polling   About   
On Line Opinion logo ON LINE OPINION - Australia's e-journal of social and political debate

Subscribe!
Subscribe





On Line Opinion is a not-for-profit publication and relies on the generosity of its sponsors, editors and contributors. If you would like to help, contact us.
___________

Syndicate
RSS/XML


RSS 2.0

ERA: The dark matter in astronomically good research ranking?

By Aynsley Kellow - posted Tuesday, 1 March 2011


This suggests that astronomy in Canada, the UK and France is well above the world average - and nobody has even mentioned the US. As it happens - and as one might expect - the US seems to lead the world in astronomy.

The impact of astronomical research carried out by different countries has been compared by analysing the 1000 most-cited astronomy papers published 1991-8 (Sánchez, and Benn, 2004: 446). The USA dominates, receiving 61% of the citations to the top 1000 papers, considerably higher than its all-science share of citations, 31%. UK comes second (11% of citations), followed by Germany (5%), Canada (4%), Italy (3%), France (3%), Japan (2%), the Netherlands (2%), Switzerland (2%) and Australia (2%). Leaving aside for the moment the fact that Australian astronomy, so outstanding that it achieved a national average of 4.2 in the ERA exercise, ranked only a remote seventh equal in this assessment, and focusing on the deflated Canadian claims, how can Canada claim pre-eminence when they come only a distant fourth? In answering this question we learn much about how Australian astronomy might have come to be ranked so highly.

The explanation for Canada lies in the methodology employed. In the global analysis, where Canada ranked fourth, papers were assigned to the country of the first-named author. This would seem to be a reasonable approach, given that the natural sciences place great emphasis on the order of authorship. (In the social sciences and humanities, no emphasis is made on order, with alphabetical order commonly the norm, but order is important in the natural sciences, and often keenly fought over). In the Canadian analysis, however, a paper was 'considered to be Canadian if there was at least one author based at a Canadian institution on the paper. The countries of all authors on a paper were given full "credit" for a paper.' Crabtree (2009: 1). Canadian astronomers, it turns out, are junior authors on a lot of good papers, but are lead authors on far fewer. The reason for this is that Canadian astronomers are involved in a large number of international collaborations. As Crabtree (2009: 1-2) put it:

Advertisement

This result can also be interpreted as a result of Canadians, on average, being members of strong international collaborations. For example, there is a Canadian on the WMAP [Wilkinson Microwave Anisotropy Probe] team so all of the (very highly cited) WMAP papers get credited to Canada along with the countries of each of the other authors.

An unusually high degree of international collaboration is a feature of astronomical research, which probably assisted in producing a high ERA ranking, and this is widely known. The reasons for it are simply astronomical. As Elizabeth Capaldi put it recently, 'Astronomy requires the telescope be placed on the right place on the globe, so astronomers cannot always work in their own country and must collaborate with the country where the telescope is placed' (Capaldi, 2010: 71). This, of course, is the reason why the transmissions form the Apollo moon landings were relayed through Australia - and featured in the movie The Dish.

Australia has a particularly fortunate location in this regard: northern hemisphere researchers wishing to research the southern sky must find a southern hemisphere collaborator, so South Africa, Chile and Australia, with clear skies and political stability provide considerable opportunities. Since most of the Milky Way is observable only from the Southern Hemisphere, Australian astronomers end up as at least junior authors on most of the leading research, but they are lead authors on few of the leading papers.

This advantage is being demonstrated again with SKA. Most social science and humanities scholars probably think Skais a music genre originating in Jamaica in the late 1950s. But SKA stands for 'Square Kilometre Array', a giant radio telescope that will be built in either South Africa or Australia, where the view of the Milky Way is best and radio interference is least. Construction, costing €1.5 billion of EU funds will begin in 2016, with first observations in 2019 and full operation by 2024. SKA needed to be located in unpopulated areas where there was a guarantee of very low levels of anthropogenic radio interference. Sites in Argentina and China were soon eliminated.

An Australian SKA Pathfinder (ASKAP) costing $100m (consisting of 36 12m dishes) is already under construction at Boolardy in Western Australia (completion due in 2013), which would be the core of SKA. So $100m is being spent with the prospect of attracting $2 billion (at current exchange rates) – all on the basis that Australia has natural advantages.

Those natural advantages, combined with the ERA methodology, got astronomy to a national average ERA score of 4.2. Call me skeptical, but I remain unconvinced that this is an accurate assessment of research in a discipline that ranks seventh equal with a mere 2 percent of lead authorships in an assessment of the discipline itself. I can understand, when we are bidding for SKA, why we as a nation would wish to talk up the quality of our research performance in astronomy. But I remain unpersuaded (as a member of the IPSA Commission on Research with some knowledge of international realtivities), that ERA has captured accurately relative research quality in political science and astronomy.

Advertisement

Not only do the results not provide any reliable basis for allocating funds between competing disciplines they heavily skewed and clustered results in astronomy alone can hardly serve as a reliable indicator for allocating funding within that discipline.

The ARC has the opportunity to improve the reliability of the exercise, at least by attempting some standardization of scores that – in the first attempt – are clearly egregious. As a political scientist, I would prefer to see the pernicious effects of the FOR codes addressed, book publisher quality and citations addressed, impact assessment reinstated, and the actual quality of individual outputs interrogated more closely. The New Zealand system - submitting the four best outputs for peer assessment - seems more likely to produce reliable results. As a member of a smaller university, I would rather see the system adjust for size and numbers and not place any great emphasis on input measures like budgets.

Minster Carr (Australian 5 February 2011) stated that "The ERA national report reveals for the first time exactly how our country's research efforts compare to the rest of the world." Sorry, Minister. I cannot accept your faith in the exactitude of the exercise.The next round is already under way - perhaps too soon to allow policy learning - and such improvements are not likely to be adopted. But let's at least try to rid the system of the faults made only too apparent in the case of astronomy.

  1. Pages:
  2. 1
  3. 2
  4. Page 3
  5. All

A complete version of this paper, complete with references can be downloaded by clicking here.



Discuss in our Forums

See what other readers are saying about this article!

Click here to read & post comments.

Share this:
reddit this reddit thisbookmark with del.icio.us Del.icio.usdigg thisseed newsvineSeed NewsvineStumbleUpon StumbleUponsubmit to propellerkwoff it

About the Author

Aynsley Kellow is Professor of Government at the University of Tasmania, and Chair of the Politics and Business Research Committee of the International Political Science Association. He has published widely on environment and resource and is completing a project on the international organisation of the mining industry. He holds shares in mining stocks, both directly and through his superannuation.

Other articles by this Author

All articles by Aynsley Kellow

Creative Commons LicenseThis work is licensed under a Creative Commons License.

Photo of Aynsley Kellow
Article Tools
Comment Comments
Print Printable version
Subscribe Subscribe
Email Email a friend
Advertisement

About Us Search Discuss Feedback Legals Privacy