The nature and significance of rankings methodologies and their applications to economics departments : and the winner is...?
- Publication Type:
- Thesis
- Issue Date:
- 2011
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
There is nowadays a stronger interest in ranking studies of tertiary institutions. However,
significant policy changes in most countries’ higher education sectors in recent times have led
prospective students, academics, governments, employers and other stakeholders to scrutinise
rankings and their methodological approaches more closely. It is also true that studies of this
nature are not universally appreciated or accepted. Nevertheless, there is an acute awareness that
they are rapidly being integrated and utilised as barometers for assessing prestige and the
distribution of government research funds. Rankings systems will no doubt form a cornerstone of
policy for a significant period into the foreseeable future.
The two major objectives of the series of five papers submitted as part of this PhD are (i) to
develop a robust, credible and defensible ranking methodology, and (ii) to use this to rank
economics teaching departments and/or individual academic teaching economists, with Australian
and New Zealand universities being the initial case studies. Although each paper makes a number
of distinct and original contributions in its own right, it is clear that there are fundamental
integrating themes underpinning the set of journal articles. These chiefly relate to the two
methodological and ranking objectives noted above.
Paper 1, using journal articles listed in the EconLit database of the American Economic
Association (AEA), examined and ranked the research output of twenty-seven Australian teaching
economics departments on the basis of both citations-based and perceptions-based (peer-review)
quality-adjusted journal weights. In addition, this paper updated and corrected several
fundamental methodological flaws in the earlier seminal Towe and Wright (1995) study and, used
a range of alternative measures to check for the robustness of the final rankings results.
Paper 2 ranked the research performance of approximately 600 Australian university teaching
economists, ranked lecturer and above, on the basis of approximately 400 quality-adjusted
refereed EconLit listed journal articles. This study was the first of its kind to employ two different
journal rankings, one based on citations (Liebowitz and Palmer, 1984; Laband and Piette, 1994;
and Kalaitzidakis, Mamuneas and Stengos, 2003) and the other on perception-based (peer-review)
to rank individual academics (Mason, Steagall and Fabritius, 1997).
Paper 3 addressed a broader range of objectives. First, it presented a comprehensive review of the
rankings methodologies commonly used in the literature. Second, it was the first international
rankings paper that examined the research productivity of teaching economics departments in both
Australian and New Zealand Universities on a total and per capita basis and on the basis of a
broad range of quality-adjusted journals. Third, economics and econometrics departments and
663 individual academic economists from Australian and New Zealand universities were ranked
on the basis of quality-adjusted journals listed in the EconLit database. Fourth, ‘Star Performers’
were identified and ranked on the basis of the quality-adjusted journal publications.
Paper 4 focused on the research performance of Economics Professors in both Australian and
New Zealand universities with the objective of establishing whether or not they had made a
significant difference to the overall rankings of their respective teaching economics departments.
This paper concluded that Professors, particularly from top-ranked departments, had contributed
significantly to both the quantity and quality of their department’s publications.
Paper 5 evaluated the robustness of rankings of Australian and New Zealand economics teaching
departments using previous alternative rankings methodologies, and compared these with the
results obtained by Macri and Sinha (2006). This paper found that the various methodologies that
were benchmarked against the Macri and Sinha (2006) paper were quite robust when compared to
citations, but were not as robust when compared to perceptions (peer-review) of journal quality.
The integrating essay concludes by commenting on future areas of research in the area of
departmental rankings and its methodologies.
Please use this identifier to cite or link to this item: