The nature and significance of rankings methodologies and their applications to economics departments : and the winner is...?

Publication Type:
Issue Date:
Full metadata record
There is nowadays a stronger interest in ranking studies of tertiary institutions. However, significant policy changes in most countries’ higher education sectors in recent times have led prospective students, academics, governments, employers and other stakeholders to scrutinise rankings and their methodological approaches more closely. It is also true that studies of this nature are not universally appreciated or accepted. Nevertheless, there is an acute awareness that they are rapidly being integrated and utilised as barometers for assessing prestige and the distribution of government research funds. Rankings systems will no doubt form a cornerstone of policy for a significant period into the foreseeable future. The two major objectives of the series of five papers submitted as part of this PhD are (i) to develop a robust, credible and defensible ranking methodology, and (ii) to use this to rank economics teaching departments and/or individual academic teaching economists, with Australian and New Zealand universities being the initial case studies. Although each paper makes a number of distinct and original contributions in its own right, it is clear that there are fundamental integrating themes underpinning the set of journal articles. These chiefly relate to the two methodological and ranking objectives noted above. Paper 1, using journal articles listed in the EconLit database of the American Economic Association (AEA), examined and ranked the research output of twenty-seven Australian teaching economics departments on the basis of both citations-based and perceptions-based (peer-review) quality-adjusted journal weights. In addition, this paper updated and corrected several fundamental methodological flaws in the earlier seminal Towe and Wright (1995) study and, used a range of alternative measures to check for the robustness of the final rankings results. Paper 2 ranked the research performance of approximately 600 Australian university teaching economists, ranked lecturer and above, on the basis of approximately 400 quality-adjusted refereed EconLit listed journal articles. This study was the first of its kind to employ two different journal rankings, one based on citations (Liebowitz and Palmer, 1984; Laband and Piette, 1994; and Kalaitzidakis, Mamuneas and Stengos, 2003) and the other on perception-based (peer-review) to rank individual academics (Mason, Steagall and Fabritius, 1997). Paper 3 addressed a broader range of objectives. First, it presented a comprehensive review of the rankings methodologies commonly used in the literature. Second, it was the first international rankings paper that examined the research productivity of teaching economics departments in both Australian and New Zealand Universities on a total and per capita basis and on the basis of a broad range of quality-adjusted journals. Third, economics and econometrics departments and 663 individual academic economists from Australian and New Zealand universities were ranked on the basis of quality-adjusted journals listed in the EconLit database. Fourth, ‘Star Performers’ were identified and ranked on the basis of the quality-adjusted journal publications. Paper 4 focused on the research performance of Economics Professors in both Australian and New Zealand universities with the objective of establishing whether or not they had made a significant difference to the overall rankings of their respective teaching economics departments. This paper concluded that Professors, particularly from top-ranked departments, had contributed significantly to both the quantity and quality of their department’s publications. Paper 5 evaluated the robustness of rankings of Australian and New Zealand economics teaching departments using previous alternative rankings methodologies, and compared these with the results obtained by Macri and Sinha (2006). This paper found that the various methodologies that were benchmarked against the Macri and Sinha (2006) paper were quite robust when compared to citations, but were not as robust when compared to perceptions (peer-review) of journal quality. The integrating essay concludes by commenting on future areas of research in the area of departmental rankings and its methodologies.
Please use this identifier to cite or link to this item: