67) The Story of Academic Rankings: Nian Cai Liu - Shanghai Jiao Tong University

The Story of Academic Rankings
Nian Cai Liu
Professor and Dean of the Graduate School of Education, Shanghai Jiao Tong University, Shanghai, China.
E-mail

Building world-class universities has been the dream of generations of Chinese. At the 100th anniversary of Peking University in May 1998, the then president of China declared that the country should have several world-class universities—resulting in the 985 Project, which is especially for building world-class universities in China.

In 1998, Shanghai Jiao Tong University was selected by the Chinese government to be among the first group of nine universities in the 985 Project. At that time, many top Chinese universities drew up their strategic goals as world-class universities, and most of them set up a timetable. Shanghai Jiao Tong University was no exception. As a professor and vice-dean of the School of Chemistry and Chemical Engineering of the university, I became involved in the strategic planning process of building Shanghai Jiao Tong University into a world-class university.

During the process, I asked myself many questions. What is the definition of a world-class university? How many world-class universities should there be globally? What are the positions of top Chinese universities in the world higher education system? How can top Chinese universities reduce their gap with world-class universities? In order to answer these questions, I started to benchmark top Chinese universities with world-class universities and eventually to rank the world universities.

The Global Position of Chinese Universities
From 1999 to 2001, with Dr. Ying Cheng and two other colleagues, I worked on the project of benchmarking top Chinese universities with four groups of US universities, from the very top to ordinary research universities. The main conclusions include that top Chinese universities were estimated to be in the position of 200 to 300 in the world. The results of these comparisons and analyses were used in the strategic planning process of Shanghai Jiao Tong University. Eventually, a consultation report was written and provided to the Ministry of Education of China.

The publication of the report resulted in numerous positive comments, many of which involved the possibility of making a real ranking of world universities. During the time, many foreign friends, who visited us for other purposes, learned about our study and encouraged us to do world rankings. They reminded us that not only in China but also universities, governments, and other stakeholders in the rest of the world are interested in the ranking of world universities. Therefore, I decided to undertake this project, and with three colleagues spent another two years until the Academic Ranking of World Universities was completed in early 2003.

In June 2003, the ranking was published on our Web site (http://www.arwu.org). Although about 1,200 institutions from all over the world have actually been ranked, only the lists of the top 500 institutions have been published on the Web. Considering the significance of differences in the total scale, the ranking results include groups of 50 institutions in the range of 100 to 200 and groups of 100 institutions in the range of 200 to 500.

Ranking by Broad Subject Fields
Ever since its publication, the ranking has attracted attention from all over the world. Numerous requests have been received, asking us to provide a ranking of world universities by broad subject fields or by schools and colleges. We have tried to respond to these requests and the results were published on our Web site in February 2007. The five broad subject fields include the natural sciences and mathematics, engineering/technology and computer sciences, life and agriculture sciences, clinical medicine and pharmacy, and the social sciences.

Arts and humanities were not ranked because of the technical difficulties in finding internationally comparable indicators with reliable data. Psychology and other cross-disciplinary fields were not included in the ranking because of their interdisciplinary characteristics. Two new indicators were introduced: first, the percentage of articles published in the top 20 percent journals of each broad subject field and, second, the research expenditures (for engineering ranking). The list of top 100 universities in each broad subject field was published.

Ongoing Efforts to Diversify the Ranking
The Academic Ranking of World Universities sought to rank research universities in the world by their academic or research performance based on internationally comparable third-party data that everyone could check. The project was carried out for our academic interests, with potential impact on the strategic planning of Chinese universities.

Methodological problems involve the balance of research with teaching and service in ranking indicators and weights—inclusion of non-English publications, the selection of awards, and the experience of award winners. Technical problems exist in the definition and name given to institutions, data searching and cleanup of databases, and attribution of publications to institutions and broad subject fields. We have been working hard to study all the above-mentioned problems and to improve our ranking.

In addition to the broad subject field ranking, we are surveying the possibilities of providing more diversified ranking lists, particularly rankings based on different types of universities with different functions, disciplinary characteristics, history, size, and budget, as well as other topics. Furthermore, we have been doing theoretical research on ranking in general, seeking to contribute to the understanding of ranking. We have also been actively participating in international societies related to ranking such as the International Ranking Expert Group—International Observatory on Academic Ranking and Excellence (http://www.ireg-observatory.org).

Conclusion
Any ranking is controversial, and no ranking is absolutely objective. Nevertheless, university rankings have become popular in almost all major countries in the world. Whether universities and other stakeholders agree, ranking systems clearly are here to stay. The key issue then becomes how to improve ranking systems and how to use their results properly. Ranking methodologies should always be examined carefully before looking at any ranking lists, and ranking results should be used with caution.

Authors note: For additional information about the Shanghai higher education rankings, see http://www.arwu.org.

[Online] Available: http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/Number54/p2_Liu.htm