Document Type : Articles
Authors
1 Institute for Science Citation (ISC), Research Department of Scientometric, Shiraz, Iran
2 institute for Science Citation (ISC), Research Department of university rankings, Shiraz, Iran
Abstract
The present study intended to examine ISC World University Ranking (ISC WUR) as an emerging ranking system. It draws on a descriptive-correlational method. First, the internal correlation held between the criteria of the ISC WUR was examined to clarify the representation of the total university score from each of its functional criteria in the ISC WUR system. Afterwards, the study assessed the extent to which the ISC WUR results diverge or converge at the general level and examined the similarities and differences between ISC WUR criteria and indicators results and those of Leiden, Nature Index, THE, and QS as complicated university rankings. The research sample included the universities ranked via the five global ranking systems in 2020. All ISC WUR’s indicators revealed significant and positive correlations with the total ranking scores of the universities. There was a significantly positive correlation between the ISC WUR and those of Leiden, Nature Index, THE, and QS.
Moreover, the indicators similarly present in ISC WUR and other ranking systems were significantly and positively correlated. In conclusion, the overall performance of each university in ISC WUR reflects all the functional criteria of that university, especially the criteria of research and international activities. Besides, although the ICS WUR and the other systems have differences and similarities in methodology, the results obtained from these systems are similar to a certain extent. Nevertheless, these similarities are not strong enough to claim that the ISC WUR presents the same results as other ranking systems.
Keywords
- Aguillo, I., Bar-Ilan, J., Levene, M. & Ortega, J. (2010). Comparing university rankings. Scientometrics, 85(1), 243-256. https://doi.org/10.1007/s11192-010-0190-z
- Alma, B., Coşkun, E. & Övendireli, E. (2016). University ranking systems and proposal of a theoretical framework for ranking of Turkish universities: A case of management departments. Procedia-Social and Behavioral Sciences, 235, 128-138. https://doi.org/10.1016/j.sbspro.2016.11.008
- Anowar, F., Helal, M. A., Afroj, S., Sultana, S., Sarker, F. & Mamun, K. A. (2015). A critical review on world university ranking in terms of top four ranking systems. In: Elleithy, K., Sobh, T. (Eds) New Trends in Networking, Computing, E-learning, Systems Sciences, and Engineering(pp. 559-566). Springer, Cham. https://doi.org/10.1007/978-3-319-06764-3_72
- Baty, P. (2011). THE Global Rankings: Change for the better. Retrieved from https://www.timeshighereducation.com/world-university-rankings/2011-12/world-ranking/analysis/the-global-rankings
- Baty, P. (2014). Times higher education world university rankings, 2004-2012. Ethics in science and environmental politics, 13(2), 125-130. http://dx.doi.org/10.3354/esep00145
- Billaut, J. C., Bouyssou, D. & Vincke, P. (2010). Should you believe in the Shanghai ranking? An MCDM view. Scientometrics, 84(1), 237-263. https://doi.org/10.1007/s11192-009-0115-x
- Bookstein, F., Seidler, H., Fieder, M. & Winckler, G. (2010). Too much noise in Times Higher Education rankings. Scientometrics, 85(1), 295-299. https://doi.org/10.1007/s11192-010-0189-5
- Bornmann, L. & Glänzel, W. (2018). Which differences can be expected when two universities in the Leiden Ranking are compared? Some benchmarks for institutional research evaluations. Scientometrics, 115(2), 1101-1105. https://doi.org/10.1007/s11192-018-2700-3
- Bornmann, L. & Haunschild, R. (2017). An empirical look at the nature index. Journal of the Association for Information Science and Technology, 68(3), 653-659. https://doi.org/10.1002/asi.23682
- Çakır, M. P., Acartürk, C., Alaşehir, O. & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813-848. https://doi.org/10.1007/s11192-015-1586-6
- Chen, K. H. & Liao, P. Y. (2012). A comparative study on world university rankings: a bibliometric survey. Scientometrics, 92(1), 89-103. https://doi.org/10.1007/s11192-012-0724-7
- Chen, W., Li, Q., & Hildebrand, A. J. (2019). Comparing University Rankings: Statistical Analysis of Four Global University Ranking Systems. Nebraska Conference for Undergraduate Women in Mathematics, Lincoln, NE, Jan 25-27.
- Chen, W., Zhu, Z. & Jia, T. (2021). The rank boost by inconsistency in university rankings: Evidence from 14 rankings of Chinese universities. Quantitative Science Studies, 2(1), 335-349. https://doi.org/10.1162/qss_a_00101
- Cheng, S. K. (2011). World university rankings: Take with a large pinch of salt. European Journal of Higher Education, 1(4), 369-381. https://doi.org/10.1080/21568235.2012.662837
- CWTS (2019). Leiden Ranking 2019, Methodology. Retrieved from https://www.leidenranking.com/Content/CWTS%20Leiden%20Ranking%202019.pdf
- Daraio, C., Bonaccorsi, A. & Simar, L. (2015). Rankings and university performance: A conditional multidimensional approach. European Journal of Operational Research, 244(3), 918-930. https://doi.org/10.1016/j.ejor.2015.02.005
- Dobrota, M. & Dobrota, M. (2016). ARWU ranking uncertainty and sensitivity: What if the award factor was excluded? Journal of the Association for Information Science and Technology, 67(2), 480-482. https://doi.org/10.1002/asi.23527
- Doğan, G. & Al, U. (2019). Is it possible to rank universities using fewer indicators? A study on five international university rankings. Aslib journal of information management, 71(1), 18-37. https://doi.org/10.1108/AJIM-05-2018-0118
- Florian, R. V. (2007). Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics, 72(1), 25-32. https://doi.org/10.1007/s11192-007-1712-1
- Frenken, K., Heimeriks, G. J. & Hoekman, J. (2017). What drives university research performance? An analysis using the CWTS Leiden Ranking data. Journal of informetrics, 11(3), 859-872. https://doi.org/10.1016/j.joi.2017.06.006
- Haunschild, R. & Bornmann, L. (2015). Discussion about the new Nature Index. Scientometrics, 102(2), 1829-1830. https://doi.org/10.1007/s11192-014-1505-2
- Holmes, R. (2006). THES university rankings: Are they really world class? Asian Journal of University Education (AJUE), 2(1). Retrieved from https://education.uitm.edu.my/ajue/wp-content/uploads/2019/04/Chapter-1-The-THES-University-Rankings-Are-They-Really-World-Class.pdf
- Huang, M. H. (2011). A comparison of three major academic rankings for world universities: From a research evaluation perspective. Journal of Library & Information Studies, 9(1), 1-25. http://doi.org/10.6182/jlis.2011.9(1).001
- Huang, M. H. (2012). Opening the black box of QS World University Rankings. Research Evaluation, 21(1), 71-78. https://doi.org/10.1093/reseval/rvr003
- Hussein, K., Buhari, S. M., Tsaramirsis, G. & Basheri, M. (2017). A new methodology for ranking international universities. Indian Journal of Science and Technology, 10(36). https://doi.org/10.17485/ijst/2017/v10i36/104094
- Ioannidis, J. P., Patsopoulos, N. A., Kavvoura, F. K., Tatsioni, A., Evangelou, E., Kouri, I., Contopoulos-Ioannidis, D. G. & Liberopoulos, G. (2007). International ranking systems for universities and institutions: A critical appraisal. BMC medicine, 5(1), 30. https://doi.org/10.1186/1741-7015-5-30
- ISC (2019a). The results of the ISC global ranking in 2019 have been published. News. Retrieved from https://b2n.ir/g29525 [in Persian].
- ISC (2019b). Our methodology. Retrieved from https://wur.isc.ac/Home/Methodology
- Ismail, E. M. (2010, December). Ranking of universities. In 2nd International Conference on Assessing Quality in Higher Education (pp. 6-8).
- Jajo, N. K. & Harrison, J. (2014). World university ranking systems: An alternative approach using partial least squares path modelling. Journal of Higher Education Policy and Management, 36(5), 471-482. https://doi.org/10.1080/1360080X.2014.936090
- Jöns, H. & Hoyler, M. (2013). Global geographies of higher education: The perspective of world university rankings. Geoforum, 46, 45-59. https://doi.org/10.1016/j.geoforum.2012.12.014
- Khosrowjerdi, M. & Kashani, Z. S. (2013). Asian top universities in six world university ranking systems. Webology, 10(2). Retrieved from https://www.webology.org/data-cms/articles/20200515052750pma114.pdf
- Liu, N. C. & Cheng, Y. (2005). The academic ranking of world universities. Higher education in Europe, 30(2), 127-136. https://doi.org/10.1080/03797720500260116
- Luke, G. & Walsh, A. (2010).World University Ranking. Retrieved from https://documents.manchester.ac.uk/display.aspx?DocID=8422
- Lukman, R., Krajnc, D. & Glavič, P. (2010). University ranking using research, educational and environmental indicators. Journal of cleaner production, 18(7), 619-628. https://doi.org/10.1016/j.jclepro.2009.09.015
- Moed, H. F. (2017). A critical comparative analysis of five world university rankings. Scientometrics, 110(2), 967-990. https://doi.org/10.1007/s11192-016-2212-y
- Moshfeghi, N. & Nadi, M. A. (2018). The comparative study of ranking system of islamic countries universities and national ranking of universities in iran using the most famous ranking systems in the world. Czech-Polish Historical & Pedagogical Journal, 10(2), 77-106. https://doi.org/10.5817/cphpj-2018-016
- Moshtagh, M. & Sotudeh, H. (2021). Correlation between universities’ Altmetric Attention Scores and their performance scores in Nature Index, Leiden, Times Higher Education and Quacquarelli Symonds ranking systems. Journal of Information Science, https://doi.org/10.1177/01655515211030868
- Nature Index (2018). Revisions to the Nature Index: A revised list of journals improves the balance of disciplines between the natural sciences. Retrieved from https://www.natureindex.com/news-blog/revisions-to-the-nature-index
- Olcay, G. A. & Bulu, M. (2017). Is measuring the knowledge creation of universities possible?: A review of university rankings. Technological Forecasting and Social Change, 123, 153-160. https://doi.org/10.1016/j.techfore.2016.03.029
- Rauhvargers, A. (2013). Global university rankings and their impact: Report II. Brussels: European University Association. Retrieved from https://eua.eu/downloads/publications/global%20university%20rankings%20and%20their%20impact%20-%20report%20ii.pdf
- Shehatta, I. & Mahmood, K. (2016). Correlation among top 100 universities in the major six global rankings: policy implications. Scientometrics, 109(2), 1231-1254. https://doi.org/10.1007/s11192-016-2065-4
- Soh, K. C. (2011). Don't read university rankings like reading football league tables: Taking a close look at the indicators. Higher Education Review, 44(1), 15-29. Retrieved from file:///C:/Users/Reza/Downloads/DontReadUniversityRankingsLikeReadingFootballLeaguesTableHER44-1.pdf
- Soh, K. C. (2015). Multicolinearity and indicator redundancy problem in world university rankings: An example using THEWUR 2013-2014 data. Higher Education Quarterly, 69(2), 158-174. http://dx.doi.org/10.1111/hequ.12058
- Soh, K. (2017). The seven deadly sins of world university ranking: A summary from several papers. Journal of Higher Education Policy and Management, 39(1), 104-115. https://doi.org/10.1080/1360080X.2016.1254431
- Sohail, S. S., Siddiqui, J., Shakil, M. T., Ubaid, S., Ahmed, J. & Alam, M. A. (2020). Sustainable Approach for University Ranking System. ICIDSSD. http://dx.doi.org/10.4108/eai.27-2-2020.2303115
- Tofallis, C. (2012). A different approach to university rankings. Higher Education, 63(1), 1-18. https://doi.org/10.1007/s10734-011-9417-z
- Van Raan, A. F. (2005b). Challenges in ranking of universities. In Invited paper for the First International Conference on World Class Universities, Shanghai Jaio Tong University, Shanghai(pp. 133-143).
- Van Raan, A. F. (2005a). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133-143. https://doi.org/10.1007/s11192-005-0008-6
- Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S. & van Raan, A. F. (2011). Towards a new crown indicator: Some theoretical considerations. Journal of informetrics, 5(1), 37-47. https://doi.org/10.1016/j.joi.2010.08.001
- Waltman, L., Calero‐Medina, C., Kosten, J., Noyons, E. C., Tijssen, R. J., van Eck, N. J., van Leeuwen, T. N., van Raan, A.F. J., Visser, M. S. & Wouters, P. (2012). The Leiden Ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American society for information science and technology, 63(12), 2419-2432. https://doi.org/10.1002/asi.22708
- Waltman, L. & Van Eck, N.J. (2015). Field-normalized citation impact indicators and the choice of an appropriate counting method. Journal of Informetrics, 9(4), 872-894. https://doi.org/10.1016/j.joi.2015.08.001