Annex 4.1. Measuring the quality of education

Details

Human capital

Primary and secondary education: PISA, TIMSS and PIRLS

The rankings for primary and secondary education shown in Chart 4.2a and 4.2b  in the main text are based on the combined average country scores in three international student assessments: the Progress in International Reading Literacy Study (PIRLS), the Trends in International Mathematics and Science Study (TIMSS) and the Programme for International Student Assessment (PISA).¹ Of the transition countries, only Belarus, Kosovo, Tajikistan, Turkmenistan and Uzbekistan have yet to participate in any of these three assessments.

Reading literacy in the fourth grade (PIRLS) is reasonably good in most transition countries. Russia shared the second place with Finland in 2011 (after Hong Kong). However, Azerbaijan and Georgia lagged significantly behind other transition countries.

Proficiency in mathematics in the eighth grade lags only slightly behind the average for advanced economies. Russia was the best performer of the 10 countries in the transition region that participated in the mathematics TIMSS in 2011. It was also ahead of all western European countries, but trailed South Korea, Singapore, Taiwan, Hong Kong and Japan. Countries in the SEMED region lagged behind other transition countries, although Tunisia was roughly on a par with FYR Macedonia and Georgia in 2011.

Transition countries’ average level of attainment in science in the eighth grade was roughly the same as that seen in the advanced economies in 2011. The leading countries were Slovenia and Russia, which performed better than the United States and all EU countries except Finland.

More countries from the transition region participated in the 2006 and 2009 rounds of the PISA programme, which pointed to an increase in the gap between those countries and advanced economies. The order of countries in terms of the achievements of 15-year-old school pupils was similar to that seen for fourth grade pupils. Estonia performed the best in reading, mathematics and science in 2009 – on a par with Switzerland in reading, Germany and Belgium in mathematics and sixth out of all participating countries in science.

Tertiary education

This annex presents more detailed information and data on some of the proxies for the quality of tertiary education presented in the main text.

League tables of top universities typically rank the world’s top 500 universities (out of a total of approximately 17,000 universities worldwide). The Academic Ranking of World Universities (ARWU) was first published in 2003 by the Shanghai Jiao Tong University. That was followed by the Times Higher Education (THE) World University Rankings, which were first produced in cooperation with Quacquarelli Symonds (QS) in 2004. In 2010 THE partnered Thomson Reuters in producing new rankings, while QS continued using the same methodology as before in partnership with US News & World Report.

A purely research-based ranking, the Performance Ranking of Scientific Papers for World Universities – also known as the Higher Education Evaluation and Accreditation Council of Taiwan Ranking (HEEACT Ranking) – was first published in 2007. Since 2012 the ranking has been known as the National Taiwan University Ranking (NTU Ranking).²

Table A.4.1.2 shows the number of top 500 universities by region according to the ARWU, QS and NTU rankings, respectively. The number of universities from transition and SEMED countries in the top 500 varies across the rankings; universities from the Czech Republic, Poland and Russia are included in the latest available versions of each ranking.

The 2013 ARWU includes 10 universities from seven transition countries (Croatia, Czech Republic, Hungary, Poland, Russia, Serbia and Slovenia) and one from Turkey. The 2013 ranking also included a university from one SEMED country – Egypt. The 2012 QS ranking contained 11 universities from four countries in the transition region (the Czech Republic, Kazakhstan, Poland and Russia) and two from Turkey, while the SEMED region was again represented by a single Egyptian university. Turkey and the SEMED countries are missing from the 2012 NTU ranking, which features seven universities from six countries in the transition region (Croatia, the Czech Republic, Hungary, Poland, Russia and Slovenia).

Table A.4.1.3 shows, for each year, the number of citable articles published by academic journals in the three previous years, the average h-index for the period 1996-2011 and the number of citations (excluding self-citations) per document. The number of citable documents has increased across the world, particularly in Turkey and the SEMED countries (especially Tunisia). In the transition region, Serbia and Bosnia and Herzegovina have also recorded large rises. This could be due to increases in the number of peer-reviewed journals over time; unfortunately, the aggregate data do not provide any information about this, nor about the quality of journals.

Advanced economies lead in terms of the average number of citations per document (excluding self-citations), with 9.8 between 1996 and 2011, followed by the transition region, Turkey and SEMED countries with around 4.5. Hungary and Estonia are the two countries that are closest to the advanced economies’ average, with 8.8 and 8.6 citations per document respectively.

The United States leads in terms of the average h-index (with 1,305), followed by the United Kingdom, Germany, France and Canada. Turkish scholars follow relatively closely behind, with an average of 193.0 (similar to the level in Chile), while academics in the transition and SEMED regions trail some way behind with averages of 100.6 and 91.0 respectively. Among the transition countries, the best performers are Russia, Poland and Hungary, while Egypt leads in the SEMED region.

Table A.4.1.4 shows the number of recipients of S&E doctorates at US universities for the period 1982-2011.³ More than half of all doctoral degrees in the United States are awarded in the field of S&E. This yardstick does not take into account recipients of doctorates in other advanced economies, nor the attractiveness of doctoral training in recipients’ home countries. However, it is still likely to be a reasonable proxy for the quality of undergraduate education in recipients’ countries of origin, given the high ranking positions of US universities (which account for half of the world’s top 100 universities and almost half of the world’s top 200 universities according to ARWU). Also, US universities typically offer paid graduate assistantships to the majority of graduate students accepted, making doctoral studies more attractive there than in other countries.

With the exception of students from the former Yugoslavia, S&E doctorate recipients at US universities from the transition region were rare prior to the collapse of communism. This was probably a reflection of restrictions on travel, rather than the quality of tertiary education in those countries. The following two decades saw large increases, due mainly to recipients from Bulgaria, Romania, Serbia, Croatia and Slovenia. Moreover, between 2002 and 2011 Bulgaria was among the top three sources of undergraduate students from transition countries. The number of doctorate recipients from Turkey also increased sharply, while the number of Jordanian recipients declined (albeit from a high level).

1. See Altinok et al. (2013) for details. PIRLS and TIMSS are conducted by the International Association for the Evaluation of Educational Achievement (IEA), while PISA is conducted by the OECD. PIRLS is designed to measure reading literacy in the fourth grade and is a successor to the IEA’s Reading Literacy Study, which ran from 1970 to 1991. Since 2001 PIRLS has been conducted every five years. TIMSS measures trends in achievement levels for mathematics and science in the fourth and eighth grades. It has been conducted at regular four-year intervals since 1995. PISA has measured the reading, mathematics and science achievements of 15-year-old school pupils at regular three-year intervals since 2000. [back]
2. See Rauhvargers (2011) for an excellent overview of the existing university rankings. [back]
3. We would like to thank Mark K. Fiegener from the National Science Foundation for sharing this breakdown of earned S&E doctorates for all countries of origin. [back]

• Prev
• Next