Revised results confirm large improvements in South Africa’s reading outcomes, albeit from a low base
1. Overview
While South Africa’s poor reading outcomes in the foundation phase are widely acknowledged, what is poorly understood is whether things are getting worse, getting better, or staying the same. A recent paper by Martin Gustafsson shows that there have been significant improvements in reading in South Africa in recent years. Gustafsson’s estimates suggest the speed of the improvements in reading has in fact been impressive compared to what the fastest-improving countries in recent history have achieved. While it is important for policy makers to take note of these trends as they point to promising positive changes in the basic education sector, it must be kept in mind that South Africa’s reading outcomes in the foundation phase remain poor even by regional comparison. The improvements noted by Gustafsson will have to be sustained in the long-term in order to realise the DBE’s vision of providing quality education for all.
2. Main Findings
2.1 The trend in South Africa’s PIRLS scores was inconsistent with trends in other testing programmes.
According to official reports, South Africa showed no improvements in PIRLS 1, which tests Grade 4 reading competencies, over the period 2011 to 2016. This trend was inconsistent with trends in other international testing programmes, which showed significant improvements over the same period. As shown in Figure 1, South Africa’s TIMSS2 results point to large gains in Grade 9 mathematics and science performance over the period 2002 to 20153. Similarly, South Africa’s performance in SACMEQ4 shows large gains in Grade 6 reading and mathematics over the period 2007 to 2013. Earlier work done by Gustafsson for the UNESCO Institute of Statistics (UIS)5 indicates that these gains are as large as one could hope for in a developing country. These improvements constituted evidence of systematic improvements in the school system, both at the primary and high school level. It was therefore curious that South Africa showed no improvement in Grade 4 reading results during this period.
FIGURE 1: South Africa’s performance in TIMSS, 2002–2015
2.2 Revised scores imply that South Africa’s performance in PIRLS has improved steadily since 2006.
Gustafsson shows that South Africa’s official results in the 2011 round of PIRLS had been calculated incorrectly due to an error in converting the 2011 results from one scale to another, a special process that had to be applied only to South Africa. This error resulted in an inflated score of 323 points in the 2011 round of the assessment. When correcting for the error in the conversion of results, Gustafsson calculates a score of 295 for South Africa in the 2011 PIRLS assessment. Comparing South Africa’s 2016 PIRLS score of 320 with the new score for 2011 points to a 25-point increase over this period, which constitutes the third-largest increase in PIRLS scores (after Morocco and Oman) over this period. Figure 1 shows that when taking into South Africa’s score of 253 in the 2006 round of PIRLS, the revised trend indicates that South Africa’s reading results have improved steadily over the period 2006 to 2016. The figure also shows that the revised trend is continuous, without an implausibly large gain between 2006 and 2011, followed by no gain between 2011 and 2016. Moreover, a continuous gain in PIRLS would be in line with the continuous improvements seen in TIMSS and ACMEQ during this period. The IEA has acknowledged that South Africa’s initial 2011 score was calculated incorrectly.
FIGURE 2: Initial vs. revised trend for South Africa’s PIRLS results, 2006–2016
2.3 South Africa’s reading outcomes remain a cause for concern.
Although the findings from Gustafsson’s analysis point to movement in the right direction, these improvements were off a low base, and poor reading outcomes in the foundation phase remain a great cause for concern. A host of research6 has shown that the country’s poor reading outcomes constitute a binding constraint to realising the DBE’s goal of providing quality education for all, since children who have not ‘learnt to read’ cannot ‘read to learn’. Despite the improvements noted by Gustafsson, it is still the case that 78% of South African learners cannot read for meaning in any language by the end of Grade 4 – the low international benchmark in terms of PIRLS scores. Further analysis by Gustafsson shows that this means as many as 18% of South African learners got a score of only 1 or 2 out of 17 in questions that required a written response (as opposed to multiple choice questions, which constituted roughly half of the questions in the assessment). These poor reading outcomes in the early grades leads to a situation where the majority of learners cannot access the curriculum after Grade 4, which assumes that children have learnt to read by the end of Grade 3. The improvements noted by Gustafsson will therefore have to be sustained in the long-term for South Africa’s reading results to catch up to countries at similar levels of development.
3. Policy suggestions
3.1 Capacity for detailed and policy-relevant analysis of microdata from international assessment should be developed. It is clear from Gustafsson’s analysis that South Africa’s PIRLS assessment statistics were not interrogated upon their release, even though the country’s results pointed to an inexplicably flat trend in performance between the 2011 and 2016 rounds of the assessment. Moreover, this trend was inconsistent with trends in other international assessments that spanned the same period. The fact that these strange results were not interrogated upon their release points to a sense that the ‘black box’ of international assessment data is difficult to understand among South African education policy makers and analysts alike. Developing capacity for the analysis of international assessment data is therefore important if we are to use this data to obtain an accurate reflection of the state of basic education in South Africa.
3.2 It is important to consider trends in learning outcomes over time.
While using data from only one point in time provides a good indication of levels of performance, Gustafsson’s work points out the importance of considering trends in performance in order to understand the dynamics that occur within education systems. Understanding these dynamics is of enormous importance for education policy, as they lie at the heart of a country’s educational development.
3.3 Future research should aim to understand why the noted improvements have occurred.
Understanding the causes of South Africa’s improvements in PIRLS, TIMSS and SACMEQ is of paramount importance for sustaining these improvements in the long-term. Research aimed at identifying these causes is required for understanding whether current education policy and practice is protecting the factors that are responsible for these improvements, and whether these policies and practices are sufficient for sustaining the upward trajectory in future.
1 Progress in International Reading Literacy Study
2 Trends in International Mathematics and Science Study
3 Van der Berg, S. and Gustafsson, M. (2019). Educational outcomes in post-apartheid South Africa: Signs of progress despite great inequality, in Spaull,
N. and Jansen, J. (eds) South African Schooling: The Enigma of Inequality. Springer.
4 Southern and Eastern African Consortium for the Monitoring of Education Quality
5 UNESCO (2019). Projecting attainment of SDG 4.1.1. Available at: https://gaml.uis.unesco.org/wp-content/uploads/sites/2/2019/05/GAML6-REF-15-Projecting-attainment-of-SDG-4.1.1.pdf
6 Van der Berg et al. (2016) Binding Constraints in Education. Synthesis Report to the PPSD.
DOWNLOAD PDF
