Quantitative Education Research Conference 2018

Coverage of the conference

The two-day conference on Quantitative Education Research, held on 6 and 7 September 2018 in Stellenbosch, featured work presented by RESEP’s own Servaas van der Berg, as well as esteemed local and international guests including Paul Glewwe of the University of Minnesota, Abhijeet Singh of the Stockholm School of Economics and Justin Sandefur of the Center for Global Development, and others. This annually-held conference offers a valuable opportunity for researchers and stakeholders to share and discourse on recent empirical findings in quantitative educational research and policy relevant to the South African context. New voices in quantitative education research were also given a unique platform in the form of an afternoon session dedicated to presentations from researchers currently engaged in doctoral studies. Download the full programme here.

Day one highlights (6 September 2018):

Servaas van der Berg (Stellenbosch University): Flows through school systems in southern Africa: Patterns, trends and policies

Van der Berg showed that despite most countries in the region having near-universal enrollment rates in the primary school phase, repetition and dropout are very common. This results in years in school not necessarily translating into grade attainment. The trend generally is that repetition is a precursor to dropout, with repetition being particularly high in grades immediately preceding national examinations, such as the primary school leaving examinations that most countries in this region have at the end of Grade 7 (see table below). It appears that schools hold weak learners back in Grade 6 to maintain high pass rates in the “high stakes” exams; this “weeding” or “culling” of students is a near-universal phenomenon in countries with high stakes exams. 

Paul Glewwe (University of Minnesota): What explains Vietnam’s exceptional performance in education relative to other countries? Analysis of the PISA and Young Lives data

Vietnam outperformed both the UK and the USA in PISA 2012 and PISA 2015, despite being the poorest participating country. One slide shown by Glewwe indicated Vietnam’s clear outlier status (performing far above expected) when SES was plotted against PISA performance (see graph below). Glewwe finds very little by way of explanation despite applying econometric techniques that usually allow us to explain exceptional performance despite the socioeconomic disadvantage. The most likely explanation that emerges from his analysis is that the value added (e.g. per school day) is higher in Vietnam than the other countries participating in PISA, as well as some suggestive evidence that Vietnamese students may be more intrinsically motivated to perform academically. 

Edward Mosuwe (Gauteng Department of Education): The value of research to inform policy

Highlighted the importance of taking into account the contextual realities within which policies are to be implemented. 

Carol Nuga Deliwe & Stephen Taylor (Department of Basic Education (DBE)): EGRS II: From research to policy and EGRS II results

The Early Grade Reading Study (EGRS) II results indicated that of the three interventions instituted (parent involvement, printed lesson plans and on-site coaching), on-site coaching was the most effective in raising learners’ reading scores in the foundation phase; learners whose teachers had an on-site coach gained about 40% of a year of learning. These findings are to be presented to parliament for national implementation (the study was conducted in the North West). 

Martin Gustafsson (DBE): Better ways of using education spending ratios to advance equity in tough budgetary times

The age profile of the current teaching force implies that there will be a surplus in the budget for more than 10 years after these teachers retire since new teachers coming into the force will have lower salaries owing to their relative inexperience (see map below). Treasury should plan for this windfall and think carefully about how to spend it. Gustafsson pointed towards the likelihood of a “perfect storm” in our education system, whereby increases in the population combined with economic stagnation will result in even more decreases in per-pupil spending. 

Abhijeet Singh (Stockholm School of Economics): Evaluating reforms for system-level change in education: Evidence from multiple Indian States 

Governments are becoming increasingly aware of the low quality of schooling in their education systems – we take this for granted now, but stating something like this would have been controversial 10 years ago. He highlighted the importance of evaluating education reforms and presenting evidence of their success or failure. Encouraging findings have been evidenced from an intervention in India (Mindspark) that issued laptops to children. Rather than the technology itself, it was the fact that the technology allowed children to learn at a pace that matched their individual learning needs that brought the gains in test scores. Importantly, the gains were uniform across the test score distribution. This finding is particularly important in a developing country such as India where there is evidence of large disparities in the levels of academic preparedness within a single class. One graph shown by Singh indicated that the knowledge levels of learners in a single classroom could span fives years of curriculum, presenting a real challenge to teachers’ classroom management and pedagogical practices (see graph below). 

Adaiah Lilenstein (Stellenbosch University): Access to learning in Francophone Africa: Combining measures of schooling access and quality for comparable estimates across countries and over time

Quantitative education data on school enrollment and literacy rates for the region were combined to obtain a measure of the quality of schooling in eight Francophone African countries. Findings point towards extremely low literacy levels in all countries considered. Compared with low SES boys, low SES girls are more likely to be illiterate, denoting the intersection of poverty and gender in determining learning outcomes in these countries. 

Sahawal Alidou (University of Antwerp): Family size and schooling in Sub-Saharan Africa: Testing the quantity-quality trade-off

Existing literature suggests children with more siblings are likely to achieve worse learning outcomes, especially among poor households as parents have less money to spend on the education of each child. Using an econometric technique based on comparing the learning outcomes of twins, however, Alidou finds evidence of economies of scale in households with more children (which arise when children can share uniforms, transport costs, etc), and that children with more siblings actually achieve better learning outcomes than their counterparts (see table below). 

Day two highlights (7 September 2018):

Justin Sandefur (Centre for Global Development): Internationally comparable mathematics scores for fourteen African countries

Researchers and policymakers are often interested in making comparisons across countries and school systems. No low-income countries participate in large-scale international assessments, making it difficult to judge how they perform by international comparison. Sandefur links regional assessment data (e.g. SACMEQ) with international assessments (e.g. PIRLS) using overlapping countries such as South Africa and Botswana, or overlapping test items that were common to international and regional assessments. He finds that the majority of SACMEQ countries performed below the 5th percentile of the USA on TIMSS (see graph below); given the low variation in performance across Southern and Eastern African countries, he conjectured that the TIMSS items may be too difficult for learners in this region.  He also compares teacher performance on international tests and finds teachers in African countries perform at the level of OECD learners. 

Ursula Hoadley (UCT) & Nick Taylor (JET): Leadership and literacy: Exploring linkages in rural and township schools in South Africa

Four low-performing, zero-fee schools across Gauteng, KZN and Limpopo were selected and compared to the leadership and management practices of four high-performing matched schools. It was discovered that very few teachers and principals knew the correct grade during which learners should be introduced to different aspects of literacy, e.g. the main idea of a story, and there was also very little convergence on these ideas across schools. The general consensus among teachers and principals was that teaching reading is easy since it “comes naturally to children” (vs. something like maths, which teachers thought was harder to teach) (see table below).

Gabrielle Wills (RESEP): Can we meaningfully measure school management and leadership in South Africa

Wills’ presentation explored findings from a 2-year mixed methods project that aimed to develop a measurement instrument that captures the practices and behaviours of school leaders and managers in challenging contexts in South Africa. Better ‘leadership for literacy’ practices were not evidenced to be systematically associated with better literacy performance (noting that there were challenges in identifying exceptional schools).  However, consistent associations were identified between better human resources practices and English literacy scores measured both in terms of levels and learning gains. The study further highlighted how management practices may vary notably within school organisations; educators in the same school reported very different experiences and perceptions of their managers, indicating that measures of management competence will be strongly influenced by who provides this information (see graph below).

Nicky Roberts (UJ): Evaluating a province-wide early grade mathematics intervention: The cases of R-Maths

Roberts presented findings from an intervention that involved training subject advisers, with the aim to increase mathematics teachers’ subject knowledge. The study finds no difference in test score gains between intervention and control groups in urban areas and a very small positive effect of the intervention (although not significant) in rural areas. (see tables below). 

Rural:

Urban:

Other presentations from the Conference on Quantitative Education Research (2018)

• Reading on paper or reading digitally?  | Nangamso Mtsatse (UNISA)
• Gender equity or new complexities  | Unathi Beku (HSRC)
• Academic resilience in challenging school contexts in South Africa | Heleen Hofmeyr (SU)
• Early reading skills related to Grade 1 English Second Language literacy in rural South African Schools | Maxine Schaefer (UNISA)
• Reading panel | Nic Spaull (SU), Elizabeth Pretorius (UNISA), Nompumelelo Mohohlwane (DBE), and Cally Ardington (UCT)

In conclusion

RESEP would like to extend gratitude to all presenters and attendees for their valuable contributions to the success of this year’s conference on quantitative education research (2018). We look forward to our continued and future collaborations.