RESEP hosts Conference on Quantitative Applications in Education

On the 19th and 20th of September this year, RESEP hosted the “Conference on Quantitative Applications in Education” at STIAS. The conference was attended by 85 participants from government, academia and civil society, with a keynote address by the Minister of Basic Education Ms Angie Motshekga

In her address, the Minister highlighted the importance of basing education policy on rigorous research. The Minister reiterated the important links between her Department and the research community:

“I want to acknowledge the valuable work that has been generated by the RESEP group recently. And not only be those directly part of RESEP, but also by many others here today, who are part of a growing network of people focused on understanding the challenges of the education sector and on offering solutions that can be implemented at a policy level. The two reports released earlier this year by RESEP have been tremendously influential in shaping thinking around what should be the priorities in education policy. These are the report on “Binding Constraints” in the sector and the report on “Laying Firm Foundations” through getting reading right.”

The conference heard from two international researchers based the Research Triangle Institute: Dr Luis Crouch and Dr Benjamin Piper. Further presentations were given by researchers from the Department of Basic Education, the Human Sciences Research Council (HSRC), the Joint Education Trust (JET), as well as those based at the University of Johannesburg, the University of Stellenbosch, the University of Cape Town, the University of KwaZulu-Natal, and the University of Witwatersrand. Links to the powerpoint presentations can be found below:

Day 1

 Day 2

 

Prominent media coverage of ‘Binding Constraints’ and ‘Laying Firm Foundations’ reports

The recent ReSEP reports on “Identifying Binding Constraints in Education for the PSPPD (Project to Support Pro-Poor Policy Development) and “Laying Firm Foundations: Getting Reading Right” for the Zenex Foundation have generated widespread media coverage over the last few weeks.

On radio, Gabrielle Wills and Martin Gustafsson were invited for a discussion on SAFM while Servaas van der Berg spoke on RSG and on some local radio stations. The reports were also discussed on TV when Janeli Kotzé and Nompumelelo Mohohlwane appeared on Freek Robinson’s programme, Regstreeks on KykNet.

These reports have featured in at least 12 newspaper articles, amongst others in Mail & Guardian, Die Burger, Beeld, The Mercury, Business Day Live and Times Live. Mail & Guardian even devoted an editorial to our findings. You can read some of the articles below:

The reports also caught the attention of Deputy President, Cyril Ramaphosa, who requested ReSEP to present to the Human Resource Development Council (HRDC), which he chairs.

On 10 June, Servaas van der Berg presented the highlighted key findings of both the reports to the HRDC in Johannesburg. His key message articulated the considerable reading deficits of children across the country and the relevance of a binding constraints framework for addressing issues of basic service delivery in education.

He also drew attention to concrete suggestions for addressing the reading crisis in the reports. The six key policy recommendations stressed at the end of his presentation included:

  • Emphasizing reading as a unifying goal for primary schooling
  • Continuing to test students regularly through reformulated ANAs as a measure of learning outcomes in primary schools
  • Using DBE workbooks to measure curriculum coverage
  • Teaching primary school teachers how to teach reading in African languages and in English 
  • Prioritizing the elimination of extreme class sizes in the Foundation Phase
  • Giving more attention to reading in African languages. 

It is satisfying to see our hard work coming to fruition. We hope that ReSEP’s research will have a lasting effect on policy debates and developments to the benefit of all South African children.

 

New ESRC/DFID Research Project: Outlier Township & Rural Schools in South Africa

At RESEP we are excited to announce our latest research initiative funded by the Economic and Social Research Council (ESRC) in conjunction with the Department for International Development (DFID). As part of the “Raising Learning Outcomes in Education Systems Research Programme” our proposal was one of eight international projects to be funded in 2015.

The focus of the research initiative is to better understand how some schools in challenging contexts manage to succeed against the considerable odds that they are up against. For this project we will be collaborating with our long-term colleagues Nick Taylor (JET), Ursula Hoadley (UCT) and Jaamia Galant (UCT). From the RESEP side Nic Spaull will be leading the project together with Servaas van der Berg and Gabrielle Wills.

From our earlier research and collaborations with qualitative researchers we have seen that there are considerable benefits when combining different approaches, and specifically the analysis of large scale datasets in conjunction with in-depth qualitative research. In this project we will be focussing on 60 primary schools in three provinces, namely, the Western Cape, KwaZulu-Natal, and Limpopo. The study takes the form of a matched-pair analysis where in each province we will select 10 outlier schools from rural areas and/or townships, and each exceptional school will be matched to a neighbouring school that is performing typically for that area.

By selecting schools that are serving the same community, that typically have the same government resources and district support, we hope to better understand which school factors lead to success in these contexts and whether they are common or different across the three provinces. We are especially interested in the School Leadership and Management (SLM) dynamics associated with high functioning schools. To that end we will also be developing a new SLM instrument to better capture the SLM practices in these schools. Almost all the surveys that have been developed to measure SLM come from places like the United Kingdom, Canada and the United States. It is perhaps unsurprising that these instruments aren’t well suited to a developing country like South Africa, or many other African countries.

We aim to answer the following questions:

1. What practices and behaviours of school leadership and management practices contribute to high student achievement of schools in challenging contexts (specifically townships and rural areas)?

2. How do we measure and codify these practices?

Our research project involves not only in-depth qualitative interviews and observation, but also gathering information on student achievement. We want to understand to what extent SLM practices are correlated with student outcomes in these schools and which of the various measures of SLM are the most closely associated with achievement.

We have already begun our initial research processes of gathering the team and charting the course ahead. We are currently in discussions with the national and provincial education systems and actively recruiting fieldworker researchers for our initial school visits later in the year.

If you are an honours or a masters student and are fluent in at least one African language (this is a non-negotiable  prerequisite) please email mschreve@sun.ac.za with your CV and a covering letter.

The uphill battle of establishing accurate performance trends for schools

Dr Martin Gustafsson is based at the Department of Basic Education and is a member of the Research on Socioeconomic Policy group at Stellenbosch University. This article first appeared in the City Press on the 26th of May and is also available here.

The technical report which informs the newspaper article, and which was discussed at a ReSEP workshop on 18 January 2016, is available here.

By Dr Martin Gustafsson


Schools dragging SA down

The South African Institute of Race Relations, in its February 2016 Fast Facts publication, concludes that “schools drag South Africa down” and that pupil performance is declining substantially.

The latter conclusion is based almost entirely on two numbers: the number of Grade 12 pupils obtaining a score of 70% or more in mathematics in 2008, which was 25 027, and the corresponding figure for 2015, 17 452. The two figures point to a decline of around 30%. At first glance, this appears to be a national disaster.

However, analysis done by myself and others in the basic education department, to be published shortly, paints a completely different picture, of large and encouraging improvements in mathematics in Grade 12. Many of the challenges facing the schooling system are accurately described by the institute’s report, but I disagree completely that the numbers should be pointing to a deterioration.

The problem is, first, that the institute looked selectively at the numbers, and ignored important figures, including a few appearing in their own report. This seems irresponsible.

Second, analysing pupil performance trends is exceedingly complex, not just in South Africa. The British education analyst John Jerrim has written extensively about how the data on mathematics trends have been spectacularly misinterpreted in his country. In South Africa, the complexities are particularly daunting in the case of Grade 12 mathematics.

So what did the institute’s analysts get wrong? They failed to point out that all of the 30% decline they refer to happened between 2008 and 2009. From 2009 to 2015, the trend, using values from all years (as one should), is a weakly positive 2% overall. The number of passes at the 70% level in 2008 was exceptionally high relative to all other years, something which should make any analyst suspicious.

The institute’s analysts also fail to point out that the overall increase in the number of physical science passes at the 70% level, over the entire 2008 to 2015 period, was a whopping 85%. In contrast to mathematics, what appears suspicious here is at least one exceptionally low value at the start of the period, in 2009. The question is why two such closely related subjects would move in completely opposing directions.

Another suspicious trend discussed in the analysis to be released by the basic education department is that the percentage of white and Indian pupils achieving high marks in mathematics has declined markedly over the 2008 to 2015 period. There appears to be no plausible explanation for this trend among these two relatively advantaged groups.

We zoomed into a sample of particularly stable and well-performing schools, with about 4000 mathematics candidates each year, to find explanations to the apparent anomalies. What emerged clearly is that variations across years in the difficulty of obtaining certain marks, for instance 70% in mathematics, explain most of the anomalies.

Mostly these variations are small, but for certain years they are large. Pupils who obtained 69% in the years 2012 to 2015, when levels of difficulty appeared particularly stable, would have obtained 68% in 2011, a marginally more difficult year, and at least 72% in the years 2008 to 2010. 2008 was a particularly easy year for obtaining high marks.

Changing the criteria for our sample did not change the picture substantially. Over the years, the mathematics examination became more difficult, while for physical science the opposite was true.

So is the problem then poor standardisation in the examinations system? Yes and no. There appears to be scope for improving the comparability of marks across years and this is receiving the attention of the basic education department and the council for quality assurance in general and further education and training, Umalusi.

At the same time, it is technically impossible to achieve anything approaching perfectly comparable marks, at all mark levels, in an examination system such as ours, or in similar systems in other countries. We need to learn to live with some variation over the years and rely on other systems, such as the international testing programmes, for more rigorous assessment of trends.

When we recalibrated results for all pupils over the 2008 to 2015 period, using what we found to be equivalent scores, we found that the number of pupils achieving a 70% level of performance in mathematics increased by 27% overall.

For black African pupils the increase was 61%. Physical science improvements, on the other hand, were found to be smaller than what published statistics would suggest, but were still encouraging. By far the largest improvements were in historically disadvantaged schools and top mathematics performers are spread across more schools in 2015 than they were in 2008.

We do not dispute that the under-performance of schools is a key factor holding the country’s development back. This is made clear in the National Development Plan. However, where we do disagree strongly with the institute for race relations is the direction the schooling system has been taking in recent years.

If the movement has been in the right direction and improvements as large as one might realistically expect, then one could hardly hope for more.

The evidence suggests the quality of school education is improving, that the improvements have been substantial and encouraging, and that they are helping to overcome historical race-based inequalities.

But trends seen in a few other countries, such as Brazil, suggest we should be aiming for an even steeper improvement.

This is what ongoing changes to our interventions, of which there are many, should aim to achieve. We also need a more rigorous national debate, involving a wider range of stakeholders, about the actual performance trends of schools.

ReSEP Launches Two Reports on Education

 

On the 24th of May the Research on Socioeconomic Policy (ReSEP) Group launched two reports which were the culmination of two years of research on South African education. The first report was for the Programme to Support Pro-Poor Policy Development (PSPPD), a joint initiative between the South African Presidency and the European Union. That report is titled “Identifying Binding Constraints in South African Education.” The second and related report was funded by the Zenex Foundation and focussed specifically on Grades 1-3. That report is titled “Laying Firm Foundations: Getting Reading Right.”

Both of the these reports highlight the centrality of ensuring that all children learn to read for meaning by the end of Grade 3. At the moment in South Africa about 60% of children cannot read at even a basic level at the end of Grade 4. These children never fully engage with the curriculum and fall further and further behind the curriculum even as they are promoted into higher grades.

The research has culminated in 10 journal articles, 12 policy briefs, 1 monograph, 2 reports and 2 policy-engagement workshops. The research covered topics ranging from excessive class sizes and learning deficits to the salaries and qualifications of Early Childhood Practitioners. It included quantitative analyses, a literature review and a special issue of the South African Journal of Childhood Education. The researchers have created a “Roadmap for Action” with detailed suggestions and recommendations for how the Department of Basic Education can ensure that all children learn to read.

 

The full reports and policy briefs can be found below:

Projects:

  1. Laying Firm Foundations: Getting Reading Right
  2. Identifying Binding Constraints in Education
  3. Teaching Reading (and Writing) in the Foundation Phase: A Concept Note

Policy Briefs:

  1. Limited support for the foundation phase:A misallocation of district resources
  2. Increasing the supply of teacher graduates
  3. Education datasets in South Africa
  4. Rethinking pre-grade R
  5. Is school based assessment in matric achieving its potential?
  6. Improving the calibre of school leadership in South Africa
  7. The DBE’s workbooks as a curriculum tool
  8. Adding randomised control trials (RCTs) to the education research toolkit
  9. What the ANAs tell us about socioeconomic learning gaps in South Africa
  10. Learning to read and reading to learn
  11. Excessive class sizes in the Foundation Phase
  12. Building an evidence base for inclusive education in South Africa: Focusing on learners with disabilities