New ESRC/DFID Research Project: Outlier Township & Rural Schools in South Africa

At RESEP we are excited to announce our latest research initiative funded by the Economic and Social Research Council (ESRC) in conjunction with the Department for International Development (DFID). As part of the “Raising Learning Outcomes in Education Systems Research Programme” our proposal was one of eight international projects to be funded in 2015.

The focus of the research initiative is to better understand how some schools in challenging contexts manage to succeed against the considerable odds that they are up against. For this project we will be collaborating with our long-term colleagues Nick Taylor (JET), Ursula Hoadley (UCT) and Jaamia Galant (UCT). From the RESEP side Nic Spaull will be leading the project together with Servaas van der Berg and Gabrielle Wills.

From our earlier research and collaborations with qualitative researchers we have seen that there are considerable benefits when combining different approaches, and specifically the analysis of large scale datasets in conjunction with in-depth qualitative research. In this project we will be focussing on 60 primary schools in three provinces, namely, the Western Cape, KwaZulu-Natal, and Limpopo. The study takes the form of a matched-pair analysis where in each province we will select 10 outlier schools from rural areas and/or townships, and each exceptional school will be matched to a neighbouring school that is performing typically for that area.

By selecting schools that are serving the same community, that typically have the same government resources and district support, we hope to better understand which school factors lead to success in these contexts and whether they are common or different across the three provinces. We are especially interested in the School Leadership and Management (SLM) dynamics associated with high functioning schools. To that end we will also be developing a new SLM instrument to better capture the SLM practices in these schools. Almost all the surveys that have been developed to measure SLM come from places like the United Kingdom, Canada and the United States. It is perhaps unsurprising that these instruments aren’t well suited to a developing country like South Africa, or many other African countries.

We aim to answer the following questions:

1. What practices and behaviours of school leadership and management practices contribute to high student achievement of schools in challenging contexts (specifically townships and rural areas)?

2. How do we measure and codify these practices?

Our research project involves not only in-depth qualitative interviews and observation, but also gathering information on student achievement. We want to understand to what extent SLM practices are correlated with student outcomes in these schools and which of the various measures of SLM are the most closely associated with achievement.

We have already begun our initial research processes of gathering the team and charting the course ahead. We are currently in discussions with the national and provincial education systems and actively recruiting fieldworker researchers for our initial school visits later in the year.

If you are an honours or a masters student and are fluent in at least one African language (this is a non-negotiable  prerequisite) please email mschreve@sun.ac.za with your CV and a covering letter.

The uphill battle of establishing accurate performance trends for schools

Dr Martin Gustafsson is based at the Department of Basic Education and is a member of the Research on Socioeconomic Policy group at Stellenbosch University. This article first appeared in the City Press on the 26th of May and is also available here.

The technical report which informs the newspaper article, and which was discussed at a ReSEP workshop on 18 January 2016, is available here.

By Dr Martin Gustafsson


Schools dragging SA down

The South African Institute of Race Relations, in its February 2016 Fast Facts publication, concludes that “schools drag South Africa down” and that pupil performance is declining substantially.

The latter conclusion is based almost entirely on two numbers: the number of Grade 12 pupils obtaining a score of 70% or more in mathematics in 2008, which was 25 027, and the corresponding figure for 2015, 17 452. The two figures point to a decline of around 30%. At first glance, this appears to be a national disaster.

However, analysis done by myself and others in the basic education department, to be published shortly, paints a completely different picture, of large and encouraging improvements in mathematics in Grade 12. Many of the challenges facing the schooling system are accurately described by the institute’s report, but I disagree completely that the numbers should be pointing to a deterioration.

The problem is, first, that the institute looked selectively at the numbers, and ignored important figures, including a few appearing in their own report. This seems irresponsible.

Second, analysing pupil performance trends is exceedingly complex, not just in South Africa. The British education analyst John Jerrim has written extensively about how the data on mathematics trends have been spectacularly misinterpreted in his country. In South Africa, the complexities are particularly daunting in the case of Grade 12 mathematics.

So what did the institute’s analysts get wrong? They failed to point out that all of the 30% decline they refer to happened between 2008 and 2009. From 2009 to 2015, the trend, using values from all years (as one should), is a weakly positive 2% overall. The number of passes at the 70% level in 2008 was exceptionally high relative to all other years, something which should make any analyst suspicious.

The institute’s analysts also fail to point out that the overall increase in the number of physical science passes at the 70% level, over the entire 2008 to 2015 period, was a whopping 85%. In contrast to mathematics, what appears suspicious here is at least one exceptionally low value at the start of the period, in 2009. The question is why two such closely related subjects would move in completely opposing directions.

Another suspicious trend discussed in the analysis to be released by the basic education department is that the percentage of white and Indian pupils achieving high marks in mathematics has declined markedly over the 2008 to 2015 period. There appears to be no plausible explanation for this trend among these two relatively advantaged groups.

We zoomed into a sample of particularly stable and well-performing schools, with about 4000 mathematics candidates each year, to find explanations to the apparent anomalies. What emerged clearly is that variations across years in the difficulty of obtaining certain marks, for instance 70% in mathematics, explain most of the anomalies.

Mostly these variations are small, but for certain years they are large. Pupils who obtained 69% in the years 2012 to 2015, when levels of difficulty appeared particularly stable, would have obtained 68% in 2011, a marginally more difficult year, and at least 72% in the years 2008 to 2010. 2008 was a particularly easy year for obtaining high marks.

Changing the criteria for our sample did not change the picture substantially. Over the years, the mathematics examination became more difficult, while for physical science the opposite was true.

So is the problem then poor standardisation in the examinations system? Yes and no. There appears to be scope for improving the comparability of marks across years and this is receiving the attention of the basic education department and the council for quality assurance in general and further education and training, Umalusi.

At the same time, it is technically impossible to achieve anything approaching perfectly comparable marks, at all mark levels, in an examination system such as ours, or in similar systems in other countries. We need to learn to live with some variation over the years and rely on other systems, such as the international testing programmes, for more rigorous assessment of trends.

When we recalibrated results for all pupils over the 2008 to 2015 period, using what we found to be equivalent scores, we found that the number of pupils achieving a 70% level of performance in mathematics increased by 27% overall.

For black African pupils the increase was 61%. Physical science improvements, on the other hand, were found to be smaller than what published statistics would suggest, but were still encouraging. By far the largest improvements were in historically disadvantaged schools and top mathematics performers are spread across more schools in 2015 than they were in 2008.

We do not dispute that the under-performance of schools is a key factor holding the country’s development back. This is made clear in the National Development Plan. However, where we do disagree strongly with the institute for race relations is the direction the schooling system has been taking in recent years.

If the movement has been in the right direction and improvements as large as one might realistically expect, then one could hardly hope for more.

The evidence suggests the quality of school education is improving, that the improvements have been substantial and encouraging, and that they are helping to overcome historical race-based inequalities.

But trends seen in a few other countries, such as Brazil, suggest we should be aiming for an even steeper improvement.

This is what ongoing changes to our interventions, of which there are many, should aim to achieve. We also need a more rigorous national debate, involving a wider range of stakeholders, about the actual performance trends of schools.

ReSEP Launches Two Reports on Education

 

On the 24th of May the Research on Socioeconomic Policy (ReSEP) Group launched two reports which were the culmination of two years of research on South African education. The first report was for the Programme to Support Pro-Poor Policy Development (PSPPD), a joint initiative between the South African Presidency and the European Union. That report is titled “Identifying Binding Constraints in South African Education.” The second and related report was funded by the Zenex Foundation and focussed specifically on Grades 1-3. That report is titled “Laying Firm Foundations: Getting Reading Right.”

Both of the these reports highlight the centrality of ensuring that all children learn to read for meaning by the end of Grade 3. At the moment in South Africa about 60% of children cannot read at even a basic level at the end of Grade 4. These children never fully engage with the curriculum and fall further and further behind the curriculum even as they are promoted into higher grades.

The research has culminated in 10 journal articles, 12 policy briefs, 1 monograph, 2 reports and 2 policy-engagement workshops. The research covered topics ranging from excessive class sizes and learning deficits to the salaries and qualifications of Early Childhood Practitioners. It included quantitative analyses, a literature review and a special issue of the South African Journal of Childhood Education. The researchers have created a “Roadmap for Action” with detailed suggestions and recommendations for how the Department of Basic Education can ensure that all children learn to read.

 

The full reports and policy briefs can be found below:

Projects:

  1. Laying Firm Foundations: Getting Reading Right
  2. Identifying Binding Constraints in Education
  3. Teaching Reading (and Writing) in the Foundation Phase: A Concept Note

Policy Briefs:

  1. Limited support for the foundation phase:A misallocation of district resources
  2. Increasing the supply of teacher graduates
  3. Education datasets in South Africa
  4. Rethinking pre-grade R
  5. Is school based assessment in matric achieving its potential?
  6. Improving the calibre of school leadership in South Africa
  7. The DBE’s workbooks as a curriculum tool
  8. Adding randomised control trials (RCTs) to the education research toolkit
  9. What the ANAs tell us about socioeconomic learning gaps in South Africa
  10. Learning to read and reading to learn
  11. Excessive class sizes in the Foundation Phase
  12. Building an evidence base for inclusive education in South Africa: Focusing on learners with disabilities

Why mothers aren’t accessing antenatal care early in their pregnancies

This article is based on a post by Dr Anja Smith that originally appeared on The Conversation, and can be accessed, in full, here. Dr Smith is a Postdoctoral Fellow at Research on Socio-Economic Policy (ReSEP) unit in the Economics Department, Stellenbosch University.


 

New research by Dr Anja Smith, Prof Ronelle Burger, and Dr Vivian Black has investigated the reasons why South Africa faces such high numbers of late antenatal clinic visits by pregnant women. By usual measurements, nearly 80% of pregnant women in the study, who were interviewed after giving birth in an urban area in the Western Cape, had visited antenatal clinics for the first time, late.

This trend is seen as correlated to the extremely high maternal mortality levels prevalent in South Africa. The WHO found that, in 2015, for every 100,000 live births in South Africa, 138 women died due to pregnancy and childbirth complications. Compared to both other developed- and developing nations, this figure is startlingly high.

Dr Smith, et al. found that late antenatal clinic visits are largely due to women not becoming aware of their pregnancies soon enough, and was more prevalent among women with lower levels of education and socio-economic standing.

To combat this, Smith, et al. suggest two areas of policy intervention that could affect valuable change:

  1. They suggest that the availability of urine-based pregnancy tests should be increased to match the distribution of free condoms, which would also assist in the treatment of the unborn children of HIV-positive mothers, when identified early enough.
  2. Furthermore, the manner in which contraceptives themselves are distributed at clinics needs to be reconsidered, as many women in the study had unplanned pregnancies. This strategy could start by emphasising the roles of contraceptives not solely as HIV and other STI preventative measures.

To read the full article and learn more about the findings and insights of the research, please read the full article by Dr Smith, here.

Black youths are NOT educationally worse off than 20 years ago – Dr Stephen Taylor

By Dr Stephen Taylor. Dr Taylor is a research associate of ReSEP who works full-time at the Department of Basic Education.


Figure 1: Proportions of the population achieving secondary education (left) and a bachelors degree (right). (Source: Stats SA (2015): CENSUS 2011: A profile of education enrolment, attainment and progression in South Africa, page 41.)

On the 18th of April the Business Day published a piece titled, “Black youth less educated now than 20 years ago.” This statement is simply wrong and unsupported by any data set. Yet the story is now gaining momentum and has been published by other news outlets, such as the Daily Maverick, reporting that “Stats SA claims black youth are less skilled than their parents.”

The article asserts that “black and coloured youths have regressed in their educational achievements” and that the proportion of black and coloured youth that complete a university degree as a share of the population has decreased. This is factually incorrect.

The article references a recent Stats SA report on the status of the youth as well as comments by the Statistician General, Pali Lohohla, as the basis for these assertions.

But in fact, the Stats SA published reports (as with all other analysis I have seen or done) indicate that the proportions of black and coloured youths who attain grade 9, grade 12, and a university degree, have all increased consistently in recent decades and are still increasing. It is thus not clear where this misconception arose.

I suspect the mistake may have arisen through a misunderstanding of a statistic which has been presented by the Statistician General recently and which appears in Stats SA’s report on educational enrolment, attainment and progression (December 2015). The statistic shows that the proportion of black and coloured youths who achieve a bachelors degree “after completing grade 12” has been declining over the last 20 to 30 years.

It needs to be understood that this statistic is the proportion of matriculants who go on to attain a degree. In other words, the denominator in this calculation is matriculants as opposed to the entire black and coloured population.

The improvement in matric attainment among black and coloured youth has been larger than the improvement in degree attainment among black and coloured youth, but – and this is the important part – there have been big improvements in both. The fact that the increase in degree completion has been slower than the increase in matric completion is not at all an indication that youth are worse off now than 20 years ago.

So the ‘bad’ news is that degree completion, although it has increased, has not kept pace with the fast increase in the attainment of matric amongst black and coloured youths. But this certainly does not mean that educational outcomes are worse than 20 years ago.

So what do the numbers actually say? The Stats SA report issued in December shows that the proportion of black people completing matric has been consistently increasing from about 20% to about 50% over the last 50 years.That report also indicates that the proportion of black people completing a degree has increased from about 2% to about 4% over the same period.

Whether you read official Stats SA reports or do your own calculations on the various Stats SA datasets – I have analysed Census data from 1996, 2001 and 2011 as well as General Household Survey data from 2002 to 2014 – it is clear that both matric attainment and degree attainment has been increasing amongst the black and coloured population.

It is also useful to consider the Department of Basic Education’s matric statistics from recent years. In 1990, there were 191 000 matric passes. By 2015 this number had more than doubled to 465 863. This increase has been driven mainly by growing numbers of black youth passing – and this growth has easily outstripped population growth, which has been about 1% a year. Even since 2008, the number of black matric passes has increased from about 250 000 to over 350 000. And the number of black people achieving a bachelors pass in matric has increased from about 60 000 to about 120 000 since 2008.

I am by no means suggesting that everything is fine in our education system. Despite the progress, there are still too many youths who do not get to grade 12, the main reason being that educational foundations laid in earlier grades have been inadequate. And completion rates at our higher education institutions should worry us. But there have been improvements in both of these areas relative to 20 years ago.

Although improved access at lower levels of education (primary and secondary school completion) has been faster than access at higher levels, paradoxically the solutions must focus on the early grades if sustainable progress is to be made.

The most alarming education statistics to me are the low proportions of children achieving basic literacy and numeracy in the early grades. International assessments of education quality point to serious deficiencies in this area, even compared to some other countries in the region. If children are not learning to read in the early grades, they will not be able to make it to higher education.

But even in the area of learning quality, the evidence points to improvement. The Trends in International Mathematics and Science study (TIMSS) showed substantial improvements in mathematics and science achievement at the grade 9 level between 2002 and 2011. However, this improvement is off a very low base.

Educational outcomes in South Africa remain far too low, especially amongst youths from poor communities. But claims that education was better under apartheid or that outcomes have deteriorated over the last 20 years are alarmist and have no basis in reality.

Dr Stephen Taylor is a researcher in the South African Department of Basic Education. His work includes impact evaluation of education interventions, measuring educational performance and equity in educational outcomes. In 2010 he completed a PhD in economics at the University of Stellenbosch,analysing educational outcomes of poor South African children.

(This article first appeared in the Business Day on Friday the 29th of April 2016)