ReSEP report on flows from matric to, and through, university enriching debate around student fees

Recent research looking at “higher education access and outcomes for the 2008 national matric cohort”, by Hendrik van Broekhuizen, Servaas van der Berg and Heleen Hofmeyr of Stellenbosch University are colouring the debate around student fees in South Africa with in-depth analysis that are attracting welcome interest.

The Mercury recently reported on the findings of the research outputs of the ReSEP authors, and highlighted the much more pervasive extent of the challenges in resolving the student fees crisis.

Looking at matric exam data from 2008 to 2013, data from South African universities from 2009 to 2014, data from the Educational Management Information System master-list and data from the 2011 national census, the authors conducted an investigation of university access in South Africa.

Their findings included that, for those who started school, only:

  • 60% wrote matric.
  • 37% passed matric.
  • 12% gained access to university within six years of finishing school.
  • 6% completed an undergraduate qualification within six years of finishing school.
  • 4% completed a degree within six years of finishing school.

To read the full story and what underlying factors are contributing to the challenges of basic education in South Africa to contribute to the resolution of the student fees crisis, please click here to go to the IOL website, and read the original article as published in The Mercury.

New data of challenges in tertiary education presented at LMIP conference

Hendrik van Broekhuizen, Servaas van der Berg and Heleen Hofmeyr recently published a report on higher education access and outcomes for the 2008 national matric cohort. The study makes use of a unique dataset that combines data at the school and university level in order to track the transition of the 2008 matric cohort from school to university. This enables analysis of the relationship between school-level factors such as matric performance and university outcomes such as access, completion and dropout. Servaas van der Berg presented the main findings of the report at the Research Colloquium on Skills Planning on the 29th of September in Johannesburg. The colloquium was hosted by the Labour Market Intelligence Partnership and the Department of Higher Education and Training.

The main findings of the report are as follows:

• Approximately one-third of learners who write the matric examinations and obtain Bachelor’s passes never go to university.

• Amongst matriculants that obtained Bachelor’s passes, university access is not skewed against black students or students from poorer schools.

• A large proportion of matriculants who do go to university do not enter in the year following matric, but only one or more years later.

• Matric marks are a good indicator of university access, but only weakly related to eventual university completion rates.

• There are large differences across universities in the average matric performance of students who attend these universities.

• It takes a long time for many students to successfully obtain university qualifications.

• Dropout rates at university, although high, are not as high as often reported, because many students that are considered “drop-outs” from university in official statistics did not leave the university system, but changed their degree programme, switched from a degree to a diploma or certificate programme, or enrolled in a different university.

Prof Van der Berg’s presentation highlighted the extremely small proportion (14%) of learners who start school who end up with a Bachelor’s matric pass. 12% achieve Diploma passes. This implies only 26% of learners who start school will be eligible for enrolment in university. Delayed entry to university and low completion rates result in only 6% of the original group who started school obtaining some kind of undergraduate qualification within six years of completing matric. Prof Van der Berg also pointed out the importance of matric performance in ‘gateway subjects’ such as Mathematics, Physical Sciences and Mathematical Literacy in university completion rates.

These findings suggest that many patterns of university access and success are influenced strongly by school results. The weak school system has a major influence on who reaches matric and how they perform in matric. This, and particularly the achievement of Bachelor’s passes, explains much of the racial differences in access to university.

RESEP hosts Conference on Quantitative Applications in Education

On the 19th and 20th of September this year, RESEP hosted the “Conference on Quantitative Applications in Education” at STIAS. The conference was attended by 85 participants from government, academia and civil society, with a keynote address by the Minister of Basic Education Ms Angie Motshekga

In her address, the Minister highlighted the importance of basing education policy on rigorous research. The Minister reiterated the important links between her Department and the research community:

“I want to acknowledge the valuable work that has been generated by the RESEP group recently. And not only be those directly part of RESEP, but also by many others here today, who are part of a growing network of people focused on understanding the challenges of the education sector and on offering solutions that can be implemented at a policy level. The two reports released earlier this year by RESEP have been tremendously influential in shaping thinking around what should be the priorities in education policy. These are the report on “Binding Constraints” in the sector and the report on “Laying Firm Foundations” through getting reading right.”

The conference heard from two international researchers based at the Research Triangle Institute: Dr Luis Crouch and Dr Benjamin Piper. Further presentations were given by researchers from the Department of Basic Education, the Human Sciences Research Council (HSRC), the Joint Education Trust (JET), as well as those based at the University of Johannesburg, the University of Stellenbosch, the University of Cape Town, the University of KwaZulu-Natal, and the University of Witwatersrand. Links to the powerpoint presentations can be found below:

Day 1

 Day 2

 

The uphill battle of establishing accurate performance trends for schools

Dr Martin Gustafsson is based at the Department of Basic Education and is a member of the Research on Socioeconomic Policy group at Stellenbosch University. This article first appeared in the City Press on the 26th of May and is also available here.

The technical report which informs the newspaper article, and which was discussed at a ReSEP workshop on 18 January 2016, is available here.

By Dr Martin Gustafsson


Schools dragging SA down

The South African Institute of Race Relations, in its February 2016 Fast Facts publication, concludes that “schools drag South Africa down” and that pupil performance is declining substantially.

The latter conclusion is based almost entirely on two numbers: the number of Grade 12 pupils obtaining a score of 70% or more in mathematics in 2008, which was 25 027, and the corresponding figure for 2015, 17 452. The two figures point to a decline of around 30%. At first glance, this appears to be a national disaster.

However, analysis done by myself and others in the basic education department, to be published shortly, paints a completely different picture, of large and encouraging improvements in mathematics in Grade 12. Many of the challenges facing the schooling system are accurately described by the institute’s report, but I disagree completely that the numbers should be pointing to a deterioration.

The problem is, first, that the institute looked selectively at the numbers, and ignored important figures, including a few appearing in their own report. This seems irresponsible.

Second, analysing pupil performance trends is exceedingly complex, not just in South Africa. The British education analyst John Jerrim has written extensively about how the data on mathematics trends have been spectacularly misinterpreted in his country. In South Africa, the complexities are particularly daunting in the case of Grade 12 mathematics.

So what did the institute’s analysts get wrong? They failed to point out that all of the 30% decline they refer to happened between 2008 and 2009. From 2009 to 2015, the trend, using values from all years (as one should), is a weakly positive 2% overall. The number of passes at the 70% level in 2008 was exceptionally high relative to all other years, something which should make any analyst suspicious.

The institute’s analysts also fail to point out that the overall increase in the number of physical science passes at the 70% level, over the entire 2008 to 2015 period, was a whopping 85%. In contrast to mathematics, what appears suspicious here is at least one exceptionally low value at the start of the period, in 2009. The question is why two such closely related subjects would move in completely opposing directions.

Another suspicious trend discussed in the analysis to be released by the basic education department is that the percentage of white and Indian pupils achieving high marks in mathematics has declined markedly over the 2008 to 2015 period. There appears to be no plausible explanation for this trend among these two relatively advantaged groups.

We zoomed into a sample of particularly stable and well-performing schools, with about 4000 mathematics candidates each year, to find explanations to the apparent anomalies. What emerged clearly is that variations across years in the difficulty of obtaining certain marks, for instance 70% in mathematics, explain most of the anomalies.

Mostly these variations are small, but for certain years they are large. Pupils who obtained 69% in the years 2012 to 2015, when levels of difficulty appeared particularly stable, would have obtained 68% in 2011, a marginally more difficult year, and at least 72% in the years 2008 to 2010. 2008 was a particularly easy year for obtaining high marks.

Changing the criteria for our sample did not change the picture substantially. Over the years, the mathematics examination became more difficult, while for physical science the opposite was true.

So is the problem then poor standardisation in the examinations system? Yes and no. There appears to be scope for improving the comparability of marks across years and this is receiving the attention of the basic education department and the council for quality assurance in general and further education and training, Umalusi.

At the same time, it is technically impossible to achieve anything approaching perfectly comparable marks, at all mark levels, in an examination system such as ours, or in similar systems in other countries. We need to learn to live with some variation over the years and rely on other systems, such as the international testing programmes, for more rigorous assessment of trends.

When we recalibrated results for all pupils over the 2008 to 2015 period, using what we found to be equivalent scores, we found that the number of pupils achieving a 70% level of performance in mathematics increased by 27% overall.

For black African pupils the increase was 61%. Physical science improvements, on the other hand, were found to be smaller than what published statistics would suggest, but were still encouraging. By far the largest improvements were in historically disadvantaged schools and top mathematics performers are spread across more schools in 2015 than they were in 2008.

We do not dispute that the under-performance of schools is a key factor holding the country’s development back. This is made clear in the National Development Plan. However, where we do disagree strongly with the institute for race relations is the direction the schooling system has been taking in recent years.

If the movement has been in the right direction and improvements as large as one might realistically expect, then one could hardly hope for more.

The evidence suggests the quality of school education is improving, that the improvements have been substantial and encouraging, and that they are helping to overcome historical race-based inequalities.

But trends seen in a few other countries, such as Brazil, suggest we should be aiming for an even steeper improvement.

This is what ongoing changes to our interventions, of which there are many, should aim to achieve. We also need a more rigorous national debate, involving a wider range of stakeholders, about the actual performance trends of schools.

Black youths are NOT educationally worse off than 20 years ago – Dr Stephen Taylor

By Dr Stephen Taylor. Dr Taylor is a research associate of ReSEP who works full-time at the Department of Basic Education.


Figure 1: Proportions of the population achieving secondary education (left) and a bachelors degree (right). (Source: Stats SA (2015): CENSUS 2011: A profile of education enrolment, attainment and progression in South Africa, page 41.)

On the 18th of April the Business Day published a piece titled, “Black youth less educated now than 20 years ago.” This statement is simply wrong and unsupported by any data set. Yet the story is now gaining momentum and has been published by other news outlets, such as the Daily Maverick, reporting that “Stats SA claims black youth are less skilled than their parents.”

The article asserts that “black and coloured youths have regressed in their educational achievements” and that the proportion of black and coloured youth that complete a university degree as a share of the population has decreased. This is factually incorrect.

The article references a recent Stats SA report on the status of the youth as well as comments by the Statistician General, Pali Lohohla, as the basis for these assertions.

But in fact, the Stats SA published reports (as with all other analysis I have seen or done) indicate that the proportions of black and coloured youths who attain grade 9, grade 12, and a university degree, have all increased consistently in recent decades and are still increasing. It is thus not clear where this misconception arose.

I suspect the mistake may have arisen through a misunderstanding of a statistic which has been presented by the Statistician General recently and which appears in Stats SA’s report on educational enrolment, attainment and progression (December 2015). The statistic shows that the proportion of black and coloured youths who achieve a bachelors degree “after completing grade 12” has been declining over the last 20 to 30 years.

It needs to be understood that this statistic is the proportion of matriculants who go on to attain a degree. In other words, the denominator in this calculation is matriculants as opposed to the entire black and coloured population.

The improvement in matric attainment among black and coloured youth has been larger than the improvement in degree attainment among black and coloured youth, but – and this is the important part – there have been big improvements in both. The fact that the increase in degree completion has been slower than the increase in matric completion is not at all an indication that youth are worse off now than 20 years ago.

So the ‘bad’ news is that degree completion, although it has increased, has not kept pace with the fast increase in the attainment of matric amongst black and coloured youths. But this certainly does not mean that educational outcomes are worse than 20 years ago.

So what do the numbers actually say? The Stats SA report issued in December shows that the proportion of black people completing matric has been consistently increasing from about 20% to about 50% over the last 50 years.That report also indicates that the proportion of black people completing a degree has increased from about 2% to about 4% over the same period.

Whether you read official Stats SA reports or do your own calculations on the various Stats SA datasets – I have analysed Census data from 1996, 2001 and 2011 as well as General Household Survey data from 2002 to 2014 – it is clear that both matric attainment and degree attainment has been increasing amongst the black and coloured population.

It is also useful to consider the Department of Basic Education’s matric statistics from recent years. In 1990, there were 191 000 matric passes. By 2015 this number had more than doubled to 465 863. This increase has been driven mainly by growing numbers of black youth passing – and this growth has easily outstripped population growth, which has been about 1% a year. Even since 2008, the number of black matric passes has increased from about 250 000 to over 350 000. And the number of black people achieving a bachelors pass in matric has increased from about 60 000 to about 120 000 since 2008.

I am by no means suggesting that everything is fine in our education system. Despite the progress, there are still too many youths who do not get to grade 12, the main reason being that educational foundations laid in earlier grades have been inadequate. And completion rates at our higher education institutions should worry us. But there have been improvements in both of these areas relative to 20 years ago.

Although improved access at lower levels of education (primary and secondary school completion) has been faster than access at higher levels, paradoxically the solutions must focus on the early grades if sustainable progress is to be made.

The most alarming education statistics to me are the low proportions of children achieving basic literacy and numeracy in the early grades. International assessments of education quality point to serious deficiencies in this area, even compared to some other countries in the region. If children are not learning to read in the early grades, they will not be able to make it to higher education.

But even in the area of learning quality, the evidence points to improvement. The Trends in International Mathematics and Science study (TIMSS) showed substantial improvements in mathematics and science achievement at the grade 9 level between 2002 and 2011. However, this improvement is off a very low base.

Educational outcomes in South Africa remain far too low, especially amongst youths from poor communities. But claims that education was better under apartheid or that outcomes have deteriorated over the last 20 years are alarmist and have no basis in reality.

Dr Stephen Taylor is a researcher in the South African Department of Basic Education. His work includes impact evaluation of education interventions, measuring educational performance and equity in educational outcomes. In 2010 he completed a PhD in economics at the University of Stellenbosch,analysing educational outcomes of poor South African children.

(This article first appeared in the Business Day on Friday the 29th of April 2016)