Education trends point to notable improvements being made

Dr Martin Gustafsson is based at the Department of Basic Education and is a member of the Research on Socioeconomic Policy group at Stellenbosch University. This article first appeared in the Business Day on the 25th of April and is also available here.

The technical report which informs the newspaper article, and which was discussed at a ReSEP workshop on 18 January 2016, is available here.

By Dr Martin Gustafsson


Picture: THINKSTOCK

The debate over whether the performance of schools is improving is a messy one, but one worth cleaning up. If one looks through the maze of often ambiguous statistics, there are a few indicators we can use to reveal actual trends. Fortunately for the country’s future, they point to notable improvements.

This is not to say all is well in the schooling system. In many respects it is not. Yet it would be an incomplete and incorrect diagnosis to ignore steps in the right direction.

John Jerrim, a British education analyst, in an interrogation of Britain’s school performance numbers, demonstrates how even in a country with supposedly good monitoring capacity it is alarmingly easy to get the trends wrong, and for this to lead to unnecessary or inappropriate policy reforms.

In SA, at the secondary school level, the matric pass rate is deeply entrenched in the policy discourse. Its dominance is perhaps unfortunate as it is an exceedingly difficult statistic to interpret because it is influenced by such a variety of factors, from demographic trends to subject selection.

One of the measures we have that gets close to being truly and simply comparable over time is our Trends in International Mathematics and Science Study (Timss) score. The Timss tests, which use common “anchor” questions across years and were administered in representative samples of schools in 2002 and 2011, point to large improvements, off a low base, in Grade 9 maths and science over the nine years.

However, legitimate questions have been raised over the ability of Timss to accurately measure progress in SA, given that Timss is designed primarily for countries with high average levels of performance, and is thus not good at differentiating between relatively weaker learners.

Patterns in our Grade 12 examinations data seem to confirm that substantial improvements in mathematics and science have occurred, and that the trend has continued beyond 2011. However, the care that needs to be taken in interpreting the data is a salutary reminder of what a minefield education performance data can be.

The analysis work I did for the Department of Basic Education was partly prompted by the insistence of the Department of Planning, Monitoring and Evaluation that a “missing middle” be addressed. Traditionally, the focus in basic education has been at a relatively low level, or the numbers of learners passing a minimum threshold, and then also at the absolute top end, or the numbers of learners passing with at least 80% in a subject, a distinction. What matters a lot for the future of engineering, financial management and other mathematically-oriented fields is the number of Grade 12 learners achieving 60% or 70% in critical subjects such as maths, given that these are cut-offs universities typically use in their entry requirements.

If one takes the Grade 12 results over the years at face value, one obtains peculiar results. Physical science outcomes have risen sharply between 2008, when the current National Senior Certificate was introduced, and last year, while maths outcomes have dropped. One would not expect closely related subjects to display such divergent trends. Moreover, it emerges that the percentage of white and Indian Grade 12 learners becoming “60 plus” maths performers, meaning they obtain a mark of 60% or more, has declined markedly, from 32% to 23% over the period.

For black African and coloured learners this probability dropped too, from 5% to 4%. These race-based inequalities are, of course, at the core of the enormous challenges the schooling system must deal with. But why would the performance of, in particular, whites and Indians drop in this fashion? There are no reasons to believe this trend to be real.

Closer analysis of the data in fact showed that marks in different years were not exactly equivalent. This should not come as a surprise. While there is almost certainly scope for improvement to the annual marks standardisation process, it would be technically impossible for a nationwide examination system such as our matric to yield exactly comparable marks year after year, through, for instance, anchor items. Common questions would soon become widely known, and their utility thus undermined.

More or less equivalent scores were found by examining the distribution of marks in a sample of stable schools that were unlikely to experience large performance shifts over time. These were high-performing schools with stable enrolment, subject participation and demographic patterns. To illustrate, in maths it seemed a mark of 60 in 2013 was about as difficult to obtain as a mark of 59 last year, 62 in 2010 and 70 in 2008. Across many subjects, 2008 emerged as a year when it was particularly easy to obtain high marks.

Equivalent scores were used to recalibrate marks across the system. More consistent trends emerged as a result. Maths and physical science both displayed improvements over time, roughly in line with each other. Whites and Indians did not experience dramatic declines.

A key finding was that strong improvements emerged among black African and coloured learners. The number of these learners reaching the level of mathematics performance represented by a mark of 60 in 2013 increased by 66%, from around 11,300 a year in 2008-09, to 18,800 in 2014-15. The group of top performers clearly became more diverse in terms of race. The analysis does imply that universities and students should not be overly rigid in interpreting marks when they plan for the future. But the positive trends also imply that one can expect students to repeat and drop out less at university.

The bulk of the improvements have occurred in historically disadvantaged schools, and high performers have been found in an increasing number of schools. In 2008, 60% of Grade 12 learners were in schools with at least one “60 plus” maths student. By 2015, this figure had risen to 77%. This is important. The presence of at least one “whizzkid” in a school, whom fellow learners and even teachers can turn to for advice, can be seen as a sign of a more vibrant mathematics class.

In the light of concerns around the effect of a large number of “progressed” learners in Grade 12 last year, one finding from the analysis is noteworthy. Across key subjects, obtaining 60% and 70% appeared no easier last year than in previous years. In fact, in maths and physical science this was slightly harder in 2015 than in the previous two years. At least at this level of the performance spectrum, historical standards were upheld.

Rather than introducing a completely new type of intensive care for, say, maths, the actual trends suggest we should instead learn from those schools and provinces that are improving fastest, and then promote practices that seem to work. More investigative data work needs to be undertaken, by a wider range of analysts. School-level subject data for Grade 12 are now generally available to researchers through an online data portal, thanks to a data sharing agreement between the Department of Basic Education and DataFirst at the University of Cape Town.

The next step should be to make learner-level data more widely available, partly through the introduction of a rigorous anonymisation process that would protect the privacy of individual citizens.

Grade RR: Priority or Pie in the Sky?

Children play beneath illegally connected electrical wires in Kliptown, Soweto. November 19, 2014. Photo: REUTERS, Siphiwe Sibeko

The education crisis has many proposed solutions, but new evidence points to the problem that our best bet – catching the problem early – might be a practically unattainable goal.

The Daily Maverick reports that findings from reports and research conducted on the feasibility of a high-quality early learning programme roll-out that could drastically improve the South African education problem, are not encouraging, both for now and in the future.

The research indicates that despite the enormous benefit to pursuing high-quality pre-school education programmes, South Africa cannot ensure the necessary quality of such a strategy for it to be meaningful, and new reports add that the quality cannot be bettered without drastic changes.

Without the widespread availability of the proper tools to measure the quality of ECD programmes, the feasibility of any new early learning programmes of meaningful quality will remain dubious.

To read the full article on The Daily Maverick, please click here.

Improving the school principal appointment process is a national priority

The education sector is in the spotlight this week with the release of an interim investigation report into corruption in the appointment process for educators and principals in South African schools. The investigation, headed up by Professor John Volmink, was initiated after allegations of high levels of nepotism in the appointment process, as well as the sale of posts for cash – with ensuing violent crimes – that have been linked to principal appointments in the past year.

The interim report confirms that there has been widespread manipulation and corruption and arrests are imminent. The gravity of the findings have been acknowledged by the National Department of Education as they note weaknesses in the system and the need to freeze new appointments  until improvements to the process have been implemented (see media report).

While extremely sobering, the investigation’s findings are a move in the right direction to eliminating corruption in education. But the findings are also very timeous as the country faces a rising and increasing number of principal replacements, as identified in new research by Gabrielle Wills.

With a rapid increase in the average age of school principals and around 24 000 public schools, South Africa is likely to require at least 1 000 principal replacements per year for the next decade just to replace retiring principals. The opening of these large numbers of vacant school leadership posts raises the urgency to eliminate corruption in appointments and to improve weak national policy leading to bad appointments, while the opportunity exists.

This is particularly important in a context where the average tenure of principals is long (almost ten years) and they face little threat to job-security, especially in provinces with strong SADTU representation. Wills identifies that, on average, less than 1 in a 1 000 principals were dismissed in South Africa in 2011.

Nationally, processes and short-listing criteria governing appointments require that to apply for a principal post, a candidate must possess at least 7 years of teaching experience and a Relative Educational Qualifications Value (REQV) of 14. But rough calculations suggest that about 87% of all educators (excluding principals) meet these requirements, as is. The existing national criteria, therefore, provide little to no value in sifting out weaker candidates.

Even in the absence of corruption, highly unsuitable principals may be appointed despite following ‘due process’. Yet raising qualification criteria is not the answer to improving the appointment process.  Consistent with international research, Wills finds that in the majority of South African schools, principals’ years of service and REQV level ranking have little observable relationship with student performance in the school’s they lead, but are correlated with remuneration scales.

The National Development Plan does address this issue, recommending the introduction of competency-based testing in the appointment process for principals in efforts to improve the calibre of school leadership. It also recommends the introduction of performance contracts which, if implemented successfully, would require higher levels of effort from school leaders and provide substantiation for dismissing under-performers.

Wills’ research finds strong support for both recommendations. Competency-based testing may assist in identifying better quality candidates and where managed by an independent third party may limit the influence that union members and corrupt stakeholders have on the appointment process.

 

A published version of Wills’ research on the principal labour market can be accessed through the South African Journal of Childhood Education, here.

Research by ReSEP’s Dr Nic Spaull Featured Prominently in 2015 Child Gauge

Earlier this month marked the release of the 2015 Child Gauge focussing on “Youth and the Intergenerational Transmission of Poverty.’ The work of RESEP researcher Dr Nic Spaull featured prominently in the report and built on earlier work conducted by the broader RESEP team. Dr Spaull wrote the education chapter for the 2015 Gauge, titled Schooling in South Africa: How low-quality education becomes a poverty trap.

ABSTRACT: “The strong legacy of apartheid and the consequent correlation between education and wealth helps drive an intergenerational cycle of poverty where children inherit the social standing of their parents or caregivers, irrespective of their own abilities or effort. This essay provides an overview of educational outcomes in South Africa, and describes the linkages between the education system and the labour market. It illustrates clearly how low-quality education in South Africa is a key mechanism in the reproduction of inequality and proposes policy and programmatic interventions to alleviate the situation”.

The findings from this chapter were influential in the overall direction of the Gauge and can be seen as the main focus of the poster at the back of the Gauge report. All the Gauge articles are available here.

Gustafsson v Friedman: Taking education off the pedestal of economic growth “incomplete”, “misleading”

In a recent letter published on Business Day‘s website, ReSEP researcher and Department of Basic Education adviser Martin Gustafsson, challenges an opinion piece by Steven Friedman, Director of the Centre for the Study of Democracy, titled “Education is not the cure-all it is made out to be”.

In light of the ongoing student protests and the state of access to, and quality of, education in South Africa, Friedman’s original opinion piece had drawn from the work of Ricardo Hausmann, former Venezuelan cabinet minister and academic economist.

Friedman writes: “[Hausmann's] numbers show that huge growth in education levels over the past 50 years have not translated into equivalent growth in the economy. [He] pointed out that China’s educational progress lags behind that of Tunisia, Mexico, Kenya and Iran — yet it grew much faster than any of them.”

Gustafsson is quick to point out that Hausmann and Friedman’s argument provides a misleading and incomplete view of the total body of research in this field.

“Yes, there was considerable disagreement around the education-growth link when we still relied on educational participation as an indicator of investment in education. But since about 2005, the use of educational-quality indicators, drawing from test data, have made the link clearer and far less disputable. In fact, over the longer term, education tends to emerge as the most powerful single factor promoting growth,” Gustafsson writes.

But Friedman also argues in his piece that the near ubiquitous notion that education is the central problem facing slowly developing and unequal economies, can have drawbacks.

“This is why harping on education can hold us back — by diverting our attention from other problems that need attention. A frequent problem is that the education argument becomes a handy excuse for those who are doing well and do not want anything to change… [This type of argument] places all the blame on people at the bottom and requires no change at all from those at the top. Like the claim that voters need “education” whenever they behave in a way that elites dislike, this is more about making people at the top feel better than about solving problems.”

However, Gustafsson points out that Hausmann’s own article faces wanting scrutiny in its lack of reasonable time dimensions in evaluating time-dependent variables such as education.

“The idea that there should be a worrying education myth within the growth discourse is itself a myth, it would seem, and is an unnecessary distraction,” he concludes.

 

To access the original articles that form this debate, please follow these links in chronological order: Hausmann; Friedman; Gustafsson

To learn more about Martin Gustafsson and his role with the ReSEP group, please click here.