The release this week of the latest round of international comparative education results produced some fascinating results. Not least of these was the outcome for Wales, characterised by the Wales’ Education Minister as alarming and “unacceptable”.
The PISA (Programme for International Student Assessment) results derive from a standardised international assessment of 15-year-olds, run by the OECD. They show that Wales has fallen further behind since the last tests in 2006, and scored worse than before in each of reading, maths and science. Scores in Wales have fallen relative to England and are now “cast adrift from England, Scotland and Northern Ireland”. The Wales Education Minister, Leighton Andrews, described the results as reflecting “systemic failure”.
What might that systemic failure be? One leading candidate is highlighted in our recent research on accountability mechanisms for state schools. We argue that the decision in 2001 by the Welsh Assembly Government (WAG) to stop the publication of school performance tables or “league tables” has resulted in a significant deterioration in GCSE performance in Wales. The effect is sizeable and statistically significant. It amounts to around 2 GCSE grades per pupil per year; that is, achieving a grade D rather than a B in one subject. This is a substantial effect, equivalent to the impact of raising class size from 30 to 38 pupils.
Although our results are based on a study of the GCSE scores school-by-school, Figure 1 gives a very stark impression of the overall effect. Students in England and Wales were performing very similarly up to 2001, but thereafter the fraction gaining 5 good passes has strongly diverged.
We take each secondary school in Wales, and match it up to a very similar school in England. This “matching” is based on pupils’ prior attainment, neighbourhood poverty and school funding among other factors. We then track the progress (or value added) students make in these schools before and after the league tables reform, comparing the Welsh school with its English match. Our analysis explicitly takes account of the differential funding of schools in England and Wales, and the greater poverty rates found in neighbourhoods in Wales.
Why should the removal of school league tables lead to a fall in school performance? Part of the effect is though the removal of information to support parental choice of school. The performance tables allow parents to identify and then apply to the higher scoring schools, and to identify and perhaps avoid the low scoring schools. This lack of applications puts pressure on the latter schools to improve. But this is not all of the story. Perhaps as important is the simple public scrutiny of performance, and in particular the public identification of the low scoring schools. This “naming and shaming” means that low scoring schools in England are under great pressure to improve, whereas the same schools in Wales are more able to hide and to coast.
Our work has attracted criticism, including a charge of using an “ideological theory” from teacher unions . A more thoughtful critic has accused us of a “howler” in the analysis: not noting the introduction of the original GCSE-equivalent qualifications. In fact, since these were introduced equivalently in both countries they simply net out of the comparison.
Responding to our research, the Welsh Assembly Government said “wait for the PISA results”. These results are now in, and do not make happy reading. No doubt there are many factors underlying the relative performance of Wales and England, but the diminution of public accountability for the country’s schools is surely one of them.