Tag Archives: GCSE

Changing the exam system

Leave a Reply

 

Author: Simon Burgess

It will take a while to fully understand the scope of the proposed changes to the exam system in England.  The exams are obviously very important for students, but also for schools and teachers. Here are some initial thoughts.

A key pledge is to ‘restore rigour’: exams are going to be more rigorous (of course, England has always had at least one exam noted for its rigour. More rigorous might mean different things: it might simply mean more failures or more broadly, greater differentiation between students.

It seems inevitable that one implication of this (in the short run at least) is greater educational inequality. One outcome of making an exam more rigorous is that more students will fail it. It seems hard to see how that could not be true. This is not fatalism; it may well be that over time, teaching brings more students above the line, but there will be a number of cohorts first with more failures. A tougher exam will not be credible if more people pass it. This coupled with a hint of no retakes seems particularly harsh. ‘No second chances’ is an uncompromising view of how to run an education system, and perhaps at odds with much of the rest of public life.

An alternative sense of ‘more rigorous’ is that the new exam will allow for greater differentiation between students. This is as much to do with the marking and reporting as the nature of the exam itself. Rather than proliferating letters and multiple stars, one way to do this that has been mooted is to simply report percentages. More or less differentiation – separating or pooling in economics terms – has a number of implications. On the plus side, it may enable more efficient matching of workers and jobs, raising average productivity; it is likely to incentivise greater effort at school as students currently more-or-less assured of an A* grade push on to shoot for 90%+. On the down side, there will be greater inequality of attainment and consequently greater inequality of earnings. It is not obvious how sizeable these pros and cons are, but there is no guarantee that the outcome will be a happy one.

We have a school system described by the Department for Education as increasingly autonomous.  Diversity of supply is the maxim, and there is reduced oversight of what schools actually do. But what brings the system together is the common exam system. Autonomous schools can do their own thing with less outside ‘interference’ … as long as its students achieve good grades. Schools’ freedom to operate is constrained by close public scrutiny of their performance in common high stakes exams. If there is less monitoring of the process of education, then regulation of the outcome (attainment measures) becomes even more important.

School performance tables, the ‘league tables’, and Ofsted are the central performance components of the regulatory framework. While Ofsted is important, it is largely driven by the school performance tables reporting exam scores. So getting the exams right is very important. Schools will react to changes in the regulatory environment as they will react to any incentives (implicit or explicit) embedded in the system. So in addition to its implications for students, the exam system will very strongly influence how schools operate. Changing the exam system is a big opportunity to get things right or to get things wrong.

One possibility would be to use the reform to move away from the current sharp threshold recording attainment, the fraction of students in a school getting at least 5 good passes. If the students are given percentage scores not letter grades (see below), the metric for the school could simply be the mean score. There is a lot to be said for not having a sharp threshold, or for locating it at the point that policy makers truly do want schools to focus. But this could be done now under the present system – GCSEs have continuous scores that are then converted to letter grades (controversially this year). Switching from a threshold to a continuous average is a separate issue to the rest of the proposed changes, and is not dependent on them.

The proposal to have one exam board per subject seems like a good idea. While monopolies may well raise costs, the nature of the competition between the exam boards was dysfunctional. And the idea of ‘competition to be a monopoly’ is well established (train companies, power transmission and other infrastructure companies) and reasonably well understood.

 

Focussing now on the students, what is the best form of tests for students? The proposals favour a return to a single exam for each subject at the end of the course, and the end of coursework, retakes and modules. Doing well in such exams will require a particular type of skill, so these are what students will try to master and what schools will try to teach.

What sort of skills should education foster? This is not a straightforward question. One plausible answer is that education should test and certify the levels of competence that students have achieved in the skills that employers want.

No one knows what skills business will need in ten years time, and so we can only speculate as to what we should grade students on. But given the ubiquity of the web, it seems very unlikely that ‘remembering large amounts of information and writing it down quickly’ will figure high up on the list. It is hard to see this as a highly prized capability. Some other much-lauded education systems  (keen on rigour) test abilities more likely to be of value. A great deal of research now focusses on different cognitive and non-cognitive skills, how they are built up and how they relate to inequality. I don’t know if there is any evidence whether the ability to remember large amounts of information is more or less socially graded than broader ranges of skills.

One argument for one-shot, high-stakes, closed-book exams is that anything more open is susceptible to parental help, and thus more likely to favour the middle class. But there are other ways to make sure that the summative assessment is solely based on the student’s work alone. Furthermore, parents support their children’s education in so many ways (conversation; providing books and computers; a quiet place to study; role models; trips) and this is unlikely to be the most important. It is worth emphasising that parents helping their children to learn is a good thing, and should be encouraged where lacking, not banned where present.

This preliminary set of points suggests that the proposed reforms of assessment will not promote skills likely to be valued in the labour market. They are likely to lead to more students failing and leaving school with nothing, and/or delaying their one-shot take, or just dropping out. This spells greater inequality in educational attainment and so in life chances.

A Report of Two Halves

1 Reply

Simon Burgess

We published some research last Friday showing that students perform less well in their crucial GCSE exams in years when there is a major international football tournament taking place at the same time. For example, the FIFA World Cup in the summer of 2010, or the UEFA European Championship next summer, both overlap in part with the GCSE exam timetable.

With the draw for the groups in the European Championship taking place earlier that day, much of the comment naturally and sensibly focussed on the specific issue of the impact of next year’s tournament on exam scores. This is important: we estimate that the concurrence of the exams and saturation media coverage of the football reduces exam scores on average by around 0.12 standard deviations of pupil performance and by a lot more for some groups who reduce their effort a lot. These groups tend to be from poorer areas and predominantly (but by no means exclusively) male students.  Since these groups are already lower-performing groups, this means that education gaps will widen. We think of this impact arising through a reduction in student effort, with that time being spent instead on watching the football tournament. The variation in impact arises because of differing tastes for football, arising in turn from cultural norms and idiosyncratic factors, and from the differential effectiveness of an hour of study on exam performance.

However, there is also a broader significance to the research: finding that effort matters matters.

Recent research by economists has broadened out from the previous focus on cognitive ability, and a great deal of work has investigated the role of non-cognitive factors in educational attainment. Non-cognitive factors can be identified with personality traits (see Heckman), and one of the ‘big 5’ personality traits is ‘conscientiousness’, with the related traits of self-control, accepting delayed gratification, and a strong work ethic. Conscientiousness has been shown to be an excellent predictor of educational attainment and course grades. These aspects of self control and ability to concentrate are clearly related to the broad notion of effort we are using here. Our results on the importance of effort strengthen this evidence by isolating the effect of decisions on effort and time allocation in addition to the general ability to concentrate and exert self-control.

There is a great deal of policy interest in England arising from recent studies of US Charter schools with what is called a “No Excuses” ethos. This includes the KIPP (Knowledge Is Power Program) network of schools and schools in the Harlem Children’s Zone. These schools all feature a long school day, a longer school year, very selective teacher recruitment, strong norms of behaviour, as well as other characteristics. Some of the profession’s very top researchers have produced evidence showing that such schools produce very powerful positive effects on student achievement. While this overall effect could be due to different aspects of the KIPP/HCZ ethos, part of it is very likely to be increased effort from the students. Our results complement this by showing the impact of just a change in effort, and that that can have very substantial effects.

This matters for a number of reasons. First, unlike genetic characteristics, cognitive ability or non-cognitive traits, effort is almost immediately changeable. Our results suggest that this could have a big effect. The fact that we find changes in student effort to be very potent in affecting test scores suggests that policy levers to raise effort through incentives or changing school ethos are worth considering seriously. Such interventions would be justified if the low effort resulted from market failures due to lack of information on the returns to schooling, or time-inconsistent discounting.  Second, the importance of a manipulable factor such as effort for adolescents’ educational performance provides evidence of potentially high value policy interventions much later than “early years” policies. This is encouraging, offering some hope that low performing students’ trajectories in life can perhaps be effectively improved even after a difficult environment early in life.