Tag Archives: Students

Changing the exam system

Leave a Reply

 

Author: Simon Burgess

It will take a while to fully understand the scope of the proposed changes to the exam system in England.  The exams are obviously very important for students, but also for schools and teachers. Here are some initial thoughts.

A key pledge is to ‘restore rigour’: exams are going to be more rigorous (of course, England has always had at least one exam noted for its rigour. More rigorous might mean different things: it might simply mean more failures or more broadly, greater differentiation between students.

It seems inevitable that one implication of this (in the short run at least) is greater educational inequality. One outcome of making an exam more rigorous is that more students will fail it. It seems hard to see how that could not be true. This is not fatalism; it may well be that over time, teaching brings more students above the line, but there will be a number of cohorts first with more failures. A tougher exam will not be credible if more people pass it. This coupled with a hint of no retakes seems particularly harsh. ‘No second chances’ is an uncompromising view of how to run an education system, and perhaps at odds with much of the rest of public life.

An alternative sense of ‘more rigorous’ is that the new exam will allow for greater differentiation between students. This is as much to do with the marking and reporting as the nature of the exam itself. Rather than proliferating letters and multiple stars, one way to do this that has been mooted is to simply report percentages. More or less differentiation – separating or pooling in economics terms – has a number of implications. On the plus side, it may enable more efficient matching of workers and jobs, raising average productivity; it is likely to incentivise greater effort at school as students currently more-or-less assured of an A* grade push on to shoot for 90%+. On the down side, there will be greater inequality of attainment and consequently greater inequality of earnings. It is not obvious how sizeable these pros and cons are, but there is no guarantee that the outcome will be a happy one.

We have a school system described by the Department for Education as increasingly autonomous.  Diversity of supply is the maxim, and there is reduced oversight of what schools actually do. But what brings the system together is the common exam system. Autonomous schools can do their own thing with less outside ‘interference’ … as long as its students achieve good grades. Schools’ freedom to operate is constrained by close public scrutiny of their performance in common high stakes exams. If there is less monitoring of the process of education, then regulation of the outcome (attainment measures) becomes even more important.

School performance tables, the ‘league tables’, and Ofsted are the central performance components of the regulatory framework. While Ofsted is important, it is largely driven by the school performance tables reporting exam scores. So getting the exams right is very important. Schools will react to changes in the regulatory environment as they will react to any incentives (implicit or explicit) embedded in the system. So in addition to its implications for students, the exam system will very strongly influence how schools operate. Changing the exam system is a big opportunity to get things right or to get things wrong.

One possibility would be to use the reform to move away from the current sharp threshold recording attainment, the fraction of students in a school getting at least 5 good passes. If the students are given percentage scores not letter grades (see below), the metric for the school could simply be the mean score. There is a lot to be said for not having a sharp threshold, or for locating it at the point that policy makers truly do want schools to focus. But this could be done now under the present system – GCSEs have continuous scores that are then converted to letter grades (controversially this year). Switching from a threshold to a continuous average is a separate issue to the rest of the proposed changes, and is not dependent on them.

The proposal to have one exam board per subject seems like a good idea. While monopolies may well raise costs, the nature of the competition between the exam boards was dysfunctional. And the idea of ‘competition to be a monopoly’ is well established (train companies, power transmission and other infrastructure companies) and reasonably well understood.

 

Focussing now on the students, what is the best form of tests for students? The proposals favour a return to a single exam for each subject at the end of the course, and the end of coursework, retakes and modules. Doing well in such exams will require a particular type of skill, so these are what students will try to master and what schools will try to teach.

What sort of skills should education foster? This is not a straightforward question. One plausible answer is that education should test and certify the levels of competence that students have achieved in the skills that employers want.

No one knows what skills business will need in ten years time, and so we can only speculate as to what we should grade students on. But given the ubiquity of the web, it seems very unlikely that ‘remembering large amounts of information and writing it down quickly’ will figure high up on the list. It is hard to see this as a highly prized capability. Some other much-lauded education systems  (keen on rigour) test abilities more likely to be of value. A great deal of research now focusses on different cognitive and non-cognitive skills, how they are built up and how they relate to inequality. I don’t know if there is any evidence whether the ability to remember large amounts of information is more or less socially graded than broader ranges of skills.

One argument for one-shot, high-stakes, closed-book exams is that anything more open is susceptible to parental help, and thus more likely to favour the middle class. But there are other ways to make sure that the summative assessment is solely based on the student’s work alone. Furthermore, parents support their children’s education in so many ways (conversation; providing books and computers; a quiet place to study; role models; trips) and this is unlikely to be the most important. It is worth emphasising that parents helping their children to learn is a good thing, and should be encouraged where lacking, not banned where present.

This preliminary set of points suggests that the proposed reforms of assessment will not promote skills likely to be valued in the labour market. They are likely to lead to more students failing and leaving school with nothing, and/or delaying their one-shot take, or just dropping out. This spells greater inequality in educational attainment and so in life chances.

Paying to fail?

Simon Burgess

Today, a very special education policy experiment was revealed.

In the past, policies have been introduced aiming to incentivise schools and teachers to raise educational attainment. These have been effective to a degree: our evidence shows that performance pay for teachers does raise educational attainment; competition among schools also has some impact, albeit much weaker.

This new policy incentivises students themselves. And in a break from past policies, this scheme directly incentivises students to fail their exams.  It is reported that Blackburn College will pay £5000 to each student who fails her/his exams. This intriguing new policy will certainly add to the research evidence on how (not) to raise attainment.

This issue is taken seriously in the US with a number of landmark policy experiments raising attainment for some of the more deprived and low-attaining groups in the country. At Harvard, Roland Fryer reports on the results of a large scale experiment in which students were incentivised in different ways. In some schools students were paid on results, and in some schools they were paid for activities leading towards better results, such as attendance and completing homework. The results were mixed, but the latter class of experiments were effective and cost-effective. Similarly, C. Kirabo Jackson at Cornell has shown that the Advanced Placement Incentive Program in Texas produced some very exciting results from paying 12th grade students for test-passing scores. Such students are more likely to attend college, do better when they attend college and are less likely to drop out. Similar experiments have taken place in the Harlem Children’s Zone

These experiments give us an idea of the value of a policy to incentivise student achievement. As far as I know, Blackburn’s policy is the first chance we have had to study the value of a policy to incentivise student failure.

More seriously, the idea of a commitment device is standard: something that penalises the provider if something does not work out as planned. A long warranty on a car is one way of the manufacturer raising the cost to itself of the car failing. But in a case such as studying for exams, where student effort is so hard to observe, and where good or bad luck can play such a role, what economists call the “moral hazard” problem is very severe.

There are obvious alternatives – the College could pledge to give £5000 to a local charity for every student that fails the exam. That would still appropriately hurt the provider for failure on their part, without giving marginal students a very high temptation for failing at the last.