Tag Archives: Incentives

Teacher performance pay without performance pay schemes

Teacher performance pay without performance pay schemes

Amid the macroeconomic gloom, the Autumn Statement contained a line about teachers’ pay. The School Teachers’ Review Body recommends “much greater freedom for individual schools to set pay in line with performance”. Consultations and proposals are expected in the near future.

But simply giving schools the freedom to do this may be a rather forlorn hope of anything much happening. It is not clear that there is a substantial demand from schools for performance-related pay (PRP) schemes that has only been thwarted by bureaucratic restrictions. It is hard to see high-powered, tough-minded PRP schemes being introduced by more than a handful of schools, not least because we have not seen large scale deviations from national pay bargaining in academies in England despite their new freedoms to do so.

If that path seems unpromising, there are other ways of facilitating a greater reflection of performance in pay, discussed shortly. But first – is PRP for teachers a good idea in the first place? Does it raise pupil attainment? What are the ‘side effects’?

This is a question that economists have produced a good deal of research on. And to summarise a lot of diverse work briefly, the international evidence is mixed. Those on both sides of the argument can point to high quality studies by leading researchers that find substantial positive effects, or no effects. In both cases, interestingly, there appeared to be little evidence of gaming or other unwanted effects of the incentives.

There is little evidence specifically for England. Our own research found a substantial positive effect of the introduction of a PRP scheme, but given the varied results found elsewhere it would seem unwise to place too much weight on this one study. The underlying performance pay scheme was poorly designed but nevertheless had a positive effect on the progress of pupils taught by eligible teachers relative to ineligible ones.

And design is key. There are many reasons why a simple high-powered incentive pay scheme might be detrimental to pupil progress, which we have discussed here and here. These include the fact that teachers have multiple tasks to do, the problems of measuring the outcomes of some of those tasks, the complex mixture of team and individual contributions, and the potential impacts on implicit motivation. The overall message is that incentives work, but schemes have to be very carefully designed to achieve what the schemes’ proponents truly intend.

There is another way to facilitate a closer link between pay and performance that does not require any school to introduce a performance pay scheme.

Published performance information in a labour market can change the way that the market rewards that performance. The critical features are first that the organisation’s own output depends in an important way on this performance characteristic of an individual; second that the organisation has some discretion in the pay offers it can make to new hires; and thirdly that the performance information is public – is available and verifiable outside the current employer. In this case, the pay structure of the market will reflect the performance rankings: high-performing individuals will be paid more.

In teaching, the first two of these three conditions are met: teacher quality matters hugely for schools, and schools have some discretion over pay. Now, suppose we had a simple, useful and universal measure of each teacher’s performance in raising the attainment of her pupils (obviously we don’t at the moment; I come back to this below), and that this was published nationally, primarily for the attention of Headteachers. The idea is that Headteachers trying to improve the attainment of their pupils would be on the look-out for high performing teachers when they had a vacancy to fill. Armed with this performance information, they might try offering a higher wage (or something else – it doesn’t have to be money) to tempt them to join their own school. Equally, the teacher’s current school may respond by raising the offer there. Over time, this process will tend to raise the relative pay of high-performing teachers relative to low-performing ones, whom no-one is trying to bid for.

This idea should not be a strange one. A number of professions have open measures of performance. Just today it is reported that performance measures for more surgeons will be made public in the summer of 2013; this is already true for heart surgeons.

It is well-known that PRP does two things: it motivates and it attracts. The outcome for pay described here will tend to make teaching more attractive to people who are excellent teachers and less attractive to those who aren’t.

There are a number of problems with this idea, though perhaps less than might appear at first glance. First, it could be argued that a performance measure derived from teaching in one school is not relevant to teaching in another school. Obviously each child and each school is unique, but it seems very unlikely that there is no commonality of context between one school and the next. Observation suggests this: teachers moving from one school to another are not counted as having zero experience, and Headteachers are often appointed from outside a school.

Second, there might be a fear that the teacher labour market would become chaotic, with everyone churning around from school to school in search of a quick gain. We have to recognise that there is substantial turnover of teachers now < http://www.bristol.ac.uk/cmpo/publications/papers/2012/wp294.pdf >. But the main point is that it does not require much actual movement to make the market work. Schools can make counter offers to try to retain their star teachers and the end result is the same – higher salaries for high-performing teachers.

Third, any measure would be noisy, partial and imperfect. Of course, all such measures are. Whether a measure is perfect is not really the question, the question is how noisy and imperfect is it, and whether it contains enough information to be useful. One advantage in this case is that the consumers of these performance indicators are the people best able to judge their usefulness and their shortcomings: Headteachers. If such metrics are not useful, Headteachers will simply ignore them; there would be no compulsion to use them. Even in labour markets with some of the most detailed and finely measured performance indicators (for example, football or baseball) there are many moves between employers that do not work out. It is worth re-emphasising that these performance measures are bound to be imperfect and incomplete, but broad measures of performance may nevertheless be very useful.

There are useful parallels to be drawn from another profession: academics. For academics, the combination of very detailed and public performance information and a context where research performance matters a great deal to universities seems to have had a substantial effect on academics’ pay.

The Research Assessment Exercise (RAE) and more recently the Research Excellence Framework (REF) have made a strong research performance very important to a university’s standing and its income. But the critical factor for academics is that an individual’s research performance is public knowledge, through very detailed recording of the impact of their research papers. Departments and universities aiming to improve their ranking seek out star researchers and attempt to bid them away with higher salaries (plus other things such as research facilities). These offers may well be matched by their current employer, but the end result is that salaries now seem to be much more closely correlated with research productivity than before the RAE/REF (I say “seem” as there does not appear to be any evidence on this, so this is casual empiricism). This is a lot of what drives many young researchers to put in very long work hours: having a paper published in a top scientific journal early in a career has a substantial lifetime payoff even in a world with few or low-powered incentive schemes. If you check out academics’ websites you will invariably see their academic output prominently displayed.

Again, an important feature is that these indices of research output are largely consumed by other academics who are aware of their strengths and weaknesses. So although they are far from perfect, they are used by precisely the people best placed to calibrate their usefulness appropriately.

If we are to go down a path of tying teacher pay more closely to performance, and yet respect the rights of increasingly autonomous schools to determine their own pay systems, then this might be an option to consider. The challenge is to devise a measure that is simple, useful and universal. It would measure the progress made by the pupils that teachers taught, it would have to deal with normal variations in performance by averaging over a number of classes and a few years, and be on a common metric. This is not straightforward, but if it gave rise to a robust broad measure of performance it could form a part of performance pay for teachers, and performance management more broadly. It could also have substantial effects on the pay of high-performing teachers.

Response to Education White Paper I: Teachers and teaching

Leave a Reply

Rebecca Allen and Simon Burgess

The focus of the new Education White Paper (WP) is advertised in the title: “The Importance of Teaching”. Teachers are rightly lauded as the most important single factor in creating a good education. The reforms relate principally to training new teachers, with additional discussion of the constraints and bureaucracy that teachers face. The White Paper calls for shifting the emphasis of teacher training from university-based to school-based training, the argument being that this is where the “craft” of teaching is better learnt, and that this will generate more effective teachers.

We believe that the WP presumes more robust evidence on this issue than actually exists. It is hard to legislate on the best way to train teachers when we are not really sure what makes a good teacher, or what effective teachers do. We need to be realistic in terms of what we know, and also in terms of the wider context around teacher development.

There are a number of prior questions that need more robust answers than they currently have to properly address this policy issue. For example: To what extent are good teachers born or made? What do effective teachers do? What motivates teachers? We discuss new teachers first and then existing teachers.

The two key issues around new teachers are recruitment and training. The research evidence suggests that the recruitment of teachers matters a great deal. This evidence can be used to design the ideal personnel policy, the ideal contract for teachers. The facts are that teachers are very different in effectiveness but that this is hard to spot pre-hire as it does not appear to be well correlated with characteristics such as degree class or subject; and that this level of effectiveness tends not to increase with experience after the first two or three years. The current teacher entry system involves making the sharpest selection before training (to be raised to a good university degree), giving training, but thereafter only mild selection: that is, most people pass their training, and then passing probation (achieving QTS) is relatively straightforward in most schools. The evidence suggests a better policy would be exactly the reverse: a much more open and inclusive approach to who can begin teacher training, coupled with a much tougher probationary policy.

It is hard to give strong advice about a model for teacher training, given only a sketchy idea of how effective teachers operate. But in practical terms, students on teacher training courses already spend about two thirds of their time in school rather than in the university lecture hall; the scope for major gains from further time in school does not seem large. Furthermore, a timely OFSTED report on initial teacher training found more outstanding university-based teacher training courses than outstanding school-based ones. The implications for schools of taking a larger role in teacher training also need some consideration, particularly given the squeeze in resources that is coming.

There are about 400,000 teachers in England, and the turnover is about 20,000 per year. So even if the average effectiveness of new teachers can be significantly improved, this will only have a marginal impact on overall effectiveness for at least a decade. Increasing the effectiveness of existing teachers offers much greater scope for rapid improvements in standards.

The counter-part to focussing initial training on schools is to emphasise and enhance training on the job, continuing professional development (CPD). The picture painted by the economics evidence suggests a model of informal, small-scale, within-school or even within-department groups would work well, with colleagues learning from the most effective teachers. Whilst CPD is discussed at some length in the WP, it has not been the focus of interest and discussion that it should be.

The broader question is why this has to be pushed towards teachers, why there isn’t much of a demand for it from most teachers. Raising the value of being an effective teacher might help fuel this demand. We know that teachers do raise their teaching effort given incentives, and it seems likely that they would also be keener to invest in their own capability to be effective. This incentivisation could be very simple and need not be personal financial gain. It could be simple pride and satisfaction from being top of a list of teachers in the staff room, or additional resources for a project chosen by the teacher, or it could be a pay bonus for the teacher.

The focus on teachers and teacher effectiveness is to be applauded. It is less clear that the right policies have been selected to enhance this.

Paying to fail?

Simon Burgess

Today, a very special education policy experiment was revealed.

In the past, policies have been introduced aiming to incentivise schools and teachers to raise educational attainment. These have been effective to a degree: our evidence shows that performance pay for teachers does raise educational attainment; competition among schools also has some impact, albeit much weaker.

This new policy incentivises students themselves. And in a break from past policies, this scheme directly incentivises students to fail their exams. It is reported that Blackburn College will pay £5000 to each student who fails her/his exams. This intriguing new policy will certainly add to the research evidence on how (not) to raise attainment.

This issue is taken seriously in the US with a number of landmark policy experiments raising attainment for some of the more deprived and low-attaining groups in the country. At Harvard, Roland Fryer reports on the results of a large scale experiment in which students were incentivised in different ways. In some schools students were paid on results, and in some schools they were paid for activities leading towards better results, such as attendance and completing homework. The results were mixed, but the latter class of experiments were effective and cost-effective. Similarly, C. Kirabo Jackson at Cornell has shown that the Advanced Placement Incentive Program in Texas produced some very exciting results from paying 12^th grade students for test-passing scores. Such students are more likely to attend college, do better when they attend college and are less likely to drop out. Similar experiments have taken place in the Harlem Children’s Zone

These experiments give us an idea of the value of a policy to incentivise student achievement. As far as I know, Blackburn’s policy is the first chance we have had to study the value of a policy to incentivise student failure.

More seriously, the idea of a commitment device is standard: something that penalises the provider if something does not work out as planned. A long warranty on a car is one way of the manufacturer raising the cost to itself of the car failing. But in a case such as studying for exams, where student effort is so hard to observe, and where good or bad luck can play such a role, what economists call the “moral hazard” problem is very severe.

There are obvious alternatives – the College could pledge to give £5000 to a local charity for every student that fails the exam. That would still appropriately hurt the provider for failure on their part, without giving marginal students a very high temptation for failing at the last.

The Browne Review and Incentives for Teaching in Universities

Simon Burgess

Incentives matter. Our research has shown repeatedly that this is true for the public sector as it is for the private sector: for teachers and schools, for doctors and hospitals and for civil servants. It is very likely also to be true for universities and those of us who work in them.

For the past couple of decades, universities have been very strongly incentivised to improve their research profiles. The evolving formats of the Research Assessment Exercise (now the Research Excellence Framework) have rewarded Departments and universities on the basis of their research output in a high powered way. This has been ferociously effective. As a whole, UK universities have vastly improved the quality and quantity of their research and now stand close to the very top of the international rankings.

One key insight is that while the RAE/REF itself is a collective Department-level incentive, this has trickled down to incentives for individual lecturers and professors. Universities keen to improve their research rating have created a “transfer market” for star researchers, and this has meant that recruitment effort, salary and respect have been focussed overwhelmingly on research ability. Young academics, wanting to get on, are aware of this and so spend their scarce time and energy on research.

This is not necessarily a bad thing – research is extremely important to a nation’s prosperity and cultural wealth. But it does mean that universities and individual academics have been incentivised to spend more time and resources on research than teaching. Does the Browne Review change any of this?

One of the less discussed points in the Browne Review is that new institutions can provide higher education (HE). Obviously, a new start-up university may find it hard to develop credibility for its degrees, but David Willetts, the Minister for universities, has floated the idea that they could teach towards the degree exams of established universities. This has worked in the past, and would give instant credibility to the degrees.

This opens up a range of possibilities. It seems unlikely that any single new institution would attempt to offer degrees across the whole range of disciplines. Instead we might see institutions offering, say, just a BSc in Computer Science, or just a BA in Spanish. This is reminiscent of the Independent Treatment Centres that transformed outcomes in health care; centres just doing cataracts or just hips. Obviously this does not provide the breadth of three years spent in a traditional university – chatting to people outside your subject, quizzing the great researchers in your field – but it would allow students to choose between these options and put a price on those factors.

Would this affect traditional universities, and alter the incentive structure for lecturers there? After all, this is where the bulk of students will be taught for the immediate future. It might. A new source of demand for talented degree teachers would raise their outside option and might force the traditional universities to pay more. The outcome depends in part on the co-production of teaching and research. Are good teachers good researchers, and vice versa, or not? What evidence there is suggests no strong correlation either positive or negative. In which case, there will definitely be an overlap in demand between traditional universities and new providers.

Of course, traditional universities will respond and make clear that their products are different, are distinctive. But they are likely to be more expensive too, and this gives students choices. There is likely to be a lot of innovation in institutional form and contracts following this path. How this will all pan out is unclear – the market for higher education is a complex one.

But it also creates a new market for talented degree-level teachers, and this may spill over into the pay and status of good teachers in universities. This in turn will encourage a re-balancing of lecturers’ effort towards teaching at the margin, and may have a greater impact on the quality of teaching in universities than any increased resources that may flow into the sector.