Tag Archives: Teachers

Teacher performance pay without performance pay schemes

Teacher performance pay without performance pay schemes

Amid the macroeconomic gloom, the Autumn Statement contained a line about teachers’ pay. The School Teachers’ Review Body recommends “much greater freedom for individual schools to set pay in line with performance”. Consultations and proposals are expected in the near future.

But simply giving schools the freedom to do this may be a rather forlorn hope of anything much happening. It is not clear that there is a substantial demand from schools for performance-related pay (PRP) schemes that has only been thwarted by bureaucratic restrictions. It is hard to see high-powered, tough-minded PRP schemes being introduced by more than a handful of schools, not least because we have not seen large scale deviations from national pay bargaining in academies in England despite their new freedoms to do so.

If that path seems unpromising, there are other ways of facilitating a greater reflection of performance in pay, discussed shortly. But first – is PRP for teachers a good idea in the first place? Does it raise pupil attainment? What are the ‘side effects’?

This is a question that economists have produced a good deal of research on. And to summarise a lot of diverse work briefly, the international evidence is mixed. Those on both sides of the argument can point to high quality studies by leading researchers that find substantial positive effects, or no effects. In both cases, interestingly, there appeared to be little evidence of gaming or other unwanted effects of the incentives.

There is little evidence specifically for England. Our own research found a substantial positive effect of the introduction of a PRP scheme, but given the varied results found elsewhere it would seem unwise to place too much weight on this one study. The underlying performance pay scheme was poorly designed but nevertheless had a positive effect on the progress of pupils taught by eligible teachers relative to ineligible ones.

And design is key. There are many reasons why a simple high-powered incentive pay scheme might be detrimental to pupil progress, which we have discussed here and here. These include the fact that teachers have multiple tasks to do, the problems of measuring the outcomes of some of those tasks, the complex mixture of team and individual contributions, and the potential impacts on implicit motivation. The overall message is that incentives work, but schemes have to be very carefully designed to achieve what the schemes’ proponents truly intend.

There is another way to facilitate a closer link between pay and performance that does not require any school to introduce a performance pay scheme.

Published performance information in a labour market can change the way that the market rewards that performance. The critical features are first that the organisation’s own output depends in an important way on this performance characteristic of an individual; second that the organisation has some discretion in the pay offers it can make to new hires; and thirdly that the performance information is public – is available and verifiable outside the current employer. In this case, the pay structure of the market will reflect the performance rankings: high-performing individuals will be paid more.

In teaching, the first two of these three conditions are met: teacher quality matters hugely for schools, and schools have some discretion over pay. Now, suppose we had a simple, useful and universal measure of each teacher’s performance in raising the attainment of her pupils (obviously we don’t at the moment; I come back to this below), and that this was published nationally, primarily for the attention of Headteachers. The idea is that Headteachers trying to improve the attainment of their pupils would be on the look-out for high performing teachers when they had a vacancy to fill. Armed with this performance information, they might try offering a higher wage (or something else – it doesn’t have to be money) to tempt them to join their own school. Equally, the teacher’s current school may respond by raising the offer there. Over time, this process will tend to raise the relative pay of high-performing teachers relative to low-performing ones, whom no-one is trying to bid for.

This idea should not be a strange one. A number of professions have open measures of performance. Just today it is reported that performance measures for more surgeons will be made public in the summer of 2013; this is already true for heart surgeons.

It is well-known that PRP does two things: it motivates and it attracts. The outcome for pay described here will tend to make teaching more attractive to people who are excellent teachers and less attractive to those who aren’t.

There are a number of problems with this idea, though perhaps less than might appear at first glance. First, it could be argued that a performance measure derived from teaching in one school is not relevant to teaching in another school. Obviously each child and each school is unique, but it seems very unlikely that there is no commonality of context between one school and the next. Observation suggests this: teachers moving from one school to another are not counted as having zero experience, and Headteachers are often appointed from outside a school.

Second, there might be a fear that the teacher labour market would become chaotic, with everyone churning around from school to school in search of a quick gain. We have to recognise that there is substantial turnover of teachers now < http://www.bristol.ac.uk/cmpo/publications/papers/2012/wp294.pdf >. But the main point is that it does not require much actual movement to make the market work. Schools can make counter offers to try to retain their star teachers and the end result is the same – higher salaries for high-performing teachers.

Third, any measure would be noisy, partial and imperfect. Of course, all such measures are. Whether a measure is perfect is not really the question, the question is how noisy and imperfect is it, and whether it contains enough information to be useful. One advantage in this case is that the consumers of these performance indicators are the people best able to judge their usefulness and their shortcomings: Headteachers. If such metrics are not useful, Headteachers will simply ignore them; there would be no compulsion to use them. Even in labour markets with some of the most detailed and finely measured performance indicators (for example, football or baseball) there are many moves between employers that do not work out. It is worth re-emphasising that these performance measures are bound to be imperfect and incomplete, but broad measures of performance may nevertheless be very useful.

There are useful parallels to be drawn from another profession: academics. For academics, the combination of very detailed and public performance information and a context where research performance matters a great deal to universities seems to have had a substantial effect on academics’ pay.

The Research Assessment Exercise (RAE) and more recently the Research Excellence Framework (REF) have made a strong research performance very important to a university’s standing and its income. But the critical factor for academics is that an individual’s research performance is public knowledge, through very detailed recording of the impact of their research papers. Departments and universities aiming to improve their ranking seek out star researchers and attempt to bid them away with higher salaries (plus other things such as research facilities). These offers may well be matched by their current employer, but the end result is that salaries now seem to be much more closely correlated with research productivity than before the RAE/REF (I say “seem” as there does not appear to be any evidence on this, so this is casual empiricism). This is a lot of what drives many young researchers to put in very long work hours: having a paper published in a top scientific journal early in a career has a substantial lifetime payoff even in a world with few or low-powered incentive schemes. If you check out academics’ websites you will invariably see their academic output prominently displayed.

Again, an important feature is that these indices of research output are largely consumed by other academics who are aware of their strengths and weaknesses. So although they are far from perfect, they are used by precisely the people best placed to calibrate their usefulness appropriately.

If we are to go down a path of tying teacher pay more closely to performance, and yet respect the rights of increasingly autonomous schools to determine their own pay systems, then this might be an option to consider. The challenge is to devise a measure that is simple, useful and universal. It would measure the progress made by the pupils that teachers taught, it would have to deal with normal variations in performance by averaging over a number of classes and a few years, and be on a common metric. This is not straightforward, but if it gave rise to a robust broad measure of performance it could form a part of performance pay for teachers, and performance management more broadly. It could also have substantial effects on the pay of high-performing teachers.

Reforming teacher training

3 Replies

Rebecca Allen and Simon Burgess

This week the House of Commons Education Select Committee published its report on the teaching profession. This post gives the main points of our evidence to the Committee.

We think of Initial Teaching Training (ITT) as encompassing both the initial training and the probationary year. How should this be set up to produce the most effective teachers who will have the greatest impact on pupil progress? ITT plays two roles for the profession – training and selection with the emphasis typically placed on the former. Both are important and neither should be neglected, but we argue that the evidence suggests that if anything, selection is the more important, and this is our focus here. An important role for selection is completely standard for any professional accreditation system in either public or private sectors.

The key argument is this: the sharpest selection should be made at the point when the evidence on ability is strongest. The final decision on who can become a teacher should be made when we have accumulated enough evidence on the candidate’s teaching effectiveness. Where is this point in teaching? The two central relevant facts are that variations in teacher effects on pupil progress are very substantial, and that the future effectiveness of a potential teacher is hard to judge from their own academic record.

We believe that the current operation of selection in ITT (tight at the beginning, negligible thereafter) is the wrong way round. Instead, we should let a broader group try out to be teachers, but enforce a much stricter probation policy based around measures of teacher effectiveness in facilitating pupil progress. Full certification and an open-ended first job would only be granted once performance data showed a teacher to be effective. The expectation would be that only the most effective teachers would make it through to full certification.

Selection into ITT is about gaining a place on a course. The difficulty faced in identifying people likely to be good teachers is very relevant here. It is very hard to tell who will be a good teacher and therefore a high degree of agnosticism would be appropriate when faced with applicants. This is certainly true for selection based on objective criteria from the applicants’ own academic records. We know that these are unrelated to teaching ability, and so should be irrelevant in selection into ITT. Beyond that, even if selectors are highly skilled at spotting potential, and it is not clear that they are, it is impractical to ask each applicant to teach a practice lesson. Therefore, selection into ITT should be very broad, with a relatively low academic entry requirement. This of course is not the situation now, nor the direction of travel of current policy. The tightening of academic entry requirements into teaching is not helpful: it will restrict the quantity of recruits and have no impact at all on average teaching effectiveness.

Graduation from ITT should also be tough. Given that much of an ITT course is now school-based, time spent in the classroom will form an important part of the assessment. Arguably the classroom experience is the key part of the course. However, in such a short space of time it will not generate sufficient data for a robust and objective view of the trainee’s effectiveness. It will nevertheless allow the trainee to discover whether teaching is for them.

Once in a job in a school, the progression to being a qualified teacher should be very different to the typical experience now. The key decision on final certification should be made after a probation period of say three years and ideally, the probation should involve classes of varying ability and year group. The period probably cannot be less, though the appropriate length of the probation would need to be analysed properly, depending on the statistical reliability of any pre-hire indicators, school-based performance data, and the cost of being wrong. This is the point when enough data is available to make a reliable judgement on the effectiveness of the teacher. There should be an expectation that not all will make it through to final certification, and indeed only the most effective should be retained. The key judgement should be a minimum threshold of progress that the probationer’s pupils make. Obviously, the measurement of that progress and the parameters of the threshold require a great deal of careful work. Like any statistical data, estimates of teacher effectiveness will never be perfect, and a good deal of evidence over a number of years will be necessary to reach a decision, but this is clearly necessary to raise the average effectiveness of the teaching profession in England.

Another innovative route into teaching is through Teach First. In some ways this is a positive development, as it allows a lot of people to try out teaching and also gives the schools which employ them an ‘out’ at the end of the two years. On the other hand, it restricts entrants based on their academic background.

It is important to see the teacher labour market as a whole, and to see how the different stages of a teacher career fit together. It seems to be very hard to fire ineffective teachers. While the regulations on this have recently changed, generating a culture that encourages headteachers to take a more proactive stance seems harder. While this may change, it may be that the best way to reduce the problem of low-performing teachers is to make it very difficult for ineffective teachers to get into the profession in the first place.

These changes would make starting out on a teaching career much more risky financially. In order to maintain the same average lifetime expected income from the profession, the pay rate of those making it through to final full certification will need to be higher. And the lower is the chance of making it through, the higher is the full professional pay.

In summary, we think that the evidence shows that the selection aspect of ITT is completely the wrong way round. Selection is tight to get into ITT in the first place, but once in, progression to full certification is normal and expected. The process needs to be more appropriately agnostic about likely teaching ability in the first place. It should also allow a broader group of people to try out teaching, but have a much tougher probation regime before trainees be given final certification. It makes much more sense to make final decisions later once more evidence on effectiveness has accrued.

Revising the Draft School Admissions Code

1 Reply

Rebecca Allen and Simon Burgess

From next week, officials in the Department for Education are going to be busy sifting through responses to the consultation exercise around the new School Admissions Code.

Two important issues in the proposed code relate to the priority given to school staff, and to random allocation. We believe that as they currently stand, these provisions will set back the goals that the Government has set for its education policy.

1. Prioritising the children of staff

Paragraph 1.33 of the code says: “If admissions authorities decide to give priority to children of staff, they must set out clearly in their admission arrangements how they will define staff and on what basis children will be prioritised.”

This suggests that admissions authorities are to be allowed to prioritise the children of staff, reversing the policy of recent Admissions Codes.

One group very likely to be included in most definitions of “staff” are teachers. For those teachers with children, this will add a new aspect to their decision on which school to seek a job at. Like many other parents, teachers will be keen for their children to attend high-performing schools.

Following the White Paper “The Importance of Teaching”, one of the leading education policy issues is how to attract the particularly effective teachers into the more challenging schools. Research evidence does not tell us whether teachers who are parents are on average more effective teachers, but there are two points to make:

This policy change will differentially increase the flow of applicants to high-performing schools. If the Headteachers of those schools are skilled at spotting effective teachers, then simply having access to a much bigger applicant pool will raise the average effectiveness of teachers the hired at those schools.
They are less likely to be novices, which is one of the few clear findings on teacher effectiveness, so in that sense alone teachers who are parents are likely to be more effective.

Given that, this policy change is very likely to work against any efforts to attract effective teachers to challenging schools, and thus set back the Government’s stated educational policy goals of narrowing the outcome gap between affluent and disadvantaged pupils.

The proposed code change will also complicate disciplinary procedures because firing a teacher from a school would also have implications for her/his children. This is likely to make it even less likely that headteachers will engage in robust performance management.

We know that any work-based privileges that are specific to particular establishments tend to cement people in that job and reduce turnover. Such privileges include health insurance, pension rights, and so on. This reduces labour mobility and typically will make the labour market less efficient. This proposed change will have the same effect in the teacher labour market as teachers will be less willing to move as it will disrupt their children’s education.

The core issue is where the most effective school staff work. It would clearly be in high-performing schools’ interests to define staff quite widely to attract highly motivated and effective governors and other staff. School leadership is key to excellent schools, ever more so as more schools become autonomous. If the children of governors were also prioritised for admission, then we would likely see an improvement in the quality of governors at high-performing schools and a decline at more challenging schools.

These points are about the effectiveness and efficiency of schools. There is also a fairness point about equity of access to high-performing schools. Some schools might define “staff” quite widely to include Teaching Assistants, lunchtime supervisors, and governors as well as teachers. In such cases, becoming one of those staff is an attractive route of access into the school, as an alternative to buying a nearby house. In extreme circumstances, individuals could offer to work for free as lunchtime supervisors, or could offer to provide goods or services to the school to become governors. While many may apply for such positions, schools will naturally choose the more able and articulate applicants, particularly those with a professional background. This will not help to foster more equal access to high-performing schools.

It may be that the provision to allow this prioritisation of staff was a way of finessing the problem of how to guarantee that the founders of free schools can get their children into ‘their’ school. If so, it might solve a problem affecting possibly 5% of pupils at the cost of the problems outlined here for the rest.

2. Random allocation

Paragraph 1.28 says “Local authorities must not use random allocation as the principal oversubscription criterion for allocating places at all the schools in the area for which they are the admission authority.”

This paragraph bans the use of lotteries as the main mechanism for resolving over-subscription across an LA. The problem of who is admitted to the high-performing over-subscribed schools is highly relevant to the Government’s goal of closing the outcome gaps between affluent and disadvantaged pupils.

The Government has stated that the primary goal of its social policy is to raise social mobility. One of the central ways in which where you were born influences your life chances is through the assignment of children to schools, governed by the Admissions Code.

Currently, the main mechanism to resolve over-subscription is proximity. It is clear that the widespread use of the proximity rule widens the socio-economic gap in the available quality of schooling. In fact, the gap in accessible school quality between rich and poor families widens by over 50% once a proximity criterion is imposed.

We can illustrate this straightforwardly, using data from the annual Census of all state school pupils in England. We can approximate the neighbourhood of a pupil as the area within 3km of her home, and calculate the average difference in the academic performance of schools in the neighbourhood of poor and rich pupils. As we would expect, more affluent families tend to live in neighbourhoods with higher quality schools. We can also use the database to estimate an approximate proximity criterion, and to look at the schools that a particular pupil could reasonably expect to be admitted to. The difference in school “quality” is now much starker, over 50% higher in fact. This is the impact of the proximity rule, and operates over and above the simple fact that rich and poor live in different places.

This shows the mean academic quality of primary school attended (measured by the mean Keystage 2 score achieved in that school) available to high and low socio-economic status (SES) families in England. ‘Neighbourhood’ is defined simply as an area within 3km of the student’s house; ‘Catchment’ is defined as schools that the student would have a very high chance of getting into based on residential location and the school ’s catchment area. Standard error bars are displayed on the levels.

Clearly some criterion has to be used to resolve over-subscription. One possibility for use in cities is a lottery. But by its very nature, a lottery ensures that places are allocated in a way that ignores social background. All those families who have applied to an over-subscribed school are entered into a ballot, and as many names are drawn out as there are free places available (that is, once looked-after children, children with special educational needs, and siblings of current pupils have been assigned slots).

It is clear that lotteries are not a panacea and are not without difficulties, as our own research has shown. However, they do offer one potentially important way of raising social mobility.

We wait to see which way the new Admissions Code turns in the end.

Response to Education White Paper I: Teachers and teaching

Where do star teachers come from?

Rebecca Allen and Simon Burgess

This Sunday sees the culmination of the National Teachers Awards weekend, with a televised presentation of prizes. This seems very appropriate – in terms of the impact on learning outcomes, hardly anything matters as much as having a good teacher. This is not an empty platitude – research shows that the effect size of having effective versus ineffective teachers is very large relative to most educational interventions. For example, in terms of higher grades achieved, having more effective teachers beats smaller class sizes.

So where do good teachers come from? Are they born or are they made? Research undertaken here and in the US can throw some light on this.

First, the evidence shows convincingly that being a good teacher does not come with experience. Student progress improves for the first two years of a teacher’s career, but not thereafter. It seems that, after the first two years at least, good teachers have always been good teachers.

Second, having the kind of intelligence that is measured by a good university degree really doesn’t matter. CMPO evidence for England shows that a teacher’s effectiveness was uncorrelated with the degree class that s/he obtained. This finding mirrors others from the US: the skills needed to be a great teacher are just different to those needed to pass degree exams. The best teachers perform well across age ranges and abilities of pupils, and are capable of showing regard for the student perspective, which highlights why cognitive skills are not particularly important.

Some recent intriguing evidence from the US correlates teaching styles with the new measures of teacher effectiveness used by economists. If supported by further studies, this offers the hope that researchers can identify how good teachers teach and emphasise these elements in teacher training. Before that time, good teachers are essentially born not made.

More generally, it seems that picking a good teacher pre-hire is hard. Writing in the New Yorker, Malcolm Gladwell asked ‘Who do we hire when we can’t tell who’s right for the job?’. He described the teaching profession as: “There are certain jobs where almost nothing you can learn about candidates before they start predicts how they’ll do once they’re hired.” US economists Kane and Staiger suggest that we need to try out four candidates to find one good teacher. Gladwell suggests that, given cognitive skills are relatively unimportant, we should lower entry standards to “having a pulse and a basic college education”. As he says: “We should be lowering [standards], because there is no point in raising standards if standards don’t track with what we care about.”

We should rightly celebrate the star teachers, even if we don’t know much about how we found them, or how to make some more.