Simon Burgess (University of Bristol)
Shenila Rawal (OPERA)
Eric Taylor (Harvard)
Most people can remember an inspirational teacher, someone who ignited their interest in a topic and helped them make rapid learning gains during their time together. Many people can also remember a dull turn-off teacher, demotivating them completely. The sense that teachers are very different in their professional abilities, and that teachers matter immensely for learning, has been confirmed and quantified by researchers over the last decade or so. Pupils with highly effective teachers make dramatically more progress in their studies, gain better qualifications and earn more in the labour market.
The natural follow-up question is: what policies or practices will increase teacher effectiveness, and raise skills all round? But so far, such practices have proved elusive. Strong, causal evidence of student learning gains from adjusting different aspects of teacher practices, recruitment, and contracts is thin on the ground. The situation can be summed up: clear evidence of the importance of teachers, absence of any proven school management tools to increase it.
This is the issue that we address in our paper. Teachers teach, but they also learn, and the process by which teachers improve their effectiveness is at the centre of our analysis. We made a relatively low-key intervention in the operating practices of 82 large secondary (high) schools in England, over two years. The intervention was focussed on teachers of maths and English for the two GCSE years, and covered over 28,000 students and around 1,300 teachers. With funding from the Education Endowment Foundation, we introduced a system of no-stakes teacher peer observation. The intervention was run as a randomised controlled trial – schools that volunteered to take part were randomised into ‘treatment’, the teacher peer observation, or ‘control’, business as usual. Teachers from the school visited each other’s classrooms and observed their peers teaching. Teachers doing the observing were given a structure, a set of questions, to base their observation and assessment on; this was not simply sitting at the back of a colleague’s class with a pen and pad. The idea was that the observer and the observee would later meet and talk through the session.
The peer observations were formally no-stakes: they did not affect pay, promotion, dismissal or any aspect of a teacher’s contract; in practice there may have been some “low stakes” in the form of social pressures or career concerns. Crucially, at set-up we randomised the roles in this process. That is, which teachers were to be observed and which were to do the observing was not based on rank, on experience, or on previous job performance, but simply random.
Did it work? Yes – we find statistically significant evidence that the intervention meaningfully improved student achievement in maths and English GCSEs. Students in treatment schools scored 0.076 student standard deviations higher, on average, than their counterparts in control schools. That is similar to the difference between having a teacher with five years of experience instead of a novice teacher.
While schools did fewer observations than we had suggested, most treatment schools completed at least some peer observations. Using the appropriate statistical techniques to take account of the fact that schools varied in their implementation of the programme, we estimate that the impact of the scheme on the treatment schools which took at least some part was higher still, at least 0.097. These results represent educationally and economically meaningful improvements in student learning. Given that this is a cheap intervention, this is particularly valuable.
The experiment also allowed us to identify which teachers benefited from the observations. If the effects we see are due to the (slight) additional pressure on the observed teachers to prepare more and to “perform” better, then we would expect only the students taught by observees to benefit. This seems to be the general assumption made by schools (and maybe other institutions more widely) when they undertake any job evaluations. Typically, schools (or school systems) will either outsource the evaluations to external providers, or it will become just another task for the school senior leadership team to find time for. Both these approaches imply the belief that the only gainers are those observed; for the observers, it’s just a task and does nothing for their own teaching effectiveness.
Our results emphatically reject that view. We find that the students taught by teachers doing the observing benefit as much as the students of observees (in fact, we find the observers’ students benefit more but the difference between them is not significantly different). How can this be? The most likely explanation is that they themselves learn something from watching: maybe something seemingly quite minor such as an idea for dealing with a particular situation; or perhaps something much more substantial like a different approach to teaching a topic. Taking the idea further, it is interesting to wonder why even quite a brief exposure to other teachers’ methods can be effective. It may be that despite recent trends to greater openness, teaching is still to a degree a “closed door” profession in which the actual doing of the job is done for the most part with no other adults present.
Of course, to some degree, most schools do some lesson evaluations, particularly for apprentice teachers. But we think our intervention has some key elements that distinguished it from the “business as usual” procedures in control schools. These are: the observation process has a clear and strong structure to guide the observer; the observation is explicitly about “continuous improvement” not about judging for pay or progression; the observation is done by peers, not by management nor by outsiders; and the observations are reasonably frequent, not just once every few years. While clearly more work needs to be done to validate or amend the key parameters, we feel that peer observation is a very promising addition to schools’ management of their teachers in pursuit of greater student achievement.