Category Archives: Uncategorized

Who is Aiming High?

What do you hope for in life? More specifically – and perhaps more mundanely – what did you hope for educationally? Were you always intent on going to university? Determined to leave school as soon as you could?

Students’ aspirations now feature in current education policy debates. This has formed an important part of the debate on social mobility. It is generally believed that having high aspirations is an important factor behind good performance in school. Whether it then follows that it is a good policy idea to attempt to raise students’ aspirations across the board is less obvious, but that is not the subject of this comment.

How are aspirations formed? Why do some students have higher aspirations than others? To make things concrete, one of the first big decisions that students have to make is whether to stay on at school at 16. Students continuing in school open up the possibility of university and a graduate career. Finishing education as soon as possible closes those possibilities down, at least in a straightforward way.

We can see a few straightforward patterns from a simple data analysis.

We studied responses to the question: ‘What do you want to do when you are 16?’ in the Next Steps dataset. The question was asked of 14 year olds in schools in England, and we focus on the simple distinction of whether they wanted to carry on at school or not. Figure 1 shows some simple raw means of educational aspirations by ethnicity and gender.

Figure 1: Percentages Aspiring to Stay at School at 16

The first point to note is that staying on at school is by far the most common aspiration for both male and female pupils across all ethnic groups.

Within this there are, however, huge differences between the different ethnic groups. For female pupils there is a ten point gap between White students at 85% and the South Asian and Black Caribbean groups at 94 or 95%. The proportion of Black African girls wanting to stay on at school is even higher, at 99%.

There is generally a similar pattern for male students across the ethnic groups. Again, over 90% of boys in the South Asian and Black African groups want to stay on, and again the proportion of White boys staying on is the lowest of all the ethnic groups at only 73%. Black Caribbean boys display a slightly higher proportion on average, at 81%.

It is interesting to briefly note that the aspirations of the respondents in this survey reflect actual activity at the age of 16 of a previous cohort: a higher percentage of students from Indian, Bangladeshi, Pakistani and Black African heritages stay on at school than do white British students.

Very similar large ethnic differences can be found in the students’ views regarding how likely it is that they will apply to university.

What influences this decision? Clearly, a full answer to this would involve a number of approaches including at least psychology and sociology as well as economics. Research evidence, including our own, suggests that the student’s prior attainment matters (the more able have higher aspirations), and that more broadly the students’ perceptions of their talent also matters; this is called “academic self-concept”. Some aspects of family circumstances matter, for example the qualifications of the students’ parents are correlated with their child’s aspirations, and some studies find that community or neighbourhood factors also matter.

However, the most important single factor is the aspirations of the student’s parents. Parental aspirations are a very strong correlate: 89% of students whose parents want them to stay on at school express the same wish.

The overall average hides differences across ethnic groups, however. In Figure 2 we plot differences in parental aspirations and differences in the students’ aspirations conditional on their parents’ views for each ethnic group in our sample. The group percentages of parents wanting their children to stay on at school are displayed along the horizontal axis. The vertical axis shows the group percentages of students wanting to stay on specifically for the group of students whose parents also want this. The figure illustrates the great disparity in parental aspirations between White and Mixed White-Black Caribbean and the South Asian groups.

There are marked differences in parental aspirations across the groups: just under 60% of low-income White boys’ parents want their sons to stay on at school, compared to around 90% or over for Indian, Pakistani, Bangladeshi and Black African boys. There are high parental aspirations within these ethnic groups across the income distribution.

Figure 2: Student and Parent Aspirations

It seems that the passing on of high aspirations from parents to children is another dimension of the complex and multi-faceted transmission of opportunities and assets from one generation to the next.

Are school league tables any use to parents?

Leave a Reply

Rebecca Allen and Simon Burgess

Today is “school league tables” day. Performance tables are released for schools and colleges in England, reporting a number of different measures of the exam performance of their students. While much attention this year will focus on the reporting of the new “English Baccalaureate”, we ask a more fundamental question: are school league tables in general any use to parents? One of the major aims for school league tables is to support and inform parents in choosing a school for their child: but are they fit for this purpose? The answer is “yes” – we show that using school league tables does help parents to identify the school in which their own specific child will do best in her future exams.

Parents consistently rank academic standards as being one of the most important criteria for choosing a school. The performance tables provide outcome measures that are very widely reported and easy to get hold of. The idea is that parents can scrutinise the results and weigh up the merits of the local schools, considering the academic performance, travel distance, the child’s own wishes and other factors before deciding which schools to write down on their application form.

But this idea has been subject to a number of critiques. There are three main lines of argument. First, it is argued that differences in raw exam performance largely reflect differences in school composition; they do not reflect teaching quality and so are not informative about how one particular child might do at a school. Second, schools might be differentially effective so that even measures of average teaching quality or test score gains may be misleading for students at either end of the ability distribution. Different school practices and resources might be more important for gifted students or others for low ability. Third, it is argued that the scores reported in performance tables are so variable over time that they cannot be reliably used to predict a student’s future performance. After all, today’s league tables reflect last year’s students’ exams, but a parent wants to know how her child will do in five years time.

It is an empirical question how quantitatively important these points are: are league tables helpful or not? The question on academic standards that parents want answered is: “In which feasible choice school will my child achieve the highest exam score?”. We argue that the best content for school performance tables is the statistic that best answers this question.

To answer this question, we use the long run of pupil data now available to researchers. We can follow students through their years at secondary school and see how they did in the exams at the end; that is standard. But we can also use statistical procedures (details) to estimate the counter-factuals of how that student would have done if s/he had gone to a different local school. We can then ask: if families had picked schools according to the league table information available at the time, would that have turned out to have been a good choice in terms of subsequent exam performance for that specific child? Focussing on the simplest measure of the school’s %5A*-C score, the results show that while it certainly does not produce a good choice for everyone, it produces a good choice for twice as many students than it produces a poor choice for. So on average, a family using the schools’ %5A*-C scores from the league tables to help identify a school that would be good academically for their child will do much better than the same family ignoring the league table information.

So are the league tables useful for parents? Definitely. Can they be improved? Certainly. The measures included in the performance tables should be judged according to their functionality, relevance, and comprehensibility. The test of functionality is the analysis just described. A measure is relevant if it informs parents about the performance of children very similar to their own in ability and social characteristics. It is comprehensible if it is given to them in a metric that they can meaningfully interpret. In fact, none of the current leading performance measures score very well across our three criteria. We have proposed an alternative measure that performs better on these criteria. No measure can be perfect because there are important trade-offs between relevance, functionality and comprehensibility: the more disaggregate the form in which performance tables are provided (increased relevance), the less precision they will have (decreased functionality). The more factors are taken into account in describing school performance for one specific child (increased relevance), the more complex the reported measure will be (decreased comprehensibility). Any choice on the content of league table information has to make decisions on these trade-offs.

Response to Education White Paper I: Teachers and teaching

Leave a Reply

Rebecca Allen and Simon Burgess

The focus of the new Education White Paper (WP) is advertised in the title: “The Importance of Teaching”. Teachers are rightly lauded as the most important single factor in creating a good education. The reforms relate principally to training new teachers, with additional discussion of the constraints and bureaucracy that teachers face. The White Paper calls for shifting the emphasis of teacher training from university-based to school-based training, the argument being that this is where the “craft” of teaching is better learnt, and that this will generate more effective teachers.

We believe that the WP presumes more robust evidence on this issue than actually exists. It is hard to legislate on the best way to train teachers when we are not really sure what makes a good teacher, or what effective teachers do. We need to be realistic in terms of what we know, and also in terms of the wider context around teacher development.

There are a number of prior questions that need more robust answers than they currently have to properly address this policy issue. For example: To what extent are good teachers born or made? What do effective teachers do? What motivates teachers? We discuss new teachers first and then existing teachers.

The two key issues around new teachers are recruitment and training. The research evidence suggests that the recruitment of teachers matters a great deal. This evidence can be used to design the ideal personnel policy, the ideal contract for teachers. The facts are that teachers are very different in effectiveness but that this is hard to spot pre-hire as it does not appear to be well correlated with characteristics such as degree class or subject; and that this level of effectiveness tends not to increase with experience after the first two or three years. The current teacher entry system involves making the sharpest selection before training (to be raised to a good university degree), giving training, but thereafter only mild selection: that is, most people pass their training, and then passing probation (achieving QTS) is relatively straightforward in most schools. The evidence suggests a better policy would be exactly the reverse: a much more open and inclusive approach to who can begin teacher training, coupled with a much tougher probationary policy.

It is hard to give strong advice about a model for teacher training, given only a sketchy idea of how effective teachers operate. But in practical terms, students on teacher training courses already spend about two thirds of their time in school rather than in the university lecture hall; the scope for major gains from further time in school does not seem large. Furthermore, a timely OFSTED report on initial teacher training found more outstanding university-based teacher training courses than outstanding school-based ones. The implications for schools of taking a larger role in teacher training also need some consideration, particularly given the squeeze in resources that is coming.

There are about 400,000 teachers in England, and the turnover is about 20,000 per year. So even if the average effectiveness of new teachers can be significantly improved, this will only have a marginal impact on overall effectiveness for at least a decade. Increasing the effectiveness of existing teachers offers much greater scope for rapid improvements in standards.

The counter-part to focussing initial training on schools is to emphasise and enhance training on the job, continuing professional development (CPD). The picture painted by the economics evidence suggests a model of informal, small-scale, within-school or even within-department groups would work well, with colleagues learning from the most effective teachers. Whilst CPD is discussed at some length in the WP, it has not been the focus of interest and discussion that it should be.

The broader question is why this has to be pushed towards teachers, why there isn’t much of a demand for it from most teachers. Raising the value of being an effective teacher might help fuel this demand. We know that teachers do raise their teaching effort given incentives, and it seems likely that they would also be keener to invest in their own capability to be effective. This incentivisation could be very simple and need not be personal financial gain. It could be simple pride and satisfaction from being top of a list of teachers in the staff room, or additional resources for a project chosen by the teacher, or it could be a pay bonus for the teacher.

The focus on teachers and teacher effectiveness is to be applauded. It is less clear that the right policies have been selected to enhance this.

“Systemic failure” in Welsh Education

Leave a Reply

Simon Burgess

The release this week of the latest round of international comparative education results produced some fascinating results. Not least of these was the outcome for Wales, characterised by the Wales’ Education Minister as alarming and “unacceptable”.

The PISA (Programme for International Student Assessment) results derive from a standardised international assessment of 15-year-olds, run by the OECD. They show that Wales has fallen further behind since the last tests in 2006, and scored worse than before in each of reading, maths and science. Scores in Wales have fallen relative to England and are now “cast adrift from England, Scotland and Northern Ireland”. The Wales Education Minister, Leighton Andrews, described the results as reflecting “systemic failure”.

What might that systemic failure be? One leading candidate is highlighted in our recent research on accountability mechanisms for state schools. We argue that the decision in 2001 by the Welsh Assembly Government (WAG) to stop the publication of school performance tables or “league tables” has resulted in a significant deterioration in GCSE performance in Wales. The effect is sizeable and statistically significant. It amounts to around 2 GCSE grades per pupil per year; that is, achieving a grade D rather than a B in one subject. This is a substantial effect, equivalent to the impact of raising class size from 30 to 38 pupils.

Although our results are based on a study of the GCSE scores school-by-school, Figure 1 gives a very stark impression of the overall effect. Students in England and Wales were performing very similarly up to 2001, but thereafter the fraction gaining 5 good passes has strongly diverged.

We take each secondary school in Wales, and match it up to a very similar school in England. This “matching” is based on pupils’ prior attainment, neighbourhood poverty and school funding among other factors. We then track the progress (or value added) students make in these schools before and after the league tables reform, comparing the Welsh school with its English match. Our analysis explicitly takes account of the differential funding of schools in England and Wales, and the greater poverty rates found in neighbourhoods in Wales.

Why should the removal of school league tables lead to a fall in school performance? Part of the effect is though the removal of information to support parental choice of school. The performance tables allow parents to identify and then apply to the higher scoring schools, and to identify and perhaps avoid the low scoring schools. This lack of applications puts pressure on the latter schools to improve. But this is not all of the story. Perhaps as important is the simple public scrutiny of performance, and in particular the public identification of the low scoring schools. This “naming and shaming” means that low scoring schools in England are under great pressure to improve, whereas the same schools in Wales are more able to hide and to coast.

Our work has attracted criticism, including a charge of using an “ideological theory” from teacher unions . A more thoughtful critic has accused us of a “howler” in the analysis: not noting the introduction of the original GCSE-equivalent qualifications. In fact, since these were introduced equivalently in both countries they simply net out of the comparison.

Responding to our research, the Welsh Assembly Government said “wait for the PISA results”. These results are now in, and do not make happy reading. No doubt there are many factors underlying the relative performance of Wales and England, but the diminution of public accountability for the country’s schools is surely one of them.

Where do star teachers come from?

Rebecca Allen and Simon Burgess

This Sunday sees the culmination of the National Teachers Awards weekend, with a televised presentation of prizes. This seems very appropriate – in terms of the impact on learning outcomes, hardly anything matters as much as having a good teacher. This is not an empty platitude – research shows that the effect size of having effective versus ineffective teachers is very large relative to most educational interventions. For example, in terms of higher grades achieved, having more effective teachers beats smaller class sizes.

So where do good teachers come from? Are they born or are they made? Research undertaken here and in the US can throw some light on this.

First, the evidence shows convincingly that being a good teacher does not come with experience. Student progress improves for the first two years of a teacher’s career, but not thereafter. It seems that, after the first two years at least, good teachers have always been good teachers.

Second, having the kind of intelligence that is measured by a good university degree really doesn’t matter. CMPO evidence for England shows that a teacher’s effectiveness was uncorrelated with the degree class that s/he obtained. This finding mirrors others from the US: the skills needed to be a great teacher are just different to those needed to pass degree exams. The best teachers perform well across age ranges and abilities of pupils, and are capable of showing regard for the student perspective, which highlights why cognitive skills are not particularly important.

Some recent intriguing evidence from the US correlates teaching styles with the new measures of teacher effectiveness used by economists. If supported by further studies, this offers the hope that researchers can identify how good teachers teach and emphasise these elements in teacher training. Before that time, good teachers are essentially born not made.

More generally, it seems that picking a good teacher pre-hire is hard. Writing in the New Yorker, Malcolm Gladwell asked ‘Who do we hire when we can’t tell who’s right for the job?’. He described the teaching profession as: “There are certain jobs where almost nothing you can learn about candidates before they start predicts how they’ll do once they’re hired.” US economists Kane and Staiger suggest that we need to try out four candidates to find one good teacher. Gladwell suggests that, given cognitive skills are relatively unimportant, we should lower entry standards to “having a pulse and a basic college education”. As he says: “We should be lowering [standards], because there is no point in raising standards if standards don’t track with what we care about.”

We should rightly celebrate the star teachers, even if we don’t know much about how we found them, or how to make some more.

Paying to fail?

Simon Burgess

Today, a very special education policy experiment was revealed.

In the past, policies have been introduced aiming to incentivise schools and teachers to raise educational attainment. These have been effective to a degree: our evidence shows that performance pay for teachers does raise educational attainment; competition among schools also has some impact, albeit much weaker.

This new policy incentivises students themselves. And in a break from past policies, this scheme directly incentivises students to fail their exams. It is reported that Blackburn College will pay £5000 to each student who fails her/his exams. This intriguing new policy will certainly add to the research evidence on how (not) to raise attainment.

This issue is taken seriously in the US with a number of landmark policy experiments raising attainment for some of the more deprived and low-attaining groups in the country. At Harvard, Roland Fryer reports on the results of a large scale experiment in which students were incentivised in different ways. In some schools students were paid on results, and in some schools they were paid for activities leading towards better results, such as attendance and completing homework. The results were mixed, but the latter class of experiments were effective and cost-effective. Similarly, C. Kirabo Jackson at Cornell has shown that the Advanced Placement Incentive Program in Texas produced some very exciting results from paying 12^th grade students for test-passing scores. Such students are more likely to attend college, do better when they attend college and are less likely to drop out. Similar experiments have taken place in the Harlem Children’s Zone

These experiments give us an idea of the value of a policy to incentivise student achievement. As far as I know, Blackburn’s policy is the first chance we have had to study the value of a policy to incentivise student failure.

More seriously, the idea of a commitment device is standard: something that penalises the provider if something does not work out as planned. A long warranty on a car is one way of the manufacturer raising the cost to itself of the car failing. But in a case such as studying for exams, where student effort is so hard to observe, and where good or bad luck can play such a role, what economists call the “moral hazard” problem is very severe.

There are obvious alternatives – the College could pledge to give £5000 to a local charity for every student that fails the exam. That would still appropriately hurt the provider for failure on their part, without giving marginal students a very high temptation for failing at the last.

The Browne Review and Incentives for Teaching in Universities

Simon Burgess

Incentives matter. Our research has shown repeatedly that this is true for the public sector as it is for the private sector: for teachers and schools, for doctors and hospitals and for civil servants. It is very likely also to be true for universities and those of us who work in them.

For the past couple of decades, universities have been very strongly incentivised to improve their research profiles. The evolving formats of the Research Assessment Exercise (now the Research Excellence Framework) have rewarded Departments and universities on the basis of their research output in a high powered way. This has been ferociously effective. As a whole, UK universities have vastly improved the quality and quantity of their research and now stand close to the very top of the international rankings.

One key insight is that while the RAE/REF itself is a collective Department-level incentive, this has trickled down to incentives for individual lecturers and professors. Universities keen to improve their research rating have created a “transfer market” for star researchers, and this has meant that recruitment effort, salary and respect have been focussed overwhelmingly on research ability. Young academics, wanting to get on, are aware of this and so spend their scarce time and energy on research.

This is not necessarily a bad thing – research is extremely important to a nation’s prosperity and cultural wealth. But it does mean that universities and individual academics have been incentivised to spend more time and resources on research than teaching. Does the Browne Review change any of this?

One of the less discussed points in the Browne Review is that new institutions can provide higher education (HE). Obviously, a new start-up university may find it hard to develop credibility for its degrees, but David Willetts, the Minister for universities, has floated the idea that they could teach towards the degree exams of established universities. This has worked in the past, and would give instant credibility to the degrees.

This opens up a range of possibilities. It seems unlikely that any single new institution would attempt to offer degrees across the whole range of disciplines. Instead we might see institutions offering, say, just a BSc in Computer Science, or just a BA in Spanish. This is reminiscent of the Independent Treatment Centres that transformed outcomes in health care; centres just doing cataracts or just hips. Obviously this does not provide the breadth of three years spent in a traditional university – chatting to people outside your subject, quizzing the great researchers in your field – but it would allow students to choose between these options and put a price on those factors.

Would this affect traditional universities, and alter the incentive structure for lecturers there? After all, this is where the bulk of students will be taught for the immediate future. It might. A new source of demand for talented degree teachers would raise their outside option and might force the traditional universities to pay more. The outcome depends in part on the co-production of teaching and research. Are good teachers good researchers, and vice versa, or not? What evidence there is suggests no strong correlation either positive or negative. In which case, there will definitely be an overlap in demand between traditional universities and new providers.

Of course, traditional universities will respond and make clear that their products are different, are distinctive. But they are likely to be more expensive too, and this gives students choices. There is likely to be a lot of innovation in institutional form and contracts following this path. How this will all pan out is unclear – the market for higher education is a complex one.

But it also creates a new market for talented degree-level teachers, and this may spill over into the pay and status of good teachers in universities. This in turn will encourage a re-balancing of lecturers’ effort towards teaching at the margin, and may have a greater impact on the quality of teaching in universities than any increased resources that may flow into the sector.

Two observations on the National Audit Office’s Evaluation of Academies

Simon Burgess and Rebecca Allen

Much of the media comment on today’s National Audit Office’s (NAO) report on academies has rightly focussed on issues of governance and financial management. In this post, we dig a little deeper into some of the other claims in the report. We are less optimistic than the NAO – less optimistic that the academy programme has had a direct impact on the improvement of deprived and very poorly performing schools; and less optimistic that these schools are no longer avoided by the middle class.

Are academies outperforming comparator schools?

One key issue is whether academies are outperforming other comparator schools. This question – and the possible answers – illustrates the fundamental problem in policy evaluation. It is never possible to truly observe what would have happened in the absence of the policy. In this case, if a school became an academy it is simply not possible to know for sure what would have happened to it if it had remained as a community school. So researchers have to make assumptions to produce estimates of the effect of the policy. One way is to look at what happened to close comparator schools and to assume that something similar would have happened to the academy: for obvious reasons, this is called matching.

The NAO analyse the GCSE results for the 62 academies that have at least two years of post-opening exam results available. They do this by matching the academies to a set of non-academy schools that have a similar demographic profile in their pupil intake. If we take the widely used indicator of the proportion of pupils gaining five or more A*-C grades at GCSE (including English and maths) academies do indeed appear to have achieved slightly higher growth in this headline statistic than matched non-academies; this is clear in Figure 8, page 19 of the main report.

This analysis answers one specific question – have the results of these schools grown more or less quickly than schools with similar levels of deprivation. This is clearly an interesting question, but it may not be the right question. We need to remember that the original intention of the academy programme was to act as a tool to turn around failing schools. So the early academies were necessarily very poorly performing schools. The right question to ask is whether becoming an academy as part of this programme helped this growth: whether, to use the technical language, there was a causal impact of academy status on exam grades.

The issue is that the schools chosen to become academies early on were very poorly performing schools. For many such schools, the only way to go is up. In other words, a lot of the poorest performing schools would have improved anyway, regardless of academy status. We can see that easily in Figure 1. This shows the improvement in GCSE performance from 2002 to 2009 of schools in the lowest 5% of the ability intake (and by extension, the most deprived schools) and excludes all academies. This Figure suggests that “reversion to the mean” is an important component of the recovery of all poorly performing schools, and we should be cautious in assigning all of that recovery to the academy programme.

NAO show that, having underperformed similarly deprived schools in the early years of data, by 2009 they have indeed caught up and are no different in their performance on the metric of five or more A* to C grades, including English and maths. However, it is worth noting that academies are outperforming schools with a similar demographic profile on the old-style measure of the proportion of pupils achieving five or more A* to C grades, excluding English and maths. This suggests academies have been focusing on non-core subjects and NAO note that they are making greater use of vocational qualifications to boost their results than other schools. Michael Gove should therefore bear in mind that downgrading the equivalencies of vocational subjects might substantially change the perceived success of the academies programme.

Are academies becoming less deprived? And is this good or bad?

The second important point made in the report is that the academies programme is becoming substantially less deprived over time. Almost all of this change in the overall average is because succeeding generations of academies have been less deprived, rather than any individual cohort of academies filling up with less deprived children.

But we would go a little further than this. Much of the small decline in the percentage of free school meals eligible (FSM) students within a cohort of academies will be accounted for by the general growth in the economy over that period. The percentage of FSM students was declining in all schools over this time, so nothing special was happening in academies.

In order to shed a little more light on this, we look at another indicator of school composition: we use some of our own schools data on the proportion of pupils who scored in the bottom quartile in the Key Stage two tests at the end of primary school. This is not susceptible to the economic cycle. In the average secondary school, 25% of pupils will have scored in the bottom quartile but academies are more deprived than the typical school so this figure is usually much higher.

Figure 2 shows that the proportion of low ability pupils in year 7 (age 11) cohorts has indeed fallen from a very high figure of 43% in 2002/3 to 34% in 2009/10 – this is the red line. However, any particular cohort of Academies does not appear to follow a pattern of increasing or decreasing ability profile of pupil intakes. For example, the first 2002/3 cohort had 43% low ability pupils in its first year versus 45% in the latest available year of data. Only the 2006/7 and 2007/8 cohorts appear to have improved their ability profile slightly, but both are still substantially more deprived than the typical school.

One perspective on the (un)changing demographic profile of academies is that this is a success: these well-resourced and high-profile schools have not been colonised and taken over by the middle class, elbowing the poorer students out. But another perspective is that these schools remain far more deprived than average, and an increase in the fraction of more affluent students would be beneficial to students and to the school. It appears that neither of these things has occurred – the intakes remain about the same on average as when the schools joined the programme, and the typical academy does not appear to have experienced a substantial new influx of middle class students.

Using Lotteries in School Admissions

Rebecca Allen and Simon Burgess

This week about half a million students are starting their first term in secondary school. For many of their families, the process of choosing that school will have been very stressful. Is that process fair? The system of school admissions is a major topic of policy controversy, with a lot of debate highlighting the differences in access to high-performing schools.

One leading policy proposal is to use lotteries to decide who gets in to over-subscribed schools. Our paper analyses the experience so far of the only Local Authority in England to introduce this system, Brighton and Hove. Understandably, other Local Authorities have been watching the policy experiment very carefully. While the paper attracted a good deal of interest and comment, this post draws out some general lessons for other cities.

First, lotteries can work: they do equalise the chances for everyone in the lottery of getting a place in the high-performing schools. This is the great hope for the system. As long as access to schools is tied to anything that can be bought, the outcome is bound to favour wealthier families. This might be buying an expensive house near a popular school, paying for coaching in grammar school tests, or paying for music or art lessons to help the child get into specialist schools. Lotteries simply offer everyone an equal chance of access to an over-subscribed school and sever the link between access and income.

And our study shows that this worked in Brighton and Hove, within catchment areas. Two of the catchment areas each have two community schools. The lottery worked perfectly within these catchment areas to equalise chances wherever you lived, next door to the school or just within the boundary. As a consequence, the characteristics of these schools’ intakes are converging. This point was missed by some of the coverage.

However, looking across the city as a whole, covering all six catchment areas, schools did not become more similar in terms of the socio-economic characteristics of their intakes. This is chiefly because the catchment areas delineate parts of the city that are different in their degrees of deprivation. Since neighbourhood matters a lot, admissions were still largely local and across catchment areas chances of entering the different schools did not equalise.

This is not to say that nothing much happened after the reform. There were significant winners and losers in terms of the academic quality of school attended. These were predictable: under the old system, proximity to an over-subscribed school got you in; under the new system if you’re close to the school but just beyond the catchment boundary, your chance of admission is much reduced. Conversely, some families living further away from the popular school but nevertheless within the catchment area were now able to access much more highly performing schools than previously. So there has been some rebalancing of intakes in terms of access to high performing schools, despite little rebalancing in terms of the fraction of poor students in each school.

So the second broad lesson is that the design of the catchment areas is key. To a degree any Local Authority is constrained by the geography it faces – the location of schools and the clustering of deprived and affluent neighbourhoods. But there are general points that can be made. An open city-wide lottery would be globally fair: it would indeed equalise chances of access to the high-performing schools for everyone in the city. This is what we have seen within catchment areas in Brighton. But in many places a city-wide lottery would be very impractical. Apart from transport issues, it would also complicate the transition from primary to secondary school, and reduce the sense of a ‘neighbourhood’ school. We do have to acknowledge, however, that a system of neighbourhood schools almost inevitably means a segregated system.

But there are compromises between these two extremes, and these perhaps offer the best lessons for the use of lotteries to help achieve fair admissions. It is important for catchment areas to contain multiple schools, and the lottery will then act to equalise the intakes of all the schools in that area. It is important that catchment zones enclose socially mixed communities, so that all of the families in those communities will have equal chances of accessing the popular schools regardless of the location of their house. If the catchment boundaries can be set so that they cover different neighbourhood demographics, and each catchment is similar, this will yield much of the benefit of city-wide lotteries in terms of fair access.