Thursday, 30 March 2017

Nine to one - not as easy as ABC

I blogged at the Sutton Trust on the dangers of an increasingly complex accountability system

Back in 1995, I helped David Blunkett commit a heretical act – at least in the eyes of the teaching unions. With the help of the late David Frost and a closely argued column in The Times, we embraced the need for school performance tables. Yes, we would look at improvement and not just absolute results, but we would still publish both to hold schools accountable and to inform parents.

Over two decades on and we have a lot more data available to us. Admittedly some of it – the detail in the invaluable National Pupil Database – is restricted to those meeting stringent data protection rules. But parents can access a pretty good summary of how well a school is doing on the DFE website. The only problem is that it has become a lot more complex. And confused.

That confusion can only have been increased by the latest announcements from the DFE this week. This year is the first time that pupils will be judged on a new 1-9 scale, replacing the current A*-G scale. The idea is that this will allow finer judgements at the top where gaining a 9 will be a lot harder than an A* - indeed, Tim Leunig, the DFE’s chief analyst, mused to his Twitter followers that only two pupils in the country might get all top marks in the new system.

But it is not at the top that the confusion and concern has been concentrated. Rather it is at the borderline. An important feature of the new system was supposed to be an ending of the focus by schools on the dreaded D-C borderline. I’ve always been slightly bemused by this concern: after all, a C is far more impressive to an employer than a D and it is deluding young people to pretend that their E is of any use to them at all. There does, of course, need to be more focus on encouraging Bs and As, but as a minimum the C grade was a reasonable one.

And despite the introduction of Progress 8 – the hugely complex statistical measure of progress on which schools are now supposed mainly to be judged – yesterday’s news shows that the C grade remains important. Teachers have been struggling for months to understand whether a score of 4 or 5 will see them over the line in the new system.

Ministers had previously indicated that key school targets would focus on the tougher 5 grade – a good pass – but pupils who gained a 4 could be eligible for progression to the sixth form or college. On Tuesday, Justine Greening tweaked this yet again saying that the performance tables will include two pass rates – those getting a 4 and above and those getting a 5 or above – particularly for the English Baccalaureate scores.

Confused? Parents will be. But more importantly, the whole thing threatens to undermine nearly three decades of school reform. Of course, the 5 A*-C measure was not perfect. But sometimes statisticians need to recognise that perfection may not be attainable if it reduces clarity. The data was a compromise, but with floor targets and minimum standards it did a lot to drive up standards, especially in the half of secondary schools where fewer than 30 per cent of pupils gained five good GCSEs twenty years ago. The danger is all this change makes it impossible to see where improvements are being sustained.

That matters to narrowing the attainment gap as well as to social mobility, because many of the schools which were performing badly in the past had a disproportionate number of disadvantaged pupils. Their decent results have spurred them to further improvement. Progress 8 is a tough sell to explain how well a school is doing because of its complexity and because of the distorting impact of a few individual pupils Now the nine to one scale is layered on top. Comparisons over time become meaningless and past successes may appear lost. All this at the same time as many of these schools bear the brunt of cuts and changes in funding.

I’ll be honest: I was a bit sceptical about the English Baccalaureate when it was introduced, in part because of concerns that it would hurt those improved schools. But research we published last year showed that it benefited early adopter schools and improved opportunities for poorer pupils. However, the target of 90% or 100% of pupils achieving it is not realistic, and the case for a technical option remains strong. But as a way of simply demonstrating a pupil’s or a school’s success in core subjects, it has proved to be not a bad idea. And crucially it is comprehensible.

But that is not the case with these latest changes. If even the head of the exams regulator admits that parents and employers will be “confused” by the new system, and that communicating what it means will be a struggle, there are real problems ahead. And it is not just individuals and pupils that could be the losers, it is the credibility of an accountability system that has delivered real improvements in our education system.

Thursday, 23 February 2017

The data deficit effect

In my latest Sutton Trust blog, how a dearth of data in Scotland propelled a Sutton Trust report onto the front pages.
A funny thing happened with the Sutton Trust's Global Gaps report a couple of weeks ago. John Jerrim’s excellent look at the different performance of highly able 15 year-olds from different social backgrounds gained some good – but not spectacular – coverage in the London media.
But on the same day it became the top political news story in Scotland. The report included breakdowns for the four UK nations and the Trust had targeted stories at outlets in each.
The Scottish data was marginally worse than that in England – and crucially it showed that science results had dipped over the last ten years significantly – but this was enough to create front page splashes in some papers and much bigger stories in Scottish editions than in their English counterparts.
Crucially, too, the opposition took the data and ran with it. The two year gap in performance between poor and better off teenagers hit a nerve, and fed a narrative that the Scottish government has been failing on education. So much so that both Ruth Davidson, the Scottish Conservative leader and Keiza Dugdale, the Labour leader, majored on the report at First Minister’s Questions.
That took the story into a second day of front page news and saw the BBC’s Scotland political editor filing a lengthy report for the evening news bulletins. By the time last Thursday’s Question Time was broadcast from Glasgow the story was still fresh enough to warrant a separate discussion.
I’ve been reflecting on why this happened. There were some strong political reasons. Opposition politicians clearly leapt on the report with a vigour long lacking in their London counterparts, and that certainly gave the story more legs than had it been solely a Sutton Trust press release and report.
Education is also a much bigger issue in Scotland, both because Nicola Sturgeon and her education secretary John Swinney have made narrowing the attainment gap their big issue in this term, which means that any signs of failure get seized upon.
But I think another factor is just as important – the data deficit North of the border. I became acutely aware of this when I served last year on the Commission on Widening Access in Scotland. The dearth of data was the main reason I subsequently commissioned researchers at Edinburgh to produce the Access in Scotland report for the Sutton Trust.
At school level, this data deficit is particularly significant. Swinney is now introducing a more rigorous – if controversial – testing system this autumn. Scotland scrapped national testing in the mid-2000s, along with Wales. The result was predictably disastrous in Wales, which has been edging back towards testing, and the PISA results suggest it saw a slide in Scottish results too.
Potentially the reintroduction of national testing could do a lot for research into social mobility in Scotland, something the critics of testing often wilfully ignore, as well as ensuring that aspirations for able disadvantaged students are stretching.
Combined with the introduction of a Scottish version of the Teaching and Learning Toolkit, currently being developed by the Education Endowment Foundation with Education Scotland, this could have a genuinely beneficial impact on less advantaged pupils’ results.
Contrast the dearth of data in Scotland (and Wales) with its abundance in England. The National Pupil Database is an invaluable resource with the potential to improve social mobility as it shows schools how others succeed in similar circumstances and with linkage to other databases including HMRC it allows researchers to measure how well students from different backgrounds progress from the start of school to the workplace.
PISA is useful for its comparability in that respect, but is not sufficient – hence the excitement surrounding our recent report. Gratifying as it was to have such great coverage, I look forward to the day when such data doesn’t cause so much of a stir in Scotland because there is much more data available on the progress of Scottish children – and teachers have the tools to compare their pupils with similar pupils elsewhere in the country.

Friday, 20 January 2017

Breaking the class ceiling

I wrote this for the Sutton Trust blog....
Education Secretary Justine Greening recalled yesterday how she’s missed out on a banking job because she hadn’t taken a gap year. “I was too embarrassed to admit that I simply couldn’t afford one,” she told an event organised jointly by the Sutton Trust and PriceWaterhouseCoopers on Wednesday.
Outlining her vision for social mobility, she admitted that she was fortunate to get a job at PWC and to progress to become an MP and a cabinet minister despite her modest beginnings. And she was perhaps fortunate to face that particular mobility barrier as the guilty bank was Barings.
As the first comprehensive educated Conservative education secretary, Justine Greening has shown an admirable determination to place social mobility at the top of her political agenda. Yesterday she announced funding for new research schools in her flagship social mobility programme of Opportunity Areas. The new schools will be run by the Education Endowment Foundation with the York-based Institute for Effective Education and will help transmit evidence on what works across other schools in their locality to address educational inequalities.
Today’s GCSE results show some signs that disadvantaged students are doing better in school – more are doing the EBacc than before and English and Maths results are improving. But the gap in attainment in the core subjects remains stubbornly high and the new Progress 8 measure underlines just how far behind many disadvantaged students are even allowing for where they started. Those gaps are still much more pronounced outside London.
The extent of the challenge was laid bare on Tuesday in a new report, Class Ceiling, from the All-Party Parliamentary Group on Social Mobility. The APPG, for which the Sutton Trust acts as the secretariat, took as its starting point the Leading People 2016 report last year which showed how across most major professions over half of all the top jobs are taken by those who went to private schools, and many were also Oxbridge graduates.
A lot of the coverage on the APPG focused on the call for a ban on unpaid internships, something the Trust has also called for in the past linked to research on the cost working without pay. Justine Greening was instinctively against a ban when questioned about this yesterday. But unless firms that hire people for months unpaid start to pay at least the minimum wage these opportunities will remain beyond those unable to access the Bank of Mum and Dad or with a family home near their workplace.
The APPG’s recommendations, based on evidence from a host of professions over the last six months, also urged fairer and more transparent recruitment practices by employers, including contextual practices that place attainment and successes achieved in the context of disadvantage, including underperforming schools and less advantaged neighbourhoods.
They argued that employers should be conscious of the impact of recruiting from a narrow pool of universities in the graduate ‘milk round’, and the social mix of institutions, building on the work already being done in some elite professions.
This is not without controversy, as some rather excitable Daily Mail coverage showed, wrong suggesting that employers should ignore qualifications and ban all internships. In fact, as with similar programmes in universities, this is about recognising that an able young person who went to a tough school and got good results will have had to show far more grit and resilience than a pupil who went to a fee paying school.
However, this doesn’t mean that they necessarily have the same social skills. And this remains a challenge. Our research has also shown that not only do privately educated graduates earn more than those with similar degrees who went to state schools. It underlines the importance of developing those skills and school and university, particularly for those the Education Secretary likes to call ‘rough diamonds’.
It is great that social mobility is now so high up the government’s agenda. And there are clearly lots of things schools need to do to improve opportunities for disadvantaged young people, not least for those whose ability shines at eleven but isn’t properly harnessed through secondary school.
But this is not just an agenda for schools. It is about what business and universities do to foster and develop talent – and to remove the financial and social barriers that prevent success