On comparing poets and computer nerds
Glenys Stacey's speech to Ofqual's Inter-Subject Comparability Conference, 2016.
Welcome everyone and thank you for attending today’s event.
Having been the Chief Regulator of Ofqual for five years, and as I am poised to leave the organisation as my term of office comes to an end, I have been reflecting on the things that have occupied Ofqual in that time, and why – why those things and not others.
Sometimes we have been reacting to events: exam paper errors in 2011, the Daily Telegraph’s teacher seminar exposure a little while later; one exam board stretched to complete its marking in time in summer 2014. Regulators always have to respond to events, and indeed they do their best to anticipate likely events and reduce the risk of them actually happening.
In other cases we have been driven to do what we have done by public perceptions, and public confidence in qualifications and results, or the lack of it. Often enough, regulators’ determine priorities with a keen eye to public confidence, and trust, and quite properly so. So the work we have been doing with exam boards since 2010 to oversee awarding, to be as confident as possible about exam results each series and each summer, and to explain those results and what lies behind them – that seemed to us necessary in order to build public confidence, faith and increasing trust in those results.
We are concerned about public confidence and of course teacher confidence, where we aim to recognise and respond to teacher concerns. That drives much of our work on the quality of marking, for example. And then fairness – regulators can find themselves between a rock and a hard place when faced with competing considerations relating to fairness, in our case fairness to students – and they must act, and find and do the fairest thing. Here I think of 2012, and unexpected GCSE English results for what turned out to be unexpected, unprecedented reasons.
And finally of course, we respond to government policy, we implement government policy and so we find ourselves living through and delivering, with exam boards, a significant programme of qualifications’ reform. On casual reflection then, it seems that most of our work has been driven by events, or public concern, or government policy – and one might worry in my shoes that in retrospect, we are too often in reactive mode.
But looking more closely, we have done more purposeful work as well: our international comparison of A levels for example, where we found that A levels stack up well when compared to other international qualifications; our detailed and definitive analysis of how exam marking has developed and how it works in practice now; and – the reason we are gathered here today – the work we have done to consider how different subjects compare.
And it is a genuine pleasure for me and for other Ofqual folk to be here and to talk with you about what is a significant, longstanding, enticing and, to some, intractable set of questions: how do subjects compare; how should one compare subjects; can one fairly compare any two or more subjects – say chemistry and media studies - and should qualifications in subjects differ in relative difficulty and if so to what extent, and what is to be done about it anyway, if anything?
I recollect my first tentative conversations about this with various members of our Standards Advisory Group a good while ago. This group of wise and experienced qualification and assessment experts often chew the fat with us at Ofqual as we consider complex or finely balanced matters. They enjoy it, I think, and they unerringly give us wise advice. So I was interested early on in their individual views on inter-subject comparability.
They had different and contrary views of course, as I have come to expect (and respect) from those steeped in assessment - and indeed people generally do have contrasting views about inter-subject comparability and what if anything should be done about it - but the first comment to me on the matter by one member of our Standards Advisory Group, Professor Paul Black - has stuck with me: he said ‘Glenys, we have computer nerds amongst us in the world, and we have poets also. How do the two compare, and how do you compare the two?’ Hmm. Tricky. And I thought he had elegantly encompassed not just subject comparison itself, but the motivations, preferences and differing innate characteristics of learners, in what he said. Computer nerds and poets – how do they compare?
Increasingly this is more than an intellectual or esoteric question. Society, government, higher and further education institutions all place increasing reliance on results and on individual grades and what they represent by way of specific subject understanding, and what they might imply about an individual’s potential ability in other subjects yet to be studied by the individual. Language teachers dismay at the seemingly unstoppable reduction in the take-up of modern foreign languages at A level, and question whether it is because potential students think it harder to get top grades in languages and so play safe and study other subjects instead. Government expectations for school performance, as measured by graded results are greater than ever. So, where does the truth lie?
The speakers we have here today are here to inform your thinking on where the truth lies. As is so often the case in the work that we do, what looks like a simple matter is actually remarkably complex and the truth is in some respects remarkably counterintuitive. By one measure, media studies IS harder than chemistry. To get things started, we published a series of papers on inter-subject comparability in December, and I hope that most of you have had a chance to have a look at those and that they have stimulated your thinking ready for the discussions today.
A word of reassurance. We are not about to do anything rash with subjects. Instead, we want to talk. We want informed, constructive conversations here and elsewhere. We want to chew on it, as they say in the States. We have a questionnaire for you to complete after today, by way of beginning. What you and our speakers say today and in response to the questionnaire will inform our thinking. It will give us one set of views as we consider over the coming months inter-subject comparability and whether any changes of any sort are desirable. A long period of conversation, discussion, careful deliberation.
Please do give us your feedback, not only on how useful today is to you, but also by completing our inter-subject comparability survey. A link to that survey will be included in our feedback questionnaire that you will be sent by email at the end of today.
I do hope you find the subject of inter-subject comparability just as fascinating as we do. Thank you for your time, and I hope today proves to be a productive and enjoyable exercise for us all.