Research and analysis

The impact of COVID-19 on 2020 to 2021 assessment arrangements

Published 20 December 2022

Applies to England

Authors

• Dr Tina Isaacs, Associate Professor, UCL Institute of Education
• Professor Roger Murphy, Professor Emeritus, University of Nottingham

Executive Summary

In November 2020 Ofqual commissioned Dr Tina Isaacs, Emeritus Associate Professor of Educational Assessment, UCL Institute of Education and Prof Roger Murphy, Emeritus Professor of Education, University of Nottingham to conduct a study examining views on the impact of coronavirus (COVID-19) on learning.

The research tracked and explored experiences of COVID-19’s impact, with an emphasis on general qualifications (A levels and GCSEs) and certain vocational and technical qualifications (VTQs) used to support progression to further or higher education such as BTECs.

The approach focused on the perceptions of year 11 and 13 pupils, and their parents and teachers, of the arrangements being put in place for qualifications in the 2020 to 2021 academic year. More specifically, the research aimed to explore insights into perceptions of the delivery and awarding of GCSEs, A levels and VTQs during 2021 in terms of practicality, validity, fairness, and differences in preparedness across learners in different circumstances, including home educated students.

The project ran alongside a portfolio of other related research that Ofqual was conducting into the impact of COVID-19. This study delved into the lived experiences of key stakeholders when quite unprecedented circumstances prevented them from completing qualifications in the way that had been anticipated.

The project mainly used a focus group approach to collect evidence. The focus group data was supplemented by some individual interviews and questionnaires.

Ofqual organised the recruitment of participants for this study and attempted to achieve a wide spread of school and college types (referred to as “centres” throughout this report) and individuals within the groups. The study also included some home educated students. Factors such as socio-economic status, region of England, centre type, and Ofsted rating were used in guiding the choice of centres to recruit.

The study included 50 remote video conference focus groups and 63 remote video conference interviews with students, teachers and parents, each focus group consisting of between 5 and 10 participants. A questionnaire for parents produced over 600 responses. The project consisted of 6 phases:

• November to December 2020
• January to February 2021
• March to April 2021
• May to June 2021
• June to July 2021
• August to October 2021

Due to the ongoing situation with the COVID-19 pandemic during the research, the issues addressed through it changed somewhat from one phase to another. Key announcements about how qualifications would be assessed and graded in 2021 were made throughout the research period, and the 6-phase research design spread over 11 months allowed the project to adjust its focus as events unfolded. This allowed the researchers to provide regular feedback to Ofqual on very specific current issues, with findings from each research phase being fed back to Ofqual usually within 10 days of data collection finishing.

Four major themes that students, teachers and parents emphasised when speaking in focus groups and interviews were:

• the diversity of COVID-19’s impact on individual students
• broader general impacts of COVID-19 on centres in different regions and sectors
• differing reactions to the proposed revisions to the 2021 qualification assessment and grade awarding arrangements.
• the evolving nature of the 2021 qualification assessment arrangements

It became apparent early in the study that the pandemic’s impact on individual learners was dramatic, unprecedented and highly differentiated. This produced a situation where the normal approaches to providing the fairest system of assessment possible for all year 11 and 13 students required considerable adaptation. Differences were noted between students pursuing more practical or vocational subjects and those pursuing more general subjects, with the former expressing more concerns about their opportunities to partake fully in rich experiences as part of their courses. Home educated students also experienced quite different challenges especially when moves were made to base 2021 grades on teacher judgements.

The provision of revised arrangements for supporting student learning varied greatly across centres. Some stated that they made quick and successful changes once remote learning became a necessity. The majority, however, stated that they struggled to do this effectively, especially in the first period of lockdown and to some extent through the entire period of this research investigation. The factors that influenced these differences related, for example, to the extent to which individual teachers were able successfully to adapt their learning support so that it was effective in a remote setting, the extent to which individual students were able to adapt to remote learning, and the availability of laptops, dependable internet connections, suitable study space and learner support from siblings, parents or others. It was also noticeable that students in socially deprived settings tended to be affected more than other students with better access to learning support, study facilities and equipment.

One of the major concerns that participants expressed was in relation to the nature and timing of announcements of decisions about changes to assessment arrangements. Some participants were frustrated that the decisions to change arrangements were announced ‘late’, which had a detrimental effect on their ability to tailor their learning effectively to the new arrangements. They also believed that anxiety levels for students, parents and teachers were unnecessarily heightened by the delays in reaching conclusions about which approaches to assessment would be followed.

In line with the comments above, considerable concerns were raised about the impact of the pandemic on students’ mental health in general, including stress, feelings of anxiety and upset over the disruption of normal social life. These concerns were expressed in all 6 phases and by all participant groups.

Nevertheless, the revised assessment arrangements put in place in 2021 were received favourably by a majority of our participants. Most were happy with the move to teacher-assessed grades, the structures put in place to guide the process for arriving at those grades, and the grade profiles that emerged from that process.

Home educated students were on the whole less satisfied with the revised assessment arrangements mainly because many were either not able to get teacher assessed grades or had much greater difficulty accessing the alternative assessment arrangements because of their situation.

Introduction

Background

In November 2020 the Office of Qualifications and Examinations Regulation (Ofqual), the regulator of qualifications, examinations and assessments in England, commissioned Dr Tina Isaacs, Emeritus Associate Professor of Educational Assessment, UCL Institute of Education and Prof Roger Murphy, Professor of Education Emeritus, University of Nottingham to conduct a study on what was then titled ‘The impact of COVID-19 on learning loss’. The study was subsequently re-titled ‘The impact of COVID-19 on learning’ (hereafter ‘the project’) in response to negative feedback received from centres around the use of the term “learning loss”.

The aim of the research was to track and explore perceptions of COVID-19’s impact, with an emphasis on general qualifications (A levels and GCSEs) and certain vocational and technical qualifications (VTQs) used to support progression to further or higher education such as BTECs. Ofqual wanted to learn more about changes in student learning that occurred as a result of the disruption to their education during the coronavirus (COVID-19) pandemic in 2020 and 2021.

The approach chosen focused on the perceptions of key stakeholders (year 11 and 13 pupils, and their parents and teachers) of the arrangements being put in place for these qualifications in the 2020 to 2021 academic year. This was done in the context of a methodology that allowed an exploration of individual and local, as well as national, circumstances during the progression of the pandemic and the various phases of lockdown and moves to remote learning. More specifically, the research aimed to gauge insights into perceptions of the delivery and awarding of GCSEs, A levels and VTQs during 2021 (the project later expanded to include the time period after the qualifications’ assessment and awarding) in terms of practicality, validity, fairness, and differences in preparedness across learners in different circumstances, including home educated students.

The project ran alongside a portfolio of other related research that Ofqual commissioned into the impact of COVID-19. Some of that work is now in the public domain, including a survey-based study looking at stakeholder responses to the Ofqual arrangements put in place during 2021 (Piggott, Gosschalk, and Webb, 2022). The distinctive contribution of this study is the opportunity that it gave those involved to delve into the lived experiences of key stakeholders during a vital year in their educational journeys, when quite unprecedented circumstances prevented them from completing qualifications in the way that had been anticipated. The experiences in 2020 when assessment and grading processes had been subject to major pandemic-related changes formed an important backdrop to the current study. Ofqual was clear in commissioning this research investigation that it wanted to stay closely in touch with what was happening on the ground, both in terms of the impact of the pandemic on the learning experiences of year 11 and 13 students preparing for 2021 qualifications, and in terms of the way teachers, students and parents were reacting to emerging ideas and arrangements for modifying the 2021 national qualification assessment arrangements.

Methodology

The project mainly used a focus group approach to collecting evidence. The rationale for this was to allow a reasonable quantity of data to be collected in a context where issues could be explored in some depth and could be related to local and individual circumstances and experiences. The student and teacher focus groups in almost every case included groups of individuals from the same centres (the only exception being where a multi-academy trust (MAT) drew from more than one centre). On the other hand, the parent focus groups were always more diverse than that, and generally included parents of students in different centres in different parts of the country. The reason for this was that Ofqual also recruited parents independently of centres through social media and parent organisations, whereas they recruited student and teacher participants exclusively through their centres. At the planning stage it was agreed that the focus group data would be supplemented by some individual interviews, and the intention of these was to allow evidence to be collected from those who could not, for a variety of reasons, participate in focus groups, and who might add quite different perspectives.

The research was conducted in 6 phases. In each phase of the research, between 6 and 20 individual interviews were conducted mostly with individuals from the selected centres, and home educated students, whose experiences and views were likely to be somewhat different from those attending the focus group sessions. We also developed a questionnaire, which covered the same areas as the focus groups and interviews, and which allowed further participation from a small number of students, teachers and parents who preferred to respond in that way. This questionnaire was also used with one parent organisation, where there was a big demand from parents wanting to participate in the research. In this case it had a dual function as it gave us additional evidence from a wide sample of parents and allowed us to select a group of individuals with known differences in experiences and viewpoints to join our focus groups.

Ofqual research staff worked closely with the us throughout the process, and the planning and conduct of the research was a collaborative exercise at every stage. We, however, retained complete independence in both the actual collection of data and the analyses of those data. Participants were offered anonymity throughout and no Ofqual staff were ever present when data was being collected from participants.

Ofqual staff helped with the recruitment of participants and took responsibility for briefing them about the nature of the research study, the ground rules for the use of data and issues such as GDPR and safeguarding. The agreed approach to recruitment was to attempt to achieve a wide spread of centre types and individuals within the groups. Ofqual employed factors such as socio-economic status, region of England, centre type, and Ofsted rating, in guiding the choice of centres to recruit into the study. Over the course of the study we heard from students, teachers and parents from all over England including deprived, average and affluent areas, from both independent and state centres, from Further Education (FE) and Sixth Form colleges, from home educated students (and their parents). Centres ranged from those with excellent Ofsted ratings to those that had recently got out of special measures. Guidance to centres, in turn, asked that they try to ensure representation from various groups including students and staff from ethnic minorities, and students with a Special Educational Need or Disability (SEND).

Despite these attempts to stratify the sample it remained in essence an opportunity sample in that it depended upon institutions and individuals being willing and available to participate in in-depth remote discussions, and in some phases of the research Ofqual experienced some challenges in recruiting participants from institutions and networks that they were able to contact. The challenges presented by the pandemic meant that Ofqual - and we - had to be methodologically flexible and responsive to ongoing changes, regulations and government recommendations.

Focus groups

Over the course of 11 months we conducted 50 remote video conference focus groups and 63 remote video conference interviews with students, teachers and parents, each focus group consisting of between 5 and 10 participants. A questionnaire aimed at parents produced over 600 responses. The project consisted of 6 phases:

• November to December 2020
• January to February 2021
• March to April 2021
• May to June 2021
• June to July 2021
• August to October 2021

Originally the project was going to focus on November 2020 through to April 2021, but Ofqual requested that the researchers continue to run focus groups and interviews through the spring and summer of 2021.

Each phase of the study normally consisted of 3 student focus groups, 3 teacher and staff focus groups, and 2 parent focus groups. If students, teachers or parents found it difficult to participate fully in a focus group, they were invited either to complete a short questionnaire or participate in a short telephone interview instead. The focus groups were a maximum of one hour long. The interview length varied, although most were approximately 30 to 50 minutes long. Both the focus groups and interviews were semi-structured, that is, there was a framework to the questions that the interviewers asked, but the conversations were allowed to be fairly open-ended and new themes could be introduced, depending upon those that were particularly relevant to the research phase that they occurred within.

All focus groups and interviews were recorded, with participants’ written consent. Due to the ongoing situation with COVID-19 during the 11 months of the research, the issues addressed through it changed somewhat from one phase to another. Key announcements about how qualifications would be assessed and graded in 2021 were being made throughout the period during which the research was being conducted, and the 6-phase research design spread over 11 months allowed the project to adjust its focus as events unfolded. Since the principal purpose of the project was to keep Ofqual continually appraised of stakeholder perceptions throughout the period, on a phase-by-phase basis, this had the considerable advantage of allowing us to provide regular feedback to Ofqual on very specific issues of the moment. It does, however, make the writing of a single amalgamated final report of a six-phase research study quite complicated. The purpose of this overview report is to document the general research approach and to highlight some of its key findings.

A dynamic action research design

As described above, this research study adopted an approach that contained elements of action research. Findings from each research phase were fed back to Ofqual usually within 10 days of data collection finishing. Those findings were shared formally with senior policy makers within Ofqual, and informally with some others with a close interest in the conduct of 2021 qualification assessments. As the research progressed, we received feedback that acknowledged ways in which the research study was being used to help inform Ofqual’s thinking on the key policy recommendations that it would need to make in the run-up to the 2021 qualification assessments.

Our reports at the end of the first 5 phases of the study were either thematically or participant-type based and were submitted in December 2020, February 2021, April 2021, June 2021 and July 2021. This final report highlights themes emerging from the 6 phases of the study and is presented thematically. It is organised around 4 major themes that students, teachers and parents emphasised when speaking in focus groups and interviews. They concern:

1. the diversity of COVID-19’s impact on individual students
2. broader general impacts of COVID-19 on centres in different regions and sectors
3. differing reactions to the proposed revisions to the 2021 qualification assessment and grade awarding arrangements
4. the evolving nature of the 2021 qualification assessment arrangements

Not surprisingly, the themes overlap somewhat – and some general issues such as student stress and mental health concerns feature as key issues that are addressed in relation to several of them.

The design of the study had considerable advantages in terms of allowing us to inform the policy-making process regularly and specifically in relation to the development of the pandemic during the 11 months and the different stages of modifying the qualification assessment arrangements for 2021. As we have noted already, it makes the drawing of overall conclusions across the six phases of the research more challenging. What we attempt to do in what follows is to highlight some key themes that emerged across the six phases and in doing that we will be drawing upon different data collected at unique moments from particular groups of participants.

Major research themes

Diversity of COVID-19’s impact on individual students

COVID-19’s impact was not consistent across regions, socio-economic status, centre type, subject learned or taught, families or individual students. We were able to speak with teachers, parents and students from a reasonably wide geographic area (in England), to people who worked and studied in both schools and colleges, to tutors and students on general and vocational courses, to teachers and students in independent and mainstream centres, the latter ranging from socio-economically secure to much less so, to students from a variety of ethnic groups, to both SEND and non-SEND students (and their teachers) and to home educated students (and their parents or carers).

In normal times, individual students have varied educational experiences for a whole variety of reasons, and these are known to be affected by social class, gender, home background circumstances, teaching provision, centre type, illness, motivation and much more. We are familiar with such differences and acknowledge that any qualifications assessment system cannot compensate for them, and that it can only try to be fair by offering equivalent assessment opportunities to all candidates. During the COVID-19 pandemic, things became much more complicated, as some of these inequalities were exacerbated and other new ones noticeably cut across many students’ preparation for 2021 qualification assessments. Among the more significant of the new COVID-19-related inequalities were:

1. • Students who were unable to engage adequately with their educational courses at times when they had to be away from their centre.
   This could be because of a lack of equipment, broadband signal strength, or home circumstances, such as caring responsibilities for other household members, or a lack of suitable quiet space where they could work.

2. • The amount of time that individual students had to spend away from their normal place of study, because of illness, illness of other family members or teachers, centre closures and partial closures or restricted timetables.

3. • The variability of the mechanisms that institutions were able to put in place when the pandemic started to have an impact in March 2020.
   Centre leaders and staff self-reported on their ability to respond to the pandemic’s many challenges. Some stated that the actions they took were quick and effective, and others stated they were far less ready or able to do that. As with the other points above, such a statement covers a great deal of complexity when one drills down to individual institutions that participated in the study, subjects studied, teachers and students. Even institutions that quickly got a high proportion of teachers and students engaging in online teaching and learning often had a lot to learn about how to make such new styles of learning effective so that students found it engaging and teachers were able to monitor student engagement and progress.

4. • The different challenges experienced by students and teachers engaged in individual general and vocational qualifications.
   We heard a lot in the early phases about how some subjects with a considerable emphasis on practical skills, such as art and design, drama, engineering, and the sciences were challenged by the move to remote learning. Although the most clear-cut examples tended to be vocational, as for example workshop-based BTEC-type courses, this was not simply a vocational versus general education distinction, as many teachers of general education courses reported difficulties in providing adequate specification coverage through remote learning. In some such areas we were told about students being set project work to do, which was not directly related to the courses that they were following, but which they were being given to keep them focused on learning during periods when they could not access relevant resources for their course requirements.

5. • Different individual students reacted quite differently when they were required to move from institution-based learning to remote learning.
   At one end of the spectrum, we were told of individual students who thrived in this new environment. Some teachers told us about students who were working longer hours and more effectively in the remote setting where they preferred more individual independent learning undertaken in a quiet environment away from busy classrooms. Such effective self-motivated and self-sufficient learners were capable of thriving in this new context, especially where they had access to good quality learning resources. On certain courses some such learners were reported to be racing ahead with coursework assignments and ending up well ahead of where they would have been had it not been for the pandemic.
   In contrast a much higher proportion of other students we spoke to fared less well when pitched unexpectedly into the world of remote learning. Depending upon the support structures that they were able to access wherever they were, during this period we heard that some students quickly became discouraged, isolated and unhappy. Teachers were from the early stages reporting higher than normal student mental health concerns and also commenting upon the difficulties they were having monitoring student engagement and welfare. Some students seemed to ‘disappear off the radar’, perhaps not attending online classes at all or in other cases attending silently with their camera off so that it was impossible to gauge their level of involvement. In some cases, teachers reported concerns about such students becoming lost or disconnected from their courses, indicating that the long-term consequences for them were much more severe than any learning deficits experienced during periods when institutions were largely closed.

6. • Home educated students tended also to have very different experiences of COVID-19 than those attending centres.
   In some ways they might be thought of as being an advantaged group, especially during lockdown periods when they already had well established remote learning arrangements. That said, they tended to feel very keenly that the alternative assessment arrangements put them at a big disadvantage in relation to other students. Many struggled to – and some stated they could not access centre assessment grades (CAGs) or teacher assessed grades (TAGs), because they did not all have teachers who could provide those for them. Even though some examination centres offered such candidates access to TAGs, this was seen as an additional type of extra provision which required the purchase of a package of assessment activities and considerable increases in costs.

7. • Concerns about student wellbeing and mental health issues were frequently reported to us throughout the period of this study.
   It is hardly surprising that the severe changes to their routines and ability to participate in normal social and recreational activities had a big impact on many students. Other research studies have looked at this broad area in more depth than we can (Banks and Xu, 2020; Impact Ed, 2020; Open Data Institute, 2020; The Children’s Society, 2020a, 2020b; Young Minds 2020a, 2020b), but in this report we are concerned to emphasise the ways in which uncertainties about qualification assessment arrangements and the possible impact of these on students’ likelihood of not getting the grades that they needed to progress with their future plans, seem to have exacerbated levels of anxiety and ill health.

All of the 7 points above impacted individual students in different ways. In our research we heard very different accounts from students following similar courses in similar or the same centres and this brought home to us how hard it was to generalise about the impact of the pandemic on individuals. Because of the focus that our study had on the impact of COVID-19 on the qualification assessment arrangements for year 11 and 13 students in 2021, it also became clear to us the challenges that faced those trying to make necessary adjustments to the assessment and grading processes in order to allow students to progress in the face of the disruption that COVID-19 caused.

Broader general impacts of COVID-19 on centres in different regions and sectors

Some highly significant regional differences were a major factor in the pandemic, as rates of incidence started to push certain regions into tiered arrangements in the autumn 2020 term that disrupted normal learning. Those differences inevitably changed over time as levels of COVID-19 in different areas and local lockdown restrictions increased and decreased. Eventually most centres were subject to the same national lockdowns and institutions generally settled into online provision for differing proportions of their students.

In our first set of focus groups and interviews we asked our participants about how much learning they believed they had ‘lost’ due to shutdowns, illness and teaching methods. In the 2019 to 2020 academic year, differences in time spent in live lessons (either in centres or online) varied greatly from centres that had only provision for one day a week or just a few hours a day, to centres that almost immediately embarked on full – or nearly full – timetables. The differences here were especially stark during the first set of restrictions, which involved mainly remote learning (March 2020 to end of summer term 2020); less so in the second period of the pandemic when centres reopened, which was characterised by Ofqual as mainly ‘new normal’ learning (autumn term 2020); and even less so in the third period of the pandemic, involving mainly remote learning again (January to early March 2021). Between early March 2021 and mid May 2021 students again experienced mainly ‘new normal’ learning. Centres had reopened but participants were still reporting being sent home for periods of time in various-sized bubbles up to the end of the study.

During all these periods of the pandemic some students continued to be provided for in their centres. Such provision was mostly offered to the children of key workers, but some centres were able to extend this to different types of vulnerable learners. Such provision was generally reported by participants in this study to be centre-supported independent study rather than normal course-specific teaching and learning provided by subject specialists.

The nature of resource and readiness varied greatly among our participants – one of the largest gaps we noticed was that independent and grammar schools were sometimes more ready to provide online lessons that featured live teaching from the outset and by and large their students were adequately resourced with laptops and tablets, either provided by the centre or by their families. We heard from centres that had already started online lessons through Microsoft Teams or similar provision at the beginning of this research (November 2020) and from those that were still struggling in the second wave of national centre closures. In these centres, we heard that additional government technical provision ^{[footnote 1]} proved inadequate to meet their needs. Some students did not have laptops or tablets at home – some resorted to using their phones for lessons – and shared technology among siblings was more common. In a few cases, we heard that children needed to share the available technology with their parents as well.

Not surprisingly, the more adequately resourced the institution, and the families that sent children to that institution, the sooner students tended to get access to technology – or already had it. We heard of institutions that routinely issued tablets to their students and had trained them to use course-related emails and shared areas before the pandemic, some in preparation for a potential lockdown, but others as routine provision, so they were able to begin teaching remotely in 2020 immediately. Households of participants that were affluent also generally had enough tablets, laptops – and quiet places to learn and study – than less affluent ones. In some cases, institutions that participated in the study were providing wake-up phone calls and meals for their students while simultaneously trying to access as much technology to share with their students as possible. This difference in provision – starker in 2020 than in 2021 – led to some centres initially using written materials such as handouts, in lieu of online resources, and when they went online the lessons were pre-recorded lectures with little interaction between students and their teachers and tutors (and between students themselves).

We frequently heard from our participants that however good the remote provision, it did not measure up to face-to-face learning. Students missed the interactions with their teachers and the ability to ask for clarification about topics in real time. They also missed interactions with their peers, citing group work, debates and projects as features that could not be captured as well online – even when the lessons were live – as in the classroom. Many of the parents and students we spoke to stated that remote learning, especially when that consisted of recorded lessons, was demotivating. Others, however, appreciated the ability to re-watch lessons. Teachers in some centres taught both remotely and face-to-face when key workers’ children came back to the classroom ahead of others or students were sent home because of COVID-19 within their learning bubble. Needless to say, those teachers found this way of working challenging.

Provision varied by type of course as well. More practical subjects – vocational courses such as, coursework-heavy courses such as art and design and music, and subjects that rely on experimentation and practicals, such as the sciences – found it challenging to cover the necessary material and provide students with resources that they could use at home. We heard from centres where lessons needed to revolve around everyday items that most students had in their homes rather than the materials to which they would have access in centres. When it came to assessment, however, (see below) some vocational courses were able to assess their students in a manner not that different from usual. For example, assessment for art and design courses has historically relied on teacher judgement as opposed to the externally set and marked assessments used in many other subjects.

Experiences of the home educated students we spoke to generally differed markedly from other students. While their learning was often not disrupted as much – most had already been learning using resources at home or had access to online lessons that were purpose-built for home study – they had more extreme challenges accessing alternative centre and teacher assessments, in both the summer of 2020 and the summer of 2021. In some cases they did not have teachers who could make judgements about either their potential (in 2020) or their achievement (in 2021) and we heard about long and sometimes fruitless searches for centres that would record any type of assessment for private candidates. As many of these students had to defer their entries for their qualifications, they had an especially keen interest in when normal exams would become available to them.

Because of the specialist issues facing home educated students, Ofqual asked us to include a number of such students and their parents or carers in our research study. We were struck again by what a diverse group they are and how different their study arrangements are, depending as they do to varying degrees on external provision from specialist tutors, tutorial colleges and examination centres. Although such students and their families welcomed their inclusion in the study, any advantages they had in being outside the mainstream system of education at a time when it was forced to move to new types of provision seemed to be countered by a sense that they were a disadvantaged minority when it came to putting in place alternative assessment arrangements. Possibly more so than any other group, they resented the total cancellation of the 2021 summer examinations, especially so as many things started to return to normal by the dates when those examinations would have been held.

The home educated students who could not access teacher assessed grades generally had no option other than waiting for normal examinations to be reinstated in order to obtain grades from courses that they might well have completed during the period of the pandemic. This they saw as being unfair and disruptive to their learning and chances of obtaining the grades that they deserved. Before it had been decided that exams would be offered in the autumn, students potentially faced a wait of 12 months or more to be assessed, and some feared that specifications might be changed and that they might do much less well than if they had been assessed straight after finishing their courses.

Differing reactions to the proposed revisions to the 2021 qualification assessment and grade awarding arrangements

From the first set of focus groups and interviews to the last, many participants bemoaned the lack of timely, clear and sufficient information from Ofqual, the government and the examination boards. Decisions about the timing of examinations or any revised assessment arrangements, exactly what their content would be, how they were going to be operationalised and by whom, and what steps would be in place to make them as fair as possible featured strongly as concerns in the focus groups held prior to the cancellation of examinations in January 2021.

After the commotion over 2020’s ‘algorithm’ driven grade awarding issues and centre assessed grades, participants were disbelieving that there was no clear set of options for general qualifications put forward for 2021, including what might happen if exams could not take place. The level of uncertainty about how plans were being made in the context of differential learning loss in the different regions of England also featured in the earlier groups and this uncertainty was reported to be adding to student anxiety and mental health problems.

Such anxiety was exacerbated by fears about what would happen with examinations in 2021, affecting students’ ability to engage with independent learning at home during lockdown as well as their ability to re-engage with effective learning when they were able to return to centres. Almost all of the people we spoke to reported unusually high anxiety and stress levels among students, teachers, parents and carers, and this was related to both the educational provision and the pressure of preparing for 2021 assessments. We frequently also heard about students becoming demotivated over time.

In January 2021, once the government’s decision was made to cancel exams – a decision made too late in the opinion of many of the people we spoke to – a different set of concerns became evident. There were considerable levels of frustration among the teachers, students and parents we spoke to about the timing of the decision-making process about alternative assessment arrangements and the way in which those arrangements were communicated, especially to students.

While most of the education professionals we spoke to understood the process – although many remained unconvinced that ‘the institution down the road’ did – it seemed to be left to teachers to explain the situation to their students among whom the level of understanding varied considerably. The fact that there was another long lockdown in the spring 2021 term only exacerbated the situation. While many of the students we spoke to expressed relief that exams were ‘cancelled’, we heard fears from students, teachers and parents about potential bias in TAGs that was only partially ameliorated by the internal standardisation of marking and grading that was put in place by centres. Throughout the study we heard a great deal about the pressures that year 11 and 13 students were under. The pandemic had a large impact on their courses and lives, in many cases, and the whole experience was made worse by high levels of uncertainty about assessment. None of that has helped with the considerable mental health issues that were reported both to us and have featured widely in other detailed research reports throughout the pandemic.

This concern over uncertainty has emerged from our study as one of the strongest issues that could inform future strategies were a similar situation to arise again. Ofqual as a public regulator has a prime responsibility to ensure that the qualifications it regulates are conducted in a way that is fair, appropriate, accurate and meaningful. In circumstances where external factors dictate that normal processes may not be possible, Ofqual naturally has a responsibility to recommend changes to those processes. In the case of the COVID-19 emergency that put huge strain on all interested parties, and it seems that the wellbeing of many students was negatively impacted by both their levels of uncertainty about what was happening, and by the timing and nature of guidance, which they were given at critical points during this emergency situation.

Whether the communication of the various stages of the creation of alternative qualification assessment arrangements could have been managed in a way that would have served the needs of those students better is for others to decide. We nevertheless heard about a great deal of unhappiness about the timing of the decisions about the changes, and also about how such changes were communicated to students as opposed to their institutions and teachers.

Evolving nature of the 2021 qualification assessment arrangements

In the autumn term 2020, teachers and students were preparing for examinations that they believed would be carried out in as normal a fashion as possible. At the time, we did hear from quite a few of the participants that they would like to see some sort of combination of external exam and centre (or teacher) assessed grades based on what students had been able to achieve between autumn 2019 and summer 2021. It was perceived that such an arrangement could take into account individual circumstances while still retaining the benefits of standardised judgements made by exam board markers for general qualifications.

In January 2021 the possibility of external judgement disappeared and ‘the exams are cancelled’ message had its own disruptive impact because of the uncertainty of exactly what would replace them. As mentioned above, the time that elapsed between the announcement about exam cancellation and the guidelines for what would constitute valid evidence of student achievement added to people’s frustrations. Many believed that the delay in provision of clear guidance and sample assessment materials had a large, negative impact. Initially many centres were working on a predicted grades basis until they were told otherwise.

For many of the participants, the fact that most of the decisions about what evidence could be used was largely left up to centres led to some disquiet. When centres were allowed to re-open in March 2021, those centres initially went about it differently. Some only invited year 11 and 13 students into centres initially. Some had truncated centre days with less instruction available. In many cases a barrage of extra assessments and exams were laid on by centres in a period of what many of the teachers we spoke to described as frantic evidence gathering after students returned from their Easter break.

Although the exam boards provided sample assessment materials for general qualifications, many teachers – as well as some students – complained to us that the materials only comprised questions from already publicly available past papers and that no new questions were offered. Some disquiet was expressed about different centres’ assessment arrangements. In some places we were told that students were given frequent short, sharp assessments. Other centres tried to replicate the usual examination experience by conducting longer exams with students, sitting them in halls. Some centres included evidence from year 9, which surprised affected students and their parents because they could not know at the time they produced the work that it would count towards their grades. Some centres used evidence from work that students created at home, others only used work done on site. Again, using home-based work unsettled some students. Some of the teachers we spoke to stated that there were more assessments in 2021 than in a normal examination year and the workload for them was much greater than usual. They also pointed out the inherent unfairness of each centre deciding on how it was going to assess its students.

There were mixed messages from the students we spoke to about assessment arrangements. Many appreciated the fact that they were given multiple opportunities to show what they had learned and that they were given the topics on which they would be assessed in advance. Some expressed the opinion that TAGs were fairer than exams because teachers and tutors were best placed to make judgements, given how well they knew their students. But others were concerned about teacher bias and expressed the view that only external examinations were reliable. Almost everyone we spoke to accepted the need to assess less content in centres that were unable to cover the whole specification, although quite a few centres were able to get through the entire specification and examined students on the full course.

When the grades were announced almost all of the students (who we interviewed as they began year 12 as well as older students about to embark on university courses, apprenticeships or employment) were satisfied with the grades they received. A few students stated that their predicted grades were higher than their actual grades and a few said their predicted grades were lower than their actual grades. No one we spoke to went through a formal external appeals process, although a couple of students said they had appealed internally within their college. None of the teacher focus groups had anything to say about the appeals process.

Despite the reservations expressed about the TAG system most of our participants had sufficient levels of confidence in it and were reasonably content with their results. Certainly compared to 2020 assessment arrangements, 2021 went more smoothly from the point of view of producing grades that were seen as acceptable. There was a mixed response in relation to students’ readiness for the next phase of education – some of the students were confident that they had learned enough to progress, but others believed that they had missed out on fundamental content and skills. Teachers and tutors were also concerned that students were embarking on qualifications less well prepared than usual. Some of the people we spoke to expressed concern that those not involved in education would not recognise the hard work that had gone into assessment in 2021 and wouldn’t value students’ grades as much as in other years, thinking that students had ‘got off easy’. However, almost everyone accepted the extraordinary nature of the past 18 months and no one was overly anxious about what the outside world was thinking, although some were resentful about the way the media reported on the situation.

Overall conclusions

The action research strategy adopted for this investigation along with the predominantly focus group approach both appear to have worked well. Together they allowed detailed information to be fed back to Ofqual throughout a highly demanding and fast-changing period of disruption to the educational experiences of year 11 and 13 students during the COVID-19 pandemic in 2021.

It became apparent early in the period that the impact of the pandemic on individual learners was dramatic, unprecedented and highly differentiated. This produced a situation where the normal approaches to providing the fairest system of assessment possible for all year 11 and 13 students required considerable adaptation.

Compared to the challenges faced in 2020, the revised assessment arrangements put in place in 2021 were received favourably by the majority of our participants, with the exception of home educated students. Most were happy with the move to teacher assessed grades, the structures put in place to guide the process for arriving at those grades, and the national grade profiles that emerged from that process.

As noted in the ‘Differing reactions to the proposed revisions to the 2021 qualification assessment and grade awarding arrangements’ section of this report, the greatest concerns we heard were in relation to the nature and timing of the announcements of the decisions about the changes to the assessment arrangements. Many of our participants were frustrated that the decisions to change the assessment arrangements were announced too late in the 2020 to 2021 academic year and this had a detrimental effect on their ability to tailor their learning effectively to the new arrangements. They also felt strongly that anxiety levels for students, parents and teachers were unnecessarily heightened by the delays in reaching conclusions about which approaches to assessment would be followed.

As also noted in the ‘Differing reactions to the proposed revisions to the 2021 qualification assessment and grade awarding arrangements’ section, considerable concerns have been raised with us about the impact of the pandemic on students’ mental health, which of course can impact directly on both their learning and the grades that they attain from national qualifications. While other research studies will delve into this subject more deeply than we can, we note that the way in which changes to assessment arrangements for high stakes qualifications are communicated to students during a disrupted year such as 2021 is a potentially sensitive area. We hope that time will be taken to review whether any key lessons can be learned from this experience about how assessment organisations and exam boards communicate with students in such a period of uncertainty.

References

Banks, J., & Xu, X. (2020). The mental health effects of the first two months of lockdown during the COVID‐19 pandemic in the UK. Fiscal Studies, 41(3), 685-708. https://doi.org/10.1111/1475-5890.12239

Impact Ed (2020). Lockdown Lessons. Pupil learning and wellbeing during the Covid-19 pandemic. ImpactEd, Impact Ed Lockdown Lessons

Open Data Institute (2020). Data about children’s lives during the pandemic.

Piggott, L., Gosschalk, K., & Webb. S. (2022). Ofqual Public Sentiment 2021. London: YouGov. https://www.gov.uk/government/publications/ofqual-public-sentiment-2021

The Children’s Society (2020a). The good childhood report 2020.

The Children’s Society (2020b). Life on hold: children’s well-being and Covid-19.

Young Minds (2020a). Coronavirus: Impact on young people with mental health needs. Survey 2: Summer 2020.

Young Minds (2020b). Coronavirus: Impact on young people with mental health needs. Survey 2: Autumn 2020. – return to school.

1. At the start of the 2021 spring term, the government increased support via the ‘Get Help with Technology’ programme ↩