Research and analysis

​​Delivery and award of vocational and technical qualifications in 2022​

Published 15 December 2022

Applies to England

Introduction

The academic year 2021 to 2022 marked a return to large-scale exams and formal assessments in vocational and technical qualifications (VTQs) and other general qualifications for the first time since 2019. This was also a period where Ofqual’s responsibility for apprenticeship end-point assessment (EPAs) grew significantly as the majority of EPAs transitioned into the safety of regulation.

Government expected exams and formal assessments for VTQs to go ahead in this period as the fairest way of providing results for students. The government recognised, however, that students who would be taking exams and assessments had experienced significant disruption to their education and that these students may have also needed continued support in the face of any further disruption.

Ofqual worked with the Department for Education (DfE) to launch a joint consultation on the arrangements for the assessment and awarding of VTQs and other general qualifications in the academic year 2021 to 2022. Following this consultation, DfE confirmed its policy position that exams and other assessments should go ahead. Ofqual confirmed details of its rules, the Vocational and Technical Qualifications Contingency Regulatory Framework (VCRF), which gave awarding organisations flexibility to successfully award hundreds of thousands of VTQ certificates during 2021 to 2022.

The VCRF applied to all qualifications except for GCSE, AS, and A level qualifications, AEA, Project qualifications and EPAs. It required awarding organisations to only award qualifications based on evidence from exams and other formal assessments taken after 1 September 2021 but permitted them to make adaptations to their qualifications and assessments, and to carry forward any adaptations from academic year 2020 to 2021 where they were still necessary and appropriate.

With the agreement of the Institute for Apprenticeships and Technical Education (IfATE), awarding organisations were able to put in place flexible approaches to apprenticeship EPAs during the coronavirus (COVID-19) pandemic. From 1 April 2022 those flexibilities were governed by IfATE’s flexibility framework. Ofqual released guidance to all awarding organisations delivering EPA to clarify how the flexibility framework aligned with the regulation of EPAs.

The delivery of all exams and assessments was possible because of the substantial effort of everyone involved. In most cases, awarding organisations swiftly implemented adaptations to mitigate the disruption caused by the coronavirus (COVID-19) pandemic to teaching and learning, provided appropriate support to training providers, teachers and exams officers, and issued valid and reliable results to enable students and apprentices to progress. Teachers and support staff helped the smooth delivery and adaptation of exams and assessments, and students and apprentices demonstrated incredible resilience and commitment to learning and assessment during the pandemic.

This report is in two parts. It describes the steps Ofqual took to oversee awarding organisations’ delivery and award of qualifications in the year 2021 to 2022. It also describes Ofqual’s approach to the regulation of apprenticeship EPAs from April 2021 to August 2022 and brings regulation of EPAs into alignment with all other VTQs. As with any delivery of assessments, there were issues to manage. This report describes some of the issues that occurred across the full breadth of the VTQ and EPA landscape and how Ofqual and awarding organisations responded to them.

Individual awarding organisations are responsible for the safe delivery of their qualifications. Awarding organisations had to manage and report to Ofqual any issues that arose in the delivery and award of their qualifications. Ofqual monitors the actions awarding organisations take and intervenes where it is necessary to ensure fairness for students and apprentices, or to protect standards and public confidence.

Two particular issues stood out in summer 2022.

First, just under 21,000 results were missing or incorrect on the respective results days for level 3 (18 August) and level 2 (25 August) VTQs. Our priority in August was making sure those results were corrected and issued quickly so that students could progress to further or higher education. We then launched a review and an investigation into the systems and processes used by 2 of the awarding organisations: Pearson and OCR. That work is ongoing, but we have already committed to a number of actions we are taking to protect against late results in 2023. The data findings on late and inaccurate results are contained in this report. Our analysis of the root causes behind the late and inaccurate results of 2022 will be published in 2023, alongside recommendations for further streamlining and efficiency in the delivery of awarding VTQs.

Second, there were problems with the quality of the core assessments in the Technical Qualification (TQ) for the new T Levels in Health and Science. Students received revised results for their core assessment, based on the higher of their issued overall core grade or the grade for their employer-set project. We have already taken action against the awarding organisation, NCFE, and we continue to monitor the situation closely.

After results are successfully issued, Ofqual evaluates the root causes of each issue that occurred during delivery, its impact and how effectively it was managed. Ofqual decides if any regulatory response is necessary. Specific issues are followed up with individual awarding organisations and the focus of ongoing monitoring is determined. Where appropriate, we conduct additional work to understand how to minimise the likelihood of certain types of issue from reoccurring.

Part 1 – Delivery and award of VTQs in 2022

Background

Part 1 of this report covers all qualifications (except for GCSE, AS, A level, Project, AEA qualifications and apprenticeship end-point assessments) awarded between 1 September 2021 and 31 August 2022. During the period covered by this report, Ofqual regulated 223 awarding organisations offering over 14,000 regulated vocational, technical, and other general qualifications.

These qualifications are delivered and taken in colleges, schools, training providers and employers (referred to as ‘centres’ in this report). The qualifications cover a wide variety of subjects or sectors and are assessed through timetabled assessments, on demand assessments, centre-set or marked tasks, practical activities, the production of portfolios or a combination of these.

Some of these qualifications, such as VTQs that are included in performance tables, are taken instead of or alongside GCSEs, AS and A levels. Others, such as Functional Skills qualifications (FSQs) and English for speakers of other languages (ESOL), are used more flexibly, including for progression to further or higher education or progression to employment. Many more VTQs test occupational competency or are used as a licence to practise.

From October 2021 to September 2022, awarding organisations issued a total of 4.9 million certificates, a 6% increase on the 4.6 million certificates issued from October 2020 to September 2021 (for more detail see Ofqual’s vocational qualifications dataset). As an illustration, in spring and summer 2022 alone, awarding organisations worked closely with centres (schools, colleges, training providers, employers or other exam centres) to issue over 974,000 qualification results to students taking:

• Functional Skills qualifications (FSQs)
• Core Mathematics qualifications
• VTQs approved for inclusion in DfE’s performance tables, such as Applied Generals and Tech Levels

In addition, awarding organisations issued just over 5,100 results for the Core and Occupational Specialism components of the Technical Qualifications in T Levels^{[footnote 1]}.

Chief Regulator’s listening tour

Ofqual’s Chief Regulator, Dr Jo Saxton, visited schools and colleges across the length and breadth of the country, from Blackpool to Plymouth, to hear directly from students, teachers, and school leaders. Discussions included a return to exams and formal assessments in the year 2021 to 2022.

Dr Saxton spoke to more than 100 senior staff and more than 200 students from more than 60 schools and colleges. These visits allowed the Chief Regulator to hear feedback directly from students and apprentices and sits alongside a range of school and college visits undertaken by other colleagues across the organisation.

Students across the country consistently said they wanted to sit exams and assessments. They wanted the chance to show what they know, understand, and can do. T Level students spoke about the value they saw in this new qualification, and the opportunities it opened up for them, while other VTQ students emphasised the flexibility of assessments as key to enable them to progress. The feedback received supported Ofqual in making decisions with a firm commitment to regulate in the interests of students and apprentices of all ages. Their feedback also helped us understand additional advice students, teachers and leaders would like, and led us to commission materials on how to revise and manage exam preparation, for example.

Phases of delivery

Awarding organisations took a variety of approaches to safely deliver results in the academic year 2021 to 2022, using the adaptations that were most appropriate for their qualifications. With assessments returning to normal, all awarding organisations went through 4 broadly similar phases:

1. Planning
2. Delivery
3. Grading
4. Results and post results

We have structured this report to reflect these phases.

Phase 1 – Planning

Supporting the sector

Ofqual expected that awarding organisations should, where possible, work with other awarding organisations and within their sector, industry or qualification type (for example FSQs) to decide if a common approach to adapting and awarding qualifications could be achieved. Where a common approach was agreed, awarding organisations should have complied with it where possible and appropriate.

Throughout the year, we met with groups of awarding organisations at fortnightly policy advisory groups and technical working group meetings to discuss policy development and to monitor awarding organisations’ delivery of assessments and adaptations as well as their communications with centres and the wider sector.

Ofqual asked awarding organisations to work with their centres when determining any adaptations, so that they did not impinge on the work that centres were doing to mitigate disruption, such as blended learning or education recovery activities. To promote consistency in their approaches to adaptations, and to discuss how they worked with centres and students, Ofqual regularly brought awarding organisations together for briefings and working group discussions.

Ofqual also met with centre representative organisations such as the Association of Colleges, and with individual schools and colleges and training providers via monthly centre reference groups, to gather feedback about communications from awarding organisations, adaptations, and preparation for results. Where appropriate, Ofqual forwarded the feedback to awarding organisations so that they could address any issues swiftly. We provided a centre guide for the awarding of VTQs in the academic year 2021 to 2022, in which we outlined what we expected from awarding organisations and what centres and their students should expect to happen.

For students, Ofqual published a collection of resources, links and documents aimed to help them prepare for their exams and assessments, including a series of blogs on how to manage exam pressure. We published a comprehensive student guide to explain what was in place to support students before, during and after assessments. During the approach to results days, we produced blog posts so that students knew what to expect, and also worked with UCAS to provide a letter to students receiving results with information about next steps.

Entries

Students take assessments that are internally assessed (that is, typically set by awarding organisations and marked by centres), over a period of several weeks. There is usually no entry for that assessment required in advance. For assessments that are externally assessed (that is, set and marked by awarding organisations), entries are made in different ways.

Many VTQ external assessments are timetabled and must be taken at a specific date and time, such as those VTQs used in school performance tables and Technical Qualifications in T Levels. In these cases, awarding organisations usually require entries several months in advance so that they can plan for delivery at scale. Awarding organisations use entry data to inform significant delivery decisions, such as how many question papers to print and deliver, and how much resource to allocate to the marking of scripts and the support of centres and students.

Other VTQ external assessments such as those used in FSQs are on demand, giving students the flexibility to take an assessment whenever they are ready. It can take as little as 3 days between a centre entering a student for an online assessment to that student sitting it. While this model provides flexibility to centres and students, awarding organisations are less able to anticipate when a student intends to certificate for a qualification. The awarding organisations would not necessarily know when a student started their course, and the student’s entries for different assessments within that qualification could be made months apart.

In summer 2022 alone, there were around 589,600 entries for external assessments for nearly 380 VTQs in performance tables. In the academic year 2021 to 2022, there were 3,490 entries for 10 Technical Qualifications in T Levels.

Adaptations

For the academic year 2021 to 2022, Ofqual required awarding organisations to award qualifications based on evidence from exams and other assessments^{[footnote 2]}. Awarding organisations were permitted to make adaptations to their qualifications to mitigate disruption caused by the pandemic to teaching, learning and delivery of assessments where it was appropriate to do so.

Awarding organisations could make a wide range of adaptations, from allowing alternative conditions under which assessments could take place, to adapting assessment methods. Ofqual did not prescribe a single approach for awarding organisations to deliver or award their qualifications. Our role was to monitor whether the awarding organisations’ adaptations enabled sufficiently valid and reliable assessments.

For VTQs in performance tables, Technical Qualifications in T Levels, and FSQs, DfE defined the scope of adaptations that awarding organisations could implement and DfE reviewed awarding organisations’ adaptation plans for VTQs in performance tables, to check whether they were in line with current policy. Where necessary, Ofqual challenged and tested awarding organisations’ rationale for their decisions, which led to them revising or improving their approach. For Technical Qualifications in T Levels, Ofqual met with awarding organisations in technical groups to discuss their plans for adaptations and to check progress. For FSQs, Ofqual monitored awarding organisations’ continued implementation of adaptations from the previous academic year^{[footnote 3]}. This allowed for consistency in adaptation approaches across similar qualifications.

For most of the remaining qualifications within the scope of the VCRF, Ofqual collected data from awarding organisations on the different types of adaptations they would apply to different qualifications as well to the assessments within a qualification. Ofqual worked with the Joint Council for Qualifications (JCQ) to define the common types of adaptations to promote consistent understanding and interpretation among awarding organisations and centres. Annex A sets out these definitions.

For a qualification that had external, internal, and practical assessments, the awarding organisation submitted to Ofqual details of the adaptations to each assessment. Some adaptations, such as alternative external quality assurance and flexibility in rules of combination, were applied to whole qualifications. Other adaptations, such as remote invigilation, would have only been appropriate for a specific assessment in a qualification.

Awarding organisations made around 39,500 adaptations to 7,382 vocational and technical qualifications

^{[footnote 4]}

Figure 1 and Table 1 show the number of adaptations broken down by adaptation type. At qualification level, the most prevalent adaptation was alternative external quality assurance arrangement. At assessment level, the most prevalent adaptations were alternative forms of assessment evidence for internal and practical, spoken or performance-based assessments, and remote invigilation for external assessments.

Figure 1: Number of adaptations by adaptation type

Table 1: Number of adaptations by adaptation type

Adaptation type	Number of adaptations
Alternative forms of evidence	7,362
Alternative external quality assurance arrangements	5,735
Remote assessment	4,588
Relaxation of or alternate controls for the creation of work and evidence	3,784
Adaptations to guidance	3,694
Streamlining within assessment	2,857
Carrying forward assessments	2,152
Flexibility of exam dates and windows for assessments	1,764
Remote invigilation	1,736
Alternative forms of employer involvement	1,320
Online rather than paper	1,302
Reduced assessment	1,025
Revised exam or assessment date	984
Adjustments to work placement requirements	593
Changes to distribution	536
Advance information	74
Flexibility in rules of combination	8
Exam aids	5

Awarding organisation readiness

Ofqual expected awarding organisations to deliver exams and assessments, and implement any adaptations appropriately and efficiently, while keeping their centres fully informed of the decisions and processes that affected them.

Ofqual undertook in-depth readiness reviews of 17 awarding organisations that together hold over 67% of the VTQ market share to check that they had appropriate plans, resource, and expertise to identify, prevent or mitigate any risks that could affect qualification delivery. The readiness reviews took place in February and March 2022 and were semi-structured interviews during which Ofqual explored:

• awarding organisations’ governance and capability
• their management of a range of operational risks relating to the delivery of all qualifications, such as assessment material production and marking, which were heightened because it was the first time in 2 years that assessments for some VTQs would be delivered
• their management of risks relating to specific qualifications, such as VTQs included in performance tables

Through these reviews, Ofqual identified isolated issues within awarding organisations’ delivery plans, such as managing upgrades to their IT systems and their approach to reviewing special consideration applications. Where necessary, Ofqual provided feedback to awarding organisations and asked them to strengthen their controls.

Ofqual also wrote to those awarding organisations that delivered an assessment series in summer 2022 to set out how it would monitor their delivery and issuing of results, and to outline the types of issues of which it expected to be notified.

Assessment material production

Ofqual requires awarding organisations to produce assessment materials which are clear, appropriate, and fit for purpose. Awarding organisations are responsible for making sure their assessments are error-free. Ofqual did not review, and has never reviewed, live assessment materials for VTQs before they are released to students. During readiness reviews, we discussed ways that awarding organisations avoided errors during assessment material production, and we sought assurances about how they managed the production risks.

Awarding organisations produce assessment materials, such as question papers and stimulus materials, in different ways. For assessments that are on demand, some awarding organisations use software to generate assessments, from a bank of standardised questions or items, such that each student takes an individualised version of an assessment. Awarding organisations tend to update their question banks frequently, adding fresh items to the bank and retiring old ones to ensure the questions are functioning appropriately and to prevent predictability. For assessments that are timetabled and take place at a specific date and time, such as for some VTQs in performance tables, awarding organisations write unique assessment materials for each session, usually a year or more before the assessments take place.

VTQs in performance tables are made up of a combination of external and internal assessments. External assessments are set and marked by awarding organisations, while internal assessments are set by awarding organisations or occasionally by centres, marked by centres and externally moderated by awarding organisations.

Awarding organisations produced over 800 unique external assessments for more than 400 performance table qualifications in the academic year 2021 to 2022, including modified assessments

Most of the external assessments were written tests offered on paper or on screen, which included a variety of different question types, including multiple choice questions, short or extended answers or essays.

Marker recruitment

Some VTQ assessments went ahead throughout the pandemic, albeit to a limited extent and where social distancing and public health guidance allowed. The risk to the recruitment of sufficient and competent markers for other VTQs, especially those included in performance tables, was heightened in the academic year 2021 to 2022 because exams had not taken place in the previous two academic years.

Ofqual collected fortnightly data from the awarding organisations who had the highest volumes of performance table qualification certifications. Ofqual used the data to monitor their recruitment of markers against their forecast requirements from January until May, and across multiple assessment series. Where marker recruitment in specific units was slower than anticipated, Ofqual discussed progress with the awarding organisation, but there was no evidence of a systemic shortfall in marking capacity.

Phase 2 – Delivery

Ofqual expects awarding organisations to work with their centres to identify, prevent and manage risks, instead of having to deal with issues that may adversely affect students. Ofqual recognises, however, that it is not always possible to eliminate all risks upfront.

Through ongoing engagement with stakeholders, other regulators, and government departments, Ofqual actively scanned for emerging risks and issues. Ofqual highlighted any risks to awarding organisations, for example, through rolling updates for all awarding organisations and the fortnightly meetings with awarding organisation members of the policy advisory and technical working groups. Ofqual monitored awarding organisations’ responses to potential issues by collecting and reviewing their data, meeting with them to review individual progress, and convening working groups to resolve issues collectively.

January 2022 assessments

In the wake of a rise in the number of cases resulting from the Omicron variant of the coronavirus in December 2021, Ofqual was alert to risks of disruption to students due to assessments being unable to take place, or students being unable to sit assessments. For this reason, Ofqual closely monitored the delivery of the January 2022 assessments.

Ofqual asked awarding organisations to contact their centres to make it clear that assessments should go ahead and reminded awarding organisations to put special consideration arrangements in place if students were unable to sit assessments due to reasons outside of their control.

Ofqual met the VTQ awarding organisations offering the highest volumes of January assessments to discuss their resource and preparation for the series and asked for weekly updates on student attendance and special consideration application volumes. There were 592,000 entries in January 2022 compared to 378,000 in January 2020 (the last pre-pandemic exam series). All assessment sessions in January 2022 went ahead without cancellation. Awarding organisations reported that the absence rate (8.4%) was slightly higher than that in January 2020 (6.4%), but the special consideration application rate in January 2022 (0.93%) was broadly comparable to that of January 2020 (0.66%) for students who missed assessments and those whose performance in assessments were adversely affected for reasons outside of their control.

While the pandemic had some impact on the January 2022 assessments, the impact was limited to individual cases, and was generally managed effectively by awarding organisations using their usual policies and systems.

Awarding organisations’ issue management

Awarding organisations must promptly notify Ofqual of any actual or potential incident which could have an impact on standards, public confidence in qualifications, or the awarding organisation’s ability to develop, deliver or award qualifications in a way which complies with our rules. Awarding organisations must also show how they have mitigated any impact. These are referred to as Adverse Effects under Condition B3 of our General Conditions of Recognition. Appropriate and prompt management of these events and incidents is crucial to making sure students are treated fairly across qualifications, and between awarding organisations and centres.

Ofqual actively monitors the events and incidents that are reported by awarding organisations to assess the impact of the issues and to evaluate awarding organisations’ management of them. This determines not only whether Ofqual needs to take any formal action, but also feeds into the ongoing monitoring of awarding organisations.

There were 106 reported incidents concerning VTQs in performance tables in the academic year 2021 to 2022, which is the same number Ofqual received for these qualifications in the academic year 2018 to 2019 (the last year when exams and formal assessments took place). A breakdown of the main incidents from 2021 to 2022 compared with 2018 to 2019 can be seen in Figure 2 and Table 2. The two most common types of incident reported in the year 2021 to 2022 were delivery failures (42 events) and suspected security breaches (22 events). This is different to the year 2018 to 2019, during which the two most common types of reported incident were assessment material errors (28 events) and delivery failures (22 events).

Figure 2: Incident types, 2021 to 2022

Table 2: Event types, 2021 to 2022 compared to 2018 to 2019

Event type	Number of reports, 2022	Number of reports, 2019
Delivery failure	42	22
Security breach	22	13
Assessment material error	16	28
Incorrect result	7	16
Malpractice	6	17
Marking issue	5	4
Other	5	6
Incorrect certificate	3	0

Delivery failures

Ofqual requires awarding organisations to deliver their assessments effectively, efficiently and to set timescales. Delivery, in this context, includes a range of processes from printing and dispatch of question papers to issuing results and processing appeals. Awarding organisations are required to report any actual or potential delivery issue which could impact the validity of the assessment result or delay the issue of results.

Figure 3 and Table 3 show the number of potential delivery failure events reported in 2021 to 2022 for VTQs in performance tables, broken down by reason.

Figure 3: Delivery failure events by reason

Table 3: Delivery failure events by reason

Table note: Percentages are rounded to the nearest whole percentage and may not add up to 100%.

Event reason	Number of reports	Percentage of total delivery failure reports
Cyber-attack	13	31%
IT failure	8	19%
Missing scripts	6	14%
Other	4	10%
Human error	3	7%
Information error	2	5%
Missed deadline	2	5%
Resource capacity	2	5%
Incorrect content	1	2%
Process weakness	1	2%

Cyber-attacks and IT failures were the most frequent types of delivery problem reported to Ofqual by awarding organisations. This was the main cause of the increase of reported delivery problems from the year 2018 to 2019 to the year 2021 to 2022. All the reported cyber-attacks were on centres. Most of the time, these issues are out of the awarding organisation’s control, but Ofqual expects them to have effective mitigations and contingency plans so that the effect of any disruption is minimised.

None of the reportable incidents about cyber-attacks or IT failures were substantial enough to threaten the ongoing operation of awarding organisations or centres. Although individual students were affected, their awarding organisations were able to put in place mitigations to minimise the adverse effects on them. For example, one awarding organisation reported that a centre had experienced a ransomware attack. Although the centre recovered its network swiftly, 16 students’ work in one subject unit could not be recovered and submitted for marking. The awarding organisation generated a result for this unit for these students based on their assessment evidence from other units of the qualification, so that they were still awarded a qualification.

Cyber-attacks and IT failures were also the most frequent types of delivery problems reported by awarding organisations in the academic year 2020 to 2021, amounting to 33 events. As a result, Ofqual required all regulated awarding organisations to provide details of how they each managed cyber security, data security and insider risks as part of their annual statement of compliance. Ofqual reviewed responses from all 158 awarding organisations that were required to submit this information, shared good practice with the regulated community and required them to work with their centres and third parties to continue managing cyber risks. Therefore, although the number of reported cyber-attack and IT failure events was greater in 2022 than compared to 2019, it in fact decreased from 2021.

Security breaches

Ofqual requires all awarding organisations to have appropriate measures in place to maintain the confidentiality of assessment materials. It is in everyone’s interests that the security of assessment materials is maintained, so that the assessment system is fair for all students. Ofqual monitors media and social media to identify potential breaches. During summer 2022, Ofqual identified examples of individuals on social media claiming to have question papers and mark schemes for sale for some VTQs that are taken alongside or instead of GCSEs, AS and A levels. In some instances, sellers were seeking large payments for these hoax papers, uploading a doctored copy of the front cover of the question paper as ‘proof’ of access. Ofqual issued a warning on our website to students on the risks of attempting to obtain question papers, making clear that trying to buy or share exam questions or papers, whether real or fake, is malpractice and could mean students are disqualified from some or all their exams.

Ofqual expects awarding organisations to report when there has been a loss or theft of, or breach in confidentiality in, any assessment materials. The report can refer to an actual security breach (for example, where the content of a live question paper is shared) and a potential security breach (for example, where procedures are not followed and there is the potential for the assessment to be compromised, even where this is not realised).

If a breach is suspected, awarding organisations are expected to investigate and to take all reasonable steps to mitigate its impact. Security breaches can be accidental, for example, if a student is accidentally given paper 2 instead of paper 1, or if results are accidentally released early to students. Breaches can also be deliberate, for example where papers are leaked or stolen.

Figure 4 and Table 4 show the number of potential security breach incidents reported in 2021 to 2022 for VTQs in performance tables, broken down by reason.

Figure 4: Security breach events by reason

Table 4: Security breach events by reason

Table note: Percentages are rounded to the nearest whole percentage and may not add up to 100%.

Event reason	Number of reports	Percentage of total security breach reports
Leak of materials (centre)	8	36%
Early results	5	23%
Loss in transit	4	18%
Incorrect paper	1	5%
Incorrect timing	1	5%
Leak of materials (awarding organisation)	1	5%
Leak of materials (candidate)	1	5%
Other	1	5%

Overall, potential leaks of confidential assessment materials originating from centres have been the most common type of security breach reported. Examples included centres handing out incorrect question papers and losing question papers on site, and unusually, a very small number of centres taking photos of questions to report an error to the awarding organisation, not realising that these are live assessment materials.

In each case, Ofqual’s priority was to ensure that awarding organisations took appropriate action to mitigate the impact of any potential or actual breaches on students. Ofqual also closely monitored awarding organisations to ensure that they thoroughly investigated the causes and the extent of the breaches and minimised likelihood of any recurrences. There was no evidence that the wider confidentiality of live assessment materials had been compromised in any of the cases reported. For timetabled assessments, awarding organisations did not need to replace the questions papers. For assessments that were on demand and where the confidentiality of isolated questions was potentially breached, the relevant awarding organisations took the necessary precautions and where appropriate withdrew these questions from future use.

Marking and moderation progress

The risk to the recruitment of sufficient and competent markers and moderators for VTQs, especially those included in performance tables, was heightened in the academic year 2021 to 2022 because most exams and assessments did not take place in the previous two academic years.

Consequently, Ofqual closely monitored awarding organisations’ marking and moderation progress of VTQs in performance tables. Awarding organisations marked students’ scripts of these qualifications on paper or online. Where marking is online, it can be either by item (an individual question or a part to a question) or at whole paper level. Ofqual collected marking data weekly, tracking progress both in terms of items and whole scripts.Where this data indicated that an awarding organisation was making slower progress than expected, Ofqual followed up on these issues to ensure that the awarding organisation put effective mitigations in place.

Ofqual also regularly monitored awarding organisations’ moderation progress against their expected completion dates.

Overall, awarding organisations made steady and sufficient progress in marking and moderation, which enabled them to make grading decisions in a timely way.

Special consideration

Ofqual requires awarding organisations to give special consideration to a student who has temporarily experienced an illness or injury, or some other event outside of their control, and this has materially affected their ability to take or perform in an assessment. Awarding organisations must have clear arrangements for special consideration and must publish details of them.

The purpose of special consideration is to mitigate as far as possible the adverse effects on those students who have covered their whole course and would have been prepared to take an assessment but were unable to do so or could not perform as expected because of circumstances outside of their control. Students are not eligible for special consideration if their education was disrupted by the pandemic or for other reasons.

In the VCRF, Ofqual set out specific expectations of awarding organisations where assessments were disrupted because of COVID-19. For example, where a national or local lockdown impacted on an assessment opportunity, awarding organisations needed to consider whether they could adapt or further adapt their qualifications and assessments in the first instance before considering whether special consideration could apply. Awarding organisations were also expected to ensure that where special consideration was available to students who had not completed all the assessments (because of previous or future lockdowns, for example), centres and students were clear about the amount of assessment evidence a student must have completed before a qualification could be awarded.

For the academic year 2021 to 2022, Ofqual collected data from awarding organisations about special consideration for the first time, for VTQs included in performance tables. Awarding organisations received around 28,000 requests for special consideration for 297 Applied General, Tech Level, Tech Cert and Tech Award qualifications (out of a possible total of 467 qualifications). Of the 28,000 total requests, approximately 20,000 requests were made on behalf of students who were present in an assessment but whose performance in that assessment was disrupted, and approximately 8,000 requests were made on behalf of students who were unable to take an assessment for reasons outside of their control. Awarding organisations have approved around 91% of the requests received to mitigate the disadvantages experienced by students.

Functional Skills qualifications

Awarding organisations continued with the range of adaptations they had put in place in previous years to enable students to take FSQs assessments. Where students and apprentices were unable to take these assessments due to the pandemic, Ofqual expected awarding organisations to take all reasonable steps to enable students to take assessments. Such steps included extending windows during which a student could take an assessment, changing invigilation requirements so that students could be invigilated by their own tutors or employers if necessary (albeit with controls in place), and offering remote invigilation so that students or apprentices could take assessments at their own home or at an employers’ premises. While the awarding organisations introduced these adaptations, Ofqual expected them to put in place appropriate controls to protect the integrity of the assessments.

In the academic year 2021 to 2022, awarding organisations issued 325,140 qualification results, 26,180 of which were achieved through remotely invigilated assessments

Phase 3 – Grading

Many VTQs test occupational competency and/or are used as a licence to practise. For these qualifications, awarding organisations have maintained performance standards from pre-pandemic.

For those VTQs that are taken alongside or instead of GCSEs, AS and A levels, Ofqual expected the awarding organisations to consider the grading approach of GCSEs, AS and A levels where possible. This meant that there was typically some generosity in outcomes for these VTQs compared to pre-pandemic levels.

Monitoring of awarding

During the academic year 2021 to 2022, Ofqual observed 22 awarding panel meetings held by awarding organisations for VTQs in performance tables and Technical Qualifications in T Levels, to check that they followed their own processes for awarding. Ofqual did not identify any issues through its observations.

Grading of VTQs in performance tables

VTQs in performance tables are often modular, and assessment typically happens throughout the course of study, which may span one, two or more years. This means that in 2021 to 2022, final grades for these VTQs, which are often taken in schools and colleges, were based on student outcomes from a range of sources, including one or more of:

• results from exams or formal assessments, some of which may have been adapted by the awarding organisation
• results based on teacher assessed grades that were awarded in summer 2021
• results based on centre assessment grades from summer 2020

The combination of assessment methods means that it was not possible to compare results directly with previous years. The results this year represented a staging post towards normal assessment and grading.

Ofqual also worked with awarding organisations to help them to develop their awarding approaches. In discussions with individual awarding organisations, Ofqual tested their rationales and checked for any unintended consequences so that we could help secure valid results and fair treatment of students.

Grading of Technical Qualifications in T Levels

The November 2021 exam series was the first during which assessments for Technical Qualifications were sat, with teacher assessed grades awarded for the core assessments of wave 1 Technical Qualifications in summer 2021. In summer 2022, core (wave 1 and 2) and occupational specialism (wave 1) assessments for Technical Qualifications were also available.

As for other VTQs typically taken in schools and colleges, Ofqual expected awarding organisations to take account of the approach to grading GCSEs, AS and A levels when awarding TQs in summer 2022. Awarding organisations were also expected to consider that these are new qualifications, and to ensure that students were not disadvantaged because of this.

Phase 4 – Results and post results

Awarding organisations and centres carry out a substantial amount of preparation ahead of results release, especially for those VTQs that are taken alongside or instead of GCSEs, AS and A levels and are used by students to progress to further or higher education. Ofqual expects awarding organisations to provide information to students and centres about which results will be released and when, and to work with their centres to maintain confidentiality of results and grade profiles ahead of results days.

To help centres and students better understand how VTQs would be awarded in 2022 and what their results meant, Ofqual published bespoke resources for centres and students, including but not limited to:

• guidance for centres: awarding VTQs in 2021 and 2022
• blog for VTQ students
• guide to the 2022 Level 3 qualification results for VTQs in England
• guide to the 2022 Level 1, Level 1/2, and Level 2 qualification results for VTQs in England
• interactive visualisations for qualification results of VTQs in performance tables and component results of Technical Qualifications in T Levels (the T Level qualification results are published by DfE)

Delays in results in summer 2022

For some VTQs, the return to and the adaptation of exams and formal assessments in the year 2021 to 2022 brought additional complexity for centres and awarding organisations. In light of this, Ofqual asked awarding organisations to consider whether they should take additional steps to improve their understanding and record-keeping in respect of which qualifications were being offered by their centres, which students were taking their qualifications and when those students expected to take assessments.

Between March and August 2022, awarding organisations issued approximately 724,700 results across 410 Level 1/2, Level 2 and Level 3 VTQs in performance tables. Nevertheless, some students did not receive VTQ results when they expected to in August.

When issues about delays in results emerged on Level 3 results day, Ofqual immediately monitored awarding organisations’ daily progress to resolve missing or late result queries. Ofqual also contacted the National Association of Examinations Officers to ask that they did all they could to support centre staff and students.

Senior officials at Ofqual, including the Chief Regulator, met directly with school and college leaders to assure them of the steps Ofqual was taking. Our focus was to ensure that the relevant awarding organisations were taking all possible steps to identify affected students and to issue results to those who were eligible.

Throughout this period, Ofqual met regularly with DfE and UCAS to update them. We facilitated the sharing of data between UCAS and awarding organisations so that students holding university offers could be prioritised.

Ofqual made clear to Pearson and OCR, the awarding organisations with the largest share of the affected VTQs, that they should publish the volumes involved and the progress they were making in resolving the issues. This led to both awarding organisations regularly publishing data.

Ahead of results day for Level 1/2 and Level 2 qualifications, Ofqual wrote to awarding organisations to ensure that they worked with their centres to obtain all the necessary information for them to process and issue results to those students who expected them. While there were a small number of delayed results, results day for L1/2 and Level 2 went relatively smoothly.

In total, just under 21,000 Pearson and OCR grades were affected by delays or errors. The table below shows the breakdown by awarding organisation and level.

Table 5: Numbers of issues by awarding organisation and qualification level

Table notes: Figures rounded to the nearest 5. Data for England only.

Qualification Level	OCR	Pearson	Total
Level 2	6,900	1,670	8,570
Level 3	4,060	8,280	12,350
Total	10,960	9,960	20,920

This represents just over 3% of the overall L2 and L3 VTQ grades released by Pearson and OCR. These missing or incorrect results affected around 20,540 students in just over 1,550 schools and colleges.

The issues affected students taking qualifications of all sizes, but students taking smaller-sized qualifications (up to 180 guided learning hours) at level 2 were particularly affected.

At level 2, the 5 qualifications most affected, in terms of absolute numbers, are shown below. The table shows the number of grades affected, and the percentage of the total grades for the qualification.

Table 6: The 5 Level 2 qualifications with the largest number of issues: number and percentage reported with respect to number of grades awarded ahead of results day.

Table notes: Figures rounded to the nearest 5. Data for England only.

Qualification title	Number	%
OCR Level 1/Level 2 Cambridge National Certificate in Creative iMedia	2,245	5.6
OCR Level 1/Level 2 Cambridge National Certificate in Sport Studies	1,430	5.8
OCR Level 1/Level 2 Cambridge National Certificate in Enterprise and Marketing	895	6.1
OCR Level 1/Level 2 Cambridge National Certificate in Health and Social Care	700	5.4
OCR Level 1/Level 2 Cambridge National Certificate in Sport Science	695	5.0

At level 3, the 5 qualifications most affected are shown below.

Table 7: The 5 Level 3 qualifications with the largest number of issues: number and percentage reported with respect to number of grades awarded ahead of results day.

Table notes: Figures rounded to the nearest 5. Data for England only.

Qualification Title	Number	%
Pearson BTEC Level 3 National Extended Diploma in Business	645	7.8
Pearson BTEC Level 3 National Foundation Diploma in Business	570	8.4
OCR Level 3 Cambridge Technical Extended Certificate in Health and Social Care	475	11.2
Pearson BTEC Level 3 National Extended Diploma in Health and Social Care	430	5.9
Pearson BTEC Level 3 National Extended Diploma in Applied Science	390	7.9

Ofqual launched a full review to establish and understand the factors that may have contributed to these results delays. This work is ongoing but comprises the following:

• a thorough investigation, which is ongoing, of the exam boards’ systems and processes
• working closely with awarding organisations and sector bodies (including exams officer associations) with recommended steps and actions to protect against the issue recurring in 2023
• working on a longer-term transformation of the VTQ system so that students holding higher education offers can receive their level 3 VTQ results at the same time as A level students

Our actions and findings in respect of this work to date are disseminated as follows:

• the scope and scale of the incident is set out within this report
• actions we are taking to ensure awarding organisations deliver VTQ results in 2023 on time

Full analysis of the root causes and our sector-wide recommendations on streamlining the delivery and awarding of VTQs beyond 2023 will be published in 2023.

T Level health and science results

Ofqual regulates the Technical Qualifications (TQs) within T Levels, which comprise the core assessments (core examinations and employer-set project) and the occupational specialism. Students can take the core assessments at the end of the first year of their T Level programme, with two further assessment opportunities within the two-year programme. The academic year 2021 to 2022 saw both the assessments of the second wave of T Levels come on stream in new routes such as health and science, as well as the assessments of T Levels launched in the first wave in the academic year 2020 to 2021.

In the week following Level 3 results day in August 2022, some centres raised significant concerns about low levels of attainment in the core examinations of the health TQ, which had the highest entry size in the route and is offered by NCFE as the single provider. Ofqual swiftly worked with centre representative organisations such as the Association of Colleges and spoke directly with centres to better understand their concerns. At the same time, Ofqual met with NCFE to discuss these concerns.

Nonetheless, Ofqual completed a thorough review of the health and science core assessment papers. In the core exams, we identified question errors, weak mark schemes, and questions covering areas not explicitly in the specification. We therefore determined that the core examinations were not a sufficiently valid or reliable measure of student performance. Our review of the employer-set project did not identify any issues that undermined the validity or reliability of that assessment.

Ofqual worked with NCFE, DfE and the Institute for Apprenticeships and Technical Education (IfATE) to determine the best possible resolution for the 1,115 students who were affected by the issues. It was agreed that the students’ Health and Science core grade (that which is carried into their final T Level certificate) would be the higher of the employer-set project grade or the overall core grade already issued. Any students who wished to improve their grade were able to resit the employer-set project in autumn 2022 or later, and a higher overall or ESP grade through the resit would count as their new overall core grade. NCFE confirmed that there would be no charge for resits.

As an immediate step, Ofqual has taken formal action to require a detailed undertaking from NCFE to ensure that assessments for the autumn 2022 series for all of its TQ are fit for purpose. NCFE has provided the undertaking and Ofqual is closely monitoring NCFE’s compliance with it. We will also continue to scrutinise NCFE’s development and delivery of assessments for summer 2023 and beyond. Ofqual is investigating NCFE to establish what led to the issues in summer 2022. Once the investigation concludes, Ofqual will determine the best course of action.

Appeals

Ofqual requires awarding organisations to permit appeals on the basis that the awarding organisation did not apply procedures consistently or that procedures were not followed properly or fairly. Awarding organisations must provide for the appeal of:

• the results of assessments
• decisions regarding reasonable adjustments and special consideration
• decisions relating to any action to be taken against a student or a centre following an investigation into malpractice or maladministration

In the academic year 2021 to 2022, the arrangements for appeals of VTQ results continued as normal. In November 2022 Ofqual collected provisional data on appeals for the following qualifications and students in centres in England:

• all Tech Awards, Tech Certs, Tech Levels and Applied Generals
• all Functional Skills Qualifications (FSQs)
• all ESOL Skills For Life Qualifications, and
• all Core Mathematics, IB Diplomas, and Cambridge Pre-Us

As of the date of this provisional data collection, awarding organisations have completed approximately 900 appeals (98% of those received so far), and have upheld 25% of those completed.

Most of the appeals received were appeals of component or qualification results, of which awarding organisations have completed over 800. Of the completed appeals, 21% were upheld.

In terms of grounds of appeal, the largest number of appeals so far (over 500 appeals) were made on the grounds that the awarding organisation did not follow its procedures correctly. Awarding organisations have upheld 25% of the appeals made on this ground.

Awarding organisations have also received approximately 100 appeals made on the grounds of unreasonable exercise of academic judgement (for example, where an awarding organisation made an unreasonable judgement while awarding marks to a task completed by a candidate). Awarding organisations upheld 23% of the appeals made on this ground.

More definitive data will become available later in the academic year when the appeals process is more complete.

Part 2 Regulation of Apprenticeship end-point assessments

Background

In May 2020 the Institute for Apprenticeships and Technical Education (IfATE) consulted on a simplified external quality assurance (EQA) system. As a result, the majority of apprentices now benefit from taking end-point assessments (EPAs) which are regulated by Ofqual. The transition to Ofqual regulation began in the autumn of 2020 and by the end of August 2022 Ofqual regulated 475 Apprenticeship standards and 120 awarding organisations (AOs) that offer EPAs. In total we regulate 927 EPAs although not all are yet on the Ofqual Register.

Figure 8: Scope of EPA regulation increase 2019 to 2022

Table 8: Scope of EPA regulation increase 2019 to 2022

	End March 2019	End March 2020	End March 2021	End March 2022	End August 2022
EPAs on Ofqual’s Register	84	136	240	589	780
Standards regulated by Ofqual	62	77	204	402	475
Standards with EPAs on Ofqual’s Register	29	47	116	251	342
Standards which have certificated	N/A	31	92	198	241
AOs with EPAs on Ofqual’s Register	21	26	41	71	84

Regulatory Approach

Ofqual regulates EPAs in the interests of apprentices of all ages. We make sure EPAs are delivered by fit-for-purpose organisations that are capable of developing and awarding valid and reliable assessments. This means we:

• ensure that only organisations that can demonstrate sufficient capacity and capability to deliver EPAs become regulated (62 to the end of August)
• work with IfATE to strengthen the assessment plans which determine how EPAs must be delivered
• implement a rolling programme to monitor AOs’ delivery of assessments

Providing advice on assessment plan development

IfATE is responsible for developing, approving and publishing assessment plans. These are the blueprint on which AOs build their assessments and must enable different AOs to consistently design assessments that are sufficiently valid, reliable, comparable, manageable, affordable, and accessible.As such, Ofqual pays close attention to the content of assessment plans and has worked with IfATE to improve the quality of plans.

The issues below are some of the greatest challenges for AOs in their interpretation and delivery of plans and have been prominent areas of focus in working with IfATE on new and revised assessment plans. Ofqual continues to work closely with IfATE, providing advice on the development of assessment plans to ensure apprentices are assessed in a sufficiently valid and reliable way.

Issues with grading criteria

Grading criteria must provide sufficient detail to guide consistent judgements from assessors over time. Issues occur where the criteria are too high level, or merely repeat the knowledge, skill or behaviour (KSB) being assessed. Weaker grading criteria may contain subjective terms, the differentiation between pass and higher grades may not be sufficiently clear and criteria may lack alignment with the knowledge, skill or behaviour. Any or all these factors may mean the criteria will not promote reliable assessment of the relevant KSB.

Assessment method clarity and validity

Assessment plans set out the assessment methods AOs should use to assess different KSBs. The plans should be clear about when approaches to assessment must be consistent or where differences are permitted. The nature or requirements of one or more assessment methods are not always sufficiently clear and this may result in very different approaches or interpretations by AOs delivering the same apprenticeship standard, where that is not the intention.

Resit or retake requirements

These may not always be sufficiently clear.

Advice on existing assessment plans

From April 2021 to August 2022, because of transition, 216 additional standards became Ofqual regulated, bringing the total number of regulated standards to 475. Apprenticeship standards describe what an apprentice will be doing and the skills required of them, by job role. Many of the assessment plans for these standards were developed some time ago, were not developed with a view to be being regulated, pre-date the existence of IfATE, and are undergoing, or scheduled for, review. Ofqual reviewed the assessment plans for these standards to identify potential risks and challenges which might be faced by AOs in their development and delivery of sufficiently valid EPAs.

Ofqual found that:

• 30% of plans contained gaps in information about how assessments should be delivered or graded
• 40% of plans contained some instructions which, if followed, could inadvertently cause an AO to be non-compliant with Ofqual regulations

These issues can lead to:

• inconsistent interpretation of the assessment requirement by AOs on the same standard
• an inconsistent assessment experience between apprentices completing the same standard
• inconsistent assessment decisions where the judgement criteria are interpreted differently by AOs and their assessors

These risks are mitigated to an extent by Ofqual’s regulation, as regulated AOs delivering these standards are required to develop assessments that are manageable, valid and reliable. In addition, Ofqual has helped to mitigate many of these risks by issuing guidance or reaching consensus in AO approaches through co-regulation and facilitating inter-AO forums. We also continue to work closely with IfATE’s group assessment leads to ensure our programme of monitoring activity supports the review process, particularly for those plans which pose risks to an AO’s compliance with regulation.

Assessment plan queries

Where an assessment plan is in delivery and an AO requires clarification on an aspect of the plan, the AO can submit an assessment plan query. From April 2021 to August 2022 Ofqual reviewed 274 assessment plan queries from 47 AOs and 19 other stakeholders (including prospective AOs, training providers, apprentices, and employers) relating to 125 different Apprenticeship standards.

Figure 9: Assessment plan query increase year on year (April to March)

Table 9: Assessment plan query increase year on year (April to March)

Category	April 2020 to March 2021	April 2021 to March 2022
Number of queries	98	168
AOs who submitted queries	26	36
Number of standards involved in queries	54	77

Where possible, Ofqual resolves all queries concerning assessment plans so AOs deliver valid and reliable assessments to apprentices.

Recognition

Ofqual’s recognition process tests an organisation’s capacity and capability in the areas of governance, finance, integrity, and assessment competence.

More than 135 prospective awarding organisations contacted Ofqual during the period to August 2022. Of these, 62 organisations were successful in achieving Ofqual recognition and 32 were unable to demonstrate that they met our requirements. Other applications were considered outside the reporting period and some withdrew from the process.

Ofqual has confidence that organisations that meet the Criteria for Recognition will deliver robust, fair assessments for apprentices for the long term. All organisations that fail to meet Ofqual’s requirements are offered feedback meetings where issues with their application are clarified. The majority of the AOs that were not successful by August 2022 have been able to reapply.

Special conditions

Where an organisation meets Ofqual’s Criteria for Recognition, but specific risks are identified during the process that require additional monitoring and support, special conditions or limitations may be imposed on that organisation.

Ofqual imposed special conditions on 27 AOs during the reporting period.

Other conditions applied include:

• requiring the organisation to review its appeals process to ensure that independence in the final stages of appeals was maintained
• where it was clear, in a more niche organisation, that expertise was resident in a limited number of individuals, that they notify Ofqual of any changes in their senior officers
• restricting the number of learners that the organisation could register over a certain period where resources of the organisation were limited
• requiring the organisation to review its policies and procedures in relation to specific areas

Ofqual monitors AOs’ compliance with special conditions. Of the 27, 8 AOs have had special conditions fully or partially revoked during the period to August 2022, having demonstrated to us that they had fulfilled the requirement of the special condition. For 5 others the duration of their special conditions was extended to provide additional time to meet our requirements. The remainder are subject to ongoing monitoring.

Case Study: Expanding a scope of recognition with a special condition

A recently recognised AO had a special condition in place which limited the number of registrations they could make on Ofqual-regulated EPAs. It was seeking to expand its scope of recognition. The rationale for the special condition centred on the organisation’s capacity and capability to both develop and deliver EPAs. Its initial application for Ofqual recognition had been for a single standard, through its expansion this would increase to more than 10 standards.

Ofqual observed the processes and systems the AO used to design, develop, and deliver its existing standard. This included a focus on recruiting, training, and deploying assessors, reviewing a sample of recorded professional discussions, checking assessment material completed by assessors, and testing the related internal quality assurance processes and decisions applied.

The review concluded that the AO had consistently followed its own design and development processes, and the AO had fully considered the assessment plan when deciding on the specialist subject capability required in development of its assessments. Moreover, it was evident the processes for training and quality assuring assessors were appropriate, focused on the quality of the outcome for the apprentice, and its approach was resulting in sufficiently valid assessments.

In short, there were no concerns identified regarding the ability of the AO to develop and deliver compliant assessments provided it had adequate access to the relevant resource. This information was subsequently used as the basis for the decision regarding the status of the special condition.

Statement of Compliance 2021

The annual Statement of Compliance is a mandatory activity for all regulated awarding organisations. It provides assurance of each awarding organisation’s compliance with Ofqual’s General Conditions of Recognition and associated conditions.For awarding organisations that offer EPAs, part of the Statement of Compliance in 2021 focused on their EPA delivery through a specific line of enquiry.

AOs were asked to answer 2 questions in relation to all the EPAs they delivered:

1. Considering your obligations under General Condition D3.2 and E4.2(a), and your approach to the development of assessment materials, to what extent do you engage with employers and other stakeholders within the relevant sector to ensure your approach is fit for purpose? Please include an outline of the nature and scope of any engagement undertaken.
2. Considering your obligations under General Condition G1.2(c), please describe how you ensure that any question banks used in your EPAs generate assessments which are of a consistent level of demand between assessments, and over time.

Question 1: stakeholder engagement

While no responses to this question resulted in findings of non-compliance, there was a range in the quality of response. Weaker responses concentrated on how assessment materials were developed and gave limited evidence of proactive engagement. Stronger responses set out a range of engagement mechanisms including considering the apprentices themselves as stakeholders and the incorporation of learner feedback.

Only 3 responses led to more acute concerns: 2 due to the brevity of response, and one because of a concern regarding adherence to the assessment plan. All concerns were resolved following discussions with the AOs.

Question 2: question banks

This question provoked an even greater range of responses and was more technical in nature. Stronger responses included aspects such as the use of multiple technical experts in the creation and review of test specifications as well as content and item creation. Some responses referred to the use of external subject matter experts in addition to in-house expertise, and many responses provided outlines of training and standardisation approaches for item writers. Some responses identified that historical information, such as archives of previous items, were also used to support consistency in level of demand in question and paper setting (which also applied to how banks were maintained over time).

Stronger responses also included information on specific approaches to marking assessments from new question banks, including double marking. They also provided information on how questions were reviewed in relation to relevant industry changes, such as advances in technology, legislation, or industry trends. In addition, some responses included references to collaborative working when dealing with the challenge of maintaining a comparable level of demand across different EPAs, including working with the regulator, employer groups and other AOs.

There were no substantive areas of concern related to question banks. Some additional information was requested in a minority of cases to enable a full understanding of the AOs’ approach.

Awarding organisations’ issue management

AOs must tell us about any incident which has, or could have, an adverse effect (EPA Qualification Level Condition EPA3.1). Ofqual’s EPA Qualification Level Conditions include the following as specific examples of events which could have an adverse effect:

• substantial errors in assessment materials
• issuing incorrect results
• loss, theft, or a breach of confidentiality in any assessment materials
• inadequate numbers of assessors to deliver the assessment effectively

In the reporting period AOs reported 18 incidents regarding EPAs.

Figure 10: EPA incidents during the reporting period

Table 10: EPA incidents during the reporting period

Incident type	Number
Assessment material error	4
Incorrect results issued	4
Failure to deliver assessment in line with assessment plan	8
Security breach	2

For each incident Ofqual engaged with the AO to understand the background, the potential adverse effect to apprentices, standards or public confidence, and the steps the AO had taken to mitigate those adverse effects. We also require the AO to identify the root cause of the event and to put in place measures to prevent reoccurrence.

Of the 18 reported events received in the reporting period, 9 have been closed with all 9 resulting in a determination of non-compliance.

Assessment design and delivery

Ofqual uses a variety of regulatory tools to check end-point assessments and to safeguard the assessment experience of apprentices. From April 2021 to August 2022, 147 separate Apprenticeship standards have either been considered through technical review or observed in the field.

Technical evaluation

Technical evaluation is a qualification-level check of an AO’s ability to develop assessment materials that support valid assessment, and reflect the requirements of the assessment plan. The process involves a check of the EPA materials provided by an AO in 2 stages, by assessment experts, and subject experts. From April 2021 to August 2022, Ofqual conducted technical evaluations of 72 EPAs, from 29 AOs across 42 standards.

The two most frequently occurring issues during this period were: a lack of clarity surrounding task requirements in the guidance provided to assessors, and a lack of exemplification of grading criteria. These two issues remain consistent with those seen in previous years.

Figure 11: Six most frequent issues from tranche 1 to 16 of technical evaluation 2017 to 2022

Table 11: Six most frequent issues from tranche 1 to 16 of technical evaluation 2017 to 2022

Rank	Category	Number of EPAs	% of EPAs	Number of issues	% of issues
1	Unclear task requirements	100	63%	223	22%
2	Little or no exemplification of grading criteria or performance requirements	75	47%	131	13%
3	General errors, contradictions, or inaccuracies in materials	69	44%	125	12%
4	EPA not in keeping with intention of assessment plan	79	50%	128	12%
5	Not all KSBs are covered	41	26%	52	5%
6	Assessment or performance requirements differs from the assessment or grading criteria set out in the assessment plan	39	25%	49	5%

From April 2021 to August 2022 Ofqual provided 72 findings reports to AOs and followed up to ensure AOs rectified the issues identified. In addition, we have requested one letter of assurance and identified 6 EPAs for further monitoring.

Over a period of more than 4 years Ofqual completed 16 tranches of technical evaluation, totalling 158 EPAs from 45 AOs across 85 standards. Analysis of data from this period shows a correlation between those AOs that have been subject to multiple evaluations and an improvement in the quality of their materials. Ofqual is seeing fewer issues in the materials submitted from these AOs. The AOs in the analysis were involved in 6 or more technical evaluation tranches with 9 or more EPAs in total.

Figure 12: Average number of identified issues in technical evaluation tranches per AO

Table 12: Average number of identified issues in technical evaluation tranches per AO

Table notes: Blank cells indicate that the AO was not involved in that tranche. The table can be scrolled to the right to see more data.

AO	Tranche 1	Tranche 2	Tranche 3	Tranche 4	Tranche 5	Tranche 6	Tranche 7	Tranche 8	Tranche 9	Tranche 10	Tranche 11	Tranche 12	Tranche 13	Tranche 14	Tranche 15	Tranche 16
AO A		11					9			4			4	8.5		5
AO B			7.5	14	12.2	12		5	4.5							5
AO C	8		12			14	7.5				2	4	7	2
AO D			11	6					8		2			8		7
AO E			5	7				1		1				4.5		3
AO F		10	8	8.3		12.7	9			4	1			4

Assessment observations

Ofqual completed 122 AO visits from April 2021 into autumn 2022, with 81 AOs across 88 standards. Understandably, during this period most visits were remote. Over 50% of visits included observation of practical and professional discussion assessment.

Assessment observation tests an AO’s approach to the design, development and delivery of EPAs, seeking assurance that they are following the assessment plan, are compliant with Ofqual’s General Conditions of Recognition, and are resolving issues where identified.

Emerging themes from observations

The critical role of the assessor

Ofqual observed a broad range of assessors in the reporting period and in a small number of cases identified concerns, for example, AOs that subjected assessors to only minimal quality assurance. Some observed assessors in this category were not adhering to the AOs’ own procedures or following the assessment plan. Instances were also observed where assessors unintentionally undermined an AO’s processes by confirming the apprentice’s grade prior to any quality assurance from the AO taking place.

In these instances Ofqual investigated thoroughly and in some cases took enforcement action.

EPA outcomes data return

In January 2022, Ofqual required all AOs with EPAs on the Register of Regulated Qualifications as of the end of 1 January 2022 to provide outcome data for their EPAs to cover the period from 1 March 2021 until the end of February 2022.The data provides insight into overall and component-level (assessment method) pass rates and grade distributions within and across standards.

This is the third year Ofqual has collected data on EPA outcomes, with a number of refinements made to the specification for the 2022 data return. The 2022 return provided data for 50,800 apprentices across 205 standards and 75 AOs. This represents a significant increase compared to the equivalent collections in 2021 and 2020, reflecting the impact of the transition programme.

Table 13: Increased scope of component data collections 2020 to 2022

Category	2020	2021	2022
Standards covered by data collection	32	88	205
AOs involved in data collection	22	43	75
Apprentices reflected in data collection	11,490	15,090	50,800

Conclusions and next steps

From October 2021 to September 2022, awarding organisations issued a total of 4.9 million certificates, which is a 6% increase compared to the same period in the previous year. Awarding organisations and centres worked together, sometimes in complex and challenging circumstances, to seek to achieve fairness and to award results that enabled students to progress onto the next stages of their lives. The Chief Regulator thanks awarding organisations and centre staff, especially exams officers, for all their work this year to ensure that students received the grades that enabled them to progress to the next stages of their lives.

Although the delivery of VTQs in 2021 to 2022 was largely successful, some significant issues occurred, such as the poor quality of assessment materials in the Health and Science T Levels and the delays in results in summer 2022. These are unacceptable, and Ofqual has taken swift action to initiate investigations into what caused these issues. Where these incidents are found to have been preventable by awarding organisations, Ofqual will take appropriate and proportionate action, including enforcement as necessary.

For the academic year 2022 to 2023, DfE has set out guidance on the subject content and assessment arrangements and Ofqual has confirmed the arrangements for grading exams and assessments. The government’s intention is to return to the carefully designed and well-established pre-pandemic assessment arrangements as quickly as possible, given they are the best and fairest way of assessing what students know and can do. As such, adaptations to VTQs are not expected to be used going forward. However, awarding organisations are permitted to keep some changes to the way in which they delivered assessments during the pandemic, for example remote assessment and remote invigilation, as normal arrangements, provided they are sufficiently valid and secure. Ofqual will play its part and support awarding organisations to test and implement new approaches to assessment delivery such as digitisation, so that greater flexibility can be provided for students and apprentices taking the assessments.

The VCRF remains in force until Ofqual publishes a notice bringing it to a close. In the very unlikely event that VTQ assessments cannot proceed as planned in 2023 because of system-wide disruptions caused by COVID-19, awarding organisations would be able to reintroduce adaptations during the academic year 2022 to 2023. Should exams and formal assessments no longer be considered viable, the provisions which permit the use of teacher assessed grades could be brought back into force, following a short consultation.

Ofqual will continue to monitor all awarding organisations’ preparation for, and management of risks related to assessment delivery for the year 2022 to 2023, with a focus on the implementation of recommendations from our review of results and awarding organisations’ production of assessment materials. Ofqual also considers centre support, contingency arrangements (from changes in public health guidance to the impact of industrial actions) and the issuing of accurate and timely results some of the key risks that awarding organisations should mitigate and manage. Ofqual’s regulatory activities will reflect these risks.

Annex A – Definitions of adaptation types

Adaptations to guidance

Adaptations to guidance for individuals involved in assessment delivery, for example invigilators, readers and/or scribes, internal assessors.

Adjustments to work placement requirements

Waiving or adjusting work experience or placement requirements, for example allowing learners to undertake a shorter period of work experience.

Advance information

Information about the focus of the content of the exams provided in advance of the exam taking place. Advance information met the principles set out in Ofqual’s consultation on proposed changes to the assessment of GCSEs, AS and A levels in 2022.

Alternative external quality assurance arrangements

Changes to the way in which an assessment or qualification is externally quality assured as set out in the published specification or guidance, for example by allowing this to take place remotely or online.

Alternative forms of employer involvement

Allowing employer engagement to take place in a format that differs to that published in the qualification specification, for example employers may be invited to give talks and/or practical demonstrations within a centre rather than sending students into workplaces.

Alternative forms of evidence

Allowing the learner to generate alternative forms of evidence which might be different to those published in the specification, for example, allowing a professional discussion in place of certain practical activities or using a practical simulation in place of an observation.

Carrying forward assessments

Internal or external tasks or assessment briefs that would normally be retired will remain ‘live’ for an additional period, and/orstudents who produced evidence for assessment in 2020 to 2021 but were unable to receive a grade can submit this evidence for assessment in 2021 to 2022.

Changes to distribution

Changes to the regulations regarding the distribution and collection of papers, for example, collation of scripts after learners have left exam rooms, quarantining papers before and after use, and extended timescales to return scripts after exams.

Exam aids

For example, the use of formulae and equation sheets.

Flexibility of exam dates and windows for assessments

Additional flexibility in exam dates and windows for assessments are permitted by the awarding organisation. For example, longer assessment windows and/or additional exam or assessment opportunities may be provided.

Flexibility in rules of combination

Flexibility in the way in which students can combine different units to achieve a qualification. This could include making mandatory units optional, so students can still achieve a qualification.

Online rather than paper

Using online or on-screen rather than paper-based assessments.

Reduced assessment

A reduction in the number of internal assessments a learner must complete to achieve the qualification. Note that centres are still required to teach the full course content.

Relaxation of or alternative controls for the creation of work and evidence

Changes to the conditions under which some or all assessment tasks are taken, for example, the requirement for candidates to be directly supervised when completing tasks may be lifted. There will be alternative requirements to ensure the authenticity of the student’s work or evidence.

Remote assessment

Ofqual’s definition:

Remote assessment is where an Assessor examines a Learner, or a recording of them, while they complete the required and timed assessment tasks from a different location to that of the Assessor. Remote assessment is used to assess practical, spoken and performance tasks, and is therefore different to on-screen and online assessment. On-screen assessment describes where a Learner reads and answers the questions of an assessment on-screen, either by typing or clicking the correct response or by assembling digital evidence of achievement.

Online assessment describes where the assessment materials for an on-screen assessment are delivered to the Centre, and where a Learner’s responses to that assessment are returned to the awarding organisation, via the internet.

Remote assessment also does not cover independent completion of required tasks or generation of evidence without direct supervision and/or observation by an Assessor.

Remote invigilation

Ofqual’s definition:

Remote Invigilation is the supervision of a Learner where the Invigilator is in a different location to that of the Learner. This may be achieved using a live feed via an internet connection and/or by the post-hoc scrutiny of recorded footage of the Learner completing the required assessment tasks. Remote Invigilation uses human Invigilators, but they may be supported by artificial intelligence software where appropriate. It ensures that the Learner completes the assessment under the required conditions so that the awarding organisation can assure itself of the Validity of the assessment and secure the award of the qualification. The Invigilator is not assessing the Learner.

Revised exam or assessment date

The awarding organisation moves the date of a timetabled exam or assessment.

Streamlining within assessment

Amendments to an assessment. For example, removal of duplication of assessment of the same or similar skill, or removing specific requirements, tasks, or assessment criteria, or combining related assessments.

Note that centres are still required to teach the full course content. Information about the focus of the content of the exams provided in advance of the exam taking place. Advance information will meet the principles set out in Ofqual’s consultation on proposed changes to the assessment of GCSEs, AS and A levels in 2022.

---

1. The overall T Level result for a student is issued by DfE. ↩

2. However, students who had unit results based on alternative evidence, such as centre assessment grades (CAGs) or teacher-assessed grades (TAGs) from previous years could have carried them forward for certification in 2021 to 2022 if they wished. ↩

3. In the academic year 2020 to 2021, FSQ assessments were expected to go ahead, adapted or otherwise. But for those students and apprentices who had genuine barriers to take these assessments despite support from their centres and awarding organisations, Ofqual’s framework permitted awarding organisations to issue results based on teacher assessed grades and with clear eligibility criteria. ↩

4. The VTQs were available to students in England and at Levels 1 to 5. They did not include VTQs in performance tables, FSQs, ESOL Skills for Life, or qualifications aimed to enable or enhance preparation for work. ↩