Research and analysis

Background Information for: Ofqual Public Sentiment 2021

Published 27 October 2022

Applies to England

© Crown copyright 2022

This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.

Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.

This publication is available at https://www.gov.uk/government/publications/ofqual-public-sentiment-2021/background-information-for-ofqual-public-sentiment-2021

Introduction

Background

This document accompanies the findings of Ofqual’s Public Sentiment 2021 study and should be read alongside the full findings report. The study was commissioned by Ofqual and conducted by YouGov. A report of public perceptions for qualifications that Ofqual regulates can also be found in Ofqual’s suite of published Annual Perceptions Surveys, of which wave 20 also explores views for qualifications in 2021.

The purpose of this background information document is to give detailed information about the method, samples and processes used in conducting this piece of research.

Context

The objective of the research was to explore public and stakeholder perceptions, and confidence in relation to arrangements for assessments and grading in England in 2021. It was conducted in response to COVID-19 related changes to assessments. To carry out the research, a series of surveys were conducted across four waves between March and September 2021 to monitor perceptions of the arrangements for assessments and grading in 2021.

Ofqual regulates general and vocational qualifications in England. The research explored perceptions about the arrangements for grading and assessments across a range of general and vocational and technical qualifications. Perceptions were gathered from senior leaders in schools and colleges, teachers, students, parents, the general public, employers, and Higher Education Institutions (HEIs). Samples of these groups were drawn from England only.

Methods and data analysis

Overview of method

The Public Sentiment 2021 study was carried out by YouGov, on behalf of Ofqual. The surveys were conducted online using YouGov’s proprietary survey system. The fieldwork was conducted across four waves between March and September 2021. Specific fieldwork dates for each wave were as follows: initial perceptions (wave 1: 24th March to 30th April 2021), pre-assessment (wave 2: 5th to 27th May 2021), post-assessment (wave 3: 28th June to 21st July 2021) and post-results (wave 4: 19th August to 24th September 2021). Note that when this and accompanying reports refer to ‘this year’ in relation to wording of the survey questions, it refers to 2021.

Survey design

The survey was designed by Ofqual in conjunction with the YouGov project team. The survey explored issues related to the arrangements for assessments and grading in 2021, including:

  • Awareness of arrangements for grading students in 2021;
  • Views on grades in the context of in 2021 and in the context of ‘before the pandemic’ (note that all views were collected in 2021) about issues related to utility and dependability of grades, and indications of proficiency. These questions were asked across a range of general and vocational and technical qualifications:
    • GCSEs
    • A levels
    • Functional Skills
    • BTECs, Cambridge nationals and other similar vocational qualifications
    • Tech Awards and Tech Levels
    • Licence to Practise, competency, professional, proficiency qualifications (i.e. those equivalent to GCSE and A Level, but not degree level), collectively referred to in these reports as ‘professional qualifications’
  • Views about teacher judgements;
  • Views about the arrangements for appeals in 2021; and
  • Overall perceptions of confidence.

The questions predominantly required closed responses, for instance: ‘yes’/’no’ and Likert scale responses. A copy of the questions included in the Public Sentiment 2021 study is available in Appendix A. Overall, the survey took respondents approximately 15 minutes to complete. Questions that evaluated perceptions specific to certain qualifications were only shown to respondents who were aware of the qualification. These audiences are noted in the question schedule in the appendix.

Part of the survey asked respondents about 11 issues related to grade utility and dependability. For these 11 dimensions of confidence, and for indications of proficiency, stakeholders were only asked to respond about qualifications they were aware of. If respondents were aware of both GCSEs and A levels, they were asked about one of these qualification types (assigned at random). Following this, respondents were asked about vocational and technical qualifications: Functional Skills; BTECs, Cambridge nationals and other similar vocational qualifications; Tech Awards and Tech Levels; and professional qualifications. If they were aware of more than one of these qualifications, they were only shown the set of 11 statements about one of these qualification types (assigned at random).

Sampling

The sampling of participants was structured to be representative of the relevant target populations: senior leaders in schools and colleges, teachers, students, parents, the general public, employers, and HEIs. Table 1 outlines the base sizes for each stakeholder group. Sampling took a repeated cross-sectional approach, whereby new respondents undertook the survey at each wave.

Questions were routed to ensure the qualification-specific questions were only asked to those aware of the relevant qualification. Table 2 outlines the relevant base sizes for the groups who saw the routed questions. While the sampling approach was successful in generating representative sample groups, it should be acknowledged that responses attributed to stakeholder groups reflect only the views of those respondents included in the study.

Table 1: base sizes for each stakeholder group

Wave Higher Education Institutions (HEIs) Students Senior Leaders Teachers Parents The General Public Employers All stakeholders
Wave 1 345 631 537 582 604 429 405 3,533
Wave 2 165 606 360 586 602 411 434 3,164
Wave 3 180 606 489 415 613 454 497 3,254
Wave 4 373 605 438 554 606 549 440 3,565
Total 1,063 2,448 1,824 2,137 2,425 1,843 1,776 13,516

Table 2: base sizes for those aware of each qualification

Wave GCSEs A levels Functional Skills BTECs, Cambridge nationals and other similar vocational qualifications Tech Awards and Tech Levels Professional qualifications
Wave 1 1,560 1,902 678 1,156 664 616
Wave 2 1,470 1,613 634 1,081 599 528
Wave 3 1,510 1,689 613 1,096 609 565
Wave 4 1,559 1,920 688 1,171 675 654
Total 6,099 7,124 2,613 4,504 2,547 2,363

Senior leaders and teachers sample

Sample coverage

The senior leaders and teachers samples were designed to be representative of senior leaders (defined as head teachers, deputy head teachers, and other members of senior leadership teams within schools) and teachers of AS and A levels, GCSEs or other academic and vocational qualifications across England. This provided coverage across the following types of establishment:

  • Local authority maintained schools
  • Academy and Free schools
  • Independent schools
  • Further Education (FE) colleges and sixth form centres

To make sure that the survey represented establishments that were teaching qualifications for people aged 14 to 19 years, only certain types of schools/colleges were eligible for the survey. These were: secondary and middle-deemed-secondary schools, academies teaching key stage 4 or post-16, independent schools teaching key stage 4 or post-16, and post-16 institutions.

Sample frame

The samples of senior leaders and teachers were drawn from two sources: the Education Company contact database, and the YouGov panel.

Education Company contact database

Firstly, contacts in schools were drawn from the Education Company’s database of publicly available education sector contacts. This list is the most accurate and comprehensive education dataset available anywhere in the UK. It contains over 5 million education data profiles, including 400,000 named teachers or lecturers, and details of more than 100,000 schools.

The Education Company’s database was used to select contacts to be surveyed. As data is held on school characteristics, this allowed us to develop a sample that was representative of school size, type and region. Sample targets were set and the total sample was structured to meet the proportions relevant to the school population in England. The database contains named contacts with generic school email addresses. This allowed the survey to be targeted at specific subject teachers in the specified schools.

YouGov Panel

The survey was also conducted using teachers and senior leaders registered with the YouGov panel; who have consented to participate in surveys with YouGov. A quota sampling approach was used on the sample drawn from the YouGov panel to ensure broad representation by school type and region.

Sample selection

The sample was selected to ensure results are representative in line with the school and teacher population in England. The sample frame was based upon official government statistics on the school and teacher population. The sample frame contained the following variables:

  • School type: maintained, academy, special school or PRU, independent
  • English region

Using the Education Company’s database, a stratified random sample of school contacts was drawn in line with the above parameters. The sample drawn was compared with the Department for Education (DfE) school population statistics to ensure it was broadly in line with the overall population. In total, a sample of 8,000 school contacts that matched the sample criteria was drawn at random from the Education Company database.

The senior leader sample was drawn at random from the list of eligible schools. The teacher sample was drawn to take into account the numbers of teachers at different types of establishment. As such, multiple teacher contacts were included for schools of a larger size.

Alongside the sample drawn from the Education Company database, a random sample of 4,000 senior leaders and teachers from the YouGov panel was contacted to take part in the survey. Tables 3 to 6 outline the senior leader and teacher base sizes, by school type and region.

Table 3: base sizes for senior leaders across the sample frame, by school type

School type Wave 1 Wave 2 Wave 3 Wave 4 Total
Maintained school 115 78 93 91 377
Academy 286 191 288 233 998
Independent 91 51 70 76 388
Further Education College 63 19 29 24 135
Other 11 21 9 14 55

Table 4: base sizes for senior leaders across the sample frame, by region

Region Wave 1 Wave 2 Wave 3 Wave 4 Total
North East England 30 19 27 21 97
North West England 72 46 67 59 244
Yorkshire and Humberside 45 45 50 53 193
East Midlands 46 24 44 27 141
West Midlands 51 38 47 41 177
Eastern England 55 35 47 37 174
London 91 44 62 57 254
South East England 102 69 90 96 357
South West England 45 40 56 47 188

Table 5: base sizes for teachers across the sample frame, by school type

School type Wave 1 Wave 2 Wave 3 Wave 4 Total
Maintained school 118 116 67 136 437
Academy 316 302 229 279 1,126
Independent 66 78 52 54 250
Further Education College 109 57 47 65 278
Other 18 33 20 20 91

Table 6: base sizes for teachers across the sample frame, by region

Region Wave 1 Wave 2 Wave 3 Wave 4 Total
North East England 33 40 28 35 136
North West England 74 74 48 77 273
Yorkshire and Humber 69 55 51 54 229
East Midlands 58 46 33 50 187
West Midlands 50 86 48 57 241
Eastern England 41 57 32 50 180
London 81 68 52 74 275
South East England 109 104 87 105 405
South West England 61 56 36 52 205

Public, parents, and students samples

Sample coverage

The public, students, and parents’ samples were drawn from the YouGov online panel of over 1.8 million adults who have consented to participate in surveys with YouGov.

Three distinct samples were drawn. These were:

  • The general public: A nationally representative survey sample of adults aged 16 years or over in England - this was sampled to be representative of the adult population in England by age interlocked with gender, region, and socio-economic classification.
  • Students: A sample of young people aged 14 to 19 years who were formally assessed on one of the in-scope qualification in 2021 - this was sampled to be representative of 14- to 19-year-olds in England by gender, age, and region.
  • Parents: A sample of parents and carers of young people who met the ‘student’ criteria defined above – this was sampled to be representative of parents in England by gender, socio-economic classification, and region.

The samples were selected to ensure the results were representative in line with the actual population of each group in England.

Sample frame

The samples of the public, parents, and students were drawn from the YouGov panel. Over the last 22 years, YouGov has carefully recruited a panel of over 1.8 million UK adults to take part in our surveys. Tables 7 to 16 outline the base sizes against the variables within the sample frames for the general public, parents and students.

Sample selection

For nationally representative samples, YouGov draws a sub-sample of the panel that is representative in terms of age and gender combined, socio-economic classification, and region - inviting this sub-sample to complete a survey.

YouGov has a proprietary, automated sampling system that invites respondents based on their profile information and how that aligns with targets for surveys that are currently active. Respondents are automatically, randomly selected based on survey availability and how that matches their profile information.

Table 7: base sizes for the general public across the sample frame, by gender

Gender Wave 1 Wave 2 Wave 3 Wave 4 Total
Male 199 191 211 287 888
Female 230 220 243 262 955

Table 8: base sizes for the general public across the sample frame, by age

Age Wave 1 Wave 2 Wave 3 Wave 4 Total
18-29 86 85 67 98 336
30-39 76 61 91 80 308
40-49 69 59 76 59 263
50-59 81 64 80 109 334
60+ 117 142 140 203 602

Table 9: base sizes for the general public across the sample frame, by region

Region Wave 1 Wave 2 Wave 3 Wave 4 Total
North East England 19 20 24 27 90
North West England 55 54 61 48 218
Yorkshire and the Humber 42 41 48 54 185
East Midlands 37 37 36 55 165
West Midlands 47 39 48 61 195
East of England 48 47 42 71 208
London 69 66 75 75 285
South East England 69 66 70 77 282
South West England 43 41 50 81 215
Table 10: base sizes for the general public across the sample frame, by socio-economic classification
Socio-economic classification Wave 1 Wave 2 Wave 3 Wave 4 Total
ABC1 (middle class) 261 235 270 337 1,103
C2DE (working class) 168 176 184 212 740

Table 11: base sizes for parents across the sample frame, by gender

Gender Wave 1 Wave 2 Wave 3 Wave 4 Total
Male 296 295 285 272 1,148
Female 308 307 328 334 1,277

Table 12: base sizes for parents across the sample frame, by region

Region Wave 1 Wave 2 Wave 3 Wave 4 Total
North East England 30 29 30 35 124
North West England 80 77 82 87 326
Yorkshire and the Humber 60 59 60 67 246
East Midlands 54 54 55 36 199
West Midlands 66 66 66 75 273
East of England 68 68 69 68 273
London 89 90 83 57 319
South East England 97 98 101 110 406
South West England 60 61 67 71 259

Table 13: base sizes for parents across the sample frame, by socio-economic classification

Socio-economic classification Wave 1 Wave 2 Wave 3 Wave 4 Total
ABC1 (middle class) 362 358 368 380 1,468
C2DE (working class) 242 244 245 226 957

Table 14: base sizes for students across the sample frame, by gender

Gender Wave 1 Wave 2 Wave 3 Wave 4 Total
Male 312 279 251 241 1,038
Female 319 327 355 364 1,365

Table 15: base sizes for students across the sample frame, by age

Age Wave 1 Wave 2 Wave 3 Wave 4 Total
15 110 102 112 107 431
16-17 275 268 227 303 1,073
18-19 246 236 267 195 944

Table 16: base sizes for students across the sample frame, by region

Region Wave 1 Wave 2 Wave 3 Wave 4 Total
North East England 32 33 28 28 121
North West England 82 77 76 65 300
Yorkshire and the Humber 56 51 57 52 216
East Midlands 55 54 56 56 221
West Midlands 70 65 57 62 254
East of England 71 63 69 63 266
London 96 99 97 98 390
South East England 105 106 101 126 438
South West England 64 58 65 55 242

Employer sample

Sample coverage

The employer sample was designed to be representative of employment in England by organisation size and sector. The sample definition ensured that only those in middle management and above roles, with the ability to answer on behalf of their organisation, were eligible to take the survey. A further criterion was added to ensure that each employer had recruited a young person aged 16 to 25 years in the last 12 months.

Sample frame

The employer sample was drawn from the YouGov business panel, targeting senior decision makers and employees with decision making responsibility for HR or personnel within their organisation.

Sample selection

The sample was selected to ensure the results are representative in line with the business population in England using the following criteria drawn from the Office of National Statistics Business Population Statistics. Tables 17 and 18 outline the base sizes against the variables within the sample frame for employers.

  • Employer size: proportionate number of employers in the 2 to 9, 10 to 49, 50 to 99, 100 to 249, and 250+ employee size band brackets; this was calculated using the percentage on employment that each size band represents rather than the number of enterprises within each.
  • Sector: broad sector coverage was ensured across the private, public, and third or voluntary sector.
Table 17: base sizes for employers across the sample frame, by work sector
Work sector Wave 1 Wave 2 Wave 3 Wave 4 Total
Private sector 314 281 329 322 1,246
Public sector 63 94 109 76 342
Third or voluntary sector 28 59 59 42 188
Table 18: base sizes for employers across the sample frame, by business size
Business size Wave 1 Wave 2 Wave 3 Wave 4 Total
2-49 employees 129 166 109 150 554
50-249 employees 65 71 128 69 333
250 or more employees 211 197 260 221 889

Higher Education Institution sample

Sample coverage

The Higher Education Institution (HEI) sample was designed to be representative of the English university population by university type. The type of university was drawn from official Higher Education Statistics Authority data. Universities who responded to the survey were grouped into the following categories:

  • Russell Group
  • Other old Universities
  • Post 1992 or new Universities
  • Other HEI

The sample definition ensured that only those in academic roles with responsibility for and knowledge of the admissions process were eligible to complete the survey.

Sample frame

The HEI sample was drawn from two sources:

  • The YouGov education panel
  • The Oscar Research data base of public sector contacts: Oscar Research are the UK Public Sector database specialists. They provide the largest, and most accurate, database of Government and Public Sector contacts and organisations in the UK.

Sample selection

The samples were selected to ensure the results are representative in line with the university population in England by university type. No further restrictions were placed on the sample but responses were monitored throughout the fieldwork period to ensure broad coverage across age, gender, and job role. Tables 19 to 21 outline the base size across these key demographics.

Table 19: base size for higher education institution respondents, by gender

Gender Wave 1 Wave 2 Wave 3 Wave 4 Total
Male 175 95 87 180 537
Female 161 70 93 190 514

Table 20: base size for higher education institution respondents, by age

Age Wave 1 Wave 2 Wave 3 Wave 4 Total
18-29 14 6 7 24 51
30-39 80 43 56 83 272
40-49 79 42 54 91 266
50-59 95 45 42 105 287
60+ 51 29 21 50 151

Table 21: base size for higher education institution respondents, by job role

Job role Wave 1 Wave 2 Wave 3 Wave 4 Total
Chancellor or Vice-chancellor 2 2 0 0 4
Dean of faculty 22 2 2 9 35
Head of department or school 61 6 10 32 109
Department or school director of studies 14 11 12 11 48
Professor 36 21 14 43 114
Senior lecturer or senior teaching fellow 86 52 59 82 279
Lecturer, or clinical lecturer or teaching fellow 79 43 53 51 226
Researcher 13 8 4 15 40
Head of admissions 4 0 1 2 7
Admissions manager or officer 7 3 4 27 41

Ethical considerations

The majority of the responses came from YouGov panellists, all who have consented to taking surveys on the YouGov panel.

Respondents are contacted by email and invited to take part in an online survey without knowing the subject prior. YouGov uses a brief, generic email invitation, which only informs the respondent that they are invited to a survey. This helps to minimise bias from those opting in or out based on level of interest in the survey topic.

Respondents were shown an introductory page, which explained that all results would remain anonymous and confidential. This means that it is not possible to identify individual respondents from their responses. After reading the full introductory page, respondents consented to taking part in the full survey.

The data was collected and stored in accordance with the General Data Protection Regulation (GDPR) and Data Protection Act 2018 which protects personal information. YouGov and Ofqual acted as joint data controller for this project, and take responsibility for the protection of personal information collected in the study.

Composite confidence measure

The ‘trends by stakeholder’ chapter in the findings report presents a composite confidence measure, which was developed as a means of quantifying overall confidence in qualifications and exploring changes between waves.

The composite confidence measure was calculated using 11 themes: reliability, bias, comparability, usefulness as a measure of knowledge, usefulness for university admissions, usefulness for job recruitment, fairness, trust, aptitude for studying in general, aptitude for studying specific subjects, and the potential for success in students’ future careers. It was calculated separately for perceptions of ‘before the pandemic’ and ‘this year’ (i.e. 2021).

These 11 themes were presented to respondents as separate statements within a Likert-scale question, which ranged from strong agreement to strong disagreement. The questions were asked separately for each of the six qualifications examined. Based on respondents’ selections, responses were assigned a score between 1 and 5 for each of the 11 criteria. ‘Don’t know’ responses were excluded from the calculation. The scoring system is detailed below.

  • Strongly agree = 5
  • Agree = 4
  • Neither agree nor disagree = 3
  • Disagree = 2
  • Strongly disagree = 1

Next, an average of the 11 individual criteria scores was calculated for each respondent, which represents the overall confidence measure. Finally, an average of the overall scores for all respondents was calculated to produce one composite confidence score per stakeholder group as well as for the total combined sample. This was done individually for each of the six qualifications, looking separately at perceptions of ‘before the pandemic’ and ‘this year’ (i.e. 2021).

Guidance on analysis

The survey was undertaken with seven different sample groups. Data for each sample group has been combined and weighted so that each group represents an equal proportion within the ‘All stakeholder’ group.

Throughout the findings report, percentages are rounded to the nearest whole number. Due to rounding, percentages may not always add up to 100%. The report summarises the data for each question using charts and tables. The intention of the findings report is to summarise the main themes in the data and not produce a full replication of the entire dataset collected.

The main report focuses on exploring perceptions at the level of ‘All stakeholders’, by wave and by qualification, with some narrative on perceptions of individual stakeholder groups. Note that all analyses, including those for perceptions of individual stakeholder groups, is provided in the data tables. The data tables also provide a breakdown of responses by demographic variables: region, social grade and ethnicity – note that these breakdowns are available only for some respondent groups, namely the general public, students and parents. This is because this data is already routinely held by YouGov for panellists in these groups, and therefore allowed for this analysis post-hoc.

Reported differences are based on statistical significance testing as tested at the 95% confidence level. Whenever a difference between waves or groups within the sample is reported, this refers to a statistically significant difference. Statistically significant differences are reported in full in the data tables.

Appendix – Survey

The full questionnaire can be found below. Note that the questions were adapted slightly to bring the tense of the wording into the appropriate context, depending on the wave it was delivered in (e.g. changing ‘will be’ to ‘were’ in the post-results phase).

The full base for each question can be found in the data tables.

Survey Questions

Qualification routing question

Base: All

Which, if any, of the following are you aware of? (Please select all that apply)

  1. a. GCSEs
  2. b. A levels
  3. c. Functional Skills
  4. d. BTECs, Cambridge (OCR) nationals or other similar vocational qualifications
  5. e. Tech Awards and Tech Levels
  6. f. Professional qualifications (those equivalent to GCSE and A Level, not degree level)
  7. g. Don’t know
  8. h. None of these

Q1

Base: All

This year, the following arrangements for grading students have been put in place.

  • Grades for GCSEs and A levels will be based on teachers’ assessments this year.
  • Grades for qualifications such as BTECS and OCR Nationals (And other qualifications that are not GCSEs or A levels, but are often taught alongside or instead of them, and taught in schools and colleges) will be based on teachers’ assessments this year.
  • Teachers’ assessment grades will be based on what a student knows, understands and can do at the time of the assessment, and will be based only on the content students have been taught.
  • Exam boards are required to provide exam questions and mark schemes that teachers could (but don’t have to) use to help grade their students.
  • School and colleges are required to check and confirm the accuracy of their teacher assessment grades for all students.
  • Exam boards are required to check schools’ and colleges’ approaches to quality assurance, and to inspect the evidence used in a sample of schools and colleges.

Based on the information you just read…

To what extent do you agree or disagree with the following statement? It is clear to me how students [will be (W1-W3); were (W4) graded this year

  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. vi. Don’t know

Q2[footnote 1]

Which, if any, of the following sources of information made you aware of the arrangements for grading students this year? (Please select all that apply)

  1. School, college or training provider
  2. Social media
  3. News articles
  4. Radio
  5. TV
  6. Word of mouth
  7. Friends or family
  8. I know a teacher - they told me about the arrangements
  9. Government publication
  10. Exam board or awarding organisation
  11. Professional body or union
  12. Other, please specify
  13. I don’t know or I can’t recall
  14. N/A – I wasn’t previously aware of any of the arrangements for grading students this year

Q3a

Base: Those aware of GCSEs[footnote 2]

For the following question, when we say ‘before the pandemic’ we mean any time before March 2020. Comparing your views on the accuracy and usefulness of grades before the pandemic with your views on the accuracy and usefulness of [grades (W1-W3); grades students have received (W4)] this year, indicate the degree to which you agree or disagree with the following statements.

In general, GCSE grades…

  1. Were reliable before the pandemic
  2. [Will be (W1-W3); are (W4)] reliable this year
  3. Were free from bias before the pandemic
  4. [Will be (W1-W3); are (W4)] free from bias this year
  5. Were comparable across schools and colleges (representing the same standard) before the pandemic
  6. [Will be (W1-W3); are (W4)] comparable across schools and colleges (representing the same standard) this year
  7. Were useful for judging what a student knows, understands, and can do before the pandemic
  8. [Will be (W1-W3); are (W4)] useful for judging what a student knows, understands, and can do this year
  9. Were useful for university admissions before the pandemic
  10. [Will be (W1-W3); are (W4)] useful for university admissions this year
  11. Were useful for job recruitment before the pandemic
  12. [Will be (W1-W3); are (W4)] useful for job recruitment this year
  13. Were fair to students before the pandemic
  14. [Will be (W1-W3); are (W4)] fair to students this year
  15. Were trusted by people before the pandemic
  16. [Will be (W1-W3); are (W4)] trusted by people this year
  17. Were a good indicator of a student’s aptitude to study a particular subject before the pandemic
  18. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study a particular subject this year
  19. Were a good indicator of a student’s aptitude to study in general before the pandemic
  20. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study in general this year
  21. Were a good indicator of a student’s potential for success in their future career *before the pandemic
  22. [Will be (W1-W3); are (W4)] a good indicator of a student’s potential for success in their future career this year
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q3b

Base: Those aware of A levels[footnote 3]

For the following question, when we say ‘before the pandemic’ we mean any time before March 2020. Comparing your views on the accuracy and usefulness of grades before the pandemic with your views on the accuracy and usefulness of [grades (W1-W3); grades students have received (W4)] this year, indicate the degree to which you agree or disagree with the following statements.

In general, A level grades…

  1. Were reliable before the pandemic
  2. [Will be (W1-W3); are (W4)] reliable this year
  3. Were free from bias before the pandemic
  4. [Will be (W1-W3); are (W4)] free from bias this year
  5. Were comparable across schools and colleges (representing the same standard) before the pandemic
  6. [Will be (W1-W3); are (W4)] comparable across schools and colleges (representing the same standard) this year
  7. Were useful for judging what a student knows, understands, and can do before the pandemic
  8. [Will be (W1-W3); are (W4)] useful for judging what a student knows, understands, and can do this year
  9. Were useful for university admissions before the pandemic
  10. [Will be (W1-W3); are (W4)] useful for university admissions this year
  11. Were useful for job recruitment before the pandemic
  12. [Will be (W1-W3); are (W4)] useful for job recruitment this year
  13. Were fair to students before the pandemic
  14. [Will be (W1-W3); are (W4)] fair to students this year
  15. Were trusted by people before the pandemic
  16. [Will be (W1-W3); are (W4)] trusted by people this year
  17. Were a good indicator of a student’s aptitude to study a particular subject before the pandemic
  18. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study a particular subject this year
  19. Were a good indicator of a student’s aptitude to study in general before the pandemic
  20. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study in general this year
  21. Were a good indicator of a student’s potential for success in their future career *before the pandemic
  22. [Will be (W1-W3); are (W4)] a good indicator of a student’s potential for success in their future career this year
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q4a

Base: Those aware of GCSEs who were shown Q3a

To what extent do you agree or disagree with the following statements?

  1. Before the pandemic, a GCSE grade 4 (equivalent of an old grade C) indicated that a student has a good understanding of the subject.
  2. This year, a GCSE grade 4 (equivalent of an old grade C) [will indicate (W1-W3); indicates (W4)] that a student has a good understanding of the subject.
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q4b

Base: Those aware of A levels who were shown Q3b

To what extent do you agree or disagree with the following statements?

  1. Before the pandemic, an A level pass (grade E) indicated that a student has a satisfactory understanding of the subject.
  2. This year, an A level pass (grade E) [will indicate (W1-W3); indicates (W4)] that a student has a satisfactory understanding of the subject.
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q5a

Base: Those aware of Functional Skills[footnote 4]

Earlier you told us that you were aware of Functional Skills. Please respond to these next questions about accuracy and usefulness of Functional Skills grades.

For the following question, when we say ‘before the pandemic’ we mean any time before March 2020. Comparing your views on the accuracy and usefulness of grades before the pandemic with your views on the accuracy and usefulness of [grades (W1-W3); grades students have received (W4)] this year, indicate the degree to which you agree or disagree with the following statements.

In general, Functional Skills grades…

  1. Were reliable before the pandemic
  2. [Will be (W1-W3); are (W4)] reliable this year
  3. Were free from bias before the pandemic
  4. [Will be (W1-W3); are (W4)] free from bias this year
  5. Were comparable across schools and colleges (representing the same standard) before the pandemic
  6. [Will be (W1-W3); are (W4)] comparable across schools and colleges (representing the same standard) this year
  7. Were useful for judging what a student knows, understands, and can do before the pandemic
  8. [Will be (W1-W3); are (W4)] useful for judging what a student knows, understands, and can do this year
  9. Were useful for university admissions before the pandemic
  10. [Will be (W1-W3); are (W4)] useful for university admissions this year
  11. Were useful for job recruitment before the pandemic
  12. [Will be (W1-W3); are (W4)] useful for job recruitment this year
  13. Were fair to students before the pandemic
  14. [Will be (W1-W3); are (W4)] fair to students this year
  15. Were trusted by people before the pandemic
  16. [Will be (W1-W3); are (W4)] trusted by people this year
  17. Were a good indicator of a student’s aptitude to study a particular subject before the pandemic
  18. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study a particular subject this year
  19. Were a good indicator of a student’s aptitude to study in general before the pandemic
  20. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study in general this year
  21. Were a good indicator of a student’s potential for success in their future career *before the pandemic
  22. [Will be (W1-W3); are (W4)] a good indicator of a student’s potential for success in their future career this year
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q5b

Base: Those aware of BTECs, Cambridge nationals and other similar vocational qualifications[footnote 5]

Earlier you told us that you were aware of BTECs or Cambridge (OCR) nationals or other similar vocational qualifications. Please respond to these next questions about accuracy and usefulness of these grades.

For the following question, when we say ‘before the pandemic’ we mean any time before March 2020. Comparing your views on the accuracy and usefulness of grades before the pandemic with your views on the accuracy and usefulness of [grades (W1-W3); grades students have received (W4)] this year, indicate the degree to which you agree or disagree with the following statements.

In general, BTEC grades…

  1. Were reliable before the pandemic
  2. [Will be (W1-W3); are (W4)] reliable this year
  3. Were free from bias before the pandemic
  4. [Will be (W1-W3); are (W4)] free from bias this year
  5. Were comparable across schools and colleges (representing the same standard) before the pandemic
  6. [Will be (W1-W3); are (W4)] comparable across schools and colleges (representing the same standard) this year
  7. Were useful for judging what a student knows, understands, and can do before the pandemic
  8. [Will be (W1-W3); are (W4)] useful for judging what a student knows, understands, and can do this year
  9. Were useful for university admissions before the pandemic
  10. [Will be (W1-W3); are (W4)] useful for university admissions this year
  11. Were useful for job recruitment before the pandemic
  12. [Will be (W1-W3); are (W4)] useful for job recruitment this year
  13. Were fair to students before the pandemic
  14. [Will be (W1-W3); are (W4)] fair to students this year
  15. Were trusted by people before the pandemic
  16. [Will be (W1-W3); are (W4)] trusted by people this year
  17. Were a good indicator of a student’s aptitude to study a particular subject before the pandemic
  18. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study a particular subject this year
  19. Were a good indicator of a student’s aptitude to study in general before the pandemic
  20. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study in general this year
  21. Were a good indicator of a student’s potential for success in their future career *before the pandemic
  22. [Will be (W1-W3); are (W4)] a good indicator of a student’s potential for success in their future career this year
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q5c

Base: Those aware of Tech Awards and Tech Levels[footnote 6]

Earlier you told us that you were aware of Tech Awards and Tech Levels. Please respond to these next questions about accuracy and usefulness of these grades

For the following question, when we say ‘before the pandemic’ we mean any time before March 2020. Comparing your views on the accuracy and usefulness of grades before the pandemic with your views on the accuracy and usefulness of [grades (W1-W3); grades students have received (W4)] this year, indicate the degree to which you agree or disagree with the following statements.

In general, Tech Awards and Tech Levels grades…

  1. Were reliable before the pandemic
  2. [Will be (W1-W3); are (W4)] reliable this year
  3. Were free from bias before the pandemic
  4. [Will be (W1-W3); are (W4)] free from bias this year
  5. Were comparable across schools and colleges (representing the same standard) before the pandemic
  6. [Will be (W1-W3); are (W4)] comparable across schools and colleges (representing the same standard) this year
  7. Were useful for judging what a student knows, understands, and can do before the pandemic
  8. [Will be (W1-W3); are (W4)] useful for judging what a student knows, understands, and can do this year
  9. Were useful for university admissions before the pandemic
  10. [Will be (W1-W3); are (W4)] useful for university admissions this year
  11. Were useful for job recruitment before the pandemic
  12. [Will be (W1-W3); are (W4)] useful for job recruitment this year
  13. Were fair to students before the pandemic
  14. [Will be (W1-W3); are (W4)] fair to students this year
  15. Were trusted by people before the pandemic
  16. [Will be (W1-W3); are (W4)] trusted by people this year
  17. Were a good indicator of a student’s aptitude to study a particular subject before the pandemic
  18. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study a particular subject this year
  19. Were a good indicator of a student’s aptitude to study in general before the pandemic
  20. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study in general this year
  21. Were a good indicator of a student’s potential for success in their future career *before the pandemic
  22. [Will be (W1-W3); are (W4)] a good indicator of a student’s potential for success in their future career this year
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q5d

Base: Those aware of professional qualifications[footnote 7]

Earlier you told us that you were aware of professional qualifications. Please respond to these next questions about accuracy and usefulness of these grades

For the following question, when we say ‘before the pandemic’ we mean any time before March 2020.Comparing your views on the accuracy and usefulness of grades before the pandemic with your views on the accuracy and usefulness of [grades (W1-W3); grades students have received (W4)] this year, indicate the degree to which you agree or disagree with the following statements.

In general, professional qualifications grades awarded grades…

  1. Were reliable before the pandemic
  2. [Will be (W1-W3); are (W4)] reliable this year
  3. Were free from bias before the pandemic
  4. [Will be (W1-W3); are (W4)] free from bias this year
  5. Were comparable across schools and colleges (representing the same standard) before the pandemic
  6. [Will be (W1-W3); are (W4)] comparable across schools and colleges (representing the same standard) this year
  7. Were useful for judging what a student knows, understands, and can do before the pandemic
  8. [Will be (W1-W3); are (W4)] useful for judging what a student knows, understands, and can do this year
  9. Were useful for university admissions before the pandemic
  10. [Will be (W1-W3); are (W4)] useful for university admissions this year
  11. Were useful for job recruitment before the pandemic
  12. [Will be (W1-W3); are (W4)] useful for job recruitment this year
  13. Were fair to students before the pandemic
  14. [Will be (W1-W3); are (W4)] fair to students this year
  15. Were trusted by people before the pandemic
  16. [Will be (W1-W3); are (W4)] trusted by people this year
  17. Were a good indicator of a student’s aptitude to study a particular subject before the pandemic
  18. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study a particular subject this year
  19. Were a good indicator of a student’s aptitude to study in general before the pandemic
  20. [Will be (W1-W3); are (W4)] a good indicator of a student’s aptitude to study in general this year
  21. Were a good indicator of a student’s potential for success in their future career *before the pandemic
  22. [Will be (W1-W3); are (W4)] a good indicator of a student’s potential for success in their future career this year
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q6a

Base: Those aware of Functional Skills and shown Q5a

To what extent do you agree or disagree with the following statements:

  1. Before the pandemic, a pass in Functional Skills indicated that a student has a satisfactory understanding of the subject.
  2. This year, a pass in Functional Skills [will indicate (W1-W3); indicates (W4)] that a student has a satisfactory understanding of the subject.
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q6b

Base: Those aware of BTECs, Cambridge nationals or other similar vocational qualifications and shown Q5b

To what extent do you agree or disagree with the following statements:

  1. Before the pandemic, a pass in BTECs or Cambridge (OCR) nationals or other similar vocational qualifications indicated that a student has a satisfactory understanding of the subject.
  2. This year, a pass in BTECs or Cambridge (OCR) nationals or other similar vocational qualifications [will indicate (W1-W3); indicates (W4)] that a student has a satisfactory understanding of the subject.
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q6c

Base: Those aware of Tech Awards and Tech Levels and shown Q5c

To what extent do you agree or disagree with the following statements:

  1. Before the pandemic, a pass in Tech Awards and Tech Levels indicated that a student has a satisfactory understanding of the subject.
  2. This year, a pass in Tech Awards and Tech Levels [will indicate (W1-W3); indicates (W4)] that a student has a satisfactory understanding of the subject
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q6d

Base: Those aware of professional qualifications and shown Q5d

To what extent do you agree or disagree with the following statements:

  1. Before the pandemic, a pass in professional qualifications indicated that a student has a satisfactory understanding of the subject.
  2. This year, a pass in professional qualifications [will indicate (W1-W3); indicates (W4)] that a student has a satisfactory understanding of the subject.
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q7

Base: All

The following questions will ask about your views around teachers’ assessments this year.

To what extent do you agree or disagree with the following statements regarding teachers’ assessments this year?

  1. I believe that teachers have sufficient expertise in assessing students to grade them accurately
  2. I believe that teachers [will be (W1-W3); were (W4)] provided with sufficient support and guidance to enable them to grade students accurately
  3. I believe that teachers [will have (W1-W3); had (W4)] enough evidence for each student to enable them to grade students accurately
  4. I trust that teachers [will grade (W1-W3); graded (W4)] students neither too harshly nor too generously
  5. I believe that students [will generally get (W1-W3); generally got (W4)] the results they deserve this year
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q8

Base: All

Thinking now about different types of students, to what extent do you agree or disagree that the following students [will generally get (W1-W3); generally got (W4)] the results they deserve this year:

  1. Students from advantaged backgrounds
  2. Students from disadvantaged backgrounds
  3. Students with Special Educational Needs
  4. Disabled students
  5. BAME students
  6. Adult learners
  7. Private candidates
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q9

Base: All

The following questions will ask about your views around the appeals process this year.

If a student believes that they have been awarded the wrong grade, they may submit an appeal: either to their school or college (if they believe there has been a procedural error, e.g. typing the wrong grade in); or to their exam board (if they believe their school or college has made an error of professional judgement). [This process will be available until the 17th December (W4)].

Grades will be changed where there is evidence of a procedural error, or a significant error of professional judgement. However, grades will not be changed where there is only a marginal difference of professional opinion over the appropriate grade to award. To what extent do you agree or disagree with the following statements?

  1. I am confident that incorrect grades as a result of a procedural error will be corrected.
  2. I am confident that significant errors of professional judgement by the school or college will be corrected.
  3. It is right that marginal differences of professional opinion about a particular grade will not be addressed by the appeals process.
  4. In general, the appeals process is fair.
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

Q10

Base: All

Given the impact of the pandemic on teaching and learning, to what extent do you agree or disagree with the following statements about grading students in summer 2021?

  1. The arrangements for grading students that [are (W1-W3); were (W4)] in place are as good as can be given the circumstances
  2. [If everything goes to plan, students will be (W1-W3); Overall, students were (W4)] graded fairly in 2021
  1. i. Strongly agree
  2. ii. Slightly agree
  3. iii. Neither agree nor disagree
  4. iv. Slightly disagree
  5. v. Strongly disagree
  6. i. Don’t know

  1. Note that the responses to this question are not provided in the main report, but are provided in the data tables. ↩

  2. Those who were aware of both GCSEs and A levels were allocated to answer either Q3a (GCSEs) or Q3b (A levels) ↩

  3. Those who were aware of both GCSEs and A levels were allocated to answer either Q3a (GCSEs) or Q3b (A levels) ↩

  4. Those who were aware of more than one qualification out of Functional Skills; BTECs, Cambridge nationals or other similar vocational qualifications; Tech Levels and Tech Awards; and professional qualifications were shown Q5 in the context of one of these qualifications. ↩

  5. Those who were aware of more than one qualification out of Functional Skills; BTECs, Cambridge nationals and other similar vocational qualifications; Tech Levels and Tech Awards; and professional qualifications were shown Q5 in the context of one of these qualifications. ↩

  6. Those who were aware of more than one qualification out Functional Skills; BTECs, Cambridge nationals and other similar vocational qualifications; Tech Levels and Tech Awards; and professional qualifications were shown Q5 in the context of one of these qualifications. ↩

  7. Those who were aware of more than one qualification out of Functional Skills; BTECs, Cambridge nationals and other similar vocational qualifications; Tech Levels and Tech Awards; and professional qualifications were shown Q5 in the context of one of these qualifications. ↩

Back to top