An investigation of inter-subject comparability in GCSEs and A levels in summer 2021
Published 12 July 2022
Applies to England
© Crown copyright 2022
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/publications/an-investigation-of-inter-subject-comparability-in-gcses-and-a-levels-in-summer-2021/an-investigation-of-inter-subject-comparability-in-gcses-and-a-levels-in-summer-2021
Summary
This technical report, which is a follow-up to analysis that was published in summer 2020 (He and Black, 2020), uses statistical methods to investigate the comparability of grading standards across subjects for GCSEs and A levels in summer 2021.
This inter-subject comparability is explored over a period of 3 years: summer 2021, when teacher-assessed grades (TAGs) were used, summer 2020, when, for most students, centre assessment grades (CAGs) were used, and summer 2019, when grades were awarded through exams, in the normal way. The analysis contributes to our understanding of the impact of the coronavirus (COVID-19) pandemic on grading in 2020 (CAGs) and 2021 (TAGs).
Before describing the findings from this analysis, it is important to be clear what we mean by ‘subject difficulty’, which has a specific statistical meaning in this report. Statistical approaches to inter-subject comparability use cohort-level data to compare the relative likelihood of students achieving a grade in certain subjects, on the basis of past attainment or concurrent attainment derived using grades achieved in all GCSE or A level subjects. This involves an assumption that there is a common underlying dimension of ‘ability’ which allows us to make meaningful comparisons between subjects as diverse as, for example, French and Physics. This needs to be borne in mind when interpreting the findings presented in this report. Although such statistical information is valuable, there are limitations to the extent to which strong conclusions may be drawn. It would be inappropriate to use such data in isolation from other evidence when deciding, for example, whether or not specific subjects can justifiably be said to be graded more harshly than others. For a more detailed discussion of this topic, you may wish to read some of Ofqual’s previous work on inter-subject comparability (eg. Ofqual, 2015).
With this caveat in mind, the main findings from the analysis are as follows.
-
Based on relationships between subject grade outcomes and prior attainment for matched candidates, GCSE subjects were graded similarly overall in both 2021 and 2020 in terms of grading leniency, while A level subjects were graded more leniently in 2021 than in 2020. Based on Rasch modelling of all candidates, both GCSE and A level subjects were graded more leniently in 2021 than in 2020 at the top grades. Both GCSE and A level subjects were considerably more leniently graded in 2021 and 2020 when compared to 2019. This reflects the overall outcomes for 2020 and 2021 that we have reported previously (Ofqual, 2020c, 2021b).
-
For both GCSEs and A levels, the range and standard deviation (variability) of relative Rasch grade difficulties in 2021 are similar to, or slightly larger than, those in 2020 and noticeably smaller than those in 2019. This is likely to reflect the fundamentally different approaches taken to assessment and grading in both 2021 and 2020, compared to normal years in which examinations take place.
-
Our Rasch modelling approach allows us to broadly rank subjects by their mean difficulty and difficulties at individual grades. It is important to note that these statistically determined rank orders usually vary between years. For GCSEs, changes in subject difficulty rank orders between 2021 (TAGs) and 2020 (CAGs) are slightly smaller than the changes we have previously observed between 2 consecutive exam years (before the COVID-19 pandemic) at the overall subject level. The differences are slightly larger at grades 7 and 9. For A levels, both at the overall subject level and for grades C, A and A*, the size and standard deviation of changes in subject difficulty rank orders are generally slightly larger than changes we have previously seen between two consecutive years of normal exams.
In summary, the different assessment and grading approaches (CAGs in 2020, TAGs in 2021, and external exams before the pandemic) resulted in differences in inter-subject comparability in GCSEs and A levels. This technical report provides information about the nature of how the relative difficulty of subjects, as defined statistically, has differed between years.
1. Introduction
Exams in GCSEs and A levels in both 2020 and 2021 were cancelled due to the pandemic. In 2020, students received either the grade provided by their centre (the centre assessment grade – CAG), or the “calculated grade” produced through a standardisation process, whichever was higher (see Ofqual, 2020a, b). In 2021, grades were determined by teachers (teacher assessment grades – TAGs) (see Ofqual, 2021a, 2021c; JCQ, 2021).
He and Black (2020) investigated the impact of the different grades (calculated grades, CAGs and the final awarded grades) in 2020 on inter-subject comparability in GCSEs and A levels using a number of statistical difficulty measures. They found that, for both GCSEs and A levels, subjects in 2020 were generally graded more leniently than in 2019 at the overall subject level and at individual grades. For both GCSEs and A levels, changes in relative grade difficulties between 2020 and 2019 were generally larger than the estimated changes between 2019 and 2018 (for both 2019 and 2018, grades were derived through normal examinations).
This report represents a follow up to the report by He and Black (2020) and forms part of a set of research reports evaluating the impact of the 2021 summer awarding. The analysis aims to answer the following main research question:
How did the 2021 teacher assessment grades impact on inter-subject comparability in relation to grades awarded in 2020 and grades awarded through normal exams?
2. Data and analysis
2.1 Data
The primary data analysed in this investigation were the results data supplied to Ofqual by exam boards (EBs), covering 2019 to 2021 for both GCSEs and A levels. The 2020 data analysed here are the final awarded grades which comprise primarily centre assessment grades (CAGs). In addition, GCSE data from 2011 to 2013[footnote 1] and A level data from 2017 and 2018 were also analysed to provide information on inter-subject comparability based on grades derived from exams taken in usual circumstances. Candidates’ prior attainment data were also used. Prior attainment was based on normalised mean KS2 scores for GCSEs, extracted from the National Pupil Database (NPD), and normalised mean GCSE scores for A level candidates, calculated from their GCSE grades awarded 2 years previously. Overall, the match rate with prior attainment data ranges from 74% to 84% for GCSE candidates and from 89% to 91% for A level candidates.
Tables 1 and 2 list the GCSE and A level subjects from 2019 to 2021 that were included in the analysis.
Table 1 GCSE subjects analysed in this report (including acronyms for some subjects used in subsequent figures and tables)
| Subject | 2019 | 2020 | 2021 |
|---|---|---|---|
| Art and Design: Three-dimensional design (AD_3DStudies) | Y | Y | Y |
| Art and Design: Art, craft and design) (AD_ACD) | Y | Y | Y |
| Art and Design: Fine art (AD_FA) | Y | Y | Y |
| Art and Design: Graphics (AD_Graphics) | Y | Y | Y |
| Art and Design: Photography (AD_Photog) | Y | Y | Y |
| Art and Design: Textile design (AD_Textiles) | Y | Y | Y |
| Arabic | Y | N | Y |
| Biology | Y | Y | Y |
| Business | Y | Y | Y |
| Combined science (Cbd_Science) | Y | Y | Y |
| Chemistry | Y | Y | Y |
| Chinese | Y | N | Y |
| Citizenship studies (Citi_Studies) | Y | Y | Y |
| Classical civilisation (Class_Civil) | Y | Y | Y |
| Computer Science (Computing) | Y | Y | Y |
| Design and technology (D&T) | Y | Y | Y |
| Dance | Y | Y | Y |
| Drama | Y | Y | Y |
| Economics | Y | Y | Y |
| English language (Eng_Lang) | Y | Y | Y |
| English literature (Eng_Lit) | Y | Y | Y |
| Film_Studies (Film studies) | Y | Y | Y |
| Food preparation and nutrition (Food_P&N) | Y | Y | Y |
| French | Y | Y | Y |
| Geography | Y | Y | Y |
| German | Y | Y | Y |
| History | Y | Y | Y |
| Italian | Y | Y | Y |
| Latin | Y | Y | Y |
| Mathematics | Y | Y | Y |
| Media studies (Media_Studies) | Y | Y | Y |
| Music | Y | Y | Y |
| hysical education (PE) | Y | Y | Y |
| Physics | Y | Y | Y |
| Polish | Y | N | N |
| Psychology | Y | Y | Y |
| Religious studies (RS) | Y | Y | Y |
| Religious studies: short course (RS_SC) | Y | Y | N |
| Sociology | Y | Y | Y |
| Spanish | Y | Y | Y |
| Statistics | Y | Y | Y |
| Urdu | Y | Y | Y |
| Total | 42 | 39 | 40 |
Table 2 A level subjects analysed (including acronyms for some subjects used in subsequent figures and tables)
| Subject | 2019 | 2020 | 2021 |
|---|---|---|---|
| Accounting | Y | Y | Y |
| Art and Design: Three-dimensional design (AD_3DS) | Y | Y | Y |
| Art and Design: Art, craft and design (AD_ACD) | Y | Y | Y |
| Art and Design: Fine art (AD_FA) | Y | Y | Y |
| Art and Design: Graphics (AD_Graphics) | Y | Y | Y |
| Art and Design: Photography (AD_Photog) | Y | Y | Y |
| Art and Design: Textile design (AD_Textiles) | Y | Y | Y |
| Biology | Y | Y | Y |
| Business studies (Bus_Studies) | Y | Y | Y |
| Chemistry | Y | Y | Y |
| Chinese | Y | Y | Y |
| Classical civilisation (Class_Civil) | Y | Y | Y |
| Computer science (Computing) | Y | Y | Y |
| Dance | Y | Y | Y |
| Drama and theatre studies (Drama_TS) | Y | Y | Y |
| Design and Technology: Product design (DT_PD) | Y | Y | Y |
| Economics | Y | Y | Y |
| Eng_Lang (English language) | Y | Y | Y |
| Eng_LangLit (English language and literature) | Y | Y | Y |
| Eng_Lit (English literature) | Y | Y | Y |
| Env_Studies (Environmental studies) | N | N | Y |
| Film_Studies (Film studies) | Y | Y | Y |
| French | Y | Y | Y |
| Further mathematics (Fur_Maths) | Y | Y | Y |
| General Studies | N | N | N |
| Geography | Y | Y | Y |
| Geology | Y | N | N |
| German | Y | Y | Y |
| History | Y | Y | Y |
| Latin | Y | N | Y |
| Law | Y | Y | Y |
| Mathematics | Y | Y | Y |
| Media studies (Media_Studies) | Y | Y | Y |
| Music | Y | Y | Y |
| Music technology (Music_Tech) | Y | Y | Y |
| Physical education (PE) | Y | Y | Y |
| Philosophy | Y | Y | Y |
| Physics | Y | Y | Y |
| Polish(M) | Y | N | N |
| Politics | x | x | x |
| Psychology | Y | Y | Y |
| Religious studies (RS) | Y | Y | Y |
| Russian | N | N | N |
| Sociology | Y | Y | Y |
| Spanish | Y | Y | Y |
| Turkish(M) | Y | N | N |
| Total | 43 | 39 | 41 |
2.2 Methods of analysis
Methods used for analysing the data were presented in detail in the report by He and Black (2020). In brief, 2 approaches were used in the analysis:
-
Variability in the relationship between subject grade outcomes and prior attainment, which was used as a proxy for ability (based on prior attainment-matched candidates).
-
Variability in the relationship in grade outcomes between subjects through mathematical modelling using the Rasch model (based on all candidates).
Analysis based on prior attainment-matched candidates
In the first approach, a prior attainment weighted mean grade (WMG) for a subject was calculated for each year. This involved classifying all prior attainment-matched candidates in a specific year into one of 10 prior attainment bands based on their mean KS2 scores (for GCSEs) or mean GCSE scores (for A levels). Each prior attainment score band has a similar number of candidates. For candidates taking a specific subject, average grade in the subject for those falling into each score band was calculated. The mean of these average grades across the 10 score bands are the weighted mean grade for the subject. This weighted mean grade represents the expected average grade from all prior attainment-matched candidates, were it to be taken by all these matched candidates from the population (in other words, if it had been taken by the entire cohort of GCSE or A level students with valid prior attainment measures, not just those who actually took it). WMG is used as a measure of overall difficulty of the subject. Subjects with high WMG values (higher grade outcomes) are said to be “easier” than subjects with low values (lower grade outcomes). The difficulty (or facility) of a subject defined this way is independent of the ability distribution of the candidates taking the subject and can be compared over time and between subjects. Since normalised KS2 scores and GCSE scores were used, WMG values can be compared between different years.
Analysis based on Rasch modelling of all candidates
In Rasch modelling, for each grade in a subject (except for the one used as the reference grade – grade 1 for GCSEs and U for A levels[footnote 2]), a difficulty measure is estimated[footnote 3]. Assuming that the ability distributions of the candidates included in the analysis are similar in 2019, 2020 and 2021, the grade difficulty measures will be on the same measurement scale and can be compared directly when the average ability of candidates is set to zero in each year and the unit of logits is used. The difficulty of a subject at a specific grade in a specific year is related to the average Rasch ability of the candidates achieving that grade. Subject 1 is said to be “more difficult” (or “harder”) than Subject 2 at grade X if the average ability of the candidates receiving X in Subject 1 is higher than that of the candidates in Subject 2. In other words, candidates with similar Rasch abilities would have achieved lower grades in Subject 1 than in Subject 2. Similarly, a subject is more “difficult” in year Y1 than year Y2 at grade X if the average ability of the candidates receiving X in Y1 is higher than that of the candidates in Y2. The mean of the difficulties from 3 to 8 for a GCSE subject and from D to A for an A level subject is used as the overall measure of difficulty for the subject.
Limitations
It is to be emphasized that care needs to be taken when interpreting the statistical measures of subject difficulty reported in this paper. The difficulty measures (overall or at specific grades) are not direct measures of performance standards which are generally subject specific. They are also not direct measures of subject demand or students’ efforts. Differences in statistical difficulty between subjects reflect differences in grade outcomes between subjects for candidates with similar levels of prior attainment or ability derived from the Rasch model. Such differences can be caused by many factors which can vary substantially between the subjects and over time but were not considered in the methods used in this study. These factors, among others, could include: the nature of the subject in terms of skills and knowledge to be learnt, the level of demand, allocation of teaching time and other resources, motivation of students, efficiency and effectiveness of teaching and learning, uptake by different population subgroups, and leniency or severity in grading. There are also limitations associated with the use of the Rasch model, including violation of the unidimensionality assumption (that is, the exams are assumed to measure a single ability in common), unrepresentativeness of the samples analysed, missing data, and imperfect data-model fit. All this must be kept in mind when reading this report.
As with the 2020 report by He and Black, the focus of this analysis is on comparability at grades 4, 7 and 9 between GCSE subjects and grades C, A and A* between A level subjects.
3. Results and discussion
This section discusses the main results from the analysis. Results for GCSEs are presented first, followed by results for A levels.
3.1 GCSE Subjects
3.1.1 Relationship with attainment at KS2
Figure 1 shows the relationship between mean GCSE grade and normalised KS2 score in 2019, 2020 and 2021 for candidates with a valid KS2 score. As is clear from Figure 1, candidates’ mean GCSE grade and KS2 score are highly correlated. Since values of slope of the regression lines are similar, differences in intercept between years to a large extent reflect differences in overall difficulty. Subjects in 2020 and 2021 are considerably easier statistically than in 2019 based on attainment at KS2. The intercept in 2021 is slightly lower than in 2020, whereas the gradient is slightly higher (steeper) which probably suggests candidates with higher prior attainment received slightly higher GCSE grades in 2021 than in 2020. Overall, it appears that GCSE subjects in 2021 and 2020 are of similar difficulty.
Figure 1 Relationship between candidates’ mean GCSE grade and normalised KS2 score in 2019 (top-left), 2020 (top-right) and 2021 (bottom).
Figure 2 compares weighted mean grade (WMG) between 2019, 2020 and 2021 across the GCSE subjects analysed. It is important to bear in mind the methodological limitations described on page 7 when considering this graph. In Figure 2, subjects are arranged in ascending order of difficulty in 2019 based on their weighted mean grades. That is, subjects to the right are more difficult, based on this measure, than those to the left in 2019. The WMG values in 2020 and 2021 for all the subjects are considerably higher than in 2019. This reflects the increase in outcomes in 2020 and 2021, relative to 2019 (Ofqual, 2020c, 2021b). While the general trend in WMG in 2020 and 2021 follow that in 2019, substantial variability also exists between the subjects. The distributions of WMG in 2020 and 2021 are similar.
Figure 2 Comparison of weighted mean grades between GCSE subjects from 2019-2021. Subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019
Figure 3 further compares changes in subject WMGs between 2019/2020 and 2020/2021[footnote 4]. In the top graph, subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019 (see also Table B1 in Appendix B). Difference in WMG between 2020 and 2019 varies from 0.320 for Italian to 1.127 for Classical Civilization. Between 2021 and 2019, the difference varies from 0.349 for Photography to 1.309 for Classical Civilization.
Differences in subject WMG between 2021 and 2020 are considerably smaller than differences between 2020 and 2019 or between 2021 and 2019. These vary from -0.137 for photography to 0.237 for PE. There is little change in WMG between 2021 and 2020 for subjects like dance, RS, drama, English literature, mathematics, French, German and combined sciences. Subjects like Urdu, film studies, photography and 3D studies have seen their WMG decreased by over 0.1, whereas subjects like classical civilization, history, business studies, sociology, economics, computing and PE have seen their WMG increased by over 0.1. Overall, average difficulty of the subjects in 2021 analysed here appears to be similar to that in 2020, with a very small increase of 0.024 in mean WMGs from 2020.
Figure 3 Comparison of changes in population weighted mean grade between GCSE subjects from 2019-2021. In the top graph, subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019
3.1.2 Rasch modelling
Having discussed the use of WMGs to examine how the overall difficulties of the subjects changed between years, this section discusses how subject difficulties changed between years at individual grade level through Rasch modelling of all candidates included in the analysis.
The distributions of difficulty at grades 4, 7 and 9, and the overall subject level, for GCSE subjects from 2019 to 2021 are shown in Figure A1 in Appendix A. There is variability in the difficulty distributions at the overall subject level and at individual grades between the subjects.
Relative Rasch grade difficulty distribution
The graphs in Figure 4 compare subject mean difficulty and difficulties at grades 4, 7 and 9 between the GCSE subjects analysed. In each of the graphs, subjects are arranged in ascending order of difficulty in 2019 (that is, in 2019 subjects on the right were more difficult, based on these measures, than those on the left).
It is clear from Figure 4 that subjects are considerably more leniently graded in 2020 and 2021, both overall and at individual grades, than in 2019. The only exceptions are three of the subjects (chemistry, physics and English literature) at grade 9, where the difficulties in 2019 are very slightly higher than in 2020. It is also clear that at grades 7 and 9, most subjects are considerably easier, based on this statistical measure, in 2021 than in 2020. Again, this likely relates to the general difference in the outcomes that we have reported elsewhere (Ofqual, 2020c, 2021b).
Figure 4 Comparison of mean difficulty (top) and difficulties at grades 4 (upper-middle), 7 (lower-middle) and 9 (bottom) for GCSE subjects from 2019-2021 estimated using the Rasch model. Subjects are arranged in ascending order of difficulty in 2019.
Figure 5 further illustrates the distributions of subject mean difficulty and difficulties at grades 4, 7 and 9. The range and standard deviation of grade difficulties are listed in Table 3. To make the comparison more meaningful, the unit of difficulty in logits was converted to a unit of grade width (GW) by dividing the original difficulty value by the average difficulty in logits between grade 8 and grade 3 (the average difficulty in logits between grade 8 and grade 3 is equivalent to one grade). When producing these plots, the original difficulty of a subject at a grade was subtracted from the mean difficulty of all subjects at that grade. This makes it easier to compare relative difficulties between the subjects at different grades.
It is clear from Figure 5 and Table 3 that relative grade difficulties (in GW unit) between the subjects in 2020 and 2021 have been reduced in terms of both range and standard deviation compared with grade difficulties estimated for 2019. The standard deviations of relative difficulty at grades 7 and 9 in 2021 are slightly larger than those in 2020, whereas they are slightly smaller at grade 4 and the overall subject level. The ranges of grade difficulties in 2021 are slightly wider than in 2020.
Figure 5 Comparison of distributions of average difficulty (top-left) and difficulties at grades 4 (top-right), 7 (bottom-left) and 9 (bottom-right) between GCSE subjects from 2019-2021.
Table 3.1 Standard deviation of grade difficulties (in GW) for GCSE subjects from 2019-2021.
| Year | Mean | G4 | G7 | G9 |
|---|---|---|---|---|
| 2019 | 0.563 | 0.672 | 0.495 | 0.560 |
| 2020 | 0.471 | 0.601 | 0.391 | 0.413 |
| 2021 | 0.446 | 0.565 | 0.413 | 0.489 |
Table 3.2 Range of grade difficulties (in GW) for GCSE subjects from 2019-2021.
| Year | Mean | G4 | G7 | G9 |
|---|---|---|---|---|
| 2019 | 2.436 | 2.724 | 2.742 | 3.150 |
| 2020 | 1.744 | 2.345 | 1.894 | 2.131 |
| 2021 | 1.790 | 2.416 | 2.205 | 2.612 |
Change in grade difficulty
To examine how subject grade difficulties have changed between academic years 2019 to 2020 and 2020 to 2021, Figure 6 shows distributions of changes in difficulties at grades 4, 7, 9 and subject mean difficulty for the GCSE subjects analysed (also see Table B2 in Appendix B). In Figure 6 (and Table B2), subjects are arranged in ascending order of mean difficulty in 2020. This is to focus on the comparison between 2021 and 2020. Positive values indicate an increase in apparent difficulty whereas negative values indicate a decrease. There is substantial variability in grade difficulty changes both between the subjects and within the same subject.
Changes in subject mean difficulty are 0.71 logits and 0.75 logits respectively between 2020 and 2019 and between 2021 and 2019. The average grade gap is about 1.22 logits in 2020, and this suggests that on average subjects in 2020 and 2021 were about three-fifths of a grade more leniently graded than in 2019. Similarly, subjects were graded about three-quarters of a grade (in 2020) and two-thirds of a grade (in 2021) more leniently than in 2019 at grade 4. At grade 7 subjects were graded about one-third of grade (in 2020) and nearly two-thirds of a grade (in 2021) more leniently than in 2019. At grade 9, subjects were about one-tenth of a grade easier in 2020 and one-third of a grade in 2021 easier than in 2019.
Figure 6 Changes in Rasch grade difficulties for GCSE subjects from 2019 to 2020 (top), 2019 to 2021 (middle), and 2020 to 2021 (bottom). Subjects are arranged in ascending order of mean difficulty in 2020 in order to focus on comparison between 2021 and 2020.
The variability of changes in subject grade difficulty from 2019 to 2021 follows a similar pattern to that from 2019 to 2020 (also see He and Black, 2020). For example, Italian, German, combined science and statistics are among the subjects with the smallest changes in difficulty, whereas textiles, 3D studies, classical civilisation and music are among the subjects with the largest changes. Further, within-subject variability in difficulty changes between 2019 and 2021 is substantially smaller than that between 2019 and 2020.
Changes in subject grade difficulties between 2020 and 2021 are considerably smaller than changes between 2019 and 2020 or between 2019 and 2021 (see the bottom graphs in Figure 6).
At grade 4, for a substantial number of subjects (and also a small number of subjects at the overall subject level), the difficulties in 2021 are higher than the difficulties in 2020 (the differences between 2021 and 2020 are positive). This may appear to be contradictory to the increased outcomes in 2021 compared with 2020 (see Ofqual, 2021b). However, this is likely to be a statistical artefact that reflects the increase in the proportion of students of lower to middle abilities receiving grade 5 and above in 2021 relative to 2020, a change which makes grade 4 appear to be more difficult in 2021 than in 2020. This effect could also result in increased overall subject difficulty in 2021 for some of the subjects (defined as the mean of the difficulties from grade 3 to grade 8 for GCSEs and from D to A for A levels).
Average changes in subject mean difficulty and difficulties at grades 4, 7 and 9 from 2020 to 2021 are -0.013 logits, 0.141 logits (about one-tenth of a grade), -0.269 logits (about a quarter of a grade) and -0.344 logits (slightly over one-fifth of a grade) respectively. Subjects in 2021 are slightly more difficult at grade 4 and easier at grades 7 and 9 than in 2020. At grades 7 and 9, photography, textiles, graphics, Urdu, English literature, Spanish and combined science are among the subjects with smallest changes in difficulty, whereas Italian, drama, music and classical civilization are among the subjects with the largest changes in difficulty.
Changes in grade difficulty rank orders
Subjects were rank ordered based on mean difficulty and difficulties at grades 4, 7 and 9 separately in 2019, 2020 and 2021, with the lowest rank (first) representing the easiest subject and highest rank the most difficult subject. Figure 7 shows the distribution of changes in subject difficulty rank order positions between academic years 2019 to 2020 and 2020 to 2021 (also see Table B3). It is clear from Figure 7 that changes in grade difficulty orders from 2020 to 2021 are smaller than changes from 2019 to 2021 or from 2019 to 2020. Changes in difficulty order positions at grade 9 between 2021 and 2019 are the largest in terms of both range and standard spread, whereas changes between 2021 and 2020 at grade 4 and the overall subject level are the smallest.
Figure 7 Distributions of changes in grade difficulty rank order positions in GCSE subjects between 2020 and 2019, 2021 and 2019, and 2021 and 2020.
Between 2021 and 2020, changes in difficulty rank orders at grades 7 and 9 are considerably higher than changes at grade 4 and the overall subject level. Average changes in difficulty position per subject are 1.37 based on subject mean difficulty, 1.74 at grade 4, 2.37 at grade 7 and 3.68 at grade 9 respectively (see Table B3).
Figure 8 shows detailed changes in difficulty order positions between 2019 to 2020 and 2020 to 2021 for individual subjects (also see Table B3. In Table B3, the number of changes greater than 5 or less than -5 are highlighted). In Figure 8, subjects are arranged in ascending order of grade difficulty in 2020. Positive values indicate increase in rank order (becoming more difficult) whereas negative values indicate decrease in difficulty rank order. The pattern of changes from 2019 to 2021 is similar to that from 2019 to 2020, with music, drama, classical civilization and PE showing the largest changes and Italian, film studies, media studies and German showing the smallest changes.
Figure 8 Distributions of change in difficulty rank order positions overall and at individual grades for GCSE subjects from 2019-2021. Subjects are arranged in ascending order of difficulty in 2020.
Between 2021 and 2020, based on subject mean difficulty, only history changed its position by more than 5 (moved down by 7). At grade 4, changes in rank order positions for all subjects are within 5. At grade 7, history and classical civilization dropped their rank order positions by 7 and 9 respectively, and 3D studies and graphics increased their position by 6 and 9. At grade 9, history, music, psychology and classical civilization had their rank order positions decreased by 7, 8, 9 and 17 respectively, while ACD, 3D studies, graphics, fine art, and textiles increased their positions by more than 5.
Comparison with 2011, 2012 and 2013
As centre assessment grades (CAGs) constitute the vast majority of the 2020 final awarded grades, the analysis discussed above essentially looked at the impact of centre assessment grades in 2020 and teacher assessment grades (TAGs) in 2021 on inter-subject comparability, particularly in relation to grades awarded in 2019 through normal exams. To understand further how the comparability of grading standards between subjects varied for CAGs and TAGs, the variability of GCSE subject grade difficulties between years under normal exam conditions was investigated. GCSE data from 2011, 2012 and 2013 (using letter grades) extracted from the NPD were analysed. Grades A, A and C were used as proxies for grades 9, 7 and 4 for the reformed GCSEs which were introduced in phases from 2015, realizing that A is not a good proxy for grade 9 since they are not comparable in terms of performance standards.
The top-left graph in Figure 9 shows distributions of differences in subject grade difficulties between pairs of consecutive years (2012 to 2011, 2013 to 2012, 2020 to 2019 and 2021 to 2020), and the top-right graph shows differences between three years (2013 to 2011 and 2021 to 2019) (also see Tables B4). The ranges and standard deviations of the distributions are listed in Table 4. The range and standard deviation of changes in grade difficulties between 2020 (CAGs) and 2021 (TAGs) are larger than those of changes between two consecutive years of normal exams at grades 4/C, 7/A and the overall subject level, and similar or slightly smaller at 9/A*.
The bottom-left graph in Figure 9 shows the distributions of differences in subject grade difficulty rank order positions between two consecutive years, and the bottom-right graph shows differences between 3 years (also see Tables B5). Values of absolute average change and standard deviation of changes are also listed in Table 4.
Between 2 consecutive years, based on subject mean difficulty, the magnitude and standard division of changes in subject rank order positions between 2021 (TAGs) and 2020 (CAGs) is slightly smaller than changes between two consecutive years of normal exams (the absolute average change per subject and standard deviation between 2021 and 2020 are 1.37 and 1.93 respectively, whereas these are 1.60 and 2.33 between 2012 and 2011 and 1.71 and 2.44 between 2013 and 2012). At grade 4/C, the size and standard deviation of changes in rank orders between 2021 and 2020 are similar to those associated with normal exams. Between 2020 and 2021, changes in grade difficulty rank order at grades 7 and 9 are larger than those associated with normal exams, with average change per subject and standard deviation of changes of 2.37 and 3.46 at grade 7 and 3.68 and 4.96 at grade 9 respectively, which are higher than the values at A and A* between 2012 and 2011 or between 2013 and 2012.
Figure 9 Changes in difficulties overall and at individual grades between two consecutive years (top) and between three years (upper-middle) for GCSE subjects from 2011 to 2013 and 2019 to 2021, and associated changes in grade difficulty rank order positions (lower-middle and bottom).
Table 4 Standard deviation and range of changes in grade difficulties and average absolute change per subject in grade difficulty rank order and standard deviation of changes for GCSE subjects between 2011 to 2013 and 2019 to 2021.
| Grade | Years | Difficulty change (range) | SD of change | Rank order change (average) | SD of change |
|---|---|---|---|---|---|
| Mean | 12-11 | 0.564 | 0.101 | 1.600 | 2.330 |
| Mean | 13-12 | 0.390 | 0.098 | 1.714 | 2.438 |
| Mean | 20-19 | 0.806 | 0.197 | 3.105 | 4.161 |
| Mean | 21-20 | 0.753 | 0.147 | 1.368 | 1.933 |
| Mean | 13-11 | 0.484 | 0.133 | 1.943 | 2.461 |
| Mean | 21-19 | 1.021 | 0.209 | 3.421 | 4.560 |
| C | 12-11 | 0.514 | 0.089 | 1.086 | 1.586 |
| C | 13-12 | 0.457 | 0.108 | 1.543 | 2.402 |
| 4 | 20-19 | 1.146 | 0.277 | 3.474 | 4.560 |
| 4 | 21-20 | 0.907 | 0.189 | 1.737 | 2.176 |
| C | 13-11 | 0.661 | 0.135 | 2.057 | 2.859 |
| 4 | 21-19 | 1.164 | 0.248 | 3.211 | 3.954 |
| A | 12-11 | 0.486 | 0.079 | 1.657 | 2.541 |
| A | 13-12 | 0.367 | 0.095 | 1.771 | 2.849 |
| 7 | 20-19 | 0.747 | 0.157 | 3.158 | 4.267 |
| 7 | 21-20 | 0.657 | 0.125 | 2.368 | 3.456 |
| A | 13-11 | 0.427 | 0.111 | 1.943 | 2.662 |
| 7 | 21-19 | 0.894 | 0.213 | 4.684 | 6.100 |
| A* | 12-11 | 1.097 | 0.207 | 2.743 | 4.113 |
| A* | 13-12 | 0.937 | 0.174 | 2.057 | 2.839 |
| 9 | 20-19 | 0.833 | 0.172 | 3.842 | 5.094 |
| 9 | 21-20 | 0.707 | 0.148 | 3.684 | 4.963 |
| A* | 13-11 | 1.193 | 0.257 | 3.029 | 4.641 |
| 9 | 21-19 | 1.090 | 0.276 | 7.053 | 8.718 |
3.2 A level Subjects
3.2.1 Relationship with attainment at GCSE
Figure 10 shows the relationship between mean A level grade and normalised mean GCSE score in 2019, 2020 and 2021 for candidates with valid mean GCSE scores. Candidate’s mean A level grade and GCSE score are reasonably highly correlated, although the correlation is not as high as that between mean GCSE grade and KS2 score. While values of slope of the regression lines are similar over the 3 years, the intercepts are different. Subjects in 2020 and 2021 are considerably easier than in 2019 based on attainment at GCSE. Further, the intercept in 2021 is 0.123 higher than in 2020, suggesting that overall subjects in 2021 are substantially easier than in 2020 based on GCSE-matched candidates. Again, this reflects analysis that has been previously reported by Ofqual about qualification outcomes in 2021 (Ofqual, 2020c, 2021b).
Figure 10 Relationship between candidate’s mean A level grade and normalised mean GCSE score in 2019 (top-left), 2020 (top-right) and 2021 (bottom).
As with GCSEs, a prior attainment (represented by mean GCSE score) weighted mean grade (WMG) for each subject was also calculated for each year. Figure 11 compares WMG between 2019, 2020 and 2021 across the A level subjects analysed. Again, it is important to bear in mind the methodological limitations (p.7) when interpreting this graph. In Figure 11, subjects are arranged in ascending order of difficulty in 2019 based on their weighted mean grades in Figure 11. Similar to GCSEs, the WMG values in 2020 and 2021 for all the subjects are considerably higher than in 2019. Although the overall trend in WMG for the subjects in 2020 and 2021 follows that in 2019, substantial variability also exists between the subjects. The distribution of WMGs in 2021 is similar to that in 2020. However, the values are noticeably higher in 2021 than in 2020 for almost all the subjects.
Figure 11 Comparison of weighted mean grades in A level subjects from 2019 to 2021. Subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019.
Figure 12 compares changes in subject WMG between 2019 to 2020 and 2020 to 2021. In Figure 12, subjects are arranged in descending order of weighted mean grade in 2019 (see also Table B6 in Appendix B). Difference in WMG between 2020 and 2019 for the subjects varies from 0.417 for media studies to 0.893 for Chinese. Between 2021 and 2019, the difference varies from 0.422 for 3D studies to 1.349 for Chinese.
Differences in subject WMG between 2021 and 2020 are considerable smaller than the differences between 2020 and 2019 or between 2021 and 2019. These vary from -0.017 for 3D studies to 0.456 for Chinese. All subjects except for 3D studies have seen their WMG values increased from 2020 to 2021. Subjects like dance, classical civilization, German, Spanish, product design, PE, music, philosophy and biology have seen their WMG go up by over 0.2. Overall, average difficulty of the subjects analysed in 2021 is noticeably lower than in 2020, with an increase of 0.169 in WMG from 2020.
Figure 12 Comparison of changes in population weighted mean grade in A level subjects from 2019-2021. Subjects are arranged in ascending order of difficulty based on their weighted mean grade in 2019 in the top graph.
3.2.2 Rasch modelling
The distributions of difficulty at grades C, A and A* and the overall difficulty for the A level subjects from 2019 to 2021 based on the Rasch model are shown in Figure A2 in Appendix A. Similar to GCSEs, there is variability in the difficulty distributions at the over subject level and at individual grades between the subjects.
Relative Rasch grade difficulty distribution
Figure 13 compares subject mean difficulty and difficulties at C, A and A* between the A level subjects analysed. In each of the graphs, subjects are arranged in ascending order of difficulty in 2019. As with GCSEs, subjects are considerably more leniently graded in 2020 and 2021 than in 2019 except for 3D Studies at A* where the difficulty in 2019 is very slightly higher than in 2020. At A and A, particularly at A, most of the subjects are noticeably easier in 2021 than in 2020. This is most noticeable in the case of Chinese. This analysis does not allow us to identify why the differences are more pronounced in some subjects than others.
Figure 13 Comparison of subject mean difficulty (top) and difficulties at C (upper-middle), A (lower-middle) and A* (bottom) for A level subjects from 2019 to 2021 estimated using the Rasch model. Subjects are arranged in order of difficulty in 2019.
Figure 14 further illustrates the distributions of subject mean difficulty and difficulties at C, A and A*. The range and standard deviation of relative grade difficulties of the subjects are listed in Table 5 (to avoid potential distortion to difficulty distribution, Chinese was excluded when producing this table). GW is used in Figure 14 and Table 5. For A levels, the average gap in difficulty in logits between A and D is equivalent to one grade (GW).
Relative grade difficulties (in GW unit) between the subjects in 2020 and 2021 have been reduced in terms of both range and standard deviation compared with grade difficulties estimated for 2019. The range and standard deviation of difficulty in 2021 are similar to those in 2020.
Figure 14 Comparison of distributions of subject mean difficulty (top-left) and difficulties at C (top-right), A (bottom-left) and A* (bottom-right) between A level subjects from 2019-2021.
Table 5.1 Standard deviation of grade difficulties (in GW) for A level subjects from 2019-2021 (excluding Chinese).
| Year | Mean | C | A | A* |
|---|---|---|---|---|
| 2019 | 0.627 | 0.693 | 0.510 | 0.738 |
| 2020 | 0.583 | 0.632 | 0.460 | 0.554 |
| 2021 | 0.566 | 0.633 | 0.448 | 0.542 |
Table 5.1 Range of grade difficulties (in GW) for A level subjects from 2019-2021 (excluding Chinese).
| Year | Mean | C | A | A* |
|---|---|---|---|---|
| 2019 | 2.625 | 2.845 | 2.123 | 2.681 |
| 2020 | 2.503 | 2.569 | 1.889 | 2.140 |
| 2021 | 2.348 | 2.607 | 1.781 | 2.203 |
Change in grade difficulty
Figure 15 shows distributions of changes in subject mean difficulty and difficulties at C, A and A* (also see Table B6 in Appendix B. Note that to avoid potential distortion to the distribution of differences in difficulty, some values for Chinese were excluded when producing Figure 15). In Figure 15 (and in Table B6), subjects are arranged in ascending order of mean difficulty in 2020. There is substantial variability in the changes both between the subjects and within the same subject.
Between 2019 and 2020 to 2021, at the overall subject level, average changes in difficulty are -1.764 logits and -1.773 logits respectively between 2020 and 2019 and between 2021 and 2019. Average grade gap is about 3.215 logits in 2020. On average, subjects in 2020 and 2021 were slightly over half of a grade more leniently graded than in 2019. At C, subjects in 2020 and 2021 were graded about three-fifths of a grade more leniently than in 2019. At A, subjects were graded about one-third of a grade in 2020 and nearly half of a grade in 2021 more leniently than in 2019. At A*, subjects were about one-fifth of a grade in 2020 and half of a grade in 2021 easier than in 2019.
Changes in grade difficulty between subjects from 2019 to 2021 are considerably different from changes from 2019 to 2020. While the between-subject variability follows a similar pattern, within-subject variability from 2019 to 2021 is substantially smaller than from 2019 to 2020 as a result of reduced difficulty at A and A* in 2021.
Figure 15 Changes in Rasch grade difficulties for A level subjects from 2019 to 2020 (top), 2019 to 2021 (middle), and 2020 to 2021 (bottom) for A level subjects from 2019-2021. Subjects are arranged in ascending order of mean difficulty in 2020.
Changes in subject grade difficulties between 2020 and 2021 are considerably smaller than changes between 2019 and 2020 or between 2019 and 2021 (see Table B6. Also see He and Black, 2020). As with GCSEs, a large number of subjects appear to be more difficult in 2021 than in 2020 at C and the overall subject level. The explanations provided for GCSEs discussed above also apply here. Average changes in subject mean difficulty and difficulties at C, A and A* from 2020 to 2021 are -0.015 logits, 0.066 logits, -0.674 logits (nearly one-fifth of a grade) and -1.057 logits (about a quarter of a grade) respectively. Subjects in 2021 are considerably easier at A and A* than in 2020. At A, subjects like Chinese, computing, further mathematics, German and music in 2021 are about one-third of a grade easier than in 2020. At A, accounting, computing and music are over half of a grade easier in 2021 than in 2020. Chinese is about one grade easier in 2021 than 2020 at A.
Changes in grade difficulty rank orders
As with GCSEs, subjects were rank ordered based on mean difficulty and difficulties at C, A and A* separately in 2019, 2020 and 2021. Figure 16 shows distributions of changes in subject difficulty rank order positions between academic years 2019 to 2020 and 2020 to 2021 (also see Table B8. Note that to avoid potential distortion to the distribution of differences in difficulty order, the value for Chinese at A* between 2021 and 2019 was excluded from the bottom-right graph in Figures 16, 17 and 18). Changes in grade difficulty orders from 2020 to 2021 are smaller than changes from 2019 to 2021 or from 2019 to 2020. Changes in difficulty orders at A and A* between 2021 and 2019 are the largest in terms of both range and spread, whereas changes between 2021 and 2020 at C are the smallest. Between 2021 and 2020, changes in difficulty rank order at A* are slightly larger than at C, A and the overall subject level. Average changes in difficulty position per subject are 2.05 overall, 1.90 at C, 1.95 at A and 2.77 at A* respectively (see Table B8).
Figure 16 Distributions of changes in grade difficulty rank order positions in A level subjects between academic years 2020 to 2019 and 2021 to 2020.
Figure 17 shows detailed changes in grade difficulty orders between academic years 2019 to 2020 and 2020 to 2021 for individual subjects (also see Table B8). In Figure 17, subjects are arranged in ascending order of grade difficulty in 2020. The patterns of changes in subject rank orders between 2021 and 2019 are broadly similar to those between 2020 and 2019.
Between 2021 and 2020, at the overall subject level, German and music changed their rank order positions by more than 5 (moved down by 7 and 9 respectively). At C, only Chinese changed its rank order position by more than 5 (moved down by 8). At A, classical civilization, German and music all had their rank order dropped by 6. At A*, four subjects changed their difficulty rank order positions by more than 5: geography went up by 8, accounting and music both came down by 8, and Chinese came down by 15.
Figure 17 Distributions of changes in difficulty rank order positions overall and at individual grades for A level subjects from academic years 2019 to 2020 and 2020 to 2021. Subjects are arranged in ascending order of mean difficulty in 2020.
Comparison with 2017, 2018 and 2019
Similar to GCSEs, A level results data from 2017 and 2018 were analysed to provide further information on changes in grade difficulty under normal exam conditions. The top graph in Figure 18 shows the distribution of differences in subject grade difficulties between two consecutive years (2018 to 2017, 2019 to 2018, 2020 to 2019 and 2021 to 2020), and the upper-middle graph shows the differences between three years (2019 to 2017 and 2021 to 2019) (also see Tables B9). The range and standard deviation of the distributions are listed in Table 6. It is clear from these graphs and Table 6 that the range and standard deviation of changes between years associated with the 2020 CAGs and 2021 TAGs are larger than those of changes based on grades from normal exams at the overall subject level and at individual grades.
The lower-middle graph in Figure 18 shows the distribution of differences in subject grade difficulty rank orders between 2 consecutive years, and the bottom graph shows the differences between three years (also see Tables B10). Values of absolute average change per subject and standard deviation of changes are also listed in Table 6.
Between 2 consecutive years, at the overall subject level and at individual grades, the range and standard deviation of changes in subject rank orders between 2021 (TAGs) and 2020 (CAGs) is generally larger than changes associated with normal exams. The absolute average change per subject and standard deviation of changes between 2021 and 2020 are 2.051 and 2.926 at the overall subject level, 1.897 and 2.582 at C, 1.949 and 2.828 at A, and 2.769 and 3.968 at A*. These are generally larger than the values associated with changes between normal examination years.
Figure 18 Changes in difficulties overall and at individual grades between two consecutive years (top) and between three years (upper-middle) for A level subjects from 2017 to 2021, and the associated changes in grade difficulty rank order positions (lower-middle and bottom).
Table 6 Standard deviation and range of changes in grade difficulties and average absolute change per subject in grade difficulty rank order and standard deviation of changes for A level subjects from 2017 to 2021.
| Grade | Years | Difficulty change (range) | SD of change | Rank order change (average) | SD of change |
|---|---|---|---|---|---|
| Mean | 18-17 | 0.976 | 0.154 | 0.900 | 1.304 |
| Mean | 19-18 | 1.185 | 0.321 | 1.300 | 2.049 |
| Mean | 20-19 | 2.055 | 0.464 | 2.410 | 3.266 |
| Mean | 21-20 | 4.910 | 0.737 | 2.051 | 2.926 |
| Mean | 19-17 | 1.579 | 0.352 | 1.500 | 2.191 |
| Mean | 21-19 | 3.657 | 0.687 | 3.487 | 4.782 |
| C | 18-17 | 1.290 | 0.206 | 1.300 | 2.121 |
| C | 19-18 | 1.825 | 0.456 | 1.700 | 2.559 |
| C | 20-19 | 3.682 | 0.592 | 2.154 | 3.274 |
| C | 21-20 | 8.278 | 1.239 | 1.897 | 2.582 |
| C | 19-17 | 1.976 | 0.462 | 1.650 | 2.145 |
| C | 21-19 | 5.890 | 0.940 | 2.923 | 3.929 |
| A | 18-17 | 0.499 | 0.102 | 1.000 | 1.449 |
| A | 19-18 | 1.095 | 0.186 | 1.150 | 1.549 |
| A | 20-19 | 3.408 | 0.591 | 2.923 | 4.070 |
| A | 21-20 | 2.785 | 0.500 | 1.949 | 2.828 |
| A | 19-17 | 1.255 | 0.231 | 1.350 | 1.732 |
| A | 21-19 | 5.587 | 0.949 | 4.308 | 5.747 |
| A* | 18-17 | 0.921 | 0.256 | 1.750 | 2.429 |
| A* | 19-18 | 2.374 | 0.396 | 1.750 | 2.924 |
| A* | 20-19 | 3.390 | 0.685 | 3.231 | 4.723 |
| A* | 21-20 | 3.674 | 0.652 | 2.769 | 3.968 |
| A* | 19-17 | 2.936 | 0.513 | 2.050 | 3.500 |
| A* | 21-19 | 6.832 | 1.240 | 5.128 | 7.749 |
4. Concluding remarks
In this report, the comparability of grading standards between subjects in GCSE and A level from summer 2021 (TAGs) was investigated using statistical measures of grade difficulty derived from:
- (a) empirical analysis of the relationships between subject grade outcomes and prior attainment
- (b) Rasch modelling of the relationships between grades in different subjects
These difficulty statistics were compared with those calculated for 2020 (where CAGs were used, primarily) and those calculated for grades determined by external exams. This allowed us to examine the impact of different approaches to assessment on inter-subject comparability. It is stressed, once again, that these relative difficulty measures are not direct measures of performance standards, nor subject demand, nor students’ efforts. They reflect differences in grade outcomes between subjects for candidates with a similar level of prior attainment or ability, derived from the Rasch model. This needs to be borne in mind when considering the findings from this study.
Based on relationships between subject grade outcomes and prior attainment for prior attainment-matched candidates, overall, though there was some variation, GCSE subjects were graded similarly in both 2021 and 2020 in terms of grading leniency or difficulty. For A levels, subjects in 2021 on average were graded about one-sixth of a grade more leniently than in 2020. Both GCSE and A level subjects were more leniently graded in 2021 and 2020 than in 2019.
Based on analysis using the Rasch model for all candidates taking the subjects analysed, for both GCSEs and A levels, at the top grades (grades 7, 8 and 9 for GCSEs and A and A* for A levels), subjects in 2021 were graded more leniently than in 2020 and substantially more leniently than in 2019. On average, GCSE subjects were graded about a quarter of a grade at grade 7 and slightly over one-fifth of a grade at grade 9 more leniently in 2021 than in 2020, whereas A level subjects were graded nearly one-fifth of a grade at A and slightly over a quarter of a grade at A* more leniently.
For both GCSEs and A levels, the range and standard deviation of relative grade difficulties (in grade width unit estimated using the Rasch model) are similar to or slightly larger than those in 2020 and noticeably smaller than in 2019.
For GCSEs, the range and standard deviation of changes in grade difficulties between 2021 (TAGs) and 2020 (CAGs) are larger than changes between two consecutive years (for example, between 2012 and 2013) of normal exams at grades 4 (C), 7 (A) and the overall subject level and similar or slightly smaller at 9 (A*). For A levels, the range and standard deviation of changes between 2021 and 2020 are considerably larger than changes between two consecutive years (for example, between 2018 and 2019) of normal exams overall and at individual grades. This suggests that the relative difficulty of subjects varied more between 2021 and 2020 than tends to be observed between consecutive years where exams take place, likely reflecting the differing approaches to assessment and grading in those 2 years.
For GCSEs, based on subject mean difficulty, the magnitude and standard deviation of changes in subject rank order positions between 2021 (TAGs) and 2020 (CAGs) is slightly smaller than changes between 2 consecutive years associated with normal exams. At grade 4 (C), the size and standard deviation of changes in difficulty rank orders between 2021 and 2020 are similar to those associated with normal exams. At grades 7 and 9, changes in rank orders are larger than those associated with normal exams, with absolute average change per subject and standard deviation of 2.37 and 3.46 at grade 7 and 3.68 and 4.96 at grade 9 respectively, which are noticeably higher than the values at A and A* between 2 consecutive years of normal exams. For A levels, at the overall subject level and at individual grades, the size and standard deviation of changes in subject difficulty rank order positions between 2021 (TAGs) and 2020 (CAGs) are generally larger than changes between two consecutive years of normal exams. The absolute average change per subject and standard deviation between 2021 and 2020 are 2.051 and 2.926 at the overall subject level, 1.897 and 2.582 at C, 1.949 and 2.828 at A, and 2.769 and 3.968 at A*. As with GCSE, these values are generally larger than changes between 2 consecutive years of normal exams.
Different grading approaches (CAGs, TAGs, and external exams) impacted on inter-subject comparability in GCSEs and A levels differently, which, to a certain extent, reflects differences in the nature of how grades were determined by the different approaches. Results from this analysis indicate that variability in changes in subject grade difficulty and difficulty rank orders over time associated with grading through teacher assessment (CAGs and TAGs) was generally larger than the variability associated with grading employing external exams.
References
Coe, R. (2008). Comparability of GCSE examinations in different subjects: An application of the Rasch model. Oxford Review of Education, 34, 609–636.
He, Q. and Black, B. (2020). Impact of calculated grades, centre assessment grades and final grades on inter-subject comparability in GCSEs and A levels in 2020. Ofqual: Coventry, UK.
Joint Council for Qualifications (JCQ) (2021). JCQ Guidance on the determination of grades for A/AS Levels and GCSEs for Summer 2021: processes to be adopted by exam centres and support available from awarding organisations.
Ofqual (2015). Comparability of Different GCSE and A Level Subjects in England: An Introduction. Ofqual: Coventry, UK.
Ofqual (2020a). Awarding GCSE, AS & A levels in summer 2020: interim report. Ofqual: Coventry, UK.
Ofqual (2020c). Summer 2020 results analysis – GCSE, AS and A level. Ofqual: Coventry, UK.
Ofqual (2021a). Decisions on how GCSE, AS and A level grades will be determined in summer 2021. Ofqual: Coventry, UK.
Ofqual (2021b). Summer 2021 results analysis and quality assurance. Ofqual: Coventry, UK.
Ofqual (2021c). Information for heads of centre, heads of department and teachers on the submission of teacher assessed grades: summer 2021. Ofqual: Coventry, UK.
Appendix A Additional figures
Figure A1 Relative mean difficulty and difficulties at grades 4, 7 and 9 expressed in logits for GCSE subjects from 2019 (top), 2020 (middle) and 2021 (bottom). In each of the graphs, subjects are arranged according to their mean difficulty. Subjects on the right are more difficult than those on the left in terms of average statistical difficulty. At individual grade level, the order of grade difficulty can be different from that based on mean Difficulty.
Figure A2 Relative mean difficulty and difficulties at grades C, A and A* expressed in logits for A level subjects from 2019 (top), 2020 (middle) and 2021 (bottom). In each of the graphs, subjects are arranged according to their mean difficulty.
Appendix B Additional tables
Table B1 Difference in weighted mean grade between 2021 to 2020 and 2020 to 2019 for GCSE subjects from 2019-2021.
| Subject | 2020-2019 | 2021-2019 | 2021-2020 |
|---|---|---|---|
| Italian | 0.320 | 0.513 | 0.193 |
| Urdu | 0.614 | 0.479 | -0.135 |
| Art and Design: Photography | 0.485 | 0.349 | -0.137 |
| Art and Design: Fine art | 0.548 | 0.497 | -0.052 |
| Art and Design: Textile design | 0.630 | 0.544 | -0.086 |
| Art and Design: Art, craft and design | 0.540 | 0.501 | -0.038 |
| Physics | 0.638 | 0.614 | -0.024 |
| Dance | 0.901 | 0.930 | 0.029 |
| Biology | 0.641 | 0.641 | 0.000 |
| Chemistry | 0.646 | 0.567 | -0.079 |
| Religious studies | 0.433 | 0.453 | 0.020 |
| Latin | 1.126 | 1.056 | -0.071 |
| Drama | 0.748 | 0.727 | -0.021 |
| Food preparation and nutrition | 0.602 | 0.575 | -0.027 |
| English literature | 0.341 | 0.349 | 0.008 |
| Art and Design: Graphics | 0.674 | 0.623 | -0.051 |
| Media studies | 0.625 | 0.604 | -0.021 |
| Film studies | 0.650 | 0.525 | -0.125 |
| Music | 0.897 | 0.924 | 0.027 |
| English language | 0.446 | 0.512 | 0.066 |
| Physical education | 0.663 | 0.900 | 0.237 |
| Citizenship studies | 0.555 | 0.588 | 0.033 |
| Mathematics | 0.383 | 0.400 | 0.017 |
| Sociology | 0.674 | 0.792 | 0.118 |
| Design and technology | 0.723 | 0.684 | -0.038 |
| Geography | 0.560 | 0.706 | 0.146 |
| Classical civilisation | 1.127 | 1.309 | 0.182 |
| History | 0.569 | 0.768 | 0.200 |
| Art and Design: Three-dimensional design | 0.616 | 0.499 | -0.117 |
| Business | 0.699 | 0.836 | 0.137 |
| Spanish | 0.617 | 0.670 | 0.052 |
| Economics | 0.959 | 1.112 | 0.153 |
| Statistics | 0.690 | 0.707 | 0.017 |
| French | 0.621 | 0.636 | 0.016 |
| German | 0.697 | 0.719 | 0.022 |
| Combined science | 0.383 | 0.410 | 0.026 |
| Psychology | 0.930 | 0.996 | 0.066 |
| Computing | 1.001 | 1.179 | 0.178 |
| Average | 0.657 | 0.681 | 0.024 |
Table B2 Differences in overall difficulty and difficulties (logits) at grades 4, 7 and 9 between 2020/2021 and 2020/2019 for GCSE subjects from 2019-2021.
| Subject | 20-19 (mean) | 21-19 (mean) | 21-20 (mean) | 20-19, (G4) | 21-19, (G4) | 21-20, (G4) | 20-19, (G7) | 21-19, (G7) | 21-20, (G7) | 20-19, (G9) | 21-19, (G9) | 21-20, (G9) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Art and Design: Three-dimensional design | -0.957 | -0.761 | 0.196 | -1.688 | -1.173 | 0.515 | -0.330 | -0.480 | -0.150 | -0.010 | -0.179 | -0.169 |
| Art and Design: Art, craft and design | -0.805 | -0.773 | 0.033 | -1.342 | -1.112 | 0.230 | -0.345 | -0.573 | -0.228 | -0.043 | -0.342 | -0.299 |
| Art and Design: Fine art | -0.857 | -0.811 | 0.046 | -1.390 | -1.166 | 0.224 | -0.392 | -0.590 | -0.198 | -0.075 | -0.291 | -0.216 |
| Art and Design: Graphics | -0.986 | -0.855 | 0.131 | -1.555 | -1.307 | 0.248 | -0.448 | -0.506 | -0.058 | -0.139 | -0.389 | -0.250 |
| Art and Design: Photography | -0.755 | -0.509 | 0.246 | -1.221 | -0.748 | 0.473 | -0.359 | -0.386 | -0.027 | -0.014 | -0.205 | -0.191 |
| Art and Design: Textile design | -0.913 | -0.854 | 0.059 | -1.506 | -1.313 | 0.193 | -0.432 | -0.537 | -0.105 | -0.040 | -0.216 | -0.176 |
| Biology | -0.638 | -0.644 | -0.007 | -1.052 | -0.832 | 0.220 | -0.249 | -0.550 | -0.301 | 0.031 | -0.223 | -0.254 |
| Business | -0.755 | -0.901 | -0.146 | -1.062 | -1.028 | 0.034 | -0.495 | -0.847 | -0.352 | -0.329 | -0.769 | -0.440 |
| Combined science | -0.410 | -0.491 | -0.080 | -0.652 | -0.622 | 0.030 | -0.187 | -0.438 | -0.251 | 0.006 | -0.129 | -0.135 |
| Chemistry | -0.615 | -0.548 | 0.067 | -1.068 | -0.752 | 0.316 | -0.191 | -0.435 | -0.244 | 0.084 | -0.128 | -0.212 |
| Citizenship studies | -0.574 | -0.641 | -0.068 | -0.867 | -0.743 | 0.124 | -0.319 | -0.599 | -0.280 | -0.116 | -0.534 | -0.418 |
| Classical civilisation | -1.046 | -1.285 | -0.239 | -1.543 | -1.462 | 0.081 | -0.586 | -1.176 | -0.590 | -0.308 | -1.150 | -0.842 |
| Computing | -0.779 | -1.091 | -0.312 | -1.142 | -1.333 | -0.191 | -0.414 | -0.876 | -0.462 | -0.360 | -0.792 | -0.432 |
| Design and technology | -0.868 | -0.831 | 0.037 | -1.307 | -1.037 | 0.270 | -0.473 | -0.715 | -0.242 | -0.233 | -0.614 | -0.381 |
| Dance | -1.058 | -1.015 | 0.043 | -1.475 | -1.258 | 0.217 | -0.709 | -0.938 | -0.229 | -0.553 | -0.934 | -0.381 |
| Drama | -0.909 | -0.852 | 0.057 | -1.294 | -0.960 | 0.334 | -0.611 | -0.932 | -0.321 | -0.284 | -0.805 | -0.521 |
| Economics | -0.964 | -1.069 | -0.104 | -1.325 | -1.256 | 0.069 | -0.582 | -0.935 | -0.353 | -0.357 | -0.741 | -0.384 |
| English language | -0.559 | -0.662 | -0.103 | -0.929 | -0.887 | 0.042 | -0.308 | -0.635 | -0.327 | -0.061 | -0.428 | -0.367 |
| English literature | -0.398 | -0.456 | -0.058 | -0.685 | -0.637 | 0.048 | -0.148 | -0.356 | -0.208 | 0.087 | -0.060 | -0.147 |
| Film studies | -0.716 | -0.566 | 0.150 | -1.169 | -0.827 | 0.342 | -0.286 | -0.410 | -0.124 | -0.058 | -0.323 | -0.265 |
| Food preparation and nutrition | -0.824 | -0.806 | 0.018 | -1.246 | -1.059 | 0.187 | -0.478 | -0.705 | -0.227 | -0.258 | -0.540 | -0.282 |
| French | -0.514 | -0.653 | -0.138 | -0.854 | -0.947 | -0.093 | -0.209 | -0.470 | -0.261 | -0.280 | -0.446 | -0.166 |
| Geography | -0.573 | -0.799 | -0.226 | -0.887 | -0.986 | -0.099 | -0.286 | -0.662 | -0.376 | -0.200 | -0.666 | -0.466 |
| German | -0.648 | -0.799 | -0.151 | -0.931 | -0.990 | -0.059 | -0.387 | -0.685 | -0.298 | -0.263 | -0.534 | -0.271 |
| History | -0.548 | -0.907 | -0.358 | -0.897 | -1.164 | -0.267 | -0.213 | -0.686 | -0.473 | -0.058 | -0.563 | -0.505 |
| Italian | -0.253 | -0.265 | -0.012 | -0.542 | -0.298 | 0.244 | 0.038 | -0.282 | -0.320 | 0.280 | -0.154 | -0.434 |
| Latin | -0.705 | -0.693 | 0.012 | -1.077 | -0.909 | 0.168 | -0.330 | -0.482 | -0.152 | -0.070 | -0.504 | -0.434 |
| Mathematics | -0.418 | -0.495 | -0.077 | -0.672 | -0.653 | 0.019 | -0.166 | -0.383 | -0.217 | -0.091 | -0.312 | -0.221 |
| Media studies | -0.785 | -0.735 | 0.050 | -1.182 | -0.909 | 0.273 | -0.423 | -0.653 | -0.230 | -0.225 | -0.540 | -0.315 |
| Music | -1.050 | -1.116 | -0.066 | -1.554 | -1.374 | 0.180 | -0.649 | -1.040 | -0.391 | -0.321 | -0.839 | -0.518 |
| Physical education | -0.746 | -0.895 | -0.149 | -1.056 | -1.059 | -0.003 | -0.506 | -0.901 | -0.395 | -0.454 | -0.913 | -0.459 |
| Physics | -0.639 | -0.627 | 0.012 | -1.106 | -0.901 | 0.205 | -0.182 | -0.471 | -0.289 | 0.070 | -0.138 | -0.208 |
| Psychology | -0.793 | -0.934 | -0.141 | -1.149 | -1.078 | 0.071 | -0.429 | -0.802 | -0.373 | -0.315 | -0.845 | -0.530 |
| Religious studies | -0.472 | -0.592 | -0.120 | -0.805 | -0.784 | 0.021 | -0.177 | -0.478 | -0.301 | -0.030 | -0.388 | -0.358 |
| Sociology | -0.668 | -0.845 | -0.176 | -0.975 | -0.948 | 0.027 | -0.384 | -0.782 | -0.398 | -0.349 | -0.866 | -0.517 |
| Spanish | -0.485 | -0.654 | -0.169 | -0.840 | -1.023 | -0.183 | -0.184 | -0.393 | -0.209 | -0.057 | -0.293 | -0.236 |
| Statistics | -0.651 | -0.706 | -0.055 | -0.978 | -0.763 | 0.215 | -0.414 | -0.733 | -0.319 | -0.313 | -0.818 | -0.505 |
| Urdu | -0.809 | -0.414 | 0.395 | -1.209 | -0.569 | 0.640 | -0.446 | -0.379 | 0.067 | -0.232 | -0.417 | -0.185 |
| Average | -0.714 | -0.751 | -0.037 | -1.111 | -0.970 | 0.141 | -0.360 | -0.629 | -0.269 | -0.156 | -0.501 | -0.344 |
| Average (GW) | -0.584 | -0.615 | -0.030 | -0.754 | -0.659 | 0.096 | -0.357 | -0.623 | -0.266 | -0.108 | -0.345 | -0.237 |
Table B3 Differences in difficulty order overall and at grades 4, 7 and 9 between 2021/2020 and 2020/2019 for GCSE subjects from 2019-2021 (changes greater than 5 are highlighted).
| Subject | 20-19 (mean) | 21-19 (mean) | 21-20 (mean) | 20-19, (G4) | 21-19, (G4) | 21-20, (G4) | 20-19, (G7) | 21-19, (G7) | 21-20, (G7) | 20-19, (G9) | 21-19, (G9) | 21-20, (G9) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Art and Design: Three-dimensional design | -0.957 | -0.761 | 0.196 | -1.688 | -1.173 | 0.515 | -0.330 | -0.480 | -0.150 | -0.010 | -0.179 | -0.169 |
| Art and Design: Art, craft and design | -0.805 | -0.773 | 0.033 | -1.342 | -1.112 | 0.230 | -0.345 | -0.573 | -0.228 | -0.043 | -0.342 | -0.299 |
| Art and Design: Fine art | -0.857 | -0.811 | 0.046 | -1.390 | -1.166 | 0.224 | -0.392 | -0.590 | -0.198 | -0.075 | -0.291 | -0.216 |
| Art and Design: Graphics | -0.986 | -0.855 | 0.131 | -1.555 | -1.307 | 0.248 | -0.448 | -0.506 | -0.058 | -0.139 | -0.389 | -0.250 |
| Art and Design: Photography | -0.755 | -0.509 | 0.246 | -1.221 | -0.748 | 0.473 | -0.359 | -0.386 | -0.027 | -0.014 | -0.205 | -0.191 |
| Art and Design: Textile design | -0.913 | -0.854 | 0.059 | -1.506 | -1.313 | 0.193 | -0.432 | -0.537 | -0.105 | -0.040 | -0.216 | -0.176 |
| Biology | -0.638 | -0.644 | -0.007 | -1.052 | -0.832 | 0.220 | -0.249 | -0.550 | -0.301 | 0.031 | -0.223 | -0.254 |
| Business | -0.755 | -0.901 | -0.146 | -1.062 | -1.028 | 0.034 | -0.495 | -0.847 | -0.352 | -0.329 | -0.769 | -0.440 |
| Combined science | -0.410 | -0.491 | -0.080 | -0.652 | -0.622 | 0.030 | -0.187 | -0.438 | -0.251 | 0.006 | -0.129 | -0.135 |
| Chemistry | -0.615 | -0.548 | 0.067 | -1.068 | -0.752 | 0.316 | -0.191 | -0.435 | -0.244 | 0.084 | -0.128 | -0.212 |
| Citizenship studies | -0.574 | -0.641 | -0.068 | -0.867 | -0.743 | 0.124 | -0.319 | -0.599 | -0.280 | -0.116 | -0.534 | -0.418 |
| Classical civilisation | -1.046 | -1.285 | -0.239 | -1.543 | -1.462 | 0.081 | -0.586 | -1.176 | -0.590 | -0.308 | -1.150 | -0.842 |
| Computing | -0.779 | -1.091 | -0.312 | -1.142 | -1.333 | -0.191 | -0.414 | -0.876 | -0.462 | -0.360 | -0.792 | -0.432 |
| Design and technology | -0.868 | -0.831 | 0.037 | -1.307 | -1.037 | 0.270 | -0.473 | -0.715 | -0.242 | -0.233 | -0.614 | -0.381 |
| Dance | -1.058 | -1.015 | 0.043 | -1.475 | -1.258 | 0.217 | -0.709 | -0.938 | -0.229 | -0.553 | -0.934 | -0.381 |
| Drama | -0.909 | -0.852 | 0.057 | -1.294 | -0.960 | 0.334 | -0.611 | -0.932 | -0.321 | -0.284 | -0.805 | -0.521 |
| Economics | -0.964 | -1.069 | -0.104 | -1.325 | -1.256 | 0.069 | -0.582 | -0.935 | -0.353 | -0.357 | -0.741 | -0.384 |
| English language | -0.559 | -0.662 | -0.103 | -0.929 | -0.887 | 0.042 | -0.308 | -0.635 | -0.327 | -0.061 | -0.428 | -0.367 |
| English literature | -0.398 | -0.456 | -0.058 | -0.685 | -0.637 | 0.048 | -0.148 | -0.356 | -0.208 | 0.087 | -0.060 | -0.147 |
| Film studies | -0.716 | -0.566 | 0.150 | -1.169 | -0.827 | 0.342 | -0.286 | -0.410 | -0.124 | -0.058 | -0.323 | -0.265 |
| Food preparation and nutrition | -0.824 | -0.806 | 0.018 | -1.246 | -1.059 | 0.187 | -0.478 | -0.705 | -0.227 | -0.258 | -0.540 | -0.282 |
| French | -0.514 | -0.653 | -0.138 | -0.854 | -0.947 | -0.093 | -0.209 | -0.470 | -0.261 | -0.280 | -0.446 | -0.166 |
| Geography | -0.573 | -0.799 | -0.226 | -0.887 | -0.986 | -0.099 | -0.286 | -0.662 | -0.376 | -0.200 | -0.666 | -0.466 |
| German | -0.648 | -0.799 | -0.151 | -0.931 | -0.990 | -0.059 | -0.387 | -0.685 | -0.298 | -0.263 | -0.534 | -0.271 |
| History | -0.548 | -0.907 | -0.358 | -0.897 | -1.164 | -0.267 | -0.213 | -0.686 | -0.473 | -0.058 | -0.563 | -0.505 |
| Italian | -0.253 | -0.265 | -0.012 | -0.542 | -0.298 | 0.244 | 0.038 | -0.282 | -0.320 | 0.280 | -0.154 | -0.434 |
| Latin | -0.705 | -0.693 | 0.012 | -1.077 | -0.909 | 0.168 | -0.330 | -0.482 | -0.152 | -0.070 | -0.504 | -0.434 |
| Mathematics | -0.418 | -0.495 | -0.077 | -0.672 | -0.653 | 0.019 | -0.166 | -0.383 | -0.217 | -0.091 | -0.312 | -0.221 |
| Media studies | -0.785 | -0.735 | 0.050 | -1.182 | -0.909 | 0.273 | -0.423 | -0.653 | -0.230 | -0.225 | -0.540 | -0.315 |
| Music | -1.050 | -1.116 | -0.066 | -1.554 | -1.374 | 0.180 | -0.649 | -1.040 | -0.391 | -0.321 | -0.839 | -0.518 |
| Physical education | -0.746 | -0.895 | -0.149 | -1.056 | -1.059 | -0.003 | -0.506 | -0.901 | -0.395 | -0.454 | -0.913 | -0.459 |
| Physics | -0.639 | -0.627 | 0.012 | -1.106 | -0.901 | 0.205 | -0.182 | -0.471 | -0.289 | 0.070 | -0.138 | -0.208 |
| Psychology | -0.793 | -0.934 | -0.141 | -1.149 | -1.078 | 0.071 | -0.429 | -0.802 | -0.373 | -0.315 | -0.845 | -0.530 |
| Religious studies | -0.472 | -0.592 | -0.120 | -0.805 | -0.784 | 0.021 | -0.177 | -0.478 | -0.301 | -0.030 | -0.388 | -0.358 |
| Sociology | -0.668 | -0.845 | -0.176 | -0.975 | -0.948 | 0.027 | -0.384 | -0.782 | -0.398 | -0.349 | -0.866 | -0.517 |
| Spanish | -0.485 | -0.654 | -0.169 | -0.840 | -1.023 | -0.183 | -0.184 | -0.393 | -0.209 | -0.057 | -0.293 | -0.236 |
| Statistics | -0.651 | -0.706 | -0.055 | -0.978 | -0.763 | 0.215 | -0.414 | -0.733 | -0.319 | -0.313 | -0.818 | -0.505 |
| Urdu | -0.809 | -0.414 | 0.395 | -1.209 | -0.569 | 0.640 | -0.446 | -0.379 | 0.067 | -0.232 | -0.417 | -0.185 |
| Average | -0.714 | -0.751 | -0.037 | -1.111 | -0.970 | 0.141 | -0.360 | -0.629 | -0.269 | -0.156 | -0.501 | -0.344 |
| Average (GW) | -0.584 | -0.615 | -0.030 | -0.754 | -0.659 | 0.096 | -0.357 | -0.623 | -0.266 | -0.108 | -0.345 | -0.237 |
Table B4 Differences in overall difficulty and difficulties (logits) at C, A and A* between 2013/2012 and 2012/2011 for GCSE subjects from 2011-2013.
| Subject | 12-11 (mean) | 13-11 (mean) | 13-12 (mean) | 12-11 (C) | 13-11 (C) | 13-12 (C) | 12-11 (A) | 13-11 (A) | 13-12 (A) | 12-11 (A*) | 13-11 (A*) | 13-12 (A*) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Additional science | -0.025 | 0.171 | 0.195 | -0.062 | 0.087 | 0.149 | -0.048 | 0.128 | 0.176 | 0.010 | 0.363 | 0.353 |
| Art | -0.050 | -0.099 | -0.048 | -0.032 | -0.143 | -0.111 | -0.077 | -0.064 | 0.013 | -0.197 | -0.200 | -0.003 |
| Biology | 0.021 | 0.182 | 0.161 | 0.011 | 0.240 | 0.229 | -0.032 | 0.203 | 0.235 | -0.034 | 0.153 | 0.187 |
| Business | 0.020 | -0.004 | -0.024 | 0.015 | -0.038 | -0.053 | 0.042 | 0.044 | 0.002 | 0.182 | 0.158 | -0.024 |
| Chemistry | -0.087 | 0.167 | 0.254 | -0.066 | 0.217 | 0.283 | -0.053 | 0.218 | 0.271 | -0.046 | 0.236 | 0.282 |
| Core science | -0.021 | 0.209 | 0.230 | -0.040 | 0.119 | 0.159 | 0.012 | 0.271 | 0.259 | -0.055 | 0.396 | 0.451 |
| Drama | 0.005 | 0.016 | 0.011 | 0.034 | 0.017 | -0.017 | 0.016 | 0.053 | 0.037 | 0.042 | 0.090 | 0.048 |
| Design and technology | -0.031 | -0.145 | -0.114 | -0.007 | -0.166 | -0.159 | -0.028 | -0.016 | 0.012 | -0.069 | 0.018 | 0.087 |
| Electronic products | -0.079 | -0.129 | -0.050 | -0.035 | -0.056 | -0.021 | -0.106 | -0.106 | 0.000 | -0.270 | -0.260 | 0.010 |
| English literature | 0.058 | 0.157 | 0.099 | 0.111 | 0.088 | -0.023 | 0.025 | 0.101 | 0.076 | -0.136 | 0.045 | 0.181 |
| English | 0.075 | 0.036 | -0.039 | 0.156 | 0.031 | -0.125 | 0.136 | 0.160 | 0.024 | 0.265 | 0.284 | 0.019 |
| Fine art | -0.040 | -0.073 | -0.033 | 0.007 | -0.104 | -0.111 | -0.071 | -0.080 | -0.009 | -0.183 | -0.221 | -0.038 |
| Food technology | -0.124 | -0.183 | -0.059 | -0.050 | -0.155 | -0.105 | -0.062 | -0.073 | -0.011 | -0.183 | -0.087 | 0.096 |
| French | -0.058 | -0.107 | -0.049 | -0.012 | -0.079 | -0.067 | -0.047 | -0.057 | -0.010 | -0.110 | -0.135 | -0.025 |
| Geography | -0.019 | -0.028 | -0.009 | -0.019 | -0.051 | -0.032 | -0.038 | -0.012 | 0.026 | 0.059 | 0.055 | -0.004 |
| Germany | -0.085 | -0.179 | -0.094 | -0.024 | -0.099 | -0.075 | -0.078 | -0.022 | 0.056 | -0.145 | -0.085 | 0.060 |
| Art and Design: Graphics | -0.089 | -0.124 | -0.034 | -0.004 | -0.113 | -0.109 | -0.092 | -0.065 | 0.027 | -0.160 | -0.044 | 0.116 |
| Home economics: child development | -0.006 | -0.123 | -0.117 | 0.006 | -0.106 | -0.112 | -0.005 | -0.097 | -0.092 | -0.011 | -0.125 | -0.114 |
| History | -0.034 | -0.051 | -0.017 | -0.029 | -0.065 | -0.036 | -0.035 | -0.003 | 0.032 | -0.048 | 0.028 | 0.076 |
| Information technology | 0.368 | 0.302 | -0.067 | 0.267 | 0.220 | -0.047 | 0.183 | 0.113 | -0.070 | 0.485 | 0.265 | -0.220 |
| Information technology short course | 0.165 | 0.223 | 0.058 | 0.231 | 0.133 | -0.098 | 0.003 | 0.270 | 0.267 | 0.111 | 0.828 | 0.717 |
| Latin | 0.191 | 0.245 | 0.053 | 0.094 | 0.103 | 0.009 | -0.005 | 0.082 | 0.087 | 0.011 | 0.207 | 0.196 |
| Maths | -0.117 | -0.130 | -0.013 | -0.247 | -0.421 | -0.174 | -0.050 | 0.020 | 0.070 | -0.184 | -0.078 | 0.106 |
| Modern foreign languages | -0.040 | -0.079 | -0.039 | -0.028 | -0.089 | -0.061 | -0.032 | -0.022 | 0.010 | -0.107 | -0.064 | 0.043 |
| Music | -0.083 | -0.036 | 0.047 | -0.077 | -0.046 | 0.031 | -0.039 | 0.092 | 0.131 | -0.042 | 0.137 | 0.179 |
| Office technology | 0.050 | 0.053 | 0.003 | 0.055 | 0.008 | -0.047 | 0.019 | 0.063 | 0.044 | 0.101 | 0.283 | 0.182 |
| Physical education | 0.082 | 0.033 | -0.049 | -0.009 | -0.089 | -0.080 | 0.086 | 0.144 | 0.058 | 0.771 | 0.933 | 0.162 |
| Physics | -0.014 | 0.166 | 0.181 | 0.010 | 0.235 | 0.225 | -0.016 | 0.192 | 0.208 | 0.010 | 0.197 | 0.187 |
| Religious education | -0.051 | -0.026 | 0.025 | -0.029 | -0.040 | -0.011 | -0.110 | -0.021 | 0.089 | -0.150 | -0.007 | 0.143 |
| Resistant materials technology | 0.039 | -0.098 | -0.137 | -0.022 | -0.152 | -0.130 | 0.028 | -0.068 | -0.096 | 0.142 | 0.031 | -0.111 |
| Religious studies | -0.067 | -0.065 | 0.002 | -0.052 | -0.097 | -0.045 | -0.071 | -0.038 | 0.033 | -0.090 | -0.014 | 0.076 |
| Spanish | -0.090 | -0.119 | -0.029 | -0.049 | -0.126 | -0.077 | -0.075 | -0.079 | -0.004 | -0.099 | -0.062 | 0.037 |
| Social science citizenship | -0.096 | -0.110 | -0.015 | -0.037 | -0.033 | 0.004 | -0.119 | -0.052 | 0.067 | -0.167 | -0.008 | 0.159 |
| Statistics | -0.063 | 0.009 | 0.072 | -0.090 | -0.089 | 0.001 | -0.079 | 0.040 | 0.119 | 0.191 | 0.464 | 0.273 |
| Additional applied science | -0.195 | -0.034 | 0.162 | -0.051 | -0.069 | -0.018 | -0.303 | -0.156 | 0.147 | -0.326 | 0.088 | 0.414 |
| Average | -0.014 | 0.001 | 0.015 | -0.002 | -0.027 | -0.024 | -0.032 | 0.033 | 0.065 | -0.012 | 0.111 | 0.123 |
| Average (GW) | -0.011 | 0.001 | 0.012 | -0.002 | -0.021 | -0.019 | -0.025 | 0.026 | 0.051 | -0.010 | 0.086 | 0.096 |
Table B5 Differences in difficulty order overall and at grades C, A and A* between 2013/2012 and 2012/2011 for GCSE subjects from 2011-2013 (changes greater than 5 are highlighted).
| Subject | 12-11 (mean) | 13-11 (mean) | 13-12 (mean) | 12-11 (C) | 13-11 (C) | 13-12 (C) | 12-11 (A) | 13-11 (A) | 13-12 (A) | 12-11 (A*) | 13-11 (A*) | 13-12 (A) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| English | 2 | 2 | 0 | 2 | 2 | 0 | 6 | 3 | -3 | 6 | 1 | -5 |
| Art | -1 | -1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -1 | -1 | 0 |
| Fine art | -1 | -1 | 0 | -1 | -1 | 0 | -1 | -1 | 0 | -2 | -1 | 1 |
| Design Technology | 1 | 0 | -1 | 1 | -1 | -2 | 0 | 0 | 0 | 0 | 0 | 0 |
| Home economics: child development | 1 | 1 | 0 | 1 | -2 | -3 | 0 | 0 | 0 | 4 | 1 | -3 |
| English literature | 2 | 5 | 3 | 2 | 4 | 2 | 1 | 4 | 3 | -4 | -2 | 2 |
| Food technology | -3 | -2 | 1 | -2 | -6 | -4 | 0 | 0 | 0 | -1 | 0 | 1 |
| Physical education | 5 | 1 | -4 | -1 | -4 | -3 | 2 | 2 | 0 | 16 | 18 | 2 |
| Maths | -2 | -2 | 0 | -5 | -5 | 0 | 0 | 0 | 0 | -3 | -3 | 0 |
| Biology | 2 | 3 | 1 | 0 | 7 | 7 | 1 | 4 | 3 | 1 | 3 | 2 |
| Modern foreign languages | -1 | -3 | -2 | -1 | -2 | -1 | 1 | 1 | 0 | -3 | -6 | -3 |
| Physics | -1 | 2 | 3 | 2 | 7 | 5 | 0 | 5 | 5 | 0 | 3 | 3 |
| Chemistry | -4 | 3 | 7 | -2 | 4 | 6 | -1 | 5 | 6 | 1 | 6 | 5 |
| Drama | 0 | -2 | -2 | 3 | 1 | -2 | 2 | 2 | 0 | 1 | -1 | -2 |
| Resistant materials technology | 1 | -5 | -6 | 0 | -5 | -5 | 1 | -6 | -7 | -2 | -3 | -1 |
| Core science | -1 | 2 | 3 | 0 | 2 | 2 | 1 | 1 | 0 | -4 | 6 | 10 |
| Information Technology | 9 | 5 | -4 | 3 | 2 | -1 | 8 | -1 | -9 | 6 | 0 | -6 |
| Additional science | 0 | 1 | 1 | -1 | 1 | 2 | -2 | 2 | 4 | 4 | 7 | 3 |
| Religious studies | -2 | -4 | -2 | -1 | -3 | -2 | 0 | 0 | 0 | 0 | 1 | 1 |
| Office technology | 1 | 1 | 0 | 2 | 1 | -1 | 1 | 0 | -1 | -1 | 1 | 2 |
| Electronic products | -1 | -4 | -3 | -1 | -2 | -1 | -2 | -5 | -3 | -8 | -10 | -2 |
| Additional applied science | -3 | 1 | 4 | 0 | -1 | -1 | -3 | -2 | 1 | 0 | -1 | -1 |
| Geography | 1 | 2 | 1 | 0 | 0 | 0 | -1 | -2 | -1 | 1 | 0 | -1 |
| Music | -2 | 0 | 2 | 1 | 2 | 1 | -2 | 1 | 3 | 0 | 0 | 0 |
| Art and Design: Graphics | -2 | -5 | -3 | 0 | -1 | -1 | -6 | -5 | 1 | -5 | -4 | 1 |
| Statistics | 1 | 2 | 1 | -2 | -1 | 1 | 0 | 0 | 0 | 3 | 3 | 0 |
| Social science citizenship | -2 | -1 | 1 | 0 | 1 | 1 | -2 | -3 | -1 | -5 | -4 | 1 |
| Religious Education | 0 | 1 | 1 | 0 | 0 | 0 | -6 | -2 | 4 | -1 | -3 | -2 |
| History | 0 | -2 | -2 | 0 | 0 | 0 | -2 | -4 | -2 | 1 | 0 | -1 |
| Business | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 2 | 0 | 4 | 2 | -2 |
| Spanish | 0 | 0 | 0 | -2 | -2 | 0 | -1 | -3 | -2 | 0 | -3 | -3 |
| Information technology short course | 2 | 2 | 0 | 2 | 2 | 0 | 1 | 2 | 1 | 0 | 1 | 1 |
| French | -1 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | -1 | -4 | -6 | -2 |
| German | -1 | -2 | -1 | 0 | 0 | 0 | 1 | 0 | -1 | -3 | -5 | -2 |
| Latin | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -1 | 0 | 1 |
| Total changes | 56 | 68 | 60 | 38 | 72 | 54 | 58 | 68 | 62 | 96 | 106 | 72 |
| Average change | 1.60 | 1.94 | 1.71 | 1.09 | 2.06 | 1.54 | 1.66 | 1.94 | 1.77 | 2.74 | 3.03 | 2.06 |
Table B6 Difference in weighted mean grade between 2021/2020 and 2020/2019 for A level subjects from 2019-2021.
| Subject | 2020-2019 | 2021-2019 | 2021-2020 |
|---|---|---|---|
| Chinese | 0.893 | 1.349 | 0.456 |
| Art and Design: Textile design | 0.524 | 0.622 | 0.098 |
| Art and Design: Graphics | 0.428 | 0.472 | 0.044 |
| Art and Design: Photography | 0.477 | 0.574 | 0.097 |
| Art and Design: Three-dimensional design | 0.439 | 0.422 | -0.017 |
| Art and Design: Fine art | 0.441 | 0.555 | 0.115 |
| Art and Design: Art, craft and design | 0.468 | 0.618 | 0.150 |
| Dance | 0.622 | 0.851 | 0.229 |
| Sociology | 0.430 | 0.516 | 0.086 |
| Film studies | 0.453 | 0.575 | 0.121 |
| Media studies | 0.417 | 0.533 | 0.116 |
| Business studeis | 0.477 | 0.601 | 0.124 |
| Drama and theatre studies | 0.667 | 0.880 | 0.213 |
| Politics | 0.489 | 0.672 | 0.183 |
| Classical civilisation | 0.505 | 0.744 | 0.239 |
| English language | 0.440 | 0.555 | 0.115 |
| English language and literature | 0.493 | 0.682 | 0.189 |
| Religious studies | 0.542 | 0.729 | 0.187 |
| Law | 0.572 | 0.671 | 0.099 |
| English literature | 0.517 | 0.680 | 0.162 |
| Spanish | 0.682 | 0.962 | 0.280 |
| Design and Technology: Product design | 0.677 | 0.929 | 0.251 |
| History | 0.460 | 0.616 | 0.156 |
| Geography | 0.439 | 0.575 | 0.136 |
| Economics | 0.468 | 0.583 | 0.116 |
| German | 0.649 | 0.913 | 0.264 |
| Psychology | 0.530 | 0.720 | 0.190 |
| French | 0.583 | 0.777 | 0.193 |
| Further maths | 0.750 | 0.780 | 0.030 |
| Physical education | 0.708 | 0.973 | 0.266 |
| Accounting | 0.600 | 0.739 | 0.138 |
| Music | 0.818 | 1.189 | 0.371 |
| Mathematics | 0.480 | 0.560 | 0.080 |
| Music technology | 0.784 | 0.983 | 0.199 |
| Philosophy | 0.719 | 1.012 | 0.293 |
| Computing | 0.782 | 0.962 | 0.180 |
| Physics | 0.608 | 0.706 | 0.098 |
| Chemistry | 0.668 | 0.761 | 0.093 |
| Biology | 0.690 | 0.924 | 0.234 |
| Average | 0.574 | 0.743 | 0.169 |
Table B7 Differences in overall difficulty and difficulties (logits) at grades C, A and A* between 2021/2020 and 2020/2019 for A level subjects from 2019-2021.
| Subject | 20-19 (mean) | 21-19 (mean) | 21-20 (mean) | 20-19 (C) | 21-19 (C) | 21-20 (C) | 20-19 (A) | 21-19 (A) | 21-20 (A) | 20-19 (A*) | 21-19 (A*) | 21-20 (A*) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Accounting | -1.803 | -1.996 | -0.193 | -2.028 | -1.951 | 0.077 | -1.036 | -2.060 | -1.024 | -2.251 | -4.116 | -1.865 |
| Art and Design: Three-dimensional design | -1.565 | -0.768 | 0.797 | -1.840 | -0.671 | 1.169 | -0.814 | -0.842 | -0.028 | 0.220 | -0.110 | -0.330 |
| Art and Design: Art, craft and design | -1.600 | -1.338 | 0.262 | -1.780 | -1.378 | 0.402 | -0.944 | -1.282 | -0.338 | -0.102 | -0.696 | -0.594 |
| Art and Design: Fine art | -1.613 | -1.336 | 0.277 | -1.923 | -1.523 | 0.400 | -0.946 | -1.160 | -0.214 | -0.220 | -0.618 | -0.398 |
| Art and Design: Graphics | -1.845 | -1.133 | 0.712 | -1.974 | -1.067 | 0.907 | -0.970 | -1.126 | -0.156 | -0.410 | -0.639 | -0.229 |
| Art and Design: Photography | -1.908 | -1.235 | 0.673 | -2.168 | -1.340 | 0.828 | -1.006 | -1.089 | -0.083 | -0.447 | -0.611 | -0.164 |
| Art and Design: Textile design | -2.481 | -1.648 | 0.833 | -2.842 | -1.873 | 0.969 | -1.127 | -1.106 | 0.021 | -0.307 | -0.405 | -0.098 |
| Biology | -1.212 | -1.640 | -0.428 | -1.472 | -1.773 | -0.301 | -0.455 | -1.269 | -0.814 | -0.276 | -1.346 | -1.070 |
| Business studies | -1.699 | -1.413 | 0.286 | -1.893 | -1.439 | 0.454 | -1.049 | -1.423 | -0.374 | -0.533 | -1.172 | -0.639 |
| Chemistry | -1.059 | -1.308 | -0.250 | -1.275 | -1.322 | -0.047 | -0.362 | -1.191 | -0.829 | -0.426 | -1.547 | -1.121 |
| Chinese | -0.755 | -4.425 | -3.669 | 0.548 | -6.561 | -7.109 | -3.665 | -6.429 | -2.764 | -3.170 | -6.942 | -3.772 |
| Classical civilisationl | -1.824 | -1.859 | -0.035 | -2.003 | -1.695 | 0.308 | -1.338 | -2.421 | -1.083 | -1.123 | -2.647 | -1.524 |
| Computing | -2.230 | -2.743 | -0.513 | -2.407 | -2.741 | -0.334 | -1.638 | -2.878 | -1.240 | -2.372 | -4.307 | -1.935 |
| Dance | -2.811 | -1.570 | 1.241 | -2.726 | -1.743 | 0.983 | -1.676 | -1.711 | -0.035 | -0.476 | -1.304 | -0.828 |
| Drama and theatre studies | -2.449 | -2.315 | 0.134 | -2.753 | -2.386 | 0.367 | -1.872 | -2.405 | -0.533 | -1.171 | -2.288 | -1.117 |
| Design and Technology: Product design | -2.162 | -2.361 | -0.199 | -2.430 | -2.320 | 0.110 | -1.547 | -2.419 | -0.872 | -1.019 | -2.588 | -1.569 |
| Economics | -1.731 | -1.451 | 0.280 | -2.041 | -1.538 | 0.503 | -0.898 | -1.344 | -0.446 | -0.786 | -1.687 | -0.901 |
| English language | -1.340 | -1.033 | 0.308 | -1.403 | -0.948 | 0.455 | -1.175 | -1.692 | -0.517 | -0.821 | -1.869 | -1.048 |
| English language and literature | -1.640 | -1.470 | 0.170 | -1.948 | -1.574 | 0.374 | -1.032 | -1.659 | -0.627 | -0.450 | -1.458 | -1.008 |
| English literature | -1.751 | -1.542 | 0.209 | -2.149 | -1.770 | 0.379 | -0.777 | -1.352 | -0.575 | -0.139 | -0.993 | -0.854 |
| Film studies | -2.012 | -1.561 | 0.451 | -2.083 | -1.467 | 0.616 | -1.635 | -2.059 | -0.424 | -1.410 | -2.151 | -0.741 |
| French | -1.680 | -1.951 | -0.271 | -1.923 | -2.035 | -0.112 | -0.938 | -1.797 | -0.859 | -0.505 | -1.813 | -1.308 |
| Further maths | -1.413 | -2.028 | -0.615 | -1.654 | -2.030 | -0.376 | -0.681 | -2.048 | -1.367 | -0.539 | -2.298 | -1.759 |
| Geography | -1.452 | -1.185 | 0.267 | -1.644 | -1.228 | 0.416 | -0.790 | -1.130 | -0.340 | -0.653 | -1.311 | -0.658 |
| German | -1.550 | -2.302 | -0.753 | -1.877 | -2.491 | -0.614 | -0.774 | -1.922 | -1.148 | -0.370 | -2.010 | -1.640 |
| History | -1.454 | -1.378 | 0.076 | -1.684 | -1.466 | 0.218 | -0.707 | -1.315 | -0.608 | -0.348 | -1.287 | -0.939 |
| Law | -1.670 | -1.566 | 0.103 | -2.007 | -1.689 | 0.318 | -0.834 | -1.171 | -0.337 | -0.354 | -0.828 | -0.474 |
| Mathematics | -1.035 | -1.230 | -0.196 | -1.342 | -1.336 | 0.006 | -0.257 | -1.013 | -0.756 | -0.102 | -1.154 | -1.052 |
| Media studies | -1.722 | -1.024 | 0.697 | -1.732 | -0.881 | 0.851 | -1.280 | -1.656 | -0.376 | -1.098 | -1.865 | -0.767 |
| Music | -2.797 | -3.207 | -0.410 | -3.134 | -3.358 | -0.224 | -1.972 | -3.263 | -1.291 | -1.578 | -3.444 | -1.866 |
| Music technology | -2.748 | -2.571 | 0.177 | -2.901 | -2.461 | 0.440 | -2.148 | -2.800 | -0.652 | -1.464 | -2.259 | -0.795 |
| Physical education | -1.972 | -2.113 | -0.141 | -2.101 | -2.168 | -0.067 | -1.393 | -2.011 | -0.618 | -0.968 | -1.891 | -0.923 |
| Philosophy | -2.158 | -2.642 | -0.484 | -2.254 | -2.685 | -0.431 | -1.624 | -2.658 | -1.034 | -1.784 | -2.944 | -1.160 |
| Physics | -1.214 | -1.756 | -0.542 | -1.370 | -1.735 | -0.365 | -0.617 | -1.781 | -1.164 | -0.629 | -2.215 | -1.586 |
| Politics | -1.812 | -1.832 | -0.020 | -2.101 | -1.899 | 0.202 | -1.078 | -1.678 | -0.600 | -0.675 | -1.622 | -0.947 |
| Psychology | -1.429 | -1.397 | 0.031 | -1.569 | -1.361 | 0.208 | -0.811 | -1.408 | -0.597 | -0.334 | -1.259 | -0.925 |
| Religious studies | -1.685 | -1.692 | -0.008 | -2.001 | -1.919 | 0.082 | -0.962 | -1.394 | -0.432 | -0.868 | -1.600 | -0.732 |
| Sociology | -1.451 | -1.021 | 0.430 | -1.713 | -1.126 | 0.587 | -0.703 | -0.904 | -0.201 | -0.294 | -0.682 | -0.388 |
| Spanish | -2.055 | -2.330 | -0.275 | -2.343 | -2.422 | -0.079 | -1.231 | -2.154 | -0.923 | -0.790 | -2.291 | -1.501 |
| Average | -1.764 | -1.779 | -0.015 | -1.947 | -1.881 | 0.066 | -1.148 | -1.821 | -0.673 | -0.796 | -1.853 | -1.057 |
| Average (GW) | -0.549 | -0.553 | -0.005 | -0.610 | -0.589 | 0.021 | -0.330 | -0.523 | -0.193 | -0.215 | -0.501 | -0.286 |
Table B8 Differences in difficulty order overall and at grades C, A and A* between 2021/2020 and 2020/2019 for A level subjects from 2019-2021 (changes greater than 5 are highlighted).
| Subject | 20-19 (mean) | 21-19 (mean) | 21-20 (mean) | 20-19 (C) | 21-19 (C) | 21-20 (C) | 20-19 (A) | 21-19 (A) | 21-20 (A) | 20-19 (A*) | 21-19 (A*) | 21-20 (A*) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Chinese | 5 | 0 | -5 | 8 | 0 | -8 | -8 | -8 | 0 | -18 | -33 | -15 |
| Art and Design: Photography | 1 | 0 | -1 | -1 | -1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 |
| Media studies | 4 | 5 | 1 | 3 | 5 | 2 | 0 | 1 | 1 | 0 | 1 | 1 |
| Art and Design: Graphics | 0 | 2 | 2 | 1 | 3 | 2 | 2 | 2 | 0 | -1 | 0 | 1 |
| Dance | -4 | -2 | 2 | -5 | -2 | 3 | -3 | -3 | 0 | -1 | -2 | -1 |
| Art and Design: Three-dimensional design | 2 | 5 | 3 | 1 | 4 | 3 | 2 | 3 | 1 | 1 | 3 | 2 |
| Art and Design: Textile design | -5 | -3 | 2 | -6 | -4 | 2 | 1 | 1 | 0 | 0 | 1 | 1 |
| Film studies | -3 | -3 | 0 | 0 | -1 | -1 | -3 | -3 | 0 | -5 | -3 | 2 |
| Art and Design: Fine art | 0 | 0 | 0 | 0 | -1 | -1 | 2 | 3 | 1 | -1 | 2 | 3 |
| Art and Design: Art, craft and design | 0 | 0 | 0 | 3 | 2 | -1 | 2 | 0 | -2 | 2 | 2 | 0 |
| Sociology | 1 | 2 | 1 | 1 | 2 | 1 | 3 | 5 | 2 | 0 | 4 | 4 |
| Drama and theatre studies | -1 | -5 | -4 | -6 | -7 | -1 | -5 | -5 | 0 | -3 | -4 | -1 |
| English language | 2 | 3 | 1 | 2 | 3 | 1 | -1 | 1 | 2 | 3 | 5 | 2 |
| English language and literature | -1 | 0 | 1 | -1 | -2 | -1 | 1 | 3 | 2 | 2 | 3 | 1 |
| Business studies | 1 | 5 | 4 | 1 | 4 | 3 | 3 | 5 | 2 | 4 | 6 | 2 |
| Religious studies | 1 | 2 | 1 | 1 | -1 | -2 | 3 | 8 | 5 | -1 | 3 | 4 |
| Design and Technology: Product design | -3 | -5 | -2 | -3 | -4 | -1 | -1 | -4 | -3 | 0 | -4 | -4 |
| Law | 3 | 4 | 1 | 1 | 4 | 3 | 3 | 6 | 3 | 2 | 6 | 4 |
| English literature | 1 | 5 | 4 | 0 | 2 | 2 | 6 | 8 | 2 | 4 | 5 | 1 |
| Classical civilisation | -2 | -1 | 1 | 1 | 2 | 1 | -3 | -9 | -6 | -1 | -5 | -4 |
| Geography | 5 | 9 | 4 | 5 | 6 | 1 | 3 | 10 | 7 | 3 | 10 | 7 |
| History | 5 | 6 | 1 | 5 | 5 | 0 | 5 | 6 | 1 | 6 | 10 | 4 |
| Psychology | 5 | 6 | 1 | 5 | 7 | 2 | 4 | 7 | 3 | 2 | 5 | 3 |
| Politics | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 3 | 3 | 2 | 5 | 3 |
| Physical education | -2 | -4 | -2 | 0 | -3 | -3 | -1 | -2 | -1 | -1 | -1 | 0 |
| Spanish | -4 | -9 | -5 | -2 | -6 | -4 | -3 | -4 | -1 | 3 | 0 | -3 |
| Economics | 2 | 6 | 4 | 1 | 5 | 4 | 3 | 6 | 3 | 1 | 4 | 3 |
| French | 2 | 3 | 1 | 1 | 1 | 0 | 2 | 0 | -2 | 6 | 4 | -2 |
| Music technology | -10 | -6 | 4 | -8 | -3 | 5 | -13 | -13 | 0 | -7 | -3 | 4 |
| German | 3 | -4 | -7 | 2 | -2 | -4 | 5 | -1 | -6 | 5 | 2 | -3 |
| Accounting | 1 | 1 | 0 | 0 | 1 | 1 | 1 | -4 | -5 | -10 | -18 | -8 |
| Music | -8 | -17 | -9 | -8 | -13 | -5 | -12 | -18 | -6 | -4 | -12 | -8 |
| Philosophy | -2 | -6 | -4 | -2 | -6 | -4 | -2 | -7 | -5 | -6 | -7 | -1 |
| Mathematics | 2 | 2 | 0 | 0 | 0 | 0 | 4 | 4 | 0 | 9 | 10 | 1 |
| Computing | -1 | -1 | 0 | 0 | 0 | 0 | -2 | -3 | -1 | -4 | -8 | -4 |
| Biology | -1 | -1 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 5 | 5 | 0 |
| Chemistry | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 | 2 | 2 | 0 |
| Physics | 0 | -1 | -1 | 0 | -1 | -1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Further maths | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Total changes | 94 | 136 | 80 | 84 | 114 | 74 | 114 | 168 | 76 | 126 | 200 | 108 |
| Average change | 2.41 | 3.49 | 2.05 | 2.15 | 2.92 | 1.90 | 2.92 | 4.31 | 1.95 | 3.23 | 5.13 | 2.77 |
Table B9 Differences in overall difficulty and difficulties (logits) at C, A and A* between 2019/2018 and 2018/2017 for A level subjects from 2017-2019.
| Subject | 18-17 (mean) | 19-17 (mean) | 19-18 (mean) | 18-17 (C) | 19-17 (C) | 19-18 (C) | 18-17 (A) | 19-17 (A) | 19-18 (A) | 18-17 (A*) | 19-17 (A*) | 19-18 (A*) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Accounting | -0.068 | 0.114 | 0.182 | -0.056 | 0.039 | 0.095 | 0.098 | 0.440 | 0.342 | 0.497 | 2.450 | 1.953 |
| Art and Design: Art, craft and design | -0.036 | -0.486 | -0.451 | -0.068 | -0.857 | -0.789 | 0.097 | 0.291 | 0.194 | 0.171 | 0.407 | 0.236 |
| Art and Design: Fine art | -0.100 | -0.502 | -0.402 | -0.171 | -0.763 | -0.592 | -0.083 | 0.089 | 0.172 | 0.078 | 0.324 | 0.246 |
| Art and Design: Graphics | 0.048 | -0.166 | -0.214 | -0.040 | -0.588 | -0.548 | 0.175 | 0.424 | 0.249 | 0.460 | 0.641 | 0.181 |
| Art and Design: Photography | -0.065 | -0.530 | -0.465 | -0.133 | -0.795 | -0.662 | 0.123 | 0.094 | -0.029 | 0.067 | 0.301 | 0.234 |
| Art and Design: Textile design | -0.133 | -0.141 | -0.008 | -0.278 | -0.291 | -0.013 | -0.084 | 0.082 | 0.166 | 0.125 | 0.280 | 0.155 |
| Biology | -0.002 | 0.177 | 0.179 | -0.057 | 0.086 | 0.143 | 0.130 | 0.489 | 0.359 | 0.404 | 1.062 | 0.658 |
| Business studies | -0.126 | -0.262 | -0.136 | -0.180 | -0.428 | -0.248 | 0.031 | 0.170 | 0.139 | 0.102 | 0.370 | 0.268 |
| Chemistry | -0.061 | 0.146 | 0.207 | -0.115 | 0.059 | 0.174 | 0.085 | 0.426 | 0.341 | 0.306 | 1.100 | 0.794 |
| Chinese | -0.411 | -0.725 | -0.315 | -0.497 | -1.023 | -0.526 | -0.224 | 0.006 | 0.230 | -0.065 | -0.486 | -0.421 |
| Classical civilisation | 0.028 | -0.404 | -0.432 | -0.085 | -0.712 | -0.627 | 0.111 | 0.344 | 0.233 | 0.542 | 0.858 | 0.316 |
| Computing | 0.055 | 0.110 | 0.055 | 0.001 | -0.051 | -0.052 | 0.127 | 0.556 | 0.429 | 0.566 | 0.943 | 0.377 |
| Dance | 0.117 | 0.122 | 0.005 | 0.013 | -0.029 | -0.042 | -0.013 | 0.178 | 0.191 | -0.075 | 0.194 | 0.269 |
| Drama and theatre studies | -0.096 | -0.395 | -0.299 | -0.120 | -0.696 | -0.576 | -0.067 | 0.045 | 0.112 | 0.064 | 0.438 | 0.374 |
| Design and Technology: Product design | -0.020 | -0.124 | -0.103 | -0.006 | -0.338 | -0.332 | -0.044 | 0.426 | 0.470 | 0.205 | 0.791 | 0.586 |
| Economics | -0.029 | -0.058 | -0.029 | -0.037 | -0.167 | -0.130 | 0.115 | 0.378 | 0.263 | 0.306 | 0.858 | 0.552 |
| English language | -0.164 | -0.546 | -0.382 | -0.174 | -0.863 | -0.689 | -0.084 | 0.326 | 0.410 | 0.073 | 0.726 | 0.653 |
| English language and literature | 0.062 | -0.237 | -0.299 | 0.070 | -0.376 | -0.446 | 0.172 | 0.333 | 0.161 | 0.358 | 0.531 | 0.173 |
| English literature | 0.040 | 0.052 | 0.012 | 0.007 | -0.074 | -0.081 | 0.147 | 0.434 | 0.287 | 0.199 | 0.657 | 0.458 |
| Film studies | 0.139 | 0.854 | 0.714 | -0.083 | 0.953 | 1.036 | 0.115 | 0.234 | 0.119 | -0.014 | 0.672 | 0.686 |
| French | 0.038 | -0.064 | -0.102 | -0.006 | -0.307 | -0.301 | 0.058 | 0.425 | 0.367 | 0.274 | 0.618 | 0.344 |
| Further maths | 0.123 | 0.782 | 0.659 | 0.037 | 0.630 | 0.593 | 0.275 | 1.261 | 0.986 | 0.580 | 1.787 | 1.207 |
| Geography | -0.244 | -0.203 | 0.041 | -0.351 | -0.353 | -0.002 | -0.023 | 0.215 | 0.238 | 0.086 | 0.508 | 0.422 |
| Geology | -0.205 | 0.470 | 0.675 | -0.449 | 0.498 | 0.947 | 0.086 | 0.322 | 0.236 | 0.625 | 0.818 | 0.193 |
| German | 0.565 | 0.277 | -0.289 | 0.793 | 0.399 | -0.394 | 0.223 | 0.285 | 0.062 | 0.117 | -0.019 | -0.136 |
| History | 0.088 | -0.011 | -0.099 | 0.053 | -0.120 | -0.173 | 0.123 | 0.291 | 0.168 | 0.449 | 0.849 | 0.400 |
| Latin | -0.216 | 0.505 | 0.721 | -0.244 | 0.764 | 1.008 | 0.058 | 0.608 | 0.550 | 0.688 | 0.964 | 0.276 |
| Law | -0.094 | -0.228 | -0.133 | -0.221 | -0.303 | -0.082 | 0.162 | 0.053 | -0.109 | 0.481 | 0.132 | -0.349 |
| Mathematics | -0.112 | 0.408 | 0.520 | -0.205 | 0.493 | 0.698 | 0.046 | 0.455 | 0.409 | 0.606 | 0.916 | 0.310 |
| Media studies | 0.042 | -0.342 | -0.384 | 0.052 | -0.608 | -0.660 | -0.070 | 0.073 | 0.143 | 0.054 | 0.520 | 0.466 |
| Music | 0.127 | 0.203 | 0.076 | 0.173 | 0.124 | -0.049 | 0.086 | 0.418 | 0.332 | 0.406 | 0.918 | 0.512 |
| Music technology | 0.035 | 0.042 | 0.006 | -0.054 | -0.195 | -0.141 | 0.187 | 0.775 | 0.588 | 0.677 | 1.843 | 1.166 |
| Physical education | -0.235 | -0.322 | -0.087 | -0.391 | -0.606 | -0.215 | 0.132 | 0.351 | 0.219 | -0.228 | 0.229 | 0.457 |
| Philosophy | -0.131 | -0.107 | 0.024 | -0.202 | -0.149 | 0.053 | -0.034 | 0.223 | 0.257 | -0.233 | 0.487 | 0.720 |
| Physics | -0.160 | -0.088 | 0.071 | -0.236 | -0.282 | -0.046 | 0.038 | 0.420 | 0.382 | 0.197 | 0.878 | 0.681 |
| Politics | -0.107 | -0.444 | -0.336 | -0.133 | -0.642 | -0.509 | -0.058 | 0.074 | 0.132 | -0.062 | 0.373 | 0.435 |
| Psychology | -0.090 | -0.108 | -0.018 | -0.125 | -0.250 | -0.125 | 0.039 | 0.315 | 0.276 | 0.142 | 0.539 | 0.397 |
| Religious studies | -0.024 | -0.147 | -0.123 | -0.035 | -0.186 | -0.151 | 0.045 | 0.154 | 0.109 | 0.416 | 0.784 | 0.368 |
| Sociology | -0.170 | -0.388 | -0.218 | -0.226 | -0.526 | -0.300 | -0.011 | 0.075 | 0.086 | -0.029 | 0.250 | 0.279 |
| Spanish | 0.054 | -0.093 | -0.146 | 0.126 | -0.178 | -0.304 | -0.020 | 0.164 | 0.184 | -0.131 | 0.299 | 0.430 |
| Average | -0.038 | -0.071 | -0.033 | -0.091 | -0.218 | -0.126 | 0.057 | 0.318 | 0.261 | 0.237 | 0.670 | 0.433 |
| Average (GW) | -0.015 | -0.029 | -0.013 | -0.037 | -0.087 | -0.051 | 0.023 | 0.127 | 0.105 | 0.095 | 0.268 | 0.173 |
Table B10 Differences in difficulty order overall and at grades C, A and A* and between 2019/2018 and 2018/2017 for A level subjects from 2017-2019 (changes greater than 5 are highlighted).
| Subject | 18-17 (mean) | 19-17 (mean) | 19-18 (mean) | 18-17 (C) | 19-17 (C) | 19-18 (C) | 18-17 (A) | 19-17 (A) | 19-18 (A) | 18-17 (A*) | 19-17 (A*) | 19-18 (A*) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Film studies | 1 | 6 | 5 | 1 | 3 | 2 | 1 | 2 | 1 | -2 | 2 | 4 |
| Chinese | -1 | -1 | 0 | -1 | -1 | 0 | -2 | -2 | 0 | -1 | -3 | -2 |
| Dance | 0 | 2 | 2 | 0 | 3 | 3 | 1 | 1 | 0 | 0 | 0 | 0 |
| Art and Design: Graphics | 1 | 0 | -1 | 1 | -1 | -2 | 2 | 2 | 0 | 2 | 2 | 0 |
| Art and Design: Photography | -1 | -3 | -2 | 0 | -2 | -2 | 0 | -1 | -1 | -1 | -1 | 0 |
| Media studies | 1 | -3 | -4 | 0 | -2 | -2 | 1 | 1 | 0 | 0 | -1 | -1 |
| Art and Design: Textile design | -1 | -1 | 0 | -1 | 1 | 2 | -2 | -1 | 1 | -1 | -1 | 0 |
| Art and Design: Art, craft and design | 1 | 1 | 0 | 0 | -2 | -2 | 1 | 1 | 0 | 1 | 0 | -1 |
| Art and Design: Fine art | -1 | -1 | 0 | 0 | 1 | 1 | -2 | -2 | 0 | -1 | 0 | 1 |
| Sociology | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| English language and literature | 1 | 2 | 1 | 2 | 2 | 0 | 2 | 2 | 0 | 3 | 1 | -2 |
| Drama and theatre studies | -1 | -1 | 0 | 0 | 0 | 0 | -3 | -6 | -3 | -2 | -2 | 0 |
| English language | 0 | -1 | -1 | -2 | -2 | 0 | -2 | 1 | 3 | 0 | -1 | -1 |
| Business studies | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -1 | -1 | 0 | -1 | -1 |
| English literature | 1 | 3 | 2 | 0 | 2 | 2 | 3 | 3 | 0 | 0 | 1 | 1 |
| Religious studies | -1 | -1 | 0 | 0 | 1 | 1 | 0 | -2 | -2 | 3 | 1 | -2 |
| Design and Technology: Product design | 0 | -1 | -1 | 4 | -1 | -5 | -1 | 1 | 2 | 2 | 3 | 1 |
| Law | 0 | -1 | -1 | -2 | -1 | 1 | 1 | -1 | -2 | 5 | 0 | -5 |
| History | 3 | 2 | -1 | 4 | 3 | -1 | 1 | -1 | -2 | 0 | -2 | -2 |
| Geology | 0 | 7 | 7 | -5 | 5 | 10 | 0 | 1 | 1 | 3 | 0 | -3 |
| Geography | -2 | -1 | 1 | -4 | 0 | 4 | -2 | 1 | 3 | -2 | -2 | 0 |
| Psychology | -1 | 0 | 1 | 1 | 1 | 0 | -1 | 0 | 1 | -2 | -1 | 1 |
| Classical civilisation | 0 | -4 | -4 | 1 | -4 | -5 | 0 | -2 | -2 | 3 | 0 | -3 |
| Spanish | 1 | 1 | 0 | 4 | 2 | -2 | 0 | -1 | -1 | -6 | -6 | 0 |
| Physical education | -1 | -1 | 0 | -4 | -4 | 0 | 0 | 1 | 1 | -6 | -4 | 2 |
| Economics | 1 | 0 | -1 | 1 | 1 | 0 | 2 | 2 | 0 | 0 | 1 | 1 |
| Politics | -1 | -4 | -3 | -1 | -4 | -3 | -2 | -4 | -2 | -4 | -4 | 0 |
| German | 4 | 2 | -2 | 7 | 4 | -3 | 4 | 0 | -4 | 0 | -7 | -7 |
| Music technolgy | 0 | 0 | 0 | -1 | -2 | -1 | 0 | 1 | 1 | 1 | 9 | 8 |
| Music | 1 | 2 | 1 | -1 | 0 | 1 | 0 | -1 | -1 | -1 | -1 | 0 |
| French | -1 | -3 | -2 | -1 | -3 | -2 | -1 | 1 | 2 | -2 | -3 | -1 |
| Accounting | -4 | -1 | 3 | -1 | 0 | 1 | 1 | 2 | 1 | 4 | 15 | 11 |
| Philosophy | 0 | 0 | 0 | -1 | 0 | 1 | 0 | -1 | -1 | -4 | -1 | 3 |
| Mathematics | 1 | 0 | -1 | 0 | 1 | 1 | -2 | 1 | 3 | 3 | 1 | -2 |
| Latin | -1 | 2 | 3 | -1 | 3 | 4 | 0 | 1 | 1 | 3 | 1 | -2 |
| Computing | 1 | -1 | -2 | 0 | -2 | -2 | 0 | 0 | 0 | 0 | 0 | 0 |
| Biology | -1 | -1 | 0 | 0 | -2 | -2 | 0 | 1 | 1 | 1 | 2 | 1 |
| Chemistry | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Physics | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -1 | -1 | 0 | 0 | 0 |
| Further maths | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 1 |
| Total changes | 36 | 60 | 52 | 52 | 66 | 68 | 40 | 54 | 46 | 70 | 82 | 70 |
| Average change | 0.9 | 1.5 | 1.3 | 1.3 | 1.65 | 1.7 | 1 | 1.35 | 1.15 | 1.75 | 2.05 | 1.75 |
-
The reformed GCSEs were introduced in phases from 2015. In 2017, only mathematics, English language and English literature were awarded with numerical grades. In 2018, a considerable number of subjects were still awarded with letter grades. Most subjects were awarded with numerical grades in 2019. For convenience, the data from 2011 to 2013 used in previous studies were analysed for this investigation. ↩
-
Grade 1 rather than U was used as the reference category for GCSEs in order to resolve the problem of disordered categories and disordered thresholds and large misfit associated with the reference categories when running the Rasch model. U was treated as missing and excluded from the analysis. ↩
-
Traditionally, the term “item difficulty” is used in Rasch analysis of educational and psychometric test data. When the Rasch model is used in the context of inter-subject comparability investigations, the term “subject difficulty” is used (see Coe, 2008). ↩
-
In this report, “between 2019/2020 and 2020/2021” means: between 2019 and 2020, between 2019 and 2021, and between 2020 and 2021. ↩