Research and analysis

Inter-subject comparability in GCSEs and A levels in summer 2022

Published 9 August 2023

Applies to England

Summary

This technical report, which follows on from analyses undertaken by He and Black (2020) and He and Cadwallader (2022), uses statistical methods to investigate the comparability of grading standards across subjects for GCSEs and A levels in summer 2022. Inter-subject comparability is explored over a period of 4 years:

• summer 2022, when grades were awarded through exams (with adaptations)
• summer 2021, when Teacher Assessed Grades (TAGs) were used
• summer 2020, when, for most students, Centre Assessment Grades (CAGs) were used, and
• summer 2019, when grades were awarded through normal exams (prior to the COVID-19 pandemic)

In this report, ‘subject difficulty’ is based on statistical analysis of inter-subject comparability. Statistical approaches to inter-subject comparability use cohort-level data to compare the relative likelihood of students achieving a grade in certain subjects, on the basis of past attainment or concurrent attainment derived using grades achieved in all GCSE or A level subjects. This involves an assumption that there is a common underlying dimension of ‘ability’ which allows us to make meaningful comparisons between subjects as diverse as, for example, French and Physics. The methods are described in more detail in the ‘Methods of Analysis’ section.

Although statistical approaches are valuable, there are limits to the extent to which strong conclusions may be drawn without considering other evidence. For example, it would be inappropriate to conclude whether or not specific subjects can be said to be graded more harshly than others on the basis of statistical evidence alone. This is because differences in statistical difficulty between subjects can be caused by many factors, which can vary substantially between subjects and over time but were not considered in the methods used to derive the difficulty measures. These factors, among others, could include: the nature of the subject in terms of skills and knowledge to be learnt, the performance standards required to achieve individual grades, the level of demand, allocation of teaching time and other resources, motivation of students, efficiency and effectiveness of teaching and learning, and uptake by different population subgroups. For a more detailed discussion of this topic, you may wish to read some of Ofqual’s previous work on inter-subject comparability (Ofqual, 2015).

Two strands of analysis were conducted and reported here, which were:

• empirical analysis of the relationships between subject grade outcomes and prior attainment for prior attainment matched candidates
• Rasch modelling of the relationships between grade outcomes in different subjects for all candidates

The main findings from this study are:

1. Both GCSE and A level subjects were generally graded less leniently in 2022 than in 2021 and 2020 but more leniently than in 2019 at the overall subject level and at individual grades. This reflects the overall outcomes for 2022, 2021 and 2020 reported elsewhere (Ofqual, 2020, 2021a, 2022b, 2022c) and the impact of the grading policies implemented for summer 2022 relative to the other years (Ofqual, 2021b).

2. For both GCSE and A level subjects, the ranges of statistical grade difficulties across subjects in 2022 are generally larger than those in 2020 or 2021 but smaller than those in 2019. The standard deviation of relative Rasch grade difficulties in 2022 is generally slightly larger than in 2021 or 2020 but slightly smaller than (or similar to) in 2019.

3. The rank order of statistical grade difficulty changed across years:

   • For GCSE subjects, the rank orders are more similar between 2022 and 2019 than they are between 2021 and 2019 or between 2020 and 2019. The exception to this pattern is for grade 9, where the difference between 2022 and 2019 is larger than between 2020 and 2019. Changes in statistical difficulty orders between 2022 and 2019 or between 2022 and 2021 are larger than changes in difficulty orders between years of normal exams. The exception is at 9/A*, where changes between 2022 and 2021 are similar.
   • For A level subjects, the rank orders are more similar between 2022 and 2019 than they are between 2021 and 2019 or between 2020 and 2019. At the overall subject level and at C, changes in statistical difficulty orders between 2022 and 2019 are similar to changes between years of normal exams, whereas changes between 2022 and 2021 are larger. At A, changes in statistical difficulty orders between 2022 and 2019 or between 2022 and 2021 are larger than changes between years of normal exams. At A*, changes between 2022 and 2019 are considerably larger than changes between years of normal exams, whereas changes between 2022 and 2021 are considerably smaller.

4. In combination, the analysis of the distributions of statistical difficulties and changes to rank order positions suggest that the grading approach in 2022 has resulted in relationships between subjects that are more similar to those from 2019 compared with grades awarded by CAGs in 2020 and TAGs in 2021.

As with 2021, the different assessment and grading approaches between years (CAGs in 2020, TAGs in 2021, and external exams in 2022 and prior to the pandemic) and the grading policies implemented in 2022 have resulted in changes in the inter-subject comparability of both GCSEs and A levels. This technical report provides information about how the relative difficulty of subjects, as defined statistically, has differed between these years.

1. Introduction

In summer 2022, GCSE and A level exams were resumed, having been cancelled for the previous two summers (2020 and 2021) due to the coronavirus (COVID-19) pandemic (Ofqual, 2021b, 2022a). Exam boards provided advanced information about the exams to schools to help students revise before the exams. Further, it was decided that both GCSE and A level national results in summer 2022 were to be lower than in 2021 but higher than in 2019 (Ofqual, 2021b), as a stage in the trajectory back to normal outcomes in 2023.

He and Black (2020) investigated the impact of the different grades (calculated grades, Centre Assessment Grades – CAGs, and the final awarded grades) in 2020 on inter-subject comparability in GCSEs and A levels using a number of statistical difficulty measures. Similarly, He and Cadwallader (2022) explored inter-subject comparability over a period of 3 years using similar measures: summer 2021, when Teacher Assessed Grades (TAGs) were used, summer 2020, when for most students CAGs were used, and summer 2019, when grades were awarded through exams in the normal way.

This study represents a follow-up to the analyses undertaken by He and Black (2020) and He and Cadwallader (2022) and explores inter-subject comparability over a period of 4 years: summer 2022, summer 2021, summer 2020, summer 2019. This study also forms part of a set of research reports evaluating the impact of the 2022 summer awarding. The analysis aims to answer the following main research question:

How did the 2022 grades awarded through normal exams (with adaptations) impact on the statistical inter-subject comparability in GCSE and A level in relation to grades awarded in 2021, 2020 and 2019?

2. Data and analysis

2.1 Data

The primary data analysed in this investigation was the results data supplied to Ofqual by exam boards (EBs), covering 2019 to 2022 for both GCSEs and A levels. Prior attainment was based on normalised mean Key Stage 2 (KS2) scores for GCSEs, extracted from the National Pupil Database (NPD), and normalised mean GCSE scores for A level candidates, calculated from their GCSE grades awarded 2 years previously. Overall, the match rate with prior attainment data ranges from 74% to 84% of candidates for GCSE subjects and from 89% to 91% of candidates for A level subjects.

Tables 1 and 2 list the GCSE and A level subjects from 2019 to 2022 that were included in the analysis (Note: For results to be more reliable, only subjects with entries greater than 3,000 for GCSEs and 1,000 for A levels were included in the analysis in a particular year).

Table 1

GCSE subjects analysed in this report (including acronyms for some subjects used in subsequent figures and tables)

Subject	2019	2020	2021	2022
Art and Design: Three-dimensional design (AD_3DStudies)	X	X	X	X
Art and Design: Art, craft and design) (AD_ACD)	X	X	X	X
Art and Design: Fine art (AD_FA)	X	X	X	X
Art and Design: Graphics (AD_Graphics)	X	X	X	X
Art and Design: Photography (AD_Photog)	X	X	X	X
Art and Design: Textile design (AD_Textiles)	X	X	X	X
Arabic	X	-	X	X
Biology	X	X	X	X
Business studies (Business)	X	X	X	X
Combined science (Cbd_Science)	X	X	X	X
Chemistry	X	X	X	X
Chinese	X	-	X	X
Citizenship studies (Citi_Studies)	X	X	X	X
Classical civilisation (Class_Civil)	X	X	X	X
Computer Science (Computing)	X	X	X	X
Design and technology (D&T)	X	X	X	X
Dance	X	X	X	X
Drama	X	X	X	X
Economics	X	X	X	X
English language (Eng_Lang)	X	X	X	X
English literature (Eng_Lit)	X	X	X	X
Film_Studies (Film studies)	X	X	X	X
Food preparation and nutrition (Food_P&N)	X	X	X	X
French	X	X	X	X
Geography	X	X	X	X
German	X	X	X	X
History	X	X	X	X
Italian	X	X	X	X
Latin	X	X	X	X
Mathematics	X	X	X	X
Media studies (Media_Studies)	X	X	X	X
Music	X	X	X	X
Physical education (PE)	X	X	X	X
Physics	X	X	X	X
Polish	X	-	-	X
Psychology	X	X	X	X
Religious studies (RS)	X	X	X	X
Religious studies: short course (RS_SC)	X	X	-	-
Sociology	X	X	X	X
Spanish	X	X	X	X
Statistics	X	X	X	X
Urdu	X	X	X	X
Total	42	39	40	41

Table 2

A level subjects analysed (including acronyms for some subjects used in subsequent figures and tables)

Subject	2019	2020	2021	2022
Accounting	X	X	X	X
Art and Design: Three-dimensional design (AD_3DS)	X	X	X	X
Art and Design: Art, craft and design (AD_ACD)	X	X	X	X
Art and Design: Fine art (AD_FA)	X	X	X	X
Art and Design: Graphics (AD_Graphics)	X	X	X	X
Art and Design: Photography (AD_Photog)	X	X	X	X
Art and Design: Textile design (AD_Textiles)	X	X	X	X
Biology	X	X	X	X
Business studies (Bus_Studies)	X	X	X	X
Chemistry	X	X	X	X
Chinese	X	X	X	X
Classical civilisation (Class_Civil)	X	X	X	X
Computer science (Computing)	X	X	X	X
Dance	X	X	X	X
Drama and theatre studies (Drama_TS)	X	X	X	X
Design and Technology: Product design (DT_PD)	X	X	X	X
Economics	X	X	X	X
English language (Eng_Lang)	X	X	X	X
English language and literature (Eng_LangLit)	X	X	X	X
English literature (Eng_Lit)	X	X	X	X
Environmental studies (Env_Studies)	-	-	X	X
Film studies (Film_Studies)	X	X	X	X
French	X	X	X	X
Further mathematics (Fur_Maths)	X	X	X	X
Geography	X	X	X	X
Geology	X	-	-	X
German	X	X	X	X
History	X	X	X	X
Latin	X	-	X	X
Law	X	X	X	X
Mathematics	X	X	X	X
Media studies (Media_Studies)	X	X	X	X
Music	X	X	X	X
Music technology (Music_Tech)	X	X	X	-
Physical education (PE)	X	X	X	X
Philosophy	X	X	X	X
Physics	X	X	X	X
Polish	X	-	-	-
Politics	X	X	X	X
Psychology	X	X	X	X
Religious studies (RS)	X	X	X	X
Sociology	X	X	X	X
Spanish	X	X	X	X
Turkish	X	-	-	-
Total	43	39	41	41

2.2 Methods of analysis

Methods used for analysing the data were presented in detail in the report by He and Black (2020) and He and Cadwallader (2022). In brief, two approaches were used in the analysis. These were:

• variability in the relationship between subject grade outcomes and prior attainment, which was used as a proxy for ability (based on prior attainment-matched candidates)
• variability in the relationship in grade outcomes between subjects through mathematical modelling using the Rasch model (based on all candidates)

Analysis based on prior attainment-matched candidates

In the first approach, a prior attainment weighted mean grade (WMG) for a subject was calculated for each year. This involved classifying all prior attainment-matched candidates in a specific year into one of 10 prior attainment bands based on their mean KS2 scores (for GCSEs) or mean GCSE scores (for A levels). Each prior attainment score band has a similar number of candidates.

For candidates taking a specific subject, the average grade in the subject for those falling into each score band was calculated. The mean of these average grades across the 10 score bands are the weighted mean grade for the subject. This weighted mean grade represents the expected average grade from all prior attainment-matched candidates, were the subject to be taken by all matched candidates from the population (in other words, if it had been taken by the entire cohort of GCSE or A level students with valid prior attainment measures, not just those who actually took it).

WMG is used as a measure of overall difficulty of the subject. Subjects with high WMG values (higher grade outcomes) are said to be ‘easier’ than subjects with low values (lower grade outcomes). The difficulty (or facility) of a subject defined this way is independent of the ability distribution of the candidates taking the subject and can be compared over time and between subjects. Since normalised KS2 scores and GCSE scores were used, WMG values can be compared between different years.

Analysis based on Rasch modelling of all candidates

In Rasch modelling, for each grade in a subject (except for the one used as the reference grade – grade 1 for GCSEs and U for A levels^{[footnote 1]}, a difficulty measure is estimated^{[footnote 2]}. Assuming that the ability distributions of the candidates included in the analysis are similar across the years of analysis, the grade difficulty measures will be on the same measurement scale and can be compared directly when the average ability of candidates is set to zero in each year and the unit of logits is used.

The difficulty of a subject at a specific grade in a specific year is related to the average Rasch ability of the candidates achieving that grade. Subject 1 is said to be ‘more difficult’ (or ‘harder’) than Subject 2 at grade X if the average ability of the candidates receiving X in Subject 1 is higher than that of the candidates in Subject 2. In other words, candidates with similar Rasch abilities would have achieved lower grades in Subject 1 than in Subject 2. Similarly, a subject is more ‘difficult’ in year Y1 than year Y2 at grade X if the average ability of the candidates receiving X in Y1 is higher than that of the candidates in Y2. The mean of the difficulties from grade 3 to grade 8 for a GCSE subject and from D to A for an A level subject is used as the overall measure of statistical difficulty for the subject. The use of these particular grades ensures a more stable estimate of the overall difficulty for the subject.

Limitations

As with our previous reports, care needs to be taken when interpreting the statistical measures of subject difficulty reported in this paper. The difficulty measures (overall or at specific grades) are not direct measures of the performance standards required to achieve the grades in a subject. They are also not direct measures of subject demand or students’ efforts. Differences in statistical difficulty between subjects reflect differences in grade outcomes between subjects for candidates with similar levels of prior attainment or ability derived from the Rasch model. Such differences can be caused by many factors, which can vary substantially between subjects and over time but were not considered in the methods used for this study. These factors, among others, could include:

• the nature of the subject in terms of skills and knowledge to be learnt
• the level of demand
• the performance standards required to achieve individual grades
• allocation of teaching time and other resources
• motivation of students
• efficiency and effectiveness of teaching and learning
• uptake by different population subgroups
• relative leniency or severity in grading

There are also limitations associated with the use of the Rasch model, including violation of the unidimensionality assumption (that is, the exams are assumed to measure a single ability in common), unrepresentativeness of the samples analysed, missing data, and imperfect data-model fit. All of this must be kept in mind when reading this report and interpreting the results.

As with the 2020 and 2021 reports (see He and Black, 2020; He and Cadwallader, 2022), the focus of this analysis is on comparability at grades 4, 7 and 9 between GCSE subjects and grades C, A and A* between A level subjects.

3. Results and discussion

This section discusses the main results from the analysis. Results for GCSEs are presented first, followed by results for A levels.

3.1 GCSE Subjects

3.1.1 Relationship with prior attainment at KS2

Figure 1 shows the relationship between mean GCSE grade and normalised KS2 score for candidates with a valid KS2 score from 2019 to 2022. To a large extent, differences in intercept between years reflect differences in overall grade outcomes (or difficulty) for candidates with similar prior attainment at KS2. This shows that subjects in 2022 are statistically more difficult than in 2020 and 2021 but are still statistically easier than in 2019.

Figure 1

Relationship between candidates’ mean GCSE grade and normalised KS2 score in 2019 (top-left), 2020 (top-right), 2021 (bottom-left), and 2022 (bottom-right).

Figure 2 compares weighted mean grade (WMG) across the GCSE subjects from 2019 to 2022. In Figure 2, subjects are arranged in ascending order of difficulty in 2019 based on their weighted mean grades. That is, subjects to the right are more difficult, based on this measure, than those to the left in 2019. The WMG values from 2020 to 2022 for all the subjects are considerably higher than in 2019, reflecting the increase in outcomes in these years, relative to 2019 (see Ofqual, 2020, 2021a, 2022b). The WMG values for 2022 are noticeably lower than those in 2020 and 2021, reflecting the different grading policies implemented in summer 2022 (Ofqual, 2021b). While the general trend in WMG from 2020 to 2022 follows that in 2019, substantial variability also exists between the subjects in terms of changes in WMG from 2019. Further, the variability for 2022 is smaller than for 2020 and 2021.

Figure 2

Comparison of weighted mean grades between GCSE subjects from 2019-2022. Subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019.

The top graph in Figure 3 shows changes in WMGs between pairs of years for individual subjects and the bottom graph shows the distributions of changes across subjects. In the top graph, subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019 (see also Table A1 in Appendix A). Positive values indicate a decrease in apparent statistical subject difficulty whereas negative values indicate an increase. For example, GCSE economics has a value of -0.5 when comparing 2022 and 2021, indicating that there has been an increase in apparent subject difficulty in 2022 compared to 2021. Of particular interest are the changes in subject WMGs between 2022 and 2019 and between 2022 and 2021. These changes allow us to compare subject WMGs from before the pandemic (2019) with 2022. Similarly, it is interesting to compare WMGs from 2021, when TAGs were awarded, and 2022, when exams resumed.

Figure 3

Comparison of changes in population weighted mean grade between GCSE subjects from 2019-2022. In the top graph, subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019.

Differences in WMG between 2022 and 2019 vary from 0.097 for Polish to 0.767 for computing, with a mean of 0.459 (over two-fifths of a grade). These differences are considerably smaller than the differences between 2021 or 2020 and 2019. Differences in WMG between 2022 and 2021 vary from -0.633 for classical civilization to 0.010 for graphics, with a mean of -0.219 (over one-fifth of a grade). Classical civilization, Latin, economics and computing all have WMGs in 2022 that are more than two-fifths of a grade lower than in 2021, whereas subjects such as photography, fine art and mathematics have their WMGs slightly lower than in 2021.

3.1.2 Rasch modelling

This section discusses how subject difficulties changed between years at individual grades and the overall subject level through Rasch modelling of all candidates included in the analysis.

Relative Rasch grade difficulty distribution

The graphs in Figure 4 compare statistical subject mean difficulty and statistical difficulties at grades 4, 7 and 9 between GCSE subjects across the 4 years of analysis. In each of the graphs, subjects are arranged in ascending order of statistical difficulty in 2019 (that is, in 2019 subjects on the right were more difficult, based on these measures, than those on the left).

Figure 4

Comparison of mean difficulty (top-left) and difficulties at grades 4 (top-right), 7 (bottom-left) and 9 (bottom-right) for GCSE subjects from 2019-2022 estimated using the Rasch model. Subjects are arranged in ascending order of difficulty in 2019.

Based on the Rasch difficulty measures, compared with 2019, subjects were more leniently graded from 2020 to 2022, both overall and at individual grades, with the exception of chemistry, physics and English literature at grade 9 where the statistical difficulties in 2020 were very slightly higher. At the overall subject level and at grade 4, subjects in 2022 are more statistically difficult than in 2020 and 2021. At grades 7 and 9, most of the subjects in 2022 are more statistically difficult than in 2021 but are still statistically easier than in 2020.

To make the comparison more meaningful, the unit of difficulty in logits was converted to a unit of grade width (GW) by dividing the original difficulty value by the average grade gap in logits between grade 8 and grade 3 (the average grade gap in logits between grade 8 and grade 3 is calculated as the difference in logits between average difficulty at grade 8 and that at grade 3 divided by 5, which is equivalent to one grade width). Table 3 lists the range and standard deviation of grade difficulties in GW. Note that, the smaller the range and standard deviation of difficulties, the closer the alignment in terms of grading difficulty between the subjects. Figure 5 further illustrates the distributions of the relative statistical subject mean difficulty and difficulties at grades 4, 7 and 9. When producing these plots, the original statistical difficulty of a subject at a grade was subtracted from the mean difficulty of all subjects at that grade for easy comparison.

Table 3

Standard deviation and range of statistical grade difficulties (in GW) for GCSE subjects from 2019-2022.

Standard deviation (GW)

Year	Mean	G4	G7	G9
2019	0.563	0.672	0.495	0.560
2020	0.471	0.601	0.391	0.413
2021	0.446	0.565	0.413	0.489
2022	0.542	0.680	0.469	0.508

Range (GW)

Year	Mean	G4	G7	G9
2019	2.436	2.724	2.742	3.150
2020	1.744	2.345	1.894	2.131
2021	1.790	2.416	2.205	2.612
2022	2.225	3.126	2.359	2.372

Figure 5

Comparison of distributions of average difficulty (top-left) and difficulties at grades 4 (top-right), 7 (bottom-left) and 9 (bottom-right) between GCSE subjects from 2019-2022.

Table 3 indicates that the standard deviation of relative grade difficulties (in GW unit) between the subjects in 2022 is generally slightly larger than that in 2021 or 2020 but slightly smaller than or similar to the standard deviation in 2019 at the overall subject level and at grades 4, 7 and 9. At the overall subject level and at grades 4 and 7, the ranges of statistical grade difficulties in 2022 are larger than in 2020 and 2021. At grade 9, the range in 2022 is larger than in 2020 but smaller than in 2021. Compared with 2019, the range of statistical grade difficulties in 2022 are smaller at the overall subject level and at grades 7 and 9 but slightly larger at grade 4.

Change in statistical grade difficulty

To examine how statistical subject grade difficulties have changed between 2022 and 2019, and between 2022 and 2021, Figure 6 shows distributions of changes in statistical difficulties at grades 4, 7, 9 and statistical subject mean difficulty between years for the GCSE subjects analysed. In the top graphs in Figure 6, subjects are arranged in ascending order of mean difficulty in 2019. Positive values indicate an increase in apparent difficulty whereas negative values indicate a decrease.

Figure 6

Changes in Rasch grade difficulties for GCSE subjects from 2019 to 2022 (left) and from 2021 to 2022 (right). In the top graphs, subjects are arranged in ascending order of mean difficulty in 2019.

Changes in difficulty between 2022 and 2019 are -0.515 logits (subject mean), -0.560 logits (grade 4), -0.477 logits (grade 7) and -0.458 logits (grade 9). The average grade gap is about 0.993 logits in 2022, and this suggests that on average subjects in 2022 are about half of a grade more leniently graded than in 2019 at the overall subject level (this is also true at grade 4). Subjects were slightly less than half of a grade more leniently graded at grades 7 and 9. In general, Italian, film studies and combined science are among the subjects with the smallest changes in statistical difficulty between 2022 and 2019, whereas many art and design subjects, classical civilization, economics, music and German are among the subjects with the largest changes.

Subjects in 2022 are more statistically difficult than in 2021 by about slightly less than a quarter of a grade (overall subject level), one-third of a grade (grade 4) and one-seventh of a grade (grade 7). At grade 9, they are only marginally more difficult (less than one-twentieth of a grade) than in 2021. Photography, combined science, English literature and Spanish are among the subjects with smallest increases in statistical difficulty from 2021 to 2022, whereas classical civilization, computing, economics, dance, history and PE are among the subjects with largest increases.

Changes in statistical grade difficulty rank orders

Subjects were rank ordered based on mean statistical difficulty and difficulties at grades 4, 7 and 9 separately from 2019 to 2022, with the lowest rank (first) representing the statistically easiest subject and highest rank the most statistically difficult subject. Figure 7 shows the distribution of changes in statistical subject difficulty rank order positions between years (also see Table A2 in Appendix A). Positive values indicate increase in statistical difficulty order whereas negative values indicate decrease in difficulty order.

Figure 7

Distributions of changes in grade difficulty rank order positions in GCSE subjects between 2020 to 2022 and 2019 and between 2022 and 2021.

At the overall subject level and at grades 4 and 7, changes in statistical grade difficulty orders between 2022 and 2019 are smaller than changes between 2021 or 2020 and 2019. This means that the statistical grade difficulty orders in 2022 and 2019 were more similar than when comparing 2019 with 2021 or 2020, when CAGs/TAGs were awarded. At grade 9, changes in statistical difficulty order between 2022 and 2019 are smaller than changes between 2021 and 2019 but larger than the changes between 2020 and 2019. Changes in rank order positions between 2022 and 2021 are similar to changes between 2022 and 2019 at the overall subject level and at grades 4 and 7 but are smaller at grade 9.

Figure 8 shows detailed changes in statistical difficulty order positions between 2020, 2021 and 2022 compared with 2019 and between 2022 and 2021 for individual subjects (also see Table A2 in Appendix A). In Figure 8, subjects are arranged in ascending order of statistical grade difficulty in 2019. Again, positive values indicate increase in rank order position (becoming more statistically difficult ) whereas negative values indicate decrease in statistical difficulty rank order position. Changes in rank order statistical difficulty positions between years at grades 7 and 9 are larger than changes at grade 4 and at the overall subject level.

Figure 8

Distributions of change in difficulty rank order positions overall and at individual grades for GCSE subjects from 2019-2022. Subjects are arranged in ascending order of difficulty in 2019.

Between 2022 and 2019, at the overall subject level, only music and English literature changed their statistical difficulty position by more than 5 positions (7 and -7, respectively). At grade 4, only classical civilization and physics changed their rank order position by more than 5 (6 and -6, respectively). At grade 7, music, dance, business studies, sociology and mathematics changed their statistical difficulty order position by more than 5. At grade 9, some of the art and design subjects, combined science, physics and English literature increased their statistical difficulty position by more than 5, whereas business studies, drama, PE, music, psychology and sociology decreased their statistical difficulty order position by more than 5.

Between 2022 and 2021, at the overall subject level, only three subjects changed their rank order positions by more than 5. These were: history (6 positions), music (7 positions) and physics (-6 positions). At grade 4, only dance and physics changed their rank order positions by more than 5 (6 and -6, respectively). At grade 7, graphics, 3D studies, drama, music and classical civilization show more than 5 changes in statistical difficulty order positions. At grade 9, art, fine art, graphics, French and classical civilization changed their statistical difficulty order by more than 5.

Changes of this kind are very common, even between years in which normal examinations take place under normal conditions. To see how changes in statistical difficulty orders between 2022 and 2019/2021 compare with changes between years of normal exams, Table 4 lists values of absolute average change and standard deviation of changes of statistical difficulty orders at the overall subject level and at 4/C, 7/A and 9/A* between 2022 and 2019/2021 and between 2011, 2012 and 2013, when normal exams took place. (Note, grades A, A and C were used as proxies for grades 9, 7 and 4 for the reformed GCSEs, which were introduced in phases from 2015. We recognise that A is not an ideal proxy for grade 9, since they are not comparable in terms of performance standards. See He and Cadwallader, 2022.

Table 4

Absolute change per subject in statistical grade difficulty rank order and standard deviation of changes in statistical difficulty orders for GCSE subjects.

Grade	Years	Change in rank order (average change)	Change in rank order (standard deviation)
Mean	2012 and 2011	1.600	2.330
Mean	2013 and 2012	1.714	2.438
Mean	2013 and 2011	1.943	2.461
Mean	2022 and 2019	2.263	2.965
Mean	2022 and 2021	2.000	2.646
C or 4	2012 and 2011	1.086	1.586
C or 4	2013 and 2012	1.543	2.402
C or 4	2013 and 2011	2.057	2.859
C or 4	2022 and 2019	2.000	2.675
C or 4	2022 and 2021	2.474	2.982
A or 7	2012 and 2011	1.657	2.541
A or 7	2013 and 2012	1.771	2.849
A or 7	2013 and 2011	1.943	2.662
A or 7	2022 and 2019	2.789	3.678
A or 7	2022 and 2021	2.421	3.387
A* or 9	2012 and 2011	2.743	4.113
A* or 9	2013 and 2012	2.057	2.839
A* or 9	2013 and 2011	3.029	4.641
A* or 9	2022 and 2019	5.158	6.481
A* or 9	2022 and 2021	2.632	3.811

Changes in statistical difficulty orders between 2022 and 2019/2021 are larger than changes between years of normal exams at the overall subject level and at grades 4/C and 7/A. At 9/A*, changes between 2022 and 2019 are larger than changes between years of normal exams, whereas changes between 2022 and 2021 are broadly similar. While it is true that the time span between 2022 and 2019 is three years and that slightly larger changes in subject rank order would be expected, compared with the expected changes associated with a time span of one or two years of normal exams, the observed changes between 2022 and 2019 at grade 9 would still likely be substantially larger than the changes expected had exams taken place in both 2020 and 2021. These changes in statistical subject difficulty orders between years to a certain extent reflect the different approaches to assessment and grading and the grading policy applied in 2022.

3.2 A level Subjects

This section discusses results from analysis of the A level subjects.

3.2.1 Relationship with prior attainment at GCSE

Figure 9 shows the relationship between mean A level grade and normalised mean GCSE score for candidates with valid mean GCSE scores from 2019 to 2022. The two measures are reasonably highly correlated, although the correlation is not as high as that between mean GCSE grade and KS2 score. While values of slope of the regression lines are similar over the 4 years, the intercepts are different, again indicating differences in grade outcomes between the years after taking into account candidates’ prior attainment at GCSE. Although subjects in 2022 are more statistically difficult than in 2020 and 2021, they are still considerably statistically easier than in 2019.

Figure 9

Relationship between candidates’ mean A level grade and normalised mean GCSE score in 2019 (top-left), 2020 (top-right), 2021 (bottom-left), and 2022 (bottom-right).

As with GCSEs, a prior attainment (represented by mean GCSE score) weighted mean grade (WMG) for each subject was also calculated for each year. Figure 10 compares WMG across the A level subjects from 2019 to 2022. In Figure 10, subjects are arranged in ascending order of difficulty in 2019 based on their weighted mean grades. Similar to GCSEs, the WMG values in 2022 are noticeably lower than in 2020 and 2021 but are still considerably higher than in 2019. Again, this reflects the increase in outcomes in these years, relative to 2019 (see Ofqual, 2020, 2021a, 2022c) and the impact of the grading policies implemented in summer 2022 (Ofqual, 2021b). Of the subjects investigated, compared with 2019, Chinese was the most leniently graded subject in 2020, 2021 and 2022, based on these statistical measures. It is stressed that, once again, the methods used to derive the different subject difficulty measures have important limitations. It would be inappropriate to use the findings from this analysis in isolation from other evidence when deciding, whether or not specific subjects can justifiably be said to be graded more harshly or leniently than others. For a more detailed discussion of this topic, you may wish to read some of Ofqual’s previous work on inter-subject comparability (Ofqual, 2015).

Figure 10

Comparison of weighted mean grades in A level subjects from 2019 to 2022. Subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019.

Figure 11 compares changes in subject WMG between pairs of years. In Figure 11, subjects are arranged in descending order of weighted mean grade in 2019 (see also Table A3 in Appendix A). Positive values indicate a decrease in apparent subject difficulty whereas negative values indicate an increase.

Figure 11

Comparison of changes in population weighted mean grade in A level subjects from 2019-2022. Subjects are arranged in ascending order of difficulty based on their weighted mean grade in 2019 in the top graph.

Difference in WMG between 2022 and 2019 are noticeably smaller than between 2020 or 2021 and 2019. Differences in WMG between 2022 and 2019 vary from 0.14 for mathematics to 0.60 for music, with a mean of 0.40 (two-fifths of a grade). Between 2022 and 2021, differences in subject WMG vary from -0.76 for Chinese to -0.11 for graphics, with a mean of -0.34 (about one-third of a grade). Further maths, music, philosophy, computing, chemistry and biology have their WMGs reduced by more than 0.50 from 2021 to 2022 whereas some of the art and design subjects, history, geography, economics, film studies and business studies are among subjects with their WMGs reduced by less than 0.25.

3.2.2 Rasch modelling

Relative Rasch grade difficulty distribution

Figure 12 compares subject mean difficulty and difficulties at C, A and A* between the A level subjects from 2019 to 2022 analysed using the Rasch model. In each of the graphs, subjects are arranged in ascending order of difficulty in 2019. At the overall subject level and at grade C, subjects were less leniently graded in 2022 than in 2021 and 2020 but were more leniently graded than in 2019. At grade A, the majority of the subjects were less leniently graded in 2022 than in 2021 and 2020. At A*, subjects were less leniently graded in 2022 than in 2021 but were more leniently graded than in 2020 and 2019. Again, Chinese was the most leniently graded subject in 2020, 2021, and 2022, based on Rasch difficulty measures. As stressed before, it would be inappropriate to use the outcomes of this statistical analysis in isolation from other evidence when deciding, for example, whether or not specific subjects can justifiably be said to be graded more leniently than others.

Figure 12

Comparison of subject mean difficulty (top-left) and difficulties at C (top-right), A (bottom-left) and A* (bottom-right) for A level subjects from 2019 to 2022 estimated using the Rasch model. Subjects are arranged in order of difficulty in 2019.

Table 5 lists the range and standard deviation of relative statistical grade difficulties of the subjects in grade width (GW). For A levels, the average grade gap in logits between grades A and D is equivalent to one grade width (GW). Given the behaviour of Chinese in these analyses, it was excluded from the summary analyses presented in the table. Figure 13 illustrates the distributions of relative statistical subject mean difficulty and difficulties at C, A and A*. Again, the original statistical difficulty of a subject at a grade was subtracted from the mean statistical difficulty of all subjects at that grade when producing Figure 13.

Table 5

Standard deviation and range of statistical grade difficulties (in GW) for A level subjects from 2019-2022 (excluding Chinese).

Standard Deviation

Year	Mean	C	A	A*
2019	0.627	0.693	0.510	0.738
2020	0.583	0.632	0.460	0.554
2021	0.566	0.633	0.448	0.542
2022	0.651	0.755	0.462	0.612

Range

Year	Mean	C	A	A*
2019	2.625	2.845	2.123	2.681
2020	2.503	2.569	1.889	2.140
2021	2.348	2.607	1.781	2.203
2022	2.526	2.901	1.835	2.430

Figure 13

Comparison of distributions of subject mean statistical difficulty (top-left) and difficulties at C (top-right), A (bottom-left) and A* (bottom-right) between A level subjects from 2019-2022.

The ranges and standard deviations of relative statistical grade difficulties (in GW) in 2022 are larger than those in 2022 or 2021 but similar to or slightly smaller than those in 2019 at the overall subject level and at individual grades.

Change in statistical grade difficulty

Figure 14 shows distributions of changes in statistical subject mean difficulty and statistical difficulties at C, A and A* between 2022 and 2021 and between 2022 and 2019. In Figure 14, subjects are arranged in ascending order of mean statistical difficulty in 2019. Positive values indicate an increase in apparent difficulty whereas negative values indicate a decrease.

Figure 14

Changes in Rasch grade difficulties for A level subjects from 2019 to 2022 (left) and from 2021 to 2022 (right). In the top graphs, subjects are arranged in ascending order of mean difficulty in 2019.

Subjects in 2022 were slightly over three-tenths of a grade more leniently graded than in 2019 at the overall subject level and at grades C and A. At A, they were graded about two-fifths of grade more leniently. Compared with 2019, Chinese, dance and textiles are most leniently graded at C or A in 2022 based on these measures. Drama, product design, accounting, music, philosophy and computing are most leniently graded at A. Law, mathematics and chemistry are least leniently graded in 2022 in comparison with 2019 from these analyses.

Compared with 2021, subjects in 2022 were graded about a third of a grade less leniently overall and at grades C and A. At A*, they were graded about a quarter of a grade less leniently. Chinese, philosophy, Spanish, French, German and music are among the subjects with the largest increases in statistical difficulty in 2022, while some of the art and design subjects, history, psychology and sociology are among the subjects with the smallest increases in statistical difficulty.

Changes in statistical grade difficulty rank orders

As with GCSEs, subjects were rank ordered based on mean statistical difficulty and difficulties at C, A and A* separately from 2019 to 2022. Figure 15 shows distributions of changes in statistical subject difficulty rank order positions between 2020 to 2022 and 2019 and between 2022 and 2021 (also see Table A4 in Appendix A).

Figure 15

Distributions of changes in statistical grade difficulty rank order positions in A level subjects between 2020 to 2022 and 2019 and between 2022 and 2021.

Changes in statistical difficulty orders overall and at individual grades between 2022 and 2019 are generally smaller than changes between 2020 or 2021 and 2019. At the overall subject level and at grade C, changes in rank order between 2022 and 2021 are slightly larger than changes between 2022 and 2019, whereas they are slightly smaller at A and A*.

Figure 16 shows detailed changes in statistical grade difficulty orders between 2020, 2021 and 2022 compared with 2019 and between 2022 and 2021 for individual subjects (also see Table A4 in Appendix A). In Figure 16, subjects are arranged in ascending order of statistical grade difficulty in 2019.

Figure 16

Distributions of changes in statistical difficulty rank order positions overall and at individual grades for A level subjects from 2019 to 2022. Subjects are arranged in ascending order of mean statistical difficulty in 2019.

Between 2022 and 2019, at the overall subject level and at grade C, no subjects changed statistical difficulty order position by more than 5. At grade A, three subjects changed their difficult position by more than 5: Chinese (-8), classical civilization (-9) and music (-10). At A*, five subjects changed their position by more than 5: accounting (-13), Chinese (-32), law (6), maths (9), and music (-9).

Between 2022 and 2021, at the overall subject level, three subjects changed their rank order positions by more than 5: German (7), music (12) and Spanish (9). At grade C, music, psychology and Spanish changed their rank order positions by more than 5. At grade A, PE and philosophy had their rank order increased by 7 and 6 respectively. At A*, changes in rank order positions are less than 5 for all subjects.

Table 6 lists values of absolute average change and standard deviation of changes of statistical difficulty orders at the overall subject level and at grades C, A and A* between years from 2017 to 2022 (also see He and Cadwallader, 2022).

Table 6

Average absolute change per subject in statistical grade difficulty rank order and standard deviation of changes in statistical difficulty orders between years for A level subjects.

Grade	Years	Change in rank order (average change)	Change in rank order (standard deviation)
Mean	2018 and 2017	0.900	1.304
Mean	2019 and 2018	1.300	2.049
Mean	2019 and 2017	1.500	2.191
Mean	2022 and 2019	1.526	1.974
Mean	2022 and 2021	2.421	3.671
C	2018 and 2017	1.300	2.121
C	2019 and 2018	1.700	2.559
C	2019 and 2017	1.650	2.145
C	2022 and 2019	1.474	2.039
C	2022 and 2021	2.158	3.154
A	2018 and 2017	1.000	1.449
A	2019 and 2018	1.150	1.549
A	2019 and 2017	1.350	1.732
A	2022 and 2019	2.579	3.532
A	2022 and 2021	1.737	2.395
A*	2018 and 2017	1.750	2.429
A*	2019 and 2018	1.750	2.924
A*	2019 and 2017	2.050	3.500
A*	2022 and 2019	4.000	6.692
A*	2022 and 2021	1.105	1.762

At the overall subject level and at grade C, changes in statistical difficulty orders between 2022 and 2019 are similar to changes between years of normal exams, whereas changes between 2022 and 2021 are larger. At grade A, changes in statistical difficulty orders between 2022 and 2019 or 2021 are larger than changes between years of normal exams. At A, changes in statistical difficulty order between 2022 and 2019 are considerably larger than changes between years of normal exams, whereas changes between 2022 and 2021 are considerably smaller. Again, the observed changes in statistical difficulty order between 2022 and 2019 at A are likely to be larger than the changes expected had exams taken place in both 2020 and 2021. These changes in statistical subject difficulty orders between years, again, to a certain extent reflects the different approaches to assessment and grading across the years and the impact of the grading policies implemented in 2022.

4. Concluding remarks

In this report, the comparability of grading standards between subjects in summer 2022 was investigated using statistical measures of grade difficulty. The approach involves empirical analysis of the relationships between subject grade outcomes and prior attainment, and Rasch modelling of the relationships between grade outcomes in different subjects. These difficulty statistics were compared with those estimated for previous years, particularly the difficulty measures estimated for 2019 and 2021. This allowed us to examine the impact of different approaches to assessment and grading and the grading policies implemented in summer 2022 on inter-subject comparability over the years of investigation.

As with previous reports, the limitations of the statistical methods used in this study, which are discussed in the ‘Methods of Analysis’ section, must be borne in mind when considering the findings from this study. It would be inappropriate to draw strong conclusions about whether specific subjects are harshly or leniently graded, without considering other evidence (see, for example, Ofqual, 2015 for further discussion).

Based on relationships between subject grade outcomes and prior attainment for prior attainment-matched candidates, GCSE subjects on average were graded about one-fifth of a grade less leniently in 2022 than in 2021 but more than two-fifths of a grade more leniently than in 2019. There was, however, variation between subjects. For A levels, subjects in 2022 on average were graded about one-third of a grade less leniently than in 2021 but two-fifths of a grade more leniently than in 2019. Again, there was variation between subjects. The differences in subject grading severity or leniency between 2022 and 2019 or 2021 reflect the impact of the grading policies implemented in summer 2022.

Based on analysis using the Rasch model for all candidates taking the subjects analysed, for both GCSEs and A levels, subjects in 2022 were generally graded less leniently than in 2021 but more leniently than in 2019. For GCSEs, compared with 2019, on average, subjects in 2022 are about half of a grade more leniently graded at the overall subject level and at individual grades. Compared with 2021, in 2022, subjects on average are more statistically difficult. At the overall subject level, subjects are slightly less than a quarter of a grade more difficult statistically, while at grade 4 they are one-third of a grade more difficult, and at grade 7 one-seventh of a grade more difficult. At grade 9, subjects in 2022 are only marginally more difficult.

For A levels, subjects in 2022 were slightly more than three-tenths of a grade more leniently graded than in 2019 at the overall subject level and at grades C and A. At A, they were graded about two-fifths of grade more leniently. Compared with 2021, subjects in 2022 were graded about a third of a grade less leniently overall and at grades C and A. At A, they were graded about a quarter of a grade less leniently. For both GCSEs and A levels, at the overall subject level, results based on Rasch analysis are broadly similar to those based on relationships between subject grade outcomes and prior attainment.

For GCSE subjects, the standard deviation of relative statistical grade difficulties (in GW unit) in 2022 is generally slightly larger than that in 2021 or 2020 but similar to or slightly smaller than that in 2019 at the overall subject level and at individual grades. The ranges of statistical grade difficulties in 2022 are generally larger than those in 2020 or 2021 but smaller than those in 2019 at the overall subject level and at grades 7 and 9. For A level subjects, the ranges and standard deviations of relative statistical grade difficulties in 2022 are larger than those in 2022 or 2021 but similar to or slightly smaller than those in 2019 at the overall subject level and at individual grades. For both GCSEs and A levels, the distributions of relative statistical subject difficulties in 2022 are closer to the distributions in 2019 than the distributions in 2021 or 2020.

For GCSE subjects, changes in statistical grade difficulty orders are smaller between 2022 and 2019 than they are between 2021 and 2019 or between 2020 and 2019. The exception to this pattern is for grade 9, where the difference between 2022 and 2019 is larger than between 2020 and 2019. Changes in statistical difficulty orders between 2022 and 2019 or between 2022 and 2021 are larger than changes in statistical difficulty orders between years of normal exams. The exception is at 9/A*, where changes between 2022 and 2021 are similar.

For A level subjects, changes in statistical difficulty orders overall and at individual grades between 2022 and 2019 are generally smaller than changes between 2020 and 2019 or between 2021 and 2019. At the overall subject level and at grade C, changes in statistical difficulty orders between 2022 and 2019 are similar to changes between years of normal exams, whereas changes between 2022 and 2021 are larger. At grade A, changes in statistical difficulty orders between 2022 and 2019 or between 2022 and 2021 are larger than changes between years of normal exams. At A*, changes between 2022 and 2019 are considerably larger than changes between years of normal exams, whereas changes between 2022 and 2021 are considerably smaller.

In combination, the analysis of the distributions of statistical difficulties and changes to rank order positions suggest that the grading approach in 2022 has resulted in relationships between subjects that are more similar to those from 2019 compared with grades awarded by CAGs in 2020 and TAGs in 2021.

Different assessment and grading approaches over recent years and the grading policies implemented in 2022 are likely to have affected statistical inter-subject comparability in GCSEs and A levels differently over the years of investigation.

References

Coe, R. (2008). Comparability of GCSE examinations in different subjects: An application of the Rasch model. Oxford Review of Education, 34, 609–636.

He, Q. and Black, B. (2020). Impact of calculated grades, centre assessment grades and final grades on inter-subject comparability in GCSEs and A levels in 2020. Ofqual: Coventry, UK.

He, Q. and Cadwallader, S. (2022). An investigation of inter-subject comparability in GCSEs and A levels in summer 2021. Ofqual: Coventry, UK

Ofqual (2015). Comparability of Different GCSE and A Level Subjects in England: An Introduction. Ofqual: Coventry, UK.

Ofqual (2020). Summer 2020 results analysis – GCSE, AS and A level. Ofqual: Coventry, UK.

Ofqual (2021a). Summer 2021 results analysis and quality assurance. Ofqual: Coventry, UK.

Ofqual (2021b). Ofqual’s approach to grading exams and assessments in summer 2022 and autumn 2021. Ofqual: Coventry, UK.

Ofqual (2022a). Student guide to exams and formal assessments in 2021 to 2022. Ofqual: Coventry, UK.

Ofqual (2022b). Guide to GCSE results for England, summer 2022. Ofqual: Coventry, UK.

Ofqual (2022c). Guide to AS and A level results for England, summer 2022. Ofqual: Coventry, UK.

Appendix A - Additional tables

Table A1

Difference in weighted mean grade between 2020 to 2022 and 2019 and between 2022 and 2021 for GCSE subjects (see Table 1 for full titles of the subjects).

Subject	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021
Polish	-	-	0.097	-
Italian	0.320	0.513	0.313	-0.200
Urdu	0.614	0.479	0.250	-0.229
Chinese	-	0.819	0.591	-0.228
Arabic	-	0.786	0.266	-0.520
AD_Photog	0.485	0.349	0.316	-0.032
AD_FA	0.548	0.497	0.455	-0.041
AD_Textiles	0.630	0.544	0.458	-0.085
AD_ACD	0.540	0.501	0.404	-0.097
Physics	0.638	0.614	0.454	-0.160
Dance	0.901	0.930	0.614	-0.315
Biology	0.641	0.641	0.442	-0.199
Chemistry	0.646	0.567	0.393	-0.174
RS	0.433	0.453	0.328	-0.126
Latin	1.126	1.056	0.520	-0.535
Drama	0.748	0.727	0.495	-0.233
Food_P&N	0.602	0.575	0.401	-0.174
Eng_Lit	0.341	0.349	0.271	-0.078
AD_Graphics	0.674	0.623	0.613	-0.010
Media_Studies	0.625	0.604	0.414	-0.190
Film_Studies	0.650	0.525	0.363	-0.162
Music	0.897	0.924	0.607	-0.317
Eng_Lang	0.446	0.512	0.418	-0.094
PE	0.663	0.900	0.601	-0.299
Citi_Studies	0.555	0.588	0.447	-0.142
Mathematics	0.383	0.400	0.317	-0.084
Sociology	0.674	0.792	0.573	-0.219
D&T	0.723	0.684	0.452	-0.233
Geography	0.560	0.706	0.483	-0.223
Class_Civil	1.127	1.309	0.676	-0.633
History	0.569	0.768	0.485	-0.283
AD_3DStudies	0.616	0.499	0.407	-0.093
Business	0.699	0.836	0.584	-0.252
Spanish	0.617	0.670	0.488	-0.182
Economics	0.959	1.112	0.608	-0.504
Statistics	0.690	0.707	0.479	-0.227
French	0.621	0.636	0.492	-0.145
German	0.697	0.719	0.565	-0.154
Cbd_Science	0.383	0.410	0.276	-0.133
RS_SC	0.516	-	-	-
Psychology	0.930	0.996	0.637	-0.359
Computing	1.001	1.179	0.767	-0.412
Average	0.654	0.687	0.459	-0.219

Table A2

Differences in difficulty order overall and at grades 4, 7 and 9 between 2020 to 2022 and 2019 and between 2022 and 2021 for GCSE subjects.

Subject	Mean				Grade 4				Grade 7				Grade 9
	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021
AD_3DStudies	-1	1	-1	-2	-6	-5	-3	2	2	8	2	-6	6	13	8	-5
AD_ACD	2	2	1	-1	-2	-1	-2	-1	1	4	1	-3	4	10	4	-6
AD_FA	-1	-1	-2	-1	-1	-1	-2	-1	-1	0	-1	-1	4	12	3	-9
AD_Graphics	-2	-2	-5	-3	-6	-8	-8	0	0	9	0	-9	0	7	6	-1
AD_Photog	-1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	-1
AD_Textiles	-3	-3	-1	2	-3	-4	-2	2	0	4	0	-4	3	12	7	-5
Biology	0	0	-2	-2	-1	1	-4	-5	2	3	2	-1	7	10	10	0
Business	-3	-5	-2	3	-1	-1	1	2	-7	-7	-6	1	-9	-10	-10	0
Cbd_Science	6	7	5	-2	5	7	2	-5	4	5	4	-1	4	8	7	-1
Chemistry	1	5	2	-3	0	2	-2	-4	0	5	4	-1	11	14	12	-2
Citi_Studies	4	3	2	-1	4	4	4	0	4	4	1	-3	1	-2	-2	0
Class_Civil	-5	-6	-2	4	-7	-6	0	6	-4	-13	-5	8	-2	-19	-5	14
Computing	0	-2	-1	1	-1	-3	0	3	-2	-7	-3	4	-3	-5	-1	4
D&T	-7	-4	0	4	-8	-4	-3	1	-4	-3	0	3	-6	-5	-4	1
Dance	-4	-7	-4	3	-9	-8	-2	6	-7	-7	-7	0	-5	-6	-5	1
Drama	-5	-4	-4	0	-5	-2	0	2	-9	-11	-5	6	-8	-10	-10	0
Economics	-5	-4	-2	2	-4	-2	0	2	-2	-3	-1	2	-2	-4	1	5
Eng_Lang	3	2	0	-2	6	2	1	-1	3	0	-1	-1	5	2	4	2
Eng_Lit	9	10	7	-3	9	6	4	-2	4	9	5	-4	5	8	9	1
Film_Studies	3	4	5	1	1	2	4	2	1	2	2	0	1	2	1	-1
Food_P&N	-1	-1	0	1	-1	-3	0	3	1	0	0	0	-1	1	0	-1
French	4	3	2	-1	5	5	1	-4	2	2	2	0	0	1	-5	-6
Geography	2	1	2	1	3	2	3	1	5	4	6	2	-2	-5	-2	3
German	1	0	0	0	2	3	-1	-4	0	0	0	0	0	0	-3	-3
History	2	-5	1	6	1	-3	0	3	7	0	3	3	3	-4	0	4
Italian	2	0	0	0	6	6	4	-2	0	0	0	0	0	0	1	1
Latin	1	3	3	0	0	0	0	0	0	4	5	1	0	-3	-2	1
Mathematics	7	8	5	-3	10	6	3	-3	7	11	7	-4	3	6	4	-2
Media_Studies	3	3	3	0	-1	0	2	2	0	-1	0	1	1	2	1	-1
Music	-13	-14	-7	7	-9	-7	-3	4	-11	-15	-8	7	-8	-16	-12	4
PE	-6	-7	-5	2	2	-3	1	4	-5	-10	-5	5	-11	-14	-12	2
Physics	0	4	-2	-6	0	2	-4	-6	5	5	1	-4	10	14	11	-3
Psychology	0	1	-1	-2	-1	1	0	-1	-4	-6	-3	3	-5	-14	-10	4
RS	5	3	3	0	1	0	-1	-1	8	6	5	-1	3	5	5	0
Sociology	0	1	-2	-3	0	2	-1	-3	-3	-5	-6	-1	-10	-13	-12	1
Spanish	4	4	1	-3	5	2	0	-2	4	4	3	-1	2	6	4	-2
Statistics	-1	0	1	1	3	2	4	2	-1	-1	-2	-1	-1	-4	-2	2
Urdu	-1	0	0	0	3	6	4	-2	0	0	0	0	0	0	-1	-1
SD of change	4.16	4.56	2.96	2.65	4.56	3.95	2.68	2.98	4.27	6.10	3.68	3.39	5.09	8.72	6.48	3.81
Min of change	-13	-14	-7	-6	-9	-8	-8	-6	-11	-15	-8	-9	-11	-19	-12	-9
Max of change	9	10	7	7	10	7	4	6	8	11	7	8	11	14	12	14
Absolute average change	3.11	3.42	2.26	2.00	3.47	3.21	2.00	2.47	3.16	4.68	2.79	2.42	3.84	7.05	5.16	2.63

Table A3

Difference in weighted mean grade between 2020 to 2022 and 2019 and between 2022 and 2021 for A level subjects (see Table 2 for full titles of the subjects).

Subject	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021
Chinese	0.893	1.349	0.591	-0.75864
AD_Textiles	0.524	0.622	0.383	-0.23943
AD_Graphics	0.428	0.472	0.360	-0.11146
AD_Photog	0.477	0.574	0.353	-0.22099
AD_3DS	0.439	0.422	0.252	-0.17027
AD_FA	0.441	0.555	0.367	-0.18873
AD_ACD	0.468	0.618	0.350	-0.26834
Dance	0.622	0.851	0.580	-0.27114
Sociology	0.430	0.516	0.253	-0.26271
Film_Studies	0.453	0.575	0.363	-0.21196
Media_Studies	0.417	0.533	0.333	-0.20062
Bus_Studies	0.477	0.601	0.385	-0.21589
Drama_TS	0.667	0.880	0.543	-0.3363
Politics	0.489	0.672	0.443	-0.22926
Class_Civil	0.505	0.744	0.378	-0.36596
Eng_Lang	0.440	0.555	0.352	-0.20314
Eng_LangLit	0.493	0.682	0.431	-0.25071
RS	0.542	0.729	0.475	-0.2546
Law	0.572	0.671	0.280	-0.39099
Eng_Lit	0.517	0.680	0.421	-0.2585
Spanish	0.682	0.962	0.500	-0.46151
DT_PD	0.677	0.929	0.580	-0.34899
History	0.460	0.616	0.402	-0.21438
Geography	0.439	0.575	0.366	-0.20942
Economics	0.468	0.583	0.336	-0.24708
German	0.649	0.913	0.466	-0.44727
Psychology	0.530	0.720	0.422	-0.29805
French	0.583	0.777	0.381	-0.39625
Fur_Maths	0.750	0.780	0.298	-0.48209
PE	0.708	0.973	0.516	-0.45685
Accounting	0.600	0.739	0.352	-0.38676
Music	0.818	1.189	0.602	-0.58678
Mathematics	0.480	0.560	0.142	-0.41751
Music_Tech	0.784	0.983	-	-
Philosophy	0.719	1.012	0.471	-0.54046
Computing	0.782	0.962	0.446	-0.5161
Latin	-	1.717	-	-
Physics	0.608	0.706	0.292	-0.41402
Chemistry	0.668	0.761	0.222	-0.53947
Biology	0.690	0.924	0.384	-0.53978
Env_Studies	-	-	-	-0.27908
Average	0.574	0.767	0.397	-0.340

Table A4

Differences in difficulty order overall and at grades C, A and A* between 2020 to 2022 and 2019 and between 2022 and 2021 for A level subjects.

Subject	Mean				C				A				A*
	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021	2020 and 2019	2021 and 2019	2022 and 2019	2022 and 2021	2020 and 2019	2021 and 2019	2022 and 2019	**2022 and 2021	** 2020 and 2019**	2021 and 2019	2022 and 2019	2022 and 2021
Accounting	1	1	-1	-2	0	1	-2	-3	0	-5	-1	4	-10	-17	-13	4
AD_3DS	2	5	3	-2	1	4	3	-1	2	3	2	-1	1	3	1	-2
AD_ACD	0	0	0	0	3	2	1	-1	2	0	2	2	2	2	2	0
AD_FA	0	0	-2	-2	0	-1	-2	-1	2	3	2	-1	-1	2	2	0
AD_Graphics	0	2	1	-1	1	3	1	-2	2	2	2	0	-1	0	1	1
AD_Photog	1	0	1	1	-1	-1	0	1	1	1	1	0	0	1	1	0
AD_Textiles	-5	-3	-3	0	-6	-4	-5	-1	1	1	2	1	0	1	1	0
Biology	-1	-1	-1	0	0	0	0	0	1	1	1	0	5	5	5	0
Bus_Studies	1	5	1	-4	1	4	0	-4	3	5	3	-2	4	6	7	1
Chemistry	0	1	1	0	0	1	1	0	0	0	0	0	2	2	2	0
Chinese	5	0	0	0	8	0	0	0	-8	-8	-8	0	-17	-32	-32	0
Class_Civil	-2	-1	-1	0	1	2	3	1	-4	-10	-9	1	-1	-5	-3	2
Computing	-1	-1	-1	0	0	0	0	0	-2	-3	-1	2	-4	-8	-4	4
Dance	-4	-2	-3	-1	-5	-2	-4	-2	-3	-3	-4	-1	-1	-2	-1	1
Drama_TS	-1	-5	-1	4	-6	-7	-2	5	-5	-5	-4	1	-3	-4	-4	0
DT_PD	-3	-5	-2	3	-3	-4	-2	2	-1	-4	-2	2	0	-4	-4	0
Economics	1	5	1	-4	0	4	1	-3	2	5	5	0	1	4	4	0
Eng_Lang	2	3	1	-2	2	3	2	-1	-2	0	-2	-2	2	4	1	-3
Eng_LangLit	-1	0	-1	-1	-1	-2	-2	0	0	2	-1	-3	2	3	1	-2
Eng_Lit	0	4	-1	-5	0	2	-1	-3	5	7	5	-2	3	5	3	-2
Film_Studies	-3	-3	-2	1	0	-1	1	2	-3	-3	-2	1	-5	-3	-3	0
French	1	2	2	0	1	1	2	1	1	-1	0	1	6	4	4	0
Fur_Maths	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1	0
Geography	4	8	3	-5	4	5	0	-5	2	9	7	-2	2	9	7	-2
German	3	-4	3	7	2	-2	2	4	4	-2	1	3	5	2	2	0
History	4	5	0	-5	4	4	0	-4	4	5	2	-3	5	9	5	-4
Law	2	4	3	-1	0	4	4	0	3	6	3	-3	2	6	6	0
Mathematics	2	2	2	0	0	0	0	0	3	3	2	-1	8	9	9	0
Media_Studies	4	5	5	0	3	5	5	0	0	1	1	0	0	1	1	0
Music	-8	-16	-4	12	-8	-12	-2	10	-12	-17	-10	7	-4	-11	-9	2
PE	-3	-4	0	4	-1	-3	0	3	-1	-2	-1	1	-1	-1	1	2
Philosophy	-2	-6	-1	5	-2	-6	-1	5	-2	-7	-1	6	-6	-7	-3	4
Physics	0	-1	-1	0	0	-1	-1	0	0	0	0	0	0	0	0	0
Politics	0	0	-4	-4	-1	0	-3	-3	-1	2	1	-1	1	4	3	-1
Psychology	4	5	0	-5	4	6	-1	-7	3	6	2	-4	2	5	2	-3
RS	1	2	1	-1	1	-1	0	1	3	7	4	-3	-2	2	2	0
Sociology	1	2	1	-1	1	2	2	0	3	5	1	-4	0	4	2	-2
Spanish	-5	-9	0	9	-3	-6	0	6	-3	-4	-3	1	2	0	0	0
SD of change	2.76	4.47	1.97	3.67	2.96	3.72	2.04	3.15	3.36	5.17	3.53	2.40	4.40	7.49	6.69	1.76
Min of change	-8	-16	-4	-5	-8	-12	-5	-7	-12	-17	-10	-4	-17	-32	-32	-4
Max of change	5	8	5	12	8	6	5	10	5	9	7	7	8	9	9	4
Absolute average change	2.05	3.21	1.53	2.42	1.95	2.79	1.47	2.16	2.47	3.89	2.58	1.74	2.95	4.95	4.00	1.11

1. Grade 1 rather than U was used as the reference category for GCSEs in order to resolve the problem of disordered categories and disordered thresholds and large misfit associated with the reference categories when running the Rasch model. U was treated as missing and excluded from the analysis. ↩

2. Traditionally, the term ‘item difficulty’ is used in Rasch analysis of educational and psychometric test data. When the Rasch model is used in the context of inter-subject comparability investigations, the term ‘subject difficulty’ is used (see Coe, 2008). ↩