Research and analysis

​​Inter-subject comparability in GCSEs and A levels in summer 2023​

Published 25 July 2024

Applies to England

Summary

This technical report, which follows on from analyses undertaken by He and Black (2020) and He and Cadwallader (2022, 2023), use statistical methods to investigate the comparability of grading standards across subjects for GCSEs and A levels in summer 2023. Inter-subject comparability is explored over a period of 3 years: 

• summer 2023, when grades were awarded through normal exams (first time after the COVID-19 pandemic) with protection applied to subjects for which the national performance was found to be lower than in 2019 (Department for Education, 2023; Ofqual, 2022, 2023),
• summer 2022, when grades were awarded through exams (with adaptations and different grading policies applied; see Department for Education, 2021), and
• summer 2019, when grades were awarded through normal exams (prior to the COVID-19 pandemic).

In this report, ‘subject difficulty’ is based on statistical analysis of inter-subject comparability. Statistical approaches to inter-subject comparability use cohort-level data to compare the relative likelihood of students achieving a grade in certain subjects, on the basis of past attainment or concurrent attainment derived using grades achieved in all GCSE or A level subjects. This involves an assumption that there is a common underlying dimension of ‘ability’ which allows us to make meaningful comparisons between subjects as diverse as, for example, art and design and physics. The methods are described in more detail in the ‘Methods of Analysis’ section. 

Although statistical approaches are valuable, there are limits to the extent to which strong conclusions may be drawn without considering other evidence. For example, it would be inappropriate to conclude whether or not specific subjects can be said to be graded more harshly than others on the basis of statistical evidence alone. This is because differences in statistical difficulty between subjects can be caused by many factors which can vary substantially between subjects and over time but were not considered in the methods used to derive the difficulty measures. These factors, among others, could include: the nature of the subject in terms of skills and knowledge to be learnt, the performance standards required to achieve individual grades, the level of demand, allocation of teaching time and other resources, motivation of students, efficiency and effectiveness of teaching and learning, and uptake by different population subgroups. More detailed discussion of this topic is provided in Ofqual’s previous work on inter-subject comparability (Ofqual, 2015).

Two strands of analysis were conducted and reported here:

1. Empirical analysis of the relationships between subject grade outcomes and prior attainment for prior attainment matched candidates.

2. Rasch modelling of the relationships between grade outcomes in different subjects for all candidates.

The main findings from this study are:

1. Overall, both GCSE and A level subjects were graded less leniently in 2023 than in 2022 and similarly to 2019. This reflects the overall outcomes for 2023 and 2022 reported elsewhere (Ofqual, 2023) and the policy of returning to pre-pandemic grading standards in 2023.

2. The rank order of statistical grade difficulty changed across the 3 years of study:

• To provide a basis for comparison, the changes in statistical difficulty were compared with those analysed in previous studies from before the pandemic. For GCSE subjects, changes in statistical grade difficulty orders between 2023 and 2019 are smaller than changes between 2013 and 2011 at the overall subject level and at individual grades. Changes between 2023 and 2022 are larger than changes between years of normal exams, except for grade C/4 where the change is similar.

• For A level subjects, at the overall subject level and at individual grades except for grade A, changes in statistical difficulty orders between 2023 and 2019 are smaller than changes between 2019 and 2017 of normal exams before the pandemic. At grade A, changes are larger between 2023 and 2019 than between 2019 and 2017. Changes in statistical difficulty orders between 2023 and 2022 are generally larger than changes between years of normal exams before the pandemic.

3. In combination, the analysis of the distributions of statistical grade difficulties and changes to rank order positions suggest that the grading approach in 2023 has resulted in relationships between subjects that more closely resemble those from 2019 than those from 2022 when grades were awarded by exams with adaptations and different grading policies.

The different assessment and grading approaches between 2019, 2022 and 2023 have resulted in changes in the relationships between subjects at both GCSEs and A levels. This technical report provides information about how the relative difficulty of subjects, as defined statistically, has differed between these years.

1. Introduction

In summer 2023, most GCSE and all A level exams returned to pre-pandemic (COVID-19) arrangements, although some measures were in place to protect against the impact of disruption in teaching and learning experienced by students in the last 4 years (Department for Education, 2023; Ofqual, 2022, 2023). Similar to 2022, formulae equation sheets were provided for GCSE maths, physics and combined science exams. Further, protection was built into the process of setting grade boundaries in 2023 to recognise the disruption experienced by students, such that allowances were made where national performance for subjects was found to be lower than in 2019. All these protective measures were designed as a stage in the trajectory back to normal in 2024. 

This study represents a follow-up to the analyses undertaken by He and Black (2020) and He and Cadwallader (2022, 2023) and explores inter-subject comparability over a period of 3 years when national exams took place, although with different approaches and grading policies applied: summer 2023, summer 2022, and summer 2019. The analysis aims to answer the following main research question:

How did the 2023 grades awarded through normal exams impact on the statistical inter-subject comparability in GCSE and A level in relation to grades awarded in 2019 and 2022?

2. Data and analysis

2.1 Data

The primary data analysed in this investigation were the results data supplied to Ofqual by exam boards (EBs), covering 2019, 2022 and 2023 for both GCSEs and A levels. Prior attainment was based on normalised mean KS2 scores for GCSEs, extracted from the National Pupil Database (NPD), and normalised mean GCSE scores for A level candidates, calculated from their GCSE grades awarded 2 years previously. For the 2023 data, the match rate with prior attainment data is over 80% for most of the GCSE subjects and over 85% for most of the A level subjects.

Tables 1 and 2 list the GCSE and A level subjects for 2019, 2022 and 2023 that were included in the analysis (Note: For results to be more reliable, only subjects with entries greater than 3,000 for GCSEs and 1,000 for A levels were included in the analysis in a particular year except for A level geology and Latin, whose entries in 2023 were slightly below 1,000 but higher in 2019 and 2022).

Table 1: GCSE subjects analysed in this report (including acronyms for some subjects used in subsequent figures and tables)

Subject	2019	2022	2023
Art and Design: Three-dimensional design (AD_3DStudies)	X	X	X
Art and Design: Art, craft and design (AD_ACD)	X	X	X
Art and Design: Fine art (AD_FA)	X	X	X
Art and Design: Graphics (AD_Graphics)	X	X	X
Art and Design: Photography (AD_Photog)	X	X	X
Art and Design: Textile design (AD_Textiles)	X	X	X
Arabic	X	X	X
Biology	X	X	X
Business studies (Business)	X	X	X
Combined science (Cbd_Science)	X	X	X
Chemistry	X	X	X
Chinese	X	X	X
Citizenship studies (Citi_Studies)	X	X	X
Classical civilisation (Class_Civil)	X	X	X
Computer Science (Computing)	X	X	X
Design and technology (D&T)	X	X	X
Dance	X	X	X
Drama	X	X	X
Economics	X	X	X
English language (Eng_Lang)	X	X	X
English literature (Eng_Lit)	X	X	X
Film_Studies (Film studies)	X	X	X
Food preparation and nutrition (Food_P&N)	X	X	X
French	X	X	X
Geography	X	X	X
German	X	X	X
History	X	X	X
Italian	X	X	X
Latin	X	X	X
Mathematics	X	X	X
Media studies (Media_Studies)	X	X	X
Music	X	X	X
Physical education (PE)	X	X	X
Physics	X	X	X
Polish	X	X	X
Psychology	X	X	X
Religious studies (RS)	X	X	X
Religious studies: short course (RS_SC)	X	N/A	N/A
Russian	N/A	N/A	X
Sociology	X	X	X
Spanish	X	X	X
Statistics	X	X	X
Urdu	X	X	X
Total	42	41	42

Table 2: A level subjects analysed (including acronyms for some subjects used in subsequent figures and tables)

Subject	2019	2022	2023
Accounting	X	X	X
Art and Design: Three-dimensional design (AD_3DS)	X	X	X
Art and Design: Art, craft and design (AD_ACD)	X	X	X
Art and Design: Fine art (AD_FA)	X	X	X
Art and Design: Graphics (AD_Graphics)	X	X	X
Art and Design: Photography (AD_Photog)	X	X	X
Art and Design: Textile design (AD_Textiles)	X	X	X
Biology	X	X	X
Business studies (Bus_Studies)	X	X	X
Chemistry	X	X	X
Chinese	X	X	X
Classical civilisation (Class_Civil)	X	X	X
Computer science (Computing)	X	X	X
Dance	X	X	X
Drama and theatre studies (Drama_TS)	X	X	X
Design and Technology: Product design (DT_PD)	X	X	X
Economics	X	X	X
English language (Eng_Lang)	X	X	X
English language and literature (Eng_LangLit)	X	X	X
English literature (Eng_Lit)	X	X	X
Film studies (Film_Studies)	X	X	X
French	X	X	X
Further mathematics (Fur_Maths)	X	X	X
Geography	X	X	X
Geology	X	X	X
German	X	X	X
History	X	X	X
Latin	X	X	X
Law	X	X	X
Mathematics	X	X	X
Media studies (Media_Studies)	X	X	X
Music	X	X	X
Music technology (Music_Tech)	X	N/A	X
Physical education (PE)	X	X	X
Philosophy	X	X	X
Physics	X	X	X
Politics	X	X	X
Psychology	X	X	X
Religious studies (RS)	X	X	X
Sociology	X	X	X
Spanish	X	X	X
Total	41	40	41

2.2 Methods of analysis

Methods used for analysing the data were presented in detail in the reports by He and Black (2020) and He and Cadwallader (2022, 2023). In brief, 2 approaches were used in the analysis:

1. Variability in the relationship between subject grade outcomes and prior attainment, which was used as a proxy for ability (based on prior attainment-matched candidates).

2. Variability in the relationship in grade outcomes between subjects through mathematical modelling using the Rasch model (based on all candidates).

Analysis based on prior attainment-matched candidates

In the first approach, a prior attainment weighted mean grade (WMG) for a subject was calculated for each year. This involved classifying all prior attainment-matched candidates in a specific year into one of 10 prior attainment bands based on their normalised mean KS2 scores (for GCSEs) or normalised mean GCSE scores (for A levels). Each prior attainment score band has a similar number of candidates. For candidates taking a specific subject, the average grade in the subject for those falling into each score band was calculated. The mean of these average grades across the 10 score bands are the weighted mean grade for the subject. This weighted mean grade represents the expected average grade from all prior attainment-matched candidates, were the subject to be taken by all matched candidates from the population (in other words, if it had been taken by the entire cohort of GCSE or A level students with valid prior attainment measures, not just those who actually took it). WMG is used as a measure of overall difficulty of the subject. Subjects with high WMG values (higher grade outcomes) are said to be statistically ‘easier’ than subjects with low values (lower grade outcomes). The difficulty (or facility) of a subject defined this way is independent of the ability distribution of the candidates taking the subject and can be compared over time and between subjects. Since normalised KS2 scores and GCSE scores were used, WMG values can be compared between different years.

Analysis based on Rasch modelling of all candidates

In Rasch modelling, for a specific year, all GCSE (or A level) subjects are analysed using the Rasch model to generate an ability estimate for each candidate and a difficulty estimate for each grade in a subject (except for the one used as the reference grade – grade 1 for GCSEs and U for A levels^{[footnote 1]}), a difficulty measure is estimated^{[footnote 2]}. Assuming that the average ability of the candidates included in the analysis is the same across the years of analysis, the grade difficulty measures from different years can be placed on the same measurement scale and can be compared directly (for example, when the average ability of candidates is set to zero in each year and the unit of logits is used). The difficulty of a subject at a specific grade in a specific year is related to the average Rasch ability of the candidates achieving that grade. Specifically, in this study, the Rasch difficulty at a specific grade N in a subject is defined as the ability at which the expected grade is N-0.5. Subject 1 is said to be statistically ‘more difficult’ (or ‘harder’) than Subject 2 at grade X if the average ability of the candidates receiving X in Subject 1 is higher than that of the candidates in Subject 2. In other words, candidates with similar Rasch abilities would have achieved lower grades in Subject 1 than in Subject 2. Similarly, a subject is more ‘difficult’ in year Y1 than year Y2 at grade X if the average ability of the candidates receiving X in Y1 is higher than that of the candidates in Y2.

The mean of the difficulties from grade 3 to grade 8 for a GCSE subject and from D to A for an A level subject is used as the overall measure of statistical difficulty for the subject. The use of these particular grades ensures a more stable estimate of the overall difficulty for the subject.

To make the comparison of grade difficulties between subjects in the same year or across different years more meaningful, for GCSEs, the unit of difficulty in logits was converted to a unit of grade width (GW) by dividing the original difficulty value by the average grade gap in logits between grade 8 and grade 3 (the average grade gap in logits between grade 8 and grade 3 is calculated as the difference in logits between average difficulty at grade 8 and that at grade 3 divided by 5, which is equivalent to one grade width) for each year. For A levels, the average grade gap in logits between grades A and D is equivalent to one grade width (GW). The difficulty measures in unit of GW will also be directly comparable between years. An advantage of the use of GW for Rasch difficulty measures is that the effect of differences in candidates’ ability distributions (in terms of the shape of the distribution, not the mean) across years is eliminated.

Limitations

As with our previous reports, care needs to be taken when interpreting the statistical measures of subject difficulty reported in this paper. The difficulty measures (overall or at specific grades) are not direct measures of the performance standards required to achieve the grades in a subject. They are also not direct measures of subject demand or students’ efforts. Differences in statistical difficulty between subjects reflect differences in grade outcomes between subjects for candidates with similar levels of prior attainment or ability derived from the Rasch model. Such differences can be caused by many factors which can vary substantially between subjects and over time but were not considered in the methods used for this study. These factors, among others, could include: the nature of the subject in terms of skills and knowledge to be learnt, the level of demand, the performance standards required to achieve individual grades, allocation of teaching time and other resources, motivation of students, efficiency and effectiveness of teaching and learning, uptake by different population subgroups, and relative leniency or severity in grading. There are also limitations associated with the use of the Rasch model, including violation of the unidimensionality assumption (that is, the exams are assumed to measure a single ability in common), unrepresentativeness of the samples analysed, missing data, and imperfect data-model fit. All this must be kept in mind when reading this report and interpreting the results.

As with previous reports (see He and Black, 2020; He and Cadwallader, 2022, 2023), the focus of this analysis is on comparability at grades 4, 7 and 9 between GCSE subjects and grades C, A and A* between A level subjects.

3. Results and discussion

This section discusses the main results from the analysis. Results for GCSEs are presented first, followed by results for A levels.

3.1 GCSE Subjects

3.1.1 Relationship with prior attainment at KS2

Figure 1 shows the relationship between mean GCSE grade and normalised KS2 score for candidates with a valid KS2 score for 2019, 2022 and 2023. The strength of the correlation between the 2 variables is strong and similar across the 3 years. To some extent, differences in intercept between years reflect differences in overall grade outcomes (or difficulty) for candidates with similar normalised prior attainment at KS2 when the values of the slope of regression lines are similar. 

Figure 1: Relationship between candidates’ mean GCSE grade and normalised KS2 score in 2019 (top-left), 2022 (top-right) and 2023 (bottom-left).

Figure 2 compares weighted mean grade (WMG) across the GCSE subjects for 2019, 2022 and 2023. In Figure 2, subjects are arranged in ascending order of difficulty in 2019 based on their weighted mean grades. That is, subjects to the right are statistically more difficult, based on this measure, than those to the left in 2019. The WMG values from 2023 for all the subjects, except of Italian, graphics and Chinese, are very similar to those in 2019 and are considerably lower than the values in 2022 for most of the subjects. The close resemblance between the patterns of the WMG values from 2023 and 2019 indicates the intended return to the pre-pandemic grading standards from 2019.

Figure 2: Comparison of weighted mean grades between GCSE subjects for 2019, 2020 and 2023. Subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019.

The top graph in Figure 3 shows changes in WMGs between pairs of years for individual subjects and the bottom graph shows the distributions of changes across subjects in more detail. In the top graph, subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019 (see also Table A1 in Appendix A). Positive values indicate a decrease in apparent statistical subject difficulty whereas negative values indicate an increase. For example, GCSE in psychology has a value of -0.623 when comparing 2023 and 2022, which indicates that there has been an increase in apparent subject difficulty in 2023 compared to 2022. Of particular interest are the changes in subject WMGs between 2023 and 2019 as they allow us to compare subject WMGs from before the pandemic (2019) with 2023. Differences in WMG between 2023 and 2019 vary from -0.088 for classical civilization to 0.354 for Italian, with a mean of 0.036 across the subjects. These differences are much smaller than the differences between 2022 and 2019. Differences in WMG between 2023 and 2022 vary from -0.763 for classical civilization to 0.041 for Italian, with a mean of -0.423. Classical civilization, computing, economics, psychology, PE, and music all have WMGs in 2023 that are over three-fifths of a grade lower than in 2022. Italian is the only subject that has its WMG in 2023 to be closely similar to that in 2022.

Figure 3: Comparison of changes in subject population weighted mean grade between 2023/2022 and 2019 and between 2023 and 2022 for the GCSE subjects analysed. In the top graph, subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019.

3.1.2 Rasch modelling

This section discusses how subject difficulties changed between years at individual grades and the overall subject level through Rasch modelling of all candidates included in the analysis.

Relative Rasch grade difficulty distribution

The graphs in Figure 4 compare statistical subject mean difficulty and statistical difficulties at grades 4, 7 and 9 between GCSE subjects in GW unit across the 3 years of analysis based on the Rasch model. In each of the graphs, subjects are arranged in ascending order of statistical difficulty in 2019 (that is, in 2019 subjects on the right were more difficult, based on these measures, than those on the left). 

At the overall level and at individual grades, subjects in 2023 were graded similarly to 2019. Compared with 2022, almost all subjects in 2023 were graded noticeably less leniently at the overall subject level and at the individual grades. 

Figure 4: Comparison of mean difficulty (top) and difficulties at grades 4 (upper-middle), 7 (lower-middle) and 9 (bottom) for GCSE subjects for 2019, 2022 and 2023 estimated using the Rasch model. Subjects are arranged in ascending order of difficulty in 2019.

Table 3 lists the range and standard deviation of grade difficulties in GW. Note that, the smaller the range and standard deviation of difficulties, the closer the alignment in terms of grading difficulty between the subjects. 

Table 3 indicated that the standard deviation of relative grade difficulties (in GW unit) between the subjects in 2023 is generally noticeably larger than in 2022, and slightly larger than or similar to 2019 at the overall subject level and at all grades. 

At the overall subject level and at grades 4 and 7, the ranges of statistical grade difficulties in 2023 are slightly larger than in 2022 but comparable to those in 2019. At grade 9, the range in 2023 is slightly larger than in both 2019 and 2022. 

Table 3: Standard deviation and range of statistical grade difficulties (in GW) for GCSE subjects for 2019, 2022 and 2023.

Standard deviation (GW)

Year	Mean	G4	G7	G9
2019	0.833	0.924	0.802	0.823
2022	0.686	0.781	0.670	0.738
2023	0.934	0.997	0.955	0.999

Range (GW)

Year	Mean	G4	G7	G9
2019	4.871	5.395	4.808	4.335
2022	3.858	4.274	3.845	3.671
2023	4.631	5.103	4.747	4.603

Change in statistical grade difficulty

To examine how statistical subject grade difficulties have changed in more detail between 2023 and 2019, and between 2023 and 2022, Figure 6 shows distributions of changes in statistical difficulties at grades 4, 7, 9 and at the overall subject level between years for the GCSE subjects analysed (again in GW unit). In the top graphs in Figure 6, subjects are arranged in ascending order of mean difficulty in 2019. Positive values indicate an increase in apparent difficulty whereas negative values indicate a decrease.

Average changes in difficulty between 2023 and 2019 across the subjects analysed are -0.18 grade (subject mean), -0.22 grade (grade 4), -0.16 grade (grade 7), and -0.17 (grade 9). Urdu, graphics, Chinese, French and German are noticeably easier in 2023 than in 2019 at the overall subject level and at individual grades, whereas Polish is substantially harder in 2023 than in 2019. For French and German, the decrease in statistical difficulty in 2023 compared to 2019 likely results from the adjustments made in 2023 to support the alignment of grading standards in GCSE French and German with GCSE Spanish more closely (see Ofqual, 2019). Film studies, Italian and computing are slightly harder in 2023 than in 2019.

Figure 5: Changes in Rasch grade difficulties for GCSE subjects between 2023 and 2019 (left) and between 2023 to 2022 (right). In the top graphs, subjects are arranged in ascending order of mean difficulty in 2019.

Subjects on average in 2023 are statistically more difficult than in 2022 by around 0.31 grade at the overall subject level. At grades 4, 7 and 9, differences in difficulty between 2023 and 2022 are 0.36 grade, 0.28 grade and 0.26 grade respectively. Polish is the subject with the smallest increase in statistical difficulty from 2022 to 2023 at the overall subject level and at all grades, followed by Urdu. Computing, music and PE are among the subjects with largest increases in difficulty in 2023 relative to 2022. Given the approach to awarding in 2022 was influenced by the distribution of grades based on teacher assessed grades (TAGs) in 2021, it should be noted that the magnitude of these changes in difficulty will also be influenced by the 2021 grade distribution in the subject. 

Changes in statistical grade difficulty rank orders

Subjects were rank ordered based on mean statistical difficulty and difficulties at grades 4, 7 and 9 separately for 2019, 2022 and 2023, with the lowest rank (first) representing the easiest subject and highest rank the most statistically difficult subject. Figure 7 shows the distribution of changes in statistical subject difficulty rank order positions between 2 years (also see Table A2 in Appendix A). Positive values indicate increase in statistical difficulty order whereas negative values decrease in difficulty order.

At the overall subject level and at grades 4, 7 and 9, changes in statistical grade difficulty orders between 2023 and 2019 are smaller than changes between 2022 and 2019, suggesting that statistical grade difficulty orders in 2023 were more similar to those in 2019 than the difficulty orders in 2022. Changes in rank order positions between 2023 and 2022 are slightly larger than changes between 2023 and 2019 at the overall subject level and at grade 4, and considerably larger at grades 7 and 9.

Figure 6: Distributions of changes in statistical grade difficulty rank order positions in GCSE subjects between 2023/2022 and 2019 and between 2023 and 2022.

Figure 7 shows detailed changes in statistical difficulty order positions between 2023/2022 and 2019, and between 2023 and 2022 for individual subjects (also see Table A2 in Appendix A). In Figure 7, subjects are arranged in ascending order of statistical grade difficulty in 2019. Again, positive values indicate increase in rank order position (becoming more difficult statistically) whereas negative values decrease in statistical difficulty order position. Changes in rank order positions between 2023/2022 and 2019 and between 2023 and 2022 at grade 9 are generally larger than changes at grades 4 and 7 and at the overall subject level.

Between 2023 and 2019, only a few subjects changed their statistical difficulty by more than 5 positions at the overall subject level and at grades 4 and 9. At the overall subject level, citizenship studies changed its statistical difficulty position by 7 and Chinese by -8. At grade 4, graphics and PE changed their rank order position by -6 and 6 positions, respectively. At grade 9, citizenship studies (10) and Spanish (6) increased their statistical difficulty position by more than 5, whereas French (-11), graphics (-9), and Latin (-5) decreased their statistical difficulty order position by more than 5.

Figure 7: Distributions of change in statistical difficulty rank order positions overall and at individual grades for GCSE subjects between 2023/2022 and 2019 and between 2023 and 2022. Subjects are arranged in ascending order of difficulty in 2019.

Between 2023 and 2022, at the overall subject level, only 7 subjects changed their rank order positions by more than 5. These were: mathematics (-9 positions), dance (7 positions), music (7 positions), PE (7 positions), English literature (-7 positions), drama (6) and RS (-6 positions). At grade 4, no subject changed its rank order positions by more than 5. At grade 7, mathematics (-9 positions), dance (8 positions), drama, music and PE (7 positions each) and English literature (-6 positions) changed their statistical difficulty order positions by more than 5. At grade 9, half of the subjects changed their statistical difficulty order position by more than 5, with the most significant changes being noted for graphics (-15 positions), citizenship studies and sociology (12 positions each), music and PE (11 positions each), and physics (-11 positions). 

To see how changes in statistical difficulty orders between 2023 and 2019/2022 compare with changes between years of normal exams prior to the pandemic, Table 4 lists values of absolute average change and standard deviation of changes of statistical difficulty orders at the overall subject level and at 4/C, 7/A and 9/A* between 2023 and 2019/2022 and between 2011, 2012 and 2013, when normal exams took place (see He and Cadwallader, 2022). 

Changes in statistical difficulty orders between 2023 and 2019 are generally smaller than changes between 2013 and 2011 of normal exams before the pandemic at the overall subject level and at individual grades. Changes between 2023 and 2022 are larger than changes between years of normal exams, except for grade C/4 where the change is midrange. These changes in statistical subject difficulty orders to some extent reflect the different approaches to assessment and grading and the grading policy applied in these years.

Table 4: Absolute change per subject in statistical grade difficulty rank order and standard deviation of changes in statistical difficulty orders for GCSE subjects.

Grade	Years	Change in rank order (average change)	Change in rank order (standard deviation)
Mean	2012 and 2011	1.600	2.330
Mean	2013 and 2012	1.714	2.438
Mean	2013 and 2011	1.943	2.461
Mean	2023 and 2019	1.805	2.595
Mean	2023 and 2022	2.293	3.379
C or 4	2012 and 2011	1.086	1.586
C or 4	2013 and 2012	1.543	2.402
C or 4	2013 and 2011	2.057	2.859
C or 4	2023 and 2019	1.463	2.107
C or 4	2023 and 2022	1.707	2.198
A or 7	2012 and 2011	1.657	2.541
A or 7	2013 and 2012	1.771	2.849
A or 7	2013 and 2011	1.943	2.662
A or 7	2023 and 2019	1.073	1.498
A or 7	2023 and 2022	2.927	3.806
A* or 9	2012 and 2011	2.743	4.113
A* or 9	2013 and 2012	2.057	2.839
A* or 9	2013 and 2011	3.029	4.641
A* or 9	2023 and 2019	2.488	3.761
A* or 9	2023 and 2022	5.756	7.033

3.2 A level Subjects

This section discusses results from analysis of the A level subjects.

3.2.1 Relationship with prior attainment at GCSE

Figure 8 shows the relationship between mean A level grade and normalised mean GCSE score for candidates with valid mean GCSE scores for 2019, 2022 and 2023. The 2 measures are reasonably highly correlated over the 3 years of study, although the correlation is not as high as that between mean GCSE grade and normalised KS2 score. The values of the regression coefficient and intercept in 2023 are similar to the values in 2019. The intercept in 2022 is more positive than those in the other years, reflecting the differences in grading policy. 

Figure 8: Relationship between candidates’ mean A level grade and normalised mean GCSE score in 2019 (top-left), 2022 (top-right), and 2023 (bottom-left).

As with GCSEs, a prior attainment (represented by normalised mean GCSE score) weighted mean grade (WMG) for each A level subject was calculated for 2019, 2022 and 2023. This WMG comparison is presented in Figure 9 (subjects are arranged in ascending order of difficulty in 2019 based on their WMG). Similar to GCSEs, the WMG values in 2023 are noticeably lower than in 2022, reflecting the decrease in outcomes in 2023 compared to 2022. The WMG values in 2023 are generally comparable to those in 2019. It is stressed that, once again, the methods used to derive the different subject difficulty measures have important limitations. It would be inappropriate to use the findings from this analysis in isolation from other evidence when deciding, whether or not specific subjects can justifiably be said to be graded more harshly or leniently than others. 

Figure 9: Comparison of weighted mean grades in A level subjects for 2019, 2022 and 2023. Subjects are arranged in ascending order of difficulty based on their weighted mean grades in 2019.

Figure 10 compares changes in subject WMG between pairs of years in more detail. In Figure 10, subjects are arranged in descending order of weighted mean grade in 2019 (see also Table A3 in Appendix A). Positive values indicate a decrease in apparent subject difficulty whereas negative values an increase. Difference in WMG between 2023 and 2019 are considerably smaller than between 2022 and 2019. Differences in WMG between 2023 and 2019 vary from -0.10 for physics to 0.14 for textiles with a mean of 0.02. Between 2023 and 2022, differences in subject WMG vary from -0.63 for music to -0.20 for mathematics, with a mean of -0.38 (nearly two-fifths of a grade). Music, product design, and PE have their WMGs reduced by over 0.50 (half of a grade) from 2022 to 2023, whereas mathematics, some of the art and design subjects, and chemistry reduced their WMGs by less than 0.25 (a quarter of a grade).

Figure 10: Comparison of changes in subject population weighted mean grade between 2023/2022 and 2019 and between 2022 and 2023 for the A level subjects analysed. Subjects are arranged in ascending order of statistical difficulty based on their weighted mean grade in 2019 in the top graph.

3.2.2 Rasch modelling

Relative Rasch grade difficulty distribution

Figure 11 compares subject mean difficulty and difficulties at C, A and A* between the A level subjects for 2019, 2022 and 2023 analysed using the Rasch model (in GW unit). In each of the graphs, subjects are arranged in ascending order of difficulty in 2019. At the overall subject level and individual grades, subjects were less leniently graded in 2023 than in 2022. Compared with 2019, on average subjects were graded similarly at the overall subject level and at individual grades. As stressed before, it would be inappropriate to use the outcomes of this statistical analysis in isolation from other evidence when deciding, for example, whether or not specific subjects can justifiably be said to be graded more leniently than others.

Figure 11: Comparison of subject mean difficulty (top) and difficulties at C (upper-middle), A (lower-middle) and A* (bottom) for A level subjects for 2019, 2022 and 2023 estimated using the Rasch model (in GW). Subjects are arranged in order of difficulty in 2019.

Table 5 lists the range and standard deviation of relative statistical grade difficulties of the subjects in grade width (GW). As with previous years, Chinese was excluded from the summary analyses presented in the table because it had been graded particularly leniently from 2020 to 2022. Figure 13 illustrates the distributions of relative statistical subject mean difficulty and difficulties at C, A and A*.

The range and standard deviation of relative statistical grade difficulties (in GW) in 2023 are similar to or slightly smaller than those in 2022 and 2019 at the overall subject level and at grade C. At grades A and A*, the ranges and standard deviations of relative statistical grade difficulties in 2023 are larger than those in 2022, but slightly smaller than those in 2019. 

Table 5: Standard deviation and range of statistical grade difficulties (in GW) for A level subjects for 2019, 2022 and 2023 (excluding Chinese).

Standard Deviation

Year	Mean	C	A	A*
2019	0.630	0.702	0.503	0.731
2022	0.648	0.756	0.451	0.598
2023	0.616	0.697	0.477	0.635

Range

Year	Mean	C	A	A*
2019	2.620	2.838	2.118	2.675
2022	2.526	2.901	1.835	2.430
2023	2.542	2.769	1.920	2.490

Change in statistical grade difficulty

Figure 12 shows distributions of changes in statistical subject mean difficulty and difficulties at C, A and A* between 2023 and 2022 and between 2023 and 2019 (in GW unit). In Figure 12, subjects are arranged in ascending order of mean statistical difficulty in 2019. Positive values indicate an increase in apparent difficulty whereas negative values a decrease.

Although, subjects on average were graded similarly in both 2023 and 2019 at the overall subject level and at individual grades, there is variability in changes of grade difficulty between subjects. At the overall subject level, difference in difficulty between 2023 and 2019 varies from -0.18 grade for further maths to 0.26 grade for dance. This varies from -0.17 grade for film studies to 0.29 grade for dance for grade C, from -0.23 grade for further maths to 0.22 for RS for grade A, and from -0.48 for accounting to 0.17 grade for 3D studies for grade A*, respectively.

Figure 12: Changes in Rasch grade difficulties for A level subjects from 2019 to 2023 (left) and from 2022 to 2023 (right) (in GW unit). In the top graphs, subjects are arranged in ascending order of mean difficulty in 2019.

Compared with 2022, on average, subjects in 2023 were graded about a third of a grade less leniently overall and at individual grades. Subjects like dance, music and Chinese are among the subjects with the largest increases in statistical difficulty while mathematics, biology, chemistry and some of the art and design subjects are among the subjects with the smallest increases in statistical difficulty.

Changes in statistical grade difficulty rank orders

As with GCSEs, subjects were rank ordered based on mean statistical difficulty and difficulties at C, A and A* separately for 2019, 2022 and 2023. Figure 13 shows distributions of changes in statistical subject difficulty rank order positions between 2023/2022 and 2019, and between 2023 and 2022 (also see Table A4 in Appendix A).

Figure 13: Distributions of changes in statistical grade difficulty rank order positions in A level subjects between 2023/2022 and 2019, and between 2023 and 2022.

Changes in statistical difficulty orders overall and at individual grades between 2023 and 2019 are generally smaller than changes between 2022 and 2019. At the overall subject level and at each grade, changes in rank order between 2023 and 2022 are larger than changes between 2023 and 2019, particularly at A and A*.

Figure 14 shows detailed changes in statistical grade difficulty orders between 2023/2022 and 2019, and between 2023 and 2022 for individual subjects (also see Table A4 in Appendix A). In Figure 14, subjects are arranged in ascending order of statistical grade difficulty in 2019.

Figure 14: Distributions of changes in statistical difficulty rank order positions overall and at individual grades for A level subjects between 2023/2022 and 2019 and between 2023 and 2022. Subjects are arranged in ascending order of mean statistical difficulty in 2019.

Between 2023 and 2019, at the overall subject level, only media studies (6) changed their statistical difficulty order position by more than 5. At grade C, no subject changed their statistical difficulty order position by more than 5. At grade A, 2 subjects changed their difficult position by more than 5: accounting (-9) and RS (9). At A*, only accounting (-10) and geology (-9) changed their position by more than 5.

Between 2023 and 2022, only 2 subjects changed their statistical difficulty order position by more than 5 at the overall subject level and at grade C, respectively: dance (8) and music (6). At grade A, 7 subjects changed their statistical difficulty order positions by more than 5: music (11), accounting (-8), geography (-7), mathematics (-7), classical civilisation (7), Chinese (7) and economics (-6). At A*, 11 subjects changed their statistical difficulty order positions by more than 5: Chinese (35), geology (-12), music (10), mathematics (-10), film studies (7), geography (-7), Latin (6), business studies (-6), economics (-6), law (-6), and politics (-6). 

Table 6 lists values of absolute average change and standard deviation of changes of statistical difficulty orders at the overall subject level and at grades C, A and A* between years from 2017 to 2023 (also see He and Cadwallader, 2022). At the overall subject level and at individual grades except for grade A, changes in statistical difficulty orders between 2023 and 2019 are smaller than changes between 2019 and 2017. At grade A, changes are larger between 2023 and 2019 than between 2019 and 2017. Changes in statistical difficulty orders between 2023 and 2022 are generally larger than changes between years of normal exams before the pandemic. The observed changes in statistical difficulty order to some extent reflect the impact of the different approaches to assessment and grading and the grading policies implemented in these years.

Table 6: Average absolute change per subject in statistical grade difficulty rank order and standard deviation of changes in statistical difficulty orders between years for A level subjects.

Grade	Years	Change in rank order (average change)	Change in rank order (standard deviation)
Mean	2018 and 2017	0.900	1.304
Mean	2019 and 2018	1.300	2.049
Mean	2019 and 2017	1.500	2.191
Mean	2023 and 2019	1.250	1.987
Mean	2023 and 2022	1.750	2.480
C	2018 and 2017	1.300	2.121
C	2019 and 2018	1.700	2.559
C	2019 and 2017	1.650	2.145
C	2023 and 2019	1.250	1.761
C	2023 and 2022	1.700	2.313
A	2018 and 2017	1.000	1.449
A	2019 and 2018	1.150	1.549
A	2019 and 2017	1.350	1.732
A	2023 and 2019	1.450	2.480
A	2023 and 2022	3.150	4.050
A*	2018 and 2017	1.750	2.429
A*	2019 and 2018	1.750	2.924
A*	2019 and 2017	2.050	3.500
A*	2023 and 2019	1.550	2.665
A*	2023 and 2022	4.650	7.249

4. Concluding remarks

In this report, the comparability of grading standards between subjects in summer 2023 was investigated using statistical measures of grade difficulty. The approach involves empirical analysis of the relationships between subject grade outcomes and prior attainment, and Rasch modelling of the relationships between grade outcomes in different subjects. These difficulty statistics were compared with the difficulty measures estimated for 2019 and 2022. This allowed us to examine the impact of assessment and grading policies implemented in summer 2022 and 2023 on inter-subject comparability. As with previous reports, the limitations of the statistical methods used in this study, which are discussed in the ‘Methods of Analysis’ section, must be borne in mind when considering the findings from this study. It would be inappropriate to draw strong conclusions about whether specific subjects are harshly or leniently graded, without considering other evidence (see, for example, Ofqual, 2015 for further discussion).

Based on relationships between subject grade outcomes and prior attainment for prior attainment-matched candidates, GCSE subjects on average were graded about two-fifths of a grade less leniently in 2023 than in 2022 and similarly to 2019. There was, however, variation between subjects. For A levels, subjects in 2023 on average were graded about two-fifths of a grade less leniently than in 2022 and similarly to 2019. Again, there was variation between subjects. The differences in subject grading severity or leniency between 2023 and 2019 or 2022 mostly reflect the impact of the return of grading standards in 2023 to those before the covid-19 pandemic (2019).

Based on analysis using the Rasch model for all candidates taking the subjects analysed, for both GCSEs and A levels, subjects in 2023 were noticeably less leniently graded than in 2022 at the overall subject level and at individual grades. On average, both GCSEs and A levels were graded similarly in 2023 and 2019.

For GCSE subjects, the standard deviation of relative statistical grade difficulties (in GW unit) between the subjects in 2023 is generally noticeably larger than in 2022 and slightly larger than in or similar to 2019 at the overall subject level and at individual grades. At the overall subject level and at grades 4 and 7, the ranges of statistical grade difficulties in 2023 are slightly larger than those in 2022 but similar to those in 2019. At grade 9, the range in 2023 is slightly larger than in both 2019 and 2022. For A level subjects, the range and standard deviation of relative statistical grade difficulties in 2023 are similar to or slightly smaller than those in 2022 and 2019 at the overall subject level and at grade C. At grades A and A*, they are larger in 2023 than in 2022, but slightly smaller than in 2019. 

For GCSE subjects, changes in statistical grade difficulty orders between 2023 and 2019 are smaller than changes between 2013 and 2011 of normal exams before the covid-19 pandemic at the overall subject level and individual grades. Changes between 2023 and 2022 are larger than changes between years of normal exams, except for grade C/4 where the change is similar. For A level subjects, at the overall subject level and at individual grades except for grade A, changes in statistical difficulty orders between 2023 and 2019 are smaller than changes between 2019 and 2017. At grade A, changes are somewhat larger between 2023 and 2019 than between 2019 and 2017. Changes in statistical difficulty orders between 2023 and 2022 are generally larger than changes between years of normal exams before the pandemic.

In combination, the analysis of the distributions of statistical difficulties and changes to rank order positions suggest that the grading approach in 2023 has resulted in relationships between subjects that are very similar to those from 2019. Statistical inter-subject comparability in GCSEs and A levels over the 3 years of investigation has likely been affected by the different assessment and grading approaches and the trends are consistent with the grading policies implemented over these years.

References

Coe, R. (2008). Comparability of GCSE examinations in different subjects: An application of the Rasch model. Oxford Review of Education, 34, 609–636.

Department for Education (2021). Adaptations in 2022 summer exams to ensure fairness for students.

Department for Education (2023). Exams in 2023 – everything you need to know. 

He, Q. and Black, B. (2020). Impact of calculated grades, centre assessment grades and final grades on inter-subject comparability in GCSEs and A levels in 2020. Ofqual: Coventry, UK. 

He, Q. and Cadwallader, S. (2022). An investigation of inter-subject comparability in GCSEs and A levels in summer 2021. Ofqual: Coventry, UK

He, Q. and Cadwallader, S. (2023). Inter-subject comparability in GCSEs and A levels in summer 2022. Ofqual: Coventry, UK

Ofqual (2015). Comparability of Different GCSE and A Level Subjects in England: An Introduction. Ofqual: Coventry, UK.

Ofqual (2019). Grading standards in GCSE French, German and Spanish. Ofqual: Coventry, UK.

Ofqual (2020). Summer 2020 results analysis – GCSE, AS and A level. Ofqual: Coventry, UK. 

Ofqual (2021a). Summer 2021 results analysis and quality assurance. Ofqual: Coventry, UK. 

Ofqual (2021b). Ofqual’s approach to grading exams and assessments in summer 2022 and autumn 2021. Ofqual: Coventry, UK.

Ofqual (2022). Grading exams and assessments in summer 2023 and autumn 2022. Ofqual: Coventry, UK.

Ofqual (2022a). Student guide to exams and formal assessments in 2021 to 2022. Ofqual: Coventry, UK.

Ofqual (2022b). Guide to GCSE results for England, summer 2022. Ofqual: Coventry, UK.

Ofqual (2022c). Guide to AS and A level results for England, summer 2022. Ofqual: Coventry, UK.

Ofqual (2023). Exam results 2023: 10 things to know about GCSE, AS and A level grades. Ofqual: Coventry, UK.

Appendix A: Additional tables

Table A1: Difference in weighted mean grade between 2022 and 2023 and 2019, and between 2023 and 2022 for GCSE subjects (see Table 1 for full titles of the subjects).

Subject	2022 and 2019	2023 and 2019	2023 and 2022
Polish	0.097	-0.079	-0.177
Italian	0.313	0.354	0.041
Urdu	0.250	-0.013	-0.262
Chinese	0.591	0.157	-0.434
Arabic	0.266	-0.064	-0.330
AD_Photog	0.316	0.006	-0.310
AD_FA	0.455	0.099	-0.356
AD_Textiles	0.458	0.017	-0.441
AD_ACD	0.404	0.011	-0.393
Physics	0.454	0.046	-0.408
Dance	0.614	0.015	-0.600
Biology	0.442	0.044	-0.397
Chemistry	0.393	0.016	-0.376
RS	0.328	0.015	-0.313
Latin	0.520	0.002	-0.518
Drama	0.495	0.036	-0.458
Food_P&N	0.401	-0.008	-0.410
Eng_Lit	0.271	-0.020	-0.290
AD_Graphics	0.613	0.273	-0.340
Media_Studies	0.414	0.038	-0.376
Film_Studies	0.363	0.027	-0.336
Music	0.607	-0.005	-0.613
Eng_Lang	0.418	0.032	-0.386
PE	0.601	-0.014	-0.615
Citi_Studies	0.447	0.034	-0.413
Mathematics	0.317	0.049	-0.267
Sociology	0.573	0.073	-0.500
D&T	0.452	0.020	-0.432
Geography	0.483	-0.002	-0.485
Class_Civil	0.676	-0.088	-0.763
History	0.485	0.033	-0.452
AD_3DStudies	0.407	0.080	-0.327
Business	0.584	0.048	-0.536
Spanish	0.488	0.021	-0.467
Economics	0.608	-0.034	-0.642
Statistics	0.479	0.004	-0.475
French	0.492	0.068	-0.423
German	0.565	0.084	-0.481
Cbd_Science	0.276	0.013	-0.263
Psychology	0.637	0.014	-0.623
Computing	0.767	0.059	-0.708
Average	0.459	0.036	-0.423

Table A2: Differences in difficulty order overall and at grades 4, 7 and 9 between 2022 and 2023 and 2019, and between 2023 and 2022 for GCSE subjects.

Subject	2022 and 2019 mean	2023 and 2019 mean	2023 and 2022 mean	2022 and 2019 grade 4	2023 and 2019 grade 4	2023 and 2022 grade 4	2022 and 2019 grade 7	2023 and 2019 grade 7	2023 and 2022 grade 7	2022 and 2019 grade 9	2023 and 2019 grade 9	2023 and 2022 grade 9
AD_3DStudies	-1	0	1	-3	-1	2	2	0	-2	8	0	-8
AD_ACD	1	1	0	-2	1	3	1	0	-1	4	-3	-7
AD_FA	-2	-2	0	-2	-2	1	-1	-3	-2	3	-5	-8
AD_Graphics	-5	-5	0	-9	-6	3	0	-2	-2	6	-9	-15
AD_Photog	-1	0	1	0	0	0	0	1	1	0	0	0
AD_Textiles	-1	-1	0	-3	0	3	0	1	1	7	-1	-8
Arabic	2	2	0	3	1	-2	1	0	-1	0	-1	-1
Biology	-2	-2	0	-4	-3	1	2	2	0	10	3	-7
Business	-2	0	2	1	0	-1	-6	-1	5	-10	0	10
Cbd_Science	5	0	-5	2	-1	-3	4	-1	-5	7	0	-7
Chemistry	3	1	-2	-2	-1	1	4	-1	-5	12	3	-9
Chinese	-6	-8	-2	-1	-2	-1	-1	-2	-1	0	0	0
Citi_Studies	2	7	5	4	0	-4	1	3	2	-2	10	12
Class_Civil	-2	-1	1	0	-1	-1	-5	0	5	-5	2	7
Computing	-1	2	3	0	2	2	-3	1	4	-1	4	5
D&T	1	1	0	-3	-1	2	0	-1	-1	-4	1	5
Dance	-4	3	7	-2	-1	1	-7	1	8	-5	4	9
Drama	-4	2	6	0	1	1	-5	2	7	-10	0	10
Economics	-2	-1	1	0	-1	-1	-1	0	1	1	1	0
Eng_Lang	0	-2	-2	1	0	-1	-1	0	1	4	2	-2
Eng_Lit	8	1	-7	4	2	-2	5	-1	-6	9	2	-7
Film_Studies	5	4	-1	4	5	1	3	1	-2	1	0	-1
Food_P&N	0	0	0	0	1	1	0	0	0	0	3	3
French	2	0	-2	1	1	0	2	0	-2	-5	-11	-6
Geography	2	2	0	3	2	-1	6	1	-5	-2	1	3
German	0	-3	-3	-1	-1	0	0	0	0	-3	-5	-2
History	1	-2	-3	1	0	-1	3	-2	-5	0	-3	-3
Italian	0	1	1	4	1	-3	0	0	0	2	1	-1
Latin	3	3	0	0	0	0	5	2	-3	-2	-6	-4
Mathematics	6	-3	-9	3	0	-3	7	-2	-9	4	-5	-9
Media_Studies	3	5	2	2	3	1	0	3	3	1	5	4
Music	-7	0	7	-3	2	5	-8	-1	7	-12	-1	11
PE	-5	2	7	1	6	5	-5	2	7	-12	-1	11
Physics	-1	-2	-1	-4	-2	2	1	2	1	11	0	-11
Polish	0	0	0	0	0	0	0	0	0	0	0	0
Psychology	-1	0	1	0	0	0	-3	0	3	-10	0	10
RS	4	-2	-6	-1	-4	-3	5	0	-5	5	3	-2
Sociology	-2	0	2	-1	-2	-1	-6	-4	2	-12	0	12
Spanish	1	0	-1	0	0	0	3	0	-3	4	6	2
Statistics	1	0	-1	4	2	-2	-2	-1	1	-2	0	2
Urdu	-1	-3	-2	4	-1	-5	-1	0	1	-2	0	2
SD of change	3.17	2.60	3.38	2.75	2.11	2.20	3.57	1.50	3.81	6.25	3.76	7.03
Min of change	-7	-8	-9	-9	-6	-5	-8	-4	-9	-12	-11	-15
Max of change	8	7	7	4	6	5	7	3	8	12	10	12
Absolute average change	2.44	1.81	2.29	2.05	1.46	1.71	2.68	1.07	2.93	4.83	2.49	5.76

Table A3: Difference in weighted mean grade between 2022 and 2023 and 2019, and between 2023 and 2022 for A level subjects (see Table 2 for full titles of the subjects).

Subject	2022 and 2019	2023 and 2019	2023 and 2022
Chinese	0.591	0.068	-0.522
AD_Textiles	0.383	0.140	-0.242
AD_Graphics	0.360	0.106	-0.254
AD_Photog	0.353	0.124	-0.229
AD_3DS	0.252	-0.044	-0.296
AD_FA	0.367	0.081	-0.286
AD_ACD	0.350	0.092	-0.258
Dance	0.580	0.029	-0.551
Sociology	0.253	-0.040	-0.293
Film_Studies	0.363	0.040	-0.323
Media_Studies	0.333	-0.037	-0.370
Bus_Studies	0.385	0.007	-0.378
Drama_TS	0.543	0.037	-0.506
Politics	0.443	0.003	-0.440
Class_Civil	0.378	-0.033	-0.411
Eng_Lang	0.352	0.008	-0.344
Eng_LangLit	0.431	0.048	-0.383
RS	0.475	0.023	-0.452
Law	0.280	-0.038	-0.318
Eng_Lit	0.421	0.085	-0.337
Spanish	0.500	0.012	-0.488
DT_PD	0.580	0.001	-0.578
History	0.402	0.039	-0.363
Geography	0.366	0.021	-0.345
Economics	0.336	0.021	-0.315
German	0.466	0.136	-0.329
Psychology	0.422	-0.024	-0.446
French	0.381	0.025	-0.356
Fur_Maths	0.298	-0.047	-0.345
PE	0.516	-0.050	-0.567
Accounting	0.352	0.001	-0.351
Music	0.602	-0.027	-0.629
Mathematics	0.142	-0.059	-0.201
Philosophy	0.471	0.092	-0.379
Computing	0.446	-0.030	-0.476
Physics	0.292	-0.095	-0.387
Chemistry	0.222	-0.027	-0.249
Biology	0.384	0.035	-0.349
Average	0.397	0.019	-0.378

Table A4: Differences in difficulty order overall and at grades C, A and A* between 2022 and 2023 and 2019, and between 2023 and 2022 for A level subjects

Subject	2022 and 2019 mean	2023 and 2019 mean	2023 and 2022 mean	2022 and 2019 C	2023 and 2019 C	2023 and 2022 C	2022 and 2019 A	2023 and 2019 A	2023 and 2022 A	2022 and 2019 A*	2023 and 2019 A*	2023 and 2022 A*
Accounting	-1	-2	-1	-3	-3	0	-1	-9	-8	-15	-10	5
AD_3DS	3	0	-3	3	1	-2	2	0	-2	1	1	0
AD_ACD	0	-2	-2	1	1	0	2	0	-2	2	0	-2
AD_FA	-2	-2	0	-2	-3	-1	2	0	-2	2	0	-2
AD_Graphics	1	-1	-2	1	-1	-2	2	0	-2	1	-1	-2
AD_Photog	1	0	-1	1	1	0	1	0	-1	1	0	-1
AD_Textiles	-3	-3	0	-5	-3	2	2	0	-2	1	0	-1
Biology	0	-1	-1	1	1	0	1	-1	-2	6	1	-5
Bus_Studies	1	0	-1	0	0	0	3	1	-2	7	1	-6
Chemistry	1	0	-1	1	0	-1	0	-1	-1	2	0	-2
Chinese	0	0	0	0	0	0	-8	-1	7	-34	1	35
Class_Civil	-1	-1	0	3	1	-2	-9	-2	7	-3	1	4
Computing	-1	1	2	0	0	0	-1	1	2	-4	0	4
Dance	-3	5	8	-4	4	8	-4	0	4	-1	0	1
Drama_TS	-1	1	2	-2	1	3	-5	0	5	-4	1	5
DT_PD	-2	0	2	-2	0	2	-2	1	3	-4	-1	3
Economics	2	0	-2	1	-2	-3	6	0	-6	5	-1	-6
Eng_Lang	1	1	0	2	0	-2	-2	2	4	1	-3	-4
Eng_LangLit	-1	-2	-1	-2	-1	1	-1	1	2	1	-2	-3
Eng_Lit	-1	-3	-2	-1	-3	-2	5	3	-2	3	0	-3
Film_Studies	-2	-3	-1	1	-2	-3	-2	0	2	-3	4	7
French	2	2	0	3	4	1	1	1	0	5	2	-3
Fur_Maths	0	0	0	0	0	0	0	0	0	1	0	-1
Geography	3	0	-3	0	-2	-2	7	0	-7	8	1	-7
Geology	0	-4	-4	1	-1	-2	2	-3	-5	3	-9	-12
German	3	0	-3	2	0	-2	1	-3	-4	3	0	-3
History	0	0	0	0	0	0	3	3	0	6	3	-3
Latin	-2	0	2	-2	0	2	-4	-1	3	-5	1	6
Law	3	0	-3	4	2	-2	3	-1	-4	6	0	-6
Mathematics	3	0	-3	1	-1	-2	3	-4	-7	10	0	-10
Media_Studies	5	6	1	4	2	-2	1	1	0	1	4	3
Music	-5	1	6	-3	3	6	-10	1	11	-9	1	10
PE	0	3	3	0	2	2	-2	2	4	1	3	2
Philosophy	-1	-1	0	-1	-3	-2	-1	1	2	-3	1	4
Physics	-1	0	1	-1	0	1	0	0	0	0	0	0
Politics	-4	1	5	-3	0	3	1	1	0	3	-3	-6
Psychology	0	0	0	-1	0	1	2	-2	-4	2	-1	-3
RS	1	4	3	0	2	2	5	9	4	2	2	0
Sociology	1	0	-1	2	0	-2	1	1	0	2	0	-2
Spanish	0	0	0	0	0	0	-4	-1	3	-1	3	4
SD of change	2.05	1.99	2.48	2.07	1.76	2.31	3.69	2.48	4.05	7.08	2.67	7.25
Min of change	-5	-4	-4	-5	-3	-3	-10	-9	-8	-34	-10	-12
Max of change	5	6	8	4	4	8	7	9	11	10	4	35
Absolute average change	1.55	1.25	1.75	1.60	1.25	1.70	2.80	1.45	3.15	4.30	1.55	4.65

1. Grade 1 rather than U was used as the reference category for GCSEs in order to resolve the problem of disordered categories, disordered thresholds and large misfit associated with the reference categories when running the Rasch model. U was treated as missing and excluded from the analysis. For A levels, letter grades were converted into numerical values representing ordered category scores: U0, E1, D2, C3, B4, A5, and A*6. ↩

2.  Traditionally, the term ‘item difficulty’ is used in Rasch analysis of educational and psychometric test data. When the Rasch model is used in the context of inter-subject comparability investigations, the term ‘subject difficulty’ is used (see Coe, 2008). ↩