Equalities analysis: what the 2023 results tell us
Ofqual routinely analyses how attainment gaps for students with different protected characteristics and socio-economic status vary over time. Here, Dr Ian Stockford, interim Executive Director for Standards, Research and Analysis, offers analysis of results for different groups of students taking GCSEs, A levels and a subset of vocational and technical qualifications.
Today, Ofqual has published equalities analysis for the qualification results issued in summer 2023. Ofqual has conducted this analysis every year since 2020 to monitor how attainment gaps for students with different protected characteristics and socio-economic status vary over time.
As in previous years, this analysis focuses on GCSE, A level, and those vocational and technical qualifications that are taken alongside or instead of them in schools and colleges. The Department for Education (DfE) has recently published statistics on attainment of students at Key Stage 4 and for 16- to 18-year-olds. Ofqual’s analysis provides further insights on how attainment gaps have changed in recent years.
This analysis explores how the results for different groups of students have changed over time, when controlling for other variables, through a common statistical technique called multivariate regression modelling. This allows us to measure the impact of each student characteristic once all others have been held constant, rather than looking purely at more simple breakdowns of results by student characteristics. For example, we can compare the results of 2 different ethnic groups, without differences in their overall prior attainment or socio-demographic make-up affecting the findings.
The student characteristics we considered are:
- prior attainment
- ethnicity
- gender
- special educational needs and disabilities (SEND) status
- free school meal eligibility
- socio-economic status (using the Income Deprivation Affecting Children Index [IDACI])
- first language (GCSE and A level only)
- region (GCSE and A level only)
- the type of school or college (GCSE and A level only)
Although we have used the same methodology in the past, this year’s findings are presented as an interactive report. In this way, users can more easily navigate the findings and focus on the results they are interested in.
In interpreting the results of this analysis, it is important to remember that exams and other formal assessments should not be considered the cause of attainment gaps, nor can they be the solution. Indeed, an important feature of exams and formal assessments is that the rules are the same for all, so that students have the same opportunity to demonstrate what they know, understand and can do. The findings of this analysis should not be interpreted as an indication that the assessments do not function for a particular group of students. Rather, the findings of this analysis are likely to reflect societal differences and other factors, such as the uneven impact of the pandemic on teaching and learning on different groups of students.
This is why the analysis looks at changes in attainment gaps, not just at existing attainment gaps, which we know are long-standing. The analysis identifies ‘notable changes’ – those changes that we believe go beyond normal year-on-year variation.
The data for the period 2018 to 2023 are presented in this analysis. Given that 2023 saw the return to pre-pandemic standards as part of a 2-year, 2-step plan to return to normal grading arrangements after the pandemic, gaps in 2023 results are compared primarily with 2 different time points:
- 2022, when exams returned after the pandemic (though with a package of support for students)
- 2019, the last year before the pandemic
Main findings
Firstly, the modelled attainment gaps (that show the impact of each student characteristic while controlling for other factors) are always different in size to the raw comparisons (based on the raw breakdowns of results by student characteristics). The modelled attainment gaps tend to be smaller than the raw comparisons. This is an important result, not only because it indicates that there is an interplay between different student characteristics (for example, ethnicity and first language), but also because it shows how simple breakdowns can be misleading.
Secondly, of the many different comparisons between groups of students, the majority showed no notable change in attainment gaps in 2023. When controlling for other variables, attainment gaps are still there, but in most cases they have not changed since either 2022 or 2019. When there has been a change worthy of note , the gap has sometimes narrowed but in some cases widened.
The term ‘notable’ is used here and throughout the reported analysis. The use of this term differs from common usage and is used to identify where changes have been highlighted taking into account both technical statistical significance and normal between-year fluctuations. It is not intended to convey any judgement of the importance of the change in question.
The patterns are different for A level, GCSE, and vocational and technical qualifications. Below we draw out the notable changes in modelled attainment gaps, once other variables are controlled for.
A level
At A level, the analysis highlights a notable change only in relation to school and/or college type.
School and college type
The analysis compared each school and/or college type with academies, as that is the type of school and/or college with the most exam entries.
Students in further education colleges have had lower outcomes than students in academies over the past 6 years, with the gap widening over time. In 2023, the gap was around a third of a grade wider than in 2019, but was about the same as the gap in 2022 (at around 0.45 of a grade).
Students in sixth form colleges had lower outcomes than students in academies in 2023. This continues the trend of the pandemic years, but it is a change from 2019 when sixth form students had slightly higher outcomes than academy students.
Independent school students had consistently higher outcomes than students in any other type of school or college. In 2023, the difference of 0.37 of a grade compared to academies was broadly in line with 2022, but 0.10 of a grade narrower than in 2019. Once other factors are controlled for, the independent school gap in terms of the probability of attaining a grade A (or above) in 2023 was the smallest recorded since 2018.
GCSE
At GCSE, the analysis highlights notable changes in relation to gender, ethnicity, socio-economic background and school or college type. Further information on gender, ethnicity and school or college type can be found in the executive summary of the analysis.
Socio-economic background
The picture is complex when it comes to socio-economic background. This is partly because we use 2 measures of disadvantage, the individual free school meals eligibility (which has changed over time) and the area-level IDACI score of deprivation, which show slightly different results. It’s also complex as the impact seems to differ across the attainment range.
Over the past 5 years, students eligible for free school meals have had lower outcomes compared to those not eligible. In line with DfE’s statistics on the disadvantage gap, our analysis shows that in 2023 the gap had widened slightly compared to previous years, by 0.12 of a grade compared to 2019 for example. A closer look shows that the gap for the probability of attaining a grade 7 (or above) is very small and broadly stable, but has continued to widen at grade 4 (or above).
Given that free schools meals eligibility has changed over time, we also considered the IDACI scored of deprivation. As in 2022, analysis using the IDACI index shows students from the most socio-economically deprived areas continued to slowly close the gap with those from better-off areas. This change was not sufficiently large to be flagged as notable using our methodology, but even if we take it to mean that the IDACI index gap is stable, it contrasts with the findings for free school meals.
Vocational and technical qualifications (VTQs)
VTQs have different structures and different grade scales, so the analysis focused on the probability of achieving the ‘top grade’, that is, the highest grade that can be achieved in each qualification. Findings vary across the different groups of qualifications considered, but overall we found even fewer notable changes than for GCSEs and A levels.
For level 2 Technical Awards and level 3 Applied Generals there were no notable changes. In 2023, however, the socio-economic gap was narrower than in recent years.
For level 2 (and 1/2) Technical Certificates, usually taken post-16, the notable changes related to prior attainment and ethnicity.
In 2023, the gap between students with low or very low prior attainment and those with medium prior attainment narrowed 6 and 5 percentage points respectively compared to 2019. This means students with low or very low prior attainment were about as likely as students with medium prior attainment to obtain top grades in 2023.
There were notable changes relating to some groups with different ethnic backgrounds. Indeed, for all the notable changes, the gap in 2023 was narrower than in 2022. The number of students is relatively small, however, and so these findings should be treated with caution.
For level 3 Tech Levels, the notable changes were in relation to ethnicity only. Similar to the findings for Technical Certificates, the changes showed a narrowing of the gap. In the cases of white and asian and those from any other mixed or multiple background, this even led to a reverse of the differences seen in 2019 when those students were less likely to achieve top grades.
Conclusion: a mixed picture
It is difficult to draw firm conclusions from these mixed findings.
This analysis shows that most attainment gaps have not changed significantly and, where they have changed, a number of them have narrowed. For example, this was the case for many ethnic groups across a range of vocational and technical qualifications, for independent schools at A level.
Some gaps have widened, like the higher results for GCSE students with a Chinese background or the lower results for A level students in further education colleges. In some instances, the attainment gaps are difficult to interpret, as in the case of socio-economic status, where the gap varies depending on the measure of deprivation considered, and along the GCSE grading scale. Or, as in case of independent schools, where the gap at GCSE widened compared to 2022, but narrowed compared to pre-pandemic years.
The picture of attainment gaps and how they change over time is a complex one. In reporting this, it is important to carefully consider the many factors at play, and how they interact. Societal differences and the disruption to teaching and learning caused by the pandemic may have affected students differently, according to characteristics that we can observe, such as protected characteristics, socio-economic background, or ability profile. But there are also other characteristics that we cannot observe, such as motivation and parental support. And those effects may change at different points in time. This is why it is important to recognise the complexity of these findings. Making this year’s report interactive is intended to help people more easily engage with and interpret the data.
For all that this analysis presents a mixed picture, it nevertheless shows clearly that attainment gaps existed before and during the pandemic, and they are not all fixed – this should warrant reflection. The objective measurement of knowledge, skills and understanding that qualifications provide can help the education system understand the progress of different groups of students.