Evaluating the education inspection framework: for schools and further education and skills providers
Chris Jones discusses some of Ofsted's findings from an evaluation of the impact of the education inspection framework (EIF).
In September 2019, we introduced the education inspection framework (EIF) for our inspections of schools, further education (FE) and skills providers and early years settings. When designing the framework, we drew on a significant amount of research into what really makes a difference for children and young people. We consulted with stakeholders and ran many events to explain our thinking about the changes.
It seems like a short time between the introduction of the EIF and the start of the first national lockdown in March 2020, but in that time we carried out:
- over 7,300 ‘full’ early years register inspections
- 3,250 school inspections, including 2,900 full inspections and section 8 visits (previously known as ‘short inspections’) to good and non-exempt outstanding schools
- over 200 FE and skills inspections
From the outset, we were committed to evaluating the impact of the EIF to find out what was going well and what might need to improve.
In this commentary, I want to share some of our findings from the first piece of evaluation we have carried out. This small-scale study looked at what was working, what might have gone better and whether there were any unintended consequences.
To do this, we held focus groups with a range of leaders and staff from primary and secondary schools and FE and skills providers. We considered evidence from inspections and from providers across a range of locations, sizes and levels of deprivation. We talked to inspectors and looked at post-inspection surveys, including the free-text responses. We also carried out a survey of leaders of small schools to find out more about how the EIF inspections were affecting them.
My thanks to Dr Sue Morris-King, Senior HMI and the evaluation team for their work on this project.
Judging the quality of education
Deep dives
Under the EIF, inspectors use a deep-dive methodology to evaluate the quality of education. This involves looking at the quality of the curriculum through a range of evidence, focusing on different subjects in order to draw conclusions about the overall curriculum. Inspectors hold discussions with senior leaders, look at curriculum planning and discuss this with curriculum leaders, and visit lessons and talk to pupils about their work. The research that we carried out in order to develop the EIF showed that this methodology should allow inspectors to collect valid, reliable evidence on which to base their judgements.
So how has this worked in practice?
Responses from FE and skills providers to the 2019/20 post-inspection surveys indicate that the deep-dive methodology has generally been welcomed by the sector. The vast majority agreed that the deep-dive methodology allowed inspectors to understand the quality of education in their setting, for example:
You very rarely have the opportunity to bring all the teachers and all the work and all the evidence to the same place and sit there and look at it, and actually it’s quite a powerful opportunity to see the difference you are making.
When we talked to curriculum leaders in schools, they had similar views, for example:
There’s a lot of value in… thinking about the intent of your curriculum and how it’s planned.
I thought the whole experience of the deep dive was oddly pleasurable in the fact that you could talk about your subject and showcase what we were doing.
The ‘joined up’ nature of the deep dives – an important part of the validity of this methodology – seemed to be something that both leaders and inspectors found valuable. All our inspectors are well versed in the purpose of deep dives. They reported that these had substantially improved inspection methodology from previous frameworks:
I think it’s more focused, more interrelated, which I think is the whole point of the deep-dive methodology.
It’s the connectedness that gives us the strength to the methodology really.
And as well as improving the connection of evidence across an inspection, inspectors thought it gave them deeper evidence across the whole inspection:
We gather a quantity of information that goes way deeper and beyond what we used to be able to get to.
The use of internal data
Under the EIF, inspectors put more emphasis on the curriculum than they have under past frameworks, and no longer consider providers’ non-statutory internal progress and attainment data. During our consultations about the EIF, teachers told us they believed that this would reduce unnecessary workload. When we talked to school staff as part of our evaluation work, it was apparent that they had noticed this shift. For some, the new approach to inspection felt more aligned to what they were trying to achieve, enabling better and broader conversations about the curriculum to take place.
A small number of the leaders and staff that we spoke to in our focus groups, though, felt that the pendulum had swung too far away from considering attainment. There was a perception among some of these that Ofsted no longer considers pupils’ attainment at all. This is not the case: inspectors still use published national performance data as a starting point in inspection, and pupils’ performance in national tests and examinations is included in one of the grade descriptors for the quality of education. But national test and examinations are only one indicator of the quality of the curriculum, so we will continue to use published attainment information in a balanced way, alongside all the other valuable information we gather during inspections. Because of the disruption to the 2020 and 2021 exam series, we will have to use published data from 2019 in future inspections, recognising all its limitations.
Challenging but fair
We are always aware of the intense nature of the inspection process, and it is clear that being inspected under the EIF is demanding for leaders. And many middle leaders – particularly those who lead areas of the curriculum – are more involved in an inspection now than they might have been in the past. Our initial evaluation of the EIF explored leaders’ perspectives on their involvement. Responses reflected a process that was seen as demanding but fair:
This was the fairest inspection process I have experienced as a headteacher; it was thorough, intense and therefore exhausting but very fair and the inspection team genuinely listened to us.
Some school leaders said that they had experienced a more open approach from inspectors under the EIF, leading to better professional dialogue than they had in the past. One head of department, for example, said:
It was more of a conversation and a dialogue… the inspector who I was with really reassured me. I think for me that actually calmed me down a little bit and I could actually just focus on what we do well within a department and I could really showcase what we did.
A leader of a further education college had similar views:
It felt very supportive, it really did, you know, it very much was a collaboration… That there was significant challenge and that transparency to it… it didn’t feel like there’s an agenda or that they can sort of have preconceived ideas, it was very much ‘let’s do it together’.
Feedback and reports
Inspectors provide formal feedback to senior leaders and those responsible for governance at the end of each EIF inspection, and keep in touch with the headteacher or principal as the inspection progresses. But we found that some middle leaders who had been involved in the inspection had expected more feedback than they had received.
In some cases, curriculum leaders said that they had had valuable, in-depth professional dialogue with inspectors, which they had really appreciated. In others, curriculum leaders felt that they were not receiving enough informal feedback throughout the inspection, despite their significant investment of time in the process. Some had received feedback from their headteacher, but not until a while after the inspection. We also heard that some staff who did not receive direct feedback during the inspection were left feeling a bit worried. This was especially true for those who had been involved in a deep dive:
I was expecting an end-of-day conversation… [it] didn’t happen. It was quite late in the day but I was expecting to have something. But there was nothing, there was absolutely nothing. No debrief.
We do all of this preparation. You know, teachers work incredibly, incredibly, hard and then to not have any sort of indication…
Some leaders also felt that direct feedback from inspectors to staff could work well, particularly when it was positive:
I then fed back to staff… It would have been lovely for the staff to have had that from the [inspection] team. I was scribbling it down as they were saying it… and there was a lot and it would have been really lovely for the team to share that with the staff.
In general, when teachers, curriculum leaders or headteachers did receive feedback directly, they were extremely positive about the experience – even when they were disappointed by the feedback and final judgement.
There were also mixed views about the new-style inspection reports. Prior to the EIF, our reports were aimed more towards schools. Now, EIF reports are written with parents in mind, so they can get a real flavour of what a school is like for their child. Some senior leaders we spoke to felt that the report was now too short to cover the depth and breadth of the inspection adequately. And we heard from some curriculum leaders that they had been expecting to see more detail about their subject in the final report.
When we carry out a deep dive, we are not inspecting the subject itself. Instead, we are looking to see what is ‘systemic’ in the curriculum: what are the common reasons why things are as they are, and what needs to happen next? So feeding back or reporting in detail on individual subjects is not something we do. But our findings here about how some providers feel about the process are important ones, and we will consider how this aspect could be improved further.
Small providers
For the smallest providers of any type, inspections can require a lot of effort to organise and facilitate. Middle and senior leaders often spend much of their time teaching, and usually hold a number of different roles. Under the EIF, the shift away from discussion with headteachers, which made up a large part of inspection under the previous framework, and towards time spent with teachers and curriculum leaders therefore has a particular impact on smaller providers. We heard from leaders in small schools and small FE and skills providers about the logistical demands of releasing staff to spend time with inspectors. A few leaders also told us that they felt that some staff had been somewhat overwhelmed by the amount of time that inspectors had been with them during the inspection.
As well as talking to leaders, we carried out a survey of leaders of small schools that had been inspected under the EIF to find out whether they thought that 2 days on site with one inspector, or one day with 2, would be better. There was no real consensus. Those who preferred the one-day model saw this as reducing the impact on staff and learners by ‘getting it over and done with’ in a short space of time and therefore not extending the amount of time that the school had to accommodate inspection activities. One respondent suggested that “the stress and workload on a limited number of staff would be much greater over 2 days.”
Interestingly, however, the same reason was given by those who preferred the 2-day model. These respondents had all experienced a one-day inspection and thought that having another day would reduce the burden by spreading out the workload:
While I understand the rationale for the one day, 2 inspector model, the pressure put on members of staff in one day was quite immense.
The one-day inspection process with 2 inspectors was completely overloaded.
It was clear when we looked at evidence bases for the inspection of small primary schools that some inspectors had taken the school’s size into account really well when planning their inspection activities. Inspectors want to talk to those who are responsible for curriculum intent. In small schools, that is most likely to be the headteacher and deputy headteacher, where there is one.
Inspectors had thought carefully about the most efficient ways to do the deep dives. For example, it was clear that they had listened carefully to the headteacher’s views about which staff should be involved and when, to ensure that the process was manageable. We want to make sure that this happens more consistently as we return to full section 5 inspections.
What are we doing next with the EIF?
In his recent blog, Sean Harford, National Director, Education, explained how inspectors will take schools’ COVID-19 context into account during this term’s inspections, as well as how we will inspect safely. He talked about how we will consider the impact of COVID-19 on the curriculum and how we might use some remote methods of inspection, such as video calls, to gather some of our evidence when needed. He also said that we want to play our part in helping schools get back on track, through inspection and professional dialogue that helps schools to reflect and develop well. And as we do this, we will continue to reflect on and evaluate how well the EIF is being implemented.