GCSE science: An evaluation of the expected difficulty of items
A research paper looking at the expected difficulty of GCSE science.
Documents
Details
The overall difficulty of a qualification is determined by many factors, including the quantity and demand of the content to be studied, the assessment structure and item (question) difficulty. It is intended that the new GCSEs in all subjects are of greater demand than the qualifications they are replacing. In addition to the subject content that is designed to be of greater demand, new GCSEs include questions that reward students who are able to draw together their understanding of content from across the course and who write extended responses. There is less non-exam assessment and all exams must be taken at the end of the course. For science GCSEs, the new subject content requires students to understand and use complex scientific terminology and to develop their mathematical skills in a scientific context. Exam questions will also reward students who can draw on their experience of data analysis and experimental design. These features were all required in the draft specifications put forward by the exam boards for accreditation. The panels that considered the drafts also considered findings from a research exercise on the relative difficulty of the draft sample assessment questions. The research followed the same methodology used for reformed GCSE maths.
For the research, we carried out 2 comparative judgement studies of the expected difficulty of items used in 2014 for the outgoing qualifications and the early (first and second) submissions of the reformed GCSE science specifications, submitted by exam boards in 2015/16. For several of the specifications, the draft sample assessments used in the research were changed before the qualifications were accredited. The items used in the research did not, therefore, in all cases reflect the level of the demand of the items that were included with the final, successful submission.
Overall, the distribution of expected difficulty of items in the early submissions that were used for the research was very similar between the legacy and reformed sample assessment material specifications. The reformed chemistry and physics items had marginally higher difficulty than the 2014 assessments. The biology sample assessments had marginally lower difficulty than the 2014 assessments, but these differences were so small that they would have no practical effect.
Unlike our work looking at GCSE mathematics assessments, this was not an inter-board comparability exercise. There was a small variation of difficulty across exam boards’ reformed specifications, very similar to the variability observed across legacy specifications. It is normal for these small differences to be accommodated when grade boundaries are set during awarding, and they therefore have no substantive impact. The accreditation panels took the study findings into account when they considered, in the round, the wide range of elements that contribute to demand: the new content requirements including mathematics, assessment types including the requirement for synoptic assessment, the change to a linear assessment structure, and removal of non-exam assessment. The panels concluded that the final specifications represented an appropriate increase in demand over legacy specifications.