Research and analysis
Marker effects and examination reliability
This study looked at how three different analysis methods could help us to understand 'rater' effects on exam reliability.
Marker effects and examination reliability: A comparative exploration from the perspectives of generalizability theory, Rasch modelling and multilevel modelling
Ref: Ofqual/13/5261 PDF, 1.89MB, 87 pages
This file may not be suitable for users of assistive technology. Request an accessible format.
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email firstname.lastname@example.org. Please tell us what format you need. It will help us if you say what assistive technology you use.
This study looked at how three different analysis methods could help us to understand rater effects on exam reliability. The techniques we looked at were:
- generalizability theory (G-theory)
- item response theory (IRT): in particular the Many-Facets Partial Credit Rasch Model (MFRM)
- multilevel modelling (MLM)
We used data from AS component papers in geography and psychology for 2009, 2010 and 2011 from Edexcel.