Myths of Official Measurement: Auditing and Improving Administrative Data in Developing Countries
A central, yet understudied, component of state capacity is the quality of administrative data, which is a key enabler of effective policy making.
Abstract
A central, yet understudied, component of state capacity is the quality of administrative data, which is a key enabler of effective policy making. The author studies this in the context of education systems in India. Using direct audit evidence, They show that levels of student achievement in Madhya Pradesh, from a large census covering approximately 7 million students annually, are severely inflated due to cheating. This distortion affects students at all levels of achievement but is particularly severe for low-performing students. In a follow-up randomized experiment, covering over 2400 schools in a different state (Andhra Pradesh), They evaluate whether tablet-based testing, which makes cheating harder, could reduce distortion. They find paper-based assessments proctored by teachers severely exaggerate achievement, in both private and government schools, but find no evidence of such distortion in tablet-based assessments. These results suggest that business-as-usual learning assessments may be compromised even without high-powered incentives, limiting their usefulness for policy action or research, although it may be possible to curtail such manipulation even at scale. This challenge of data corruptibility, and potentially evidence on reforms to address it, is likely to generalize across sectors.
This research is part of the ‘Research on Improving Systems of Education’ programme
Citation
Singh, A. 2020. Myths of Official Measurement: Auditing and Improving Administrative Data in Developing Countries. RISE Working Paper Series. 20/042. https://doi.org/10.35489/BSG-RISE-WP_2020/042.
Link