What evidence is available and what is required, in humanitarian assistance? 3ie Scoping Paper 1
Abstract
This paper investigates the current landscape of evidence, with particular emphasis on evidence from impact evaluation, in the humanitarian sector. This is in an attempt to identify areas in which actionable evidence is available and those where more evidence is needed so as to direct research to where it will be most valuable.
The study incorporates a wide array of methods to assess available evidence including an online survey of 395 participants who are knowledgeable with regards to the humanitarian sector, semi-structured interviews with 53 experts from the humanitarian sector, extensive literature reviews of repositories of humanitarian studies and strategy documents of major humanitarian organisations, and a gap map that presents the results of a thorough search for completed, ongoing and planned impact evaluations of humanitarian interventions.
The authors find that, with the exception of health and nutrition, most areas in the humanitarian sector suffer from a paucity of evidence. An evidence gap map included in this study provides an illustration of the complete landscape of evidence in the sector. The upshot is that there is agreement amongst policymakers that decisions should be based on research evidence and it is recognised that impact evaluations can and should have a greater role to play in building the evidence base.
Key recommendations coming out of this study beyond informing the areas and questions for impact evaluation are that:
- Humanitarians must agree upon a way of prioritising research needs. This study puts forward a framework for choosing areas for further research.
- Efforts need to be made to index and classify existing evidence and a single unifying repository or portal should be made to improve the ease of accessibility to existing evidence.
- A single set of templates and reporting guidelines should be agreed upon to aid in the indexing and classification of evaluation studies. It would also be beneficial to agree upon standards for data collection in these studies.
One of the big challenges encountered in undertaking this project was that there is a lack of databases and repositories that index these studies and assessments. We confirmed that there is a general lack of evidence of good high quality evidence in humanitarian assistance, especially of studies that show a causal relationship between assistance and changes in targeted results. Our findings show that operational research within humanitarian assistance can be very useful and can be undertaken. Criteria that may inform prioritising further impact evaluation related research areas are assessing the feasibility of undertaking impact evaluation, seeing whether these address current or anticipated knowledge gaps, synthesising currently available results in different contexts to learn better, focusing on innovation and choosing the populations to study with care. Last but not least we also discuss the various phases of humanitarian assistance to gauge when evaluation related research is most required. Methods protocols can go a long way to plan and help programmes if planned in advance and included in a prospective manner in humanitarian programming.
Citation
Clarke, M.; Allen, C.; Archer, F.; Wong, D.; Eriksson, A.; Puri, J. What evidence is available and what is required, in humanitarian assistance? 3ie Scoping Paper 1. International Initiative for Impact Evaluation (3ie), New Delhi, India (2014) vii + 89 pp.
Links