Inter-rater and test-retest reliability of three contingent valuation question formats in south-east Nigeria
Abstract
This paper examines the inter-rater and test-retest reliability of willingness to pay (WTP) for insecticide-treated mosquito nets and net re-treatment using the bidding game BG;: binary with follow-up (BWFU) and a novel structured haggling technique (SH). Inter-rater reliability was evaluated by having two sets of interviewers administer questionnaires to 109 BG;: 110 BWFU and 103 SH randomly selected household heads. Test-retest reliability was investigated by repeating interviews on 146 BG;: 161 BWFU and 139 SH household heads one month after an initial survey. Data analysis used testing of means, the Spearman correlation and the Pearson correlation coefficient for test of reliability, while non-parametric analysis was used to determine factors causing a variation in WTP. The study was conducted in Southeast Nigeria. Inter-rater reliability coefficients were estimated for the individual WTP for own nets, WTP for others and WTP for re-treatment. Using WTP for own nets as the best reliability estimate, the coefficients were high at values of 0.77 C.I. 0.72-0.86;: 0.75 C.I. 0.64-0.81 and 0.74 C.I. 0.63-0.82 in the BG, BWFU and SH, respectively. In test-retest reliability coefficients, the coefficients for WTP for own nets were low-to-moderate at values of 0.51 C.I. 0.40-0.62;: 0.41 C.I. 0.28-0.53 and 0.56 C.I. 0.41-0.65 for the BG, BWFU and SH groups, respectively. Factors such as gender, change in income, unplanned expenditures, stated WTP in first survey, time-to-think, external information, and subjecting respondents to more than one interview explained the lower test-retest reliability coefficients. We conclude that the CVM was reliable in the study area and the question formats had similar levels of reliability. The lower coefficients in the test-retest reliability were due to the influence of factors affecting demand that had changed in the intervening period. Standard formats for determining reliability within CVM should be developed for easy comparison of results from different studies.
Citation
Health Economics (2005) 14 (5) 529-536 [DOI: 10.1002/hec.928]