How can the impact of development activities intended to benefit poor men, women and children caught up in complex processes of rural transformation best be assessed?
The research sets out to develop and evaluate a protocol for impact assessment based on self-reported attribution without the use of comparison groups as an alternative to experimental or quasi-experimental designs based on statistically inferred attribution.
The three year project, starting in September 2012, is led by James Copestake at the University of Bath, and being conducted in collaboration with three NGOs - Self Help Africa, Farm Africa, and Evidence for Development. It is jointly funded by the UK Economic and Social Research Council (ESRC) and Department for International Development DFID. The research has three strands that will be applied to four projects: two in Ethiopia and two in Malawi.
Strand 1 comprises a baseline and two rounds of annual monitoring of food security and income at the household level by NGO staff.
Strand 2 comprises two rounds of annual in-depth interviewing to elicit self-reported attribution from intended project beneficiaries.
Strand 3 comprises two rounds of qualitative evaluation of what Strand 2 added to the understanding of project stakeholders.
There is continued and growing demand for high quality evidence of the impact of development activities, both in countries where such activities are taking place and in those that provide financial and technical support. In the context of some scepticism about "what works" demand from the general public is also important (e.g. Horton, 2010). The project aims to have an impact on methodological debates over appropriate forms of M&E and impact assessment at various levels: (a) within SHA and FA, and through them within the NGO networks in which they participate; (b) among other development agencies engaged in work similar to SHA and FA; (c) within academia - see note on academic beneficiaries; (d) within aid policy networks within the UK and globally - e.g. through policy forums run by ODI, 3ie, Oxfam; (e) at the national level within the two countries where field work will be undertaken; (f) among the general public. Positive impact on methodologies for strengthening feedback and organisational learning within development agencies and along the aid chain should result in improved rural development practices and outcomes both within the countries where the research is undertaken and potentially more widely. There is widespread uncertainty about the effectiveness of alternative strategies and modalities for promoting household level economic security in the context of complex rural livelihood transformations: over the terms of smallholder participation in national and global value chains, for example. While the primary goal of the research is methodological, the pilot studies will also deliver more immediate and tangible benefits in terms of better understanding of the impact of the specific rural development activities of SHA and Farm Africa. These insights will impact most immediately on their own programmes in each country and more widely across the other countries across Africa where they work. They should also contribute to national and global debates over rural development strategies more widely. The impact of both methodological and substantive empirical findings will be maximised through networks at global, national and local levels. Given the primacy that SRA based IA places on the validity of beneficiary perceptions of change and attribution of impact, ensuring feedback and enabling dialogue within the communities where the studies are undertaken will also be important. The NGOs conducting the research will lead in supporting communities to use the information to increase understanding of the relative impact of specific development activities in their locality for different community members. This will help inform decisions and be used in participatory planning processes. The attached "Pathways to impact" plan sets out target audiences, forms of engagement, outputs and potential impacts more fully.