Introduction
Little information is available on results achieved. Even self-reported results are rare, and may not be credible. There are many reasons for this, which are explored in a Reader on the subject. Briefly, however, recent trends in PSD have made results measurement more tricky. Market-based approaches tend to benefit a self-selecting sample; isolation and randomisation are major challenges, which few have either the money or the expertise to address reliably. Every situation is different, so there is little hope of 'proving' that an approach always (or never) works.
As a result, the theme of results measurement seems to have been, in practice, rather 'stuck'. On the one hand are those who focus on achieving results in very complex environments, and who see problems more than solutions in trying to measure those results. On the other hand are those who say 'if you cannot measure the results of your work, maybe they just aren't there'. Existing methodologies have not (yet) delivered solutions to this conundrum, despite an emphasis on traditional logframes, baselines and some statistical studies.
Thankfully, the evaluation community is moving away from a reliance on purely statistical techniques and macro-level correlations, and towards mixed methods, based on the results chain or logic of the programme. Indeed, articulating and validating the logic of the programme is a very powerful (if under-used) tool. The log-frame format was designed in a simpler age to encourage exactly this clarity; expanding this format to capture the sequencing and parallel activities now being implemented in many programmes can yield great benefits - in terms both of effectiveness and measurement.
For some published results, visit our Evidences of Impact page. Additional input is always welcome - particularly given the lack of results data mentioned above.
The DCED Standard for results measurement
The DCED's own Standard for results measurement provides an innovative yet practical framework, whereby programmes can measure their own results according to good practice. Some training providers offer courses featuring the Standard
for the page dedicated to the DCED Standard for results measurement.
Agency Policies and Methodologies for results measurement
An overview member agency methodologies in results measurement, both generally and relating to value chain development and business environment reform in particular, is available here.
The Foundation Center has compiled a list of tools and resources used by foundations and philanthropic organisations for assessing social impact. The list, including brief summaries of the tools and resources, is available here.
Methodological Papers on measuring results in PSD
- Gertler, Paul et.al. (2011) Impact Evaluation in Practice. Washington, DC: The World Bank.
- GTZ. (2009) Measuring Employment Effects of Technical Cooperation Interventions: Some Methodological Guidelines.
- OECD-DAC. Glossary of Key Terms in Evaluation and Results Based Management. Paris: OECD.
- Creevey, Lucy. (2008) Common Problems in Impact Assessment Research, USAID Impact Assessment Primer Series, Washington, DC: USAID
- Creevey, Lucy and Don Snodgrass. (2006) Collecting and Using Data for Impact Assessment, USAID Impact Assessment Primer Series, Washington, DC: USAID
- de Janvry, Alain, Andrew Dustan and Elisabeth Sadoulet. (2010) Recent Advances in Impact Anallysis Methods for Ex-Post Impact Assessments of Agricultural Technology: Options for the CGIAR.
- Leuww, Frans and Jos Vaessen on behalf of the Network of Networks on Impact Evaluation (NONIE). (2009) Impact and Evaluation: NONIE Guidance. Washington, DC: World Bank Group.
- Vaessen, Jos. (2010) Challenges in impact evaluation of development
interventions: opportunities and limitations for randomized experiments. Discussion Paper. Antwerp, Belgium: University of Antwerp.
- White, Howard. (2011) An Introduction to the Use of Randomized Control Trials to Evaluate Development Interventions. 3ie Working Paper. New Delhi, India: International Initiative for Impact Evaluation.
- Woller, Gary. (2007) Developing a Causal Model for Private Sector Development Programmes. USAID Impact Assessment Primer Series, Washington, DC: USAID.
- Woller, Gary and Jeanne Downing. (2007) Causal Models as a Useful Programme Management Tool: Case Study of PROFIT Zambia, USAID Impact Assessment Primer Series, Washington, DC: USAID
Photographs courtesy of Sudipto Das, Stefan Erber