Version VIII builds on the experience of implementing Version VII, drawing mainly on feedback from programmes implementing the Standard, and from auditors auditing the implementation of the Standard. The changes represent mainly clarifications and streamlining.
Implementing the DCED Standard
1) Articulating the results chain
The DCED Standard requires that programmes first articulate a results chain, a hypothesis about how the activities of the programme are expected to lead to outputs, outcomes, and eventually development impact. Results chains can:
- Illustrate what you are doing, and why you are doing it.
- Clarify the expected outcomes and impact of your programme.
- Identify key assumptions underlying the programme.
- Communicate the programme to partners and external stakeholders.
Ultimately, you cannot monitor progress if you do not understand what you are doing. Making the logic of your programme clear, in the results chain format, provides the framework for your results measurement system. Regularly updating results chains allows the monitoring system to evolve as staff learn more about the programme and the context in which they work.
- DCED Guide to Results Chains: Explains what results chains are, how to design them, and how to use them. It also explains the requirements of the DCED Standard.
- Case studies of the DCED Standard: Each case study includes example results chains. Look for the sector or country most relevant to your project.
- Representing Theories of Change: A Technical Challenge with Evaluation Consequences. Video presentation by Rick Davies for EDIL, 2018
An indicator shows what you think success will look like if your outputs are produced and outcomes achieved. Indicators specify how you will measure whether the changes anticipated in the results chains are really occurring.
The DCED Standard requires indicators to be derived from the logic of the results chain. Once you have clarified what you expect to happen, you can then be clear about what you expect to change – and what you would measure, at each step, to see whether this change occurred. The Standard also requires “a small number of indicators at the impact level [that] can be aggregated” (2.3); DCED’s work on indicator harmonisation for this purpose is also linked below.
- DCED Guide to Developing Indicators.
- Case Studies: Several case studies of the Standard contain example indicators. See in particular case studies of tea production in Vietnam and livestock markets in Kenya with SNV, tofu production in Indonesia with Swisscontact and Mercy Corps, or maize production in Bangladesh, with Katalyst.
- DCED RMWG proposal for harmonised indicators in PSD, 2016
- DCED BEWG Practical guidance on measuring results in business environment reform, 2013
3) Measuring attributable change
Once the indicators are identified, programmes must develop a system for measuring how they change. A ‘results measurement plan’ is required to summarise what indicators will be measured, when, how, and by whom. The Standard assumes that programmes will monitor progress at output and early outcome level, on a frequent basis; this monitoring information is used by programme staff to adjust the approach – in other words, for adaptive management. The Standard also assumes that more intensive research is carried out from time to time, to assess outcomes and impact; these impact assessments are used for reporting and communications.
Measurement generates information about what is changing during the life of the programme. It does not necessarily say much about the extent to which those changes were caused by the programme. Perhaps they would have happened anyway? This is the question of attribution; to what extent were observed improvements actually caused by you? The Standard requires programmes to address this issue of attribution, to a level that would convince a reasonable but sceptical observer.
- Implementing the DCED Standard: 3a Measuring Changes in Indicators.
- Practical Guidelines for Conducting Research.
- Practical Guidance for Selecting Sample Sizes, accompanied by an online Sample Size Calculator.
- Working Paper: Measuring Job Creation in Private Sector Development, by MarketShare Associates for the Donor Committee for Enterprise Development (June 2014)
- Implementing the DCED Standard: 3b Estimating Attributable Change.
- DCED Case Studies: In particular, see palm oil production in Thailand with GTZ, and maize production in Bangladesh, with Katalyst.
4) Capturing wider change in the system or market
Traditionally, programmes aim to directly improve the lives of aid recipients. For example, they may distribute seeds, provide healthcare, or sponsor education. However, this type of assistance is limited; it will only benefit the direct recipient of the aid. Moreover, it is frequently unsustainable, as it ceases when the project ends.
In response to this challenge, PSD programmes often seek to create ‘systemic change’. This is change in systems, such as markets, government, or civil society. This can have a greater impact than direct assistance, as it affects everyone in that system. People benefit indirectly from systemic change even if they had no contact with the programme. It is more likely to be sustainable, as the change may continue even once the programme is over.
The Standard calls on programmes to make efforts to capture these wider changes, so that they do not under-report their achievements.
- Implementing the DCED Standard: Assessing Systemic Change.
- Recording of DCED Webinar on Systemic Change and presentations.
- Case studies. In particular, see ‘Making Sense of Messiness’, a paper by Samarth-NMDP Nepal on monitoring changes in market systems.
- Assessing Systemic Change, by Alexandra Miehlbradt and Hans Posthumus, 2018
- Related resources are available on the Springfield, USAID Leveraging Economic Opportunities and BEAM Exchange websites.
5) Tracking costs and impact
The Standard requires programmes to state their annual and cumulative costs, so that the achievements of the programme can be put into perspective. Clearly, a larger and more costly programme can be expected to achieve greater results and scale.
6) Reporting costs and results
The Standard requires programmes to document the key changes in the indicators at least annually, so that they can be communicated both internally (to donors, management staff, programme staff) and externally if deemed appropriate. When doing so, it is necessary to acknowledge the contributions made by other projects or the private sector. Key indicators should be dis-aggregated by gender, to the extent possible.
7) Managing the system for results measurement
An effective programme will use real-time monitoring data to adjust their approach as they implement. This allows information on results to guide decision-making at all levels, from strategic choices to implementation methods.
Results measurement should consequently be integrated into all aspects of programme management, from design through implementation. Indeed, the achievement of results should drive the programme, orienting staff efforts and guiding key decisions. This requires clear responsibilities, adequate planning, appropriate skills and sufficient human and financial resources.
An article in the Development in Practice Journal (Maclay, 2015) points out that “through its Standard for Results Measurement, the DCED has provided a unique example of donors building consensus around adaptive management concepts in the field of market sector development” and that replication of such processes and principles should be encouraged in other sectors.
- Implementing the DCED Standard: Managing the System for Results Measurement.
- In search of the sweet spot in implementing MSD programmes: why and how embracing messiness is the key to success. MDF, 2018
- The Science in Adaptive Management. ILO, 2016
- Navigating complexity. Adaptive management at the Northern Karamoja Growth, Health and Governance Program, Engineers without borders and Mercy Corps, 2014
- Experiences in M&E to date, with Peter Roggekamp, CAVAC Cambodia, 2012.
In November 2017, Alexandra Miehlbradt and Hans Posthumus conducted a training workshop with advanced practitioners in results measurement. Based on the presentations and discussions, they developed five Practitioners’ Notes. Each Note outlines key challenges and provides solutions and tips from the field. You can read a one page guide to the notes here.
- Gathering Information from Businesses (2018)
- Using Technology in Monitoring and Results Measurement (2018)
- Monitoring (2018)
- Using Multipliers to Estimate Impact (2018)
- Assessing Systemic Change (2018)
Measuring Women’s Economic Empowerment
In 2017, the DCED published a Synthesis Document that draws together resources and guidance on integrating gender and WEE into PSD programmes. This guidance is based on a review of documentation relating to the integration of gender considerations into PSD programmes and the practical experience of a broad range of economic growth programmes integrating WEE.The document is structured according to the elements of the DCED Standard.
In 2014, the DCED also published Practical Guidelines for Measuring Results in Women’s Economic Empowerment in Private Sector Development. The guidelines include practical advice on integrating women’s economic empowerment into each element of the DCED Standard. Drawing on good practices and lessons learned from Making Markets Work for the Chars (M4C) and the Alliances Lesser Caucasus Programme (ALCP) in Georgia, the guidelines in particular offer suggestions for measuring women’s economic empowerment at the household level.
Also in 2014, the DCED organised a Webinar on Women’s Economic Empowerment: Measuring Household Level Results in Practice. Click here to access all webinar resources, including the full recording, presentation and additional materials.
Implementation in Challenge Funds
In 2013, the DCED published Practical Guidelines for Measuring Results in Challenge Funds, using the DCED Standard. The guidelines contain practical advice on topics including the selection of indicators, how to design a practical results measurement system, and how to share responsibilities between the business and the fund manager. They are also applicable to other forms of partnership with the private sector.
- Experience of the Australian-funded Enterprise Challenge Fund: To read about the experiences of a challenge fund using the DCED Standard for Results Measurement, download “Designing a results measurement system for the Enterprise Challenge Fund for the Pacific and South-East Asia”.
- In 2014, the DCED organised a Webinar on Measuring Results in Challenge Funds. Click here to access all webinar resources, including the full recording and presentation, additional materials, and answers to frequently asked questions.
Implementation in Conflict-Affected Environments
The DCED’s Practical Guidelines for Measuring Achievements in PSD in Conflict-Affected Environments (2015) provide supplementary examples and advice for programmes implementing the DCED Standard in conflict-affected environments. They outline the challenges that programmes may face when implementing each of the eight elements of the Standard, and give practice guidance for results measurement in these contexts.
The DCED has produced two supplementary case studies, which show how elements of the DCED Standard can be applied in a conflict affected environment. They are of the Sustainable Employment and Economic Development Programme in Somalia (FAO) and the Employment Promotion Programme in Sierra Leone (GIZ).
The DCED Standard and evaluation
Many programmes commission external evaluations. These could benefit from the improved clarity and measurement that the DCED Standard requires, and can strengthen accountability and learning from the programme.
- The DCED’s Paper on Evaluation and the DCED Standard (2014) addresses questions such as: Why should evaluators be interested in monitoring systems? How can the DCED Standard support evaluations, and vice versa?
- In 2015, DFID and DCED held a workshop in London on Evaluation of PSD Programmes, with a number of consulting firms providing evaluation services. The Minutes document the issues arising, and include a sample agreement between DFID, Implementers and Evaluators.
- For a deeper review of evaluation methods in M4P programmes, which provides recommendations in line with the approach taken by the DCED Standard, read Itad’s Review of M4P Evaluation Methods and Approaches, 2013.
Other Programme Manuals
For a review of several RM Manuals aligned with the Standard, click here; this section lists some others.
The DCED web-page on results measurement methodologies contains policies and methodologies on results measurement from different agencies, and general papers on measuring results.
- USAID (2014): Evaluating Systems and Systemic Change for Inclusive Market Development.
- USAID (2014): Monitoring Facilitation Activities. Report to USAID/ Uganda.
- GIZ Example Results Models and Indicators, 2014, gives results chains for 10 key areas of PSD. The results chains are hyperlinked to example indicators.
- ILO (2013), Intervention models and performance indicators of the ILO Small Enterprises Unit: A mini guide for project managers
- TradeMark East Africa (2012): Monitoring and Evaluation Guidelines. TMEA is a trade facilitation organisation. These guidelines are aligned with the DCED Standard, and cover developing results chains, indicators, baseline surveys, and monitoring plans.
- Effective Monitoring for Pro-poor Cluster Development: Guidelines for Practitioners (UNIDO, 2012). These guidelines provide advice on monitoring in cluster development inspired by, and based on, the DCED Standard.