Evaluation

Printer-friendly Version  Send to a Friend  Give Feedback

2007 Annual Evaluation Review: The Challenge of Capacity Development
Completed: 2007

The general purposes of the Annual Evaluation Review are twofold:

• outline evaluation activities of the Operations Evaluation Department over the preceding 12 months; and
• summarize the performance of ADB's sovereign projects and programs from OED evaluations, and draw lessons to guide future ADB operations.

The theme of the 2007 AER is The Challenge of Capacity Development. This is in line not only with the global development agenda as stated in the Paris Declaration on Aid Effectiveness, which emphasize capacity development as one of the most critical issues in achieving aid effectiveness and sustainable development, but also with ADB's current Reform Agenda, which emphasize the importance of capacity development in its own right. Given this theme, the third purpose of this AER is to synthesize the findings on effectiveness of ADB's capacity development-related assistance from existing and ongoing OED evaluations, and to identify factors affecting effectiveness as well as lessons of good practices for improving future ADB assistance for capacity development.

Summary of Findings

• Among the most noteworthy features of OED's activities in the previous year were
  
  

  (i) continued use of a rolling 3-year work program with extensive consultation

  (ii) increased emphasis on country, sector, and thematic evaluations of strategic importance, with an increased focus on timing to feed evaluation findings into ADB's decision-makin

  (iii) a knowledge management unit established to increase the dissemination and use of OED findings

  (iv) introduction of the Evaluation Information System, revamp of the evaluation webpages in ADB's website, and production of a range of new knowledge products; and

  (v) continued involvement in the international evaluation community.

  
  
• The findings on the performance of ADB's sovereign projects and programs show that of the 19 projects/programs evaluated in 2006, 11% were rated highly successful, 63% successful, 21% partly successful, and 5% unsuccessful, better than the long-term average of 65% successful (including those rated highly successful), 27% partly successful, and 8% unsuccessful. The findings also show a long-term trend towards improved performance that started in the mid-1980s. The long-term results show that projects are more likely to be successful in certain sectors and countries. The performance of programs was very low in 1984–1991 but has improved since then to approach the project success rate.
• The findings from existing and ongoing capacity development-related evaluations conducted by OED show mixed results, with no consistent patterns of success across sectors, countries, regions, or time. However, consistent factors affecting capacity development performance have been observed, including factors driving success (success drivers or strengths) and deterring success (deterrence factors or weaknesses). These factors can be classified into four groups-design/quality-at-entry factors within and beyond ADB's control. While these four groups of factors consist of both positive and negative factors, only the positive ones (success drivers) are shown in a four-quadrant table below to roughly represent good practices for capacity development:
  
  

  1. Design/QAE Factors within ADB's Control	2. Design/QAE Factors beyond ADB's Control
  Clear results framework to measure/monitor CD Strategic direction with realistic CD objectives Adequate diagnostic baseline assessments at all CD levels (individual, organizational, network, and contextual levels) Long-term continuity to institutionalize CD, careful phasing/sequencing, and exit strategy Appropriate mix of modalities Mainstreaming project implementation/ management unit activities into target agencies' normal operations Adequate staff time and skills, and financial resources Inclusive participatory approach, with strong commitment of and ownership by target agencies Cooperation and harmonization with other DPs	Conducive political environment Conducive economic/fiscal environment Conducive policy/institutional environment Conducive sector capacity
  3. Implementation Factors within ADB's Control	4. Implementation Factors beyond ADB's Control
  Sufficient and qualified staff for implementation and supervision , including optimal use of resident missions Flexibility during implementation and supervision Selection of qualified consultants and limited delays in implementation	Continued conducive enabling environments Continued commitment of and ownership by target agencies Continued cooperation and harmonization with other DPs

  CD = capacity development, DP = development partner, QAE = quality-at-entry.

Conclusions on the Capacity Development Theme

• Lessons to improve ADB's future capacity development performance can be drawn from the success drivers identified above. These lessons should be considered as a set of good practices for ADB's future capacity development assistance.
• Since the success drivers in groups 2 and 4 are design/QAE and implementation factors at the contextual/external level beyond ADB's control, they are likely to act as incentives (opportunities) to capacity development performance. However, the negative side of these factors will act as risks/constraints (threats) to capacity development performance. Although ADB has no control over these risks, some of them could be identified and mitigation mechanisms formulated (e.g., through policy dialogue/conditionalities and loan covenants) during the design stage.
• An important conclusion is that doing less but doing it well is likely to produce better results. However, even less ambitious capacity development initiatives need to be supported by a set of good practices or success drivers as identified above, particularly the design/QAE and implementation factors within ADB's control in groups 1 and 3 (e.g., clear results frameworks, adequate diagnostics to identify and mitigate risks beyond ADB's control, long-term engagement, appropriate mix of modalities, and an inclusive participatory approach). Much remains to be done by ADB through specific and actionable recommendations to put the preconditions for these good practices in place.