The Khulna-Jessore drainage project in Bangladesh showed the need to account for conditions outside a project’s immediate area
When operations evaluation began in 1978, it consisted of assessing after completion the extent to which projects had achieved their expected economic and social benefits. Now evaluation shapes decision making throughout the project cycle and in ADB as a whole. Independent Evaluation at the Asian Development Bank traces the early steps, describes recent accomplishments, and looks to the future.
The Operations Evaluation Department (OED) strives to help ADB become a learning organization. OED is able to do so because it is self-directed, avoids conflicts of interest, is insulated from external influence, and is organizationally independent. Its work demonstrates that ADB is prepared to be accountable for the results it produces. OED now reports to the Board of Directors through the Development Effectiveness Committee. (OED’s reports to the committee are uploaded onto the ADB website at the time they are submitted to the committee.) The change has made evaluation a dedicated tool—governed by the principles of usefulness, credibility, transparency, and independence—for greater accountability and better development assistance (Enhancing the Independence and Effectiveness of the Operations Evaluation Department).
OED seeks to increase value added from its work. The chapter Generating and Sharing Knowledge (page 84) shows how ADB leverages operational and developmental wisdom and increases learning. Backed by perception surveys, OED also employs measures to become more strategic, extend its outreach and respond appropriately to client needs, measure success by value added and not by number of reports produced, distinguish between types of recommendations and make better recommendations, innovate in knowledge management, and be a more active global player in developing evaluation capacity and best practices.
CONDUCTING AND DISSEMINATING STRATEGIC EVALUATIONS
OED’s work program has shifted substantially from evaluation of individual projects to broader and more strategic studies. To select priority topics, OED consults with the Development Effectiveness Committee, Management, and the heads of departments and offices. OED’s goals are to improve the quality of evaluations by using more robust methods; give priority to country and sector assistance program evaluations; increase the number of joint evaluations; validate self-evaluations to shorten the learning cycle; evaluate development impact more rigorously; develop evaluation capacity, both in ADB and in developing member countries; improve portfolio performance; evaluate business processes; and circulate findings and recommendations and see to it that these are carried out.
Influential evaluation studies in 2007 dealt with indigenous peoples safeguards; performance of technical assistance; private sector development and operations; ADB’s energy policy; projects cofinanced by ADB and the Global Environment Facility; ADB’s Japan Funds; ADB’s support for public resource management programs; the effect of microfinance on poor rural households and the status of women; ADB’s resident mission policy and related operations; ADB’s approaches to partnering and harmonization; the implementation of ADB’s long-term strategic framework; managing for development results; Asian Development Fund operations; country assistance program evaluations for the People’s Republic of China (PRC), India, Pakistan, and Sri Lanka; and sector assistance program evaluations for the energy and transport sectors in India and the road and railway sector in the PRC.
HARMONIZING PERFORMANCE INDICATORS AND EVALUATION METHODOLOGIES
OED is a member of the Evaluation Cooperation Group (ECG), a community of practice organized in 1996 by the heads of evaluation of multilateral development banks to strengthen the use of evaluation for more effective and accountable development lending; share and spread lessons from evaluations; harmonize evaluation criteria, methods, and approaches; enhance professionalism and collaboration among their members; and pave the way for borrowing member countries to build their capacity to perform evaluations.
OED began to administer, maintain, and improve ECGnet in 2006, and revamped the system in 2007. In June, the ECG also agreed to create a part-time secretariat to synchronize activities; con-tinue to administer, maintain, and develop ECGnet; and provide support services to the chair. The secretariat is housed in OED.
DEVELOPING CAPACITY IN EVALUATION AND EVALUATIVE THINKING
Since 1990, OED has helped develop evaluation capacity to raise awareness of the value of evaluation in transparent, accountable, results-oriented, and effective management systems. OED’s projects have proved that sound monitoring and evaluation systems are beneficial to good governance and public sector reform, results-based management, and internal auditing.
ADB approved regional technical assistance, funded by the Regional Cooperation and Poverty Reduction Fund of the PRC, to help strengthen the skills, resources, and systems for results-based monitoring and evaluation in Cambodia, the Lao People’s Democratic Republic, and Viet Nam. Part of the project will be research and special studies for capacity development and increased sharing of knowledge in monitoring and evaluation. The project supports the Shanghai International Program for Development Evaluation Training and is expected to propose a strategy for developing evaluation capacity.
|
ADB studied the effect of microfinance on the status of women in Nepal
Emerging Practices in Policy-Based Lending |
Policy-based lending, known as program lending in ADB, was introduced to ease the balance-of- payments difficulties of developing member countries. Over the last 30 years, it has also helped improve incentives, enabling environments and institutions, and more recently public resource management. At the end of 2006, program lending had taken up 24% of ADB's total public sector lending. Does experience disclose good practices? Can ADB’s program lending policy be improved? Read Special Evaluation Study on Policy-Based Lending: Emerging Practices in Supporting Reforms in Developing Member Countries. |
OED’s work program has shifted substantially from evaluation of individual projects to broader and more strategic studies
OED is self-directed, avoids conflicts of interest, is insulated from external influence, and is organizationally independent
Working the Internet |
OED has intensified its efforts to make more effective use of Internet technology. The evaluation pages on ADB’s website have been redesigned to emphasize content, navigational ease, appearance, load time, and cross-media accessibility. The results are telling. Requests for the department’s evaluation homepage rose sharply in 2006 and have remained consistently high since then. Source: www.adb.org/evaluation/ |
OED has helped develop evaluation capacity to raise awareness of the value of evaluation in transparent, accountable, results-oriented, and effective management systems |
|
|
Better Evaluation-Based Learning
|
|
The Operations Evaluation Department (OED) promotes the adoption of knowledge management concepts used in the corporate sector:
Concentration. ADB’s work plan has country, sector, and thematic areas of concentration.
Learning. To become a learning organization, ADB is restructuring its management information systems. OED has leveraged its greater institu-tional independence and assigned more resources to knowledge management to better support learning.
Quality of evaluation. OED’s more sophisticated methods, increased orientation toward impact assessment, and broader evaluations suggest that more knowledge is being created and used to reinforce the concepts and quality of develop-ment and to increase external accountability. The quality of the lessons identified and the recom-mendations made must constantly improve.
Feedback. With OED’s help, the feedback system for evaluation-based learning and accountability is being systematized and institutionalized.
Information and documentation. ADB is mak-ing increasing use of modern, Internet-based information and documentation systems, which cut information-gathering and search costs and strengthen institutional memory. But the systems could be made more functional, especially given the rising flood of data and information. ADB is networking the systems and encouraging upward feedback in a decentralized organization. OED manages the evaluation pages on ADB’s website for better functionality, design, content, original-ity, professionalism, and effectiveness.
Internalization. OED is looking into the possi-bility of using a more systematic and innovative way of internalizing lessons from evaluation in an overall strategic framework for knowledge management.
Monitoring. OED promotes the monitoring of actions taken on evaluation findings and recom-mendations, using well-established monitoring and evaluation systems.
Disclosure. Evaluation approach papers and the comments of external stakeholders are posted on the evaluation pages on ADB’s website.
Partners and stakeholders. ADB largely directs feedback to internal audiences, but increasingly recognizes the need for more active participation by in-country partners and stakeholders, including the media.
Broad-based evaluations. OED continues to move from individual project evaluation to a broader country, sector, and thematic focus, increasing the potential impact of evaluation, especially when it is timed to coincide with pol-icy review. Findings and recommendations from broad-based evaluations feed in straightforward fashion into ongoing projects, given the exist-ence of relatively well-established monitoring and evaluation systems; if not, challenges present themselves.
Self-evaluation. OED’s advocacy and support for better self-evaluation by units responsible for programs and activities have encouraged better design and monitoring frameworks, monitoring and evaluation systems, and completion reports.
Growing interest in developing member coun-tries (DMCs). ADB now places a higher premium on achieving results and helps DMCs strengthen their abilities to measure and manage for results. DMCs themselves are showing greater interest in developing their evaluation capacity, as reflected in stronger demand for knowledge products and services from the International Program for De-velopment Evaluation Training, and in the growth of evaluation associations.
|
|
|
Top of page
