Home 
Publications Catalog Online Publications ADB Review Article
Impartial Observer
|
In January 2004, the Asian Development Bank made the Operations Evaluation Department (OED) its first independent department reporting directly to the Board of Directors.
As OED helps ADB improve its development effectiveness and accountability to its stakeholders, the change reflects ADB’s commitment to enhanced accountability.
OED’s three keys to maintaining ADB’s credibility and accountability are integrity, independence, and impartiality. OED’s independence is vital to maintain the credibility of its evaluations—and of ADB as a whole.
OED’s current position in ADB’s organizational hierarchy and the scope of its evaluations are products of years of evolving ideas and needs.
OED was established in 1978 as the Postevaluation Office. At that time, evaluation was geared toward specific ADB projects and programs.
The realization among donor countries of the need for increased accountability and results manifested in the growing focus and scope of evaluations, and in the change of name to Operations Evaluation Office. Subsequently, the office became a department in 2001.
With the establishment of OED’s independence in 2004, ADB Management went from approving evaluation reports to responding to and deciding on actions ADB should take on OED recommendations (see Evaluation: Making A Difference).
OED reports directly to the Board of Directors’ Development Effectiveness Committee (DEC) (see Development Effectiveness Committee Aims for Results).
One of the keys to OED’s effectiveness is selecting the right topics for its studies. The strategic context of OED’s current 3-year rolling work program 2006–2008 (www.adb.org/Evaluation/2006-Work-Program.pdf) is based on ADB’s Poverty Reduction Strategy, the ADB President’s Address to the Board of Governors at ADB’s 38th Annual Meeting, the 2005 Paris Declaration on Aid Effectiveness, and the ADF IX Donors’ Report: Development Effectiveness for Poverty Reduction.
The work program is now a product of discussions with DEC members, the ADB President, ADB’s four vice presidents, and suggestions by operational departments and OED staff.
OED evaluates ADB projects, programs, and technical assistance through performance evaluation reports; the ongoing project portfolio, through its annual report on portfolio performance; broader thematic issues, policies, practices, and procedures through special studies; and the effectiveness of ADB operations at a country level through country assistance and sector assistance program evaluations.
OED evaluation reports, Management responses, and a summary of DEC discussions are posted on ADB’s website (www.adb.org/evaluation).
The timing of strategic OED reports is crucial to maximize their impact: reports must be prepared before decisions are made. In this context, the 3-year rolling work program is flexible so new studies can be added and others deferred or dropped as strategic topics emerge.
The biggest hurdle facing OED—and evaluation departments in other development institutions—is ensuring that lessons from evaluations are monitored and taken into account in project and strategy design
A common misconception is that evaluation comes only at the end of the project cycle. OED is beginning to provide realtime feedback to ongoing operations, including the design of new operations. It also conducts selective review and validation of project completion reports prepared by operations departments.
Because of the growing concern among donor countries that project quality be improved and more tangible results achieved—and the subsequent need for an improved strategic agenda—OED is focusing more toward broader country, sector, impact, and policy evaluations. More private sector evaluations are also being conducted.
Fewer evaluations of individual public sector projects will be undertaken. These will, however, be continued on a sample basis to accumulate lessons on the ground and to allow OED to monitor performance trends.
Development Effectiveness Committee Aims for ResultsThe Development Effectiveness Committee (DEC) of the Asian Development Bank was established in December 2000. It comprises six members of ADB’s Board of Directors whose term on the committee ends in June 2007. They are Agus Haryanto, chairperson, Indonesia; Richard Moore, Australia; Nima Wangdi, Bhutan; Emile Gauvreau, Canada; Xiaosong Zheng, People’s Republic of China; and David Taylor, United Kingdom. The Committee’s general mandate is to assist the Board in ensuring that ADB’s programs and activities are achieving the desired development objectives, and making efficient use of ADB resources. Among the responsibilities DEC carries out on behalf of the Board, are the following: reviewing the Operations Evaluation Department (OED)’s annual work program; reviewing selected OED evaluation reports and the actions taken by ADB on them; reporting to the Board on highpriority operations evaluation issues, if any, that have a significant bearing on the relevance, efficiency, and effectiveness of ADB; and making recommendations on such issues to the Board. It monitors and reports to the Board on the implementation of its decisions, reviews the annual programs preparing project completion reports and technical assistance completion reports, and reviews the annual report on the loan and technical assistance portfolio. In its Annual Report 2005, DEC reported that it considers unsatisfactory or not yet satisfactory ADB actions on five of the 13 previous DEC recommendations; that it expects ADB to undertake the necessary action on major issues identified in OED’s 2004 Annual Report; that the weakest link in the evaluation process is in monitoring and reporting on actions taken on evaluation recommendations; and that DEC activities should feed into Board discussions and decisions on policies and strategies. To view DEC’s Annual Report 2005, see www.adb.org/Documents/Board/ Committee-Reports/2005/Dev-Effect/decar05. pdf. |
The biggest hurdle facing OED—and evaluation departments in other development institutions—is ensuring that lessons from evaluations are monitored and taken into account in project and strategy design, both by the organization and its developing member countries.
OED is working to improve the quality of its lessons and recommendations to make them clearer, more actionable, and easier to monitor. This requires increased interaction with operations staff, while consciously safeguarding OED’s independence.
OED helps ADB improve its development effectiveness and accountability to its stakeholders
Efforts are also ongoing to improve evaluation methodologies, including harmonization and the establishment of good practice standards with other multilateral development banks.
The change in focus of OED’s work program toward more complex studies also has budget and staffing implications. Currently, OED has 23 professional staff and 19 local staff.
Evaluation: Making A DifferenceIn July 2005, ADB’s Operations Evaluation Department (OED) issued its fourth Annual Report on Loan and Technical Assistance (TA) Portfolio Performance. Among others, the report found stagnation in the volume of ordinary capital resources (OCR) loan approvals; delays in project implementation; problems with year-end bunching of loan and TA approvals; difficulties experienced by many developing member countries (DMCs) in meeting conditions of program loan tranche releases; a heavy concentration of lending in a few DMCs; and a decrease in OCR net income during 2001–2004. The report recommended that ADB enhance the efficiency of project implementation; improve sector selectivity and focus in country strategies and programs (CSPs); improve TA portfolio management; improve planning and timing of Board consideration of loans; and strengthen the monitoring of results attainment during implementation. In August 2005, the Development Effectiveness Committee (DEC) of ADB’s Board of Directors recommended that ADB Management prepare an action plan to address issues raised by the report. In November 2005, Management completed its Action Plan to Improve Loan and TA Portfolio Performance, which outlined efforts ADB is undertaking in its Reform Agenda to improve organizational effectiveness and development impact. The action plan also proposed a more rigorous application of existing procedures within ADB and additional measures to address specific recommendations in OED’s report. To view the action plan, go to www.adb.org/Documents/ Reports/LTAPP/IN292-05.pdf. To view OED’s 2004 Annual Report on Loan and Technical Assistance Portfolio Performance, go to www.adb.org/Documents/Reports/Portfolio_Performance/2005/rpe-oth-2005-06.pdf. And in Another Case… In December 2003, OED’s Special Evaluation Study on Project Performance Management in ADB and Its Projects in DMCs found that ADB had made substantial progress in improving the context for project performance management and developing and adopting specific processes for managing on the basis of results. It said ADB had worked hard to promote this approach in its DMCs, with limited resources, but that the quality of performance specification and progress data should be improved, particularly in the area of development results. Strategic alignment also needed to be enhanced. The report recommended ways in which ADB could improve its project performance management through a concerted effort to upgrade existing and future project frameworks (now called design and monitoring frameworks). The report also proposed ADB Management produce a time-bound action plan to respond to the report’s recommendations. In April 2004, ADB Management responded with the Action Plan for Improving Project Performance Management in ADB. The action plan recommended measures to ensure improved quality assurance and quality control of project frameworks and project performance reports; institutionalization of the responsibility for the project performance management system; and that all staff undergo training in the system. As of March 2006, the preliminary results of a regular OED assessment on the quality of design and monitoring frameworks show that those for projects and programs are progressively improving from 32% rated satisfactory or better in 2000, to 52% in 2002, 71% in 2004, and 83% in 2005. The quality of technical assistance frameworks has improved, but requires more progress. To view the Special Evaluation Study on Project Performance Management in ADB and Its Projects in DMCs, go to www.adb.org/Documents/ses/reg/ses_oth_200329.pdf. |
Go back to current issue
| © 2008 Asian Development Bank Privacy | Terms of Use |
 Top of page |