|Project Name||Implementing Impact Evaluation at ADB|
|Project Type / Modality of Assistance||Technical Assistance
|Source of Funding / Amount||
|Strategic Agendas||Inclusive economic growth
|Drivers of Change||Partnerships
|Sector / Subsector||
Public sector management - Economic affairs management
|Gender Equity and Mainstreaming||Some gender elements|
|Description||The outputs of the TA will include (i) at least 5 impact evaluation studies (1 to 2 per RD) with published case study illustrations on the IE studies, (ii) IE awareness building and capacity development for DMC officials and ADB staff, and awareness building for Management; and (iii) provision of technical support to RDs in selecting and implementing IE interventions. The TA will provide financial and technical support for selected IE interventions.|
|Project Rationale and Linkage to Country/Regional Strategy||
Bilateral and multilateral development agencies spend billions of dollars each year supporting development programs, and developing countries spend many times more on programs from their own resources. There is a substantial amount of empirical work on, for example, the effects of economic growth and human capital development on poverty reduction. However, evidence on the development effectiveness of individual projects and programs is limited. In particular, questions often remain on whether a given intervention produced its intended benefits, by how much, and how efficiently such intervention contributed to the intended outcomes and impacts. Unless the evidence of project and program impact is available, it is not possible to adequately inform policy makers and development partners as to how further assistance can be more effectively designed and implemented. While a number of multilateral and bilateral development institutions have introduced programs to institutionalize IE analyses, ADB lags behind.
Since its establishment, ADB has supported its developing member countries (DMCs) in designing, financing, and implementing project and policy interventions aimed at fostering economic development and reducing poverty in the region. ADB's routine economic sector work, cost-benefit analysis, and project performance monitoring, as well as project completion and self-evaluation exercises currently provide the basis for identifying project outcomes and impacts ex-ante and ex-post. The ex-ante analysis used in ADB for identifying causal links between development interventions, outcome and impact is relatively comprehensive. Monitoring of implementation progress and processes, especially inputs and outputs, is also attended through project performance reporting system. However, once projects are completed, the ex-post assessment of the causal links between outputs, outcomes and impact is often limited by the absence of carefully constructed counterfactuals, which are needed to assess of what would have happened to beneficiaries in absence of the interventions. Often, evaluations at project completion are conducted on the basis of monitoring of partial indicators, which may lead to selection bias due to focus on available rather than needed data. The lack of good counterfactuals and baselines also leads to reliance on qualitative assessments and identification of ADB contribution rather than attribution, limiting the accountability for development outcomes.
IED evaluates projects following their completion, but the extent to which it can carry out impact evaluations is limited by the availability of reliable counterfactuals and baseline surveys. It is recognized that good counterfactual identification and baselines surveys are best started at the beginning of the project cycle as part of project monitoring and evaluation system, as then these can be used to carry out the IEs at, to the extent possible, project mid-term and at project completion. As such, IEs should be implemented by RDs responsible for operations.
ADB recognizes the need to more systematically measure outcomes and impact as a way of strengthening the management of effective development. This issue was also underscored by member countries in General Capital Increase (GCI V) and Asian Development Fund (ADF X) discussions. In response ADB has taken actions toward upscaling and mainstreaming IE. Better measurement of the outcome and impact of ADB interventions will help the institution and DMC policy makers to assess the consequences of the development initiatives and help allocate limited resources more efficiently, guide the design of more effective interventions, and enable evidence-based policy dialogue.
Upscaling and mainstreaming IE into ADB operations require additional resources. These resources are needed to initiate DMC awareness raising and capacity building, and to provide seed funds to initiate start-up of new and highly selective IEs with aim to promote and expand IE activities. All departments are expected to have a direct or indirect role in an IE initiative, but the key implementing role is recognized to lie with the RDs, with technical support from ERD. As such, a RETA was selected as the appropriate modality. Designing and implementing IE studies is a complex undertaking and is not necessary or feasible for every project. The proposed TA will design and implement (or initiate implementation of) IEs of strategically selected project interventions so as to develop the knowledge base required to sharpen ADB's development effectiveness. The TA will be an important step toward implementing the initial ADB Impact Evaluation Work Program (2009 2010).
|Impact||Impact evaluation firmly established in ADB operations.|
|Description of Outcome||Impact evaluation piloted in each regional department with an effective centralized technical support system.|
|Progress Toward Outcome|
|Description of Project Outputs||
1. ADB staff and DMC counterparts understand impact evaluation concepts and practices.
2. Impact evaluation methodology guidelines selected, advice provided to ADB staff and DMC counterparts on project design and implementation, and at least five new impact evaluation projects implemented
3. Lessons from pilot activities presented to Management and disseminated within ADB and DMCs
|Status of Implementation Progress (Outputs, Activities, and Issues)|
|Geographical Location||1 or 2 projects in each subregion (Central and West Asia, East Asia, Pacific, South Asia, Southeast Asia)|
|Stakeholder Communication, Participation, and Consultation|
|During Project Design|
|During Project Implementation|
|Consulting Services||Although the exact requirements of the type of expertise and the length of their inputs will only be known during the implementation of the individual studies, it is envisaged that the TA will require about 18 person-months of international and 25 person-months of national consultant services. Given this limitation and the intermittent nature of assignments and anticipated variety of expertise required, the TA will primarily fulfill its consulting requirements by recruiting experts on an individual basis. If the studies require involvement of a research institution, the recruitment will be based on the Consultants Qualification Selection (CQS). On an exceptional basis, and where the subproject selections result in a country context where only one research institution can provide the required expertise, the recruitment will be conducted on a single-source selection (SSS) basis. All consultants, individual and research institutions, will be engaged in accordance with ADB's Guidelines on the Use of Consultants (2010, as amended from time to time).|
|Procurement||Procurement of equipment (computer hardware and software) by ADB will be in accordance with its Procurement Guidelines (2010, as amended from time to time) and in coordination with ADB's Office of Information Systems and Technology, and the Office of Administrative Services. At TA completion, any equipment procured will be disposed off in accordance with project administration instructions.|
|Responsible ADB Officer||Sakiko Tanaka|
|Responsible ADB Department||Economic Research and Regional Cooperation Department|
|Responsible ADB Division||Economic Analysis and Operations Support Division|
Asian Development Bank
6 ADB Avenue,
Mandaluyong City 1550, Philippines
|Concept Clearance||26 Aug 2010|
|Fact Finding||28 Feb 2010 to 13 Mar 2010|
|Last Review Mission||-|
|Last PDS Update||27 Jan 2011|
|Approval||Signing Date||Effectivity Date||Closing|
|07 Dec 2010||-||07 Dec 2010||31 Oct 2012||30 Sep 2015||-|
|Financing Plan/TA Utilization||Cumulative Disbursements|
|1,000,000.00||0.00||0.00||0.00||0.00||0.00||1,000,000.00||07 Dec 2010||881,090.68|