|Project Name||Mainstreaming Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries|
|Country / Economy||Regional
|Project Type / Modality of Assistance||Technical Assistance
|Source of Funding / Amount||
|Operational Priorities||OP1: Addressing remaining poverty and reducing inequalities
OP2: Accelerating progress in gender equality
OP3: Tackling climate change, building climate and disaster resilience, and enhancing environmental sustainability
OP5: Promoting rural development and food security
|Sector / Subsector||
Agriculture, natural resources and rural development / Agricultural policy, institutional and capacity development
Education / Education sector development
Energy / Energy sector development and institutional reform
Finance / Inclusive finance
Public sector management / Social protection initiatives
Transport / Transport policies and institutional development
|Gender||Some gender elements|
|Description||The knowledge and support technical assistance (TA) cluster aims to mainstream impact evaluation (IE) to improve the development effectiveness of project and policy interventions in Asian Development Bank (ADB) developing member countries (DMCs). The TA will support (i) IE studies of ADB-financed sovereign interventions or pilot interventions under consideration for finance, and (ii) increase awareness and capacities of the DMCs to carry out IE and use results.|
|Project Rationale and Linkage to Country/Regional Strategy||
ADB commits more than $30 billion each year to investments, but rigorous evidence on attributable impacts of the financed interventions is not regularly generated. After investment completion, self-evaluations are conducted to rate performance and are subject to validation by the Independent Evaluation Department (IED). In selected cases, IED also conducts project performance evaluation, and rating patterns are synthesized in various evaluation studies. Yet, these analyses often include little empirical basis for the effects of project interventions on outcomes of interest, and understanding of effects often remains rooted in assumptions that are not subjected to testing.
The development profession has increasingly recognized the need for rigorous IE methodologies (including randomized experiments and appropriate econometric approaches) to isolate the effects of interventions from those of confounding factors. Annual publication of rigorous IEs has risen from tens of studies in the early 2000s to over 500 by 2015. During this period, the conduct of IEs to test development interventions has become a central focus of the world's top research economists. Fast-paced private industries, such as information technology, medicine, and marketing, have also rapidly escalated the use of IE methods to test consumer responses to innovations before they are scaled widely.
In parallel, recent developments in data collection technologies and analysis tools present opportunities to undertake IEs that are nimble and responsive to operational needs in shorter timeframes. More 'big data are increasingly collected and available through automated processes, including sensorized 'smart infrastructure and remote sensing, often at high frequency. Administrative and billing information are increasingly available, and microdata from national statistical systems are increasing in coverage and quality in many DMCs. Computerized service delivery can also enable new forms of large-scale experiments to rapidly test innovations.
IE is essential to strengthen ADB's Strategy 2030 ambition to be a knowledge provider. Initially, IEs at ADB were largely carried out through individual efforts of regional departments (RDs). Recognizing the need for a more systematic approach to measure project outcomes and impacts, ERCD developed a research and development TA (TA 7680-REG: Implementing Impact Evaluation at ADB) in 2010, which funded five pilot IEs. This work was supplemented in 2012 with TA cluster (CTA) 0012-REG: Developing Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries to support (i) approximately 15 IE study phases distributed over ADB's five RDs and ERCD, (ii) development of IE methodologies, and (iii) awareness raising and capacity building on IE among the DMCs. Under both TAs, IE was to have support from ERCD, but conduct remained largely an RD responsibility.
More effective mainstreaming of IE requires improved support. Continued support is needed to ensure that promising initiated studies are completed and appropriate new studies are initiated. At the same time, support needs to be improved by consistently ensuring that financial resources are accompanied by appropriate technical expertise and oversight via more of a 'One ADB approach, in which research economists from ERCD can be fully engaged in studies, along with global partners. TA support to IE studies will be further improved to focus support more strategically to add value to operations. This includes (i) adopting nimble approaches for IEs to be completed in shorter periods, (ii) targeting under-studied themes, and (iii) supporting IE of pilot interventions under consideration for future investments. Thus, additional resources will be required to (i) support ongoing IE studies for completion; (ii) support new IE studies; (iii) disseminate study findings and lessons; and (iv) contribute to capacity building of DMC stakeholders.
Development effectiveness of ADB finance and partnerships enhanced
|Description of Outcome||
Consideration of evidence on the effectiveness of interventions in ADB operations and discussions informing DMC policies expanded*
*Impact evaluation has three main outcome pathways: 1) direct pathway of study findings informing the design of similar subsequent projects; 2) process pathway of interaction between the impact evaluation and project teams leading to improvements prior to any findings; and 3) broader conceptual pathway in which findings inform other research and discussions to influence policies in a range of contexts and countries. The third pathway is often the most important over the long term, but is also difficult to track or predict in the short term, so it has no direct target.
|Progress Toward Outcome|
|Description of Project Outputs||
IE studies conducted.
IE awareness and capacity enhanced
|Status of Implementation Progress (Outputs, Activities, and Issues)|
|Summary of Environmental and Social Aspects|
|Stakeholder Communication, Participation, and Consultation|
|During Project Design|
|During Project Implementation|
|Responsible ADB Officer||Yamano, Takashi|
|Responsible ADB Department||Economic Research and Development Impact Department|
|Responsible ADB Division||Economic Analysis and Operational Support Division (EREA)|
Asian Development Bank
|Concept Clearance||30 Sep 2019|
|Approval||18 Dec 2019|
|Last Review Mission||-|
|Last PDS Update||15 Jan 2020|
|Financing Plan/TA Utilization||Cumulative Disbursements|