Regional : Mainstreaming Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries

Sovereign Project | 53343-001

The knowledge and support technical assistance (TA) cluster aims to mainstream impact evaluation (IE) to improve the development effectiveness of project and policy interventions in Asian Development Bank (ADB) developing member countries (DMCs). The TA will support (i) IE studies of ADB-financed sovereign interventions or pilot interventions under consideration for finance, and (ii) increase awareness and capacities of the DMCs to carry out IE and use results.

Project Name
Mainstreaming Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries
Project Number
53343-001
Country / Economy
  • Regional
Project Status
Active
Project Type / Modality of Assistance
  • Technical Assistance
Source of Funding / Amount
TA 0045-REG: Mainstreaming Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries
Source Amount
Technical Assistance Special Fund US$ 4.50 million
Operational Priorities
  • OP1: Addressing remaining poverty and reducing inequalities
  • OP2: Accelerating progress in gender equality
  • OP3: Tackling climate change, building climate and disaster resilience, and enhancing environmental sustainability
  • OP5: Promoting rural development and food security
Sector / Subsector
  • Agriculture, natural resources and rural development / Agricultural policy, institutional and capacity development

  • Education / Education sector development

  • Energy / Energy sector development and institutional reform

  • Finance / Inclusive finance

  • Public sector management / Social protection initiatives

  • Transport / Transport policies and institutional development

Gender
Some gender elements
Description
The knowledge and support technical assistance (TA) cluster aims to mainstream impact evaluation (IE) to improve the development effectiveness of project and policy interventions in Asian Development Bank (ADB) developing member countries (DMCs). The TA will support (i) IE studies of ADB-financed sovereign interventions or pilot interventions under consideration for finance, and (ii) increase awareness and capacities of the DMCs to carry out IE and use results.
Project Rationale and Linkage to Country/Regional Strategy

ADB commits more than $30 billion each year to investments, but rigorous evidence on attributable impacts of the financed interventions is not regularly generated. After investment completion, self-evaluations are conducted to rate performance and are subject to validation by the Independent Evaluation Department (IED). In selected cases, IED also conducts project performance evaluation, and rating patterns are synthesized in various evaluation studies. Yet, these analyses often include little empirical basis for the effects of project interventions on outcomes of interest, and understanding of effects often remains rooted in assumptions that are not subjected to testing.

The development profession has increasingly recognized the need for rigorous IE methodologies (including randomized experiments and appropriate econometric approaches) to isolate the effects of interventions from those of confounding factors. Annual publication of rigorous IEs has risen from tens of studies in the early 2000s to over 500 by 2015. During this period, the conduct of IEs to test development interventions has become a central focus of the world's top research economists. Fast-paced private industries, such as information technology, medicine, and marketing, have also rapidly escalated the use of IE methods to test consumer responses to innovations before they are scaled widely.

In parallel, recent developments in data collection technologies and analysis tools present opportunities to undertake IEs that are nimble and responsive to operational needs in shorter timeframes. More 'big data are increasingly collected and available through automated processes, including sensorized 'smart infrastructure and remote sensing, often at high frequency. Administrative and billing information are increasingly available, and microdata from national statistical systems are increasing in coverage and quality in many DMCs. Computerized service delivery can also enable new forms of large-scale experiments to rapidly test innovations.

IE is essential to strengthen ADB's Strategy 2030 ambition to be a knowledge provider. Initially, IEs at ADB were largely carried out through individual efforts of regional departments (RDs). Recognizing the need for a more systematic approach to measure project outcomes and impacts, ERCD developed a research and development TA (TA 7680-REG: Implementing Impact Evaluation at ADB) in 2010, which funded five pilot IEs. This work was supplemented in 2012 with TA cluster (CTA) 0012-REG: Developing Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries to support (i) approximately 15 IE study phases distributed over ADB's five RDs and ERCD, (ii) development of IE methodologies, and (iii) awareness raising and capacity building on IE among the DMCs. Under both TAs, IE was to have support from ERCD, but conduct remained largely an RD responsibility.

More effective mainstreaming of IE requires improved support. Continued support is needed to ensure that promising initiated studies are completed and appropriate new studies are initiated. At the same time, support needs to be improved by consistently ensuring that financial resources are accompanied by appropriate technical expertise and oversight via more of a 'One ADB approach, in which research economists from ERCD can be fully engaged in studies, along with global partners. TA support to IE studies will be further improved to focus support more strategically to add value to operations. This includes (i) adopting nimble approaches for IEs to be completed in shorter periods, (ii) targeting under-studied themes, and (iii) supporting IE of pilot interventions under consideration for future investments. Thus, additional resources will be required to (i) support ongoing IE studies for completion; (ii) support new IE studies; (iii) disseminate study findings and lessons; and (iv) contribute to capacity building of DMC stakeholders.

Impact

Development effectiveness of ADB finance and partnerships enhanced

Project Outcome

Description of Outcome

Consideration of evidence on the effectiveness of interventions in ADB operations and discussions informing DMC policies expanded*

*Impact evaluation has three main outcome pathways: 1) direct pathway of study findings informing the design of similar subsequent projects; 2) process pathway of interaction between the impact evaluation and project teams leading to improvements prior to any findings; and 3) broader conceptual pathway in which findings inform other research and discussions to influence policies in a range of contexts and countries. The third pathway is often the most important over the long term, but is also difficult to track or predict in the short term, so it has no direct target.

Progress Toward Outcome

Implementation Progress

Description of Project Outputs

IE studies conducted.

IE awareness and capacity enhanced

Status of Implementation Progress (Outputs, Activities, and Issues)
Geographical Location
Regional

Summary of Environmental and Social Aspects

Environmental Aspects
Involuntary Resettlement
Indigenous Peoples

Stakeholder Communication, Participation, and Consultation

During Project Design
During Project Implementation

Contact

Responsible ADB Officer
Yamano, Takashi
Responsible ADB Department
Economic Research and Development Impact Department
Responsible ADB Division
Economic Analysis and Operational Support Division (EREA)
Executing Agencies
Asian Development Bank

Timetable

Concept Clearance
30 Sep 2019
Fact Finding
-
MRM
-
Approval
18 Dec 2019
Last Review Mission
-
Last PDS Update
15 Jan 2020

Funding

TA 0045-REG

Financing Plan/TA Utilization
ADB Cofinancing Counterpart Total
Gov Beneficiaries Project Sponsor Others
4,500,000.00 0.00 0.00 0.00 0.00 0.00 4,500,000.00
Cumulative Disbursements
Date Amount
- 0.00

Project Data Sheets (PDS) contain summary information on the project or program. Because the PDS is a work in progress, some information may not be included in its initial version but will be added as it becomes available. Information about proposed projects is tentative and indicative.

The Access to Information Policy (AIP) recognizes that transparency and accountability are essential to development effectiveness. It establishes the disclosure requirements for documents and information ADB produces or requires to be produced.

The Accountability Mechanism provides a forum where people adversely affected by ADB-assisted projects can voice and seek solutions to their problems and report alleged noncompliance of ADB's operational policies and procedures.

In preparing any country program or strategy, financing any project, or by making any designation of, or reference to, a particular territory or geographic area in this document, the Asian Development Bank does not intend to make any judgments as to the legal or other status of any territory or area.


Safeguard Documents See also: Safeguards
Safeguard documents provided at the time of project/facility approval may also be found in the list of linked documents provided with the Report and Recommendation of the President.

None currently available.


Evaluation Documents See also: Independent Evaluation

None currently available.


Related Publications

None currently available.


The Access to Information Policy (AIP) establishes the disclosure requirements for documents and information ADB produces or requires to be produced in its operations to facilitate stakeholder participation in ADB's decision-making. For more information, refer to the Safeguard Policy Statement, Operations Manual F1, and Operations Manual L3.

Requests for information may also be directed to the InfoUnit.

Tenders

No tenders for this project were found.

Contracts Awarded

No contracts awarded for this project were found

Procurement Plan

None currently available.