46185-001: Developing Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries (Cluster TA) | Asian Development Bank

Regional: Developing Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries (Cluster TA)

Sovereign (Public) Project | 46185-001 Status: Active

Impact evaluation (IE) is a cross-cutting theme that is relevant in all countries, sectors, and regions. Conducting impact evaluation for strategically selected interventions will help improve relevance and effectiveness of country and regional partnership strategies. Properly implemented impact evaluations will help in the proper selection of a strategic mix of effective interventions. By implementing impact evaluations of development interventions in the Asia-Pacific region, the proposed cluster TA will contribute to the creation of knowledge on what works in development, how it works, and why.

 
Project Name Developing Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries (Cluster TA)
Project Number 46185-001
Country Regional
Project Status Active
Project Type / Modality of Assistance Technical Assistance
Source of Funding / Amount
TA 0012-REG: Developing Impact Evaluation Methodologies, Approaches and Capacities in Selected Developing Member Countries
Technical Assistance Special Fund US$ 4.50 million
Strategic Agendas Inclusive economic growth
Drivers of Change Governance and capacity development
Private sector development
Sector / Subsector

Public sector management / Economic affairs management

Gender Equity and Mainstreaming
Description

Impact evaluation (IE) is a cross-cutting theme that is relevant in all countries, sectors, and regions. Conducting impact evaluation for strategically selected interventions will help improve relevance and effectiveness of country and regional partnership strategies. Properly implemented impact evaluations will help in the proper selection of a strategic mix of effective interventions. By implementing impact evaluations of development interventions in the Asia-Pacific region, the proposed cluster TA will contribute to the creation of knowledge on what works in development, how it works, and why.

The proposed cluster TA will comprise three sub-projects with each sub-project covering 4 to 5 impact evaluation studies, or a total of 12 to 15 studies distributed over the five regional departments. By the end of the proposed TA, impact evaluation will have been mainstreamed in ADB operations, i.e., it is envisioned that impact evaluation is incorporated in the design of ADB's investment projects, especially in project preparatory technical assistance (PPTA) projects.

Sub-projects

Each subproject will include 4 to 5 impact evaluation studies and will comprise two main components, namely, (a) the conduct of IE studies applying various methodologies and survey instruments, and (b) the development of impact evaluation methodologies, approaches, and capacities in both ADB and the DMCs.

(a) Conduct of IE studies

The IE studies component will cover support to conduct of a number of IE studies while ensuring that at least one study will be supported in each of the five regions (Central and West Asia, East Asia, the Pacific, South Asia, and Southeast Asia). Activities under the individual studies may include design of the study methodology and survey instruments; conduct of baseline, mid-line and end-line surveys; and production of the study reports. The proposed TA will endeavor to support the conduct of full IE studies from beginning to end within the timeframe of the proposed TA. However, as learned from the experience of TA 7680, the TA may support key phases of individual studies; for example, as to be approved by the IE Committee, only the design and conduct of the baseline survey may be implemented under the proposed TA, while the mid-line and end-line surveys can be financed through separate initiatives. Through this component, the TA will provide direct technical support by individual and firm consultants, and ERD specialists to individual studies by advising on study design, methodology, and application in all phases of the study. This technical support will also include partnerships with renowned IE research institutes (e.g., International Initiative for Impact Evaluation or the Jameel Lateef Poverty Action Lab) to provide expertise, as well as ground-level support for DMCs in the design of IE frameworks and methodologies (e.g., visits to project sites, pre-testing of survey instruments, and other fieldwork activities).

(b) Development of impact evaluation methodologies, approaches, and capacities

The second component on development of IE-methodologies, approaches and capacities will entail a number of activities including (i) review of IE methodologies and adoption of available techniques and tools for sector and theme specific studies; (ii) conduct of pilot studies to test sector- or theme-specific techniques and tools; (iii) IE awareness raising and capacity building for DMC officials and project staff; (iv) establishment of a central IE data source containing information on project baseline, mid-line and end-line surveys (e.g., questionnaires, sampling methodology, interviewer manuals, field data, etc.); and (v) conduct of knowledge-sharing activities (i.e., regional conferences, where IE studies conducted will be presented; workshops and seminars, where experts will be invited to share methods, practices and lessons on IE) and dissemination of IE studies (publishing IE studies for distribution to ADB and government counterparts, and uploading studies in the ADB website).

Project Rationale and Linkage to Country/Regional Strategy

Bilateral and multilateral development agencies spend more than $30 billion each year on social development programs, with developing countries spending another $300 billion annually. However, the development effectiveness of this spending is not always clear, and evidence on their impact, if they are available, is scant. In particular, it is not clearly known how much of Asia's recent improvements in poverty reduction and human capital outcomes have been due to development spending and how much are trickle-down effects of the region's rapid economic growth. This lack of information limits the ability of policy makers and development partners to further improve effectiveness of the development spending through proper design of programs and projects.

ADB has supported its developing member countries (DMCs) in designing, financing, and implementing project and policy interventions aimed at fostering economic development and reducing poverty in the region. ADB's economic sector work, cost benefit analysis and past effort at benefit monitoring and evaluation, as well as design and monitoring frameworks and social safeguard requirements, collect information that provide a basis for identifying project performance ex-ante and ex-post. While these are useful exercises in designing projects and monitoring their performance, they do not necessarily identify the actual impacts of ADB's development support. Although it may not be possible to identify specific impacts of development spending at the macro level, it is possible to do so at the project level through the use of rigorous impact evaluation (IE) studies.

Thus, mainstreaming IE in ADB operations is highly beneficial not only for the Bank but also, and more importantly, for DMCs. Gaining accuracy in the evaluation of interventions will permit policymakers in DMCs and ADB to anticipate the expected consequences of these interventions. In this way, IE will allow DMCs and ADB to allocate limited resources more efficiently, guide the design of more effective interventions, and enhance evidence-based policy dialogue.

Impact Improved development effectiveness of ADB-designed projects responsive to the development needs of DMCs
Project Outcome
Description of Outcome Impact evaluation is mainstreamed in ADB operations
Progress Toward Outcome As of September 2017, a total of 14 projects in ADB incorporated impact evaluation in project design and implemented in collaboration with project executing agencies in the DMCs.
Implementation Progress
Description of Project Outputs

1. Impact evaluation studies adopting appropriate methodology with effective technical support

2. Impact evaluation awareness-raising seminars and capacity building workshops for ADB staff and DMC officials

3. Reports and dissemination workshops on lessons learned from review of methodologies and conduct of pilot studies

Status of Implementation Progress (Outputs, Activities, and Issues)

1) Fourteen impact evaluation studies are conducted, covering 8 sectors (education, energy, employment, finance, transport, water supply, social protection, and other municipal infrastructure and services) across 9 DMCs (India, Mongolia, Nepal, Pakistan, People's Republic of China, Philippines, Papua New Guinea, Sri Lanka and Viet Nam). Of the 14 studies, 3 were completed, 2 with completed baseline, and the rest are ongoing at various stages of implementation.

2a) Under Subproject 1, four conferences and workshops on impact evaluation methodologies and practices and their applications on various sectors were conducted from 2012 to 2015.

2b) Under Subproject 2, six seminars were conducted where methodologies and results of impact evaluation studies in various sectors (e.g., migration, transport and power investments, disasters and emergencies preparedness and infrastucture) were presented. Resource persons include reputable experts in their respective field.

2c) An impact evaluation guidebook, intended as a practical resource for those interested in planning impact evaluation studies, has been drafted and target for release in 2018.

3a) The Impact Evaluation (IE) portal, a repository of IE information, materials, and activities within ADB, was launched in 2017 and continuously updated and improved since then. It is intended for bankwide dissemination of IE materials (approved proposals, status of implementation, financial status, evaluation reports, survey instruments, etc.) and was made possible by a cross-departmental undertaking led by ERCD.

3b) Policy brief disseminated for one IE study (CWRD-PAK Transport for Employment in Lahore, Pakistan) titled "Overcoming barriers to women's mobility".

3c) A pilot study that analyzes the impact of the introduction of IT on customer acquisition and retention under off-grid roof-top solar power business in rural India is being implemented through close collaboration between ERCD and PSOD.

Geographical Location Regional
Summary of Environmental and Social Aspects
Environmental Aspects
Involuntary Resettlement
Indigenous Peoples
Stakeholder Communication, Participation, and Consultation
During Project Design RDs were consulted in identifying potential projects in which to do impact evaluation studies for funding under the TA.
During Project Implementation

Each study supported by the TA was/is being conducted in close collaboration with project executing agencies in the DMCs, e.g., district agricultural offices, Ministry of Education, Ministry of Transportation, etc. For instance, the Impact Evalution of the Pilot Graduation Program of SERD is being conducted in collaboration with the Department of Social Welfare and Development (DSWD) of the Philippine Government.

Established in 2013, the Impact Evaluation Committee (IEC), in which representatives from the 5 regional departments (RDs) sit as members, provides overall guidance and oversight for the cluster TA (C-TA 0012), and reviews/approves proposals (impact evaluation studies) submitted by RDs for funding under the TA. IEC meetings are conducted, as needed, to discuss relevant issues relating to the implementation of the impact evaluation studies funded under the TA.

Responsible ADB Officer Jiang, Yi
Responsible ADB Department Economic Research and Regional Cooperation Department
Responsible ADB Division Development Indicators and Policy Research Division
Executing Agencies
Asian Development Bank
6 ADB Avenue,
Mandaluyong City 1550, Philippines
Timetable
Concept Clearance 28 Aug 2012
Fact Finding -
MRM -
Approval 14 Jan 2013
Last Review Mission -
Last PDS Update 29 Sep 2017

TA 0012-REG

Financing Plan/TA Utilization Cumulative Disbursements
ADB Cofinancing Counterpart Total Date Amount
Gov Beneficiaries Project Sponsor Others
4,500,000.00 0.00 0.00 0.00 0.00 0.00 4,500,000.00 - 0.00

Project Data Sheets (PDS) contain summary information on the project or program. Because the PDS is a work in progress, some information may not be included in its initial version but will be added as it becomes available. Information about proposed projects is tentative and indicative.

The Public Communications Policy (PCP) recognizes that transparency and accountability are essential to development effectiveness. It establishes the disclosure requirements for documents and information ADB produces or requires to be produced.

The Accountability Mechanism provides a forum where people adversely affected by ADB-assisted projects can voice and seek solutions to their problems and report alleged noncompliance of ADB's operational policies and procedures.

In preparing any country program or strategy, financing any project, or by making any designation of, or reference to, a particular territory or geographic area in this document, the Asian Development Bank does not intend to make any judgments as to the legal or other status of any territory or area.

Safeguard Documents See also: Safeguards

Safeguard documents provided at the time of project/facility approval may also be found in the list of linked documents provided with the Report and Recommendation of the President.

None currently available.

Evaluation Documents See also: Independent Evaluation

None currently available.

Related Publications

None currently available.


The Public Communications Policy (PCP) establishes the disclosure requirements for documents and information ADB produces or requires to be produced in its operations to facilitate stakeholder participation in ADB's decision-making. For more information, refer to the Safeguard Policy Statement, Operations Manual F1, and Operations Manual L3.

Requests for information may also be directed to the InfoUnit.