Implementing Impact Evaluation at ADB | Asian Development Bank

Implementing Impact Evaluation at ADB

"Impact Evaluation" provides empirical evidence regarding the outcomes or results attributable to development projects. Evidence on what works and does not work is important in improving accountability and learning from the development interventions. Before–after comparisons or comparisons between areas with and without projects often lead to false conclusions about project effects, because the differences observed are due to many other factors than the project alone. Impact evaluation offers a set of tools to separate the consequences of projects from other drivers of change, in order to understand what changes are truly attributable to projects. Impact evaluation has a variety of applications: from understanding which approach to an intervention is most effective, to ascertaining causal relationships that will effectively reduce poverty. All impact evaluation techniques are premised on the concept of a “counterfactual,” which is either identified as a randomly selected “control group” and is approximated econometrically.

Implementing impact evaluation at ADB

Although impact evaluation is a growing field, there remains little evidence about the contribution of individual infrastructure projects and programs, which receive a majority of ADB’s finance, to development goals. Moreover, there are many choices and assumptions that underpin the designs of projects in all sectors, of which few are empirically tested. More systematic measurement of project/program outcomes through impact evaluation supports evidence-based policymaking and helps development partners design and implement projects more effectively. ADB recognizes that impact evaluation will help ADB and its developing member countries (DMCs) allocate limited resources more efficiently, guide the design of more effective interventions and enable evidence-based policy.

In line with its priority on managing for development results and mainstreaming impact evaluation in the project cycle, ADB approved the following technical assistance projects—TA 7680: Implementing Impact Evaluation at ADB (approved on 7 December 2010) and Cluster TA 0012: Developing Impact Evaluation Methodologies, Approaches, and Capacities in Selected Developing Member Countries (approved on 14 January 2013).

Completed IE studies

Completed baseline

  • China, People's Republic of: Ningxia Liupanshan Poverty Reduction Rural Road Development Project

    The study developed methodology for measuring the impact of the loan project on poverty reduction in the Liupanshan target area. It involved the conduct of a baseline survey to collect comparative data before the road projects are constructed and an endline survey to analyze the project outcome and impacts, as well as necessary follow up surveys to evaluate longer term impacts.

  • Georgia: Sustainable Urban Transport Investment Program Tranche 1

  • Nepal: Raising Incomes of Small and Medium Farmers

  • Papua New Guinea: Papua New Guinea Financial Literacy Program

    The study provided a snapshot of the existing level of financial literacy, financial awareness, use of financial products, among others; and validated that on the average there is no significant difference between treatment and control wards and that any differences in the end line survey will be due to the Financial Literacy Program.

  • Sri Lanka: Eastern and North Central Provincial Road Project

    The study assessed the impact of road projects on poverty, economic activities, small and medium enterprises, access to education and educational outcomes, and mechanism by which road projects contribute to improved living conditions.

Ongoing studies

  • Bangladesh: Evaluating the Effects of Modernized Irrigation Infrastructure, Volumetric Water Pricing and Water Saving Technology on Water Use Efficiency (Baseline and midline)

    The study will evaluate the effects of modernized irrigation infrastructure, volumetric water pricing and water saving technology on water use efficiency. Outcomes to be tested include reduced periods of water deficit, improved yield and land productivity; increased water use efficiency, and enhanced equity in agricultural incomes.

  • China, People's Republic of: Baseline Survey for TVET Sector Reforms in the Guangzxi Zhuang Autonomous Region

    The proposed results-based lending program will help to strengthen the relevance of technical and vocational education and training (TVET) in Guangxi, the quality of TVET teaching and learning, and TVET's contribution to local and regional economic development. The project will also support building capacity of government for effective management of the TVET system.

  • India: Baseline Survey for Rural Road Investment Program in the States of Chhattisgarh and Madhya Pradesh

    The proposed program will connect around 800 habitations in the states of Chhattisgarh and Madhya Pradesh by constructing to all-weather standard about 7,000 kilometers of rural roads. The immediate outcome of the project will be improved connectivity of rural communities in the participating states to markets, district headquarters, and other centers of economic activity.

  • India: Enhancing Roof-top Solar Power Business Using Digital Technology

    The study will evaluate if using digital technology would enhance the productivity and profitability of Simpa Network Inc. business by analyzing if the introduction of IT (i.e., smart phones) would improve customer acquisition and retention under off-grid roof-top solar power business in rural India.

  • Mongolia: Pilot Models to Improve School Dormitory Environment and Services (Baseline survey)

    The project will support reform of the school dormitory system in Mongolia to ensure access to quality education for all students, particularly those from herder families who live a nomadic life, by establishing and implementing models to improve the physical dormitory environment and related services.

  • Nepal: Support to Collective Action for Agricultural Value Chain Development in Nepal (Endline Survey)

    The study will measure the effects of projects—Raising Incomes of Small and Medium Farmers Project (RISMFP), World Bank's Agricultural Commercialization and Trade (PACT), IFAD's High Value Agriculture Project (HVAP)—on a range of outcomes among farmer groups/cooperatives such as agricultural net revenues and household incomes/wages; participation in off-farm activities; and agricultural production practices.

  • Pakistan: Transport for Employment in Lahore, Pakistan

    Urban public transport facilitates specialization of firms and workers. Yet, despite increasing urbanization, most of Pakistan's cities suffer from a poorly connected public transport network. This limits the pool of workers that firms can attract within the city, reducing the benefits of urbanization. To help address this problem, with support from ADB and other development partners, the government of Pakistan has embarked on urban public transport investments including construction of mass transit lines in all the country's major cities.

  • Pakistan: Warehousing and Transport Services for Rice Farmers in Hafizabad district, Punjab (Endline)

    The IE will study the size and distribution of benefits from providing small- and medium-scale rice farmers in Hafizabad district, Punjab access to warehousing and transportation services. The interventions will provide them opportunity to engage in a more organized value chain that would lead to increased competition in the local economies through the elimination of middlemen or the reliance on ad-hoc markets, and better prices for farmers.

  • Papua New Guinea: Financial Literacy Program (Phase 2)

    The study will evaluate the effectiveness of financial literacy programs in supporting financial inclusions in remote parts of Papua New Guinea (PNG); how the knowledge gained has influenced financial decision making, including its effects on savings and budgeting behaviors which are likely to impact upon both personal and business activities.

  • Papua New Guinea: Road Access on Poverty and Other Social Welfare Indicators in Rural Highlands Communities

    The study will measure the impact of road development on household welfare in rural PNG over the period between 1996 and 2010, using two cross-sectional household surveys and corresponding road maps. Assessment will cover the entire highlands region with focus on rural areas where approximately 85% of the population live.

  • Philippines: DSWD Pilot Graduation Program: Baseline Data Collection

    The study will evaluate whether the existing convergence efforts, specifically of the Conditional Cash Transfer program (CCT) and Sustainable Livelihood Program (SLP) will more effectively increase the well-being of the extreme poor and make them more economically active.

  • Philippines: Employment Facilitation for Inclusive Growth

    The project focuses on impact of MyFirstJob program on young persons’ integration into the labor market. Using randomized IE, both a control and treatment group are selected through a lottery of eligible applicants to the MyFirstJob program.