Regional: Selected Evaluation Studies for 2014

Sovereign (Public) Project | 48117-001 Status: Active

This research and development technical assistance (TA) for Selected Evaluation Studies for 2014 supports the 2014-2016 program cycle, focusing on studies due in 2015 and 2016. It will fund select evaluations and related outreach activities from both the derived operational (i.e., country and sector program evaluations) and knowledge (i.e., thematic, corporate and impact evaluations, topical working papers) programs of the Independent Evaluation Department (IED). These include, but are not limited to:

Project Details

Project Officer
Van Rijn, Johannes Jacobus Frederik Independent Evaluation Department Request for information
  • Regional
  • Transport
Project Name Selected Evaluation Studies for 2014
Project Number 48117-001
Country Regional
Project Status Active
Project Type / Modality of Assistance Technical Assistance
Source of Funding / Amount
TA 8653-REG: Selected Evaluation Studies for 2014
Technical Assistance Special Fund US$ 2.50 million
Strategic Agendas Environmentally sustainable growth
Inclusive economic growth
Regional integration
Drivers of Change Governance and capacity development
Knowledge solutions
Private sector development
Sector / Subsector

Agriculture, natural resources and rural development - Agricultural policy, institutional and capacity development - Agricultural production - Agro-industry, marketing, and trade - Irrigation - Land-based natural resources management - Rural water policy, institutional and capacity development

Education - Education sector development - Education sector development - social protection initiatives - Non-formal education - Non-formal education - social protection initiatives - Pre-primary and primary - social protection initiatives - Secondary - Secondary - social protection initiatives - Technical and vocational education and training

Energy - Electricity transmission and distribution - Energy efficiency and conservation - Energy sector development and institutional reform - Large hydropower generation

Finance - Finance sector development - Inclusive finance - Infrastructure finance and investment funds - Money and capital markets - Small and medium enterprise finance and leasing - Trade finance

Health - Health insurance and subsidized health programs - Health sector development and reform - Health system development - Mother and child health care

Public sector management - Decentralization - Public administration - Public expenditure and fiscal management - Social protection initiatives

Transport - Multimodal logistics - Rail transport (non-urban) - Road transport (non-urban) - Transport policies and institutional development - Urban public transport - Urban roads and traffic management - Water transport (non-urban)

Water and other urban infrastructure and services - Other urban services - Urban flood protection - Urban policy, institutional and capacity development - Urban sanitation - Urban slum development - Urban solid waste management - Urban water supply

Gender Equity and Mainstreaming No gender elements

This research and development technical assistance (TA) for Selected Evaluation Studies for 2014 supports the 2014-2016 program cycle, focusing on studies due in 2015 and 2016. It will fund select evaluations and related outreach activities from both the derived operational (i.e., country and sector program evaluations) and knowledge (i.e., thematic, corporate and impact evaluations, topical working papers) programs of the Independent Evaluation Department (IED). These include, but are not limited to:

1. Country assistance program evaluations (CAPEs) and validations of country partnership strategy final reviews. The TA will fund at least two CAPEs (second-generation CAPEs for Papua New Guinea and the Peoples Republic of China), and validations of at least three country partnership strategy (CPS) final reviews (Bangladesh, Lao Peoples Democratic Republic, and Timor Leste). Each CAPE will evaluate ADB support to the country and cover ADB strategy, policy dialogue, completed and ongoing projects and TA, and ADB performance. It will provide ADB and the developing member country (DMC) with an evaluative basis of use for designing the future CPS. When appropriate, country sector assessments will provide inputs during the preparation of a CAPE. Each sector assessment will evaluate ADB performance within a given sector in the country, focusing again on strategy, policy dialogue, ongoing and completed projects and TA, and performance.

2. Review of Asian Development Fund (ADF) Operations. The thematic evaluation study (TES) will review the commitments made under the ADF X and XI replenishments and assess the relevance, responsiveness, and results of ADF operations over the said period. It will comprise a desk review of selected project documents, evaluation reports, and pertinent literature. DMC consultations may be undertaken in a range of countries, a perception survey held, along with country portfolio reviews covering all five regions.

3. Environmentally Sustainable Growth. The TES will review the design and implementation of ADB's 2002 Environment Policy and subsequent directional documents. It will assess the responsiveness, results, and relevance of the policy and environmental elements of Strategy 2020 (S2020), to draw out lessons for future operations. The study will initially undertake a desk review of project documents, evaluation reports, and pertinent literature. It will also take stock of prior environmental evaluations to learn from best practice of development partners. Perception surveys, country case studies of important environmental projects, and key informant interviews of regional DMC stakeholders and other development partners undertaking environment-related work will also be conducted to elicit views about ADB's achievements.

4. Effectiveness of ADB's Partnerships. The TES will evaluate the effectiveness of ADB partnerships through a review of such agreements, co-financing partnership arrangements, and related documents. It will analyze projects and TAs, and conduct in-depth interviews and a stakeholders survey. The study will cover at least 2 countries/regions (tentatively the Greater Mekong and Central Asian regions). The study will first examine the type of partnerships that ADB signed, and assess their relevance, efficiency and effectiveness. Second, it will look into various financing partnership agreements ADB is handling, administering, or participating in, on behalf of development partners (e.g., project-specific co-financing, framework arrangements, trust funds, financing partnership facilities) in terms of relevance, effectiveness, and efficiency of procedures. Third, it will look at progress made in co-financing operations as part of partnerships with other agencies, e.g., international financial institutions, commercial and private sector partners such as banks. Lastly, it will administer a survey questionnaire to resident missions and headquarters staff (country economists).

5. ADB Risk Management Practices for Non-sovereign Operations (NSOs). The TES will assess the relevance, efficiency, and effectiveness of ADB's risk management policies (e.g., exposure limits), systems and processes for NSO to identify any required enhancements that can facilitate the development effectiveness and sustainability of NSO. It is currently envisaged that the TES will have a special focus on how ADB manages the triangle of risk, reward, and development impact, in particular, how it prioritizes these objectives and handles potential trade-offs and whether it achieves an optimal mix. The review will look at ex ante strategies and policies, their application, as well as results based on a review of completed transactions. ADB risk management standards and practices will be compared to good international practices, especially those utilized by comparable multilateral financial institutions. Relevant findings of a planned internal audit of ADB's risk management and NSO pricing by the Office of Auditor General will be utilized as appropriate.

6. Impact evaluations. The TA will support at least one impact evaluation (most likely related to energy). The study will use quasi-experimental designs and emphasize validity and reliability of evaluation results. It will involve: (i) literature review, (ii) secondary data analysis and identification of counterfactuals, (iii) primary data collection (usually a large questionnaire survey), and (iv) data analyses and report preparation. The evaluation will build on data collected during the project's implementation and identify relevant comparison groups (counterfactuals) based on questionnaires, key informant interviews, and secondary data.

7. Other sector and thematic assessments, and topical working papers. The TA will support selected country, sector, and/or thematic assessments needed to feed into high-level evaluations in the IED work program. This will also include selective evaluative research on high impact topics of specific interest and relevance to DMCs.

8. Evaluation outreach, Knowledge sharing, and Partnership initiatives. The TA will support interactive knowledge-sharing events (i.e., trainings, seminars and conferences) on its completed major evaluations. These will be a venue for learning and showcasing of evaluation products; informing stakeholders of formative, summative, and prospective evaluation approaches, and findings and lessons. The learning events are to be held in selected DMCs in partnership with national think tanks, evaluation societies, public and/or nonprofit research institutions, and other partners. From each evaluation report or learning event, IED may also develop derivative communication and outreach products (such as podcasts, multimedia products, and synopses) to promote further learning. Other dissemination efforts may include preparation and publishing of syntheses, briefs, and occasional papers tailored to widen readership, and the use of new mediums for knowledge sharing (e.g., online readers, tablets) to extend the reach of IED products.

Project Rationale and Linkage to Country/Regional Strategy

The yearly work program of IED focuses on higher-level evaluations covering multiple countries, sectors, and themes to complement project evaluations. These evaluations give a more strategic perspective to ensure that ADB's directions and operations are captured and supported by more intense knowledge and outreach activities to maximize usefulness and impact. The Board of Directors of the Asian Development Bank (ADB) approved the IED's 2014-2016 work program, with effect from 11 December 2013.

ADB has made significant efforts to track and achieve the goals of Strategy 2020 (S2020). This TA will help IED fulfill its mandate, role, and functions (as guided by ADB's evaluation policy) in promoting learning and accountability by: (i) supplying evaluative feedback on the effectiveness of ADB operations to gauge the relevance and effectiveness of S2020 in several important thematic and operational areas, and (ii) facilitating the sharing and use of evaluation knowledge.

1. Results of ADF operations. IED will conduct real time evaluation on both recently completed and ongoing replenishment of the ADF, as an input to its next replenishment. Management asked IED to conduct an evaluation of ADF X and XI operations in 2015, to feed into the ADF XII replenishment process. It succeeds a 2011 thematic evaluation of ADF VIII and ADF IX (2001-2008) and their development effectiveness. This round of evaluation is of special interest given rapid development in Asia, and the likelihood of further graduations of countries out of ADF. Other areas which may be looked into include (i) use of ADF by relatively new modalities such as the multitranche financing facility and results-based lending; and (ii) the importance of addressing inequality in poor and middle income countries, and the potential need to continue ADF operations in for particular groups and areas in those countries.

2. Supporting environmentally sustainable growth. This evaluation will come more than a decade after the approval of ADB's environment policy in 2002, 7 years after S2020's first formulation of environmentally sustainable growth as a key strategic agenda, 6 years after the energy policy of 2009, 3-5 years after several infrastructure sector operational plans were approved with environmental guidance, and 2 years after the issuance of ADB's new Environment Operational Directions (2013-2020). ADB's Development Effectiveness Review series over the past five years reports a gradual increase in the number of operations with environmental themes beyond original targets, and their good performance. But still, issues of environmental degradation, waste, urbanization, deforestation, flooding, environmental protection, and climate change have become starker since the approval of S2020 in 2008.

3. Pursuing effective partnerships. S2020 identified this as a key objective in order to mobilize resources, leverage knowledge, and meet the need for highly specialized development projects, with special attention for non-core areas of operations. In past evaluations, independent evaluation has argued that ADB should ensure that its work is complementary to that of its development partners, realizes synergies, and achieves well rounded development results. All of this calls for an increasing reliance on effective partnerships. This evaluation will assess how the S2020 aspirations in this area have been operationalized, how effective they have been, and how much scope there is for untapped partnerships. Good practices in existing partnerships in both ADB and other aid agencies will be identified. Lessons and recommendations may also lead to confirmations or proposals to adjust ADB's overall programmatic approach and level of specialization in operational areas, with a view to improve development effectiveness. The review will also aim to provide insights on the costs and benefits of such arrangements and suggestions for improvement.

4. Strengthening risk management practices for NSOs. During its midterm review of S2020, ADB management stated that Private Sector Operations Department should seek investment opportunities that carry higher development returns while managing higher risks. Despite recent improvements in the risk management of NSOs, Management continues to identify the need for further strengthening ADB's risk management systems and reviewing existing country and sector exposure limits. Also, without taking equity risk (and specifically, early development equity risk), ADB will remain limited to financing already bankable projects rather than contributing to taking many more projects from concept stage to the bankable stage. This implies that to add value, ADB will have to take higher risks than private sector parties. To maintain its financial viability, ADB needs to be adequately compensated for its risks. ADB's current pricing of NSOs is directly linked to credit risk ratings, which some stakeholders consider to not always be in line with market conditions.

5. Improving evaluation utilization. Like other evaluation units of multilateral and bilateral development institutions, IED is continually challenged to optimize the use of evaluation. A well-targeted outreach to relevant decision makers can make evaluations more influential. Likewise, if evaluations are shared proactively, they will benefit from enhanced credibility and provide better incentives to act upon them. IED has made steady improvements in identifying and generating evaluation knowledge and has moved to strengthen sharing and use. If there is a good outreach plan, and resources are available, IED can efficiently and effectively communicate results to multiple audiences. Development agencies value the role of participatory communication channels for learning and new communication approaches are needed (for instance, interactive knowledge sharing and learning events that communicate specific aspects of evaluations with emphasis on interpretation, knowledge exchange, and potential application).

Impact Improved development effectiveness of ADB operations, policies, strategies, and business processes
Project Outcome
Description of Outcome Increased use of evaluation findings, lessons, and recommendations by the ADB Board of Directors and its DEC, by ADB Management and operations staff, and also by DMC agencies and their practitioners, and other interested parties in the general public
Progress Toward Outcome
Implementation Progress
Description of Project Outputs

1. Practical (country/sector/theme-specific) lessons and recommendations provided through completed evaluations

2. Knowledge sharing and Learning events (training, seminars, conferences, and other events)

3. Derivative products from the knowledge sharing events

Status of Implementation Progress (Outputs, Activities, and Issues)
Geographical Location
Summary of Environmental and Social Aspects
Environmental Aspects
Involuntary Resettlement
Indigenous Peoples
Stakeholder Communication, Participation, and Consultation
During Project Design

Selected Evaluation Studies. The evaluation process will be guided by an approved evaluation approach paper (EAP) (or in some specific cases a concept paper) which defines the intended study, i.e., key evaluation issues and questions, methods and approach, inputs and activities. The formulation of the EAP will involve interactions between IED staff and ADB operations (including interdepartmental review or commenting on the draft paper) and non-operations staff, representatives from the international evaluation community, non-government organizations, and, where appropriate, consultations with relevant study countries.

Evaluation Outreach, Knowledge Sharing, and Partnership Initiatives. With assistance from ADB's in-country Events Management Coordinator (national consultant), the host or collaborating partner in the DMC will help organize the planned knowledge sharing and learning event for policymakers, decision makers, and associated stakeholders.

During Project Implementation

Selected Evaluation Studies. The implementation of of each evaluation study will include a mixed set of (i) country visits to meet beneficiaries and affectees (including vulnerable groups), (ii) in-country meetings with DMC government staff, non-government organizations, and development partners, (iii) consultations with operations and non-operations staff at ADB headquarters and resident missions, (iv) targeted field interviews/surveys, and (iv) video conferences with relevant evaluation units, international think tanks etc.

Evaluation Outreach, Knowledge Sharing, and Partnership Initiatives. With help from ADB's in-country Events Management Coordinator (national consultant), the host or collaborating partner in the DMC will facilitate the planned knowledge sharing and learning event for policymakers, decision makers, and associated stakeholders.

Business Opportunities
Consulting Services

Evaluation studies under the TA are tentatively estimated to require about 46 and 88 person-months of international and national consulting inputs, respectively. Specific requirements for each evaluation will be reviewed and finalized during the preparation of an appropriate concept paper or evaluation approach paper (EAP).

For the learning events, in-country national consultants (i.e., events management coordinator) will be recruited to help organize, implement, and coordinate the events. Resource persons will also be engaged to serve either as panelists and moderator for every learning event.

Procurement Procurement requirements for training/seminars/conferences and other goods will be determined during the preparation of an appropriate concept paper or EAP for each major evaluation.
Responsible ADB Officer Van Rijn, Johannes Jacobus Frederik
Responsible ADB Department Independent Evaluation Department
Responsible ADB Division Office of the Director General, IED
Executing Agencies
Asian Development Bank
6 ADB Avenue,
Mandaluyong City 1550, Philippines
Concept Clearance 11 Dec 2013
Fact Finding -
Approval 21 May 2014
Last Review Mission -
Last PDS Update 18 Feb 2014

TA 8653-REG

Approval Signing Date Effectivity Date Closing
Original Revised Actual
21 May 2014 - 21 May 2014 30 Jun 2017 31 Dec 2018 -
Financing Plan/TA Utilization Cumulative Disbursements
ADB Cofinancing Counterpart Total Date Amount
Gov Beneficiaries Project Sponsor Others
2,500,000.00 0.00 0.00 0.00 0.00 0.00 2,500,000.00 21 May 2014 1,785,793.69

Project Data Sheets (PDS) contain summary information on the project or program. Because the PDS is a work in progress, some information may not be included in its initial version but will be added as it becomes available. Information about proposed projects is tentative and indicative.

The Public Communications Policy (PCP) recognizes that transparency and accountability are essential to development effectiveness. It establishes the disclosure requirements for documents and information ADB produces or requires to be produced.

The Accountability Mechanism provides a forum where people adversely affected by ADB-assisted projects can voice and seek solutions to their problems and report alleged noncompliance of ADB's operational policies and procedures.

In preparing any country program or strategy, financing any project, or by making any designation of, or reference to, a particular territory or geographic area in this document, the Asian Development Bank does not intend to make any judgments as to the legal or other status of any territory or area.

None currently available.

Safeguard Documents See also: Safeguards

Safeguard documents provided at the time of project/facility approval may also be found in the list of linked documents provided with the Report and Recommendation of the President.

None currently available.

Evaluation Documents See also: Independent Evaluation

None currently available.

Related Publications

None currently available.

The Public Communications Policy (PCP) establishes the disclosure requirements for documents and information ADB produces or requires to be produced in its operations to facilitate stakeholder participation in ADB's decision-making. For more information, refer to the Safeguard Policy Statement, Operations Manual F1, and Operations Manual L3.

Requests for information may also be directed to the InfoUnit.