Country Assistance Program Evaluation for Pakistan | Asian Development Bank

Country Assistance Program Evaluation for Pakistan

Evaluation Document | 12 November 2013

This evaluation assesses the performance of ADB operations in Pakistan, and provides lessons and recommendations for ADB's new country partnership strategy (CPS), scheduled for approval in 2014.

This second independent evaluation of the ADB program in Pakistan (the first was in 2007) covers operations and policy dialogue during 2002-2012. The focus is on the results of two country strategies, one covering 2002-2006 and the other 2009-2013. One hundred and sixteen loans and grants, totalling $11.5 billion, are assessed, including 10 private sector loans. The evaluation also considers the major portfolio restructuring in 2007-2009.

Previous work has identified an unusually high number of projects and programs in Pakistan that were rated less than successful or unsuccessful. While the evaluation tries to delineate the causes, the answers remain in part elusive. The program's performance was in the lower mid-range of ADB's client countries before the evaluation period. This reflects the rather difficult country context for operations, with special challenges from a weak fiscal and governance position, problematic federal-provincial-local government relations, and a large gap between ADB procurement and safeguards practices and those of the country.

The evaluation points to a confluence of factors, from a sharp rise in lending, particularly program lending at the start of the period, leading to some weak lending choices and program design; to government fatigue with the conditionality of program lending; to policy reversals that reduced outcomes of several program loans; and to the difficult security situation over the decade, which led to cancellations of components in inaccessible areas.

The Pakistan program may also have been affected more than others by the reorganizations in ADB in 2002 and 2006 and concomitant staff changes. ADB's resident mission in Pakistan was too small for the many loans in the first half of the decade. Changes in strategies over the years meant that there was an increased focus on particular types of programs. This may have hurt continuity. Rightful and gradually increasing attention to safeguard compliance in the infrastructure portfolio may have led to slower implementation and some cancellations. ADB's support for International Monetary Fund (IMF) programs led to some ADB-supported reform programs being discontinued, as the 2009 IMF standby credit agreement went off-track in 2010.

The evaluation presents a number of positives and more recent operational improvements. It finds that ADB's country strategies matched Pakistan's priorities well, and that ADB has retained the trust of the government. ADB is furthermore regarded as a lead partner in energy and to some extent in roads, especially in highways and regional corridors. ADB has done relatively well in responding to some natural disasters, and in private sector operations. Due to a more manageable portfolio in 2013, and the recent democratic election of a government with a clear mandate, the outlook should be positive. As reflected in this evaluation, the program in Pakistan going forward can be closely aligned with ADB's support for inclusive and environmentally sustainable growth.

Contents 

  • Introduction
  • Country Context
  • Country Strategies and Program
  • Program Implementation
  • Assessment of the Program
  • Lessons and Recommendations
  • Appendixes

Note on IED's Country Evaluations and Validations

Using its 2015 Guidelines for the Preparation of Country Assistance Program Evaluations and Country Partnership Strategy Final Review Validations, the Independent Evaluation Department (IED) intends to provide an objective and informed judgement of the performance and results of country partnership strategies (CPSs), particularly in terms of their relevance, effectiveness, efficiency, sustainability and development impacts.

In ascertaining relevance, IED considers not only the alignment of the program with country needs and government objectives, but also cross-sector CPS objectives, appropriateness of modalities and sector program designs, and sufficiency of donor coordination.

The effectiveness of a country program in delivering results is also an important aspect of an IED evaluation. Primary focus is on the achievement of the outcomes and outputs of ADB interventions (and the likelihood of achievement if the program is still ongoing), as worded in CPSs and their results frameworks. These include knowledge products and institutional development efforts.

Performance and results are likewise judged from an efficiency perspective, that is, whether the program was delivered in a cost-effective and timely manner, and generated value for money. It also considers the capacity of executing agencies that may contribute to start up and implementation delays, and cost overruns.

Another critical element of IED's evaluation is the likely sustainability of results over the medium term, technically, financially, environmentally, socially, politically, and institutionally. Further, IED looks at how results led to development impacts. Specifically, whether ADB contributed to achieving the CPS objectives, directly through its sector programs and implementing cross-cutting agenda(s) across various modalities in different sectors and by various development partners.

IED gives special importance to cross-cutting objectives by considering how the cross-sectoral and thematic objectives of the CPS are articulated in the results framework and provided with appropriate indicators and targets; and how the program achieved the cross-sector thematic results. Equal weights are given to the achievement of sector and cross-cutting objectives in the relevance and development impact assessments, in both country assistance program evaluations (CAPEs) and CPS final review validations (CPSFRVs). This aligns with ADB's increasing emphasis on achieving corporate strategic priorities.

In preparing its country evaluations and validations, IED conducts document reviews, consults with concerned departments, staff, governments and other stakeholders, and undertakes evaluation missions. IED has put in place a quality assurance system to ensure consistent application of its 2015 guidelines. In CPSFRVs, IED's primary focus is to validate the evidence presented in the CPS Final Review.