Country Assistance Program Evaluation for Pakistan (2007)
This country assistance program evaluation assessed the performance of the Asian Development Bank's (ADB) development assistance to Pakistan during 1985-2006, the first whole-program study of operations in Pakistan.
This Country Assistance Program Evaluation (CAPE) assessed the performance of the Asian Development Bank's (ADB) development assistance to Pakistan during 1985-2006, the first whole-program study of operations in Pakistan prepared by the Operations Evaluation Department of ADB.
Summary of findings
Pakistan has provided a very challenging context in which to implement a development program. The report outlines a number of implications of this reality for ADB operations, a number of which have not been addressed. This has contributed to less successful operations. Prior strategies have not produced a coherent and focused program of operations. As of year-end 2006, ADB had 80 loans in its portfolio. It has operated across 10 sectors and around 30 subsectors. On the other hand, there have been very few private sector operations. The CAPE placed particular attention on client perceptions of ADB performance. Clients see ADB as responsive, flexible, empathetic, and non-prescriptive in its dealings with the Government. On the other hand, clients universally disliked ADB's business processes, which are seen as slow and cumbersome. Clients see that the Pakistan Resident Mission (PRM) lacks delegated authority and is weak in sector expertise.
The performance of ADB's operations in Pakistan over the last 20 years was rated "partly successful." Performance varied by sector. Operations in the power and roads sectors were rated "successful." Those in the agriculture and natural resources; education; finance; and law, economic management, and public policy sectors were rated "partly successful" (the latter being a tentative assessment given that most operations are ongoing). The programs in support of health, nutrition and social protection; and water supply, sanitation, and waste management were rated "unsuccessful."
This evaluation provides conclusions based on its assessment of the performance of ADB development assistance in Pakistan, grouped into four interrelated categories:
- resource allocation decisions
- design of operations
- program management, and
- business processes.
The overall conclusion is that ADB has too many loans in its Pakistan portfolio across too many sectors and subsectors for the staff and TA resources it has available, and given the requirements of its business processes. These need to be brought into alignment. In addition to a strategy that contributes to a more focused program, firm strategic management is required.
- In the absence of a major increase in the number of staff assigned to support operations in Pakistan, ADB should reduce the number of sectors and subsectors in which it is involved. Four core lending and two core nonlending sectors would be appropriate, with a more focused approach within these sectors (i.e., fewer subsectors covered). Staff resources allocated to PRM need to be aligned better with the requirements of the program of this major client.
- ADB should try to increase significantly its private sector operations in Pakistan in a manner that creates synergies with its public sector operations.
- The balance needs to be adjusted between (a) lending; and (b) economic, sector, and thematic work and policy dialogue. The second element should be emphasized more to ensure that operations are underpinned by a more rigorous analysis, and that ADB becomes recognized as a leading source of ideas in its sectors of core focus.
- The preparation of ADB-funded projects in Pakistan (including the use of project preparatory technical assistance [TA]) should be reviewed with a view to moving the responsibility for project design primarily from ADB to the Government. ADB would continue to provide support from funds currently allocated for project preparatory TA, and training to executing agencies where needed.
- If ADB increases PRM staff resources significantly, greater efforts are needed to ensure compliance with the Resident Mission Policy for full involvement of resident mission staff in processing projects. Similarly, and subject to the same proviso, the delegation of project administration needs to be accelerated, and projects should be delegated as soon as possible after approval before they run into implementation problems.
- To complement zero tolerance to corruption in its projects, ADB needs to ensure that it understands the nature, extent, and drivers of corruption in each sector in which it engages. The aim is to ensure that project design and/or separate initiatives incorporate effective anticorruption measures at the sector or country level. As required under the Second Governance and Anticorruption Action Plan, this will involve conducting risk assessments and preparing risk management plans. Overcoming corruption is likely to involve greater use of country systems (particularly for procurement), as well as capacity development to implement these.
- Executive Summary
- Development Context and Government Priorities
- The Asian Development Bank's Strategy and Program
- Program Implementation
- Results Achieved, Performance Assessment, and Rating
- Conclusions and Recommendations
Note on IED's Country Evaluations and Validations
Using its 2015 Guidelines for the Preparation of Country Assistance Program Evaluations and Country Partnership Strategy Final Review Validations, the Independent Evaluation Department (IED) intends to provide an objective and informed judgement of the performance and results of country partnership strategies (CPSs), particularly in terms of their relevance, effectiveness, efficiency, sustainability and development impacts.
In ascertaining relevance, IED considers not only the alignment of the program with country needs and government objectives, but also cross-sector CPS objectives, appropriateness of modalities and sector program designs, and sufficiency of donor coordination.
The effectiveness of a country program in delivering results is also an important aspect of an IED evaluation. Primary focus is on the achievement of the outcomes and outputs of ADB interventions (and the likelihood of achievement if the program is still ongoing), as worded in CPSs and their results frameworks. These include knowledge products and institutional development efforts.
Performance and results are likewise judged from an efficiency perspective, that is, whether the program was delivered in a cost-effective and timely manner, and generated value for money. It also considers the capacity of executing agencies that may contribute to start up and implementation delays, and cost overruns.
Another critical element of IED's evaluation is the likely sustainability of results over the medium term, technically, financially, environmentally, socially, politically, and institutionally. Further, IED looks at how results led to development impacts. Specifically, whether ADB contributed to achieving the CPS objectives, directly through its sector programs and implementing cross-cutting agenda(s) across various modalities in different sectors and by various development partners.
IED gives special importance to cross-cutting objectives by considering how the cross-sectoral and thematic objectives of the CPS are articulated in the results framework and provided with appropriate indicators and targets; and how the program achieved the cross-sector thematic results. Equal weights are given to the achievement of sector and cross-cutting objectives in the relevance and development impact assessments, in both country assistance program evaluations (CAPEs) and CPS final review validations (CPSFRVs). This aligns with ADB's increasing emphasis on achieving corporate strategic priorities.
In preparing its country evaluations and validations, IED conducts document reviews, consults with concerned departments, staff, governments and other stakeholders, and undertakes evaluation missions. IED has put in place a quality assurance system to ensure consistent application of its 2015 guidelines. In CPSFRVs, IED's primary focus is to validate the evidence presented in the CPS Final Review.