Project Performance and the Project Cycle

Date: September 2008
Type: Evaluation Reports
Subject:
ADB administration and governance; Evaluation
Series: Special Evaluation Studies

Description

OED conducted this study to better understand why problems of project performance and project quality continue to occur in ADB projects despite a general trend to higher numbers of projects being rated successful or better over the past few years. This evaluation was also a response to the recommendation in the 2006 Annual Evaluation Review that OED identify factors that influence project success and, ultimately, country outcomes.

The evaluation was designed to look at project performance within a structured framework-the sequence referred to as the project cycle. Looking at the problems of project performance and project quality in the context of specific activities undertaken during various stages of the project cycle, as well as the individuals and agencies responsible for undertaking them, shed new light on why about 30% of ADB's project portfolio is rated less than successful during the evaluation phase.

The evaluation was implemented through three distinct steps:

  • a retrospective analysis based on available documentation, consisting of an in-depth review of all 75 project completion reports for projects rated less than successful for the period 2002-2006 inclusive, and a further review of 40 projects rated successful during the same period;
  • a real-time analysis based on questionnaires and interviews with key informants in Indonesia, Philippines, and Sri Lanka, third-party individuals such as consultants, and ADB staff; and
  • a synthesis of the first two steps based on the findings of both analyses and subsequent verification by reference to published data.

This evaluation is one of the many initiatives taken by ADB to improve project performance. The distinctive feature of this report is that the analysis is based on "internal" feedback from operational staff, and is in line with ADB's new emphasis on improved corporate knowledge management for enhanced development effectiveness. Many suggestions ensue directly or indirectly from the analysis in the report. These should be usefully considered for further studies and possible implementation.

Issues and recommendations

  Issue Recommendation
Overloading the Project Team Leader The project team leader is a key figure during project identification, preparation, approval, and implementation. As the key individual involved in formulating projects, supervising project preparation technical assistance projects, preparing the report and recommendation of the President, and shepherding project proposals through the internal review process, the responsibilities of the project team leader have always been challenging. However, over time the role and duties of the project team leader have grown to such an extent that this may have a negative effect on project quality and future project performance. The responsibilities are now well beyond the technical capabilities of any one individual. The current system of teamwork may not be mitigating the project team leader's burden. Project teams have been less than effective in adequately supporting the project team leaders. Management should continue to explore modalities for strengthening team work (including through appropriate incentives) to have better burden sharing in teams.
Budgetary Support not Commensurate with Expanding Project Preparatory Technical Assistance Scope The scope of PPTAs has been expanding over time to sometimes include even preparation of the draft report and recommendation of the President. However, PPTA budgets have not changed as much. Management's initiative in the Work Plan and Budget Framework 2006-2008 to increase PPTA to 40% of the TA portfolio was very appropriate. However, the actual figures for 2006 and 2007 do not reflect this intention. The TA budget allocated to PPTA went up marginally to 25% in 2006 (from 22% in 2005) but dropped to 17% in 2007, the lowest level in the past 11 years. Hence, Management's well-intended goal has yet to be achieved. Management should review PPTA funding requirements in light of the current and future scope of PPTAs and ensure the necessary PPTA funds.
Need for Greater Project Supervision to Avoid Implementation Delays Adequate supervision of ADB projects during the project implementation phase is very important in influencing ultimate project success. However, the number of professional staff days devoted to implementation review missions dropped by 36% between 1998 and 2005. This drop may be compensated to some extent by the implementation functions undertaken by national staff in resident missions for delegated projects which account for about a third of all active projects. Nevertheless, implementation delays have not reduced over the years. Current resource allocation for project implementation supervision should be reexamined for adequacy. Start-up delay, which is a common feature for most projects, should receive special attention, and adequate resources should be made available including thorough flexible use of TA and loan funds and/or greater internal human resource allocation.

Contents

  • Executive Summary
  • I. Introduction and Background
  • II. Methodology
  • III. Retrospective Analysis—Factors Affecting Project Performance
  • IV. Real-Time Analysis—Overview of the Key Stages in the Project Cycle and their Link to Project Performance
  • V. Summary of Retrospective and Real-Time Analyses
  • VI. Some Key Issues and Recommendations
  • Appendixes
  • Supplementary Appendixes