The joint evaluation of the Almaty-Bishkek Regional Road Project was a pilot exercise, in support of the harmonization agenda under the 2005 Paris Declaration on Aid Effectiveness* and in line with the aspirations of the Evaluation Cooperation Group*.
The joint evaluation report, prepared by ADB's Independent Evaluation Department and the European Bank for Reconstruction and Development's Evaluation Department, presents a joint assessment of the entire project, which was cofinanced on a parallel basis by the two multilateral development banks.
The Project rehabilitated 226 kilometers of regional road between Almaty in Kazakhstan and Bishkek in the Kyrgyz Republic. In addition to improving transport efficiency between key economic centers, the Project contributed to regional cooperation between landlocked Central Asian states and dealt with border crossing facilities and nonphysical barriers to trade. These mainly related to road sector reforms; and road sector-related laws, policies, and procedures.
Summary of Findings of the Joint Evaluation Report
Overall Rating. The evaluation rated the Project "partly successful," although ADB's Independent Evaluation Department and EBRD's Evaluation Department used different rating perspectives. While the Project delivered the main targeted outputs and partly removed road transport barriers, deficiencies in project design and implementation resulted in a large cost overrun, and changes in scope reduced the overall project rating. Moreover, the uncertainty surrounding project sustainability affected the overall rating.
Relevance. The Project was assessed as "relevant." Its goal and purpose were consistent with the development strategies of the governments and the development partners. The Project is in line with ADB's Central Asia Regional Economic Cooperation program, which aims to develop an integrated and efficient transport system in support of sustainable economic growth and poverty reduction. It was found "compliant" with EBRD's transport operations policy and consistent with EBRD's strategy for Kazakhstan. However, the quality-at-entry was found to be deficient in terms of project design and imposition of unrealistic loan condition.
Effectiveness. The Project is rated "effective" in achieving its main purpose of improving the efficiency of the regional road and "satisfactory" in fulfilling its objectives. The traffic on the project road has increased with higher vehicle speeds. However, the Project's effectiveness was reduced for two reasons. First, although the Cross-Border Agreement was ratified in 2002, no evidence on its implementation was found during the evaluation. Second, the project road relies on an old bridge at Akzhol-Chu, which was not improved as part of the Project. This has resulted in traffic congestion and longer waiting time at the border crossing.
Efficiency. The Project is rated "less efficient" based on the findings of an economic analysis using a 2008 traffic count and assumptions relating to operation and maintenance, vehicle operating cost savings, and travel time savings. The uncertainty surrounding the financing (through budgetary allocations by central governments) for road maintenance has increased the risk of lowering the cost and time savings in the medium to long term. This rating could be reviewed if the allocations for road maintenance from central budgets improve in the medium term.
Sustainability. The Project is rated "less likely to be sustainable" in view of concerns relating to lack of assurance of provision of adequate allocations, resources for road maintenance, and capacity building. While the government budgetary funds provided for maintenance have increased over the years, they are not sufficient to meet the requirements. Moreover, the absence of a system to control and charge heavy goods vehicles appropriately is likely to affect road quality in the medium term. Overall, the risk relating to provision of funds for maintenance and the deployment of adequate human resources and equipment remains high.
The Project resulted in several positive socioeconomic impacts. It introduced international best practices to improve project implementation. New sources of livelihoods were opened, such as retail shops, taxi driving, car washing, roadside cafes, and hair salons. The border-crossing impact was diluted because no major change in cross-border movements resulted. The border crossing was perceived to have become more difficult due to bureaucratic procedures on the Kazakhstan side. Despite this, trade between Kazakhstan and the Kyrgyz Republic at the Akzhol-Chu border crossing point on the Almaty-Bishkek road increased by an average annual rate of 38% from 2000 to 2007. This could also be attributed to growth in the national economies. The transition impact, the main EBRD mandate focus, is rated "marginal/satisfactory." Despite the general attribution problem, the rehabilitation of the regional road can be assumed to have substantially helped increase traffic and trade figures.
Although the Project introduced international best practices to both Kazakhstan and Kyrgyz Republic, the performance of ADB-funded TA projects in both countries was rated "partly successful" due to the lack of implementation of the TA outputs. ADB could have added further value in the areas of institutional strengthening and capacity development. The delays in taking decisions relating to civil works contracts in both countries indicate that project management and institutional capacity could be improved. Implementation performance of the ministries of transport and communications in both countries was "less than satisfactory." Similarly, the performance of ADB and EBRD could have been better.
Lessons Identified by the Joint Evaluation Report
- Effectiveness of the Project was reduced due to the lack of an adequate bridge on the Chu River at the international border point at Akzhol-Chu;
- Given the limited capacity of the governments in terms of human resources and their emergence from the planned economic structure, the Project would have benefited from a simpler project design structure. Taking into account the delays in decision making as well as poor contractor performance in Kazakhstan, the use of project implementation units should have been reconsidered;
- Loans to developing member countries should have realistic conditions for loan effectiveness. Moreover, loan covenants should have full ownership from within the government to ensure implementation;
- Funding allocations for road maintenance should be based on a rigorous planning regime and be transparent. Sufficiency and continuity of funding and human resources need to be ensured. Both countries need to comply with the loan covenant for maintenance;
- Harmonizing of the project financing could be considered under the Paris Declaration to improve implementation efficiency and reduce transaction costs.
- Executive Summary
- Executive Summary
- Design and Implementation
- Assessment of Project Outputs and Performance
- Assessment of Project Outcomes and Impacts
- Issues, Lessons, and Follow-up Actions