Home 
Publications Catalog Online Publications Document
Economic Analysis Retrospective: 2003 Update
II. Economic Analysis in 2003: An Overview
Retro 2003 reviews 63 loan proposals for public sector projects and programs (Table 1) to assess the quality of the economic analysis in these documents. The loan documents included in Retro 2003 refer to proposals that reached Management Review Meeting (MRM) stage during the period, November 2002 to December 2003. Retro 2003 also draws on reviews of PPTA reports, CSP papers, and sector studies.
The loan proposals reviewed for Retro 2003 were distributed relatively evenly among sectors. Energy, transport and communications sectors account for 27% of the total number of loan proposals. Agriculture and natural resources comprised 19%, while social infrastructure (health, education, urban development, water supply and sanitation) accounted for 27% of the total. The balance (27%) consists of loan proposals in finance, industry, multi-sector and other projects not classified elsewhere. In terms of lending modality, investment project loans (60% of the total number of loan proposals) continue to be the dominant lending instrument. Quick disbursing program loans together with sector development programs accounted for another 27% of loan proposals. Many of the issues raised in Retro 2002 arise again in Retro 2003. Retro 2002 emphasized the need for ADB staff involved in country operational programming and project processing to recognize that economic analysis goes beyond the limited perspective of rate of return estimation. It is a broader concept that includes country and sector contexts, economic rationale, market demand, and the analysis of feasible alternatives. Economic analysis from these perspectives help to assure that the right choice of intervention is made. To assess the robustness of a project’s economic viability, economic analysis must also analyze risk, financial and fiscal impact, and sustainability. Retro 2002 identified a number of areas in the project cycle where economic analysis needed strengthening. A summary is provided in Box 1. Box 1. Retro 2002 Recommendations
Retro 2002 observed that there was insufficient attention paid to establishing a sound economic rationale for a project. The link between country/sector strategies and individual projects was often weak; the need for public sector (as opposed to private sector) intervention or the role of ADB was usually not justified; and the choice of lending modality was usually not explained. Project concept papers were also often not sufficiently developed at the CSP stage. Hence, the economic rationale was established as project preparation progressed. At the technical assistance (TA) implementation and later stages, alternative approaches to addressing the specified development problem were often not investigated. This was identified as one of the weakest aspects of economic analysis. Only 30% of the proposals reviewed included specific considerations of alternatives. There were also weaknesses in identifying a counterfactual, the so-called “without project” scenario, that plays an important role in identifying and valuing project economic benefits. A related issue that was also not well established was the relationship between project investment and outcome. In many economic analyses, sensitivity analysis was limited to a mechanistic exercise that gave little information on the robustness of the project results. As a result, the analysis did not serve its key purpose, that is, to provide insight for identifying critical variables for monitoring during implementation. Project sustainability analysis also lacked systematic examination of the financial, fiscal, and institutional implications of projects. For many projects, for example, the possibility of inadequate maintenance of project facilities (especially after project completion) was usually not recognized as a risk. Retro 2002 made a number of recommendations for improving the scope and conduct of economic analysis. It was not expected that all of these recommendations would be immediately adopted because changes in approaches and processes take time to implement. Yet, as changes are under way, there is a need to assess whether changes move broadly in the right direction, and to provide clarifications accordingly. Therefore, there is a need to assess how the practice of economic analysis in 2003 incorporated and responded to the recommendations of 2002. There is also a need to highlight new initiatives and good practice to encourage learning and knowledge sharing. QUALITY ASSURANCE. Partly in response to Retro 2002, several regional departments introduced measures and processes to enhance and assure project design quality. Some measures are formal, such as Quality Assurance and Facilitation meetings introduced by South Asia Regional Department (SARD), to ensure that proposed project designs are based on sound technical, economic and financial analyses. Others are informal in the form of peer consultation or review. These are significant initiatives, signaling senior staff’s commitment to improving project quality-at-entry. The effectiveness of these initiatives will need to be assessed. As will be discussed in Section IV, Economics and Research Department (ERD) will be actively involved in working with country and project teams at the early stages of the operations cycle. ECONOMIC RATIONALE. The rationale for a project comprises three levels of inquiry: the problem diagnosis, the articulation of public sector intervention, and the role of ADB in the intervention along with choice of lending modality. The rationale for a project needs to be outlined at the project concept stage when it first enters the CSP, firmed up during TA implementation, and ultimately carried forward to the RRP stage. The rationale for projects at the CSP stage has generally been weaker than in the RRP in all sectors and across all regional departments. This may reflect a more limited understanding of key issues at the CSP stage. Lack of sufficient sector studies prior to identification of specific operations is perhaps a critical contributing factor. Better linkage to sector issues in the CSP would therefore help clarify the basis for the project’s rationale. (See Section III for detailed discussion.) Some measures were instituted to help operations staff increase their understanding of economic and technical issues and improve their capacity in problem diagnosis. This will likely help to improve their ability to formulate a sound project rationale. For example, South Asia Agriculture, Environment, and Natural Resources Division (SAAE) organized a week workshop on agribusiness subsector analysis, which provided staff with an overview of the nature of agribusiness subsector and some essential analytical tools for problem diagnosis. DEMAND ANALYSIS. Retro 2002 stressed that demand and market analyses are the cornerstones of identifying and quantifying the project benefit stream. In 2003, for example, demand forecasts for energy projects usually included high-low scenarios to test the sensitivity of demand uncertainty on project viability. There were a few good practices in the area of market and demand analysis in other sectors. FIJ: Alternative Agriculture Project included a detailed analysis of export markets—overall market prospects by commodity, market structure, marketing channels, and related infrastructure. The analysis supported identification of key bottlenecks in the marketing system that needed to be addressed, so that Fijian farmers could effectively realize the benefits of shifting to alternative farming. BAN: Second Primary Education Program presents another good practice analysis of demand-side factors affecting school attendance, particularly girls’ attendance. The analysis helped to refine a stipend scheme as part of the Program design. Thus, the Program considered both supply and demand-side measures to meet the objective of increasing school attendance. Demand and market analyses can help to understand behavioral responses to project interventions and therefore contribute to improving the optimality of project design. ALTERNATIVE ANALYSIS. Comprehensive analysis of alternatives are not routinely applied or reported in RRPs. Typically in physical infrastructure sectors (transport, power and water supply), alternatives were considered and assessed in technical terms and limited to capital investments. Policy options (vis-à-vis investment) were rarely considered. Even in the case of methodology, there is room for improvement in terms of clearly presenting technical options and applying sound analysis. For example, in the case of BHU: Rural Electrification and Network Expansion, an alternative analysis should have been carried out for electrification of villages at different distances from the power grid because diesel generation may have been a better option in cases where villages were far from the grid. Also, comparison of alternatives based on life cycle costs was inappropriate because this approach does not factor in growth in electricity demand. In the case of CAM: Greater Mekong Region Transmission, the least cost analysis compared options on the basis of marginal costs at the customer class level. This approach does not take into account that marginal cost varies by voltage level and that consumers in a customer class may take power at different voltages, especially industrial and commercial consumers. Also, marginal cost should not be looked at a single value because it varies with time of day, time of year, and geography. It would have been preferable if the conventional approach of comparing present values of the alternatives were adopted in the least cost analysis. Given the network characteristics of infrastructure operations, a specific infrastructure investment must be clearly established as part of the least-cost network development plan. This requires analysis and clear presentation of the plan at sector level and further clarification of investment options at project level. During 2003, three road sector loans were processed in India: National Highway Corridor (Sector) I, Rural Roads Sector I, and Chhattisgarh State Roads Sector Development. The road subprojects under consideration were part of the Government’s strategy to improve road infrastructure and were identified as priority projects by the government planning agency. However, most were rehabilitation efforts where alternatives were limited to a choice of engineering design standard. In the case of agriculture and social sector operations, analysis and reporting of alternatives was generally weak, although there were a few good practice cases during 2003. For example, LAO: Northern Community-Managed Irrigation presented three mutually exclusive alternatives that were compared in terms of investment, operating cost, and returns. However, analysis of alternatives need not be restricted to technical options, as in the case of TAJ: Education Sector Reform. A discussion of lending modality can provide insights into different approaches for addressing a specific development issue. As Tajikistan had just begun to stabilize from a protracted civil war, the environment was not deemed yet appropriate for a program or sector development program loan. There was still a need to establish a better understanding of the sector, develop an initial information base and institutional capacity, and coordinate donors prior to embarking on more comprehensive policy reforms. A project loan was therefore selected. Alternative analysis in policy-based lending operations still needs to clearly specify feasible policy options, while considering technical and political factors. The analysis needs to consider the costs of adjustment of the competing policy options and their impact on the medium term fiscal framework. Also, as discussed in Retro 2002, a program under consideration may be one of a sequence of operations to support an overall reform agenda, and the focus of the alternative analysis should also be one of understanding what was achieved so far and what will be the impact of subsequent phases. However, this approach is rarely considered. Overall, the analysis of alternatives needs significant improvement. Retro 2002 presents broad, sector wise outlines of what may be addressed in the analysis of alternatives. Further assessment is provided in Section III. VALUING COSTS AND BENEFITS. Retro 2002 recommended that program economists prepare and update country estimates of key price variables such as shadow exchange rates and shadow wage rates. These estimates are important to both project analysis and policy analysis. To date, program economists for Azerbaijan and Mongolia have done so using the abbreviated methods. For policy analysis, estimates using this method are inadequate and a more sophisticated analysis is required, such as the one found in the Guidelines for the Economic Analysis of Projects (February 1997). Nevertheless, the initiative on the part of these two economists is recognized and there is a need to encourage more program economists to engage in shadow price analysis as part of their economic and sector work. On the other hand, there have been several instances of project economists using unsubstantiated and unverified shadow exchange rates and shadow wage rates. In these instances, the project economist used a shadow exchange rate and shadow wage rate from other RRPs for the same developing member country (DMC) on the basis that the RRP was recent and assuming that the values of these parameters are also recent. It is often the case that the same approach was used in the earlier RRP and thus it is possible that the parameter values are out of date. Therefore, it is important that shadow exchange rate and shadow wage rate parameters be recalculated each year. In another instance, three agriculture and rural development projects were processed in Mekong Department (MKRD), but it was found that the methodologies for market price analysis were not consistently applied across these projects. With the assistance of EREA staff, Mekong Agriculture, Environment and Natural Resources Division (MKAE) prepared a staff study to develop a consistent database for commodity market prices for the Greater Mekong Subregion for future use. SOME INNOVATIONS. Retro 2002 noted initiatives where project teams tried different approaches for analyzing project distribution impacts and assessing project sustainability. Many such initiatives continued in 2003 and new ideas were experimented with. A few such initiatives could be considered for wider application. PRC: Fujian Soil Conservation Project applied a qualitative method of poverty impact assessment to identify channels of effect and further develop measures for project-level poverty monitoring. VIE: Central Highland Health Project used a benefit incidence technique and household survey data to assess ex ante how project interventions will result in more equitable access to the provision of health services and more equitable distribution of project benefits. PAK: Punjab Public Resource Management Program was the first policy operation that built its fiscal implication analysis through a Medium Term Fiscal Framework (MTFF). The MTFF presented a clear and transparent account of the total resource envelope for province-wide policy reforms. Indeed, the introduction of MTFF was part of the policy recommendation under the Program. During 2002 and 2003, EREA staff continued to build awareness among regional department staff of the need to develop a broader perspective of economic analysis. Nevertheless, the transition from simple awareness to effective application of the principles of economic analysis seems to be still far off. There are challenges for ensuring that country and sector studies are linked closely to project-level analysis. There are also challenges to foster more effective teamwork among staff of different disciplines and work units. As noted in the Retro 2002, much of the effort for quality improvement needs to come from attention and interest of senior staff in the regional departments.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© 2008 Asian Development Bank Privacy | Terms of Use |
 Top of page |
Back