Indonesia: Engineering Education Development Project [Loan 1432-INO]

Date: December 2009
Type: Evaluation Reports
Country:
Subject:
Education; Evaluation
Series: Project Performance Evaluation Reports
Project Number: 27527-013

Description

Background

This report presents the findings of the performance evaluation for Indonesia's Engineering Education Development Project, approved in February 1996, to assist the Government in improving the quality, relevance, and capacity of engineering education, while enhancing the access of economically disadvantaged but qualified students who wish to study in the seven project universities and seven polytechnic institutions located in strategic locations in the country. This evaluation aims to draw lessons from the project’s experience that can be used in formulating new education projects and in improving the implementation of ongoing education assistance, not only in Indonesia, but also in other ADB developing member countries.

The evaluation approach involved

  • a desk review of relevant project documents and reports;
  • consultations with concerned operations divisions and government agencies to solicit their views on key issues;
  • field visits to some project and nonproject universities and polytechnic institutions;
  • focus group discussions and/or interviews with major stakeholder groups (Ministry of National Education; Directorate General of Higher Education; central project implementation unit; local project implementation unit (LPIU); and university administrators, lecturers, alumni, and students);
  • field surveys of sample beneficiaries including photographs; and
  • analysis of available nationwide secondary data, supplemented by primary data collected during the field surveys.

Summary of Findings

The project is rated successful. It was highly relevant at the time of appraisal and at evaluation to the government’s and the Asian Development Bank's objectives and strategies; effective in achieving its objectives, with project outputs and outcomes sometimes exceeding targets despite the financial crisis experienced during project implementation; efficient in terms of resource utilization; and likely sustainable in terms of tangible and intangible benefits.

To improve the efficiency and sustainability of the project’s impact, outcomes, and inputs, and the evaluation of a project, the following issues need to be addressed in the near future:

Maintenance and management of project-supported facilities and equipment need to be improved and properly funded, particularly those needing repairs to ensure long term sustainability. This would require project institutions to develop other sources of revenue and/or improve cost recovery.

  • Setting up and proper accessibility of baseline data on output and outcome indicators for each university and polytechnic institution should be done prior to a project, and implementation should be regularly maintained even after project completion.
  • Coordination and consultation between the archives section of the Office of Administrative Services, resident mission, regional department, and the Independent Evaluation Department (IED) should be encouraged, as records storage needed could span several years for different kinds of IED studies. In this particular case, LPIU progress reports submitted to Indonesia Resident Mission were believed to have been disposed of, since no LPIU report was found in the archives section, and the evaluation mission had to resort to the availability of LPIU reports in the project institutions visited.

Lessons Identified

Reviewing and analyzing the information gathered, the evaluation mission identified the following lessons for future projects in higher technical education in Indonesia:

  • Monitoring and evaluation should be done not only once or twice, but should be integrated and implemented within the regular activities of the project institutions and should also be linked to the management information system of the Directorate General of Higher Education. Baseline data for specific outcome indicators like enrollment rate, pass rate or graduation rate of students, job search period, etc., should be established and continuously updated in an easily accessible database for each project institution.
  • Partnering with other educational institutions, organizations, and enterprises can improve the use of facilities and be a source of revenue to increase cost recovery.
  • Integrated management of equipment for each university and polytechnic institution, involving an informative inventory system, operation and maintenance manuals, and staff training and accountabilities, should be instituted to permit easy monitoring of its status to promote optimal use.
  • Entrepreneurship capability development would be useful as one of the future project programs due to limits in government funding for higher education institutions to be capable of developing or acquiring alternative means of funding.
  • Real-time assessments or postevaluation studies on quality of engineering education and academic standards would always need time-series trends of key education indicators. These indicators would help each university or polytechnic institution identify needed improvements for curriculum development, industry-institution linkage, staff development, and access to engineering education.
  • An effective screening method used in enhancing college access to the underprivileged and women could be institutionalized for each project university and polytechnic institution. Expansion of coverage of the scholarship scheme to far-flung areas may be encouraged including the achievement of 50:50 female and male ratio goal. In line with this, mechanics of scholarship disbursement should be properly designed to avoid delays in disbursement.

Follow-up Actions and Recommendations

Sustainability. The evaluation mission identified follow-up actions and recommendations for future projects in higher technical education in Indonesia to improve sustainability especially for equipment. An increase in the allocation for maintenance and of equipment and physical facilities at each project institution is needed. For earthquake prone areas, design of civil works should also take into account human safety concerns, and emergency funds should be allocated as needed. In addition, a development plan needs to be established for improving the quality of program curricula and staff development, as well as for improving networking with other private universities and private enterprises.

Monitoring and Evaluation. A post completion monitoring and evaluation system should be set up and maintained regularly to obtain meaningful impact assessment and identify needed improvements to sustain the project.

Contents

  • Basic Data
  • Executive Summary
  • Introduction
  • Design and Implementation
  • Performance Assessment
  • Other Assessments
  • Issues, Lessons, and Follow-up Actions
  • Appendixes