Impact Evaluation Study of Rural Water Supply and Sanitation in Punjab, Pakistan (Urdu Translation)

Date: August 2009
Type: Evaluation Reports
Country:
Subject:
Evaluation; Water
Series: Impact Evaluation Studies

Description

This impact evaluation, an Urdu translation of the English original, assesses the performance of (i) Punjab Rural Water Supply and Sanitation (Sector) Project, and (ii) Punjab Community Water Supply and Sanitation (Sector) Project and identifies lessons for maximizing the development effectiveness of rural water supply and sanitation (WSS) interventions by conducting a rigorous impact evaluation and sustainability analysis. The evaluation aimed to quantify the impact of WSS assistance on health, education, labor force participation including hours worked by applying quasi-experimental design. The sustainability analysis focused on two key aspects of the WSS interventions:

  1. technical and physical status of subprojects, and
  2. the capacity assessment of community-based organizations (CBOs) responsible for the operation and maintenance (O&M)of infrastructure.

The study employed a mixed method approach and the findings are based on data collected from household surveys; community survey; focus group discussions; key informant interviews; technical inspection and assessment of infrastructure and surrounding environment; and knowledge, attitude and practice surveys. Both sector projects have adopted a community-driven development approach and were funded by ADB.

Summary of findings

The study shows that the projects

  1. significantly improved households' access to water supply,
  2. reduced drudgery among the lowest socioeconomic group,
  3. improved high school attendance of the girls in middle socioeconomic group, and
  4. increased leisure time for female members of the households.

Overall, the project interventions had no significant impact on primary health such as the incidence and intensity of diarrhea; significant reduction in incidence was found in the middle socioeconomic group. Similarly, at the aggregate level, the projects had no impact on labor force participation and hours worked, although disaggregated analysis showed a significant but negative impact in the middle socioeconomic group. Thus, increase in high school attendance rates either came from the withdrawal of working children from the labor force, particularly in the middle socioeconomic group, or from the reduction of time spent in fetching water. The lack of impact on labor force participation and work hours indicates that the time saved from fetching water documented in the study had not been translated into more income generation, contrary to projects' expectation.

Limited project assistance for sanitation, facilitating access to credit and improving hygiene education also had no significant impact on households. Overall, 80% of the subprojects were functional and had no problem with presence of heavy metals, but majority of them had bacteriological contamination and sanitary hazard problems, both at the source and distribution points. Similarly, only 43% of the CBOs managing these subprojects were partly or fully functional and had low functional maturity and reflected weak capacity in managing WSS systems. Similarly, while several CBOs could meet O&M costs from user charges, majority of them lacked resources for capital replacement and routine maintenance work.

Main lessons

  • Rural WSS projects significantly benefit female household members especially in reducing drudgery and increasing high school attendance.
  • The current focus of WSS projects designs needs to extend beyond improving access to water supply, and include wastewater and solid waste management; increased role of nongovernment organizations, and private sector in supporting CBOs responsible for subprojects, improving water quality, and additional provisions for improving access to WSS for the poor and other disadvantaged groups who cannot afford piped water connection to their homesteads.
  • To maximize benefits from WSS investment, ADB needs to collaborate with other development partners in the areas it does not have core strength.
  • Valid baseline data based on relevant indicators with relevant counterfactuals (comparisons) are crucial for results monitoring and evaluation and, hence, efforts are required to collect such data prior to the commencement of the projects.
  • It is important to safeguard and maximize project benefits by undertaking post-project monitoring and undertaking corrective measures on time. Such measures may include, but not limited to, assistance to bridge finance O&M in the initial years of operations even after project completion; strengthening the capacity of CBOs to address technical, managerial, and financial management issues; and strengthening linkages between CBOs and provincial, district, and local level agencies involved in WSS delivery and services.

Overall Assessment and Recommendations

Overall, ADB assistance to rural WSS projects has been rated successful, but at the low end of the scale. The following are recommended for future WSS operations:

  • Provide prominence to gender benefits from rural WSS projects.
  • Address wastewater and solid waste management concurrently with improving access to safe water in future project designs.
  • Strengthen existing collaborations and partnerships and foster new ones in WSS with other development partners in developing member countries.
  • Include baseline studies as a requirement in project designs and establish a user-friendly depository of all available baseline studies and associated databases for result monitoring, evaluation, and future project designs.
  • Follow up with relevant agencies so that required actions are taken on time to ensure the sustainability of project benefits.