Evaluating Development Results and Poverty Reduction

Feature | 8 May 2012

Asia has been very successful in reducing the rate of poverty yet nearly 1.8 billion people in the region still live on less than $2 a day. The effectiveness of development spending is more important than ever before.

ADB.org speaks to Vinod Thomas, Director General of Independent Evaluation at ADB about a new paper which shows that "connecting the dots" is one way to get better development results.

You've studied a number of evaluations of projects - what did you discover?

We looked at many evaluations done at ADB as well as at other institutions and basically there is no silver bullet when it comes to development results. But when you compare successes and failures in projects, the one thing that leaps off the page is that where interventions have been connected across sectors, across partners and over time, you see a much better chance of getting strong results. While many projects have been rated successful, the gap is substantial in terms of what they can be.

What does it mean to "connect the dots"?

Projects aim to deliver certain outputs, such as kilometers of roads or classrooms. For some time now, ADB and others have realized that putting in the money to deliver these physical outputs is not enough. We also need to ensure there are adequate linkages with related efforts so that more roads lead to market development or more classrooms translate into learning outcomes.

Take the case of efforts to improve people's health. Investments in traditional health projects are important, but the links to water and sanitation projects could also be the key to improving health outcomes. Connecting the dots is being aware of what else is important for things to work better and if there are big gaps, how those gaps can be addressed. For such a scenario to materialize, we need to develop good connections and conversations with other players involved.

Did rural electrification improve life in Bhutan? An evaluation interviewed over 2,000 electrified and non-electrified households and found that while the project brought electricity to homes, it had not been instrumental in creating income generating activities.

Asian Development Bank's Assistance for Rural Electrification in Bhutan - Does Electrification Improve the Quality of Rural Life?

Building roads in the Pacific A recent evaluation found that for rural roads financed by ADB in the Pacific to function and be sustainable, there needs to be equally strong support from partner organizations for capacity development so the roads can be properly looked after.

Transport Sector in Pacific Developing Member Countries (1995-2010)

Why have development institutions struggled to be connected?

There are positive examples and the question would be why they aren't scaled up and why they aren't the norms. At one level there are behavioral or motivational factors: people pursue their individual areas or sectors and they are rewarded for delivering a particular product.

Evaluations have noted that the rewards systems in the multilateral organizations - including ADB and the World Bank - are shaped by the lending culture to deliver loans. Incentives are much less geared to delivering - let alone measuring - work that reduces poverty or brings about more sustainable development. This translates into a focus on delivering the road or the bridge, rather than being a contributor to bringing about real change for the poor. Keeping an eye on outcomes and changing the incentive structure can help.

What impact do you think the Paris Declaration and the Accra Agenda for Action have had?

There is undoubtedly a recognition that we need to harmonize more across partners and with country systems. It is also important to note that many people are in the game now - it used to be a dozen donors in a country and now you have 30 plus. This is all the more reason we need to have creative and effective linkages.

How can you improve the way we "connect the dots"?

I think the focus on outcomes is one way to turn this around. ADB is a leader in putting the focus on a results framework, even though it recognizes that it lags in helping to generate outcomes. The results framework that ADB has linked to decisions on budget and strategy is an innovation that was ahead of many others.

ADB Board, Management and staff as well as governments play a huge role in this. With any project, the various ministries within a government need to be connected. If we have a water and sanitation project, do we have the health professionals at the table as well? They should be because half the benefits are going to be in health.

What role do evaluators play in changing the system?

Evaluators have a responsibility to reward the extra effort that has been made to get a good outcome and not just reward outputs. But going after the outcomes can be risky, so evaluators should also recognize these risks.

It is also tempting to say: "Oh this is a successful project, let's repeat it." Or sometimes: "This has a poor rating so don't do this anymore." But sometimes the implication could be different. Expanding access to water might have worked in the past, but in the future, we might need to take a risk and focus more on ground water or coastal management or pollution control. A project in forestry may not have worked in the past, but if the need in this area is even greater going forward, why not do it better instead of pulling back?

The comment to evaluation is: are we seeing the bigger picture? Are we encouraging innovation? If we don't do that, then we are not connecting the dots either.