Boosting Collaboration in Evaluation in Times of Crisis

Video | 12 April 2021

Evaluation Headlines is an online interview series that started in 2020 to understand how evaluators from different multilateral development banks and international financial institutions are adapting their evaluation work to remain relevant and responsive during this time of coronavirus disease (COVID-19) pandemic. This interview series is produced by the Independent Evaluation Department (IED) of the Asian Development Bank (ADB).

In this edition of Evaluation Headlines, Veronique Salze-Lozac’h, IED’s Deputy Director General, and Megan Kennedy-Chouane, Head of Evaluation Unit, Development Co-operation Directorate at the Organisation for Economic Co-operation and Development (OECD), talk about how evaluators can collaborate better to address the COVID-19 crisis. Interview by IED's Saleha Waseem.

Transcript

Saleha Waseem

Evaluation Knowledge Management, Independent Evaluation, Asian Development Bank

Hello and welcome to Evaluation Headlines, an online series that was started a year ago when the globe was gripped by the COVID-19 pandemic. It's been a year now. Development organizations have been working to contain the far-reaching consequences of COVID-19 crisis and have committed huge funds.

While these efforts are imperative, evaluators worry that the pressure for urgent action can lead to critical missteps. Evidence from past interventions during global or regional crises has shown that not all well-intended actions are effective. What is most important and perhaps most challenging is striking the right balance between quick and flexible responses on the one hand,

and coordination, quality planning, and context-sensitivity on the other hand.

At the start of this crisis, evaluators came together to share challenges and experiences and to see how they could collaborate better to address this unknown situation. This led to the creation of the COVID-19 Global Evaluation Coalition. And that is what we are going to talk about today.

I'm very pleased to have with me today Véronique Salze-Lozach, she is the Deputy Director General at Independent Evaluation Department at the Asian Development Bank. And Megan Chouane. Megan is the Head of Evaluation at OECD. Thank you both so much for

taking time out to be with us today.

Véronique, I'm going to come to you straight. What is the role of evaluators in crisis situations such as these and what is the role that IED is playing during these times?

Véronique Salze-Lozac’h

Deputy Director General, Independent Evaluation, Asian Development Bank

Thank you Saleha, and thank you for this opportunity to interact with you and Megan. The role of evaluation in times of crisis, a lot of people would say that actually it's not a good time for evaluation because it is a crisis, because there is a need for action. And one can think that evaluation will come later when the crisis is over.  But on the contrary I think there is an important role to play for evaluation in times of crisis. It's a time when MDBs like the Asian Development Bank have been very active, responding fast, and with high volumes of money. And it's important to make sure that evaluation is there to play its role. To see whether the money has been spent in the right place. That it’s reaching the beneficiary that it’s supposed to be reaching.  That it’s used in an efficient, effective way, and that it’s sustainable.  So it’s really important that the accountability is maintained. And we’re here for that.

But more than that, we are also here for the learning aspect of it. I think there is a huge role for evaluation to play in times of crisis by bringing lessons from past evaluations, of what worked, what didn’t work, and why.  So that we can inform the decision makers as they are making very fast decision and as they have to react to the crisis.  This crisis is really bringing new events on regular basis. We have to adapt on regular basis. And of course the results have to be now. It's not in two years. It’s not like building a bridge, it’s not coming in two or three years. We need results now. Countries need results now. And board members need information from independent evaluation also very rapidly to inform their decision making.

So I think that’s a role we have to play.  And IED has been playing it by providing comments to ADB's responses to the crisis, and playing its role as independent evaluation. It has also provided lessons from the past to inform the decision making.

IED has been also creating new platforms for knowledge sharing, reaching out,  making sure that there were interaction and connection  among evaluators and with broader public. And of course we have been adapting our program. We’ve been adapting our methodology to continue to implement evaluation even though we cannot travel to the field. And finally we are implementing something very innovative for an MDB which is the real time evaluation of ADB's responses to the crisis. So that we can bring early findings and early lessons for decision makers and for management, to improve development effectiveness.

Saleha Waseem

Evaluation Knowledge Management, Independent Evaluation, Asian Development Bank

Absolutely, I agree Véronique. Megan if I may come to you next. Joint evaluation enables agencies to learn from each other and to share evaluation knowledge. What, to your mind, are the other benefits of collaboration such as these?

Megan Kennedy-Chouane

Head of Evaluation Office, Organisation for Economic Co-operation and Development (OECD)

Thank you so much Saleha, it's great to be with you today. And thank you to Véronique for this conversation. I think it's very timely. So the COVID-19 Global Evaluation Coalition works like an umbrella. We've created a platform through which evaluation units can collaborate, share information, and also carry out joint work.

And what we've seen already in the early days of the coalition in 2020 is that this collaborative approach is incredibly valuable for creating efficiencies to avoid that we do duplication of work.

And also to help inspire the different members.  They can learn from one another about the evaluation approaches, how they are adjusting to COVID-19, what types of questions they are asking about the COVID-19 related support that their institutions are providing. And also to share the lessons, both procedural or methodological lessons, and also the substantive lessons that are coming out of their individual evaluation work.

What we've found is that a joint approach can really help answer some of the key critical strategic level questions that are very difficult for individual evaluation units to answer themselves. So for example, looking at questions of coordination and the coherence of different institutions collaborating at the country level. To what extent are the partners dividing up the tasks and working together in a way that creates synergies and maximizes the value for the resources each of them are providing without creating duplication or waste.

We've also managed to support the launching of several joint evaluations. And a joint evaluation, of course, is a step further in terms of the collaboration spectrum. Joint evaluation really supports shared analysis, shared evaluation processes, and ultimately mutual accountability in terms of looking at the findings of joint evaluations.

Saleha Waseem

Evaluation Knowledge Management, Independent Evaluation, Asian Development Bank

Absolutely, Megan. Véronique, if I may come to you next. IED is among the independent evaluation offices collaborating with OECD in this project. Why join forces, what is the motivation?

Véronique Salze-Lozac’h

Deputy Director General, Independent Evaluation, Asian Development Bank

Well we’ve been really supporting this initiative from the beginning. I think it’s a great initiative and we thank Megan and the secretariat for that.  Originally it was just an idea. They were really taking a chance collecting, combining everybody, and asking everybody whether there was any appetite for collaboration. And we definitely, as part of this coalition and with all of the other members of this coalition, responded positively. We thought that it was a great idea, and because it’s a time of crisis there was even more need than in normal times to collaborate,

to cooperate, and to join forces.

As Megan was saying, it’s really important in times of crisis to find synergies and to make sure that we joined efforts and forces to have better impact.  It’s also a unique platform for sharing experiences. We’re all on the same boat. All facing the same challenges. How do you implement evaluation when you can’t even go to the countries? So it was a good opportunity

also to share our issues, the challenges we’re facing, and how we were responding to these challenges.

Third, I think it’s a great opportunity also for combining lessons, collecting lessons, and sharing these evidence-based lessons from evaluation with the broader community. And finally I think it’s actually very important as a platform that we collect data, that we agree also on ways to collect data so that down the road we will be able to combine our information, to compare our findings, and to make sense out of this crisis.  In IED we are not embarked into a joint evaluation.

But as I mentioned earlier we are doing real-time evaluation. And the coalition has been very helpful and supportive because we’ve been able to bring our ideas on the table and share with others.  We have shared our approach paper. And there is right now a collective effort

in the coalition to identify some evaluation questions that all members could use so that at the end we could compare the findings.  And I think what is important is that in times of crisis, we need to adapt very fast, so the coalition helps us also see how others are adapting.

We need to be responsive. And hopefully this coalition will also help us share information and share knowledge on how to be better prepared for the next crisis.

Saleha Waseem

Evaluation Knowledge Management, Independent Evaluation, Asian Development Bank

Absolutely, Véronique. Megan, can you take us through some  concrete examples

from what has been achieved by this coalition.

Megan Kennedy-Chouane

Head of Evaluation Office, Organisation for Economic Co-operation and Development (OECD)

Yes, so in 2020 we were really focused on helping gather and communicate evidence from past evaluations to support those that are taking decisions on the COVID-19 response and recovery right now. And I know this is something that our colleagues, Véronique and other colleagues in

Asian Development Bank evaluation team are also working on.

So one example would be around food security. The strength of the coalition is really being able to draw the evidence base of all of the forty-some partners that are active in the coalition. And so we had, through our colleagues in Global Affairs Canada, an opportunity to provide a lessons from evaluation note on food security. And we worked with both the bilateral evaluation departments and the specialized United Nations agencies including FAO, WFP, and IFAD. And we drew together key evaluation lessons around food security. And we tried to really pass what would be the core messages for how to support food security in current context.

Now this evaluation note was done very rapidly. It was a rapid review. So it does have some limitations. But it also inspired both further systematic review work by some of the colleagues in the coalition, and the note was actually transmitted to a meeting of over twelve development ministers who have a regular meeting around COVID-19 response and recovery.

So that was the first, and then we had three more examples of where this real-time evidence was

able to be fed into the decision making and the discussions. And it was actually an amazing opportunity to have evaluation evidence on the table in those high-level discussions to support the COVID response.

Saleha Waseem

Evaluation Knowledge Management, Independent Evaluation, Asian Development Bank

Well, thank you so much Véronique and Megan, for taking time out to be with us today.

Véronique Salze-Lozac’h

Deputy Director General, Independent Evaluation, Asian Development Bank

Thank you very much , thank you Saleha and thank you Megan for this interaction

it was really a delight. Thank you.

Megan Kennedy-Chouane

Head of Evaluation Office, Organisation for Economic Co-operation and Development (OECD)

Thank you Saleha, it's been wonderful to speak with you today, and also with Véronique. I think this is a fantastic series and I look forward to continuing the conversations with you throughout the year.

Saleha Waseem

Evaluation Knowledge Management, Independent Evaluation, Asian Development Bank

And for those of you who've joined us online thank you so much for joining us. We'll be back again with another topic, another set of panelist. Till then, stay safe.