 | |
 |
Table of Contents |
 |
|
|
Managing Knowledge
In June 2004, ADB prepared a knowledge management framework to guide its transition to a knowledge-based organization.23 The framework defines knowledge as understanding the why, what, how, who, when, and where relative to taking action. It reviews
practices with reference to formal knowledge products and services and significant but often unrecognized knowledge byproducts. It highlights initiatives to enhance the capacity to process knowledge, analyzes challenges and opportunities, and specifies the
goal and objectives of knowledge management, guiding principles, expected outputs, and associated action plans.
 |
As regards operations evaluation, Knowledge Management Framework (2004)
identifies a need to improve evaluation of knowledge products and services. This will
enable ADB to capture lessons, incorporate them into new and ongoing products, and
enhance effectiveness. This entails development of a more systematic approach to, and
consistent methodology for, evaluating knowledge products and services.
OED conducts evaluations to find out what results are being achieved, what
improvements should be considered, and what is being learned about policies, strategies,
programs, and projects, including their design, implementation, and results.24
Producing credible, timely, and objective data, information, and knowledge that describe
ADB’s organizational performance promotes development impact if what is learned
informs decision making. Sharing lessons also demonstrates good governance and
advances understanding of what ADB aims to accomplish, thereby generating support
for it.25 Figure 3 illustrates the principal audiences for evaluations.
Lessons are of two types: operational and developmental. Operational lessons
have a managerial and administrative component offering ideas for the establishment
of an environment enabling identification and ownership of good practices. They relate, among others, to performance measurement, aid coordination, resource requirements,
team building and coordination, procurement practices, delivery and reporting systems,
and logistics. Developmental lessons pertain to realization of development results,
improvement of developmental practice, and delivery on priorities.
Evaluation reports that are simply placed on a shelf provide no return on investment.
The highest value can be realized only when what is learned from evaluation
impacts decision making and improves practice in a relevant, effective, efficient, and
sustainable manner. Specifically, what key audiences, both inside ADB and outside it,
can gain from lessons creates opportunities to
- identify shortcomings in policies, strategies, programs, projects, and associated processes, methods, and techniques;
- set out remedial courses of action to address issues and problems;
- increase institutional learning that builds capacity to manage for development results and development effectiveness; and
- inform key audiences about how ADB is performing.
Each benefit from evaluation is defined by what value the evaluation offers, the
strategies developed for sharing results, and the manner in which the results are used.
Building value means staying committed and focused. Evaluation is both an opportunity
to contribute to the improvement of development activities and a process to forge
findings and recommendations. The results that are then shared should
- improve developmental practices,
- enhance institutional learning,
- validate hypotheses, and
- facilitate identification of issues and resolution of problems.
To accomplish this, it is necessary to think of the broader picture, focus on results, maintain flexibility, keep messages clear and simple, and extend them in timely fashion and the right format with appropriate dissemination techniques.
Knowledge must not be seen as something that is supplied from one person to another, or from better-off countries to developing countries, but rather as something that can flow back and forth and be continually improved, adapted, and refreshed. Knowledge management tools can be harnessed in support of this. The advent of the Internet has brought information technologies that facilitate this practice. They include e-learning, web conferencing, collaborative software, content management systems, and email lists. There are also organizational enablers
including knowledge audits, communities of practice, action reviews, peer assists, information taxonomies, coaching, and mentoring. Each expands the level of inquiry and provides a platform to achieve a specific purpose or action.
ADB’s annual lending volume is typically about $6 billion, with technical assistance totaling
about $180 million a year. Therefore, the return on investment in lesson learning for operational and developmental impact is likely to be high, and maximizing it is a legitimate concern. In late 2006, OED established the Knowledge Management Unit. This initiative is expected to
- promote client awareness among staff of OED,
- accelerate dissemination of findings and recommendations to key audiences, and
- increase learning and exchanges in OED and ADB.
Improvements are being made to OED’s website. Invitations are being sent to link to it. Evaluation reports are being recycled by compiling summaries. However, knowledge management also requires proficiency in areas of competence other than knowledge sharing and learning. The other the priority areas that OED must concentrate on are strategy development, management techniques, collaboration mechanisms,
and knowledge capture and storage.
Knowledge management applied to lesson learning is at its genesis in ADB. In 2006, improvements
were made that hold promise not only in OED but, more importantly, vis-à-vis its interface with
other ADB departments and offices, DMCs, and the international evaluation community. In the medium term, OED must continue to improve the organizational culture, management system, business processes, information technology solutions, community of practice, and external relations and networking for lesson learning. It must connect better to ADB’s knowledge management framework. For this, a plan is needed to build on recent achievements and create more value with measurable results against each interface. Learning Lessons in ADB: Strategic Framework, 2007–2009 26 sets the stage for regular annual knowledge audits for systematic identification and analysis of knowledge needs, products and services, gaps, flows, uses, and users from the perspective of learning lessons. It also permits formulation of annual business plans to deliver outputs against
each interface. As a result, OED will be better placed to accomplish its mission.
|
The Growing Importance of the Internet
The Internet has already surpassed newspapers and Yellow
Pages as a vehicle for knowledge products and services.
As the Internet becomes more popular, the importance of
a successfully established and managed Internet presence
is self-evident. In 2006, OED made efforts to increase the
dissemination of the findings and recommendations from
evaluation using the web and introduced hyperlinks and
meta tags in documents. It began to contribute some of its
knowledge products and services to portals of development
information such as Development Gateway, Eldis, and Livelihoods
Connect. In 2006, OED improved its website to ensure
that all knowledge products and services are accessible to key
audiences in reader-friendly ways. In 2007, OED will increase
its Internet presence with an emphasis on:
- website content,
- navigational structure,
- website and identity appearance,
- website load time, and
- cross-media accessibility.
OED’s website is available at http://www.adb.org/evaluation/default.asp
|
|
Signs of Improvements in Evaluation-Based Learning
Concepts of knowledge management borrowed from the corporate sector
have made inroads into development agency thinking. They can also be applied
in an evaluation setting:
- Concentration. ADB’s strategic planning frameworks are moving towards
greater concentration on country, sector, and thematic areas. They also
display a parallel trend towards greater decentralization.
- Learning. Increasingly, ADB sees itself as a learning organization. It is
restructuring its management information systems accordingly. Operations
evaluation is expected to play a key part in this; it has been given greater
institutional independence and is being given greater priority.
- Quality of evaluation. More sophisticated methods, more impact orientation,
and a greater number of broader based evaluations suggest that
in the future more knowledge will be generated that is needed for both
quality improvement and conceptual advancement of development, and
for greater external accountability.
- Feedback. ADB must increase the extent to which it systematizes and
institutionalizes its feedback system for evaluation-based learning and
accountability.
- Information and documentation. ADB is making increasing use of
modern, Internet-based information and documentation systems. These cut
information gathering and search costs and strengthen institutional memory.
However, the functionality of the systems leaves room for improvement.
In particular, efforts are needed to manage the quickening flood of data
and information. Another problem is how to network the systems and
how to cater for upward feedback in a decentralized organization.
- Internalization. ADB must do more to promote internalization of lessons from evaluation, taking a more systematic and innovative approach. OED is currently appraising the inclusion of this in an overall strategic framework for knowledge management.
- Monitoring. Implementation monitoring, i.e., implementation of findings
and recommendations from evaluation, should be based on established
monitoring and evaluation systems.
- Disclosing. By publicly releasing position papers at early stages of evaluations and posting on OED’s website the comments of external stakeholders,
OED has advanced ADB’s public disclosure policy.
- Partners and stakeholders. ADB has, in the past, largely addressed its feedback to internal audiences. The need for greater participation of incountry
partners and stakeholders, including the media, in the feedback process is increasingly recognized.
- Broad-based evaluations. Moving from project evaluations to a higher level country, sector, and thematic focus is an important way of increasing potential evaluation impact, especially when evaluations are timed to coincide with policy reviews. Given relatively well-developed monitoring and evaluation systems, feeding findings and recommendations from evaluation into ongoing projects is comparatively straightforward. In contrast to this, using the lessons learned from broader based evaluations presents more challenges.
|
____________________
23 ADB. 2004. Knowledge Management in ADB. Manila.
24 Evaluations can be formative or summative. Formative evaluations are undertaken early to understand what is being achieved and identify how that can be improved. Summative evaluations are conducted during implementation or ex-post to assess effectiveness and determine results and overall value. Evaluations can be categorized by focus or in terms of the stage in the life cycle of the operation being evaluated.
25 2006. Operations Evaluation Department: Knowledge Management Team—Proposed Work Plan, 2006. ADB, Manila; 2006. Establishment of a Knowledge Management Unit in OED. ADB, Manila.
26 ADB. 2007. Learning Lessons in ADB: Strategic Framework, 2007–2009. Manila.
 Back
Influential Evaluations | Next
Conclusions |
|
|
Topics 
Top of page
