|
|
Appendix 2: Results Management at the Project Level
The design and monitoring framework. Fundamentally, managing for results
begins at the project level. Achieving results at this level requires
a systematic approach to design, implementation, and evaluation. The most
common approach is to use a design and monitoring framework (logical framework)
to structure project planning. That framework should communicate essential
information about the project to stakeholders in an efficient, easy-to-read
format. More specifically, the framework should make the following immediately
clear:
- how the project will achieve results by converting inputs into a defined
set of outputs that are expected to achieve a desired development result
or outcome;
- which time-bound and quantifiable indicators and targets will be
used to monitor and evaluate the project;
- what risks may adversely affect achieving results, and what appropriate
measures are there to mitigate those risks;
- the specific assumptions that must remain valid if the project is
to succeed.
This appendix presents a general project-planning framework based on
the Asian Development Bank’s (ADB) design and monitoring framework;14
however, the principles are generally applicable to project design and
management in any organizational setting.
Template: The design
and monitoring framework (DMF). As shown in Figure A2.1, the
DMF is a 14-box matrix that answers a number of key questions.
- Why do we want to do this project in the first place (impact)?
- What is the project going to accomplish (outcome)?
- What is the scope of the project (outputs) and what key activities
need to be carried out (activities)?
- What resources (inputs) are required?
- What are the potential problems (risks) that may
affect the success of the project?
- What are the fundamental assumptions that the project design is
based on?
- How do we measure (performance indicators) and
verify (data sources) that have been successful?
Key role of participation. The participation of stakeholders
is essential from the earliest phases of project design. Groups generally
make better decisions than individuals, and broad participation allows
diverse perspectives to be taken into account. At the same time, the act
of participating, contributing, and seeing inputs translated into elements
of project design builds commitment to and ownership of the project. Participatory
processes in project planning are part of, both the situation
analysis and project identification phases.
Situation Analysis
Projects are generally intended to address a specific development problem,
often related to alleviating constraints in access to public services.
Early on, it is essential to reach a consensus that the current situation
is unsatisfactory and that changes need to be made. After that general
consensus with stakeholders is reached, a full situation analysis needs
to be conducted. The two main components of this are stakeholder
analysis and problem analysis.
Stakeholder analysis. Key stakeholders should be carefully
analyzed using a format like that in Figure A2.2.
Initially, it is essential to clarify which people and organizations
are involved in and/or affected by the development problem. Which groups
are supportive and which groups are opposed? Regardless of whether a group
supports the project or not, it should be involved in identifying and
understanding the development problem. In most cases, this participation
takes place in a workshop setting with the dialogue facilitated by a consultant,
a member of the project team, or one of the participants (see Box A2.1).
| Box A2.1: Tips for Effective Stakeholder Analysis |
| Use a series of focus group meetings and facilitated workshops to
deepen understanding of the development problem and more clearly identify
the way forward. |
| Define group categories flexibly. The definition may
be narrow or broad, depending on the situation. |
| Ensure that you have all fundamental information on key stakeholders. |
| Conduct an in-depth analysis of key stakeholders. |
| Update stakeholder analysis during project implementation. |
Problem analysis. After conducting the stakeholder analysis,
the focus shifts to a comprehensive analysis of the development problem.
The primary tool is the problem tree (see Box A2.2) that does the following.
- analyzes the context and situation of the problem;
- identifies various problems associated with the developed problem;
- provides a visual diagram of cause-and-effect relationships (see Figure
A2.3).
In most cases, there is already an initial/general understanding of the
nature of the problem, whether from sector or thematic assessments, monitoring
of development outcomes, or identification of specific performance problems.
Nevertheless, the problem tree analysis adds significant value in that
its participatory development builds on the initial diagnosis and clarifies
the true nature of the problem.
| Box A2.2: The Seven Steps in the Problem Tree Analysis |
| Step 1: Discuss the situation and underlying problems
and identify the development problem with stakeholders. Clearly state
the development problem as a negative situation and pin it in the
center of the board. |
| Step 2: Write the problems that are
the direct causes of the development problem on cards and place them
in a horizontal line under the development problem. |
| Step 3: Repeat step 2 by determining the direct
causes for each of these problems and placing them on cards in a horizontal
line under each problem card. Broaden the problem tree as you work
downwards until you reach very specific root causes. |
| Step 4: Identify the direct effects
of the development problem. Write them on cards and place them in
a horizontal line above the core problem. |
| Step 5: Continue to work upward by placing the
perceived effects of the level below on a horizontal line above. |
| Step 6: Review the problem tree and
analyze the interrelationship of problems at different levels. Move
the cards around to reflect the evolving consensus. |
Another useful tool for brainstorming is the mind map. Begin your mind
map by writing down a central idea. Then think freely—non-lineary—and
write down new and related ideas radiating out from the center of the
map. Focus on key ideas, then seek out the branches and connections between
ideas. The mind map helps you “tease out” hidden connections
and identify previously unknown cause-and-effect relationships (see Figure
A2.4).
Summary of problem analysis. The problem tree represents
an evolving stakeholder consensus and is likely to be revised and refined
even during project implementation. It is important to remember the following.
- Problem statements may need verification (i.e. do the problem statements
still reflect what is actually happening now).
- Cause-effect links may need ongoing verification.
- A second or third development problem may need to be fully analyzed
before a comprehensive picture emerges.
- Other stakeholders may need to be consulted as new issues are uncovered
during the analysis.
 Top
Project Identification
After you have a clear understanding of the development problem and the
underlying causes, you need to specify the desired future situation. This
may be referred to as the project identification phase and involves analyzing
objectives and alternatives.
Analyzing objectives. The problems identified in the
problem tree must now be transformed into objectives, i.e., future solutions
of the problems. The analysis of objectives describes the desired situation
after the problems have been resolved; the key analytic tool is the objective
tree (see Figure A2.5 and Box A2.3).
| Box A2.3: Analyzing Objectives |
| Step 1: Reformulate the development problem at
the highest level of the problem tree into a desired condition, i.e.,
into an objective. Objectives should be realistic and achievable. |
| Step 2: Write the potentially direct
means for achieving the development objective as statements on cards
and place them in a horizontal line under the core objective. |
| Step 3: Revise the statements as required and add
new statements as relevant and necessary to achieve the stated objective. |
| Step 4: Repeat step 2 by determining
the means for achieving each of the objectives and place them on cards
in a horizontal line under each statement card. You may have more
objectives than you have problems in the problem tree. |
| Step 5: The space above the development objective
is for direct objectives. Examine each statement and convert it into
a positive, desirable objective. |
| Step 6: Repeat step 5 by determining
the direct objective for the statements listed below. |
| Step 7: Review the objectives, checking that all
means-ends relationships are valid and that none are missing. Complete
the tree by connecting the cards with lines. |
Analyzing alternatives. This analysis (see Box A2.4)
is used to do the following:
- identify alternative means of achieving the development objective;
- assess the feasibility of each alternative;
- reach consensus on the project strategy.
| Box A2.4: Analyzing Alternatives |
| Step 1: Confirm the desired objective. This then
becomes the one and only outcome of the proposed project. Only one
outcome should be identified for each project. |
| Step 2: Identify sets of objectives
from the objective tree that represent alternative strategies to achieve
the desired situation. |
| Step 3: Present and discuss potential alternatives
with appropriate stakeholders. Make sure that each group understands
the implications of each alternative for them, i.e., if it will affect
them positively or negatively. |
| Step 4: Confirm selection criteria,
i.e. economic, financial, socioeconomic, environmental, technical,
and institutional. |
| Step 5: Based on the decision made during step
4, carry out necessary assessments, analyses, and feasibility studies.
|
| Step 6: Decide on the most appropriate
strategies for implementing the proposed project. The final decision
should be consensual to ensure ownership and to maximize the probability
of achieving desired results. |
Top
Using the Design and Monitoring Framework
The design and monitoring framework (DMF) is a simple but powerful design
and management tool. It helps build consensus with stakeholders and creates
ownership of the proposed project. It organizes thinking and relates activities
to expected results. It provides a structure for monitoring and evaluation
where planned and actual results can be compared.
The DMF matrix, similar to the logical frameworks employed by multilateral
and bilateral aid agencies, is made up of 14 frames with 4 major headings/columns.
The design summary outlines the main elements of the project and shows
the vertical logic of the DMF—the means-ends relationship or the
results chain. The vertical logic tests the soundness of the results chain
by checking if the inputs are sufficient to carry out the activities that
will produce the outputs. In turn, outputs are expected to achieve the
desired outcome at the completion of the project. This will contribute
towards achieving desired impact. The results chain is derived from the
objective tree and the analysis of alternatives (see Figure A2.6).
Top
Performance Targets and Indicators
Performance indicators, which may be either qualitative or quantitative,
measure project results. The two main points to bear in mind when determining
indicators are the following.
- If we can measure it, we can manage it.
- All indicators have to be measurable in terms of quality, quantity,
and time (target).
Performance measures allow you to recognize the successful accomplishment
of objectives. Listed in the second column of the DMF, they precisely
specify each result at the output, outcome, and impact levels (see Box
A2.5).
| Box A2.5: Steps to Determining Performance Indicators |
| Process |
Education |
Agriculture |
| |
Outcome: The education system in southeast
province provides higher quality, equitable education. |
Outcome: The rural, agrarian population in province
X has improved its income-earning potential. education. |
| Step 1: Determine the basic indicator (what you
intend to measure). |
More and better-educated students graduate. |
Rice yields of small farmers increased. |
| Step 2: Decide on the quantity (increase/decrease). |
The number of graduates increases from 10,000 to 25,000. |
Rice yields of at least 1,000 small farmers (owning 3 hectares or
less) increase by at least 30% from 5 tons to 6.5 tons. |
| Step 3: Describe the quality (what kind of change). |
The number of graduates (55% female and 45% male) passing
national standard examination from lower income families (US$5,000
per year) in southeast province increases from 10,000 to 25,000. |
Rice yields of at least 1,000 small farmers (owning 3 hectares or
less) increase by 30% from 5 tons to 6.5 tons while maintaining the
same quality (average weight of grain) as in 2004. |
| Step 4: Add the time frame (by when). |
The number of graduates (55% female and 45% male) passing
national standard examination from lower income families (US$5,000
per year in southeast province increases from 10,000 to 25,000 per
annum starting in year 4 of project implementation. |
Rice yields of at least 1,000 small farmers (owning 3 hectares or
less) increase by 30% from 5 tons to 6.5 tons annually starting in
2007 while maintaining the same quality (average weight of grain)
as in 2004. |
Top
Understanding Project Assumptions and Risks
Projects are not isolated from external events and are influenced by
factors outside the direct control of project management. These include
political, social, financial, environmental, institutional, and climatic
factors. Termed “assumptions” and “risks,” these
are highlighted in the fourth column of the DMF.
Assumptions are positive statements of conditions, events,
or actions that are necessary to achieve results at each level of the
DMF. Risks are negative statements of conditions, events,
or actions that will adversely affect or make it impossible to achieve
the intended results. Assumptions and risks may fall into one of two categories:
- those that involve the wider policy and institutional environment
and thus depend on external decision makers (e.g., policy environment;
institutional capability, political will);
- those associated with uncontrollable events or conditions (i.e., political
stability, world prices, interest rates, absence of natural disasters).
Top
Vertical Logic of Assumptions and Risks
Assumptions and risks complete the cause-effect logic of the DMF (see
figure A2.7 and Box A2.6). The completion of the activities will result
in the outputs only if certain assumptions hold true and the risks do
not materialize. This applies in the same way to the project outcome and
the intended impact.
After assumptions and risks have been clearly understood and specified,
they provide important insights and guide decisions during project design
and management. The following management options are available to deal
with assumptions and risks.
| Box A2.6: Determining Assumptions and Risks |
| Step 1: Assumptions and risks are best considered
during the objective tree analysis. For each statement, consider the
assumptions and the risks of achieving the level above, the cause-effect
relationship. |
| Step 2: The analysis of alternatives
should also evaluate assumptions and risks. Formulate assumptions
and risks for those alternatives that are not going to be addressed
by the project. |
| Step 3: Assess each assumption by its necessity
and sufficiency– if/then–to the logic of the project design. |
| Step 4: Refine the assumption and risk
statements. Overly vague statements cannot be monitored or mitigated
by project activities. For example, "the government remains supportive
of the reform program" is too general an assumption and cannot
be measured. Restate it as, "parliament approves the legislative
changes as per agreed schedule." |
| Step 5: Rate the importance and probability of
each assumption and risk. Focus on assumptions and risks that are
critical (positive/negative) to the success of the project and determine
their likelihood. |
- Do nothing. This is certainly the best option if
none of the assumptions and risks are critical enough to endanger the
achievement of the results. These fall into the low/low category.
- Change the project design. If a risk is high and/or
an assumption is critical to the success of the project, consider adding
an output, activities, or inputs to address the assumption.
- Add a new project. If the possibility of addressing
the risks and assumptions is beyond the scope of the project, consider
an additional one.
- Abandon the project. If a risk is high and likely
to occur or an assumption is absolutely necessary and there are no outputs
or activities that can reduce the risk or increase the probability the
assumption will hold true, you may well conclude that the project is
too risky and abandon it.
- Monitor and mitigate assumptions and risks. There
are a number of possible scenarios where close monitoring is appropriate.
In these cases, it is essential to have clearly specified indicators.
Make provisions for mitigating measures by including activities and
adding resources.
|
Publications 
RSS Feeds