Next: 2.3 Step 3: Collect information
2.2 Step 2: Design monitoring and evaluation process
In
addition to typical outputs from traditional monitoring and evaluation,
conflict sensitive monitoring and evaluation assesses the interaction between
the context and the project. In order
to understand this interaction the process should be designed around three
primary issues: (a) understanding the context and changes in the context; (b)
understanding the intervention, including its implementation; and (c) measuring
the interaction between the two.
(a) Understanding the changing context
As outlined in Chapter 2, a conflict analysis can be used to provide an understanding of the context in which project interventions are situated, and to track changes that occur. In particular, the conflict indicators developed at the conflict analysis stage will help systematically monitor changes in the context in terms of conflict profile, causes, actors, and dynamics.
However, some organisations may not have a conflict analysis at the time they want to start sensitising their monitoring or evaluation; or they may have a conflict analysis that has become outdated.
For monitoring purposes, if a conflict analysis does not exist it will suffice to conduct a current analysis and to begin incorporating conflict indicators from this point forward. This conflict analysis will provide the baseline from which to monitor and later evaluate changes in the context. The depth and scope of the conflict analysis should be appropriate to the existing or anticipated intervention and your organisation’s capacity. If on the other hand the conflict analysis is outdated, there is no need to redo it – simply develop (if none exist) or use the conflict indicators from the initial analysis to monitor changes in the operating context.
If you are conducting an evaluation, then given the importance of a conflict analysis to create a baseline, a retroactive conflict analysis should be undertaken using past reports and other information sources to estimate the situation prior to the start of the intervention.
(b) Understanding the project implementation
As conflict sensitive monitoring and evaluation focuses on the interaction between the context and the intervention, it is important to understand the project’s intended and actual implementation.
Intended implementation, activities and approaches
- purpose and scope of the activity
- geographic location of the project
- project beneficiaries and partners
- timeframe
- funding level and sources.
The information outlined above can generally be found in the project proposal and approved implementation plans.
Actual implementation, activities and approaches
- who are the project partners and beneficiaries? And why?
- what have been successes and challenges?
- were any activities undertaken that had not been envisaged during the planning? Why?
- were any adjustments made from the initial strategy? Why?
- have any activities been changed or cancelled?
- were there problems with staff (eg security, motivation)?
This information is typically found through the monitoring of traditional project indicators that were designed in the planning stage. You may want to ensure that the questions above can be answered through your initial project monitoring indicators, and add or adjust indicators as necessary.
When gathering this information for an evaluation, reference can be made to previous monitoring reports. It is important, however, to gather other perspectives that may not be reflected in these reports: designed as they usually are for a specific audience they may not fully capture the project’s implementation realities (see triangulation below, Chapter 2 Box 10, and Module 1 section 3.2 of this chapter).
(c) Understanding the interaction between the context and the project
As described in Module 1 of this chapter, there are three elements to conflict sensitive indicators:
- c onflict indicators are used to monitor the progression of conflict factors against an appropriate baseline, and to provide targets against which to set contingency planning (see Chapter 2).
- p roject indicators monitor the efficiency, effectiveness, impact and sustainability of the project (see Module 1 and Annex 1 of this chapter).
- i nteraction indicators (see Module 1 of this chapter) are created at the planning phase of the project in order to measure the interaction between the context and the project.
Specifically, interaction indicators are used to monitor the impact of the project on the context, and of the context on the project. For example, if the context tells you that corruption amongst local government officials is a contributing conflict cause, and the project involves building the capacity of local government officials, then an interaction indicator will measure both:
- the project’s effect on corruption amongst local government officials
- the effect of corruption amongst local government officials on the project.
A key challenge practitioners face when undertaking conflict sensitive monitoring and evaluation is the issue of agency or causality. For example, an NGO may be working in a remote village to provide access to water resources in a way that is equitable between two ethnic groups – an issue identified as key in a conflict analysis. Following the successful implementation of the project, evaluators using interaction indicators find that inter-marriage rates between the two ethnic groups have increased. The challenge of conflict-sensitive evaluation resides in the attribution of this change: is increased inter-marriage a result of the project intervention? Of interventions by other actors operating at the same and other levels? Or of changes in the context that are unrelated to external actors?
The highly simplified schematic that follows demonstrates the difficulty of determining the causal link between – in this example – the project intervention and a change in inter-ethnic marriage rates. Most contexts are substantially more complex than outlined in this diagram.
Diagram 1
1
Conflict-sensitive monitoring and evaluation must recognise that there is not always a direct cause-and-effect relationship between the context and the project. In this sense, ‘good enough’ thinking is required as conflict-sensitive monitoring and evaluation can never provide absolute certainty. It is nevertheless important to anticipate the challenge posed by causality when developing conflict-sensitive indicators. Good indicators often seek not to address directly the interaction between the project and the context, but to focus instead on more indirect causal manifestations of this interaction (eg not “did my project contribute to reduce discrimination?” but “are there parts of the district that are safe for some groups and not for others?”).
Because every context is unique and can change dramatically over short periods of time, it is not possible to provide a definitive list of conflict-sensitive indicators that practitioners can use or adopt to their own situations. In addition to the guidelines outlined above, it is, however, possible to outline a general approach to developing indicators for conflict-sensitive monitoring and evaluation. Annex 1 uses a fictionalised context to provide a detailed breakdown of the type of analysis and indirect questioning that is useful for developing conflict sensitive indicators. Box 1 below provides some actual examples of using indirect indicators to help determine impact.
|
Oxfam Sri Lanka have developed a series of conflict sensitive indicators to evaluate their peacebuilding work (this seeks to build relationships and supporting links within and between communities, to empower people to transform conflict, and to develop the analysis and resolution skills of partners). In one programme the relationships are built using inter-community exchanges. Indicators – quantitative and qualitative – were developed by the beneficiary communities, and are crosschecked by Oxfam. Indicators of the growing relationships between two previously divided communities include:
In order to gauge whether the relationship building has had a wider peacebuilding effect, Oxfam has looked at those who were not directly involved in the actual project (both within each family and in the community more broadly) to see if they have been affected by the project. Indicators include:
Showing attribution continues to prove a difficult task, and remains an open question for Oxfam. |
The three dimensions of conflict sensitive monitoring and evaluation outlined above – understanding the changing context, understanding the project implementation, understanding the interaction between the context and the project – provide a means of designing a conflict-sensitive monitoring and evaluation process. They may also inform the identification of required skills within the monitoring or evaluation team, which are likely to include:
- conflict analysis skills
- good knowledge of the context and related history
- sensitivity to the local context
- local language skills
- monitoring and evaluation expertise (including interviewing skills).
Finally, it is important to keep in mind that there is currently no clear way of assigning attribution for the consolidation of peace to any one particular actor. Given the complexity of most contexts, intervening actors will at best be able to demonstrate that their positive interventions coincided with positive changes in the context. Project and programme goals and objectives for building peace will need to be humble and realistic.
Next: 2.3 Step 3: Collect information
© FEWER, International Alert and Saferworld, 2003. All rights reserved.
Site design and publishing system by Jez Humble.