Next: 2.4 Step 4: Analyse information
2.3 Step 3: Collect information
Collecting information is fundamental to the process of monitoring and evaluation. Conflict-sensitive information will need to include a combination of perception-based and objective data.
Perception-based information
As explained above, conflict-sensitive monitoring and evaluation cannot assume a direct causal relationship between the context and the project. In order to increase confidence in information collected, the perceptions of respondents can provide additional perspectives on causal relationships. Perception-based information can be derived from the following sources:
- executors of the activity: eg project staff, partners and implementing agencies
- beneficiaries of the activity: eg recipients of project outcomes (services, goods, training)
- observers of the activity: eg other organisations operating inside and outside the area, experts, academics, national and local leaders. Although this may not be an obvious category of respondents, their indirect involvement in the project and / or presence in the context may help ensure a more balanced understanding of the interaction.
The strength of perception-based information primarily depends on an honest and impartial composition of the list of respondents. If it is not possible to find unbiased respondents, it may help to get a balance of biases from among all interviewees. Evaluators also face a unique perception-related issue, as former project beneficiaries may use an end of project evaluation as an opportunity to deliver positive and uncritical feedback on the interaction between the project and the context, in the hope of securing future assistance or employment.
Objective information
Just as perception-based information helps address the issue of causality, objective information can be used to provide additional perspectives. Where perception-based information relies on views, beliefs and feelings of respondents, objective information seeks to provide less controversial or more ‘factual’ data. Sources for objective data are entirely context specific – eg news media may sometimes be a good source of objective information, but in a different context or at a different time information reported may be entirely perception-based.
The principal reason for combining objective and perception-based information in the process of conflict sensitive monitoring and evaluation is triangulation. In other words, information received from one source is compared and contrasted to similar information received from another, in an effort to gain a more comprehensive understanding of the interaction between the intervention and the context. From a conflict-sensitive perspective, perceptions sometimes provide more information than ‘facts’ or the ‘truth’.
It is important to triangulate data within one information source, just as it is important to triangulate information sources. For example, within one community interviewers should talk to a representative cross-section of the population, from government officials to unemployed youth, as well as individuals (although perhaps not leaders) from major social and occupational groups. As mentioned above, project staff and observers not directly related to the project also provide a means of triangulating perception-based information from the field.
| The way in which information is gathered can also be diversified to elicit a variety of perspectives. In Northern Uganda, for example, interviewers using open questions asked respondents ‘what has been done about the local situation and by whom?’ Closed questions, on the other hand, elicit a yes or no response: ‘do you feel safe?’ Scaling asks respondents to rank their responses: ‘compared to five years ago, are local government officials today much more, more, the same, less, or much less corrupt?’ Each form of questioning has advantages and disadvantages, and the best results are achieved by using a variety of different techniques. |
However, the perspectives of people involved in the community provide only one source of information (albeit diversified within the source), so it is also important to triangulate sources, for example, by reviewing secondary materials such as foreign government-sponsored country reports through a desk study, as well as soliciting the views of specialists. Focus groups, stakeholder and feedback workshops, and quantitative surveys provide other means of triangulating information sources.
Conflict-sensitive monitoring and evaluation requires that organisations acknowledge the potential impact of the monitoring or evaluation process itself on the conflict dynamics. Gathering information for monitoring and evaluation may have negative outcomes, such as putting community members at risk by raising suspicion or asking sensitive questions. Questions that are acceptable in one context may endanger interviewers and respondents in another. Interviewers may inadvertently upset respondents with probing, insensitive questions. Dialogue must always be based on mutual consent and respect, and the understanding that the consequences of the interview may last well beyond the discussion. Measures must be taken to ensure the safety at all times of both interviewers and respondents.
In situations of violent conflict, monitoring or evaluating projects often becomes quite difficult. While it is often possible to implement projects in such environments through local partners and community-based organisations or other means, it is sometimes not feasible to send external staff or consultants into the area to monitor or evaluate activities. Organisations tend to fear that ‘outsiders’ may be at risk in violent environments, and that locals may be endangered by talking to these ‘outsiders’. The struggle to monitor or evaluate their projects effectively under circumstances of violent conflict sometimes leads organisations to rely on telephone conversations and photographic evidence. Conflict-sensitive monitoring and evaluation must find ways of safely interacting with respondents in these types of environments; unfortunately this challenge remains unresolved.
Next: 2.4 Step 4: Analyse information
© FEWER, International Alert and Saferworld, 2003. All rights reserved.
Site design and publishing system by Jez Humble.