What is it?
The methodology for the assessment should allow you – as much as possible – to make relevant and pragmatic recommendations in an environment where there is, at best, only partial or incomplete knowledge of the issues and the power relationships. Details on ways to undertake this are covered in the next activity on building the assessment framework. When forming the plan for the field phase, it is important to explore the use of quantitative and qualitative data and consider triangulation. If time is too short to conduct a comprehensive assessment at the outset of a programme, then plans for more detailed assessments should be built into its design.
Why is it important?
It is important to develop a methodology to be used during the field mission in order to ensure the coherence of the analysis and recommendations for any eventual programme and to structure the team work. You may have a team of various experts with different thematic background and country of origin. Having a clear methodology helps the entire team to move in the same direction and gather data in a consistent way. However, the methodology for an assessment mission may have to be adapted during the field mission according to the context, the initial data gathered and the possibility or not to meet relevant interlocutors.
How do we do it?
Develop a snapshot of the situation. You need to undertake an in-depth analysis in order to complete as clear a picture as possible. As analysis is inherently subjective, you should ensure that as many of the team as possible are involved in this step – especially national team members – in order to minimise bias. Although the focus of the analysis will depend on the particular purpose of the assessment, two key aspects that should always be considered are a Political Economy Analysis and a Conflict Analysis / Do No Harm Analysis.
The Do No Harm analysis is designed to help understand what impact an assistance programme can have on relationships in a fragile state environment. If the analysis shows that assistance will actually make tensions in relationships worse, it then prompts those conducting a Do No Harm analysis to think through alternative assistance programming in order to eliminate these negative influences. A Do No Harm analysis can be conducted in seven steps:
Step 1: Understanding the context
Step 2: Analyse dividers and sources of tension
Step 3: Analyse connectors and local capacities for peace
Step 4: Analysing the assistance programme
Step 5: Analysing the assistance programme’s impact on dividers and connectors through resource
transfers and implicit ethical messages
Step 6: Generate programming options
Step 7: Test options and redesign programme
Visit www.cdainc.com for more details, or the ISSAT website for related resources at issat.dcaf.ch. Example: In preparing to support a reform of the justice and security sector in Zimbabwe, a group of national and international experts, with the support of a donor group, decided to undertake a thorough Do No Harm analysis. Zimbabwe had experienced severe and recurrent political crises and the justice and security sector had played a central role. However, the newly created coalition Government had been identified as a window of opportunity for sector transformation. Little was known about the current state of affairs and an assessment seemed as a natural first step. A Do No Harm analysis revealed several political risks associated with an overarching justice and security assessment, highlighting the important of getting initial buy-in from key stakeholders. As a result it was decided to initiate a series of studies on sub-sectors in a sequence that the national parties could agree to, with the aim of identifying policy options. This in turn exposed the need for a participatory and consensusbased mechanism where the national parties could directly influence the justice and security programme.
Consider aspects for Monitoring & Evaluation of any eventual programme: Although the decision to implement a programme will not take place until after the assessment, it is important to already be thinking of issues relating to Monitoring and Evaluation (M&E). Relevant tasks that need to be incorporated into the plan for when you are in the field include:
- Collecting as much as relevant information as possible that will be useful to identify or generate information that can be used to set baselines for any eventual programme.
- Reviewing the M&E capacities and needs of relevant international actors at HQ and field level.
- Assessing the capacities and needs in the partner country to introduce, manage and sustain a results-based M&E system. This can be achieved by looking at the incentives and demands for designing and building a results-based M&E system; current roles, responsibilities and structures for assessing performance, and capacity building requirements to set up an M&E system.
Keep up to date the risk management matrix: You need to consider the risks of positive and negative outcomes to undertaking an assessment (the fact of just doing one and the way in which it will be carried out). For example, how is the assessment likely to affect the power relationships within the country? To mitigate against potential negative interference and to capitalise positive influences, who can assist in identifying / recognising champions and spoilers during the assessment? Following the update of the risk matrix, a GO/NO GO decision should be made in consultation with the mandator.
There are many different research tools that could be considered depending on the context. These include surveys, direct interviews (structured and non-structured), focus groups, network analysis, building visual maps to represent the relative importance/impact of different sectors or relationships, workshops, observation, etc. Longer-term methods such as surveys and ethnographic research, especially those that require local partners, should be commissioned in advance and mindful of the fact that there may be a need for research capacity building. The choice of methodology will need to take into account the cultural norms, level of political will for reform, security situation, time and capacity, as well as drawing on the technical or thematic knowledge of your team members. Remember that a lot of imagination and empathy may be required to determine the best way to ask certain more sensitive questions.
There are several resources that list possible questions that may be applicable, such as the OECD DAC Handbook on SSR (see the "resources" box on this page) or the UNODC Criminal Justice Assessment Handbook (see the "resources" box on this page).
When determining the exact methodology and formulating more specific questions, remember you will also need to verify information already received from previous assessment reports and open-source material, as well as triangulating information received on the ground from interlocutors.
Communication skills are vital for undertaking assessments, and approaches with different communities must take into account the social and cultural norms. Moreover, care must be taken to manage expectations on what can be expected by those sharing their views and information.
To address these issues, the team should prepare a communications strategy in advance with the mandator (although ideally a representative of the mandator should be on the team) to cover:
- The extent to which information can be shared with different actors;
- The extent to which the results of the assessment will be shared and how;
- Culturally appropriate ways to explain to the interlocutors how the information they give will be used;
- How do deal with possible ‘expectations problems’such as NGOs expecting contracts to implement any projects that come out of the assessment;
- How to get sensitive issues on the table.