Assess

Powered by mxGraph

Introduction

An assessment is a process of data gathering and information analysis carried out in support of a predetermined purpose. Purposes of assessments can include the following, and may be multiple:

  • Inform policy towards a particular country, sector or grouping;
  • Support the design of programmes to assist the development of security and justice systems;
  • Identify suitable entry points to support longer-term security and justice processes;
  • Identify and mitigate against risks and pitfalls of engagement;
  • Identify individuals, communities, networks, organisations or institutions to champion security and justice processes;
  • Identify possible suitable local, national, regional or donor partners for future engagement;
  • Inform funding priorities for support to SSR;
  • Create a baseline from which to evaluate programme outcomes and impact.

Preliminary remarks on what makes security and justice assessments different

While many of the principles behind conducting an assessment are the same in security and justice development as in other development activities, the nature of donor support to security and justice initiatives requires a different framework. Security and justice development is inherently a highly sensitive and political process. The extent of political will to undertake (or to avoid) security and justice development must be understood. Assessments must not just focus on technical issues and institutions, but include political dynamics, individual influences, informal networks and customs. In addition, a realistic understanding is required of the extent of the impact a donor can have (both positive and negative). It is important to unpack and acknowledge the relationships within the partner country, as well as donor capabilities to support effective development given the constellation of partner country relationships.

In addition to a good mix of sectoral technical experts, security and justice assessment teams need an understanding of security and justice development as being a holistic and political process that requires the combination of effectiveness with accountability. There is also a requirement for project management skills, cultural and local political awareness, knowledge of good practice in assessment processes and change management experts.

These challenges are exacerbated in the sorts of fragile or conflict-affected state environments in which security and justice support is often undertaken. The security and justice system in a post-conflict society is characterised by political volatility, collapsed structures, institutional fluidity and general uncertainty: the nature and number of personnel, the levels of organisational capacity, the effectiveness of the security and justice institutions, the activities of non-state security and justice providers, and the security and justice needs of the population, are often not clearly known.

Fragile or conflict-affected societies are often confronted by legacies of massive abuse generating a lack of trust and a crisis of legitimacy of the security institutions. Typical other post-conflict processes such as DDR, elections, mine action, and return of refugees are highly resource intensive and politically demanding. The tension between the numbers of top priorities (everything is urgent) and limited resources often calls for a careful identification of suitable entry points, rather than just focusing on long-term reform.

Entry points are limited programme activities that are feasible and not threatening, but build confidence, open doors for long-term involvement, and contribute to a holistic reform vision. 

Entry points should, therefore, be able to be implemented straight away and can be completed in a relatively short timescale.

Overview of ISSAT's approach to undertaking assessments

Assessments have traditionally focused on the analysis of security and justice in a country from the perspective of the provider – using institutional analyses that look for gaps when comparing the situation in the host country against an ‘ideal’ model. This, however, risks creating recommendations that fail to impact on the end-user of security and justice services and can end up being supply driven. They also fail to take into account the realities of the country and how security and justice are actually provided. 

Based on lessons identified and evolving good practice, ISSAT has selected a five phase process:

Phase 1. Determine the feasibility and appropriateness of an assessment  

This phase covers the main questions that need to be asked on whether an assessment is the most appropriate action (looking at relevance and feasibility); checking against the main principles for supporting SSR and reviewing how these influence the decision-making process; formulating the assessment request (including defining the objectives and scope); and identifying resources.

Phase 2. Plan the assessment 

This phase concentrates on ensuring the ground work has been made to incorporate the Paris Declaration and Accra Agenda for Action principles, as well as undertaking preparatory analysis covering past assessment reports, and awareness of existing political dialogue and the history of SSR in the country. The procedural aspects of the assessment mission that are covered at this stage include details on the assessment team, terms of reference and beginning to develop a context-specific methodology, including commissioning surveys where possible and appropriate. 

Phase 3. Build an assessment framework

This phase suggests a framework within which an assessment team can work and that combines a ‘Needs Assessment’ approach (looking at the institutional gaps) with a ‘What is Used’ approach (whereby focus is placed on the end user, where they actually go to get security and justice, how security and justice services are delivered and by whom). The process is founded on four identified good practices: 

  1. Evaluating the nature and structure of the State, particularly taking into account the governance structures and cultural norms that historically exist. This allows evaluations of justice and security institutions to be placed into the appropriate context, rather than gauging effectiveness against a Westphalian state model.
  2. Recognising the importance of personalised politics: the team need to map individual networks and examine leadership abilities, rather that just looking at institutional and capacity analyses. Unpacking the dynamic balance of power between different actors, communities and organisations – as well as their self-interests – is key, albeit extremely difficult. Particular care is taken where possible to vary the sources and examine opinions and judgements according to their power interests. 
  3. Recognising the importance of the relationship between security and justice service providers and their customers. These relationships are examined to see how these can be progressively made more accountable and effective, taking into account the myriad of ways (state and non-state) that people currently use to find security and justice. 
  4. Examining which constituencies are likely to support what initiatives, combined with an analysis of likely sources and levels of resistance. 

A comprehensive risk management approach is adopted throughout due to the fact that knowledge will inherently be partial and incomplete, and therefore unintended consequences are virtually a certainty that need to be mitigated against. 

Phase 4. Conduct the assessment in the field

This phase provides some guidance for conducting the assessment mission in the field and collect data according to the framework set in the previous phase.

Phase 5. Analyse the data and formulate recommendations

This is a specific stage of the process that provides the link between the assessment and any subsequent SSR support activities. It also provides the opportunity to collate and review lessons learned for future assessments. The exact process to determine recommendations will have been worked out with the mandator during the initiating/planning phases: for example, recommendations could be produced by the assessment team or could be explored through a stakeholder workshop. Four main parameters are considered: the life-span of the recommendation, the cost, potential spin-offs, and the political impact. Furthermore, recommendations need to be focused and feasible, with preconditions and consequences made explicit. Next steps are considered to clarify how the process will move from diagnosis to action and this will invariably require close liaison between the assessment team, the mandator and the host government. If national stakeholders are not party to the development of recommendations, bilateral or open meetings are highly recommended to validate findings, enhance national ownership of the process and build trust with key implementers.

For full access to the Prezi Presentation, ISSAT Assessment Methodology, kindly follow the link. 

Be the first to add a comment!

Short guide to Assess

Sub-activities

1. Integrate overarching principles

What is it?

These overarching principles should be adhered to throughout the assessment process. They are:

  • The Paris Declaration, Accra Agenda for Action, Busan Partnership and 3C commitments
  • Local ownership
  • Risk management
  • Conflict sensitivity / Do No Harm

Why is it important?

It is important to respect these principles when carrying out an assessment in order to maximize the chances of success, not only for your assessment but also for the subsequent phases (programme design, implementation, monitoring and evaluation).

How do we do it?

a) Adherence to the Paris Declaration, Accra Agenda for Action, Busan Partnership and 3C commitments

The Paris Declaration (2005), the subsequent Accra Agenda for Action (2008) and Busan Partnership (2011), and the 3C Obligations (2009) commit signatories to improve aid effectiveness through, inter alia, harmonisation, alignment, mutual accountability, coordination and reinforcing national ownership. One of the Paris Declaration targets is that joint assessments should make up 2/3 of all assessments conducted by 2010. In order to achieve aid effectiveness in the area of security and justice, the country’s ownership of the process is essential. The importance of procedural arrangements, or different coordination groups, should be emphasised in fostering ownership, alignment and harmonisation. Ideally, there will be a Government lead Security and Justice Group with donor participation in line with the Aid Effectiveness principles: harmonisation of the donors at the political and technical level through coordination mechanisms, such as a working group, is fundamental. If well functioning, it will facilitate both political dialogue and technical arrangements for the planning and execution of an assessment, as well as the broader security and justice process (including programme design, implementation, monitoring & evaluation and efficient funding mechanisms).

The Poverty Reduction Strategy (PRS) of a country can be an entry point, especially if a Poverty Reduction Strategy Paper (PRSP) will be produced or updated. If conditions for a joint assessment with other donors and the Government do not exist, it is still important to try to seek collaboration with key national actors, involve one/some donors and facilitate an open dialogue on the results with key stakeholders. It is also vital to seek, when possible, to delegate the management of the assessment process to the local donor representatives. Usually, the more an assessment is lead by HQ, the weaker the national ownership and the donor alignment and harmonisation will be.

It is easier to recognise what is not local ownership rather than attempt to capture the different components in a checklist, as these will differ depending on the context and levels of capacity. For example, local ownership does not mean having pre-written terms of reference accepted by national partners, or sharing plans only after they have been finalised. The bottom line is that donors do not do security and justice development – they support it.

b) Local ownership

The Paris Declaration stipulates that donors must “respect partner country leadership and help strengthen their capacity to exercise it”. In turn, the partner country must “exercise leadership in developing and implementing their national development strategies through broad consultative processes” (including thematic strategies such as those pertaining to security and justice). In this context, the assessment team should plan and conduct their activities so as to facilitate and enhance the leadership and participation of national actors. Achieving this can be extremely difficult, especially in fragile and conflict-affected environments. It requires looking at three areas: 

  1. Actors – are you talking to the right people? There needs to be a combination of those who are powerful or influential enough to lead change and those who are representative of the views of the peoples in a country. 
  2. Process – how will you achieve ownership? This needs to be a negotiated process that takes into account how support will be aligned with national visions, as well as considering the capacities available. It is worth remembering that the level of ownership is unlikely to be static and will increase in line with improved capacities within the national partners. 
  3. Systems and structures – what indigenous structures, systems or institutions exist (or can be built up) that reinforce ownership? This should look at, for example, who will be implementing, or monitoring and evaluating implementation of any of the assessment recommendations, or who is ultimately responsible for the project or programme (such as a joint or government-led steering group). 

It should be remembered that if local ownership is not strived for, the reform process will not have legitimacy and is unlikely to be sustainable.

c) Risk management

The environment in which security and justice assessments are undertaken is inherently complex and uncertain. There are many different actors in play with a myriad of agendas. Information will be difficult to obtain, and equally difficult to verify. Time is a constant pressure, both in terms of the time available to the assessment team to evaluate the situation, and regarding the often rapid evolution of events on the ground. This means that, despite all attempts to limit them, there will most likely be many unintended consequences arising. 

Unintended consequences can be as a result of undertaking the assessment, due to the likes of interactions whilst in country, raised expectations, impacts on other programmes, or empowerment of particular interlocutors. They can also come from the recommendations or options emanating from the assessment. You must take a proactive risk management approach to undertaking the assessment, seeking to identify and mitigate against negative consequences, as well as being able to react to and exploit positive consequences previously unforeseen.

d) Conflict sensitivity / Do No Harm

The risk management approach should be combined with adopting a conflict sensitive or Do No Harm approach. Do No Harm, or conflict sensitive analysis is analysis that examines the impact of the support on conflict and helps to unpack the complexity of the environments in which security and justice assessments often takes place. Questions to ask include: Does the assessment risk weakening factors that minimise conflict and strengthen factors that enhance it? How can the assessment approach be changed so as to maximise the positive? If national experts are part of the team, a Do No Harm Exercise can be done within the assessment team. If possible more actors should be involved. However, the most important aspect is to conduct the conflict sensitivity analysis and make sure it influences the process and the results and recommendations from the assessment. A full description of how to undertake such analyses can be found in the Do No Harm Handbook at www.cdainc.com.


2. Conduct the assessment in the field

What is it?

This is the fourth activity of an assessment. The main actors are all the members of the assessment team, led by the team leader. The aim is to collect data in the field to provide sound analysis and potential recommendations and options for eventual further support for a SSR programmes.

The following sub-activities should be carried out in the field:

  • Organise and hold a briefing with the entire assessment team;
  • Organise and hold a briefing with the mandator and other stakeholders (e.g. national counterparts);
  • Collect data using various data collection tools;
  • Debrief with the team each day;
  • Debrief with the mandator and other stakeholders on the preliminary recommendations and options.

At the end of this activity, the output should be a presentation of the preliminary recommendations and options to the mandator and other stakeholders in the field. They should be able to comment on these preliminary recommendations and options in order to rectify and complement them if necessary.

Why is it important?

Going to the field to collect data is crucial in an assessment. An assessment team needs to take the measure of the work being done on the ground, understand the politics and dynamics of the main stakeholders, ask questions to the security and justice providers and recipients and see for itself the context in which people live.

Presenting the preliminary recommendations and options at the end of the field mission is important in order to obtain feedback from the main stakeholders and to manage their expectations. They should not be surprised by the assessment report at the end of the process.

How do we do it?

Field missions will be different according to each request. The following should thus be adapted to the context, the time available, the number of experts involved and the main stakeholders' role (donor and beneficiaries).

4.1: Organise a team briefing at the beginning of the field mission 

4.2: Organise a briefing with the mandator and other main stakeholders involved in the assessment (e.g. the government, other donors, non state actors, etc.) 

4.3: Collect the data using various data collection tools 

4.4: Organise team debriefings at the end of each day to update the others on the work and data collected; on the difficulties encountered and how they may be overcome; on some preliminary analysis.

4.5: Organise a debriefing with the mandator and other main stakeholders involved in the assessment (e.g. the government, other donors, non state actors, etc.) to present the preliminary recommendations and options. Depending on the length of the field mission, a midpoint briefing might also be useful as well as an end of mission debriefing.


3. Types of assessments

Here are several assessments, which have been tailored for use in specific reform contexts.


4. Determine feasibility and appropriateness

What is it?

This is the first activity of an assessment. It covers issues to be taken into account when first receiving a request to assist with a security and justice assessment. It provides an overview of the main steps that you should consider in order to ensure that you have as clear a picture as possible of what is required in order to start planning, as well as gathering initial information. It assumes that the request has come from a donor, but that the national partners will be brought into the process as soon as possible, in line with commitments under the Paris Declaration, Accra Agenda for Action and Busan Partnership.

This activity covers the main questions that need to be asked to determine whether the assessment is feasible and appropriate.

The output of this activity should be an explicit decision from the mandator on the initiation (or not) of the proposed assessment and some initial Terms of References (to be finalized in the next activity).

The decision may be to: 

  1. Initiate the security and justice assessment;
  2. Not initiate the assessment; 
  3. Hold until certain conditions are met;

The potential positive and negative consequences associated with each of these three decision options should be identified before a decision is made. An analysis of the potential positive and negative consequences will not only facilitate the decision-making itself but will also help to promote opportunities associated with a decision or to manage undesirable consequences once the decision is made.

A security and justice assessment may have to be delayed, adapted or may not be possible at all for a variety of reasons. Examples include: 

  • Upcoming elections that focus all attention and maychange the political environment;
  • The security risks for an assessment team are too high;
  • The rainy season makes in-country movements difficult

Such factors have to be identified and considered in deciding whether and when to go ahead with an  assessment request.

Potential positive and negative consequences of decision options relate to changes of expectations of potential beneficiaries, as well as changes to the role and perception of the donor country in the partner country. For instance, the decision of a donor country previously involved in humanitarian assistance to launch an SSR assessment will draw the attention of security actors and other donors involved in SSR, and will raise the donor country’s political profile due to the sensitivity of security and justice.

Depending on the circumstances and whether they have already been involved at this stage in the process, partner country representatives should be involved. Other government actors and departments at Headquarters and in the field who are affected by the decision should be informed about its outcome.

Why is it important?

This first activity is crucial for the rest of the assessment process because it will set the basis for a constructive and efficient assessment mission. Before engaging further and raise expectations of both the mandator and the local stakeholders, it is important to make sure that the main conditions are met for maximizing the chances of success (of the assessment but also of the subsequent phases of programme design and implementation).

How do we do it?

In order to determine whether an assessment is feasible and appropriate, you should carried the following sub-activities:

1.1: Clarify the purpose of the assessment.

1.2: Start analyzing the context.

1.3: Check the human and financial resources available.

1.4: Consider alignment, harmonization and ownership


5. Plan the assessment

What is it?

This activity concentrates on ensuring the ground work has been made to incorporate the Paris Declaration and Accra Agenda for Action principles, as well as undertaking preparatory analysis covering past assessment reports, and awareness of existing political dialogue and the history of SSR in the country. The procedural aspects of the assessment mission that are covered at this stage include details on the assessment team, terms of reference and beginning to develop a context-specific methodology, including commissioning surveys where possible and appropriate.

At the end of this activity, the output should be an official written approval of the Terms of References. Moreover, the assessment team should be composed, the budget decided and the methodology developed.

Why is it important?

Planning is crucial for an assessment. It allows starting the assessment on solid grounds with a clear mandate and objective, an adequate team and a realistic work plan and methodology. Assessment can be difficult to undertake and manage due to the complex settings and country situation. Knowing where you are going and how you are going to get there allow you to manage the resources and the expectations efficiently.

How do we do it?

Note that the sub-activities will not always follow in sequence and there will be a degree of moving backwards and forwards between sub-activities as new information or events unfold.

2.1: Ensure political buy-in

2.2: Coordinate between HQ and country office

2.3: Assemble the assessment team

2.4: Gather additional data on the country context

2.5: Develop the assessment approach and methodology

2.6: Plan the work

2.7: Hold a kick-off team meeting

2.8: Prepare field deployment and logistics

2.9: Support drafting of the Terms of Reference


6. Build the assessment framework

What is it?

This activity combines a ‘Needs Assessment’ approach (looking at the institutional gaps) with a ‘What is Used’ approach (whereby focus is placed on the end user, where they actually go to get security and justice, how security and justice services are delivered and by whom). The approach should not be prescribed, but developed in line with the specific context and purpose of each assessment. The following does not set out a series of detailed steps or lists of questions to ask, but instead provides direction on good practice approaches and ideas on how to undertake different aspects of the assessment. You should use this as guidance when determining the specific methodological framework for your assessment.

Why is it important?

Taking the ‘Needs Assessment’ approach and the a ‘What is Used’ approach allows you to:

  • Evaluate the nature and structure of the State, particularly taking into account the governance structures and cultural norms that historically exist. This allows evaluations of justice and security institutions to be placed into the appropriate context, rather than gauging effectiveness against a Westphalian state model. 
  • Recognize the importance of personalised politics: the team need to map individual networks and examine leadership abilities, rather that just looking at institutional and capacity analyses. Unpacking the dynamic balance of power between different actors, communities and organisations – as well as their self-interests – is key, albeit extremely difficult. Particular care is taken where possible to vary the sources and examine opinions and judgements according to their power interests. 
  • Recognize the importance of the relationship between security and justice service providers and their customers. These relationships are examined to see how these can be progressively made more accountable and effective, taking into account the myriad of ways (state and non-state) that people currently use to find security and justice. 
  • Examine which constituencies are likely to support what initiatives, combined with an analysis of likely sources and levels of resistance. 

A comprehensive risk management approach is adopted throughout due to the fact that knowledge will inherently be partial and incomplete, and therefore unintended consequences are virtually a certainty that need to be mitigated against.

How do we do it?

3.1: Assess the context

3.2: Assess service provision


7. Analyse data and formulate recommendations

What is it?

This is the last activity of the assessment process that provides the link between the assessment and any subsequent SSR support activities. It also provides the opportunity to collate and review lessons learned for future assessments. The exact process to determine recommendations will have been worked out with the mandator during the initiating/planning activities: for example, recommendations could be produced by the assessment team or could be explored through a stakeholder workshop.

Four main parameters are considered:

  • the life-span of the recommendation,
  • the cost,
  • potential spin-offs, and
  • the political impact.

Furthermore, recommendations need to be focused and feasible, with preconditions and consequences made explicit. Next steps are considered to clarify how the process will move from diagnosis to action and this will invariably require close liaison between the assessment team, the mandator and the host government. If national stakeholders are not party to the development of recommendations, bilateral or open meetings are highly recommended to validate findings, enhance national ownership of the process and build trust with key implementers.

Why is it important?

This activity is important because it allows you to step back from all the information you have collected in the previous activities. It allows the analysis of the data collected in order to extract the most appropriate recommendations/options. It is also important in order to draft the final report and communicate the results of your assessment with some detachment from the field mission.

How do we do it?

Note that the sub-activities will not always follow in sequence and there will be a degree of moving backwards and forwards between steps as new information or events unfold.

5.1: Confirm recommendations/options

5.2: Debrief with the mandator

5.3: Write the report

5.4: Provide feedback and updates

5.5: Distribute the reports

5.6: Collate and distribute the lessons identified


Resources