Evaluation

General Description

Evaluations are an important activity to be carried out for SSR projects, programmes and policies. As defined by the OECD, an evaluation is "the systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors.” (Glossary of Key Terms in Evaluation and Results Based Management, OECD 2002)

In this section, you will find a useful and operational methodology to undertake mid-term, on-going or formative evaluations, as well as ex-post or summative evaluations of simple SSR projects, complex SSR programmes and SSR policies.

This methodology is not appropriate for ex-ante prospective evaluations or assessments or for the design and implementation of monitoring systems for a project or programme.

Throughout this methodology, specific evaluation terms are used, most of them have been hyperlinked to their document of origin, the Glossary of Key Terms in Evaluation and Results Based Management, OECD 2002.

PRELIMINARY REMARKS ON THE SPECIFICITIES OF EVALUATING SSR

A Saferworld study on evaluating SSR and the OECD SSR handbook, section 10 on evaluation and monitoring, highlight a number of specificities/challenges for evaluating SSR. Some are common to all development aid evaluation exercises, and some are more specific to SSR. Below are the most relevant challenges for evaluating security and justice projects, programmes and policies:

SSR is often undertaken in fragile and conflict-affected environments:

Lack of data
Rapidly changing and insecure situation
Limited design and bad planning due to time pressure
Important to ensure conflict-sensitivity of the evaluation process and results

SSR has a multi-sectoral and integrated/holistic dimension:

The performance of one sub-sector is influenced by others. It is thus a challenge for attributing the performance when evaluating only one sub-sector.
Evaluating a whole-of-government approach and/or an integrated approach requires specific methodology

SSR is highly politically sensitive and has a culture of secrecy:

Difficult access to data (documents but also through interviews)
Official objectives may be different than unstated goals

SSR actors (e.g. soldiers, policemen, lawyers, judges and intelligence officers) often have limited understanding of evaluation processes:

Hostility towards evaluation processes
Difficult to reach agreement on what to measure and how
Difficult to integrate the lessons learned
Limited numbers of specialist SSR evaluators

Local ownership of SSR and of the evaluation process is crucial:

Difficult to evaluate local ownership
Results of the evaluation process have to be locally owned. Evaluation findings should be shared with local partners and recommendations adapted to their needs.

The ISSAT evaluation offers a structured approach to deal with these difficulties. It builds awareness and provides access to knowledge to SSR practitioners on the evaluation process in a simple and accessible manner. However, it needs to be adapted to the context of each evaluation.

For access to the Prezi Presentation, ISSAT Evaluation Methodology, kindly follow the link.