General Description
Evaluations are an important activity to be carried out for SSR projects, programmes and policies. As defined by the OECD, an evaluation is "the systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors.” (Glossary of Key Terms in Evaluation and Results Based Management, OECD 2002)
In this section, you will find a useful and operational methodology to undertake mid-term, on-going or formative evaluations, as well as ex-post or summative evaluations of simple SSR projects, complex SSR programmes and SSR policies.
This methodology is not appropriate for ex-ante prospective evaluations or assessments or for the design and implementation of monitoring systems for a project or programme.
Throughout this methodology, specific evaluation terms are used, most of them have been hyperlinked to their document of origin, the Glossary of Key Terms in Evaluation and Results Based Management, OECD 2002.
PRELIMINARY REMARKS ON THE SPECIFICITIES OF EVALUATING SSR
A Saferworld study on evaluating SSR and the OECD SSR handbook, section 10 on evaluation and monitoring, highlight a number of specificities/challenges for evaluating SSR. Some are common to all development aid evaluation exercises, and some are more specific to SSR. Below are the most relevant challenges for evaluating security and justice projects, programmes and policies:
SSR is often undertaken in fragile and conflict-affected environments:
- Lack of data
- Rapidly changing and insecure situation
- Limited design and bad planning due to time pressure
- Important to ensure conflict-sensitivity of the evaluation process and results
SSR has a multi-sectoral and integrated/holistic dimension:
- The performance of one sub-sector is influenced by others. It is thus a challenge for attributing the performance when evaluating only one sub-sector.
- Evaluating a whole-of-government approach and/or an integrated approach requires specific methodology
SSR is highly politically sensitive and has a culture of secrecy:
- Difficult access to data (documents but also through interviews)
- Official objectives may be different than unstated goals
SSR actors (e.g. soldiers, policemen, lawyers, judges and intelligence officers) often have limited understanding of evaluation processes:
- Hostility towards evaluation processes
- Difficult to reach agreement on what to measure and how
- Difficult to integrate the lessons learned
- Limited numbers of specialist SSR evaluators
Local ownership of SSR and of the evaluation process is crucial:
- Difficult to evaluate local ownership
- Results of the evaluation process have to be locally owned. Evaluation findings should be shared with local partners and recommendations adapted to their needs.
The ISSAT evaluation offers a structured approach to deal with these difficulties. It builds awareness and provides access to knowledge to SSR practitioners on the evaluation process in a simple and accessible manner. However, it needs to be adapted to the context of each evaluation.
For access to the Prezi Presentation, ISSAT Evaluation Methodology, kindly follow the link.
OUR EVALUATION MANDATES
2020:
- Mid-term evaluation of UNDP Promoting the Rule of Law in Palestine Programme (UN)
- Evaluation of UNDP CAR RoL Programme (UN)
- Evaluation of the joint UNDP Rule of Law Portfolio in DRC (UN)
2019:
- Mid-term review of the project “Introduction of a Full-Scale Probation Service in Ukraine" (Norway)
- Evaluation of the Swedish Juvenile Justice Project in Albania (Sweden)
- Evaluation of the Swedish Police Programme in Moldova (Sweden)
2018:
- Mid-term evaluation of the UNDP/MINUSCA joint project in support to the SCC in CAR (UN)
- Re-socialisation of ex-combatants project evaluation (Norway)
- Evaluation of Swedish Police Programme in Liberia (Sweden)
- Evaluation of UNDP Colombia RoL Programme (UN)
2017:
- Evaluation of UNDP Jordan RoL Programme (Germany, UN)
- Support for SSR Monitoring and Evaluation Guidelines (EU)
2016:
2015:
- Evaluation of UNDP DRC Community Policing Programme (UN)
- Programme Evaluation - International Commission against Impunity in Guatemala (UN)
2014:
- Evaluation of the SSD programme Phase II in Burundi (Netherlands)
- Review of SSR Process in Burundi: From Arusha to present (Netherlands)
- Review of the Rule of Law Advisory Mission on Moldova (Norway)
- Fourth Annual Review of Security Sector Accountability and Police Reform (SSAPR) programme in DRC (UK)
2013:
- Review of the implementation of the Community Policing Project in Albania (Sweden)
- Mid Year review - DFID DRC SSAPR programme (UK)
2012:
- Evaluation and Identification Mission for the Continuation of the (bilateral) Legal Cooperation between Indonesia and the Netherlands (Netherlands)
- Security Sector Development Programme in Burundi - Phase I Evaluation (Netherlands)
- Support to European Court of Auditors Performance Audit (EU)
2011:
- Advisory Support to Evaluation of EC Support to Justice and Security System Reform (EU)
- Support to Evaluation of Security Sector Development Programme in Burundi (Netherlands)
2010:
2005: