Creating an SSR index and indicators

Boris Vollenweider
Nov 4th, 2011 @ 3:34 pm
by Boris Vollenweider

Various indices exist to compare, analyze and explain different facts and phenomena. They are usually based on census, empirical research in various scientific fields or statistics. Among these indices we find a broad variety of topics such as the Corruption Perception Index (Transparency International), which highlights the perceived level of corruption in the public and private sector, or the Human Development Index (UN) which ranks countries based on health, education and living standards.

As stated in SSR literature, every country has a need for SSR. Even though countries with a post-conflict background usually show an increased need for SSR activities, highly developed and stable countries also need an SSR strategy (albeit with a more limited scope).

Based on this I was thinking if it would make sense to create an SSR index. Such an index could be used to show which countries have a high or low need for SSR programs and to show improvements of the security sector that were achieved over time.

The difficult part attempting to create such an index would be to define the right criteria and indicators.

There are also potential risks that must be considered while creating such a tool. Even though such an index can assist in the decision making process of allocating limited resources to projects and prioritize them, there is also a potential danger to be considered. What if a country who is desperately looking for donors behaves in a way that makes it look worse in the index only to receive that external support that otherwise would be go to another government?

I welcome any inputs and thoughts on this and looking forward to a productive discussion!

Jan 9th, 2017 @ 12:01 pm
by Antoine Hanin

Hello Petra,

Thanks for re-launching this debate/forum. It indeed remains very valid. My experience in measuring results of donors support to SSR tells me that you cannot do it with generic indicators. A basket of indicators may give you information on whether some aspects of the reform process have improved, stayed the same or decreased. This is useful for reporting, for monitoring purposes, mainly for the country undergoing a reform process. It can be used for political purposes, for capturing more funds, for informing the population, etc.

Indicators will never give you information on the results of donors support to the reform process. In order to do so, you will need to carry out a fully-fledged evaluation that may give you some evidence on the extent to which the donors have contributed to the reform process. You could for example carry out an evaluation of the support from the Netherlands to SSR in all the countries you are working over the last 5-10 years. Such type of evaluations are often being carried out by the EU on several topics, of which SSR published in 2011: http://ec.europa.eu/europeaid/strategic-evaluation-eu-support-justice-and-security-system-reforms-third-countries-2001-2009_en

Happy to discuss more if you want.

Best regards,

Antoine   

Dec 13th, 2016 @ 5:52 pm
by Petra van Oijen

Dear colleagues,

It is interesting to read about these topics - which remain very current. At the NL MFA we are at the moment discussing issues related to measuring results - and how to improve this for the various activities we have. In this context, of course, also the formulation of indicators is discussed. I was asked to think about one or a few indicators that capture our SSR activities. This is quite a challenge as our activities are quite diverse (ranging from Iraq to Burundi, Mali and North Africa, as well as different focus areas). Of course, a few aspects always return (e.g. governance, civil society).

I have already looked into a few documents of DCAF (e.g. ISSAT, TFNA) as well as SDG 16 and its indicators. Below a few of the things that came up. 

  • SDG 16 uses a few relevant indicators, including:
    • Indicator 16.6.2. Proportion of population satisfied with their last experience of public services, disaggregated by service
    • Indicator 16.7.1b. Percentage of population who believe decision-making at all levels is inclusive and responsive
    • Indicator 16.b.1. Proportion of the population who believe that state institutions are treating people of all groups fairly, equitably and without discrimination
  • A few of the indicators we tried to use at the MFA in the past (still quite some challenges regarding the measuring):
    • Strengthened capacity of (security) institutions with oversight functions in the security sector (qualitative, disaggregated per country)
      • Perhaps a link can be made to “# accountability mechanisms in place” – but also difficult, as not every project focusses on this.
    • Access to information on security sector strengthened for society (qualitative, disaggregated per country) 
      • Also difficult to measure as several projects/activities can be included by this, but these are all defined differently, and have a different focus.
  • And these are a few indicators, mentioned in a MFA guideline for measuring results (again, it is important to think about the method of measuring - %? Based on (subjective) questions tot he population/involved actors?)
    • Inclusion of civil society in security sector is promoted
    • Needs assessments conducted by security institutions, community dialogues 
      • This in combination with a perception survey of the local population in the area of security & justice can be useful
      • However, not all organisations can do this (due to knowledge, costs, resources)  – it is actually a different/separate activity – not sure if we can ask this of all organizations we support in the area of SSR
    • Capacity of institutions with oversight functions strengthened
    • Percentage of community members declaring to be involved in dialogue to find solutions for insecurity and instability with other community members, including of other ethnicities. 
  • Important to realise that not everything can be measured easily – and perhaps an extra activity is needed (e.g. regarding perception surveys)
  • It remains a challenge to define “a general indicator” for SSR (or perhaps even two or three) – which is at the same time relevant for all our activities, as these can be quite diverse
  • Several aspects will be based on subjective reactions of involved people/organisations
  • All this is not necessarily a bad thing, but it needs to be thought through

Any ideas and/or suggestions related to the questions above are welcome!

Thank you!

Petra

Jul 15th, 2015 @ 5:17 pm
by Thammy Evans

Hello All,

The subject of indicators for measuring progress (not just counting output) comes up periodically in security and justice sector reform programming. I thought, therefore, that it is worth highlighting a couple more sources for potential indicators

Security Sector Monitor publications from 2009-2011 (CIGI)

Creating Indicators for Measuring Gender in SSR (DCAF)

Please add others to this list!

May 31st, 2013 @ 3:33 am
by Natacha Meden

You may find the 2010 DCAF paper on "Measuring SSG - a Guide to Relevant Indicators" by Ursula Schroeder useful to fuel the discussion further.