Tools

Review of the Implementation of the Action Plan for the Development Strategy of the Ministry of Interior of Serbia

Main challenges

  • Time constraints:while prior knowledge of the MoI from ISSAT team and assistance from Sida/SPAP greatly contributed to carry out the field mission, the team had only two weeks in Belgrade, which restricted the number of meetings and activities possible.
  • Culture of secrecy: like if by default, all kinds of documents and information are classified in the MoI and it is difficult to navigate a mandate of this nature when the prevailing institutional culture and legal/policy framework is one of secrecy and opacity.
  • Empowering while doing no harm– Capacities are uneven among mid- and senior management at the MoI and between units. The imbalance of capacity across units is compounded by uneven attention given to different units by senior managers and donors. In this mandate, the risk was that those units needing the most support would be undermined further by being highlighted as deficient in the report due to the inherent blame culture (the Analytics Directorate being an example). Asymmetrical support to some units while others take on expanding responsibilities posed another challenge: how to reinforce the most performing units (e.g. BSP) whilst not creating too much of a gap in capacity with the rest of the MoI, making them the object of jealously by their peers.
  • MoI in transition– the MoI was looking ‘very inwards’ (sic) until Brussels gave its feedback to the screening under Chapter 24, and then preparing Serbia’s follow-up on the results; afterwards, the new Cabinet has been focusing attention in drafting the new Police Law. Discussion on gaps identified by ISSAT will thus depend on the Cabinet being more free of those processes to think on strategic management. The report ‘is a seed that can grow, but in other conditions’, as pointed out in the AAR. (See below also).

Review of the Implementation of the Action Plan for the Development Strategy of the Ministry of Interior of Serbia

Success factors

  • Objectivity: ISSAT mandate was carried out in a way that safeguarded the team and the report against considerations of political bias and confirmed the reputation of the organisation within the MoI as an independent body.
  • Contextual knowledge: previous ISSAT mandate with the MoI and in Serbia, and in the region provided a robust knowledge of the realities and issues at stake in this review.
  • Balance and relevance:  the report was perceived as addressing all the relevant issues against a benchmark that was rational, balanced and that made justice to the current capacities and overall context of the MoI. This meant in turn that recommendations are credible in that they contain attainable outcomes, when the opposite is quite often true in the experience of the MoI: evaluations against  an ideal scenario with no relation to realities on the ground and existing baselines, thus immediately discarded by those to whom they are addressed.
  • Language/Tone: the mandate touched sensitive topics and the report raised sensitive questions, including transparency, stakeholder engagement etc, in a tone that was “diplomatic but straight to the point” to convey the important messages. The subtle but sharp relevant language also contributed to empower without undermining (see below).

Lessons identified for improving the chances of success:

Link-up the review with building the capacity of the institution(s) being assessed:ISSAT team was requested to share its review methodology with the BSP, in order to share additional tools and practices that could provide the Bureau with ideas to inform their role in the context of the Accession process. While the use of such methodology by the Bureau needs further confirmation, the fact that it is perceived as useful to other programmes within the MoI is a relevant ‘collateral’ gain. (See also below).

Review of the Implementation of the Action Plan for the Development Strategy of the Ministry of Interior of Serbia

Outputs and Outcomes

The mid-term evaluation report delivered by ISSAT was considered very useful, pertinent and objective  by both the mandator and the MoI. Both stakeholders noted nonetheless that the report has yet to produce it’s full impact due to circumstances external to the mission: first, the catastrophic floods that put the country in a status of national emergency for several months; second, the general elections held in March that redrew Serbian political landscape and brought a new party to power. At the MoI, both events took attention and resources away from discussions on strategic management and disrupted ordinary activities. This was compounded by an overriding priority given by the new Government in its first months in office to the revision of the National Programme for the Adoption of theacquis communautaire(NPAA) and the follow-up to the EU screening on Chapter 24 (Serbia recently submitted a new action plan to meet the goals set by Brussels on Justice, Freedom and Security).

Only recently life returned to normal in the Ministry. The action plan for Chapter 24 and the new plan for the Development Strategy provided an opportunity for BSP to use ISSAT’s report within the MoI, at this stage to raise awareness of issues raised in the document and the importance of building capacities in areas like budgeting and strategic analysis. ISSAT findings and recommendations are likely to have a bigger impact long after the mandate finished, into 2015. Indeed, the review seems to be recognised at senior level in the MoI as a reference source to inform planning at the Ministry, in two dimensions that seem interconnected. First, the mandate report is used as a stand-alone document considered for its sensitive analysis and recommendations - even if the latter were not yet or not entirely acted upon. Second, the mandate (report, methodology and ISSAT’s overall approach) left a set of tools and good practice, meaning that ISSAT contribution is clearly understood to have high value beyond the review conducted. New management in the Directorate for Analytics has also enhanced opportunities for the recommendations to actually be taken in consideration in terms of addressing some of the imbalances identified in the report.

The item of capacities for budgetary inputs into strategic planning was not included on the initial Terms of Reference for this review but it was considered relevant during the field mission and included in the report. The resulting remarks and evaluation were credited as being very useful to the MoI, well beyond the new Action Plan recently approved. Adding budgeting to the mandate contributed to raise awareness of the importance of this focus-area at the MoI, as previously mentioned.

Strategic Management - Baseline Study Moldovan MIA

Outputs and outcomes

ISSAT produced a comprehensive report that evolved from the original request of doing a baseline report to one that looked at the various needs and made substantial recommendations. Note that the term ‘baseline’ suggests that programme areas have been already identified and the study is to see where the process evolved from. This mandate actually needed the programme areas to be identified.

In order to capture a broader array of views, an electronic survey was developed by the team and distributed to key personnel in the Central Apparatus/MIA and the heads and deputy heads of a number of departments within the General Inspectorate of Police, Border Police, and the “Stefan cel Mare” Academy (SCM) following the first main mission.

The findings of the review were considered robust, sharp and timely both by the mandator and the MIA. They point to the exceptional progress made in 18 months in strategic management at the MIA  under the current minister, while identifying gaps in different areas. The report was used by the mandator (Sweden) to develop a programme by DCAF with the MIA, with a suggestion made in the AAR that findings from the baseline assessment can be carried forward by DCAF.

The recommendations are meant to quickly shore up gains obtained so far with the need to sustain reform in the longer term, also against the prospect of a change of tide in Moldovan politics. The question raised in the AAR is whether those recommendations can be implemented in the existing conditions, due to political constraints and limited human and financial resources.

Early on the mission, the team  had discussions with Swedish embassy staff on what the role of ISSAT was regarding capacity building and the question remains if ISSAT’s ‘reinforce not replace’ role was effectively delivered. There were plans to have an afternoon capacity building session with the embassy staff, which never materialised due to time constraints and uneven engagement. From the perspective of the mandator, though, the results were tangible in terms of apprehending methods and tools.

Strategic Management - Baseline Study Moldovan MIA

Success factors

Scoping and survey - The development of a template specific to the mission was vital in order to maintain a coherent framework for the analysis. The template questions were also shared widely so we were better able to manage expectations. The work carried out earlier on in the mandate allowed ISSAT to get to a point towards the end of the mandate where the team had a fairly comprehensive overview of key findings that could be tested out with key interlocutors (e.g. the EU High Level Advisor in the ministry). It also gave the team the confidence to be able to support the idea that detailed design of the subsequent programme could be initiated based on the draft report in order to save time.

Access to information/transparency– The degree of openness at the MIA is remarkable. If anything, there is too much information : over 100 strategies/concept papers across the MIA, all available to be shared. The progress made in transparency is quite extraordinary if we take in consideration that in 2012 the MIA was still perceived as “a closed system that lacked transparency, with Soviet-totalitarian governing practices” (MIA Functional Analysis).

Team composition- The team composition was important. ISSAT experts on team could count on the extensive knowledge of David Clarke, drawing on his understanding of the political undercurrents. This was a great example of ISSAT being able to bring together different resources.

Local support– Excellent support was provided by the advisor to the Minister.

Local knowledge– The mandate gained from the insight gathered by ISSAT during the NORLAM Moldova mission that had started slightly earlier.

Relevance of the report– As mentioned above, dozens of concept papers exist across the MIA, adding to the three functional assessments previously carried out by various donors (including EU). The risk for ISSAT baseline to be redundant was avoided by rooting the findings in very comprehensive information collected from a range of sources and using different tools (including an electronic survey, interviews, document analysis, etc.). From the MIA, the report is perceived as adding depth and scope to the existing analysis.

Credibility– The findings are not disputed in the MIA and the report was ‘generally accepted’, including by those who don’t accept that a review can be performed by a foreign organisation and experts from abroad. Such actors tend, if anything, to question the legitimacy of the approach – not the credibility and validity of the actual findings.

Audit /assessment of the Inspectorate General of Public Security in Burundi

Main challenges

Concepts and processes– Uneven understanding among different stakeholders, within the IGSP and across the MSP of the role, importance and mechanisms entailed by internal control in a functioning democracy.

Political engagement– As noticed in the AAR, even the most immediate quick-win activities recommended in the report need that IGSP has a legal framework in place, but political commitment from Burundian Government to move forward on this has been vague, in fact compromising IGSP critical role.

Programme support– The audit sets clear directions and suggests an action plan to address fundamental gaps in internal control, but any progress on this depends on a robust and immediate support from SSD to democratic oversight and accountability.

Programme overstretch– SSD is a complex, far-reaching SSR programme spanning for eight years and supporting the main security institutions in Burundi (Police and Defence Forces), but there is the risk that whatever is not included or prioritised and funded under SSD is not purposely sustained by the Burundian counterparts.

Uneven and unused capacity– IGSP badly lacks capacity across the board when it comes to main and secondary/procedural functions of an internal control body, from legal expertise to strategic management, while some of the actual existing capacity among staff is lost by the way IGSP functions.

Lessons identified:

  • Civil society organisations are usually keen to engage in discussions about internal control mechanisms of police services. Such occasions offer entry-points to bring-in external stakeholders, otherwise kept fairly outside relevant processes and activities, as it happened this time, in spite of clear demands from ISSAT for such opening from the Burundian authorities.
  • Training in internal control and democratic oversight, linking up internal audit bodies with the role of external stakeholders (Parliament, Ombudsman, etc.), are the natural and crucial follow-up of this mandate. These activities can be a contribution from ISSAT to pursue support in areas identified as major capacity gaps, and keeping the momentum for a reinforcement of the SSD Governance pillar. It’s worth reminding that “Internal Control” goes on being  one of the focus areas under the action plan for ‘Police’ pillar of SDD Phase III.
  • Dedicate the first day of the team in the field to set up interviews and organise agenda, since gaps in pre-arranging interviews are recurrent and almost unavoidable in such contexts.

Audit /assessment of the Inspectorate General of Public Security in Burundi

Success factors

  • A good preparation of the mandate, starting with a one-day meeting in Geneva aligning approaches and planning work among Polfed and ISSAT experts on team, starting from scratch and designing the mission.
  • Scoping visit from the team leader to Burundi.
  • Wealth of local/contextual knowledge in the team.
  • Mixed team composition bringing complementary skills and approaches to democratic control mechanisms: technical/Polfed and SSR/ISSAT.
  • Trust build over time between ISSAT and SSD in Burundi.
  • Good local interaction between SSD, CTB (Belgian development agency) and GIZ (German aid agency).
  • ISSAT methodology.
  • Facilitation from IGSP staff.

Lessons identified for improving the chances of success:

    • Recommendations can be discussed with local stakeholders in a way that allows for some knowledge transfer on programme design, in the case restricted to the elaboration of an action plan for ‘quick-wins’. Such an exercise can bridge recommendations with whichever projects that SSD decides to carry forward following the review.
    • SSD needs to act on the audit in terms of pushing to have some of the recommendations followed on; this implies that the Programme has to keep representing to the Burundian authorities that crucial decisions have to be taken and operationalised soon to have IGSP fulfilling its important role.

Audit /assessment of the Inspectorate General of Public Security in Burundi

Outputs and Outcomes

This mandate fully delivered the expected outputs, including an audit report with recommendations in accordance to the mandator’s needs. The recommendations were taken already in consideration and two subsequent missions from Polfed to Burundi worked on how to implement some of them. The reaction to the recommendations was immediate and positive also from the IGSP, which already took suggested points of action in consideration and welcomes the clarification of roles and responsibilities between the Inspectorate and SSD. During the mandate, the position of Inspector-general was vacant but the new head of IGSP took office in the same day of the end of the field mission. A working group was set up just after the restitution session to discuss a set of activities aimed at obtaining quick-wins on the recommendations and that ISSAT proposed to be carried out until Q4 2014.

To this date, the IGSP still didn’t act on most of these actions and further steps depend on the completion of the new legal framework for the IGSP. The Organic Law and the Internal Statute of the IGSP have been drafted and provide finally a legal framework for organising  the work of the Inspectorate, although a decree on the function of internal audit is still to be drafted. Until the legal framework is approved and fully in effect, the scope of activities of IGSP  is nonetheless very limited, specially when it comes to investigations and filing of cases.

The extent to which recommendations are fully implemented will need to be checked in one or two years, since a previous audit to IGSP was basically left in the drawer in terms of acting on suggested points. The review report delivered by the team provides a strong base for the design of SSD projects in support of the IGSP but it also implies and needs that the mandator follows up on the results by pushing for tangible actions (see below).

Throughout the report, attention is given to technical and procedural notions that help unpack the concept of internal control and that lay out a comprehensive set of principles, mechanisms and tools. This corpus can be easily taken as part of training materials for IGSP staff and the PNB overall, as acknowledged by the SSD/MSP coach visiting ISSAT in October to discuss further areas of collaboration related to training in police integrity.

As part of the second/main field mission, in March 2014, a one-day workshop was organised by the team in Bujumbura to conduct a SWOT analysis of the IGSP with a group of fifteen staff of the Inspectorate. The SWOT exercise was considered to provide simple yet powerful analytical tools, with gains in terms of ownership of the review itself.

The evaluation team was assisted in Burundi by two members from IGSP that facilitated contacts and participated in several meetings. Future will tell what will result  from this exposure of IGSP staff to ISSAT’s evaluation methodology and to intense discussions that articulated the principles of internal control with the realities on the ground and the political and social context in Burundi.

For ISSAT, the fact that the evaluation team included two experts working actually in police internal control in their own country provided an opportunity to gain a more technical view of the questions involved. It is considered a relevant gain of capacity for future reviews and an important enhancement of existing in-house knowledge on police reform, integrity and responsibility.

It is the perception of the IGSP that the mandate resulted in opening some doors to potential collaboration of the institution with civil society organisations.

Review of the Rule of Law Advisory Mission on Moldova (NORLAM)

Innovative approaches

NORLAM is working across most of the criminal justice chain, including the correctional and rehabilitation aspects. This is not in itself entirely a unique approach, but it is considered as rare for a single development partner to have such a wide reaching programme that aims to adopt a holistic approach across the chain. It is worthwhile to consider broader use of such an approach across the world of SSR. NORLAM has the resources and skill sets to work across the chain, including if necessary using the police prosecutor to work with the police. It understands the linkages between the different elements of the chain and has good relationship with all its partners. Working on only one element of the chain can cause imbalances within the chain that undermine the effectiveness of reform efforts and not yield sustainable results. Working on the entire chain requires more support but has higher potential for sustainability.

NORLAM resource pool of rule of law experts from Norway is an interesting modality. The particularity is that all experts are active practitioners in their field – which is also very rare in development programmes in SSR. It is also noteworthy that these experts come from the same country, knowing the same criminal justice system and speaking the same language. Communication among them is easier, than for example in ESDP mission where experts can come from 27 different EU states, with 27 different systems and languages. Communication and understanding the linkages between the different elements of the criminal justice chain is the key to success. Everyone has to stay informed on the others’ work and understand where the delays/misunderstandings in the chain are. 

The resource pool is, as such, not really an innovative modality, but it is interesting in terms of mapping global good practice in SSR to further study the opportunities and challenges that stem from using such a pool and in particular look at the Norwegian experience – including inter-institutional cooperation in Norway, recruitment and availability of staff, and pre-deployment training and selection.

Lessons identified on innovative approaches:

  • Resource pool of experts is an interesting modality to explore in the world of SSR. It is not a unique modality but there are ways to enhance its effectiveness.
  • Communication and understanding the linkages in the criminal justice chain are key factors for success.