Team selection and collaboration

Strategic Management - Baseline Study Moldovan MIA

Success factors

Scoping and survey - The development of a template specific to the mission was vital in order to maintain a coherent framework for the analysis. The template questions were also shared widely so we were better able to manage expectations. The work carried out earlier on in the mandate allowed ISSAT to get to a point towards the end of the mandate where the team had a fairly comprehensive overview of key findings that could be tested out with key interlocutors (e.g. the EU High Level Advisor in the ministry). It also gave the team the confidence to be able to support the idea that detailed design of the subsequent programme could be initiated based on the draft report in order to save time.

Access to information/transparency– The degree of openness at the MIA is remarkable. If anything, there is too much information : over 100 strategies/concept papers across the MIA, all available to be shared. The progress made in transparency is quite extraordinary if we take in consideration that in 2012 the MIA was still perceived as “a closed system that lacked transparency, with Soviet-totalitarian governing practices” (MIA Functional Analysis).

Team composition- The team composition was important. ISSAT experts on team could count on the extensive knowledge of David Clarke, drawing on his understanding of the political undercurrents. This was a great example of ISSAT being able to bring together different resources.

Local support– Excellent support was provided by the advisor to the Minister.

Local knowledge– The mandate gained from the insight gathered by ISSAT during the NORLAM Moldova mission that had started slightly earlier.

Relevance of the report– As mentioned above, dozens of concept papers exist across the MIA, adding to the three functional assessments previously carried out by various donors (including EU). The risk for ISSAT baseline to be redundant was avoided by rooting the findings in very comprehensive information collected from a range of sources and using different tools (including an electronic survey, interviews, document analysis, etc.). From the MIA, the report is perceived as adding depth and scope to the existing analysis.

Credibility– The findings are not disputed in the MIA and the report was ‘generally accepted’, including by those who don’t accept that a review can be performed by a foreign organisation and experts from abroad. Such actors tend, if anything, to question the legitimacy of the approach – not the credibility and validity of the actual findings.

Audit /assessment of the Inspectorate General of Public Security in Burundi

Outputs and Outcomes

This mandate fully delivered the expected outputs, including an audit report with recommendations in accordance to the mandator’s needs. The recommendations were taken already in consideration and two subsequent missions from Polfed to Burundi worked on how to implement some of them. The reaction to the recommendations was immediate and positive also from the IGSP, which already took suggested points of action in consideration and welcomes the clarification of roles and responsibilities between the Inspectorate and SSD. During the mandate, the position of Inspector-general was vacant but the new head of IGSP took office in the same day of the end of the field mission. A working group was set up just after the restitution session to discuss a set of activities aimed at obtaining quick-wins on the recommendations and that ISSAT proposed to be carried out until Q4 2014.

To this date, the IGSP still didn’t act on most of these actions and further steps depend on the completion of the new legal framework for the IGSP. The Organic Law and the Internal Statute of the IGSP have been drafted and provide finally a legal framework for organising  the work of the Inspectorate, although a decree on the function of internal audit is still to be drafted. Until the legal framework is approved and fully in effect, the scope of activities of IGSP  is nonetheless very limited, specially when it comes to investigations and filing of cases.

The extent to which recommendations are fully implemented will need to be checked in one or two years, since a previous audit to IGSP was basically left in the drawer in terms of acting on suggested points. The review report delivered by the team provides a strong base for the design of SSD projects in support of the IGSP but it also implies and needs that the mandator follows up on the results by pushing for tangible actions (see below).

Throughout the report, attention is given to technical and procedural notions that help unpack the concept of internal control and that lay out a comprehensive set of principles, mechanisms and tools. This corpus can be easily taken as part of training materials for IGSP staff and the PNB overall, as acknowledged by the SSD/MSP coach visiting ISSAT in October to discuss further areas of collaboration related to training in police integrity.

As part of the second/main field mission, in March 2014, a one-day workshop was organised by the team in Bujumbura to conduct a SWOT analysis of the IGSP with a group of fifteen staff of the Inspectorate. The SWOT exercise was considered to provide simple yet powerful analytical tools, with gains in terms of ownership of the review itself.

The evaluation team was assisted in Burundi by two members from IGSP that facilitated contacts and participated in several meetings. Future will tell what will result  from this exposure of IGSP staff to ISSAT’s evaluation methodology and to intense discussions that articulated the principles of internal control with the realities on the ground and the political and social context in Burundi.

For ISSAT, the fact that the evaluation team included two experts working actually in police internal control in their own country provided an opportunity to gain a more technical view of the questions involved. It is considered a relevant gain of capacity for future reviews and an important enhancement of existing in-house knowledge on police reform, integrity and responsibility.

It is the perception of the IGSP that the mandate resulted in opening some doors to potential collaboration of the institution with civil society organisations.

Assessment to inform potential options for future international support on Police reform in DRC

Mandat et équipe conjointe DFID et UE

From Internal AAR:

Officiellement dans les TdR, le mandat était un mandat conjoint DFID et UE. Mais les TdR ne clarifiaient pas du tout les vrais besoins de chaque mandataire. Officieusement, ce mandat était demandé par DFID pour leur donner des éléments pour la conception de leur business case pour leur prochain programme. L’UE n’était impliquée que pour des raisons de visibilité et de coordination. L’UE a ses propres procédures pour l’identification et la conception de leur programme. Heureusement que l’équipe ISSAT/SU était flexible et avait les compétences pour répondre aux objectifs officieux et s’adapter aux besoins des mandataires.

Leçons identifiées:

-          Clarifier au maximum dans les TdR les objectifs de tous les mandataires lors de mandat conjoint.Connaître le rôle de chaque mandataire et les raisons officieuses de leur demande de soutien. Soyez attentif au fait que ceci peut apparaitre petit à petit au cours du mandat.

-          Les mandats conjoints sont habituellement plus compliqués que les mandats provenant d’un seul mandataire. Il faut donc s’assurer que l’équipe soit flexible et très compétente. 

Assessment to inform potential options for future international support on Police reform in DRC

Planification

From Internal AAR:

La période des vacances n’a pas du tout été propice à la bonne planification du mandat. Ceci depuis la demande initial des mandataires à ISSAT jusqu’au déploiement de l’équipe sur le terrain. Les personnes de contact du côté des mandataires (DFID et UE) étant en vacances durant la phase de planification. L’équipe a donc observé un manque de communication, de soutien logistique et administratif. Il y a eu également un manque de référence aux documents clés à analyser avant la mission de terrain. De plus, les membres de l’équipe d’ISSAT, issus du roster, n’ont pas reçu toute l’information nécessaire sur les tenants et aboutissants de la négociation du mandat afin de bien se préparer pour leur mission de terrain. ISSAT ayant soutenu de manière continue DFID RDC depuis 2012, il aurait été intéressant de briefer l’équipe d’experts sur les résultats des précédents mandats.

Les TdR ont probablement été copié-collé d’un modèle de l’UE, assez peu détaillés et pas finalisés ni approuvés officiellement avant le déploiement de l’équipe sur le terrain. Les objectifs officiels, détaillés dans les TdR, étaient fort différents des objectifs officieux qui n’ont été communiqués à l’équipe qu’une fois sur le terrain.

Les partenaires locaux n’ont que très peu été impliqué lors de la phase de planification. De manière générale, les partenaires locaux ne reçoivent pas les documents de DFID qui ne sont pas traduits en français. Ceci est le cas par exemple pour les rapports d’évaluation du SSAPR rédigés en 2012 et 2013 conjointement par ISSAT et la SU. Ceci dit en passant, ces rapports étant très (voir trop) long selon certains, il est difficile et fort coûteux de les traduire. Il serait intéressant d’essayer de réduire la longueur des rapports auxquels ISSAT contribue.

Leçons identifiées:

-          Besoin de TdR clairs, détaillés, identifiant les responsabilités de chacun, ainsi qu’une personne de contact pour la logistique du côté du mandataire. Les TdR devraient être approuvés avant le déploiement de l’équipe sur le terrain.

-          Mener une mission préalable(scoping mission) de quelques jours est une des leçons apprises par ISSAT afin de mener à bien la phase de planification, voir de finaliser les TdR et obtenir une approbation officielle. Ceci n’a pas été possible pour ce mandat à cause de raisons diverses et variés (temps, disponibilités des experts, volonté des mandataires).

-         Traduire les documents clés pour les partenaires locaux. Intégrer le coût de la traduction dans le budget du mandat afin de mener à bien ce travail (car ceci n’est jamais fait par DFID). ISSAT et le mandataire doivent convenir ensemble de qui s’occupera de la traduction et partager les contacts de bons traducteurs (qui ne sont pas facile à trouver).

-         Communiquer avec les membres de l’équipe d’ISSAT, issus du roster, sur les tenants et aboutissants des négociations du mandat avant leur déploiement sur le terrain. Dans le cas où le mandat est la suite d’un mandat précédent mené par ISSAT, s’assurer de bien communiquer les résultats de ce dernier à la nouvelle équipe d’expert. Si cela est opportun, faire venir le Team Leader chez ISSAT à Genève pour 1-2 jours afin de discuter avec les membres d’ISSAT ayant suivi les précédents mandats.

Review of the Rule of Law Advisory Mission on Moldova (NORLAM)

Capacity building of the mandator

Capacity building activities were mainly carried out individually, through the participation to the field mission of one staff from Norad. It was clear from the start that someone on the mandator side had to participate to the mandate and it proved to be very useful. The evaluation methodology was not new to the mandator but applying it made the terms and concepts more understandable. The mandate took a structured and systematic approach in collecting and sharing the information, with day-to-day team briefings, planning the interview questions ahead of time, etc. This was considered to be very good. The involvement of Norad staff was limited to the field mission. It could have been better if he would have been involved also during the planning and reporting phases.

The role of ISSAT in terms of capacity building was not clear. It should have been clarified since the beginning in the ToR. NORLAM staff could have benefitted from capacity building activities on the methodology and the use of terms such as output, outcome, impact, etc. Moreover, they do not have the in-house expertise to implement some of the recommendations on strategic planning, project design, etc.

NORLAM staff considered that the ISSAT field mission was very time consuming in terms of planning interviews and logistics. They did not expect that at all and were very busy with their day-to-day work. The mission added a lot of work on top of everything else. Even though the ToR clarifies that “Overall responsibility for co-ordination of the team’s activities on the ground will be provided by NORLAM. i.e. making appointments, providing transport, etc.”, this investment should have been better communicated and someone should have been dedicated to that. There seems to be some confusion on the role of the local expert, part of the review team, in terms of planning and logistics.

It is recognised that time was a big constraint for this mandate. The field mission was very intense with a lot of interviews and only 10 days in Moldova. But this mandate was very well organised and very well led by the team leader. Team members were impressed by that. An additional 2 days on the ground could have been foreseen in order to collect additional information or meet again with some interlocutors to clarify certain issues.

Lessons identified on capacity building:

  • Include a staff member from the mandator side since the beginning of the mandate, from the planning to the reporting phase. Clarify her/his role in the ToR as well as capacity building activities to be undertaken during the mandate.
  • Propose a workshop on the evaluation methodology to the mandator staff in the field. This could be done in half a day at the beginning of the field mission.
  • Clarify and communicate as much as possible on who should plan interviews and take care of the logistics. Recognise that it takes time and plan that well in advance (3 weeks before the deployment of the team in the field).
  • If possible, leave two free days at the end of the field mission in order to be flexible to investigate further certain issues and plan additional interviews.