Collaboration with team members, international organizations and local partners

• Time constraints:while prior knowledge of the MoI from ISSAT team and assistance from Sida/SPAP greatly contributed to carry out the field mission, the team had only two weeks in Belgrade, which restricted the number of meetings and activities possible.
• Culture of secrecy: like if by default, all kinds of documents and information are classified in the MoI and it is difficult to navigate a mandate of this nature when the prevailing institutional culture and legal/policy framework is one of secrecy and opacity.
• Empowering while doing no harm– Capacities are uneven among mid- and senior management at the MoI and between units. The imbalance of capacity across units is compounded by uneven attention given to different units by senior managers and donors. In this mandate, the risk was that those units needing the most support would be undermined further by being highlighted as deficient in the report due to the inherent blame culture (the Analytics Directorate being an example). Asymmetrical support to some units while others take on expanding responsibilities posed another challenge: how to reinforce the most performing units (e.g. BSP) whilst not creating too much of a gap in capacity with the rest of the MoI, making them the object of jealously by their peers.
• MoI in transition– the MoI was looking ‘very inwards’ (sic) until Brussels gave its feedback to the screening under Chapter 24, and then preparing Serbia’s follow-up on the results; afterwards, the new Cabinet has been focusing attention in drafting the new Police Law. Discussion on gaps identified by ISSAT will thus depend on the Cabinet being more free of those processes to think on strategic management. The report ‘is a seed that can grow, but in other conditions’, as pointed out in the AAR. (See below also).

ISSAT produced a comprehensive report that evolved from the original request of doing a baseline report to one that looked at the various needs and made substantial recommendations. Note that the term ‘baseline’ suggests that programme areas have been already identified and the study is to see where the process evolved from. This mandate actually needed the programme areas to be identified.

In order to capture a broader array of views, an electronic survey was developed by the team and distributed to key personnel in the Central Apparatus/MIA and the heads and deputy heads of a number of departments within the General Inspectorate of Police, Border Police, and the “Stefan cel Mare” Academy (SCM) following the first main mission.

The findings of the review were considered robust, sharp and timely both by the mandator and the MIA. They point to the exceptional progress made in 18 months in strategic management at the MIA  under the current minister, while identifying gaps in different areas. The report was used by the mandator (Sweden) to develop a programme by DCAF with the MIA, with a suggestion made in the AAR that findings from the baseline assessment can be carried forward by DCAF.

The recommendations are meant to quickly shore up gains obtained so far with the need to sustain reform in the longer term, also against the prospect of a change of tide in Moldovan politics. The question raised in the AAR is whether those recommendations can be implemented in the existing conditions, due to political constraints and limited human and financial resources.

Early on the mission, the team  had discussions with Swedish embassy staff on what the role of ISSAT was regarding capacity building and the question remains if ISSAT’s ‘reinforce not replace’ role was effectively delivered. There were plans to have an afternoon capacity building session with the embassy staff, which never materialised due to time constraints and uneven engagement. From the perspective of the mandator, though, the results were tangible in terms of apprehending methods and tools.

Scoping and survey - The development of a template specific to the mission was vital in order to maintain a coherent framework for the analysis. The template questions were also shared widely so we were better able to manage expectations. The work carried out earlier on in the mandate allowed ISSAT to get to a point towards the end of the mandate where the team had a fairly comprehensive overview of key findings that could be tested out with key interlocutors (e.g. the EU High Level Advisor in the ministry). It also gave the team the confidence to be able to support the idea that detailed design of the subsequent programme could be initiated based on the draft report in order to save time.

Access to information/transparency– The degree of openness at the MIA is remarkable. If anything, there is too much information : over 100 strategies/concept papers across the MIA, all available to be shared. The progress made in transparency is quite extraordinary if we take in consideration that in 2012 the MIA was still perceived as “a closed system that lacked transparency, with Soviet-totalitarian governing practices” (MIA Functional Analysis).

Team composition- The team composition was important. ISSAT experts on team could count on the extensive knowledge of David Clarke, drawing on his understanding of the political undercurrents. This was a great example of ISSAT being able to bring together different resources.

Local support– Excellent support was provided by the advisor to the Minister.

Local knowledge– The mandate gained from the insight gathered by ISSAT during the NORLAM Moldova mission that had started slightly earlier.

Relevance of the report– As mentioned above, dozens of concept papers exist across the MIA, adding to the three functional assessments previously carried out by various donors (including EU). The risk for ISSAT baseline to be redundant was avoided by rooting the findings in very comprehensive information collected from a range of sources and using different tools (including an electronic survey, interviews, document analysis, etc.). From the MIA, the report is perceived as adding depth and scope to the existing analysis.

Credibility– The findings are not disputed in the MIA and the report was ‘generally accepted’, including by those who don’t accept that a review can be performed by a foreign organisation and experts from abroad. Such actors tend, if anything, to question the legitimacy of the approach – not the credibility and validity of the actual findings.

• A good preparation of the mandate, starting with a one-day meeting in Geneva aligning approaches and planning work among Polfed and ISSAT experts on team, starting from scratch and designing the mission.
• Scoping visit from the team leader to Burundi.
• Wealth of local/contextual knowledge in the team.
• Mixed team composition bringing complementary skills and approaches to democratic control mechanisms: technical/Polfed and SSR/ISSAT.
• Trust build over time between ISSAT and SSD in Burundi.
• Good local interaction between SSD, CTB (Belgian development agency) and GIZ (German aid agency).
• ISSAT methodology.
• Facilitation from IGSP staff.

Lessons identified for improving the chances of success:

• • Recommendations can be discussed with local stakeholders in a way that allows for some knowledge transfer on programme design, in the case restricted to the elaboration of an action plan for ‘quick-wins’. Such an exercise can bridge recommendations with whichever projects that SSD decides to carry forward following the review.
  • SSD needs to act on the audit in terms of pushing to have some of the recommendations followed on; this implies that the Programme has to keep representing to the Burundian authorities that crucial decisions have to be taken and operationalised soon to have IGSP fulfilling its important role.

This mandate fully delivered the expected outputs, including an audit report with recommendations in accordance to the mandator’s needs. The recommendations were taken already in consideration and two subsequent missions from Polfed to Burundi worked on how to implement some of them. The reaction to the recommendations was immediate and positive also from the IGSP, which already took suggested points of action in consideration and welcomes the clarification of roles and responsibilities between the Inspectorate and SSD. During the mandate, the position of Inspector-general was vacant but the new head of IGSP took office in the same day of the end of the field mission. A working group was set up just after the restitution session to discuss a set of activities aimed at obtaining quick-wins on the recommendations and that ISSAT proposed to be carried out until Q4 2014.

To this date, the IGSP still didn’t act on most of these actions and further steps depend on the completion of the new legal framework for the IGSP. The Organic Law and the Internal Statute of the IGSP have been drafted and provide finally a legal framework for organising  the work of the Inspectorate, although a decree on the function of internal audit is still to be drafted. Until the legal framework is approved and fully in effect, the scope of activities of IGSP  is nonetheless very limited, specially when it comes to investigations and filing of cases.

The extent to which recommendations are fully implemented will need to be checked in one or two years, since a previous audit to IGSP was basically left in the drawer in terms of acting on suggested points. The review report delivered by the team provides a strong base for the design of SSD projects in support of the IGSP but it also implies and needs that the mandator follows up on the results by pushing for tangible actions (see below).

Throughout the report, attention is given to technical and procedural notions that help unpack the concept of internal control and that lay out a comprehensive set of principles, mechanisms and tools. This corpus can be easily taken as part of training materials for IGSP staff and the PNB overall, as acknowledged by the SSD/MSP coach visiting ISSAT in October to discuss further areas of collaboration related to training in police integrity.

As part of the second/main field mission, in March 2014, a one-day workshop was organised by the team in Bujumbura to conduct a SWOT analysis of the IGSP with a group of fifteen staff of the Inspectorate. The SWOT exercise was considered to provide simple yet powerful analytical tools, with gains in terms of ownership of the review itself.

The evaluation team was assisted in Burundi by two members from IGSP that facilitated contacts and participated in several meetings. Future will tell what will result  from this exposure of IGSP staff to ISSAT’s evaluation methodology and to intense discussions that articulated the principles of internal control with the realities on the ground and the political and social context in Burundi.

For ISSAT, the fact that the evaluation team included two experts working actually in police internal control in their own country provided an opportunity to gain a more technical view of the questions involved. It is considered a relevant gain of capacity for future reviews and an important enhancement of existing in-house knowledge on police reform, integrity and responsibility.

It is the perception of the IGSP that the mandate resulted in opening some doors to potential collaboration of the institution with civil society organisations.

From Internal AAR:

Officiellement dans les TdR, le mandat était un mandat conjoint DFID et UE. Mais les TdR ne clarifiaient pas du tout les vrais besoins de chaque mandataire. Officieusement, ce mandat était demandé par DFID pour leur donner des éléments pour la conception de leur business case pour leur prochain programme. L’UE n’était impliquée que pour des raisons de visibilité et de coordination. L’UE a ses propres procédures pour l’identification et la conception de leur programme. Heureusement que l’équipe ISSAT/SU était flexible et avait les compétences pour répondre aux objectifs officieux et s’adapter aux besoins des mandataires.

Leçons identifiées:

-          Clarifier au maximum dans les TdR les objectifs de tous les mandataires lors de mandat conjoint.Connaître le rôle de chaque mandataire et les raisons officieuses de leur demande de soutien. Soyez attentif au fait que ceci peut apparaitre petit à petit au cours du mandat.

-          Les mandats conjoints sont habituellement plus compliqués que les mandats provenant d’un seul mandataire. Il faut donc s’assurer que l’équipe soit flexible et très compétente. 

From Internal AAR:

La période des vacances n’a pas du tout été propice à la bonne planification du mandat. Ceci depuis la demande initial des mandataires à ISSAT jusqu’au déploiement de l’équipe sur le terrain. Les personnes de contact du côté des mandataires (DFID et UE) étant en vacances durant la phase de planification. L’équipe a donc observé un manque de communication, de soutien logistique et administratif. Il y a eu également un manque de référence aux documents clés à analyser avant la mission de terrain. De plus, les membres de l’équipe d’ISSAT, issus du roster, n’ont pas reçu toute l’information nécessaire sur les tenants et aboutissants de la négociation du mandat afin de bien se préparer pour leur mission de terrain. ISSAT ayant soutenu de manière continue DFID RDC depuis 2012, il aurait été intéressant de briefer l’équipe d’experts sur les résultats des précédents mandats.

Les TdR ont probablement été copié-collé d’un modèle de l’UE, assez peu détaillés et pas finalisés ni approuvés officiellement avant le déploiement de l’équipe sur le terrain. Les objectifs officiels, détaillés dans les TdR, étaient fort différents des objectifs officieux qui n’ont été communiqués à l’équipe qu’une fois sur le terrain.

Les partenaires locaux n’ont que très peu été impliqué lors de la phase de planification. De manière générale, les partenaires locaux ne reçoivent pas les documents de DFID qui ne sont pas traduits en français. Ceci est le cas par exemple pour les rapports d’évaluation du SSAPR rédigés en 2012 et 2013 conjointement par ISSAT et la SU. Ceci dit en passant, ces rapports étant très (voir trop) long selon certains, il est difficile et fort coûteux de les traduire. Il serait intéressant d’essayer de réduire la longueur des rapports auxquels ISSAT contribue.

Leçons identifiées:

-          Besoin de TdR clairs, détaillés, identifiant les responsabilités de chacun, ainsi qu’une personne de contact pour la logistique du côté du mandataire. Les TdR devraient être approuvés avant le déploiement de l’équipe sur le terrain.

-          Mener une mission préalable(scoping mission) de quelques jours est une des leçons apprises par ISSAT afin de mener à bien la phase de planification, voir de finaliser les TdR et obtenir une approbation officielle. Ceci n’a pas été possible pour ce mandat à cause de raisons diverses et variés (temps, disponibilités des experts, volonté des mandataires).

-         Traduire les documents clés pour les partenaires locaux. Intégrer le coût de la traduction dans le budget du mandat afin de mener à bien ce travail (car ceci n’est jamais fait par DFID). ISSAT et le mandataire doivent convenir ensemble de qui s’occupera de la traduction et partager les contacts de bons traducteurs (qui ne sont pas facile à trouver).

-         Communiquer avec les membres de l’équipe d’ISSAT, issus du roster, sur les tenants et aboutissants des négociations du mandat avant leur déploiement sur le terrain. Dans le cas où le mandat est la suite d’un mandat précédent mené par ISSAT, s’assurer de bien communiquer les résultats de ce dernier à la nouvelle équipe d’expert. Si cela est opportun, faire venir le Team Leader chez ISSAT à Genève pour 1-2 jours afin de discuter avec les membres d’ISSAT ayant suivi les précédents mandats.

Capacity building activities were mainly carried out individually, through the participation to the field mission of one staff from Norad. It was clear from the start that someone on the mandator side had to participate to the mandate and it proved to be very useful. The evaluation methodology was not new to the mandator but applying it made the terms and concepts more understandable. The mandate took a structured and systematic approach in collecting and sharing the information, with day-to-day team briefings, planning the interview questions ahead of time, etc. This was considered to be very good. The involvement of Norad staff was limited to the field mission. It could have been better if he would have been involved also during the planning and reporting phases.

The role of ISSAT in terms of capacity building was not clear. It should have been clarified since the beginning in the ToR. NORLAM staff could have benefitted from capacity building activities on the methodology and the use of terms such as output, outcome, impact, etc. Moreover, they do not have the in-house expertise to implement some of the recommendations on strategic planning, project design, etc.

NORLAM staff considered that the ISSAT field mission was very time consuming in terms of planning interviews and logistics. They did not expect that at all and were very busy with their day-to-day work. The mission added a lot of work on top of everything else. Even though the ToR clarifies that “Overall responsibility for co-ordination of the team’s activities on the ground will be provided by NORLAM. i.e. making appointments, providing transport, etc.”, this investment should have been better communicated and someone should have been dedicated to that. There seems to be some confusion on the role of the local expert, part of the review team, in terms of planning and logistics.

It is recognised that time was a big constraint for this mandate. The field mission was very intense with a lot of interviews and only 10 days in Moldova. But this mandate was very well organised and very well led by the team leader. Team members were impressed by that. An additional 2 days on the ground could have been foreseen in order to collect additional information or meet again with some interlocutors to clarify certain issues.

Lessons identified on capacity building:

• Include a staff member from the mandator side since the beginning of the mandate, from the planning to the reporting phase. Clarify her/his role in the ToR as well as capacity building activities to be undertaken during the mandate.
• Propose a workshop on the evaluation methodology to the mandator staff in the field. This could be done in half a day at the beginning of the field mission.
• Clarify and communicate as much as possible on who should plan interviews and take care of the logistics. Recognise that it takes time and plan that well in advance (3 weeks before the deployment of the team in the field).
• If possible, leave two free days at the end of the field mission in order to be flexible to investigate further certain issues and plan additional interviews.

• There was excellent collaboration within the ISSAT team (and its various partners) as well as with the EU, UN, IADB, regional organizations, government counterparts and civil society. Indeed, many countries took the mandate VERY seriously setting-up extensive agendas with literally hundreds of meetings and site visits. The mandate successfully circulated the ISSAT brand. 

• When the EU HQ relationship with countries was strong and positive, the reception in country was (predictably) positive. Where EU country offices were less engaged in JSSR activities, or had tense relations with Brussels, the experience was less positive. This is, in some ways, unavoidable. 

• The ISSAT team devoted considerable time (4 months!)  to defining countries to visit in consultation with the EU. This was a long and pain-staking process and was entirely dependent on the client’s priorities and interests. The only suggestion is that the ToR reflect these challenges (obliquely, since they are political in nature) and expectations be managed accordingly. 

• Team collaboration was extremely positive including with the Programme Design Team who were very much dedicated to the task at hand.
• Team had some meetings with relevant partners such as the Usalama Forum which was a great meeting providing a lot of insight and information.

• Team should have planned more time to meet with other international and local actors for coordination.

• EU contributions were counterproductive.

• More efforts before the workshop might have been necessary to bring the EU on board.

• ISSAT was criticised by UNDP that they led the process too much in the field. ISSAT presented the preliminary results in front of stakeholders and UNDP saw that as a UN job.
• Working with the different partners on the field was challenging. Some partners refused to commit staff and those who were assigned to the process were not really committed or lacked the right capacity (UNDPKO and UNDP didn’t want to commit people to this process).

• ISSAT held the main pen throughout the whole process because they assessed if they didn’t, the output won’t be delivered on time.
• Team member was Special Assistant of SRSG in charge of RoL and this was a significant link in terms of organizing the whole mission.
• Related to the above point the role of this team member was key in terms of mobilizing the DRSG office to exert pressure on stakeholders to contribute.
• UNDP CO was in every major meeting and was kept in the loop regarding any relevant issues to the mission.
• Special Assistant to UN police Commissioner, SSR Adviser to SRSG and DR SRSG Office for RoL were kept in the loop.
• This high level mobilisation ensured that stakeholders remained committed and contributed actively.
• Police sub-team worked very well. There was clear leadership by UN Police Commissioner and LNP Commissioner. The UNDP representative was extremely experienced and contributed actively to the process.
• The sub-team had also a clear set of objectives that everybody was working towards. The team was task oriented and didn’t question the objectives.
• Partly related to that was the fact that the police sector was a clear-sighted one that knew exactly what kind of support they needed from the government.
• Also, this work dynamic could have been a result of the fact that the police force is a task-oriented disciplined one.

• Ownership and partnership is important but people have to be committed.

• Having ISSAT on the team worked very well to allow for an independent aspect to the review. The police experts on the team were part of the programme design with MoFA funding. As a result Mark D carried out some of the meetings on the side to gather objective information. An independent team gives stronger legitimacy to the report.
• Having ISSAT PO as a M&E resource person since this is the first evaluation for this programme and 2 members of the mission team have no previous experience doing evaluations.
• ISSAT PO and M&E resource person contributed significantly to the methodology design and ToR development. In particular in terms of the evaluation process chain: desk-top review, develop programme ToC, etc.
• Team split up to undertake interviews in order to optimise productivity.
• Balance within the team was very good. The team member who was part of the programme previously was an important addition to the team in order to open doors and as the institutional memory for the programme and the programmatic/political process.
• Stakeholders shared information more easily when they spotted the team member whom they knew from the program.

• When team was being set up SU was aware of the importance of having a UK core staff member (since the mandate needed to have a proper SU hat because of the nature of the programme which has a strong SU tie), military background and knowledge of the programme and the context
• The value of a SU core staff member was his knowledge of SU. Perceptions on the ground can be very different if you have a core staff member. In particular vis a vis the implementing company (ASI); but also with respect to national stakeholders. Team felt that they were taken more “seriously” with core staff member on board.
• The value of having Mathiew on the team was high because of the fact that he was a military person who has already served in South Sudan.
• ISSAT Advisor was the lead on the ground but not during the report-writing process. SU owned entirely that process.
• ASI is the implementing partner for this program. Their role is a positive one dues to many factors:
  • Most of ASI members had already worked for HMG and maintained strong knowledge of the processes
  • The provided continuity on the field in the absence of HMG presence
  • The represented an a-political face to HMG which allowed them to push their agenda further to wider horizons
  • Because of their expertise (all ex-senior civil servants), the team believes that the British embassy will never be able to match their added value in the field.
  • HMG was very much focused on Afghanistan so ASI was a necessary partner to support HMG in its work in South Sudan

• Training participants were negative especially in the beginning. They didn’t understand the point of this training and they were sure as to what they were doing in that room.

• Mission Lead: One of the mission leads lacked basic leadership skills related to strategic vision and facilitation of team efforts for optimisation of results. Mission lead played the role of micro-manager of which team member goes where or does what, weighing too much on individual styles of work and thus frustrating the team. Also the team leader in question obviously lacked minimum levels of motivation for this mission. This might be linked to the fact that she was losing her job right after the mission. Team debriefings: end-of-day team debriefings were mostly absent and when organised they were unstructured and dealt with irrelevant issues thus becoming an obstructive practice to the performance of the team.

• ISSAT’s policy for deploying two experts:this worked very well in terms of creating an ISSAT nucleus to counter-balance the bad practice of the team leader. Both ISSAT Advisors agree that the presence of the other was a very important factor of support and facilitated exchanges vis-à-vis the mandator.
• Division of labour: In cases like this where team dynamics are more obstructive than constructive, insisting on division of labour and agreeing on specific tasks for team members is key for progress. ISSAT team pushed for a clear list of deliverables that they then sent around in a written email to ISSAT leadership and mandators so as to clarify as much as possible work flows and expectations for post-mission work.
• Transition of information between ISSAT teams: the mission team was briefed by ISSAT Advisor who was previously involved with the same mandator in the target country. However, the change in the employment situation of the mandator (loss of post) influenced immensely a change in behaviour and performance. 

• Mission Lead: Mission leads need to dispose of minimum leadership skills. This is something that lies outside the scope of ISSAT’s direct influence. However, due to the important negative effects of lack in leadership on the performance of the mission, ISSAT should use its Advocacy and Outreach service line to raise awareness on this issue. Also, it is preferable not to send persons who are about to lose their job on missions. This is another factor that might lie beyond ISSAT’s direct scope of influence but could be tackled through the Advocacy and Outreach Service Line.
• Attitude of ISSAT Advisors in absence of leadership: Some strategies that have been identified by ISSAT Advisors who were supposed to operate under team leaders that did not dispose of leadership skills included one-on-one discussions with team leader to clarify areas of misunderstanding and also confidence-building for the team leader in parallel with the mission.

• ISSAT Advisor’s role and image:
  • ISSAT Advisor was introduced by mandator (PBSO) to national authorities in an official letter as a “consultant” and no mention of ISSAT was made. As a result, she was perceived and dealt with as a separate entity from the mandator (PBSO) and with no relation to ISSAT. This did not help the image ISSAT is trying to promote of itself (reinforce and not replace).
  • This perception was aggravated by the fact that ISSAT Advisor was deployed alone on this field mission. In theory, she was supposed to deploy on a team composed of ISSAT and mandator representatives (PBSO and/or UNMIL) but due to lack in time and resources, this did not happen.

• UNMIL leadership in flux:
  • The SRSG, who was instrumental in many aspects of the PBF and PBC engagement in Liberia, left the country at the end of January (her replacement has not yet been announced); the DSRSG for Rule of Law had arrived in Liberia only a month or two before the mission deployed.

• Mission-planning and communication on mission timeline and objectives:
  • New York-based and Liberia-based interlocutors (PBO and UNMIL representatives) were not consulted and ill-informed of this mission which limited to a large degree team collaboration. For more details, see above comments under the planning section (Communication for mission-planning)

• ISSAT Advisor role and image:
  • It might be useful for ISSAT to consider distributing ISSAT Business cards and ISSAT email accounts to its Roster level 2 members, inorder to strengthen their ISSAT image vis-à-vis the stakeholders.

• Disconnect between ISSAT interlocutors for mission-planning and ISSAT’s stakeholders when mission-implementing:
  • The mandator’s team on the field was not aware of the mission upon arrival, and were not aware of its expected outcomes. As this differed from the ISSAT understanding and the fact that the Chief was away during the start and significant amount of time during the mission resulted in wasted time and efforts.
  • Field staff members were continuously travelling in and out of office which made it difficult to have enough time with substance experts. This should have been flagged during mission planning as an important risk to be mitigated.

• Mandator’s hierarchy:
  • The mandator’s office is very much managed on a personal-based approach as opposed to institutional or organisational approach; this made it very difficult for staff to take initiatives in the absence of the Chief and complicated the progress of the mission

• Team dynamics were very positive. Qualities such as patience, mutual understanding and ability to challenge allowed team to gain better understanding of issues at hand.
• Three person teams work very well.

- Excellent collaboration between the Kosovo NGO and the international partners (DCAF and OSCE).

- Excellent.

- Coordination is always challenging, try to support national efforts to coordination international support: It was difficult to be able to map out or even understand the large number of international activities support police and justice reform, as there was no clear coordination mechanism or central repository of such information. Supporting national efforts to coordination international support, would help overcome such confusion.

- Pre-planning meeting of team ensure coherence of approach: Meeting ahead of time to prepare and plan, divide responsibilities etc was viewed as extremely useful.

- Focus groups can facilitate information gathering: Hosting a series of focus group meeting with members of civil society etc to gather information and to build interest on SSR issues was also viewed as useful. A number of focus groups were held outside of the capital which ensured more representative views on security and justice service provision were obtained.

- Engage with ‘national coordination mechanism’ to gather information on reform efforts and international support: If possible aim to plan the mission to coincide with a Coordination meeting (if they exist) of international actors working in this area, this can help to cut down on the large number of interviews with the international community, it can facilitate contacts and speed up understanding of activities already on-going. 

- In this case, Guinea’s international partners called the shots and dictated many aspect of the mission, indeed imposing some language and agendas that may not have been of the choosing of Guineans.

- I pushed as much as I could to remind the central dimension of SSR assessment of local ownership and succeeded in some instances to get local partners’ views and interests influence decision.

- Thought this will continue to be a challenge in international donor dependent SSR, it is essential for parties that fund SSR missions not to dictate their views and impose agendas just because of their control of the funding.

- The inclusion of a member of the Albanian State Police on the team was important for several reasons. Firstly, it ensured that the design of the programme received a reality check. Secondly, the ASP representative ensured that key persons within the ASP were kept informed of developments throughout the process, thus maintaining support and enthusiasm for the design project. Thirdly, the ASP representative will now take on a general coordination role between the ASP and the programme implementation team, which means that there will be continuity throughout the process.

- The fixer took on the additional role of interpreter between the ASP representative and the rest of the team – both during missions and during the interim planning phases. This was crucial to creating the conditions for the team to benefit fully from the contributions of the ASP representative. 

- The national consultant brought expertise to the team and her knowledge and networks in the area of local government were invaluable.In addition, the national consultant was able to provide an insight into cultural and contextual issues from outside of the State institutions. This provided a useful balance to other opinions held within the team.

- The team had very close coordination with the key international actors active in the police sector and related areas. This was, in part, due to previous relationships, but the level of cooperation was enhanced by the timely sharing of information and the level of transparency maintained throughout the process. 

- If possible, include representatives from the State and Non-State sectors in the team. In addition to technical knowledge, awareness of process (in general and specific to the local context) is very important.

- Proactive sharing of information with other donors and international organisations is key to building cooperation. 

• Great team work,  good division of labour between members of the team due to:
  • Very good expertise in terms of thematic (justice and security) and context (SL and DFID working methods);
  • Team members knowing each other well;and
  • Good ToR, discussed beforehand and circulated to all team members.
• Great support from DFID SL and the implementing partner.

Lessons identified for improving the chances of success:

• Have a mixed team with:
  • strong and relevant thematic expertise;
  • knowledge of the country and programme context;
  • knowledge of mandator’s methods and procedures;and
  • who knows each other.
• Circulate and provide comments on the ToR before hand.
   
• Ensure strong support from mandator’s field office/Embassy and programme implementers.

• Newly formed teams: The challenge of newly formed teams is that members are not familiar with each other, their skills, competencies and methods of work.
• See comments on Burundian team members’ contribution under methodology

• The size and skills of the team: The team as it was defined worked very well in terms of its size and type of experience. Having 3 experts deployed for the evaluation of a big programme was the appropriate number as each advisor had one pillar to focus their efforts, while collaborating with the others on the other pillars. In terms of the skills and competencies deployed, this worked very well also. There was an expert on the thematic area (evaluation) and one expert on the technical questions (police and military) and one team leader who also dealt with governance aspects.
• Pre-deployment preparation: All the team met in Brussels before deploying to Burundi to harmonize information, approaches and work plans.
• Time dedicated for team work factored in daily: This team met once every day at the end of each day to recap with the team main findings, challenges etc this helped harmonise the picture for all team members. These working sessions were of crucial value to ensure a certain coherence of work between members thus allowing for building on each other’s work as opposed to duplicating or collating information without knowledge of its value to the objective of the evaluation.
• The Mandator’s role in evaluations: The mandators were very much hands-off when it came to the team’s work. They did not interfere with the design or planning of the mission and left the team entirely free in undertaking the evaluation.

• Collaboration with the Burundian team members should have been managed better to get better results on both sides (see section on methodology and approach).
• Time dedicated for team work factored in daily: This is absolutely key to the success of a mission. These meetings should be at least an hour long. These meetings were missing in a similar mission in Congo which had a big negative impact on the team’s work).
• The Mandator’s role in evaluations: A hands-off approach by the mandator in terms of the substantive part of the evaluation and assuming the role of a partner resourceful of the historical background of the programme is a very positive framework to encourage. 
• Collaboration with the Burundian team members should have been managed better to get better results on both sides (see section on methodology and approach)
• Time dedicated for team work factored in daily:  This is absolutely key to the success of a mission. These meetings should be at least an hour long. These meetings were missing in a similar mission in Congo which had a big negative impact on the team’s work).
• The Mandator’s role in evaluations: A hands-off approach by the mandator in terms of the substantive part of the evaluation and assuming the role of a partner resourceful of the historical background of the programme is a very positive framework to encourage.

• Excellent collaboration support and interaction with the international partners. A strong show of support from key players throughout (through the presence of senior management during the whole workshop). The two donors organising this event shared professional, dynamic and extremely healthy partnerships. Collaboration was very positive and both parties shared common understanding of the context and worked for the same goals. This excellent model for cooperation contributed to the enabling environment for the success of the event.
• Some of the enabling factors for such a collaboration are:
  • Collocation of offices
  • Common understanding of issues
  • A sense of importance attached to the workshop
  • Highly experienced professionals

• Personality factors

• Not all trainers had had training experiences with ISSAT and so, not everyone was familiar with the material and ISSAT’s approach. This was aggravated by the lack of preparation time for the team before deployment;
• The local partners were very much involved, but sometimes there were misunderstandings (perhaps due to cultural differences and/or expectations)

• Overall the training team worked very well together, especially taking into account the challenges expressed previously;
• High interest in this exercise was expressed by Rwandan counterparts and keen commitment to take things forward was apparent, this facilitated interactions and the level of commitment by all concerned;

• More time needed for preparation and informing facilitators about ISSAT’s way of operating, including the material. This can be done through ToT courses and should be based on the training skills of the members.

• Inter-ministerial approach: The mission was a MoD driven initiative but financed and headed by Norwegian Agency for Development Cooperation (NORAD).  The team was representative of these two bodies in addition to independent ISSAT team member. This was reflected in a difference in approach and methodology that had to be managed on the field. The team overcame its differences but near the start of the work, clear dynamics emerged with tensions between the Norway MoD team members and the development members.

• ISSAT Policy Advisor as a neutral mediator: At the beginning of the mission, the team leader asked for a meeting to try and reach common understanding amongst different team members on the objectives of the mission. ISSAT representative facilitated this meeting adding the value of being external to Norway’s internal debate. This meeting allowed all parties to put forth their ideas and concerns and thus move forward in a more homogeneous direction on mission objectives. This was enhanced by ISSAT’s Advisor’s efforts to tie the debate to ISSAT’s professional experience and tools on international actors’ good practice.

• Mission team lead by the development component: In this mission, where the team was composed of NORAD and MoD representatives, it was certainly beneficial for the development component to be the lead. This allowed for the necessary flexibility in the approach.

• Solid network of contacts for ISSAT PA: Collaboration with international actors went well as a result of the previous collaboration ISSAT PA had with many of them as a result of her past assignment.  

• See recommendation on Who’s Who Directory under Methodology section.

ISSAT team work:

• Team worked well together through complimentary roles, professionalism, openness, flexibility, listening, and confidence in the topic. All members of the team had not worked as one group previously

Integrating new members to team  

• The above mentioned values for team work facilitated integrating experience of new team member
• Mission preparatory activities allowed for team building
• New team member used first day of training to observe other trainers and understand their operating procedures

Preparatory work:  

• Advance preparation days in country allowed for planning and team building
• Preparatory activities resulted in clearly shared plan with responsibilities, tasks and time lines

• Reinforce team presence at the end of the session: At the end of sessions or at the end of the day, the current facilitator may ask the other facilitators if they have any final comments. If time permits, this gesture reinforces the “team” image of the trainers vis-à-vis the participants.
• Shuffling of groups on daily basis: Maintain changing of groups as opposed to consistent team’s composition to facilitate exchange of experiences and networking.
• GB members’ inclusion on team: When possible, GB members should be included on the teams for stronger and consolidated collaboration. In the cases where GB members are participants, their contributions could be brought forward during the various sessions.

• Relationship Between Local NGOs: Some other NGO participants who were invited to the workshop did not see the point in participating as they considered any work done, to be work done for Fundasaun Mahein  and not for themselves.

Good connections in DRC: Major IO, international and national NGOs involved in the justice system were consulted with fairly open dialogue.

• Collaboration with international partners: There appeared to be insufficient dialogue between internationals and Guineans. Most dialogue was narrowly targeted which can create programmes and projects that are not complementary and can divert resources from a nationally led holistic process. In an opportunity such as National Seminar that brings so many stakeholders together, partners should be working with each other and avoid trying to implement pet projects that can distract the process.

• Collaboration with mission team: It was very positive. All members were dedicated to the success of the seminar and worked hard to that end.
• The role of Gén. Cissé was key to the success to the mission. His position as coordinator and personal attributes as a well-respected, experienced and well-reputed senior politician supplied the political cover necessary for the technical mission.
• His presence within the technical team was also crucial as opposed to being positioned within UNDP HQ in Conakry. His presence shielded the mission’s technical team from political pressure and communicated their recommendations through appropriate channels.
• The role of Col. Mahamane (UNDP SSR field focal point) was crucial to the success of the seminar. His impact as the agent responsible for longer term capacity development (1 month before seminar in addition to support after seminar) was key in getting National Technical Committee up to speed. His multidisciplinary competencies and knowledge and skills in facilitating, coaching and accompanying the Guineans was key to success of whole mission and constituted needed link between ISSAT mission and the Committee.
• Collaboration with national partners: It  was easy and it was clear that our interlocutors were committed to improve, handling matters as best as they could; although, lack in capacities and/or awareness was visible

• Capacity development: Capacity development of partners and local actors should have been done right after the national assessment in order to create continuity and prepare partners for programme design.

Clarity: There was a very good working relationship among the team as roles, responsibilities, and objectives were all made clear ahead of the mission.

Uniform Culture: When working with security actors, different uniforms (i.e. police, military) hold different traditions and institutional mentalities. If you can be aware of these factors well in advance, respect them, and work with them as opposed to working against them, then the collaboration with partners from these groups will greatly improve.

Team Coherence: Without roles and guiding standards set out well in advance of a mission this can complicate team roles and responsibilities.

• Politicisation of security forces
• Reluctance/resistance to obtain technical assistance from the international community
• Impunity and indiscipline in the rank of security forces

• Local partner

- Strong buy-in from one Prime Minister of the Government on National Reconciliation
- Good working relations with the leaders in the Forces Armees des Forces Nouvelles (FAFN) zone
- Constant and ongoing communication with the heads of the security forces even during public disorder and violent manifestations (Nov. 2004, Jan 2006)
- Regular meetings with national security actors from the south and the north

• United Nations

- Good leadership within the mission
- Fairly good integration of all UN sectors (work in progress)
- Good operational interaction among UN and International actors

• None

• Politicisation of security forces
• Reluctance/resistance to obtain technical assistance from the international community
• Impunity and indiscipline in the rank of security forces

On the local partners' side:

• Strong buy-in from the President of Haiti and the Director General of Police
• Good working cooperation with most leaders  of the HNP  
• Good leadership within the Rule of Law pillars
• Good integration within Rule of Law
• Good operational interaction among National,UN and International actors

• None

None

• The commission drew extensively on published academic and policy material regarding intelligence reform, including the material available on the website of DCAF.

None

• Time for Collaboration: The ISSAT team visit was too short and not prepared well enough to establish a form of structural collaboration with the international institutions in the DRC nor with local DRC partners of the DFID project.

None

• Coordination for Confidence Building: Coordination is not just a matter of putting people and organisations together, it is a form of confidence-building.
• Preparation Time: ISSAT should insist on better preparation of contacts with international donors and with local counterparts before the ISSAT team visit.

• The team consisted of one program officer from HQ, a parliamentary expert, and myself (country and area expertise). The team was small, which allowed us to maneuver smoothly and make adjustments as needed. However, not having any national staff attached to the team posed a number of challenges, including on logistics, understanding recent political/security developments, and contextual analysis.

• The team met with a number of national civil society organizations to explore possible partnerships. These meetings provided the team an opportunity to assess potential national partners. We also met with a number of nationals (unaffiliated with any organization) who were knowledgeable about political affairs who could provide an unbiased assessment of parliament and assistance to the Assembly.
• The team consulted with a number of international organizations to cross-check information and better understand the working environment on the ground. These meetings focused on understanding the political situation, rather than on finding strategic partners.
• The team also met with the donor agency putting out the call for proposals. We did this mid way through the mission to allow us enough time to form our own thoughts about assistance to parliament, but still have enough time to follow up relevant leads.

• Team composition matters a great deal, especially in unstable security environments. Having national staff to provide input, feedback, as well as assist with planning, would have been very helpful. Consulting with national organizations and individuals provided us some of that feedback, although it would have been useful to have someone we knew and trusted.

• Collaboration within the team was compromised by having one member in the US while we were in Asia.

• For both proposals, expat and national staff were consulted extensively on drafts once completed. Given that they are responsible for implementing the program once it is awarded, it is obviously key to ensure their buy-in.
• Staff in all relevant offices were consulted and asked for input throughout the process. While this rarely involved their outreach to local stakeholders, at least they would be brought into the process and not just handed a product to implement.

• In assembling a team for a particular mission, that team should be deployed as a unit. Having additional team members contribute virtually is likely to compromise the process. I would suggest establishing a team that can work together, and if needed, bringing on experts later in the process for quality control or for a particular expertise (not reflected in the team composition).

None

• Partnership: A core concept in drafting the White Paper was that of 'partnership', with the main partners being the ministers responsible for defence, the parliamentary defence committee, the civilian defence secretariat, and the leadership of the armed forces. It was recognized that the relationship between these actors would entail tension and disputes from time to time because of their different responsibilities and perspectives, but that the concept of partnership would ensure stable civil-military relations.  
  The international community was not involved in the White Paper process.

None

• Nature of Trust funds: Trust Funds usually work through the lowest common denominator. Each member around the table has the opportunity to make the process less ambitious by being less of a risk taker. Because there needs to be consensus, if one project partner is not on the same policy level for one reason or the other, his/her position will imperatively need to be facilitated. Solutions and concessions need to be made. One partner could skew the whole process, even if all the others do not agree.

• Nature of Trust funds: Having international and national actors around a table all tied in with the same project forces people to speak and share information. It also forces them to find common visions and try to get to the same understanding on things.

• Trust Fund implementation: We need to be much clearer and tighter when we establish Trust Funds, in particular in terms of  modalities of implementation of the Trust Fund, instructions on how it should work and its specific objectives, leaving nothing to chance.

• Donor coordination: As the study was financed and administratively managed by Sweden, donor harmonization was a specific challenge. However, by informing and listening to bilateral and multilateral agencies and, finally, integrate the final results into the overall donor agenda, the process could benefit from the full weight of the international community.

• Benefit of the lead actor: In spite of the strong focus on national ownership, the fact that Sweden coordinated the study, and not a donor consortium, meant that technical transaction costs could be kept low.
• Benefit of substative collaboration: Even though Sweden had this role as a catalyst, the national ownership over time and donor engagement in the final process meant that the reach of the political impact increased and a technical study could help provide better policy options for a variety of actors.

• Joint assessment: Aware of the increased transaction costs, it would have been interesting to maybe have 2-3 core donors as catalysts for the study. This could have increased the benefits of the process in itself and the likelihood of sustainable outcomes. Still, one should be lead donor as to assure that substantive, and not only administrative, aspects are harmonized.

• Coherence across donor programmes: No clear coherence was established from the support provided by other donors had been established with the Swiss support for the DGMMA. This can lead to the conflicting use of terminology and the overlap of goals in each area of support. For example, the DGMMA and a training institution at a parallel or lower level could be providing the same courses for the same target audience when there are limited resources at the SPLA’s disposal for training.

• Multi-donor participation: Having different members of the international community who are engaging in SPLA training roles attend the workshop allowed information from all parties to be brought forward and clarified. It also motivated the international actors and SPLA to coordinate training development further.
• Filling donor gaps: Constraints faced by the Swiss on providing assistance on military operational effectiveness allowed for the opportunity of greater donor coordination by beginning to identify other partners who could fill in this gap and provide the DGMMA with assistance.

• Donors meetings: Further improvements on coordination are required to make the DGMMA fit in with what is being developed in other parts of the SPLA Training Strategy. Regular meetings between the donors supporting SPLA training development would be a first step with the SPLA driving these meetings.
• Co-location: The Swiss establishing an office in the SPLA headquarters would also improve collaboration on moving the DGMMA forward by creating a closer relationsip.