Timor-Leste economic policy forum

final report External Evaluation of Land and Housing Justice Action

4/30/2016

Final Report
External Evaluation of Land and Housing Justice Action
EC Funded Project 2013-2016
November 2015
Prepared by
Tomas Freitas
Independent Evaluator
Dili, 30th of November 2015

Contents                                                                                                            2
Acronyms                                                                                                           4
Acknowledgments                                                                                               5
Executive Summary                                                                                             6
Introduction                                                                                                          9
Research Methodology                                                                                      10
Chronology Evaluation                                                                                       12
Program Rationale and Logic                                                                          13
Interview findings - Haburas                                                                             14
Interviews with members of Haburas                                                             15
Interviews with Rede Ba Rai Secretariat                                                          16
Interviews with members about the RBR Secretariat                                    17
Interviews with Matadalan Ba Rai                                                                    20
Interviews with Land Defence Group of Covalima                                        20
Interviews with HAK                                                                                          22
Interviews with Grupo Alvu Stratejiku of Baucau                                          23
Interviews with DNTPSC of Baucau                                                                 24
Interviews with Grupo Alvu Stratejiku of Viqueque                                       25
Interviews with Partners of HAK and FTM in Viqueque                                26
Interviews with BELUN                                                                                     27
Interviews with Partners of BELUN                                                                 28
Interviews with DNTPSC of Viqueque                                                           29
Interviews with FTM                                                                                        29
Interviews with Partners of FTM in Maun-Fahe                                            30
Interviews with DNTPSC of Maun-Fahe                                                         30
Interviews with Luta Ba Futuru in Maun-Fahe                                              31
Interviews with the Community of Betano in Maun-Fahe                          31
Interviews with Kdadalak Sulimutu Institute                                                32
Interviews with UNAER in Ermera                                                                33
Interviews with Grupo Defende Rai of Oecusse                                         35
Table of Summary of Findings per activity                                                   37
Conclusion and Recommendations                                                               40
Annex of List of Interviewees                                                                        41

Acronyms
BELUN                         Instituisaun BELUN
CEFOBOM                   Centro Foin Sae Boa Ventura
CO                               Committee Organiser
DNTPSC                       Dirasaun Nasional Teras Propiadade Sistema Cadastral
EC                                European Commission
ETM                             Ekipa Tau Matan
FECM                           Fundasaun Edukasaun Comunidade Matebian
FTM                             Forum Tau Matan
GAS                              Grupo Alvu Stratejiku
GDR                             Grupo Defende Rai
HABURAS                    Fundasaun HABURAS
HASATIL                       Hadomi Agrikultura Sustentavel Timor-Leste
HAK                             Asosiasaun Hak Azasi dan Keadilan
JOHALD                       Joventude Hamahon Dezenvolvementu
KSI                               Kdadalak Sulimutu Institute
LAIFET                         Labour Institution for East Timor
LBF                              Luta Ba Futuru
LHJA                            Land Housing Justice Action
LH                                Lao Hamutuk
LUTA HAMUTUK         Institusaun Luta Hamutuk
UNAER                         Uniaun Agricultura Ermera
MANEO                       Masine Neo Oecusse
MBR                            Matadalan Ba Rai
PMU                            Project Management Unit
RBR                             Rede Ba Rai
RDTL                            Republica Democratica of Timor-Leste

Acknowledgments
The author would like to thank the logistical and communications support provided by HABURAS during this assessment. Special thanks to Fernando Antonio, Roberto A da Cruz, Horacio Goncalves Da Costa, Antonio Delimas, Rozito, Bendito, and Januario for their assistance during the interviews in Dili and also in the municipalities. The author would also like to thank the HABURAS Director as well as the Project Management Unit and all the staff who have contributed their time and knowledge during the evaluation process. Lastly, I would like to thank my research assistance Nelson Seixas Miranda for his hard work in taking notes during the evaluation process. The views expressed in this document in no way reflect the views of HABURAS or the delegation of the European Commission in Timor-Leste. For further clarifications or additional information the author can be contacted on thomas_freitas@yahoo.com

Executive Summary
This external evaluation was an assessment to measure the general achievements of each and every partner in delivering the planned activities, as well as the way they managed the activities. Haburas is the main partner; other consortium members include HAK, KSI, FTM, BELUN, and RBR, implementing the Land and Housing Justice Action program, which is funded by the European Commission (EC).
The program has been implemented for the last 14 months; according to the set goals, Haburas and its partners have to collaborate with State Actors (SAs) and Non-Sate Actors (NSAs) at the municipal and national level. This project was developed through various activities such as training, dissemination of information, prevention, awareness-raising, promoting dialogue, promoting the rights of vulnerable communities to land and housing, advocacy, networking, lobbying, and provision of legal assistance.

While there are achievements there have also been challenges, both for the partners as well as for the Project Management Unit (PMU).

To better understand this situation, the HABURAS foundation recruited an independent evaluator to do an assessment of the program and activities during the contract.

Besides this main report, the evaluator also provided a collection of Annexes which includes a document on management of Rede Ba Rai, an analysis of the human resources for implementing this project, and a description of challenges and obstacles faced by the PMU.

The evaluation took more than twenty working days, involving people from Dili, and communities from Baucau, Viqueque, Maun-Fahe, Covalima, Ermera and Oecusse. This assessment gathered around 20 organisations, 4 state actors, 16 community leaders, and 28 ordinary people; altogether more than 76 people were involved in this assessment.

There are a number of achievements that have been gained by HABURAS and partners of the Consortium such as: HAK, which has been very successful in empowerment of its partners in Baucau and Viqueque; members of GAS such as the Village Chief of village of Tirlolo who is able to explain the right to equal land and housing, and a female lawyer in a local NGO who can articulate in simple terms Articles 54 and 58 of the Constitution. The evaluator also admired other GAS members in Viqueque who are strongly committed to success in this program even though they are volunteers only.

The evaluator has been amazed with the work from FTM in Viqueque and Maun-Fahe (Same) which has socialised the right to equal land and housing at the village level, even though it has at times faced difficult questions from the community.

The evaluator was excited by all the hard work that has been done by Matadalan Ba Rai, in improving the capacity of their partners Grupo Defende Rai (GDR); for example in Covalima, they have defended victims in Camenasa and Oques that has been affected by mega-projects such as the Suai Supply Base, Airport and Auto Estrada. In Oecusse they have defended the rights of people in Sakato, Nepani, Lifau, Lalisu, Cunha and Tono, that has been affected by the mega-project of ZEEMS.

The evaluator is also impressed with KSI and the hard work that they have done with their partners UNAER in Ermera. Based on the interviews with seven of the members of UNAER in Gleno, the evaluator was impressed with their answers to the evaluators questions; for example two of participants who were still of quite a young age and had only attended training 3-4 times, were able to explain very well about the concept of land reform. The evaluator was also impressed by the explanation by the Village Chief village of Buruma, who was able to explain how to facilitate a land dispute mediation, which can be seen to be a result of the hard work done by BELUN with their mediation training.

There are however also a number of issues that need to be addressed for the future.
According to the findings, Haburas as the main actor for the program, needs more discipline and needs to be more proactive in managing the three different divisions including Matadalan Ba Rai (MBR), the Project Management Unit (PMU) and Rede Ba Rai (RBR). Haburas needs to show an example for the others in terms of how to prevent conflicts of interest. Haburas needs to be cleverer in managing activities and programs, think outside the box to help other members of the consortium that might struggle in the execution of activities, and think like a leader of the land network with nineteen organisations behind it.

The Secretariat of Rede Ba Rai needs to lead the movement to help people who are looking for justice like the communities of Aitarak-Laran, Kaisidu, Ossurua, Betano, Suai and Oecusse.
The Secretariat needs to be more organised and better at time management – it does not need big funding to write a letter to the authority of ZEEMS in Oecusse, does not need one week to organise a press conference about starving farmers in Tono, or two weeks to facilitate the community at the Suai Supply Base to go to Maun-Fahe and share their experience with the community in Betano.

Rede Ba Rai has extensive human resources in civil society who are experts in analysing the draft laws of the Land Acts. Rede Ba Rai would do well to reflect on the past when the Secretariat worked without the EC grant, with the volunteer spirit and success in advocating on land issues.
Members of the Consortium need to work together as a team not as competitors. According to the program, each partner in the Consortium has to meet four targets; however, if one of the implementing partners fails in achieving their targets, this means failure of the whole program. There is a requirement for members of the consortium to unify all their training modules.
Partners in the municipalities such as GAS, GDR, UNAER and the Victims Association have to be more proactive in advocating on issues from the municipalities. If the community wants to move fast, partners need to help them to do this because it is better to resolve problems in the municipalities themselves rather than in Dili.

Introduction
The European Commission (EC) to Timor-Leste has funded the Land and Housing Justice Action (LHJA) program. This program has been offered to Haburas Foundation as the main lead and Consortium members such as HAK, FTM, KSI, BELUN, and RBR for the implementation of programs and activities. According to the contract the duration of the project is forty two months starting from February 2013 to November 2016. This external evaluation was conducted after 34 months of the project. The Haburas Foundation as main lead for this project recruited Tomas Freitas as independent evaluator for the evaluation of the program. The evaluation was conducted over twenty working days, with a total budget of US$5000. The aim of the assessment was to evaluate the achievements of each and every partner in delivering the planned activities, as well as the way that they managed the activities. The evaluation project was part of the total grant from the EC to Haburas and each partner, which in total was approximately EUR 800.000 for the duration of the project, according to the contract with the reference number EuropeanAid ID: TL-2009-EUP-2901959605, as agreed between Haburas and EC on 12 February 2013.
Twenty days to conduct this evaluation is a limited amount of time, as the Terms of Reference had high expectations. The evaluator would like to recommend to Haburas that in the future more time and budget be allocated for this kind of evaluation.

Research methodology
@Literature review or desk review - review of relevant documents, for instance; Matrix log-frame, Activity Plan, and narrative reports. Desk review also included analysing report documents, to try to identify the logic and rationale of the program.

@Face to face interviews or direct interviews with Haburas and other partners such as RBR, HAK, FTM, KSI, BELUN and other partners of RBR such as HASATIL, LAIFET, LAO HAMUTUK and the community of Aitarak Laran in Dili; as well as partners in municipalities such as Baucau, Viqueque, Maun-Fahe, Covalima, Ermera, and Oecusse. The objective of the interviews was to identify the achievement of each and every partner in delivering planned activities, as well as the way they managed the activities.

@Focus Group Discussions with partners of HAK in Baucau and Viqueque, with partners of FTM in Viqueque and Maun-Fahe, with partners of Haburas in Covalima and Oecusse, and with partners of KSI in Ermera.

@ In-depth interviews - further interviews after focus group discussions, or after first interviews, the objective of which is to conduct the investigation in more detail. In-depth interviews were also conducted in Dili, Baucau, Viqueque, Maun-Fahe, Covalima, Ermera and Oecusse.

@Specific interviews were conducted with special target groups including in Dili with the Chief of the Aldeia of Aitarak Laran, in Maun-Fahe with LBF and the community of Betano, and in Oecusse with farmers of Suco Tono.

Strengths and weaknesses of the methodology

The strengths and weaknesses of the methodology are varied; in Dili the face to face interviews meant that most of the time the evaluator met only with the Director him/herself, or just the program manager or even sometimes with just one staff member alone. This meant that it was difficult to verify whether what he or she said was the truth. During the interviews in Dili with Haburas, MBR, RBR, PMU, HAK, FTM, KSI, BELUN, LAIFET, LAO HAMUTUK, and HASATIL, only four organisations had more than one person participate in the interview. The majority of the time the person involved in the interview was the one in charge of the organization.

The strength of the direct interviews or one on one meetings is the ability to find out exactly the capability of each person in answering the questions, and we can also investigate in more detail if there is any doubt.

The weakness of direct interviews or one on one meetings, is that many times the responder only relies on one source, or he/she is not confident to explain something that he or she is not responsible for.

The strength of focus group discussions is that everyone who is involved in the meeting can be accountable to each other; they can each add more facts and provide many reasons, and can remember exact facts related to the activities.

The weakness of focus group discussions, is that many times the Village Chief or the elders or especially males always talk first and set the opinion, which makes it hard for other participants to intervene or disagree; female always talk after all males have spoken, and many times there are only very few women in the meetings so it makes them feel uncomfortable to speak up and support their reasons.

Chronology of the evaluation
The evaluation began on 5th of November. From 5-6 November the evaluator reviewed the contract document between Haburas and EC, as well as the interim reports 12 December 2013 - 30 November 2014 from PMU to EC, and the 2014 quarterly report of the Secretariat of Rede Ba Rai. From 7-8 November, the evaluator contacted and made appointments for the interviews in Dili. from 9-14 November interviews were conducted with partners in Dili including with Haburas, MBR, PMU, RBR, HAK, KSI, FTM, BELUN, LAIFET, HASATIL, LAO HAMUTUK, and the community of Aitarak-Laran. On 16 November, interviews were conducted in Baucau with partners of HAK; on 17 November in Viqueque with partners of HAK and FTM; on 18 November the in Maun-Fahe with partners of FTM; on 19 November in Covalima with partners of MBR/Haburas, on 20 November in Ermera with partners of KSI. On 21-23 interviews were conducted in Oecusse with partners of MBR/Haburas. The preliminary findings report was prepared from 24-27 November, and from 30 Nov – 1st Dec more interviews were conducted with RBR staff. Preliminary findings were presented to PMU, EC and members of the Consortium on 2 December. From 3-7 December was spent preparing the final report.

Number of interviews during the evaluation process

Program rationale and logic
According to the contract between Haburas and the EC, the Consortium led by Haburas with its partners was required to implement several programs to achieve four types of results. The activities of Result 1 include: 1.1 Production of training and public information materials; 1.2 Training on land and housing rights; 1.3 Training on land defence; 1.4 Training on agrarian reform; 1.5 Radio programming. The targets for Result 1 include increased land and housing rights literacy in key target groups of citizens, NSAs, local authorities and political actors.
The activities for Result 2 include: 2.1 Training in financial management for NSA partners in this action; 2.2 Stakeholders analysis; 2.3 Comprehensive capacity and training needs assessment of key NSAs and other target groups; 2.4 Workshop: Mapping of existing LHR information materials; 2.5 Training of trainers on popular education techniques; 2.6 Paralegal training to NSAs on defending land and housing rights; 2.7 Technical training on dispute resolution for partner NSAs; 2.8 Training on gender in relation to housing, land and environment; 2.9 Training of media on LHR; 2.10 Comparative study tour on land rights and experiences in Indonesia; 2.11 Technical training for district based NSAs; 2.12 Technical training for community and members. The target for Result 2 include: strengthened capacity of NSA’s at all levels to engage in local and national land and housing related development processes.
The activities for Result 3 include: 3.1 Establishment of case management systems (CSM); 3.2 Regular meetings of mediation and legal aid core group; 3.3 Provision of legal aid services; 3.4 Orientation meeting for disputants; 3.5 Disputants mediation; 3.6 Coordination and support to local mediators; 3.7 Mediation training to Las. The target for Result 3 is the effective provision of legal aid and mediation services to LHV communities.
The activities for Result 4 include: 4.1 Training RBR members in network management; 4.2 Regular round table discussion on civic education; 4.3 Regular RBR members meetings; 4.4 Production of advocacy materials; 4.5 Annual advocacy activities; 4.6 Media exposure; 4.7 Website development; 4.8 RBR Yearly internal evaluation; 4.9 Annual General Meeting; 4.10 RBR Strategic Plan for 2016-2020; 4.11 Research on land ownership and inheritance of women and men in Ermera; 4.12 Monitoring of the impact of land law implementation on farmers; 4.13 Sub-grants for district NSA activities and research. The target for Result 4 is the improvement in quality of land rights advocacy and increased opportunities for dialogue between LHV communities, NSAs, SAs and LAs.

Findings
Interviews with HABURAS Foundation
According to the interview with Deometrio Amaral, former director of HABURAS Foundation, the program of Land and Housing Justice Action is based on good quality research which was published through a document called “Komunidade Nian Lian Kona Ba Rai”, Community voices of the land (Matadalan Ba Rai, 2010). Demetrio stated that the expected result from the implementation of LHJA program was improved communication with communities about the needs of the land. The former director also stated that based on research documents the President of the Republic Democratic of Timor Leste has vetoed the proposed Land Act laws. Strategic advocacy is not only at a top level, but also Haburas Foundation with other members of the Land Network (Rede Ba Rai) accompanied and provided legal assistance to communities that were affected by evictions, in places such as in Timor-Plaza, Brimob, and Aitarak Laran.

On the other hand, Virgilio Guterres the current director of HABURAS Foundation stated the land is very important and in the future it will become a big problem. He therefore recommended that the European Commission continue to support this program. In the interviews he emphasised that HABURAS is institutionally responsible for three divisions including Matadalan Ba Rai, Rede Ba Rai and the Project Management Unit, all of which work on the Land and Housing Justice Action program. He recognised it is huge for HABURAS to look after all of them, therefore he proposed that Rede Ba Rai should have its own Constitution and internal regulations, independent and separate from HABURAS. Independent means that Rede Ba Rai can look for their own funding and be registered as an independent institution. The director is also dissatisfied with the Oversight Body (Ekipa Tau Matan) which according to him is not functioning well, causing the Secretariat of Rede Ba Rai to also not perform well.

Interviews with members of RBR about HABURAS
Five members of the Consortium including MBR, FTM, HAK, BELUN, and KSI are dissatisfied with the high structure of HABURAS, which they see as being slow to resolve problems at the Secretariat of RBR, which had previously been considered to be inactive and not making any progress over the past eighteen months. After the previous coordinator of the Secretariat was replaced by a new interim coordinator, the Secretariat appeared to regain trust from its members. However, according to four of the members of Consortium, the intervention from HABURAS Foundation in the restructure of the secretariat of RBR was considered too late, because the problem of mismanagement had been unresolved for more than eighteen months. The same dissatisfaction was also expressed by sixteen member organisations of RBR. However, around half of the members, about eight organisations were not aware of problems at the Secretariat.

Analysis from evaluator
It appears that the problem as explained above has been ignored with little attention paid to what has been happening at the Secretariat of Rede Ba Rai. The implementation of activities has been delayed for more than eighteen months. This is seen to be a serious problem because the Secretariat of RBR has managed to achieve 80% of total activities and the remaining 20% of activities were divided among HAK, MBR, FTM, KSI and BELUN. In interviews some of the members of RBR stated that “the previous coordinator of the Secretariat of RBR has a family relationship with the former director of Haburas”; if this is true, there is in fact a conflict of interest. After analysing the contract and activities plan which lists approximately 31 activities, the evaluator does not see any responsible activities for Haburas as an institution. There does not appear to be a mechanism for holding the Secretariat of Rede Ba Rai accountable. There is no mechanism and monitoring system, which has to written into the activities plan, which could take place through a regular meeting (weekly, monthly, quarterly) between the director of Haburas, the Coordinator of the Secretariat of RBR, and the Coordinator of the divisions of MBR and Grant Manager of PMU.

Recommendation
The evaluator would like to recommend to Haburas to hold a regular meeting, which could be weekly, every two weeks or monthly, just between these four people, which could be called the “EC management team meeting”. The EC representative could at times be invited to attend the meeting. This is a simple mechanism that could contribute towards preventing future problems. It should also be noted that the Oversight Body (Ekipa Tau Matan) is not part of the EC project and does not take part in the monitoring mechanism. If Haburas continues to rely on the Oversight Body, the same mistake could happen again.
The yearly internal evaluation could pick up the problem, but because the Secretariat of RBR is responsible for realisation of this activity, this activity did not happen last year, because the yearly internal evaluation only took place this year.
The evaluator also recommends to Haburas not to appoint someone or anyone that has any family relationship with the institution of Haburas. To prevent miss-management of finance in the future, the evaluator would like to propose to the EC to conduct financial auditing of this program.

Interviews with the Secretariat of Rede Ba Rai (RBR)
Rede Ba Rai in English word means Land Network, which is composed of sixteen organisations and some individuals.
Based on the interviews, the staff and current coordinator recognise that some of the activities and programs were not going well before, but after the change in the coordinator position and the replacement by the interim coordinator from September – November 2015, there are signs to show that the Secretariat is improving. According to staff, the hierarchy of HABURAS as an institution is unable to resolve problems that have been occurring internally at the Secretariat of RBR.
The current structure of RBR is also why the implementation of some activities and programs has been delayed; because recruitment of the PMU coordinator happened late, the activation of PMU did not start until April 2013. Rede Ba Rai also recognised that the capacity of RBR members of RBR is insufficient for conducting activities and programs that have been scheduled, and that the time for the inception process is not enough. Inception in this case means preparation for briefing on how to execute the activities and programs based on the timeline that has been scheduled. However, in the last three months from September to November 2015, according to staff, the RBR Secretariat RBR seems to be going well, particularly after they had five days for strategic planning meetings, which resumed planning some of the programs and activities and finalising their own constitutions and internal regulations for the network. They were able to decide on rules and responsibilities for the Oversight Body Ekipa Tau Matan, for supervising the work of the Secretariat. Although, the evaluator has not seen the report from strategic planning, it seems that confidence about the Oversight Body (Ekipa Tau Matan) has increased.
The Secretariat recognises that it does not get full support from members of RBR; for example when the members had planned to do monitoring of a case of land dispute in the municipalities, on the due date for travel, the members cancelled the trip without a fundamental reason. The Secretariat also feels disappointed with some institutions which are also a part of the RBR, for example when people come to HABURAS or HAK to complain about land dispute issues, they are told “The Secretariat of RBR are not in the office, perhaps you can come back tomorrow”. The interviews confirmed that the Secretariat has just reactivated regular meetings between members, however the Secretariat doesn’t feel comfortable at the meetings when they are asked by members about how many land dispute cases have been registered, and how many cases have received assistance from the Secretariat. Despite slowing down in the execution of activities, currently the Secretariat has called for sub-grant proposals which are open for members of RBR. The idea of calling for proposals from the members is to help execute all activities that have been planned by the Secretariat. Based on the interview with the current coordinator of the Secretariat, the Secretariat has received a few proposals so far from UNAER (Agriculture Union of Ermera), ITA BA PAS Foundation, FORUM TAU MATAN, LAIFET, HASATIL and is expecting more applications from the members.

Interviews with members about the Secretariat of RBR
According to an interview with one of the national organisations which has been active since the initial establishment of the network Rede Ba Rai, this organisation is disgruntled with the performance of the RBR Secretariat, which they see as not being transparent. The organisation is also disappointed with one of their staff members who is supposed to represent the organisation at the RBR, however the staff member has not provided any update on the work of the network back to the organisation.

Moreover, another national NGO based in Dili and also a member of RBR stated that it does not get any support from the Secretariat, even while this organisation worked hard in advocating for an eviction case in the sub-village of Aitarak-Laran. However, in contrast to this, the Chief of the sub-village Aitarak-Laran stated that “the advocacy for the eviction case of Aitarak-Laran has not only involved one single organisation, however has been a combined force from the community of Aitarak-Laran itself, involving some veteran members who are living in the area, together with some NGOs likes LAIFET, HABURAS, HASATIL, and RBR, who have all joined in one advocacy committee to defend the rights of the community of Aitarak-Laran. Despite the case, there is a concern around a continued risk of eviction, because the community has already accepted compensation from the government from $1500-$3500 per household. The Village Chief said, “If one day the government asks my community to leave the place, we have to go but we don’t know where to go”. Currently the number of families who live in that area has increased to fifty.

Analysis from evaluator
The Secretariat of RBR has described the delay in the implementation of program as being not only the fault of the Secretariat, but also came about because of the late appointment of PMU staff, and delays in conducting the inception process for all Consortium members on how to adjust their program and activities.

With regard to the “yearly internal evaluation”, the only internal evaluation report the evaluator has received is four pages from internal evaluator Meabh Cryan; the content of the report doesn’t refer to any reasons of delay in implementation of activities.

The reluctance of the other NGOs to attend to clients that come to complain about their land issues, is a signal of jealousy between the members as well as a disregard to advocate on land issues. It appears to be the case that every member of the Consortium works alone and competes against each other and not as a team to gain the same objective.

It is good to know that the Secretariat of RBR has called for proposals from its members. The case of Aitarak-Laran is like time bomb which will become a problem when come to eviction. Where will these families go? Who will be responsible for the settlement? What can RBR do for the solution?

Recommendation
The “yearly internal evaluation” needs to be reactivated because it could pick up any problems or obstacles that are faced by the Consortium and the members of RBR every twelve months not every twenty four months. This activity has to be organised and responsible for by HABURAS or PMU not the Secretariat. Experience has shown us that the first years didn’t have an internal evaluation due to failure in the Secretariat; if RBR has a good person as the coordinator, the same mistakes do not have to be repeated. Regular meetings between the consortium members should take place every month, and the agenda for the regular meetings should be clear, which includes updating the progress of activities and declaring any problems that might need assistance from other members. The PMU needs to be tougher at this meeting and might need an auto-criticism mechanism to evaluate the programs.
The evaluator would like to propose to the Secretariat of RBR that before granting a proposal, the Secretariat needs to make sure that they are clear on how they are going to execute the program - do they have enough time and human resources for implementation? What is Plan B if Plan A fails? These questions come up as a reflection on the failures of the Consortium in the implementation of activities. To overcome those difficulties, the secretariat of RBR needs to map the resources of each of its members, and to identify the human resources that are available in civil society, for example there needs to be a list of each person that has skills in a certain area, such as good facilitation, evaluation and mediation skills, good analysis, report writing and mentoring skills. The list could be shared among all members.

In the case of Aitarak-laran, the Secretariat now has plenty of time to take the initiative with the members of RBR, and needs to re-plan the strategy on how to find the solution for the community in the area, where now the number of households is increasing. The RBR needs to organise a meeting with government to find the solution for those residents.

Interviews with MATADALAN BA RAI (MBR)
MBR is involved in the current structure of the division, with Antonio Delimas as coordinator and Carlos Salsinha as deputy. The MBR is one division of HABURAS Foundation, and this division has been established in 2010 long before HABURAS obtained support funding from the European Commission. This section only focused on land issues, advocating and mediating on cases of land disputes. According to the current coordinator, they published a book which was based on research, called “Komunidade Nian Lian Kona Ba Rai” which in English means “Community Voices of the Land. They produced a training manual with the title “Complexity of land and property and the actual political land in Timor-Leste”. The MBR also produced 200 pamphlets for campaigning on land issues and provided training to Non State Actors in the municipalities of Covalima and Oecusse about the rights of land and housing for its citizens based on Articles 54 and 58 of the Constitution of the Democratic Republic of Timor-Leste (RDTL). Although MBR had joining the Consortium, during the first year of the contract with EC, MBR recognised that the execution was too slow at the beginning. The structure of MBR meant that after Roberto da Cruz the former coordinator of MBR left, it affected programs and activities of the division. The MBR not only lost Roberto but also two other staff members of MBR resigned and went to work for international organisations. Despite the change in the MBR structure however, under the new coordinator Antonio Delimas they believe that the MBR will catch-up. The present team consider themselves proactive in advocacy, for instance strong involvement of the MBR in mediating land disputes in Covalima municipality such as at the Airport and Suai Supply Base. The MBR is also considered to have strong links with Land Defence Group (LDG) in Covalima, and has successfully accompanied evictees of the mega-project called Zona Economic Exclusive and Social Market (ZEEMS) in the municipality of Oecusse. The evictees lost their land and houses which were taken by ZEEMS for the enlargement of roads, roundabouts, central electricity, hotels and more mega-projects of the ZEEMS.

Interviews with Land Defence Group (LDG) in the Municipality of Covalima
Seven people participated in these interviews; one person was a representative of the Camenasa Village Chief, two of sub-village chiefs, two of the youth council, and two people were representatives of the community. From the seven interviewees which were all part of the Land Defence Group, two of the participants attended training on land issues 6-7 times; three participants were involved in training 4-5 times and two others attended training on land issues 3-4 times. One of the participants confirmed that MBR has provided training at the village level, for example in the village of Oques. Regarding the material used for the training, seven of them agreed that the material was technical and needs to be made much simpler and given more local examples so that they can understand better. They stated that half of the training manual which was around six pages just explained about the general concept of international conventions. Despite the complexity of the material, three of them explained that they did try to re-read the material just to have better understanding, however, the other four participants when asked were unable to remember the context of the material.

The Land Defence Group asked if MBR could socialise the details about the draft of the Land Act, because they still do not have an understanding of the draft law. One of the members of the Land Defence Group his under pressure from his community to defend his village Oques from a new freeway project which will take an extra 100 meters of land; he therefore believes that understanding the draft Land Act will help his community negotiate with the government.

One of the seven people interviewed is Tito Abilio, who is responsible as Acting Village Chief in Suco Camenasa. Tito is very actively involved in the advocacy process and is confident in explaining the land cases of Novo Suai, Suai Supply Base and Airport in Camenasa. The seven interview participants stated that they heard about Rede Ba Rai but they don’t know exactly what they are doing in Covalima, but only knows HABURAS through MBR work. The participants agreed that the Committee Organiser (Beni) is very proactive in accompanying the advocacy process with other Land Defence Group members, however, they would like to see more clear and systematic planning activities from him. The members of the Land Defence Group agreed that they have a very good relationship with the former director of DNTPSC of Suai Sr. Afonso, however they expressed that the new director of DNTPSC does not really cooperate because he is not from Suai-Covalima.

Analysis from evaluator
The comments from the partners of MBR in Covalima specifically about the training material indicate that they all found it very hard to understand the content of the material. However, it can be asked whether these target groups actually have an interest in this training. They have an interest in the draft laws of the Land Act, because they have all been victims of the mega-project. For example Sr. Tito Abilio from Camenasa, is very enthusiastic to know how many people are working on the Airport project, and also wants to know why no one from his village of Camenasa is working on the project. Despite the training material, the partners also questioned the Committee Organiser’s (Maun Beni’s) activity plan, as they would like to see a work plan that is clear and systematic.

Recommendation
The training that would appear to be the most beneficial at the moment is with regard to effective and efficient advocacy, not just an understanding of the right to equal land and housing.
With regard to the proposition from partners about the CO, the evaluator recommends to Haburas, MBR and the Secretariat of RBR that they should design an activity plan for the CO of COVALIMA, and the plan must correlate to the program, at least for a twelve month period.
Interviews with HAK Association
HAK Association is one of the veteran NGOs which was established back in Indonesian times. As an expert on human rights issues, HABURAS invited them to join the Consortium and together designed the proposal for Land and Housing Justice Action. According to the interview with the Director of HAK Manuel Monteiro, thus far HAK is confident about the programs and activities that have been scheduled to be implemented. Moreover, Manuel emphasised that HAK has produced two training materials including ‘Saida Mak Direitus Humanos’ and ‘Saida Mak Advokasia’, and in addition to that he also mentioned that they have feedback on commentary form from participants after attended training. Based on the report form for March – July 2015, training has been implemented six times about the right for equal land and housing, and six times about monitoring and advocacy of the right to equal land and housing. He also recognised they have a good relationship between HAK and FTM and sometimes with KSI as well. Furthermore, he is very proud of HAK that has 25 members in its Strategic Target Group which is called Grupo Alvu Stratejic (GAS), which is considered very strong in advocating on human rights issues.

Interviews with Grupo Alvu Stratejic (GAS) in Baucau
The interviews in the municipality of Baucau involved five people which included three community representatives, one lawyer from a local NGO in Baucau called Fundasaun Edukasaun Comunidade Matebian (FECM), and one director of DNTPSC Baucau, Sr. Sancho Guterres. The GAS partner in Baucau stated that they have attended training around 4-6 times - they felt that the material on human rights was too simple, and they recommended that extra detail be added as well as more examples from a culture perspective. In contrast to the human rights material, the partners considered the advocacy material to be complex and used too many high technical terms, and the content of the material does not provide any local examples. However, they did enjoy facilitation by a particular trainer who is according to them is always energetic and entertaining which made the participants not get bored. At the interview the evaluator found that that Village Chief of Tirlolo and the lawyer from FECM had enough capacity to interpret the training materials, and can explain very well Articles 54 and 58 of the RDTL Constitution which grants land and property rights to every citizen.

In addition to quality interpretation by its members, the GAS partner also recognised Rozito as being very proactive in facilitating mediation and advocacy land disputes in the municipalities of Baucau and Viqueque. As one of the long term staff members with HAK, according to the GAS Rozito is at the moment focused on advocacy of community land in Kaisidu village, where the land will be taken by government for cement mining. Furthermore, Rozito together with another GAS member is actively involved in mediation of land disputes in the villages of Tirlolo, Buruma and Wailili. According to the Village Chief Tirlolo, between 2013 – 2015 between 40 and 50 land dispute cases have occurred, however not all have been documented.

The GAS partner recommended that future training should be implemented in the villages and sub-villages. They also stated that they will submit a proposal to Rede Ba Rai Secretariat which will includes programs and activities for twelve months. If the proposal is agreed to by RBR, they will conduct an evaluation every three months. In their final recommendation they proposed to hold in future training about the civil code (kodiku prosesu civil) and socialising the draft laws of the land.

Interviews with DNTPSC of Baucau Municipality
In the interview that involved the director of DNTPSC, the director Sancho Guterres recognised that has attended one training organised by HAK, however because this is new position for him (four months), he actually does not know much about the relationship between HAK and DNTPSC in the past.

Analysis of evaluator
Based on the interviews in Baucau, the evaluator can understand why some partners of HAK understand the training material and some do not. This is because some partners have an interest in knowing, because they have responsibilities – for example the Village Chief of Tirlolo needs to know because his community could ask him about the training; the same could apply to the young lawyer - even though she is new in that NGO because she is lawyer she has to use her intelligence to absorb the material and share it with other staff in her office.

With regard to HAK staff in Baucau and Viqueque, or Beni and Ajanu as CO for Haburas in Covalima and Oecusse, this is not something unusual, because usually focal points, committee organisers, district officers, or liaison officers are close to their communities and the community tends to say good things about them. In a short time it is not easy to identify their weaknesses. However, it is the case that Rozito complained that when it comes to the rainy session, it is difficult for him to mobilise activities in two municipalities with only a motorbike, at the same time.

Recommendation
This recommendation is not only for HAK but for all members of the consortium and members of RBR, that need to map their partners capability. For example, HAK needs to have a database of their GAS members, listed individually, and of how many times that individual has attended any kind of training from HAK or other organisations. Such a database system will help any organisation to see how strong their partners are.
The evaluator would also like to propose to HAK to encourage their partners to write down any activities or any cases of land disputes in their area.
The evaluator would also like to recommend to HAK to employee one more staff member, or Rozito can still perform the tasks but HAK has to provide him with a four-wheel drive.

Interviews with partners of HAK in Municipality of Viqueque
In the interview with HAK partners in Suco Ossurua, Postu Ossu and the municipality of Viqueque, the interviews involved one Village Chief, one Sub-Village Chief and one ordinary person who was also part of the 25 members of GAS. In the interviews the Village Chief of Ossurua could explain very well about what is human rights, but for the advocacy material all three participants felt that it was very technical and hard to understand. Therefore they recommended to hold training at the village and sub-village level, and stated that the community of Ossurua needs to know about the draft of land laws, because there are people in the village from Indonesian times who are concerned that they may one day be forced to leave, as there are others claiming that they are the landlord, which has also pushed these people to renovate their houses as well. The fundamental question from the community as stated by the Village Chief is will the community who has lived in this village for more than forty years have a right to a claim on the land?

Analysis of evaluator
The case above shows that the partners are not really invested in the trainings; they only want HAK to socialise the draft laws of the Land Act. And because the community has lived in that place for nearly forty years (even though culturally it does not belong to them), including the Village Chief, they do need socialisation of the draft Land Act as soon as possible, to clarify their rights to land.

Recommendation
For HAK to immediately begin socialisation of the draft Land Acts.

Interviews with partners of HAK and FTM in the Municipality of Viqueque
The interview involved five people which was composed of one Sub-Village Chief, two youth representatives, one local NGO, one Coordinator of the Victims Association and the Director DNTPSC of Viqueque Sr. Mateus Ramos Pereira. Both HAK and FTM claim that these five participants are their members. According to HAK they are part of GAS, and the same was also expressed by FTM. Out of the five interviewees, two of them are local NGOs. Representatives of the victims association stated that they have attended training more than 5-7 times, and the Sub-Village Chief plus two others acknowledge that they have attended more than 3-4 times. All of them remember having attended three days training in Viqueque which was organised by HAK and FTM in June 2014. Interestingly from these interviews, one of the participants still had his notebook with him, and when the evaluator asked if they remembered what kind of material they have received from the training back then, this participant was able to easily answer the question because he had everything written down in his notebook. He could explain very well the difference between material presented by HAK and FTM, and stated that he felt that the material from the two organisations did not correlate with each other, therefore sometimes confusing the participants of training, and he stated that the training manual from HAK ‘Saida Mak Direitus Humanos’ was very simple, but that the other material from FTM which is a manual on training of trainers (ToT) he felt was very technical and to become a good facilitator it is necessary to have more practice to use the material. During the interviews the partners all agreed that generally speaking in trainings participants found it a struggle to stay focused. Two of those partners stated that they had attended a comparative study that was organised by HAK back in 2014. They felt that the program was good and that more of these were needed in future. Besides that, four of those partners told the evaluator they attended the socialisation program conducted by FTM in the villages of Uaniuma, Maluru, Vessoru, Uma Kiik, Uma Wain Kraik and Babulu. Moreover, one of the partners recommended to have more training of trainers (TOT). Furthermore, five of them proposed to organise a Strategic Planning (SP) together with HAK and FTM and develop a proposal, the aim of which would be to access the EC grant through the Secretariat of RBR.

Analysis of evaluator
The interviews showed that the training material of FTM and HAK which includes training on equal land and housing rights, how to become a good facilitator, human rights and how to do advocacy, confused the participants if presented at once in one training activity. The polarity of knowledge also became an issue for participants to be able to absorb the material.

Recommendation
Refer to the same recommendation as proposed to partners of HAK in Baucau, which is to organise a human resources database for the partners.
Regarding correlation of material, the evaluator would like to refer to a suggestion from BELUN that all members of the Consortium need to sit down together and re-evaluate their training materials.
The evaluator would like to endorse the idea from partners in Viqueque to develop a proposal together between HAK, FTM and partners to submit to the Secretariat of RBR.

Interviews with BELUN
An interview was conducted with Izalde Correia Pinto, as Program Coordinator for Land and Housing Justice Action from BELUN. According to Izalde, the activities that were scheduled for implementation this year (2015), is one training in each of the municipalities of Baucau, Manatuto and Bobonaro, and the training was about ‘mediation of land disputes’. The training targeted village chiefs and village councils; for example in Baucau, the training was participated in by 19 participants from the villages of Tirlolo, Bahu, and Buruma and it was held from 26-28 of October 2015. In Manatuto the training was participated in by 17 participants from the villages of Sahul, Kaiteas, Ai-lili, Ma’abat, and Kribas, and it was held in March 2015. In Bobonaro, the training was participated in by 22 participants from the villages of Holsa, Rahinea, Ritabou, Raihu and Lahimeu. Training material was called “Modul Mediasaun” and was about 18 pages long.

Interviews with partners of BELUN
An interview was conducted via telephone call with the Village Chief of Buruma, Sr. Antonio Belo. “The training was good and the training material was perfect, because it explained the options of how to resolve any problem, and those options included negotiation, mediation, arbitration, and tribunal”. Sr. Antonio acknowledged that this is not the first time for him to attend training about land disputes and mediation; he acknowledged having received this kind of training in the past from other institutions such as GIZ, PDHJ, and DNTPSC. As Village Chief, the community always came to him asking to resolve land disputes. Since he became the Village Chief of Buruma in 2012, he has facilitated a mediation process for more than fifty case of land disputes; around thirty cases were resolved through a mediation process, while the rest went to court. When the evaluator ask if he had ever written any of the cases down, he explained that some of the cases are huge and very complicated and therefore he had only documented some of them.

Analysis of evaluator
Based on the interview the evaluator can conclude that Sr. Antonio Belo has the capacity to facilitate mediation because he is the Village Chief. Culturally the community only believes in and trusts the Village Chief or the elders (katuas lia-nain) to facilitate mediation. Sr. Antonio understands very well the training material, but wishes to focus on one technique and that is mediation. This is interesting because there are other options including negotiation, arbitration and tribunal, however he is interested only in the mediation process because according to him that is the most successful process that he has used to solve any problems in his villages.

Recommendation
The evaluator believes that this is a good start for BELUN because it targets the local authority as their beneficiaries. The evaluator would like to recommend to BELUN to facilitate training based on specific targets such as facilitated mediation and resolution of cases. The evaluator also would like to propose to BELUN to organise a database which would only focus on registration of land disputants. BELUN also needs to expand the location of training to other municipalities.

Interviews with DNTPSC of the Municipality of Viqueque
In terms of its relationship with state actors, in this case its relationship with Dirasaun National Terras Propriadade Sistema Cadastral (DNTPSC), five of the interviewees questioned why the DNTPSC has started the measuring system (Sistema Cadastral) of people’s lands and housing while the Land Act has not become law yet. Moreover, if the DNTPSC keeps doing this, the community will think that certain people have the right to certain land and housing even though in some cases the land and housing is still subject to a dispute. To answer the above questions, the director of DNTPSC Sr. Mateus Ramos, says “the measuring system is based on a government plan which is a national program, and this has already begun in other municipalities, so accepted or not it has to keep going”. According to the director, they have a good relationship with civil society in Viqueque; he also mentioned that he had attended and was a speaker at a workshop organised by civil society, and he also presented a topic about UNTAET Regulation 03/2001. At the workshop he also listened to some other speakers from civil society who explained about how to become a good facilitator for the resolution of land dispute, which he thought was good material for a training. In the interview, he stated that DNTPSC will continue to collaborate with civil society however, DNTPSC will not socialise the draft Land Act because it is still considered to be a draft proposal. Before end of the interview, Sr. Mateus Ramos complained that DNTPSC Viqueque doesn’t have any transportation for facilitating their works; he asked that civil society refer this to the national level.

Interviews with Forum Tau Matan (FTM)
According to the Program Manager of Forum Tau Matan Zelia Fernandes, FTM has produced a training manual that includes two modules that are training of trainers and the right to equal land and housing. She also stated that FTM always documents pre-test and post-test before and after training. The pre-test and post-test is a questionnaire form for the training participants to answer questions about the training material. Forum Tau Matan recognises that it has a good relationship with HAK. FTM also agreed that the RBR Secretariat RBR has not been working well for the past eighteen months, however in the last three months many activities have been executed since the RBR Secretariat RBR now has a new interim coordinator. Based on her perspective about the Consortium, she stated that the regular meeting which was supposed to have been organised by PMU every three months sometimes doesn’t really work, because the Consortium members seem not really open to talk about their difficulties in the execution of their own programs.

Interviews with FTM Partners in the Municipality of Maun-Fahe
Interviews in the municipality of Maun-Fahe involved eight people, five of whom are from Centro Foinsae Boa Ventura (CEFOBOM), one person from the local NGO Luta Ba Futuru (LBF), one person from the community of Betano and the last interview with the director of DNTPSC of Maun-Fahe, Sr. Mariano Cortereal Tilman. In the first interview with five participants from the local organisation CEFOBOM, of the five, two of them confirmed that they had attended training 3-4 times, and three others stated that they had only attended training 1-2 times. Of the five only one was able to explain very well about the right to equal land and housing; the remaining four considered the language of the material to be very technical. Five of them recommended that more simple words should be used in the training material, and suggested that the facilitator use more games to keep the attention of participants, especially after lunch time. Based on the interviews, the five participants do not really know of any cases of land disputes in the municipality of Maun-Fahe, including Betano.

Interviews with DNTPSC in the Municipality of Maun-Fahe
During the interview, the director of DNTPSC, Sr. Mariano Cortereal Tilman stated that he just started in this position about four months ago, and so far he had not seen any local NGOs based in Maun-Fahe accompany the community of Betano. He also stated that the community of Betano really needs the presence of civil society to advocate for their problems, which are due to the government taking their land for the oil refinery mega project in Betano. The director stated that the mega-project will affect the community in Betano, because the Village Chief has declared 236 hectares of the land as belonging to the state. From 236 hectares according to Sr. Mariano, 230 hectares will be used for a refinery plant and 6 hectares will be used for new settlements in Novo Betano, which in this case the community of Betano are actually still confused as to who will settle in Novo Betano. According to data owned by DNTPSC, the mega-project of Betano will affect 20 households, and so far 236 hectares of land will get reimburse at $3 per meter square from the government. In a short time DNTPSC will publish information regarding the land that will be used for the mega-project refinery in Betano, and will then see if there are any complaints from the community of Betano.

Interview with local organisation LBF
Luta Ba Futuru (LBF) is a local NGO based in the municipality of Maun-Fahe. This organisation was established in 2009, focusing on the provision of clean water and sanitation to the community of Maun-Fahe. According to the interview with Armando da Silva, Coordinator of LBF, he has participated in training organised by FTM about 5-6 times. Armando could explain very well the concept of the right to equal land and housing. He can describe how to become a good facilitator, and he also still remembers what he learned from the training, for example he stated “first to become a good facilitator you have to become a good leader so you can lead the process; secondly, to become a good facilitator you have to act like a referee which means that you have to control time very well, thirdly, you have to be neutral and impartial”. When the evaluator asked if he knows if any socialisation has been implemented by FTM in Maun-Fahe, Armando stated that he personally accompanied the activities in Sucos Taitudac, Holarua and Betano in 2014, and for 2015 some of his staff accompanied activities in Sucos Fatukahe, Dotik, and Uma Berloic. He also stated that LBF will be considering about whether to monitor land issues in Betano.

Interviews with the community of Betano
During the interview with Sr. Virgilio Salsinha, who works at the Centro Saude of Same, he stated that as a member of the community of Betano he has also been affected by the refinery plant mega-project. Sr. Virgilio stated that the government has already measured his land, of which about 900 square meters will be affected by the project. However, he has been unhappy to learn that the government will only pay $3 per square meter. He also stressed that he has been very disappointed about the declaration by the village chiefs of Betano, and he rejected the idea that the land belongs to the government; he stated that even though they don’t have land title, every single community who lives in Betano is recognised that their ancestors used to live in the area. Sr. Virgilio also stated that they have never received any assistance from civil society in the municipality of Maun-Fahe to accompany their case. They questioned with regard to Novo Betano, who will settle in that place? Will it be the community of Betano who has been affected by the project, or will the new Betano village be only for company workers who will work for the project. He also recommended to NGO Luta Hamutuk to monitor and accompany their problems.

Analysis of Evaluator
Based on the interviews in Maun-Fahe, the evaluator would like to stress that this program of Land and Housing Justice Action is to guarantee that all the community have access to equal land and housing; what is the point of all the hard work in providing training to the partners, if in the end the community of Betano will become victims of injustice. What is the point of all the socialising activities that have been done in the villages when in the end twenty families in Betano don’t have anywhere to go.

Recommendation
FTM and RBR need to re-plan their strategy of how to accompany the community in Betano. Local organisation like LBF can be considered as partners to conduct monitoring in the field. The evaluator would like to recommend to RBR to facilitate partners like Land Defence Group from Covalima to go to Betano and discuss with the community that is going to be affected by the refinery mega-project. It will be a valuable lesson for the people of Betano if partners from Covalima can share their experience about Suai-Supply-Base, Airport, and Novo Suai.

Interview with Kdadalak Sulimutu Institute (KSI)
Based on the interviews with Program Manager of KSI, Jenito Santana, KSI has produced a training manual about the right to defend the land. In the manual there are seven sub-topics which includes (1) the right of citizens to claim the land based on article 54 and 58 of the RDTL Constitution (2) limitations on ownership of the land (3) the right to indigenous land (4) the right to customary land (5) how to obtain title (6) legal structure of Timor-Leste laws and regulations (7) What is land reform?
The KSI recognises that it has just started research on land issues in Ermera, conducted last month in October 2015, and will now be moving to the next step which is data entry data and analysis. Jenito also referred to the radio program which was conducted as a one-off live transmission at community radio in Ermera, which involved several speakers including the Director of DNTPSC of Ermera, coordinator of UNAER, and one community representative from Maudio; the topic of discussion was land disputes and land reform in Ermera, and the moderator was a KSI staff member. When the evaluator asked about production of the film, the director stated that it is now in the process of editing and will be finalised soon. However, according to him, KSI didn’t anticipate a budget for publication of the film which was around $20 per minute. When the evaluator questioned why the execution of activities had been delayed, Jenito responded that he recognised his organisation had been delayed in implementation of the program because one of his staff who had been responsible for implementation had resigned (and a replacement had now been recruited). KSI recommended to PMU that based on the contract that had been signed off between KSI and PMU, that there needed to be submitted a quarterly report. KSI also questioned why PMU had released a payment of 10% after the activities had been implemented.

Interviews with UNAER in Ermera
These interviews involved 7 participants all from a group called Uniaun Agricultura Ermera (UNAER). The participants were composed of five people from Commite Baze (Based Committee) and another two were president and vice-president of the Union. According to the interviewees, two of the members had attended training 7-8 times, another two had attended 5-6 times, and three others had also attended 3-4 times. When the evaluator asked questions to the three participants that had attended training 3-4 times, the evaluator was surprised that they could explain very well about the concept of land reform and also the right of each citizen to land and housing, based on Articles 54 and 58 of the RDTL Constitution. Unfortunately, when the evaluator asked about the other five sub-topics that were included in the training manual, it appeared that all of them felt uncomfortable to answer the questions. Three of the participants admitted that they had accompanied training on land reform in sucos; Mau-Ubu, Hatolia Vila, Mausae, and Fatukero in September 2015. According to the President of the union, Sr. Amaro, UNAER have fully supported all activities and programs conducted by KSI, and as an example he told the evaluator that recently his member had been actively involved actively in a research program which was the responsibility of KSI. The research was conducted from 23-28 October 2015, and samples were taken from four sub-districts including Railaco, Ermera, Hatolia and Letefoho, and from four sub-districts they took three villages, and in each village they took only two sub-villages and then interviewed 25 people from each aldeia or sub-village. The evaluator also asked questions about how they conducted the interviews, and they answered “it’s easy because all of the questions already have been written in the questionnaire form, so we just read the questions”, they stated that besides direct interviews they also conducted focus group discussions. The evaluator kept asking what kind of questions they have to ask to community, and they said the questions were around, how big is the size of your land? Where did you get your land from? When did you get your land, in which regime? Do you have any one to testify? The land that you have, is it secure?

The interviews then moved on to different issues, with the president and vice president asking about the follow-up of the establishment of re-forestation centre in Suco Talo and Manusae, questioning KSI about the reforestation implementation plan. They said that the “trees covering the coffee plantation are getting old”, therefore it needs to be reforested with new plants. UNAER asked about how to re-establish the cooperative that has not functioned for some time. Sr. Amaro also stated that UNAER has not yet been registered as a legal entity, and because of that the local authorities in Ermera have always questioned UNAER’s legal status to defend land issues in Ermera. The Union also asked if the secretariat of RBR can support the implement of some of their activities. According to the president, UNAER is very disappointed with the KSI liaison officer who spends too much time in Dili rather than Ermera. UNAER also asked the Secretariat of RBR to please consult with them first regarding the implementation of activities, before going directly into the field.

Analysis of evaluator
KSI appears to have an attitude that they can resolve all problems; not being aware of their limitations will cause difficulties in the future. The evaluator is concerned in particular about their capacity to execute one of their listed activities that is “Research on land ownership and inheritance of women and men in Ermera” - the evaluator believes that KSI needs assistance from other partners to do data entry and analyse the data and evidence.

Recommendation
The evaluator would like to recommend to Kdadalak Sulimutu Institute to discuss with PMU to find a solution to completing their film. The organisation also needs to talk to Lao Hamutuk about recording of the radio program, as well talk to members of RBR to find out about any resources that can help them to complete the above-mentioned research. The organisation needs to discuss with UNAER any planned activities that have not yet been implemented.

Interview with Haburas partners in the Region of Oecusse
Interviews in the region of Oecusse were a little bit different to interviews in the other municipalities. In Oecusse, the evaluator met four members of local NGO Machine Neo Oecusse (MANEO) and thirteen victims that had been affected by the mega-project of Zona Economic Exclusive Social Market (ZEEMS). In this region the evaluator did not have a chance to discuss training or capacity building that they had received from Haburas. When the evaluator arrive on Saturday morning at the office of MANEO NGO, the meeting began immediately involving eleven people. The participants were composed of a spokesperson for the victims, five members of the victims, one regional coordinator of Haburas, and four staff from MANEO organisation. Sr. Agustino da Costa Magno as spokesperson for the victims stated that the mega project of roads, power station and hotels has affected the communities of Ambeno
.
Firstly, the speakers stated that the project for the widening of the roads has affected the communities of Sakato, Nepani, Lifau, Lalisu, Cunha and Tono. In Suco Sakato, eleven families had to move out of their homes and give away four hectares of their land to ZEEMS for building of the new central electrical power plant. According to one of the victims Patricio Quebo, ZEEMS had promised them to pay them $3 dollar per meter square that they had lost but this had not happened to date.

At the same village, Sakato, Sr. Francisco Lopes Colo, used to have a house of 7x8 meter squared; ZEEMS asked him to pull down his house, which he did, however after the road had been widened, his original house was still three meters away from the edge of road. Sr. Francisco stated that ZEEMS had given him some materials that were not enough to rebuild a house of 7x8 meter squared. The same thing had also happened to Sr. Jose Koa, a journalist for the local newspaper STL, who had to stop working as a journalist because he had to rebuild his own house.

The interviews with Sr. Francisco Xavier, in the village of Umbei, Aldeia Sanane revealed that Sr. Francisco was dissatisfied with the ZEEMS project of road enlargement which had taken his house of 9x13 meters squared and 1100 meters square of land. A widow named Sra. Carolina Fui also experienced a similar situation, where ZEEMS had bulldozed half of house leaving the other half very unstable and exposed to wind and rain. Even after removing the side of the house, the existing house was still three meters away from the edge of the expanded road. Though they were compensated like the others, the amount of compensation was minimal compared to what they needed.

In the interview in Suco Tono, Sr. Joao Batista and Sra. Elisabeth Bobo expressed their dissatisfaction with the decision from ZEEMS which told them not to work on their rice field because if they spread the seeds, it would disturb the road project. However, over a year later the project has not yet started, and now 270 families who live in Suco Tono and who owns 42.1 hectares of rice fields cannot farm their rice.

Analysis of evaluator
Based on the interviews with the speakers and the victims, it appears that they have trust and believe in Haburas to advocate on their issues. They were very enthusiastic to meet with the evaluator and open up about the issues. They hope Haburas and RBR will follow-up their case to the national level.

Recommendation
Haburas needs to talk to other members of RBR about how to follow up the cases of those who have been affected by the ZEEMS project. The evaluator would like to recommend immediately find the solution for the 270 families that have been told that they cannot farm their land in Tono.

Conclusion and recommendations
The hard work that the members of the Consortium have each put in to achieving their goals must be congratulated. Some of the achievements are:

HAK for their achievements in empowering their local partners. This empowerment could be seen when the Village Chief of Tirlolo and the female lawyer in Baucau explained very well about the right to equal land and housing according to Articles 54 and 58 of the RDTL Constitution.
HAK partners in Viqueque who have a strong commitment to continue working on land issues even though they are only volunteers.
The work of FTM in Viqueque and Maun-Fahe (Same) which socialised the right to equal land and housing through the village level, even though they are at times facing difficult questions from the community.
Matadalan Ba Rai, for all the hard work and effort in improving the capacity of their partners Grupo Defende Rai (GDR), for example in Covalima for defending the victims in Camenasa and Oques that have been affected by mega-projects such as the Suai Supply Base, Airport and Auto Estrada, and in Oecusse for defending the rights of people in Sakato, Nepani, Lifau, Lalisu, Cunha and Tono, that have been affected by the mega-project of ZEEMS.
KSI for their hard work that they have done with their partners UNAER in Ermera, particularly seen through the knowledge of interview participants on land reform.
BELUN for their commitment to providing capacity development at the village level specifically training on mediation for land disputes.

Although there is still much work to be done, each member has demonstrated their commitment as could be seen in the interviews with their partners in the municipalities. A few points from the evaluation report can be re-emphasised here:

Haburas as lead organisation on the project needs to show a good example in preventing conflict of interest inside the organisation;
Haburas needs to lead the land network movement to find solutions for communities in Aitarak-Laran, Baucau, Betano, Covalima, Ermera and Oecusse.
Haburas needs to think of themselves as a leader of sixteen organisations under the Land Network movement, not just a leader of the Consortium.
The five organisations under the Consortium need to open to each other, as collaboration and solidarity are important – there is no competition.
Consortium partners need to be committed to attending to suggestions from their own partners in the municipalities, such as GAS, GDR and UNAER.
The partners of HAK, FTM, MBR, BELUN, and KSI in the municipalities need to take more responsibility for solving community problems instead of relying so heavily on their partners in Dili. Cases such as mining cement in Kaisidu, the refinery project in Betano, the mega-project in Suai, and ZEEMS in Oecusse need more proactive collaboration from partners in the municipalities; the Consortium members as part of the RBR network need to re-plan their advocacy strategies for these mega-projects.
The consortium partners need to unify their training modules, in particular with regard to Activity 1.2, Training on Rights to Equal Land and Housing.

0 Comments

Timor-Leste Civil Society capacity assessment

4/30/2016

0 Comments

Final Report of Baseline Organisational Capacity Assessment (BOCA)
EU Funded Project February 2012 – December 2013
Prepared by
Tomas Freitas
Independent Evaluator
Dili, 10th of March 2014

Acronyms
AJEC                Associasaun Estudante Joventude Estacao Cajularan
CDC                 Centro Dezenvolvementu Comunitaria - Baucau
CDHD              Centru Feto Haburas Dezenvolvementu
CDP                 Centro Dezenvolvementu Popular – INURITIL
CDTL                Community Development Timor-Leste – Covalima
CFHD               Centro Feto Haburas Dezenvolvementu - Baucau
CFS                  Centro Formasaun de Servisu
CIACS               Centro Informasaun de Agricultura Cantina do Suco - Viqueque
CTI                   Community Transformation Institute - Baucau
CYC                  Children and Youth Centre – Ponta Leste, Baucau
DFW                Dili Film Work - Dili
DISIS                Dili Institute of Strategic and International Studies - Dili
EP                    Estrela Plus – Dili
ETADEP           Ema Mata Dalan Ba Progresio
FB                    Fundasaun BELUN – Baucau
FBHTL              Fundasaun Beneleba Haburas Timor-Leste - Liquica
FCBB                Foinsae Cdalak Buka Banati – Viqueque
FC                    Fundasaun CAILALO - Baucau
FECM               Fundasaun Educasaun Comunidade Matebian – Baucau
F-GRACA          Fundasaun Graca - Covalima
F-HABEER        Fundasaun HABEER - Bobonaro
F-HACSOLOK   Fundasaun Hacsolok - Bobonaro
F-HADER          Fundasaun Hadian Dezenvolvimentu EKonomia Rural - Liquica
F-HAMAHON   Fundasaun HAMAHON - Dili
FHM                Fundasaun Hadomi Malun – Covalima
FHTL                Fundasaun Haksolok Timor-Leste
FMF                 Fundasaun Moris Foun – Liquica
FMH                Fundasaun Moris Hamutuk – Liquica
F-MALAEDOI   Fundasaun MALAEDOI – Liquica
F-MANKLEDU Fundasaun MANKLEDU - Liquica
F-NETIL            Fundasaun NETIL - Liquica
FINA                 Fitun Naraoman - Covalima
Loro Matan     Loron-Matan - Covalima
FOLSETIL         Fo Liman Serbi Timor-Leste - Liquica
FKSH                Feto Kbiit Servisu Hamutuk – Dili
F-KAEBAUK      Fundasaun KAEBAUK - Covalima
FRN                  Fundasaun ROMAN NUTETU – Covalima
FSS                   Fundasaun Sadan Saneti - Covalima
F-TOHA            Fundasaun Timor Oan Hamutuk - Viqueque
GHC                 Grupo Haburas Covalima
GFFTL              Grupo Feto Foin Sae Timor Lorosae – Dili
HAFOTI            Hamahon Feti Timor - Dili
HADEZTA         Haburas Dezenvolve Talento - Dili
HDTL               Haburas Dezenvolvementu Timor-Leste - Liquica
HDMTL            Halibur Deficiente Matan Timor Leste – Dili
HLT                  Hametin Lia Tatoli - Covalima
HIAM               Hamutuk Ita Ajuda Malu – Health – Dili
IBP                   Ita Ba Paz - Dili
IHL                   Institutu Luta Hamutuk – Dili
IT                     Info Timor - Baucau
JEF                   Juventude Esperanca ba Futuru – Covalima
JRB                  Juventude Rai Binan - Covalima
KDP                  Kolegas Das Pas – Viqueque
KDT                  Klibur Domin Tibar - Liquica
LABEH             Lalenok Ba Ema Hotu – Dili
LODA               Luta Organiza Dezenvolve Area - Liquica
KHC                 Knua Haberan Komunidade – Viqueque
MIPG - ET        Multi Interest Peace Group of Tasi Timor - Bobonaro
MOKO             Moris ho Comunidade - Viqueque
OHM               Organizasaun Haburas Moris – Bobonaro
PJF                   Prepara Juventude ba Futuru – Covalima
RHTO               Rae’es Hadomi Timor Oan – Dili
RCT                  Radio Comunidade Tokodede - Liquica
RPV                  Radio Povo Viqueuque
RN                   ROMAN NUTETU – Covalima
Sadan Saneti   Sadan Saneti - Covalima
TAHA               Tane Hamutuk - Liquica
TTAA                Technical Trainees Alumni Association Timor-Leste - Liquica
TECOMTIL       Technology Computer Timor-leste - Viqueque
UFD                 Unidade Feto ba Dezenvolvementu - Baucau

Acknowledgments
The author would like to thank the logistical and communications support provided by FONGTIL during this assessment. Special thanks to Tito Lopes, Cornelio do Rosario, Isaura Dias, Ricardo Ximenes, Natercia Mendes and Olivio Sequeira for their assistance during the interviews in Dili and also in the districts. The author would also like to thank the FONGTIL Director as well as the management team and all the staff who have contributed their time and knowledge during the evaluation process. The views expressed in this document in no way reflect the views of FONGTIL or the delegation of the European Commission in Timor-Leste. For further clarifications or additional information the author can be contacted on thomas_freitas@yahoo.com.

Executive Summary
The European Commission to Timor-Leste has funded the Baseline Organisational Capacity Assessment (BOCA), an assessment to measure the capacity development of its members, staff and Secretariat, as well as to ascertain what the agencies, donors and government think about the quality of the 267 FONGTIL members around the country. This last point the evaluator was unable to measure, as the baseline report has not yet been published. Regarding this, the evaluator has attempted to identify what has caused this delay and what factors have affected this activity. The evaluator has identified internal and external factors that have contributed to the failure of this activity.

This assessment has involved 67 member organisations, interviews with 96 people, including 39 directors, 16 program managers, 24 key staff, six DLO’s, and four board members, including a former board member, a director and one former director of FONGTIL and five MOCA members. This evaluation was conducted in six districts including Dili, and also involved a desk review, face to face interviews and focus group discussions as the methodology for evaluation.
Face to face interviews were used as the evaluation methodology in Dili, while focus group discussions were used in the districts; Viqueque, Baucau, Liquica, Bobonaro and Covalima.
This evaluation has come to the conclusion that there has been a failure of the MOCA project. Internal factors that have contributed to this failure are included in the 15 Activities Plan for 2013; however according to the evaluation and documents review, this included the fact that two activities could not be implemented due to limited time, and six main activities were requested for removal, meaning that then another two activities could not be executed. Due to a delay in re-entering in the database for MOCA assessment, this has caused a delay in analysing the baseline assessment and therefore the final report was also delayed for submission. The final result of the baseline assessment therefore could not be published, and because of that part of the BOCA assessment (point two) could not be done.
External factors that have contributed to the failure of the MOCA project is the complexity of problems that its members face in their own organisation, which have accumulated for a long time and which have impacted on the organisation through, in some cases, an unwillingness to work to implement programs and activities. In this evaluation of 67 member organisations in six districts, 20% of them are still looking for donors, 8% do not care what FONGTIL is doing, 23% gave negative comments about FONGTIL, and 33% are still waiting for their problems to be fixed by FONGTIL. These attitudes have been accumulating for some time and have contributed to a souring of the relationship between the FONGTIL Secretariat and its member organisations.

Introduction
The Baseline Organisational Capacity Assessment (BOCA) was developed to evaluate another project called the Membership Organisational Capacity Assessment (MOCA). In accordance with the contract between FONGTIL and the European Commission, the MOCA project began in February 2012 and finished in December 2013. At the beginning of February 2014, FONGTIL recruited Tomas Freitas as the independent consultant to conduct the external evaluation. The evaluation was conducted over twenty (20) working days, with a total budget of US $5000. Both assessment tools are funded by the European Union based in Timor-Leste, with the aim of increasing capacity development among non-government organisations (NGOs) in Timor-Leste. This evaluation was part of the activities plan of the MOCA project, with a total budget of EUR192.966,45, divided between the European Union grant totalling approximately EUR173.644,45 and from FONGTIL approximately EUR19.322,00 according to the contract (DCI-NSAPVD/2011/277-927).

Background of FONGTIL
As the umbrella organisation of local, national and international NGOs, civil society organisations and development partners in Timor-Leste, FONGTIL (Forum ONG Timor-Leste), according to the Annual Report of 2013, currently has 367 member organisations operational across the entire country. FONGTIL’s mandate is to do coordination, advocacy, and dissemination of information, and facilitate capacity development. These activities are integrated into FONGTIL’s strategic planning. This includes approximately 323 local and national NGOs, 57 international NGOs, as well as seven candidates for local NGOs and five for international NGOs, all of which are under FONGTIL’s responsibility. FONGTIL has seven board members composed of five males and two females, six of whom are representatives of local and national organisations and one from an international NGO. As of 31 December 2013, FONGTIL had 33 permanent staff.
Non-governmental organisations have existed in Timor-Leste since the 1980s. During that time the work of NGOs was focused on emergency programs; in the 1990s some NGOs began to expand their programs, including community development. In the late 1990s, advocacy and human rights became central issues and the main program for NGOs in that period, and in 1998, fourteen organisations together took the initiative to establish Forum ONG Timor-Leste (FONGTIL).

Chronology of the MOCA assessment
One of FONGTIL’s mandates is to facilitate capacity development of its members. A questionnaire was developed to map the quality of organisations through identifying their strengths and weaknesses. The MOCA assessment project involved 267 samples, comprising of 195 NGOs, 52 Foundations, and 19 Associations. To run this assessment, FONGTIL employed 14 facilitators which included DLO’s and the MOCA team itself. The pilot testing in Dili began on the 9th of October and finished on the 30th of November 2012. It assessed 20 NGOs, and was conducted by seven members of the MOCA team. Before beginning the assessment, the facilitators were required to attend interviews training which was conducted over three days (29-31 January, 2013), the purpose of which was to guarantee the quality of the assessment. The assessment then began on the 1st of February 2013 in twelve districts, with a target of 80% of the 367 members, which totalled about 294 members. By the 22nd of July 2013, 267 had returned the questionnaire. The process of data entry began on 23rd of May and continued up to the 28th of August 2013, and took more than 97 days to complete.

This chronology only partially reflects the activities of the MOCA project; there are fifteen more activities which needed to be completed over four months, between September and December 2013. Some training was done for the capacity development of the members and staff of FONGTIL. The recruitment of an independent consultant for analysing the baseline database was delayed, and this affected the timing for the publication of the baseline assessment report.

Program rationale and logic
According to the Term of Reference (TOR) the objective of the BOCA is to assess the outcome of the MOCA project, which reviewed the progress of activities against the estimated results, and which targeted three main objectives as described below;
1. Increased understanding of Member-Organizations capacity development needs, by themselves, their development partners/funders and FONGTIL, leading to better and realistic actions to address their priority needs, either by themselves and/or through the facilitation of FONGTIL, or with their partner donors, other International NGOs or key National NGOs.
2. Published results of the baseline assessment have effectively served as benchmark for FONGTIL-member organisations for future improvement and as a valuable reference for development partners in relation to human resource development in Timor-Leste.
3. FONGTIL Secretariat is effectively capacitated to efficiently and more effectively address its core service of providing sustainable organizational capacity development assistance to its members.
To evaluate these three main objectives, the evaluator used three different approaches as the methods for assessment, including data review and analysis, direct interviews and focus group discussions.

Methods of Evaluation
This evaluation was conducted by using the methodology of document review, face to face interviews and focus group discussions, as well as data analysis of the contract agreement between the European Commission and FONGTIL, the MOCA action plan, the interim and annual reports, as well as several related important documents. Details of these documents can be seen on the references list below. The evaluation began with two days of document review (10 – 11 Feb, 2014), followed by three days of face to face interviews (12, 13, 14 Feb, 2014) in Dili (five NGOs per day - fifteen in total). During the time in Dili, the evaluator also interviewed the MOCA teams. Questions included, what do they think about the MOCA assessment; what kind of benefits have they received from FONGTIL since they become members; and what kind of training do they want from FONGTIL that will help them to increase their capacity.

After Dili, the evaluation continued in the districts over five days, beginning in Viqueque, then Baucau, Liquica, Bobonaro and Covalima. In the districts the evaluators used a different approach for conducting the evaluation, that is focus group discussions rather than individual interviews. Due to limited time the discussion was conducted in one place and was organised by the FONGTIL District Liaison Officer. In Viqueque district, the meeting was attended by eleven NGOs (twelve were expected), in Baucau ten NGOs (ten were expected), in Liquica twelve members (fourteen were expected), in Bobonaro five members (of the expected fifteen). One of the reasons for the low attendance in Bobonaro was because it was raining heavily and the roads were slippery, making access to the location was difficult. Low attendance did mean however that those who did attend had a lot of time to express their views. In Suai the meeting was attended by fourteen of the expected eighteen participants. Details of participant attendance can be seen in the below table.

The evaluation process in the districts consisted of three topics on the agenda; first, verification of key documents including the Constitution of the organisation, internal regulations, rules and guidelines for finance, human resources, assets, as well as monitoring, and evaluation reports. Secondly, evaluation of the relationship between the members and FONGTIL, which focused on capacity development, training and advocacy. Thirdly, the issue was raised by the members about the limits of donor support from external and internal sources, especially civil society funding from the Prime Minister’s Office. As described in the analytical report of the baseline assessment by an independent consultant, INSIGHT, 125 of 267 members rely on short term funding for specific projects, with a budget not exceeding $25,000. Therefore, the discussion about FONGTIL facilitating donor access to their members became an important topic of discussion.

The total number of individuals involved in this process of evaluation was 96 people, which included 39 directors, 16 program managers, 24 key staff, six DLO’s, and four board members, including a former board member, a director, one former director of FONGTIL and five MOCA members. The evaluation in the five districts (not Dili) involved interviews with the DLO’s, in order to determine the working relationship between the DLO’s and the Secretariat of FONGTIL in Dili. After concluding the initial interviews in Dili and the other five districts, the interviews continued on in Dili, which included interviews with the DLO from Maubisse (responsible for Aileu and Ainaro districts). The interviews also involved the MOCA teams, some of team leaders, board members and the Executive Director of FONGTIL. The objective of these additional interviews in Dili was to clarify some issues that had been raised by the members and the DLO’s from districts.

The questions asked in the interviews revolved around the organisation having their own constitution, internal regulations, human resource guidelines, financial regulations, asset management guidelines, and guidelines for planning, monitoring, evaluation and reporting. The evaluator considered these questions to be key to identifying the capacity development of each member. The organisations were asked to share their experience of preparing and finalising these documents. It should be noted that many of the member organisations did not understand what the differences between some of these key documents.

Strengths and weaknesses of the methodology
The strengths and weaknesses of this evaluation are varied; in Dili the face to face interviews meant that most of the time the evaluator met only with the Director him/herself, or just the program manager or even sometimes with just one staff member alone. This meant that it was difficult to verify whether what he or she said is the truth. During the interviews in Dili with fifteen organisations, only three organisations had more than one person participate in the interview. The majority of the time the person involved in the interview was the one who had filled out the MOCA questionnaire.

In the districts the focus group meetings had advantages and disadvantages; the advantage was that there was time to talk because the evaluator as the moderator gave opportunity to all of the participants to say something. But much of the time the men talked a lot compared to the women, and the women were not inclined to say anything contrary to what the men had said. For example in Baucau, when it came to a discussion about funding for civil society from government, the men expressed that they believed funding should be awarded based on the quality of the proposal, whereas the women preferred that all members had the same rights to receive funding, irrespective of the quality of the proposal. The discussion was not balanced due to the dominance of the men in the discussion.

Findings
This report has the findings of the assessment into three categories, and sub-categories according to the terms of reference of the evaluation. First, to identify the capacity development of each member before and after they received training from FONGTIL or other organisation. Second, to evaluate the published results of the baseline assessment. However because the analytical report of the baseline assessment has not yet been published, the evaluator could not conduct this part of the evaluation; instead, the evaluator has attempted to identify what has caused the delay in publishing the report, including looking at identifying internal and external factors that may have contributed to the failure of this activity. Third, according to the terms of reference, the evaluator should identify the capacity of staff at the FONGTIL secretariat in providing sustainable organisational capacity development assistance to its members.

During the process of evaluation, the evaluator gathered that from beginning of the consultation process the design process for the questionnaire was not optimal. This information was identified through interviews with the members and some of the FONGTIL staff. According to the analytical report of the baseline assessment, there was an interviews training session conducted between 29 - 31 of January 2013; and the time provided for doing the interviews should have been from 4th of February until 30th of June 2013; however, this evaluator did not find any evidence that the MOCA questionnaire form had been completed by the interviewees, and during the interviews there was also no evidence that any of the interviewees had been questioned by the interviewers. The pilot project that was conducted in Dili from 9th of October until 30th of November 2012 was not optimised to identify weaknesses of the questionnaire, and even some of the board members did not understand the methodology of the assessment.

According to the interim narrative report which was published on 30 September 2013, the seventeen essential activities are correlated to each other, therefore if there is failure to implement one activity, this will affect the outcome for the other objectives.
The below table shows the findings of the evaluation.

Internal Findings
Internal findings includes evidence that has been identified from internal activity which has been planned but not executed. All evidence has to be justified against the result of planned activities.
The main objective that is aimed for from this MOCA assessment will not be accomplished, because the six main activities that would justify the capacity of the Secretariat and its members was removed from the plan. It is therefore difficult for the evaluator to analyse the capacity development of each of the actors who play an important role. In order to bring about an outcome, this report will analyse each activity against the progress result.

Activity - M01
Beginning with the design of the IT-based database for internal use and for baseline analysis - this activity has not yet begun; according to FONGTIL the time was limited in order to implement this activity. The intention of this particular activity was not only for internal use but also for baseline analysis, which means that this IT-based database was also intentionally designed for analysis of the data from the MOCA assessment questionnaire. However, because the independent consultant (INSIGHT) has declared that the database design was technically incorrect, no one has wanted to continue the job that was left.

Comment
The FONGTIL database is still very basic and not comprehensive; if a donor wanted to find out information about a specific NGO based in a specific region, with all the details about organisational performance, it would be difficult to find the answer from the database.

According to the interview with the Information and Technology (IT) Officer of FONGTIL, it appears that the job description for the IT Officer is to increase the knowledge and capacity of FONGTIL members about LINUX software, focusing on how to introduce the new program and not about how to fix the database system. The IT Officer has produced a manual for the new software LINUX program, which is unique as it is written in Tetum, however is this really what is expected of the IT Officer as per the job description.

Activity - S03
Regarding the Intensive training on Organisational Development for DLO’s and the Capacity Building Team, this activity was implemented, according to the FONGTIL staff, and was also reported in the interim narrative report of 2013. The idea of this activity was to improve staff capacity in the Secretariat as well as the DLO’s. The training was attended by twelve participants, including five DLO’s from Baucau, Maubisse, Liquica, Lospalos and Manufahi and seven members of MOCA’s team, as well as two staff from different divisions of FONGTIL. The training was conducted by a local private firm (INVEST PEOPLE).

Comment
After reading the report of the training, the evaluator has concluded that the training appears to have been very general, and has little engagement to the reality of FONGTIL activities. The training materials can be considered standard for any NGO staff around the world, which has been translated into Tetum. The evaluator questions the capacity of local Timorese staff to absorb the material and understand the topics. The evaluator also did not see any feedback assessment or evaluation of the methodology of the training. During the evaluation, the evaluator spent much time with the MOCA team, as well as the DLO’s from Maubisse, Baucau and Liquica. Upon attempting to explore how far they have absorbed the material from the training, there was little evidence to show that they had.

Recommendation
For the future, for this activity FONGTIL needs to appoint someone to write a detailed special report about the training, for which the content would include the objective of the training, a copy of the training materials, an evaluation of the training material and the facilitator, and details of follow up after the training. This special report could be developed into a handbook, which could become a guideline for planning and evaluation for the next training.

Activity – S04a
Training on Finance Information System (FIS) for all staff is very important and crucial for the successful financial management of organisations, therefore FONGTIL has seriously taken this activity into consideration. The activity was executed in mid-December 2013, contrary to the plan in which it should have been implemented in November. However, it was attended by 28 participants, most of whom were internal staff at the Secretariat and DLO’s of FONGTIL. The training was facilitated by the finance officer and the team, and objective of the training was to increase the knowledge of rules of financial management of an organisation, which was one of the recommendations of the external financial auditor.

Comment
The training was attended by almost all FONGTIL staff including DLO’s, which was very good. The evaluator did not see training materials and did not have a chance to speak to the Finance Officer who was busy with the financial audit at the time.

Recommendation
For future training it is recommended that two different materials are used - the FONGTIL finance manual and the finance manual from another organisation, so that the two can be compared. As mentioned in the previous recommendation, a special report should be written about the training which can be developed into a handbook of guidelines.
Activity - S06
Intensive training on Monitoring & Evaluation for all Program Officers and District Liaison Officers - this training was conducted twice with different participants, in September and October 2013. The training involved DLO’s and NGO members who are involved in monitoring networking organisations (Rede Monitorizasaun) and the facilitator for the trainings was the same person from INVEST PEOPLE.

Comment
FONGTIL already have their own guidelines for planning, monitoring, evaluation and reporting, prepared by the Logistics Manager with the President of the FONGTIL Board and approved by the Executive Director in May 2013. The evaluator has analysed the existing guidelines, which appear to be very general, and which would need more input and contribution to make them simpler and easier to understand and implement. This training could have been used to discuss and complete the existing guidelines rather than hire a facilitator to produce a new template.
According to the verification form that the evaluator received from participants during the evaluation process, fifty percent (50% of 45) respondents stated that they do have guidelines for planning, monitoring, evaluation and reporting (PMER). According to baseline report, six months after the training, fourteen percent (14% of 267) members stated that they do have PMER documents. There is not much difference between the two statistics: six months ago 38 of 267 members have said they do have these guidelines, and now 22 of 45 members state that they have them. To clarify, the 45 members are those that were involved during the evaluation process.
The evaluator recognises that is not easy to measure the outcome of this activity in the short term, because it is dependent on the ability of each participant to understand the topic, and also on organisational capability to implement the guidelines. The evaluator believes that as most of the members are in receipt of donor funding, they must be able to apply PMER principles and activities into their funding proposals.

Recommendation
In the future, FONGTIL should be more diligent in measuring training outcomes. If the DLO’s have already received the training, it should be followed up by challenging them to provide similar training in their own districts, by using similar guidelines. If the DLO from Covalima is not confident enough to facilitate the training, the methodology of an individual facilitator can be changed to a team facilitator, making it easier to provide good quality training and produce good quality guidelines.

Interim Financial and Narrative Report
The Interim Financial & Narrative Report was published on 30 of September 2013. The document clearly explains the progress of each activity; some of the explanations are very detailed. However the report is in English only, making it difficult for most of the staff to understand.

Recommendation
In future the document needs to be prepared in both languages, English and Tetum. If this is difficult to organise, the Performance Development Review document needs to be reviewed to include multi-language as an indicator for the PDR evaluation.

Activity - S07a
Hiring of a Technical Writer for the Baseline Report - according to the 2013 interim report, the recruitment process began in March, but due to the low budget to pay for the consultant, nobody applied. After altering the budget line, finally FONGTIL awarded the job an independent consultant from INSIGHT. The contract began on 9 September and finished on 16 December.
For the process of data analysis, the independent consultant for technical reasons refused to use the SPSS program; he proposed to FONGTIL to increase payment with the reason that he would need time to re-enter the data and change the previous SPSS program into ODK. This therefore affected the due date for submission of the draft document, which was over by one month. The draft document was received on 17 of January 2014, and a few weeks after that the final document was received on 9 February. This process was very complicated, because at the beginning of the data entry process, two FONGTIL staff members received training from an SPSS specialist and, according to them, they and the specialist had though that the data entered was correct until the INSIGHT consultant said otherwise.

Comment
The evaluator has analysed this case and has concluded that from the start, the design of the MOCA project was not optimum. According to one of the former board members interviewed, it had not yet been decided how FONGTIL were going to analysis the data after the collection of the questionnaires. The SPSS program was introduced by one of the international advisors who was working with one of the international NGOs in Dili, however when the data was ready for entry into the SPSS system, the advisor who introduced the system had to leave because her contract had finished. The new advisor was not familiar with the SPSS program.

Recommendation
In future, if FONGTIL were to design a similar project, a simple program should be used for data analysis, and FONGTIL should ensure that they have enough human resources to execute the activity. The time that it takes for data entry should also be considered, and it should also be ensured that there is a budget line allocated in the proposal to hire an advisor to work together with the team from the start until the completion of the project.
The following six activities were requested to be removed by FONGTIL, as described in the interim narrative report 2013. Even though this was the case, it is still possible to implement them if there is any funding available in the future.

Activities – S08 & M07
Hiring of technical writers and development of three top priority capacity development modules, activity - S08, and activity - M07. These activities were requested to be removed from the 2013 Activity Plan.

Recommendation
These tasks be joined into one activity, in order to avoid confusion for the writers and for FONGTIL itself. The Terms of Reference for the technical writer did not clearly express that the consultant would need to identify three priorities of capacity development needs, however the identification of these three priorities should be not be difficult to do. Now FONGTIL has to do this process on their own, by going through again the 267 MOCA questionnaires and identifying those priorities.
According to key documents that were collected by the evaluator during the consultation process, 45 respondents of the 69 participants who attended the consultation meeting in Viqueque, Baucau, Liquica, Bobonaro and Covalima stated their top three priorities. One hundred percent of the organisations have their own Constitution otherwise they could not be registered with FONGTIL; eighty two percent (82% of 45) stated that they have internal regulations; thirty four percent (34% of 45) have guidelines for human resources management, and forty five percent (45% of 45) have a regulation for managing their assets, while fifty five percent (55% of 45) have a finance manual and fifty percent (50% of 45) stated that they do have guidelines for planning, monitoring, evaluation and reporting.

These key documents from 45 organisations identified that the members’ training priorities are, in order of priority, training in drafting of guidelines for human resources management; drafting of guidelines for assets management; and drafting of guidelines for planning, monitoring, evaluation and reporting.

Activities M08 & M10
Intensive Training of Capacity Development Planning Facilitators – M08 and Capacity Development Planning Workshop – M10 - these two activities were requested for removal from the 2013 Activities Plan.

Recommendations
These two activities need to be combined into one activity, because to plan a workshop for capacity development of members depends upon capacity of facilitators that have received the intensive training. The capacity of facilitators is important in order to ensure that there is consistency of results.

Activity M10a & M12
FONGTIL and 180 members (60% of 375) undertook Capacity Development Planning – M10a and Training on the Top Three Capacity Priorities for 54 members (30% of 180) – M12. These activities have been requested for removal in the 2013 Activities Plan.

Recommendations
Even though these activities were requested for removal from the 2013 Activity Plan, the evaluator considers that this could be included in the 2014 activity plan, because the final outcome of the MOCA projects depends upon these activities. To measure the capacity development of its members including staff of the secretariat depends upon this kind of activity; therefore the evaluator recommends that this activity be implemented.

Activity M13
The Advocacy Campaign was to be based upon the Baseline Results (see SO7b), however this activity has not yet been implemented because the final baseline report was delayed and has affected this activity. This activity includes producing 1000 copies of a CD Rom, and pamphlets, and can be considered to be a very big activity requiring a lot of human resources to execute. Because it has been requested for removal from the 2013 Activity Plan, it is difficult for the evaluator to assess point number two of the Term of Reference.

Capacity Development Training
There are a number of trainings that were conducted by FONGTIL with the members, and some of these activities have already been identified in the MOCA project; other activities were not included as part of the MOCA project, however the evaluator still considers that the benefit of the training can be justified as the capacity development of members, measured against objectives number 1 and 3. The table below shows us that the number of participants was very small. Three trainings which were part of the MOCA project activity plan such as for Financial Information System (FIS- S04a), Monitoring & Evaluation (ME-S06), and also Organisational Development (OD-S03), were implemented in 2013. The six trainings listed below have so far capacitated 94 people, representing 54 organisations and 40 staff from the Secretariat and DLO’s of FONGTIL. Appendix 2 shows the list of training participants. 54 organisations of 267 members, is only 20% of training over a period of twelve months. Moreover this data only shows us the rate of participation itself and does not tell us anything about the level at which the participants understood the training material.

Even though the training involved 40 FONGTIL staff members, this does not mean that all FONGTIL staff have participated in the training. Table 2 shows us that of the ten DLO’s only five have attended two trainings, and another five participated in one training. Of the management team, one person attended three trainings, two attended two trainings, and another two participated in one training. In the Secretariat there was one staff member who attended 3 trainings and five staff attended two trainings, and the remaining nineteen attended one training.

Table 2 shows that the capacity development activities of FONGTIL staff is minimal - how can the Secretariat capacitate 267 members if the Secretariat themselves do not have enough capacity themselves.

External findings
External findings are external factors that have contributed to the successes and failures of the MOCA process. For example, concern about services that were supposed to have been provided by FONGTIL to their members; concern about funding for civil society from the Prime Minister’s Office; concern about the separation of powers between the Board and the Executive, etc. The evaluator aimed to identify other influences that may have caused some activities to not be implemented.
During the process of evaluation, the evaluator also asked questions to participants regarding how far they understood the MOCA process. According to sixty seven organisations from the six districts who participated in the consultation process, forty percent (40% of 67) understood the MOCA assessment, and eleven percent (11% of 67) stated that they did not understand the objective; a large forty seven percent (47% of 67) answered that they did understand the MOCA assessment, but when the evaluator asked them to explain it, it was clear that they actually did not understand.

Dili
The interviews in Dili began on 13, 14, and 15 of February 2014. Consultation was divided into three days of face to face interviews; for each day the evaluator targeted five organisations - fifteen all together at the end of day three. The evaluator has found very variable findings, some of which are very interesting to explore. For example; seven of the fifteen NGOs interviewed in Dili stated that FONGTIL was not pro-active in introducing members to donors, and eight organisations stated that they do not need help from FONGTIL. One of the members had a creative idea which involved FONGTIL facilitating a joint proposal, combining four or five members together to apply to one donor. Two vulnerable organisations stated that they need assistance from FONGTIL to do advocacy with them in order to campaign about their rights of living. In contrast, thirteen organisations in Dili said that they do not really care if the advocacy division of FONGTIL could assist them. Two organisations in Dili had appreciated work that had been done by the DLO’s of Maubisse and Baucau, who according to them are very pro-active. Eight organisations stated that they have some experience with assessments from FONGTIL, but they did not see any results or follow up. Eight members declared that they were willing to collaborate with FONGTIL in order to facilitate capacity development training to members in the districts but they did not see any plan or engagement from FONGTIL. All fifteen members in Dili questioned their member fee payment – they believed that expenditure of the member fee should be more transparent. Only one organisation recognised that some of the FONGTIL Secretariat staff had helped them in writing a funding proposal. Thirteen organisations stated that they never receive any training from FONGTIL, or facilitated by the FONGTIL Secretariat. Two of the fifteen informed that they had previously participated in FONGTIL training, about preparing a draft manual for finance and advocacy using social networking media.

Viqueque
On 17 of February 2014, the evaluator held a consultation meeting with FONGTIL members in Viqueque district, involving eleven local organisations. The meeting used a method called focus group consultation, in which all the members including the evaluator talked and discussed on previously determined topics for discussion, which included a verification of the MOCA assessment, about the key documents as referred to in Section II; and the members’ opinion of FONGTIL services to members. Another topic was raised by the focus group participants, about access to the civil society fund. There was concern from many members about the predominance of donor funding in Dili only; how they rarely or never receive any training from FONGTIL about how to write a good proposal. One member recognised that FONGTIL had facilitated her organisation to attend an international conference overseas. Ten organisations agreed that the DLO of Viqueque is very active, however some questioned his job description. However, all of them agreed that the DLO should have his/her own office. The eleven organisations also agreed that FONGTIL should work with international NGOs in collaborating with local NGOs in implementing their programs and activities in Viqueque. They provided an example were collaboration did not occur, even though the international and local NGO have the same program and activity. Nine organisations questioned why last year only two local organisations were able to access the civil society fund from the Prime Minister’s Office. All eleven members of FONGTIL agreed that the policy around the civil society fund should be changed and adopt a flat rate or equal distribution to all NGO members with total budget amount of $25,000 per NGO.

Baucau
The meeting in Baucau was held on 18 of February 2014, and according to the list of participants, the ten organisations who were involved in the MOCA project all participated again in this consultation meeting. Three of the ten members has criticised a lot about FONGTIL services to members, however most of these criticisms were without evidence and fact. Five organisations did not agree about the flat rate policy for the civil society fund, which had also been referred to by the members from Viqueque. One organisation supported by another four organisations agreed with the policy of equal $25,000 distribution for all NGOs to use for planned programs and activities.

Liquica
On 19 of February 2014, a consultation meeting was held in Liquica, attended by twelve organisations. All members agreed that there is always a regular meeting every month between members and the DLO, however the topic of discussion is always around the monitoring of infrastructure projects in Liquica district; there is little discussion about how to increase member’s needs on capacity development. Eleven of twelve members stated that FONGTIL never facilitates trainings about how to draft key documents. Two members recognised that DLO does not have a clear task, because many times national and international NGOs in Dili have utilised the DLO for their own programs. Twelve members agreed that FONGTIL should facilitate meetings between donors and members in the districts;, the twelve members were also concerned that the DLO should be advocating for the member’s needs. One member has declared that they do not get any benefits to becoming a member of FONGTIL, even though they have paid a membership fee. Three members asked FONGTIL to finance monitoring network activities (Rede Monitorizasaun). One member of Rede Monitorizasaun asked the advocacy division of FONGTIL to advocate for any problems with infrastructures projects that were identified by the group (Rede), so that there would not be any negative impact. All of members of the meeting agreed that policy of civil society fund should be that it is available equally to everyone.

Bobonaro
This consultation was attended by five local organisations on Thursday 20 of February 2014. Only thirty three percent (33%) of those members who had been involved in the MOCA assessment took part. Only five organisations attended the meeting; while fifteen were expected, the bad weather was an issue on the day. The members complained that previously they had regular meetings with the former Executive Director of FONGTIL, however this was no longer happening. Only one organisation stated that they had an effective finance manual. Two members complained that FONGTIL applied for the same funding that they did, and that FONGTIL received the money but they did not. Two members complained that they had experienced many assessments from a range of organisations, but never saw any results or follow up. Five members remembered that an NGO had collected database information; while they did not know what for, they did see the FONGTIL logo at the top of the questionnaire form, which is why they were willing to provide information for the questionnaire. Two members questioned the job description of the DLO, who they felt did not have a specific set of tasks assigned. One member was concerned that the DLO was not active in advocacy. However they all agreed that it was important that the DLO should have his/her own office. Five members agreed that the policy of the civil society fund should be equal for every member, at $25,000.

Covalima
This consultation was held in Suai on Friday 21 of February 2014; there were fourteen participants. Three members complained that FONGTIL was no longer pro-active in meeting with its members in Covalima. Three participants recognised that they had been involved in some training provided by FONGTIL, for example training on finance manuals, and as result one organisation already had their own finance manual and another two were in the process of drafting theirs. Twelve members recognised that the DLO was very pro-active in advocacy, especially in response to an emergency situation or crisis, and that the DLO organised a regular meeting with them and the local authority. Despite this, five organisations complained that the DLO did not follow up their proposal that was submitted to donors. Three members said, and another five members agreed, that their organisations had not received any donor funding for the past three years. Eight out of fourteen members believed that FONGTIL does not have any fixed plan for capacity development of its members. Ten NGOs had experienced the database collection, however they did not know what it was for. Thirteen NGOs agreed to propose that FONGTIL should organise a mini Annual General Meeting in Covalima district, which would involve local authorities, donors and local members. Lastly, all of the fourteen NGOs had agreed to discuss the policy of the civil society fund.
The above findings have been classified into six different categories below:

Twenty percent (20% of 67) members commented that they need help from the FONGTIL Secretariat in facilitating donor access to them. Eight percent (8% of 67) said that they do not care, that FONGTIL is not relevant to them. Twenty three percent (23% of 67) members gave negative comments about FONGTIL. Three percent (3% of 67) members expressed that they willing to helps FONGTIL to capacitate other members. Thirteen percent (13% of 67) members stated that they are appreciative of FONGTIL’s assistance to their organisations, for example through trainings and facilitations. Thirty three percent (33% of 67) members recommended many points for FONGTIL’s follow up.

District Liaison Officer (DLO)
During the consultation process in the six districts, the evaluator also interviewed the DLO’s from each district. All of the DLO’s agreed that the FONGTIL Secretariat in Dili should have a plan for regular trainings for capacity development for its members. Four of them complained about difficulties working without an office. Six DLO’s questioned the coordination link from the Secretariat in Dili, because they felt that at times the Secretariat forced work on specific issues on them without enough coordination with the DLO’s. Six DLO’s were concerned about the minimum amount of financial support for monitoring of activities. One DLO in particular was upset with about the lack of financial support on a specific issue, and the same DLO also complained that the Secretariat did not want to support local NGOs for fundraising. All of them believed that there was no or limited planning agenda from the Secretariat, and that most of the time there were emergency agendas or activities put in place, which sometimes interfered with other activities already planned in the districts.

FONGTIL Board
The consultation process involved two active and two former board members; former board members were consulted because they had been involved in and knew the process of the MOCA assessment. One of the board members had been directly involved in designing the questionnaire for the MOCA assessment; another three had known about it but had not been directly involved. Some of the board members questioned the structure of the FONGTIL Executive which they believed to be too big and inefficient, and two board members stated that FONTIL’s operational costs were too big in comparison with the budget for programs and activities. Two board members felt that the Executive did not pay enough attention to prioritising the main program, preferring to execute small grants rather than big project. Some of the board members expressed their opinion about a lack of qualification and skills in the Secretariat, meaning that it was difficult to expect them to be able to provide capacity building to other members. One member stated that the FONGTIL Constitution did not really mandate for regular meetings, therefore many times there was no regular board meetingor the meetings did not achieve a quorum.

Comment
Board candidates must have knowledge of the separation of powers between the board and the staff as executive, and the difference between their functions. Some FONGTIL staff felt that there had been some interference from the Board to the staff Executive, an example being when the board decided to terminate the contracts of FONGTIL staff members and DLO’s; this has affected some project activities, such as the publishing of the final baselines assessment report results. This is one example where lack of knowledge for interpretation of the mandates has resulted in a negative impact to other projects.

Recommendation
The Board should consider the issue of separation of powers between the Executive and the Board at the next Annual General Meeting (AGM). In order to minimise the conflict of interest between the Board and the Executive, FONGTIL needs to re-evaluate the composition of its Board, and perhaps trial new board members from outside of the FONGTIL membership, who can provide good advice to the FONGTIL Executive, for example three independent members or experts, and four who still represent the members. This new system could be trialled for one board term.
The Board also needs to evaluate the duration of the contract for the FONGTIL Executive Director, whether three years is sufficient to measure the progress of results by the Executive Director. The evaluator would recommend a five year term.

Recommendations from this evaluation to the Management Team
In the future if there is the opportunity to implement a similar project that similar, it is recommended that FONGTIL discuss and analyse the benefits and impact to FONGTIL, before applying for the project. Before submitting the proposal, FONGTIL would need to measure the capacity of staff and available human resources, make sure that there will be enough time to implementing the project, and that there will not be, or only minimal clashes with other projects that will implemented at the same time. If lack of human resources is an issue, it would be better to recruit an advisor to accompany the project from the beginning until the end, ensuring that the cost for the advisor is included in the proposal. If the project is considered to be a big project that is likely to continue on for some time, it would be necessary to consolidate human resources and prioritise the project, while re-allocating the delivery of smaller projects to the DLO’s or other members of FONGTIL to execute.

The structure of the FONGTIL executive needs to change, as currently it is unclear who is in charge if the director is not available. There is not a clear management team; currently each division has their own mandate, which makes it very difficult for each team to listen to the other. For example, the MOCA project involved seven FONGTIL members, including 1) Manager of Membership Service and Training (MST), 2) Office Manager, 3) Membership Services Officer, 4) Human Resources Officer, 5) Capacity Building Officer, 6) Resource Centre Officer, and 7) Manager of Finance. If we consider that the Manager of MST is to lead the MOCA team, then the Office Manager should not be in that team, because if we consider the Office Manager to be second in charge after the Director, he/she should be acting as the Program Manager who is managing all the other divisions, and also ensuring that the division managers should be reporting to and are accountable to him or her. The Office Manager can take on the role of Acting Director if the Director is not available, not the President of the Board – this is confusing for the staff and managers because decisions are coming from the President of the Board which can prejudice the competence of the Executive.

Recommendations to the MOCA team
The Team Coordinator must be able to ensure that all activities are executed according to the established plan and time line. The Coordinator has to be responsible for the timetable, and each activity needs to be evaluated regularly, at minimum once per week. If there is an obstacle, this should be tried to be solved within the team first before taking to the office manager or director level. All team members have to have a specific responsibility for an activity and all team members should be expected to submit their plan of how to guarantee that each activity will be implemented according to the timeline.

Recommendation to the Executive Director
The Executive Director of FONGTIL needs to clarify the mandate, competence and duty and responsibility of the Executive Director as per the FONGTIL Constitution. The evaluator recommends that there is a submission put to the next Annual General Meeting for ratification of mandates and the competence of the Executive Director. The Executive Director should also be organising regular meetings with all members in the districts, and creating a healthy work environment at the FONGTIL Secretariat as well as out in the districts. If there is tension arising or problems between staff and staff, or staff with the board or staff with FONGTIL members, it is up to the Director to initiate or facilitate any activity that can re-open the lines of communication between them.

Conclusion
The data from Appendix 2 showed that in the past twelve months, FONGTIL has organised six trainings for building the capacity of their staff and members. Three of the six trainings were included in the 2013 Activity Plan 2013, which is Organisational Development, Financial Information System, and Monitoring and Evaluation training, while the other three trainings were not a part of the MOCA project activity. A total of 94 participants attended the training, comprised of 54 participants from the membership and 40 from the Secretariat and DLO’s of FONGTIL. 54 of 267 members is only twenty percent (20%) of the FONGTIL membership that has benefitted from these trainings; twenty percent over twelve months is very low as an indicator for capacity development of its members. This indicator moreover is only for the level of participation, not yet for the level of comprehension of the material and the ability to put it into practice.

Objective number two cannot be assessed because the baseline result has not yet been published. However, the evaluator did try to ascertain the factors that had caused this delay. According to the evidence and analysis, the evaluator concluded that the project design was not optimal from the beginning, the lack of experience of FONGTIL staff also contributed to the problem, and the Board was not proactive in assisting the execution of some of the required activities. The result is that the outcome now cannot be measured.

The capacity FONGTIL staff including the Secretariat and the DLO’s is minimal; according to Appendix 2 below and Table 2 above, over a twelve month period only two staff members attended training three times; twelve staff attended twice and twenty seven staff attended training once. It can be questioned how the FONGTIL Secretariat is to provide capacity development to 267 members if the staff themselves have received very minimal training.

Above all, the removal of six main activities from the activity plan for the MOCA project has affected the ability to measure the capacity development of FONGTIL’s members. The internal factors which included the initial process for design of the project, regarding issues of time management and appropriate human resources, and the lack of experience of the staff, as well as lack of supervision by the management team has caused the failure of the project.

External factors that have contributed to the failure of the project is the complexity of problems that FONGTIL’s members face in their own organisations, and their relationship with FONGTIL, which has accumulated and worsened over time. The different situations and opinion of the 67 interviewed members can be seen in the pie chart above, most notably their ambivalent relationship with and criticisms of FONGTIL. The above recommendations regarding these external factors would go some way to repairing these relationships.

References
FONGTIL, 2011, Grant Contract External Actions Of The European Union, DCI-NSAPVD/2011/277
FONGTIL., 2011. ‘Grant Contract External Actions of the European Union’: DCI-NSAPVD/2011/277-972
FONGTIL., 2012. ‘Interim Narrative Report: Annex VI’.
FONGTIL.,2012. ‘MATADALAN LIBRE OFFICE’.
FONGTIL.,2012. ‘Matadalan Planeamentu, Monitorizasaun, Evaluasaun no Reportazem’.
FONGTIL.,2012. ‘Matadalan Oinsa Uza MOCA’.
FONGTIL.,2012. ‘MOCA Questionary Tools’
FONGTIL., 2012. ‘Relatoriu Annual Forum ONG Timor-Leste’.
FONGTIL., 2013. ‘Interim Narrative Report: Annex VI’.
FONGTIL., 2013. ‘Mata Dalan Papel no Funsaun Konsellu FONGTIL’, Konsellu FONGTIL.
FONGTIL., 2013. ’Matadalan Membership Membru FONGTIL’.
Soares, E., 2013. ‘FONGTIL Members Organisational Capacity Assessment: MOCA, Analytical Report’. INSIGHT.
Notaras, M., 2013. ‘Mid-term External Evaluation – 2012/2013 EU Project including FONGTIL MOCA Assessment.

0 Comments