How do you carry out a participatory research project?
- What are the models of participatory research?
- How do you design a participatory research project? What are the steps?
- What is important to keep in mind when desigining a participatory research project?
- How does each model address decisions around participation and verification?
How is it evaluated?
- How do you evaluate participatory research?
- How do you measure the impact?
- How do you know when participatory research is successful?
Share your thoughts, ideas, questions and stories below by adding a new comment to this thread or replying to existing comments.
One effective means to evaluate a participatory research program or project is to begin with creating indicators or "benchmarks" with respect to the conditions or types of outcomes that are being studied or trying to be improved. And those who are the intended beneficiaries should be the first set of actors involved in helping to identify or create those benchmarks.
If, for example, a particular research project is intended to test ways in which local peoples can maintain their stewardship over tradtional ecological resources and knowledge (and by virtue of that, the benefits that such knowledge bestows on their ability to maintain their way of life), one approach is to work with those peoples to determine benchmarks. How much access do they have to their traditional knowledge before the research program begins? How effectively can they use that knowledge to maintain or improve their well-being? Such questions of course have to be placed in a cultural context, but if posed clearly, the beneficiaries of a participatory research project/program should be able to provide answers that establish a baseline.
From there, it is up to all parties involved in the research project/program to continually monitor and evaluate outcomes, and compare them to the benchmarks. Are conditions improving? Is there a net benefit or change for the better? In that respect, evaluating the effectiveness of participatory research is an ongoing process--not merely something that is done after the project or program is conducted. Too often, the "evaluation" aspect tends to go overlooked, but it is crucial.
Hi Michael,
Thanks for starting this thread on evaluating participatory research! I couldn't agree more with your conclusion that evaluation is crucial. It seems to me like evaluation is almost inherent in the participatory action research concept. I think it has to do with the collective empowerment of the community participating and also the collective ownership of the process and the outcomes that leaves no room to just leave out the evaluation part.
It would be great to hear from you all - some examples of how evaluation has seemlessly been carried out in a participatory action research project, and also examples where it has not worked so well!
Hi everyone,
Together with several colleagues, I just finished working on a four year participatory action research study with young mothers who were former child soldiers in Liberia, Sierra Leone, and northern Uganda. There are two written materials we have recently produced that include detailed information about how the young mothers in the program developed indicators that we used to quantitatively evaluate the program. We also used process indicators along the way to evaluate how well the program was proceeding in a participatory manner. The first material is our final report and the second is a handbook for practitioners. The report gives detailed information about the study, while the handbook uses case studies and examples from the study to help practitioners think through new ways of either doing a fully PAR program or incorporating more participatory methods into ongoing programming.
Both the report and the handbook can be downloaded from our website: www.pargirlmothers.com
Best regards,
Michael, I can't agree with you more. So many research projects run into trouble when they try to figure out ex post facto what they've achieved, and no one established benchmarks at the outset. The benchmark process also helps implementers refine their goals, and decide what can and cannot be measured. (It is my belief that many worthwhile projects can come up with good metrics -- but not all.)
Some of the most elegant work I've seen in M&E in the media field is with the BBC World Service Trust's Research and Learning Group: http://www.bbc.co.uk/worldservice/trust/research/reports/
My Columbia graduate student class is preparing a wiki publication on new mobile platforms for M&E, which should be completed by the end of the year.
My career has been in media and human rights, and the media focus is an advantage in documentation in many ways. I was the director of the Committee to Protect Journalists between 1988 and 1992. The organization began by responding to individuals appeals to assist journalists and protest abuses of press freedom. But we found that formalizing the research component was vital to the rest of the work. Over the years, CPJ has assembled an impressive collection of journalists around the world who report on both local press conditions and specific abuses. As mentioned above, the issue of trust, and often confidentiality, is fundamental. Sometimes well-meaning practitioners can "burn" a source in the attempt to be transparent with sourcing. The best response is to support local institutions. In the freedom of expression sector of human rights, some of this work has been developed through the IFEX Clearinghouse (International Freedom of Expression Exchange, www.ifex.org). In this way, threatened journalists join an international network of mutual support.
Very good points. I believe human rights researchers and practitioners should be somewhat like medical doctors in their thinking: first of all, do no harm. If your action/research is going to cause an individual to be hurt or killed, that is definitely harm. In my field (survey methods, statistics) the problem isn't that the researchers don't care about the community being researchered; it is that they don't always think through the ethical ramifications of their methods or even what the existence of the dataset created might mean!
Thanks for bringing up issues around security and the potential risks of having/storing datasets...
It's important to think about how the data that is being collected will be stored securely - meaning, be sure to back up the information and make sure that the data is stored in a way that others can't see it! For tools and ideas on how to do this (such as using TrueCrypt to encrypt files on your computer and on external hard drives) I would recommend looking at Tactical Tech's Security in a Box toolkit and also New Tactic's online dialogue on Staying Safe: Security Resources for Human Rights Defenders.
Martus, product of Benetech: www.benetech.org
Martus uploads data to a "safehouse" in another country, allowing local offices to flush data if being raided, recover data if offices are damaged/destroyed, and so on.
Jana has given some great examples of participatory action over in the thread where we define PAR:
This now raises a different question for me that i wanted to move into this thread where we talk about how to carry out a PAR project. By now i have got the impression that several sets of skills and tasks are required in order to do PAR. And probably different actors will bring in these skills:
I am speaking from the perspective of the scientist who has only marginal experience in advocacy and community organizing. Therefore, i am now wondering whether PAR actually involves at least three types of actors who triangulate their skills and resources.
I am now thinking of one example from Liberia, which i presented in another thread. This project was a conjoint initiative of a local Liberian women organization that recruited from the local community, together with an international NGO - Women's Rights International, and assisted by Herbert Spirer a renowned scholar in the quantitative documentation of human rights violations.
Who can tell from past experience what triangulation of actors is the most promising setup to ensure successful implementation and effective impact of PAR projects that range from research to action?
Thank you, Jule, for bringing up the question of required skills for PAR. I think it closely relates to a post in the very first thread of the dialogue that highlight the importance of cultural competency and context in PAR work:
I would like to ask: How do you select researchers for a particular project? How do you determine their cultural competency? What resources are available to research groups to help build cultural competency?
PAR research can play a huge role in guiding policy decisions. However, in one of my community-organizing experiences (organizing immigrant and refugee communities in an urban area to alter data collection standard in order to address health disparities) I learned that governing authorities can often be reluctant to involve people of the same ethnic background as the population in question. While some claim that this allows for more "objective" research, community members sometimes challenge the findings of such research (for it may contain flawed results, due to cultural misconceptions/misunderstandings). The very spirit and goals of PAR emphasize the importance of representing the voice and objectives of the community, and the work that the participants of this dialogue are engaged in sets a powerful example that governmental authorities can learn from.
Dear Vera,
thank you, i think you have touched upon another important aspect of carrying out PAR. Let me try to answer your questions. I actually think they go in another direction as to the question i asked beforehand.
As far as i have understood the debate so far, PAR is driven by the people who want to do it. So i guess researchers self-select themselves when the current situation of a population is important to them. Or, if a community drives to conduct a project, they select the researcher upon the criteria important to them. If a third party wanted to do a PAR project they would search for a researcher that is familiar for working on the issues (similiar to the one) concerned, familiar with PAR, or both. I guess if PAR is done properly, ideally the researcher would be very open and receptive to the needs of the population she is studying. Hence, in the ideal PAR project both sides learn: the population acquires new research skills, the scientist deepens her cultural competency.
I am interested what others think about this. Maybe my reasoning is too simple.
Let me come back to my question above. I intended to point in another direction. If the final outcome of PAR is to start a campaign (= action) involving the population that was studied, e.g. submitting a petition to actors on the international agenda, collaborating with actors in the (international) non-governmental sphere, then i could imagine that the researcher and the community alone might not be too familiar with the sector they are targeting, in terms of the important stakeholders with whom one has to build alliances or who are important adversaries. Such 'action' requires advocacy skills, i.e. with running campaigns at the levels above the locale. Speaking from the perspective of the scientist, i do not have much experience in that field, but the main goal of PAR is action. But maybe, scientists specialised in PAR also bring advocacy competences?
New Tactics has a great approach to map the stakeholders around the goal and issue that is important to activists. In this mapping process, you map potential alliances and adversaries to your goal along this pro/con-dimension and think hard about whom you can move closer towards your own goals, designing the tactics required for that. So, i am wondering whether the researcher and the community alone are able to do this, or, whether at such tactical stage of the project it would be helpful to have a third actor who can identify key players in the field, strategize tactics etc. And if so, i imagine this third party had to be involved in a PAR project from the outset, because he/she would know what kind of information research had to investigate in order to feed a campaign--i.e. defining milestones, project benchmarks, goals etc. at the initial planning stage of PAR.
Research is a method to produce findings and make conclusions. In PAR, research involves the local expertise of the population. We agreed in this thread that the presentation and use of the research results is another important part of PAR that should empower the people to act upon themselves. Is the researcher the best person to provide for this? What set of skills is key for PAR to be really successful from start to finish, to result in action and have an impact, meaning it improves the situation of the population studied? Who brings what skills and competency? Would it be a good idea to collaborate with an advocacy group that is skilled in acting at the (supra)national level?
I think there are many examples of 'action' that don't require any expert advocates or activists (I guess it depends on your definitions of all these terms). For example, the 'action' that was taken in Mozambique after Wona Sanana facilitated a participatory research project focused on children's rights, included:
Many of these actions don't require a change in law or a court case - it sometimes, instead, requires raising awareness and education. So I think questions around 'who would be the best person to make a change' would depend on the outcomes of the research.
Although this raises another question - how are people made aware of all the possible interventions? Jule - I appreciate you mentioning our tool, Tactical Mapping, in your comment above! I agree - I think it could be a very helpful tool for people to figure out what those action steps will be.
Tactical Mapping is a method of visualizing the institutions and relationships sustaining human rights abuses, and then tracking the nature and potency of tactics available to affect these systems, ultimately serving as a tool to monitor the implementation of strategy.
It would be great to hear from others that have carried out PAR projects on facilitating the 'action' part of PAR. Did groups decide to collaborate with advocacy groups? With international groups? How do communities determine the best actions and interventions to address the issues addressed/raised in the research?
Thank you, Kristin, for presenting the Mozambique example. That indeed seems like an ideal initiative to bring about institutional change at the local level. I now assume that the subject and context of the PAR project determine what kinds of actors should be involved. For example, in health, education, cultural or some environmental projects i could imagine that the local population and researchers are possibly able to induce a lot of shift, whereas some political issues (e.g. minority rights) might require some lobbying and campaigning if the case has to be made at the supra-local level.
I second your invitation to others for sharing their experience with regard to the facilitation of action through PAR. Please do share your expertise.
Best wishes to everyone,
I would like to add that there likely are several "informal" things that can be done to enhance to the goals and purpose of a participatory research project, with an eye toward better understanding how conditions can be improved for the intended beneficiaries. Yes: sound, balanced evaluation plans are critical, but I wonder whether we--as practitioners--consider what could be done outside the scope of a project (but ethically) to further assess needs.
In the survey sample world, for example, it is possible to administer surveys with structured questions whose answers ultimately will be coded for data analysis. However, it also is possible to include open-ended, qualitative responses (for example, at the end of a survey), thereby allowing respondents to articulate their personal needs--all of which places a human face on a research project, and gives researchers insights as to concerns or problems that they otherwise might not have considered or thought of when designing their survey or data collection instrument. Doing so also helps respondents to feel less like research "subjects" and more like active participants, which I would hope would be a given in any participatory research project or program!
~Michael Kisielewski
Thank you, Michael. I think you have touched upon a very interesting point! It is true that as a scientist one is very focused on getting the research done. I find your recommendation for open-ended, unstructured questionnaire parts very useful. I immediately had to think of truth and reconciliation commission (TRC) processes in which everyone is invited to come forward and share their story. It is an integral part of TRCs to provide space for people to participate. I wonder if TRCs are another example of PAR, even though they are being organized at the national level given the mission and huge framework they comprise. Would you agree?
Dear Jule:
Thank you for that question--I had not given it much thought! Certainly, TRCs have the elements of participation combined with information gathering or data gathering--so on some basic level I would argue that they are a form of participatory action research. As you point out, TRCs are a national-level instrument (and one that can be very complex), which makes it somewhat harder to put them into a specific category. Perhaps we could state that they are a different "form" of PAR--a bit less traditional than the conventional understanding of participatory research?
Another question is how or whether practitioners could use the work of TRC processes to further more grassroots, traditional PAR projects. Once the work of a TRC has been completed, I think it would be invaluable to continue with some form of periodic follow-up that (ethically!) involves local peoples who can further inform the wider research community as to the nature and patterns of abuses.
~Michael Kisielewski, StatAid
When thinking about initiating an effective participatory research project, it seems like a precursor would be the capacity/confidence-building of individuals, particularly those who have not previously been included in decision-making processes. I am curious about the steps you take to prepare communities before implementing a particpatory research project to ensure that individuals feel personally empowered to participate. Furthermore, what are some strategies you recommend for building rapport with communities? What are the most effective ways for learning about the particular contexts in which the research will be conducted before beginning?
Dear Keren:
You pose some excellent questions--not all of which I think I can answer, but certainly some of which I would like to respond to.
I think that researchers and project teams must be prepared to build confidence and rapport with local communities/local peoples by acknolwedging in advance what some of the limitations of their work might be. By that, I am referring to how important it is to come to terms with the fact that some individuals will be more engaged and enthusiastic about a project than others. In a perfect world, all individuals would be equally enthused, but in reality, we must realize that not all persons will feel that way. One of the most important themes that has been raised since the first day of this dialogue has been the maxim of "do no harm," and I think that researchers must begin by building trust with local communities and their leaders, including the need to communicate what their goals are and how participation in a given project will not compromise the well-being or safety of local peoples.
It also is crucial to approach a participatory research project from an intellectually "humble" point of view, understanding that the chance to engage in such a project as a researcher is more of a privilege than a right. By demonstrating a firm understanding of the problems faced by the intended beneficiaries of a given project, those beneficiaries will be more likely to develop a stronger sense of trust and feel more engaged. And, of course, the practice of listening to the concerns of local participants cannot be understated. In the context of participatory research concerning human rights or humanitarian situations, we must not forget that local participants oftentimes have suffered greatly already, and that they are not seeking to be treated in a paternalistic way. That is something not easily taught when studying the practice of field work or in a strictly academic research setting.
~Michael Kisielewski, StatAid