Ronnie Vernooy
In the previous modules, you learned how to prepare, design, and conduct participatory research around resilient seed systems and climate change. You followed six steps to understand the local context and analyze how best to develop an adaptation strategy, together with farmers, that is focused on the acquisition and evaluation in the field of new crop varieties. Now that you have almost completed the research cycle, it is time to ask what has been achieved and what has beenlearned. In keeping with the overall approach of the resource box, answers to these questions can be found using a participatory method in which researchers and farmers and possibly other stakeholders work together to assess the achievements, reflect on problems and challenges encountered, and identify lessons learned about the research process itself.
In this module, you will learn how to conduct participatory evaluation with regard to the resilient seed systems research cycle. When should evaluation questions be defined? The key questions are: Who should be involved? What are the key aspects of the research that participatory evaluation questions can usefully address? What types of tools can be used for what purpose? Where can the tools be found? What are core tasks or steps in conducting an effective participatory evaluation? What factors can have an influence on the process and the results?
At the end of the module you will be able to prepare a comprehensive participatory evaluation plan.
Research evaluation is the analysis of the effectiveness and direction of a particular research activity or project and involves judging progress and outcomes. Participatory evaluation is a joint effort or a partnership between researchers and other stakeholders, such as farmers, government officials, or extension workers, to evaluate, systematically, the research carried out. By participatory we mean various types and degrees of involvement, control over, and decision-making in an activity or the whole research process.
Two important reasons for using a participatory process are to increase the relevance and effectiveness of the research to stakeholders and to contribute to empowerment and social transformation. Combining these two reasons, participation can be seen as both a means and an end to strengthen local people’s capacity to make decisions, shape their future adaptation choices, and enhance their ability to create an environment for change. Participatory evaluation can also increase the accountability of everyone involved in the research process because it is constructed as a collective effort to learn from successes and failures. Last but not least, participatory evaluation can help identify gaps in the research process and suggest how these gaps can be filled through possible follow-up activities.
Participatory evaluation has emerged because of a recognition of the limitations of conventional evaluation. Conventional evaluation mainly serves the needs of project implementers and donors and ignores the interests of other groups involved in research and development efforts, especially local people. Such evaluations are normally carried out by outside experts, with the result that a gap exists between the experts’ perception of the project and its results and that of the people who are directly involved.
In this module, we introduce the concept and practice of participatory evaluation. Using such an approach will allow you to do more than just write a report for the donor agency or the agency that supports your work technically; it will allow you to put the results to good use by others as well.
This part of the module will help you answer the following questions: When should evaluation questions be defined? Who should be involved? What are the key aspects of the research that participatory evaluation questions can usefully address?
Participatory evaluation emphasizes participation of the stakeholders in deciding how project progress should be measured and the results acted on. Broadening the involvement of the various stakeholders in identifying and analyzing change can create a clearer picture of what is really happening on the ground according to the perspectives of both women and men of various ages, classes, and ethnic groups in the community. It allows people involved in the research to share successes and learn from each other. At the same time, it is potentially empowering as it puts local people in charge, helps develop their skills, shows that their views count, and provides an opportunity for joint learning. Scientists can also use results from participatory evaluation to learn from and redesign their interventions (see Work Group for Community Health and Development 2014 listed in the Recommended reading for this module).
Well-designed participatory evaluation starts with evaluation questions, which are relevant and of practical use to the research team and others involved in the research process. They must also be effectively answered within a reasonable time frame and with the amount of resources available. To integrate participatory evaluation into the whole research cycle and plan adequately for its execution, one or more evaluation questions should be defined during the initial stages of the cycle. They may be based on the objectives of the research, which are established before research begins, then later refined or adjusted. In this way, the risk that participatory evaluation becomes a last-minute and, thus, largely futile activity can be avoided (Estrella et al. 2000, Vernooy 2005).
Useful evaluation questions usually address three aspects of the research:
*Note: Evaluation of the immediate research outputs in the form of novel varieties takes place in step 5 of the research cycle (Module 5. Field experimentation).
The specific evaluation questions should be introduced by the rationale (why carry out a participatory evaluation?) and address what will be evaluated and for whom. In the following steps in the preparation of the evaluation plan, the remaining questions of who will carry out, how to carry out, and when to carry out can then be dealt with. A sound strategy requires paying attention to the six key questions (Why? For whom? What? Who? How? When?).
 Vernooy, R., 2005 Participatory research and development for sustainable agriculture and natural resource management: a sourcebook Gonsalves, J., Becker, T., Braun, A., Campilan, D., de Chavez, H., Fajber, E., Kapiriri, M., Rivaca-Caminade, J., Vernooy, R. (editors), CIP-UPWARD, Los Baños, Philippines, and International Development Research Centre, Ottawa, Canada, pp. 104–112 This chapter provides practical guidance for the formulation of a participatory monitoring evaluation plan including evaluation questions. |
|
 Work Group for Community Health and Development, 2014 The community toolbox Participatory evaluation. Work Group for Community Health and Development, University of Kansas, Lawrence, Kansas, Chapter 36: Introduction to evaluation, Section 6: Participatory evaluation This is an excellent, concise overview of participatory evaluation covering rationale, theory, and practice. |
|
Estrella, M., 2000 Learning from change: issues and experiences in participatory monitoring and evaluation Blauert, J., Campilan, D., Gaventa, J., Gonsalves, J., Guijt, I., Johnson, D., Ricafort, R. (editors) , Intermediate Technology Press, London, UK, and International Development Research Centre, Ottawa, Canada One of the first books about the participatory approach, this book provides an overview of common themes and experiences in participatory monitoring and evaluation across different institutions and sectors as well as case studies. Chapter 1, Learning from change (pages 1–14), introduces key concepts and synthesizes four major steps for implementation. |
This part of the module will help you answer the following questions: What types of tools can be used for what purpose? Where can these tools be found?
To find answers to the evaluation questions, one or more tools must be selected. There is no blueprint for selecting tools. Usually, a mix of complementary tools will allow you to obtain the required information and provide some basis for triangulation or cross-checking of validity. Tools from conventional social science research, such as interviews and surveys, can be combined with tools from participatory research, such as participatory ranking and mapping.
In recent years, a large set of tools has been developed by researchers with various academic backgrounds, but a common interest in participatory evaluation. Among them are oral histories and testimonials, key informant interviews, brainstorming, focus groups, surveys, network analysis, various rating and ranking exercises, mapping, diagramming, community photography and video, theatre, and role playing (Gawler 2005).
The nature of the evaluation questions will be the best guide for identifying the most appropriate tool or tools (Community Sustainability Engagement Evaluation Toolbox 2010). For the three types of questions related to design and planning, implementation, and outcomes, it would be useful to combine tools that capture the individual perspectives of participants in the research as well as their collective perspective (e.g., how the community has benefited from the research in terms of improved crops and cropping practices; what the community has learned beyond the introduction of new crop diversity). It is important to keep the process doable while obtaining useful information. The readings below include lists of tools and how to use them. Numerous resources are available online.
 Community Sustainability Engagement Evaluation Toolbox, 2010 Tool Selector This list of monitoring and evaluation tools offers useful advice on the selection of tools according to type of data to collect (quantitative or qualitative) and category of outcome to measure (efficiency, effectiveness, and outcomes). |
|
 Participatory methods,Useful methods and ideas Institute of Development Studies, Brighton, UK This online document has a useful section on participatory methods. |
|
 Gawler, M., 2005 Useful tools for engaging young people in participatory evaluation UNICEF Regional Office for CEE/CIS and the Baltics, Geneva, Switzerland, 42 pg. This document presents clear and concise instructions for the use of 15 tools that can be used by young and not so young people alike. |
This part of the module will help you answer the following questions: What are the core tasks or steps in conducting an effective participatory evaluation? What factors can have an influence on the process and the results?
Careful planning of the evaluation process is as important as planning in the earlier stages of the research cycle, but, all too often, researchers only begin to think about evaluation once the research is nearly complete. At that point, energy and financial resources may have run out.
Participatory evaluation integrates the results of the previous steps, defining good questions and selecting appropriate tools, and provides answers to six key questions: why? for whom? what? who? how? and when? (Patton 2005).
If all goes well, answers to the evaluation questions will be found and put to good use. One way to do that is to synthesize them in the form of recommendations for further action to be undertaken by specific users. A sound recommendation includes not only the concrete action(s) to be undertaken, but also a feasible time frame and the actors who can take responsibility for the action(s). In Module 8, you will learn more about sharing the results of research.
The results generated by participatory research depend not only on asking sound questions, using good tools, and having a feasible plan, but also on the context in which the research takes place. This includes the socioeconomic and political situation; local culture; resource access and rights; social identities and relationships along the lines of gender, class, kinship, ethnicity, and age; and the attitudes, interests, and abilities of the various stakeholders including the researchers. In other words, as with all science, we need to be aware that knowledge is socially structured and that this implies a process of representation, discussion, and potential conflict and negotiation. For example, in countries with a strong government system from national to local levels, politics usually play a key role in the process of rural change. Participatory evaluation can lead to greater transparency and accountability, but it should be introduced and practised with prudence (Vernooy et al. 2003, 2006).
 Xu Jianchu, Sun Qiu, Vernooy, R. , 2006 The power of participatory monitoring and evaluation Development in Practice 16(5), A synthesis article on the Chinese experience. Not freely downloadable. |
|
 Patton, M.Q., 2005 Utilization-focused evaluation (U-FE) checklist A simple to follow checklist of 12 core tasks (or steps) and challenges for the facilitator of an evaluation. |
|
|
Patton, M.Q, 1997 Utilization-focused evaluation. The new century text. Sage Publications, Thousands Oaks, USA This classic book on utilization-focused evaluation is the source of the U-FE checklist |
|
 Vernooy, R., Sun Qiu, Xu Jianchu (editors), 2003 Voices for change: participatory monitoring and evaluation in China Yunnan Science and Technology Press, Kunming, China, and International Development Research Centre, Ottawa, Canada This book describes in detail how two Chinese research teams learned about and successfully integrated participatory monitoring and evaluation into their research projects in the field of natural resource management. The two case studies also explain the context of the research, the challenges faced, and how they were dealt with. Of special interest are chapters 3, 4, and 5 (pp. 55–147), which describe the field experiences of the two teams. |
Here is a quiz that will help you test your newly acquired knowledge. Once you have covered the content sections and completed the assigned readings, please answer the Participatory evaluation quiz.
Please, document this step of the research process by summarizing your participatory evaluation plan. Please specify:
The next module in our research process is Knowledge sharing and communication. Let us begin!
