Central to the evaluation enterprise is the challenging practice of valuing, of making of empirically-supported judgments of quality about the program being evaluated. In evaluation, values guide the selection of evaluation purposes and audiences, the key evaluation questions, and the criteria by which the quality of the evaluand is judged. Guidance for the practice of valuing, however, remains muted in some evaluation theories, and muffled or muddied in others. Little is known about the processes and parameters that shape which particular values are chosen, purposefully or not, to anchor an evaluation study. Toward better understanding of how values are defensibly engaged in evaluation practice, this paper presents highlights from an empirical study of the evaluative reasoning of eight seasoned evaluators (mostly North-American), in particular, how and with what justifications values show up in their practice. Interviews and evaluation report reviews constituted the study data.
Evaluation research is research on evaluation itself as a practical and societal phenomenon. The field of evaluation is a body of a knowledge-producing practice that is characterized by its own logic, its own “imaginaries” (Schwandt, 2009), and its own problematics. It is also characterized by its own blind spots, its own dynamics, and its own discussions. One such discussion pertains to the ways in which values are present in evaluation practice.
The conversation on valuing was long marginalized due to a dominant values-neutral view that endorsed a fact-value dichotomy (House & Howe, 1999; Shadish, Cook, and Leviton, 1991). In current scholarly debate, however, the previous conviction that value judgments cannot be examined or justified rationally is a contested issue, and there exists considerable disagreement regarding how evaluators should attend to values and make value judgments (Julnes, 2012; Scriven, 2006, 2012; House, 2015, Stake, 2004). This disagreement has accelerated partly with the increasing scholarly recognition that the methodologies employed to enact evaluation practices themselves advance values, purposefully or not, as they, among other things, incorporate philosophical assumptions about what counts as legitimate knowledge (Greene, 1997, 2002). In addition, diverse views on who constitutes legitimate audiences for evaluation also remain, thus engaging debate about the political and moral dimensions of evaluation practice (Schwandt, 2003, 2009). Hence, evaluators’ work can be understood as neither scientifically nor politically neutral, but inherently values-committed (Greene, 2001).
Drawing on Greene’s (1997, 2006; Hall, Greene and Ahn, 2012) view of what it means to be “values-engaged,” we ask: how do values show up in evaluation practice? Our study of values in evaluation involved interviews and evaluation report reviews with eight seasoned evaluators. Our results illuminated features of both descriptive and prescriptive valuing, alongside the strategies employed by our respondents to make the valuing strands of evaluation more explicit, and thereby more contestable. We share selected findings from this study in the form of case vignettes that illustrate how values show up in evaluation, how they proceed along different pathways, and how they are present in evaluation results and the uses made of these results. The varied contexts of our evaluator sample add culture and custom to these values portraits. We position this study within the ongoing research on the relationship between evaluation theory and practice and address, in particular, the fundamental questions of purpose and role of the evaluator. We direct attention to the practices of evaluation and the built-in assumptions that bear, among other things, different value commitments regarding the purpose and location of our work in society, and the methods and procedures that enable enactment of these commitments.
Greene, J. C. (1997). Evaluation as advocacy. American Journal of Evaluation, 18(1).
Greene, J. (2001). Evaluation extrapolations. American Journal of Evaluation, 22(3): 397-402.
Greene, J. C. (2002). With a Splash of Soda, Please: Towards Active Engagement with Difference. Evaluation, Vol 8(2): 259–266
Greene, J. C. (2006). Making the World a Better Place Through Evaluation. In M. C. Alkin (Ed.), Evaluation Roots: A Wider Perspective of Theorists' Views and Influences. Thousand Oaks, CA: SAGE Publications, Inc.
Greene, J.C. (2012). Values-engaged evaluation. In Marco Segone (ed.), Evaluation for equitable development results (pp. 192-206). New York: UNICEF.
Hall, J.N., Greene, J.C., & Ahn, J. (2012). Values-engagement in evaluation: Ideas, implications, and illlustrations. American Journal of Evaluation, 33(2), 195-207.
House, E., & Howe, K. (1999). Values in Evaluation and Social Research. Thousand Oaks, CA: Sage Publications, Inc.
House, E.R. (2015). Evaluating values, biases, and practical wisdom. Charlotte, NC: IAP.
Julnes, G. (ed). (2012). Promoting valuation in the public interest: Informing policies for judging value in evaluation, New Directions in Evaluation, (133), 3-15.
Schwandt, T. A. (2003). In Search of the Political Morality of Evaluation Practice. Studies in Educational Policy and Educational Philosophy, 1-6.
Schwandt, T. A. (2009) ’Globalization influences on the western evaluation imaginary’, in K. E. Ryan and J. B. Cousins (eds), The Sage International Handbook of Educational Evaluation. Thousand Oaks, CA: Sage.
Scriven, M. (2006). Conceptual Revolutions in Evaluation: Past, Present, and Future In M. C. Alkin (Ed.), Evaluation Roots: A Wider Perspective of Theorists' Views and Influences. Thousand Oaks, CA: SAGE Publications, Inc.
Scriven, M. (2012). The Logic of Valuing. New Directions for Evaluation, 133, 17-28.
Shadish, W.R., Cook, T.D., & Leviton, L. (1991). Foundations of program evaluation, theories of practice. Thousand Oaks, CA: Sage.
Stake, R.E. (2004). How far dare an evaluator go toward saving the world? American Journal of Evaluation, 25 (1), 103-107.
American Evaluation Association (AEA). Track: Research on Evaluation. Atlanta, Georgia, United States, October 24-29, 2016.