Tamarack
FacebookTwitterRSSContactStaff Directory
About Tamarack Learn Events Join a Community
<
  Developmental Evaluation
 

In this session, Mark Cabaj speaks with Michael Quinn Patton about his new book on the topic of Developmental Evaluation – an approach that has proven particularly useful and effective at addressing the unique challenges of evaluating the real nature of community change work.

Learning Objectives:

  • To deepen appreciation for the role of evaluation and evaluation thinking
  • To explore the distinctions between various types of evaluation
  • To understand more about developmental evaluation and its use
  • To investigate concepts, tools and resources available to support working with developmental evaluation methods

On this page you'll find:

Meet the Thought Leaders

Michael Quinn Patton Michael Quinn Patton - Michael is an independent organizational development consultant who has worked with organizations and programs at the international, national, state, and local levels, and with philanthropic, not-for-profit, private sector, and government programs. He is a generalist whose aim is to improve human effectiveness and results.  His work spans a full range of programs including: leadership development, education, human services, the environment, public health, employment, agricultural extension, arts, criminal justice, poverty, transportation, diversity, managing for results, performance indicators, effective governance, and futuring.

Michael has a doctorate in Sociology from the University of Wisconsin and spent 18 years on the faculty of the University of Minnesota.  He is the only recipient of both the Alva and Gunner Myrdal Award from the Evaluation Research Society for "outstanding contributions to evaluation use and practice" and the Paul F. Lazarsfeld Award for lifetime contributions to evaluation theory from the American Evaluation Association. 

Michael has authored six evaluation books including a 4th edition of Utilization-Focused Evaluation and co-authored Getting to Maybe: How the World Is Changed which applies complexity theory to social innovation and presents developmental evaluation as an approach for evaluating innovations.  Michael’s latest book, Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use was released in June 2010.

Mark Cabaj

Mark Cabaj - Mark Cabaj is a founding Principal of Tamarack – An Institute for Community Engagement, an organization based in Waterloo, Ontario, focused on assisting people build strong communities through local action.

Mark joined Tamarack in 2002 and is currently the Executive Director of Vibrant Communities – a network of communities and national organizations that use collaborative, comprehensive approaches to substantially reduce poverty within Canada.

Back to top.

The Power of Evaluation Thinking

For some, the words “evaluation” or “evaluation thinking” generate negative images of something that we’d prefer to avoid.  However, fundamentally evaluation is about increasing the effectiveness of change efforts of all kinds.  Evaluation thinking involves a commitment to ask ourselves questions such as: are we doing what we wanted to be doing?  Are we accomplishing what we wanted to accomplish?  

While there is value in conducting evaluations to satisfy the needs and expectation of others, Michael believes that the highest form of accountability is to oneself – as someone involved in a change effort – to engage in considering whether or not your work is unfolding as you thought and hoped that it might.  Ultimately, evaluation is a leadership function that involves being systematic about making developmental changes to the implementation of a plan based on feedback.  Listen here as Michael offers his definition of evaluation and evaluation thinking.

Back to top.

Developmental Evaluation: A Definition

Developmental evaluation is outcomes-oriented in its focus and is particularly useful in situations where the outcomes are emergent and changing.  It emerged as a niche for use in situations of ongoing development.  It is particularly well-suited for helping to monitor the connections between short-term outcomes and longer-term social change efforts.  Programs that do not expect to create a standardized model, but rather anticipate the need to be continuously evolving are examples where Developmental Evaluation is particularly useful.  The emphasis in Developmental Evaluation is on documenting decisions and formalizing the learning and the knowledge-bases that drive decisions.  In this clip, Michael shares the story of how he coined the term developmental evaluation in response to a client’s frustration with the limitations of both formative and summative evaluation methods. 

Back to top.

An Overview of Evaluation Purposes & Types

Historically, there have been two primary types of evaluation: Formative Evaluation and Summative Evaluation.  Formative evaluation is used to help improve a program or policy.  Summative evaluation is used to assess the merit or worth of a program or policy in order to determine whether it should be sustained, discontinued or scaled up.  However, both formative and summative evaluations are quite rigid.  They are also most effective when work is finite or can be paused while an evaluation is conducted.  Listen here as Michael uses the metaphor of “making soup” to describe the distinctions between formative, summative and developmental evaluation.

Back to top.

The Value of Evaluation in Developmental Situations

Developmental situations are ones that require a willingness to move beyond making slight improvements and instead continually adapt or change in response to the current environment.  Using this “lens”, one can begin to identify many developmental situations where evaluative thinking could be extremely useful.  These include situations where people are innovating; people are trying out a new approach where no model exists; or, people do not anticipate their work ever arriving at a fixed model. 

Developmental Evaluation works at the intersection between “top-down” models of social change which emphasize generalized knowledge about how change should occur and “bottom up” models which engage people in generating their own solution to problems.  It adds value by helping to track the development of an innovation over time.  Rather than being something apart, Developmental Evaluation is embedded into the innovation process, tracking changes over time and bringing some regular “reality-testing” to the innovation process.  In the clip below, Michael summarizes Developmental Evaluation’s value-added role. 

Back to top.

The What & Why of “Patch Evaluation”

Evaluators are primarily trained to assume that all situations are simple and can be predicted at the outset.  However, in real life, programs can be operating on multiple tracks simultaneously. The term “Patch Evaluation” is used to describe these types of situations: where evaluative thinking is being used simultaneously in different ways according to the different stages or types of work that is being undertaken.  A critical beginning step in any large program or community effort is to analyze and determine which aspects of work are simple, which are complicated and which are complex and design an approach to evaluation that responds to these different needs.  Listen to the clip below as Michael describes this process.

Back to top.

Concepts & Tools for Working with Developmental Evaluation

One of the most commonly asked questions about developmental evaluation is: what methods or tools are best used when working in developmental situations?  Michael’s simple answer is, “You can use any evaluative method in a developmental context.”  Listen here as Michael elaborates on this point.

Back to top.

Reflection Questions

  1. How might a developmental evaluation approach enhance the effectiveness of your work?
  2. What would you anticipate to be the challenges and/or benefits of implementing a developmental evaluation approach?
  3. What are the important questions that you would need to answer in order to determine the most effective approach to evaluation within your program or change effort?

Find additional information to help guide your continued reflection on these and other important questions, in the Resources and Links below.

Back to top.

Links & Resources

Michael's new book, Developmental Evaluation, can be via ordered via the Amazon links below.

Amazon.ca (CDN)
Amazon.com (USA)

Amazon.co.uk
(UK)

  • Measuring Learning – In this 2007 Tamarack online audio seminar, Michael offers an explanation of Developmental Evaluation.
  • Learn more about Michael Quinn Patton.
  • A Conversation with Michael Quinn Patton – In this interview with the Harvard Family Research Project, Michael shares his thoughts on the emerging trends in evaluation practice.
  • In Conversation – In this interview with the International Development Research Centre, Michael discusses his approach for making sure evaluations are useful for decision-makers.
  • DE 201: A Practitioner's Guide to Developmental Evaluation – This just-released guide - developed for the McConnell Foundation - builds on the contribution of an earlier booklet by Jamie Gamble entitled A Developmental Evaluation Primer, published by the Foundation in 2008.
  • Eva the Evaluator – This children’s book is written by Roger Miranda – the recently retired director of evaluation at the United Nations.  It provides a clear and simple explanation of the role of an evaluator.

Back to top.

 

Vibrant Communities Podcast Subscription iTunes subscribers can click to subscribe to Vibrant Communities Podcast http://feeds.
feedburner
.com/Vibrant
Communities


Help Need Help?


Download full Audio from this Seminar

Developmental Evaluation

Run time 00:52:18

Sponsors:

The Ontario Trillium Foundation

Maytree