A significant challenge facing comprehensive or collaborative community change efforts is to identify appropriate indicators to measure and track change. When there are multiple stakeholders, a variety of change efforts, and a dynamic and shifting complex problem, evaluation can fall to the bottom of the agenda. Approaches to Measuring Community Change Indicators provides a unique lens on four metrics - measuring less poverty in communities, more vibrant communities, more citizen engagement and more collaboration and highlights tools for effectively monitoring change.
On this page you’ll find:
Approaches to Measuring: Community Change Indicators
Several years ago when Tamarack was reviewing its mission and mandate we spoke of things we would like to be known for. The ideas we stood for and the outcomes we wanted to achieve. After much thought and consultation we came up with the following vision and mission. It was built around four key words:
- Less poverty
Tamarack exists to build vibrant and engaged communities in Canada. Our work will result in more collaborative approaches and less poverty.
A vibrant community is one where committed citizens work together to build a community that is caring, prosperous and sustainable.
Our mission is to engage citizens in inspired action as they work and learn together on behalf of their communities to create and realize bold visions for the future.
Once having agreed on the vision and mission of the organization our next challenge was a way to measure the outcomes of our work. Approaches to Measuring Community Change Indicators is a first step in trying to capture how others might have done that. Measuring outcomes is not foreign to us. In Vibrant Communities Canada we have gone to great lengths to measure specifically engagement and less poverty. Evaluation has always interested us not only as a means for monitoring our activities but also an important way to tell the story of our work.
Tamarack, in partnership with The Ontario Trillium Foundation, is pleased to bring you this resource and webpage. We hope it will increase understanding of community change indicators and be an important reference tool.
The Vibrant Communities Approach to Evaluation
Comprehensive community change efforts are not easy to evaluate. The Aspen Institute’s Roundtable on Community Change has reviewed dozens of examples over twenty years and has concluded that the breadth, scale and complexity of the efforts means traditional forms of evaluation are inadequate.
The Vibrant Communities network is experimenting with a new approach: developmental evaluation. At the local level, this means reflecting on the theory of change underlying a group’s work and upgrading it as required to better achieved desired outcomes, respond to a changing environment, and capture the emerging insights and questions of participants. At the national level, it is about mining the on-the-ground experience of communities for patterns and themes that help us understand the value of this approach to reducing poverty. So far, the experiment has been successful.
In his latest book on developmental evaluation, Michael Quinn Patton, a leading evaluation expert, commented on VC’s contribution to the emerging field of developmental evaluation: "Community-based developmental evaluation is hard to do and do well. Tamarack’s work with Vibrant Communities is the best I’ve seen." (Patton, Michael 2010. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use, Page 232)
Back to top.
Key Measurement Tools and Approaches
Ontario Deprivation Index
The Daily Bread Food Bank, in partnership with the Caledon Institute, began looking at a different way to measure poverty beyond levels of income, by asking a very large sample of people what they saw as necessary to have a standard of living above poverty, and secondly, what they have and do not have.
The deprivation index is not a list of basic necessities, but is based on prevailing social and economic conditions in a certain area. For example, ‘clean running water’ does not distinguish poor from non-poor households in Ontario, with some exceptions, though it might in developing countries. But being able to afford fresh fruits and vegetables every day could distinguish poor from non-poor households, even in a wealthy place such as Ontario.
Vital Signs is an annual check-up conducted by community foundations across Canada that measures the vitality of communities, identifies trends, and shares opportunities for action in at least ten areas critical to quality of life.
Canadian Index of Wellbeing
The Canadian Index of Wellbeing (CIW) is a new way of measuring wellbeing that goes beyond economic measures like GDP. It provides insights into the quality of life of Canadians – overall, and in specific areas that matter.
The CIW shines a spotlight on the interconnections among areas: for example, how changes in income and education are linked to changes in health. When the index is complete at the end of 2010, the CIW will measure eight interconnected areas of wellbeing including living standards: healthy populations: community vitality; education; democratic engagement; time use; arts, culture and recreation; and environment.
Often evaluation processes are static and focus on end of project results. Instead, Vibrant Communities is asking key stakeholders about the questions that should be asked, the format of the end product and how, as a participant in the evaluation process, they might use the evaluation results to inform their work. This process has increased engagement in the evaluation design and development and potentially in how end-users will engage with results.
Additional Links and Resources
Poverty Compendium: A Tool for Communities - This online audio seminar and paper identify 147 strategies and frameworks that groups have used to try and reduce poverty in their communities. Access the audio seminar here and the Poverty Compendium resource here.
Strategic Drivers for Comprehensive Approaches - This online audio seminar and paper discusses the use of strategic drivers to achieve a comprehensive approach to community change efforts. Access the audio seminar here and the Strategic Drivers resource here.
Comprehensive Strategies for Deep and Durable Outcomes - This online audio seminar and paper focus on six nested strategies for pursuing comprehensiveness in community change efforts. Access the audio seminar here and the Comprehensive Strategies paper here.
Poverty Reduction Strategies and Provincial Policies - Six provinces and three territorial governments are pursuing poverty strategies to reduce the impact on low-income citizens. In a series of on-line audio seminars, senior government officials from Newfoundland & Labrador, Nova Scotia, New Brunswick, Ontario and Manitoba discuss their poverty reduction priorities and progress to date. Sherri Torjman, Vice President, Caledon Institute of Social Policy completes the series by providing an overview of the progress and outcomes achieved. Access the online provincial poverty reduction strategies series here.
Back to top.