Meaningful Evaluation: a holistic and systemic approach to understanding and assessing outcomes

Zappala, G

Meaningful Evaluation: a holistic and systemic approach to understanding and assessing outcomes

Zappala, G

Permalink

Publisher:: University of Technology, Sydney (UTS)
Publication Type:: Journal Article
Citation:: Cosmopolitan Civil Societies: An Interdisciplinary Journal, 2020, 12, (2-3)
Issue Date:: 2020-11-25

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download full textAdobe PDF (2.36 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Zappala, G
dc.date.accessioned	2021-01-19T01:32:05Z
dc.date.available	2021-01-19T01:32:05Z
dc.date.issued	2020-11-25
dc.identifier.citation	Cosmopolitan Civil Societies: An Interdisciplinary Journal, 2020, 12, (2-3)
dc.identifier.issn	1837-5391
dc.identifier.issn	1837-5391
dc.identifier.uri	http://hdl.handle.net/10453/145476
dc.description.abstract	<jats:p>The concept and practice of Social Impact and Social Impact assessment has developed and matured over the last decade. Despite this growth, confusion still exists with respect to definitions of social impact as well as which tools and frameworks are most appropriate. Central to the concept and practice of social impact is a focus on outcomes. This paper argues that a discernible albeit problematic pattern and approach has emerged in outcomes measurement within the social impact field. After briefly reviewing some of the key problems with how outcomes measurement is practised, it presents some recent approaches from the evaluation field that attempt to address some of these concerns before introducing a new approach to understanding and evaluating outcomes – Meaningful Evaluation (ME). A pilot Meaningful Evaluation of volunteer ethics teachers in the Primary Ethics program in NSW is used to illustrate the approach. ME combines the Map of Meaning (MoM) with insights from next generation evaluation approaches to understand and assess outcomes. Informed by appreciative inquiry perspectives, it provides a means to bridge the divide between positivist and interpretivist approaches in evaluation. Key to ME is the assumption that it is more likely that immediate outcomes lead to medium and long-term outcomes (changes in behaviour) that are sustainable and lead to impact if participants experience program interventions as ‘meaningful’. Meaning is an important internal outcome that is essential if longer-term external outcomes are to occur. It also shifts the focus to capture unintended outcomes, key to developing holistic and systemic rather than linear and mechanistic Theories of Change. </jats:p>
dc.language	en
dc.publisher	University of Technology, Sydney (UTS)
dc.relation.ispartof	Cosmopolitan Civil Societies: An Interdisciplinary Journal
dc.relation.isbasedon	10.5130/ccs.v12.i2-3.7034
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	15 Commerce, Management, Tourism and Services, 16 Studies in Human Society
dc.title	Meaningful Evaluation: a holistic and systemic approach to understanding and assessing outcomes
dc.type	Journal Article
utslib.citation.volume	12
utslib.for	15 Commerce, Management, Tourism and Services
utslib.for	16 Studies in Human Society
pubs.organisational-group	/University of Technology Sydney/Provost
pubs.organisational-group	/University of Technology Sydney
utslib.copyright.status	open_access	*
dc.date.updated	2021-01-19T01:31:37Z
pubs.issue	2-3
pubs.publication-status	Published online
pubs.volume	12
utslib.citation.issue	2-3

Abstract:

The concept and practice of Social Impact and Social Impact assessment has developed and matured over the last decade. Despite this growth, confusion still exists with respect to definitions of social impact as well as which tools and frameworks are most appropriate. Central to the concept and practice of social impact is a focus on outcomes. This paper argues that a discernible albeit problematic pattern and approach has emerged in outcomes measurement within the social impact field. After briefly reviewing some of the key problems with how outcomes measurement is practised, it presents some recent approaches from the evaluation field that attempt to address some of these concerns before introducing a new approach to understanding and evaluating outcomes – Meaningful Evaluation (ME). A pilot Meaningful Evaluation of volunteer ethics teachers in the Primary Ethics program in NSW is used to illustrate the approach. ME combines the Map of Meaning (MoM) with insights from next generation evaluation approaches to understand and assess outcomes. Informed by appreciative inquiry perspectives, it provides a means to bridge the divide between positivist and interpretivist approaches in evaluation. Key to ME is the assumption that it is more likely that immediate outcomes lead to medium and long-term outcomes (changes in behaviour) that are sustainable and lead to impact if participants experience program interventions as ‘meaningful’. Meaning is an important internal outcome that is essential if longer-term external outcomes are to occur. It also shifts the focus to capture unintended outcomes, key to developing holistic and systemic rather than linear and mechanistic Theories of Change.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/145476