Assessment of psychology competencies in field placements: Standardized vignettes reduce rater bias

Gonsalvez, CJ; Bushnell, J; Blackman, R; Deane, F; Bliokas, V; Nicholson-Perry, K; Shires, A; Nasstasia, Y; Allan, C; Knight, R

Assessment of psychology competencies in field placements: Standardized vignettes reduce rater bias

Gonsalvez, CJ Bushnell, J Blackman, R Deane, F Bliokas, V Nicholson-Perry, K Shires, A

Nasstasia, Y Allan, C Knight, R

Permalink

Publication Type:: Journal Article
Citation:: Training and Education in Professional Psychology, 2013, 7 (2), pp. 99 - 111
Issue Date:: 2013-08-13

Closed Access

	Filename	Description	Size
	ContentServer (98).pdf	Published Version	354.35 kB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Gonsalvez, CJ	en_US
dc.contributor.author	Bushnell, J	en_US
dc.contributor.author	Blackman, R	en_US
dc.contributor.author	Deane, F	en_US
dc.contributor.author	Bliokas, V	en_US
dc.contributor.author	Nicholson-Perry, K	en_US
dc.contributor.author	Shires, A https://orcid.org/0000-0001-8153-1126	en_US
dc.contributor.author	Nasstasia, Y	en_US
dc.contributor.author	Allan, C	en_US
dc.contributor.author	Knight, R	en_US
dc.date.issued	2013-08-13	en_US
dc.identifier.citation	Training and Education in Professional Psychology, 2013, 7 (2), pp. 99 - 111	en_US
dc.identifier.issn	1931-3918	en_US
dc.identifier.uri	http://hdl.handle.net/10453/116044
dc.description.abstract	Supervisors' ratings of psychology trainees' competence in field settings are a critical component of training assessment. There has been little systematic research regarding the validity of these assessments, but the available evidence suggests we have a problem! Supervisors' judgments may be affected by systemic biases that pose a serious threat to assessment credibility. The current study is part of a research collaboration among six universities that endeavors to develop and evaluate a new method-the use of vignettes-against outcomes derived from a conventional rating scale. Individual vignettes were designed and subjected to a rigorous process of peer-review and revisions, before final vignettes were assigned calibration scores by a group of experts. A catalogue of vignettes (n = 41) that represent various domains of competence across several developmental stages was compiled. University and field super-visors used the conventional rating scale and the vignette-matching procedure (VMP) to evaluate competencies at end-placement. Data from a pilot (n = 20) and a follow-up study (n = 57) suggest that compared with a conventional rating scale, the VMP reduced leniency and halo biases. The VMP has the potential to improve outcomes of competency assessments in field placements and merits further research and development. © eat to assessment credibility. The current study is part of a research collaboration among six universities that endeavors to develop and evaluate a new method-the use of vignettes-against outcomes derived from a conventional rating scale. Individual vignettes were designed and subjected to a rigorous process of peer2013 American Psychological Association.	en_US
dc.relation.ispartof	Training and Education in Professional Psychology	en_US
dc.relation.isbasedon	10.1037/a0031617	en_US
dc.subject.classification	Clinical Psychology	en_US
dc.title	Assessment of psychology competencies in field placements: Standardized vignettes reduce rater bias	en_US
dc.type	Journal Article
utslib.citation.volume	2	en_US
utslib.citation.volume	7	en_US
utslib.for	1701 Psychology	en_US
utslib.for	1302 Curriculum and Pedagogy	en_US
utslib.for	1303 Specialist Studies in Education	en_US
pubs.embargo.period	Not known	en_US
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Graduate School of Health
utslib.copyright.status	closed_access
pubs.issue	2	en_US
pubs.publication-status	Published	en_US
pubs.volume	7	en_US

Abstract:

Supervisors' ratings of psychology trainees' competence in field settings are a critical component of training assessment. There has been little systematic research regarding the validity of these assessments, but the available evidence suggests we have a problem! Supervisors' judgments may be affected by systemic biases that pose a serious threat to assessment credibility. The current study is part of a research collaboration among six universities that endeavors to develop and evaluate a new method-the use of vignettes-against outcomes derived from a conventional rating scale. Individual vignettes were designed and subjected to a rigorous process of peer-review and revisions, before final vignettes were assigned calibration scores by a group of experts. A catalogue of vignettes (n = 41) that represent various domains of competence across several developmental stages was compiled. University and field super-visors used the conventional rating scale and the vignette-matching procedure (VMP) to evaluate competencies at end-placement. Data from a pilot (n = 20) and a follow-up study (n = 57) suggest that compared with a conventional rating scale, the VMP reduced leniency and halo biases. The VMP has the potential to improve outcomes of competency assessments in field placements and merits further research and development. © eat to assessment credibility. The current study is part of a research collaboration among six universities that endeavors to develop and evaluate a new method-the use of vignettes-against outcomes derived from a conventional rating scale. Individual vignettes were designed and subjected to a rigorous process of peer2013 American Psychological Association.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/116044