Interrater reliability and agreement of subjective judgments. Tinsley, H. E. & Weiss, D. J. Journal of Counseling Psychology, 22(4):358–376, July, 1975.
Interrater reliability and agreement of subjective judgments. [link]Paper  doi  abstract   bibtex   
Indexes of interrater reliability and agreement are reviewed, and suggestions are made regarding their use in counseling psychology research. The distinction between agreement and reliability is clarified, and the relationships between these indexes and the level of measurement and type of replication are discussed. Indexes of interrater reliability appropriate for use with ordinal and interval scales are considered. The intraclass correlation as a measure of interrater reliability is discussed in terms of the treatment of between-raters variance and the appropriateness of reliability estimates based on composite or individual ratings. The advisability of optimal weighting schemes for calculating composite ratings is also considered. Measures of interrater agreement for ordinal and interval scales are described, as are measures of interrater agreement for data at the nominal level of measurement. (54 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
@article{tinsley_interrater_1975,
	title = {Interrater reliability and agreement of subjective judgments.},
	volume = {22},
	copyright = {© American Psychological Association 1975},
	issn = {0022-0167},
	url = {https://sremote.pitt.edu/psycarticles/docview/614268449/13E234C70206B816F03/,DanaInfo=search.proquest.com+18?accountid=14709},
	doi = {http://dx.doi.org/10.1037/h0076640},
	abstract = {Indexes of interrater reliability and agreement are reviewed, and suggestions are made regarding their use in counseling psychology research. The distinction between agreement and reliability is clarified, and the relationships between these indexes and the level of measurement and type of replication are discussed. Indexes of interrater reliability appropriate for use with ordinal and interval scales are considered. The intraclass correlation as a measure of interrater reliability is discussed in terms of the treatment of between-raters variance and the appropriateness of reliability estimates based on composite or individual ratings. The advisability of optimal weighting schemes for calculating composite ratings is also considered. Measures of interrater agreement for ordinal and interval scales are described, as are measures of interrater agreement for data at the nominal level of measurement. (54 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)},
	language = {English},
	number = {4},
	urldate = {2013-05-19},
	journal = {Journal of Counseling Psychology},
	author = {Tinsley, Howard E. and Weiss, David J.},
	month = jul,
	year = {1975},
	keywords = {Counseling Psychology (major), Experimental Methods (major), Rating (major), Rating Scales (major)},
	pages = {358--376},
}

Downloads: 0