Application of Generalizability Theory to Concept Map Assessment Research. Yin, Y. & Shavelson, R. J. Applied Measurement in Education, 21(3):273–291, 2008.
doi  abstract   bibtex   
In the first part of this article, the use of Generalizability (G)\${\textbackslash}backslash\$ntheory in examining the dependability of concept map assessment scores\${\textbackslash}backslash\$nand designing a concept map assessment for a particular practical\${\textbackslash}backslash\$napplication is discussed. In the second part, the application of\${\textbackslash}backslash\$nG theory is demonstrated by comparing the technical qualities of\${\textbackslash}backslash\$ntwo frequently used mapping techniques: construct-a-map with created\${\textbackslash}backslash\$nlinking phrases (C) and construct-a-map with selected linking phrases\${\textbackslash}backslash\$n(S). Some mea- surement facets that influence concept-map scores\${\textbackslash}backslash\$nare explored and how to opti- mize different concept mapping techniques\${\textbackslash}backslash\$nby varying the conditions for different facets is shown. It is found\${\textbackslash}backslash\$nthat C and S are not technically equivalent. The G coefficients for\${\textbackslash}backslash\$nS are larger than those for C under the same condition. Further-\${\textbackslash}backslash\$nmore, a decision(D) study shows that fewer items (propositions) would\${\textbackslash}backslash\$nbe needed for S than C to reach desired level of G coefficients if\${\textbackslash}backslash\$nonly one occasion could be afforded. On the other hand, C seems to\${\textbackslash}backslash\$nreveal students' understanding about dif- ferent concepts than S\${\textbackslash}backslash\$nbetter. For practical purposes, one might prefer S because it is\${\textbackslash}backslash\$neasier to score and produces higher reliability. However, this efficiency\${\textbackslash}backslash\$ncomes at the cost of validity. We would trade off validity and reliability\${\textbackslash}backslash\$nfor effi- ciency by including more propositions in C map.
@article{yin_application_2008,
	title = {Application of {Generalizability} {Theory} to {Concept} {Map} {Assessment} {Research}},
	volume = {21},
	issn = {0895-7347},
	doi = {10.1080/08957340802161840},
	abstract = {In the first part of this article, the use of Generalizability (G)\${\textbackslash}backslash\$ntheory in examining the dependability of concept map assessment scores\${\textbackslash}backslash\$nand designing a concept map assessment for a particular practical\${\textbackslash}backslash\$napplication is discussed. In the second part, the application of\${\textbackslash}backslash\$nG theory is demonstrated by comparing the technical qualities of\${\textbackslash}backslash\$ntwo frequently used mapping techniques: construct-a-map with created\${\textbackslash}backslash\$nlinking phrases (C) and construct-a-map with selected linking phrases\${\textbackslash}backslash\$n(S). Some mea- surement facets that influence concept-map scores\${\textbackslash}backslash\$nare explored and how to opti- mize different concept mapping techniques\${\textbackslash}backslash\$nby varying the conditions for different facets is shown. It is found\${\textbackslash}backslash\$nthat C and S are not technically equivalent. The G coefficients for\${\textbackslash}backslash\$nS are larger than those for C under the same condition. Further-\${\textbackslash}backslash\$nmore, a decision(D) study shows that fewer items (propositions) would\${\textbackslash}backslash\$nbe needed for S than C to reach desired level of G coefficients if\${\textbackslash}backslash\$nonly one occasion could be afforded. On the other hand, C seems to\${\textbackslash}backslash\$nreveal students' understanding about dif- ferent concepts than S\${\textbackslash}backslash\$nbetter. For practical purposes, one might prefer S because it is\${\textbackslash}backslash\$neasier to score and produces higher reliability. However, this efficiency\${\textbackslash}backslash\$ncomes at the cost of validity. We would trade off validity and reliability\${\textbackslash}backslash\$nfor effi- ciency by including more propositions in C map.},
	number = {3},
	journal = {Applied Measurement in Education},
	author = {Yin, Yue and Shavelson, Richard J.},
	year = {2008},
	pages = {273--291}
}

Downloads: 0