Similarity to Molecules in the Training Set Is a Good Discriminator for Prediction Accuracy in QSAR. Sheridan, R., Feuston, B., Maiorov, V., & Kearsley, S. J.~Chem.~Inf.~Comput.~Sci., 44(6):1912--1928, 2004.
Similarity to Molecules in the Training Set Is a Good Discriminator for Prediction Accuracy in QSAR [link]Paper  doi  abstract   bibtex   
How well can a QSAR model predict the activity of a molecule not in the training set used to create the model? A set of retrospective cross-validation experiments using 20 diverse in-house activity sets were done to find a good discriminator of prediction accuracy as measured by root-mean-square difference between observed and predicted activity. Among the measures we tested, two seem useful: the similarity of the molecule to be predicted to the nearest molecule in the training set and/or the number of neighbors in the training set, where neighbors are those more similar than a user-chosen cutoff. The molecules with the highest similarity and/or the most neighbors are the best-predicted. This trend holds true for narrow training sets and, to a lesser degree, for many diverse training sets and does not depend on which QSAR method or descriptor is used. One may define the similarity using a different descriptor than that used for the QSAR model. The similarity dependence for diverse training sets is somewhat unexpected. It appears to be greater for those data sets where the association of similar activities vs similar structures (as encoded in the Patterson plot) is stronger. We propose a way to estimate the reliability of the prediction of an arbitrary chemical structure on a given QSAR model, given the training set from which the model was derived.
@article{Sheridan:2004aa,
	Abstract = {How well can a QSAR model predict the activity of a molecule not in
	the training set used to create the model? A set of retrospective
	cross-validation experiments using 20 diverse in-house activity sets
	were done to find a good discriminator of prediction accuracy as
	measured by root-mean-square difference between observed and predicted
	activity. Among the measures we tested, two seem useful: the similarity
	of the molecule to be predicted to the nearest molecule in the training
	set and/or the number of neighbors in the training set, where neighbors
	are those more similar than a user-chosen cutoff. The molecules with
	the highest similarity and/or the most neighbors are the best-predicted.
	This trend holds true for narrow training sets and, to a lesser degree,
	for many diverse training sets and does not depend on which QSAR
	method or descriptor is used. One may define the similarity using
	a different descriptor than that used for the QSAR model. The similarity
	dependence for diverse training sets is somewhat unexpected. It appears
	to be greater for those data sets where the association of similar
	activities vs similar structures (as encoded in the Patterson plot)
	is stronger. We propose a way to estimate the reliability of the
	prediction of an arbitrary chemical structure on a given QSAR model,
	given the training set from which the model was derived.},
	Author = {Sheridan, R.P. and Feuston, B.P. and Maiorov, V.N. and Kearsley, S.K.},
	Date-Added = {2007-12-11 17:01:03 -0500},
	Date-Modified = {2009-04-16 15:15:10 -0400},
	Doi = {10.1021/ci049782w},
	Journal = {J.~Chem.~Inf.~Comput.~Sci.},
	Keywords = {domain applicability; qsar},
	Number = {6},
	Owner = {rajarshi},
	Pages = {1912--1928},
	Pmid = {15554660},
	Timestamp = {2007.04.16},
	Title = {Similarity to Molecules in the Training Set Is a Good Discriminator for Prediction Accuracy in QSAR},
	Url = {http://dx.doi.org/10.1021/ci049782w},
	Volume = {44},
	Year = {2004},
	Bdsk-File-1 = {YnBsaXN0MDDUAQIDBAUIJidUJHRvcFgkb2JqZWN0c1gkdmVyc2lvblkkYXJjaGl2ZXLRBgdUcm9vdIABqAkKFRYXGyIjVSRudWxs0wsMDQ4RElpOUy5vYmplY3RzViRjbGFzc1dOUy5rZXlzog8QgASABoAHohMUgAKAA1lhbGlhc0RhdGFccmVsYXRpdmVQYXRo0hgMGRpXTlMuZGF0YU8RAWwAAAAAAWwAAgAAA212IAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMU5bQNIKwAAABCNbQxzaGVyaWRhbi5wZGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEI/kwl+gGwAAAAAAAAAAAAIAAwAACSAAAAAAAAAAAAAAAAAAAAAIYXJ0aWNsZXMAEAAIAADFObNTAAAAEQAIAADCX9hbAAAAAQAQABCNbQAKTIAACkxpAAB8EwACAC9tdiA6VXNlcnM6cmd1aGE6RG9jdW1lbnRzOmFydGljbGVzOnNoZXJpZGFuLnBkZgAADgAaAAwAcwBoAGUAcgBpAGQAYQBuAC4AcABkAGYADwAIAAMAbQB2ACAAEgArVXNlcnMvcmd1aGEvRG9jdW1lbnRzL2FydGljbGVzL3NoZXJpZGFuLnBkZgAAEwABLwAAFQACAAz//wAAgAXSHB0eH1gkY2xhc3Nlc1okY2xhc3NuYW1lox8gIV1OU011dGFibGVEYXRhVk5TRGF0YVhOU09iamVjdF8QJS4uLy4uL0RvY3VtZW50cy9hcnRpY2xlcy9zaGVyaWRhbi5wZGbSHB0kJaIlIVxOU0RpY3Rpb25hcnkSAAGGoF8QD05TS2V5ZWRBcmNoaXZlcgAIABEAFgAfACgAMgA1ADoAPABFAEsAUgBdAGQAbABvAHEAcwB1AHgAegB8AIYAkwCYAKACEAISAhcCIAIrAi8CPQJEAk0CdQJ6An0CigKPAAAAAAAAAgEAAAAAAAAAKAAAAAAAAAAAAAAAAAAAAqE=},
	Bdsk-Url-1 = {http://dx.doi.org/10.1021/ci049782w},
	Bdsk-Url-2 = {http://dx.doi.org/10.1021/ci049782w}}

Downloads: 0