Quine’s semantic space: Count or predict semantic vectors from small in-domain data? [unpublished draft]. Bloem, J., Betti, A., Oortwijn, Y., Reynaert, M., & Ossenkoppele, T. In 2020.
abstract   bibtex   
We propose extrinsic evaluation as a solution to the problem of evaluating distributionalsemantic models trained on small, in-domain data sets, where standard datasets of wordsimilarity scores may not capture domain-specific meanings. By relying on a ground truthof text passages relevant to a research question in the history of ideas, we are able to testwhat model best retrieves those passages in an information retrieval task, quantifyingthe performance of different types of distributional semantic models in a challengingdomain. Our results show that count-based models outperform prediction-based modelsas the latter do not generalize well to queries they are not tuned on. The finding thatcount-based models generate better word embeddings from small data in the 1-2 milliontoken range also holds for document embeddings.
@inproceedings{bloem_quines_2020,
	title = {Quine’s semantic space: {Count} or predict semantic vectors from small in-domain data? [unpublished draft]},
	abstract = {We propose extrinsic evaluation as a solution to the problem of evaluating distributionalsemantic models trained on small, in-domain data sets, where standard datasets of wordsimilarity scores may not capture domain-specific meanings. By relying on a ground truthof text passages relevant to a research question in the history of ideas, we are able to testwhat model best retrieves those passages in an information retrieval task, quantifyingthe  performance  of  different  types  of  distributional  semantic  models  in  a  challengingdomain. Our results show that count-based models outperform prediction-based modelsas the latter do not generalize well to queries they are not tuned on. The finding thatcount-based models generate better word embeddings from small data in the 1-2 milliontoken range also holds for document embeddings.},
	author = {Bloem, Jelke and Betti, Arianna and Oortwijn, Yvette and Reynaert, Martin and Ossenkoppele, Thijs},
	year = {2020},
}

Downloads: 0