Incremental Sense Weight Training for In-depth Interpretation of Contextualized Word Embeddings

Incremental Sense Weight Training for In-depth Interpretation of Contextualized Word Embeddings. Jiang, X., Yang, Z., & Choi, J. D. In Proceedings of the 34th AAAI Conference on Artificial Intelligence: Student Abstract and Poster Program, of AAAI:SAP'20, pages 13823–13824, New York, NY, 2020.

Paper

Slides abstract bibtex

We present a novel online algorithm that learns the essence of each dimension in word embeddings. We first mask dimensions determined unessential by our algorithm, apply the masked word embeddings to a word sense disambiguation task (WSD), and compare its performance against the one achieved by the original embeddings. Our results show that the masked word embeddings do not hurt the performance and can improve it by 3%.

@inproceedings{jiang:20a,
	abstract = {We present a novel online algorithm that learns the essence of each dimension in word embeddings. We first mask dimensions determined unessential by our algorithm, apply the masked word embeddings to a word sense disambiguation task (WSD), and compare its performance against the one achieved by the original embeddings. Our results show that the masked word embeddings do not hurt the performance and can improve it by 3%.},
	address = {New York, NY},
	author = {Jiang, Xinyi and Yang, Zhengzhe and Choi, Jinho D.},
	booktitle = {Proceedings of the 34th AAAI Conference on Artificial Intelligence: Student Abstract and Poster Program},
	date-added = {2019-10-25 18:14:36 -0400},
	date-modified = {2020-07-20 18:22:06 -0400},
	keywords = {emorynlp,selected},
	pages = {13823--13824},
	series = {AAAI:SAP'20},
	title = {{Incremental Sense Weight Training for In-depth Interpretation of Contextualized Word Embeddings}},
	url = {https://aaai.org/Conferences/AAAI-20/},
	url_paper = {https://www.aaai.org/ojs/index.php/AAAI/article/view/7183},
	url_slides = {https://www.slideshare.net/jchoi7s/incremental-sense-weight-training-for-indepth-interpretation-of-contextualized-word-embeddings},
	year = {2020},
	Bdsk-Url-1 = {http://www.aaai.org/Conferences/AAAI/2016/aaai16accepted-papers.pdf}}

Downloads: 0