BERT Has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model. Wang, A. & Cho, K.
BERT Has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model [link]Paper  abstract   bibtex   
We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality.
@article{wangBERTHasMouth2019,
  archivePrefix = {arXiv},
  eprinttype = {arxiv},
  eprint = {1902.04094},
  primaryClass = {cs},
  title = {{{BERT}} Has a {{Mouth}}, and {{It Must Speak}}: {{BERT}} as a {{Markov Random Field Language Model}}},
  url = {http://arxiv.org/abs/1902.04094},
  shorttitle = {{{BERT}} Has a {{Mouth}}, and {{It Must Speak}}},
  abstract = {We show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality.},
  urldate = {2019-04-11},
  date = {2019-02-11},
  keywords = {Computer Science - Computation and Language,Computer Science - Machine Learning},
  author = {Wang, Alex and Cho, Kyunghyun},
  file = {/home/dimitri/Nextcloud/Zotero/storage/S8QLHLQN/Wang and Cho - 2019 - BERT has a Mouth, and It Must Speak BERT as a Mar.pdf;/home/dimitri/Nextcloud/Zotero/storage/7JNBY733/1902.html}
}

Downloads: 0