Paper Website abstract bibtex
Twitter, or the world of 140 characters poses serious challenges to the efficacy of topic models on short, messy text. While topic mod-els such as Latent Dirichlet Allocation (LDA) have a long history of successful application to news articles and academic abstracts, they are often less coherent when applied to microblog content like Twitter. In this paper, we investigate methods to improve topics learned from Twitter content without modifying the basic machin-ery of LDA; we achieve this through various pooling schemes that aggregate tweets in a data preprocessing step for LDA. We empir-ically establish that a novel method of tweet pooling by hashtags leads to a vast improvement in a variety of measures for topic co-herence across three diverse Twitter datasets in comparison to an unmodified LDA baseline and a variety of pooling schemes. An additional contribution of automatic hashtag labeling further im-proves on the hashtag pooling results for a subset of metrics. Over-all, these two novel schemes lead to significantly improved LDA topic models on Twitter content.