Transformers in Time Series: A Survey. Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., & Sun, L. May, 2023. arXiv:2202.07125 [cs, eess, stat]![Transformers in Time Series: A Survey [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.
@misc{wen_transformers_2023,
title = {Transformers in {Time} {Series}: {A} {Survey}},
shorttitle = {Transformers in {Time} {Series}},
url = {http://arxiv.org/abs/2202.07125},
doi = {10.48550/arXiv.2202.07125},
abstract = {Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.},
urldate = {2023-10-17},
publisher = {arXiv},
author = {Wen, Qingsong and Zhou, Tian and Zhang, Chaoli and Chen, Weiqi and Ma, Ziqing and Yan, Junchi and Sun, Liang},
month = may,
year = {2023},
note = {arXiv:2202.07125 [cs, eess, stat]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Electrical Engineering and Systems Science - Signal Processing, Statistics - Machine Learning},
}
Downloads: 0
{"_id":"aiKiofaQwZxR9e5XT","bibbaseid":"wen-zhou-zhang-chen-ma-yan-sun-transformersintimeseriesasurvey-2023","author_short":["Wen, Q.","Zhou, T.","Zhang, C.","Chen, W.","Ma, Z.","Yan, J.","Sun, L."],"bibdata":{"bibtype":"misc","type":"misc","title":"Transformers in Time Series: A Survey","shorttitle":"Transformers in Time Series","url":"http://arxiv.org/abs/2202.07125","doi":"10.48550/arXiv.2202.07125","abstract":"Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.","urldate":"2023-10-17","publisher":"arXiv","author":[{"propositions":[],"lastnames":["Wen"],"firstnames":["Qingsong"],"suffixes":[]},{"propositions":[],"lastnames":["Zhou"],"firstnames":["Tian"],"suffixes":[]},{"propositions":[],"lastnames":["Zhang"],"firstnames":["Chaoli"],"suffixes":[]},{"propositions":[],"lastnames":["Chen"],"firstnames":["Weiqi"],"suffixes":[]},{"propositions":[],"lastnames":["Ma"],"firstnames":["Ziqing"],"suffixes":[]},{"propositions":[],"lastnames":["Yan"],"firstnames":["Junchi"],"suffixes":[]},{"propositions":[],"lastnames":["Sun"],"firstnames":["Liang"],"suffixes":[]}],"month":"May","year":"2023","note":"arXiv:2202.07125 [cs, eess, stat]","keywords":"Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Electrical Engineering and Systems Science - Signal Processing, Statistics - Machine Learning","bibtex":"@misc{wen_transformers_2023,\n\ttitle = {Transformers in {Time} {Series}: {A} {Survey}},\n\tshorttitle = {Transformers in {Time} {Series}},\n\turl = {http://arxiv.org/abs/2202.07125},\n\tdoi = {10.48550/arXiv.2202.07125},\n\tabstract = {Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.},\n\turldate = {2023-10-17},\n\tpublisher = {arXiv},\n\tauthor = {Wen, Qingsong and Zhou, Tian and Zhang, Chaoli and Chen, Weiqi and Ma, Ziqing and Yan, Junchi and Sun, Liang},\n\tmonth = may,\n\tyear = {2023},\n\tnote = {arXiv:2202.07125 [cs, eess, stat]},\n\tkeywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Electrical Engineering and Systems Science - Signal Processing, Statistics - Machine Learning},\n}\n\n\n\n\n\n\n\n","author_short":["Wen, Q.","Zhou, T.","Zhang, C.","Chen, W.","Ma, Z.","Yan, J.","Sun, L."],"key":"wen_transformers_2023","id":"wen_transformers_2023","bibbaseid":"wen-zhou-zhang-chen-ma-yan-sun-transformersintimeseriesasurvey-2023","role":"author","urls":{"Paper":"http://arxiv.org/abs/2202.07125"},"keyword":["Computer Science - Artificial Intelligence","Computer Science - Machine Learning","Electrical Engineering and Systems Science - Signal Processing","Statistics - Machine Learning"],"metadata":{"authorlinks":{}},"html":""},"bibtype":"misc","biburl":"https://bibbase.org/zotero/mh_lenguyen","dataSources":["iwKepCrWBps7ojhDx"],"keywords":["computer science - artificial intelligence","computer science - machine learning","electrical engineering and systems science - signal processing","statistics - machine learning"],"search_terms":["transformers","time","series","survey","wen","zhou","zhang","chen","ma","yan","sun"],"title":"Transformers in Time Series: A Survey","year":2023}