Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance. Barbosa, T. M. S., Souza, R., Cruz, S. M. S., Campos, M. L. M., & Cottrell, L. In Proceedings of the 4th International Conference on Internet Applications, Protocols and Services. ![Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance [pdf]](https://bibbase.org/img/filetypes/pdf.svg "pdf")
Paper abstract bibtex Measuring the quality of Internet is essential to evaluate the performance of data links around the world and to keep track of how countries have improved their connections throughout the years. Moreover, Internet performance measurements provide understanding for network bottlenecks, trouble-shooting and even insights about the impact of major events such as tsunamis, fiber cuts or social upheavals. For this reason, since 1998, the PingER (Ping End-to-end Reporting) initiative at SLAC National Accelerator Laboratory monitors end-to-end performance of Internet links spread over 160 countries, providing a worldwide history of Internet performance. Data containing network measurements are daily collected from PingER Measurement Agents (MAs) and stored into flat files. As a result, PingER maintains a valuable fine- grained big dataset consisting of Internet performance data around the world. However, due to the large amounts of data, performing sophisticated joint analyses on those files may be so difficult that it becomes unfeasible in some scenarios. In this paper, we apply data warehousing techniques to transform the data on those flat files into structured data using a data model that facilitates complex analyses. We load the transformed data into a big distributed data warehouse that is able to perform complex analytical queries on large volumes of data in seconds. Finally, we show some data analyses correlating Internet performance data to hypothetical real-world scenarios.
@inproceedings{barbosa_applying_2016,
location = {Kuala Lumpur, Malaysia},
title = {Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance},
url = {https://renan-souza.github.io/files/NETAPPS2015.pdf},
abstract = {Measuring the quality of Internet is essential to evaluate the performance of data links around the world and to keep track of how countries have improved their connections throughout the years. Moreover, Internet performance measurements provide understanding for network bottlenecks, trouble-shooting and even insights about the impact of major events such as tsunamis, fiber cuts or social upheavals. For this reason, since 1998, the {PingER} (Ping End-to-end Reporting) initiative at {SLAC} National Accelerator Laboratory monitors end-to-end performance of Internet links spread over 160 countries, providing a worldwide history of Internet performance. Data containing network measurements are daily collected from {PingER} Measurement Agents ({MAs}) and stored into flat files. As a result, {PingER} maintains a valuable fine- grained big dataset consisting of Internet performance data around the world. However, due to the large amounts of data, performing sophisticated joint analyses on those files may be so difficult that it becomes unfeasible in some scenarios. In this paper, we apply data warehousing techniques to transform the data on those flat files into structured data using a data model that facilitates complex analyses. We load the transformed data into a big distributed data warehouse that is able to perform complex analytical queries on large volumes of data in seconds. Finally, we show some data analyses correlating Internet performance data to hypothetical real-world scenarios.},
booktitle = {Proceedings of the 4th International Conference on Internet Applications, Protocols and Services},
author = {Barbosa, T. M. S. and Souza, R. and Cruz, S. M. S. and Campos, M. L. M. and Cottrell, Les},
urldate = {2016-10-17},
date = {2016},
file = {NETAPPS2015_submission_52.pdf:/Users/rfsouza/Library/Application Support/Zotero/Profiles/33j3px8o.default/zotero/storage/QPEHH8ND/NETAPPS2015_submission_52.pdf:application/pdf}
}
Downloads: 0
{"_id":"N4uBKAtCGPiLQWTmq","bibbaseid":"barbosa-souza-cruz-campos-cottrell-applyingdatawarehousingandbigdatatechniquestoanalyzeinternetperformance","downloads":0,"creationDate":"2016-10-18T02:58:27.161Z","title":"Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance","author_short":["Barbosa, T. M. S.","Souza, R.","Cruz, S. M. S.","Campos, M. L. M.","Cottrell, L."],"year":null,"bibtype":"inproceedings","biburl":"http://raw.githubusercontent.com/renan-souza/renan-souza.github.io/master/Renans-Publications.bib","bibdata":{"bibtype":"inproceedings","type":"inproceedings","location":"Kuala Lumpur, Malaysia","title":"Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance","url":"https://renan-souza.github.io/files/NETAPPS2015.pdf","abstract":"Measuring the quality of Internet is essential to evaluate the performance of data links around the world and to keep track of how countries have improved their connections throughout the years. Moreover, Internet performance measurements provide understanding for network bottlenecks, trouble-shooting and even insights about the impact of major events such as tsunamis, fiber cuts or social upheavals. For this reason, since 1998, the PingER (Ping End-to-end Reporting) initiative at SLAC National Accelerator Laboratory monitors end-to-end performance of Internet links spread over 160 countries, providing a worldwide history of Internet performance. Data containing network measurements are daily collected from PingER Measurement Agents (MAs) and stored into flat files. As a result, PingER maintains a valuable fine- grained big dataset consisting of Internet performance data around the world. However, due to the large amounts of data, performing sophisticated joint analyses on those files may be so difficult that it becomes unfeasible in some scenarios. In this paper, we apply data warehousing techniques to transform the data on those flat files into structured data using a data model that facilitates complex analyses. We load the transformed data into a big distributed data warehouse that is able to perform complex analytical queries on large volumes of data in seconds. Finally, we show some data analyses correlating Internet performance data to hypothetical real-world scenarios.","booktitle":"Proceedings of the 4th International Conference on Internet Applications, Protocols and Services","author":[{"propositions":[],"lastnames":["Barbosa"],"firstnames":["T.","M.","S."],"suffixes":[]},{"propositions":[],"lastnames":["Souza"],"firstnames":["R."],"suffixes":[]},{"propositions":[],"lastnames":["Cruz"],"firstnames":["S.","M.","S."],"suffixes":[]},{"propositions":[],"lastnames":["Campos"],"firstnames":["M.","L.","M."],"suffixes":[]},{"propositions":[],"lastnames":["Cottrell"],"firstnames":["Les"],"suffixes":[]}],"urldate":"2016-10-17","date":"2016","file":"NETAPPS2015_submission_52.pdf:/Users/rfsouza/Library/Application Support/Zotero/Profiles/33j3px8o.default/zotero/storage/QPEHH8ND/NETAPPS2015_submission_52.pdf:application/pdf","bibtex":"@inproceedings{barbosa_applying_2016,\n\tlocation = {Kuala Lumpur, Malaysia},\n\ttitle = {Applying Data Warehousing and Big Data Techniques to Analyze Internet Performance},\n\turl = {https://renan-souza.github.io/files/NETAPPS2015.pdf},\n\tabstract = {Measuring the quality of Internet is essential to evaluate the performance of data links around the world and to keep track of how countries have improved their connections throughout the years. Moreover, Internet performance measurements provide understanding for network bottlenecks, trouble-shooting and even insights about the impact of major events such as tsunamis, fiber cuts or social upheavals. For this reason, since 1998, the {PingER} (Ping End-to-end Reporting) initiative at {SLAC} National Accelerator Laboratory monitors end-to-end performance of Internet links spread over 160 countries, providing a worldwide history of Internet performance. Data containing network measurements are daily collected from {PingER} Measurement Agents ({MAs}) and stored into flat files. As a result, {PingER} maintains a valuable fine- grained big dataset consisting of Internet performance data around the world. However, due to the large amounts of data, performing sophisticated joint analyses on those files may be so difficult that it becomes unfeasible in some scenarios. In this paper, we apply data warehousing techniques to transform the data on those flat files into structured data using a data model that facilitates complex analyses. We load the transformed data into a big distributed data warehouse that is able to perform complex analytical queries on large volumes of data in seconds. Finally, we show some data analyses correlating Internet performance data to hypothetical real-world scenarios.},\n\tbooktitle = {Proceedings of the 4th International Conference on Internet Applications, Protocols and Services},\n\tauthor = {Barbosa, T. M. S. and Souza, R. and Cruz, S. M. S. and Campos, M. L. M. and Cottrell, Les},\n\turldate = {2016-10-17},\n\tdate = {2016},\n\tfile = {NETAPPS2015_submission_52.pdf:/Users/rfsouza/Library/Application Support/Zotero/Profiles/33j3px8o.default/zotero/storage/QPEHH8ND/NETAPPS2015_submission_52.pdf:application/pdf}\n}\n\n","author_short":["Barbosa, T. M. S.","Souza, R.","Cruz, S. M. S.","Campos, M. L. M.","Cottrell, L."],"key":"barbosa_applying_2016","id":"barbosa_applying_2016","bibbaseid":"barbosa-souza-cruz-campos-cottrell-applyingdatawarehousingandbigdatatechniquestoanalyzeinternetperformance","role":"author","urls":{"Paper":"https://renan-souza.github.io/files/NETAPPS2015.pdf"},"downloads":0},"search_terms":["applying","data","warehousing","big","data","techniques","analyze","internet","performance","barbosa","souza","cruz","campos","cottrell"],"keywords":[],"authorIDs":[],"dataSources":["NetmjR9Zk4dDJod22"]}