Using Low-Rate Flow Periodicities for Anomaly Detection: Extended. Bartlett, G., Heidemann, J., & Papadopoulos, C. Technical Report ISI-TR-2009-661, USC/Information Sciences Institute, August, 2009.
Using Low-Rate Flow Periodicities for Anomaly Detection: Extended [link]Paper  abstract   bibtex   
As desktops and servers become more complicated, they employ an increasing amount of automatic, non-user initiated communication. Such communication can be good (OS updates, RSS feed readers, and mail polling), bad (keyloggers, spyware, and botnet command-and-control), or ugly (adware or unauthorized peer-to-peer applications). Communication in these applications is often periodic but infrequent, perhaps every few minutes to few hours. This infrequent communication and the complexity of today's systems makes these applications difficult for users to detect and diagnose. We show that there are several classes of applications that show low-rate periodicity and demonstrate that they are widely deployed on public networks. In this paper we present a new approach to identify changes in low-rate periodic network traffic. We employ signal-processing techniques, using discrete wavelets implemented as a fully decomposed, iterated filter bank. This approach allows us to cover a large range of low-rate periodicities, from seconds to hours, and to identify approximate times when traffic changed. Network administrators and users can use our techniques for network- or self-surveillance. To measure the effectiveness of our approach, we show that it can detect changes in periodic behavior caused by events such as installation of keyloggers, an interruption in OS update checks, or the P2P application BitTorrent. We quantify the sensitivity of our approach, showing that we can find periodic traffic when it is at least 5–10% of overall traffic.
@TechReport{Bartlett09a,
	author = "Genevieve Bartlett and John Heidemann and Christos Papadopoulos",
	title = "Using Low-Rate Flow Periodicities for Anomaly
                  Detection: Extended",
	institution = 	"USC/Information Sciences Institute",
	year = 		2009,
	sortdate = 		"2009-08-01", 
	number =	"ISI-TR-2009-661",
	month =		aug,
	keywords =	"low-rate periodic detection, wavelet, traffic",
	project = "ant, lander, madcat",
	jsubject = "spectral_network",
	jlocation =	"johnh: pafile",
	url =		"https://ant.isi.edu/%7ejohnh/PAPERS/Bartlett09a.html",
	pdfurl =		"https://ant.isi.edu/%7ejohnh/PAPERS/Bartlett09a.pdf",
	otherurl = "ftp://ftp.isi.edu/isi-pubs/tr-661.pdf",
	myorganization =	"USC/Information Sciences Institute",
	copyrightholder = "authors",
	abstract = "As desktops and servers become more complicated,
                  they employ an increasing amount of automatic,
                  non-user initiated communication.  Such
                  communication can be good (OS updates, RSS feed
                  readers, and mail polling), bad (keyloggers,
                  spyware, and botnet command-and-control), or ugly
                  (adware or unauthorized peer-to-peer applications).
                  Communication in these applications is often
                  periodic but infrequent, perhaps every few minutes
                  to few hours.  This infrequent communication and the
                  complexity of today's systems makes these
                  applications difficult for users to detect and
                  diagnose.  We show that there are several classes of
                  applications that show low-rate periodicity and
                  demonstrate that they are widely deployed on public
                  networks.  In this paper we present a new approach
                  to identify changes in low-rate periodic network
                  traffic.  We employ signal-processing techniques,
                  using discrete wavelets implemented as a fully
                  decomposed, iterated filter bank.  This approach
                  allows us to cover a large range of low-rate
                  periodicities, from seconds to hours, and to
                  identify approximate times when traffic changed.
                  Network administrators and users can use our
                  techniques for network- or self-surveillance.  To
                  measure the effectiveness of our approach, we show
                  that it can detect changes in periodic behavior
                  caused by events such as installation of keyloggers,
                  an interruption in OS update checks, or the P2P
                  application BitTorrent.  We quantify the sensitivity
                  of our approach, showing that we can find periodic
                  traffic when it is at least 5--10\% of overall
                  traffic.",
}

Downloads: 0