@TechReport{Guo17a,
	author = 	"Hang Guo and John Heidemann",
	title = 	"Detecting {ICMP} Rate Limiting in the {Internet} (extended)",
	institution = 	"USC/Information Sciences Institute",
	year = 		2017,
	sortdate = 		"2017-05-01", 
	project = "ant, retrofuturebridge, lacrend",
	jsubject = "topology_modeling",
	jlocation = 	"johnh: pafile",
	number = 	"ISI-TR-717",
	month = 	may,
	jlocation = 	"johnh: pafile",
	keywords = 	"icmp, rate limiting",
	url =		"https://ant.isi.edu/%7ejohnh/PAPERS/Guo17a.html",
	pdfurl =	"https://ant.isi.edu/%7ejohnh/PAPERS/Guo17a.pdf",
	abstact = "
Active probing with ICMP is the center of
many network measurements, with tools like ping, traceroute, 
and their derivatives used to map topologies and as
a precursor for security scanning. However, rate limiting
of ICMP traffic has long been a concern, since undetected
rate limiting to ICMP could distort measurements, silently
creating false conclusions. To settle this concern, we look
systematically for ICMP rate limiting in the Internet. We
develop a model for how rate limiting affects probing,
validate it through controlled testbed experiments, and
create FADER, a new algorithm that can identify rate
limiting from user-side traces with minimal requirements
for new measurement traffic. We validate the accuracy
of FADER with many different network configurations in
testbed experiments and show that it almost always detects
rate limiting. Accuracy is perfect when measurement
probing ranges from 0 to 60x the rate limit, and almost
perfect (95\%) with up to 20\% packet loss. The worst
case for detection is when probing is very fast and blocks
are very sparse, but even there accuracy remains good
(measurements 60x the rate limit of a 10\% responsive
block is correct 65\% of the time). With this confidence,
we apply our algorithm to a random sample of whole
Internet, showing that rate limiting exists but that for slow
probing rates, rate-limiting is very, very rare. For our random
sample of 40,493 /24 blocks (about 2\% of the responsive
space), we confirm 6 blocks (0.02\%!) see rate limiting at
0.39 packets/s per block. We look at higher rates in public
datasets and suggest that fall-off in responses as rates
approach 1 packet/s per /24 block (14M packets/s from
the prober to the whole Internet), is consistent with rate
limiting. We also show that even very slow probing (0.0001
packet/s) can encounter rate limiting of NACKs that are
concentrated at a single router near the prober.
",
}

{"_id":"oH9CD56BZaAxHu7Hp","bibbaseid":"guo-heidemann-detectingicmpratelimitingintheinternetextended-2017","author_short":["Guo, H.","Heidemann, J."],"bibdata":{"bibtype":"techreport","type":"techreport","author":[{"firstnames":["Hang"],"propositions":[],"lastnames":["Guo"],"suffixes":[]},{"firstnames":["John"],"propositions":[],"lastnames":["Heidemann"],"suffixes":[]}],"title":"Detecting ICMP Rate Limiting in the Internet (extended)","institution":"USC/Information Sciences Institute","year":"2017","sortdate":"2017-05-01","project":"ant, retrofuturebridge, lacrend","jsubject":"topology_modeling","jlocation":"johnh: pafile","number":"ISI-TR-717","month":"May","keywords":"icmp, rate limiting","url":"https://ant.isi.edu/%7ejohnh/PAPERS/Guo17a.html","pdfurl":"https://ant.isi.edu/%7ejohnh/PAPERS/Guo17a.pdf","abstact":"Active probing with ICMP is the center of many network measurements, with tools like ping, traceroute, and their derivatives used to map topologies and as a precursor for security scanning. However, rate limiting of ICMP traffic has long been a concern, since undetected rate limiting to ICMP could distort measurements, silently creating false conclusions. To settle this concern, we look systematically for ICMP rate limiting in the Internet. We develop a model for how rate limiting affects probing, validate it through controlled testbed experiments, and create FADER, a new algorithm that can identify rate limiting from user-side traces with minimal requirements for new measurement traffic. We validate the accuracy of FADER with many different network configurations in testbed experiments and show that it almost always detects rate limiting. Accuracy is perfect when measurement probing ranges from 0 to 60x the rate limit, and almost perfect (95%) with up to 20% packet loss. The worst case for detection is when probing is very fast and blocks are very sparse, but even there accuracy remains good (measurements 60x the rate limit of a 10% responsive block is correct 65% of the time). With this confidence, we apply our algorithm to a random sample of whole Internet, showing that rate limiting exists but that for slow probing rates, rate-limiting is very, very rare. For our random sample of 40,493 /24 blocks (about 2% of the responsive space), we confirm 6 blocks (0.02%!) see rate limiting at 0.39 packets/s per block. We look at higher rates in public datasets and suggest that fall-off in responses as rates approach 1 packet/s per /24 block (14M packets/s from the prober to the whole Internet), is consistent with rate limiting. We also show that even very slow probing (0.0001 packet/s) can encounter rate limiting of NACKs that are concentrated at a single router near the prober. ","bibtex":"@TechReport{Guo17a,\n\tauthor = \t\"Hang Guo and John Heidemann\",\n\ttitle = \t\"Detecting {ICMP} Rate Limiting in the {Internet} (extended)\",\n\tinstitution = \t\"USC/Information Sciences Institute\",\n\tyear = \t\t2017,\n\tsortdate = \t\t\"2017-05-01\", \n\tproject = \"ant, retrofuturebridge, lacrend\",\n\tjsubject = \"topology_modeling\",\n\tjlocation = \t\"johnh: pafile\",\n\tnumber = \t\"ISI-TR-717\",\n\tmonth = \tmay,\n\tjlocation = \t\"johnh: pafile\",\n\tkeywords = \t\"icmp, rate limiting\",\n\turl =\t\t\"https://ant.isi.edu/%7ejohnh/PAPERS/Guo17a.html\",\n\tpdfurl =\t\"https://ant.isi.edu/%7ejohnh/PAPERS/Guo17a.pdf\",\n\tabstact = \"\nActive probing with ICMP is the center of\nmany network measurements, with tools like ping, traceroute, \nand their derivatives used to map topologies and as\na precursor for security scanning. However, rate limiting\nof ICMP traffic has long been a concern, since undetected\nrate limiting to ICMP could distort measurements, silently\ncreating false conclusions. To settle this concern, we look\nsystematically for ICMP rate limiting in the Internet. We\ndevelop a model for how rate limiting affects probing,\nvalidate it through controlled testbed experiments, and\ncreate FADER, a new algorithm that can identify rate\nlimiting from user-side traces with minimal requirements\nfor new measurement traffic. We validate the accuracy\nof FADER with many different network configurations in\ntestbed experiments and show that it almost always detects\nrate limiting. Accuracy is perfect when measurement\nprobing ranges from 0 to 60x the rate limit, and almost\nperfect (95\\%) with up to 20\\% packet loss. The worst\ncase for detection is when probing is very fast and blocks\nare very sparse, but even there accuracy remains good\n(measurements 60x the rate limit of a 10\\% responsive\nblock is correct 65\\% of the time). With this confidence,\nwe apply our algorithm to a random sample of whole\nInternet, showing that rate limiting exists but that for slow\nprobing rates, rate-limiting is very, very rare. For our random\nsample of 40,493 /24 blocks (about 2\\% of the responsive\nspace), we confirm 6 blocks (0.02\\%!) see rate limiting at\n0.39 packets/s per block. We look at higher rates in public\ndatasets and suggest that fall-off in responses as rates\napproach 1 packet/s per /24 block (14M packets/s from\nthe prober to the whole Internet), is consistent with rate\nlimiting. We also show that even very slow probing (0.0001\npacket/s) can encounter rate limiting of NACKs that are\nconcentrated at a single router near the prober.\n\",\n}\n\n","author_short":["Guo, H.","Heidemann, J."],"bibbaseid":"guo-heidemann-detectingicmpratelimitingintheinternetextended-2017","role":"author","urls":{"Paper":"https://ant.isi.edu/%7ejohnh/PAPERS/Guo17a.html"},"keyword":["icmp","rate limiting"],"metadata":{"authorlinks":{}}},"bibtype":"techreport","biburl":"https://bibbase.org/f/dHevizJoWEhWowz8q/johnh-2023-2.bib","dataSources":["YLyu3mj3xsBeoqiHK","fLZcDgNSoSuatv6aX","fxEParwu2ZfurScPY","7nuQvtHTqKrLmgu99"],"keywords":["icmp","rate limiting"],"search_terms":["detecting","icmp","rate","limiting","internet","extended","guo","heidemann"],"title":"Detecting ICMP Rate Limiting in the Internet (extended)","year":2017}