Why do certain hops show high pings in my traceroute?

Post by **Edge100x** » Tue Aug 17, 2010 2:33 pm

Sometimes, you might run a tracert to a server and it will look like this, with high pings for a single hop:

 1   0.1 ms   0.1 ms   0.1 ms  router.internap-la.nfoservers.com [64.94.101.254]
 2   1.0 ms   0.3 ms   0.5 ms  border1.ge1-3.nuclearfallout-48.ext1.lax.pnap.net [63.251.209.181]
 3   0.6 ms   1.6 ms   0.5 ms  core2.po2-20g-bbnet2.lax.pnap.net [216.52.255.66]
 4   0.9 ms   0.3 ms   0.8 ms  sl-st21-la-9-2-2.sprintlink.net [144.232.154.205]
 5   1.7 ms   1.5 ms   1.6 ms  sl-crs1-ana-0-4-0-3.sprintlink.net [144.232.20.207]
 6  30.1 ms  29.9 ms  30.1 ms  sl-crs1-fw-0-0-0-2.sprintlink.net [144.232.20.65]
 7  45.9 ms  46.1 ms  46.3 ms  sl-crs1-atl-0-8-0-0.sprintlink.net [144.232.18.146]
 8  46.1 ms  45.7 ms  45.8 ms  sl-st21-atl-0-0-0.sprintlink.net [144.232.20.117]
 9  48.6 ms  48.6 ms  48.4 ms  208.30.205.130
10 148.5 ms 248.8 ms 198.8 ms  border10.tge3-1-bbnet1.acs.pnap.net [64.94.0.12]
11  48.9 ms  49.2 ms  48.4 ms  atlanta.nuclearfallout.net [64.94.238.5]

If you see this behavior, you should re-run the tracert several more times, to see if high pings also show at the endpoint. If you don't see high pings at the endpoint, there is not actually a problem. When using the service, your experience always matches what you see at the endpoint. Additionally, even if the destination does show a problem, it might not start with that hop, unless all hops between the two show the same behavior.

More specifically, you'll see situations like the one above whenever a router is processing a BGP update, which is quite common. BGP updates cause the core CPU usage on the router to spike, which leads to increased latencies for other processes running on it -- processes that include the one that handles self-generated ICMP TTL-expired replies. Normal routed traffic typically flows without core CPU involvement because it switches directly between line cards, and remains unaffected by this. Forwarded traffic is also prioritized high enough that it would not be affected by BGP updates with hardware that does involve the core CPU.

Further, sometimes routers rate-limit or drop their self-generated ICMP TTL-expired replies completely, as a general policy. In these cases, you might see hops that won't respond at all. Again, if later hops (and, critically, the endpoint) don't show the same problem, there is nothing to be concerned about.