Seattle Comcast routing problem

Use this forum if you have a relatively general question or comment about a game, Ventrilo, TeamSpeak, or Murmur/Mumble server with us. If you have a server-specific question that might not help out the community, please directly contact us through your control panel instead.
sprtn
New to forums
New to forums
Posts: 4
https://www.youtube.com/channel/UC40BgXanDqOYoVCYFDSTfHA
Joined: Sat Apr 14, 2007 9:23 am

Seattle Comcast routing problem

Post by sprtn »

Hey it seems all seattle comcast users are getting routed through california to all internap seattle servers. here's a tracert

Tracing route to c-208-146-45-249.premium-seattle.nuclearfallout.net[208.146.45.249] over a maximum of 30 hops:

1 2 ms <1 ms <1 ms 192.168.1.1
2 * * * Request timed out.
3 9 ms * 9 ms GE-1-3-ur01.seattle.wa.seattle.comcast.net [68.86.99.33]
4 9 ms * 10 ms te-9-3-ur02.seattle.wa.seattle.comcast.net [68.86.96.98]
5 * 9 ms * te-9-1-ar01.seattle.wa.seattle.comcast.net [68.86.96.102]
6 9 ms 9 ms 9 ms 68.86.96.174
7 10 ms 35 ms 11 ms 68.86.90.217
8 26 ms 25 ms 26 ms te-0-0-0-0-cr01.sacramento.ca.ibone.comcast.net [68.86.84.38]
9 * * 29 ms 68.86.85.78
10 29 ms 27 ms 28 ms TenGigabitEthernet3-1.ar1.snv2.gblx.net [64.214.174.109]
11 41 ms 42 ms 45 ms InterNAP-Ken-Schmid.ge-3-0-0.402.ar2.SEA1.gblx.net [208.49.181.130]
12 50 ms 48 ms 45 ms border26s.ge1-1-bbnet1.sea.pnap.net [206.253.192.163]
13 47 ms 41 ms 41 ms nuclearfallout-5.border26s.sea.pnap.net [206.191.144.46]
14 33 ms 31 ms 30 ms c-208-146-45-249.premium-seattle.nuclearfallout.net [208.146.45.249]

Trace complete.

The problem is probably on comcast's end, i've contacted them and they don't seem to want to do anything about it. There anything you guys can do? or do you have any suggestions?
User avatar
To'mas
Founder
Founder
Posts: 212
Joined: Fri Apr 19, 2002 11:35 am
Location: Los Angeles
Contact:

Post by To'mas »

We've activated a Fiber connection with mzima to try to help this but it didn't. We are trying to get mzima to advertise our netblock directly to bypass the problem with comcast.
Also, we've been selected to be connected to the SIX in Seattle witch peers with comcast directly in another attempt to fix this problem.
Stay tuned for further developments.
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Post by Edge100x »

Aye. This problem has actually existed for several weeks now. It's entirely on Comcast's end, and we have tried our best to contact them to discuss it, without any luck. It's difficult to see why they would be doing this.
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Post by Edge100x »

I posted this today to the Events log for all Seattle servers.
We have been troubleshooting higher-than-normal latencies to Comcast customers in the Seattle area from our Seattle locations for some time now. Since the beginning, the issue has been on Comcast's side; our outbound traces stay in Seattle, while their inbound traces go down through California. The result is that pings are about 20ms higher than they should be. We have attempted to remedy this through numerous adjustments on our end -- installing a router, adding additional transit and peering, changing our BGP settings, etc, with no luck.

Recently, we were finally able to get ahold of a couple of backbone engineers at Comcast, who let us know what the real problem was: Comcast is arbitrarily sending our traffic through California as part of widespread changes they are making to their nationwide network. They were worried about overloading their connection to AT&T in Seattle, so they shifted a chunk of traffic down through their California network segment, and we happen to be in that chunk.

The engineers also told me that within 2-4 weeks, Comcast will be adding a Level3 peer in Seattle, at which point our latencies would return to normal. But, we want a solution now, so we've gone out of our way to find one.

Today, we were assigned a new IP block from one of our upstreams, Mzima. This new IP block does not exhibit the problem from Comcast customers because it is not assigned by InterNAP, as our other blocks are. We have modified our systems to allow us to shift any customers that need lower Comcast latencies over to the new block temporarily, while we wait for Comcast to perform its network upgrades.

If you want to be moved to one of the new Comcast-friendly IPs, please send us an email ASAP (our Live chat staff cannot perform these moves). We do not recommend that public servers make this change, as the new IP would only be temporarily, and public servers suffer greatly from IP changes (due to clients losing track of the server).

We are also continuing to communicate with Comcast and attempting to get them to modify their network's routing behavior. If we succeed, the routing to all of our IPs may return to normal for Comcast customers.
sprtn
New to forums
New to forums
Posts: 4
Joined: Sat Apr 14, 2007 9:23 am

Post by sprtn »

Thanks for the info, boys.
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Post by Edge100x »

Comcast added Level3 peering in Seattle today and the pings to most of our servers from Comcast customers are back to normal. We are still working on some lingering issues and customers will see an event in the log and get an email with more complete information.
sprtn
New to forums
New to forums
Posts: 4
Joined: Sat Apr 14, 2007 9:23 am

Post by sprtn »

Just following up. The same issue still exists for me -- several months later.
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Post by Edge100x »

I'm a Comcast customer in Seattle myself and I check this on a daily basis, so it's definitely still fixed. Are you sure you're not connecting to a server that did not move to the newer IP block (not the Mzima one, just the /24 that we have better control over)? We discussed this in the Events post and email that I referenced.
nofxpunk6
New to forums
New to forums
Posts: 4
Joined: Thu Aug 23, 2007 11:52 am

Post by nofxpunk6 »

I am also having an issue with my Internap routing from Wave Cable just accross the water from seattle. I will post my tracert aswell:


C:\>tracert seattle.nuclearfallout.net

Tracing route to seattle.nuclearfallout.net [208.146.45.132]
over a maximum of 30 hops:

1 1 ms <1 ms <1 ms 192.168.1.1
2 18 ms 10 ms 10 ms 24-113-46-1.wavecable.com [24.113.46.1]
3 11 ms 25 ms 21 ms 24-113-127-41.wavecable.com [24.113.127.41]
4 22 ms 12 ms 18 ms wavebroadband-gw.g11-0.oly-cor0.noanet.net [64.1
46.232.253]
5 15 ms 17 ms 16 ms srp3-0.wtn-cor0.noanet.net [66.119.192.25]
6 17 ms 14 ms 16 ms 64.209.96.85
7 151 ms 73 ms 69 ms InterNAP-Ken-Schmid.ge-3-0-0.402.ar2.SEA1.gblx.n
et [208.49.181.130]
8 70 ms 67 ms 74 ms border25s.ge2-1-bbnet2.sea.pnap.net [206.253.192
.226]
9 72 ms 69 ms 71 ms nuclearfallout-5.border25s.sea.pnap.net [206.191
.144.42]
10 69 ms 70 ms 72 ms c-208-146-45-132.premium-seattle.nuclearfallout.
net [208.146.45.132]

Trace complete.

C:\>

Any help would be most appreciated.

Bryan
nofxpunk6
New to forums
New to forums
Posts: 4
Joined: Thu Aug 23, 2007 11:52 am

Post by nofxpunk6 »

after looking at mine more closely it seems that its still in seattle just getting latency from an internap router for some reason :/
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Post by Edge100x »

nofxpunk6, yours is an entirely different problem and appears to be due to InterNAP's outbound route selection. Please send me your client IP in email to rentals @ nuclearfallout.net and I will take a closer look.
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Post by Edge100x »

nofxpunk6, I'll also ask noanet.net if they're interested in peering with us through the SIX (Seattle Internet Exchange), since they appear to have a presence there. This would reduce latencies even further, if they agree to it.
nofxpunk6
New to forums
New to forums
Posts: 4
Joined: Thu Aug 23, 2007 11:52 am

Post by nofxpunk6 »

Thank you! I seen my ip address via email to rentals@nuclearfallout.net as asked.

Bryan
sprtn
New to forums
New to forums
Posts: 4
Joined: Sat Apr 14, 2007 9:23 am

Post by sprtn »

I get routed through sacramento still...

Tracing route to c-208-146-45-132.premium-seattle.nuclearfallout.net [208.146.45.132]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 * * * Request timed out.
3 19 ms 10 ms 10 ms GE-1-3-ur01.seattle.wa.seattle.comcast.net [68.86.99.33]
4 10 ms 11 ms * te-9-3-ur02.seattle.wa.seattle.comcast.net[68.86.96.98]
5 10 ms 12 ms 18 ms te-9-1-ar01.seattle.wa.seattle.comcast.net [68.86.96.102]
6 9 ms 9 ms 9 ms te-7-2-ar02.seattle.wa.seattle.comcast.net [68.86.96.174]
7 12 ms 11 ms 15 ms 68.86.90.217
8 26 ms 29 ms 29 ms te-0-0-0-0-cr01.sacramento.ca.ibone.comcast.net [68.86.84.38]
9 30 ms 28 ms 28 ms 68.86.85.78
10 35 ms 30 ms 28 ms TenGigabitEthernet3-1.ar1.snv2.gblx.net [64.214.174.109]
11 35 ms 38 ms 35 ms InterNAP-Ken-Schmid.ge-3-0-0.402.ar2.SEA1.gblx.net [208.49.181.130]
12 38 ms 34 ms 35 ms border25s.ge2-1-bbnet2.sea.pnap.net [206.253.192.226]
13 39 ms 37 ms 36 ms nuclearfallout-5.border25s.sea.pnap.net [206.191.144.42]
14 35 ms 35 ms 35 ms c-208-146-45-132.premium-seattle.nuclearfallout.net [208.146.45.132]

Trace complete.
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Post by Edge100x »

sprtn, yes, as I mentioned, the server you are tracing to (seattle.nuclearfallout.net) is on the old netblock. I suggested to our Premium Seattle customers back in May that they move to our new 208.146.44.xxx netblock to avoid this problem, and our gaming community chose not to do it. Most of our regular customers did choose to move and are enjoying lower latencies to Shaw, Comcast, and other local ISPs.

I will ask the community again whether it is willing to move.
Post Reply