Chicago bandwidth upgrade

[stickz]

We have added two more transit providers in Chicago.

Is Teksavvy currently a Chicago transit provider like in New York? Using your ISP to carry packets has various benefits. They have a server there and i'm wondering why my packets aren't using it.

[Edge100x]

No. The two providers that we added are both global tier 1s that are able to provide upstream filtering that assists in mitigating very large DDoS attacks. It looks like we have good latencies to TekSavvy, though.

[stickz]

No. The two providers that we added are both global tier 1s that are able to provide upstream filtering that assists in mitigating very large DDoS attacks. It looks like we have good latencies to TekSavvy, though.

"Premium locations should offer the best possible latencies in their respective cities -- not just lower than Internap alone, but lower than anyone else out there."

I do not have this in Chicago but got it in New York. Pings are lower with less variance when my ISP is used to carry the traffic directly to their Toronto pass-through.

And any routing issues are non exist when immediately routing through their network. If a filter was added for Teksavvy, 1000's of clients would never have a routing issue and better networking, in both New York and Chicago where their servers are located.

My variance (best to average) is approximately 0.8ms lower than any tier 1 provider in New York. The mtr I ran was flicking between 1ms and 2ms which leads me to believe around 1.8ms variance.

To be fair though, my inbound is using route ae7 instead of ae6 which is stronger on the gtt carrier.

[stickz]

p.s Teksavvy has an agreement signed with League of Legends for Internap Los Angeles. It essentially states that traffic will use a certain route, so people rarely if ever experience networking issues.

[Edge100x]

TekSavvy is a smaller Canadian consumer ISP. It is unlikely that they even sell transit in Chicago and I have not seen demand from customers to purchase from them. It is unlikely that they offer ACL support. They are very well-reached through our existing transit links, already with very low latencies. Priority to connect to them would come below all tier 1 providers and larger Canadian and US ISPs.

As far as I know, TekSavvy does not have a PoP in LA, so transit would just be used to reach them to/from that location. That works fine, because, in many cases, peering with a provider far outside of its served area (such as if TekSavvy were in LA, whereas TekSavvy customers are all in Canada) does not decrease latencies over using a transit provider that peers with them closer to the edge of their customer base.

[stickz]

TekSavvy is a smaller Canadian consumer ISP. It is unlikely that they even sell transit in Chicago and I have not seen demand from customers to purchase from them.

They sometimes carry traffic to Chicago through their network before passing it off to anther carrier. It's highly likely they do sell transit in Chicago, like in New York. But the point is, the premium sticker on Chicago doesn't mean lower latencies and better routing, if these opportunities aren't being taken advantage of. It's more of a sales gimmick than anything else, which confuses people into buying Chicago. It's definitely apparent the DDoS migration is there, which is at least half the truth.

I will quote this exact line from your website again.
"Premium locations should offer the best possible latencies in their respective cities -- not just lower than Internap alone, but lower than anyone else out there."

They are very well-reached through our existing transit links, already with very low latencies. Priority to connect to them would come below all tier 1 providers and larger Canadian and US ISPs.

Far western clients have HUGE issues with your existing transit links. Chicago may not be as much of a concern, but Frankfurt for instance is absolutely horrible. It would be an absolute dream for one of those clients to download a file at 15% bellow their maximum speed. I'm eastern and had to get a shunt from Internap to force the correct route just to resolve more minor issues.

I know this because I've tried it. New York is the best compromise, which hurts some of my European playerbase. It's a complete joke, how one EU client has 140ms and the other 115ms in the same city. I red flagged this right away, and shortly after they started complaining about lag.

[theRadAleks]

You do have to take in consideration that one of your players may not have as good internet as the other. For example if one of them has an average upload speed of around ~2mb and the other has ~6mb then you will definitely see at least a 20ms difference

[Edge100x]

But the point is, the premium sticker on Chicago doesn't mean lower latencies and better routing, if these opportunities aren't being taken advantage of.

We do not define Premium as "the absolutely best possible path, inbound and outbound, all the time, for every single client." That is simply not possible, even with an unlimited budget. (And for you to suggest that we prioritize buying a large transit link to a smaller provider based in another country, you must think our budget is quite large!)

Premium means that we are using a mix of Internap and other bandwidth and engineer the combination to lead to the same or better average latencies for the client base overall, when compared to all other providers in the same city. And, realistically, Internap is otherwise the highest-performance option in most cities, so it is primarily a comparison to Internap.

Far western clients have HUGE issues with your existing transit links.

I have not seen any evidence of widespread, or even not-so-widespread, routing issues out of Chicago. I'd be happy to look at whatever data you are referring to in a support request.

Chicago may not be as much of a concern, but Frankfurt for instance is absolutely horrible. It would be an absolute dream for one of those clients to download a file at 15% bellow their maximum speed. I'm eastern and had to get a shunt from Internap to force the correct route just to resolve more minor issues.

You personally had a very difficult-to-pin-down issue out of Frankfurt due to an occasional routing choice to your ISP through Internap (we use Internap exclusively in Frankfurt). This was not a problem that occurred for a large segment of the client base there and it is completely separate from anything in Chicago.

[stickz]

For example if one of them has an average upload speed of around ~2mb and the other has ~6mb then you will definitely see at least a 20ms difference

Incorrect. You will only see a difference if your upload speed reaches near 80% which rarely happens on the source engine. I went from 15/2 to 30/10 in the past month with no difference in latency.

I do however migrate inference by using quality of server controls (QoS) to schedule my packets. If your router supports it, I heavily recommend flashing dd-wrt, enabling QoS and checking off "optimize for gaming". I can start youtube videos without spiking my in-game latency. (they try to eat up all your bandwidth for a short burst)

You do have to take in consideration that one of your players may not have as good internet as the other.

I heavily take this factor into consideration.

I got custom additions which automatically reduce bandwidth by 50% when the server reaches higher counts. By scheduling less packets, I can reduce my CPU usage, as VDS offerings are pushed to the red-line with my game-types. Also, many clients running on weaker carriers, experience less issues as the server count raises. Far western and oceanic clients often end up using those weaker carriers and expense issues with current transit routes.

[theRadAleks]

For example if one of them has an average upload speed of around ~2mb and the other has ~6mb then you will definitely see at least a 20ms difference

Incorrect. You will only see a difference if your upload speed reaches near 80% which rarely happens on the source engine. I went from 15/2 to 30/10 in the past month with no difference in latency.

I recently went from 18/2 to 100/10 and i have defiantly seen an improvement in my latency to over-sea servers for example i had around ~150 ping to the Frankfurt location and now i average around ~130

[stickz]

Far western clients have HUGE issues with your existing transit links.

I have not seen any evidence of widespread, or even not-so-widespread, routing issues out of Chicago. I'd be happy to look at whatever data you are referring to in a support request.

My clients are long gone before I can get them to send me an mtr. They never show up the server again. The most common misconception is "my countries internet is horrible, it's just not strong enough for overseas." This is absolutely incorrect. I see their latency spiking and routes using weaker carriers; When their next-door neighbor is using level-3 combined with gtt for a competitive latency.

[theRadAleks]

Also my ping to the Seattle would hit around ~90 and now on average its ~70

[stickz]

I recently went from 18/2 to 100/10 and i have defiantly seen an improvement in my latency to over-sea servers for example i had around ~150 ping to the Frankfurt location and now i average around ~130

This stuff here is the magic which decides your latency, not the amount of bandwidth you have. Keep in mind, I got a shunt which forces the lowest latency route. I also had to hide my ip address for security reasons due to my personal configuration. Toronto to Frankfurt are the two major locations. If I lived in New York, I would expect a 90ms latency on Frankfurt. Toronto would be about 100ms. (nothing but the best)

Inbound to Frankfurt.

Code: Select all

|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                                  DD-WRT -    0 |   50 |   50 |    0 |    0 |    3 |    0 |
|                            xxx.xxx.xxx.xxx -    0 |   50 |   50 |    6 |    8 |   13 |    9 |
|              xxx.xxx.xxx.xxx.cpe.pppoe.ca -    0 |   50 |   50 |   17 |   18 |   25 |   20 |
|              xxx.xxx.xxx.xxx.cpe.pppoe.ca -    0 |   50 |   50 |   17 |   19 |   38 |   19 |
|         ae0_2160-bdr04-tor.teksavvy.com -    0 |   50 |   50 |   16 |   18 |   23 |   19 |
|                   ae6.tor10.ip4.gtt.net -    0 |   50 |   50 |   17 |   20 |   46 |   20 |
|              xe-2-2-0.fra23.ip4.gtt.net -    0 |   50 |   50 |  109 |  112 |  138 |  115 |
|                 internap-gw.ip4.gtt.net -    0 |   50 |   50 |  110 |  112 |  117 |  113 |
|      border3.ae2-bbnet2.fra002.pnap.net -    0 |   50 |   50 |  109 |  112 |  124 |  111 |
|v-31-186-251-151.unman-vds.internap-frankfurt.nfoservers.com -    0 |   50 |   50 |  109 |  111 |  116 |  113 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider

Code: Select all

Outbound from Frankfurt
My traceroute  [v0.86]
rna1 (0.0.0.0)                                         Sat Jul 25 23:09:08 2015
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                       Packets               Pings
 Host                                Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. edge2.xe-0-0-14.nuclearfallout-5  0.0%    54    2.8   0.5   0.3   2.8   0.5
 2. core3-0-0-0-0.fra002.pnap.net     0.0%    54    0.9   0.9   0.7   1.2   0.0
 3. xe-10-1-2.fra23.ip4.gtt.net       0.0%    54    0.5   0.9   0.4  10.9   1.9
 4. xe-10-0-0.tor10.ip4.gtt.net       0.0%    54   92.9  94.5  92.8 126.1   6.0
 5. tek-savvy-solutions-gw.ip4.gtt.n  0.0%    54   92.9  93.3  92.9 112.1   2.6
 6. ae3_2170-bdr01-tor.teksavvy.com   0.0%    53   93.0  94.5  92.9 129.1   6.2
 7. xxx.xxx.xxx.xxx.cpe.pppoe.ca        0.0%    53   94.4  94.5  94.2  94.8   0.0
 8. ???
 9. xxx.xxx.xxx.xxx.cpe.teksavvy.com   0.0%    53  110.3 112.1 110.1 120.3   2.0