Yep, I did it again.

[stryfe]

So, what's the deal now with the nightly blaze crashes we are seeing in the Chicago Datacenter? TimSad and myself have both noticed a ton of them and I got players now telling me it's happen 3-5 times a week at night?

[Edge100x]

We have not been seeing location-wide Blaze-related drops in Chicago every night, thankfully.

I did see one last night right around midnight, but it was the first in awhile. The drop only happened for Blaze-using games and it was not accompanied by an attack against a customer here. From the timing, it was highly likely to be an unannounced upstream maintenance event occurring somewhere along the path to or from the ISP hosting Blaze and Internap in Chicago (as well as other places, certainly).

[TimSad]

It has Blaze player dumped on me either 4 or 5 times late at night within the past week while my LA servers remain stable. Might have to relocate my Chicago servers to another location if this continues. Hopefully it will settle down because I'd really prefer for my Chicago servers to stay in that area since it tends to get the best player traffic.

[Edge100x]

This is not a common problem in Chicago or any other location. You should have us check to see if your server is crashing instead of experiencing problems with Blaze.

If it's occurring at a specific time of day, we can also run MTRs to Blaze at that time and see if we can detect any loss along the path.

[stryfe]

Yeah this seems to happen nightly I don't know exactly when I would need to look at my control panel to see the time of the drop. Like Tim said though, Los Angeles server was up for over a week and 3 days and saw ZERO problems.

[Edge100x]

The usage graph is usually the easiest way to see the time of the drop.

[stryfe]

You can see a couple days where it a did a dramatic drop in player count.

[Edge100x]

The one early on the 30th is the sharpest and most likely to be a Blaze issue or crash. The others are harder to tell, since they are not as sudden -- and keeping in mind that servers also usually empty out organically at the end of the day, often fairly quickly because there can be a cascade of drops at round end.

What you'll want to do is to contact us after a specific drop and have us check it out.

[stryfe]

That's the thing, I don't know when it happens. If it's 300am eastern I'm dead asleep as I have a career. IF I can catch it I'll definitely let you guys know and I am sure TimSad will as well.

[Edge100x]

It's OK if it's not immediately after it happens. We should still be able to look at logs.

[TimSad]

I'd really hate to jinx myself here but here it goes... After that really rocky week, it seems that I've now had a week of flawlessness. No late night Blaze drops for about a whole week now.