VDS Down 3'rd day out of 4 due to DDOS?

[MikeGracia]

Hi all,

I know it's not NFO's fault, but at the same time, it's a Friday night, and for the 3rd time in 4 days we have an outage

Is there a reason NFO is having the problem? Is someone targeting them specifically for some reason?

I think NFO are a GREAT host, with excellent support, and I feel bad they keep getting hit - but at the same time, I spent ages trying to sort our gameserver to be as I wanted, finally built up a group of players, and for the last few days we have had serious issues and player loss

I don't really know why I'm posting this as such, wondered if it was effecting everyone, or just a few people?

Would moving to a different Datacentre help? - I know that ANYWHERE can get a DDOS, but with NYC being hit 3 times in 4 days, would somewhere else be safer? Even if just at the moment?

[MikeGracia]

VDS came back up, then went down again

I just want to know what to expect this evening - will we be in for another night with intermittent connections to the server, or is this just a blip?

Thanks guys

[MikeGracia]

So I have just been told in support that moving to a new host may indeed help with the DDOS attack issues, but I can't move to a new location until the DDOS issues are resolved...

So no point in moving then?!

NFO support has always been amazing, and I do feel bad for you guys getting hit so bad with the DDOS in NYC, but at the same time, my players are starting to wander off elsewhere, and I can't blame them! 3rd day of packets loss, disconnects and outages, who could

I hope this gets resolved soon :S - Why would anyone want to DDOS NFO! They have the best support and prices around :S

[Vanderburg]

To clarify, a new host was not recommended, but a new location. I know this is what you meant, but that's not how it looks. The problem with moving locations while the location's connection is saturated is that there's no room for your game server/VDS data to be sent over and if our system can't talk to the machine hosting your VDS, something can definitely get broken in the process.

[MikeGracia]

Hi Vanderburg,

I understand, thanks for clearing it up (got an explanation via support ticket too - thanks guys!).

I meant new location yup, not new host - sorry! I meant a move to one of your places other than NYC - No intention of LEAVING NFO!!! You guys are great, I know that!

Sorry if I seemed stressed, it's just, well... I was stressed! Players don't always understand, and I've spent quite a while building them up, so get worried that all the effort will be wasted if they leave - After 4 days of intermittent issues on a server, some of them do start to wander I think - understandable.

Anyhow, I hope it's all sorted Let's hope the eeeediots doing the DDOS stop. Do you think it was someone trying to hit you guys directly, or the DC?

To clarify, a new host was not recommended, but a new location. I know this is what you meant, but that's not how it looks. The problem with moving locations while the location's connection is saturated is that there's no room for your game server/VDS data to be sent over and if our system can't talk to the machine hosting your VDS, something can definitely get broken in the process.

[MikeGracia]

Aaaand down again

[wolfn00b]

Our servers have been lagging and crashing for several days too.

About 8:30PM (central US time) our vds in New York crashed, again, disconnecting all our gamers on five servers we were running on the vds.

I have been tinkering with our game servers inside the vds for a couple days, thinking maybe we were having some kind of memory leak or maybe running too many files on our mods.

Or maybe even some vindictive player we banned was trying to hack our server with ddos or some kind of ftp hack.

So I am kind of relieved to see this is a problem with the whole Internap network lol (hey it's not my fault!)

Please get it fixed so my players will stop bitching at me

[soja]

Wow I had no idea NY was having so many issues, I am over in chicago and I get no notifications that NY is getting hit. That must really suck, we have 2 dedicated machines here and a VDS in chicago and if someone can take down NY with 20Gbit then I really hope they don't come to chicago D:

Good luck guys, I know NFO is doing everything they can!

[Edge100x]

We are doing everything we can and posting events to the Events log page when these happen (so make sure to check there for updates).

I wouldn't recommend jumping to a location switch just yet, as the attacks in NYC will be resolved one way or another. We can't tolerate having a location taken offline this frequently.

[MikeGracia]

I wouldn't recommend jumping to a location switch just yet, as the attacks in NYC will be resolved one way or another. We can't tolerate having a location taken offline this frequently.

I totally understand where you're coming from, but can I ask, you do realise that this may result in us losing a decent percentage of our player base - that has been hard to build up over the past few months?

I do hope for some level of loyalty from our players, however with this going on for so many days, I couldn't blame them for moving off.

I just want to make sure that you realise the issue here for us - we are being asked to not move DC's, and to wait until the DDOS is sorted before moving, yet it's impossible for you to give us an ETA on the solution to the DDOS which has already been going on for several days.

NFO are great, however the solution offered here I don't think is - sorry.

I am at the stage now where two to three times a day I have to say sorry to players, un-kill them in the database, and try to restore their lost gear (server doesn't go down just gets disconnected, keeps running, they die - rejoin dead). <--- To have to keep doing this for an unspecified amount of time, hurts :S

I know you guys can't give an accurate ETA on fixing this guy's attacks, but do you THINK it will be a matter of hours, or days, or could we be talking weeks?

[MikeGracia]

Right, server's gone down again.

When it comes up, I think I'm going to copy all game files and database files etc to be safe, as if this carries on for much longer, and NFO won't 'allow' me to move away from the DDOS'd datacentre that is being effected until it's resolved, I will have to move elsewhere

I HATE to say that, but for the last 4 days, the connection has been on/off, we've lost players, and we've been told to sit tight for several days now - unable to either get access to the product we pay for, OR be allowed to move to another NFO datacentre.

I understand you guys are likely too busy to handle a DC move right now (sorting the DDOS etc), but we only run ONE Dayz instance, and I can backup everything, and restore it to a new box myself within 1/2hrs max.

What I don't want to do, is pay for 2 boxes whilst I carry out the move.

NFO - If I do the move myself, are you still saying I can't move away from the Datacentre that's been DDOS'd consistently for the last 3 or 4 days, until those DDOS's stop?

[soja]

DDOS attacks can be launched at any time, and the pro-active DDOS protection can only do so much.

I assume being this intense of an attack, that John has to personally talk with internap to mitigate/null route the attack, and it is 4:40am on the west coast atm(where john lives).

I don't think they aren't "allowing" you to move, John is just saying the attacks will be resolved soon and you would've lost your IP for nothing.

It seems like your server type gets really screwed up in an attack, more so than others, and for that I am truly sorry, and it might be a better idea to leave the server offline and communicate with your players via forums/social networking/ts3 or vent letting them know what is happening.

[Edge100x]

MikeGracia, it's up to you on whether to move or not. VDS moves are not done manually here; they are automatic, with the virtual drive copied by the system when you independently trigger the move via the order page. It doesn't make a difference on our end whether you move. My advice was based on what I would personally suggest doing for long-term satisfaction, and applies to anyone seeing a temporary problem with NYC because of the attacks over the last two days.

There was no DDoS attack this morning, and no other observed problem at or around 3:50am PDT. If your server restarted at that time, you were seeing something unrelated to these events.

[wolfn00b]

John it has occurred to me that this may be a bit too professional to be some kid in his mom's basement with a homemade botnet.

Mike try talking to your players, post a link to this thread in your forum if you have one. I sent out a mass email on my clan forum to let everyone know what was going on.

A lot of my players thought it was something wrong with their computer, or their game client, or something I was doing wrong. If people know what is going on they are more sympathetic.

The whole point of the ddos is to get customers and players mad so they will leave nfo, to punish nfo for some perceived wrongdoing, or to make them lose clients, so don't let them win.

[joshh20]

These attacks are unfortunately because I banned a player who was breaking rules on my server after he donated.