NYC Datacenter and What it means to customers

[RainMotorsports]

Wish i could edit my posts more than 2 minutes out Elochai

Building was built in 1929 has 34 floors and 2 sub basements. Battery backup in event of total failure of generator and main power is rated for 30 minutes at full load so probably an hour or so normally. Might be why the data center took so long to go offline who knows but Internap.

[Edge100x]

dyno, that does not match what InterNAP posted to us -- they have never used the word "destroyed" in their updates to us, although it is possible that the pumps were destroyed (even if they weren't, they may still be underwater, the fuel may be contaminated, etc). I posted the exact wording that InterNAP used (it was a copied-and-pasted quote) to everyone's "Events log" page soon after they sent it to us.

The latest word from InterNAP is this:

At this time, the NYM-EXT1 facility is still without power. We have coordinated fuel deliveries and pumps, as well as engineers to fabricate pipe to bring the fuel directly to the generators on the mezzanine level. There is currently no ETA available for the completion of these activities, but we are working through all options to bring the UPS infrastructure back online as soon as possible. We will continue to work toward resolution and provide status updates with any new information. Thank you for your continued patience and understanding during this crisis.

I assume that there is some reason for the fuel being stored below-ground and for the pumps also being there, but I do not know it myself. I can only guess that historical reasons and/or NYC building codes on the storage of fuel are involved. I am sure that the setup can be improved for the future, such as by using submersible pumps or relocating the pumps to the mezzanine level.

Some other DCs in NYC were affected by flooding in the same way. Others were on higher ground and only had to worry about the power outage.

Looking up how much fuel these generators use, they had quite a bit of fuel on the mezzanine (though clearly not enough to get them through a catastrophe like this). Diesel burn rate for a fully-loaded 2 MW generator is 150 gal/hr.

[IcEWoLF]

Easy fix, get servers in the west coast or central.

Just a note, other datacenters are experiencing the same issues, so this is not just NFo, take a look at WHT (webhostingtalk.com)

[IcEWoLF]

BTW this is a good article to check out:

http://www.datacenterknowledge.com/arch ... a-centers/

[rustydusty1717]

The gen sets seems to be saving the day for many locations on the east coast as we speak!

[dyno]

Edge100x -

I do apologize -- shouldn't have inferred the message was changed without first checking with someone at NFO. It was simply peculiar that another, nearly identical, message was posted online near the same time. BUT - I really don't/didn't have any reason to doubt the message that was posted in our events logs. Everyone at NFO has always been honest / straight with me.

Went to edit the earlier post but it seems too much time has elapsed.

[Edge100x]

InterNAP likely just sent out a slightly different message to different classes of customers. The note you posted seems to have gone out to their managed dedicated server clients.

Power has been restored as of late Tuesday night, thankfully!

[Elochai]

Lets hope they can get the link back up as I been having nothing but issue with the temp server. It took hours for 2 game servers to be created on it. The shut down every few mins, the notice went out that there was an issue with xen and that the system needed a reboot at 1 am. Woke up just 10 mins ago to find the two servers I was able to get created were both offline.

So really so far I haven't got any use out of the temp server or settled anything with my client.

[Elochai]

My client just message me that the servers are back online. This is great news and I hope the fuel keeps coming till the power has be restored

[rustydusty1717]

http://gigaom.com/cloud/how-good-prep-a ... ane-sandy/

[Elochai]

http://gigaom.com/cloud/how-good-prep-a ... ane-sandy/

Now that dedication !

[soja]

http://gigaom.com/cloud/how-good-prep-a ... ane-sandy/

If i lived closer I would've helped out

[rustydusty1717]

Here's a pale of gas, thanks for the poweredge, here's some more fuel, thanks for another poweredge.....

[Edge100x]

I wish InterNAP could have done that! Unfortunately InterNAP likely didn't have the staffing for it, as most of their people were evacuated and their generators are bigger (their website says 2 MW, which would have a burn rate of up to 150 gal/hr).