With a new wave of extremely large (100+ Gbps) attacks, we've been forced to issue more null-routes than usual recently -- sometimes several per day. Though we discuss how we handle null-routes in detail in the first post above, we still frequently hear specific questions about null-routes that we will answer here, for easy reference.
Can't you just block my attack?
We do filter attacks. We block attacks internally, the instant they occur. But, that doesn't prevent our upstream links from being saturated by the attacks -- it only stops our internal links from being saturated. With most null-routes, the attack saturated our upstream links, instead.
At all our locations, we exclusively buy from upstreams that can help filter certain types of attacks on their end, as well. We leverage these filters whenever possible.
The attack that I saw looks small! Why did you need to null-route?
We have a highly-tuned internal mitigation system that prevents most attack traffic from reaching target customers. Because customers see little of the attack traffic, attacks can look small. Most machines also have 1 Gbps network adapters, so the tens of Gbps that these attacks generate simply can't be seen.
If we issue a null-route, it means that the attack saturated our links to upstreams and/or caused problems for one of our upstream providers -- which means that it was extremely large. Automatic safeguards prevent null-routes from being issued in other circumstances, and every single null-route is manually reviewed to make sure that it was truly necessary.
Can you remove our null-route early?
In most cases, if a null-route was issued, it was necessary, and can't be removed early. When we remove a null-route before the posted period has passed, it exposes the location to a much higher risk of further impact.
There are occasional cases where we can remove null-routes early, such as when we are able to work with an upstream to implement a new filter that would help with that specific type of attack, or when a false positive is determined. In those cases, we remove the null-route without requiring that the customer contact us. This means that you should not contact us to ask for a null-route to be removed.
The null-route event says that it will stay in place for some number of hours. I don't believe that the attack would have lasted more than a few minutes. Why does the null need to last so long?
A null-route is a huge deal -- an emergency, stopgap, last-resort measure that we take to protect our network, and other customers, from the serious impact of a major attack. With that in mind, we base the amount of time that we null-route each target IP address on a number of factors, including:
- The likelihood of additional attacks (and additional collateral damage to other customers)
- How impactful the attack was or could be (if repeated)
- The rate at which similar attacks and null-routes are occurring at the same location
- The length of time that it takes our system to send out emails to other involved hosts
- The amount of time it takes the most responsive hosts to investigate and clean the compromised devices that we report to them
If there is a belief that the attacker wouldn't have continued attacking for very long, that's good, but that is not an important factor in this equation.
We manually evaluate every null-route action and remove a null-route early when we determine that it makes sense to do so -- based on our calculation of these factors for that specific attack. For instance, if an attack only barely saturated connections to our upstreams and caused only light packet loss, we may remove the null-route before the communicated amount of time, because having a small percentage of the compromised devices removed from the botnet would make additional attacks small enough that we could mitigate them effectively.
How big was a specific attack? What type of traffic was involved?
For security reasons, we do not generally release more information than what we state publicly about all attacks (including in these KB articles and in the network capacity numbers given at
https://www.nfoservers.com/networklocations.php) and what is posted to the control panel.
Additional information beyond that is very valuable to attackers, who use it to tune their attacks and market their illegal services.
The attacker told me something about an attack that triggered a null-route, or warned me that more attacks are coming, or told me that I had to do something to prevent more attacks. What should I do?
You should stop communicating with the attacker, and make sure that anyone associated with your service also does not communicate with the attacker. Universally, communicating with the attacker leads to further attacks.
I am panicking because I received attacks over the last couple of days that triggered null-routes. Will these continue to happen forever?
No. Attack sizes rarely stay large for very long. Botnets often shrink in size over time as the devices that attackers use are fixed, rebooted, or taken offline -- or are compromised by different groups. Spoofed traffic sources (used for very large reflection attacks) are typically noticed and taken offline, and reflection attacks are also generally easier to filter than botnet attacks.
It seems like it's really easy to trigger a null-route (with just a few minutes of traffic) that takes down my service (for hours)! Is that true?
No. Attacks that trigger null-routes are exceptional. Run-of-the-mill attackers can't run this level of attack, and as we mention in another answer, attacks of this size can't keep recurring indefinitely.
In terms of the length of an attack, if an attacker can run an attack for a few minutes, then that attacker could also run it for much longer -- hours, if necessary.
Will I receive a credit or discount for time that my service is null-routed?
We are not able to provide credits to the victims of attacks. We are not to blame for these attacks, and we do not receive a credit from our upstreams for them, so a credit would not make sense.
We also put a large amount of time and money into defending targeted customers against attacks, and we take the most damage from them, but we do not issue customers a bill for handling them.
I want to go after the attacker! What can I do?
You should report the attack to the law enforcement authority that handles cybercrime within your jurisdiction. In the US, that is generally the FBI, and the crime can be reported through
https://www.ic3.gov.
Make sure that your report to law enforcement includes any information that you have on who might have launched the attack, and any evidence that you can present of that. For instance, if the attacker told you that he or she was going to attack, you can pass along that conversation, along with all contact details that you have for the attacker.
Make sure to also mention NFOservers as your host, and let your law enforcement contact know that we have more data that we can provide about the attack size and devices that were involved. We already work with law enforcement sources, including the FBI, to provide targeted information on large-scale DDoS events, and we can loop in your new contact.
You should
not retaliate for an attack by launching an attack of your own. Not only is this illegal, but it is also an escalation that will increase the likelihood that you will receive further attacks yourself. You may also target the wrong person, or cause serious collateral damage to an ISP or other host.
Can't NFO just upgrade its network to handle the type of large-scale attack that led to my null-route?
Sometimes, we have the option of upgrading our network, at a significant cost. We have to weigh this large cost against how many customers would be helped by the upgrade, and we have to determine if the attacks will continue after the upgrade is done (upgrades typically take months to perform, due to waiting for upstreams to provision the new circuits). When it makes sense, we proceed with such upgrades. We have upgraded our Chicago location repeatedly, for instance, and continue to add to its capacity.
At other times, dependent on the attack sizes and location, we can't upgrade our upstream links because the upstreams themselves are being saturated. We must wait for upstreams to upgrade first, if so. We start those conversations when we see such a situation occur.
I think that another host can do better in handling attacks that cause null-routes with NFO. Why should I stay with NFO?
In most cases, it is not true that another host could do better than what we can do in our Chicago location. We have advanced mitigation systems at all locations that are generally seen as the best available, and our Chicago location, specifically, has a larger capacity and better upstream filters than most providers can dedicate to mitigation, even the largest ones.
We have heard reports that some hosts do not issue null-routes and instead simply let large attacks come through and bring down the target service -- meaning that when the attacker stops running the attack, the service recovers. This is not a positive thing, as it means that other customers connected to the same network see major connection interruptions from their "noisy neighbors". Null-routes, issued appropriately, are an important component in ensuring a stable, high-performance network.
