Sat, 05 Feb 2011

Black Hole Routing

A number of Tier-1/Tier-2 network service providers support a feature called real time black hole routing triggered via BGP. In simple terms, what this means is that with providers that support this, you can advertise a route to your transit provider(s) that tells them "I'd like you to null route this instead of routing it to me." Why would this be useful? The most likely situation is an IP on your network is being DDoS'd (Distributed Denial of Service attack) hard enough that it's congesting your transit pipe(s) causing increased latency and/or packet loss for all of your internet traffic.

The usual way to do this (or express any other sort of desired upstream routing policy to your transit provider) is via BGP communities. These are set in the output route-map for your eBGP peering with the provider. The following is an example of how you might setup a system to allow for easy creation/removal of real time black hole routes (on cisco gear).

First:

1. Make sure your provider(s) support this.
2. Look up what community strings they use for this.
3. Step three, you'll probably need to contact each provider and make sure they're setup to receive /32 IPv4 routes from you. Assuming they do prefix filtering, they may not automatically be setup to accept such specific routes from you.

Using Level3 as the example, a search for Level3 BGP communities will turn up that 3356:9999 is Level3's customer accepted community for telling Level3 to discard traffic for the tagged route.

Now, you could edit your Level3 output route-map, BGP config (insert a network statement), and then do a static route every time you want to create a real time black hole route, but doing it that way is time consuming and error-prone and you may not want everyone who has enable access mucking around in your eBGP config. Instead, why not set things up so all you have to do is create a special static route anywhere on your network?

This config assumes you have separate routers for transit connections and for internal routing, there are config changes that will need to be done on each.

On the transit router that talks to Level3:

ip community-list standard BLACKHOLE permit <your-ASN>:9999
!
route-map LEVEL3-OUTPUT permit 5
 match community BLACKHOLE
 set community 3356:9999

On your internal routing router(s):

ip access-list extended match32
 permit ip any host 255.255.255.255
!
route-map blackhole permit 10
 match ip address match32
 match tag 9999
 set community <your-ASN>:9999
!
router bgp 
 redistribute static route-map blackhole
 redistribute ospf 1 route-map blackhole

If you have much experience with BGP, you're hopefully saying to yourself "but redistribution, especially of the IGP, into BGP is really dangerous". Well, the blackhole route-map is limiting the redistribution to only those routes tagged with 9999 and which are /32s. This means if someone gets clumsy and creates a route for a shorter network with the tag 9999, the route-map will not match that route, and it won't be redistributed into BGP. So this setup won't let you accidentally real time blackhole an entire CIDR block. The reason for redistributing both OSPF (or you might use ISIS) and static is, this way the route can be created on this router (as a static route) or on another device participating in your IGP.

Once these config changes have been made, all you need to do to real time black hole route an IP is log into the internal router or any router in your IGP and

ip route <IP to black hole> 255.255.255.255 null0 tag 9999

A really neat side effect of this setup is, you can real time black hole an IP without null routing it internally.

Suppose the IP you want to real time black hole is part of a customer's /28, and that /28 is configured as the IP on the customer's access port. i.e.

interface FastEthernet0/1
 ip address <customer IP network> 255.255.255.240

You can log into that device, and

ip route <IP to black hole> 255.255.255.255 FastEthernet0/1 tag 9999

Now, the IP is still routed to the customer, but because it's tagged with 9999, assuming your customer aggregation routers redistribute static into your IGP (which for me is OSPF), the route will be in your IGP with the tag, your internal router will see this and redistribute it into BGP with the internally used real time black hole community, and your transit router(s) will tag the route with the appropriate community to have your transit provider(s) real time black hole route it. The IP is still reachable inside your ASN, but to the rest of the internet, it's dead as your transit providers are null routing it.

[/internet/routing] permanent link