Distributed denial-of-service (DDoS) attacks are causing havoc in networks around the world, but in many instances there are steps that you can take to mitigate these types of attacks. This appendix attempts to detail some of the steps that you may take on network infrastructure routers to cause the least amount of harm in any network.
Denial-of-service (DoS) attacks are common on the Internet. The first step in responding to such an attack is to find out exactly what sort of attack it is. Many of the commonly used DoS attacks are based on high-bandwidth packet floods, or on other repetitive streams of packets. Many of these attacks were described in Chapter 5, “Threats in an Enterprise Network.” Attackers choose common exploits because they are particularly effective, particularly hard to trace, or because tools that enable execution of the attack are widely available. Many DoS attackers lack the skill or motivation to create their own tools and use programs found on the Internet; these tools tend to fall in and out of fashion.
A wide variety of DoS attacks are possible. Even ignoring attacks that use software bugs to shut down systems with relatively little traffic, the fact remains that any IP packet that can be sent across the network can be used to execute a flooding DoS attack. When you are under attack, you must always consider the possibility that what you're seeing is something that does not fall into the usual categories.
There are many schools of thought regarding filtering and rate limiting, especially on routers. Filters (or access lists) permit or deny traffic, whereas rate limiting constrains the amount of traffic being sent or received. The worst offenders of propagating DDoS attacks are people who refuse to apply any kind of filtering on their routers. It will not stop more sophisticated attacks, but even simple “no-brainer” filters will mitigate having obviously spoofed packets traverse corporate or Internet-connected networks.
The characteristics of the source IP address is typically a large factor in deciding whether to filter and/or rate limit traffic. The source IP addresses from DDoS attacks can have one of four characteristics:
Spoofed RFC 1918 and special-use addresses—. These are well-known addresses that should never be used or seen as part of the global Internet. Most can be found from an Internet draft by Bill Manning titled “Documenting Special-use IPv4 Address Blocks That Have Been Registered with IANA/RIR.” This list includes the network addresses in Table D-1.
Table D-1. Special-Use IPv4 Address Blocks Registered with IANA/RIR
Description
Network
Default
0.0.0.0 /8
Loopback
127.0.0.0 /8
RFC 1918
10.0.0.0 /8
RFC 1918
172.16.0.0 /12
RFC 1918
192.168.0.0 /16
Net test
192.0.2.0 /24
Testing devices
192.18.0.0 /15
IPv6 to IPv4 relay
192.88.99.0 /24
RFC 1918 name servers
192.175.48.0 /24
End-node auto configuration
169.254.0.0 /16
Spoofed addresses that are not in the global Internet routing table—. Routable IP networks are allocated via delegations of authority from the Internet Assigned Numbers Authority (IANA). The list of IP addresses that have not been activated is easily accessible at http://www.iana.org/assignments/ipv4-address-space. This makes it easy for someone to create an attack with spoofed addresses based on this list. Because these addresses are not in the global Internet routing table, a simple check in the forwarding table can easily aid in their identification. Some tools, such as unicast reverse path forwarding (uRPF), make this check straightforward, making it harder for these attacks to be successful.
Valid address from a DDoS agent—. When a compromised host acts as a DDoS agent, it is possible that the source IP address will be that of the penetrated system. In this case, filtering must be applied with care, because dropping all packets from the penetrated machine will also drop any valid user traffic being sourced from that address. In some situations, it may make more sense to apply some rate-limiting tactics instead.
Spoofed address using a valid address from the Internet—. Sometimes the attacker's target is actually the spoofed addresses. Instead of trying to directly take down a large corporate network, the addresses from that network are spoofed so that other entities (typically upstream connected networks) will isolate the corporate network through the use of filtering. As in any valid address DDoS attack, this is a case where rate-limiting tactics may make sense depending on severity of the DDoS attack.
It is always a good idea to be proactive rather than reactive in mitigating any kind of network security attack. Chapter 8, “Incident Handling,” discussed incident handling and the importance of having procedures in place to handle attacks, incorporating intrusion detection systems into your network architecture, and using tools to perform regularly scheduled host and network audits.
Specifically for DDoS attacks, there are additional ways to prevent your networks from becoming overloaded with unwanted traffic. These include the use of filtering and rate limiting and are discussed in this appendix. All examples are for Cisco IOS Software router configurations. For more detailed explanations of any filtering techniques listed here, refer to Chapter 10, “Securing Internet Access.”
All networks should implement some degree of network ingress/egress filtering to avoid spoofed attack traffic from being propagated across networks. Ingress filtering defines the traffic that is allowed to enter your network, whereas egress filtering defines the traffic that you send out to other networks. Filtering will not stop attacks that use valid source addresses, but all routers should incorporate filters that deny packets with spoofed RFC 1918 and special-use addresses from propagating through the Internet. In addition, all networks should consider following the recommendations in RFC 2827, “Network Ingress Filtering: Defeating Denial-of-Service Attacks Which Employ IP Source Address Spoofing,” by P. Ferguson and D. Senie. It recommends ingress filtering on the ISP/customer edge of the network to ensure that a source address of a packet matches the IP address block allocated to a customer. This recommendation can be extended to corporate networks, many of which are large enough to follow security principles that apply to Internet service providers (ISPs). The corporate network should only accept traffic from outside connections with source addresses other than the corporate network block. In addition, it should only permit outbound traffic with valid corporate network addresses.
Although these may seem to be simple rules to follow, networks are complex beasts and how to apply these rules can vary. It is easy to get confused and go filter crazy or just not deploy them. A simple rule to follow is to at minimum, apply both ingress and egress filters at any router interface that connects to any remote connection, especially the Internet. In addition, for consistency, it is good practice to use the same fixed access list number for ingress and egress filters in your routers. It simplifies troubleshooting when, for example, you know that access list 103 is used for ingress filters and access list 104 is used for egress filters to remote connections. Of course, when routers have multiple remote connections, the consistency scheme may have to be modified. The following examples illustrate some common scenarios and offer recommended ways to apply these rules.
The first example is that of a corporate network illustrated in Figure D-1. Here, the corporate network has been assigned the network address 144.254.0.0/16 and connects two remote offices with network addresses 171.71.32.0/27 and 192.150.42.0/27. The filtering policy and configuration examples follow.
The branch routers are assumed to have no other connection to the Internet, and filters are only applied on the interfaces that connect to the corporate network, in this case BRI 0.
The branch router ingress filtering policy is as follows:
Deny all RFC 1918 and special-use addresses from entering the branch network. (The assumption is made that even if the corporate network is using some private address space, the traffic should stay within the corporate network.)
Deny all traffic with an IP source address that matches the branch network address allocation.
Permit all other traffic.
The branch router egress filtering policy is as follows:
Permit only traffic with an IP source address that matches the branch network.
Deny all other traffic.
The branch A router configuration is displayed in Example D-1.
Example D-1. Branch A Router Configuration
access-list 133 deny ip host 0.0.0.0 any access-list 133 deny ip 127.0.0.0 0.255.255.255 any access-list 133 deny ip 10.0.0.0 0.255.255.255 any access-list 133 deny ip 172.16.0.0 0.15.255.255 any access-list 133 deny ip 192.168.0.0 0.0.255.255 any access-list 133 deny ip 192.0.2.0 0.0.0.255 any access-list 133 deny ip 169.254.0.0 0.0.255.255 any access-list 133 deny ip 240.0.0.0 15.255.255.255 any access-list 133 deny ip 171.71.32.0 0.0.0.31 any access-list 133 permit ip any any access-list 144 permit ip 171.71.32.0 0.0.0.31 any access-list 144 deny ip any any interface BRI0 description To Corporate Network ip access-group 133 in ip access-group 144 out
Filters are placed on only the interfaces that connect to the branch networks, which in this case is a PRI.
The NAS router ingress filtering policy is as follows:
Permit only traffic with an IP source address of branch networks.
Deny all other traffic.
The NAS router egress filtering policy is as follows:
Deny all RFC 1918 and special-use addresses from propagating to branch networks.
Deny all traffic with an IP source address that matches the branch network address allocation.
Permit all other traffic.
The NAS router configuration is displayed in Example D-2.
Example D-2. NAS Router Configuration
access-list 133 permit ip 171.71.32.0 0.0.0.31 any access-list 133 permit ip 192.150.42.0 0.0.0.31 any access-list 133 deny ip any any access-list 144 deny ip host 0.0.0.0 any access-list 144 deny ip 127.0.0.0 0.255.255.255 any access-list 144 deny ip 10.0.0.0 0.255.255.255 any access-list 144 deny ip 172.16.0.0 0.15.255.255 any access-list 144 deny ip 192.168.0.0 0.0.255.255 any access-list 144 deny ip 192.0.2.0 0.0.0.255 any access-list 144 deny ip 169.254.0.0 0.0.255.255 any access-list 144 deny ip 240.0.0.0 15.255.255.255 any access-list 144 deny ip 171.71.32.0 0.0.0.31 any access-list 144 deny ip 192.150.42.0 0.0.0.31 any access-list 133 permit ip any any interface Serial 0:23 description To Branch Offices ip access-group 133 in ip access-group 144 out
Filters are placed only on the interface that connects to the Internet, which in this case is a T3 circuit.
The Internet router ingress filtering policy is as follows:
The Internet router egress filtering policy is as follows:
Permit only traffic with an IP source address of the corporate network and branch networks.
Deny all other traffic.
The Internet router configuration is displayed in Example D-3.
Example D-3. Internet Router Configuration
access-list 133 deny ip host 0.0.0.0 any access-list 133 deny ip 127.0.0.0 0.255.255.255 any access-list 133 deny ip 10.0.0.0 0.255.255.255 any access-list 133 deny ip 172.16.0.0 0.15.255.255 any access-list 133 deny ip 192.168.0.0 0.0.255.255 any access-list 133 deny ip 192.0.2.0 0.0.0.255 any access-list 133 deny ip 169.254.0.0 0.0.255.255 any access-list 133 deny ip 240.0.0.0 15.255.255.255 any access-list 133 deny ip 144.254.0.0 0.0.255.255 any access-list 133 deny ip 171.71.32.0 0.0.0.31 any access-list 133 deny ip 192.150.42.0 0.0.0.31 any access-list 133 permit ip any any access-list 144 permit ip 144.254.0.0 0.0.255.255 any access-list 144 permit ip 171.71.32.0 0.0.0.31 any access-list 144 permit ip 192.150.42.0 0.0.0.31 any access-list 144 deny ip any any interface Serial 0/0 description To Internet ip access-group 133 in ip access-group 144 out
Now look at a more complex example that is illustrated in Figure D-2. Here some complexity is added; each of the branch routers has its own Internet connection, which can act as a redundant path for the corporate network to the Internet. Additionally, the corporate network and branch networks use some private RFC 1918 addressing, which needs to only be available from within the corporate or branch networks. In other words, no private address space traffic should traverse the links connecting the corporate and branch networks.
NOTE
In some scenarios, corporations can coordinate the use of private address space between their corporate and branch sites and may want to allow propagation of these addresses between the corporate and branch networks only. Care must be taken that traffic from these private addresses are not leaked out to other networks, such as the Internet.
The filtering policy and configuration examples follow.
The branch routers will now have to take into consideration that it's a backup connection for the corporate network and possibly the other branch network. Both ingress and egress filters are applied on the interfaces that connect to the corporate network—in this case BRI 0, which connects to the corporate network, and serial 1/0, which connects to the Internet. Because there is more likelihood that spoofed traffic may enter from the Internet connections, the filters should be stricter to ensure that the corporate network will not be affected by a DDoS attack from the Internet using spoofed addresses.
The branch router ingress filtering (connecting to corporate network) policy is as follows:
Deny all RFC 1918 and special-use addresses from entering the branch network.
Deny all traffic with an IP source address that matches the branch network address allocation.
Permit all other traffic.
The branch router egress filtering (connecting to corporate network) policy is as follows:
Deny all RFC 1918 and special-use addresses from propagating to the corporate network.
Deny all traffic with an IP source address that matches the corporate network address allocation.
Permit all other traffic.
The branch router ingress filtering (connecting to Internet) policy is as follows. (Note that this is the same as ingress filter above.)
Deny all RFC 1918 and special-use addresses from entering the branch network.
Deny all traffic with an IP source address that matches the branch network address allocation.
Permit all other traffic.
The branch router egress filtering (connecting to Internet) policy is as follows:
Permit only traffic with an IP source address that matches the corporate and branch network allocations.
Deny all other traffic.
The branch A router configuration is displayed in Example D-4.
Example D-4. Branch A Configuration (Advanced Filtering)
access-list 133 deny ip host 0.0.0.0 any access-list 133 deny ip 127.0.0.0 0.255.255.255 any access-list 133 deny ip 10.0.0.0 0.255.255.255 any access-list 133 deny ip 172.16.0.0 0.15.255.255 any access-list 133 deny ip 192.168.0.0 0.0.255.255 any access-list 133 deny ip 192.0.2.0 0.0.0.255 any access-list 133 deny ip 169.254.0.0 0.0.255.255 any access-list 133 deny ip 240.0.0.0 15.255.255.255 any access-list 133 deny ip 171.71.32.0 0.0.0.31 any access-list 133 permit ip any any access-list 144 deny ip host 0.0.0.0 any access-list 144 deny ip 127.0.0.0 0.255.255.255 any access-list 144 deny ip 10.0.0.0 0.255.255.255 any access-list 144 deny ip 172.16.0.0 0.15.255.255 any access-list 144 deny ip 192.168.0.0 0.0.255.255 any access-list 144 deny ip 192.0.2.0 0.0.0.255 any access-list 144 deny ip 169.254.0.0 0.0.255.255 any access-list 144 deny ip 240.0.0.0 15.255.255.255 any access-list 144 deny ip 144.254.0.0 0.0.255.255 any access-list 144 permit ip any any access-list 166 permit ip 171.71.32.0 0.0.0.31 any access-list 166 permit ip 144.254.0.0 0.0.255.255 any access-list 166 deny ip any any interface BRI0 description To Corporate Network ip access-group 133 in ip access-group 144 out interface Serial 1/0 description To Internet ip access-group 133 in ip access-group 166 out
Filters are placed on only the interfaces that connect to the branch networks. Because you now must allow potential traffic other than that sourced by the branch network allocation itself to enter the corporate network, there is more likelihood that spoofed traffic may enter from the branch connections. Therefore, the filters should be stricter to ensure that the corporate network will not be affected by a DDoS attack propagating from the branch networks using spoofed addresses.
The NAS router ingress filtering policy is as follows:
Deny all RFC 1918 and special-use addresses from entering the corporate networks.
Deny all traffic with an IP source address that matches the corporate network address allocation.
Permit all other traffic.
The NAS router egress filtering policy is as follows:
Deny all RFC 1918 and special-use addresses from propagating to branch networks.
Deny all traffic with an IP source address that matches the corporate network address allocation.
Permit all other traffic.
The NAS router configuration is displayed in Example D-5.
Example D-5. NAS Router Configuration (Advanced Filtering)
access-list 133 deny ip host 0.0.0.0 any access-list 133 deny ip 127.0.0.0 0.255.255.255 any access-list 133 deny ip 10.0.0.0 0.255.255.255 any access-list 133 deny ip 172.16.0.0 0.15.255.255 any access-list 133 deny ip 192.168.0.0 0.0.255.255 any access-list 133 deny ip 192.0.2.0 0.0.0.255 any access-list 133 deny ip 169.254.0.0 0.0.255.255 any access-list 133 deny ip 240.0.0.0 15.255.255.255 any access-list 133 deny ip 171.71.32.0 0.0.0.31 any access-list 133 permit ip any any access-list 144 deny ip host 0.0.0.0 any access-list 144 deny ip 127.0.0.0 0.255.255.255 any access-list 144 deny ip 10.0.0.0 0.255.255.255 any access-list 144 deny ip 172.16.0.0 0.15.255.255 any access-list 144 deny ip 192.168.0.0 0.0.255.255 any access-list 144 deny ip 192.0.2.0 0.0.0.255 any access-list 144 deny ip 169.254.0.0 0.0.255.255 any access-list 144 deny ip 240.0.0.0 15.255.255.255 any access-list 144 deny ip 144.254.0.0 0.0.255.255 any access-list 144 permit ip any any interface Serial 0:23 description To Branch Offices ip access-group 133 in ip access-group 144 out
Filters are placed only on the interface that connects to the Internet. Note that the ingress filter differs from the first example in that it now must not deny traffic that may have IP source addresses from the branch networks. This is because the branch office-to-corporate network path via the Internet provides a level of redundancy in the event that the direct path fails.
The Internet router ingress filtering policy is as follows:
Deny all RFC 1918 and special-use addresses from entering the corporate network.
Deny all traffic with an IP source address of the corporate network.
Permit all other traffic.
The Internet router egress filtering policy is as follows:
Permit only traffic with an IP source address of the corporate network and branch networks.
Deny all other traffic.
The Internet router configuration is displayed in Example D-6.
Example D-6. Internet Router Configuration (Advanced Filtering)
access-list 133 deny ip host 0.0.0.0 any access-list 133 deny ip 127.0.0.0 0.255.255.255 any access-list 133 deny ip 10.0.0.0 0.255.255.255 any access-list 133 deny ip 172.16.0.0 0.15.255.255 any access-list 133 deny ip 192.168.0.0 0.0.255.255 any access-list 133 deny ip 192.0.2.0 0.0.0.255 any access-list 133 deny ip 169.254.0.0 0.0.255.255 any access-list 133 deny ip 240.0.0.0 15.255.255.255 any access-list 133 deny ip 144.254.0.0 0.0.255.255 any access-list 133 permit ip any any access-list 144 permit ip 144.254.0.0 0.0.255.255 any access-list 144 permit ip 171.71.32.0 0.0.0.31 any access-list 144 permit ip 192.150.42.0 0.0.0.31 any access-list 144 deny ip any any interface Serial 0/0 description To Internet ip access-group 133 in ip access-group 144 out
NOTE
Filtering is a fairly complex issue due to the variety of ways you can deploy filters. It is important to keep the maintenance of these filters as low as possible and to optimize the configurations for your environment. Ingress/egress filtering needs to be deployed because it facilitates the DoS originator to be easily traced to its true source, because the attacker would have to use a valid, and legitimately reachable, source address.
A number of routers in the market today have features that enable you to limit the amount of bandwidth some type of traffic can consume. Sometimes this rate limiting is also referred to as “traffic shaping,” and typically a vendor will allow you to enforce a bandwidth policy against network traffic that matches an access list.
You can use this proactively to create access list rules that match some of the network traffic used by the DDoS attack. If the attack is using Internet Control Message Protocol (ICMP) packets (as in Smurf attacks) or TCP SYN packets, for example, you could configure the system to specifically limit the bandwidth those types of packets are allowed to consume. This will allow some of these packets that may belong to legitimate network flows (such as datagrams being exchanged from established TCP microflows) to go through.
For Cisco routers, the committed access rate (CAR) functionality works with Cisco Express Forwarding (CEF), found in Cisco IOS Software Release 11.1CC and onward from 12.0. It allows network operators to rate limit certain types of traffic to specific sources or destinations. The main advantage of CAR is that it acts on packets as they arrive on a router's interface, dropping or rate limiting the DoS flow before any packet processing occurs.
The following two examples show how you can use CAR to rate limit ICMP packets and TCP SYN packets, two of the more commonly found DoS attacks.
In this example, a corporation chooses to rate limit all ICMP echo and echo-reply packets received at its 2-Mbps connection from the Internet. The ICMP echo and echo-reply traffic is limited to 256 kbps, with a maximum burst size of 8000 bytes; all packets exceeding this rate are dropped. Multiple rate-limit commands can be added to an interface to control other kinds of traffic as well. Example D-7 shows the configuration commands for the previously described scenario.
In this example, a corporation uses CAR to limit TCP SYN floods to specified servers. Typically these would be critical servers, and in this example they reside on the 144.254.2.0 network. The TCP SYN traffic is limited to 8 kbps, with a maximum burst size of 8000 bytes; all packets exceeding this rate are dropped. Multiple rate-limit commands can be added to an interface to control other kinds of traffic as well. Note that the first access list entry ensures that established TCP sessions are not limited. Example D-8 shows the configuration commands for the previously described scenario.
Example D-8. Configuration for TCP SYN Traffic Rate Limiting
access-list 108 deny tcp any 144.254.2.0 0.0.0.255 established
access-list 108 permit tcp any 144.254.2.0 0.0.0.255
interface Serial 1/1
rate-limit input access-group 108 8000 8000 8000 conform-action transmit
exceed-action drop
NOTE
It is recommended that you first measure the number of SYN packets during normal state (before attacks occur) and use those values to set the burst size. If you set the burst size too low, many legitimate SYNs may be dropped. During an attack, use the show interfaces rate-limit command to display the conformed and exceeded rates for the interface.
A routing based filtering feature called IP unicast reverse-path forwarding (uRPF) can also be used to prevent most forms of IP address spoofing. This feature examines each packet received as input on that interface. If the source IP address does not have a route in the routing table that points back to the same interface on which the packet arrived, the router drops the packet.
To use uRPF, CEF switching needs to be turned on in the router (CEF distributed switching for an RSP+VIP-based router). CEF is an enhanced Layer 3 IP switching technology in Cisco routers that avoids the potential overhead of continuous cache churn by instead using a Forwarding Information Base (FIB) for the destination switching decision. The FIB mirrors the entire contents of the IP routing table—that is, there is a one-to-one correspondence between FIB table entries and routing table prefixes; therefore no need to maintain a route cache.
There is no need to configure the input interface for CEF switching. This is because uRPF has been implemented to search through the FIB using the source IP address. As long as CEF is running on the router, individual interfaces can be configured with other switching modes. RPF is an input-side function that is enabled on an interface or subinterface and operates on packets received by the router.
Cisco IOS Software Release 12.1 and later includes an enhancement to uRPF that allows access lists to be applied to work in conjunction with uRPF. When a packet fails reverse path verification, the access list is applied. If the access list permits the packet, it is forwarded. If the access list denies the packet, it is dropped. In this manner, the access lists enable you to create exceptions to uRPF's usual functionality. In addition, it provides for a logging mechanism for packets that do not pass the uRPF check.
uRPF is not supported in any 11.2 or 11.3 images. uRPF is included in Cisco IOS Software 12.0 on all platforms that support CEF. CEF-supported platforms include the Cisco 7X00 series routers equipped with RSP7000, 7200 series, 7500 series, as well as the 12000 series, and AS5800.
Example D-9 illustrates a simple configuration that you can use on a router to use uRPF. It is useful to deploy this in scenarios such as leased-line aggregation routers or on dialup ports on access server gateways such as the AS5800 where maintenance of access lists could be a major headache.
Even with the proactive measurements that corporate networks take to mitigate DDoS attacks, because new attacks are constantly creeping up you must be ready to provide countermeasures during a new DDoS attack. When a potential new DDoS attack occurs, you need to be able to have the assistance and cooperation from your direct uplink network providers. Talk to your uplink providers and get the appropriate contact information for people who can help you with implementing routing access control that limits the amount of bandwidth and different source addresses that are let through to your network. Also, it is important to start countermeasures as soon as possible. Begin the backtracking of packets as soon as possible, and contact your uplink providers, when traces indicate that the DoS attack traffic came over their networks. Don't rely on the source addresses, because they can be arbitrarily chosen. (Of course, if everyone implemented appropriate filtering techniques in their corporate networks, the spoofed-address DoS attacks from invalid network addresses would be greatly minimized.) The overall effort of being able to determine origins of spoofed DoS attacks depends on your quick action, because the router entries that allow traffic backtracking will expire a short time after the flood is halted.
If possible, capture packet samples for analysis. The best place to capture packet samples is at the edges of your network where there is a demarcation from a trusted network to a less-trusted network. It is recommended that you use a SUN workstation or a Linux box on a fast Pentium machine to capture the packet sample. For capturing, use the TCP dump program (Linux or SUN) or snoop (SUN Solaris only). The command syntax is as follows:
tcpdump -i interface -s 1500 -w capture_file snoop -d interface -o capture_file -s 1500
The maximum transmission unit (MTU) size in this example is 1500; change this parameter if the MTU is greater than 1500.
Preserve these logs as evidence for law enforcement.
If you want to involve law enforcement and you are within the United States, contact your local FBI field office. More information is available at the http://www.nipc.gov/ website. If you are located in Europe, no single point of contact exists. Contact your local law enforcement agency and ask for assistance.
The packets in many DoS attack streams can be isolated by matching them against filtering (access list) entries. This is obviously valuable for filtering out attacks, but is also useful for characterizing unknown attacks, and for tracing “spoofed” packet streams back to their real sources, if valid source addresses are used. Access lists and access list logging are the most useful features for characterizing and tracing common attacks.
The source addresses of DoS packets are almost always set to values that have nothing to do with the attackers themselves, and are therefore of no use in identifying the attackers. The only reliable way to identify the source of an attack is to trace it back hop by hop through the network. This process involves reconfiguring routers and examining log information, and therefore requires cooperation by all network operators along the path from the attacker to the victim. Securing that cooperation usually requires the involvement of law enforcement agencies, which must also be involved if any action is to be taken against the attacker.
The tracing process for DoS floods is relatively simple. Starting at a router (call it “A”) that's known to be carrying flood traffic, one identifies the router (call it “B”) from which A is receiving the traffic. One then logs into B, and finds the router (“C”) from which B is receiving the traffic. This continues until the ultimate source is found. Note that it is often the case that more than one upstream router propagates attack traffic, so multiple upstream sources may need to be identified.
There are several complications in this method:
The “ultimate source” may in fact be a computer that has been compromised by the attacker, but which is actually owned and operated by another victim. In this case, tracing the DoS flood will only be the first step.
Attackers know that they can be traced and will usually continue their attacks only for a limited time; there may not be enough time to actually trace the flood.
Attacks may be coming from multiple sources, especially if the attacker is relatively sophisticated. It's important to try to identify as many sources as possible.
Communication problems slow down the tracing process. Frequently one or more of the network operators involved will not have appropriately skilled staff available. It is therefore important to first identify all network ingress points associated with the attack, mitigate the attack there as seen appropriate, and then (or in parallel) contact concerned upstream service providers to begin tracing the attack further upstream.
Legal and political concerns may make it difficult to act against attackers even if one is found.
The fact is that most efforts to trace DoS attacks fail. Because of this, many network operators will not even attempt to trace an attack unless placed under pressure. Many others trace only “severe” attacks, with differing definitions of what is “severe.” Some assist with a trace only if law enforcement is involved.
If you choose to trace an attack passing through a Cisco router, the most effective way of doing so is to construct an access list entry that matches the attack traffic, attach the log-input keyword to it, and apply the access list outbound on the interface through which the attack stream is being sent toward its ultimate target. The log entries produced by the access list will identify the router interface through which the traffic is arriving, and, if the interface is a multipoint connection, provide the Layer 2 address of the device from which it is being received. You can then use the Layer 2 address to identify the next router in the chain, using, for example, the show ip arp mac-address command.
To trace a SYN flood, you might create an access list similar to the following:
access-list 169 permit tcp any any established access-list 169 permit tcp any host victim-host log-input access-list 169 permit ip any any
This access list will log all SYN packets destined for the target host, including legitimate SYNs. To identify the most likely actual path toward the attacker, examine the log entries in detail. In general, the source of the flood will likely be the source from which the largest number of matching packets are arriving. Remember that the source IP addresses themselves may mean nothing; you're looking for source interfaces and source MAC addresses. Sometimes it's possible to distinguish flood packets from legitimate packets because flood packets may have invalid source addresses; any packet whose source address is not valid is likely to be part of the flood.
Remember that the flood may be coming from multiple sources—distributed SYN floods are a major concern to many large corporations.
To trace a Smurf attack stream, use an access list like this:
access-list 169 permit icmp any any echo log-input access-list 169 permit ip any any
Note that the first entry doesn't restrict itself to packets destined for the reflector address. The reason for this is that most Smurf attacks use multiple reflector networks. If you're not in contact with the ultimate target, you may not know all the reflector addresses. As your trace gets closer to the source of the attack, you may begin to see echo requests going to more and more destinations; this is a good sign.
If you're dealing with a great deal of ICMP traffic, however, this may generate too much logging information for you to read easily. If this happens, you can restrict the destination address to be one of the reflectors that's known to be used. Another useful tactic is to use an entry that takes advantage of the fact that netmasks of 255.255.255.0 are very common in the Internet. And, because of the way that attackers find Smurf reflectors, the reflector addresses actually used for Smurf attacks are even more likely to match that mask. Host addresses ending in .0 or .255 are very uncommon in the Internet, so you can build a relatively specific recognizer for Smurf stimulus streams like this:
access-list 169 permit icmp any host known-reflector echo log-input access-list 169 permit icmp any 0.0.0.255 255.255.255.0 echo log-input access-list 169 permit icmp any 0.0.0.0 255.255.255.0 echo log-input access-list 169 permit ip any any
With this list, you can eliminate many of the “noise” packets from your log, while still having a good chance of noticing additional stimulus streams as you get closer to the attacker.
The log-input keyword exists in Cisco IOS Software versions 11.2 and later, and in certain 11.1-based software created specifically for the service provider market. Older software does not support this keyword. If you're using a router with older software, you have three viable options:
Create an access list without logging, but with entries that match the suspect traffic. Apply the list on the input side of each interface in turn and watch the counters. Look for interfaces with high match rates. This method has a very small performance overhead and is good for identifying source interfaces. Its biggest drawback is that it doesn't give link-layer source addresses and is therefore useful mostly for point-to-point lines.
Create access list entries with the log keyword (as opposed to log-input). Once again, apply the list to the incoming side of each interface in turn. This method still doesn't give source MAC addresses, but can be useful for seeing IP data—for instance, to verify that a packet stream really is part of an attack. Performance impact can be moderate to high; newer software performs better than older software.
Use debug ip packet detail to collect information about packets. This method gives MAC addresses, but can have serious performance impact. It's easy to make a mistake with this method and make a router unusable. If you use this method, make sure that the router is switching the attack traffic in fast, autonomous, or optimum mode. Use an access list to restrict debugging to only the information you really need. Log debugging information to the local log buffer, but turn off logging of debug information to Telnet sessions and to the console. If possible, arrange for someone to be physically near the router, or access the router via remote out-of-band console port access, so that it can be power cycled as necessary.
Remember that debug ip packet will not display information about fast-switched packets; you will need to do a clear ip cache to capture information. Each clear command will give you one or two packets of debug output. Also note that this option applies primarily to software-based platforms.
Remember that the counter on an access list entry counts all matches against that entry. If you apply an access list to two interfaces, the counts you'll see will be aggregate counts.
Access list logging does not show every packet that matches an entry. Logging is rate limited to avoid CPU overload. What logging shows you is a reasonably representative sample, but not a complete packet trace. Remember that there are packets you are not seeing.
In some software versions, access list logging works only in certain switching modes. If an access list entry counts a lot of matches, but logging nothing, try to clear the route cache to force packets to be process switched. Be careful about doing this on heavily loaded routers with many interfaces; a lot of traffic can get dropped while the cache is rebuilt. Use CEF whenever possible.
Access lists and logging have a performance impact, but not a large one. Be careful on routers running at more than about 80-percent CPU load, or when applying access lists to very high-speed interfaces.
NOTE
The material in this section was written by Rob Thomas and is reprinted here with his permission. You can find this information and ongoing updates from Rob on the web at http://www.cymru.com/Documents/dos-and-vip.html. The document has been modified slightly here to conform to Cisco Press book presentation.
DoS attacks have become almost ubiquitous. While these attacks are often easily and quickly mitigated through the use of filters on the border routers, these filters can prevent the proper analysis of the attack by obfuscating the attack and its data. For example, an ICMP attack against a given host, from a range of source IP addresses, can be quickly blocked by an ACL [access control list] that blocks all ICMP to the victim host. However, this leaves only ACL counters as an indicator of the duration and intensity of the attack. Further, the ACL (presuming it does not include the log-input keyword) does not reveal the source IP addresses of the attack. Have the source addresses changed? Are there additional hosts joining the attack? Which source is producing the bulk of the packets? These are valid questions, and the data provided by ACL counters will not provide the answers. Clearly there must be a better way. With a Cisco VIP card and NetFlow, it is possible to both block and monitor the attack.
The Cisco VIP (Versatile Interface Processor) is a “router on a card.” It contains a CPU, memory, an IOS image, and all of the necessary structures to perform packet switching. Often used in the Cisco 7513 router, the VIP improves performance by removing the burden of packet processing, in many cases, from the RSP (Route Switch Processor). Because the VIP is a “router on a card” that runs an IOS image, it also has many of the same commands and data structures as the RSP.
NetFlow is a data collection method in Cisco routers that allows router support personnel to trace the flows that pass through the router. NetFlow is often used for performance and trend analysis as well as billing.
Through the use of the undocumented VIP console and NetFlow, a DoS attack may be closely monitored and analyzed. The VIP console can also be used to perform the more routine and mundane router performance checks, such as show proc cpu.
This method has been tested, in the lab and in production (during actual DoS attacks), on a Cisco 7513 router with the VIP2-50 line card.
Note that methods exist to secure a border router prior to an attack. DoS mitigation methods and NetFlow configuration can be reviewed in the Secure IOS Template (http://www.cymru.com/Documents/secure-ios-template.html). If BGP is used, there are additional defense methods available as documented in the Secure BGP Template (http://www.cymru.com/Documents/secure-bgp-template.html).
The if-con command used to access the VIP console is undocumented and unsupported by Cisco. Cisco will not assist anyone with the use (or misuse) of this command. While this methodology has been fully “battle tested” on both lab and production equipment, and has been successfully utilized during several DoS attacks, your mileage may vary. The author is not responsible for any damage caused by the use of this command. The author also won't take undue credit for any benefit derived from the use of this command.
My thanks to the following for input, reviews, and suggestions.
Angelito Basa
David Bergum
Ton Schoenmakers
The syntax for the if-con command is if-con slot, where slot is the chassis slot location in which the VIP is inserted. The if-con command can only be run from enable mode, and there is no online help available for it. This command will prompt for the mode of access, either console or debug. Unless you have access to the IOS source code and a tolerance for sudden router reloads, you will likely not benefit from entering debug mode. Here is an example:
secure-router01>if-con 9
Console or Debug [C]: [PRESS ENTER HERE]
Entering CONSOLE for VIP2 R5K 9
Type "^C^C^C" or "if-quit" to end this session
[PRESS ENTER HERE]
VIP-Slot9>
Note the prompt: VIP-Slot9>. This indicates that the user is on the VIP console for the VIP in slot 9 of the chassis. This makes it quite clear that this is not the RSP console. To exit the VIP console, the syntax is simply if-quit:
VIP-Slot9>if-quit
Disconnecting from slot 9 CONSOLE after 00:02:12
secure-router01>
The prompt has returned to the expected RSP CLI prompt, and the amount of time spent in the VIP console mode is noted.
Once in the VIP console, many of the RSP CLI commands will work as expected. This, combined with NetFlow, is what makes DoS monitoring possible.
To illustrate the benefit of the VIP console during a DoS attack, presume the following scenario: A single host, IP address 1.1.1.1, is the target of a DDoS attack. The attack consists of a high rate of small UDP packets to an unused port. The source IP addresses of the attack are far too numerous, dynamic, and disparate to block with black hole routes or selective ACLs. The target host is also suffering a performance degradation due to the rate of packets flowing to the host. It is decided to block all UDP to this host with a simple ACL:
access-list 105 remark Temporary ACL to block the DDoS attack against 1.1.1.1 – 20 May 2001 robt access-list 105 deny udp any host 1.1.1.1 access-list 105 permit ip any any
Due to the high rate of packets, it is decided not to log the ACL, as this might negatively impact the performance of the router and syslog server. This ACL is applied to the external interfaces of the router, which are located on the VIP in slot 9. The external interfaces are HSSI interfaces, HSSI9/0/0 and HSSI9/1/0. The victim host is saved. However, this makes tracing the attack difficult. What source IP addresses are used in the attack? Are new source addresses entering the fray? Which source addresses are the key contributors? While ACL counters can be utilized to determine if the attack is still raging, it can be quite difficult to determine the intensity of the attack, as well as the contribution to the attack of each participating host. Fortunately, the border router utilizes VIPs and NetFlow, thus the data on the attack is still available. We move to the VIP console to learn more:
secure-router01>if-con 9 Console or Debug [C]: [PRESS ENTER HERE] Entering CONSOLE for VIP2 R5K 9 Type "^C^C^C" or "if-quit" to end this session [PRESS ENTER HERE] VIP-Slot9>
While ACL 105 has been applied to the two interfaces located on this VIP, the VIP actually routes any denied packets to the Null interface. NetFlow caches these flows for a short time, allowing a user to monitor them and extract copious details about them. Note that these flows are routed to the VIP Null interface, and are not sent to the RSP. Thus, the NetFlow cache on the RSP does not contain any of these flows and cannot access this data.
The show proc cpu command can be used to determine if the VIP is still heavily utilized by the attacking packets. To wit:
VIP-Slot9>sh proc cpu
CPU utilization for five seconds: 80%/80%; one minute: 80%; five minutes: 70%
[ Output truncated. ]
Based on the CPU utilization statistics, it is clear that the VIP is spending most of its time switching packets. The first number in the 80%/80% entry indicates the CPU utilization. The second number indicates what percentage of CPU utilization is due to network interrupts—in other words, packet processing. Next, NetFlow is queried to see what sort of traffic is hitting this VIP. This is done with the show ip cache flow command, with referenced points shaded:
secure-router01#if-con 9 Console or Debug [C]: Entering CONSOLE for VIP2 R5K 9 Type "^C^C^C" or "if-quit" to end this session VIP-Slot9>show ip cache flow IP packet size distribution (489639251 total packets): 1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480 .000 .992 .000 .003 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 512 544 576 1024 1536 2048 2560 3072 3584 4096 4608 .000 .000 .000 .000 .003 .000 .000 .000 .000 .000 .000 IP Flow Switching Cache, 8913408 bytes 5088 active, 125984 inactive, 1843766371 added 805412120 ager polls, 0 flow alloc failures Active flows timeout in 30 minutes Inactive flows timeout in 15 seconds last clearing of statistics never Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec) -------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow TCP-Telnet 28084 0.0 1 45 0.0 0.1 11.7 TCP-FTP 172835 0.0 1 47 0.0 2.4 13.7 TCP-FTPD 2818 0.0 1 40 0.0 0.2 11.3 TCP-WWW 5551226 1.2 1 53 1.3 0.1 5.0 TCP-SMTP 4179 0.0 1 42 0.0 1.0 12.2 TCP-X 2594 0.0 1 40 0.0 0.6 11.2 TCP-BGP 2546 0.0 1 40 0.0 0.2 11.5 TCP-NNTP 2554 0.0 1 40 0.0 0.1 11.2 TCP-Frag 177 0.0 2 269 0.0 1.7 16.8 TCP-other 528636 0.1 1 40 65.5 0.6 35.5 UDP-DNS 11596 0.0 1 54 0.0 0.8 17.2 UDP-NTP 723 0.0 2 40 0.0 9.0 16.8 UDP-TFTP 763 0.0 3 37 0.0 10.2 16.9 UDP-Frag 25 0.0 1 40 0.0 251.4 15.0 UDP-other 169720402 39.5 1 40 46.2 0.6 11.3 ICMP 275131 0.0 10 759 0.6 7.7 14.2 IGMP 36 0.0 1789 1246 0.0 15.2 16.9 IP-other 7 0.0 19 64 0.0 18.9 17.5 Total: 176304332 41.0 2 44 113.9 0.6 11.2 SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts Hs9/1/0 192.168.2.51 Null 1.1.1.1 11 04A9 0017 614K Hs9/1/0 192.168.47.72 Null 1.1.1.1 11 05F9 0017 281K Hs9/1/0 192.168.49.52 Null 1.1.1.1 11 08EA 0017 65K Hs9/1/0 192.168.32.18 Null 1.1.1.1 11 08EC 0017 1463K Hs9/1/0 192.168.208.208 Null 1.1.1.1 11 0411 0017 8351K Hs9/1/0 192.168.77.66 Null 1.1.1.1 11 126F 0017 1763K Hs9/1/0 192.168.184.159 Null 1.1.1.1 11 0609 0017 191K Hs9/1/0 192.168.22.48 Null 1.1.1.1 11 0885 0017 1520K Hs9/1/0 192.168.22.48 Null 1.1.1.1 11 0883 0017 66K Hs9/1/0 192.168.7.44 Null 1.1.1.1 11 0F07 0017 97K Hs9/1/0 192.168.7.44 Null 1.1.1.1 11 0F09 0017 2084K Hs9/1/0 192.168.54.208 Null 1.1.1.1 11 040C 0017 3018K Hs9/1/0 192.168.248.90 Null 1.1.1.1 11 0521 0017 201K Hs9/1/0 192.168.201.177 Null 1.1.1.1 11 060C 0017 171K Hs9/1/0 192.168.201.177 Null 1.1.1.1 11 054C 0017 107K [ Output truncated. ] VIP-Slot9>if-quit Disconnecting from slot 9 CONSOLE after 00:03:25 secure-router01#
The host 1.1.1.1 is a web server, so web traffic (TCP-WWW) is expected. However, this site does not send or receive any UDP-based data, so the UDP-other entry is quite troublesome. We see very few entries for UDP fragments, so this does not appear to be a fragment attack. As noted in the 64 column of the IP packet size distribution stanza, 99.2% (.992) of the packets are 64 bytes in size. The Bytes/Pkt column of the UDP-other entry reveals that the packets are all 40 bytes in size (not counting IP header information). Thus, this is an attack of many small packets. This is the first indicator of woe, and requires further analysis.
The individual flows provide the granular detail necessary for proper analysis. Note that the listed IP addresses are directing a high rate of packets to the host 1.1.1.1. The entries indicate that the flows are entering the site through the HSSI 9/1/0 interface (SrcIf). None of the flows are entering through the HSSI9/0/0 interface. Due to ACL 105, the flows are sent to the Null interface (DstIf). The destination address is the target host, 1.1.1.1 (DstIPaddress).
The next three columns are all in hexadecimal. The protocol (Pr) is 11, which equates to decimal 17 – UDP. The source ports (SrcP) are varied. The destination port (DstP) is a consistent 17, which equates to decimal 23. UDP port 23 is an unused port on this (and most) system. Finally, we know the number of packets (Pkts) each host has generated in the poll interval. These are quite high.
So what have we learned? We know that this attack is UDP-based. We know that the packets are not fragments, nor do they claim to be fragments. We know that the majority of the packets are 64 bytes in length. We know of several source addresses used in the attack, and these are likely spoofed based on the address type (RFC 1918). These can be used in an effort to trace the source (see the “Tracking Spoofed IP Addresses” article [http://www.cymru.com/Documents/tracking-spoofed.html]). We know the source interface (HSSI9/1/0), which can be traced to an upstream router or provider. This greatly narrows the field when attempting to uncover the source of the attack. Since the attacks are all coming from the same interface, it is thus possible that the sources aren't so very disparate after all. Perhaps they are even in the same netblock or company! We know the intensity of the attack based on the packet counts. We can now monitor the duration and intensity of the attack.
In short, we have collected enough data to pursue the sources of the attack, involve the upstream ISPs in the pursuit, and possibly modify our ACL to accommodate the specific target ports.
NOTE
The material in this section was written by Rob Thomas and is reprinted here with his permission. You can find this information and continual updates from Rob on the web at http://www.cymru.com/Documents/tracking-spoofed.html. The document has been modified slightly here to conform to Cisco Press book presentation.
Tracking spoofed IP addresses back to the source can be quite a difficult task. For myriad reasons, such as limited router access, attacks of a short duration, and the manual nature of spoofed address tracking, finding the actual generator of the spoofed packets can be very difficult. For this reason, attackers often use the bogon address ranges, where a bogon address range is any unassigned and likely unrouted (by BGP4 in the Internet) netblock. This includes the RFC 1918 addresses as well as a collection of other address spaces, such as 1/8, 169.254/16, and the like.
However, with a certain combination of features enabled on a Cisco router, it is possible to determine the source of the spoofed packets. Further, this can be done without the laborious and CPU-intensive task of adding ACLs to filter the spoofed packets. The key features are CEF and NetFlow.
NOTE
Please be aware that router resources are never infinite. Both CEF and NetFlow require resources from the router, and therefore are not entirely immune to issues. NetFlow exports, in particular, may heavily load both the router and the export interface. Please take the time to test your configuration prior to deploying it in production.
While this document details a method for tracking the source of a DDoS attack that utilizes spoofed IP addresses, there are several other documents that detail methods of mitigating DDoS attacks. You can view my Secure IOS Template (http://www.cymru.com/Documents/secure-ios-template.html) and my Secure BGP Template (http://www.cymru.com/Documents/secure-bgp-template.html) to enhance your router and peering security. There are also several efforts currently underway to block DDoS attacks. Here are a few informative links. (Thanks to John Kristoff for passing these along.)
Pushback:
Traceback:
CenterTrack:
Most high-end Cisco routers on the Internet run either CEF or dCEF. This is because of the large performance gains to be realized with CEF, which stands for Cisco Express Forwarding. CEF has many benefits over fast switching, including a more reliable and sturdy method for building the forwarding table. CEF also offers some security benefits, such as RPF (reverse path forwarding). RPF provides a means of blocking packets that claim to originate from within your network, but present themselves on an external interface. Keep in mind that CEF can be a bit tricky to configure in an environment that has asymmetric data flows. You may wish to review the Cisco CEF white paper (http://www.cisco.com/warp/public/cc/pd/iosw/iore/tech/cef_wp.htm). CEF is, therefore, a wise choice for reasons of performance and security. CEF is enabled on a global basis with the command ip cef. To enable dCEF (Distributed CEF), the global command is ip cef distributed.
NetFlow provides a means of mapping traffic flows through a router. This can be of great use for capacity planning, statistical analysis of traffic patterns, and security reviews. Here is a sample of the output from NetFlow:
router1#sh ip cache flow
IP packet size distribution (11319 total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .016 .002 .002 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.000 .000 .000 .000 .976 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 278544 bytes
1 active, 4095 inactive, 19 added
1909 ager polls, 0 flow alloc failures
last clearing of statistics never
Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)
-------- Flows /Sec /Flow /Pkt /Sec /Flow /Flow
TCP-Telnet 1 0.0 204 47 0.0 71.5 1.3
UDP-other 7 0.0 3 627 0.0 8.4 15.3
ICMP 10 0.0 5 91 0.0 4.1 15.4
Total: 18 0.0 15 103 0.0 9.5 14.6
SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts
Se1 192.168.88.5 Et0 192.168.77.2 11 0013 0007 31
From NetFlow, we can determine our packet distribution, protocol distribution, and current flows. Clearly this is valuable data. Enabling NetFlow is done on a per-interface basis with the command ip route-cache flow.
Once both CEF (or dCEF) and NetFlow are enabled on the router, we are ready to begin hunting the source of a spoofed IP address attack. It is recommended that the routers run Cisco IOS 12.0 or better.
In the example scenario, a malicious user on the host sweatpants, IP address 192.168.97.2/24, wishes to flood the host spanky, IP address 192.168.77.2/24, with copious amounts of bogus UDP traffic. To avoid being caught, the miscreant on sweatpants has decided to spoof his address to be 96.170.4.8. The miscreant knows that the entire 96/8 netblock is unassigned, and therefore any packets destined for this network will not arrive at an actual site. The spoofed packets are all destined for UDP port 7, the echo port. The source port is UDP port 19, the chargen port.
The network topology is displayed in Figure D-3.
The routing is configured thusly:
Spanky
Default route, gateway 192.168.77.1 (router1)
Router 1
Default route, gateway 10.10.10.2 (router2)
Router 2
Static route 192.168.77.0/24, gateway 10.10.10.1 (router1)
Static route 192.168.97.0/24, gateway 172.17.50.1 (router3)
Router 3
Default route, gateway 172.17.50.2 (router2)
Static route 192.168.97.0/24, gateway 10.222.88.144 (router4)
Router 4
Default route, gateway 10.222.88.129 (router3)
Router 5
Default route, gateway 10.222.88.1 (router3)
Sweatpants
Default route, gateway 192.168.97.1 (router4)
While static routing was used for this experiment, the experiment is not fundamentally changed by the use of a dynamic routing protocol, such as OSPF [Open Shortest Path First] or EIGRP [Enhanced Interior Gateway Routing Protocol]. While the responses to the spoofed address from spanky might be dropped sooner in the path, the result is the same—the spoofed packets never make it back to sweatpants, the source of the malevolent data stream. It is not uncommon to find the use of default routes for networks that are singly attached to the Internet.
Using a packet generator, the attack is launched from sweatpants against spanky. A steady stream of spoofed packets now present themselves on the network interface of Spanky. Due to the interrupt saturation and higher than normal CPU load, the attack is detected by the system administrator. The use of the snoop tool (Solaris-specific packet sniffer) determines the source IP of the attack, 96.170.4.8. The network and security teams are alerted. After the source IP address (96.170.4.8) and source port (UDP 19) are noted from the output of snoop, the first step is to log in to the border router, router1, and take a look.
In this topology, it may seem quite obvious that the source of the spoofed packets, from the perspective of router1, must be the serial interface leading to router2. However, it is wise to validate this assumption to ensure that the source of the spoofed attack is not a host within the same subnet as Spanky. First, the NetFlow cache is queried thusly:
router1#sh ip cache flow | include 96.170 Se1 96.170.4.8 Et0 192.168.77.2 11 0013 0007 159
Here we see that the source interface of the flow, which is listed in column one, is Serial1. So it has been determined that the source is somewhere beyond the border router. Next, CEF is queried. CEF inserts all active sources, on a per-interface basis, in its tables:
router1#sh ip cef se1
Prefix Next Hop Interface
0.0.0.0/0 10.10.10.2 Serial1
10.10.10.0/30 attached Serial1
Here it is seen that the only next hop, according to the CEF cache, is 10.10.10.2. Consulting the topology above, it is noted that the next-hop IP address is router2. The search moves one hop further, to router2.
The process is repeated on router2. First, a check of the NetFlow cache:
router2#sh ip cache flow | include 96.170 Se0 96.170.4.8 Se1 192.168.77.2 11 0013 0007 299
The source interface of the flow is Serial0. Now for a check of the CEF tables:
router2#sh ip cef se0
Prefix Next Hop Interface
172.17.50.0/30 attached Serial0
192.168.97.0/24 172.17.50.1 Serial0
Once again, the topology is consulted and it is determined that the next hop listed in the CEF tables, 172.17.50.1, is router3.
On router3, the NetFlow tables are examined:
router3#sh ip cache flow | include 96.170 Et1 96.170.4.8 Se0 192.168.77.2 11 0013 0007 3235
Ah, perhaps the end is near! The source interface for the flow is Ethernet1. Is the source station directly attached to this router? A check of the CEF tables reveals:
router3#sh ip cef et1
Prefix Next Hop Interface
10.222.88.128/25 attached Ethernet1
10.222.88.144/32 10.222.88.144 Ethernet1
192.168.97.0/24 10.222.88.144 Ethernet1
10.222.88.73/32 10.222.88.73 Ethernet1
This presents a bit of a conundrum; there are two possible sources. It may be necessary to check both IP addresses. First, a check of router5, 10.222.88.73:
router5#sh ip cache flow | include 96.170 router5#
This command returns nothing. After verifying that the attack is still underway, it is obvious that the attacker's data flow does not pass through this router. Moving on to router4 reveals:
router4#sh ip cache flow | include 96.170 Et1 96.170.4.8 Et0 192.168.77.2 11 0013 0007 6673
Ah, this looks promising. A quick check of the CEF tables finds:
router4#sh ip cef et1
Prefix Next Hop Interface
192.168.97.0/24 attached Ethernet1
192.168.97.2/32 192.168.97.2 Ethernet1
So the only active IP address is 192.168.97.2. Since a quick check of either the MAC address (with sh arp) or other means reveals that this is not a Cisco router, this IP address begins to look more suspect. At this point, network sniffing can be performed to verify that the source IP of the attack, 96.170.4.8, is tied to the MAC address of 192.168.97.2. The source of the spoofed IP addresses has been found.
While this method is fast and presents very little impact on the routers, it is not without certain limitations.
First, NetFlow must be running on the interfaces. NetFlow can be configured, in real time, during an attack. The NetFlow data is the key to this method.
Second, router access must be available. This can be a hurdle both technically (no access to the routers) and politically (the routers are owned by another entity). However, this can be a coordinated effort, with multiple teams handing off the tracing as each autonomous system boundary is crossed. If the trace is done completely within a single AS (autonomous system), however, many of the political and technical issues may not exist.
Third, the attack must be of a duration that allows for a trace. Short, bursty attacks may not allow for a full trace. While a partial trace may help to narrow the scope of the search, it will not find the culprit.
Fourth, this method is obviously limited to the Cisco IOS platform. Other platforms, such as a Check Point FireWall-1 firewall, will provide similar tracing capabilities through the rule base or tools such as tcpdump, snoop, and iptrace. However, some platforms may provide no trace method at all.
Many companies are starting to create tools to help network operators handle DoS attacks. A rather clueful company is Arbor Networks, which provides tools to immediately identify, characterize, and trace back attacks to the network ingress. Additionally, incident data is automatically recorded for later analysis or law enforcement correspondence. You can find more information at http://www.arbor.net.
For more detailed information about DoS attacks themselves, refer to the following DoS attack resource pages: