5

Collecting Network Evidence

The traditional focus of digital forensics has been on locating evidence on a potentially compromised endpoint. More specifically, computer forensics is largely focused on a system’s storage. Law enforcement officers interested in criminal activity such as fraud or child exploitation can find the evidence required for prosecution on a single hard drive. In the realm of incident response, however, it is critical that the focus extends far beyond a suspected compromised system. For example, there is a wealth of information that can be obtained within the hardware and software in question, along with the flow of traffic from a compromised host to an external Command-and-Control (C2) server.

This chapter focuses on the preparation, identification, and collection of evidence that is commonly found among network devices and along traffic routes within an internal network. This collection is critical during incidents where an external threat source is in the process of commanding internal systems or stealing data from the network. Network-based evidence is also useful when examining host evidence, as it provides a second source of event corroboration, which is extremely useful in determining the root cause of an incident.

We will cover the following topics in this chapter:

  • An overview of network evidence
  • Firewall and proxy logs
  • NetFlow
  • TCPDump packet capture
  • Wireshark packet capture

An overview of network evidence

There are network log sources that can provide CSIRT personnel and incident responders with good information. Each network device provides different evidence based on its manufacturer and model. As a preparation task, CSIRT personnel should become familiar with how to access these devices to obtain the necessary evidence or have existing communication structures in place to engage IT personnel to assist with the proper response techniques during an incident.

Network devices such as switches, routers, and firewalls also have their own internal logs that maintain data on who accessed the device and made changes. Incident responders should become familiar with the types of network devices on their organization’s network and be able to access these logs in the event of an incident:

  • Switches: These are spread throughout a network through a combination of core switches that handle traffic from a range of network segments and edge switches that handle traffic for individual segments. As a result, traffic that originates on a host and travels out of the internal network will traverse several switches. Switches have two key points of evidence that should be addressed by incident responders. The first is the Content-Addressable Memory (CAM) table. This CAM table maps the physical ports on the switch to the Network Interface Card (NIC) on each device connected to the switch. Incident responders tracing connections to specific network jacks can utilize this information. This can aid the identification of possible rogue devices, such as wireless access points or systems connected to the internal network by an adversary. The second way in which switches can aid an incident investigation is by facilitating network traffic capture.
  • Routers: Routers allow organizations to connect multiple LANs into either a Metropolitan Area Network (MAN) or a Wide Area Network (WAN). As a result, they handle an extensive amount of traffic. The key piece of evidentiary information that routers contain is the routing table. This table holds the information for specific physical ports that map to the networks. Routers can also be configured to deny specific traffic between networks and maintain logs on allowed traffic and data flows. Another significant source of evidence that routers can provide is NetFlow data. NetFlow provides data on IP addresses, ports, and protocols of network traffic. This data can be utilized to determine the flow of traffic from various segments of the network (NetFlow will be covered in greater detail later in this chapter).
  • Firewalls: Firewalls have changed significantly since the days when they were simply considered to be a different type of router. Next-generation firewalls contain a wide variety of features, such as intrusion detection and prevention, web filtering, data loss prevention, and detailed logs about allowed and denied traffic. Often, firewalls serve as a detection mechanism that alerts security personnel to potential incidents. This can include alerts from features such as Intrusion Detection Systems (IDSs)/Intrusion Prevention Systems (IPSs), blacklists of known bad URLs or IP addresses, or alerts flagging configuration changes to the firewall without the knowledge of IT personnel. Incident responders should have as much insight as possible into how their organization’s firewalls function and what data can be obtained prior to an incident.
  • Network IDSs/IPSs: These systems are purposefully designed to provide security personnel and incident responders with information concerning potential malicious activity on the network infrastructure. These systems utilize a combination of network monitoring and rulesets to determine whether there is any malicious activity or not. IDSs are often configured to alert you to a specific malicious activity, while an IPS can detect, but also block, potential malicious activity. In either case, both types of platform logs are an excellent place for incident responders to locate specific evidence of malicious activity.
  • Web proxy servers: Organizations often utilize web proxy servers to control how users interact with websites and other internet-based resources. As a result, these devices can give an enterprise-wide picture of web traffic that both originates with and is destined for internal hosts. Web proxies also have additional features, such as alerting security personnel to connections to known malware C2 servers or websites that serve up malware. A review of web proxy logs in conjunction with a possibly compromised host may identify a source of malicious traffic or a C2 server exerting control over the host.
  • Domain controllers or authentication servers: Serving the entire network domain, authentication servers are the primary location that incident responders can leverage for details on successful or unsuccessful logins, credential manipulation, or other credential uses.
  • DHCP servers: Maintaining a list of assigned IP addresses for workstations or laptops within the organization requires an inordinate amount of upkeep. The use of Dynamic Host Configuration Protocol (DHCP) allows for the dynamic assignment of IP addresses to systems on the LAN. DHCP servers often contain logs on the assignment of IP addresses mapped to the MAC address of the host’s NIC. This becomes important if an incident responder has to track down a specific workstation or laptop that was connected to the network at a specific date and time.
  • Application servers: A wide range of applications from email to web applications is housed on network servers. Each of these can provide logs that are specific to the type of application. Also of interest during an incident investigation are any logs pertaining to remote connections. Adversaries will often pivot from a compromised system to servers to gain access to confidential data or for other follow-up activities.

Preparation

As covered in the previous section, there is a wide range of network devices, each with its own method of aggregating and reporting relevant data. The ability to acquire network-based evidence is largely dependent on the preparations that are undertaken by an organization prior to an incident. Without some of the critical components of a proper infrastructure security program, key pieces of evidence will not be readily available to incident responders. The result is that evidence may be lost while CSIRT members hunt down critical pieces of information. In terms of preparation, organizations can aid the CSIRT by having proper network documentation, up-to-date configurations of network devices, and a central log management solution, such as a SIEM, in place.

Another consideration in terms of preparation is to incorporate the specific tasks that need to be performed for proper network evidence collection into the incident response playbooks. For example, firewall logs that are not sent to a central log management solution, such as a SIEM, are stored on the device itself. Playbooks that involve acquiring network evidence should be verbose enough to walk through the process of gathering these logs.

Aside from the technical preparation for network evidence collection, CSIRT personnel needs to be aware of any legal or regulatory issues with regard to collecting network evidence. Additionally, CSIRT personnel needs to be aware that capturing network traffic can be considered an invasion of privacy if there is no policy clearly stating that network monitoring may take place. Therefore, the legal representative of the CSIRT should ensure that all employees of the organization understand that their use of the information system will be monitored. This should be expressly stated in policy prior to any evidence collection taking place.

A network diagram

To identify potential sources of evidence, incident responders need to have a solid understanding of what the internal network infrastructure looks like. One method that can be employed by organizations is to create and maintain an up-to-date network diagram. This diagram should be detailed enough that incident responders can identify individual network components such as switches, routers, or wireless access points. This diagram should also contain internal IP addresses so that incident responders can immediately access those systems through remote methods. For instance, examine the following simple network diagram:

Figure 5.1 – A sample network diagram

This diagram allows for the quick identification of potential evidence sources. For example, suppose that the laptop connected to the switch at 192.168.2.1 is identified as communicating with a known malware C2 server. A CSIRT analyst could examine the network diagram and ascertain that the C2 traffic would have to traverse several network hardware components on its way out of the internal network. For example, there would be traffic traversing the switch at 192.168.10.1, through the firewall at 192.168.0.1, and, finally, from the router out to the internet.

Configuration

Determining whether an attacker has made modifications to a network device such as a switch or a router can be made easier if the CSIRT has a standard configuration immediately available. Organizations should already have configurations for network devices stored for disaster recovery purposes but they should also have these available to CSIRT members in the event that there is an incident.

Firewalls and proxy logs

Two main sources of evidence available while investigating an incident are ingress/egress points into the network from the internet. Modern malware and other exploits will often require the ability to reach internet-based resources. This may be for the purpose of downloading additional malware or to exploit code. Other attacks that involve data exfiltration will require access to the internet. Finally, adversaries will often have to establish C2 over compromised systems. In these cases, traffic from various protocols will traverse the perimeter of the victim network. Depending on the victim, this traffic will have to traverse a firewall, internet proxy, or both. As a result, both technologies provide incident response personnel with a major source of evidence.

Firewalls

Firewalls have evolved from a simplified routing and blocking technology into platforms that provide a significant insight into the traffic coming into and leaving the network. Next-generation firewalls often combine the deny/allow ruleset with IDSs or IPSs, as well as controlling network access to applications. This creates a significant source of evidence that can be leveraged during an incident.

Acquiring evidence from firewalls is largely dependent on the manufacturer and the specific model that is used. Incident responders should thoroughly understand the feature set and specific data that can be obtained as part of their preparation. Although features differ between vendors and models, there are some key evidence points that are near-universal:

  • A connection log: The connection log provides the source and destination IP addresses and protocols of connections between internal and external systems. This is critical when determining whether any internal systems may have had contact with an adversary-controlled system or are possibly being controlled. In addition to allowed connections, the logs may also provide an insight into connections that were denied. One technique that is often used by adversaries is to use tools to attempt to connect to well-known ports that are commonly in use. If these ports are closed to external connections, there will be a deny entry in the logs. Successive denies across a range of ports are indicative of reconnaissance activity.
  • Remote access logs: Firewalls often serve as the Virtual Private Network (VPN) concentrator for remote access. If a remote user becomes infected via malware, they can introduce that infection into the internal network through the VPN. Remote access logs will show systems that are connected and at what time they connected. This may allow incident responders to correlate activities and determine whether a remote user was the source of the infection.

Web application firewalls

A special type of firewall is the Web Application Firewall (WAF). This device or software sits between the public internet and a web application. This allows an organization to protect the application from the public. WAF policies can be changed very quickly to respond to attacks such as Denial-of-Service (DoS) or adversaries attempting to compromise the web application infrastructure. From an evidentiary standpoint, WAFs provide analysts with log files of connections made from the internet, along with other data points such as the type of HTTP requests made and information on the systems that access resources. This is a good source of evidence of attempted or successful exploitation of web servers and web applications.

Web proxy servers

Adversaries often make use of scripting such as Microsoft Visual Basic or PowerShell to download secondary exploit packages or malware. These scripts will often contain a URL that points to the exploit or malware. Adversaries make use of URLs as opposed to IP addresses, as the IP addresses can be easily changed via domain name registration, allowing them to change their infrastructure without having to change their scripts.

Organizations that make use of web proxy servers for HTTP and HTTPS requests will have a record of any system on the internal network that reached out to an external site. From here, they may be able to identify the location and, potentially, the malware or exploit that has been downloaded. Additional insight may be gained from C2 traffic that makes use of similar tactics to malware.

As detecting attacks often takes months, it is imperative that incident responders can view the history of an activity that has happened over the span of weeks or even months. Given the relatively small size of proxy requests, even just the date, time, requesting system, and the URL that was visited can provide a significant piece of evidence that might not be available otherwise.

NetFlow

First designed by Cisco Systems in 1996, NetFlow is a feature found in network devices such as switches and routers that allows network administrators to monitor traffic within the network. NetFlow is not strictly a security tool but it does provide a good deal of data to incident responders in the event of an incident. NetFlow is sent by network devices via the UDP protocol to a central collection point, often called the NetFlow Collector.

In a security context, NetFlow provides deep insights into the internal traffic of systems as they communicate with each other. This is often referred to as east-west traffic, as opposed to north-south traffic, which is used to describe internal systems communicating with external systems through the perimeter firewall. For example, the following diagram shows a simple network. In a real-world scenario, an attacker may compromise a system on the 10.10.2.0/24 subnet. From there, they may attempt to pivot to a file server on the 10.10.1.0/24 subnet. Once there, they can acquire confidential data and move it back to the compromised system for exfiltration. The switches forward the NetFlow data to the collector, which includes the IP addresses, protocols, and data size. This data is critical for providing incident response analysts with details that they may not normally otherwise acquire:

Figure 5.2 – A NetFlow diagram

NetFlow data is limited and depends on the types of devices that are configured to send data to the NetFlow server. Generally, NetFlow contains the source and destination IP addresses, the source and destination ports, the protocol, and finally, the amount of traffic sent. Even with this limited information, analysts gain insight into adversary activities, such as potential lateral movement or data exfiltration.

Configuring NetFlow is dependent on the type and manufacturer of the network components. Moreover, there is a wide range of collectors and analysis tools that can be leveraged depending on the budget and other resources. One of the advantages of including NetFlow analysis in the overall network operations is that it not only provides data to the incident response team, but it is also highly useful in day-to-day network operations in terms of hunting down latency or other communication issues. This dual purpose makes including it as part of the overall network operations easier to justify.

Packet capture

Capturing network traffic is critical to having a full understanding of an incident. Being able to identify potential C2 IP address traffic may provide further information about the type of malware that has infected a host. In other types of incidents, CSIRT members may be able to identify potential exfiltration methods that an external threat actor is utilizing.

One method is to set up what is referred to as a network tap. A network tap is a system that is in line with the compromised host and the switch. For example, in the network diagram, if the compromised host is on the 192.168.1.0/24 subnet, the tap should be placed in between the host and the switch. This often involves placing a system in between the host and the switch.

Another option is to configure a Switched Port Analyzer (SPAN) port. In this configuration, the switch closest to the compromised host will have port mirroring enabled. This then sends the traffic from the entire segment the switch is on to the system that is on the mirrored port.

Finally, some network devices have built-in applications, such as tcpdump, that can be utilized to capture traffic for further analysis. This may be the quickest option, as it does not require physical access to the network or the switch and can be set up remotely. The drawback to this method is that storage on the switch may not support a large capture file and the added strain may increase the chances of some packets not being captured.

tcpdump

tcpdump is a command-line tool specifically designed for packet capture. tcpdump is often included with Linux distributions and is found on many network devices. For many of these devices, tcpdump has to be run as a root user or with root privileges, as it will be monitoring network traffic. The relevant documentation is available at http://www.tcpdump.org/. To perform a packet capture with tcpdump, the following process can be used:

  1. To access the basic help menu, type the following into Command Prompt:
    arkime@arkime: :~$ tcpdump -h

The command will produce the following help menu:

Figure 5.3 – The tcpdump help menu

The default tcpdump setting is to capture traffic on all available interfaces. Running the following command produces a list of all the interfaces that tcpdump can capture traffic on:

arkime@arkime::~$ tcpdump -D

The following screenshot shows that in this case, the ens160 (Ethernet) interface and the lo (loopback) interface are available for capturing traffic:

Figure 5.4 – tcpdump capture interfaces

  1. To configure a basic capture on the Ethernet interface located at ens33 with normal verbosity, type the following command:
    arkime@arkime:~$ sudo tcpdump -i ens160 -v

The -i switch tells tcpdump which interface to perform the packet capture on. In this case, it is on the following Ethernet interface: ens160. The -v switch sets the verbosity of the packet capture. In this case, the verbosity is set rather low. For additional data, the switch can be set to -vvv for a more detailed look at the packets. The following screenshot shows what information is displayed by the command:

Figure 5.5 – The tcpdump command output

While this method determines whether traffic is traversing that interface or not, the individual packet information is useless to an analyst due to the speed with which the individual packets appear on the screen. For the packet capture to be of any use, it is recommended that you output the file so that a later examination can be performed with a packet analysis tool such as Wireshark. Wireshark will be reviewed later in this chapter and in greater detail in Chapter 9.

  1. To configure tcpdump to output the packet capture to a file, the following command is used:
    arkime@arkime:~$ sudo tcpdump -i ens160 -vvv -w ping_capture

PING command

PING (short for Packet Internet Groper) is the utility that uses the ICMP packets being sent in this section. PING is used to determine whether there is network connectivity to various systems. In this case, a check of connectivity to the Google DNS at 8.8.8.8 is being performed.

The command tells tcpdump to capture network traffic and write the file out to capture. Unlike the previous capture, there is no traffic indicated on the screen.

  1. To stop the capture, press Ctrl + C, which produces the following information:

Figure 5.6 – The tcpdump output

  1. After navigating to the root directory, the file can then be opened via a network analysis tool such as Wireshark, as shown:

Figure 5.7 – Wireshark packet capture analysis

tcpdump can also be configured to focus the capture on specific sources or destination IP addresses and ports. For example, if an incident response analyst needs to collect packets leaving a specific host at the 192.168.10.54 IP address, the following tcpdump command will produce the desired results:

arkime@arkime:~$ sudo tcpdump -i ens33 src host 192.168.10.54

Packets going to a destination such as a known C2 server at the IP address can also be separated from the background network traffic with the following command:

arkime@arkime:~$ sudo tcpdump -i ens33 dst host 162.4.5.23

tcpdump is a powerful tool and has plenty of options. Incident response analysts are advised to examine and incorporate its various features into their toolkit.

WinPcap and RawCap

During an incident, it may become necessary to obtain a packet capture from a Windows system. In incidents such as a compromised web server or application server, a Windows system will not have a native application with which to conduct a packet capture. There are several packet capture tools available on Windows systems. The first tool that can be utilized is WinPcap. This tool is generally recognized as the standard for packet capture on Windows systems and is available for free download at https://www.winpcap.org/. The drawback to this tool from a forensics perspective is that it must be installed on the system. This can complicate a forensic analysis, as any changes to the system have to be thoroughly documented. For this reason, it is a good preparatory step to ensure that high-risk systems such as web servers, file servers, and application servers already have WinPcap installed.

Another option available to incident response analysts is the use of tools such as RawCap. RawCap has the same basic capability as WinPcap without the need to install it on the local system. RawCap can be easily run from a USB device attached to the system. To perform a packet capture with RawCap, the following process is used:

  1. Start Windows Command Prompt as an administrator.
  2. In Command Prompt, navigate to the folder containing the RawCap.exe file. For a list of options, type the following:
    D:>RawCap.exe -help

The command will produce the following output:

Figure 5.8 – The Rawcap.exe menu

The output produces a list of interfaces. One of the advantages of RawCap is that, even from a USB device, the incident response analyst can perform a packet capture on each of the interfaces. In this example, the capture will be performed on the Ethernet interface indicated by the number 0.

  1. To start the packet capture, RawCap requires the network interface where the traffic should be captured, and an output file to output the packet capture. To capture the traffic on the wireless interface and output it to a file called RawCap.pcap, the following command is used:
    C:ProgramDatachocolateyinRawCap.exe 0 RawCap.pcap

The command produces the following output:

Figure 5.9 – The output of a RawCap packet capture

  1. Pressing Ctrl + C will stop the capture. The capture file, RawCap.pcap, is saved to the same directory as the RawCap.exe file. This file can then be opened with tools such as Wireshark for further analysis:

Figure 5.10 – Analysis of the RawCap file in Wireshark

Now that we have introduced Wireshark, we will examine how the tool can also be used to capture network traffic.

Wireshark

Wireshark is a Unix or Windows packet capture and analysis tool. Unlike tcpdump or tools such as RawCap, Wireshark is a GUI-based tool and includes not only packet capture but also analysis features. As a result, Wireshark may be difficult to deploy rapidly during an incident, as the program has to be installed. Furthermore, the tool is only supported on Windows or macOS. Installing Wireshark on a Linux system requires a bit more effort. The one distinct advantage that Wireshark has over command-line options is that incident response analysts can perform a detailed inspection of the traffic as it is being captured. Wireshark can be run on the system itself or on a USB drive. Once installed, it must be run as an administrator. To perform a packet capture with Wireshark, the following process is used:

  1. The first step is to select an interface where Wireshark will capture traffic:

Figure 5.11 – Wireshark Capture interfaces

In the screenshot, there is only one interface that appears to be handling traffic. The capture will be performed on the Ethernet0 interface.

  1. Double-clicking on the interface will start a packet capture. As was stated before, unlike tcpdump or RawCap, the actual capture is outputted to the screen for immediate analysis:

Figure 5.12 – A Wireshark capture view

  1. To stop the capture, click the red box in the upper-left corner of the pane. The file can then be saved for further analysis.

Another tool that is included with Wireshark and is useful during evidence acquisition is mergecap. mergecap is a command-line tool that allows incident response analysts to combine multiple packet capture files from Wireshark, tcpdump, or RawCap. This is extremely useful in situations where incident response analysts obtain packet captures from several sources but want to check for traffic to a specific host. To access the menu for mergecap, type the following into Command Prompt:

arkimie@arkime:~$mergecap -help

That command produces the following help information:

Figure 5.13 – The mergecap help menu

To merge several packet capture files, the following command is used:

arkime@arkime:~$mergecap -w switches.pcap switch1.pcap switch2.pcap switch3.pcap

By combining the output of three packet captures into one file, the incident response analyst can examine a wider range of activities across multiple network paths. If, for example, the analyst is searching for traffic coming from an unknown host to an external C2 server, they would be able to combine captures over the entire span of the network and then search for that IP, rather than individually picking through each packet capture.

Evidence collection

In order to conduct a proper examination of log files and other network data such as packet captures, they often have to be moved from the log source and examined offline. As with any source of evidence, log files or packet captures have to be handled with due care to ensure that they are not corrupted or modified during the transfer. One simple solution is to transfer the evidence immediately to a USB drive or similar removable medium. From there, a hash can be created for the evidence prior to any examination.

The acquisition of network evidence such as a packet capture or a log file should be thoroughly documented. Incident response personnel may be acquiring log files and packet captures from several sources over the entire network. As a result, they should ensure that they can trace back every separate piece of evidence to its source, as well as the date and time that the evidence was collected. This can be recorded on a network evidence log sheet and entries can be completed for each piece of evidence. For example, see the following entry:

Figure 5.14 – A network evidence collection entry

The log entry captures the following necessary information:

  • File Name: Each log file or packet capture should have its own unique name. Within the procedures in use by the CSIRT, there should be a naming convention for different types of evidence files.
  • Description: A brief description of the file. There does not need to be too much detail unless it is a particularly unique file and a detailed description is called for.
  • Hash: A comprehensive overview of hashing will be covered in later chapters. For now, it suffices to say that a hash is a one-way algorithm that is utilized to provide a digital fingerprint for a file. This hash will be recorded at the collection phase and after analysis to demonstrate that the file was not modified during the analysis phase. There are several ways to compute the hash. In this case, the MD5 hash can be computed using the md5sum installed hashing program on Ubuntu. md5sum has several different options that can be accessed via the command line. For the help menu, type the following:
    arkime@arkime:~$md5sum --help

That produces the following help menu:

Figure 5.15 – The md5sum help menu

The MD5 hash can be calculated for the packet capture from the switch by simply entering the following command:

arkime@arkime:~$md5sum ping_capture

This produces the following output:

Figure 5.16 – A md5sum file calculation

  • Source: The source is important. In this case, the packet capture was obtained from the system located at the 192.168.0.110 IP address.
  • Date / Time Acquired: Record the date and time that the file was captured. Usually, this is the same as the date and time that the capture was stopped. If it is a long packet capture, the start and stop times of the capture can be noted.

Setting the time standard

Prior to an incident, it is important to identify what time zone will be in use. From an evidentiary standpoint, the time zone does not really matter as long as it is consistent throughout the entire incident investigation.

  • Captured By: Ensure that the person collecting the file is identified.
  • Method: Indicate what tool was used to capture the file.
  • Storage Drive: Once the evidence has been captured, transfer it to an evidence storage drive. This should also be documented.

Once the collection is complete, a chain of custody form should also be filled out for the external medium that contains the evidence files. From here, the files can be analyzed.

Summary

Evidence that is pertinent to incident responders is not just located on the hard drive of a compromised host. There is a wealth of information available from network devices spread throughout the environment. With proper preparation, a CSIRT may be able to leverage the evidence provided by these devices using solutions such as a SIEM. CSIRT personnel also has the ability to capture network traffic for later analysis through a variety of methods and tools. However, all these techniques are influenced by the legal and policy implications that CSIRT personnel and the organization at large need to navigate. By preparing for the legal and technical challenges of network evidence collection, CSIRT members can leverage this evidence and move closer to the goal of determining the root cause of an incident and bringing the organization back to its full operation.

This chapter discussed several sources of evidence available to incident response analysts. Logs from network devices, whether they report to a SIEM or through other methods, can give you an insight into what has transpired in the network. Packet captures provide details about the exact nature of network traffic. Finally, analysts must be prepared to acquire these sources of evidence in a forensically sound manner.

In the next chapter, the focus will shift from network evidence acquisition to acquiring volatile data from host-based systems.

Questions

  1. Which of these items are potential sources of network evidence?
    1. Switches
    2. Routers
    3. Firewalls
    4. All of the above
  2. Network diagrams are important in identifying potential areas where network evidence can be acquired.
    1. True
    2. False
  3. Which of the following is not a network forensic evidence capture tool?
    1. RawCap
    2. Wireshark
    3. WinPcap
    4. LogBeat
  4. When conducting evidence acquisition, it is not important to record the hash value of the file.
    1. True
    2. False

Further reading

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset