Software threat modeling is a process that has evolved quite a bit over the last few years. Microsoft Press published an entire book written by Frank Swiderski and Window Snyder titled Threat Modeling on the subject. The second edition of Michael Howard and David Leblanc’s Writing Secure Code (Microsoft Press) and Michael Howard and Steve Lipner’s The Security Development Lifecycle (Microsoft Press) also contain information about threat modeling. In addition to these books, Peter Torr, in an excellent article titled “Guerrilla Threat Modelling” at http://blogs.msdn.com/ptorr/archive/2005/02/22/378510.aspx, describes how to create threat models quickly. You can use these valuable resources to expand your understanding of threat modeling beyond what we discuss here. In this chapter, we focus on understanding how security testers can use threat modeling to create actionable test cases and help prevent security issues from entering the product in the first place. For example, the Microsoft software development cycle now requires threat modeling to be performed in the planning and design stage. By performing threat modeling at this stage in the product development cycle, design flaws are found before the code has been written.

The threat model creation process should include representatives from the design team (the team that wrote the product specification), the programming team, and the testing team. Each member brings a different point of view and different knowledge about the product. You risk overlooking valuable information about or insights into the product if the threat model creation process doesn’t include someone from each of these disciplines. External attackers don’t have access to the people who created the product or to the product specifications. Use these information sources to your advantage in security testing.

Threat models (TMs) typically consist of three key parts:

You can use each part of the threat model to find security problems and ship more secure software.

DFDs are extremely useful when you are attempting to find security threats against an entire application or pieces of it. Creating an accurate and reasonably complete DFD forces the software creator to have a clear understanding of the application and how all of the data moves between different components in the DFD.

The team on which Tom Gallagher and Lawrence Landauer, two of the authors, work performs security reviews for Microsoft Office products. Part of the team’s role involves meeting with individual application teams within the Office organization, understanding what they are building, and giving them security advice. After the product team gives the security review team a brief explanation of what they are creating, the security review team looks at the DFD created by that product team. The DFD quickly clarifies exactly how particular software features work and how a user (or attacker) can interact with the feature. By studying the DFD, the security review team can begin to understand the product at a deeper level. This deeper understanding enables the team to uncover threats to the system more easily.

Figure 2-1 shows an example of a small Web application’s DFD. The DFD represents a server application that allows a user visiting a Web page to click a link on that page and e-mail a copy of the page to other users, a feature we call the e-mail Web page feature, although it sometimes goes by other names. This functionality is common on news sites. The user can request that an e-mail message containing a brief note from the user and a link to a news article on the Web site be sent to an arbitrary e-mail address. Because the Web server doesn’t record any identifiable information about the requester, the requester’s actions are largely anonymous.

Figure 2-1 is a high-level and simple DFD. Often, more detailed DFDs are created to examine specific parts of the high-level DFD. Even with this high-level DFD, you have a much better idea of how the system works. Simply knowing some of the technologies that are used can enable you to identify some attacks that are relevant and that require testing in this application. Did you notice that the feature accepts anonymous Web requests with untrusted data? Can you follow the untrusted data through the application? Did you notice this feature sends e-mail to user-supplied addresses with user-supplied content by using Simple Mail Transfer Protocol (SMTP)? What else can you see based on this DFD? Can you spot any potential problems based on this DFD? We look at a few later, but before we do, let’s look at ways an attacker might enter the application.

Figure 2-1. High-level data flow diagram for the functionality that enables a user to send a link to a Web page to a specific e-mail address

When people secure objects in the physical world, such as their home, they first secure obvious entry points like doors and windows for good reason. These points are where attackers will most commonly attempt to gain entry. Likewise, it is important for security testers to understand the entry points and how potentially malicious data can enter the software being tested. Some entry points are fairly easy to identify just by thinking about the application. For example, an application that accepts data through the user interface or uses a user-specified data file has a few fairly obvious entry points. However, many entry points are not obvious. Identifying all entry points is so important in security testing that we dedicate Chapter 3, entirely to the topic.

In the physical world in addition to securing entry points, people attempt to secure exit points, such as trash cans. For example, sensitive information is often shredded before being discarded. Companies concerned that sensitive information might be appear in the trash restrict access to the trash receptacles by using locks and/or guards.

In an ideal world, you could test everything completely. However, this isn’t possible: you can’t test every single piece of software as thoroughly as you might like. You need to prioritize where you will spend your time testing. The DFD is extremely useful in helping you identify where to spend your time. You want to find the most accessible parts of the software—the most obvious entry and exit points—and use this data to make testing prioritization decisions. For this reason, you must remember to consider and identify the access levels required to hit the various points in the DFD. For example, if the target application listens on a socket (allowing remote computers to connect to it), you must know whether a certain access level is required for the application to accept the data from the socket. For example, an application that allows only administrators to send data after authentication (using Kerberos, Internet Protocol Security [IPSec], etc.) is less of a security testing priority than is application functionality that can be invoked by anonymous users. Anonymous users and administrators are at opposite ends of the spectrum, but most users, including authenticated users, should be considered potential threats.

In the DFD in Figure 2-1, it is clear that anonymous users can send data to the product through a Hypertext Transfer Protocol (HTTP) request. This functionality is a high-priority item to test because the HTTP requests can be made by anyone on the Internet.

Before you jump in and spend time and resources testing, it’s a good idea for you to finish developing your threat model so you have a clear, organized approach to testing. The next key part of developing a threat model is to make an actual list of threats associated with the functionality being modeled. The list of entry points can be used as a starting point. You need to think about all of the data that enters the functionality, how it is used, and how it might be maliciously used to cause undesirable results. Because testers break software as part of their job, with practice, they often come up with some of the better threats against an application.

Although we could fill several pages with threats against the e-mail Web page functionality mentioned earlier, we mention only a few here:

Software testers commonly create a test plan, which specifies exactly which features and functionalities will be tested and how they will be tested, as well as which features and functionalities will not be tested. The threat model provides the foundation for the test plan. The threats define specific scenarios of interest that should be tested. It is a tester’s responsibility to make sure threats contain enough information to make each threat a set of test cases; if the threats aren’t actionable enough to test, you are missing critical information in your threat model. Application developers and designers often must provide this information. As a tester, it is important that you encourage programmers and designers to include enough detail in the threats so that the threats are actionable enough to test. You might need to interview these people to understand the specific features better and to obtain the data you need. Each threat brainstormed and included in the threat model must be tested to assess whether it is an actual flaw in the product.

If the threats aren’t carefully verified, the value of threat modeling is quickly diminished. The Microsoft Office group, for example, is serious about verifying threats. The threat modeling tool the team uses automatically enters a bug in the bug tracking system for each threat. Next, the feature designer investigates whether there is a potential design flaw. If there is, the feature designer can modify the product’s specification to eliminate the threat or mitigate it. If the designer doesn’t think the threat is an issue, he or she assigns the threat to a developer, who verifies this by looking at the code. Assuming the threat doesn’t affect the product or has been deemed not serious enough to warrant removing or mitigating, the threat is assigned to the tester that works on the functionality in question. Testers should treat threats in the same way and can use the same process testers have for closing “fixed” or “won’t fix” functionality bugs. Testers need to think carefully about the assumptions the designer and developer might have made during their assessment of the issue.

While implementing a feature, a developer often encounters parts of the specification that are unclear and must make a decision on how to write the code. Additionally, many times the original design changes after the specification is written, and often the specification is not updated. Even if the specification matches the code, there will be bugs in that code. Testers are familiar with exactly how the product works, including which places are buggy and don’t match the specification. These discrepancies change what should be included in the threat model because they either invalidate some threats or create new ones. Threat models should be updated when the design changes.

As mentioned earlier, all software has bugs. Of the bugs found prior to shipping the software, some are deemed noncrucial and so the team deliberately leaves them unfixed. These bugs are typically small issues that happen only in rare circumstances. However, many security issues occur in the same uncommon code paths in which these bugs are found, and so it is often the case that the unfixed insignificant bugs can be combined with other issues to create an exploitable security issue. To find many of the deeper security bugs, testers start by conducting normal functionality testing and taking advantage of small functionality nuances.

For example, an application’s code to save a file might terminate the filename with a null character. A user might be able to specify a filename containing a null character and have parts of the filename ignored. This bug might not be fixed because of the belief that users shouldn’t put a null character in filenames, and users doing so is rather uncommon. Because executable files can be dangerous, the example file save routine contains filtering code to verify that only text files (using the .txt extension) are saved. The filtering code does not terminate a filename when it recognizes a null. An attacker might be able to take advantage of this and specify a filename such as “test.exe[NULL CHARACTER].txt.” The file passes the .txt extension check, and is then saved as “test.exe.” This specific example is a canonicalization issue and is discussed in more detail in Chapter 12.

Threat models are read by people who are building functionality that depends on features contained in the threat model. For this reason, it is important to keep threat models up-to-date. As the design changes to mitigate threats, these changes should be included in the threat model because these changes can introduce additional threats that must be carefully modeled and tested.

As you have seen, threat models are an important part of security testing. Testers play a key role alongside developers and designers in the quality and usefulness of the data flow diagram and threat model to help ship secure software. A few key points to remember include the following:

In the next chapter, we look at specific tools, methodologies, and tips to help identify entry points for testing and threat modeling.