Security testing is one of the most technical, time-consuming, yet rewarding areas of software testing. When people think about software testers, they often visualize individuals who use a software program the way the software company anticipates customers will use it. If you are a software tester, you know that testers are often responsible for many kinds of testing, including the following types:
Accessibility testing. Testing that the software is able to be used by people with disabilities
International testing. Testing that software versions work correctly in other locales, including functionality that might be customized in a locale or how the user interface is displayed in that locale’s language
Performance testing. Testing how fast the software operates
Upgrade testing. Testing how the new software operates when a previous version is already installed on the customer’s machine
Security testing. Testing the software’s ability to withstand attacks
This seems like a lot of testing, but, excluding security, all of these types of testing involve scenarios that legitimate users of the software are likely to encounter. Sometimes the product group making the software will come up with several different types of customers and use scenarios for the product and will test to verify that the product behaves according to the design specification for each customer and scenario combination. Accordingly, it’s crucial to understand the issue in each category. For example, for accessibility testing it is helpful for testers to consider that the user might be unable to use the mouse to click buttons. With this knowledge, a tester can verify that all functionality on the menus can be accessed using the keyboard.
Testing the product’s functionality in these legitimate use scenarios is important, but it doesn’t test whether the product is secure. Security testing is different from all other types of testing. Security testing attempts to find vulnerabilities in the software and to verify whether it’s possible for an attacker to misuse the software program for malicious purposes. Some people call security testing “negative testing” because the tester verifies that bad things don’t happen. As it happens, most good security testers are also excellent functionality testers.
A legitimate customer (nonattacker) would not use the product as it is used in many of the scenarios tested during security testing. In the subset of test scenarios that a legitimate user would experience, the security impact of a software flaw might not be realized. For example, a legitimate user might include images in e-mail messages, but testing this functionality doesn’t normally help the security tester identify whether a malicious user might employ the feature—for example, to track recipients of the e-mail message by using the image as a Web beacon. (For more information about Web beacons and tracking a user by using an HTML image, see http://office.microsoft.com/en-us/assistance/HP010440221033.aspx.)
When it comes to security, it doesn’t matter who uses the product and how the product is used if a feature or flaw can enable an attacker to compromise users or data. It doesn’t matter whether someone is abusing a coding error or using a feature that was designed to perform an insecure behavior. The security tester’s job is to uncover these types of bugs.
Tip
Good security testers also have many of the characteristics of good functionality testers. Good testers can distinguish all of an application’s individual components and subcomponents from one another. They understand how each works individually and, more important, how they all work together to provide an end-to-end solution. Good testers can then identify problems that might occur based on the tester’s knowledge of how the application works internally.
For many software products, the person responsible for testing a specific feature is also responsible for testing all aspects of that feature, including accessibility, globalization, performance, and security. Although testing all these areas involves a great amount of work and is a great responsibility, it is a sensible way to conduct testing because the feature tester knows the specific functionality extremely well and is able to focus on exactly how the feature works.
Another common approach to security testing is to assign it to security experts. Sometimes security experts—also known as penetration testers, or pen-testers for short—work on the same test team as functionality testers; other times, pen-testers work as security consultants and are hired by the software development company to perform security tests (also known as pen-tests). If functionality testers or pen-testers aren’t testing software security, don’t worry! The product’s security will still be tested—by criminals, spies, third-party pen-testers, and security hobbyists.
Important
Do not confuse the testing of security features with security testing. Testing the functionality of security features is important—if a security feature, such as password protection, fails, the product might be unusable or insecure. On the other hand, penetration-style security testing includes the deliberate testing of all the product’s features to be sure they can withstand attack. Testing security functionality is not the equivalent of security testing.
Criminals might test a product’s security so that they can find a way to perpetrate a crime. For example, a criminal might test the security of a banking Web site to find a software vulnerability that will enable access to the bank customers’ money. Spies look for software security vulnerabilities for other reasons. Perhaps an underhanded company will try to obtain confidential information about its competitors by exploiting software vulnerabilities. Spying isn’t limited to the corporate world; government agencies also spy. Some criminals and spies are prepared to use vast resources to attempt to find security vulnerabilities in target software if compromising the user or data is rewarding enough. Even a seemingly unimportant home computer might be of interest to an attacker if it can be used as a tool to launch additional attacks and promote anonymity.
Many legitimate reasons exist for external pen-testers to test software whether or not they have been hired directly by the creator of the software. Some security consulting companies are hired to test a company’s security by attempting to break into the company’s premises or networks. If the target company uses particular software, that software could be targeted for penetration. Pen-testers typically notify the software developer and sometimes the general public when a vulnerability is discovered. See Chapter 20 for more information on that notification process.
Security hobbyists test software for fun and challenge. Security testing is often like a complicated puzzle. The hobbyist attempts to figure out how all the pieces of the software work and how they can be used together to cause some insecure behavior. Because software is very complex, often developers are unclear on exactly how all test cases will be handled by their code. Security hobbyists are in a unique position because they can decide which software is interesting to them and they can test it as long as they like. They can spend extreme amounts of time examining a small piece of a program. If they find an issue, security hobbyists usually notify the software creator and perhaps the general public the same way a third-party pen-tester does. Security hobbyists should not be thought of as novices—many are extremely knowledgeable, clever, and experienced.
This book describes various attack techniques; many are related to specific technologies. Although it’s important to understand these attacks, it is more important to understand the general approach to security testing, which is as follows:
There are alternative approaches. For example, some people conduct security testing strictly by searching the source code for commonly misused function calls. In our experience, we find the approach described in this chapter to be the most effective and notice that many experienced security testers use the same or a very similar approach.
To be effective at finding security bugs, a bug finder must have a thorough and varied understanding of how the target application works. You are interested in how someone will use an application feature, but you’re even more interested in how that feature was implemented. A good understanding of how software is developed enables you to surmise several methods a developer might have used to implement the functionality you are security testing. Then you can consider the implementation and design of each software option and how the developer might have included protection of security weaknesses. You can inspect the target application to verify your assumptions of how it was implemented. (Software inspection is discussed in Chapter 3.) Next you can conduct various tests to see whether the developer was aware of the security issues or whether you’ve uncovered a security flaw.
Security weaknesses creep into software for various reasons. Software developers are usually very clever people, but it is difficult to be aware of all the many and varied attacks, several of which are discussed in this book. Also, developers have many responsibilities in addition to blocking attacks. For example, they must design and code the application’s functionality, make the application accessible for people with disabilities, and ensure the application is easily localizable. Developers must contend with aggressive ship schedules and so find their time limited, which makes it difficult to fend off all possible attacks successfully, whereas attackers with deep security knowledge can spend large amounts of time trying to break an application by focusing on very small parts at a time. As an attacker—bug hunters need to see themselves as attackers—your job is easier than that of the developer attempting to block attacks: attackers must find only one way to break into an application to be successful. The developer, on the other hand, must be aware of hundreds of attack tricks and must defend against all of them, plus as-of-yet unforeseen issues, to be successful.
Security testers employed by a software development company are in a challenging position. Although they use many methods similar to those of malicious attackers, ultimately their job is to help protect the application by finding as many vulnerabilities as possible. Each bug that is found and fixed makes it more difficult for external bug hunters to abuse the software and helps protect the customers who use the product. Whereas external attackers might be successful in their malicious endeavors if they find a few security flaws a year, security testers working on a product test team are expected to find numerous security bugs in specific products within tight production schedules.
The advantages of thinking carefully about how the application was designed or implemented are clear. In addition to brainstorming ideas about how the application might have been put together, you can take apart the target application to get a better understanding of how it works and, in turn, how you can break it. This same approach is used in the physical world: to someone who does not understand how locks work, picking a lock sounds difficult. In fact, some locks are difficult to pick, but many are not, if you know how they operate. To protect their property, many people use a combination lock. Most people assume only someone with the three-number combination or a bolt cutter can open the lock and access the protected property.
The numbers on combination locks range from 0 to 39 and the locks have a shackle with a notch in its side, as shown in Figure 1-1. Inside the combination lock, a metal bar keeps the shackle from moving upward when the lock is in the locked position. When the correct combination has been entered, the bar moves away from the notch in the shackle, allowing the shackle to move upward and the lock to open. An attacker needs only to move the metal bar away from the notch in the shackle to open the lock.
It might seem difficult or impossible to move the bar without physically damaging the lock. (Picking a lock is about opening the lock without damaging it. If you wanted to bypass the lock, you could simply use bolt cutters to cut it open.) But a close examination of the lock reveals a very narrow gap between the shackle and the body of the lock. The gap is about 0.2 millimeters wide—so narrow that not even a fingernail could fit in it. However, as it turns out, the aluminum of a soda can is just the right thickness and strength. Someone named “JasonLynn” posted a video on a message board detailing how to cut an aluminum can so that it can be used as a shim to open a combination lock. With a piece of metal cut to the proper size, a combination lock can be opened in less than 15 seconds. By using knowledge of how a combination lock works internally and the right tool, anyone can pick the lock. Locksmiths already knew this technique and used a professional tool known as a shim for the same purpose.
Tip
Bypassing software security is very similar to picking locks. You want to develop a deep understanding of how the internals work. This knowledge makes it much easier to find security vulnerabilities.
You can easily see why you want to have a deep understanding of how the software works internally. You can take applications apart to gain this knowledge. Once you have this knowledge, it is sometimes obvious how the application can be compromised. Other times, deep understanding of the inner workings provides more information about which attack method might be successful and what type of tools you might need to build to attack the target application. It is extremely important to understand the target software and whether it has any dependencies. Dependencies can include the protocols used, the code libraries called, the underlying operating system, the compiler, and so on. Much of this book focuses on how to take software apart and obtain information about how it is implemented. We also discuss common problems with different implementations and designs so that you can pick the software apart and find security vulnerabilities.
After you have a deep understanding of the software, you must think maliciously about how specific features could be used. Just as an accessibility tester must understand how a user with disabilities might think about and interact with the software, you need to think about how a malicious user could employ various features of the software being tested. Thinking maliciously isn’t something most people are accustomed to doing but is key to being a successful security tester. Attackers think maliciously about software functionality. And so, to find security bugs before attackers do, you must also think maliciously about the software. A few examples can show you how to use understanding of how an application works and a malicious mind-set to identify security issues.
Before the Internet was widely available, people used bulletin board systems (BBSs) to exchange messages (often only with other users of the same BBS), transfer files, and play very limited games (often text based). A BBS is a computer set up to receive calls. Users of the BBS used their modems to connect over the telephone lines. Many people set up BBSs on their personal computers so that callers could create new limited accounts. The owners of the systems, called sysops, wanted to allow people to use their BBSs, but usually required some valid contact information about the users because they needed to track malicious users. (Phone companies didn’t offer the Caller ID service yet. Later, when Caller ID became available, it turned out not to be good for identity verification anyway because it can be spoofed by the caller; see Chapter 6.) Some sysops manually validated new users by calling each one and talking with the user at the phone number used to register the account.
Other BBSs automatically verified new users by using a feature called callback verification (CBV). During registration, new users were instructed to type ATA in their terminal window when their phone rang for verification. The ATA command enabled the user’s modem to answer the phone call. The CBV feature then disconnected the new user’s session and immediately dialed the phone number the user registered with. Once the BBS called the new user back and a connection was reestablished, the CBV feature would prompt the new user for a user name and password. If these were correct, the new user would be granted access to the BBS.
From a functionality standpoint, the CBV feature worked, but now you need to think maliciously about it. Can you think of ways to use this feature maliciously or to bypass it? The CBV feature dials any number the user types in. Malicious users quickly realized the full potential of the CBV feature. For example, it could be abused in such a way as to have the police arrive at the sysop’s home any time the malicious user chose: in the United States, a malicious caller could call the BBS and enter 911 as the first three digits of the callback phone number. When the CBV feature called the modem number supplied, it actually dialed the 911 emergency center and police were dispatched to the location of the 911 call, which was where the BBS was running!
Clearly, the designers/developers of the CBV feature never anticipated users would use the feature to force the BBS system to call 911. However, once the potential for abuse was understood, CBV programs were modified to include a list of numbers the system was prohibited from calling. A security tester’s job is to be malicious in thinking about a product’s design. If someone had anticipated this design flaw before the CBV feature was released, many BBS sysops could have been spared much confusion.
This book outlines various understood attacks against common technologies. Understanding already-known attacks can help you catch them in your testing as well as help you establish a malicious mind-set toward software. A malicious mind-set and technical knowledge (deep understanding) are required to find new security attacks against both new and existing technologies.
People who are good at finding security flaws in software usually are able to think maliciously about everyday encounters, too, because they have developed a malicious mind-set. The CBV example is a computer-related example, but finding security flaws often involves malicious thinking independent of any technology.
Recently, we purchased a kitchen table from a local store that sells unassembled furniture. We got the box home, opened it, and noticed the table was the wrong color. We went back to the store to return the table. When we reached the front of the customer service line, the agent asked for the box that contained the pieces of the table and the sales receipt. The agent examined the receipt, noted the return in the point-of-sale software, and then gave us our money back.
This probably sounds like a pretty typical merchandise return experience. However, if you think carefully and maliciously about the situation, the customer service agent made a mistake that someone can exploit to steal merchandise from the store. The agent checked the receipt and accepted the original merchandise back from us, right? Maybe. The agent only checked the receipt and looked at the box containing the original merchandise. The agent did not look inside the box. We easily could have filled the original box with garbage weighing about the same as the kitchen table and returned that for a refund of the purchase price, while keeping the new table at home. We did not try to do this! It would be illegal and unethical to shoplift a table, but you can see that with a little thought, common behaviors can be exploited to perpetrate an attack.
Unlike this example, software can often be attacked with a great deal of anonymity, which makes it even easier for some people to forget or ignore the legalities and ethical considerations of their actions. When people return merchandise to a store, the store security tapes show them making the return, and most returns require an ID or a phone number plus a signature (certainly, these could be faked).
One of the most important concepts for a security tester to remember is that software often acts like the customer service agent in the example and blindly trusts the data it is given without properly validating it. Other times, the validation process is faulty.
To help prevent piracy, most software stores do not let you return software if the package has been opened. Often the way the store knows whether the package was opened is if the shrink wrap has been broken. The belief is that if the shrink wrap is still intact, the software hasn’t been opened. This is an example of faulty validation. A customer can break the shrink wrap, remove the software, and then re-shrink-wrap the box. Although the average person doesn’t own shrink-wrapping equipment, some people do, and if the target software is valuable enough, it is worth the time and money spent obtaining such equipment. This shrink-wrap example points out something else important: if an attack is possible but difficult for the average user to accomplish, given a target that is valuable enough, an attacker will spend the resources (time, money, etc.) to perform the attack.
Once you’ve come up with some malicious ideas about how the software might be vulnerable to attack, you need to test and determine whether the product actually is vulnerable. Attacking the product can be time-consuming. Chapter 2, discusses how to prioritize testing. Because most of this book discusses specifically how to perform various attacks against software, we briefly mention this step here and then move on to the last step in our general approach to security testing.
In the earlier combination lock example, we demonstrated how it wasn’t necessary to discover the lock’s security problem because someone else had already discovered the weakness and shared the information. Certainly, malicious people can use the information about bypassing a combination lock to steal, but others can use this information for positive purposes. For example, a company that designs combination locks could use this information to ensure its new design does not fall prey to the shim attack; someone securing property using a combination lock might consider using a higher-security lock because of the shim attack.
Much like the combination lock example, new computer security issues are discovered continuously, and the information is published. Some of the best resources providing information about new computer security issues are security mailing lists. Two of the more popular computer security mailing lists are Bugtraq (http://www.securityfocus.com/archive/1) and Full-Disclosure (http://lists.grok.org.uk/pipermail/full-disclosure). Every day, information about newly discovered security flaws is posted and discussed. Anyone can post to these mailing lists, which means the quality of the content varies. It is important for security testers to spend time understanding the issues posted. Understanding the flaws in detail helps you gain a better understanding of why a particular attack works, gives you new ideas for test cases you might want to try, and gives you some insight into how attackers think when they attack software.
Security conferences are also good resources for security testers. Black Hat (http://www.blackhat.com), CanSecWest (http://www.cansecwest.com), and Defcon (http://www.defcon.org) are popular security conferences in North America. Attendees and presenters at these conferences are diverse. In the past, presenters have included employees of European and U.S. government agencies as well as people previously convicted of computer crimes. The presenters, some of the most respected people in the security community, often unveil new attacks and/ or new tools to help attack software. In addition to the presentations, conferences provide a great environment in which to discuss and learn about attackers’ attitudes toward finding vulnerabilities and exploiting them and to network with other security-minded people.
Functionality testing does not test a product’s security. Even if a vulnerability is difficult to discover, someone will discover it given enough time. Various types of people test software security: security testers who work for software development companies, malicious users who hunt for security vulnerabilities so that they can commit crimes or spy, security consultants who are hired to break into a target, and hobbyists who do it for fun and profit.
Thorough security testing requires a deep understanding of how the tested functionality is implemented. The more information you have about how an application works, the more insight you will have in finding security vulnerabilities. Once you have a good understanding of how the tested functionality works, you need to think maliciously about how the functionality could be abused. Then you test your malicious ideas against the target. Throughout the process, it is important for you to stay up-to-date on the latest vulnerabilities and exploits by reading security mailing lists and/or attending security conferences because software security testing is a rapidly changing area.