Search engines can produce an absolute overload of information if not used efficiently. Not only can you find information about the financial aspects of your targets, but also information about key employees, usernames and passwords, confidential documents such as network diagrams, information indicating what types of software or hardware you use or have in place, and even if systems are in a default state. This information can be devastating in the wrong hands. As a penetration tester, your focus should be to bring this type of information forth and show the client how it can be used to gain access to the client's most critical assets (and hopefully, you will tell them how to fix the problem as well!).There are search engines that cache information for quick access, and there are search engines that will archive sites and documents for years on end. There are even search engines that focus strictly on networking equipment such as wireless access points or publically facing routers, switches, servers, and more.
We will continue our footprinting reconnaissance efforts with Shodan. This search engine is specialized in indexing the information found in banners served by devices attached to the Internet. The search engine primarily indexes findings from port 80, but also indexes some Telnet, SSH, and FTP banners. Shodan is a web application and can be accessed by going to https://www.shodan.io/. An example of this is shown in the following image:
With Shodan, you can find information on devices connected to the Internet. In addition to allowing you to search by IP address or hostname, it also allows you to search by geographical location. Exporting the search results into XML is a premium feature requiring you to purchase credits. There is an example export available if you want to build a transform for MagicTree or some other data centralization tool before you decide if you want to spend money on the export.
There are several free filters that make narrowing the searches down much simpler. Most filters use the same format: searchtermfilter:{filterterm}; an example would be a search for IIS6.0os:"Windows2000". These filters can also be used in conjunction with each other in order to pull some very interesting results.
Here is a listing of several important filters:
- net: Possibly one of the most useful filters for a penetration tester. You can search your IP ranges using IP/CIDR notation (for example,
127.1.1.0/24) to see if all of your devices are configured as expected or if there are indicators that a vulnerable server or network device configuration is externally facing and ready to be compromised during testing. - city: This will limit the search to the city listed.
- country: Restricts the search to devices in the country of choice. This is also very important for pentesting, as there may be times when a client provides you with IP ranges (which you validated, right?), and then places certain assets out of scope due to location. A client may choose to not test against systems located in Singapore for instance.
- port: Will restrict the search to the port indicated. Remember that Shodan does not scan index banners for all ports, only for 80, 21, 22, and 23.
- before: Search for systems scanned before a specified date.
- after: Search for systems scanned after this date.
- os: Specify operating systems you want to include or exclude in your search.
In order to perform affective searching in Shodan, you must have some understanding of the types of banners that are indexed and what sort of information they typically contain.
FTP, Telnet, and SSH banners will vary, but each will provide useful versioning information.
Banners can be collected using nc example.com:80 and then typing HEAD / HTTP/1.0, which results in the typical banner format you will see in your Shodan results. As the HTTP banners are often the most difficult to understand, we walk through some of the commonly found sections:
root@bt:~# nc example.com 80 Trying 192.168.1.1... Connected to example.com. Escape character is '^]'. HEAD / HTTP/1.0 HTTP/1.1 200 OK Content-Length: 9908 Content-Type: text/html Last-Modified: Tue, 7Jul 2015 02:35:17 GMT Accept-Ranges: bytes ETag: "6e879e69be87cc1:0" Server: Microsoft-IIS/8.0 X-Powered-By: ASP.NET Date: Sun, 12Jul 2015 02:08:55 GMT Connection: close Connection closed by foreign host.
The interpretation of the output is as follows:
- The
HTTP/1.1 200status code highlighted will provide a response to your query indicating the status of your request. In this case, theHEAD/HTTP/1.0was accepted and processed successfully, thus initiating a status code of200 OK. Content-Length: This indicated the length of the content in the decimal number of OCTETs.Content-Type: This will list the type of content being sent. It can be an image/GIF, text/HTML, or other types.Accept-Ranges: This indicates if the server will accept a byte range. Setting this to none will let the client know that range requests could be denied.ETag: This provides the client with the current entity tag value.Server: This will provide you with the version and type of software being used to service the request. This is one of the most important banner results for a penetration tester. Clients should be advised to hide this information. You will use this information to establish which attack types may be usable on the machine.X-Powered-By: Flag is not a standard header, but can provide useful information to an attacker. It can also be changed or disabled completely.
Common status codes include the following:
|
HTTP status code |
Description |
|---|---|
|
|
A successful query resulting in displaying the result. |
|
|
The document has been moved permanently. |
|
|
The document has been moved temporarily. |
|
|
A temporary redirect is being used. |
|
|
Syntax error - cannot process your request. |
|
|
Request requires authentication. Usually indicates a login is required. |
|
|
Request is forbidden. |
|
|
The page was not found on the server. |
|
|
The server is not available at the moment. Unable to get the resource on behalf of the client. |
|
|
Internal server errors cause the server to be unable to complete the request. |
|
|
An unsupported HTTP version was used. |
Just as with most search engines Shodan is extremely user-friendly. To perform a basic search, simply type the search string into the input box at the top of the screen and you will be presented with a listing of results. You can search using any of the filters we previously discussed or you can try your hand at looking for specific banner fields.
In this day and age, everything is becoming interconnected. People are using their personal devices for work, sending out corporate e-mails using personal accounts on publicly owned mail servers, and watching lots of videos. One trend that has occurred over the years is that people have become so comfortable with the Internet that they are willing to share their information with unknown individuals and websites around the world. We will now discuss some of the methods you can use to verify that your clients are not unintentionally or intentionally leaking actionable or confidential data onto the public Internet.
There have been many books written on Google hacking; as a result, discussing the details and tricks involved would quickly divert the focus of this book.
Tip
If you are not familiar with Google hacking, perform a search for Johnny Long, visit his website at http://www.hackersforcharity.com/, and check out The Google Hacking Database (GHDB), which was the original Google Dorks repository.
Exploit-DB at https://www.exploit-db.com/ has taken over and updated Mr. Long's Google Dorks database. This is now the official GHDB site. You should use these tools in tandem with good filters to ensure that you get only the data you need. Here are some examples of how this can be done.
Go to http://exploit-db.com/google-dorks and choose a query. Here is a random entry:
inurl:ftp "password" filetype:xls
Enter it into Google.com with the following modifications. Add the site: option followed by a domain name that is part of your rules of engagement:
site:example.com inurl:ftp "password" filetype:xls
In the case of this example, if there are any results found, you have located an MS Excel file that contains some form of password. Remember that results will vary and the best Google search queries are usually focused on determining the versions of installed software, seeking out known vulnerable installations that will later be targeted if allowed by the rules of engagement.
You should also be performing focused searches that locate all major document types such as .pdf, .doc, .txt, .xls, and more. However, there are some additional tools that will help us with this.
Tip
Warning: Do not open random files on your primary testing machine. You should have a separate machine (not connected to your network or the Internet) that can be used to open unknown (that is potentially harmful) files and media. One of the easiest methods of gaining access to a machine is through sending a file to a user that uses exploits to open up a system to an attacker. Opening unknown files in an uncontrolled environment would be reckless. Don't be that user.
To understand the types of queries you will see when browsing Exploit-DB's Google Hacking Database (GHDB), you must understand the types of operators that are used. Here is a list of the more common advanced operators:
|
Filter |
Description |
Example |
|---|---|---|
|
|
Search for all terms in a URL |
|
|
|
Search for all terms in the page text |
|
|
|
Search for term in the page title |
|
|
|
Displays cached pages |
|
|
|
Searches phonebook listings |
|
|
|
Search Google Groups for items by a specific author (Use Google Groups search for this) |
|
|
|
Searches for all documents of a specific type |
|
|
|
Restrict your search to a specific site (or domain) |
|
|
|
Find all pages that point to a specified URL |
|
By now, you should have some usernames, and possibly even some phone numbers and job titles. This information will come in handy if you are planning on performing a social engineering test.
Tip
Search engines such as Google can be used to search for information that corporate employees are dropping on the Internet as easily as you could search for a pie recipe. Be sure to verify that your client wants you to do research on employees before you start, not after. There are many laws that protect the privacy of an employee and only a lawyer can let you know what is acceptable and what is not.
One practice that seems to be prominent in penetration testing is to search for forum and group postings made by employees that may include information relating to work assets. Most of the information will not be shared with the world in a malicious manner, but rather innocently. This does not change the fact that attackers have access to this information and could possibly use it against a targeted company. Look for things such as an administrator of the company asking for help on configuring a specific firewall type or other network devices. A security professional that posts questions on a public forum may be unintentionally providing clues as to which standards their company complies with. These are the types of information that give you, the penetration tester, as well as an advanced attacker, the knowledge necessary to penetrate an otherwise secured environment.
Here are some tools that would assist you in finding more information:
|
Name |
Description |
Location |
|---|---|---|
|
Search Diggity |
Leverages search engines, such as Google, Bing, and Shodan, to quickly identify vulnerable systems and sensitive data in corporate networks. |
http://www.bishopfox.com/resources/tools/google-hacking-diggity/ |
|
Site Digger 3.0 |
Searches Googles cache. Finds all sorts of information. Requires .NET Framework 3.5 to work. |
http://www.mcafee.com/us/downloads/free-tools/sitedigger.aspx |
|
The Harvester |
Searches for Subdomains, Hostnames, Users, Employee e-mails, and names from search engines and PGP servers. |
Included in Kali or https://github.com/laramies/theHarvester |
|
Lullar.com | ||
|
White Pages | ||
|
PeekYou | ||
|
TinEye | ||
|
Internet Archive |
Personal favorite, archives copies of websites and files for years and years. |