The simplest definition of network latency is the time it takes for a data packet to move across a network connection. The lower the latency between your Web application tiers, the faster the response times. Latency and bandwidth are two factors that determine the speed of a network connection. An easy way to determine the network latency between your client and a Web application is to use the pathping utility, or other utilities such as the Visual Trace Route feature provided in Application Expert. A PowerPoint presentation of Compuware’s Application Expert can be found on this book’s companion CD.

Every local area network (LAN) and wide area network (WAN) connection is constrained by network latency. Factors that cause latency to increase are poor quality network devices, longer distances causing communication travel through more hops or network devices, and network congestion. For example, a typical 56-kbps local modem connection may have 200 milliseconds of latency, while a similar connection traveling a farther distance may pass through more network devices, which increases the latency to 500 milliseconds. In Table 5-1, we captured a trace with Network Monitor to measure the impact of network latency when requesting the 46-kilobyte home page of IBuySpy with our Microsoft Internet Explorer Web browser. The capture was imported into Application Expert’s Response Time Predictor, and we are able to extrapolate the response times for both a 200-millisecond and 500-millisecond latency 56-kbps connection. We’ll discuss this Application Expert feature and others in the “Using Compuware Application Expert” section later in this chapter.

The impact of the additional 300 milliseconds of latency for this particular page view and all associated elements is 2.5 seconds, not 300 milliseconds. The reason for this is latency affects every application round trip.

A network round trip is a client-server request-response pair generated by an application. A request-response pair sent from a Web browser to a Web server and back is considered a single round trip. For example, when you type in a URL in your Web browser, such as http://www.microsoft.com, each image, style sheet, or other page element defined in the response to your request (from the Web server) is counted as a separate network round trip. Additional round trips are involved in the setup of each connection to the Web server in this example. Later in this chapter we will discuss how Application Expert can be used to quickly obtain the number of round trips your ASP pages are consuming. Combined with latency, round trips have a profound effect on application response time. Imagine that your IIS servers are located on both the US East and West Coasts. You are located in Mountain View California and viewing your IBuySpy Login ASPX page on the West Coast IIS server takes 4 seconds while viewing it from the East Coast IIS server takes 6.5 seconds. Further investigation shows the latency for your connection to the East Coast server is 500 milliseconds while that to the West Coast server is 200 milliseconds. Because every round trip is affected by network latency, the East Coast response time is longer by approximately 2 seconds.

For most application developers, network latency on the Internet is out of their control. This is why it is important to develop applications that use as few round trips as possible to keep response times quick even under high network latency conditions.

The most effective method for reducing round trips is to reduce the number of objects per page. Table 5-2 shows two home pages for major Internet search sites that many of you have probably used before. The first home page has fewer than six objects and loads very quickly. The second home page has approximately 15 objects and loads much more slowly. Notice the difference in the number of round trips that are generated when each of these home pages are loaded.

Search Site 1 used in Table 5-2 was able to limit the number of objects to six and has a less than 10-second response time on most 56-kbps connections. They limit the number of objects to six by creating one large graphic at the top of their home page, which consists of eight images. Considering each object requested will create a network round trip, they have saved seven round trips by merging their eight small graphics into a single request.

Data transferred is the amount of data moved between a client Web browser and Web server and is often measured in kilobytes. The best approach for measuring data transferred is to break down the scenarios within your application by page views including their associated elements. For example, the IBuySpy home page transfers 46 kilobytes of data non-cached the first time you access it and 24 kilobytes of data cached (subsequent requests with many objects loaded from your Web browser cache located in the temporary Internet folder). Based on our experience, the amount of data transferred for a page view and associated elements of 50 kilobytes or less will give an acceptable response time. This allows low speed Internet connections like 56-kbps modems the best chance of having fast response times. This chapter uses several terms to quantify data, such as kilobits per second and kilobytes of data transferred. Therefore to better illustrate how these metrics relate, Table 5-3 shows the conversion from bits to bytes, kilobits to kilobytes, and kilobytes to megabytes.

There are several techniques for reducing the amount of data being transferred between Internet Explorer and the IIS tier. These methods include IIS compression, removing unnecessary characters and white space, reducing the number of objects on pages, and optimizing images.

HTTP Compression is a built-in feature of IIS and Internet Explorer. This tool works well for static content like HTM files, HTML files, CSS files, JS files, and uncompressed images. However, with dynamic content you may run into some problems, such as pages rendering incorrectly or becoming non-functional. There is also additional overhead associated with resources on your IIS server if you choose to use IIS compression. By default, HTTP Compression is disabled and can be enabled using the WWW Service Master Properties Services tab, shown in Figure 5-1.

Figure 5-1. Enabling IIS compression

Once HTTP Compression is enabled, you must specify the file types to be compressed by editing the metabase. The following steps show how to edit the metabase settings to compress static TXT, JS, CSS, DOC, and XLS files:

Another method to trim down the amount of data transferred between the browser and IIS tier is to package the data more tightly by removing all unnecessary characters and white space from your code. This is particularly useful around loops when a lot of data is returned to the client. Also, many developers put comments into their code, which adds additional characters and lines to the page, resulting in more data transferred.

Reducing the number of objects on your page will decrease the amount of data transferred. Many sites use images for bullets and text instead of using formatted text. By limiting the number of elements and using colored text and background colors to make sections of the page stand out, you can reduce the amount of data transferred and network round trips.

Images make a Web site or application more visually appealing but there is a significant performance cost associated with using them. Use Graphics Interchange Format (GIF) images whenever possible. Using optimized or compressed images will reduce the amount of data transferred and typically does not impact the clarity of your pictures much. Most high quality imaging software suites will include a tool to optimize and compress your image files. Reuse images for your company logo, navigational bars or other images that are used throughout your application. Make sure you use the exact same name and path or Internet Explorer will treat it as a new image. After the initial fulfillment of the request Internet Explorer will cache the images locally. Another option is to preload images using JavaScript. This will load the images in the background while the rest of the page loads. Subsequent requests for the cached image will appear more quickly to the end user because the images will be cached locally in Internet Explorer.

When client requests spend unnecessary time on the middle or back-end tier of an application, it is considered a processing delay. This can be due to poorly written code that can take several seconds to process, or poor SQL tuning and improper indexing causing stored procedures and other SQL commands to consume valuable time. Processing delays directly impact response times. To get the most gain out of our tuning efforts, we typically focus on delays longer than 1 second. However, your application may have stricter or higher standards depending on a number of variables including type of application and size of your user base. For example, if your application is designed to render large reports to a small group of decision support analysts, response time may be expected to be slower. This is because larger reports may take longer to process on your database server, and given that your user base is small, budget constraints may limit the amount of time you can spend optimizing this application. However, if your user base is very large and consists of customers ordering your products, you will spend more time optimizing the application’s response time to ensure that no sales are lost because customers are frustrated about slow response times.

The most common way to speed the response time of your application is to reduce application-processing delays. This is accomplished by speeding up slowly executing ASPX, Managed Code, and poor SQL code. More details are available in the IIS, Managed Code, and SQL sections of this book. As discussed earlier in the "Identifying Application Processing Delays" section, by using Network Monitor display data view or Application Expert bounce diagrams, you can identify whether or not your page view and associated elements have processing delays and which tier is causing it. Next, determine where in your application’s code the processing delay is happening. There are several methods for profiling and identifying the code causing processing delays:

In reference to performing an application network analysis, response time refers to how long it takes data to travel from the client’s initial request to the completion of the request. Improving performance of your application is synonymous with reducing the overall response time for each of your application’s page views including associated elements. Some factors that constrain response times are a high number of round trips, low bandwidth, end users with high latency connections, a large amount of data transferred, and processing delays on the IIS code, managed code, or SQL code. We describe the process of using Application Expert to quickly calculate response times later in this chapter.

Response time acceptability depends on a variety of factors including the type of application, the function of each requested page within the application, the bandwidth and latency of your Internet connection. For instance, a requested page that has no interaction with a back-end database (that is, a static page view) or has a small amount of content should load faster than requested pages that are heavy in content or constantly retrieve or update data from a database (that is, a dynamic page view) will be slower. Table 5-4 shows the response times we consider acceptable when performing an application network analysis. Low response times represent applications with small static page views or very light interaction with a database. High response times represent applications with large page views or heavy interaction with a database.

User scenarios are a collection of page views, including their associated elements, that implement business or functional processes. For example, searching the IBuySpy site (discussed in Chapter 3) is a good example of a real-world user scenario. The search scenario consists of several steps, such as loading the home page, entering a search string, clicking the GO button, selecting a product from the search results and viewing the product detail page. This scenario consists of two page views. Instead of capturing the entire user scenario, we recommend you conduct this analysis by separating the scenario into distinct user actions, like clicking a button or link to submit a form that makes a request to the Web server. The reason we isolate the user scenarios into separate page views or user actions is to help identify the exact causes of slow response times.

In preparing for your network captures, you should document all node MAC addresses and IP addresses. To do this, open a command prompt and type ipconfig/all from each back-end server and client. The output from this command will show the IP address information, DNS servers, DNS suffix, physical or MAC address, and other valuable information. This will make it easier to set up your capture filter and isolate network traffic created by the application. Often you can use a host file entry to force nodes to communicate with the correct NIC and IP addresses you are capturing on. The hosts file is typically located in the C:windowssystem32driversetc folder and does not have a file extension. To add or change entries you can open the hosts file with Microsoft Notepad and follow the examples provided within the file.

Network Monitor needs to be properly configured before capturing data from your user scenarios. There are three key settings—the capture filter, capture buffer, and network—that can be adjusted to make sure you are creating a correct and complete capture.

Capture filters allow you to narrow down your network captures to the tiers or nodes that are used by your application. This makes it easier to isolate data transferred and round trips without analyzing unnecessary network traffic in the trace file. Figure 5-3 shows a typical capture filter.

Figure 5-3. Network Monitor capture filter

The capture buffer is an important setting to adjust. By default the buffer is set to 1 megabyte. If a page is transferring a large amount of data you may exceed this 1 megabyte limit. Typically we adjust the capture buffer to 10 megabytes to accommodate any large captures. Figure 5-4 shows the Network Monitor Capture Buffer Settings dialog box.

Figure 5-4. Network Monitor Capture Buffer Settings dialog box

It’s important to make sure you capture on the correct network. With clients and servers that have more than one NIC, you need to set which network you want to capture on. To verify you are capturing on the correct network, start a Network Monitor trace and ping the IP address or URL of the Web application you are testing. Next, stop your Network Monitor trace and verify you collected the ping traffic. If not, switch Network Monitor to capture a different network and repeat until you capture on the correct network. Figure 5-5 shows where you set Network Monitor to capture on the proper network.

Figure 5-5. The Network Monitor Select A Network dialog box

The hardware used in an application network analysis includes the test client, and all of the back-end servers that make up the application (IIS Server, SQL Server, Application Server) and any other network devices used (switch, hub, router, bridge, load balancer, etc.). Applications that require high availability and scalability typically use more than one identical Web server to load balance traffic volume. If your application is load balanced, you should ensure that only a single node (front-end Web server) is enabled during an application network analysis. The purpose of this is to make sure you are capturing traffic on the correct Web server. There are numerous ways to use Network Monitor to capture traffic created by your application. Here are a few:

Now that Network Monitor is installed and configured correctly and your environment is set up, you are ready to begin capturing network traffic. We have discovered a few tricks that will assist you in gathering captures:

Network Monitor Capture Structure

A Network Monitor capture file consists of two views, the capture window and the display data view. As illustrated in Figure 5-12, the capture window calculates the amount of data transferred between nodes.

Figure 5-12. The Network Monitor capture window

The display data view contains nine columns and can contain many rows, depending on the amount of data collected. By default the trace file will show the MAC address of the NIC. To make the trace easier to read, edit the address to show the machine name or machine type. Figure 5-13 illustrates a raw capture from a client browser to an IIS server to a SQL server and back.

Figure 5-13. The Network Monitor display data view

All data in these views is important, however, the critical data to focus on identifies the amount of data transferred, number of round trips, and total time.

Application Expert provides advanced response time prediction capabilities and granular application-level details for tuning distributed applications to ensure optimal network performance. Application Expert is used as a packet sniffer or for importing binary capture files obtained through Network Monitor. Application Expert also can be used as a fully functional packet sniffer. We find the following features of Application Expert to be particularly useful when performing a network analysis:

Conversation Map

Once a network capture has been imported into Application Expert, you can model the conversation between applications tiers. Figure 5-14 shows the conversation map of IBuySpy home page (no cache). This figure shows how quickly the conversation map is used to determine the amount of data transferred and number of round trips for a page view including its associated elements.

Figure 5-14. Application Expert conversation map

The conversation map provides a top-level view of the nodes (Client, IIS, and SQL) and conversations associated with hitting the home page. The conversation is defined as the data transferred or communication between the various nodes in your application. You can view the conversations by bytes or select your choice of display metric, such as payload bytes, frames, or application turns or round trips by right-clicking to open the Conversation Map context menu. Referring back to Figure 5-6, we see a total of 46 kilobytes of data being transferred between the IE client and IIS server and a total of 18 round trips.

Bounce Diagram

The bounce diagram is an excellent feature that gives you the packet level view of your application tiers and allows you to identify processing delays. Figure 5-15 shows the IBuySpy Default.aspx bounce diagram.

Figure 5-15. Application Expert bounce diagram

The bounce diagram is a packet flow timing diagram for client/server conversations on the network. This view shows individual frames between nodes in the sequence in which they occurred during the capture. In Figure 5-15, the IBuySpy login page shows a very small processing delay of approximately 170 milliseconds between the Internet client and Web server.

In the bounce diagram, each vertical line represents a node and the horizontal lines on the grid represent relative time increments in seconds. The time is relative to the start of the trace, which is designated 0.00. Within the diagram, the colored horizontal lines represent captured frames. The size of each frame is indicated by a color described in the legend at the bottom of the window: red for fewer than 100 bytes, yellow for up to 512 bytes, green for up to 1024 bytes, and blue for more than 1024 bytes.

Thread Analysis

It is helpful to look at the threads and thread timing using the thread analysis view in conjunction with the bounce diagram to identify which requests are causing processing delays. This combination view lets you focus thread-by-thread. The panes are dynamically linked, so an action in the upper pane affects the display and data in the lower pane to give you multiple views. For example, when you click a thread in the upper pane, the display in the lower pane changes to show the frames associated with the selected thread. If necessary, Application Expert automatically zooms into the bounce diagram for you. When you click a frame displayed in the bounce diagram, you get the detailed information on that thread. Figure 5-16 shows a thread analysis of the IBuySpy Default.aspx page. Using the thread analysis feature in Application Expert, you can quickly identify all 16 objects associated with this page view. Identifying this information from the raw Network Monitor capture or Application Expert packet trace view takes much longer.

Figure 5-16. Application Expert thread analysis

To analyze your capture in more detail, drill into the packet trace for a specific frame by selecting the Drill Into Packet Trace option in the context menu or, in a bounce diagram view, double-clicking a frame. This view is similar to the way Network Monitor displays the raw capture. The packet trace view lists all of the packets and provides important information about each one including the frame number, acknowledgement number, window size, and total bytes (data transferred) versus payload bytes (total bytes minus overhead of headers and acknowledgement frames). The bounce diagram view, thread analysis view, and conversation map are visual representations of the raw data provided in the packet trace view. Figure 5-17 displays a packet trace of the IBuySpy Default.aspx page.

Figure 5-17. Application Expert packet trace view

Response Time Predictor Tool

The Response Time Predictor allows you to predict the impact of bandwidth and latency for overall response time. This is very useful in determining response times of your Web pages in a pre-production test environment. Having this data you can identify and fix performance problems before your customers begin using your application. For instance, if you have several page views that do not meet your criteria for response time acceptability, you have a chance to optimize them before your customers begin using your Web application. The predicted response time depends on a number of factors, including the number of network round trips, amount of data transferred, and processing delays which are modeled using algorithms developed by Compuware. In the Response Time Predictor, you specify the bandwidth and the latency parameters to characterize the production network. The bandwidth and latency should be defined for the slowest link between the communicating end nodes. Figure 5-18 shows a Response Time Predictor model for IBuySpy Home page.

Figure 5-18. Application Expert Response Time Predictor

In previous sections of this chapter we identified and defined key components that are used throughout the application network analysis. We introduced Network Monitor, the application used to capture the network traffic between application tiers. Next, we will provide our method for analyzing applications for network efficiency and identifying response time bottlenecks.

The capture file is in a format that can be viewed in Network Monitor or imported into Application Expert. The capture file is not easily interpretable and can be intimidating for novice users. By using Application Expert to interpret the capture, valuable time can be saved. To import your Network Monitor capture files into Application Expert, follow these steps:

Calculating Data Transferred

After performing the network captures, we isolate traffic between application tiers. Calculating data transferred with Application Expert is accomplished using the conversation map. Figure 5-19 shows there are approximately 15 kilobytes of data transferred between the Internet Explorer client—IIS server tier and 4 kilobytes of data transferred between the IIS server—SQL server tier for the IBuySpy Checkout.aspx page view including their associated elements.

Figure 5-19. Calculating data transferred with Application Expert

Counting Network Round Trips

Counting the number of network round trips is accomplished using the Application Expert conversation map. Figure 5-20 shows there are five round trips between the Internet Explorer Client—IIS server and 10 round trips between the IIS server—SQL server for the IBuySpy Checkout.aspx page.

Figure 5-20. Counting round trips with Application Expert

Identifying Application Processing Delays

Processing delays on the IIS or SQL server tiers will critically impact end user response time performance. These delays are caused by several factors such as slowly executing scripts or components in your ASPX pages, SQL stored procedures requiring optimization, or improper SQL indexes that add seconds to your page load time. There are a number of tools and methods you can use to determine if your application has a processing delay. One such tool is SQL Profiler to monitor the SQL server tier, looking at the IIS logs “time taken” field to estimate IIS processing duration, or using the Application Expert bounce diagram feature to identify excessive time between network frames.

Note

For debugging and testing purposes we recommend enabling logging within IIS and using the W3C Extended Log file format. This includes enabling all of the extended properties, such as time taken, bytes sent, and bytes received. For more information relating to the W3C Log file format, see Chapter 6.

If you find delays of more than a second between frames, this should be flagged and investigated for optimization. If you reduce this delay by making code changes, the overall response time for that particular page view will be reduced by the same amount. Figure 5-21 shows a 1.3-second processing delay on the IIS server tier for the IBuySpy update cart page.

Figure 5-21. Processing delay shown in Application Expert