Chapter 16

Azure Diagnostics and Debugging

What’s In This Chapter?

• Defining Windows Azure Diagnostics
• Comparing local debugging to cloud debugging
• What is Intellitrace?
• How to use profiling
• How to use Windows Azure Diagnostics
• How to view your logs
• Working with Windows Azure MMC

Whenever you write an application, you debug or diagnose it. The reasons for this are numerous—from bugs to performance issues to testing changes in your algorithms or code. Windows Azure provides both debugging and diagnostic capabilities, and you can use both together to get to the root cause of any issues that arise in your application. In addition, Visual Studio provides rich tools so that your debugging does not need to happen against the live instance of your service; you can debug against an emulated environment, which makes the debugging process available anytime and anywhere.

What Is Windows Azure Diagnostics?

Windows Azure Diagnostics enables you to collect data about your Azure application so that you can use that data to debug, troubleshoot, measure performance, or perform other development tasks. You can run Windows Azure Diagnostics locally or remotely and configure it separately from your Azure application. This flexibility enables you to work with diagnostics without changing your Azure application. In addition, you can transfer Windows Azure Diagnostics either on a schedule or on-demand to a Windows Azure storage account, which allows you to persist and build a data warehouse of performance metrics over time.

Windows Azure Diagnostics enables you to collect a number of different logs including the Windows Azure logs, IIS logs, and Windows Azure Diagnostics infrastructure logs, in addition to other sources, such as the Windows Event log and crash dumps. Beyond logs, you can also use tracing and debugging to help you understand where issues may occur in your code. Optionally, you can also leverage performance counters to understand the health of your Azure application. These performance counters can either be built-in or custom performance counters that you create.

Local Versus Cloud Debugging

For debugging you have your choice to either debug:

• Locally on your development environment using the emulated Azure environment
• Using the remote desktop and the remote debugger to debug your running Azure worker role instances

Although both have their merits, debugging is easier against your local instance while leveraging diagnostics as a viable alternative to an attached debugger for your production Azure instances running in the cloud.

Your local debugging session may not uncover all your issues because it is fundamentally different from running your code in the cloud. For example, your network or machine utilization is different locally versus running in the Microsoft datacenter. Therefore, if your application fails because of an overloaded network or starvation for resources, you may not find those issues when performing a local debugging session.

Because debugging is based on the built-in Visual Studio debugger, you do not need to perform special steps to start debugging except setting a breakpoint and starting your application with debugging enabled.

Using Intellitrace

Intellitrace is a feature of Visual Studio 2010 Ultimate that enables you to debug your applications by showing you events that occurred during the running of your application and the context surrounding those events. You can configure Intellitrace for the events to collect and allow for the collection of additional data, such as parameters and return values.

Configuring Intellitrace is easy with Azure. Check the Enable Intellitrace for .NET 4 Roles box that turns on Intellitrace for your Azure deployment, as shown in Figure 16-1.

When enabled, you need to deploy your solution for Azure. When initialized, you can view the Intellitrace logs from your Azure instance, as shown in Figure 16-2.

Figure 16-1

Figure 16-2

Using the Intellitrace logs, you can track down both the events and calls that happened in your instance and look back in time to understand what went wrong. Intellitrace is a little bit like the movie Back to the Future in that you can go back in time and understand the events that lead to the current situation, the variables and threads involved, and the state of the machine at the time. You should not use Intellitrace on production instances of your Azure application because it will slow performance.

Using Profiling

Beside Intellitrace, you can also use profiling to debug your applications. One restriction is that you can use only Intellitrace or profiling but not both at the same time. Many times you should first use Intellitrace to look for any exceptions in your application and try to fix those exceptions. Then, you can enable profiling, which can help you identify the most time-consuming areas in your code.

With Azure profiling, you can activate the following settings, as shown in Figure 16-3.

Figure 16-3

• CPU Sampling: With this option, you can collect different methods including CPU sampling, instrumentation, .NET memory allocation, and concurrency. You can use CPU sampling to track down any CPU spikes (such as runaway code). Sampling has a low impact on your application.
• Instrumentation: This collects information about the entry, exit, and time in a function. This enables you to detect functions that take a long time to execute or take a long time to get or receive data.
• .NET Memory Allocation: This lets you know if you’re utilizing memory in an efficient way, if you’re allocating too much memory, or if you’re not destroying objects correctly.
• Concurrency: This collects process and thread execution data, which enables you to identify resource contention and deadlocking in a multithreaded application. You can determine if other code in your application is blocking and therefore starving your code of CPU or resources. You can use only one collection method at once, such as CPU sampling or instrumentation. If you need to collect using different methods, you must change your profile settings and redeploy your Azure instance.
• Enable Tier Interaction Profiling: If you use ADO.NET in your application, this allows you to understand the execution times for your synchronous ADO.NET calls. For example, you can track elapsed time either in the process or in the queries, the number of queries, or the query command text.

After you deploy your Azure instance and initialize it, you can have Visual Studio download the profiling data and view the profiling reports. From there, you can troubleshoot your application to understand where the biggest demands of resources or time take place. Figure 16-4 shows a profiling report against an Azure instance.

Figure 16-4

Using Windows Azure Diagnostics

When you work with your Azure applications, there are times when you want to collect diagnostics about your application for debugging or troubleshooting purposes. By allowing you to collect diagnostic information and persisting that information, Azure ensures that your application runs at the optimal configuration over time without forcing an intrusive process.

Configuring Azure Diagnostics

To configure Azure Diagnostics, you need to import the Azure Diagnostics module into your service model. This is as simple as adding an Import element to your ServiceDefinition.csdef file, as shown in the following code:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="MyHostedService" xmlns="http://schemas.microsoft.com/
ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="WebRole1">
    <Imports>
      <Import moduleName="Diagnostics" />
    </Imports>
  </WebRole>
</ServiceDefinition>

If you persist your diagnostic information to your Azure Storage account, you must also specify a connection string in your ServiceConfiguration.cscfg file so that Azure knows how to export the diagnostic log and store it in Azure storage. The XML for the connection string is shown in the following code block. It shows how to use the storage emulator for storage and then shows passing in an account name and key to store the log in an Azure storage account.

<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="AzureService" xmlns="http://schemas.microsoft.com
/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1" osVersion="*">
  <Role name="WebRole1">
    <Instances count="1" />
    <ConfigurationSettings>
      <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
 value="UseDevelopmentStorage=true" />
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

<ConfigurationSettings>
   <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"   
   value="DefaultEndpointsProtocol=https;AccountName=AccountName;
AccountKey=AccountKey"/>
</ConfigurationSettings>

Be default, Azure does not collect all available diagnostics data so that it can save system resources. Table 16-1 shows which diagnostics data is collected and how that data is formatted.

Table 16-1: Azure Diagnostic Collection

To add the logs not collected by default, you need to perform different procedures based on the type of log. The next sections step through collecting each of the different nondefault logs.

Collecting IIS Failed Requests

To collect IIS failed requests, you need to modify the web.config for your web role in Azure. You must add code to the system.webServer section to include the providers you want to track, such as Authentication, CGI, Security, and so on. In addition, you can configure the HTTP status codes that you want for collection. The following code shows how to modify the system.webServer section to turn on the IIS failed request collection.

<tracing>
   <traceFailedRequests>
      <add path="*">
         <traceAreas>
            <add provider="ASP" verbosity="Verbose" />
            <add provider="ASPNET" areas="Infrastructure,Module,Page,
             AppServices" verbosity="Verbose" />
            <add provider="ISAPI Extension" verbosity="Verbose" />
            <add provider="WWW Server" 
                 areas="Authentication,
                        Security,
                        Filter,
                        StaticFile,
                        CGI,
                        Compression,
                        Cache,
                        RequestNotifications,
                        Module" 
                 verbosity="Verbose" />
         </traceAreas>
         <failureDefinitions statusCodes="400-599" />
      </add>
   </traceFailedRequests>
</tracing>

Collecting Windows Event Log and Crash Dumps

To collect the event log information, you need to write a bit of code after you deploy your Azure instance with the diagnostic monitoring enabled. To do this, you want to make sure your code references the Microsoft.WindowsAzure.Diagnostics.dll file. Then, you write code in your OnStart method for your role to configure the diagnostics module and add in your Windows Event Log datasource, such as System or Application. The following code shows you how to configure the Application event log. You cannot access the Security event log because of the locked down permissions that your Azure service accounts runs under.

using Microsoft.WindowsAzure.Diagnostics;
var diags = DiagnosticMonitor.GetDefaultInitialConfiguration();
diags.WindowsEventLog.DataSources.Add(“Application!*”);
//Set the timeframe for collection
diags.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
//Restart the Diagnostic monitor with changes
DiagnosticMonitor.Start(
“Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString”, diags);

A couple caveats exist when working with event log collection:

• Unless you use filtered events, you collect all events into the log, both your own applications as well as the system. The preceding sample would dump all events in the Application log. Azure will charge you for the storage and transactions.
• The log can list a lot of events, so you must make sure to limit the amount of events. To filter events, use an XPath expression to specify the filter. The easiest way to get the XPath expression is to use the local event viewer on your dev machine, filter for the event you want in the viewer, and then switch to the XML view of the event viewer custom view creator, as shown in Figure 16-5. From there, you can get the XPath expression, such as Application!*[System[Provider[@Name=‘HttpEvent’]]].

Figure 16-5

• To collect crash dumps, you need to enable the collection of crash dumps through the API. A value of false collects a partial crash dump, and true collects a full crash dump. In addition to using EnableCollection, you can also use EnableCollectionToDirectory method, which enables you to pass a string that is the directory to write the logs to.

  //Put true for complete crash dump collection
  Microsoft.WindowsAzure.Diagnostics.CrashDumps.EnableCollection(false);

Adding Tracing

Azure also enables you to use tracing as a means to monitor the execution of your code. You can use three classes: System.Diagnostics.Trace, System.Diagnostics.Debug, and System.Diagnostics.TraceSource to trace your application. To do so, you initialize the trace listener. If you use Visual Studio, this step is done for you, but if you use a different tool, the following code shows how to configure the trace listener in your web.config or app.config file.

<system.diagnostics>
   <trace>
      <listeners>
         <add type=
"Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, 
            Microsoft.WindowsAzure.Diagnostics, 
            Version=1.0.0.0, 
            Culture=neutral, 
            PublicKeyToken=31bf3856ad364e35"
            name="AzureDiagnostics">
            <filter type="" />
         </add>
      </listeners>
   </trace>
</system.diagnostics>

When enabled, you can use Trace statements to write out to the trace log. For example, if you want to write out a statement to the trace log, you can use Trace.WriteLine to add a new text line to the trace log.

Using Performance Monitors

In addition to using diagnostics and tracing, you can also use the performance monitor to track the performance of key counters in your Azure applications. As with the other methods, you must modify your OnStart method in your web role to enable your performance counter monitoring. As with event logs, you must specify the path to your performance counters. For example, if you want to collect the % Processor Time, the path would be Processor(_Total)\% Processor Time, or if you want to collect the available memory, you use MemoryAvailable MBytes. The easiest way to figure out these paths is to use performance monitor to view these counters and construct their paths from the user interface navigation.

When you have your path, you can configure Azure Diagnostics to start collecting your performance counters similar to the way you did for the other diagnostic technologies, as shown in the following code:

using Microsoft.WindowsAzure.Diagnostics;
var diags = DiagnosticMonitor.GetDefaultInitialConfiguration();
diags.PerformanceCounters.DataSources.Add(new PerformanceCounterConfiguration()
    {
        CounterSpecifier = @"Processor(_Total)\% Processor Time",
        SampleRate = TimeSpan.FromSeconds(5)
    });
diags.PerformanceCounters.ScheduledTransferPeriod = TimeSpan.FromSeconds(1d);
diags.PerformanceCounters.BufferQuotaInMB = 512;
//Set the timeframe for collection
diags.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
//Restart the Diagnostic monitor with changes
DiagnosticMonitor.Start(“
Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString”, diags);

Viewing Your Logs

Viewing your logs is straightforward whether you use the local debugging environment or your Azure Storage Account using Visual Studio. In the Server Explorer in Visual Studio, you can open either your local storage environment or the Azure Storage environment to view both your blobs and your tables. Make sure that you enable your Azure Storage account by using the Properties dialog for your Web Role in Visual Studio and provide the account and key for your Azure Storage account, as shown in Figure 16-6.

Figure 16-6

You also must configure Azure to ship the logs over to your storage account by modifying the WebRole.cs under the OnStart() method, as shown earlier in the chapter. After you make all these changes, you can deploy your solution to Azure and start viewing your logs in Visual Studio, as shown in Figure 16-7.

Figure 16-7

Windows Azure MMC

When working with diagnostics, try to leverage as many tools as you can to quickly configure and deploy your diagnostic settings and code. Visual Studio provides a number of tools (such as Intellitrace, discussed earlier in the chapter) but there are also third-party tools that you should consider. Windows Azure MMC is a tool that enables you to monitor and manage your Azure deployment. You can upload, deploy, or upgrade your Azure instances. You can also configure the diagnostics for Azure, which may be faster than writing all the configuration code. You can find the MMC tool at http://wapmmc.codeplex.com/. Figure 16-8 shows the Windows Azure MMC interface.

Figure 16-8

Summary

In this chapter, you learned how to monitor and debug your Azure applications. By leveraging technologies such as Intellitrace, profiling, and diagnostics, you can ensure that your Azure applications run in a scalable manner, and if you do encounter errors, you can figure out those errors more quickly.