Task Cancellation

To cancel a Task that is already running requires some cooperation. It is never a good idea to force-terminate a thread, and the Task Parallel Library does not allow you to access the underlying thread, let alone abort it. (If you do program directly with a Thread object, then you can call the Abort method, but this is dangerous and not recommended. Just pretend this API does not exist. Aborting a thread can leave synchronization objects in unknown states, shared state could be corrupted, and asynchronous operations may never complete.)

To cancel a Task, you need to pass a CancellationToken object to the delegate where it can then poll the token to determine if it needs to end processing. This example also demonstrates using a lambda expression as a Task delegate.

You can find this code in the sample TaskCancellation project.

static void Main(string[] args)
{
  var tokenSource = new CancellationTokenSource();
  CancellationToken token = tokenSource.Token;

  Task task = Task.Run(() =>
  {
    while (true)
    {
      // do some work...
      if (token.IsCancellationRequested)
      {
        Console.WriteLine("Cancellation requested");
        return;
      }
      Thread.Sleep(100);
    }
  }, token);
  
  Console.WriteLine("Press any key to exit");

  Console.ReadKey();

  tokenSource.Cancel();
  
  task.Wait();  

  Console.WriteLine("Task completed");    
}

Handling exceptions

If a Task throws an exception during its execution, what happens depends on how the Task and its result are handled. The naive way is to just access the Result property. If there is no result because the Task threw an exception, then the thread that accesses the Result property or calls Wait explicitly will get an AggregateException.

var task = Task<int>.Factory.StartNew(() =>
{
    int x = 42;
    int y = 0;
    return x / y;
});
task.Wait(); // exception here!
int result = task.Result;

If you are not explicitly waiting, but have a continuation, you have the same problem:

Task<int>.Factory.StartNew(() =>
{
    int x = 42;
    int y = 0;
    return x / y;
}).ContinueWith(task =>
{
    int val = task.Result; //exception here!
});

There are a few solutions. First, you can wrap the Result access in an exception handler:

Task<int>.Factory.StartNew(() =>
{
    int x = 42;
    int y = 0;
    return x / y;
}).ContinueWith(task =>
{
    try
    {
        // safely handle result
        int val = task.Result;
    }
    catch(AggregateException ex)
    {
        LogException(ex);
    }                
});

But we will learn later that throwing exceptions is bad. The Task class provides a number of properties to determine the state of a task:

• IsCanceled: The task was canceled.
• IsFaulted: An exception was thrown. The exception can be accessed via the Exception property.
• IsCompleted: True when the task is done executing, whether due to a fault, cancellation, or successful completion.
• IsCompletedSuccessfully: True when the task has completed without fault or cancellation.

You can use IsFaulted to detect whether there are unhandled exceptions:

Task<int>.Factory.StartNew(() =>
{
    int x = 42;
    int y = 0;
    return x / y;
}).ContinueWith(task =>
{
    if (task.IsFaulted)
    {
        LogException(task.Exception);
    }
    else
    {
        // safely handle result
        int val = task.Result;
    }
});

public static class ExceptionUtils
{
    public static void LogException(System.Exception exception)
    {
        LogExceptionRecursive(exception, 0, null);
    }

    private static void LogExceptionRecursive(System.Exception ex, 
                                              int recursionLevel)
    {
        if (ex == null)
        {
            return;
        }

        if (recursionLevel >= 10)
        {
            // Just to be safe, have a recursion cutoff
            return;
        }

        Console.WriteLine(
          $"Type: {ex.GetType()}, Message: {ex.Message}," +
          $" Stack: {ex.StackTrace}, Level: {recursionLevel}");

        var aggEx = ex as AggregateException;
        if (aggEx != null && aggEx.InnerExceptions != null)
        {
            foreach (var inner in aggEx.InnerExceptions)
            {
                LogExceptionRecursive(inner, recursionLevel + 1);
            }
        }
        else if (ex.InnerException != null)
        {
            LogExceptionRecursive(ex.InnerException, 
                                  recursionLevel + 1);
        }
    }
}

<configuration> 
    <runtime> 
        <ThrowUnobservedTaskExceptions enabled="true"/> 
    </runtime> 
</configuration>

static void Main()
{
    AppDomain.CurrentDomain.UnhandledException 
      += OnUnhandledException;
}

private static void OnUnhandledException(
  object sender, 
  UnhandledExceptionEventArgs e)
{
    var ex = e.ExceptionObject as Exception;
    if (ex != null)
    {
        ExceptionUtils.LogException(ex, 
                                     Events.Log.UnhandledException, 
                                     true);
    }
    else if (e.ExceptionObject != null)
    {
        var type = e.ExceptionObject.GetType().ToString();
        Console.WriteLine(
          $"Non-exception object: {type} - {e.ExceptionObject}");
    }
    else
    {
        Console.WriteLine("Unknown object");
    }
}

Child Tasks

When tasks are created from within another task, they are, by default, independent from each other. This example creates a child task that executes arbitrary code. Both parent and child tasks write to the console, but they are completely unrelated to each other and either one can return and complete first.

Task.Factory.StartNew(() =>
{
    Task childTask = Task.Factory.StartNew(() =>
    {
        Console.WriteLine("Inside child");
    });
    
}).ContinueWith(task => Console.WriteLine("Parent done"));

This is usually what you want as it maintains strong asynchrony that is a good habit in this type of processing, but it is possible to create a stronger relationship between parent and child tasks. One way is to have the parent wait on the child task (or on the Result property of the child task):

Task.Factory.StartNew(() =>
{
    Task childTask = Task.Factory.StartNew(() =>
    {
        Console.WriteLine("Inside child");
    });
    
    // Explicit Wait!
    childTask.Wait();
}).ContinueWith(task => Console.WriteLine("Parent done"));

This is less than ideal—it introduces a blocking call into an otherwise asynchronous process, which is a violation of the cardinal rule of efficient asynchronous programs. An alternative is to pass the TaskCreationOptions.AttachedToParent flag, telling the parent that it cannot complete until all of its children do.

Task.Factory.StartNew(() =>
{
    Task childTask = Task.Factory.StartNew(() =>
    {
        Console.WriteLine("Inside child");
    }, TaskCreationOptions.AttachedToParent);
    
}).ContinueWith(task => Console.WriteLine("Parent done"));

While useful in circumstances where you want to enforce some coordination between tasks, this option does introduce new complexities that need to be dealt with.

First of all, it is possible for parent tasks to deny children the ability to attach themselves:

Task.Factory.StartNew(() =>
{
    Task childTask = Task.Factory.StartNew(() =>
    {
        // do work
    }, TaskCreationOptions.AttachedToParent);
    
}, TaskCreationOptions.DenyChildAttach);

In this case, the parent’s preferences are obeyed and the child’s request to attach is ignored. It behaves the same as the first example in this section. When you start tasks via Task.Run instead of the Task.StartNew method used here, TaskCreationOptions.DenyChildAttach is set by default.

You must also handle exceptions. An exception in an attached child Task is automatically passed onto the parent Task to be handled by the parent Task’s continuation method (or whatever thread waits on the task’s result). A detached child Task that throws an exception must be handled in the same way as any other code inside the parent task (check its Exception property, or wrap the Result property check in an exception handler).

Cancellation also becomes a bit trickier. To simplify things, you should use the same CancellationToken object for all tasks in a parent-child relationship. When cancellation is signaled, what happens is determined by where and when the cancellation happens:

• Parent cancels before child Task start: Child never starts.
• Parent cancels after child Task starts: Child Task executes until it checks for cancellation.
• Attached child cancels: A TaskCanceledException is passed up to the parent’s Task and will be included in its AggregateException.

A TPL Dataflow Example

To see how TPL Dataflow blocks can work together, we will create a simple example of a text processing pipeline. This application processes a directory of text files and analyzes the frequency of all the words in each file. There are some post-processing steps that combine the analyses and weed out some common, uninteresting words.

Here is the creation of the pipeline, which returns a reference to the first block:

private static readonly HashSet<string> IgnoreWords = 
    new HashSet<string>() { "a", "an", "the", "and", "of", "to" };
private static readonly Regex WordRegex = 
    new Regex("[a-zA-Z]+", RegexOptions.Compiled);

private static ITargetBlock<string> CreateTextProcessingPipeline(
    string inputPath, 
    out Task completionTask)
{
    int fileCount = Directory.GetFiles(inputPath, "*.txt").Length;

    var getFilenames = new TransformManyBlock<string, string>(
        path =>
    {
        return Directory.GetFiles(path, "*.txt");
    });

    var getFileContents = new TransformBlock<string, string>(
      async (filename) =>
    {
        Console.WriteLine("Begin: getFileContents");
        using (var streamReader = new StreamReader(filename))
        {
            return await streamReader.ReadToEndAsync();
        }
        Console.WriteLine("Begin: getFileContents");
    }, new ExecutionDataflowBlockOptions 
       { 
        MaxDegreeOfParallelism = Environment.ProcessorCount 
       }
    );

    var analyzeContents = 
      new TransformBlock<string, Dictionary<string, ulong>>(
    contents =>
    {
        Console.WriteLine("Begin: analyzeContents");
        var frequencies = 
          new Dictionary<string, ulong>(
            10000, StringComparer.OrdinalIgnoreCase);

        var matches = WordRegex.Matches(contents);
        foreach (Match match in matches)
        {
            ulong currentValue;
            if (!frequencies.TryGetValue(match.Value, 
                                         out currentValue))
            {
                currentValue = 0;
            }
            frequencies[match.Value] = currentValue + 1;
        }
        Console.WriteLine("End: analyzeContents");
        return frequencies;
    }, new ExecutionDataflowBlockOptions 
       { 
         MaxDegreeOfParallelism = Environment.ProcessorCount 
       }
    );

    var eliminateIgnoredWords = 
      new TransformBlock<Dictionary<string, ulong>, 
                         Dictionary<string, ulong>>(
    input =>
    {
        foreach(var word in IgnoreWords)
        {
            input.Remove(word);
        }
        return input;
    });

    var batch = 
      new BatchBlock<Dictionary<string, ulong>>(fileCount);

    // This is one point of synchronization -- 
    // all processing converges on this     
    var combineFrequencies = 
      new TransformBlock<Dictionary<string, ulong>[], 
                         List<KeyValuePair<string, ulong>>>(
    inputs =>
    {
        Console.WriteLine("Begin: combiningFrequencies");
        var sortedList = new List<KeyValuePair<string, ulong>>();
        var combinedFrequencies = new Dictionary<string, ulong>(
         10000, StringComparer.OrdinalIgnoreCase);
        
        foreach (var input in inputs)
        {
            foreach (var kvp in input)
            {                        
                ulong currentFrequency;
                if (!combinedFrequencies.TryGetValue(
                       kvp.Key, 
                       out currentFrequency))
                {
                    currentFrequency = 0;
                }
                combinedFrequencies[kvp.Key] = currentFrequency + 
                                               kvp.Value;
            }
        }
        foreach(var kvp in combinedFrequencies)
        {
            sortedList.Add(kvp);
        }
        sortedList.Sort((a, b) => 
        {
            return -a.Value.CompareTo(b.Value);
        });
        Console.WriteLine("End: combineFrequencies");
        
        return sortedList;
    }, new ExecutionDataflowBlockOptions() 
        { 
          MaxDegreeOfParallelism = 1 
        }
    );

    var printTopTen = new ActionBlock<List<KeyValuePair<string, 
                                                        ulong>>>(
      input =>
    {
        for (int i=0;i<10;i++)
        {
            Console.WriteLine(
              $"{input[i].Key} - {input[i].Value}");
        }
        getFilenames.Complete();
    });

    // Hook up blocks
    getFilenames.LinkTo(getFileContents);
    getFileContents.LinkTo(analyzeContents);
    analyzeContents.LinkTo(eliminateIgnoredWords);
    eliminateIgnoredWords.LinkTo(batch);
    batch.LinkTo(combineFrequencies);
    combineFrequencies.LinkTo(printTopTen);
    
    completionTask = getFilenames.Completion;

    return getFilenames;
}

To use the pipeline, you just need to retrieve the block reference and post a message to it:

Task completionTask;
ITargetBlock<string> startBlock = 
    CreateTextProcessingPipeline(args[0], out completionTask);

startBlock.Post(args[0]);

completionTask.Wait();

In the output, you can see from the console output that there are some blocks which execute concurrently, while others execute serially after previous blocks complete, and different types of blocks can be interleaved:

Begin: getFileContents
Begin: getFileContents
Begin: getFileContents
Begin: getFileContents
Begin: analyzeContents
Begin: getFileContents
Begin: getFileContents
Begin: getFileContents
Begin: analyzeContents
Begin: getFileContents
Begin: getFileContents
Begin: getFileContents
Begin: getFileContents
Begin: getFileContents
Begin: getFileContents
Begin: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
Begin: analyzeContents
End: analyzeContents
End: analyzeContents
End: analyzeContents
End: analyzeContents
Begin: combiningFrequencies
End: combineFrequencies
I - 7693
you - 5188
in - 4266
My - 4237
that - 4158
is - 3790
NOT - 3209
FOR - 3201
d - 3117
IT - 2947

Avoid Blocking

To obtain the highest performance you must ensure that your program never wastes one resource while waiting for another. Most commonly, this takes the form of blocking the current thread while waiting for some I/O to complete. This situation will cause one of two things to happen:

1. The thread will be blocked in a waiting state, get unscheduled, and cause another thread to run. This could mean creating a new thread to handle pending work items or tasks if all current threads are in use or blocked.
2. The thread will hit a synchronization object which might spin for a few milliseconds waiting for a signal. If it does not get it in time, it will enter the same state as 1.

In both cases, it needlessly increases the size of the thread pool, and possibly also wastes CPU spinning in locks. Neither situation is desirable.

Locking and other types of direct thread synchronization are all explicit blocking calls and easy to detect. However, it is not always so obvious what other method calls may lead to blocking. They often revolve around I/O of various kinds so you need to make sure that any interactions with the network, the file system, databases, or any other high-latency service is done asynchronously. Thankfully, .NET makes it fairly easy to do that with Task objects.

When making use of any I/O API, whether it is for network, file system, databases, or anything else, make sure that it returns a Task; otherwise, it is highly suspect and probably doing blocking I/O. Note that older asynchronous APIs will return an IASyncResult and usually start with Begin-. Either find an alternate API that returns a Task instead, or use the Task.FactoryFromAsync method to wrap these methods inside of Task objects to keep your own programming interface consistent.

You should rarely or never call the Task.Wait method. Use continuations instead. Waiting on a Task is a subset of the larger problem of blocking calls.

Avoid Lock and Dispatch Convoys

A common restriction for UI applications is that the UI state can only be modified from a single thread, often termed the “dispatcher” thread. This means, that even if you want multiple threads to do some background work, they must transition their work back to the UI thread to communicate any visible changes.

In WPF, it looks something like this:

// On a background worker thread
Application.Current.Dispatcher.BeginInvoke(
  DispatcherPriority.Background,
  () => this.statusBar.Value = "Complete");

If the UI application has many threads doing this kind of thing constantly, you can start to lock up the UI with multiple updates. In these kinds of cases, you will need to either rethink the UI changes themselves or introduce an aggregation layer where you can batch up multiple updates and update all parts of the UI at once.

This is similar to the lock convoy problem, where contention is so high for a single lock that more time is spent spinning or waiting for the lock to free than actually doing work. In both cases, you need to rethink the async strategy to reduce contention on the single resource.

Use Tasks for Non-Blocking I/O

.NET 4.5 added -Async methods to the Stream class so that now all Stream-based communication can be entirely asynchronous quite easily. Here is a simple example:

int chunkSize = 4096;
var buffer = new byte[chunkSize];

var fileStream = new FileStream(filename, FileMode.Open,
     FileAccess.Read, FileShare.Read, chunkSize, useAsync: true);

var task = fileStream.ReadAsync(buffer, 0, buffer.Length);
task.ContinueWith((readTask) =>
  {
    int amountRead = readTask.Result;
    fileStream.Dispose();
    Console.WriteLine("Async(Simple) read {0} bytes", amountRead);
  });

You can no longer take advantage of the using syntax to clean up IDisposable objects such as Stream objects. Instead, you must pass those objects to the continuation method to make sure they are disposed along every path.

The above example is actually quite incomplete. In a real scenario, you will often have to make multiple reads to a stream to get the full contents. This can happen if the files are larger than the buffer you provide, or if you are dealing with a network stream instead of files. In that case, the bytes have not even arrived at your machine yet. To handle this situation asynchronously, you need to continue reading the stream until it tells you there is no data left.

An additional wrinkle is that now you need two levels of Task objects. The top level is for the overall read—the portion your calling program is interested in. The level below that is the set of Task objects for each individual chunked read.

Consider why this is so. The first asynchronous read will return a Task. If you return that up to the caller to wait or continue on, then they will continue executing after the first read is done. What you really want is for them to continue executing after all the reads are complete. This means you cannot return that first Task back to the caller. You need a fake Task that completes once all the reads are done.

To accomplish all of this, you will need to use the TaskCompletionSource<T> class, which can generate that fake Task for you to return. When your series of asynchronous reads are complete, you call the TrySetResult method on the TaskCompletionSource, which will cause it to trigger whoever is waiting or continuing on it.

The following example expands on the previous example and demonstrates the use of TaskCompletionSource:

private static Task<int> AsynchronousRead(string filename)
{
  int chunkSize = 4096;
  var buffer = new byte[chunkSize];
  var tcs = new TaskCompletionSource<int>();

  var fileContents = new MemoryStream();
  var fileStream = new FileStream(filename, FileMode.Open, 
    FileAccess.Read, FileShare.Read, chunkSize, useAsync: true);
  fileContents.Capacity += chunkSize;

  var task = fileStream.ReadAsync(buffer, 0, buffer.Length);
  task.ContinueWith(
      readTask => 
      ContinueRead(readTask, fileStream, 
                   fileContents, buffer, tcs));

  return tcs.Task;
}

private static void ContinueRead(Task<int> task, 
                 FileStream stream, 
                 MemoryStream fileContents, 
                 byte[] buffer, 
                 TaskCompletionSource<int> tcs)
{
  if (task.IsCompleted)
  {
    int bytesRead = task.Result;
    fileContents.Write(buffer, 0, bytesRead);
    if (bytesRead > 0)
    {
      // More bytes to read, so make another async call
      var newTask = stream.ReadAsync(buffer, 0, buffer.Length);
      newTask.ContinueWith(
        readTask => 
        ContinueRead(readTask, stream, 
                     fileContents, buffer, tcs));
    }
    else
    {
      // All done, dispose of resources and 
      // complete top-level task.
      tcs.TrySetResult((int)fileContents.Length);
      stream.Dispose();
      fileContents.Dispose();
    }
  }
}

Note that there is no requirement for anyone to wait on a Task before the result is set—no ordering dependency is a fairly important requirement for asynchronous coding.

const int TotalLength = 1024;
const int ReadSize = TotalLength / 4;
    
static Task<string> GetStringFromFileBetter(string path)
{
  var buffer = new byte[TotalLength];

  var stream = new FileStream(
    path, 
    FileMode.Open, 
    FileAccess.Read, 
    FileShare.None, 
    buffer.Length, 
    FileOptions.DeleteOnClose | FileOptions.Asynchronous);

  var task = Task<int>.Factory.FromAsync(
    stream.BeginRead, 
    stream.EndRead, 
    buffer, 
    0, 
    ReadSize, null);

  var tcs = new TaskCompletionSource<string>();

  task.ContinueWith(readTask => OnReadBuffer(readTask, 
                     stream, buffer, 0, tcs));

  return tcs.Task;
}

static void OnReadBuffer(Task<int> readTask, 
             Stream stream, 
             byte[] buffer, 
             int offset, 
             TaskCompletionSource<string> tcs)
{
  int bytesRead = readTask.Result;
  if (bytesRead > 0)
  {
    var task = Task<int>.Factory.FromAsync(
      stream.BeginRead, 
      stream.EndRead, 
      buffer, 
      offset + bytesRead, 
      Math.Min(buffer.Length - (offset + bytesRead), ReadSize),
      null);

    task.ContinueWith(
      callbackTask => OnReadBuffer(
        callbackTask, 
        stream, 
        buffer, 
        offset + bytesRead, 
        tcs));
  }
  else
  {
    tcs.TrySetResult(Encoding.UTF8.GetString(buffer, 0, offset));
  }
}

Task.Run( ()=>
{
  using (var inputStream = File.OpenRead(filename))
  {
    byte[] buffer = new byte[16384];
    // Calling a synchronous I/O method
    // -- this will block the thread.
    var input = inputStream.Read(buffer, 0, buffer.Length);
    ...
  }
});

async and await

In .NET 4.5, there are two new keywords that can simplify your code in many situations: async and await. Used together, they turn your TPL code into something that looks like easy, linear, synchronous code. Under the hood, however, it is really using Task objects and continuations.

The following example comes from the AsyncAwait sample project.

static Regex regex = new Regex("<title>(.*)</title>", 
                               RegexOptions.Compiled); 

private static async Task<string> GetWebPageTitle(string url) 
{ 
    System.Net.Http.HttpClient client = 
      new System.Net.Http.HttpClient(); 
    Task<string> task = client.GetStringAsync(url);

    // now we need the result so await 
    string contents = await task; 
    Match match = regex.Match(contents); 
    if (match.Success) 
    { 
        return match.Groups[1].Captures[0].Value; 
    } 
    return string.Empty; 
}

To see where the real power of this syntax lies, consider a more complex example, where you are simultaneously reading from one file and writing to another, compressing the output along the way. It is not very difficult to use Task objects directly to accomplish this, but consider how trivial it looks when you use the async/await syntax. The following is from the CompressFiles sample project.

First, the synchronous version for comparison:

private static void SyncCompress(IEnumerable<string> fileList)
{
  byte[] buffer = new byte[16384];
  foreach (var file in fileList)
  {
    using (var inputStream = File.OpenRead(file))
    using (var outputStream = File.OpenWrite(file+".compressed"))
    using (var compressStream = new GZipStream(outputStream, 
                          CompressionMode.Compress))
    {
      int read = 0;
      while ((read = inputStream.Read(buffer, 
                                      0, 
                                      buffer.Length)) > 0)
      {
        compressStream.Write(buffer, 0, read);
      }
    }
  }
}

To make this asynchronous, all we have to do is add the async and await keywords and change the Read and Write methods to ReadAsync and WriteAsync, respectively:

private static async Task AsyncCompress(
    IEnumerable<string> fileList)
{
  byte[] buffer = new byte[16384];
  foreach (var file in fileList)
  {
    using (var inputStream = File.OpenRead(file))
    using (var outputStream = File.OpenWrite(file+".compressed"))
    using (var compressStream = 
             new GZipStream(outputStream, 
                            CompressionMode.Compress))
    {
      int read = 0;
      while ((read = await inputStream.ReadAsync(buffer, 0, 
                             buffer.Length)) > 0)
      {
        await compressStream.WriteAsync(buffer, 0, read);
      }
    }
  }
}

It looks like this code will get blocked waiting for the HTTP result, but do not confuse await for “wait”; they are similar, but deliberately just different enough. Everything before the await keyword happens in the calling thread. Everything from the await onwards is in the continuation. Your code can await any method that returns a Task<T> as long as it is in a method marked async. With these keywords, the compiler will do the heavy lifting of transforming your code into a structure similar to the previous TPL examples.

Significantly, notice that there is an await within a using statement. The dispose pattern works correctly with async and await, greatly simplifying your code.

Using async/await can dramatically simplify your code, but there are some Task-based situations for which they cannot be used. For example, if the completion of a Task is nondeterministic, or you must have multiple levels of Task objects and are using TaskCompletionSource<T>,then async/await may not fit.

Story I ran into this determinism problem myself when I was implementing retry functionality on top of .NET’s HTTP client functionality, which is fully Task-enabled. I started with a simple HTTP client wrapper class and I used async/await initially because it simplified the code. However, when it came time to actually implement the retry functionality, I immediately knew I was stuck because I had given up control of when the tasks completed. For my implementation, I wanted to send the retry request before the first request had actually timed out. Whichever request finished first is the one I wanted to return up to the caller. Unfortunately, async/await will not handle this nondeterministic situation of arbitrarily choosing from multiple child Task objects that you are waiting on. One way to resolve this is to use the ContinueOnAny method described earlier. Alternatively, you could use TaskCompletionSource to manually control when the top-level Task is complete. I chose the latter for my situation.

A Note On Program Structure

There is a crucial tidbit from the previous section: all await statements must be in methods marked async, which means those methods must return Task objects. This kind of restriction does not technically exist if you are using Task objects directly, but the same idea applies. Using asynchronous programming is like a beneficial “virus” that infects your program at all layers. Once it takes root in one part, it will move up the layers of function calls as high as it can.

Of course, using Task objects directly, you can create this (bad) example:

Task<string> task = Task<string>.Run(()=> { ... });
task.Wait();

But unless you are doing only trivial multithreaded work, this completely ruins the scalability of your application. Yes, you could insert some work between the task’s creation and the call to Wait, but this is missing the point. You should almost never wait on Task objects. That wastes a thread that could otherwise be doing useful work. It may lead to more context switching and higher overhead from the thread pool as more threads are needed to handle the available work items.

If you carry this idea of never waiting to its logical conclusion, you will realize that it is very possible (even likely) that nearly all of your program’s code will occur in a continuation of some sort. This makes intuitive sense if you think about it. A UI program does essentially nothing until a user clicks or types—it is reacting to input via an event mechanism. A server program is analogous, but instead of mouse and keyboard, the I/O is via the network or file system.

As a result, a high-performance application can easily start to feel disjointed as you split the program logic based on I/O boundaries. The earlier you plan for this, the better off you will be. It is critical to settle on just a few standard patterns that most or all of your program uses. For example:

• Determine where Task and continuation methods live. Do you use a separate method or a lambda expression? Does it depend on its size?
• If you use methods for continuations, settle on a standard prefix (e.g., OnMyTaskEnd).
• Standardize error handling. Do you have one continuation that handles all errors, cancellations, and normal completions? Or do you have separate methods to handle each of these and use TaskContinuationOptions to selectively execute them?
• Decide whether to use async/await or Task objects directly.
• If you have to call old style Begin…/End… asynchronous methods, wrap them in Task objects to standardize your handling, as described earlier.
• Do not feel like you have to use every feature of the TPL. Some features are not recommended for most situations (AttachedToParent tasks, for example). Standardize on the minimal feature set you can get away with.

Asynchronous programming really is a different kind of programming than standard, synchronous, straight-line procedures and calls. It requires you to develop a new mindset, one that is aggressive about eliminating waits and blocking calls and considers program structure in terms of independent pieces that only occasionally come together to synchronize. It can take a while to develop this kind of mindset. Start small and build up.

Use Timers Correctly

To schedule a method to execute after a certain timespan and optionally at regular intervals thereafter, use the System.Threading.Timer class. You should not use mechanisms like Thread.Sleep that block the thread for the specified amount of time, though we will see below that it can be useful in some situations.

The following example demonstrates how to use a timer by specifying a callback method and passing it two timeout values. The first value is the time until the timer fires for the first time and the second value is how often to repeat it thereafter. You can specify Timeout.Infinite (which has value -1) for either value. This example fires the timer only once, after 15 milliseconds.

private System.Threading.Timer timer;

public void Start()
{
  this.timer = new Timer(TimerCallback, 
                         null, 
                         15, 
                         Timeout.Infinite);
}

private void TimerCallback(object state)
{
  // Do your work
}

Do not create an excessive number of timers. All Timer objects are serviced from a single thread in the thread pool. A huge number of Timer instances will cause delays in executing their callbacks. When the time comes due, the timer thread will schedule a work item to the thread pool, and it will be picked up by the next available thread. If you have a large number of tasks, queued work items, or high CPU usage, this means your Timer callbacks will not be accurate. In fact, they are guaranteed to never be more accurate than the operating system’s timer tick count, which is set to 15.625 milliseconds by default. This is the same value that determines thread quantum lengths. Setting a timeout value less than that will not get you the results you need. If you need more precision than 15 milliseconds, you have a few options:

1. Reduce the operating system’s timer tick resolution. This can result in higher CPU usage and severely impact battery life, but may be justified in some situations. Note that changing this could have far-reaching impacts like more context switches, higher system overhead, and worse performance in other areas.
2. Spin in a loop, using a high-resolution timer (see Chapter 6) to measure elapsed time. This also uses more CPU and power, but is more localized.
3. Call Thread.Sleep. This will suspend the thread and there is no guarantee that it will be resumed at the time you desire. On a highly loaded system, it is possible the thread could be context-switched out and you will not get it back until well after your desired interval.

When you use a Timer, be aware of a classic race condition. Consider the code:

private System.Threading.Timer timer;

public void Start()
{
  this.timer = new Timer(TimerCallback, 
                         null, 
                         15, 
                         Timeout.Infinite);
}

private void TimerCallback(object state)
{
  // Do your work
  this.timer.Dispose();
}

This code sets up a Timer to execute a callback in 15 milliseconds. The callback just disposes the timer object once it is done with it. This code is also likely to throw a NullReferenceException in TimerCallback. The reason is that it is very possible for the callback to execute before the Timer object is assigned to the this.timer field in the Start method. Thankfully, it is quite easy to fix:

this.timer = new Timer(TimerCallback, 
                       null, 
                       Timeout.Infinite, 
                       Timeout.Infinite);
this.timer.Change(15, Timeout.Infinite);

Note A project I once worked on allocated a System.Threading.Timer object to track timeouts for each node in dependency graph. This was fine while the graphs were small. Once they started hitting a few thousands nodes in size, with hundreds of concurrently running nodes, we saw significant lock contention and unreliable timeout behavior. It quickly became an untenable, very expensive design. Instead, we designed our own timeout monitoring system that relied on polling at regular intervals—essentially, we reduced our design to use a single timer that checked all running nodes at a suitable frequency.

If you only need to run the target method a single time (i.e., no recurring scenarios), then you can use Task.Delay to schedule a delegate to run once in the future:

var task = Task.Delay(1000).ContinueWith(_ =>
  { 
    Console.WriteLine("After delay");
  });

Ensure Good Thread Pool Size

The thread pool tunes itself over time, but at startup it has no history and will start in a default state. If your software is extremely asynchronous and uses a lot of CPU, then it may be hit with a steep startup cost as it waits for more threads to be created and become available. To reach the steady state sooner, you can tweak the startup parameters to maintain a minimum number of ready threads upon startup.

const int MinWorkerThreads = 25;
const int MinIoThreads = 25;
ThreadPool.SetMinThreads(MinWorkerThreads, MinIoThreads);

You do need to exercise caution here. If you are using Task objects, then they will be scheduled based on the number of threads available for scheduling. If there are too many threads then the Task objects can become over-scheduled, leading to less CPU efficiency, not more, as more context switching happens. The thread pool will still eventually switch to using an algorithm that can reduce the number of threads to below this number, if the work load is lighter.

You can also set a maximum with the SetMaxThreads method and this has the same risks.

To figure out how many threads you really need, leave this setting alone and analyze your app during its steady state using the ThreadPool.GetMaxThreads and ThreadPool.GetMinThreads methods, or by using performance counters to examine how many threads are in the process.

Do Not Abort Threads

Terminating threads without their cooperation is a dangerous procedure. Threads have their own clean up to do and calling Abort on them does not allow a graceful shutdown. When you kill a thread, you leave portions of the application in an undefined state. It would be better to just crash the program, but ideally, just do a clean restart.

To safely end a thread, you must use some shared state and the thread function itself must check that state to determine when it should end. It must be cooperative to be safe.

If you are using Task objects, as you should, there is no API provided to terminate a Task. Use a CancellationToken, as discussed earlier in this chapter, to allow cooperative termination.

Do Not Change Thread Priorities

In general, it is a bad idea to change thread priorities. Windows schedules threads according to their priority. If high-priority threads are always ready to run, then lower priority threads will be starved and rarely given a chance to run. By increasing a thread’s priority you are saying that its work must have priority over all other work, including other processes. This notion is not a safe part of a stable system.

It is more acceptable to lower a thread’s priority if it is doing something that can wait until all the normal-priority tasks have completed. One good reason to lower a thread’s priority is that you have detected a runaway thread doing an infinite loop. You cannot safely terminate threads, so the only way to get that thread and processor resource back is by restarting the process. Until you can shut down and do that cleanly, lowering the runaway thread’s priority is a reasonable mitigation. Note that even threads with lower priorities are still guaranteed to run after a while because Windows will increase their dynamic priority the longer it goes without executing. The exception is Idle priority (THREAD_PRIORITY_IDLE), which the operating system will only ever schedule if literally nothing else can run.

You may find a legitimate reason to increase thread priority, such as reacting to rare situations that require faster responses, but this should be used with care. Windows schedules threads agnostic of the process they belong to, so a high-priority thread from your process will run at the expense not only of your other threads, but all the other threads from other applications running on your system.

If you are using the thread pool, then any thread priority changes are reset every time the thread reenters the pool. If you are using the Task Parallel Library and still manipulating the underlying thread, then keep in mind that the same thread can run multiple tasks before being placed back in the pool.

Do I Need to Care About Performance At All?

Validate that you actually need to care about performance in the first place. This goes back to the principles discussed in the first chapter. Not all code is equal in your application. Not all of it should be optimized to the n^th degree. Typically, you start at the “inner loop”—the code that is most commonly executed or is most performance critical—and work outwards until the cost outweighs the benefit. There are many other areas in your code that are much less critical, performance-wise. In these situations, if you need a lock, take a lock and do not worry about it.

Now, you do have to be careful here. If your non-critical piece of code is executing on a thread pool thread, and you block it for a significant amount of time, the thread pool could start injecting more threads to keep up with other requests. If a couple of threads do this once in a while, it is no big deal. However, if you have lots of threads doing things like this, then it could become a problem because it wastes resources that should be doing real work. If you are running a program that processes a heavy, constant workload, then even parts of your program that are not performance-sensitive can negatively affect the system by context switching or perturbing the thread pool if you are not careful. As in all things, you must measure.

Do I Need a Lock At All?

The most performant locking mechanism is one that is not there. If you can completely remove your need for thread synchronization, then you are in the best place as far as performance goes. This is the ideal, but it might not be easy. It usually means you have to ensure there is no mutable shared state—each request through your application can be handled without regard to another request or any centralized volatile (read/write) data. If you can do this, this is the optimal performance scenario.

Be careful, though. It is easy to go overboard in the refactoring and make your code a spaghetti mess that no one (including you) can understand. Unless performance is really that critical and cannot be obtained any other way, you should not carry it that far. Make it asynchronous and independent, but make it clear.

If multiple threads are just reading from a variable (and there is no chance at all of any thread writing to it), then no synchronization is necessary. All threads can have unrestricted access. This is automatically the case for immutable objects like strings or immutable value types, but can be the case for any type of object if you ensure its immutability while multiple threads are reading it.

If you do have multiple threads writing to some shared variable, see if you can remove the need for synchronized access by converting usage to a local variable. If you can create a temporary copy to act on, no synchronization is necessary. This is especially important for repeated synchronized access. You will need to convert repeated access to a shared variable to repeated access of a local variable followed by a single shared access, as in the following simple example of multiple threads adding items to a shared collection.

object syncObj = new object();
var masterList = new List<long>();
const int NumTasks = 8;
Task[] tasks = new Task[NumTasks];

for (int i = 0; i < NumTasks; i++)
{       
  tasks[i] = Task.Run(()=>
  {
    for (int j = 0; j < 5000000; j++)
    {
      lock (syncObj)
      {
        masterList.Add(j);
      }
    }
  });
}
Task.WaitAll(tasks);

This can be converted to the following:

object syncObj = new object();
var masterList = new List<long>();
const int NumTasks = 8;
Task[] tasks = new Task[NumTasks];

for (int i = 0; i < NumTasks; i++)
{       
  tasks[i] = Task.Run(()=>
  {
    var localList = new List<long>();
    for (int j = 0; j < 5000000; j++)
    {
      localList.Add(j);
    }
    lock (syncObj)
    {
      masterList.AddRange(localList);
    }
  });
}
Task.WaitAll(tasks);

On my machine, the second code fragment takes less than half of the time as the first.

In the end, mutable shared state is a fundamentally performance-hostile pattern. It requires synchronization for data safety and performance begins to suffer. If, at all possible, your design can avoid locking, then you are close to implementing an ideal multithreaded system.

Synchronization Preference Order

If you decide you do need some sort of synchronization, then understand that not all of them have equal performance or behavior characteristics. Most situations call for just using a lock and that should usually be default. Using anything other than a lock should require some intense measurement to justify the additional complexity. In general, consider synchronization mechanisms in this order:

1. lock/Monitor class: Keep it simple, understandable, with a good balance of performance.
2. No synchronization at all: Remove shared mutable state, refactor, and optimize. It is harder, but if you can do it, it will usually perform better than using a lock (barring mistakes or detrimental design changes).
3. Simple Interlocked methods: They can be more appropriate in some scenarios, but once the situation becomes more complex, turn to a lock.

And finally, if you can truly prove they are beneficial, then use fancier, more complex locks, but be warned: they are rarely as useful as you might think.

• Asynchronous locks (see later in this chapter)
• Everything else

Specific circumstances may dictate or preclude the use of some of these techniques. For example, combining multiple Interlocked methods is not likely to outperform a single lock statement.

Memory Models

Before discussing the details of thread synchronization, we must take a short detour into the world of memory models. The memory model is the set of the rules in a system (hardware or software) that govern how read and write operations can be reordered by the compiler or by the processor across multiple threads. You cannot actually change anything about the model, but understanding it is critical to having correct code in all situations.

A memory model is described as “strong” if it has an absolute restriction on reordering, which prevents the compiler and hardware from doing many optimizations. A “weak” model allows the compiler and processor much more freedom to reorder read and write instructions in order to get potentially better performance. Most platforms fall somewhere between absolutely strong and completely weak.

The ECMA standard defines the minimum set of rules that the CLR must follow. It defines a very weak memory model, but a given implementation of the CLR may actually have a stronger model in place on top of it.

A specific processor architecture may also enforce a stronger memory model. For example, the x86/x64 architecture has a relatively strong memory model which will automatically prevent certain kinds of instruction reordering, but allow others. On the other hand, the ARM architecture has a relatively weak memory model. It is the JIT compiler’s responsibility to ensure that not only are machine instructions emitted in the proper order, but that special instructions are used to ensure that the processor itself does not reorder instructions in a way that violates the CLR’s memory model. These instructions may not be cheap to execute, which is another reason why avoiding synchronization at all is always preferable.

The practical difference between x86/x64 and ARM architectures has significant impact on your code, particularly if you have bugs in thread synchronization. Because the JIT compiler under ARM is more free to reorder reads and writes than it is under x86/x64, certain classes of synchronization bugs can go completely undetected on x86/x64 and only become apparent when ported to ARM.

In some cases, the CLR will help cover these differences for compatibility’s sake, but it is good practice to ensure that the code is correct for the weakest memory model in use. The biggest requirement here is the correct use of volatile on state that is shared between threads. The volatile keyword is a signal to the JIT compiler that ordering matters for this variable. On x86/x64 it will enforce instruction ordering rules, but on ARM there will also be extra instructions emitted to enforce the correct semantics in the hardware. If you neglect volatile, the ECMA standard allows these instructions to be completely reordered and bugs could surface.

Other ways to ensure proper ordering of shared state is to use Interlocked or keep all accesses inside a full lock.

All of these methods of synchronization create what is called a memory barrier. Any reads that occur before that place in code may not be reordered after the barrier and any writes may not be reordered to be before the barrier. In this way, updates to variables are done correctly and seen by all CPUs.

Use volatile When Necessary

Consider the following classic example of an incorrect double-checked locking implementation that attempts to efficiently protect against multiple threads calling DoCompletionWork. It tries to avoid potentially expensive contentious calls to lock in the common case that there is no need to call DoCompletionWork.

private bool isComplete = false;
private object syncObj= new object();

// Incorrect implementation!
private void Complete()
{
  if (!isComplete)
  {
    lock (syncObj)
    {
      if (!isComplete)
      {
        DoCompletionWork();
        isComplete = true;
      }
    }
  }
}

While the lock statement will effectively protect its inner block, the outer check of isComplete is simultaneously accessed by multiple threads with no protection. Unfortunately, updates to this variable may be out of order due to compiler optimizations allowed under the memory model, leading to multiple threads seeing a false value even after another thread has set it to true. It is actually worse than that, though: It is possible that isComplete could be set to true before DoCompletionWork() completes, which means the program could be in an invalid state if other threads are checking and acting on the value of isComplete. Why not always use a lock around any access to isComplete? You could do that, but this leads to higher contention and is a more expensive solution than is strictly necessary.

To fix this, you need to instruct the compiler to ensure that accesses to this variable are done in the right order. You do this with the volatile keyword. The only change you need is:

private volatile bool isComplete = false;

To be clear, volatile is for program correctness, not performance. In most cases it does not noticeably help or hurt performance. If you can use it, it is better than using a lock in any high-contention scenario, which is why the double-checked locking pattern is useful.

Double-checked locking is often used for the singleton pattern when you want the first thread that uses the value to initialize it. This pattern is encapsulated in .NET by using the Lazy<T> class, which internally uses the double-checked locking pattern. You should prefer to use Lazy<T> rather than implement your own pattern. See Chapter 6 for more information about using Lazy<T>.

Use Monitor (lock)

The simplest way to protect any code region is with the Monitor object, which in C# has a keyword equivalent.

This code:

object obj = new object();
bool taken = false;
try
{
  Monitor.Enter(obj, ref taken);
}
finally
{
  if (taken)
  {
    Monitor.Exit(obj);
  }
}

is equivalent to this:

object obj = new object();

lock(obj)
{
...
}

The taken parameter is set to true if no exceptions occurred. This is guaranteed so that you can correctly call Exit.

In general, you should prefer using Monitor/lock versus other more complicated locking mechanisms until proven otherwise. Monitor is a hybrid lock in that it first tries to spin in a loop for a while before it enters a wait state and gives up the thread. This makes it ideal for places where you anticipate little or short-lived contention.

Monitor also has a more flexible API that can be useful if you have optional work to do if you cannot acquire the lock immediately.

object obj = new object();
bool taken = false;
try
{
  Monitor.TryEnter(obj, ref taken);
  if (taken) // do work that needs the lock
  {...}
  else       // do something else
  {...}
}
finally
{
  if (taken)
  { 
    Monitor.Exit(obj);
  }
}

In this case, TryEnter returns immediately, regardless of whether it got the lock. You can test the taken variable to know what to do. There are also overloads that accept a timeout value.

Use Interlocked Methods

Consider this code with a lock that is guaranteeing that only a single thread can execute the Complete method:

private bool isComplete = false;
private object completeLock = new object();

private void Complete()
{
  lock(completeLock)
  {
    if (isComplete)
    {
      return;
    }

    DoCompletionWorkHere();

    isComplete = true;
  }
}

You need two fields and a few lines of code to determine if you should even enter the method, which seems wasteful. Instead, call the Interlocked.Increment method:

private int isComplete = 0;

private void Complete()
{
  if (Interlocked.Increment(ref isComplete) == 1)
  {
      DoCompletionWorkHere();
  }
}

Or consider a slightly different situation where Complete may be called multiple times, but you want only to enter it based on some internal state, and then enter it only once.

enum State { Executing, Done };
private int state = (int)State.Executing;

private void Complete()
{
  if (Interlocked.CompareExchange (ref state, (int)State.Done, 
                 (int)State.Executing) == (int)State.Executing)
  {
      DoCompletionWorkHere();
  }
}

The first time Complete executes, it will compare the state variable against State.Executing and if they are equal, replace state with the value State.Done. The next thread that executes this will compare state against State.Executing, which will not be true, and CompareExchange will return State.Done, failing the if statement.

Interlocked methods translate into a single processor instruction and are atomic. They are perfect for this kind of simple synchronization. There are a number of Interlocked methods you can use for simple synchronization, all of which perform the operation atomically:

• Add: Adds two integers, replacing the first one with the sum and returning the sum.
• CompareExchange: Accepts values A, B, and C. Compares values A and C, and, if equal, replaces A with B, and returns original value. See the LockFreeStack example below.
• Increment: Adds one to the value and returns the new value.
• Decrement: Subtracts one from the value and returns the new value.
• Exchange: Sets a variable to a specified value and returns the original value.

All of these methods have multiple overloads for various data types.

Interlocked operations are memory barriers, thus they are not as fast as a simple write in a non-contention scenario. In general, you should not count on them to be significantly faster than a simple lock scenario, but you can see the InterlockedVsLock sample project for a simple benchmark that shows a very slight advantage.

As simple as they are, Interlocked methods can implement more powerful concepts such as lock-free data structures. But a serious word of caution: Be very careful when implementing your own data structures like this. The seductive words “lock-free” can be misleading. They are really a synonym for “repeat the operation until correct.” Once you start having multiple calls to Interlocked methods, it is extremely likely to be less efficient than just using lock in the first place, even assuming your code is 100% correct (which it likely is not). It can be very difficult to get it right or performant. Implementing data structure like this is great for education, but in real code your default choice should always be to use the built-in .NET collections. If you do implement your own thread-safe collection, take great pains to ensure that it is not only 100% functionally correct, but that it performs better than the existing .NET choices. If you use Interlocked methods, ensure that they provide more benefit than a simple lock. There are very, very few situations in which you will need to do this kind of work.

As an example (from the accompanying source in the LockFreeStack project), here is a simple implementation of a stack that uses Interlocked methods to maintain thread safety without using any of the heavier locking mechanisms.

class LockFreeStack<T>
{
  private class Node
  {
    public T Value;
    public Node Next;
  }

  private Node head;

  public void Push(T value)
  {
    var newNode = new Node() { Value = value }; 
    
    while (true)
    {
      newNode.Next = this.head;
      if (Interlocked.CompareExchange(ref this.head, 
                                      newNode, 
                                      newNode.Next) 
         == newNode.Next)
      {
        return;
      }
    }
  }

  public T Pop()
  {
    while (true)
    {
      Node node = this.head;
      if (node == null)
      {
        return default(T);
      }
      if (Interlocked.CompareExchange(ref this.head, 
                                      node.Next, 
                                      node) 
        == node)
      {
        return node.Value;
      }
    }
  }    
}

This code fragment demonstrates a common pattern with implementing data structures or more complex logic using Interlocked methods: looping. Often, the code will loop, continually testing the results of the operation until it succeeds. In most scenarios, only a handful of iterations will be performed.

While Interlocked methods are simple and relatively fast, you will often need to protect more substantial areas of code and they will fall short (or at least be unwieldy and complex).

Asynchronous Locks

Starting in .NET 4.5, there is some interesting functionality that may expand to other types in the future. The SemaphoreSlim class has a WaitAsync method that returns a Task. Instead of blocking on a wait, you can schedule a continuation on the Task that will execute once the semaphore permits it to run. There is no entry into kernel mode and no blocking. When the lock is no longer contended, the continuation will be scheduled like a normal Task on the thread pool. Usage is the same as for any other Task.

To see how it works, consider first an example using the standard, blocking, wait mechanisms. This sample code (from the WaitAsync project in the sample code) is rather trivial, but demonstrates how the threads hand off control via the semaphore.

class Program
{        
    static SemaphoreSlim semaphore = new SemaphoreSlim(1);
    const int WaitTimeMs = 1000;        

    static void Main(string[] args)
    {
        Task.Run((Action)Func1);
        Task.Run((Action)Func2);
        Console.ReadKey();            
    }

    static void Func1()
    {
        while (true)
        {
            semaphore.Wait();
            Console.WriteLine("Func1");            
            semaphore.Release();
            Thread.Sleep(WaitTimeMs);
        }
    }

    static void Func2()
    {
        while (true)
        {
            semaphore.Wait();
            Console.WriteLine("Func2");            
            semaphore.Release();
            Thread.Sleep(WaitTimeMs);
        }
    }
}

Each thread loops indefinitely while waiting for the other thread to finish its loop operation. When the Wait method is called, that thread will block until the semaphore is released. In a high-throughput program, that blocking is extremely wasteful and can reduce processing capacity and increase the size of the thread pool. If the block lasts long enough, a kernel transition may occur, which is yet more wasted time.

To use WaitAsync, replace the methods with these implementations:

static void AsyncFunc1()
{
    semaphore.WaitAsync().ContinueWith(_ =>
    {
        Console.WriteLine("AsyncFunc1");                
        semaphore.Release();
        Thread.Sleep(WaitTimeMs);
    }).ContinueWith(_ => AsyncFunc1());
}

static void AsyncFunc2()
{
    semaphore.WaitAsync().ContinueWith(_ =>
    {
        Console.WriteLine("AsyncFunc2");                
        semaphore.Release();
        Thread.Sleep(WaitTimeMs);
    }).ContinueWith(_ => AsyncFunc2());
}

Note that the loop was removed in lieu of a chain of continuations that will call back into these methods. It is logically recursive, but not actually so because each continuation starts the stack fresh with a new Task.

No blocking occurs with WaitAsync, but this is not free functionality. There is still overhead from increased Task scheduling and function calls. If your program schedules a lot of Task objects, the increased scheduling pressure may offset the gains from avoiding the blocking call. If your locks are extremely short-lived (they just spin and never enter kernel mode) or rare, then it may be better to just block for the few microseconds it takes to get through a standard lock. On the other hand, if you need to execute a relatively long-running task under a lock, this may be the better choice because the overhead of a new Task is less than the time you will block the processor otherwise. You will need to carefully measure and experiment.

If this pattern is interesting to you, see Stephen Toub’s series of articles on asynchronous coordination primitives, titled Building Async Coordination Primitives (Parts 1-7), which cover more types and patterns.

Other Locking Mechanisms

There are many locking mechanisms you can use, but you should prefer to stick with as few as you can get away with. As in many things, but especially multi-threading, the simpler the better.

The simplest way to ensure that a method can only ever be called by a single thread is to decorate it with the MethodImplOptions.Synchronized option. You can apply it to static or instance methods.

[MethodImplAttribute(MethodImplOptions.Synchronized)] 
public void DoSomething(int arg)
{
    ...
}

However, this is a very blunt tool and you should rarely need to use something so coarse. It is usually better to lock on a narrower scope from within the method.

If you know that a lock is extremely short-lived (tens of cycles) and want to guarantee that it never enters a wait state, then you can use a SpinLock. It will just spin a loop until the contention is done. In most cases, Monitor, which spins first, then enters a wait if necessary, is a better default choice.

private SpinLock spinLock = new SpinLock();

private void DoWork()
{
  bool taken = false;
  try
  {
    spinLock.Enter(ref taken);
    // Do some work
  }
  finally
  {
    if (taken)
    {
      spinLock.Exit();
    }
  }
}

In general, avoid other locking mechanisms if you can. They are usually not nearly as performant as the simple Monitor. Objects like ReaderWriterLockSlim, Semaphore, Mutex, or other custom synchronization objects definitely have their place, but they are often complex and error-prone.

Completely avoid ReaderWriterLock—it is deprecated and should never be used for any reason.

If there is a -Slim version of a synchronization object, prefer it to the non-Slim version. The -Slim versions are all hybrid locks which means they implement some form of spinning before they enter a kernel transition, which is much slower. -Slim locks are much better when the contention is expected to be low and brief.

For example, both ManualResetEvent and ManualResetEventSlim classes exist, and there is an AutoResetEvent class, but there is no AutoResetEventSlim class. However, you can use SemaphoreSlim with the initialCount parameter set to one for the same effect.

Concurrency and Collections

There are a handful of collections provided with .NET that allow concurrent access from multiple threads. They are all in the System.Collections.Concurrent namespace and include:

• ConcurrentBag<T>: unordered collection
• ConcurrentDictionary<TKey, TValue>: key/value pairs
• ConcurrentQueue<T>: first-in/first-out queue
• ConcurrentStack<T>: last-in/first-out stack

Most of these are implemented internally using Interlocked or Monitor synchronization primitives and I encourage you to examine their implementations using an IL reflection tool.

They are convenient, but you need to be careful—every single access to the collection involves synchronization. Often, that is overkill and could harm your performance if there is a lot of contention. If you have necessarily “chatty” read/write access patterns to the collection, this low-level synchronization may make sense.

This section will cover a few alternatives to concurrent collections that may simplify your locking requirements. See Chapter 6 for a discussion of collections in general, including those listed here, particularly for a description of the APIs unique to these collections, which can be tricky to get right.

private volatile Dictionary<string, MyComplexObject> data = new
  Dictionary<string, MyComplexObject>();

public Dictionary<string, MyComplexObject> Data 
{ 
  get 
  { 
    return data; 
  }
}

private void UpdateData()
{
  var newData = new Dictionary<string, MyComplexObject>();
  newData["Foo"] = new MyComplexObject();
  ...
  data = newData;
}

private void CreateReport(DataSource source)
{
  Dictionary<string, MyComplexObject> data = source.Data;
  
  foreach(var kvp in data)
  {
     ...
  }
}

Copy Your Resource Per-Thread

If you have a resource that is lightweight, not thread-safe, and is used a lot on multiple threads, you could consider wrapping it in a ThreadLocal<T> object.

Here is a classic example, using the Random class, which is not thread-safe.

private static readonly ThreadLocal<Random> threadLocalRand 
    = new ThreadLocal<Random>(()=>new Random());

static void Main(string[] args)
{
    int[] results = new int[100];

    Parallel.For(0, 5000,
        i =>
        {
            var randomNumber = threadLocalRand.Value.Next(100);
            Interlocked.Increment(ref results[randomNumber]);
        });
}

The ThreadLocal<T> constructor takes a Func<T> to allow you to specify a factory method. This will be called the first time the variable is used on a thread, before the object is returned to you for usage.

ThreadLocal<T> is available in .Net 4 and higher. If for some reason, you need to use an earlier version of .Net, there is an alternative: the [ThreadStatic] attribute. It can only be applied to static variables.

Here is the same example implemented using [ThreadStatic]. This example comes from the MultiThreadRand sample project:

[ThreadStatic]
static Random threadStaticRand;

static void Main(string[] args)
{
  int[] results = new int[100];
              
  Parallel.For(0, 5000,
    i =>
    {
      // thread statics are not initialized
      if (threadStaticRand == null) 
      { 
        threadStaticRand = new Random();
      }
      var randomNumber = threadStaticRand.Next(100);
      Interlocked.Increment(ref results[randomNumber]);
    });
}

You should always assume that statics marked as [ThreadStatic] are not initialized at the time of first use—.NET will only initialize the first one. The rest will have default values (usually null). Because of its limitations, there is little reason to use [ThreadStatic] and you should prefer ThreadLocal<T> for most circumstances.

It is important to note that [ThreadStatic] and ThreadLocal<T> are slower than non-thread-aware variables. You should prefer standard variables whenever possible, but consider these two possibilities as a simple way to convert a shared resource (needing a lock) into a non-shared resource (needing no synchronization).

Performance Counters

In the Process category exists the Thread Count counter.

In the Synchronization category, you can find the following counters:

• Spinlock Acquires/sec
• Spinlock Contentions/sec
• Spinlock Spins/sec

Under System, you can find the Context Switches/sec counter. It is difficult to know what the ideal value of this counter should be. You can find many conflicting opinions about this (I have commonly seen a value of 300 being normal, with 1,000 being too high), so I believe you should view this counter mostly in a relative sense and track how it changes over time, with large increases potentially indicating problems in your app.

.NET provides a number of counters in the .NET CLR LocksAndThreads category, including:

• # of current logical Threads: The number of managed threads in the process.
• # of current physical Threads: The number of OS threads assigned to the process to execute the managed threads, excluding those used only by the CLR.
• Contention Rate / sec: Important for detecting “hot” locks that you should refactor or remove.
• Current Queue Length: The number of threads blocked on a lock.

ETW Events

Of the following events, ContentionStart_V1 and ContentionStop are the most useful. The others may be interesting in a situation where the concurrency level is changing in unexpected ways and you need insights into how the thread pool is behaving.

• ContentionStart_V1: Contention has started. For hybrid locks, this does not count the spinning phase, but only when when it enters an actual blocked state. Fields include:
  • Flags: 0 for managed, 1 for native.
• ContentionStop: Contention has ended.
• ThreadPoolWorkerThreadStart: A thread pool thread has started. Fields include:
  • ActiveWorkerThreadCount: Number of worker thread available, both those processing work and those waiting for work.
  • RetiredWorkerThreadCount: Number of worker threads being held in reserve if more threads are needed later.
• ThreadPoolWorkerThreadStop: A thread pool thread has ended. Fields are same as for ThreadPoolWorkerThreadStart.
• IOThreadCreate_V1: An I/O thread has been created. Fields include:
  • Count: Number of I/O threads in pool.

See https://docs.microsoft.com/dotnet/framework/performance/thread-pool-etw-events for more information about thread-related ETW events.

Get Thread Information

Most debuggers will show you the running threads and let you selectively pause and unpause them (or as Visual Studio calls it, freeze and thaw).

In WinDbg, you can get detailed information about managed threads with the !Threads and the !ThreadState commands:

0:007> !Threads
ThreadCount:      3
UnstartedThread:  0
BackgroundThread: 2
PendingThread:    0
DeadThread:       0
Hosted Runtime:   no
                                                                  
 ID OSID ThreadOBJ  State GC Mode     GC Alloc Context  Domain  ...
0 1 5580 034578a0   2a020 Preemptive  058ECED4:00000000 0344d1e0...
5 2 4bb8 034675c8   2b220 Preemptive  00000000:00000000 0344d1e0...
6 3 42b0 03486658   21220 Preemptive  00000000:00000000 0344d1e0...

0:007> !ThreadState 2a020
    Legal to Join
    CoInitialized
    In Multi Threaded Apartment
    Fully initialized

The command !ThreadState -special will show you CLR-owned threads and what they are doing:

0:000> !Threads -special
ThreadCount:      3
UnstartedThread:  0
BackgroundThread: 2
PendingThread:    0
DeadThread:       0
Hosted Runtime:   no
                                                                 
 ID OSID ThreadOBJ State GC Mode     GC Alloc Context  Domain  ...
0 1 5580 034578a0  2a020 Preemptive  058ECED4:00000000 0344d1e0...
5 2 4bb8 034675c8  2b220 Preemptive  00000000:00000000 0344d1e0...
6 3 42b0 03486658  21220 Preemptive  00000000:00000000 0344d1e0...

          OSID Special thread type
        4 4c30 DbgHelper 
        5 4bb8 Finalizer 
        6 42b0 GC

In this case, you can see the finalizer thread and a GC thread (presumably the background GC thread, since this application uses workstation GC).

Visualizing Tasks and Threads with Visual Studio

Visual Studio has an optional tool called the Concurrency Visualizer. This tool relies on ETW events, but shows them in a graphical form so that you can see exactly what all the Task objects and threads are doing in your application. It can tell you when Task delegates start and stop, whether a thread is blocked, doing CPU work, waiting on I/O, and a lot more.

It is important to note that capturing this type of information can significantly degrade the application’s performance, as an event will be recorded every time a thread changes state, which is extremely frequent.

As of the time of this writing, the Concurrency Analyzer is not available for Visual Studio 2017, but you can use the more rudimentary Profiling Wizard to collect this data if needed. Launch the Profile Wizard (from the Analyze | Performance Profiler…) and select the concurrency option.

Using PerfView to find Lock Contention

A strong alternative to Visual Studio, especially for analysis of production workloads is PerfView. You can use the sample program HighContention to see this in action. Run the program and collect .NET events using PerfView. Once the .etl file is ready to view, open the Any Stacks view and look for an entry named “Event Microsoft-Windows-DotNETRuntime/Contention/Start” and double-click it. It will open a stack view of your process and show stacks that contribute to thread contention.

Where Are My Threads Blocking On I/O?

You can use PerfView to collect information about threads such as when they enter various states. This gives you more accurate information about overall wall-clock time spent in your program, if it is not using the CPU. However, be aware that turning on this option slows down your program considerably.

In PerfView’s Collection dialog, check the “Thread Times” box as well as the other defaults (Kernel, .NET, and CPU events).

Once collection is done, you will see a Thread Times node in the results tree. Open that and look at the By Name tab. You should see two main categories at the top: BLOCKED_TIME and CPU_TIME. Double-click BLOCKED_TIME to see the callers for that group.