Benefits of Concurrency

Although it is not trivial, the effort of writing concurrent code usually pays off in one or more of the ways shown in the following list:

• User interface responsiveness. It is best to keep computationally heavy work and user interface (UI) updates separate, so that the UI does not freeze while the heavy work is being done. Similarly, UI should be kept interactive and responsive while your app waits for a response from a remote server, for example.

• Better resource utilization. In the last scenario waiting for a server response would keep the CPU (central processing unit) idle, unless your app can do other useful work in the meantime. For instance, instead of waiting for an entire audio file to download, in order to be played, your app could download a chunk of it and start playing it while the rest of the file is being downloaded.

• Improved performance. On multicore systems, such as today’s iPhones and iPads, designing concurrent tasks makes it possible to run them in parallel, thus saving time. It is important to keep in mind that just because tasks can run at the same time does not mean that they will–more on that next.

Challenges of Concurrency

Designing concurrent apps has its flipside too–here are some of the challenges to keep in mind.

• Parallelism is not guaranteed. Unless you are prepared to exercise very low-level control of scheduling, running, and synchronizing tasks, when and for how long each one runs is up to the operating system. Your job as a programmer is to write code that can be run in parallel if needed.

• Improved performance is not a given either. It is worth measuring your app’s performance of critical jobs before and after you break them down into concurrent tasks. Too much granularity can often hurt, rather than improve speed.

• Dealing with dependencies is important. Tasks designed to serve the same job are rarely completely independent of each other. It is especially important to consider scenarios where two or more tasks can get in a deadlock, each waiting for the other one to complete, in order to continue.

• Managing shared resources is important. Tasks might need to access the same files, network connections, data in memory, or other resources. You need to make sure that this does not lead to data corruption, where more than one task modifies the same resource at the same time, or to inconsistencies, where reading data happens before writing to it has finished.

• Debugging can be tricky. When there is more than one task running at the same time, it can be challenging to replicate and trace errors in your code, because how things happen is related to timing and order.

A few words on multithreading

Concurrency is traditionally linked with multithreaded programming. When making iOS apps you certainly have the option of accessing and using threads. Apple has also provided higher-level technologies, which can save you a lot of work related to thread scheduling and synchronization. We will use one of these technologies, Grand Central Dispatch (GCD), to build a concurrent app in the later part of this chapter. GCD itself uses threads, so it is worth understanding how these work, as this underpins GCD’s principles. Moreover, whether you write multithreaded code or not, your apps are multithreaded, as a lot of iOS SDK calls run their tasks on different threads.

When you start an app on your iOS device, it runs in a process. Each process is virtually independent of anything else that runs on the device: it has its own chunk of memory, which no other process can access and can pretend that it is the only thing running on the system.

Code in a process runs in one or more threads. You can think of a thread as a sequence of CPU instructions to be executed one after the other. A process has at least one thread of execution. When there are multiple threads, one of them is designated as the main thread. The main thread is where code that updates the UI usually runs. Other tasks can be assigned to different threads and be run at the same time or seemingly at the same time as the code on the main thread. Each thread has its own stack for instructions but does not have designated memory–threads share the same memory space.

Executing threads “seemingly at the same time” is what happens on single-core machines. As there is only one CPU, which can execute a single instruction at a time, it is impossible to have instructions run in parallel. Instead this is simulated: when a thread runs, a chunk of its instructions are executed, then the thread is put to sleep (i.e., paused), while a chunk of instructions from another thread are executed, then the second thread is put to sleep, so another one can run… This going to sleep and waking up require work: before it is paused, the running thread needs to save its state (its stack) and then needs to retrieve it when it is woken up, so it can continue from where it left off. This is called context switching and is one of the prices you pay for using multiple threads: saving and retrieving state takes additional time and memory. In the house-cleaning example this would be similar to the scenario, where all of your housemates have found better places to be and have left you to tidy up on your own. You could run multiple tasks at a time and try to wash the dishes and dust in parallel: you start the tap, put dishwashing liquid on the sponge, wash one dish, then put the sponge down, rinse your hands, turn the tap off, pick up the dust cloth and dust the top of the bookshelf; then start the tap again, put dishwashing liquid on the sponge. All of the preparation to wash a dish or to dust a shelf is what you do to switch contexts and in this case creates a lot of unnecessary overhead–you would probably be better off if you washed all of the dishes and then got on with dusting. Figure 7-1 illustrates this: threads running on a single-core CPU, rather than the house cleaning drama.

Figure 7-1. Multiple threads running on a single-core machine

On multicore machines there is the option to run more than one task at a time (Figure 7-2): Alice can do the dishwashing, while you dust.

Figure 7-2. Multiple threads running on a dual-core machine

When more than one thread can execute a piece of code, this code needs to be thread-safe. This means guaranteeing that there will be no unwanted side effects, such as data corruption or crashes, caused, for example, by a thread trying to access a variable that another thread has not yet finished initializing. Or Bob pulling the stepladder away just as you are about to climb on it to dust the chandelier.

Another couple of tricky situations that need to be considered with multithreading and when designing concurrent code in general are deadlocks and race conditions.

Deadlocking happens when two or more threads (or tasks) are paused and wait for each other to finish before they can continue execution. Say, Bob has kindly agreed to do the dishes and is waiting for you to return the dishwashing liquid, which you grabbed to treat a suspicious stain on the carpet. You, however, (prudently) want to use rubber gloves to do this, so you decide to wait until Bob has finished washing the dishes has and taken the gloves off.

A race condition typically arises when more than one thread or task can access and modify shared memory (or another resource) and the state of this memory can vary depending on who got to it first (i.e., it ends up being unpredictable). If two of your housemates decided to rearrange the bookshelves and Alice started ordering the books alphabetically, but Bob rearranged them by color and size at the same time,² Alice and Bob would be in a race condition and the final state of the bookshelves would be anybody’s guess.

When you work with multiple threads, their scheduling is typically done by the operating system, but the communication between threads and ensuring their safety is up to you.

Apple’s GCD saves you some of this work by helping you think about your code in terms of tasks and queues of tasks, rather than low-level threads and their relation to hardware. In order to explore GCD and see what is available through it, we will design an app that uses it to accomplish a task concurrently. The next section will guide you through setting up the app.

Performing a Search in the Background

A simple way of concurrently searching for prime numbers is to split the job into two tasks: one to find prime numbers and another one to show the results on the screen. The task of displaying the results will need to be part of the main queue, as it involves updating the UI. The search however can be performed on a concurrent queue. Listing 7-10 shows you the implementation, which we explain step by step below.

Listing 7-10. Adding the Search Task to a Background Queue

// MARK: Search for prime numbers, using concurrency
func concurrentlyFindPrimeNumbersLessThan(number limit: UInt) {
    // Make sure we have been given a valid limit:
    assert(limit > 0)

    // Obtain a reference to one of the existing queues:
    let queue =
        dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)

    // Create a block of code that will be added to the queue:
    let block = dispatch_block_create(DISPATCH_BLOCK_INHERIT_QOS_CLASS) {
        let primes = self.findPrimeNumbersInRange(range: 0...(limit - 1))

        // When the search is done,
        // obtain a reference to the main queue
        // and dispatch the task of updating the UI to it:
        let mainQueue = dispatch_get_main_queue()
        dispatch_async(mainQueue, {
            self.displayResults(resultsArray: primes)
        })
    }

    // Add the code block to the background queue,
    // so it can be scheduled for execution:
    dispatch_async(queue, block)
}

First we obtain a reference to one of the global queues, using the GCD function dispatch_get_global_queue and specifying an identifier for the queue we need. The queue identifier distinguishes between queues by the quality of service (QOS) they offer. Quality of service affects the priority with which tasks in the queue are run. How tasks are run also has an effect on the device’s battery life. You can normally choose between the following QOS classes⁴:

• QOS_CLASS_USER_INTERACTIVE. Tasks with this QOS are treated as critical for keeping UI responsive and are thus executed with the highest priority for accessing system resources. They are the most energy-hungry.

• QOS_CLASS_USER_INITIATED. Tasks on such a queue have usually been triggered by a user action and the user is assumed to be waiting for a result. They get a relatively high priority for execution but are not treated as critical.

• QOS_CLASS_UTILITY. This QOS is for tasks that may or may not have been initiated by the user and have even lower priority. This allows them to be executed in a way that preserves energy (i.e., battery life).

• QOS_CLASS_BACKGROUND. Tasks in background queues are run with the lowest priority and in the most energy-efficient way. This QOS is for doing work from which the user does not expect an immediate result and may not even be aware of altogether.

Our intent is for the user to start the search for prime numbers and keep the user interface responsive while he or she waits for the result, so we request a queue with QOS_CLASS_USER_INITIATED.

Next, we define the task that we will put on the queue and assign it to a constant called block. This is done by creating a dispatch block: a piece of code that can be assigned to a variable or a constant and be passed around as an argument to function calls.

Inside the code block we call the findPrimeNumbersInRange method we implemented earlier and when it returns its result, we create another task that will display it on the screen. We add this task to the main queue, as it needs to update the UI. Note that we can define a block of code inline without assigning it to a variable or a constant first–this is what the curly brackets in the call do:

dispatch_async(mainQueue, {/*Code for the task comes here*/})

Finally we call dispatch_async, which adds the task in block to the queue to be executed (Listing 7-10).

Run the app again and this time start the search for prime numbers with the Use concurrency switch turned on. There is an improvement: finding prime numbers still takes a while, but the app’s user interface does not freeze.

Displaying Progress

It would be nice to inform the user about how the search is going while he or she waits for the results. For that purpose we will first overload the findPrimesInRange function to report progress as it goes, which can then be displayed on the screen. We will have progress reported in a callback that is passed as an argument to the function.

On Listing 7-11 we first create a typealias (i.e., an alias for the type of callback we want to use), in this case a function that takes one argument of type Float and does not return a result. We then define findPrimeNumbersInRange to be almost like the original version we coded earlier, but with one extra argument that takes the callback, which is called with progress information as the search for prime numbers is going on.

Listing 7-11. Defining a Search Method That Reports Progress

// Define an alias for the callback
// we want to use to report progress:
typealias ProgressHandler = (Float) -> ()

func findPrimeNumbersInRange(range r: Range<UInt>,
    progressHandler: ProgressHandler) -> [UInt] {
    // We will collect the results in this array:
    var primeNumbers = [UInt] ()

    for i in r {
        if isItPrime(number: i) {
            primeNumbers.append(i)

            // Calculate the proportion of work done
            // and call the progress handler
            let progress = Float(i - r.first!) / Float(r.count)
            progressHandler(progress)
        }
    }

    // Job done:
    progressHandler(1.0)

    return primeNumbers
}

A quick note on syntax: we defined the typealias in Listing 7-11 for ease of reading, but this is not compulsory. Instead, you can declare findPrimeNumbersInRange like this by using the callback signature as the second argument’s type:

func findPrimeNumbersInRange(range r: Range<UInt>,
    progressHandler: ((Float) -> ())) -> [UInt]

All we need to do in our concurrentlyFindPrimeNumbersLessThan method is call the new overload of findPrimeNumbersInRange and pass it the displayProgress method we defined earlier; the rest of the code is exactly the same as it was (Listing 7-12).

Listing 7-12. Modifying concurrentlyFindPrimeNumbersLessThan to Use the New Search Method

// MARK: Search for prime numbers, using concurrency
func concurrentlyFindPrimeNumbersLessThan(number limit: UInt) {
    // Make sure we have been given a valid limit:
    assert(limit > 0)

    // Obtain a reference to one of the existing queues:
    let queue = dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)

    // Create a block of code that will be added to the queue:
    let block = dispatch_block_create(DISPATCH_BLOCK_INHERIT_QOS_CLASS) {
        let primes =
            self.findPrimeNumbersInRange(range: 0…(limit - 1),
                progressHandler: self.displayProgress)

        // When the search is done,
        // obtain a reference to the main queue
        // and dispatch the task of updating the UI to it:
        let mainQueue = dispatch_get_main_queue()
        dispatch_async(mainQueue, {
            self.displayResults(resultsArray: primes)
        })
    }

    // Add the code block to the background queue,
    // so it can be scheduled for execution:
    dispatch_async(queue, block)
}

Before you can safely run the app, however, there is something we need to do. Can you guess what it is? Have a look at Listing 7-13 to see if you guessed right.

Listing 7-13. Ensuring That Displaying Progress Happens on the Main Queue

func displayProgress(percent amountDone: Float) {
    dispatch_async(dispatch_get_main_queue(), {
        self.searchProgress.progress = amountDone
        self.resultsTextView.text = "Progress: (Int(100.0 * amountDone))%"
    })
}

The displayProgress method we defined earlier updates the UI, so we need to ensure that the update happens on the main queue. We have changed displayProgress and wrapped the UI modifications in a dispatch_async call that adds them to the main queue.

Controlling Access to Shared Resources with Dispatch Barriers

If we had to check a huge interval of numbers to find out which ones are prime, we could divide the job even further. Instead of having one task that iterates over all of the numbers, we could split the interval between several tasks–if run in parallel, they could, in theory, complete the job faster.

I hedged with throwing “in theory” there, as there is no universal recipe for improving performance. Making tasks more granular is a double-edged sword: it could speed up your code, but it could also just as easily slow it down if there is a lot of synchronization to do. Sharing resources between tasks is one tricky situation, which might require synchronization techniques that get in the way of performance.

In this section we will contrive a scenario to demonstrate how we can control access to shared resources between tasks with dispatch barriers. We will split the search for prime numbers into several tasks, which we will add to a queue to be run concurrently. But we will require that all results are combined in the same array in the end (and will not care if we end up with a sorted array of prime numbers or not). Access to this array will need to be limited to one task at a time within the queue–this is achieved by putting the code that modifies the array in a dispatch_barrier_async call.

Dispatch barriers are normally used to synchronize tasks in the same queue. The code in the barrier is executed only after all previous tasks that were submitted to the queue have finished. No other tasks are started before the barrier code completes.

We will do a few changes to concurrentlyFindPrimeNumbersLessThan to illustrate this. Listing 7-14 will guide you through these modifications.

Listing 7-14. Splitting the Search for Prime Numbers into Subtasks

// MARK: Search for prime numbers, using concurrency
func concurrentlyFindPrimeNumbersLessThan(number limit: UInt) {
    // Let us split the search into four:
    let concurrentTaskCount: UInt = 4
    // Based on the numer of concurrent tasks we want,
    // calculate the size of the interval
    // that each task will need to search through:
    let rangeSize = limit / concurrentTaskCount + limit % concurrentTaskCount

    // Keep track of how many tasks have completed:
    var tasksDone: UInt = 0

    // An array to combine results in the end:
    var primes = [UInt]()

    // Create a custom concurrent queue:
    let queue =
        dispatch_queue_create("com.diadraw.primeNumberSearchQueue",
            DISPATCH_QUEUE_CONCURRENT);

    for i in 0 ..< concurrentTaskCount {
        // Work out the interval for the search:
        let rangeStart = i * rangeSize
        if rangeStart >= limit {
            // If we have already covered the whole interval,
            // mark the task as complete:
            tasksDone += 1
            continue
        }
        let rangeEnd = min(UInt(rangeStart + rangeSize - 1), limit - 1)

        // Define a block of code to perform the search:
        let block = dispatch_block_create(DISPATCH_BLOCK_INHERIT_QOS_CLASS) {
            let primesInCurrentRange =
                self.findPrimeNumbersInRange(range: rangeStart ... rangeEnd)

            // Set a barrier around writing to the primes array,
            // so that no more than one task can write to it
            // at the same time:
            dispatch_barrier_async(queue, {
                primes.appendContentsOf(primesInCurrentRange)

                // Keep track of how many tasks have completed:
                tasksDone += 1

                // When we've got the last chunk of results in, display them:
                if tasksDone == concurrentTaskCount {
                    dispatch_async(dispatch_get_main_queue(), {
                        self.displayResults(resultsArray: primes)
                    })
                }
            })
        }

        // Add the block to the background queue:
        dispatch_async(queue, block)
    }
}

First we declare an array of unsigned integers, called primes, which will contain the results from all the tasks–this will be the shared resource between the queues. We also add a constant, concurrentTaskCount, which will determine into how many subtasks we want to split the search for prime numbers –let us arbitrarily set it to 4. In a loop from zero to concurrentTaskCount we do mostly the same as before: define a code block, which calls findPrimeNumbersInRange and receives from it an array of prime numbers. This time we will have several blocks, each of which does the search in its own subinterval of numbers. Instead of displaying the results as soon as they arrive, however, each task will add its results to the primes array and only when all tasks are done will we send the results to the main queue to be displayed on the screen. This is the code we put in a dispatch barrier.

Something to note about dispatch barriers is that they only make sense to use with custom concurrent queues. As a barrier serializes access to a resource, it will not make a difference on a serial queue. Put on a global queue, barriers are likely to be ignored: the priority of iOS’s scheduler is to keep things running and not allow your app to block the execution of other processes. For that reason, instead of requesting a reference to a global queue like we did earlier, Listing 7-14 creates a custom queue with dispatch_queue_create.

In this version of concurrentlyFindPrimeNumbersLessThan we have opted out of displaying progress to keep things simple.

Cancelling Lengthy Tasks

It is nice to allow the user to cancel a task that can take a while, especially if the user managed to start it accidentally or with the wrong input. You know the moment of “Oh, £@%^. I meant to download the latest Xcode, not all of the ‘X-Men’ series. . . .”

Let us modify the concurrentlyFindPrimeNumbersLessThan method, so that tasks can be cancelled. In order to do that, we will need access to the blocks of code that we add to the queue outside of concurrentlyFindPrimeNumbersLessThan. To help with that we will define an array to keep references to them. Add this array, called blocks, as a property to ViewController (Listing 7-15).

Listing 7-15. Defining an Array to Keep References to the Code Blocks That Are Running Concurrently

// MARK: Search for prime numbers, using concurrency
// An array of execution blocks:
var blocks = [dispatch_block_t]()

The only modification we need to make to concurrentlyFindPrimeNumbersLessThan is at the end of the for-in cycle that creates and adds each code block to the queue. Find this line,

dispatch_async(queue, block)

and replace it with the code after the last comment in Listing 7-16. In this new version we first add a reference to the block of code in the blocks array, so we can access it for cancellation later. Then we send it to the concurrent queue, but instead of dispatch_async, we call dispatch_after. This call delays the execution of the block with a given time interval, specified in nanoseconds. Note that this is not a guarantee that the task in the block will be executed exactly after that long but only that it will not start before the time interval has passed.

We have added this delay, so we can test the cancellation. One thing to keep in mind when cancelling tasks in a queue is that they can only be cancelled before they have started executing. Cancelling a task that is already running has no effect. The five-second delay we have added should give us enough time to tap the Cancel button and see the effect. To reflect that, a nice addition to the UI would be to disable the Cancel button once the concurrent tasks start executing. We have omitted this from Listing 7-16, in order to keep it simple, and are leaving it as a little homework to you.

Listing 7-16. Modifying concurrentlyFindPrimeNumbersLessThan, So That We Can Test Cancelling Tasks

func concurrentlyFindPrimeNumbersLessThan(number limit: UInt) {
    // Make sure we clear the blocks array from any old task references:
    blocks.removeAll()
    // The rest of the code below this line stays the same.

    for i in 0 ..< concurrentTaskCount {
        // The rest of the code in the for-in loop stays the same.

        // Add the block to the queue, but after a delay:
        blocks.append(block)

        let delayInSeconds = 5.0
        let startTime = dispatch_time(DISPATCH_TIME_NOW,
            Int64(delayInSeconds * Double(NSEC_PER_SEC)))
        dispatch_after(startTime, queue, block)
    }
}

Now let us implement the actual cancellation. Find the cancelSearch action we added earlier and replace the TODO annotations inside it with the code from Listing 7-17. As you can see, cancelling a block of code is quite simple–it is done with a call to dispatch_block_cancel.

Listing 7-17. Cancelling the Execution of Blocks of Code

@IBAction func cancelSearch(sender: AnyObject) {
    // Cancel any blocks that have not been run yet:
    for block in blocks {
        dispatch_block_cancel(block)
    }

    // Update the screen to show that the search was cancelled.
    resultsTextView.text.appendContentsOf("
Search cancelled.")

    // Disable Cancel, but enable Start:
    cancelButton.enabled = false
    startButton.enabled = true
}

Run the app, set the search limit with the slider, and tap the Start button. If you tap the Cancel button within five seconds after that, you should be able to cancel the search.