CS 346 (W23)
Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Async Programming

Programs typically consists of functions, or subroutines, that we call in order to perform some operations. One underlying assumption is that subroutines will run to completion before returning control to the calling function. Subroutines don’t save state between executions, and can be called multiple times to produce a result. For example:

Function Calls

Because they run to completion, a subroutine will block your program from executing any further until the subroutine completes1. This may not matter with very quick functions, but in some cases, this can cause your application to appear to “hang” or “lock up” waiting for a result.

This is not an unusual situation: applications often connect with outside resources to download data, query a database, or make a request to a web API, all of which take considerable time and can cause blocking behaviours. Delays like this are unacceptable.

The solution is to design software so that long-running tasks can be run in the background, or asynchronously. Kotlin has support for a number of mechanisms that support this. The most common approach involves manipulating threads.

Threads

A thread is the smallest unit of execution in an application. Typically an application will have a “main” thread, but we can also create threads to do other work that execute independently of this main thread. Every thread has its own instructions that it executes, and the processor is responsible for allocating time for each thread to run. 2.

Threads in a process

Multi-threading is the idea of splitting up computation across threads – having multiple threads running in-parallel to complete work more quickly. This is also a potential solution to the blocking problem, since one thread can wait for the blocking operation to complete, while the other threads continue processing. Threads like this are also called background threads.

Note from the diagram below that threads have a common heap, but each thread has its own registers and stack. Care must be taken to ensure that threads aren’t modifying the same memory at the same time! I’d recommend taking a course that covers concurrency in greater detail.

Multiple threads executing

Concurrent vs. Parallel Execution

Concurrent Execution means that an application is making progress on more than one task at a time. This diagram illustrates how a single processor, handling one task at a time, can process them concurrently (i.e. with work being done in-order).

In this example3, tasks 1 and 2 are being processed concurrently, and the single CPU is switching between them. Note that there is never a period of overlap where both are executing at the same time, but we have progress being made on both over the same time period.

img

Parallel Execution is where progress can be made on more than one task simultaneously. This is typically done by having multiple threads that can be addressed at the same time by the processor/cores involved..

In this example, both task 1 and task 2 can excecute through to completion without interfering with one another, because the system has multiple processors or cores to support parallel execution.

img

Finally, it is possible to have parallel and concurrent execution happening together. In this example, there are two processors/cores, each responsible for 2 tasks (i.e. CPU 1 handles task 1 and task 2, while CPU 2 handles task 3 and task 4). The CPU can slide up each task and alternate between them concurrently, while the other processor executes tasks in parallel.

img

So which do we prefer? Both have their uses, depending on the nature of the computation, the processor capabilities and other factors.

Parallel operations happen at the same time, and the operations logically do not interfere with one another. Parallel execution is typically managed by using threads to split computation into different units of execution that the processor can manage them independently. Modern hardware is capable of executing many threads simultaneously, although doing this is very resource intensive, and it can be difficult to manage threads correctly.

Concurrent tasks can be much easier to manage, and make sense for tasks where you need to make progress, but the tasks isn’t impeded by being interrupted occasionally.

Managing Threads

Kotlin has native support for creating and managing threads. This is done by

  • Creating a user thread (distinct from the main thread where you application code typically runs).
  • Defining some task for it to perform.
  • Starting the thread.
  • Cleaning up when it completes.

In this way, threads can be used to provide asynchronous execution, typically running in parallel with the main thread (i.e. running “in the background” of the program).

val t = object : Thread() {
	override fun run() {
  	// define the task here
    // this method will run to completion
  }
}

t.start() // launch the thread, execution will continue here in the main thread
t.stop() // if we want to halt it from executing

Kotlin also provides a helper method that simplifies the syntax.

fun thread(
  start: Boolean = true, 
  isDaemon: Boolean = false, 
  contextClassLoader: ClassLoader? = null, 
  name: String? = null, 
  priority: Int = -1, 
  block: () -> Unit
): Thread

// a thread can be instantiated quite simply.
thread(start = true) {
  // the thread will end when this block completes
  println("${Thread.currentThread()} has run.")
}

There are additional complexities of working with threads if they need to share mutable data: this a a significant problem. If you cannot avoid having threads share access to data, then read the Kotlin docs on concurrency first.

Threads are also very resource-intensive to create and manage4. It’s temping to spin up multiple threads as we need them, and delete them when we’re done, but that’s just not practical most of the time.

Although threads are the underlying mechanism for many other solutions, they are too low-level to use directly much of the time. Instead we rely on other abstractions that leverage threads safely behind-the-scenes. We’ll present a few of these next.

Callbacks

Another solution is to use a callback function. Essentially, you provide the long-running function with a reference to a function and let it run on a thread in the background. When it completes, it calls the callback function with any results to process.

The code would look something like this:

fun postItem(item: Item) {
    preparePostAsync { token ->
        submitPostAsync(token, item) { post ->
            processPost(post)
        }
    }
}

fun preparePostAsync(callback: (Token) -> Unit) {
    // make request and return immediately
    // arrange callback to be invoked later
}

This is still not an ideal solution.

  • Difficulty of nested callbacks. Usually a function that is used as a callback, often ends up needing its own callback. This leads to a series of nested callbacks which lead to incomprehensible code.
  • Error handling is complicated. The nesting model makes error handling and propagation of these more complicated.

Promises

Using a promise involves a long-running process, but instead of blocking it returns with a Promise - an object that we can reference immediately but which will be processed at a later time (conceptually like a data structure missing data).

The code would look something like this:

fun postItem(item: Item) {
    preparePostAsync()
        .thenCompose { token ->
            submitPostAsync(token, item)
        }
        .thenAccept { post ->
            processPost(post)
        }

}

fun preparePostAsync(): Promise<Token> {
    // makes request and returns a promise that is completed later
    return promise
}

This approach requires a series of changes in how we program:

  • Different programming model. Similar to callbacks, the programming model moves away from a top-down imperative approach to a compositional model with chained calls. Traditional program structures such as loops, exception handling, etc. usually are no longer valid in this model.
  • Different APIs. Usually there’s a need to learn a completely new API such as thenCompose or thenAccept, which can also vary across platforms.
  • Specific return type. The return type moves away from the actual data that we need and instead returns a new type Promise which has to be introspected.
  • Error handling can be complicated. The propagation and chaining of errors aren’t always straightforward.

  1. Yes, the OS can preemptively task switch to a different program, but this particular program will remain blocked by default. ↩︎

  2. Diagrams from https://www.backblaze.com/blog/whats-the-diff-programs-processes-and-threads/ ↩︎

  3. Diagrams taken from http://tutorials.jenkov.com/java-concurrency/concurrency-vs-parallelism.html ↩︎

  4. Threads consume roughly 2 MB of memory each. We can easy manage hundreds or thousands of threads, but it’s not unreasonable to picture building a large service that might require 100,000 concurrent operations (or 200 GB or RAM). We can’t reasonably handle that many threads on “regular” hardware.↩ ↩︎