Writing tests for modern async code with XCTest

If you’ve ever looked at unit testing your code with Xcode, you will have seen XCTest. It’s a fairly straightforward framework to test your Objective-C and Swift code with. However, testing asynchronous code has always been a bit of a pain due to constructs like delegates and callbacks.

In this post, we will start by looking at the classic way of testing asynchronous code to make sure we’re all on the same page regarding the pros and cons of a classic async test. After that, we’ll take a look at how async / await dramatically changes the way we write unit tests for asynchronous code, and how it even changes the ways in which our tests can succeed and fail.

Exploring the classic way of writing asynchronous test

By default, XCTest cases run synchronously which means that any callback based code where your callback isn’t called synchronously and immediately requires some special attention while testing. The core tool for this used to be ^{XCTestExpectation} which allows developers to not mark a test as completed (or failed) until the expectation was marked as finished. This didn’t necessarily mean that the test completed successfully, just that the asynchronous code completed.

As a reminder (or a quick introduction if this is all new to you), here’s what an expectation based test looks like:

final class MovieLoaderTests: XCTestCase {
    
    var loader: MovieLoader!
    
    override func setUp() {
        loader = MovieLoader()
    }
    
    func testMoviesCanBeLoaded() {
        // 1
        let moviesExpectation = expectation(description: "Expected movies to be loaded")
        // 2
        loader.loadMovies { movies in
            moviesExpectation.fulfill()
            
            XCTAssert(movies.count == 10, "Expected 10 movies to be loaded")
        }
        // 3
        waitForExpectations(timeout: 1)
    }

}

This code initially sets up a test expectation. We provide a nice description to make sure that we can easily identify this expectation should anything go wrong.

After that, we call our asynchronous callback based code. When the ^loadMovies method comes back with a set of ^Movie objects (which I haven’t included the definition for here because it’s not relevant to the testing code), we fulfill the expectation which allows our test to continue running, and we make some assertions.

Lastly, we call ^{waitForExpectations} with a timeout of one. This means that we expect our expectation to be fulfilled within one second. If this doesn’t happen our test will automatically fail.

The code above looks pretty simple and straightforward but imagine our callback would be a bit more complicated:

loader.loadMovies { result in
    guard let movies = try? result.get() else {
        return
    }
    
    moviesExpectation.fulfill()
    
    XCTAssert(movies.count == 10, "Expected 10 movies to be loaded")
}

In this code, we receive a ^{Result<[Movie], Error>} in our closure instead of just an array of ^Movie. This means that in order to check the number of loaded movies we need to grab the success case from our result (which we can conveniently do with ^get() and then make sure that we actually have a list of movies before we proceed.

This quickly introduced in issue in our code because instead of fulfilling the expectation before running my ^guard, I only fulfill the expectation after the guard. This effectively means that our test will always fail with an unfulfilled expectation error instead of an error that describes what actually happens (we’ve received a ^Result but it’s not what we expected).

To fix this, we would need to update our code to look like this:

loader.loadMovies { result in
    moviesExpectation.fulfill()
    
    guard let movies = try? result.get() else {
        XCTFail("Expected movies to be loaded")
        return
    }
    
    XCTAssert(movies.count == 10, "Expected 10 movies to be loaded")
}

While this code is a lot easier to reason about, you can surely imagine that a much larger and more complicated test can easily contain some mistakes regarding assertions that are never run even though we want them to, or that we end up with some unfulfilled expectations due to certain code paths that simply never hit a point that fulfills our expectations.

It’s also important to note that debugging the test above would be frustrating. If we unexpectedly receive an error from ^loadMovies we wouldn’t immediately know which error we’ve encountered.

Luckily, your tests are much, much nicer once you start migrating over to async / await. So let’s take a look at how we can write more modern async tests.

Exploring tests with async / await

Async await allows us to receive results from asynchronous methods simply by using the ^await keyword. And we can mark any methods that should be run asynchronously with ^async and the system will automatically perform any work needed to allow that method to be run asynchronously.

If you haven’t seen this before, here’s a very brief example of what this looks like:

// defined as
func loadMovies() async throws -> [Movie] {
    let data = try await loader.fetchMovieData()
    let movies = try JSONDecoder().decode([Movie].self, from: data)
    
    return movies
}

// called as
let movies = try await loadMovies()

Notice how we no longer need to define a ^Result. Our async code can throw errors or successfully return a result. This is extremely similar to how we would normally write synchronous code.

So how can we test this asynchronous ^loadMovies function then?

Well, what’s really cool is that since Xcode 13 we’re allowed to mark our test methods as async. This enables the ability for our test methods to suspend and wait for the results of asynchronous work that is initiated from with the test.

In other words, we no longer need to juggle test expectations and we can directly await the results of our asynchronous work inside of a unit test.

func testMoviesCanBeLoadedAsync() async {
    do {
        let movies = try await loader.loadMovies()
        XCTAssert(movies.count == 10, "Expected 10 movies to be loaded")
    } catch {
        XCTFail("Expected movies to be loaded")
    }
}

This is already much easier to reason about than the test we’ve had before. However, we’re still lacking the ability to easily see what went wrong if we weren’t able to load movies. The reason for this is that we catch but don’t actually handle the error. We could update our ^XCTFail line to include the error’s ^description but we can actually do better.

Our tests can’t just be ^async they can also be throwing! This means that we can greatly simplify our test as follows:

func testMoviesCanBeLoadedAsync() async throws {
    let movies = try await loader.loadMovies()
    XCTAssert(movies.count == 10, "Expected 10 movies to be loaded")
}

These two lines of code are way easier to read than the callback based version of the same test. And more importantly, it’s much harder to make mistakes in the test even if it grows in number of lines and complexity since we no longer have to think about fulfilling our expectations.

Another huge win is that we can now fail our test if any step unexpectedly throws an error and Xcode will actually include some brief information about the error in the test failure message.

For example, we might see the following failure message:

<unknown>:0: error: -[SampleApp.MovieLoaderTests testMoviesCanBeLoadedAsync] : failed: caught error: "The operation couldn’t be completed. (SampleApp.MovieError error 0.)"

It’s not much, and it might not always be enough to have a solid starting point for debugging but it sure beats seeing nothing more than a generic test failure. At least with this message we know to take a look at the ^MovieError definition to find out which error we’ve defined as the first ^case (which is what that ⁰ in the output suggests to be the thrown error).

Depending on how much information you desire in your failed test output, you can write a pretty solid async unit test in just two lines of code which, if you ask me, is pretty awesome.

My favorite part about the ability to mark test cases as ^async is that it’s so much easier to reason about the flow of a test, how and where it might fail, and it allows me to focus on just the test instead of also having to think about expectations and making sure they’re always fulfilled.

In Summary

In this post, you’ve learned everything you need to know about writing unit tests for modern asynchronous code. We started off by looking at test expectations which have been the standard too for writing async tests in XCTest for a very long time. You’ve learned that testing with expectations can be error prone, and hard to reason about . You also saw that it’s pretty easy to end up writing tests that aren’t easy to debug should anything go wrong unexpectedly.

After that, you saw how you can mark your test methods as both ^async and ^throws which allowed us to take a test that was almost 10 lines long and transform it into a test of just 2 lines while also making it easier to reason about the test, making it harder to introduce subtle test bugs, and even improving the reporting when the test fails. All by moving to async / await.

Swift Concurrency is a very exciting technology that, as you now know, impacts more than just our production code. It also has a tremendously positive impact on our testing code.