Comparing XCTest vs. XCUITest: How Do They Relate?

In this post, we'll learn about the Apple testing frameworks XCTest vs. XCUITest. Although they are related, they have their differences. One is related to white-box testing, whereas the other is related to black-box testing.

We'll briefly discuss the two frameworks and then find their advantages and disadvantages. Then we'll see code examples for both of them.

What Is XCTest?

XCTest is a unit testing framework by Apple. Developers use it to test their apps. Most enterprise apps are developed with Agile methodology. In Agile, each developer gets a small part of the app to be developed in one to two weeks, which is known as a sprint. Then it's the developer's responsibility to do their part using unit tests.

You can also test Swift or Objective-C code with XCTest. In our post on Swift Struct, we've tested our Swift code using XCTest. Apple released XCTest at WWDC (World Wide Developer Conference) 2016. The test cases are written in the language of Swift or Objective-C.

Advantages of XCTest

• XCTest is built into XCode, the default IDE (Integrated Development Environment) for Mac systems. So, you don't need to do complicated setups that other frameworks, like Appium, require.
• We can also write XCTest on an iPad because XCode is now available on iPad and isn't dependent on Mac.
• XCTest is the fastest testing framework for iOS apps. The time to run the test on the iOS simulator is the fastest compared to doing it with other frameworks.
• You can write the test cases in Swift or Objective-C. The developers creating these test cases are quite familiar with these languages because they've created the app in the same language.
• XCTest integrates well with the Continuous Integration/Continuous Development (CI/CD) pipeline. These DevOps pipelines are a must for modern enterprise development.

Disadvantages of XCTest

• You can use XCTest only with iOS apps. We can't test Android apps with it.
• We can write test cases only in the Apple languages Swift and Objective-C.
• XCTest can't be used to test cross-mobile apps written in React Native. These apps have a common code base and run on both iPhone and Android devices. Other popular testing frameworks, like Appium, aren't restricted in this way. In our earlier post, we tested a React Native app with Appium.
• Testers or nontechnical users can't write test cases in English-like language. We can write these kinds of test cases with testing frameworks like Calabash.
• XCTest doesn't run well on real connected devices.

What Is XCUITest?

XCUITest is Apple's default automation testing framework. This test framework is based on XCTest. You can do automation testing for any iOS app. Previously, manual testing was used to test apps. But because it's time-consuming, testing is moving toward automation testing, which emulates a real user.

XCUITest is limited to iOS apps, written in either Swift or Objective-C. XCode is used to write XCUITest in Swift or Objective-C language. Besides doing automation testing, we can also do performance testing with XCUITest.

Advantages of XCUITest

• XCUITest is built into XCode, just like XCTest. So, you don't need to do complicated setups that similar automation mobile testing frameworks, like Appium, require.
• Because the setup in Appium is so complicated, we even have an additional tool, Appium doctor, to check if all setup is done properly.
• XCUITest is the fastest automation testing framework for iOS apps.
• Swift or Objective-C is used to write the test cases. The developers creating these test cases are quite familiar with these languages because they created the app in the same language.
• XCUITest integrates well with the CI/CD pipeline.
• XCode can easily record interactions with the simulator.

Disadvantages of XCUITest

• iOS apps can be tested only with XCUITest, and we can't test Android apps with it.
• We can write test cases only in the Apple languages Swift and Objective-C. Automation testers are more inclined toward Java and Python to write their test cases.
• XCUITest can't be used to test cross-mobile apps written in React Native.
• Testers or nontechnical users can't write test cases in English-like language.
• XCUITest also doesn't run well on real connected devices.

White-Box vs. Black-Box Testing

XCTest falls under white-box testing and XCUITest falls under black-box testing. Now, let's understand what these are.

White-box testing is a form of testing in which the tester knows the code or the app's structure. Typically, in white-box testing, the developer is actually the tester. The developer writes these kinds of unit tests to test their code. Knowledge of programming language is a must in this kind of testing.

Black-box testing is a type of testing in which the tester doesn't know the code or the app's structure. Typically, the tester isn't the app's developer. The tester mainly tests the functionality by interacting with the app like a real user. They deploy manual or automation testing to implement the same. Manual testing doesn't require knowledge of programming languages. But automation testing requires knowledge of programming languages.

XCTest Example

With XCTest, we mainly do unit testing for an app or any Swift code. We use testing methods like XCTAssertEqual() or XCTAssert() to test our code. The details of the two in-built functions are below:

• XCTAssertEqual() checks whether the passed expression has the same value.
• XCTAssert() checks whether the given expression is true. Here, we pass only one argument.

Here, in line 15, we're checking with XCTAssertEqual() whether the redCar.color is equal to Red.

We've also tested the Calculator app in our earlier post on XCUITest with XCTAssert() function. And we did it to test if the header exists in the app.

lazy var labelTextAI = app.staticTexts["calcHeader"]

@discardableResult
func checkLabelAI(completion: Completion = nil) -> Self {
  log("Check if Label text exists by Accessibility Identifier")
  XCTAssert(labelTextAI.exists)
  return self
}

XCUITest Example

With XCUITest, we do automation testing on an app running on a simulator. Thus, it seems as if a real user is doing the interactions.

In our earlier post on XCUITest, when we ran the test cases, the Calculator app was opened automatically and numbers 1, +, 5, and = were pressed in it. Therefore, we selected the real elements of buttons 1, +, 5, and =. Through Accessibility Inspector, we got these labels. We then used the built-in method of tap and performed the tap.

That post contains the complete setup, code, and test running steps.

import XCTest

public class CalcPage: BaseTest {
  override var rootElement: XCUIElement{
   return app.staticTexts["1"]
  }

  lazy var oneButton = app.staticTexts["1"]
  lazy var plusButton = app.staticTexts["+"]
  lazy var fiveButton = app.staticTexts["5"]
  lazy var equalButton = app.staticTexts["="]

  @discardableResult
  func tapOne(completion: Completion = nil) -> Self {
   log("One button tap done")
   oneButton.tap()
   return self
 }

  @discardableResult
  func tapFive(completion: Completion = nil) -> Self {
   log("Five button tap done")
   fiveButton.tap()
   return self
 }

  @discardableResult
  func tapPlus(completion: Completion = nil) -> Self {
   log("Plus button tap done")
   plusButton.tap()
   return self
 }

  @discardableResult
  func tapEqual(completion: Completion = nil) -> Self {
    log("Equal button tap done")
    equalButton.tap()
    return self
  }
}

What You've Learned

In this post, you learned about XCTest and XCUITest. You learned about the advantages and disadvantages of both. In addition, you learned about white-box and black-box testing. XCTest is a white-box testing method because the tester has access to the code. By contrast, XCUITest is a black-box testing method because the tester doesn't have access to the code. And finally, you also learned about writing test cases with both frameworks from Apple.

If you want to test your apps more easily and without all these complicated setups and test cases, you should try Waldo. You only need to provide the APK or IPA file. After that, you can interact with the mobile app like a real user. In addition, it'll automatically generate the test cases for you.