Unit tests promise the developers to allow refactoring, which is changing the implementation without modifying the behaviour of the code. Unfortunately, this often turns out to be the false promise in complex enough systems. Is it the problem with unit testing in general?
It turns out one of the common issues in unit testing is over-isolation. What might come as a shock is that:
Mocking should be avoided at all costs. It should only be used to isolate from significant architectural boundaries.
But wait, mocking allows us to write unit test in the first place, right? The correct answer is that it depends on the context:
- It would be impossible to write reliable tests without mocking the boundaries of the system like the file system or networking services.
- Excessive mocking inside the boundaries (example: inside the module) only slows down or entirely prevents refactoring.
What are the boundaries? As a rule of thumb, let's define the boundaries for the iOS app as the contents of the singular Xcode project or framework:
- Apple-provided frameworks are the boundaries.
- External libraries are the boundaries.
- The networking and the file system code are the boundaries.
- Unless your team delivers a product composed of the independently developed frameworks, there are very few to no boundaries within the code written by the team.
Mocking as an anti-pattern
Imagine we have a product defined as follows:
struct Product {
let name: String
let photo: Data
}where the photo property is a data of .png encoded image. We want to present this product inside the ProductView:
class ProductView {
let nameLabel = UILabel()
let photoImageView = UIImageView()
// rest of the implementation
}Given the PhotoPresenting protocol:
protocol PhotoPresenting {
func present(photo: Data, in imageView: UIImageView)
}we can implement the ProductPresenter as follows:
class ProductPresenter {
init(photoPresenter: PhotoPresenting) {
self.photoPresenter = photoPresenter
}
func present(product: Product, in view: ProductView) {
photoPresenter.present(
photo: product.photo, in: view.photoImageView)
view.nameLabel.text = product.name
}
private let photoPresenter: PhotoPresenting
}Let's test the ProductPresenter with a mocked instance of ProductPresenting:
extension UIImage {
static func testImageData(color: UIColor) -> Data {
let rect = CGRect(x: 0, y: 0, width: 100, height: 100)
let image = UIGraphicsImageRenderer(size: rect.size)
.image { context in
color.setFill()
context.fill(rect)
}
return image.pngData()!
}
}
class PhotoPresenterSpy: PhotoPresenting {
var invokedPresent: (photo: Data, view: UIImageView)?
func present(photo: Data, in imageView: UIImageView) {
invokedPresent = (photo: photo, view: imageView)
}
}
class ProductPresenterSpec: QuickSpec {
override func spec() {
describe("ProductPresenter") {
var photoPresenterSpy: PhotoPresenterSpy!
var sut: ProductPresenter!
beforeEach {
photoPresenterSpy = PhotoPresenterSpy()
sut = ProductPresenter(
photoPresenter: photoPresenterSpy)
}
afterEach {
photoPresenterSpy = nil
sut = nil
}
describe("present") {
var productView: ProductView!
beforeEach {
let product = Product(
name: "Product",
photo: UIImage.testImageData(color: .systemRed)
)
productView = ProductView()
sut.present(product: product, in: productView)
}
afterEach {
productView = nil
}
it("should present the product name") {
expect(productView.nameLabel.text) == "Product"
}
it("should present the photo") {
expect(photoPresenterSpy.invokedPresent)
.toNot(beNil())
expect(photoPresenterSpy.invokedPresent?.photo)
== UIImage.testImageData(color: .systemRed)
expect(photoPresenterSpy.invokedPresent?.view)
=== productView.photoImageView
}
}
}
}
}Now, assume we want to do a very simple change: replace the PNG encoded image data with the Base64 format. We leave the input data intact:
// Previously:
struct Product {
let name: String
let photo: Data
}
// Now:
struct Product {
let name: String
let photo: Base64Image
init(name: String, photo: Data) {
self.name = name
self.photo = Base64Image(
encoded: photo.base64EncodedString())
}
}
struct Base64Image {
let encoded: String
}
Of course, we do not want to modify the behaviour of the application. Next, we change the PhotoPresenting interface to match the updated code:
// Previously:
protocol PhotoPresenting {
func present(photo: Data, in imageView: UIImageView)
}
// Now:
protocol PhotoPresenting {
func present(photo: Base64Image, in imageView: UIImageView)
}We need to update PhotoPresenterSpy accordingly:
// Previously:
class PhotoPresenterSpy: PhotoPresenting {
var invokedPresent: (photo: Data, view: UIImageView)?
func present(photo: Data, in imageView: UIImageView) {
invokedPresent = (photo: photo, view: imageView)
}
}
// Now:
class PhotoPresenterSpy: PhotoPresenting {
var invokedPresent: (photo: Base64Image, view: UIImageView)?
func present(photo: Base64Image, in imageView: UIImageView) {
invokedPresent = (photo: photo, view: imageView)
}
}What happened?
- We did the refactoring by introducing a change to our internal representation of the model.
- We have not changed the behaviour of the application.
- The tests do not compile anymore!
it("should present the photo") {
expect(photoPresenterSpy.invokedPresent)
.toNot(beNil())
// We use a different, non-equatable data type now
expect(photoPresenterSpy.invokedPresent?.photo)
== UIImage.testImageData(color: .systemRed)
expect(photoPresenterSpy.invokedPresent?.view)
=== productView.photoImageView
}We have to change the test code to make compiler happy:
it("should present the photo") {
expect(photoPresenterSpy.invokedPresent)
.toNot(beNil())
expect(photoPresenterSpy.invokedPresent?.photo.encoded)
== UIImage.testImageData(color: .systemRed)
.base64EncodedString()
expect(photoPresenterSpy.invokedPresent?.view)
=== productView.photoImageView
}That is a huge problem as it breaks the testing promise.
Our tests should support refactoring. Unfortunately, it turned out that refactoring requires us to rebuild the test code, too.
What if our tests relied on the side-effect performed by the real PhotoPresenter implementation instead?
class ProductPresenterSpec: QuickSpec {
override func spec() {
describe("ProductPresenter") {
var sut: ProductPresenter!
beforeEach {
sut = ProductPresenter(
photoPresenter: PhotoPresenter())
}
afterEach {
sut = nil
}
describe("present") {
var productView: ProductView!
beforeEach {
let product = Product(
name: "Product",
photo: UIImage.testImageData(color: .systemRed)
)
productView = ProductView()
sut.present(product: product, in: productView)
}
afterEach {
productView = nil
}
it("should present the product name") {
expect(productView.nameLabel.text) == "Product"
}
it("should present the photo") {
expect(productView.photoImageView.image)
.toNot(beNil())
expect(productView.photoImageView.image?.pngData())
== UIImage.testImageData(color: .systemRed)
}
}
}
}
}It turns out that now we can change Product internal structure without updating the test code. Our tests greatly support the refactoring process.
Is lacking isolation in the tests a problem? Not really. There is no production code usage of ProductPresenter which does not rely on PhotoPresenter, so we are exercising the application code even more.
Bottom-up approach
To understand the correct approach for testing the code we should turn to functional programming. In functional programming, the code is composed of the tiny functions which are piped together to achieve the desired outcome. Imagine we are building a method to format an order in the following format:
Somebody ordered a burger with quadruple cheeseFirstly, we need to format a quantity (single, double, ...):
func formatQuantity(_ quantity: Int) -> String {
switch quantity {
case Int.min...0:
return "none"
case 1:
return "single"
case 2:
return "double"
case 3:
return "triple"
case 4:
return "quadruple"
default:
return "\(quantity)x"
}
}We can provide an exhaustive test for each condition:
class FormatQuantitySpec: QuickSpec {
override func spec() {
describe("formatQuantity") {
it("should return none for -1") {
expect(formatQuantity(-1)) == "none"
}
it("should return none for 0") {
expect(formatQuantity(0)) == "none"
}
it("should return single for 1") {
expect(formatQuantity(1)) == "single"
}
it("should return double for 2") {
expect(formatQuantity(2)) == "double"
}
it("should return triple for 3") {
expect(formatQuantity(3)) == "triple"
}
it("should return quadruple for 4") {
expect(formatQuantity(4)) == "quadruple"
}
it("should return 5x for 5") {
expect(formatQuantity(5)) == "5x"
}
it("should return 6x for 6") {
expect(formatQuantity(6)) == "6x"
}
}
}
}Secondly, we build another method that uses formatQuantity underneath:
func formatPurchase(customer: String, quantity: Int) -> String {
let howMuch = formatQuantity(quantity)
return "\(customer) ordered a burger with \(howMuch) cheese"
}The formatPurchase tests do not have to cover all of the formatQuantity branches. Instead, we only add enough tests to:
- Go through all of the branches in the system under test.
- Make sure we pass the arguments correctly.
class FormatPurchaseSpec: QuickSpec {
override func spec() {
describe("formatPurchase") {
it("should return correct string with 4") {
expect(formatPurchase(customer: "Mike", quantity: 4))
== "Mike ordered a burger with quadruple cheese"
}
it("should return correct string with 8") {
expect(formatPurchase(customer: "Jane", quantity: 8))
== "Jane ordered a burger with 8x cheese"
}
}
}
}In this case, the formatPurchase method has no branches so we end up with 2 tests verifying the quantity parameter is wired correctly.
I like to call this approach bottom-up testing because it starts with the exhaustive tests of the lowest-level layer and then builds up each layer on the foundations of a well-tested layer below.
Solving real problems with bottom-up testing
Bottom-up approach paired with exhaustive unit testing is a great way to test edge cases that would be normally impossible or very hard to check. Imagine we are processing driving license candidates:
struct Candidate {
let name: String
let birthDate: Date
let nationality: Country
}
enum Country: String {
case argentina
case france
case iceland
case poland
}For each candidate, we know his/her name, birth date and nationality. Our task is to filter out the candidates who are eligible for driving licenses.
The rest of the example is written in a functional fashion. Please, try not to consider it too much. Each described function could be defined as a class method instead, it is just shorter this way.
Firstly, we need to determine what is the legal age for obtaining a driving license depending on a country. Since this information is crucial, we need to make sure it is calculated correctly for each of the countries we have defined:
class MinimumDrivingAgeSpec: QuickSpec {
override func spec() {
describe("minimum driving age") {
it("should return 17 for Argentina") {
expect(minimumDrivingAge(in: .argentina)) == 17
}
it("should return 15 for France") {
expect(minimumDrivingAge(in: .france)) == 15
}
it("should return 16 for Iceland") {
expect(minimumDrivingAge(in: .iceland)) == 16
}
it("should return 18 for Poland") {
expect(minimumDrivingAge(in: .poland)) == 18
}
}
}
}The minimumDrivingAge method is straightforward to implement:
func minimumDrivingAge(in country: Country) -> Int {
switch country {
case .argentina:
return 17
case .france:
return 15
case .iceland:
return 16
case .poland:
return 18
}
}Before we can implement further tests, we need a method to declare hardcoded test dates. It can be problematic. The easiest method to define a date is Date(timeIntervalSince1970:) initialiser. However, it is not much of a use to the programmer:
let date = Date(timeIntervalSince1970: 631886400)What date is it? We need to use an online date converter to find out. It is much better to implement a converter which allows to define dates in a string format:
let date = "1990-01-09 12:00:00".dateThe implementation is unrelated to the blog post itself and will be omitted.
Now, we can calculate the age of the candidate based on the birthDate[1]:
class CalculateAgeSpec: QuickSpec {
override func spec() {
describe("calculateAge") {
it("should return correct value for 18 years") {
let from = "2002-01-01 00:00:00".date
let to = "2020-01-01 00:00:00".date
expect(calculateAge(from, to)) == 18
}
it("should return correct value for 25 years") {
let from = "1994-12-31 10:00:00".date
let to = "2019-12-31 10:00:00".date
expect(calculateAge(from, to)) == 25
}
}
}
}The simplest implementation passing both tests is:
func calculateAge(_ from: Date, _ to: Date) -> Int {
Calendar.current.dateComponents(
[.year], from: from, to: to).year!
}It works, but is the code any good?
Calendar.currentis the usual suspect. We are not injecting the calendar and therefore we have no control over the timezone in which the calculations are performed. However, since bothfromandtodates are getting converted to the same timezone it should not cause any issues. Is there anything else to think of?- The
CalendarAPI does not guarantee that theyearcomponent will be returned. Although it is very unlikely (or impossible) to happen, we still should be able to test that scenario and ultimately get rid of the!.
To find out the truth, let's dive a level deeper. Instead of using the dateComponents(_:from:to:) method directly, we will wrap it in a function that allows us to inject a Calendar instance and returns the optional value in line with the API:
func yearsBetweenDates(in calendar: Calendar)
-> (Date, Date) -> Int? {
// still to be implemented
}To aid exhaustive tests of the yearsBetweenDates method, we introduce a special behaviour that encapsulates a single test case. Considering its implementation unrelated to the post itself, let's go straight to testing the specific calendar cases[1]:
class YearsBetweenDatesSpec: QuickSpec {
override func spec() {
describe("yearsBetweenDates") {
itBehavesLike(YearsBetweenDatesBehavior.self) {
.init(sut: yearsBetweenDates(in:),
from: "2002-01-01 00:00:00",
to: "2020-01-01 00:00:00",
calendar: .gregorian,
expected: 18)
}
itBehavesLike(YearsBetweenDatesBehavior.self) {
.init(sut: yearsBetweenDates(in:),
from: "1992-11-10 12:15:00",
to: "2020-01-21 19:30:00",
calendar: .gregorian,
expected: 27)
}
}
}
}Those two behave as expected if we retain our "original" implementation:
func yearsBetweenDates(in calendar: Calendar)
-> (Date, Date) -> Int? {
return { from, to in
calendar.dateComponents([.year], from: from, to: to).year
}
}However, if we flip the from and the to date the following test case will not pass:
class YearsBetweenDatesSpec: QuickSpec {
override func spec() {
describe("yearsBetweenDates") {
itBehavesLike(YearsBetweenDatesBehavior.self) {
.init(sut: yearsBetweenDates(in:),
from: "2020-01-01 00:00:00",
to: "2002-01-01 00:00:00",
calendar: .gregorian,
expected: 18)
}
}
}
}Let's fix the implementation to work regardless of the order:
func yearsBetweenDates(in calendar: Calendar)
-> (Date, Date) -> Int? {
return { from, to in
calendar.dateComponents([.year], from: from, to: to)
.year.map(abs)
}
}Next, let's check what happens if we change the calendar type. Our goal is to confirm that the maths is unrelated to the selected calendar type. We add test cases for all the calendars until we find an unexpected result[1]:
itBehavesLike(YearsBetweenDatesBehavior.self) {
.init(sut: yearsBetweenDates(in:),
from: "2002-01-01 00:00:00",
to: "2020-01-01 00:00:00",
calendar: .chinese,
expected: 18)
}
itBehavesLike(YearsBetweenDatesBehavior.self) {
.init(sut: yearsBetweenDates(in:),
from: "2002-01-01 00:00:00",
to: "2020-01-01 00:00:00",
calendar: .japanese,
expected: 18)
}
itBehavesLike(YearsBetweenDatesBehavior.self) {
.init(sut: yearsBetweenDates(in:),
from: "2002-01-01 00:00:00",
to: "2020-01-01 00:00:00",
calendar: .buddhist,
expected: 18)
}
itBehavesLike(YearsBetweenDatesBehavior.self) {
.init(sut: yearsBetweenDates(in:),
from: "2002-01-01 00:00:00",
to: "2020-01-01 00:00:00",
calendar: .hebrew,
expected: 17) // !!!
}Whoa! It turns out that for the .hebrew calendar the yearsBetweenDates method returns an unexpected age. This means our code can break depending on the calendar settings on the device. Therefore, we should hardcode a .gregorian calendar instance for the age calculation.
Next, let's rebuild the calculateAge function so that it uses .gregorian calendar and also handles the missing year scenario. To achieve that, we introduce a helper function[1]:
func age(_ yearsBetween: @escaping
(Date, Date) -> Int?)
-> (Date, Date) -> Int {
// implementation to be built
}This allows us to change the method for resolving a number of years passed between dates in tests so that we can simulate the nil value behaviour. Let's write the tests first[1]:
class AgeCalculatorSpec: QuickSpec {
override func spec() {
describe("age") {
var sut: ((Date, Date) -> Int)!
afterEach {
sut = nil
}
context("with real year calculating function") {
beforeEach {
sut = age(yearsBetweenDates(
in: .init(identifier: .gregorian)))
}
it("should return correct value for 18 years") {
let from = "2002-01-01 00:00:00".date
let to = "2020-01-01 00:00:00".date
expect(sut(from, to)) == 18
}
it("should return correct value for 25 years") {
let from = "1994-12-31 10:00:00".date
let to = "2019-12-31 10:00:00".date
expect(sut(from, to)) == 25
}
}
context("with broken year calculating function") {
beforeEach {
sut = age { _, _ in nil }
}
it("should return 0") {
let from = "2002-01-01 00:00:00".date
let to = "2020-01-01 00:00:00".date
expect(sut(from, to)) == 0
}
}
}
}
}Here is where the bottom-up approach kicks in:
- We do not stub the year calculation function. Instead, we simply use the production dependency we intend to work with.
- We do not cover all of the
yearsBetweenDatesedge-cases. We build just enough test cases to make sure we are passing correct arguments to our dependency. - We add test cases for all branches inside our brand new
agefunction. We have two branches depending on whether the year is present. If not,0is returned.
Let's build the age method implementation:
func age(_ yearsBetween:
@escaping (Date, Date) -> Int?)
-> (Date, Date) -> Int {
return { from, to in yearsBetween(from, to) ?? 0 }
}Finally, we can refactor the code to simplify the setup a little bit:
// calculateAge is the age function with
// hardcoded year calculation method:
let calculateAge = age(yearsBetweenDates(in:
Calendar(identifier: .gregorian)))
class AgeCalculatorSpec: QuickSpec {
override func spec() {
describe("age") {
var sut: ((Date, Date) -> Int)!
afterEach {
sut = nil
}
context("with real year calculating function") {
beforeEach {
// usage of the real calculateAge in tests:
sut = calculateAge
}
it("should return correct value for 18 years") {
let from = "2002-01-01 00:00:00".date
let to = "2020-01-01 00:00:00".date
expect(sut(from, to)) == 18
}
it("should return correct value for 25 years") {
let from = "1994-12-31 10:00:00".date
let to = "2019-12-31 10:00:00".date
expect(sut(from, to)) == 25
}
}
context("with broken year calculating function") {
beforeEach {
sut = age { _, _ in nil }
}
it("should return 0") {
let from = "2002-01-01 00:00:00".date
let to = "2020-01-01 00:00:00".date
expect(sut(from, to)) == 0
}
}
}
}
}Composing methods
Up to this point, we have two methods implemented:
minimumDrivingAge(in country: Country) -> IntcalculateAge(_ from: Date, _ to: Date) -> Int
They're both well-tested and we can easily build on top of them. Our task is to find candidates eligible to drive. Instead of filtering the whole array at once (it is harder to set up in tests), we will start with finding out whether the given candidate is eligible:
func isEligibleToDrive(
candidate: Candidate, now: Date) -> Bool {
// to be implemented
}As usual, we build the test cases first[1]:
class IsEligibleToDriveSpec: QuickSpec {
override func spec() {
describe("isEligibleToDrive") {
var candidate: Candidate!
var now: Date!
beforeEach {
now = "2020-01-21 20:20:20".date
}
afterEach {
now = nil
candidate = nil
}
context("for candidate aged 17 in Poland") {
beforeEach {
candidate = Candidate(name: "",
birthDate: "2003-01-12 10:50:00".date,
nationality: .poland)
}
it("should return false") {
expect(isEligibleToDrive(candidate:
candidate, now: now)) == false
}
}
context("candidate aged 18 in Poland") {
beforeEach {
candidate = Candidate(name: "",
birthDate: "2001-12-31 12:22:22".date,
nationality: .poland)
}
it("should return true") {
expect(isEligibleToDrive(candidate:
candidate, now: now)) == true
}
}
context("candidate aged 15 in France") {
beforeEach {
candidate = Candidate(name: "",
birthDate: "2005-01-01 08:23:00".date,
nationality: .france)
}
it("should return true") {
expect(isEligibleToDrive(candidate:
candidate, now: now)) == true
}
}
}
}
}
Again, we do not need to worry about complete test cases for each of the composing functions. With the bottom-up approach, we need just enough test cases to make sure that composing functions are wired with correct arguments. Also, we need to make sure that boundary conditions are handled correctly, that is why we chose ages 17 & 18 for Poland.
Let's implement the filter:
func isEligibleToDrive(candidate: Candidate,
now: Date) -> Bool {
calculateAge(candidate.birthDate, now)
>= minimumDrivingAge(in: candidate.nationality)
}The tests should pass now.
The final step is to filter the whole array of candidates. Since we have already built a solid underlying layer this becomes a very simple check:
class DrivingLicenseCandidatesFilterSpec: QuickSpec {
override func spec() {
describe("filterCandidates") {
var candidates: [Candidate]!
beforeEach {
let allCandidates = [
Candidate(name: "Janusz",
birthDate: "2003-01-12 10:50:00".date,
nationality: .poland),
Candidate(name: "Pierre",
birthDate: "2005-01-01 08:23:00".date,
nationality: .france),
]
candidates = filterCandidates(allCandidates,
now: "2020-01-21 20:20:20".date)
}
afterEach {
candidates = nil
}
it("should return filtered candidates") {
expect(candidates).to(haveCount(1))
expect(candidates.first?.name) == "Pierre"
}
}
}
}And a rather simplistic implementation:
func filterCandidates(_ candidates: [Candidate],
now: Date) -> [Candidate] {
candidates.filter {
isEligibleToDrive(candidate: $0, now: now)
}
}
Using the bottom-up approach we have built an exhaustive test suite. We can be fairly confident about using the code in production.
Drawbacks
As with every programming pattern, bottom-up testing has its' drawbacks and limitations you have to be aware of. The obvious one is the process is counterintuitive. Usually, we build features in a top-down fashion:
- Start small.
- Build a skeleton of the feature with a minimalistic subset of requirements.
- Slowly add new code to accommodate more requirements.
Therefore, the bottom-up approach requires a lot of refactoring around the test code when developing new features. It is in line with the regular TDD approach to change the shape of the code often until we get a full grasp of how the module should work. However, if we only give the refactoring a lick and a promise and jump straight to the next requirement implementation the test code gets polluted with duplications quickly.
Speaking of duplications, the bottom-up approach requires strict extraction of similar test cases. Since we often verify the same cases for both lower- and higher-level layers, the expectations should be nicely extracted to make sure the single requirements change only alters a single place in the test code.
The final drawback is that removing tests or changing method composition on a higher-level code weakens the tests heavily. Take the filterCandidates method as an example:
class DrivingLicenseCandidatesFilterSpec: QuickSpec {
override func spec() {
describe("filterCandidates") {
var candidates: [Candidate]!
beforeEach {
let allCandidates = [
Candidate(name: "Janusz",
birthDate: "2003-01-12 10:50:00".date,
nationality: .poland),
Candidate(name: "Pierre",
birthDate: "2005-01-01 08:23:00".date,
nationality: .france),
]
candidates = filterCandidates(allCandidates,
now: "2020-01-21 20:20:20".date)
}
afterEach {
candidates = nil
}
it("should return filtered candidates") {
expect(candidates).to(haveCount(1))
expect(candidates.first?.name) == "Pierre"
}
}
}
}The tests are crude because they rely on the isEligibleToDrive method existence. What if, as a part of the refactoring, we want to get rid of that method? It would require hardening of the specs first. There are two ways to look at it:
- It is still a much better scenario compared to using mocked dependencies. In the case of mocking, the
filterCandidatesmethod is only proved to invokeisEligibleToDrivemethod passing correct arguments. We do not have even a single test that proves it correctness when replacing the underlying dependency with a different contract. - We should aim to retroactively harden the top-level test suite to prove the correctness of user stories and acceptance criteria.
Speaking from the experience, the second approach is a way to go. All in all, the sole purpose of having the extensive test suite is to make sure the behaviour of the application does not change when the code changes. The behaviour of the application is described in the form of the user stories and acceptance criteria, therefore we should go out of our way to double-check those scenarios.
Final thoughts
Bottom-up (or functional) approach makes it easy to extensively verify an application's behaviour and goes hand in hand with the test-driven development. Although it requires a lot of practice to master the craft and creative thinking to pull it off for the concrete use case, it is worth a shot.
I hope you enjoyed the article and it gave you the idea of how the tests should be architected. Happy testing!
[1] Although the article is written in a TDD-like spirit ("let's write the tests first"), it does not showcase the whole TDD process. The presenting test cases stem from multiple iterations of test-driven development.
This article is a part of 101 iOS unit testing series. Other articles in the series are listed here.