Blog Infos
Author
Published
Topics
Published
Topics
Mocks, stubs, fakes, dummies, and spies on Android: from theory to (good) practice

Regardless of the technology or product you’re working with, knowing how to use test doubles is fundamental to any automated testing strategy. On Android in particular, using this kind of resource becomes even more important when dealing with the non-instrumented tests in your suite. Essentially, the concept behind test doubles is quite simple, but the great number of namings, definitions, and tools available cause a lot of confusion in the development community, you’ve probably heard something like this:

  • “We need to mock this dependency and everything will work fine” 🙌
  • “Avoid using Mocks!” 😱
  • “Mocks vs Stub?” ⚔️
  • “Prefer using Fakes than Mocks” 🤔

You may believe me or not, but the above sentences can be interpreted differently if we don’t know the correct definition of things. If you’ve never heard about test doubles or want to dig deeper into this subject, this blog post is for you!

What are test doubles?

Before explaining, I would like to recall some basic characteristics of a good unit test: speed, determinism, and easy configuration. With that in mind we can define test doubles as the following:

Test doubles are lightweight substitutes that override real dependencies needed to test a system or behavior. Test doubles are best applied when they replace slow, non-deterministic, or difficult-to-configure dependencies.

Diagram showing the use of test doubles to replace non-deterministic and slow dependencies with deterministic and fast ones.

The concept of test doubles was created by Gerard Meszaros in the book XUnit Test Patterns: Refactoring Test Code and refined by many other references in the software engineering and testing area. In the tech literature, we can find 5 categories of test doubles: Dummies, Stubs, Fakes, Mocks and Spies. Each one of them with its specific purpose.

Dummy

The Dummy is the simplest test double. It has the sole purpose of being passed as argument, while not having much relevance to the test itself.

Dummies are often used to fill in mandatory parameters and nothing else. We usually don’t need additional tools to create them, although it’s possible to do so.

@Test
fun `Update registered note count when registering a new note in empty repository`() {
val dummyNote = //Dummy
noteRepository.registerNote(dummyNote) //Just filling the parameter, the double's content is not relevant for the test
val allNotes = noteRepository.getNotes()
assertEquals(expected = 1, actual = allNotes.size)
}
view raw DummyExample.kt hosted with ❤ by GitHub

A simple example of Dummy utilization in a test

 

In the example above, dummyNote has the sole purpose of being passed as a parameter and its internal values are not very relevant to the test. Below we can find some examples of variables that can be also considered Dummies:

//----- Literal dummy -----
val dummyPrice = 10.0
//----- Generated dummy -----
val dummyCustomer = CustomerTestBuilder.build()
//----- Alternative empty implementation -----
val dummyNote = DummyNote()
class DummyNote(): Note //No implementation
class RealNote(): Note //Real implementation
view raw OtherDummies.kt hosted with ❤ by GitHub

Other types of dummies generated without any tooling

 

Dummies usually replace data entities that are difficult to configure. They also help to keep the test code small, clean, and free from external tools for its simplicity. Only use another test double or tooling if it’s really necessary.

Stub

The Stub is a test double that provides fixed or pre-configured answers to replace the actual implementation of a dependency.

Stubs are usually generated by tools added to your projects, such as Mockito or Mockk, but they can also be implemented manually. They prevent slow or non-deterministic calls from being made during the test execution.

@Test
fun `Retrieve notes count from server when requested`() {
val notesApiStub = //Stub
val note = //Dummy
val noteRepository = NoteRepository(notesApiStub) //System under test
//Stub configuration. Hard-coded value returned will be a list with 2 entries.
//This method is going to be called by noteRepository.getNoteCount()
every { notesApiStub.fetchAllNotes() } returns listOf(note, note)
val allNotes = noteRepository.getNotes()
assertEquals(expected = 2, actual = allNotes.size)
}
view raw StubExample.kt hosted with ❤ by GitHub

A simple example of Stub utilization in a test

 

In the example above, we are using MockK to configure the Stub. In line 9, we demonstrate how this configuration is done when we set up exactly what the notesApiStub dependency needs to respond to when fetchAllNotes() is called:

every { notesApiStub.fetchAllNotes() } returns listOf(note, note)

An alternative to this configuration would be creating your own Stub manually, like the example below:

interface NoteApi {
suspend fun uploadNote(note: Note): Result
suspend fun fetchAllNotes(): List<Note>
}
class RealNoteApi : NoteApi {
override suspend fun uploadNote(note: Note): Result {
//Real implementation
}
override suspend fun fetchAllNotes(): List<Note> {
//Real implementation
}
}
class StubNoteApi(
val notes: List<Note> = listOf(),
val result: Result = Result.Success
) : NoteApi {
override suspend fun uploadNote(note: Note): Result {
return result
}
override suspend fun fetchAllNotes(): List<Note> {
return notes
}
}

Manually implemented Stub providing fixed answers

 

Independently on how your Stub is created, a pre-configured response will be returned immediately, avoiding the call to a real backend. Use Stubs when you need quick, deterministic, pre-configured answers for your test.

Fake

The Fake is a test double with a very similar purpose to Stub: providing simple and quick answers to a client who consumes it. The main difference is that the Fake uses a simple and lightweight working implementation under the hoods.

The Fake usually implements the same interface as the dependency it’s replacing. Their main characteristic is to have a lightweight functional implementation and be smarter than the Stubs, not only returning pre-defined and hard-coded responses configured previously. For this reason, the Fakes come closer to the system’s actual behavior compared to other test doubles.

@Test
fun `Retrieve all notes when requested`() {
val noteApiFake = FakeNoteApi() //Fake double implementing the same interface as the original
val noteRepository = NoteRepository(noteApiFake) //System under test
val note = //Dummy
noteApiFake.uploadNote(note) //Configuring the fake
noteApiFake.uploadNote(note) //Configuring the fake
//Fake with real and lightweight implementation is going to be used under the hoods
val allNotes = noteRepository.getNotes()
assertEquals(expected = 2, actual = allNotes.size)
}
view raw FakeExample.kt hosted with ❤ by GitHub

A simple example of Fake utilization in a test

Job Offers

Job Offers


    Developer (m/w/d) Backend/ Mobile

    Payback GmbH
    Cologne, Germany
    • Full Time
    apply now

    Senior Software Engineer – OS

    Peloton
    New York, USA
    • Full Time
    apply now

    Sr. Software Engineer, Android

    Box
    Redwood City, California; USA
    • Full Time
    apply now
Load more listings

OUR VIDEO RECOMMENDATION

, ,

Rock-Solid UI and Instrumentation Testing

We Android developers use the “flakiness” word more than the “stability” word while talking about Espresso and Instrumentation tests. We cannot truly trust flaky tests. Instrumentation testing will continue to be part of our lives…
Watch Video

Rock-Solid UI and Instrumentation Testing

Sinan Kozak
Android Staff Engineer
Delivery Hero

Rock-Solid UI and Instrumentation Testing

Sinan Kozak
Android Staff Engine ...
Delivery Hero

Rock-Solid UI and Instrumentation Testing

Sinan Kozak
Android Staff Engineer
Delivery Hero

Jobs

The fact that Fakes use the same contract as the real dependency help us to spot inconsistencies in the class design, also preventing internal details of the dependency from being leaked to the test.

Fakes can also be implemented manually as shown in the example below:

interface NoteApi {
suspend fun uploadNote(note: Note): Result
suspend fun fetchAllNotes(): List<Note>
}
class RealNoteApi: NoteApi {
override suspend fun uploadNote(note: Note): Result {
//Real impl
}
override suspend fun fetchAllNotes(): List<Note> {
//Real impl
}
}
class FakeNoteApi: NoteApi {
private val notes = mutableListOf<Note>()
override suspend fun uploadNote(note: Note): Result {
notes.add(note)
return Result.Success
}
override suspend fun fetchAllNotes(): List<Note> {
return notes
}
}

Manually implemented Fake, a little smarter implementation compared to the Stubs

 

A very famous Fake we can find in the Android world is Room’s in-memory database. Despite making use of an external tool to create it, it can still be considered a Fake as it has a lightweight functional implementation that replaces the real database.

val database = Room.inMemoryDatabaseBuilder(
    context, 
    MainDatabase::class.java
).build()

Fakes are widely used in tests that are at the I/O boundaries of the system, replacing dependencies that normally share states, such as databases and backends.

Use Fakes when you need quick, deterministic answers to your test and when you need to reproduce a more complex response that Stubs wouldn’t be able to handle.

Mock

The Mock is a double that aims to verify specific interactions with dependencies during the execution of a test. In other words, Mocks replace dependencies that want to be observed when a system is being tested.

Mocks don’t need to set up a hard-coded response like Stubs, they are used to observe interactions with dependencies.

@Test
fun `Track analytics event when creating new note`() {
val analyticsWrapperMock = //Mock
val noteAnalytics = NoteAnalytics(analyticsWrapperMock) //System under test
noteAnalytics.trackNewNoteEvent(NoteType.Supermarket)
//Verifies that specific call has happened
verify(exactly = 1) { analyticsWrapperMock.logEvent("NewNote", "SuperMarket") }
}
view raw MockExample.kt hosted with ❤ by GitHub

A simple example of Mock utilization in a test

 

In the example above, we use MockK to set up a Mock. In it, we need to verify that NoteAnalytics calls the AnalyticsWrapper.logEvent(String, String) method with specific parameters when finishing the NoteAnalytics.trackNewNoteEvent(Enum) call.

verify(exactly = 1) { 
    analyticsWrapperMock.logEvent("NewNote", "SuperMarket") 
}

The Mock purpose is to observe and verify an interaction with a dependency, while the Stub/Fake purpose is to simulate the dependency behavior and return predefined values.

Use Mocks when you need to check specific interactions with dependencies, especially if the behavior tested has no concrete return value to assert (method that returns Void or Unit). Avoid Mocking dependencies that have defined return values.

Also avoid using them directly on classes that you have no control over the implementation, such as external libraries, as the contract can change at any time and this can turn into a compilation error on your test in the future. In those cases, try to create Wrappers that encapsulate external dependencies that you have no control over and create Mocks for the Wrappers instead.

Spy

In my opinion, the Spy is the most confusing test double of all, as its definition varies between different authorship. Summarizing the original definition from Gerard Meszaros and putting it into my own words:

We can say that Spies have a similar purpose to Mocks, which would be to observe and verify interactions with dependencies during the execution of a test. The difference is that Spies use a functional implementation to operate and they can record more complex states which can be used for later verification or assertion.

A Spy can replace or extend the concrete implementation of a dependency by overriding some methods to record relevant information for the test verification. Regardless of whether the Spy is tool-generated or created manually, by definition, there will always be a working implementation under the hood.

@Test
fun `Track analytics event when creating new note`() {
val analyticsWrapperSpy = //Spy
val noteAnalytics = NoteAnalytics(analyticsWrapperSpy) //System under test
//AnalyticsWrapperSpy records the interaction with NoteAnalytics under the hoods
noteAnalytics.trackCreateNewNoteEvent(NoteType.Supermarket)
//Based on the its internal implementation, the spy returns the state of the dependency
val numberOfEvents = analyticsWrapperSpy.getNewNoteEventsRegistered()
assertEquals(expected = 1, actual = numberOfEvents)
}
view raw SpyExample.kt hosted with ❤ by GitHub

A simple example of Spy utilization in a test

 

In the example above, we have a Spy that was manually implemented. What will define if the test passed is the state that the spy holds. In this test, we want to verify that a specific analytics event was triggered a certain number of times, in other words, to verify that NoteAnalytics interaction with its dependency occurred in this way. The spy method responsible for that will be the analyticsWrapperSpy.getNewNoteEventsRegistered().

You will find different ways to define Spies during your studies, even Martin Fowler’s own definition is a little abstract to me. To ease this confusion a bit, my recommendation is that you focus that Spies observe interactions, have a lightweight implementation and hold state for future assertions.

Spies should be used when you want to verify that your dependency is in a specific state and you can’t do it simply with a Mock. Doing multiple verifications using a Mock on a single test can be an evidence that you are trying to observe a complex state. You can also use a Spy if you want to make your tests more readable in complicated scenarios (through custom methods that are going to be used in later assertions).

Dummies, Stubs, Fakes, Mocks, and Spies: a summary

After explaining the concepts, I think you can see why it’s no surprise that people have trouble understanding the test doubles, many nuances define each one of them.

To summarize the explanation, we can divide the five doubles into the following categories:

  • Those that don’t simulate behavior or observe interactions: Dummies.
  • Those that simulate behavior: Stubs and Fakes.
  • Those that observe interactions: Mocks and Spies.
  • Those that don’t have a functional implementation under the hood: Dummies, Stubs, and Mocks.
  • Those that have a functional implementation under the hoods: Fakes and Spies.

 

Summary of the 5 test doubles

 

All test doubles can be generated manually or by external tools. The ones that are configured by tools are more likely to couple implementation details of the dependency with the test itself. Using tools like Mockito or Mockk can make tests easy to configure initially, but it can also incur a higher maintenance cost in the future. Manually generated doubles tend to increase the test code base size in the beginning but they also make testing easier to maintain in the long run. Choose your trade-offs wisely 😄

Why do people call all test doubles Mocks?

It won’t be rare to see people calling Mocks when they really mean Stubs (or other test doubles). For the sake of simplicity, sometimes all doubles are commonly just called Mocks. That’s because many tools that help creating them have generalized this term.

See the example of MockWebServerMockK, and Mockito. Regardless of whether the test double is a Mock, a Stub, or a Fake, the name Mock is what usually comes up. One reason for this is that some doubles can take multiple roles, being Mocks and Stubs at the same time. With these cases in mind, it became preferable to call doubles that are more generic or have multiple roles of Mocks rather than creating another nomenclature for them.

Another reason for this generalization is due to the existence of inconsistent definitions that appear in books and articles on the internet:

“Classification between mocks, fakes, and stubs is highly inconsistent across the literature. [1] [2] [3] [4] [5] [6]

— Taken from Wikipedia, Mock Object

In my opinion, it’s not such a big deal to summarize all the test doubles to Mocks, as this can help with communication and reduce cognitive load on the daily basis. However, bear in mind that it’s very important for anyone who uses this generic term to know that it’s an abstraction and the other definitions exist. You will probably find debates around Mocks vs Fakes on the Internet, so it’s always good to know the correct definition of things.

The role of test doubles in the testing methodologies/schools

As stated at the beginning of this article, using test doubles in the right way is of utmost importance for the non-instrumented part of the test pyramid. How you use the test doubles will depend on which testing school you follow:

  • The sociable testing school. Also known as classic school, Detroit style, or Chicago Style.
  • Or the solitary testing school. Also known as mockist style or London style.

 

Comparison between sociable tests and solitary tests. Taken from: https://martinfowler.com/bliki/UnitTest.html

 

Maybe you have created tests using one of these schools and don’t even realize it. The main difference between them is the definition of “unit”, which consequently defines what a unit test is. Bellow a small summary of each testing school 👇

For the sociable testing school (Detroit Style), the unit is represented by the behavior, regardless of whether this behavior is composed of more than one class or not. In this case, a unit test can be executed with multiple real classes, as long as they represent a single behavior, the test is fast and can be parallelizable (don’t hold state). The recommendation here is to reduce the usage of test doubles and just use them to replace dependencies that share states, that are too slow, or that are out of your control, eg database, backend, third-party tools.

For the solitary testing school (London Style), the unit is represented by the class. In this case, all dependencies of the class being tested should be replaced with doubles, even if the real ones are quick and easy to configure. This style of testing intends to achieve a greater level of isolation and control.

Conclusion

That’s it! Now that you’ve learned the theory behind using test doubles, it’s time to move on to the second part of this series, where we’ll see a little bit more about the specifics of Android:

If you learned something from this article, leave your claps 👏 So I can know that it was helpful to you. Any feedback or questions, send them on Twitter or in the comments of this post. Thanks!

This article was originally published on proandroiddev.com on May 23, 2022

YOU MAY BE INTERESTED IN

YOU MAY BE INTERESTED IN

blog
Automation is a key point of Software Testing once it make possible to reproduce…
READ MORE
blog
Every good Android application should be well tested to minimize the risk of error…
READ MORE
blog

Running Instrumented Tests in a Gradle task

During the latest Google I/O, a lot of great new technologies were shown. The…
READ MORE
blog
Notifications are a very important part of Android apps, showing relevant information, informing when…
READ MORE

Leave a Reply

Your email address will not be published.

Fill out this field
Fill out this field
Please enter a valid email address.

Menu