Blog Infos
Author
Published
Topics
,
Published

One of the challenges of Mobile Development is dealing with third-party integrations and particularly the automated testing of such integrations. Push Messaging using FCM is one such case, where the impact of the integration breaking can be high, particularly during a campaign or hotfix release. Let’s have a look at how we can fully automate the testing to make sure our FCM integrations never break again.

 

Android instrumented test verifying push flow

 

Firebase Cloud Messaging became the de-facto standard for Push Messaging for Android apps. It is a cross-platform messaging solution that lets you reliably send messages at no cost.

FCM architecture — Source

The whole flow of the push notification is explained here.

  1. The app receives the push token from the FCM backend via the Firebase SDK.
  2. The app uploads that token to the app server.
  3. The app server triggers a push message using the uploaded token and a HTTP request to the FCM backend.
  4. The push message reaches the app and triggers the desired action.

 

The push messaging flow to test

 

The flow involves several moving parts and it can be difficult and time consuming to test it. More importantly, if any of the integrations breaks, it can take a long time before it is discovered and it can happen at the worst moment when you actually want to send push messages to your users.

If we want to end-to-end test the push messaging flow in a reliable way, we need to solve several problems to achieve it.

  1. Synchronize sending and receiving of the push message.
  2. Have a device which receives the FCM token.
  3. Use the token with the FCM backend to trigger the push message.
  4. Make the test wait for the push message.
  5. Verify that the app received the correct push message.

We can solve problem 1 by turning the app instrumented test into the app backend, communicating with the FCM backend. By doing it this way we avoid the extra moving piece of the app backend and we include the full control of the flow in our test.

Android instrumented test verifying push flow

Job Offers

Job Offers

There are currently no vacancies.

OUR VIDEO RECOMMENDATION

,

Put Your Tests on a Diet:Testing the Behavior and Not the Implementation

How do you write tests? How much time do you spend writing tests? And how much time do you spend fixing them when refactoring?
Watch Video

Put Your Tests on a Diet:Testing the Behavior and Not the Implementation

Stelios Frantzeskakis
Staff Engineer
PSS

Put Your Tests on a Diet:Testing the Behavior and Not the Implementation

Stelios Frantzeska ...
Staff Engineer
PSS

Put Your Tests on a Diet:Testing the Behavior and Not the Implementation

Stelios Frantzes ...
Staff Engineer
PSS

Jobs

Problem 2 — the need for a real device token can be solved by using Firebase Test Lab physical devices through the Gcloud CLI and Gradle plugin to execute the tests. The instrumented test suite running on a real device will have a push token available right after the app is launched and we can use the token directly within our test code.

Then follows solving problem 3 — triggering the push through the FCM backend. All we need to do that is the push token and a simple RetrofitPushServerClient, which will implement the FCM HTTP protocol. We then need to pass the FCM authorization key into our test on the device, which can be done by passing an instrumented test argument to the device through the gcloud environment-variables parameter added by the Gradle plugin.

It can take some time for the FCM backend to send the push message and the app to receive it on the Test Lab device. We need to make sure that the test does not continue to assert before the message arrives to ensure the test is reliable.

Finally, the test looks like this:

class PushIntegrationTest {
lateinit var pushClient: PushServerClient
lateinit var thisDeviceToken: String
@get:Rule
val pushRule = PushAwaitRule()
@Before
fun setUp() {
pushClient = PushServerClient.create(apiKey())
val tokenTask = FirebaseMessaging.getInstance().token
thisDeviceToken = Tasks.await(tokenTask)
}
@Test
fun testPushIntegrationFromSettingsToAbout() {
DeepLinkLaunchTest.launchDeepLink("https://github.com/settings") // just launches screen to start from
pushRule.onViewAwaitPush() // makes Espresso wait for push to arrive
sendDeepLinkPush("https://github.com/about") // Triggers the push through FCM backend
onView(withText("by Josef Raska")).check(matches(isDisplayed())) // Asserts that the expected about screen was launched
}
}

Full version can be found here.

 

From the final assertion we can see that the whole flow passed as expected. This test verifies that the app launched on the settings screen before navigating into the about screen after the push message with this deep link arrives. (Video of the test)

We now have a fully automated test, which goes through the flow of receiving push messages from the FCM backend and verifies that the app handles them properly.

The action the push message triggers will be highly dependent on your app and so the way you verify they are handled correctly will vary. The key here is making sure the push message arrives and that the app reacts to it accordingly.

Flakiness is always a concern when it comes to tests that integrate with the live backend of a third-party like this. The referenced repo has implemented test metrics, which can give us a good answer to this question.

The tests ran reliably for a month on 2 different Firebase Test Lab devices with 260 consecutive successful runs.

Data on success rate of Push Messaging tests 260 of 260 passed

As such the answer is clear — yes, the test is reliable and is performing well with P90 time being 1.6s and less than 0.6s for the median time. This means that the duration is comparable to other Espresso UI tests and can nicely fit to any test suite.

P90 of the Push Messaging tests is 1.627 seconds

Test automation of your push messaging scenarios can save you many risks for a low cost and make your push messaging integration a pleasure to iterate on safely. Engineers can even use the tests to verify new push scenarios and achieve a form of TDD without tedious manual HTTP requests to the FCM backend.

How do you test your third-party integrations? Do you automate the tests? Let me know in the comments.

Happy coding!

YOU MAY BE INTERESTED IN

YOU MAY BE INTERESTED IN

blog
Automation is a key point of Software Testing once it make possible to reproduce…
READ MORE
blog
Every good Android application should be well tested to minimize the risk of error…
READ MORE
blog
In this article we’ll go through how to own a legacy code that is…
READ MORE
blog

Running Instrumented Tests in a Gradle task

During the latest Google I/O, a lot of great new technologies were shown. The…
READ MORE

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.

Menu