Blog Infos
Author
Published
Topics
,
Published

One of the challenges of Mobile Development is dealing with third-party integrations and particularly the automated testing of such integrations. Push Messaging using FCM is one such case, where the impact of the integration breaking can be high, particularly during a campaign or hotfix release. Let’s have a look at how we can fully automate the testing to make sure our FCM integrations never break again.

 

Android instrumented test verifying push flow

 

Firebase Cloud Messaging became the de-facto standard for Push Messaging for Android apps. It is a cross-platform messaging solution that lets you reliably send messages at no cost.

FCM architecture — Source

The whole flow of the push notification is explained here.

  1. The app receives the push token from the FCM backend via the Firebase SDK.
  2. The app uploads that token to the app server.
  3. The app server triggers a push message using the uploaded token and a HTTP request to the FCM backend.
  4. The push message reaches the app and triggers the desired action.

 

The push messaging flow to test

 

The flow involves several moving parts and it can be difficult and time consuming to test it. More importantly, if any of the integrations breaks, it can take a long time before it is discovered and it can happen at the worst moment when you actually want to send push messages to your users.

If we want to end-to-end test the push messaging flow in a reliable way, we need to solve several problems to achieve it.

  1. Synchronize sending and receiving of the push message.
  2. Have a device which receives the FCM token.
  3. Use the token with the FCM backend to trigger the push message.
  4. Make the test wait for the push message.
  5. Verify that the app received the correct push message.

We can solve problem 1 by turning the app instrumented test into the app backend, communicating with the FCM backend. By doing it this way we avoid the extra moving piece of the app backend and we include the full control of the flow in our test.

Android instrumented test verifying push flow

Job Offers

Job Offers


    Distinguished Android Engineer

    Expedia Group
    Chicago, London, San Francisco, Austin, Gurgaon, Seattle or Remote
    • Full Time
    apply now

    Android Developer

    ClearScore
    London (Hybrid role)
    • Full Time
    apply now

    API Engineer

    American Express
    Phoenix, USA
    • Full Time
    apply now
Load more listings

OUR VIDEO RECOMMENDATION

, ,

Rock-Solid UI and Instrumentation Testing

We Android developers use the “flakiness” word more than the “stability” word while talking about Espresso and Instrumentation tests. We cannot truly trust flaky tests. Instrumentation testing will continue to be part of our lives…
Watch Video

Rock-Solid UI and Instrumentation Testing

Sinan Kozak
Android Staff Engineer
Delivery Hero

Rock-Solid UI and Instrumentation Testing

Sinan Kozak
Android Staff Engine ...
Delivery Hero

Rock-Solid UI and Instrumentation Testing

Sinan Kozak
Android Staff Engineer
Delivery Hero

Jobs

Problem 2 — the need for a real device token can be solved by using Firebase Test Lab physical devices through the Gcloud CLI and Gradle plugin to execute the tests. The instrumented test suite running on a real device will have a push token available right after the app is launched and we can use the token directly within our test code.

Then follows solving problem 3 — triggering the push through the FCM backend. All we need to do that is the push token and a simple RetrofitPushServerClient, which will implement the FCM HTTP protocol. We then need to pass the FCM authorization key into our test on the device, which can be done by passing an instrumented test argument to the device through the gcloud environment-variables parameter added by the Gradle plugin.

It can take some time for the FCM backend to send the push message and the app to receive it on the Test Lab device. We need to make sure that the test does not continue to assert before the message arrives to ensure the test is reliable.

Finally, the test looks like this:

class PushIntegrationTest {
lateinit var pushClient: PushServerClient
lateinit var thisDeviceToken: String
@get:Rule
val pushRule = PushAwaitRule()
@Before
fun setUp() {
pushClient = PushServerClient.create(apiKey())
val tokenTask = FirebaseMessaging.getInstance().token
thisDeviceToken = Tasks.await(tokenTask)
}
@Test
fun testPushIntegrationFromSettingsToAbout() {
DeepLinkLaunchTest.launchDeepLink("https://github.com/settings") // just launches screen to start from
pushRule.onViewAwaitPush() // makes Espresso wait for push to arrive
sendDeepLinkPush("https://github.com/about") // Triggers the push through FCM backend
onView(withText("by Josef Raska")).check(matches(isDisplayed())) // Asserts that the expected about screen was launched
}
}

Full version can be found here.

 

From the final assertion we can see that the whole flow passed as expected. This test verifies that the app launched on the settings screen before navigating into the about screen after the push message with this deep link arrives. (Video of the test)

We now have a fully automated test, which goes through the flow of receiving push messages from the FCM backend and verifies that the app handles them properly.

The action the push message triggers will be highly dependent on your app and so the way you verify they are handled correctly will vary. The key here is making sure the push message arrives and that the app reacts to it accordingly.

Flakiness is always a concern when it comes to tests that integrate with the live backend of a third-party like this. The referenced repo has implemented test metrics, which can give us a good answer to this question.

The tests ran reliably for a month on 2 different Firebase Test Lab devices with 260 consecutive successful runs.

Data on success rate of Push Messaging tests 260 of 260 passed

As such the answer is clear — yes, the test is reliable and is performing well with P90 time being 1.6s and less than 0.6s for the median time. This means that the duration is comparable to other Espresso UI tests and can nicely fit to any test suite.

P90 of the Push Messaging tests is 1.627 seconds

Test automation of your push messaging scenarios can save you many risks for a low cost and make your push messaging integration a pleasure to iterate on safely. Engineers can even use the tests to verify new push scenarios and achieve a form of TDD without tedious manual HTTP requests to the FCM backend.

How do you test your third-party integrations? Do you automate the tests? Let me know in the comments.

Happy coding!

YOU MAY BE INTERESTED IN

YOU MAY BE INTERESTED IN

blog
Automation is a key point of Software Testing once it make possible to reproduce…
READ MORE
blog
Every good Android application should be well tested to minimize the risk of error…
READ MORE
blog

Running Instrumented Tests in a Gradle task

During the latest Google I/O, a lot of great new technologies were shown. The…
READ MORE
blog
Notifications are a very important part of Android apps, showing relevant information, informing when…
READ MORE

Leave a Reply

Your email address will not be published.

Fill out this field
Fill out this field
Please enter a valid email address.

Menu