Create an Android feature updated in real time by the server in a scalable way! Clean Architecture + SSE client + Kotlin Flow

Introduction

The Android community is very well documented and currently has thousands of libraries and architectural options for us to develop our applications. Obviously that the lack of options would be a problem but the abundance of options can also be configured into a problem, as for the hurry of delivering a final product the developer can just focus on a immediate result using libraries that gives a solution to his needs but might not be the best option for a performance point of view or for any other reason, like generating future bugs or financial costs to the company.

Given the amount of technology options we have, stating that our solution is the best possible would not be accurate, so instead of it the aim of this study is to propose a recommendable set of well known technologies and architectures to achieve an Android feature that is updated in real time by a server.

Summary

In order to structure this feature, we will try to explore the viable options we have for:

1. Client-Server Communication: Short Polling, Long Polling, WebSocket or SSE?
2. System Architecture: Why Clean Architecture? How to apply it?
3. Data streaming: Which implementation of Flow? Why not LiveData?

1. Client-Server Communication

There are many options to implement as our communication system, such as Short Polling, Long Polling, WebSocket & SSE. But let’s just consider SSE & WebSocket for this explanation as either Short Polling & Long Polling would be a waste of resources as both needs multiple requests to be made in a loop instead of having a single dedicated connection.

Server Sent Event (SSE) is a push technology that establishes a unidirectional HTTP connection by sending a GET request informing the intention on opening an event-stream in the header, which allows the server to send multiple events directly to the client over the same connection. It means that once a connection is established the client will be able to be continually updated without the need to make multiple requests.

Another technology that would also work for the same purpose is WebSocket, which is a communication protocol, other than HTTP, that provides an open connection between Server and Client. The main difference here is that the connection provided by the WebSocket is bidirectional, known as a full-duplex connection, which allows both Server and Client to send updates to each other.

Depending on the purpose of the feature that is going to be implemented one of those two would be preferable, for example:

• Chat or online Gaming: For these features we would need a full-duplex connection as we would be sending messages to the server and also getting the messages that were sent by other client(s), so it is a scenario in which a WebSocket would be the correct choice.
• News Feed or Order Status Tracking: For these features we would need a unidirectional connection, as the client would not need to send any information to the Server but just observe what is being sent in order to be updated.

For this implementation, we’ll be working with a SSE client, as we mean to have a page that is updated in real time by the server and doesn’t need to send any data.

How to implement it on Android?

There are some different client libraries created by the community, but what was used in the sample project for this study is the okhttp-sse library, which is very well maintained and documented.

2. System Architecture

So far we already know that we will have an open connection using a SSEClient to get multiple events from the Server, but now comes maybe the most important part: How to properly structure the stream from the client to the View? For that purpose we have the Clean Architecture.

I don’t think there’s a need to explain this architecture introduced by Robert C. Martin, the Uncle Bob, in 2012, as it’s very widely used by the community, but the main idea of this architecture is that a software should not only meet UI and UX requirements, but also should be easy to understand, flexible, testable and maintainable, and it works by dividing the system into layers with different levels of access.

Currently for Android a common structure for the Clean Architecture is as shown below:

In the image above we can see the following layers:

• Data Layer:
  — Responsible for providing external information, mapping it into a model structure that the app can understand.
  — Have access to the Domain layer.
  — Can be a pure Kotlin module.
• Domain Layer:
  — Responsible for providing the app’s business logic, with all abstractions that the different layers in the app use to communicate to each other.
  — Does not have access to any other layer, it just gets a Repository instance from the dependency injection tree that will be provided in the Data layer.
  — Can be a pure Kotlin module.
• Presentation Layer:
  — Responsible for the UI features.
  — Have access to the Domain layer.
  — The only one that needs to be an Android module.

The importance of having pure Kotlin modules is to promote platform independence (which makes it possible to create multi platform libraries with modules), faster build times and make it easier to test between other aspects.

3. Data Streaming

Well, as we’re going to open a connection to a server we’ll need to work with multithreading, and the most recommended way to do it currently is by using Coroutines. I believe that most of us are already used to Coroutines, but for those who aren’t it has a close concept to a light-weight thread, but it’s not.

Light-weight threads can share resources with other threads while Coroutines can actually be executed in a thread, suspended and resumed in another one, and that’s why the functions are actually named suspend function, because it can be suspended to be resumed at another moment.

But we don’t only want to open a connection, we want to keep observing it in order to be updated about the upcoming events in our downstream connection to the Server, and for that functionality we have to find a way to stream the data from the Client to the user Interface, which can be done by using a Callback strategy, a RX implementation or Kotlin Flow.

To avoid a longer discussion deep diving into all the options, let’s start excluding Callbacks and RX. Callbacks can be messy in large systems, leading to something known as the “Callback Hell”, which is when you start losing track of where your callbacks are coming from in your system. As for RX, it’s a great Reactive tool, but since we’re using Kotlin, it’s better to work with a Kotlin-specific library like Flow. Also, Flow covers all the Reactive functionalities of RX and has an easier learning curve. So, that’s why for this implementation we choose to use Kotlin Flow.

Unlike suspend functions that return only a single value to the function caller, a Flow can emit multiple values. However, for each new collector that is set a new instance of Flow is created, what is called the cold flow property.

For a scenario with a data downstream to emit real-time events it’d be interesting to have multiple collectors to the same Flow instance, what is called the hot flow property. Also, it will be necessary to emit values into different points of the client code, so there’s need for a thread-safe way to emit the new values (like the MutableLiveData, with the postValue() function). Summing up the requisites so far are for a hot flow that can emit values in a thread-safe way. Luckily for this we have the SharedFlow, which is an implementation of Flow.

The last requisite we have, is that our Flow actually will just emit a value in case it’s not equal to the one we already have and for that purpose we have the StateFlow, which is an implementation of SharedFlow.

So, we finally have a Multithreading tool that covers all our requisites, which are:

• It’s a stream that can emit multiple values.
• It can have multiple collectors.
• It just emits non repeated values.
• It has a thread-safe way to emit the new values.
• It is a library written in Kotlin.
• It is not an Android library, so the layers can still be pure Kotlin.

We can see that it fits like a glove to our purpose and will be a great tool in order to send our events downstream until the View layer.

Wouldn’t a LiveData with suspend functions do the job?

The easy answer to this question is yes, however the complete answer is, we would not be able to achieve all the requisites we just described before. By doing the same list we did on the last topic, in case we tried to do this implementation with LiveDatas it would be like:

• It’s a stream that can emit (post) multiple values.
• It can have multiple collectors (observers).
• It emits repeated values.
  (Issue that can be fixed by implementation)
• It has a thread-safe way to emit the new values.
• It is a library written in Java.
• It is an Android library.

As we can see, it has some limitations that Flow doesn’t, but the biggest problem here is that this is an Android library. It’s indeed possible to add this library to pure Kotlin modules, but it would not be the most suitable option as LiveData is a lifecycle aware tool made to work with Android Components so it’d not be correct to say that our data & domain layers are platform independent anymore.

Conclusion

After the whole proccess, looks like we found great answers to the 3 main system needs, which are:

• ​​Client-Server Communication: SSE
• Architecture: Clean Arch
• Data Streaming: State Flow

So now, let’s check the visual sequence diagram of the system and the Sample app made to support this study.

Sequence Diagram

Sample Project

To make it easier to understand the system I created a sample project with the exact same structure proposed in this study and it’s publicly available on Github (link).

In the image above each line represents a new Event, starting with some construction states and then moving to real events received from a test SSE Server, where eventCount contains how many events we had so far and OnEvent contains the received event data (which is a random word from the consumed test Server).

More Details

This article describes a study that was presented as a talk on droidcon Berlin 2023. The presentation can be checked at this link.

Thanks for reading and I hope it was useful 🙂 let me know if you have points to add and feel free to contact me through here.

References

SSE & WebSocket-> https://web.dev/eventsource-basics/#server-sent-events-vs-websockets

Lightweight threads -> https://www.techopedia.com/definition/24784/lightweight-thread#:~:text=A%20lightweight%20thread%20is%20a,context%20switching%20time%20during%20execution.

Coroutines -> https://kotlinlang.org/docs/coroutines-basics.html

Flow -> https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-flow/

Shared Flow -> https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-shared-flow/

State Flow -> https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-state-flow/

Clean Architecture -> http://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html

This article was previously published on proandroiddev.com