Imagine observing hundreds of rows of data where new records can get added, deleted, or updated anytime and showing this real-time data in a RecyclerView directly to the user. Fetching all the data is time and bandwidth-consuming. To reduce this burden you decided to paginate the data. But things just went even worse right? You just found out there is no out-of-the-box solution to do so. And to do these things efficiently is another challenge. Let’s analyze the problem a little deeper.
What’s the problem?
I’m developing a messaging module where any number of users can chat with each other in a single channel, similar to Slack channels or a WhatsApp group.
Why I need the real-time updates is because of the following requirements—
- New messages should appear instantly.
- Users can react and reply to other users’ messages.
- Users can delete the message as well, so it’s important to do so in case the user sent something accidentally.
The official documentation to paginate queries suggests using the get()method on the Query . Which means you get the latest data only once. Essentially you lose the real-time updates that are non-negotiable for my case and for you too if you are here!
Even the FirebasePagingAdapter from the FirebaseUI library doesn’t provide real-time updates because it uses Paging3 underneath and it is just not built for dynamic data.
Both of the ‘official’ ways are not fit for our case, plus we also lose the ability to get new messages. 😀
What should we do then?
Possible solution
After experimenting with many cases, I found no one-shot solution to remove all trade-offs. Sometimes I lost the ability to add new messages or lost updates in old messages. BUT there is one solution that may require the following precondition and some extra work.
You need at least one predictable field/column in your database.
By predictable, I mean whose value you should know while fetching data. For example, in my data, each message has a timestamp. And its column name is created_at .
Now, I can fetch the data in batches.
- New records — Add snapshot listener to get messages whose
created_atstarts after current system time. - Old N records — Another snapshot listener to get messages whose
created_atstarts before current system time. - Pagination — Same as 2, add more snapshot listener on-demand but based on
created_atof oldest record we got from query 2.
Here, N is the size of a single page or number of items you want to observe at once.
The query for this will look as follows —
| // To observe new records being added. | |
| firestoreDb.collection(collectionPath) | |
| .orderBy("created_at", Direction.ASCENDING) | |
| .startAfter(currentTimestamp) | |
| .addSnapshotListener(...) | |
| // To observe last N records | |
| firestoreDb.collection(collectionPath) | |
| .orderBy("created_at", Direction.ASCENDING) | |
| .endBefore(currentTimestamp) | |
| .limitToLast(10) // N = 10 | |
| .addSnapshotListener(...) | |
| // To observe last N to 2N records | |
| firestoreDb.collection(collectionPath) | |
| .orderBy("created_at", Direction.ASCENDING) | |
| .endBefore(lastRecordTimestamp) | |
| .limitToLast(10) // N = 10 | |
| .addSnapshotListener(...) |
Job Offers
OUR VIDEO RECOMMENDATION
Jobs
Get back a list of DocumentChange from these queries in your repository methods. We are using DocumentChange because it will contain only those records which are added/removed/updated. Implementation depends on you, but since I’m using Kotlin Flows here is how the repository method will look —
| fun getMessages(...): Flow<List<DocumentChange>> { | |
| return callbackFlow { | |
| ... | |
| .addSnapshotListener { snapshot, e -> | |
| if(e != null) close(e) | |
| trySend(snapshot.documentChanges) | |
| } | |
| awaitClose() | |
| } | |
| } |
Now we have our queries figured out. Now we need a data structure to store all the data coming through these queries. Remember this is a SnapsotListener so any callback can trigger anytime with new updates.
List? — You can use lists BUT doing operations especially in the case of deleting an item or updating an item is costly.
HashMap or LinkedHashMap(to maintain insertion order) will help us in this case.
| private val messageMap = linkedMapOf<String, MessageItem>() | |
| private fun updateMessagesMap(documents: List<DocumentChange>) { | |
| documents.forEach { documentChange -> | |
| val message = documentChange.document | |
| .toObject(MessageResponse::class.java) | |
| when (documentChange.type) { | |
| Type.ADDED -> { | |
| messageMap[message.id] = message | |
| } | |
| Type.MODIFIED -> { | |
| messageMap[message.id] = message | |
| } | |
| Type.REMOVED -> { | |
| messageMap.remove(message.id) | |
| } | |
| } | |
| } | |
| } |
That’s it! Convert it back to the list to expose it on the view layer and use it in your adapter. For UI efficiency make sure to use DiffUtil with your RecyclerView Adapter.
What do you think?
What do you think about this solution? I’d be happy to improve this solution so let me know your ideas and feedback. Also, feel free to reach out to me on Twitter in case of any questions.
1 Comment. Leave new
the idea seems to be very good, but I’m having difficulties to make this implementation. Well, I have the same. Do you have the complete code so I can analyze and see how I can implement this in my project?