In almost any kind of mobile project, we, mobile developers, find ourselves dealing with paginated data at some point. It’s a necessity if the list of data is too much to retrieve from server in one call. Therefore, our backend colleagues provide us with an endpoint that returns the list of data in pages, and expect us to know how to deal with it on the client side.

In this article, we will focus on how to fetch, cache, and display paginated data using the latest methods recommended by Android, as of June 2023. We will go through the following steps:

• Fetch the list of Pokemon data in pages from a public GraphQL API
• Cache the fetched data in local database using Room
• Use the latest Paging library components to handle pagination
• Display page items smartly (rendering only what is being visible) using LazyColumn

For the sample project, of which I will share the GitHub repository at the end of the article, we will utilize Hilt as our dependency injection library and use Clean Architecture (presentation → domain ← data). So, I will explain things starting from the data layer, then move to the domain layer, and conclude with the presentation layer.

Data Layer

This layer is where by far most of the stuff is going on about pagination and caching. So, if you can make it through this section, you will be mostly done with it.

Remote Data Source

As the remote data source, we will use a public GraphQL Pokemon API. As opposed to Retrofit, which is what we use to interact with REST APIs, we use Apollo’s Kotlin client for GraphQL APIs. It allows us to execute GraphQL queries and automatically generates Kotlin models from requests and responses.

First, we need to add the following lines to our module level build.gradle file:

plugins {
    // ...
    id "com.apollographql.apollo3" version "$apollo_version"
}

apollo {
    service("pokemon") {
        packageName.set("dev.thunderbolt.pokemonpager.data")
    }
}

dependencies {
    // ...
    implementation "com.apollographql.apollo3:apollo-runtime:$apollo_version"
}

Here in the apollo block, we set the configuration of the Apollo library. It provides many settings, all of which you can check through its documentation. For now, we just need to set the package name to dev.thunderbolt.pokemonpager.data so that the generated Kotlin files will be under the correct package, which is the data layer.

Then, we need to download the server’s schema, so the library will be able to generate models and we will be able to write queries with autocomplete. In order to download the schema, we use the following command provided by Apollo:

./gradlew :app:downloadApolloSchema --endpoint='https://graphql-pokeapi.graphcdn.app/graphql' --schema=app/src/main/graphql/schema.graphqls

This will download the server’s schema in the directory of app/src/main/graphql/schema.graphqls.

Now, it’s time to write our query in a file named pokemon.graphql which we created in the same folder as the schema file.

query PokemonList(
    $offset: Int!
    $limit: Int!
) {
    pokemons(
        offset: $offset,
        limit: $limit
    ) {
        nextOffset
        results {
            id
            name
            image
        }
    }
}

When we build our project, Apollo Kotlin will generate the models for this query by automatically running a Gradle task named generateApolloSources.

Going back to the Kotlin world, we will define our PokemonApi class to encapsulate all of our interactions with the GraphQL, as follows:

class PokemonApi {

    private val BASE_URL = "https://graphql-pokeapi.graphcdn.app/graphql"

    private val apolloClient = ApolloClient.Builder()
        .serverUrl(BASE_URL)
        .addHttpInterceptor(LoggingInterceptor())
        .build()

    suspend fun getPokemonList(offset: Int, limit: Int): PokemonListQuery.Pokemons? {
        val response = apolloClient.query(
            PokemonListQuery(
                offset = offset,
                limit = limit,
            )
        ).execute()
        // IF RESPONSE HAS ERRORS OR DATA IS NULL, THROW EXCEPTION
        if (response.hasErrors() || response.data == null) {
            throw ApolloException(response.errors.toString())
        }
        return response.data!!.pokemons
    }
}

Here, we initialize our Apollo Client instance with required configuration, and implement our function to execute the generated Kotlin version of the query which we wrote in the pokemon.graphql file. This function basically gets offset and limit parameters, executes the query, and if everything goes well, returns the response of the query, which is again auto-generated by Apollo.

Local Data Source/Storage

To store relational data locally and create an offline-first app, we will depend on Room, which is an Android persistence library written over SQLite.

First, we need to add the Room dependencies to our build.gradle file:

dependencies {
    // ...
    implementation "androidx.room:room-ktx:$room_version"
    kapt "androidx.room:room-compiler:$room_version"
    implementation "androidx.room:room-paging:$room_version"
}

Then, we will define two entity classes, one for storing Pokemon data in our database, and another to keep track of the number of the page to be fetched next.

@Entity("pokemon")
data class PokemonEntity(
    @PrimaryKey val id: Int,
    val name: String,
    val imageUrl: String,
)

@Entity("remote_key")
data class RemoteKeyEntity(
    @PrimaryKey val id: String,
    val nextOffset: Int,
)

In relation to these, we also need two DAO (Data Access Object) classes to define all of our database interactions in them.

@Dao
interface PokemonDao {
    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertAll(items: List<PokemonEntity>)

    @Query("SELECT * FROM pokemon")
    fun pagingSource(): PagingSource<Int, PokemonEntity>

    @Query("DELETE FROM pokemon")
    suspend fun clearAll()
}

@Dao
interface RemoteKeyDao {
    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insert(item: RemoteKeyEntity)

    @Query("SELECT * FROM remote_key WHERE id = :id")
    suspend fun getById(id: String): RemoteKeyEntity?

    @Query("DELETE FROM remote_key WHERE id = :id")
    suspend fun deleteById(id: String)
}

Here, the critical function we need to pay attention to is the pagingSource() one. Room is able to return list of data as PagingSource, so that our Pager object (which we will create later) will use it as the single source to generate a flow of PagingData.

Finally, we need to have a RoomDatabase class that creates tables for these entities in the local database and provides the DAOs to interact with the tables.

@Database(
    entities = [PokemonEntity::class, RemoteKeyEntity::class],
    version = 1,
)
abstract class PokemonDatabase : RoomDatabase() {
    abstract val pokemonDao: PokemonDao
    abstract val remoteKeyDao: RemoteKeyDao
}

Both of this PokemonDatabase and previously defined PokemonApi classes are instantiated and provided as singleton instances by the Hilt module of our data layer.

@Module
@InstallIn(SingletonComponent::class)
class DataModule {

    @Provides
    @Singleton
    fun providePokemonDatabase(@ApplicationContext context: Context): PokemonDatabase {
        return Room.databaseBuilder(
            context,
            PokemonDatabase::class.java,
            "pokemon.db",
        ).fallbackToDestructiveMigration().build()
    }

    @Provides
    @Singleton
    fun providePokemonApi(): PokemonApi {
        return PokemonApi()
    }

    // ...
}

Remote Mediator

Now, it’s time to implement our RemoteMediator class, which will be responsible for loading paginated data from the remote API into the local database whenever needed. It’s important to note that remote mediator does not provide data directly to the UI. If paginated data gets exhausted, Paging library triggers remote mediator’s load(…) method to fetch and store more data locally. Therefore, our local database can always remain the single source of truth.

class PokemonRemoteMediator @Inject constructor(
    private val pokemonDatabase: PokemonDatabase,
    private val pokemonApi: PokemonApi,
) : RemoteMediator<Int, PokemonEntity>() {

    private val REMOTE_KEY_ID = "pokemon"

    override suspend fun load(
        loadType: LoadType,
        state: PagingState<Int, PokemonEntity>,
    ): MediatorResult {
        return try {
            val offset = when (loadType) {
                LoadType.REFRESH -> 0
                LoadType.PREPEND -> return MediatorResult.Success(endOfPaginationReached = true)
                LoadType.APPEND -> {
                    // RETRIEVE NEXT OFFSET FROM DATABASE
                    val remoteKey = pokemonDatabase.remoteKeyDao.getById(REMOTE_KEY_ID)
                    if (remoteKey == null || remoteKey.nextOffset == 0) // END OF PAGINATION REACHED
                        return MediatorResult.Success(endOfPaginationReached = true)
                    remoteKey.nextOffset
                }
            }
            // MAKE API CALL
            val apiResponse = pokemonApi.getPokemonList(
                offset = offset,
                limit = state.config.pageSize,
            )
            val results = apiResponse?.results ?: emptyList()
            val nextOffset = apiResponse?.nextOffset ?: 0
            // SAVE RESULTS AND NEXT OFFSET TO DATABASE
            pokemonDatabase.withTransaction {
                if (loadType == LoadType.REFRESH) {
                    // IF REFRESHING, CLEAR DATABASE FIRST
                    pokemonDatabase.pokemonDao.clearAll()
                    pokemonDatabase.remoteKeyDao.deleteById(REMOTE_KEY_ID)
                }
                pokemonDatabase.pokemonDao.insertAll(
                    results.mapNotNull { it?.toPokemonEntity() }
                )
                pokemonDatabase.remoteKeyDao.insert(
                    RemoteKeyEntity(
                        id = REMOTE_KEY_ID,
                        nextOffset = nextOffset,
                    )
                )
            }
            // CHECK IF END OF PAGINATION REACHED
            MediatorResult.Success(endOfPaginationReached = results.size < state.config.pageSize)
        } catch (e: ApolloException) {
            MediatorResult.Error(e)
        }
    }
}

In the load(…) function, we first need to check which type of load we are dealing with. If LoadType is…

• REFRESH, it means that we are either at the initial load, or data somehow got invalidated and we need to fetch data from scratch. So, if this is the case, we set our offset value to “0” as we would like to fetch the first page of data.
• PREPEND, we need to fetch the page of data that comes before the current page. This is not needed in the scope of this example as we don’t want to fetch anything while scrolling to the top. Therefore, we simply return MediatorResult.Success(endOfPaginationReached = true) to indicate that data loading should not occur anymore.
• APPEND, we need to fetch the page of data that comes after the current page. In this case, we go and fetch the remote key object which should be already stored in our local database by the previous data load. If there is none or its nextOffset value is “0”, this means that there is no more data to be loaded and appended. This is how this API works, by the way. Your API might indicate the end of data differently, so you need to write your APPEND logic accordingly.

After we decided the correct value of offset, now it’s time to make API call using this offset and also the pageSize provided in the config. We will set the page size when we create our Pager object in the next step.

If the API call successfully returns a new page of data, we store the items and also the next offset value in our database using corresponding DAO functions. Here, we need to execute all database interactions within a transaction block, so that if any interaction fails, no change will be made to the database.

Finally, if everything goes well after database calls, we return a MediatorResult.Success checking if we’ve reached the end of pagination by comparing the number of items returned by the latest load with the page size we will define in the config.

Pager

Now, we’re going back to our data layer’s Hilt module again and we will create our Pager object. This object will bring all we’ve defined up until now together, and work as the constructor for the PagingData flow.

@Module
@InstallIn(SingletonComponent::class)
class DataModule {

    // ...

    @Provides
    @Singleton
    fun providePokemonPager(
        pokemonDatabase: PokemonDatabase,
        pokemonApi: PokemonApi,
    ): Pager<Int, PokemonEntity> {
        return Pager(
            config = PagingConfig(pageSize = 20),
            remoteMediator = PokemonRemoteMediator(
                pokemonDatabase = pokemonDatabase,
                pokemonApi = pokemonApi,
            ),
            pagingSourceFactory = {
                pokemonDatabase.pokemonDao.pagingSource()
            },
        )
    }
}

Here, we provide three things to the constructor of Pager. First, we set a PagingConfig with a desired page size, as I mentioned before. Second, we provide our remote mediator instance. And third, we set the paging source provided by Room as the single source of data for our Pager.

Repository

As we’ve done much of the work in our remote mediator, our repository implementation will be quite simple.

class PokemonRepositoryImpl @Inject constructor(
    private val pokemonPager: Pager<Int, PokemonEntity>
) : PokemonRepository {

    override fun getPokemonList(): Flow<PagingData<Pokemon>> {
        return pokemonPager.flow.map { pagingData ->
            pagingData.map { it.toPokemon() }
        }
    }
}

Using our Pager instance, we simply return its flow of PagingData to consumers. However before doing that, we also need to map PokemonEntity to domain’s Pokemon model. It’s because our domain layer does not know anything about data or presentation layers as the basis of Clean Architecture, so we should not carry our data models to domain layer.

Domain Layer

In this pure Kotlin layer, nothing much is going on actually. Here, we have our Pokemon model, our repository interface, and a simple use case class that interacts with this repository.

// REPOSITORY INTERFACE
interface PokemonRepository {
    fun getPokemonList(): Flow<PagingData<Pokemon>>
}

// USE CASE
class GetPokemonList @Inject constructor(
    private val pokemonRepository: PokemonRepository
) {
    operator fun invoke(): Flow<PagingData<Pokemon>> {
        return pokemonRepository.getPokemonList()
            .flowOn(Dispatchers.IO)
    }
}

// MODEL
data class Pokemon(
    val id: Int,
    val name: String,
    val imageUrl: String,
)

Here, one question you might have in your mind can be how to use PagingData in a pure Kotlin layer where we have no dependency on any Android component. It’s actually simple: There is a specific dependency provided by the Paging library for non-Android modules, so that we can access all simple Paging components like PagingSource, PagingData, Pager, and even RemoteMediator.

dependencies {
    // ...
    implementation "androidx.paging:paging-common:$paging_version"
}

Presentation Layer

After this quick coverage of the domain layer, let’s jump directly into the presentation layer, where the rest of the critical stuff is going on. But first, we need to add the following Paging dependencies to our build.gradle file:

dependencies {
    // ...
    implementation "androidx.paging:paging-runtime-ktx:$paging_version"
    implementation "androidx.paging:paging-compose:$paging_version"
}

Other than the runtime-ktx dependency, the compose dependency is also required here since it provides some intermediaries between our paging data flow and UI.

ViewModel

This is again one of the simple classes of this article, where we simply get the flow provided by the use case (to which it was already provided by the repository), and store it in a value.

@HiltViewModel
class PokemonListViewModel @Inject constructor(
    private val getPokemonList: GetPokemonList
) : ViewModel() {

    val pokemonPagingDataFlow: Flow<PagingData<Pokemon>> = getPokemonList()
        .cachedIn(viewModelScope)
}

We store the flow by calling cachedIn(viewModelScope) so that it’s kept active as long as the lifetime of our ViewModel. Besides that, it survives configuration changes like screen rotation, so that you get the same existing data rather than fetching it from scratch.

This method also keeps our cold flow as it is and does not turn it into a hot flow (StateFlow) as the stateIn(…) method would do. This means that if the flow is not being collected, no unnecessary code will be executed.

Screen (UI)

Now, we are at the final step of our pagination, where we will display our paging items in a LazyColumn. There are no RecylerViews or adapters anymore in Jetpack Compose. All these are now handled underneath and our large number of items are still laid out smartly, without causing any performance issues.

@Composable
fun PokemonListScreen(
    snackbarHostState: SnackbarHostState
) {
    val viewModel = hiltViewModel<PokemonListViewModel>()
    val pokemonPagingItems = viewModel.pokemonPagingDataFlow.collectAsLazyPagingItems()

    if (pokemonPagingItems.loadState.refresh is LoadState.Error) {
        LaunchedEffect(key1 = snackbarHostState) {
            snackbarHostState.showSnackbar(
                (pokemonPagingItems.loadState.refresh as LoadState.Error).error.message ?: ""
            )
        }
    }

    Box(modifier = Modifier.fillMaxSize()) {
        if (pokemonPagingItems.loadState.refresh is LoadState.Loading) {
            CircularProgressIndicator(modifier = Modifier.align(Alignment.Center))
        } else {
            LazyColumn(
                modifier = Modifier.fillMaxSize(),
                horizontalAlignment = Alignment.CenterHorizontally,
            ) {
                items(
                    count = pokemonPagingItems.itemCount,
                    key = pokemonPagingItems.itemKey { it.id },
                ) { index ->
                    val pokemon = pokemonPagingItems[index]
                    if (pokemon != null) {
                        PokemonItem(
                            pokemon,
                            modifier = Modifier.fillMaxWidth(),
                        )
                    }
                }
                item {
                    if (pokemonPagingItems.loadState.append is LoadState.Loading) {
                        CircularProgressIndicator(modifier = Modifier.padding(16.dp))
                    }
                }
            }
        }
    }
}

This article was previously published on proandroiddev.com