Today in this article we will talk about how to implement Pagination, Sorting and Custom Plugins using Ktor. This is an extension of my previous article on Developing REST APIs with Ktor. Please go through the article once before reading this one so you have a good background understanding of what Ktor is and how to develop basic REST APIs using Ktor. So without much delay, let’s get started.

Pagination

Though most of you already know, for my friends who are not aware of what pagination is, I will quickly explain it with the help of an example. When you scroll through an app having a list of items like Swiggy, Netflix, Disney Hotstar or any app which displays search results as a list of items, you may have noticed that initially, only 7–8 items of the list are visible and if you scroll more to the right (if the list is horizontal), a loading sign or shimmer effect appears before the next 7–8 items are loaded. This is called pagination.

In technical terms when we hit a paginated API, we specify the page number and number of items to be loaded in the list and accordingly only that many items are returned by the API. This is very helpful when our list is very large.

There are 2 types of pagination, Offset Pagination and Keyset Pagination. I have used Offset Pagination in this example. There are two main things in offset pagination, limit (the number of results) and offset (the number of records that need to be skipped). Let us quickly start with the code. As mentioned in the previous article, we have used Ktorm ORM in our project.

fun fetchPaginatedNotes(page: Int, size: Int): List<Note> {
    val limit: Int = size
    val pageSize: Int = size
    val skip: Int = (page - 1) * pageSize
    return db.from(NotesEntity).select().limit(offset = skip, limit = limit).map {
        val id = it[NotesEntity.id]
        val note = it[NotesEntity.note]
        Note(id ?: -1, note ?: "")
    }
}

As you can see, the function that returns the paginated list of items accepts the page and size parameters as query params.

page -: Page is the page number from which I want to start displaying the results. Let us say there are total 100 items in the list and size of each paginated list returned is 10, I want fetch starting from the 91st item of the list, my page number will be 10.

size -: Size is the number of items I want the list to return. Let’s us I want to see the list of items from 91 to 100, I will specify the size as 10.
The final URL will look like -:

https://www.url.com/endpoint?page=9&size=10

Now the question that might have come to your mind is what is the logic behind this and what is skip in the function written above?

Actually offset based pagination is nothing but a modified SQL query!! Yes, limit and offset both are a SQL concept. Under the hood the query for this would be something like -:

SELECT * FROM notes LIMIT 10 OFFSET 10;

Offset is the number of records that need to be skipped, we do accept only the pageSize and page number in query params as it makes more sense for the client. We can calculate OFFSET from pageSize and pageNumber easily by

offset = (pageNumber — 1) * pageSize

Let us take the same example, I want to fetch the list from 91st to the 100th item of the list, given the pageSize is 10 and page number is 9 as discussed before, offset will be

offset = (10 — 1) * 10 = 90

This means a total of 90 items will be skipped and items will be displayed from the 91st item, the limit is 10 so only 10 items will be shown. That is how offset pagination can be handled in Ktor 🙂

Sorting

All of you know what sorting is so without going into much detail, let us see how ascending and descending sorting can be gracefully handled in Ktor. Let us have a look /notes endpoint in our NotesRuter.

get("/notes") {
    when (val sortType = call.request.queryParameters["sort"]) {
        // Sort according to ascending or descending
        ASCENDING, DESCENDING -> {
            call.respond(
                HttpStatusCode.OK,
                service.fetchSortedNotes(isDescending = sortType == DESCENDING)
            )
        }
        // No sorting requested
        null -> {
            call.respond(HttpStatusCode.OK, service.fetchAllNotes())
        }
        // Invalid sorting type
        else -> {
            call.respond(
                HttpStatusCode.NotAcceptable,
                NoteResponse(success = false, data = "Please enter a valid sorting type(asc or desc)")
            )
        }
    }
}
object Constants {
    const val ASCENDING = "asc"
    const val DESCENDING = "desc"
}

Before sorting let me quickly tell null and else do here. It is a good practice to implement sorting in the original API endpoint instead of having a separate endpoint, as it is a basic requirement, so we want to handle it with and without sorting in our /notes endpoint.

1. null -: call.request.queryParameters[“sort”] here refers to the sort query in the endpoint. If this query is not there aka it is null, simply fetch and return all the items present in the db without any sorting.
2. else -: If the query value for sort is anything other than asc or desc, it is invalid and we do not want to return the correct response to the client in this case as it is an invalid request, so I have returned an error code 406 which means Not Acceptable.
3. asc, desc -: As the name suggests, this is used to send an ascending or descending list of items to the client. Based on the type of sorting requested by the client, the server computes (queries) the right sorting and returns the same to the client.

Let us take a look at our function which queries the same.

fun fetchSortedNotes(isDescending: Boolean? = false): List<Note> {
    val notes = NotesEntity
    val notesSortedByName = if (isDescending == true) {
        notes.note.desc()
    } else {
        notes.note.asc()
    }
    return db.from(NotesEntity).select().orderBy(notesSortedByName).map {
        val id = it[NotesEntity.id]
        val note = it[NotesEntity.note]
        Note(id ?: -1, note ?: "")
    }
}

The function is straightforward and self-explanatory. What this desc() and asc() extension function does internally is a query with order by note. The exact queries for this would be -:

SELECT * FROM notes ORDER BY note;

SELECT * FROM notes ORDER BY note desc;

Plugins

If you are a backend developer or have sometimes tried Node JS or other BE frameworks, you may have noticed that there are some actions to be performed upon receiving a request or sending a response. This action can be adding a validation or authorisation check, printing error logs in the console or uploading to Firebase logs, adding headers to each request or validating that the request was made with all the necessary headers. These actions you may want to implement for all or most of your API endpoints. So what do we do to write this logic like validation, and logging for every single endpoint? No, we use something called a Middleware in Node JS. In Ktor, it is named as Plugin. Earlier it was called as Ktor feature but was later changed to Plugin as the name feature is very generic.

There are many inbuilt plugins which Ktor offers such as CallLogging for monitoring, Authentication for normal authorisation, Compression, Cors and many more. We can also use some third-party plugins such as Koin which provides us dependencies at runtime, jwt for advanced authorisation etc.

Apart from inbuilt plugins and third-party plugins, the Ktor team makes it very easy for us to create our own custom plugins that fit our use case and use them in all or some of our routes easily.

Starting with v2.0.0, Ktor provides a new API for creating custom plugins. In general, this API doesn’t require an understanding of internal Ktor concepts, such as pipelines, phases, and so on. Instead, you have access to different stages of handling requests and responses using the onCall, onCallReceive, and onCallRespond handlers.

The process to create a plugin and what onCall, onCallReceive, and onCallRespond etc mean is very clearly defined in the documentation here so I would not go into detail about each of them, but let us quickly create and use two such small but useful plugins in our app.

import io.ktor.server.plugins.*
val ErrorLoggerPlugin = createApplicationPlugin(name = "ErrorLoggerPlugin") {
    on(CallFailed) { _, cause ->
        println("PLUGIN ERROR: API call failed because of ${cause.message}")
    }
}
val RequestLoggerPlugin = createApplicationPlugin(name = "RequestLoggerPlugin") {
    onCall { call ->
        call.request.origin.apply {
            println("Request URL: $scheme://$host:$port$uri")
        }
    }
}

As the name suggests, the first ErrorLoggerPlugin will be responsible for printing all the API call fails and the reason for the failure, in a real production app, think of it as monitoring the error logs on Firebase or other SDKs and giving alerts when there is a sudden rise in API failures.

The second RequestLoggerPlugin again as. the name suggests will log all the incoming requests from the client, another important metric for monitoring.

Installing plugins

We have seen how to create our own custom plugin, now let us see how to use the same in our app. We want to use the plugins either in 1. all endpoints or 2. selected endpoints. Ktor again makes it easy for us, for plugins that are application scoped simply use createApplicationPlugin while creating your plugin and if it is scoped to a route use createRouteScopedPlugin.

fun Application.module() {
    configureLogging()
}
fun Application.configureLogging() {
    install(ErrorLoggerPlugin)
    install(RequestLoggerPlugin)
}

I have installed both these plugins scoped to the application due to my use case, but you can install them to a particular route as well, for example in the notes route only I want to use RequestLoggerPlugin plugin, so I will remove it from Application.configureLogging() and add it in notesRoute().

fun Application.module() {
    configureLogging()
    configureRouting()
}
fun Application.configureLogging() {
    install(ErrorLoggerPlugin)
    install(RequestLoggerPlugin)
}
fun Application.configureRouting() {
    routing {
        notesRoutes()
        authenticateRoutes(secret, issuer, audience)
    }
}
fun Route.notes() {
    install(RequestLoggerPlugin)
    get("/notes") {
        
    }
}

and RequestLoggerPlugin to -:

val RequestLoggerPlugin = createRouteScopedPlugin(name = "RequestLoggerPlugin") {
    onCall { call ->
        call.request.origin.apply {
            println("Request URL: $scheme://$host:$port$uri")
        }
    }
}

This is how you can create and implement custom plugins for your application or specific routes.

For complete code, kindly refer to my GitHub repo ktor-api
There are many more interesting articles related to Ktor, Kotlin and Android coming soon 🙂 Kindly share your feedback in the comments and mention what should the next article be for Ktor. Thanks for spending your valuable time in reading this article, do share it with your friends!

This article was originally published on proandroiddev.com on October 30, 2022