In this article, we are going to create a KSP-based annotation processor that generates new code and files based on annotation usages. If you’d like to know more about code generation and make your development process more productive and fun, continue reading!
(Also, if you want a TL;DR version, you can visit this repository on GitHub for the completed code and library)
What Is KSP?
KSPÂ is an API that gives the ability to Kotlin developers to develop light-weight compiler plugins that analyze and generate code while keeping them away from unnecessary complexities of writing an actual compiler plugin.
Many libraries (including Room) are currently using KSP, instead of KAPT. You can find a list of few of them here.
How Does KSP Work?
In Kotlin developers have access to compiler APIs which they can use to develop compiler plugins. These plugins have access to almost all parts of the compilation process and can modify the inputs of the program.
Writing compiler plugins for simple code generation might get complex and that’s why KSP has been created. It is a compiler plugin under the hood which hides away the complexity (and the dependency to the compiler itself) of writing a compiler plugin by maintaining a simple API.
Bear in mind that unlike compiler plugins, KSP cannot modify the compiled code and treats them as read-only inputs.
Comparison to KAPT
KAPT is a Java-based Annotation Processor, which is tied to the JVM, while KSP is a code processor which depends only on Kotlin and can be more natural to Kotlin developers.
On another note, KAPT needs to have access to Java generated stubs to modify the program input based on annotations. This stage happens after all the symbols in the program (such as types) are completely resolved. This stage takes 30% of the compiler time and can be costly.
Since not all code generators need all of the symbol resolution (as it is the case in our example), KSP can be much faster since it happens in an earlier stage of the compiler, which not all (but enough) symbols are completely resolved.
Developing Your First KSP Project
Photo by Marc Reichelt on Unsplash
In this article, we are going to create ListGen, a KSP-based library that creates a list out of all the functions that have a specific annotation.
For example, you can add @Listed Annotation to your functions:
// can be anywhere (or in any module) in your project
@Listed("mainList")
fun mainModule() = 2
// can be anywhere (or in any module) in your project
@Listed("otherList")
fun helloModule() = "hello!"
// can be anywhere (or in any module) in your project
@Listed("mainList")
fun secondModule() = 3
And have a list of them generated like below:
// in build/.../GeneratedLists.kt val mainList = listOf(mainModule(), secondModule()) val otherList = listOf(helloModule())
So let’s get generating!
To get started with KSP, you only needs a few things. To get started, add the KSP API to your KSP-module dependencies:
| // to help the IDE recognize the KSP generated files, add the following to your app's build.gradle | |
| android { | |
| kotlin { | |
| sourceSets.debug { | |
| kotlin.srcDirs += 'build/generated/ksp/debug/kotlin' | |
| } | |
| sourceSets.release { | |
| kotlin.srcDirs += 'build/generated/ksp/release/kotlin' | |
| } | |
| } | |
| } | |
| // add the following the KSP module's build.gradle | |
| dependencies { | |
| implementation "com.google.devtools.ksp:symbol-processing-api:1.6.20-1.0.5" | |
| } |
Adding the necessary configuration to the build.gradle files
Creating The Annotation Class
Since we want to use KSP for annotation processing, we need to define our custom annotation(s) so we can use them later on. In this case, we want to have an annotation called @Listed that takes a name as an input and is defined on functions. Later we will create a list of all their usages, using the provided names.
| @Target(AnnotationTarget.FUNCTION) | |
| @Retention(AnnotationRetention.SOURCE) | |
| annotation class Listed( | |
| val name: String, | |
| ) |
Creating The Processor And Its Provider
In order to process files (and create more of them) yo need to create a SymbolProcessor and introduce it to KSP by using a SymbolProcessorProvider (which is basically a factory for the processor), since on the JVM, KSP uses a Service Loader to find the provider.
For now, we won’t process anything to complete our setup.
| import com.google.devtools.ksp.processing.Resolver | |
| import com.google.devtools.ksp.processing.SymbolProcessor | |
| import com.google.devtools.ksp.processing.SymbolProcessorEnvironment | |
| import com.google.devtools.ksp.symbol.KSAnnotated | |
| class ListedProcessor(private val environment: SymbolProcessorEnvironment): SymbolProcessor { | |
| override fun process(resolver: Resolver): List<KSAnnotated> { | |
| return emptyList() // we'll return an empty list for now, since we haven't processed anything. | |
| } | |
| } |
| import com.google.devtools.ksp.processing.SymbolProcessor | |
| import com.google.devtools.ksp.processing.SymbolProcessorEnvironment | |
| import com.google.devtools.ksp.processing.SymbolProcessorProvider | |
| class ListedProcessorProvider : SymbolProcessorProvider { | |
| override fun create(environment: SymbolProcessorEnvironment): SymbolProcessor { | |
| return ListedProcessor(environment) | |
| } | |
| } |
In order to introduce the provider to the JVM’s ServiceLoader, we need to create the following file.
| // Create the directories: yourmodule/src/main/resources/META-INF/services/ | |
| // Create a file named com.google.devtools.ksp.processing.SymbolProcessorProvider | |
| // Inside the file, add the following line (and nothing more): | |
| com.snaky.ksp.processor.provider.ListedProcessorProvider | |
| // For a reference, you can check out here: | |
| // https://github.com/adibfara/ListGen/blob/main/ksp/src/main/resources/META-INF/services/com.google.devtools.ksp.processing.SymbolProcessorProvider |
In order to use this generator (which currently does nothing) we need to declare a KSP dependency to its module, in our application module.
| // Inside your app's module, declare a dependency to your ksp module | |
| dependencies { | |
| implementation project(":ksp") // since you want to use your @Listed annotation | |
| ksp project(":ksp") // to make KSP work | |
| } |
We’re all set. Now we can start developing our processor.
Crafting The Processor
Photo by SwapnIl Dwivedi on Unsplash
Job Offers
OUR VIDEO RECOMMENDATION
Jobs
In order to create the processor, you need to ask yourself: “What is this processor supposed to generate?”. To answer that question, you can start by doing things by hand and start creating files and codes that are supposed to the generated. This way you get a lot of insight into what actually needs to be happening inside your processor.
In our case, we want to generate a file that looks like the following and has the proper imports:
val mainList = listOf(mainModule(), secondModule()) val otherList = listOf(helloModule())
In order to do this, we need to
- Add a proper package to the generated file
- Find mainModule, secondModule and helloModule
- Know where they are and add the proper imports for them above the file
- Find their name (mainList and otherList) from the annotation
- Generate a file containing the information above
- Make it efficient
Finding Annotations
KSP provides a resolver which you can use to find every symbol in the processed module that has a specific annotation.
Let’s generate a file containing a comment with the names of the functions that have the @Listed annotation.
| internal class ListedProcessor( | |
| private val environment: SymbolProcessorEnvironment, | |
| ) : SymbolProcessor { | |
| private fun Resolver.findAnnotations( | |
| kClass: KClass<*>, | |
| ) = getSymbolsWithAnnotation( | |
| kClass.qualifiedName.toString()) | |
| .filterIsInstance<KSFunctionDeclaration>().filter { | |
| it.parameters.isEmpty() | |
| } | |
| override fun process(resolver: Resolver): List<KSAnnotated> { | |
| val listedFunctions: Sequence<KSFunctionDeclaration> = | |
| resolver.findAnnotations(Listed::class) | |
| if(!listedFunctions.iterator().hasNext()) return emptyList() | |
| val functionNames = listedFunctions.map{ it.simpleName.asString() } | |
| val sourceFiles = listedFunctions.mapNotNull { it.containingFile } | |
| val fileText = buildString { | |
| append("// ") | |
| append(functionNames.joinToString(", ")) | |
| } | |
| val file = environment.codeGenerator.createNewFile( | |
| Dependencies( | |
| false, | |
| *sourceFiles.toList().toTypedArray(), | |
| ), | |
| "your.generated.file.package", | |
| "GeneratedLists" | |
| ) | |
| file.write(fileText.toByteArray()) | |
| return (listedFunctions).filterNot { it.validate() }.toList() | |
| } | |
| } |
Generated file 🎉
The process function needs to return the symbols that were not valid. KSP uses this information for its multiple round processing.
As you can see, creating a file and filling the information is a piece of cake. All that is remaining is to add imports, read annotations’ names and add the functions and we are good to go.
| internal class ListedProcessor( | |
| private val environment: SymbolProcessorEnvironment, | |
| ) : SymbolProcessor { | |
| private fun Resolver.findAnnotations( | |
| kClass: KClass<*>, | |
| ) = getSymbolsWithAnnotation( | |
| kClass.qualifiedName.toString()) | |
| .filterIsInstance<KSFunctionDeclaration>().filter { | |
| it.parameters.isEmpty() | |
| } | |
| override fun process(resolver: Resolver): List<KSAnnotated> { | |
| val listedFunctions: Sequence<KSFunctionDeclaration> = | |
| resolver.findAnnotations(Listed::class) | |
| if(!listedFunctions.iterator().hasNext()) return emptyList() | |
| // gathering the required imports | |
| val imports = listedFunctions.mapNotNull { it.qualifiedName?.asString() }.toSet() | |
| // group functions based on their given list-name | |
| val lists = listedFunctions.groupBy { it.annotations.first { it.shortName.asString() == "Listed" }.arguments.first().value.toString() } | |
| val sourceFiles = listedFunctions.mapNotNull { it.containingFile } | |
| val fileText = buildString { | |
| append("package your.desired.packagename") | |
| newLine() | |
| newLine() | |
| imports.forEach { | |
| append("import $it") | |
| newLine() | |
| } | |
| newLine() | |
| lists.forEach { (listName, functions) -> | |
| val functionNames = functions.map { it.simpleName.asString() + "()" }.joinToString(", ") | |
| append("val $listName = listOf($functionNames)") | |
| newLine() | |
| } | |
| } | |
| createFileWithText(sourceFiles, fileText) | |
| return (listedFunctions).filterNot { it.validate() }.toList() | |
| } | |
| private fun createFileWithText( | |
| sourceFiles: Sequence<KSFile>, | |
| fileText: String, | |
| ) { | |
| val file = environment.codeGenerator.createNewFile( | |
| Dependencies( | |
| false, | |
| *sourceFiles.toList().toTypedArray(), | |
| ), | |
| "your.generated.file.package", | |
| "GeneratedLists" | |
| ) | |
| file.write(fileText.toByteArray()) | |
| } | |
| private fun StringBuilder.newLine(count: Int = 1) { | |
| repeat(count){ | |
| append("\n") | |
| } | |
| } | |
| } |
Generated lists 🔥
We’re all done. You can continue to clean up the file and add more features if necessary. You can also add unit tests (you can see my unit tests here).
Note: In this example we used a simple StringBuilder to create the contents of the file. For more advanced usages you can use libraries like KotlinPoet to write the contents of the generated files more efficiently.
Using the visitor pattern
For the basic example above, we have filtered the symbols on the function types and iterated over them, since that was the only symbol that we needed. If you need to support more symbols (like classes), you can also use a KSVisitor and pass it to your symbols’s accept function, which will call the proper function on your visitor (e.g. visitFunctionDeclaration is called if your symbol is a function). You can see it in action here.
Performance
To make the processor super fast, a few things need to be considered.
Photo by Wesley Tingey on Unsplash
Minimizing the amount of processed files
Operate on as few as possible files to get the desired results. Notice the above code that operates only on the functions that have our specific annotations. All other symbols are ignored.
Informing the compiler
KSP is smart and has an incremental compilation strategy. We don’t want our generated file(s) to change, unless:
- A previously existing file that contained our annotations has been changed/deleted.
- A new file containing our annotation has been added
All other files should be ignored.
In order to achieve this, we have given our dependencies to the createNewFile function, which informs the processor about the files that we have considered for creating this file.
Avoiding expensive functions
Some functions in KSP APIs are expensive (as they are noted in their documentation). One example is resolve, which resolves a TypeReference to a Type. These functions are expensive and should be used only if knowledge about them is absolutely necessary. Try to look out for these functions (and their documentation) and use them sparingly.
Conclusion
KSP is a powerful tool that helps developers to write light-weight compiler plugins and annotation processors while maintaining a Kotlin-friendly API.
Using KSP can help developers and tech leaders to create libraries that help them achieve more productivity by generating files and boilerplate code.
I hope you enjoyed this article and it has helped you learn how to create your first KSP library.
If you enjoyed this article, be sure to check out my other articles:
đź”’ Synchronization, Thread-Safety and Locking Techniques in Java and Kotlin
đź’Ą The Story of My First A-ha Moment With Jetpack Compose | by Adib Faramarzi | ProAndroidDev
Kotlin Contracts: Make Great Deals With The Compiler! 🤜🤛
Follow me on Medium If you’re interested in more informative and in-depth articles about Java, Kotlin and Android. You can also ping me on Twitter @TheSNAKY 🍻.
This article was originally published on proandroiddev.com on May 12, 2022
