Image by StockSnap from Pixabay

As a software engineer, you’ll inevitably encounter compatibility challenges at some point in your career. These may manifest as source compatibility, binary compatibility, or backward compatibility issues. In this article, we’ll delve into these concepts, exploring what they entail and how best to navigate them.

It’s important to note that the content of this article is tailored to Kotlin users, with a focus on Gradle as the primary build tool.

You can reproduce the issues and problems faced in this article using the provided sample project.

Source compatibility

Source compatibility is the most straightforward form of compatibility, it refers to the ability of the new version of your code to seamlessly integrate without necessitating any modifications in the client’s codebase.

In other words, the updated version of your code can be recompiled with the latest version of your library without extra changes.

Let’s take the following example.

Suppose you intend to modify this interface to retrieve all vegan dishes. One approach is to introduce a specification argument like this:

Predictably, this adjustment would trigger a compilation error, as indicated below:

e: No value passed for parameter 'specification'

The reason behind this error lies in the fact that the alteration made to the code isn’t source-compatible.

An effortless way to ensure source compatibility is to provide a default value for the new parameter, as demonstrated below:

With this modification, the code becomes source-compatible with the previous version, allowing it to compile seamlessly without requiring any modifications from the client’s end.

Binary compatibility

In our earlier example, we discussed source compatibility and how to maintain it when we apply changes to our code. But is that all we need? The answer depends on a few things.

There are two situations to consider:

1. We depend directly on the sources: If we directly rely on the source code of a library, we usually just need source compatibility. This happens when we’re working on internal projects or using a mono-repo for our code, a general rule of thumb to identify if you are in this case is when you are using dependencies like implementation(project(":lib")).
2. We depend on published libraries: When we depend on a published version of a library, like when we add a line such as implementation("com.msignoretto:lib:1.0.0") in the build.gradle.kts file, source compatibility is not enough and we need to think about binary compatibility as well. This happens when we use libraries made by others or when we publish libraries that others or ourselves use.

Let’s deep dive into the second case.

Binary compatibility is stricter than source compatibility. It not only means that the client’s code doesn’t need to change, but it also doesn’t need to be recompiled. It ensures that the new version of our code can work with the old version of the client’s code without the client having to upgrade explicitly.

Let’s explain this with an example:

Imagine we initially publish our library without the dish specification as a jar file called 1.0.0 with the name com.msignoretto:lib:1.0.0. Now, if we shift to using the published version, our dependency in the client’s code changes from implementation(project(":lib")) to implementation("com.msignoretto:lib:1.0.0").

This change is significant. Now, we depend on the published version of the library’s binary, not directly on its source code.

Considering this shift, let’s see how it affects our code.

Suppose we make the changes related to the dish specification, as described earlier, and publish this as a new version, say 1.1.0.

In this case, we might face two situations:

1. Your dependency tree contains only the new version
2. Your dependency tree contains both the old and the new version

Let’s explore both scenarios in detail.

Scenario 1

In this scenario, the client doesn’t need to make any changes to their code. They can simply update the dependency to the new version of the library, resulting in the same outcome as with the source-compatible code.

To visualize the dependency tree, you can execute the following command:

./gradlew :app:dependencies

Replace app with the name of your module.

For the sample project, the dependency tree will appear as follows:

...
+--- project :moduleA
|    +--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10 (*)
|    \--- com.msignoretto:lib:1.1.0
|         \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10 (*)
\--- project :moduleB
     +--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10 (*)
     \--- com.msignoretto:lib:1.1.0 (*)

It’s important to note that the output of the dependencies command doesn’t contain the old version of the library at all.

Scenario 2

The second scenario is indeed more complex. Let’s consider what happens when we update the client to use the new version of the library, while still having a dependency compiled against the old version of the library.

To simulate this scenario, we modify moduleB in the sample project to use the old version, 1.0.0.

If you want to reproduce this exact scenario checkout the binary-incompatible branch of the sample project in here.

Now, your dependency tree displays both the old and the new versions of the library:

+--- project :moduleA
|    +--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10 (*)
|    \--- com.msignoretto:lib:1.1.0
|         \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10 (*)
\--- project :moduleB
     +--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10 (*)
     \--- com.msignoretto:lib:1.0.0 -> 1.1.0 (*)

In this scenario, we notice a difference compared to Scenario 1:

• Both the old and new versions are present in the dependency tree.
• mobuleB 1.0.0 -> 1.1.0 is a new addition that wasn’t present before.

Let’s put that aside for now and come back to it later.

From the output above, you can observe that the sample project is composed of 2 modules: moduleA and moduleB. Both modules contain identical code and both depend on the lib library but with different versions. While moduleA depends on version 1.1.0, moduleB relies on version 1.0.0.

Both ModuleA and ModuleB contain the following code:

Let’s try to execute the following function:

Upon running that function, we encountered the following error:

Exception in thread "main" java.lang.NoSuchMethodError: 'java.util.List com.msignoretto.binarycompatibility.lib.DishesRepository.getDishes()'
 at com.msignoretto.binarycompatibility.moduleA.ModuleA.run(ModuleA.kt:8)
 at com.msignoretto.binarycompatibility.app.MainKt.main(main.kt:10)
 at com.msignoretto.binarycompatibility.app.MainKt.main(main.kt)

This error serves as a clear indication that your code is not binary-compatible.

Upon decompiling the bytecode for moduleA, we encounter the following bytecode for invoking the getDishes() method:

INVOKESTATIC com/msignoretto/binarycompatibility/lib/DishesRepository$DefaultImpls.getDishes$default (Lcom/msignoretto/binarycompatibility/lib/DishesRepository;Lcom/msignoretto/binarycompatibility/lib/DishSpecification;ILjava/lang/Object;)Ljava/util/List;

Meanwhile, for moduleB, the bytecode for the invocation is as follows:

INVOKEINTERFACE com/msignoretto/binarycompatibility/lib/DishesRepository.getDishes ()Ljava/util/List; (itf)

For improved readability, let’s decompile the bytecode of both modules into Java, resulting in the following code.

ModuleA:

ModuleB:

If you diff those files you will get the following output:

This comparison highlights that moduleA expects to find a method:

DefaultImpls.getDishes$default(dishesRepository, (DishSpecification)null, 1, (Object)null);

while moduleB expects to find

dishesRepository.getDishes();

Hence explains the runtime error faced above.

To understand why and how this occurs, let’s delve into how Gradle resolves dependencies.

Gradle dependency conflict resolution

When employing Gradle as your build tool and facing a dependency conflict, Gradle defaults to a particular conflict resolution strategy. This strategy entails utilizing the latest version of the conflicting dependency.

Let’s examine again scenario 2 discussed earlier.

Our project has two modules, moduleA and moduleB which are both built together as part of the app. moduleA depends on lib version 1.1.0, while moduleB depends on version 1.0.0.

When we build the project, Gradle identifies the conflict and picks the newest version of the dependency, which is 1.1.0.

This situation can be illustrated as follows:

com.msignoretto:lib:1.0.0 -> 1.1.0 (*)

Similarly, decompiling version 1.1.0 of the DishesRepository interface reveals the following code:

The decompiled Java bytecode clearly illustrates that the second version of the library not only introduces a peculiar static class for default implementation but also incorporates a new method getDishes(@NotNull DishSpecification var1) that requires a DishSpecification as a parameter. Meanwhile, the previous version of the getDishes() method without any parameters, has been removed.

To address this specific case, a simple solution is to explicitly add a function without extra parameters and remove the default value, as demonstrated in version 1.2.0 of the DishesRepository:

Let’s observe the decompiled bytecode after implementing this change:

You’ll notice that the old function getDishes() present in version 1.0.0 remains intact, while the new one is added. This modification maintains binary compatibility because the old code compiled against version 1.0.0 will still find the function getDishes() in the new bytecode.

Backward compatibility

Now that we’ve addressed binary compatibility, it’s essential to understand how to handle the old function. Although you might assume the work is complete, there’s another critical aspect to consider: backward compatibility.

Your clients expect the code to behave as it did before. Therefore, even if you’ve maintained binary compatibility, changing the behavior of the code in the new version could break backward compatibility.

Let’s illustrate this with the same code example used earlier.

In version 1.2.0, we reintroduced the function:

public fun getDishes(): List<Dish>

Let’s hypothesize you chose to simply return an empty list, considering this function obsolete and not intended for use anymore.

Upon running the code from the main file mentioned earlier, the output is now:

>> Module A <<
>> Module B <<

Contrast this with the previous output:

>> Module A <<
Pizza - Pizza with tomato and cheese - 10.0
Pasta - Pasta with tomato and cheese - 8.0
>> Module B <<
Pizza - Pizza with tomato and cheese - 10.0
Pasta - Pasta with tomato and cheese - 8.0