We know that Kotlin goes beyond Java by adding new features and improving existing ones. But have you ever wondered why certain features are missing in Java and how Kotlin overcomes those limitations?
Let’s look at one such feature: closures. As the story goes…
It all started with one Kotlin Code Snippet.
fun counter(): () -> Int {
var i = 0
return { i ++ }
}
counter is a higher-order function—which either takes functions as parameters or returns a function. In this case, it returns a function of type () -> Int.
We can store it in a variable and invoke it multiple times:
fun counter(): () -> Int {
var i = 0
return { i ++ }
}
fun main() {
val next = counter()
println(next())
println(next())
println(next())
}
Can you spot what’s going to be printed? 👀
The function returned from counter() is an anonymous function that can both keep a state that is initialized out of its scope and perform some operations on it. Here, we see the incrementing variable i declared in counter().
Looking under the hood, the function captures variable i by making a copy into its scope — enabling the function to modify the variable even after the execution of counter() has finished. As a result, the function retains the ability to preserve the state of the variable across multiple function calls.
Our case (above) — where a function can access and manipulate variables from the scope in which it was created, even if the scope no longer exists — is an example of a closure.
In Kotlin, closures might be seen when using higher-order functions like map, filter, or forEach, as well as when passing a lambda as a function parameter. These closures can capture variables, including the implicit it.
However, we’re not here to analyze closures. Let’s get into the nitty-gritty of our main topic.
As you may know, the Kotlin compiler for JVM compiles Kotlin source files into Java class files. You might expect that once the code is rewritten in Java, the output is identical. But that’s not the case. Once it hits Java…
public final class Counter {
public Function0<Integer> counter() {
int i = 0;
return new Function0<Integer>() {
@Override
public Integer invoke() {
return i++; // error here
}
};
}
}
…the result is a compilation error:
javac Counter.java
Counter.java:9: error: local variables referenced from an inner class must be final or effectively final
return i++;
^
1 error
The goal of this blog is to understand why.
Closures and local variables
Why the error in Java?
Local variables referenced from an inner class must be final or effectively final.
The restriction for inner classes is stated in the Java Language Specification: “Any local variable, formal parameter, or exception parameter used but not declared in an inner class must either be final or effectively final.”
Note: A comparable rule also exists for lambdas.
This is due to the nature of closures. As mentioned, captured variables are independent copies of variables in the enclosing scope. Those variables must be declared as final or effectively final to ensure consistent behavior by preventing their values from changing.
If captured variables could be modified in the inner scope, any change would not be reflected in the corresponding variables in the outer scope — because these variables are distinct entities stored in separate stack entries. Enforcing such variables to be final prevents running in such a scenario.
Why no error in Kotlin?
Unlike Java, Kotlin allows modifying captured variables in a closure.
As stated in Kotlin documentation: “A lambda expression or anonymous function (as well as a local function and an object expression) can access its closure, which includes the variables declared in the outer scope. The variables captured in the closure can be modified in the lambda.”
Kotlin/JVM code is designed to be interoperable with Java and thus must comply with the Java language specification for compatibility. So how does the Kotlin compiler perform all necessary transformations to meet this requirement?
To find out, the simplest approach is decompiling the Kotlin bytecode to Java. One way of doing the conversion is by using IntelliJ IDEA.
// the result is simplified
public final class CounterKt {
@NotNull
public static final Function0 counter() {
final Ref.IntRef i = new Ref.IntRef();
i.element = 0;
return new Function0() {
public final int invoke() {
return i.element++;
}
});
}
}
Ref.IntRef is a reference type that is provided by the kotlin.jvm.internal package and designed to hold an int value.
The reason why the state of the reference type can be changed inside a closure is that the closure captures a reference to the object, not just its value. Therefore, any change made through the reference inside the closure will directly affect the original variable it refers to.
This allows closures to modify the state of variables captured from an enclosing scope while having them declared as final.
But there is another way…
Closures and instance variables
Since it applies to local variables only, the Java restriction on final values can be bypassed. Rewriting the example using i as an instance variable would compile.
public final class Counter {
private int i = 0;
public Function0<Integer> counter() {
return new Function0<Integer>() {
@Override
public Integer invoke() {
return i++;
}
};
}
}
Does this mean there’s no longer a closure?
Well, there still is a closure; the difference is in what is captured. The best way to see this is to dive into the resulting Java bytecode of both examples.
Show me the bytecode!
To speed up the investigation (because let’s face it, every programmer wants to make life easier), I used Bytecode Viewer (v2.11.2) — a handy graphical tool that enables you to view and understand the low-level bytecode instructions generated by the Java compiler.
By decompiling the first example, which uses a local variable of a reference type Ref.IntRef, we get the following:
/* --- original --- */
public final class Counter {
public Function0<Integer> counter() {
Ref.IntRef i = new Ref.IntRef();
i.element = 0;
return new Function0<Integer>() {
@Override
public Integer invoke() {
return i.element++;
}
};
}
}
/* --- decompiled --- */
public final class Counter {
public Function0 counter() {
Ref.IntRef i = new Ref.IntRef();
i.element = 0;
return new Counter$1(this, i);
}
}
class Counter$1 implements Function0 {
// $FF: synthetic field
final Ref.IntRef val$i;
// $FF: synthetic field
final Counter this$0;
Counter$1(Counter this$0, Ref.IntRef var2) {
this.this$0 = this$0;
this.val$i = var2;
}
public Integer invoke() {
return this.val$i.element++;
}
}
This snippet provides a nice demonstration of closures in action. To understand it further, let’s break it down.
The generated anonymous inner class implementing Function0 interface — Counter$1 — represents our closure. Its constructor accepts Ref.IntRef as a parameter. Thus, when a new object is instantiated, it will capture the reference to the variable i from the counter() method in the Counter class. The created object will be available in the heap until it is garbage — collected, which does not have to match the execution time of the counter() function.
Noticed the hint about what’s captured in the second example, showcasing instance variables?
/* --- original --- */
public final class Counter {
private int i = 0;
public Function0<Integer> counter() {
return new Function0<Integer>() {
@Override
public Integer invoke() {
return i++;
}
};
}
}
/* --- decompiled --- */
public final class Counter {
private int i = 0;
public Function0 counter() {
return new Counter$1(this);
}
}
class Counter$1 implements Function0 {
// $FF: synthetic field
final Counter this$0;
Counter$1(Counter this$0) {
this.this$0 = this$0;
}
public Integer invoke() {
return this.this$0.i++;
}
}
There is one more constructor parameter used in both examples that we haven’t discussed yet: the parameter this$0 of type Counter.
When a new Counter$1object is created, the reference to the enclosing class is captured instead of individual variables. This means that any modification to the variable i is made through the captured class reference.
Since Counter$1 holds a strong reference to Counter, the lifetime of Counter is at least as long as the lifetime of Counter$1. Because any change on i is preserved, it does not have to be declared as final.
Note: According to the Java Language Specification, every non-static inner class captures a reference to its enclosing class.
However, with great power comes great responsibility. For the same reason as stated above, such a capture might be a source of memory leaks. If the Counter$1 instance was stored in a long-lived object or referenced by other active objects, it would prevent the Counter instance from being garbage — collected.
Therefore, always double-check when you use inner classes or when you pass your lambda from one place to another in your codebase. They might carry more than you expect.
Wrap up
Kotlin and Java offer distinct approaches to handling closures. While we can’t mess with captured variables in Java to keep things stable, Kotlin comes with more degrees of freedom to play around with mutable states inside closures. Regardless of what you choose, it’s good to grasp these differences and be aware of the extra effort involved. Hope this blog helps with that.✌️
I’d like to thank Alexander Kovalenko for adding a spark to the article, Tomáš Mlynarič with Prashant Rathore for valuable feedback and Linda Krestanova for the English check! 🫶
Resources:
- Soshin, A. and Arhipov, A. (2022). ‘Closures’, in Kotlin Design Patterns and Best Practices: Build scalable applications using traditional, reactive, and concurrent design patterns in Kotlin. Packt Publishing Ltd, pp. 161
This article was previously published on proandroiddev.com
