Loading...
Home
  • Tech Blogs
  • Videos
  • Conferences
    • Droidcon News
    • Upcoming Conferences
    • Become a Partner
    • Past Events
    • Keep Me Informed
    • Diversity Scholarships
  • Community
    • droidcon Team
    • How to Hold a Droidcon
  • Android Careers
Sign In

Global CSS

droidcon News
The Magic of Compiler Extensions
Speaker: Andrei Shikov Source: droidcon Online 2020 Advanced Kotlin Series Webinar (Part 3 of 3)
Transcript
English
00:14
okay
00:15
welcome everybody to our droid con
00:16
online webinar this is the the final
00:18
version of our
00:19
of the final we have a three-part series
00:21
uh on advanced kotlin this is our final
00:23
uh webinar in the advanced kotlin series
00:26
and we're really excited
00:27
for our speaker today we're excited to
00:29
be with you for this final edition
00:31
uh and just wanted before we have andre
00:33
come on do his presentation wanted to do
00:35
the usual
00:36
a few housekeeping items with you uh so
00:39
i'm gonna get started because we've got
00:40
a great presentation we don't want to
00:41
take too much time away from that
00:44
as you know all of the droidcom online
00:47
webinars are available at droidcon.com
00:50
this webinar should be available
00:52
tomorrow uh androidcom.com it should be
00:55
indexed
00:55
so you can search searchable uh and you
00:58
should be able to find all those in case
00:60
you missed anything else in this series
01:01
or from our jetpack series or
01:04
or our multi-platform series or a
01:06
hands-on in-depth series
01:07
you should be able to find everything at
01:08
droidcom.com and we look forward to
01:11
seeing you over there
01:13
if you have not used a slido before
01:17
please note that uh slido is where we do
01:20
our q a
01:22
so if you would head over to slido.com
01:25
please put in this code also you can
01:27
check in the chat function in your
01:28
interface window
01:30
uh and only my colleague will be sending
01:31
that over as a direct link
01:33
so please use slido uh to to ask your
01:36
questions don't
01:37
use the the question function in the
01:39
interface here you slido
01:41
what that'll allow you to do is in case
01:43
you see a question that you wanted to
01:44
ask
01:45
that's already been asked you can vote
01:46
on it give it a thumbs up um
01:48
if you think it's a really important
01:49
question a really smart question vote on
01:51
it too
01:52
and that way when andre's doing his q a
01:54
all of the most important questions go
01:56
to the top
01:57
so we have time to answer them so make
01:59
sure you head over to slido.com right
02:00
now
02:01
open that up in your browser and get
02:03
ready to put your questions in there
02:05
just to give you a preview what's coming
02:06
up is you know we started
02:08
uh our our droid con online webinar
02:10
series in april
02:12
we've gone through as mentioned a few
02:14
different series on jetpack and
02:15
multi-platform
02:17
we're wrapping this up with with three
02:18
more on our hands-on and in-depth um
02:22
items here so we have on september 10th
02:25
that's something it's all about testing
02:27
ui testing other testing it's going to
02:28
be a great webinar
02:30
uh on september 17th we're going to be
02:32
looking at new form factors
02:34
particularly things like dual screen
02:35
form factors uh for microsoft duo that's
02:38
going to be a lot of fun
02:39
uh and then our final one on september
02:41
24th speeding up builds
02:43
building a devops pipeline so that's
02:45
what we've got coming up for you right
02:46
now
02:47
uh we have today uh andrei shiko if
02:50
you're an android engineer for magic lab
02:52
if you had registered and looked at the
02:54
program originally you had would have
02:56
expected maybe a second speaker
02:58
we apologize our second speaker had to
03:00
drop out last minute there were some
03:01
medical complications in his family
03:04
and so we wish he and his family well
03:06
but we're excited to have andre here and
03:07
also
03:08
it'll allow him in fact to do this in a
03:10
little more relaxed way
03:12
uh and take a little bit more time than
03:14
we normally would maybe we'll have some
03:15
more time for questions too
03:17
so we're really excited to welcome andre
03:18
here um
03:20
i'm going to change the moderator over
03:22
to him
03:23
uh and so we can let him be the um
03:26
presenter now and andre feel free to
03:28
turn on your screen
03:29
or turn on your webcam and we'll be able
03:31
to uh
03:32
to see you as well there we go right
03:36
yes so so take it away it's all yours
03:41
all right all right so i hope you can
03:43
see me well
03:44
and the presentation as well and you can
03:48
hear me well as well because it's just
03:49
like really hard to
03:50
check it here so i'm really actually
03:53
excited to be on droidcon online i've
03:54
been following this whole thing like the
03:56
whole webinar series
03:57
for this time and it's really exciting
03:60
to be a part of it
04:02
uh myself and today i'm gonna be
04:05
focusing on one of my
04:06
like areas that was more interesting to
04:09
me personally for the last year i was
04:11
focusing on it really a lot it is a
04:14
compiler and a particular coating
04:16
compiler
04:17
and its extension or compiler plugins
04:20
you might have heard
04:20
them about that before but it's really
04:23
interesting
04:24
area which you can actually go deeper
04:27
and check how coding
04:29
works and why there's certain things the
04:31
way it does
04:32
so let's start and the first thing about
04:35
compiler is this
04:36
actually tool that we uh
04:39
that we use to take our source code and
04:41
compile to some kind of
04:43
form of binary and we don't really care
04:45
about how it works that much we have
04:47
like our build system like gradle or
04:48
something like that depending on what
04:50
you use or maybe using command line
04:51
interface that's also fine
04:53
but we don't really need to go deeper in
04:55
that
04:56
and check what's inside this magic box
04:59
and today's talks is right about this
05:02
magic box i'm going to be talking
05:03
how it works what kind of things he does
05:05
inside of our source code and how it
05:07
produces binaries
05:09
and with that i will keep the focus on
05:12
actually extending it for
05:13
our needs and doing some kind of meta
05:16
programming congenital things
05:19
but first allow me to introduce myself a
05:21
little so as you already saw i'm like
05:23
android engineer i'm doing android for a
05:25
while and yeah that's my photo before
05:27
before i started doing compilers
05:29
um and that's where i met kotlin
05:32
while doing android and that was the
05:34
perspective that i came into this
05:36
uh kind of whole cooking compiler
05:39
area with cotton i have really long
05:43
relationship like i've been using it
05:44
since
05:45
before the release and to be honest i
05:49
never look back
05:50
and as i started going into compiler i
05:52
realized that cotton actually has an
05:54
open one so you can just go on the
05:55
github check the code and do whatever
05:57
you like
05:58
and that's where i went i learned a lot
06:01
of stuff
06:02
and today today's talk is mostly result
06:05
of this research in the code base
06:07
and made the amazing things alone there
06:08
i don't work for jetbrains just to
06:10
clarify
06:11
and i mostly like talk to them on uh
06:14
like slack and other channels but didn't
06:17
really
06:17
participate in the creation of the
06:19
compiler itself
06:22
so this talk will be in the three parts
06:25
first i will try to clear the line
06:27
between annotation rotation
06:28
and this it will be like java annotation
06:30
procession and compiler plugins from
06:32
code inside
06:33
and when this line is cleared when we
06:35
get this understanding we can go into
06:37
the plugins themselves
06:38
and we'll go that first theoretical
06:41
things
06:41
like basically look at how the compiler
06:43
works and we'll do it step by step
06:46
and later we'll go to create the plugins
06:50
like extend these compiler structures we
06:52
already have and we'll go through
06:54
more practical things like some code and
06:58
examples of apis that we have at this
06:60
particular moment of time
07:02
all right so what's the difference
07:04
between annotation procession and those
07:06
plugins
07:06
they are both kind of playing the
07:08
similar role in the ecosystem
07:10
the kind of meta programming tool which
07:12
allows you to do some kind of code
07:13
generation and some kind of things
07:15
and on the surface they're quite similar
07:18
so
07:19
when you start using that you start with
07:20
the source file so they both take
07:23
source files of each respective platform
07:25
obviously but jetbrains actually created
07:27
tools
07:27
so you can use for example gem
07:30
annotation persistent we've gotten
07:32
when you generate these steps or uh
07:35
quoting compiler actually can understand
07:37
the java files when it compiles
07:39
you can just link to link to them but at
07:42
the same time
07:43
you won't be using quotes in the java
07:44
ecosystem because like what
07:46
doesn't compile uh java files for that
07:48
you have java compiler
07:50
and similar way annotation system kind
07:52
of work with coding without those steps
07:56
the first big distinction comes of how
07:59
we approach input
07:60
with annotation processing here we
08:03
uh get source files and we emit
08:06
new source files and then we can repeat
08:08
this over and over again
08:11
uh with different processor maybe the
08:13
same one but it just like multi-step
08:14
process can be repeated
08:16
there are some uh exceptions like lone
08:18
poke or stuff like that but they usually
08:20
using some kind of internal api majority
08:22
of the processor i've seen they actually
08:24
created the new files
08:25
and compiler is on the totally opposite
08:28
side of the scale so it doesn't emit
08:30
like new source files that you will be
08:32
later feeding and compiling it actually
08:33
creates in a binary
08:34
and it's really focusing on doing that
08:36
efficiently so the
08:38
plugin here is not like something that
08:40
manipulates with like source files or
08:41
not
08:42
area it actually uh some kind of
08:45
transform
08:46
like the extension on inside the
08:49
compiler that you can use
08:50
to adjust the generated code so the
08:53
result we will see only there
08:56
there is a way to do something like
08:59
annotation processing dance in kotlin
09:00
and ksp or code and symbol processing
09:03
you may be heard of that is like tool
09:05
from
09:05
google which went with alpha like in
09:07
june so there is a link below you can
09:10
check it out if you want
09:11
so essentially it's replicating like
09:13
this single pass annotation processing
09:15
using internal
09:16
kind of plug-in api that is there or was
09:19
created originally for kpt i think
09:21
uh and it's kind of doing the same but
09:24
majority of the plugins actually just
09:26
focusing on transforming the
09:29
source to the binaries
09:33
and the last thing i wanted to draw on
09:35
this distinction is actually
09:36
documentation because when you go into
09:39
annotation processing work it will have
09:40
been
09:41
quite a long time around and you just
09:44
have a lot of different materials you
09:46
have tutorial stocks and
09:48
the api itself is really stable so you
09:52
won't really expect the uh you won't
09:54
really expect it to break
09:56
like your annotation processing to break
09:58
when you're changing versions of like
09:59
that
09:59
with cotton block is totally opposite
10:01
you kind of
10:03
have really experimental api which was
10:04
mostly created for internal needs of jet
10:06
brains
10:08
originally and right now they slowly go
10:10
in into different directions like some
10:11
more stable thing
10:12
so it's some specifics they are changing
10:15
almost every release right now
10:16
and documentation is essentially
10:18
non-existent
10:20
so thankfully the code is available for
10:21
you to research in the first place and
10:23
community is quite helpful if you're
10:26
stuck and if you want to try something
10:28
out you usually point
10:29
out where to start where to search for
10:33
one thing i wanted to note here while we
10:35
are on uh annotation process in the
10:37
plugin
10:38
area is that why we don't really like
10:42
kpt in coding
10:43
why we'll compile an ad so the uh
10:47
most problematic part here is generating
10:49
this type so basically take
10:50
all of the quotient source source files
10:53
then you take all declaration compiler
10:55
to java so you create new java source
10:57
files and this process is quite slow
10:59
and it obviously depends on the module
11:01
size but usually takes like 20 30
11:03
percent
11:03
of the compilation itself and it cannot
11:07
be skipped
11:07
because as i said java annotation
11:09
processing cannot really understand what
11:10
happens inside the button so you need to
11:12
convert these files forward
11:13
with plugins you essentially go to
11:15
coding files and just compile
11:17
yes you may have some limited uh limited
11:20
kind of understanding of the java files
11:22
as well but if you focus in on coding
11:24
here in the first place like if you have
11:25
gotten
11:26
database or code base so it's
11:29
uh much faster to do so and just work
11:32
with the plugins directly
11:37
and plugins actually allows us to do
11:38
even more things so you can focus on a
11:40
multi-platform create your plugin which
11:42
uh does some kind of meta programming or
11:44
code generation on all the platforms
11:46
that codes and supports
11:47
you can change the code that user writes
11:49
yes you can do this with some kind of
11:52
part of the uh java coupon or java
11:54
annotation processing part but
11:56
it's still much better formulated here
11:59
inside the
12:00
uh kotlin capillary plugins and you have
12:03
way more possibilities of actually
12:05
detecting the things that you want to
12:06
change so you don't have to be focused
12:07
on rotations only
12:08
you can just go around change some
12:10
function calls or probably you have some
12:12
interesting class that extends some
12:13
marker interface you want to add some
12:15
things to that or generate some
12:17
uh some basic uh implementations of the
12:20
methods for interfacing for that you can
12:22
do it and you don't need any kind of
12:23
annotation on
12:24
any additional code to do so
12:29
but obviously there are some drawbacks
12:31
in here and first of all
12:32
is not a big on my from my point of view
12:35
but
12:36
still is important for many it's like
12:37
multiple passes so intelligent
12:39
protection you can go around and apply
12:41
different processors and
12:42
try to uh together as a multi-step
12:45
approach
12:46
with coding compiler you cannot really
12:47
do so and
12:49
to be honest compiler plugins frequently
12:52
even don't work with each other
12:54
i'm not sure if in serialization for
12:55
example right now work with compose well
12:57
and so on
12:58
and the second thing here is that you
13:01
don't really see the generated code that
13:02
much
13:03
you can check the by bytecode that is
13:05
emitted by jvm
13:06
for example you can dump some internals
13:08
of the compiler
13:10
but from user perspective it's much
13:12
easier to just go to generated java
13:14
files
13:14
like like you do with annotation
13:16
procession and check whatever whatever
13:18
generated there
13:19
you can do you cannot do this with the
13:20
compiler because they just focus on
13:22
emitting the final binary
13:24
so we have to leave that
13:27
all right so i hope we kind of
13:32
give you some understanding of what is
13:33
annotation procession what the plugins
13:34
where the line is landscape of the meta
13:36
programming that we have
13:40
and now we're moving completely word of
13:42
the compile
13:43
and as i said uh this part to be
13:46
like more theoretical we will go through
13:49
the parts of compiler how they work
13:51
together and what the structures they
13:53
pray so later we can utilize this
13:55
knowledge of actually using apis
13:57
that export the compound yeah
14:00
i establish already that there is uh the
14:03
source
14:04
files that getting transformed into the
14:07
binaries and the binders can do
14:09
different platforms because we have
14:11
kotlin
14:12
if look at this a bit deeper
14:16
we can see that this process kind of
14:19
happens in two big steps
14:20
first of them is called front-end and
14:22
second one is called back-end
14:24
so the front-end is where compiler tries
14:26
to understand what the hell have you
14:28
written in the source code that are
14:29
trying to fit in there
14:31
and here it validates errors both
14:34
syntaxical
14:35
and logical ones and this understanding
14:38
later packed into this structure the
14:40
structure is really
14:41
useful for water in the back end
14:44
what we do it with that data in the back
14:46
end we actually generate it for the
14:48
target platform
14:49
so that's where we already have complete
14:51
understanding what it happens inside
14:53
of the code that you provided to the
14:55
compiler and we can generate the
14:58
binaries or whatever the uh
15:01
targets target platform sources are
15:04
here before generating backend also goes
15:07
through the
15:08
optimization steps like for example in
15:10
line-in or
15:11
it could be some plasma specific stuff
15:14
like interrupt
15:14
the intel interrupt for jvm
15:18
for getter setters for properties or
15:19
something like that
15:21
so this is basically the front end we're
15:23
trying to understand what happens and
15:25
the back end we use an asset design to
15:27
generate
15:27
the code for the target classroom let's
15:29
go one layer deeper
15:31
and we'll start with front-end and trace
15:35
uh trace this snippet through all of the
15:38
structures that it gets converted into
15:40
so we'll understand how it goes
15:42
what steps are there on the way
15:45
and the first step is to convert this
15:47
code snippet to psi
15:49
actually it's based on this kind of
15:51
multi-step process but it's the first
15:52
one that we
15:53
exposed as a compiler plug-in writer so
15:55
we'll step in there
15:57
so psi is the program structure
15:59
interface
16:00
and is really close to its function on
16:02
an abstract syntax tree if we
16:04
ever if you know what that is and
16:08
while parsing the code like parsing
16:09
these three compiler already
16:11
checked for all the syntax errors it
16:13
knows that parentheses is all in place
16:15
there are no missing commas and so on
16:18
and
16:18
this three that it gives us it also is
16:20
immutable
16:21
so it's directly tied to the source code
16:24
that we've written
16:25
so we cannot really change it that much
16:29
uh this tree is also starts with the
16:31
file so basically we have this
16:32
collection of these trees for each of
16:34
the files that we
16:35
collect there and it really helps us to
16:38
analyze effectively
16:40
for example if we interested what is the
16:42
class that continuously selected
16:44
function
16:44
we just walk up the tree we find node of
16:46
the type class
16:48
and here we go or if we're interested in
16:51
a
16:51
like the connection other way like we
16:54
want to check what are the functions
16:56
that are inside the
16:57
class that we are purchasing we can do
16:59
the same by just going down the tree
17:02
in general psi is used by jbrains in
17:04
many places if most of the it is
17:07
and they have quite good to me quite
17:09
good documentation on that but not
17:11
unlike coding part of the website but
17:13
unlike
17:13
eddy part of the website so you can
17:16
check it out if you want more answers
17:17
with your house and
17:19
wise
17:22
so right now we have this uh tree parsed
17:26
but we still don't have the complete
17:28
information about what happens
17:30
inside these source files to demonstrate
17:32
that i added a simple call
17:34
so basically right now we have println
17:36
function that we call
17:38
inside of the example function and
17:41
this printer line is not really in the
17:42
current file and it's not really
17:45
even any current like uh set of files
17:48
that we are compiling it's in totally
17:50
different artifact
17:52
but we still need to validate the
17:53
parameters we need to check that we use
17:55
string that we pass is actually correct
17:57
and probably return type we're also
17:59
interested in that and
18:00
later we have to generate a bytecode for
18:03
that so basically
18:04
forwarding to correct instructions in
18:05
the bytecode so we all need to
18:07
link this information to the description
18:09
of that function
18:10
and for that uh coding compiler has this
18:14
structure name descriptors these cuters
18:17
can be seen as a metadata that is
18:18
associated with this psi
18:21
uh and it describes more information
18:23
about the class for example
18:25
uh you can check what is the
18:26
constructors of the class what is the
18:28
type
18:29
uh what are the type parameters that it
18:32
has
18:32
and function could be like parameter
18:34
infor and like name and obviously all
18:36
these things
18:37
so with the descriptors they can also
18:40
exist
18:40
outside of psi for example every
18:42
artifact is also part into these
18:44
descriptors
18:45
that are organized in the tree as well
18:48
ns3
18:49
also has two other elements that we
18:50
don't see in psi directly
18:52
first of them is like module the
18:54
compilation unit
18:55
uh basically all of the elements that
18:59
have during current compilation like our
19:01
current files that we compiling
19:03
some dependencies that we have all of
19:04
the classes functions can be available
19:06
from that module
19:07
and they are separated with different
19:09
package fragments usually like for
19:10
different arts in fact you have package
19:12
fragments something like that
19:13
and they are using like fully qualified
19:15
name the one that you usually write in
19:16
the import
19:17
you can actually get access to the class
19:20
like description
19:20
of the class of the function or whatever
19:23
is declaration you are looking for
19:26
so let's say that we checked the imports
19:27
and we found our println function
19:29
and now we wrap it into resolved call
19:32
which keeps the context of all of the
19:34
things that we're calling here so for
19:35
example let's say that uh
19:37
right now uh here we have the function
19:39
descriptor for println
19:40
and hello as a parameter so we just
19:42
remember that there is a constant as a
19:44
parameter here in
19:45
hello we can also kind of capture some
19:49
elements of the outer context for
19:51
example we inside the class and we want
19:52
to use some field inside this println
19:54
uh call we can also do that you it also
19:57
captured using this result called
19:58
mechanism
19:60
so these are two elements uh which
20:02
extensively use an
20:03
analysis and if you're writing compiler
20:05
plugins you probably
20:07
will try to manipulate and somehow
20:09
utilize it to
20:10
acquire some knowledge about the code
20:12
that you actually combine right now
20:15
and uh this connection between psi
20:18
and all of these elements is starting
20:21
inside the building context
20:22
you can look at it as like a hash map
20:25
which contains a different connection
20:28
between different comments
20:29
you can add your own connections there
20:32
for example if you're interested in some
20:34
metadata
20:35
specific for your plugin you can add it
20:37
there and
20:38
you can sometimes alter the existing
20:41
ones
20:42
but it's probably not the best way to
20:44
approach them
20:48
so we right now parsed all this data
20:50
into psi into descriptors and into these
20:53
result codes
20:54
and we can move into generating part
20:58
plug into the back end
20:59
and the back end is quite interesting
21:00
here because you have to support all the
21:02
different
21:03
platforms
21:09
and because of that because we have to
21:10
support these platforms
21:12
uh we have a back-end separated we at
21:15
least we had a security for most power
21:17
but front it was shared between
21:20
different platforms
21:22
so it was like that before the new
21:24
back-end came in so probably you heard
21:26
about this ir
21:27
or intermediate representation back and
21:29
then we have and right now it looks
21:31
something like that so it will be coming
21:33
somewhere in like one
21:34
for something unstable you can enable it
21:36
for a long time all right
21:38
and the interesting part here is that
21:40
despite behind all of these different
21:42
platforms with their custom tools for
21:44
generation so for jvm you for example
21:46
have object web asm
21:47
for native series lvm compilation that
21:49
takes place and for javascript there if
21:51
i remember there are some custom tools
21:53
for that
21:53
so right now with this intermediate
21:56
representation you can define some
21:58
common transforms and for us as a
21:60
compiler plugin writers
22:01
it's particularly important because we
22:04
can't
22:04
write our plugin like code generation
22:06
for a plugin only once
22:08
and apply it to all the platforms which
22:10
is quite cool for my opinion
22:11
um the api by itself is also improved a
22:15
lot with
22:16
when ir came so right now it's much
22:18
easier and much more accessible from my
22:20
point of view
22:22
and right now majority of the new
22:24
plugins that
22:26
are kind of support kind of are coming
22:28
right now on the scene they
22:29
support all this new backend as a
22:32
default so for example compose
22:33
it works by default on ir back and
22:36
that's why it's
22:38
kind of yeah
22:41
uh the exceptions from that probably the
22:43
first party from jetbrains
22:45
you have like you need to support all
22:47
the backhands in there and
22:48
if i remember correctly and wheel from
22:50
square also is written using old
22:52
backhand
22:52
because they want to use tool right now
22:54
before the ir is right
22:57
so ir by itself is also a tree
22:60
and it contains all the information that
23:02
you had in psi
23:04
and descriptors together so basically
23:06
this complete information about each of
23:08
the nodes
23:09
it's much less variables on psi so you
23:11
don't have
23:12
different nodes for identifiers and so
23:14
on entry point here is
23:16
module as we saw with the descriptors
23:17
but right now it contains
23:19
all of the modules that we compile right
23:22
now so basically files that we compile
23:23
right now
23:24
and the important part of this tree is
23:27
that right now it's mutable so you can
23:28
just go there
23:29
change whatever function you want the
23:31
body of the calls whatever you're
23:32
interested in and this is
23:37
extensively used in many plugins for
23:39
example compose modifies majority of the
23:41
calls with the composable functions and
23:42
so on
23:45
uh the extension that we have for
23:49
like adding new stuff to ir is
23:52
applied right after it is created so we
23:55
can focus on generated code
23:57
as close to original sources other
23:59
original source files as it could be
24:01
and later it goes through the process of
24:03
lowering or optimization so lowering is
24:05
basically this
24:06
as i understand the places kind of this
24:08
path for all of the
24:09
files that does one particular thing so
24:12
for example
24:13
there was uh this tweet from jay quarter
24:16
some time ago
24:16
said lowering for multi-line strings
24:19
which
24:20
uh does stream marching on the compile
24:21
time or
24:23
there is lower in the for example right
24:25
function perhaps function
24:28
uh with on jvm with the classes on top
24:30
so something like that
24:31
and they usually go for all of the files
24:34
and do only one particular thing and
24:36
later
24:36
this optimized code is uh generated to
24:39
the target ladder so like bytecode
24:41
what's already interesting
24:44
all right so this is for the compiler
24:46
first we have a front end
24:48
trying to understand what happens inside
24:50
the
24:51
source files and validate that this code
24:52
is actually correct and later we go to
24:55
the back end
24:55
and here we are focusing on optimizing
24:58
that
24:59
uh structures that we had and generating
25:02
the resulting
25:03
code for the target platform that we
25:04
have compiled right now
25:06
i hope it gives you at least some
25:08
understanding about it happens
25:09
or what is happening inside so let's
25:12
right now try to add something to it
25:16
and that's where we go into actual
25:17
creatine plugins
25:22
i will be jumping here through the code
25:25
and try to run the through process of
25:27
creating a plugin
25:28
with different pieces that can change
25:30
there
25:31
and the first note i wanted to mention
25:33
is that it's actually really
25:35
experimental all api that i mentioned
25:37
right now
25:38
so as i said you frequently have your
25:41
plugin break after like version of the
25:43
eighth of the coding
25:44
with minor one and i'm pretty sure it
25:47
will gonna be changed in the future
25:48
because
25:49
right now many kind of big plugins come
25:51
to release the compose base uh
25:53
by itself is worth money so it just
25:57
uh right now it is this experimental api
25:59
that we need to be aware of
26:00
but many things at least on conceptual
26:02
level
26:03
uh will stay the same basically now is
26:06
based
26:06
on the price we pay to get in compiler
26:09
plugin development basic early
26:12
all right so what is the compiler plugin
26:14
in the first place it is basically a jar
26:16
that you compile using the dependency of
26:18
the compiler
26:19
and later you can add this chart using
26:22
the
26:23
command line parameter x plugin so here
26:25
i did it in gradle
26:27
but in gradle you actually can do much
26:28
better you have the
26:31
transformation you can use like myelin
26:33
artifacts or mymac originates
26:35
or you can use like any kind of
26:36
artifacts that you got used to in new
26:38
ground
26:39
so you can notice here that there is
26:41
different
26:42
clasp for native and it's just because
26:44
how
26:46
shadow jar is set up for jvmgs compiler
26:49
and native one there is nothing beacon
26:51
there you can just mimic it
26:53
by yourself but it's something to be
26:54
aware of
26:57
so we connected our plugin and the first
26:60
thing that gets connected is a company's
27:02
register
27:03
as i said it's connected through the
27:04
service loader and as with anything
27:06
that uses service loader in a jvm we can
27:09
use auto server
27:11
annotation processor to connect that and
27:14
when compiler gets our component
27:15
register
27:16
it calls this method uh to register the
27:19
components
27:20
and the components or extensions that
27:23
are available there
27:24
they look like this so you have the
27:25
interface some extension slide
27:27
and it's companion object has a method
27:31
to register
27:32
like instance of it and there are many
27:34
extensions like that inside the compiler
27:36
so you for example have
27:38
uh one that allows you to modify iron
27:42
some particular phase
27:43
or you have one that allows you to check
27:45
for errors or the one
27:46
that allows you to kind of other
27:49
separate descriptors like custom things
27:51
to so compile will be aware of
27:53
during analysis on the front end so
27:57
basically each compiler appliance
27:59
usually multi implements multitude of
28:01
those checks
28:02
and their combined provide the whole
28:04
functionality of the plugin
28:07
just an example that's how you connect
28:09
that you have your extension
28:11
you have the method to register it and
28:13
your instance and that's
28:15
basically extension registered
28:18
that's why if you're looking through the
28:20
some compiler plugin implementation that
28:22
you're interested in
28:23
you can go to company registrar and see
28:25
all of the
28:27
extensions that are connected right now
28:29
and this can give you at least some
28:31
overview what happened it
28:32
what happens inside that because you may
28:34
be aware that this extension that this
28:35
on this particular face
28:36
or some another and later you can go
28:39
like face by face and trying to
28:40
understand
28:41
what the compiler plugin does in each of
28:43
those
28:46
and as an example of writing the plugin
28:48
i will
28:49
look at two really small subsets of the
28:52
quote and actualization
28:54
uh code link serialization by itself is
28:56
a really good example of the plugin that
28:58
you made right
28:59
i totally recommend you go there check
29:01
it out and see
29:02
how it works inside like the one that
29:04
you actually could use in production
29:06
but here i will be taking like small
29:07
subsidy for those who do know
29:10
what that is it's basically the magic
29:12
annotation that you cannot take your
29:13
classes with
29:14
and laser regenerates some bulletproof
29:17
so you can
29:18
integrate it with a serialization
29:20
framework
29:21
library that it has what exactly does it
29:24
generate
29:25
basically two parts first is this custom
29:26
serializer that explains how you will be
29:29
serializing code it has better to
29:30
serialize deserialize and this
29:32
child serializer should actually explain
29:34
uh how the structure will be looking
29:36
and uh this
29:39
custom serializer is available through
29:42
the method
29:43
in a companion object that you can
29:46
basically use
29:47
to specify to the library how you want
29:50
the object to be serialized
29:52
so that will be focusing today so it
29:53
will be like a method and then some
29:55
implementation of the serial license
29:58
and we'll be doing that in two parts so
29:60
first of all first of them
30:01
is to add serialize because uh during
30:04
analysis because compiler doesn't really
30:06
know that we have this new method so if
30:07
you look at that
30:08
in the companion object we have a new
30:10
method we can call outside of this class
30:12
and compile is not really aware that
30:14
this method exists right now so we need
30:15
to tell it like an analysis phase there
30:17
is method that exists
30:19
and later we will actually generate
30:20
implementation of the serializer
30:22
and the functions let's start with the
30:26
first one
30:27
so here we'll be adding serialization
30:29
serializer descriptor
30:30
during analysis stage and to do that we
30:33
will use the
30:35
extension that is called synthetic
30:36
resolve extension it allows you to add
30:38
properties
30:39
methods and classes i think that's it uh
30:43
two different like different classes
30:45
that you have inside content
30:48
and you can do you do it in like two
30:50
stages
30:51
first you specify that there is this new
30:52
identifier that you are interested in so
30:55
basically saying that there is new
30:56
identify serializer
30:57
inside this companion object of the
30:59
serializable class and later you specify
31:01
the metadata associated with this
31:03
identifier
31:05
so let's go through adding actually that
31:07
method yeah that's the method
31:10
so first we specify the identifier and
31:13
here we say
31:14
okay if this companion object is
31:17
actually serializable
31:19
oh if this component is actually inside
31:22
a serializable class
31:23
which we check using the annotation we
31:26
will add like return this new identifier
31:29
that we are interested in
31:30
otherwise we just add anything
31:34
then this name like then we get called
31:37
with another method
31:38
and here it's very important to check
31:41
that
31:42
we are called with is the one that we
31:44
are
31:45
ourselves because compiler will be just
31:47
calling this with all the classrooms you
31:49
encounter with all the names
31:50
but it encounters in sizes like tool
31:52
string equals some
31:54
custom implementations and so on
31:59
right so
32:03
here we check that all of these things
32:05
that we've been called is correct is
32:07
actually the method that we are
32:08
interested in
32:09
and later we create a descriptor
32:12
and add it to this result basically it
32:15
will collect
32:16
that coding expects from us
32:20
and creating the descriptor is actually
32:22
quite simple so
32:23
you just take the class that is is
32:26
inside
32:26
you add some annotations right now we
32:28
don't have any sheets mc
32:30
e and the name and he also specified the
32:32
synthesized so cotton won't be looking
32:34
for the psi
32:35
source of this uh serial
32:38
for this descriptor there
32:41
and installation basically continuation
32:43
of creation is just
32:45
uh kind of the specifics of how compile
32:48
applied uh
32:49
how compiler parses the functions
32:51
internally
32:52
so when we initialize an app we need to
32:54
specify what are the type and
32:56
usual parameters of the function return
32:59
type of that
32:60
and we're also saying that there is some
33:02
flux that you're interested in it's like
33:04
final
33:04
and public so we added this function and
33:08
compiling our knowledge we can call
33:10
it it will generate the correct links
33:12
link in white code or something
33:14
but we will fail with the abstract
33:17
syntax of after abstract method
33:18
or something it's basically saying that
33:21
there is no implementation
33:22
in actual by that's what we need to do
33:24
on the back end
33:26
and that will be using ir generation
33:28
extension
33:30
uh it will give us the motion fragment
33:33
so basically this top element
33:34
that we are in the then and later we can
33:36
show you how to hold three base on that
33:38
and we also have the environment of this
33:41
inside
33:42
in form of the plug-in context
33:45
so this is an implementation of the
33:48
actual generation i
33:50
created this machine part is the
33:52
lowering
33:53
so lowering as i said it passed through
33:55
the whole
33:57
whole like all of the compile
33:60
uh files being compiled right now that
34:04
we want to something with and right now
34:06
i'm interested in classes because that's
34:07
what i'm marked as serializable
34:09
so i'm using class lowering past and it
34:11
will be called on each of the
34:12
instances of classes that uh
34:16
that the plugin will encounter inside
34:18
the module fragment and for each of
34:21
these
34:22
uh for these three classes i just run
34:24
generate and later i check if i need to
34:25
generate something
34:28
so the first thing when you start
34:30
generating are actually taking it is
34:32
like
34:33
which what exactly doing it's right like
34:35
which comments exactly should be there
34:36
or what instructions
34:37
and for that usually when you're writing
34:39
bytecode you just go
34:41
and check what the resulting plus files
34:43
but with ir is a bit harder because you
34:46
like on a level
34:46
higher than that so here you have
34:50
basically some compiler arcs that allow
34:52
you to dump this internal state with ir
34:54
in the component
34:55
so you specify the directory where you
34:57
want all these dots to be
34:59
and the phases that you want to download
35:01
so the phase is basically each of those
35:02
optimizations
35:03
here i've chosen the first one i found
35:05
in a compiler for jvm
35:08
but you can just write all in all tabs
35:11
and you'll just enumerate them you know
35:13
and just will be
35:14
there like after all optimizations you
35:17
will see what the
35:18
ir how it's changed if mostly you're
35:20
interested only in the first one because
35:22
that's this
35:22
that you're supposed to be generating
35:26
um and these ir is really huge so
35:30
there's just one data class in final
35:31
annotation
35:32
but it's really a lot of things that
35:34
code engineering raised there and iris
35:36
was really verbose but it's much easier
35:38
to read
35:39
uh because it just feel like structure
35:43
and you can just find the identifiers
35:46
you are
35:46
interested in and later just go for that
35:49
through the
35:50
tree that is connected so here for
35:52
example i have this object
35:54
uh definition that is there and we have
35:57
the ir that is
35:58
uh kind of associated to that and now we
36:01
can just use the
36:04
use the builders inside the compiler to
36:05
go one by one
36:07
and just generate these things for
36:09
example here i'm using builder called
36:10
build class
36:11
and add in this class
36:14
inside the parent companion object that
36:16
they have
36:17
and the super type as well which also
36:20
mentioned
36:20
like mentioned in these super types
36:24
field inside the ir the same thing you
36:27
do
36:27
for the constructor you just have this
36:29
builder and you can
36:31
just basically write all the fields that
36:33
you see inside them
36:36
and for the body it's like two commas
36:39
that you have in there
36:40
and for that you can use the builders
36:42
for the budgets that they have inside
36:43
color again and it's basically the same
36:45
thing the interesting part here is this
36:47
dsl with the only plus
36:48
which basically allows you to add your
36:51
statement to a list of statements inside
36:52
the
36:53
blog block which is the body of the
36:55
function
36:57
all right i won't go through the whole
36:58
generation of ir for that particular
37:00
case
37:02
but you can find the complete example
37:03
before or also some
37:05
examples of how to check for errors and
37:08
some other things
37:10
for using these links i just like
37:12
repository which has this compiler
37:14
plugin with some readme which you can
37:16
check what it actually does
37:19
uh just to sum up what we did with the
37:22
plugin
37:23
is that probably most plugins which are
37:24
not like generating source code
37:26
as the case be that's what they do they
37:28
usually modify a little bit on an
37:30
analysis stage
37:31
they add some checks that their things
37:33
are correct and they also add the
37:35
uh some discrete or some metadata to say
37:37
that they added some new
37:39
classes or some methods or something
37:41
like that and
37:42
when they uh go to the
37:46
backend stage they just do like pure
37:49
code changes basically altering whatever
37:51
is there on the ir level
37:52
as you saw with these builders and this
37:54
builder is really helpful because that's
37:56
how they do all of these lowerings
37:57
inside the compiler so there is a really
37:59
huge functionality in there
38:00
and majority of those is more or less
38:03
stable so
38:04
it doesn't really break that much in
38:06
that layer right now
38:08
all right uh
38:12
that's basically it for me right now
38:15
so let's sum up first of all
38:18
i wanted to say that because i guess
38:20
that's really experimental apigee brains
38:22
doesn't give us any guarantees
38:23
and uh i wouldn't just put them directly
38:27
to the production
38:28
unless you are really you really kind of
38:32
are free to
38:33
uh take this new release of courtly and
38:35
make sure that everything works right a
38:37
lot of tests for that and so on
38:38
so it's like a little bit harder to do
38:41
it with the usual the more stable things
38:43
but at the same time you have a lot of
38:45
possibilities that this compiler open
38:47
like you could not really do annotations
38:50
and you have a multi-platform thing and
38:53
also the impact on the compilation speed
38:56
is much lower because you don't have to
38:58
make java tools to understand because
39:02
and also one thing is that many
39:04
compilers that have done like exactly
39:06
the way i described they actually
39:07
support
39:08
incremental compilation out of the box
39:10
which is also good
39:11
so here i want to say that compiler
39:13
plugins is pretty
39:14
new thing but they're for sure here to
39:17
stay because
39:17
many of the companies are invested in
39:20
also for example google with their
39:21
compose
39:22
uh we have jet brands with multitude of
39:24
applicants they have written from the
39:25
conception of
39:26
cotton straight away like personalized
39:28
and all of the android things that we
39:30
did
39:31
so this is something that i've been
39:33
looking into and
39:34
was really excited about i hope i give
39:37
you some understanding what it is
39:39
and maybe you're excited about this as
39:41
well
39:43
um and also before i finish
39:46
i want to mention aroma method because
39:48
you cannot really talk about compiler
39:49
plugins and communicative without
39:50
mentioning aromatics
39:52
and quite a big community that focusing
39:54
more on the functional thing so like
39:56
we're doing a lot of interesting
39:57
stuff on uh type level extensions and
40:00
content
40:02
but more importantly they're writing a
40:03
framework for
40:05
us like developers of those plugins
40:09
and uh those
40:12
though this framework allows you to have
40:14
like more or less competitive experience
40:16
with the idea and like compile a part of
40:20
that so basically
40:21
joining those together it's more or less
40:23
in the alpha stage but the
40:25
uh opportunities that it gives to you
40:26
now is already quite huge so i recommend
40:28
to check out and there is
40:30
a hash from the question like slack if
40:32
you're interested to join the channel
40:33
ask them how that works and what other
40:35
things
40:37
and yeah thank you for listening i hope
40:39
you enjoyed this
40:41
you can find me on twitter or telegram
40:42
by using these handles on top
40:44
you can find this example with the
40:46
plugin that
40:47
went through in the bottom and if you're
40:50
interested in compiler things so you
40:52
started writing your own you beat stack
40:54
you can go to the
40:55
link channel compiler and ask questions
40:58
there are there are people from
40:58
jetbrains they
40:59
really kind of recently started to
41:01
finally answer the questions about the
41:03
plugins
41:04
and sometimes something as well so
41:05
probably even i can help you if
41:07
something
41:07
goes wrong so right that's it for me
41:10
thank you very much
41:14
well honor thank you again so much for
41:16
for doing this i know
41:18
when you and i talked yesterday we
41:20
talked a little about some
41:21
you know you've given this presentation
41:23
before in russian
41:24
uh and get up and then internally and um
41:27
it's always whether there's questions
41:30
getting astronauts always depends
41:31
because it's kind of a
41:33
a new and esoteric topic so right now
41:36
i'm seeing in slido that we don't have
41:38
any
41:38
questions currently about the
41:40
presentation but i did want to give
41:42
some people a second to think about it
41:44
um but i appreciate the fact that you've
41:46
also given your contact information
41:49
if anybody does have questions uh after
41:51
they've watched the video
41:53
or whatnot they can reach you on
41:55
telegram or on
41:56
on twitter to ask questions and i
41:58
believe um uli will also be sending out
42:01
we have a copy of the of the slides
42:03
correct
42:05
oh here you know what we do have one
42:08
question i'll have really um show this
42:11
one up on the screen then
42:13
it looks like we did finally get a
42:14
question really earlier
42:18
that's actually quite cool
42:22
okay so all right yeah
42:26
let me just move it over
42:30
right so yeah that's probably the
42:33
hardest question that i could answer
42:35
uh so what's the best way of um
42:39
get familiar with all those extensions
42:41
of the
42:42
plugins of the compiler um right now
42:45
there is
42:46
not really much so from me the best
42:48
experience was to go through the
42:50
actual plugins that other people written
42:53
because they give you clarification
42:54
experience they have a goal that you
42:55
want to achieve
42:56
and have the things how they did it for
42:58
example cod linux serialization was like
43:02
one of more like harder ones to maybe
43:04
procedure but at the same time it was
43:07
really interesting to go through that
43:08
like elements of how they add the method
43:11
here
43:11
or how they modify the
43:14
coding like what do they do in the code
43:16
gene station so on
43:17
uh there are some simple examples out of
43:20
there and unfortunately i don't remember
43:21
right now there was
43:23
uh one from it was one uh doing some
43:26
kind of
43:26
assertion like smart asserts it was
43:28
really small just doing some kind of i
43:30
really like back and forth called gene
43:32
things
43:32
and i probably that's the nice way to
43:34
start so i recommend you to like search
43:37
for
43:37
i think it's also like power resort or
43:39
something like that if i
43:41
remember correctly so maybe you can
43:43
check those
43:44
and start from there right now because
43:46
documentation is really lacking
43:48
um one of the other things maybe i'm not
43:51
sure if
43:52
people from aerometa actually given any
43:55
uh given any like books about a
43:58
particular framework that would be like
43:60
well they actually did that mostly
44:01
but you're gonna go there and ask how
44:04
these things works as well but it's
44:05
probably like
44:06
usually you go to this like channel to
44:08
ask if you don't have something working
44:10
right it's just
44:11
really hard to ask people to explain to
44:13
you what is there
44:23
well all right i think that you you
44:25
answer the hardest question
44:27
uh the one and only so anyway
44:30
well good very good well we're really
44:32
again excited that we had you with us to
44:34
wrap up our
44:34
our advanced kotlin series he certainly
44:37
did a great job and we really appreciate
44:38
you being here
44:39
and talking to us about a topic that i
44:41
think not a lot of people have a lot of
44:42
experience with
44:43
but it definitely is important opens uh
44:45
i was looking into it after you and i
44:47
talked yesterday definitely
44:49
using the compilers extensions opens a
44:51
lot of doors to extending functions and
44:53
things i think it's really
44:54
a useful tool so i'm excited that you
44:56
were able to teach us about that
44:58
uh as it anyway this video will be
44:60
available online at droidcom.com
45:02
tomorrow
45:02
feel free to ping andre on telegram or
45:05
in or
45:06
twitter for any questions and again we
45:08
uh andre thank you and we look forward
45:10
to seeing everybody
45:11
soon uh for the last three of our online
45:13
uh joycon webinar series
45:15
and we look forward to seeing you all at
45:16
droid conor of droidcon america's an
45:18
apac
45:19
coming up soon that's it we're signing
45:22
off oh
45:23
we did one final question sneak in go
45:25
ahead
45:27
right so others are available only on
45:29
intelligence so this is interesting part
45:31
of it goes like
45:32
uh if you're talking about compiler
45:34
plugins they're available on the
45:36
compile itself so it's like coding see
45:38
things like just runs through
45:39
item yeah so psi is baked into the
45:42
compiler
45:43
itself so it's not really
45:47
something limited to only of the
45:48
intelligent platform
45:50
uh i think the uh why they chosen phi in
45:53
the first place
45:54
is because they like had the in mind
45:58
this integration of the tooling and
46:00
you have to agree like in tooling for
46:02
kotlin it's truly amazing
46:04
so a big part of that is how big the
46:07
plate is how the compile is structured
46:09
on the front and where it understands
46:10
the code because they shared a lot of
46:12
common parts
46:13
with the idea and that's why it repeats
46:15
i chose
46:16
but you know it's totally it's totally
46:19
unbounded from the utility you can use
46:21
quotient without intelligent
46:22
and probably some people do for sure
46:30
all right okay
46:33
i think we're good we're going to sign
46:35
off but again thanks for being here we
46:36
look forward to seeing
46:37
you in another event take care everyone
46:39
have a good day
46:41
bye
droidcon News

Tech Showcases,

Developer Resources &

Partners

/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/home-details/EmployerBrandingHeader
EmployerBrandingHeader
https://jobs.droidcon.com/
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/jobs-droidcon/jobs.droidcon.com
jobs.droidcon.com

Latest Android Jobs

http://www.kotlinweekly.net/
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/kotlin-weekly/Kotlin Weekly
Kotlin Weekly

Your weekly dose of Kotlin

https://proandroiddev.com/
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/pad/ProAndroidDev
ProAndroidDev

Android Tech Blogs, Case Studies and Step-by-Step Coding

/detail?content-id=/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Zalando/Zalando
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Zalando/Zalando
Zalando

Meet one of Berlin's top employers

/detail?content-id=/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Academy for App Success/Academy for App Success
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Academy for App Success/Academy for App Success
Academy for App Success

Google Play resources tailored for the global droidcon community

Follow us

Team droidcon

Get in touch with us

Write us an Email

 

 

Quicklinks

> Code of Conduct

> Terms and Conditions

> How to hold a conference

> FAQs

> Imprint

Droidcon is a registered trademark of Mobile Seasons GmbH Copyright © 2020. All rights reserved.

powered by Breakpoint One