Loading...
Home
  • Tech Blogs
  • Videos
  • Conferences
    • Droidcon News
    • Upcoming Conferences
    • Become a Partner
    • Past Events
    • Keep Me Informed
    • Diversity Scholarships
  • Community
    • droidcon Team
    • How to Hold a Droidcon
  • Android Careers
Sign In

Global CSS

 

How to Use Shaders for Android View and How Android View May Use Shaders

 

 
Alexey Glukharev
Android Team Lead and Co-founder at Nova Ocean
Published: March 17, 2021
Tweet
Share
 

 

It’s not enough to use only Canvas to work with dynamic UI elements that use shadows, real-time filters for photo or video, or responsive UI or lighting. It would be best if you had something more powerful. In the past, we were able to use RenderScript for this, but is it supported anymore?

I suggest using OpenGL for your custom view, which is an inheritor of the Android View class (android.view.View) with regular GLSL shaders if you are working on one of the following:

  • Shaders or real-time color corrections for video streams.
  • Dynamic shadows and lighting for custom UI elements.
  • Advanced per-pixel animations.
  • Some UI effects, such as blurring, distortion, pixelization, etc.
  • If you are creating a brand new, neuromorphic, responsive UI.

It will provide a robust environment and many examples of shaders that you can easily use for your application. And I will show you how to do it easily!

 

Idea

We need to have a class that behaves the same as Android View (android.view.View) lying in our standard layout, and we can use the OpenGL fragment shader to render its content.

 

Demo

 

Demo app with multiple ShaderViews. Dynamic light and video filters.

 

How it Works in Abstract

Suppose we want to order a new picture painted with some magic paints from one famous artist and hang it on the wall. What we have in our situation:

  • Magic paints — OpenGL GLSL shaders.
  • Canvas — a quadrangle that will fill in all space of our custom view.
  • Famous artist — a class that implements Render interface. This artist, in turn, uses magic paints to draw the picture on the canvas.
  • Picture — custom view class that uses artist with his/her canvas and magic paints.
  • Wall — android Activity or Fragment.

 

How it Works on the Technical Side

  1. Let’s choose a parent view for our custom view class (BTW, let’s call our view class ShaderView). We have two options: SurfaceView and TextureView. I will describe the difference in a moment.
  2. Create a Render class that will render the view using shaders.
  3. Create a 3D model of a quadrangle that will fill in all the view space (3D, as far as OpenGL was made for 3D scenes). Don’t worry about this point; it’s a standard solution, and it’s easy to do.

 

OpenGL quadrangle inside TextureView.

 

SurfaceView or TextureView

SurfaceView and TextureView are both inherited from the Android View class, but there are some differences between them.

To that point, SurfaceView has an inherited class that works perfectly with OpenGL and provides excellent performance. This class is called GLSurfaceView. But the main issue of this class is that we cannot overlay one GLSurfaceView with another. Hence, we cannot use it in our layout hierarchy, and we are not able to transform, animate, or scale a view like this

TextureView behaves as a regular android.view.View, and you can animate, transform, scale or even overlay one with another. But this advantage consumes more memory than SurfaceView, and you lose performance (avg. 1–3 frames).

Coming back to the point, as we wanted that our custom view acts as a regular Android View, we should use TextureView.

The next challenge for us is that there is no built-in class that uses OpenGL render and TextureView. But hold on — GLSurfaceView works exactly for what we need but with SurfaceView, so let’s think about how we can use this class for our brand new GLTextureView.

 

Create GLTextureView

Okay, GLSurfaceView has almost everything that we need for OpenGL rendering. What we need is to copy this useful code to our class and do some modifications.

  1. Create a new class GLTextureView.kt which is inherited from TextureViewand extends TextureView.SurfaceTextureListener and View.OnLayoutChangeListener. Add constructors.

 

 
 

2. Copy-paste code from GLSurfaceView to our GLTextureView class.

3. Remove old Java-style constructors and init() method.

4. Add a new init {} block.

 

 

5. Update finalize() method to the Kotlin way :) (If you’ve got a better solution, let me know in the comments!).

6. Replace SurfaceHolder with SurfaceTexture.

7. Replace all mentions of GLSurfaceView with GLTextureView.

8. Update imports avoiding using GLSurfaceView. Also, check existing imports and delete everything related to GLSurfaceView.

9. Fix nullability issues after auto-conversion of Java code to Kotlin. In my case, I had to updates override methods and some nullable parameters (e.g., `egl: EGL10` should be `egl: EGL10?`).

10. Move constants to companion object or to the top level.

11. Remove unsupported annotations.

12. Add methods of SurfaceTextureListener interface.

 

 

13. In createSurface() method you may find a broken line, replace view.holder by view.surfaceTexture.

14. Override onLayoutChange.

 

 

As a result, you will have something like this.

 

Extensions

To make the work easier on ourselves, we will create extensions that load the shader’s source code from the RAW resources directory.

 

 

Code of Shaders

In this article, we will use simple shaders to make our code less complicated. If you’re looking for more complex shaders, you may them find here or here.

 

Vertex Shader

For our purposes, we need just a simple vertex shader to render our quad (don’t spend a lot of time on its code).

 

 

Fragment Shader

The code is quite simple, but let’s see what we have here.

First of all, we define the version of GLSL.

 

#version 300 es

 

Then, define the custom parameters that we’re going to send to the shader.

 

uniform vec4 uMyUniform;

 

Define input and outputs parameters for our fragment shader. Input — what we receive from Vertex Shader (texture coordinates, in our case) and Output — what we send as a result (color of a pixel).

 

in vec2 textureCoord;
out vec4 fragColor;

 

And write the function that will be executed for each pixel of our Android View and return its color.

 

void main() {
   fragColor = vec4(textureCoord.x, textureCoord.y, 1.0, 1.0) * uMyUniform;

}

 

As a result, we will have this:

 

 

QuadRender

The next class that we’re going to need is a rendering class. This class will draw a quadrangle with the size of the ShaderView using shaders.

 

OpenGL quadrangle in camera’s projection. The camera is a point of view of a user who looks at the device.

 

Our class should extend GLTextureView.Renderer interface with three methods:

onSurfaceCreated() — Create shader program, bind some uniform params, and send attributes to the vertex shader.

onDrawFrame() — Each frame update. In this method, we draw the screen quad and update uniform params if needed.

onSurfaceChanged() — Update the viewport.

So, let’s write the code step by step. I won’t go into detail describing how OpenGL works in detail because it’s out of the scope of this article. I also want to mention that we are focusing only on the fragment shader and are not concerned with the vertex shader’s details, as it should be the same for almost any possible fragment shader requirements.

Define constants.

 

 

Two variables which will keep the source code of our vertex and fragment shaders.

 

 

Define the list of vertices for the vertex buffer.

 

 

Define matrices.

 

 

And add initialization to init {} block.

 

 

Vertex shader, vertices attributes, and matrixes locations.

 

 

Locator of shader program.

 

 

Okay, we’re done with initialization, so let’s write the onSurfaceCreated()method. We will load and initialize our shaders and get locators for attributes, including the uMyUniform uniform parameter that we will use to send some custom vector data to the fragment shader.

 

 

Please pay attention to the last three lines where we get our custom uniform’s location (uMyUniform) for the fragment shader. For more complex shaders, we will have to add more such parameters.

In onSurfaceCreated() we’ve used some methods to create and link the program.

 

 

The next method that we should make is onDrawFrame().

 

 

Please pay attention to the lines where we send a custom value (uMyUniformValue) to uniform (uMyUniform) to the fragment shader.

And the last one, surfaceChange() is quite a simple method.

 

 

The full code of this class you may find here.

 

ShaderView

Great, everything that we need for our Shader View is ready. Now we can use the power of a fragment shader to render its content! Let’s create a ShaderView.

 

 

Advanced: Using Textures in Fragment Shaders

It’s nice that we’ve learned how to communicate and work with fragment shaders, but in many cases, you have to deal with textures (e.g. for effects like blurring or video effects).

In the fragment shader, you should define the uniform as sampler2D type and get the current pixel of texture by the texture coordinates using the texture() method from GLSL.

Here is a complete code of the shader.

 

 

Then, we will need two extensions to load and use bitmap as OpenGL textures.

 

 

Now we’re ready to load a texture from the drawable resource directory as a bitmap using loadBitmapForTexture() and then in the QuadRender.onSurfaceCreated() method. We will bind the texture to the OpenGL texture slot (slots from GL_TEXTURE0 to GL_TEXTURE31 are available).

Don’t forget to recycle the bitmap when you don’t need it anymore.

 

 

After this, we set this texture as active and visible for the fragment shader in QuadRender.onDrawFrame().

 

 

For the full code of the example of how to use a texture, you will find it in this branch.

 

Examples and references

For the source code of this article, you will find it in my repository.

 

appspell/shader_view_for_article_example

Example of ShaderView for article. Contribute to appspell/shader_view_for_article_example development by creating an…

github.com

 

Or you can explore the ShaderView library with a friendly, high-level API here.

 

appspell/ShaderView

This library is the easiest way to use OpenGL shaders as an Android View . You just simply need to add ShaderView in…

github.com

 

 

 

 

Tags: Shaders, Android, Opengl, Android App Development, AndroidDev

 

View original article at: 


 

Originally published: March 05, 2021

Android News
Getting… your BottomSheetScaffold working on Jetpack Compose Beta 03
Getting… your BottomSheetScaffold working on Jetpack Compose Beta 03

By Carlos Mota

It’s Monday, no releases this week, and… there’s a new version of Jetpack Compose — beta 03—available. What a perfect time to just increment 02 to 03 and see what’s new. The API is (almost) final so after updating from alpha to beta there weren’t any big changes to do. However, and remember that’s still in development, there’s always something that I need to update. 

By ProAndroidDev -
Android News
Noisy Code 🗣 With Kotlin Scopes
Noisy Code 🗣 With Kotlin Scopes

By Chetan Gupta

Scopes make your code more readable? think again... You are going to encounter these scope functions namely let, run, apply, also, within every Kotlin codebase, along with all the mischievous ways developers exploit their usage from the way they were intended for. Let see how popular opinion on those ends up just as a code noise.

By ProAndroidDev -
Android News
Improving Android DataBinding with Bindables library
Improving Android DataBinding with Bindables library

By Jaewoong Eum

DataBinding is one of the most important factors for MVVM architecture. The basic concept of DataBinding is to link the view and view model via observer patterns, properties, event callbacks, etc. Linking and automating communication between the view via the bound properties or something in the view model has a lot of benefits in the MVVM architecture concept.

By ProAndroidDev -
Android News
KMM QuickStart Guide
KMM QuickStart Guide

By Mayank Kharbanda

Kotlin Multiplatform (KMP) is a code-sharing technology that allows us to use the same code for different platforms whether it’s JVM, Android, iOS, watchOS, tvOS, Web, Desktop, or WebAssembly. In this article, we will use Kotlin Multiplatform Mobile (KMM) which is a subset of KMP with the focus on providing better tooling and support for sharing code on mobile platforms i.e. Android and iOS.

By ProAndroidDev -
droidcon News

Tech Showcases,

Developer Resources &

Partners

/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/home-details/EmployerBrandingHeader
EmployerBrandingHeader
https://jobs.droidcon.com/
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/jobs-droidcon/jobs.droidcon.com
jobs.droidcon.com

Latest Android Jobs

http://www.kotlinweekly.net/
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/kotlin-weekly/Kotlin Weekly
Kotlin Weekly

Your weekly dose of Kotlin

https://proandroiddev.com/
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/pad/ProAndroidDev
ProAndroidDev

Android Tech Blogs, Case Studies and Step-by-Step Coding

/detail?content-id=/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Zalando/Zalando
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Zalando/Zalando
Zalando

Meet one of Berlin's top employers

/detail?content-id=/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Academy for App Success/Academy for App Success
/portal/rest/jcr/repository/collaboration/Groups/spaces/droidcon_hq/Documents/public/employerbranding/Academy for App Success/Academy for App Success
Academy for App Success

Google Play resources tailored for the global droidcon community

Follow us

Team droidcon

Get in touch with us

Write us an Email

 

 

Quicklinks

> Code of Conduct

> Terms and Conditions

> How to hold a conference

> FAQs

> Imprint

Droidcon is a registered trademark of Mobile Seasons GmbH Copyright © 2020. All rights reserved.

powered by Breakpoint One