Loading DEX code over the network

Today we are going to see how code can be downloaded and executed over the network. This technique is useful if you want to update parts of an app without rolling out a complete update. In the code sample, we will fetch two dex files, load them dynamically, and then execute their code. Along the way, we will explain how class loading works under the hood.

This post has actually two parts: a first part about Java class loading in general and a second part about our Android example. Let’s start talking about how Java loads code. All code in Java is loaded dynamically. Every time some code access a class in some way, such as accessing one of its static fields or creating a new instance, if the accessed class is not loaded yet, the VM will load it and link it, so it can be used. This is called class loading and in Java it is performed by a ClassLoader.

Java part: Class loading basics

Class loading is actually split in three different phases: loading, linking and initializiation. Loading a class means locating its .class file, for example in a jar or over the network, and reading its binary content in some VM data structure. Linking takes the VM data structure, verifies it for errors, and merges it into the state of the VM, i.e. the class is visible to other code. Before executing though, the class needs to be initialized: this is where the VM runs static initializer blocks and set defaults values for the static fields. This phase uses a special VM lock, one per class or interface, and is what makes singletons thread-safe (see Effective Java Item 3).

Class loader phases and delegation scheme

In this post, we will use “class loading” for all the three phases together. There are two things you still need to know for loading your code dynamically:

1. ClassLoaders (should) follow the delegation scheme.
2. Each class is identified by its name and its class loader.

The delegation scheme is a contract, often breached, of a ClassLoader. It simply tells that when a class loader tries to load a class, it should first try to delegate the loading to its parent. If the parent cannot load the class, the child will try it. In the Android code below, we will use a DexClassLoader which takes a parent ClassLoader in the constructor. The root of class loaders is called bootstrap class loader and it loads fundamental classes like String or Thread from rt.jar.

In the VM each class is identified by its (complete) name and its class loader. This means that if we load a class twice, changing its class loader, then VM will see two completely different classes. If you have ever seen an exception “ClassCastException: cannot cast class X to class X“, then that was a class loader problem. We won’t go into detail of all the pitfall of class loaders, if you want more, leave a comment and I’ll write a new post on it.

Android part: Loading dex over the network

Let’s discuss now the Android specific code sample: our goal is to download a dex file over the network and load it in the VM. Remember that Android does not work with normal Java byte code but rather uses its own format called dex (which by the way it’s not strictly byte
code as instructions are longer than a byte).

Before starting, you need:

1. local.properties in the project root with sdk.dir set (just import the project in Android Studio)
2. To set the path for the dx command in the variable dxCommand in dexs/build.gradle

Alternatively to 2. you can add dx to the PATH in your environment, see also README.md for more detail. After step 1. and 2., you can build the project from the root directory with:

Finally, install the app and run the server with:

The UI of the app displays two buttons and a text field. First you need to insert host and port of the machine where the server is running into the text field. The default will work if you are running Genymotion and the server on the same machine.

A screenshot of the app

The two buttons trigger each a download, load a class from the downloaded dex file into the VM, and then cache an instance of such classes in the activity. The logic for those classes is quite simple: the first button will swap the red and green channels of the background of the text view. The second button will swap the red and blue channels. (I wrote another post about channels and in particular the alpha channel and blending modes here).

The code should be straightforward. A “Colorizer” is an interface that encapsulates an algorithm working on a color, in our case swapping some of its channels. The dex files are downloaded in ColorizerDownloader (straightforward) and linked in ColorizerLinker:

As discussed above the class loader needs a parent for its delegation scheme, here taken from Context (see ColorizerLoader). It also takes a folder in the internal storage which it uses as its code cache. The internal storage is used for security reasons, so that other apps cannot write on it and thus inject malicious code in our app. The null parameter is for native libraries, not used here.

As you see the most important method is DexClassLoader.loadClass(String), a method inherited from ClassLoader. This methods performs loading, linking and initialization of the class indicated by the string parameter. This must be a full class name, for example it.gilvegliach.learning.networkdexloading.OneTwoSwapperColorizer. From the code, you might think that, after the loading is finished, the class loader can be garbage collected. This is actually not the case. Each object contains a reference to its class (through getClass()) and each class contains a referent to its class loader (through getClassLoader()). Therefore the class loader can be garbage collected only after we release all our Colorizers, which we don’t ever do in this app.

As you see the basics of class loaders are not complicated. This post though is just a quick introduction. There are plenty of pitfalls such as problems arising from inverting the delegation scheme, security issues and signing, class loader leaks, etc. If you want to know more, leave a comment below with what interests you and I’ll try to write about it.

  • Vlado24

    Hello, I’ve checked your code on Github and I have some questions.
    1) I’m trying to create class in another project in IntelliJ Idea, then I javac this file and try to dx, then upload and download via Retrofit – have you tried it? Is it works?
    2) I can’t build your project, can’t you make another one without server and dx and maybe downloading from server via Retrofit pls? 🙂 It help me a lot, Thx!