Deep Blue Lambda

In part one of this series, I wrote about some of the current obstacles to creating a user-friendly Clojure application for Android, as well as some techniques which may be useful in creating a better user experience. In this post, I will begin concentrating on Android Clojure development from the developer's point of view by examining how to integrate Clojure into Android’s build process.

I will briefly describe the typical Android compile and deploy cycle, and then show how to modify the build to integrate Clojure. Afterwards, I will note some of the ways developing for Android with Clojure differs from other Clojure development. Finally, I will provide a link to some example code that implements the ideas presented here and provide a preview for my next post.

The Android build work flow

Broadly speaking, the standard Android build uses Ant to automate the following steps:

1. Perform automated code generation based on the application’s resources and Android Interface Definition Language¹ files.
2. Compile both the generated code and the application source code to Java class files using the standard Java compiler javac.
3. Generate Dalvim VM byte codes from the Java classes and any project libraries using the dx tool provided by the Android SDK. The result is a file called classes.dex.
4. Finally, create and sign an Android package file that includes the classes.dex file and any application resources. This package can then be installed on an Android device.

The good thing about using Ant as a build tool, is that it is relatively easy to modify the build process. In fact, Android includes a number of extension points, such as -pre-compile or -post-compile. It is also possible to completely replace Android’s stock tasks with your own.

In the following section, I will show how to modify the second step above to include Clojure compilation.

Clojure and Android integration using Ant²

The following will demonstrate how to add Clojure compilation to a stock Android project. For the purposes of this exercise, I will assume that this is a fresh project created using the Android SDK and its android create project command. If you need more help with this step, check the Android Documentation.

It will be necessary to make a number of changes to the build.xml file created by Android. Be sure to make these changes in the file before the line that reads <setup />.

Adding new input directories

The following set of lines modifies where Android will look for source code such that instead of having one set of source files under src, it will look under src/java. You can then place your code under src/clojure.

<!-- Modified input directories -->
<property name="source.dir" value="src/java" />
<property name="clojure.source.dir" value="src/clojure" />
<property name="clojure.source.absolute.dir" location="${clojure.source.dir}" />

It is important to do this even if you have no Java code, as Android will copy any non-Java source files out of its input directory and add them to the Android package. As such, if you place your Clojure code where it expects to find its Java code, your source will be included in your compiled application, resulting in unneeded bloat and an unintentional release of your code.

Adding new output directories

The next bit of code allows you to direct the output of Clojure compilation to a different location than Android expects. For now, Clojure will compile to the same location as Java.

<!-- Clojure output directories -->
<property name="out.dir" value="bin"/>
<property name="clojure.out.dir" value="${out.dir}" />
<property name="clojure.out.absolute.dir"
          location="${clojure.out.dir}" />
<property name="clojure.out.classes.dir"
          value="${clojure.out.absolute.dir}/classes" />
<property name="clojure.out.classes.absolute.dir"
          location="${clojure.out.classes.dir}" />

Adding the Clojure targets

This Ant target will ensure that the output directories for Clojure exist before compilation:

<!-- Ensure that Clojure directories exist -->
<target name="-clojure-dirs">
    <mkdir dir="${clojure.out.classes.absolute.dir}" />
</target>

Finally, this next target is in charge of doing the actual Clojure compilation:

<!-- Compile Clojure namespaces -->
<target name="-clojure-compile" depends="-clojure-dirs">
    <java classname="clojure.lang.Compile"
          classpathref="android.target.classpath"
          fork="true"
          failonerror="true">
        <sysproperty key="clojure.compile.path"
                     value="${clojure.out.classes.absolute.dir}"/>
        <classpath>
            <pathelement path="${clojure.source.absolute.dir}"/>
            <path refid="jar.libs.ref"/>
            <fileset dir="${jar.libs.dir}" includes="*.jar" />
            <pathelement path="${out.classes.absolute.dir}"/>
        </classpath>
        <!-- Add your Clojure namespace here -->
        <arg value="com.sattvik.android.clojure.basic.ClojureBrowser"/>
    </java>
</target>

You will need to modify line 16 above and replace with your namespace. If you have multiple namespaces to compile, just add multiple lines, each with a different value.

Wiring Clojure to the Android build process

All that is left is to ensure that Ant will run the targets added above. In this implementation I replace Android’s compile target with a new one. This new target will execute both Android’s standard compilation and the Clojure compilation added above.

<!-- Hook Clojure compile into Android build work flow  -->
<target name="compile" depends="android_rules.compile,-clojure-compile">

Final steps

Before you compile and deploy your Clojure application to the emulator or your device, there are some things you will have to do:

1. Remove the “Hello, world!” activity generated by Android under the src directory.
2. Create the src/java and src/clojure directories.
3. Install the Clojure JAR file to the libs directory.
4. Create your new Clojure application under src/clojure.

Once you have done these steps, you can use the following commands:

• ant compile compiles your application.
• ant debug creates a development version of your program as an Android package.
• ant install installs the Android package onto an Android device or a running emulator.

Observations

In my experience, developing with Clojure for Android is a bit different from developing in other environments. For one, the build speed is quite a bit slower, which is problematic because there is no REPL that will run on the platform itself. Also, as I noted my last post, application sizes are relatively large for a mobile. However, there are ways to address some of these issues.

Build speed

Once everything is set up, you will find that compiling your application will be fairly quick. However, when it comes time to package your application and deploy it to your device, it will be much slower. In my experience, compilation takes only a few seconds, but creating the full package takes about a half minute.

The reason for this is that once the application has been compiled, the JVM bytecodes need to be transformed to Dalvik bytecodes, a memory- and CPU-intensive task. Ideally, the Clojure library could be processed once and the result could be reused. Unfortunately, Android does not support this.

No REPL

One of the great things about developing in Clojure is being able to use a REPL within your application for the purpose of interactive development. Being able to dynamically update your code without waiting for a complete compile-deploy cycle is invigorating. Unfortunately, on Android, with the stock Clojure library, this is impossible at this time.

The reason for this is that Clojure generates JVM classes, and these are incompatible with the Dalvik VM. Off-hand, I can see two possible resolutions to this:

1. Use the Android dx tool at runtime to convert Clojure’s JVM bytecode into Dalvik executables which can then be loaded.
2. Create a whole new byte-compiling back end for Clojure that uses Dalvik VM bytecodes instead of JVM bytecodes.

The primary advantage of the first approach is that it is probably the easiest and least intrusive method. In fact, I am using it to develop a Clojure REPL application I plan to release soon. Unfortunately, it is also extremely inefficient.

The second approach is really the ideal solution, but it is going to be a much more difficult task and will quite possibly be too disruptive to include as a standard part of Clojure any time soon.

Application size

If you examine the contents of an Android package built using the above steps, you may find that it includes source files from Clojure, such as clojure/core.clj. This is because the standard Clojure JAR file includes them, and Android happily copies them into your package. However, these files are useless and add nothing but dead weight. For best results, remove these files from the Clojure JAR.

Things to come

I originally intended this to be a two-part series, but it turns out that I have much more material than I originally thought. Next time, I will show how to include ProGuard into the build cycle in order to slim your application and weed out unneeded classes. I will also show how this affects the runtime behaviour of the application.

I appreciate your feedback. If you have any requests or ideas, feel free to comment below.

Example code

I have created an example application written in Clojure that provides a very basic web browser. With each new instalment of this series, I will update the repository with a version of the code that uses the new techniques. The examples are available from GitHub.

Footnotes

1. From the Android documentation: Android Inteface Definition Language is an IDL language used to generate code that enables two processes on an Android-powered device to talk using interprocess communication (IPC).
2. If you prefer working with Maven, check out android-clojure-flashlight-example.