Direct linking for syntax-quoted vectors

Following up from a previous post about optimizing syntax quote’s expansion, I was curious exactly why it led to a notable decrease in bytecode generated for the Clojure jar.

As of Clojure 1.12.5, a syntax quoted vector expands to (apply vector ...). This has interesting implications for direct linking, and it’s worth considering whether to emit (vec ...) instead.

It turns out most of the bytecode decrease was due to making syntax quote more compatible with direct linking, under which (vec [1 2]) and (apply vector [1 2]) compile very differently. In a nutshell, (vec [1 2]) calls vec’s function directly without dereferencing #'vec, while (apply vector [1 2]) dereferences #'vector on each call. This difference also enables leaner static initialization.

We can see the difference using Clojure and a class disassembler.

Let’s compile two functions foo and bar that use (apply vector nil) and (vec nil) respectively:

$ clj
Clojure 1.12.5
user=> (.mkdir (java.io.File. "classes"))
true
user=> (defmacro compile-direct [form]
         (binding [*compiler-options* {:direct-linking true}
                   *compile-files* true]
           (eval form)))
#'user/compile-direct
user=> (compile-direct (defn foo [] (apply vector nil)))
#'user/foo
user=> (compile-direct (defn bar [] (vec nil)))
#'user/bar

We can compare the disassembled output using these commands:

$ javap -c 'classes/user$foo.class'
$ javap -c 'classes/user$bar.class'

We’ll just zoom in on the differences from here.

First, let’s note that foo contains an extra field const__1. This will contain the #'vector var needed to compile the (apply vector ...) call, since it cannot be completely directly linked.

public final class user$foo extends clojure.lang.AFunction {
  public static final clojure.lang.Var const__1;
  public static java.lang.Object invokeStatic();
  public static {};
}
public final class user$bar extends clojure.lang.AFunction {
  public static java.lang.Object invokeStatic();
  public static {};
}

This impacts the static initializer method since foo must then resolve and initialize the var constant:

public final class user$foo extends clojure.lang.AFunction {
  public static {};
     // this.const__1 = RT.var("clojure.core", "vector")
     0: ldc           #31
     2: ldc           #33
     4: invokestatic  #39
     7: checkcast     #17
    10: putstatic     #15
    13: return
}
public final class user$bar extends clojure.lang.AFunction {
  public static {};
     0: return
}

Finally, the body of foo makes one static and one virtual (instance) method call, where bar only has a static method call.

public final class user$foo extends clojure.lang.AFunction {
  public static java.lang.Object invokeStatic();
    // return clojure.core.apply.invokeStatic(
    //   const__1.getRawRoot(),
    //   null);
    0: getstatic     #15
    3: invokevirtual #20
    6: aconst_null
    7: invokestatic  #25
   10: areturn
}
public final class user$bar extends clojure.lang.AFunction {
  public static java.lang.Object invokeStatic();
    // return clojure.core.vec(null);
    0: aconst_null
    1: invokestatic  #16
    4: areturn
}

Approximately 85 classes in Clojure’s own jar can enjoy similar improvements if we update syntax-quoted vectors to emit (vec ...) instead of (apply vector ...). Removing code from static initializers is a significant way to improve Clojure’s startup time.

The patch to do this simply deletes sixteen characters from the Clojure compiler, which is kind of neat for an optimization.

-static Symbol VECTOR=Symbol.intern("clojure.core","vector");
+static Symbol VEC=Symbol.intern("clojure.core","vec");
...
-   ret = RT.list(APPLY, VECTOR, RT.list(SEQ, ...
+   ret = RT.list(VEC, RT.list(SEQ, RT.cons(CONCAT, ...

I have proposed the change as CLJ-2962.

dissoc

Direct linking for syntax-quoted vectors