Uploaded image for project: 'Clojure'
  1. CLJ-1741

deftype class literals and instances loaded from different classloaders when recompiling namespace

    Details

    • Approval:
      Incomplete
    • Patch:
      Code

      Description

      Scenario: Given two files:

      src/dispatch/core.clj:

      (ns dispatch.core (:require [dispatch.dispatch]))
      

      src/dispatch/dispatch.clj:

      (ns dispatch.dispatch)
      (deftype T [])
      (def t (->T))
      (println "T = (class t):" (= T (class t)))	
      

      Compile first core, then dispatch:

      java -cp src:target/classes:clojure.jar -Dclojure.compile.path=target/classes clojure.main
      user=> (compile 'dispatch.core)
      T = (class t): true
      dispatch.core
      user=> (compile 'dispatch.dispatch)
      T = (class t): false     ;; expected true
      dispatch.dispatch
      

      This scenario more commonly occurs in a leiningen project with :aot :all. Files are compiled in alphabetical order with :all. In this case, dispatch.core will be compiled first, then dispatch.dispatch.

      Cause:

      (compile 'dispatch.core)

      • transitively compiles dispatch.dispatch
      • writes .class files to compile-path (which is on the classpath)
      • assertion passes

      (compile 'dispatch.dispatch)

      • due to prior compile, load dispatch.dispatch__init is loaded via the appclassloader
      • ->T constructor will use new bytecode to instantiate a T instance - this uses appclassloader, loaded from compiled T on disk
      • however, T class literals are resolved with RT.classForName, which checks the dynamic classloader cache, so uses old runtime version of T, instead of on-disk version

      In 1.6, RT.classForName() did not check dynamic classloader cache, so loaded T from disk as with instances. This was changed in CLJ-979 Closed to support other redefinition and AOT mixing usages.

      Approaches:

      1) Compile in reverse dependency order to avoid compiling twice.

      Either swap the order of compilation in the first example or specify the order in project.clj:

      :aot [dispatch.dispatch dispatch.core]
      

      This is a short-term workaround.

      2) Move the deftype into a separate namespace from where it is used so it is not redefined on the second compile. This is another short-term workaround.

      3) Do not put compile-path on the classpath (this violates current expectations, but avoids loading dispatch__init)

      (set! *compile-path* "foo")
      (compile 'dispatch.core)
      (compile 'dispatch.dispatch)
      

      This is not easy to set up via Leiningen currently.

      4) Compile each file with an independent Clojure runtime - avoids using cached classes in DCL for class literals.

      Probably too annoying to actually do right now in Leiningen or otherwise.

      5) Make compilation non-transitive. This is in the ballpark of CLJ-322 Closed , which is another can of worms. Also possibly where we should be headed though.

      Screening: I do not believe the proposed patch is a good idea - it papers over the symptom without addressing the root cause. I think we need to re-evaluate how compilation works with regard to compile-path (#3) and transitivity ( CLJ-322 Closed ) (#5), but I think we should do this after 1.7. - Alex

      See also: CLJ-1650 Closed

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              alex+import import
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: