We're updating the issue view to help you get more done. 

deftype class literals and instances loaded from different classloaders when recompiling namespace

Description

Scenario: Given two files:

src/dispatch/core.clj:

1 (ns dispatch.core (:require [dispatch.dispatch]))

src/dispatch/dispatch.clj:

1 2 3 4 (ns dispatch.dispatch) (deftype T []) (def t (->T)) (println "T = (class t):" (= T (class t)))

Compile first core, then dispatch:

1 2 3 4 5 6 7 java -cp src:target/classes:clojure.jar -Dclojure.compile.path=target/classes clojure.main user=> (compile 'dispatch.core) T = (class t): true dispatch.core user=> (compile 'dispatch.dispatch) T = (class t): false ;; expected true dispatch.dispatch

This scenario more commonly occurs in a leiningen project with :aot :all. Files are compiled in alphabetical order with :all. In this case, dispatch.core will be compiled first, then dispatch.dispatch.

Cause:

(compile 'dispatch.core)

  • transitively compiles dispatch.dispatch

  • writes .class files to compile-path (which is on the classpath)

  • assertion passes

(compile 'dispatch.dispatch)

  • due to prior compile, load dispatch.dispatch__init is loaded via the appclassloader

  • ->T constructor will use new bytecode to instantiate a T instance - this uses appclassloader, loaded from compiled T on disk

  • however, T class literals are resolved with RT.classForName, which checks the dynamic classloader cache, so uses old runtime version of T, instead of on-disk version

In 1.6, RT.classForName() did not check dynamic classloader cache, so loaded T from disk as with instances. This was changed in CLJ-979 to support other redefinition and AOT mixing usages.

Approaches:

1) Compile in reverse dependency order to avoid compiling twice.

Either swap the order of compilation in the first example or specify the order in project.clj:

1 :aot [dispatch.dispatch dispatch.core]

This is a short-term workaround.

2) Move the deftype into a separate namespace from where it is used so it is not redefined on the second compile. This is another short-term workaround.

3) Do not put compile-path on the classpath (this violates current expectations, but avoids loading dispatch__init)

1 2 3 (set! *compile-path* "foo") (compile 'dispatch.core) (compile 'dispatch.dispatch)

This is not easy to set up via Leiningen currently.

4) Compile each file with an independent Clojure runtime - avoids using cached classes in DCL for class literals.

Probably too annoying to actually do right now in Leiningen or otherwise.

5) Make compilation non-transitive. This is in the ballpark of CLJ-322, which is another can of worms. Also possibly where we should be headed though.

Screening: I do not believe the proposed patch is a good idea - it papers over the symptom without addressing the root cause. I think we need to re-evaluate how compilation works with regard to compile-path (#3) and transitivity (CLJ-322) (#5), but I think we should do this after 1.7. - Alex

See also: CLJ-1650

Environment

None

Status

Assignee

Unassigned

Reporter

import

Approval

Incomplete

Patch

Code

Fix versions

Affects versions

Release 1.7

Priority

Major