Certain files are compiled twice, leading to "Can't redefine a constant ..." errors

Description

Repro:

Create a folder with the following files:

The contents shall be:

In this folder, call java -cp src;cljs.jar clojure.main. In the repl, enter:

If some conditions are met (see [1] and the explanations below), you'll see the compilation fail with "Can't redefine a constant at <...>".

NOTE: the easiest way to reproduce is to sort the value returned by cljs.closure/add-dependency-sources by a function of our choosing, (comp first rovides) for example. This can ensure that the last element will be a dependency which appears twice (see explanations below). However, in this case we have to use a hand-crafted classpath, which includes our patched clojurescript jar (+ its dependencies, + the src folder).

Cause:

NOTE: My understanding of the compiler internals (cljs.js-deps and cljs.closure in particular) is very superficial, but those familiar with it can probably follow along.

The journey starts at cljs.closure/add-dependency-sources, invoked in cljs.closure/build. The function takes some dependencies, converts them to a set, and adds some dependencies to it. Things to note here:

  • the :file entry in each dependency is nil [0]

  • the order of (seq (set inputs)) is "random" (or unspecified rather) [1]

  • the :source-file entry in each dependency is a java.io.File instance

  • the :source-file entry in each additional dependency from cljs.closure/find-cljs-dependencies is a java.net.URL instance

  • there are some dependencies (in the repro case at least) which were in inputs and were added as well with cljs.closure/find-cljs-dependencies (so they appear twice in the list)

The fact that some dependencies are java.io.File, some are java.net.URL instances may be the reason why some items appear multiple times in the set, but I haven't confirmed this.

Next, the computed dependencies are fed into cljs.js-deps/dependency-order, where they are passed to cljs.js-deps/build-index. This function reduces over each dependency (OUTER reduce), and reduces over each provided namespace (rovides entry, INNER reduce).

Here's the crucial part: the INNER reduce associates each namespace in rovides with the dependency, and the OUTER reduce then associates the :file entry with the dependency. But, since the :file entries are nil (see [0]), the value mapped to nil is overridden with each step in the OUTER reduce. Now, let X be a dependency which appeared twice (or at least twice, does not matter) in inputs. If X happens to be the last element of the dependencies (depends on [1]), then in the final map (when the OUTER reduce finishes) X will be mapped to nil. Therefore, in the final index, X will be present twice (once mapped to nil, once mapped to the namespace it provides).

Afterwards, these dependencies are compiled, and when the stars align (= sometimes it happens, sometimes it doesn't, both with parallel and non-parallel build) the compilation will fail with "Can't redefine a constant at <...>". This is most likely due to the fact that certain dependency (X), which has a ^:const var, was compiled twice (and some namespaces are compiled twice, this can be observed when :verbose is set to true).

I could reproduce the error on both windows (locally), and on linux (on a travis-ci server).

Environment

windows, linux

Activity

Show:
David Nolen
February 24, 2017, 8:44 PM

Please verify that does or does not resolve this problem.

Christian Romney
June 15, 2017, 9:05 PM

I was unable to reproduce this issue. I have setup a Github repo to make it easier for anyone who wants to give this a try:
https://github.com/christianromney/cljs-1921

David Nolen
June 16, 2017, 4:33 PM

Please reopen if someone can reproduce.

Assignee

Viktor Magyari

Reporter

Viktor Magyari

Labels

Approval

None

Patch

None

Priority

Minor
Configure