Direct-linking breaks clojure.test do-report stack-depth assumptions for failure reporting.

Description

Test failure locations are being mis-reported (wrong class/line number).

Given a test ns:

The results with 1.8-RC2 (with CLJ-1854 patch included) are:

Cause: The location of the error is calculated in test.clj by constructing a Throwable in do-report and then dropping the top 1 frame (which is do-report itself) to find the user frame where the assertion failed. However, with direct linking there are now 2 frames on top of of the failure in user code, so the same (incorrect) location in test.clj is reported every time.

Approach: Change to a different strategy to filter the top frames based on content, not hard-coded depth. This strategy will work regardless of whether direct linking is involved or not.

1. Instead of constructing an exception and using it's stack trace, instead call Thread.getStackTrace() on the current thread. Create a new function that works on stack traces rather than exceptions and no longer needs a depth check.
2. Drop top frames while their class name starts with java.lang (this filters the call to java.lang.Thread.getStackTrace()) or clojure.test. The top frame will then be in the user's code.
3. Deprecated old file-and-line function (not sure if other clojure test reporting frameworks use this, despite it being private).
4. Updated tests that check that these functions work with an empty stack trace, as the JVM may elide it.

Patch: clj-1856.patch

Environment

None

Attachments

1

Activity

Show:

Alex MillerNovember 25, 2015 at 7:26 PM

If I lein test (or run from repl), I see:

Gary TrakhmanNovember 24, 2015 at 10:54 PM

I altered a local clojure build to print the stacktrace of the Throwable created by do-report, showing the additional invokeStatic frames during a repl session, showing the first user function frame is at offset 2.

This is also with the patch from http://dev.clojure.org/jira/browse/CLJ-1854 applied

Gary TrakhmanNovember 24, 2015 at 10:35 PM

'2' works in the standard case of direct-linked clojure with non-direct-linked app code. I'm exploring if there's a way to get the line info via macro-expansion of 'try-expr' and passing the line value into do-report for those cases.

Completed

Details

Assignee

Reporter

Approval

Ok

Patch

Code

Priority

Affects versions

Fix versions

Created November 24, 2015 at 10:15 PM
Updated December 6, 2015 at 2:15 AM
Resolved December 6, 2015 at 2:15 AM