[spec] Spec'ed fn doesn't throw when called lazily

Description

Summary:

1) Given instrumented function map-f that is called lazily, e.g. as (def ls (map map-f (range))).
2) Also given instrumented function varargs-f, a varargs function. When you pass a LazySeq to varargs-f some or all elements are realized as a consequence of conforming.
3) The problem: when calling (apply varargs-f ls) some invalid calls to map-f can go unnoticed.

Repro example:

In the following code map-f is expected to throw when called with something else than a Symbol. However the call to map-f with a String slips through.

Output:

Cause:

When varargs-f's arguments are realized as a result of conforming, some calls to map-fn are made in the scope of with-instrument-disabled: https://github.com/clojure/spec.alpha/blob/f23ea614b3cb658cff0044a027cacdd76831edcf/src/main/clojure/clojure/spec/test/alpha.clj#L140

Background:

I ran into this issue when spec'ing merge-with. Spec'ed fns in some test namespaces didn't throw anymore, because this line in clojure.test has a similar problem with regards to spec as described above: https://github.com/clojure/clojure/blob/28efe345d5e995dc152a0286fb0be81443a0d9ac/src/clj/clojure/test.clj#L775

CLJ-2443.patch contains a fix, but I realize it may not be perfect yet. Therefore I provided CLJ-2443-test.patch that only contains the unit test which can be used to test an alternative solution.

Environment

None

Activity

Show:
Michiel Borkent
November 25, 2018, 1:29 PM

Updated description with CLJ-2443-test.patch

Michiel Borkent
November 25, 2018, 1:27 PM

CLJ-2443-test only contains the test, not the fix itself.

Michiel Borkent
November 24, 2018, 3:50 PM

CLJ-2443.patch fixes this problem by conforming a Cons type argument outside the scope of with-instrument-disabled.
Test provided. Conforming this patch works with speculative (https://github.com/slipset/speculative).

Assignee

Unassigned

Reporter

Michiel Borkent

Labels

Approval

Triaged

Patch

Code and Test

Priority

Major