With clojure.test, debug prn should be optional, so you can run tests with minimal output

Description

There is an unconditional prn in assert-check (https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/clojure_test.cljc#L19) that outputs the test configuration (seed, etc.) every time a defspec is used. For projects with many property tests/defspecs, this results in a large test output when tests are successful.

For tests that pass, it seems like this output is unnecessary, since failing tests always report the seed, etc. Currently, the only obvious way to suppress this message is to monkey-patch assert-check and disable prn. It would be nice to have an option to disable these messages through configuration for successful runs.

Environment

osx el capitan, leiningen 2.6.1, clojure 1.8, test.check 0.9.0

Activity

Show:
Michael Glaesemann
July 6, 2017, 11:32 PM

This patch relies upon the refactoring in TCHECK-132 and includes tests. I've suppressed the debug reporting by default and introduced report-completion to control display similar to report-shrinking. This reverses the existing behavior of displaying debug by default, but I think this is better and more consistent behavior for the common user: passing tests don't emit results by default, and you generally turn up debugging when you want to dig into an issue. I've added test-na-hook to the existing defspec tests in other test namespaces for consistency with existing behavior.

report-completion also controls the reporting of ::shrunk messages. I'm not sure if these were purposefully emitted or not, as there was no test/report defmethod implemented. They were emitted by default under clojure.test/report (but not clj.test/report, which doesn't report messages without handlers). The logic to include them under the same option is that they're both reporting the details of a run, one success, the other failure.

gfredericks
July 24, 2017, 2:23 PM

I like these changes, but I think we should keep the default as the current behavior. We can relegate changing the default behavior to the introduction of a new namespace (clojure.test.check.clojure-test-2) or a new macro (clojure.test.check.clojure-test/defspec-2), and then make a bunch of "breaking" enhancements at once, without confusing anybody.

Michael Glaesemann
November 16, 2017, 8:58 PM

As discussed, make the default the current behavior. Thanks, Gary!

gfredericks
November 20, 2017, 2:28 PM

I've applied the patch on master; thanks again!

Michael Glaesemann
November 20, 2017, 3:26 PM

Yay!

Completed

Assignee

gfredericks

Reporter

import

Labels

Approval

None

Patch

None

Priority

Minor