This problem was detected in context of this discussion https://groups.google.com/d/msg/clojure/mIlKaOiujlo/tF71zZ2BCwAJ
A minimal version of how specs error reporting failed the users intuition there looks like this:
He used an invalid ns form
The error reported by spec:
While the error is technically true, it doesn't show the user /how/ each of the alternative options of the reported s/cat failed.
To get a better understanding why the users data is not correct, he should know precisely what spec tried and how it failed.
A good example of how this works is s/alt, where all failing alternatives are always reported to the user.
The problem has been investigated, first experimentally, then in specs code. Finally, a patch that brings error reporting like s/alts comes attached.
It has been observed that specs error reporting behavior for cat with optional branches is the following:
1. If the cat failed after one or many optional branches, the entire cat is reported as failing
2. If the cat failed after one or many optional branches /and/ a subsequent required branch, only the subsequent required branch is reported with no remarks to the alternative optional branches.
Rule 1 explains the ns example.
Rule 2 can fail the users intuition significantly worse:
The report clearly doesn't address the users intent of putting in a number. Instead he is made to believe that he should have entered a keyword.
A simple patch has been programmed that changes op-explain to have the following behaviour:
- All alternatives that have been tried in a s/cat are reported individually.
It improves the reported errors significantly because it makes clearly transparent how the users data failed the validation.
It would be even better if explain-data sorted ::s/problems by length of their :path which would push the first two unintended options to the end.
While examples can be made up where this reporting produces more noise (like defn) I believe its the right tradeoff for aforementioned reasons and:
- We programmers always ask our users for the most specific information when something went wrong - It is correct to apply the same to specs error reporting
- Custom error reporters (s/explain-out) get more data to generate narrow reports matching the users intent even better