Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Handle end-of-file in object and array readers

Description

Per https://ask.clojure.org/index.php/11563/handle-end-of-file-in-object-and-array-readers

Most EOFs produce helpful error messages like JSON error (end-of-file inside string). But the object and array readers don't handle EOF and throw unhelpful exceptions.

`(clojure.data.json/read-str "{")` throws `Value out of range for char: -1` due to this line in read-key:
`(throw (Exception. (str "JSON error (non-string key in object), found " (char c) ", expected \"")))`
The `(char c)` throws because c is -1.

This is fixed by handling the -1 case in read-key:

(defn- read-key [^PushbackReader stream] (let [c (int (next-token stream))] (if (= c (codepoint \")) (let [key (read-quoted-string stream)] (if (= (codepoint \:) (int (next-token stream))) key (throw (Exception. "JSON error (missing `:` in object)")))) (codepoint-case c \} nil -1 (throw (Exception. "JSON error (end-of-file inside object)")) (throw (Exception. (str "JSON error (non-string key in object), found `" (char c) "`, expected `\"`")))))))

(clojure.data.json/read-str "{\"\":\"\"") gives an incorrect error message: “JSON error (missing entry in object)”

This is fixed by handling EOF in read-object:

(codepoint-case (int (next-token stream)) \, (recur r) \} (persistent! r) -1 (throw (Exception. "JSON error (end-of-file inside object)")) (throw (Exception. "JSON error (missing entry in object)"))))

`(clojure.data.json/read-str "[")` throws JSON error (unexpected character): ￿ (the unexpected character is (char 65535)). This is because read-array pushes -1 back onto the stream, and -1 gets pushed as 65535.

The fix is to handle EOF inside read-array:

(defn- read-array [^PushbackReader stream options] ;; Expects to be called with the head of the stream AFTER the ;; opening bracket. ;; Only handles array value. (let [c (int (next-token stream))] (codepoint-case c \] [] \, (throw (invalid-array-exception)) -1 (throw (Exception. "JSON error (end-of-file inside array)")) (do (.unread stream c) (read-array* stream options)))))

Environment

None

Attachments

1
  • 26 Feb 2022, 07:22 PM

Activity

Show:

Alex Miller December 22, 2023 at 4:37 AM

Released in data.json 2.5.0

Alex Miller December 21, 2023 at 11:36 PM

Applied a different change for next release, moved exception construction into methods, measured perf impact.

Alex Miller February 28, 2022 at 10:52 PM

Did you try my prior recommendation to move the exception construction out of the mainline code?

jshaffer2112 February 26, 2022 at 8:44 PM
Edited

Attached the patch. It does impact benchmarks a bit, but nothing I tried helped much with that.

Alex Miller February 11, 2022 at 8:29 PM

Note, need to consider perf impacts. It will be preferable to move the exception construction into a separate function to reduce bytecode size and make this more amenable to inlining.

Fixed

Details

Assignee

Reporter

Approval

Vetted

Priority

Created February 11, 2022 at 8:21 PM
Updated December 22, 2023 at 4:37 AM
Resolved December 21, 2023 at 11:36 PM