Uploaded image for project: 'tools.reader'
  1. TRDR-54

Carriage returns in files can cause Pushback buffer overflow exception

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Completed
    • Labels:
      None

      Description

      With latest master version of tools.reader (1.3.0), some files with carriage returns that are not followed by a formfeed or newline character can cause the default pushback buffer size of 1 to be overflowed, resulting in an exception thrown during reading. The root cause is that peek-char is implemented with a read-char followed by unread for some types of reader objects, and read-char ends with a call to peek-char when such an isolated carriage return is found.

      One example to reproduce:

      user=> (require '[clojure.java.io :as io])
      nil
      user=> (require '[clojure.tools.reader.edn :as tre])
      nil
      user=> (require '[clojure.tools.reader.reader-types :as rt])
      nil
      user=> (spit "test.edn" "[a\rb]")
      nil
      user=> (def rdr (rt/indexing-push-back-reader (io/reader "test.edn")))
      #'user/rdr
      user=> (tre/read rdr)
      
      IOException Pushback buffer overflow  java.io.PushbackReader.unread (PushbackReader.java:155)
      
      user=> (def rdr (clojure.lang.LineNumberingPushbackReader. (io/reader "test.edn")))
      #'user/rdr
      user=> (clojure.edn/read rdr)
      [a b]
      

      If anyone is experiencing this problem and wants a workaround, if you can replace all carriage returns with newlines in the input file/string/reader, you should avoid this problem, although if you are using a line numbering/indexing reader, that may change the line numbers of lines in the data.

        Attachments

          Activity

            People

            • Assignee:
              bronsa Nicola Mometto
              Reporter:
              jafingerhut Andy Fingerhut
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: