Source information when reading EDN

Description

I want to parse a file as EDN with full source information (line/column metadata), but tools.reader does not provide any metadata when using clojure.tools.reader.edn. I also want the source information for numeric literals and keywords, which cannot hold metadata. One approach is to parameterise the construction of the data, and allow the builder to add or discard metadata, including a place to hold metadata for keywords and numeric literals. I've attached an implementation as a patch.

Would this be suitable for inclusion into tools.reader? If not, is there a different approach that would be?

Environment

None

Activity

Show:
Andrew Childs
April 8, 2015, 3:11 AM

Hi Nicola,

  • I've attached an updated patch that addresses the first two points: removed whitespace churn and lein test-all passes.

  • For generating timing information, is there a set of benchmarks or a collection of example documents you would recommend I use?

  • This only targets the edn reader since that is what is required for my application. The same approach could be applied to the regular reader.

Thank you for considering these changes,
Andrew

Nicola Mometto
April 13, 2015, 6:45 PM

Hi Andrew,
if this is going to make into tools.reader, it has to be a change in both the edn reader and the regular clojure reader.
It's a tools.reader policy that the regular clojure reader is additive to the edn reader meaning that if a feature is available for t.r.edn then it is avaliable for t.reader too (the opposite is not guaranteed).

For timing information, I suggest you generate a bunch of random edn data using something like data.generators or test.check and benchmark it pre and post patch using criterium (https://github.com/hugoduncan/criterium)

Andrew Childs
April 30, 2015, 1:00 AM

Hi Nicola,

I've run some benchmarks, the details of which I have pushed to github. The output seems to indicate there's some performance penalty for small inputs, but for larger inputs it's less obvious or even faster. Since it's adding a layer of abstraction, I don't yet understand why it is sometimes faster for larger inputs.

I'm happy to do the work to integrate a similar abstraction for the regular clojure reader, but I'd like to know if you agree with the approach before I put in the time do so. I'd really like full line numbers in the EDN reader, but I'm not attached to any particular implementation.

Thanks,
Andrew

Nicola Mometto
May 5, 2015, 3:34 PM

This is a significant change and I don't think think I will have time to review it anytime soon, in the meantime thanks for your work, I'll let you know as soon as I get to look at it more throughly.

Nicola Mometto
March 3, 2017, 11:21 AM

Hi Andrew, Sorry for the long wait on a response, but I've decided to decline this ticket as I don't feel this is the best approach to implement this functionality on tools.reader.

Declined

Assignee

Nicola Mometto

Reporter

Andrew Childs

Labels

None

Approval

None

Patch

Code

Priority

Major