Reduce time spent creating persistent data structures
Tests (see 0001-Playing-with-mutable-collections.patch, where normal, mutable java ArrayLists and HashMaps are used) indicate that a significant amount of time is spent creating the collections needed to hold the parsed json.
Current master (d744fb19ff5528320b51316b8d5364049d34797d)
With 0001-Benching-with-non-pds.patch applied:
Explore different ways of efficiently producing persistent data structures when parsing json. One such solution would be to back fly-weight maps and vectors with normal java collections.
I don’t disagree with the purpose. I do disagree with the bottom line comparison benchmark however.
You really want to hit the nail in terms of speed ?
Look at this.
, the goal here is not to change the behavior to mutable, but to explore the costs. There are a lot of potential options in getting from reading values off a stream to a persistent collection.
Well, the goal being to return a persistent structure, that ‘extra’ time is worth it. Now if there’s a speed improvement backing parsing with Java structures and then returning a persistent structure, great.
Transient structures are already speeding things up. But if the benchmark Java structures + conversion to persistent structures is not an improvement, I can’t see how is this useful from Clojure. 😬
Converting from Json to persistent structures and vice-versa to me has more value in the rest of my code. The conversion process itself to/from Json is a very little piece of the whole processing done with the persistent structure.
I am aware of the fact that this patch does not return a PDS. The patch is more a “proof” that creating the PDS takes time.
Your test doesn’t return a persistent structure if I read carefully ? (Or my eyes are really bad that’s possible ). Or it’s not shown in the patch ?
If not then you should add the conversion step of the result in your benchmark. When I decode JSON in Clojure. I expect persistent structures, not Java ones.