Small vectors created using vector-of may have easily-reduced memory usage

Description

None

Environment

Steps to see the memory being wasted per small vector created using core.rrb-vector’s vector-of function:

 

clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.1"} org.clojure/core.rrb-vector {:mvn/version "0.1.1"} cljol {:git/url "https://github.com/jafingerhut/cljol" :sha "bb5549e9832e73e4a9fc5dfdf695c48e797729fa"}}}'

(require '[clojure.core.rrb-vector :as fv]
'[cljol.graph :as gr]
'[ubergraph.core :as uber]
'[cljol.dig9 :as d])

;; Observe in this graph of objects how the two 'root' references
;; point at the same object.
(def cpv10 (apply vector-of :long (range 10)))
(def cpv11 (apply vector-of :long (range 11)))
(d/view [cpv10 cpv11])

;; Whereas here, the two 'root' references point at different objects,
;; each consuming 184 bytes of distinct memory for each small
;; rrb-vector.

;; This increases the memory consumed from about 152 bytes = (8
;; bytes/long) * (11 elements) + 64 bytes overhead, or 1.7x more than
;; the 88 bytes required for the 11 elements themselves, up to
;; (8 bytes/long) * (11 elements) + 248 bytes, or 3.8x more than the
;; 88 bytes.

;; Hopefully this is a straightforward enhancement that could be made
;; in core.rrb-vector.
(def rpv10 (apply fv/vector-of :long (range 10)))
(def rpv11 (apply fv/vector-of :long (range 11)))
(d/view [rpv10 rpv11])

Assignee

Unassigned

Reporter

Andy Fingerhut

Labels

None

Approval

None

Patch

None

Priority

Major
Configure