-
Notifications
You must be signed in to change notification settings - Fork 378
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: significantly improve the memory usage of histogram #610
base: master
Are you sure you want to change the base?
Conversation
This is the reason |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No immediate issues jumps out when looking through this 👍 Do you have any umbers or graph to confirm this helps things?
Co-authored-by: Simen Bekkhus <[email protected]>
No systematic results. I will try get some. |
My tests show a memory saving of only 5% 😢 |
How many buckets total, and how many with values, did you test and get 5% savings? |
@zbjornson @SimenB I tested one histogram with the default buckets:
I got roughly the same results in all cases, peaking at a saving of 5%. Generally speaking (in percentages):
|
Significantly reduce the amount of memory used by histograms, especially for high cardinality/lots of buckets:
bucketValues
object with prototype of zero values instead of copying. This way counter per bucket is only allocated when its value is greater than zero (first time it's incremented).bucketExemplars
(when using exemplars) instead of pre-filling with nulls.Additional optimizations:
valueFromMap
into hash only when it's allocated (don't reinsert every time)bucketExemplars
. Instead reuse previous lookupNotes:
Object.freeze(this.bucketValues)
because it causes += 1 to fail, even though the value in the prototype isn't actually changed. (This feels like a JS or v8 bug)hasOwnProperty
to check for bucket existence