9 Comments
User's avatar
Some dude's avatar

You also mentioned HyperLogLog as a follow-up - not that anyone asked, but this is my favorite video explaining it: https://www.youtube.com/watch?v=lJYufx0bfpw

There are production-grade implementations of HyperLogLog and other sketches in Apache DataSketches (https://datasketches.apache.org/)

Expand full comment
Teiva Harsanyi's avatar

Thanks! I wanted to make a dedicated post for HyperLogLog, I'll definitely add your video as a reference.

Expand full comment
Some dude's avatar

This is referring to the CVM Algorithm, as depicted here, right?

https://www.youtube.com/shorts/47PCjlbxk74 (full vid: https://www.youtube.com/watch?v=MZI3aL1igP8)

Expand full comment
Teiva Harsanyi's avatar

Thanks, I didn't know it was called this way. Funny enough, the first time I heard about it was before 2023, when the paper was released. I'm going to reference it.

Expand full comment
Robin's avatar

Insanely beautiful

Expand full comment
Teiva Harsanyi's avatar

I loved it as well :)

Expand full comment
Gabriele Cimato's avatar

Thank you for this post! My initial thought was more oriented to UX with optimistic updates. That wouldn't have really solved the problem, but just hid it away. First time I hear about Probabilistic increments, pretty cool!

Expand full comment
Nick Ang's avatar

Hey Teiva, thanks for this insightful post. Probabilistic increment - definitely an elegant solution to a problem that general SQL databases aren't optimised for!

Expand full comment
Teiva Harsanyi's avatar

Thank you, glad you liked it.

Expand full comment