HN Companion◀︎ back | HN Companion home | new | best | ask | show | jobs
Two-Stack Sliding-Window Aggregation (orlp.net)
54 points by surprisetalk 2 days ago | 6 comments


Very cool, might I suggest you rename your `pop` function to `shift` (or if you like ugly names `popleft`)? Popping windows isn't really a thing, it made me double take on the code.

I don't like `shift` as a name because it might give the impression the entire window shifts. Push increases the window size by one and pop decreases the window size by one (evicting the earliest pushed element).

In my mental model `shift` would move the entire window, it would be one push + one pop.


Ok, but shift is the commonly used name for that operation, and pop commonly does something else than what you just described. If you could come up with a better word than shift that would be fine, but pop should be the inverse of push in my opinion. If you push, then pop, you should have the same data as you started with.

Would be nice to have some example code how the class is intended to be used

This is very cool, I can see a big use case on this for streaming analytics and anomaly detection. So if I get this right, you get:

- Amortized O(1) work per window update

- O(1) aggregate queries, assuming a fixed cost merge operation

- O(W) memory for W buckets, with sketch size treated as fixed

- No need to recompute the entire window every second


That's right, and one more property I consider very important for numerical stability:

- The aggregate only depends on the value currently in the window.

Most problems caused by NaNs/infinities/catastrophic cancellation/etc in sliding windows are covered by this property.