Research notes / Time-series storage
Time-series rollups: equal summaries, different data
Published / Source review and synthetic arithmetic
An unchanged aggregate is not evidence that a time series is unchanged. A rollup can be correct for its version and still hide changes to individual samples. Preserve the stream version, window boundaries and counts alongside the summary.
On this page6 sections
A tree that remembers versions
BTrDB, presented at FAST 2016, partitions time in a copy-on-write tree. Leaves contain time-value pairs; internal nodes contain summaries. Updated paths lead to a new root, retaining earlier versions. Version annotations support locating modified ranges without scanning all raw samples. Aggregate queries and change queries serve different purposes. [1, sections 3-4]
The paper's large-range latency evaluation returns 2048 statistical records at varying resolutions, not every underlying sample. It also distinguishes the production deployment from its optimized experimental version. Those results are not a current latency guarantee; we did not reproduce them. [1, section 6.3]
Three sequences, one summary
Consider four samples at offsets 0, 1, 2 and 3 nanoseconds in the same window. Values below are invented, in arbitrary units. Every row has four observations, a sum of 20, minimum 0, maximum 10 and mean 5.
| Sequence | Values at offsets 0, 1, 2, 3 ns | Count | Mean | Peak offset |
|---|---|---|---|---|
| rising | 0, 2, 8, 10 | 4 | 5 | 3 ns |
| falling | 10, 8, 2, 0 | 4 | 5 | 0 ns |
| redistributed | 0, 4, 6, 10 | 4 | 5 | 3 ns |
The rising and falling sequences even contain the same multiset of values. Reversing their order preserves every order-insensitive statistic, but moves the peak from 3 ns to 0 ns. The redistributed sequence changes the interior values while retaining the four reported statistics. A finer query can reveal these differences; the coarse summary cannot recover information it does not encode.
This is a constructive counterexample, not a BTrDB bug. It proves that equality of these summaries is insufficient for raw-data equality. It does not estimate how often the ambiguity occurs in real telemetry.
A mean needs its count
Suppose one window contains [0] and another contains [10, 10, 10]. Their means are 0 and 10. Averaging those two means gives 5, but pooling the four samples gives 7.5: (1 * 0 + 3 * 10) / (1 + 3).
The first answer weights windows equally; the second weights observations equally. Neither is automatically a time-weighted mean of an irregularly sampled physical signal. That requires a stated interpolation or duration model. The downloadable program uses exact integer sums and counts before presenting the small decimal result.
Window identity and query completion
Python client v5.34.2 documents aligned windows of 2**pointwidth nanoseconds, half-open time bounds, boundary alignment and omission of empty windows. An absent row is not a zero-valued sample. Its annotation-version counter is distinct from the data-stream version. [2]
Our aligned fixture requests three 1024 ns windows and returns one nonempty window. Two are absent; a point exactly at the end boundary is excluded. The example requires already aligned bounds, rather than imitating every client adjustment. Integer nanoseconds are retained as decimal strings in JSON and BigInt in the calculation: adjacent timestamps at this scale can collapse to the same JavaScript Number.
The Go v5.17.1 contract widens changed ranges at coarser resolution. Treat such ranges as areas to revisit, not exact changed-sample lists. Version zero requests the latest stream version; retain the actual returned version for repeatable analysis. Receiving values is also not proof that a streamed query finished successfully: consume the value channel and check its final error. [3]
Reproduce and interpret
Download the generated evidence JSON, calculation module, tests and generator. Run node --test btrdb-rollups.test.mjs with the module beside it. No external service, package installation or dataset is required.
These deterministic examples have no sampled population or confidence interval. They do not exercise BTrDB duplicate policies, tree recovery, rounding of floating-point measurements or concurrent updates. The relevant engineering check is narrower: do not use a summary comparison as a substitute for source identity and a complete query.
Related: storage consistency and ordering, numeric precision at import boundaries and the copy-on-write definition.
Sources and scope
- Andersen and Culler. BTrDB: Optimizing Storage System Design for Timeseries Processing. FAST 2016, pp. 39-52, sections 3-4 and 6.3.
- BTrDB Python v5.34.2 documentation: aligned_windows and stream annotations.
- BTrDB Go bindings v5.17.1: Changes, LatestVersion and query-channel completion.
Sources checked 14 November 2026. Original code and synthetic examples: CC0. Source publications retain their rights. The 2016 design and later client contracts are separate versions, not one executed experiment.