Performance & storage tuning¶
The block store has three knobs that trade disk size against read and import speed. They interact in non-obvious ways, so this page explains what each does, gives the recommended combination, and backs it with measurements (mainnet blocks ~25.15M; exact percentages depend on the chain, but the shape holds).
TL;DR — the defaults are already tuned¶
This is the default, and it targets a small archival/RPC store: fast import, smallest practical sealed size, and read latency dominated by decode rather than decompression.
| knob | governs | default | direction |
|---|---|---|---|
--compression-level |
import speed (open-segment compression) | 1 (fastest) |
low = fast import; does not affect final size |
--seal-compression-level |
final sealed size (re-pack at seal) | 6 (better) |
higher = smaller, free on reads; best costs minutes per segment under s2 for the same size |
--block-group-size |
size ↔ random-read tradeoff | 64 |
larger = smaller, slower random single-block reads |
The two compression levels point in opposite directions because they sit on different paths: import is the hot path (keep it fast and cheap), seal runs on a background goroutine (compress hard, off the hot path). The import compression is thrown away and re-done at seal, so there is no reason to raise it.
Block grouping — --block-group-size¶
A grouped sealed segment compresses N consecutive blocks as one unit, so the compressor sees the redundancy shared across neighbours (recurring addresses, log topics, selectors) that per-block compression can't reach. The cost: to read one block the store must inflate its whole group.
This is the only knob that trades size against read latency, and the size curve saturates fast while the random-read cost keeps climbing:
| group size | sealed size vs per-record | random single-block read | sequential range scan |
|---|---|---|---|
| 1 (per-record) | — | 1.4 ms | 1.4 ms/blk |
| 16 | −14.8 % | 6 ms | 1.4 ms/blk |
| 64 | −21.1 % | 21 ms | 1.4 ms/blk |
| 128 | −22.4 % | 42 ms | 1.5 ms/blk |
| 256 | −23.0 % | 84 ms | 1.7 ms/blk |
| 1024 | −23.5 % | 373 ms | 2.1 ms/blk |
64is the default — it captures 90 % of the grouping benefit with the lowest random-read cost (21 ms) and flat range scans, so it is safe for any access pattern. Beyond 128 you gain sub-percent on disk while random reads keep doubling and range scans start to slow — not worth it.- Sequential range scans are flat up to 128: a scan decompresses each group
once and serves all of its blocks (the reader keeps a one-group cache), so the
per-block cost is RLP decode, not decompression. If your reads are overwhelmingly
range scans (the explorer pattern),
128trades ~1.3 pp more disk for a random-read cost (42 ms) those scans never pay — a reasonable bump for max size. - A random single-block read is the worst case — it inflates the whole group —
so
64keeps it at 21 ms versus128's 42 ms.
--block-group-size is locked into the manifest at creation, so choose it
before the first run. 1 disables grouping (per-block layout).
Seal compression level — --seal-compression-level¶
This sets the final size of sealed segments. The seal re-pack runs on the background sealer, off the import hot path, so it can compress as hard as you like without slowing reads or imports.
- It is free on reads. Decompression speed is independent of the compression level, for s2 and zstd alike — higher-compressed data even decodes marginally faster.
- The
besttier (10or above) is ~11 % smaller than thedefaulttier at the same group size (e.g. 67.8 vs 76.6 MB for 10 k blocks at group 64; measured under zstd). - Its only cost is seal CPU — zstd
bestis ~5× slower to encode thandefault, and every zstd tier is far slower than any s2 tier — but that is background work that never blocks reads or writes.
Levels are tiers, not 1–22
UMNode's s2 exposes three tiers (fastest / better / best) and its pure-Go
zstd four (fastest / default / better / best), so
--seal-compression-level 10 and 19 produce identical bytes (both "best").
See the manifest section
for the full mapping. Use 10; there is no point going higher.
Recommendation: 6 (better), the default. Under s2 the best tier buys
nothing measurable: on BNB mainnet around block 6.8M it re-packed a 50k-block
segment in 3.5 to 9.4 minutes (1.80 and 1.98 GB) where better took 38 s
for a segment of the same size (1.82 GB), and that CPU competed with the
executor the whole time. Pick 10 only for a zstd archival store, where the
best tier is ~11 % smaller than the default tier and sealing time is irrelevant.
Import compression level — --compression-level¶
This compresses each block as it is imported into the open (still-filling)
segment. In a grouped store that compression is transient: at seal the open
file is decompressed and re-packed at --seal-compression-level. So:
- It does not change the final store size. Measured: sealed size is identical
at every import level (only
--seal-compression-leveland--block-group-sizemove it). - It only costs import-path CPU. Higher levels slow every block on the hot path for nothing (measured under zstd; every s2 tier is cheaper still):
--compression-level |
import cost / block | open-segment size (transient) | sealed size |
|---|---|---|---|
| 1 (fastest) | 1.6 ms | 91 MB | 67.8 MB |
| 3 (default) | 2.2 ms | 88 MB | 67.8 MB |
| 7 (better) | 2.7 ms | 87 MB | 67.8 MB |
| 19 (best) | 9.8 ms | 86 MB | 67.8 MB |
Recommendation: 1 (fastest) — it maximises import/backfill throughput, and
the only thing a higher level buys is a slightly smaller open segment (one
segment's worth, re-packed away at seal). Bump it to 3 only if you want that
transient open segment a touch smaller; the difference is negligible.
Codec — --compression¶
s2 (default), zstd or snappy. s2 decodes at memory speed whatever the
tier and encodes far faster than zstd, which is why it is the default: block
import and the seal re-pack sit on the sync hot path, and under zstd the
best-tier re-pack of one 50k-block BNB segment near block 6M took 5 minutes
on an M2 laptop, halving the executor's throughput while it ran. zstd is the
choice for a cold archival store where disk matters more than CPU: it is the
smallest (snappy produced a store ~74 % larger than zstd-best in testing;
s2 lands between the two). snappy is s2's predecessor and has nothing left to
offer here. The codec is not locked — switch it any time; segments already
sealed keep theirs. See Compression codecs.
Putting it together¶
The architecture is cheap fast import → background best-tier re-pack, so the default combination is fast import, best seal, balanced group:
- Small sealed store (~−21 % vs per-record best, ~−30 % vs per-record default-tier) with the lowest random-read cost.
- Fastest import (the heavy compression is hidden on the background sealer).
- Range scans bound by RLP decode, not decompression; random single-block reads ~21 ms worst case.
Adjust only the two genuinely workload-dependent knobs: raise --block-group-size
to 128 for ~1 pp more disk if reads are overwhelmingly range scans, or drop
--seal-compression-level to 6 if
a high-rate sync outpaces the background sealer.
See CLI flags and Block store for the mechanics behind each setting.