Block store

Downloaded blocks are persisted in a segmented store under --block-db (default data/chain), modelled on reth's static files. This page covers both the theory of the format and the practical settings that govern it.

Concepts

A block lives in one of three tiers, by how far it sits behind the chain tip:

• Tail. The most recent blocks — those within --seal-confirmations of the tip — are held in full (uncompressed RLP) in the pending key-value DB, keyed by number, so a reorg can overwrite them in place.
• Open segment. The moment a block falls out of the tail (becomes confirmed) its compressed bytes are appended to its segment's open file (NNNN.seg.open) and only a small locator (offset, length) is kept in the pending DB. So the bulky block data leaves the key-value DB as soon as it is confirmed — the DB only ever holds the recent tail plus locators for the one or two segments still filling.
• Sealed segment. When an open segment's whole aligned range is present it is sealed: an index file (NNNN.idx) is written and the data file atomically renamed to an immutable NNNN.seg, and its locators are dropped from the DB. Any block then reads in O(1) by number against the mmapped index. The one thing that ever replaces a sealed file is a receipts backfill, which re-packs it with receipts attached and swaps the new file in atomically; the one thing that ever removes one is pruning. A sealed segment can also arrive whole from outside — downloaded from a segment archive and attached after verification — and every sealed segment can be backed up to one.

A segment is a fixed-size group of --segment-size blocks (default 50,000); segment i covers [i·size, (i+1)·size − 1], and a block's segment is pure arithmetic: segment = blockNumber / segmentSize. A MANIFEST file records the parameters fixed at creation and is validated on every open.

Each stored block carries its header, body (transactions, uncles, withdrawals) and receipts (in go-ethereum's compact ReceiptForStorage form). With --receipts derive (the default when execution is on) receipts are not fetched and the inline list is empty; the executor keeps them in the receipts stream instead. With --receipts none the list is empty too. The store marks every block it stores without receipts so that a later run under --receipts fetch can fill them in — see Receipts backfill.

Monitoring

With a block store configured, UMNode logs a one-line occupancy summary every --storage-report-every (default 10s, 0 disables), broken out by the three tiers above:

Block store sealedSegments=N sealedBlocks=N sealedLast=N openBlocks=N openLast=N tailBlocks=N tailLast=N

• sealedSegments / sealedBlocks — immutable NNNN.seg files and the blocks in them (sealedSegments × segmentSize, since a segment seals only when its whole range is present).
• openBlocks — blocks flushed to open (still-filling) segments.
• tailBlocks — recent blocks still held in full in the pending DB.
• sealedLast / openLast / tailLast — the highest block number in each tier (0 when the tier is empty). They make sync progress visible at a glance: in steady forward sync tailLast is the chain tip, with openLast and sealedLast trailing it by the open-segment and seal depth.

The same numbers are logged once at startup as Block store ready, which is handy for seeing where each tier resumed after a restart.

Why segment size & grouping are locked

segment = blockNumber / segmentSize is baked into every on-disk path, and block grouping into every sealed byte. Changing either would make existing files unreadable, so both are written into the MANIFEST at creation and locked forever — UMNode refuses to start if a later run requests different values. The compression codec is not locked: every segment header names the codec its records were written with, so it may change between runs (see Compression codecs).

{
  "formatVersion": 3,
  "segmentSize": 50000,
  "codec": "s2",
  "groupSize": 64,
  "compressionLevel": 1,
  "sealCompressionLevel": 6,
  "pendingBackend": "pebble",
  "networkID": 1,
  "genesisHash": "0xd4e5…"
}

• segmentSize — locked. Blocks per segment.
• codec — not locked. s2 (default), zstd or snappy: what segments are written with from now on. Every segment header names its own codec and readers dispatch on it, so a switch applies to new segments while everything already sealed stays readable; the manifest is updated to the new value on open (logged as Block store codec changed), and so are the recorded compressionLevel / sealCompressionLevel — provenance for what is written from now on, refreshed unlogged when only a level changes.
• groupSize — locked. Blocks compressed together per unit in sealed segments (default 128); see Block grouping. Omitted (and read as per-block) for a database created with --block-group-size 1 or before grouping existed.
• pendingBackend — locked. pebble (default) or leveldb. The pending dir holds engine-specific files and the only copy of unsealed blocks, so switching engines would orphan them. (Databases created before this field existed are read as leveldb, the engine they were built with.)
• compressionLevel — not locked. Compression level (an encoder tier, see below) for block import only (every block compressed as it is flushed to the open segment). Grouped segments are re-packed at sealCompressionLevel, so this does not change final sealed size — only import speed (lower = faster). sealCompressionLevel falls back to it when 0. Retunable; affects only segments written afterward.
• sealCompressionLevel — not locked. Level for the grouping re-pack at seal (0/omitted = inherit compressionLevel); see Block grouping. It runs off the import hot path, so it can be higher than compressionLevel for free. Retunable; affects only segments sealed afterward.
• networkID, genesisHash — bind the database to one chain; a mismatch is refused.

Compression levels are encoder tiers, not libzstd's 1–22

A level selects an encoder tier, and there are only a few: S2 has three encoders, and the pure-Go klauspost/compress zstd exposes four — not the reference C library's distinct levels 1–22. Both --compression-level and --seal-compression-level are bucketed:

level value	s2 tier	zstd tier
1–2	fastest	fastest
3–5	better	default
6–9	better	better
10+	best	best

So many numbers are equivalent — e.g. 10 and 19 produce byte-identical segments (both "best"). Pick a value for the tier you want; there is no point going above 10. The startup Block store ready log prints the resolved tier, e.g. compressionLevel="1 (fastest)". (snappy ignores the level.)

Compression codecs

--compression picks what new segments are written with:

codec	default	on disk	encode	decode
s2	yes	between snappy and zstd	fast at the better tier (a 50k-block BNB segment re-packs in ~40 s); best took 3.5 to 9 minutes for the same size	memory speed, independent of the tier
zstd		smallest	slow at best: the re-pack of one 50k-block BNB segment near block 6M took 5 minutes on an M2 laptop while execution ran alongside	slower than s2
snappy		largest (≈74 % above zstd-best in testing)	fast, no levels	fast

S2 (klauspost/compress) is Snappy's successor: the same LZ77 family without entropy coding, so decoding runs at memory bandwidth whatever the level, with larger blocks and better matching than snappy. It is the default because compression sits on the sync hot path — the import compresses every block and the seal re-packs every segment — and under zstd the sealer competed with the executor for CPU and I/O.

The codec is not locked. Every segment header carries its codec id and readers dispatch on it, so switching --compression takes effect for segments written from now on while everything already sealed stays readable; the manifest's codec is updated on open (logged as Block store codec changed). An open (still-filling) segment keeps the codec its header names: blocks flushed into it after the switch are compressed with that codec, and the seal re-pack then writes the whole segment in the new one. (Version 0.2.120 got this wrong and flushed new-codec records into old-codec open files; records are still decoded by trying the other codecs when the header's fails, so such a segment reads and seals normally.) The same holds for the changeset and receipts segments under the state store (--state-changeset-compression). The segment archive treats the codec as provenance, not geometry: it publishes each segment in whatever codec it was sealed with.

The Pebble instances too. The state store (flat/, hist/), the hash index and the pending areas compress their SSTable blocks with Pebble's default Snappy format — and that encoder is S2 as well: the root go.mod replaces github.com/golang/snappy with a thin module (thirdparty/snappy) over the klauspost drop-in, which encodes in S2's Snappy-compatible mode. The bytes stay valid Snappy (every existing SSTable, go-ethereum's RLPx frames and goleveldb files included); only the encoder and the decoder get faster. --state-db-zstd still switches the history instance to zstd.

For a cold archival store where disk matters more than CPU, go back to zstd: --compression zstd --state-changeset-compression zstd (and --seal-compression-level 10 for zstd's best tier); segments already sealed as s2 stay as they are.

Converting an existing store

A store that has lived through a codec switch holds a mix: segments sealed before the switch in the old codec, later ones in the new. Reads do not care, but for one codec everywhere — a single decode path, one format in the archive, uniform tooling — run the node once with --repack-segments:

umnode --network bnb-mainnet --repack-segments

It opens the stores exactly as a normal run would (so pass the same --block-db / --state-db and codec flags), rewrites every sealed block segment — and, with --execute, every sealed changeset and receipts segment under the state store — whose header codec differs from the configured one, in that codec at the seal tier and the store's grouping, then exits. Segments are converted in parallel, one per core, and each is logged as Re-packed sealed segment with its size before and after. Converting the BNB mainnet store of October 2026 — 128 block segments (22.2 GB, through block 6.4M), 103 changeset (7.7 GB) and 103 receipts segments (3.6 GB), all zstd — to s2 took 4m17s on an 8-core M2 laptop, about 130 MB/s of compressed input; the s2 files came out 13 % (blocks), 16 % (changesets) and 53 % (receipts, which zstd's entropy coding suits best) larger — the price of memory-speed decoding everywhere. Each rewrite is built in a temp file next to the old one and swapped in by rename, so an interrupted run (Ctrl-C or a crash) leaves every segment either converted or untouched — a leftover *.seg.repack / *.tmp is swept on the next open — and re-running picks up the rest; one segment's worth of free disk is enough. For changeset and receipts segments the manifest's content hash is re-catalogued, since the archive addresses them by it; a detached segment is left alone and comes back from the archive in the codec it was published in. Open (still-filling) segments and the tail are not touched: they are sealed in the configured codec anyway. Run it with the node stopped — the stores' locks refuse a second process.

Choose --segment-size, --block-group-size and --pending-db before the first run.

On-disk segment layout (format v3)

A sealed segment is two sibling files: a data file NNNN.seg and an index file NNNN.idx. Splitting the index out of the data file lets a reader mmap the small idx — resolving a block number to its (offset, length) is then a memory read with no per-lookup syscall, and the bulky data file is touched only for the payload itself. The data file is written by streaming: confirmed records are appended in arrival order as they land (offsets aren't known up front, and reverse-fill delivers them newest-first), and the idx is written once at seal — the data section is never rewritten.

flowchart TB
    subgraph seg["Data file · NNNN.seg"]
        direction TB
        H["Header (24 bytes)<br/>magic UMSG · version · codec · firstNumber"]
        DATA["Data section<br/>framed records in arrival order, each:<br/>num · payloadLen · crc32 · compressed RLP"]
    end
    subgraph idxf["Index file · NNNN.idx (mmapped)"]
        direction TB
        IH["Header (40 bytes)<br/>magic UMIX · version · firstNumber · count · entriesCRC<br/>dataLen · dataTailCRC — the data file's identity"]
        IE["Entries · count × (payloadOffset · payloadLen)<br/>sorted by block number"]
    end
    H --> DATA
    IH --> IE

For block n a reader indexes entry i = n − firstNumber in the mmapped idx to get its payload's (offset, length), then reads that record from the data file — no scan, no per-lookup syscall, and no per-reader heap index. While a segment is still open (unsealed), the same (offset, length) lives as a locator in the pending DB, so open-segment blocks read in O(1) too.

The per-record num · len · crc32 frame makes the data section self-describing: an open file interrupted by a crash is rescanned on the next open and its locators rebuilt, with the crc bounding where intact records end (a torn append fails the check and is trimmed). Crucially the idx is a pure derivative of the data file — always rebuildable by rescanning — so it is never the source of truth. The idx header carries a CRC over its entries and — since a sealed data file can be replaced by a receipts backfill — the identity of the data file it was built for: its byte length and the crc32 of its last 4 KiB. The store verifies both as it reconciles each sealed segment on open and rebuilds any idx that is lost, wrong-sized, fails its CRC, or belongs to a previous data file (an older idx without the identity is spot-checked against the frames its first and last entries point at). Reads then trust the idx, just as they trust the data file's per-record CRC rather than re-checking it.

Data is compressed per record; the index is not

A record's payload is compressed with the configured codec (s2 by default, or zstd / snappy) — only the payload; the segment header and each record's num · len · crc32 frame stay plaintext. A record is one block in the per-record format, or a whole group of blocks under block grouping — either way the index points at a record, and a single transaction's bytes are never independently decompressable, which is why a future transaction index points at a (blockNumber, txIndex) rather than a byte range (see Hash indexes).

The index is stored raw — fixed 12-byte entries, uncompressed — so entry i sits at a known offset and the mmapped lookup is a direct memory read. Compressing it would destroy that fixed-offset random access (the whole point of a separate, mmappable idx), and the index is tiny next to the compressed block data anyway.

Block grouping

By default a sealed segment does not compress each block on its own. It packs --block-group-size consecutive blocks (default 64) into one compression unit — a group — so the compressor sees the redundancy shared across neighbouring blocks (recurring contract addresses, log topics, function selectors) that per-block compression cannot reach. On real mainnet-scale data this shrinks sealed segments by ≈14 % at the default zstd tier and ≈24 % at the best tier (--seal-compression-level 10; measured under zstd — grouping works the same way under s2) — on top of everything per-block compression already saves.

The cost is paid on read: to return one block the store must inflate its whole group. That is the right trade for this store because the dominant read pattern is sequential range scans (an explorer walking a block range), where each group is decompressed once and serves all of its blocks — so the per-block cost is unchanged. A small one-group cache in each sealed-segment reader makes that automatic: the lookups for a group's blocks share a single decompression instead of re-inflating it per block. An isolated random single-block read is the worst case (it inflates the whole group to use one); it stays fast in absolute terms and the cache absorbs repeats. See Performance for the size-versus-read curve and how to pick the group size.

Produced at seal, off the hot path

Grouping changes only the seal step. Blocks are still written to the open file per-record as they confirm — so the write path, the locator model, the tail and crash recovery are all unchanged, and import/backfill speed is unaffected. When a segment's range is complete, seal re-packs it: it reads each block back from the per-record open file, batches groupSize of them, compresses each group together, and streams the groups into a fresh data file — holding only one group in memory at a time.

The re-pack runs on a background goroutine, off the store lock, so it never blocks reads or writes. The completed segment stays in the open tier — readable via its open file — until the re-pack finishes and the store flips it to sealed under a momentary lock; the brief open→sealed handoff is the same race a per-record seal already handles (a reader whose open-file read loses to the seal falls through to the sealed file). A Put that completes a segment returns at once rather than stalling on the re-pack: in a best-tier seal of a 4096-block segment that took ~43 s, the store served ~24 000 reads with a worst-case latency of ~31 ms — versus freezing for the whole 43 s if it ran inline. The progress is logged at --seal-progress-every (default 30s, 0 silences it). At reconcile on open a leftover (e.g. shutdown-postponed) seal is queued for that same background sealer rather than run inline, so Open returns promptly and the re-seal finishes in the background while the store already serves traffic.

Because the re-pack is off the hot path, it has its own compression level, --seal-compression-level (default 6 = the better tier; 0 inherits --compression-level). Raising it compresses sealed segments harder without touching the per-block import path — the open file is still written at the fast --compression-level. The seal level is where the sealed size is decided: the default tier gives ≈14 % grouped savings and the best tier ≈24 %, with block import and read latency unchanged. Measured: higher seal compression is even marginally faster to read (decode speed is independent of the compression level, for s2 and zstd alike), so the only cost is more background CPU per segment at seal time — under s2's default better tier, well under a minute per segment. See Performance.

The open file is left untouched until the grouped .seg is durably renamed into place, so an interrupted re-pack is simply retried on the next open from the open file — the same crash contract as a per-record seal. A half-written NNNN.seg.repack temp is swept on reconcile. On shutdown Close postpones any in-flight or queued seal rather than blocking on it — it aborts the re-pack so shutdown stays fast. The next open's reconcile then queues that complete open segment onto the background sealer rather than re-packing it inline, so Open does not block on the re-pack either; the segment is readable via its open file and the re-seal lands shortly after startup. So both shutdown and startup stay prompt across a long best-tier re-pack — the seal is simply deferred and finished in the background at the next start.

On-disk layout (format v4)

A grouped segment reuses the per-record frame — num · payloadLen · crc32 · payload — but each frame is a whole group: num is the group's first block and payload is the compressed group. The decompressed group payload is self-describing, so a reader splits the blocks back out with no external metadata:

count · len₀ · len₁ · … · len_{c-1} · rlp₀ · rlp₁ · … · rlp_{c-1}

The index keeps its shape — one 12-byte (offset, length) entry per block — but every block in a group carries its group frame's locator. So a lookup resolves to the group, which is decompressed once and sliced: groups are aligned to groupSize within the segment, so block n is at position (n − firstNumber) mod groupSize of the group starting at ((n − firstNumber) / groupSize) · groupSize. The last group of a segment is short when groupSize does not divide segmentSize. As with per-record segments, the idx is a pure derivative — rebuilt by rescanning the group frames if it is ever lost or fails its CRC.

A grouped data file carries version 4 in its header (per-record is version 3), with the group size in the header too. Reads dispatch on that per-file version, so the two formats coexist in one store and a database created before grouping (or with --block-group-size 1) keeps working unchanged — its per-record segments are read exactly as before. An older build, which only knows version 3, refuses a version-4 file rather than misread it.

The flag

--block-group-size (default 64) sets blocks-per-group and is locked in the manifest at creation, like --segment-size. Set it to 1 to disable grouping and store one block per record (the pre-grouping layout). Because it is locked, the value is read only when the database is first created; on reopen the manifest is authoritative — a per-record database silently ignores a grouping flag (its segments are per-record regardless), while a grouped database refuses a different size rather than corrupt its bytes.

Opening a store created with a different group size

Because the group size is locked, a store created with a group size other than the current default 64 must be reopened with the matching --block-group-size (it errors otherwise: "block group size is immutable: database was created with 128, config requests 64"). A per-record store needs no flag.

Writing & sealing

The store accepts out-of-order writes — exactly what the reverse-fill sync needs. Where a block lands depends only on its depth behind the tip.

flowchart TD
    PUT["Put(block n)"] --> TIP[Advance tip = max block seen]
    TIP --> SEALED{Segment<br/>already sealed?<br/>or already flushed?}
    SEALED -->|yes| NOOP[No-op]
    SEALED -->|no| CONF{tip − n ≥<br/>seal-confirmations?}
    CONF -->|no, still recent| TAIL[Store full block in the tail]
    CONF -->|yes, confirmed| FLUSH[Append to open segment file<br/>+ record locator]
    TIP --> EVICT[Evict tail blocks the tip<br/>just buried → flush them too]
    FLUSH --> FULL{Segment now<br/>complete?}
    EVICT --> FULL
    FULL -->|no| DONE[Done · queryable immediately]
    FULL -->|yes| SEAL[Seal: build index from locators,<br/>write NNNN.idx, rename → .seg,<br/>drop locators]

Because a block only gets flushed (and a locator) once it is confirmed, a segment reaching segmentSize locators is by construction fully confirmed — so "segment complete" is the only seal trigger; the confirmation depth is enforced upstream, at the tail boundary.

The eviction scan starts at a tracked tail floor — the lowest block that can still be in the tail — rather than at the start of the tail's key range. The pending DB keeps a point tombstone for every evicted block until its next compaction, and an iterator has to step over each one, so a scan from the start of the range cost more with every block ever imported (on a BNB fill the import of a 4096-block chunk grew from 14 s to over a minute between hourly compactions before this). The floor is lowered by any tail write below it and raised past everything an eviction flushed, so a Put costs what it evicts, nothing more.

Forward backfill skips the tail altogether for blocks that cannot reorg: when a chunk's top block sits at least --seal-confirmations below the latest chain head the node has seen, every block of the chunk is appended straight to its open segment (PutConfirmed) — no full-block write into the pending DB, no eviction later, no tombstone and none of the compaction those feed. Presence ("is block n already stored?") is answered from memory on every path: a per-open-segment bitmap and the set of live tail blocks, both rebuilt from the files and the pending DB on open, so neither Put nor Has reads Pebble.

Sealing is crash-safe: the index file is written and fsynced first, then the data file is atomically renamed to .seg — so a present .seg always implies a present .idx, and even if the idx is lost it is rebuilt from the data file. The recovery cases are covered under Self-healing.

With block grouping on (the default), seal additionally re-packs the open file into groups as it writes the new data file, rather than adopting the open file as-is. The open file stays the source of truth until the grouped .seg is durably in place, so the crash contract is identical.

Put is idempotent: re-putting a block already flushed or sealed is a no-op; re-putting a tail block overwrites it, as a reorg would.

Reading

flowchart LR
    GET["Get(number)"] --> RF{Segment sealed?}
    RF -->|yes| O1[Read sealed file · O(1)]
    RF -->|no| RT{In the tail?}
    RT -->|yes| O2[Decode full block from DB]
    RT -->|no| RL{Locator in DB?}
    RL -->|yes| O3[Read open file at offset · O(1)]
    RL -->|no / read failed| RC[Re-check sealed<br/>in case of a racing seal]

Reads route sealed → tail → open segment. The final sealed re-check guards the race where a concurrent seal moves a block out of its open file (closing it) between the lookups — the reader simply falls through to the now-sealed file.

Receipts backfill

A block imported under --receipts derive or none is stored without receipts. The store keeps track of every such block so that a later run under --receipts fetch can fill them in retroactively:

• While a block is unsealed (tail or open segment) a one-byte marker keyed by its number lives in the pending DB; a fill, or a reorg overwrite that carries receipts, clears it.
• At seal the segment's markers are folded into one bitmap keyed by the segment, so the pending DB stays lean — one key per sealed segment, not one per block. debug_receiptsStatus and debug_status report the totals.

The fill itself, driven by the receipts filler worker (debug_fillReceipts), takes the cheapest route per tier:

• A tail block is overwritten in place, as a reorg would.
• An open-segment block is re-appended to the open file with its receipts and its locator moved to the new record. The stale record stays behind until seal, which packs only the current records; the open-time rescan keeps the last record of a number, so a crash between the append and the locator update is harmless.
• A sealed segment is rewritten: a new data file is built off-lock (grouped or per-record, as its header says) with the receipts attached and a new index written for it; then, under the store lock, the data file is renamed over the old one and the index after it, and the segment's bitmap is cleared for the blocks filled. Readers of the old file are retired after a grace period, so an in-flight read completes on the file it opened. The worker stages a sealed segment's receipts in a bounded buffer and rewrites once per buffer-full, so a segment costs one rewrite in the common case.

Receipts never enter the store unchecked: the fetcher validates every list against its header's receiptsRoot before handing it over, and a fill is refused when the receipt count does not match the block's transactions. Because the index now describes a data file that can be replaced, the idx header carries that file's identity (see On-disk segment layout), so a crash between the two renames is healed on the next open.

Pruning

By default the store is an archive: nothing is ever deleted. --block-history N turns it into a rolling window of the most recent N blocks. A background pruner re-evaluates the floor every minute and, when it has risen, asks the store to drop everything below it — a whole sealed segment at a time, as a file delete, the same operation the changeset store performs for state history. The open segment and the tail are never touched, so the effective tail is always at least the open segment plus --seal-confirmations; a segment straddling the floor stays whole.

Dropping a segment retires everything that described it, so the rest of the store does not quietly keep growing while the data shrinks:

• its blocks' hash-index entries are walked and deleted before the files go (the index is keyed by hash, not number, so this is the only way to reclaim them);
• its missing-receipts bitmap and the blocks' share of the transaction total are taken off the store's counts;
• its cached reader is retired after a grace period, so an in-flight read completes on the file it opened.

The floor lives in the MANIFEST (prunedBelow) and is written before any file is removed, so a crash mid-prune leaves only leftover files that the next open finishes dropping. Blocks below the floor read as absent (null for the eth_ methods; the debug_ methods say pruned), Put refuses them, and backfill treats the floor as its lower bound — and, on a fresh store, never fetches below the window in the first place (see The retention window). debug_status reports the floor as blocks.prunedBelow.

Growing the window. Pruning is not final. A node restarted with a larger --block-history (or all, or a lower --backfill-from) has a window that reaches below its floor: the backfill then lowers the floor to the window and fetches the history back — downward from the former floor, so every block chains up to a stored, verified one; whole segments come from the segment archive when it has them, each verified against the stored block above, the rest from peers. Pruned state history and receipts come back the same way, from the archive only (see there).

Extra ranges. --extra-archive-blocks A:B,… keeps block ranges outside the window: a sealed segment sharing a block with a range is never dropped — the floor passes over it — and Put takes a block inside a range, or the one just above it (the anchor its fill chains up to), below the floor. The ranges are not recorded in the MANIFEST: they are named at open on each start, and the sweep that finishes an interrupted prune on open spares the ones named — so a start without the flag drops their segments like any pruned history. The backfill fills the ranges once the window is filled; debug_status lists them as blocks.extraRanges, each with how far its fill has come.

The master retention. State and receipts for a pruned block are useless, so --state-history and --receipts-history are capped to --block-history: all or a wider window shrinks to it (logged at start), a narrower one stays. The three floors therefore line up, with blocks the outermost.

Never ahead of the executor. With --execute on, a block is dropped only once it has been executed: the floor is bounded by the executed head, so a node still catching up keeps every block until execution arrives, and the pruner starts trimming once execution has caught up. The flip side is one-way: after pruning, a state rebuild from genesis is impossible (debug_rebuildState says so; an unwind to a block at or above the floor still works), and a node whose state is behind the floor — execution turned on later than the floor, or a state directory deleted — cannot execute at all. It logs an error at start and parks; the remedy is a full block store.

Never ahead of the archive. With --backup, a sealed segment the archive does not hold yet is never dropped: the uploader installs a prune guard on the store, and the floor stops at the first block of the lowest sealed segment the guard has not cleared — decided under the store's lock before the floor is recorded, so a restart (which finishes an interrupted prune below the recorded floor without asking) cannot drop it either. The floor catches up as the uploads land; a slow or unreachable archive holds pruning back rather than lose a segment. The same guard keeps the state-history and receipts segments. See Pruning waits for the backup.

Attaching sealed segments

Because a sealed segment is self-contained, one produced by another node — the segment archive, a backup — can be adopted whole instead of fetching its blocks from peers, in one of two ways:

• AttachSealed takes a data file staged under segments/incoming/ (the store's own filesystem, so the final step is an atomic rename) whose header carries this store's codec and group size and the segment's first block — a file that is exactly what this database would have sealed. Its index is built from the self-describing frames (an archive carries none).
• AttachBlocks builds the segment in this store's layout from a stream of block records — the way a segment of another geometry (other segment size, grouping or codec) is converted. The records are the same in every layout; OpenSealedFile reads any sealed data file on its own, so an archive's segments are read back block by block and re-packed into the local data file and index, compressed at the local seal level.

Either way, every block is verified before anything changes:

• it must decode, carry its own number, link to the block before it, and match the roots in its header — transactions, uncles, withdrawals and, when stored, receipts — the checks the fetcher makes on every block a peer serves (the root derivations run in parallel across cores);
• the segment must link to the chain the store holds: block 0 to the store's genesis, any other segment to the stored block just below or just above it. A segment that links to nothing stored is refused.

Only a segment the store holds nothing of can be attached — not sealed, not sealing, no block of it in the open file or the tail — and only one whose last block is at least --seal-confirmations below the chain head the caller knows of, so the recent, still-mutable top of the chain never lands in an immutable file. Attaching indexes every block's hashes, adds its transactions to the total and records which blocks lack receipts (the archiving node may have run --receipts derive), so the segment then behaves exactly like one sealed locally — including for the receipts backfill. Bookkeeping is written before the files are put in place, so a crash at any point leaves either nothing (a half-built file is swept like any seal temp) or a complete sealed segment that the next open reconciles like any other.

Self-healing on open

On open, the store reconciles in-memory state from disk and repairs what an interrupted seal or append can leave behind:

• A torn open file (crash mid-append) → rescanned frame by frame; the first record that fails its crc bounds the data, and anything past it is trimmed. The file is the source of truth, so the segment's locators are rebuilt from it.
• A complete but unsealed open file (all blocks present, seal not yet run — including a seal that crashed before the rename, or one a shutdown postponed) → re-sealed, which writes (or rewrites) its .idx: a per-record segment inline, a grouped one queued for the background sealer so Open never blocks on the re-pack.
• A sealed segment whose index is missing, the wrong size, fails its entries CRC, or describes a previous data file (a receipts backfill rewrite interrupted between its two renames; the idx header records the data file's length and tail crc) → the idx is rebuilt from the data file, using the builder its own header version selects (per-record or grouped). A leftover NNNN.idx.tmp (an interrupted idx write), a NNNN.seg.repack (an interrupted grouped re-pack or receipts rewrite) and an orphan .idx (no sealed segment) are dropped.
• Orphan keys — tail or locator entries for an already-sealed segment, or a tail entry duplicating a flushed block → dropped.

Reconcile also recomputes the tip from disk — the max block across the tail, open files and sealed segments — so the tail boundary behaves identically across restarts. This runs single-threaded before the store is handed out, so a restart always lands in a consistent state.

The tail and reorgs

Both the open-segment append and the seal are one-way: once a block's compressed bytes are appended to a segment file they are not rewritten. So a block is not flushed the instant it arrives — it stays in the mutable tail (full, in the pending DB, overwritable) until the tip has advanced --seal-confirmations blocks past it:

flush block n  ⟺  tip − n ≥ sealConfirmations

A reorg shallower than that depth just overwrites the affected numbers in the tail. Only blocks buried beyond it — past any realistic reorg — leave the tail for a segment file. A segment seals once all of its blocks have been flushed, which is exactly when its last (highest) block clears the tail; the old whole-segment "seal lag" falls out of this per-block rule.

• The default is 256, comfortably beyond mainnet's reorg depth. Set --seal-confirmations 0 to flush (and seal) as soon as blocks arrive — the tail is then empty and every block is compressed into its segment file immediately.
• This is a runtime policy, not locked: it only governs when a block leaves the tail, never the bytes on disk, so it can change between runs.
• The tail holds at most ~sealConfirmations blocks. With reverse-fill the tip is established early (the head arrives first), so blocks below it arrive already confirmed and stream straight into segment files, bypassing the tail entirely; the tail only fills near the live head.

The realtime worker is the consumer of this mechanism. It overwrites tail blocks in place via Put to apply a reorg, uses RewindTail to drop blocks orphaned above a shorter new head, and refuses to act when a reorg's common ancestor falls below the tail — RewindTail returns ErrDeepReorg rather than touch confirmed or sealed data, and the worker halts.

Pending area backend

The pending area is a key-value database; the engine is selectable with --pending-db and locked in the manifest at creation:

• pebble (default) — the same LSM engine go-ethereum uses; actively maintained and already in our dependency tree.
• leveldb — goleveldb, the pure-Go engine the discovery and inventory databases use.

The backend is locked because the pending dir holds engine-specific on-disk files and the only copy of tail blocks — silently switching engines would orphan them (open-segment locators could be rebuilt from the files, but the tail could not). Either way the store sees the same small interface (point get/put/has, batched delete, ordered range scan, compaction); the segment file format is identical regardless of backend.

Compaction

The pending DB churns constantly — tail blocks are deleted as they flush, and a seal deletes a whole segment's worth of locators at once — and an LSM engine reclaims that space lazily, leaving tombstones until a compaction rewrites the affected files. UMNode runs an explicit compaction every --compaction-interval (default 1h, 0 disables) on a background goroutine, covering both the pending block area and the peer inventory (whose records are overwritten on every reconnect). Both engines compact online, so reads and writes continue throughout; Close stops the goroutine and waits for any in-flight compaction before closing the database.

Knowing the segment grid in advance

Because the mapping is pure arithmetic with a fixed segmentSize, the entire segment partition is determined by the head height H:

• Number of segments: H / segmentSize + 1 (indices 0 … H/size).
• All segments below the last are full and will seal — except any so close to the head that they fall within --seal-confirmations (typically just the one below it). The segment containing the head stays partial in pending until its full range arrives.

Knowing the grid is not the same as knowing what's filled: with out-of-order fill, a segment seals only when every block in its range is present.

Hash indexes (planned)

The store currently supports lookup by block number only. Lookup by hash is planned as secondary indexes in a follow-up release. The intended design:

• Index values are block numbers, not byte locations. A sealed segment already maps blockNumber → offset, length, and segment = number / size is arithmetic, so the number alone resolves the bytes. Storing (segment, offset, length) would duplicate the offset table and couple the index to the on-disk format.
  • block hash → blockNumber
  • tx hash → (blockNumber, txIndex) — not a byte range, because compression is per-block-record; a tx read decompresses its whole block and selects Body.Transactions[txIndex].
• Lives in its own goleveldb (e.g. indexes/), separate from the block bytes, so it never touches the immutable segment format or the manifest. Additive — no migration.
• A pure derivative of segments + pending, so it can always be dropped and rebuilt by scanning; a version marker lets reconcile rebuild on mismatch.
• The tx hash index is large (one entry per transaction), so it is intended to be opt-in behind a flag (cf. reth's transaction-lookup index).

See Configuration and CLI flags for the settings that govern the store.