Hash indexing¶

UMNode is block-oriented: the block store is keyed by block number, which is all the sync workers and eth_getBlockByNumber need. Looking a block or transaction up by hash needs a separate index, which this page describes. It backs eth_getBlockByHash and eth_getTransactionByHash.

It is on by default (--index) and lives in its own database under <block-db>/index/, kept out of the hot pending DB.

What it stores¶

Two mappings, both pointing at a block number — never a physical offset, so the index is immune to reseal and compaction (a number is resolved to a segment and offset through the segment's existing .idx):

blockHash → number
txHash → (number, txIndex)

A lookup resolves the number with the normal Get(number) read path, then re-checks the full hash before returning — so a wrong candidate is never returned (see truncated keys).

When blocks are indexed, and the tail¶

A block is indexed the moment it is confirmed — flushed out of the reorg window (--seal-confirmations) into a segment. Confirmed blocks are immutable, so the on-disk index never holds a stale, reorged-out entry.

The most recent blocks — those still in the mutable tail — are mirrored in small in-memory hash maps instead, rebuilt on open and kept in sync as blocks enter the tail, get confirmed, or are dropped by a reorg. A by-hash lookup therefore checks the in-memory tail first (recent blocks), then the on-disk index (everything older).

flowchart LR
    Q["eth_getBlockByHash / …ByHash"] --> T{in the tail?}
    T -->|yes| TM["in-memory tail map → number"]
    T -->|no| IX["on-disk index → number(s)"]
    TM --> G["Get(number) + verify full hash"]
    IX --> G
    G --> R["block / transaction"]

Truncated keys (disk-saving)¶

The index key keeps only the first --index-key-bytes bytes of each hash (default 8 of 32), with the location packed into the key:

block: 'b' ‖ hash[:K] ‖ number
tx:    'x' ‖ hash[:K] ‖ number ‖ txIndex

Truncation roughly halves the on-disk index versus full 32-byte-hash keys. Two hashes can now share a key prefix (a collision); a lookup prefix-scans every candidate under the prefix and resolves it by re-checking the full hash on each Get — so a collision only ever costs an extra block read, never a wrong answer.

K is locked in the index DB when it is first created (changing it would orphan every entry). 32 keeps the whole hash — no truncation, no collisions.

The transaction set is the one that matters (mainnet has billions of txs). Extra candidates a lookup must read ≈ 1 + N / 256^K:

`--index-key-bytes`	keyspace	reads/lookup (≈2³¹ txs → 4× growth)	tx index size¹
`4`	2³²	~1.5 → ~3	~37 GB
`5`	2⁴⁰	~1.002 → ~1.008	~42 GB
`8` (default)	2⁶⁴	1 → 1	~49 GB
`32`	2²⁵⁶	1 → 1 (no collisions)	~109 GB

¹ rough, at full mainnet, measured under the earlier zstd index compression — the default is now Pebble's Snappy framing encoded by S2, a few percent larger (see below). Each "extra read" is a real block decode, so going below 5 is rarely worth it; 8 is collision-free in practice with ample margin.

Coverage of existing data — the reindexer¶

The index is built as blocks are confirmed, so a database synced from scratch with indexing on ends up fully indexed. For a store that already held data before --index was turned on, a background reindexer fills the gap:

On open it captures the confirmed range [floor, tip − seal-confirmations] and scans it ascending, indexing every block, in the background — Open never blocks on it, and by-hash lookups work for whatever is already indexed while it runs.
Progress is a persisted watermark, so it resumes after a restart and aborts promptly on shutdown rather than holding up Close.
It only ever touches confirmed blocks (never the reorg-prone tail), and the live import path keeps indexing new confirmations meanwhile — the two never overlap. It runs once to clear the backlog, then marks itself done; later opens skip it.
Progress is logged every --reindex-report-every (default 30s, 0 silences) with the fill percentage, sustained rate, and ETA — e.g. Hash index reindex progress at=12,345,678 to=25,000,000 pct=49.4% rate="8500 blk/s" eta=24m.

Debug methods drive it:

debug_indexStatus — config, reindex progress (target / next / done / percent), and the index's on-disk size (diskBytes).
debug_reindex — force a fresh full re-scan. Use it to fill gaps left if you turned --index off for a while and back on — the one case the automatic pass can't detect on its own.
debug_compactIndex — compact the index DB in the background to reclaim space from churn (old key versions, levels left un-merged by abrupt restarts). The index is otherwise append-only, so this is rarely needed; pebble's automatic compaction keeps a running node's index lean.

Keeping space low¶

The index DB uses Pebble's default block compression — Snappy framing, encoded by S2 through the root go.mod's replace of golang/snappy — like every other Pebble instance in the node. zstd was ~15 % smaller on this data (the random hash bytes don't compress, but the in-key block numbers and prefixes do), but its compaction CPU sits on the import path, which is what S2 everywhere is for. Pebble reads snappy and zstd SSTables interchangeably, so an index built under the earlier zstd default converts on its next compaction — no reindex needed; debug_compactIndex forces it at once.

A running node otherwise needs no attention: pebble continuously auto-compacts the append-only index, so it tracks the chain's growth with negligible reclaimable garbage. As a safety net, a background check on the --compaction-interval tick reads the index's read amplification — high values mean writes piled up un-merged (typically after abrupt restarts) — and auto-triggers a background compaction when it crosses a threshold, so the node self-heals without operator action. debug_indexStatus.diskBytes and debug_compactIndex remain the manual levers.

Disabling¶

--index=false skips the index DB entirely (no extra disk). The by-hash RPC methods then return a clear "hash indexing is disabled" error rather than wrong or empty results.

Configuration¶

Flag	Default	Meaning
`--index` ⚙	`true`	Build the block-hash / tx-hash index. `=false` disables it and skips the index DB.
`--index-key-bytes` ⚙	`8`	Leading bytes of each hash kept in an index key (4–32; 32 = the whole hash). Locked at index creation.
`--reindex-report-every` ⚙	`30s`	How often the background reindexer logs progress (fill %, rate, ETA); `0` silences it.

Both can be set per-network in the config (Index, IndexKeyBytes); see Configuration.