Backfill sync

The backfill worker fills in chain history the store doesn't yet hold. By default it fills forward — genesis-first, in ascending order — so an indexer or state-builder can consume blocks in order as they import (state for block N needs block N−1, so it can only be built forward). It can also fill reverse (head-down, the original behaviour) with --backfill-forward-direction=false — see Direction.

It shares the fetcher with realtime sync (which always fills forward to the live head) and runs alongside it. It runs by default whenever a block store (--block-db) is enabled. Disable it with --backfill=false. The range it fills is bounded by --backfill-from / --backfill-to.

Direction — forward (default) or reverse

--backfill-forward-direction (default true) chooses how history is filled.

Forward (default). Anchors at the lower bound (--backfill-from, genesis by default) and climbs toward the target (--backfill-to, else the chain head), importing blocks in ascending order. Each block is verified by linking up to the block below it (its parentHash must match). This is the order an indexer/state-builder needs, so it can run in parallel with import.

Reverse (=false). Anchors at the trusted head and walks down its verified ancestry to the lower bound (the original behaviour, detailed in the rest of this page). The store stays contiguous [floor, head] throughout, and every stored block is provably part of the head's canonical ancestry.

The two trade off:

• Contiguity. Reverse keeps one contiguous range from the head down. Forward, while realtime fills the recent top, builds a separate genesis-up range — so the store is [genesis, X] ⋯gap⋯ [anchor, head] until the two meet. A chain contiguous from genesis to the head — and any indexing that needs it — is not complete until forward backfill reaches the head. (Reverse has the inverse caveat: genesis-up indexing can't start until it has descended all the way to the lower bound.)
• Trust. Reverse is anchored at the head realtime took from a peer and verifies every ancestor, so it cannot be fed a non-canonical fork. Forward is anchored at genesis (a constant) and is tamper-evident via parentHash, but it cannot prove the genesis-up chain is the canonical one (vs a peer's self-consistent fork) until it connects to the head-anchored range. For an honest-majority peer set this is moot; it is why reverse remains available, for the stronger guarantee.

The rest of this page describes the reverse mechanics; forward is their mirror (read the contiguous frontier above from instead of the floor, link each chunk up, commit lowest-first). The store memoizes the frontier query: a call resumes probing just above the last verified run instead of re-checking every block of the open segment (up to 50 000 of them, per chunk and per progress line), and a tail rewind or a pruned segment drops the memo. A forward chunk whose top is at least --seal-confirmations below the latest chain head is committed with the store's confirmed fast path: straight into the open segment, bypassing the reorg tail (one Backfill writes confirmed blocks straight to their segments line marks the first such chunk).

How it works

flowchart LR
    BF[Backfill worker] -->|read floor| ST[(Block store)]
    BF -->|fetch chunk below floor| F[Fetcher]
    F -->|headers / bodies / receipts| PS[(Ready peers)]
    F -->|validated blocks| BF
    BF -->|Put, highest-first| ST

1. The worker reads the store's floor. If the store is empty it has no anchor to attach history to, so it idles until realtime establishes one — or, when running standalone (see the range below), anchors at the chain head itself.
2. It logs a Backfill fetching blocks line at Info — the chunk span, its batches, and the effective concurrency, so the network step and its parallelism are visible at the default log level — and asks the fetcher for the chunk just below the floor — [floor − max-catch-up, floor − 1] — getting fully-assembled, validated blocks back. The fetcher pulls the chunk's batches in parallel across peers (up to --fetch-concurrency, scaled to the ready eth/69+ peer count).
3. It checks that the chunk links to the anchor: the top block of the chunk must hash to the stored floor block's parent. This ties the new history to verified data, the same trust check realtime makes when extending its head.
4. Before writing, the import step re-checks the chunk is one gap-free, parent-linked chain (see below), then writes it highest-first and logs a Backfilled blocks line with fetch / write / elapsed timings (the network, disk, and total portions) and a prefetched flag. The chunks are pipelined: just before a chunk commits, the next chunk's fetch is launched, so the network fetch of chunk N+1 overlaps the store write of chunk N. Commits stay strictly sequential and every seam check stays anchored to committed store data — only the fetch runs ahead, at the cost of one extra chunk of blocks in memory. A prefetch that no longer matches the plan (the target moved, a retry changed the frontier) is discarded and re-fetched in the foreground.
5. The floor has now dropped by a chunk; it loops until it reaches the lower bound (genesis by default), then logs Backfill complete and exits. When an explicit --backfill-to was clamped to a lower chain head (see below), the top ends up below to; the completion line is then a warning that names the unfilled gap instead, since backfill fills only downward and won't revisit it.

Why highest-first

Within a chunk the worker commits blocks in descending order. That is what keeps a restart safe. The store derives the resume point from its floor (the lowest contiguous block), so the stored range must never develop a hole below the anchor. Writing highest-first means an interruption — a shutdown, a crash — leaves a contiguous suffix directly below the anchor: the floor simply sits a little higher and the next run continues from there. Writing lowest-first could leave a gap above the new floor that nothing would ever revisit.

Linking to the anchor

Backfill never trusts peers blindly. Each chunk is validated end to end by the fetcher (every block against its header) and the chunk as a whole is pinned to the store: its top block must be the parent of the current floor block. A peer serving a different chain fails this check, the chunk is rejected, nothing is written, and the worker retries — so backfilled history is as trustworthy as the anchor it descends from.

On top of that anchor check, the import step (shared with realtime) verifies, just before writing, that the batch is one gap-free, parent-linked chain — consecutive block numbers with each child's parent hash matching the block below it — and rejects the whole batch otherwise. So even a number-correct but mis-linked or gappy response from a peer is caught at the write boundary and never persisted.

Limiting the range

Two flags bound the blocks backfill fills, inclusive:

• --backfill-from (default 0, genesis) — the lowest block. Backfill stops here instead of walking all the way to genesis. Use it to keep only recent history (e.g. --backfill-from 18000000).
• --backfill-to (default 0, auto) — the highest block. Auto means "the earliest block realtime has written" — i.e. fill everything below the live region — or, on an empty store, the chain head.

Most runs leave both at their defaults: backfill fills [genesis, realtime's anchor). The bounds cover two extra cases.

The retention window. With --block-history N the lower bound is also raised to head − N, rounded down to a segment boundary: blocks below it would only be pruned, so they are never fetched — a pruned node downloads its window, not the chain. With execution on, the bound is capped at the executed head, since the executor needs every block from its head on: a state restored by snap sync pulls it down to the snapshot's head, a state built from genesis to its own small head — so a full sync is unaffected — and a state with no head yet to genesis. The window is resolved once per run, when the chain head is first known (so the anchor does not chase a moving head), and logged as Backfill lower bound raised to the retention window.

The window works the other way too. When it reaches below the store's prune floor — the node was restarted with a larger --block-history, or all, or a lower --backfill-from — the fill lowers the floor to the bound and, once the forward fill is complete, fetches the gap back downward from the former floor, the reverse way: each chunk chained up to a stored, verified block, whole segments from the archive where it has them. See Pruned history, fetched back.

Extra ranges. Once the window is filled, the ranges --extra-archive-blocks names are filled too, highest first, each downward from the block above it: that block — stored, or else a peer's taken on trust as a reverse fill's top is — anchors the fill, whole segments come from the archive where it has them, the rest from peers, and a range the store already holds is left alone. debug_status shows each range's fill as blocks.extraRanges.

Standalone backfill (no realtime). With --realtime=false and an empty store, there is no anchor to descend from, so backfill anchors at the chain head (from the head poller — needs --head-poll-every > 0) and fills down to --backfill-from. This lets a node archive history without following the tip.

A bounded window below the live region. If --backfill-to is set below the lowest block realtime has stored, there is no stored parent above it to verify against. Backfill then establishes the anchor by trusting the peer's block at --backfill-to — exactly the trust realtime's head cold-start takes — and chain-verifies every block beneath it. This fills an isolated [from, to] window and leaves an intentional gap between to and realtime's region; that gap is not revisited (auto mode tracks the global floor, which now sits at from). Only the single top block is taken on trust; prefer leaving --backfill-to at auto unless you specifically want a sub-range.

A --backfill-to above the chain head. The requested top can't be fetched while it is ahead of the live chain, so backfill clamps the anchor to the current head at cold-start and fills down from there. The run then ends below to, and Backfill complete becomes a warning naming the unfilled [head+1, to] gap. Because backfill fills only downward it never closes that top gap — realtime fills it forward, or re-run later once the chain has advanced past to onto a fresh anchor. Related gotcha: a range's top segment only seals once the store holds at least --seal-confirmations blocks above it (the highest segment is always the open tier), so to seal the top segment of a backfill range, set --backfill-to that many blocks higher — and still at or below the head.

--backfill-from must not exceed --backfill-to (when to is set); the node refuses to start otherwise.

A pruned store. With --block-history N the lower bound is raised to the store's prune floor — in both directions, and re-read on every step as the floor rises — so backfill never fetches pruned history back while the window stays at the floor: reverse stops at the floor, forward starts from it, and the Backfill complete line reports the floor as from. A window reaching below the floor lowers it and fetches the history back (see The retention window).

Whole segments from the archive

With an archive source — the public archive unless --archive-source none — the fill asks the segment archive before fetching from peers whenever it reaches a segment boundary: the segment just above the forward frontier, or the one just below the reverse floor. A segment the archive has is downloaded whole, verified by the store (hash chain, header roots, linkage to the stored neighbour — see Attaching sealed segments) and attached — converted to the local layout first when the archive was sealed with other store settings; the fill continues on the far side of it. A segment the archive lacks is fetched from peers as usual. To make the frontier land on boundaries, peer chunks end at segment boundaries while an archive is configured, and the pipelined prefetch is skipped for a chunk the archive may supply. Segments that reach above the target (or into the mutable top of the chain) and the segment holding an empty store's lower-bound anchor are always fetched from peers.

Running alongside realtime

Both workers share one fetcher and the same peer set, and both write through the store's single-writer lock, so their writes are atomic with respect to each other. Realtime owns the blocks at and above the anchor (including reorgs in the recent tail); backfill only ever writes below the floor, which realtime never touches. The fronts cannot collide. Backfilled blocks sit far behind the tip, so they flush straight into segments and seal as each segment's range fills.

Forward fill with realtime. A forward backfill climbs from genesis while realtime follows the head, and the two split the chain at a handoff distance of two catch-up chunks (2 × --max-catch-up, 8,192 blocks by default):

• When a head arrives more than that far above the store's highest block — the backfill frontier, on a node filling from genesis — realtime anchors at the head instead of catching up from the frontier, which would fetch and write the very range the backfill is writing. The store then holds [genesis, frontier] ⋯gap⋯ [anchor, head].
• The backfill fills the gap and stops just below realtime's anchor, never fetching into its range; once the two ranges meet it logs Backfill complete.
• With nothing stored above it and the head within the handoff distance, the backfill stops (Backfill complete; realtime follows the head from here) and leaves the last stretch to realtime's catch-up instead of racing it to the tip.

Before 0.2.122 realtime only anchored at the head on an empty store, so when the forward backfill stored genesis first, realtime caught up from the frontier and repeated the backfill's work.

Shutdown

Like realtime, the worker checks for cancellation between blocks and between chunks, so an interrupt stops promptly rather than draining a whole chunk. Because chunks are committed highest-first, an interrupted chunk leaves the store contiguous and the next start resumes from the new floor. The node drains the worker before closing the store, so an interrupt during a write exits cleanly.

Progress logging

Every --backfill-report-every (default 10s, 0 disables) the worker logs a one-line summary of where the fill is and how fast it is importing:

Backfill progress  direction=forward  block=25,150,500  target=25,160,000  progress=5.0%  rate="850 blk/s"  eta=11s

• progress is the share of [from, target] already stored. For forward the position is block (the contiguous frontier above from); for reverse it is floor (the lowest stored block, descending toward the lower bound). target is an explicit --backfill-to, or in auto mode the live frontier the fill converges on (the chain head for forward).
• rate is the sustained import rate — blocks committed per second averaged since the worker started. Chunks commit atomically, so this running average is steadier (and a better ETA basis) than an instantaneous per-interval rate, which would mostly read zero between commits.
• eta is the time to reach the target at that rate (1d10h25m-style; - when the rate is unusable). In auto mode the target tracks the live head, so the estimate stretches slightly as the chain grows.

The other workers carry the same eta key on their progress lines: the executor's Executed blocks estimates when execution catches the stored head from how fast the lag closes across report windows (- while the gap is not shrinking — i.e. import-bound), and realtime logs a Realtime catching up line with rate and ETA while it refills a deep gap after downtime.

It is independent of the per-chunk Backfilled blocks logs (which always print) and of the periodic block-store summary.

Configuration

Flag	Default	Meaning
--backfill ⚙	true	Run the backfill worker. Needs --block-db.
--backfill-forward-direction ⚙	true	Fill forward (genesis-first, in-order) for parallel indexing; =false fills reverse (head-down). See Direction.
--backfill-report-every ⚙	10s	How often to log the progress line (fill %, import rate); 0 disables. See Progress logging.
--backfill-from	0	Lower bound of the range, inclusive (0 = genesis).
--backfill-to	0	Upper bound, inclusive (0 = auto: the chain head for forward; for reverse, realtime's earliest block, or the head on an empty store).
--max-catch-up ⚙	4096	Blocks fetched and committed per chunk (shared with realtime catch-up).
--archive-source ⚙	(public archive)	The segment archive whole sealed segments are attached from instead of fetching their blocks; none fills from peers only.

Backfill reuses the fetcher flags (--fetch-batch, --fetch-concurrency, --fetch-retries, --request-timeout, and --rpc-peers for JSON-RPC block sources tried ahead of the wire peers) and the --max-catch-up chunk size. --backfill and --backfill-forward-direction can also be set per-network in the config (Backfill, BackfillForwardDirection); see Configuration. With backfill on but no block store, the worker has nowhere to write and does not start.

Imported blocks — from realtime and backfill — are counted by the umnode_chain_imported_blocks_total metric; see Observability.