Prebuilt call traces instead of state history (concept)¶

Status

Proposed — not accepted. Measure first. This page works through the idea of materializing every block's callTracer output during execution and serving debug_trace* from that store, so that state history (the per-block changesets and their index) can shrink to the reorg window. The sizing below says a full trace store is not smaller than the history it would replace — same class or larger — so the concept is reframed: traces as an optional, additional materialized artifact for trace-heavy serving, built from history in parallel. Phase 0 is a measurement; the decision gate is at the end.

The question¶

UMNode serves exactly one tracer, the callTracer (Tracing). Every debug_traceBlockByNumber / debug_traceBlockByHash / debug_traceTransaction re-executes the block on its parent's point-in-time state. That state exists only because the state store keeps history: a pre-value changeset per block (store/changeset) plus a per-key index of the blocks that changed it (hist/). History is the bulk of an archive node — on mainnet about 1.3–1.7 TB, against ~150 GB of flat state (reth and Erigon 3 measure the same split; see Sizing expectations).

If call traces are the only historical product we need, why keep history at all? Execute each block once, record the call tree as we go, store it like any other block artifact, and answer trace requests with a read instead of a re-execution.

What history is used for today¶

Consumer	Needs history	Depth
Forward execution	no — pre-values are captured during execution	—
Reorg unwind	yes	≤ `--seal-confirmations` (256)
`debug_rebuildState` from a block	yes	down to the chosen block
`eth_call` and the state getters at a past block	yes	any
`debug_trace*`	yes — the parent's post-state	any

Shrinking history to the reorg window keeps the first three (partial rebuilds limited to the window) and gives up the historical eth_call / getters. The traces need a replacement — the trace store below.

The idea¶

flowchart LR
    EXEC[Executor] -->|flat writes + head| FLAT[(flat/ — Pebble)]
    EXEC -->|changesets, last 256 blocks only| CS[(changeset tail)]
    EXEC -->|callTracer hooks| TR[trace blob per block]
    TR --> TS[Trace segment store<br/>pending tail → sealed compressed segments]
    RPC[debug_trace*] -->|read + decode + render| TS

Capture. ExecuteBlock already drives geth's EVM; TraceBlock already attaches callTracer hooks per transaction (evm/trace.go). The forward path would run the same hooks once, so traces are produced for free in EVM time — no second execution.
Store. One blob per block in a segmented store. store/changeset is already payload-agnostic below its encoder: Append(num, blob), Get(num), RewindTail, SetSealCeiling, Prune, Reset know nothing about changesets. A second instance with a trace payload is the same machinery — immutable, content-hashed, group-compressed segments (s2 by default); detachable like any other.
Serve. debug_traceBlockByNumber reads block N's blob, decodes the frames and renders geth's JSON. debug_traceTransaction picks one transaction's tree out of the blob. ByHash resolves through the hash index first, as today.
Reorgs. Either the trace store keeps a rewritable pending tail like the changeset store, or — simpler — it holds only blocks deeper than the reorg window, and the window itself is served by re-execution from the retained 256-block history exactly as today. The second variant removes unwind from the trace store entirely; see open questions.

One canonical stored form¶

The callTracer has two switches, onlyTopCall and withLog, plus a timeout. One stored form answers all four combinations:

Store the full-depth tree with log references — the withLog: true shape. onlyTopCall is a strict reduction: the root frame with its own direct logs (geth drops nested frames and logs emitted below depth 0), so it is derived at serve time. timeout becomes meaningless — there is nothing to execute.
Omit what the block store already has. The root frame's input is the transaction calldata and its from / to / value are the transaction's; they are re-attached from the stored body at render time. Internal frames keep their input / output — nothing else holds them.
Binary, not JSON. Addresses as 20 bytes, gas and values as varints, calldata as raw bytes; JSON doubles every byte as hex and adds ~200 bytes of keys per frame. Blocks are compressed in groups at seal (like block grouping), where repeated selectors, token addresses and amounts compress well.
Exactness is the hard requirement. The rendered JSON must be byte-for-byte what geth's callTracer emits, including its omitempty rules (value absent on STATICCALL / DELEGATECALL, to absent on a failed CREATE, output absent when empty, the exact error and revertReason strings). That argues for the already-planned built-in tracer mirroring geth's callFrame (Tracing) rather than round-tripping geth's JSON into a binary form.

Sizing — why "instead of" does not pay¶

The model is bytes per frame × frames per block. A frame is one CALL, DELEGATECALL, STATICCALL, CALLCODE, CREATE or CREATE2.

Bytes per frame. Fixed fields (type, from, to, gas, gasUsed, value, keys and braces) are ~200–250 bytes of JSON; a typical internal call carries 68–200 bytes of calldata and 0–64 bytes of return data, which JSON doubles as hex. So ~500–700 bytes JSON per frame, ~200–300 bytes binary, and after grouped compression — selectors, addresses and amounts repeat heavily within a block — roughly 60–120 bytes on disk.

Frames per block. The unknown. A recent mainnet block (200–300 transactions, 20–30 Mgas used) plausibly holds 1,000–4,000 frames; swaps, lending and bridges each spawn tens of internal calls, plain transfers one. This is the number Phase 0 has to measure — it is uncertain by about 3×.

Per block → per year (today's activity, ~2.6 M blocks/yr):

Frames / block	On disk / block	Per year
1,000	~60–120 KB	~150–300 GB
4,000	~250–500 KB	~650 GB–1.3 TB

Two independent anchors land in the same range. The public BigQuery traces table grew about 40 GB a month in 2018, when blocks held 8 Mgas — ~0.2 MB of logical (uncompressed, columnar) trace rows per block, or ~0.75 MB scaled to today's 30 Mgas blocks, consistent with the model's JSON size before compression. And struct-logger traces run ~6 MB per block without stack or storage; the callTracer is 10–50× smaller because it records calls, not opcodes — the only reason this idea is thinkable at all.

Whole chain. Activity was far lower before 2021, so the cumulative store is weighted to recent years:

Network	Trace store (full chain, estimate)	History today (changesets + index)
Ethereum mainnet	~1–4 TB	~1.3–1.7 TB
BSC mainnet	~3–10 TB (≈3× mainnet in every component)	~3.8 TB
Sepolia	~0.5–2 TB (spam-heavy; ≈½ mainnet)	~0.8 TB

The history figures are measured (reth and Erigon 3 archive minus full, see Sizing expectations); the trace figures are the model above and need Phase 0.

History is the minimal representation

A block's changeset is the smallest record from which everything about that block's state — balances, storage, and its traces — can be rebuilt by re-execution. Traces are a materialized view of history: they trade re-execution time for bytes. Materialized views are never smaller than the data they are derived from unless they discard information, and a byte-exact callTracer output discards nothing. Disk is therefore not the argument for this concept.

What it buys¶

Latency. A trace becomes one group decompression plus a render — milliseconds — instead of re-executing a 20–30 Mgas block (hundreds of milliseconds on one core; seconds for attack-shaped blocks). Predictable, with no --rpc-trace-timeout to trip.
Serve-time CPU. Traces cost no EVM time at serve time, so trace QPS is bounded by disk and JSON rendering, not by cores. A full-chain trace scan — what an indexer does — runs at disk speed, hours instead of the days a from-genesis re-execution takes.
Write path. Capturing adds tracer hooks to forward execution (an estimated 5–15 % of EVM time; to be measured), but the store is append-only like the changeset store: no compaction, no write amplification. If history shrinks to the reorg window, the hist/ index — the only random-access read-modify-write load left in the state store — can go too, since unwind does not consult it. That is a real simplification of the commit path.
Determinism. A trace blob is a pure function of chain execution, so segments are byte-identical across honest nodes: content-addressable, verifiable, and fetchable from a CDN, object store or peer — the same property that makes detachable history work.

What it costs¶

Disk, per the sizing above: as much as history, or several times more.
Generality. No historical eth_call / getters outside the window, and the callTracer is the only tracer that can ever be served for blocks without history. Any tracer change — a bug fix, a new field geth adds — means rebuilding every blob: in parallel from history if history is kept, or by a from-genesis re-execution (days) if it was dropped.
Exactness burden. geth compatibility rests on our tracer and renderer instead of on geth's own code, for every block ever stored.
Moving parts. A third segmented store, one more participant in the seal / prune / unwind / WAL-less-checkpoint ordering.

Three options¶

	A — history only (today)	B — traces instead of history	C — traces in addition to history
Disk (mainnet)	~1.5 TB history	~1–4 TB traces + 256-block window	history + traces, then history pruned to a window or detached cold
Trace latency	re-execute (100 ms – s)	read (ms)	read (ms)
Historical `eth_call`	yes	no	yes while history is attached
Build traces	n/a	once, during the single forward execution	in parallel from history, N workers, resumable
Tracer change	nothing stored to rebuild	re-execute from genesis	re-trace from history in parallel
Write path	flat + index + changesets	flat + traces (+ window)	flat + index + changesets + traces
Complexity	—	trace store + exact renderer	+ a backfill worker

B's only advantage over C is a lighter write path; its disadvantage is that the one copy of history that makes everything rebuildable is gone. C keeps history as the build input and treats traces as the product; once the trace store is complete, history can be pruned to the window (--state-history 256) or, better, detached to cold storage and re-attached only to rebuild.

Decision¶

Proposed, pending Phase 0. The recommendation as of this writing:

Do not pursue B to save disk. It will not.
Pursue the trace store as C, if trace serving or trace scanning is the product and Phase 0 confirms the store lands within about 2× of history with a tolerable tracer overhead on forward execution. The win is latency, serve-time CPU and scan throughput — not footprint.
If disk is the constraint, the answer is the existing history with --state-history N plus detach, not traces.

Implementation plan¶

Measure (the gate). A one-off command on an archive node — e.g. umnode measure-traces --from A --to B --every K — runs evm.TraceBlock with callTracer + withLog over sampled blocks on ReaderAt(n-1) and reports, per block and per era: frames, JSON bytes, binary bytes (the model above applied to the real frames), group-compressed bytes, gas used, and the changeset blob size for the same block (changeset.Store.Get). Then bytes/block, bytes/gas and the trace-to-changeset ratio. A second mode executes a range with and without the hooks to measure capture overhead. Nothing is stored. Decide on the numbers.
Trace store. Factor the payload-agnostic core of store/changeset into a shared blob segment store used by both; define the trace blob format v1 (per-transaction trees, log references, omitted root fields).
Build from history. A backfill worker that traces [from, to] in parallel on point-in-time readers (blocks are independent given their pre-state), writes blobs in order, resumes from the store's tip, and logs a progress line like the other workers.
Serve. debug_trace* read the store first and fall back to re-execution when the block is absent and history is present. Both paths must render identical JSON — a differential test over the oracle chain at a tiny seal geometry, mirroring the changeset tests.
Forward capture. Hooks in the executor's forward path so the store follows the tip without a second execution; or, in the window variant, the store trails the head by the reorg window and the window is served by re-execution.
History window. --state-history 256 without the hist/ index for window-only nodes; detach / attach of history for rebuilds.

Open questions¶

Pending tail or trailing store? Keeping traces only for blocks deeper than the reorg window (served by re-execution inside it) avoids unwind and the WAL-less seal-ceiling coordination for the trace store entirely, at the cost of two serve paths. Likely the better first cut.
Log references vs copies. withLog frames can point at receipt log indices (small, needs the receipt at render time — always present in the block store) or embed the logs (larger, self-contained).
Per-block vs per-transaction addressing. debug_traceTransaction wants one tree; a per-transaction offset table inside the block blob avoids decoding the whole block.
Compression dictionary. A per-segment zstd dictionary trained on selectors and addresses may matter more here than for changesets.
Built-in tracer fidelity. Precompile frames, CREATE address on failure, gas accounting of the root frame, EIP-7702 delegation frames — each must match geth before any blob is written.