Initial Block Download & Fast Sync

This chapter covers getting a satd node to the chain tip: AssumeUTXO fast sync, the assumevalid script-verification skip options, dual-engine shadow verification, and the IBD performance and storage tuning flags. Points that differ from Bitcoin Core are called out throughout.

For the per-key table of defaults, reload disposition, and Core-vs-satd status, see the Configuration Flag Reference. This chapter explains how the pieces work and when to use them.

How satd syncs: the IBD pipeline

satd does not download and verify blocks one at a time in lockstep. IBD is a pipeline that keeps the network, the disk, and the CPU busy at the same time.

• Parallel block download. satd fetches blocks from many peers at once, the way BitTorrent downloads pieces from a swarm. Download throughput scales with aggregate peer bandwidth instead of one peer's round-trip time. -maxahead bounds how far ahead of the connect tip downloaded blocks may be staged. The 256 blocks just above the connect tip are always fetched, even with the window full, so the block the connector needs next is never held back by it. As in Bitcoin Core, blocks are asked only of peers that advertise NODE_NETWORK, or NODE_NETWORK_LIMITED for blocks within 286 of their tip. A pruned satd advertises NODE_NETWORK_LIMITED itself and serves no block more than 290 below its tip, as Core does.
• Background prefetch workers. -prefetchworkers threads pre-read blocks from the flat files without holding the chainstate lock. They deserialize each block, compute txids, run the context-free transaction validation, and resolve the block's UTXO inputs into the coin cache. All of this happens off the connect thread. When the connect thread reaches a block, its inputs are already cached.
• Speculative script verification. Prefetch workers verify scripts ahead of connection and mark the transactions they verified. The connect thread does not verify them again. In assumevalid mode, the connect step can then go straight to applying UTXO changes for the trusted range.
• Asynchronous shadow verification. The second consensus engine (see below) runs on its own worker pool, never on the connect path. It adds almost no wall-clock cost.

Network I/O, block pre-processing, script verification, and chainstate writes overlap instead of running one after another.

AssumeUTXO fast sync

AssumeUTXO makes a node usable in minutes instead of days. The node loads a UTXO-set snapshot at a recent height and serves wallets and queries from it immediately. It validates the historical chain from genesis to the snapshot in the background. satd's implementation is shipped and Bitcoin Core-compatible.

Loading a snapshot

• loadtxoutset <path> (RPC) loads a UTXO snapshot file. The node then holds two chainstates. The snapshot chainstate becomes the active tip; wallets, Esplora, Electrum, and RPC serve from it immediately. A background chainstate validates from genesis up to the snapshot's anchor. When background validation completes, the snapshot is marked validated and the node is a normal fully-validated node.
• --fast-start=<url|path> (startup flag) automates the sequence. satd downloads the snapshot (or reads a local file), waits for header sync to reach the snapshot's anchor, and calls loadtxoutset itself. Remote sources must be https://; plain http:// is refused, and TLS certificates are verified. Use --fast-start-sha256=<hex> to pin the download's digest. Download progress appears in the pre-RPC startup TUI gauge.
• getchainstates (RPC, Core 27+ compatible) reports progress. A node with no snapshot reports a single fully-validated chainstate. After loadtxoutset it reports a second, background chainstate, and the snapshot entry carries snapshot_blockhash and validated: false until background validation finishes.
• dumptxoutset <path> (RPC) writes a Bitcoin Core-compatible UTXO snapshot from your own node.

Trust model

At load, satd verifies the snapshot file against a hardcoded anchor hash. A tampered or wrong-height snapshot is rejected no matter where it came from. satd hosts no snapshots and does no P2P snapshot fetch; the operator names a trusted https:// source or a local file. The historical chain is still fully validated in the background. AssumeUTXO shortens the time to a usable node; it does not skip validation.

Difference from Bitcoin Core. The RPCs (loadtxoutset, getchainstates, dumptxoutset) and the two-chainstate model match Core. The --fast-start download-verify-load flag and --fast-start-sha256 are satd extensions. Core requires a manual loadtxoutset against a file you fetched yourself.

Where to get a snapshot

satd hosts none, and does not name one for you. The anchors compiled into the binary decide which snapshots are loadable at all — currently mainnet heights 840,000, 880,000, 910,000 and 935,000, copied verbatim from Bitcoin Core's m_assumeutxo_data. Signet, testnet and regtest have no anchors, so fast-start is mainnet-only.

Several people publish the utxo-<height>.dat files Core's dumptxoutset produces; Jameson Lopp's mirror and https://bitcoin-snapshots.jaonoctus.dev/ are two that have been around a while. Any of them will do, because the host is trusted for availability only:

satd --fast-start=https://<host>/utxo-880000.dat \
     --fast-start-sha256=<sha256 of that file>

--fast-start-sha256 pins what you downloaded, so a truncated or swapped file fails before it is parsed. That check is a convenience; the one that matters is the anchor comparison above, which satd performs against a hash compiled into the binary and which no snapshot host can influence. A snapshot from a hostile mirror is rejected at load.

Pick the highest anchor height a published snapshot exists for: the higher the base, the less history the background validation has left to walk.

--fast-start-sha256 is the file's SHA-256, not the anchor hash. hash_serialized_3 in the anchor table is a hash over the UTXO set, not over the file; sha256sum utxo-880000.dat does not produce it. Take the file digest from the publisher, or compute it after downloading once.

Script-verification skip: assumevalid

-assumevalid controls how much script verification IBD performs. satd accepts three forms. The third is a satd extension.

With a block hash, a block's scripts are skipped only when all of these hold, as in Core's ConnectBlock:

1. the assumevalid block is in the block index;
2. the block is the assumevalid block or one of its ancestors;
3. the block is on the best-header chain;
4. the best header has at least the network's minimum chain work (-minimumchainwork overrides it);
5. the best header is more than two weeks' worth of proof of work beyond the block.

Otherwise the block is verified in full: for example, a side-chain block at or below the assumevalid height, or the most recent two weeks of the chain while the headers are still close to it. When the decision changes, the log gives Core's line: Enabling script verification at block #N (hash): <reason>. or Disabling script verification at block #N (hash).

One difference from Core: when the best header is on a branch that does not carry the assumevalid block, satd verifies every block, including blocks below the fork point that Core would still skip. That is stricter than Core, never looser.

Difference from Bitcoin Core. Core's -assumevalid takes a block hash or 0. satd adds the all keyword and -assumevalidage, which trust the deep chain and verify the last day without pinning a hash. This suits recurring fast re-syncs. assumevalid is independent of AssumeUTXO, which concerns the UTXO set rather than script verification; the two compose.

Consensus engine & shadow verification

satd ships two independent script-verification engines: Bitcoin Core's C++ script interpreter, built from Core's source through libbitcoinkernel, and a from-scratch Rust verifier. It can run both together and verify every script twice. Bitcoin Core has no equivalent.

Read a mode name as "which engine is the shadow". In <engine>-shadow, the named engine is the shadow, the non-authoritative one. The other engine is primary; its verdict is what the node acts on. The shadow re-verifies in the background and logs any disagreement. So:

• rust-shadow: the Rust engine is the shadow; C++ is primary.
• cpp-shadow: the C++ engine is the shadow; Rust is primary.

-consensus=<mode>:

The Rust engine passes Bitcoin Core's script test suite. Shadow verification against Core's C++ interpreter (then libbitcoinconsensus) across the whole mainnet chain, genesis to about height 945,000, found zero divergence. Both engines hash a transaction's signature data once and reuse it for every input, as Core's own CheckInputScripts does. (Up to 0.7.x the C++ engine was libbitcoinconsensus, whose API made it redo that work for every input, so a transaction with many inputs cost time quadratic in their number.)

rust-shadow (C++ primary) stays the default out of conservatism. Running two independently written engines against each other is satd's core safety property, and Core's interpreter is the most widely deployed implementation. The plan is to promote the Rust engine to primary as it accumulates production mileage; cpp-shadow is that step. Treat the single-engine rust mode with care. The engine itself is proven, but either single-engine mode gives up the dual-engine cross-verification. satd prints a caution at startup when the single-engine rust mode is selected.

The shadow engine runs on a bounded background worker pool, so it consumes spare CPU without slowing block connection. Three flags tune it:

• -shadowworkers=<n> (default 4): background shadow-verification threads.
• -shadowqueuesize=<n> (default 4194304): the most transactions the shadow work queue holds.
• -shadowqueuemb=<n> (default 128): the most memory, in MiB, the queued work may hold. Each queued transaction holds its serialized bytes and a copy of every output it spends. On a host with few cores the shadow engine can fall behind the primary for hours during IBD, and this budget, not the entry count, is what keeps the queue from growing into the process's memory limit. 0 removes the byte bound and leaves only -shadowqueuesize.

When either bound is reached, shadow work is dropped, and an aggregated WARN is logged at most once per 5 s with the queue's depth and bytes. The primary engine still verifies every script, so correctness is unaffected. Frequent drops mean the shadow engine cannot keep up on this host; -consensus=rust or -consensus=cpp runs a single engine.

Difference from Bitcoin Core. Core has a single C++ engine and no shadow mode. satd's default runs both engines at once. -consensus, -shadowworkers, -shadowqueuesize and -shadowqueuemb are satd-specific.

IBD performance & storage tuning

These flags bound or accelerate IBD. Full defaults and semantics are in the Configuration Flag Reference.

-dbcache, -prune, -txindex, -assumevalid, and -reindex keep Core's names and meanings. The rest of the table is satd-specific tuning with no Core equivalent.

Reindexing

• -reindex rebuilds both the block index and the chainstate from the block files on disk (Core-compatible).
• -reindex-chainstate rebuilds only the chainstate (the UTXO set) from the existing block files, and preserves the flat block files (Core-compatible). It is faster than a full -reindex when only the chainstate is suspect.

Both work against block files written by Bitcoin Core, including the XOR-obfuscated files Core v28.0+ produces by default. The key in blocks/xor.dat is picked up automatically (see blocksxor in the Configuration Flag Reference).

A reindex on a synced mainnet node runs for hours. The shipped systemd unit handles this without tripping the start timeout; see "Reindex resilience" in Packaging.

A rebuild that does not finish

Both rebuilds start by wiping what they rebuild, and the wipe marks the chainstate current and every index complete before the first block is replayed. A rebuild cut short, by a kill, a power cut, a container stop, or -stopatheight below the chain's tip, therefore leaves a UTXO set and indexes that end partway while reading as finished; and satd cannot connect the rest by itself, because the block index already records those blocks as connected.

So satd writes a marker, .chainstate_rebuild in the network datadir, before the wipe, and removes it only once the rebuild has reached the chain's tip. A start that finds it acts on it:

A full -reindex that did not finish is finished on any start, whatever the flags. It continues from the height it last flushed: satd scans the block files again, checks each block it had already replayed against them (the index entry's header, file position, height and chainwork), and connects the rest. Nothing is downloaded and no block file is written. The replay flushes at least every 1000 blocks, so a kill costs at most that much replay, plus the scan. The scan reads each block's header, not the whole block: at most 4 KiB per block.

It runs again from genesis instead when it cannot be continued: its wipe may not have finished, satd 0.6.0 started it (0.6.0 did not record whether the wipe had finished), the chainstate is from another schema, or the blocks it replayed no longer match the block files. In the last case satd says so and exits without changing anything, and the next start runs the reindex from genesis. A chainstate rebuild trusts the block index, so it cannot finish a full one.

An unfinished chainstate rebuild is not continued: its flags restart it from genesis. If the chainstate directory itself is gone (removed to resync), the marker protects nothing and is removed at the next start.

-reindex-chainstate -stopatheight=H stops short on purpose and is refused all the same on the next plain start: the node could never advance past H.

A stop signal (SIGTERM or SIGINT) during a rebuild stops it at once: satd abandons the block it is connecting, flushes what it had connected, keeps the marker and exits 0, and the next start continues a full -reindex from there. See "Signals" in Packaging.

Block files that end early

A full -reindex can only rebuild what the block files hold. It reads blk00000.dat, blk00001.dat and on in order, stops at the first missing number, and stops reading a file at a record that is cut off. A file lost in a partial restore or an interrupted copy therefore ends the rebuilt chain at the hole.

satd holds the replay to the highest tip the node is known to have reached before the wipe: the chain database's tip, an unfinished rebuild's marker, or the clean-shutdown marker, whichever is highest. When the block files end below it, satd replays what they hold, then syncs the rest from peers as Core would. It also logs an error, does not log the reindex as complete, and raises the reindex.short warning (getwarnings, getblockchaininfo). The warning gives both heights and locates the hole: it names the missing file, or the first file where stored blocks pick up again without their parents. Restoring that file and running -reindex again saves the download.

A pruned node is exempt: pruning deletes the oldest block files on purpose, so its reindex always downloads the chain again. A block on the chain whose stored bytes fail validation stops the replay too, with the reindex.halted warning.

Upgrading the chainstate unprompted

-upgradechainstate=1 runs -reindex-chainstate by itself when it is needed and does nothing when it is not, so it can stay on for every start:

• the chainstate was written at an older schema (a release that changed the storage format, such as 0.6.0's schema 7): satd logs one warning and rebuilds the UTXO set and every enabled index from the block files, as -reindex-chainstate would; nothing is downloaded;
• a chainstate rebuild did not finish: satd restarts it from genesis.

It never downgrades (a chainstate a newer satd wrote is refused, as without the flag), and it refuses a pruned node, which does not have the blocks a rebuild needs and must resync. An AssumeUTXO node's background chainstate is not covered: finish or discard background validation before upgrading. The app-store packages and the reference stack set upgradechainstate=1 in the configuration they generate, which is how an app-store node crosses a schema change without a shell.

Driving a reorg by hand

invalidateblock and reconsiderblock work as in Bitcoin Core, and reach the node through sat-cli's raw-RPC passthrough:

sat-cli invalidateblock <blockhash>
sat-cli reconsiderblock <blockhash>

They are not listed in sat-cli --help — any method --help does not name is forwarded verbatim, which is how Core-compatible tooling keeps working. That makes them easy to miss when they are the tool you need.

Invalidating drives a reorg away from the named block and everything descended from it. reconsiderblock clears the mark and re-activates the best chain.

Startup integrity checks

Before serving RPC or connecting to peers, satd checks two things about the chain it is about to present.

The height→hash index is audited for gaps at or below the tip and rebuilt in place from the tip's ancestry. It is derived state, so this is a repair and startup continues. Heights whose rows disagree with the tip's ancestry are logged but never overwritten — correcting one means choosing between branches by chainwork.

The tip's ancestry is walked back one retarget period, and every block in it must be one this chainstate actually connected. This is not repaired. A block in the tip's ancestry that was never connected means the UTXO set is missing every output it created, and the only way to recover those is to replay the block. satd reports the affected heights and exits:

FATAL: this node's UTXO set does not agree with the chain its tip claims.

  * The tip stands on 8 block(s) that were never connected.
...
Refusing to start. Rebuild the UTXO set with -reindex-chainstate.

Serving in that state is worse than not starting: the tip is a real block on the real chain, the height is correct, and gettxout answers confidently and wrongly.

Two distinct faults can be reported, and they do not share a remedy. Blocks that were never connected mean the UTXO set is missing deltas, which -reindex-chainstate rebuilds. A broken parent pointer means the block index itself is wrong; -reindex-chainstate trusts that index, so only a full -reindex fixes it. Both can be present at once, and both are printed.

The exit status is 3, distinct from the 1 used for ordinary startup failures such as a bad config key, so supervision and alerting can tell "chainstate is damaged" from "the config file has a typo" without scraping stderr. Because this never heals by restarting, the shipped units set RestartPreventExitStatus=3 alongside Restart=always; without it the node would restart every few seconds forever and the unit would never settle into failed, so unit-state alerting would never fire. If you wrote your own unit, add that line.

After a -reindex or -reindex-chainstate, the audit runs again against what the replay actually rebuilt — a replay that stops short or reproduces the hole fails the same way, in the same run, rather than serving until the next restart.

On a pruned node none of these remedies can replay the missing blocks, because the block data is gone. satd says so rather than naming a remedy that cannot work: fetch the affected heights from a peer with getblockfrompeer and repair them, or resync.

On an AssumeUTXO node the history below the snapshot base is legitimately unvalidated until the background chainstate reaches it. That is recognised and logged at INFO, not treated as damage.

A coin the database cannot read

If RocksDB cannot read a coin from the chainstate (an I/O error or a checksum mismatch), or a coin's row does not decode, satd stops at once, as Bitcoin Core does. This holds wherever a coin is looked up: block connection, the mempool, gettxout, the Electrum and Esplora lookups, and the offline satd-chainstate-audit below. The log and stderr say Error reading from database, shutting down, with the outpoint and the error. satd does not carry on as if the coin were spent: that would refuse a valid block that spends it, and record the block as invalid across restarts.

The stop is an abort() (the process ends with SIGABRT), so no shutdown flush runs. The next start replays the blocks after the last flush from the block files, as after any crash. If the error comes back after a restart, the disk or the datadir is damaged: check the disk, then rebuild the UTXO set with -reindex-chainstate, or audit a copy of the datadir first (below).

Auditing a suspect datadir offline

satd-chainstate-audit answers the question the startup checks cannot afford to: does the UTXO set actually agree with the blocks on the active chain? It walks the tip's ancestry, reads each block back from the flat files, and reports every disagreement — coins that should exist and do not, spent coins still present, height-index rows naming the wrong block, txindex rows pointing at the wrong block, cumulative transaction counts that do not follow from their parent.

satd-chainstate-audit --datadir /path/to/datadir
satd-chainstate-audit --datadir /path/to/datadir --window 20000 --verbose

It takes the RocksDB lock, so the node must be stopped.

It issues no writes of its own, but it is not non-mutating: opening the chainstate opens RocksDB read-write, so the WAL is replayed and truncated, memtables may flush and compact, the MANIFEST is rewritten, obsolete files are deleted, any missing column family is created, the legacy address-history column families are dropped, and the schema version is stamped — after which an older satd will no longer open that datadir. Opening the block files creates xor.dat if absent. If the datadir is evidence — which is the case this tool exists for — copy it and audit the copy. The tool prints this warning on every run.

Note also that it is not included in the release tarballs or the Docker image; build it from source (cargo build --release --bin satd-chainstate-audit).

Exit status is 0 when consistent, 1 when it could not run, 2 when it found inconsistencies, so it scripts cleanly. A coin the database cannot read stops the tool as it stops the node (see above): it prints Error reading from database with the outpoint and the RocksDB error, which names the damaged file, and ends with SIGABRT before any report.

It diagnoses and does not repair. A missing coin is recoverable only by replaying the block that created it: -reindex-chainstate, or satd-chainstate-repair for a single block's lost delta. A broken parent pointer is a block index fault and needs -reindex — -reindex-chainstate trusts the same block index and cannot fix it.

--window bounds the walk, and its cost is not only one block read per height: every output the window creates and every outpoint it spends is held in memory until the end, so the default already runs to roughly a gigabyte on mainnet and tens of thousands of blocks runs to many. Start at the default and widen only as far as the search needs.

There is no --txindex flag: the tool reads the answer out of the datadir. An absent txindex row counts as a fault only when the chainstate's own completeness marker says the index was fully built, which rules out both shapes that would otherwise produce a false alarm — a node that does not run -txindex at all (satd's default), and one where -txindex=1 was switched on after the chain had already synced without it, leaving every historical block without a row it was never going to have. In that second case the audit says the rows went unchecked rather than counting them clean, because "not looked at" and "looked at and fine" are different answers.

This used to be a flag, and it was wrong in both directions. It defaulted to true while satd's -txindex defaults to off, so the invocation the node itself prints reported every transaction in the window as a missing row and exited 2 against a perfectly healthy node; and passing false silently disabled the txindex checks altogether, so a genuinely broken index came back consistent. An auditor cannot be expected to know a stranger's -txindex setting, and now does not have to.

Two states are reported but are not faults. Blocks the node pruned are counted separately from blocks that could not be read: pruning deletes block data deliberately, and treating that as damage would fail every healthy pruned node — at the default window, for most of the range — and then recommend -reindex-chainstate, which a pruned node refuses outright. On an AssumeUTXO node the snapshot base is read from the background chainstate's marker, so history below it is reported as not-yet-validated rather than as a hole.

Where a block could not be read, for either reason, the coin checks are skipped at and below that height — its spends are unknown, so a coin it spent would otherwise look missing. The tool prints a note when this applies. Verdicts above that height are unaffected: the walk runs newest-first, and a coin created at height H can only be spent at or above H.

Differences from Bitcoin Core at a glance

• assumevalid=all with assumevalidage: verify-recent-only mode. Core takes a hash or 0.
• Dual-engine shadow verification (-consensus, -shadowworkers, -shadowqueuesize, -shadowqueuemb): the default runs the C++ and Rust engines together. Core has one engine.
• --fast-start / --fast-start-sha256: one-flag AssumeUTXO download-verify-load. Core requires a manual loadtxoutset.
• -dbcache=auto: adaptive cache sizing. Core's is static.
• satd-only IBD and storage options: -prefetchworkers, -maxahead, -storageprofile, -maxopenfiles, the -rocksdb* and -compaction* families, -ibdl0pauseat, and the stall watchdog.
• -par is accepted for config compatibility. It does not size the connect path, but a positive value feeds -shadowworkers when that flag is unset.