A guided tour of the codebase
A Bitcoin Core-compatible full node in Rust.
One process. One RocksDB. One systemd unit.
~219,000 lines of Rust across 19 workspace crates · +24k lines of Go SDK
Every code snippet is real, verbatim from master at 4874b537 (2026-08-19).
Navigate with ← → · press t for the table of contents
Part 1 · Orientation
satd holds four surfaces byte-compatible with Bitcoin Core v30 — consensus rules, P2P wire format, JSON-RPC method shapes, and bitcoin.conf syntax — and treats any unlisted deviation as a bug. Inside that envelope, it deliberately ships more.
libbitcoinconsensus from genesis to ~945k blocks with zero divergence.bitcoin.conf starts satd unedited; unknown keys rejected as typos, recognized-but-unsupported keys warn.SIGHUP; native TLS everywhere.bitcoind + electrs + esplora + nginx + exporter stack into one tip-consistent process.Part 1 · Orientation
Everything consensus- and node-shaped lives in the node library crate; satd and sat-cli are thin binaries over it. Protocol surfaces, indexes, and the policy language are separate crates so their dependencies stay out of the consensus build.
fuzz/ is a separate cargo-fuzz workspace (nightly + sanitizers), kept out of the normal build. Toolchain pinned at Rust 1.93.0, edition 2024.
Part 1 · Orientation
The spine is the same as any full node — P2P feeds validation, validation feeds storage — but every read surface hangs off the same RocksDB instance and updates ride the same write batch as block connection, so no surface can observe an index out of sync with the tip.
CConnman → PeerManagerImpl → ChainstateManager/CChainState → LevelDB + flat files, with indexes (TxIndex, BlockFilterIndex, CoinStatsIndex) as separate LevelDB databases synced by background threads — an index can lag the tip. Esplora/Electrum/metrics are external processes with their own copies of the chain.Part 2 · Storage
All persistence flows through one trait. Four implementations: RocksDbStore (the durable backend), CoinCache (a write-back cache that is itself a Store), SplitStore (AssumeUTXO routing), and InMemoryStore (tests, and the reference implementation for the batch contract). Most of the ~60 methods have defaults so a minimal backend implements only the core dozen.
/// Abstract storage backend for block index, UTXO set, and metadata.
pub trait Store: Send + Sync {
fn get_block_index(&self, hash: &BlockHash) -> Option<BlockIndexEntry>;
fn get_coin(&self, outpoint: &OutPoint) -> Option<Coin>;
fn has_coin(&self, outpoint: &OutPoint) -> bool;
fn get_tip(&self) -> Option<BlockHash>;
fn get_block_hash_by_height(&self, height: u32) -> Option<BlockHash>;
fn write_batch(&self, batch: StoreBatch) -> Result<(), StoreError>;
/// Write with the given durability mode. Default delegates to
/// `write_batch` (ignoring the mode) — concrete backends that can honor
/// `BulkLoad` should override.
fn write_batch_mode(&self, batch: StoreBatch, _mode: WriteMode) -> Result<(), StoreError> {
self.write_batch(batch)
}The unit of writing is StoreBatch — one atomic batch per block connection or disconnection, spanning every column family. Its one subtle invariant is spelled out at the definition:
/// **Remove-wins contract:** if the same key appears in both a family's
/// puts and removes within one batch, the key must end ABSENT. Both
/// emitters rely on this: `connect_block` carries a put+remove pair for
/// an output created and spent within the same block, and
/// `disconnect_block` carries the mirror pair (undo-restore put +
/// created-output remove) — in both shapes the correct final state is
/// absent. Implementations must apply puts before removes (or net the
/// pairs); regression tests pin this for `RocksDbStore` and
/// `InMemoryStore`.
#[derive(Default)]
pub struct StoreBatch {
pub block_index_puts: Vec<(BlockHash, BlockIndexEntry)>,
pub coin_puts: Vec<(OutPoint, Coin)>,
/// (outpoint, spent_amount, spent_height) — carried for O(1) counter/histogram updates.
pub coin_removes: Vec<(OutPoint, u64, u32)>,
pub tip: Option<BlockHash>,
pub height_hash_puts: Vec<(u32, BlockHash)>,CCoinsView → CCoinsViewBacked → CCoinsViewCache → CCoinsViewDB for coins, with a separate CDBWrapper/CBlockTreeDB for the block index. satd flattens this into one trait whose batch spans all families — the property the native indexes depend on. The tradeoff: Core's narrow views make it harder to accidentally couple subsystems; satd's single batch makes cross-family atomicity structural rather than conventional.Part 2 · Storage
Chainstate, block index, undo data, and every index share one RocksDB. The CF list is the schema — and the comment explains why it must stay exhaustive:
/// Every column family this store can create, plus RocksDB's default CF.
/// `flush_durable` flushes exactly this list (filtered to the CFs that
/// exist on the open DB), so a CF missing here would silently lose its
/// WAL-less (BulkLoad) writes on process exit. `open()` asserts every
/// descriptor it creates is listed — add new CFs HERE first.
const ALL_CFS: &[&str] = &[
"default",
CF_COINS, CF_BLOCK_INDEX, CF_HEIGHT_INDEX, CF_UNDO,
CF_TX_INDEX, CF_METADATA, CF_CHAIN_TX,
CF_ADDR_FUNDING_V2, CF_ADDR_SPENDING_V2, CF_OUTPOINT_SPEND,
#[cfg(feature = "block-filter-index")] CF_FILTER,
#[cfg(feature = "block-filter-index")] CF_FILTER_HEADER,
CF_ADDR_BACKFILL_TEMP, CF_SP_TWEAKS,
];let compression_per_level = [
DBCompressionType::None, // L0
DBCompressionType::None, // L1
DBCompressionType::Lz4, // L2
DBCompressionType::Lz4, // L3
DBCompressionType::Lz4, // L4
DBCompressionType::Lz4, // L5
DBCompressionType::Zstd, // L6
];Hot upper levels stay cheap to write; the bottom level, where data lives forever, pays for zstd once.
// Atomic commit across all column families.
// In BulkLoad mode we skip the WAL — the writer (connect loop during
// IBD) is responsible for calling `flush_durable` periodically so the
// amount of work lost on crash is bounded. `atomic_flush(true)` +
// `DataStored`-vs-`Valid` block-index markers ensure recovery is
// consistent: on restart any `DataStored` block not reflected in the
// tip pointer simply gets re-connected.
let mut wopts = WriteOptions::default();
if mode == WriteMode::BulkLoad {
wopts.disable_wal(true);
}~20–50% less write I/O during sync, bounded crash-replay via periodic durable flushes.
profile.rs ships --storage-profile=ssd|hdd presets sized from a production failure: a fully-indexed mainnet sync once accumulated 13,037 SST files and ~370 GB of pending compaction because max_total_wal_size (256 MB) was smaller than total memtable capacity (~680 MB). Likewise max_open_files is capped because RocksDB's default of −1 once wedged a 78 GB process holding every SST open. The comments keep the incidents attached to the knobs.-txindex, -blockfilterindex, -coinstatsindex), each synced by a background thread that can lag the tip. No per-level compression control, no WAL-mode switching; Core instead batches state into CCoinsViewCache and writes rarely.Part 2 · Storage
The UTXO working set sits in a two-tier cache — an unbounded dirty map of unflushed mutations plus a bounded clean LRU — wrapping the durable store and implementing the same Store trait. Budget defaults to Core's -dbcache=450.
/// Dirty coin entry — must be flushed to backing store before eviction.
enum DirtyEntry {
/// Coin exists in backing store and was modified/added. `fresh` = true means
/// the coin was created in this flush window (never written to backing store).
Present { coin: Coin, fresh: bool },
/// Coin was spent. Carries (amount, height) for counter/histogram updates.
/// If `fresh` = true, the coin was created and spent in the same flush window
/// and can be discarded without touching the backing store.
Spent { amount: u64, height: u32, fresh: bool },
}/// Flush dirty coins to the backing store. Caller must hold the
/// flush-exclusion lock (via `flush` or `FlushExclusion`).
///
/// Optimizations:
/// - **FRESH elision**: coins created and spent in the same flush window never
/// touch the backing store (Core PR #17487 insight).
/// - **Move semantics**: flushed coins are moved (not cloned) to the clean LRU,
/// avoiding the allocation burst that caused glibc malloc fragmentation.
fn flush_inner(&self) -> Result<(), StoreError> {The cache is also the reorg rollback primitive: the reorg driver flushes the pre-reorg chain to disk first, applies the whole reorg (disconnect + reconnect) to the cache only, and on any failure discards the partial reorg wholesale:
/// Discard every uncommitted (un-flushed) cache mutation, returning the
/// cache to exactly the last-flushed on-disk state held by the inner
/// store. Does NOT touch the inner store.
///
/// This is the rollback primitive for the atomic-reorg path (issue
/// #262). The reorg driver flushes the pre-reorg active chain to the
/// inner store first, then applies the whole reorg (disconnect +
/// reconnect + triggering connect) to this cache *only*. On any failure
/// it calls this to drop the partial reorg wholesale — no block-body
/// replay, and it cannot itself fail.CCoinsViewCache pioneered FRESH/DIRTY flags (satd's comment credits Core PR #17487 directly). The structural difference: Core's cache is a strict layered view flushed by FlushStateToDisk, while satd's is a concurrent shared structure — so it carries extra machinery Core doesn't need: a flush-exclusion lock, a foreign-writer note_mutation check that makes rollback refuse rather than corrupt (issue #567), and a generation counter preventing racy read-through inserts from resurrecting spent coins (issue #583).Part 2 · Storage
Block bodies live in Core's exact flat-file layout — blk*.dat, 128 MB per file, magic[4] || size_le[4] record headers — including v28's XOR obfuscation. That's what lets a Core blocks/ directory be adopted wholesale on migration.
/// XOR `data` in place against the repeating 8-byte `key`, where `data[0]`
/// sits at absolute file offset `offset`. No-op for the zero key, so the
/// plaintext path costs one comparison. Processes 8 bytes per step via a
/// phase-rotated `u64` so full-file de-obfuscation during `-reindex` runs at
/// memory bandwidth rather than byte-at-a-time.
pub(crate) fn xor_in_place(data: &mut [u8], key: &[u8; 8], offset: u64) {
if *key == ZERO_XOR_KEY {
return;
}Undo data breaks with Core deliberately: no rev*.dat files. Spent-coin records live in RocksDB's undo CF, written in the same batch as the connect — and they don't store outpoints at all:
//! On-disk format: an 8-byte magic, a 1-byte version, a varint count,
//! then back-to-back `Coin::serialize_compact` records. The outpoint
//! for each spend is recoverable from the block's tx inputs (the
//! connect-order invariant guarantees `undo.spent_coins[i]` belongs to
//! the i-th non-coinbase input), so we don't store it. Per-spend cost
//! is ~28 bytes for typical P2WPKH.coinview.rs defines satd's internal compact codec for the live coins CF (~28 bytes for P2WPKH, 35% smaller than the previous bincode rows). compressed_coin.rs separately implements Core's exact compressor.cpp wire format — varint-with-increment, exponent-mantissa amounts, six special script types — used only for AssumeUTXO snapshot files, so satd reads Core-produced snapshots and writes snapshots Core can read.
The AssumeUTXO background chainstate needs a private UTXO set but must share the block store, or historical block-index positions would be lost at handoff. SplitStore is a Store that routes: block index, height→hash, txindex to the shared half; coins, undo, tip to the private half.
rev*.dat flat files parallel to blk*.dat, with outpoints included. satd's in-DB, outpoint-free form trades Core-file compatibility (those files are ignored on migration anyway) for atomicity with the connect batch and ~28 bytes per spend.Part 3 · Chain & Validation
state.rs is 15,431 lines and the single owner of chain mutation. It is deliberately lock-poor: one RwLock<ChainTip>, one mutex that serializes every mutator, and lock-free atomics that exist so the stall watchdog can observe the connector without taking the lock the connector might be wedged holding.
/// Central chain state manager.
pub struct ChainState {
store: std::sync::Arc<CoinCache>,
flat_files: Arc<Mutex<FlatFileManager>>,
/// Path to the blocks directory, for mutex-free reads.
blocks_dir: PathBuf,
tip: RwLock<ChainTip>,
pub network: Network,
script_verifier: Arc<dyn ScriptVerifier>,
assumevalid: AssumeValid,
checkpoints: Vec<Checkpoint>,
/// Lock-free monotonic counter bumped on every successful connect.
/// Read by the stall watchdog to detect connector wedges without
/// taking the `tip` RwLock, which is precisely the lock the wedge
/// might be holding.
connect_heartbeat: AtomicU64,The most important field carries its own postmortem. "Single writer by construction" was the original design; issue #567 — a mainnet UTXO corruption caused by an invalidateblock reorg racing the P2P connector — is why the lock now covers everything:
/// **Every mutator must hold it, including the connector.** The
/// original lock covered `accept_block` and the two height-row commits
/// but not `connect_stored_block` / `connect_preprocessed_block`, on
/// the reasoning that the connector *is* the single P2P writer. That is
/// true of the P2P side and false of the whole system: an
/// `invalidateblock` reorg is a second writer, and issue #567 is what
/// the two of them did to a mainnet UTXO set. "Single writer by
/// construction" is not an invariant unless something enforces it.
accept_lock: std::sync::Arc<Mutex<()>>,cs_main, shared by validation, mempool, and net_processing. satd splits that role: accept_lock serializes chain mutation only, the mempool has its own lock, and readers go through the tip RwLock or lock-free snapshots. Finer-grained and non-recursive — at the price of needing explicit lock-order rules (documented in-file: accept_lock first, then the coin cache's flush exclusion, never the reverse).Part 3 · Chain & Validation
connect_block is a pure function from ConnectParams to a StoreBatch — all contextual consensus rules (BIP 30/34/68/113, sigops, subsidy, script verification) run here, and nothing touches disk until the batch commits. Every stage is announced to a lock-free phase tracker, born from two production stalls where a 95-thread dump still couldn't pinpoint the wedge:
#[repr(u8)]
pub enum ConnectPhase {
Idle = 0,
EnterConnect = 1,
PreResolveCoins = 2,
PerTxValidate = 3,
VerifyDispatch = 4,
VerifyJoin = 5,
ShadowDispatch = 6,
PostVerifyChecks = 7,
WriteBatch = 8,
TipWrite = 9,
/// block_processor: no header yet for next_height; waiting on
/// connect_signal condvar (1s timeout).
WaitingForHeader = 10,
WaitingForBlockData = 11,
CondvarWait = 12,
FlushingCoinCache = 13,
FlushDurable = 14,
WaitingForAcceptLock = 15,
}Script verification fans out across scoped threads — and even the phase-marker placement encodes an incident:
let queue_ref = &verify_queue;
let chunk_size = verify_queue.len().div_ceil(num_threads);
mark(crate::chain::connect_phase::ConnectPhase::VerifyDispatch);
let errors: Vec<ConnectError> = std::thread::scope(|s| {
// ... one worker per chunk ...
// Joining the scope-spawned workers is the most common
// wedge site we've seen in practice (issue #178). Mark
// the phase transition before .join() so a stalled
// dump definitively pins the wedge to this step.
mark(crate::chain::connect_phase::ConnectPhase::VerifyJoin);ConnectBlock in validation.cpp does the same contextual work, parallelizing script checks within a block via CCheckQueue. It has no phase instrumentation — a wedged ConnectTip is diagnosed from thread dumps. satd additionally parallelizes across blocks (next slide's prefetch pipeline), which Core does not.Part 3 · Chain & Validation
The atomicity discipline (issue #262): flush a durable checkpoint of the pre-reorg chain, hold the cache's flush-exclusion for the whole reorg so nothing partial can reach disk, and apply every disconnect/reconnect to the in-memory cache only. Failure recovery is then infallible — drop the cache delta:
// Atomic-reorg durable checkpoint (#262): flush the pre-reorg chain so
// it is the exact rollback target, then hold the flush-exclusion for
// the whole reorg so no external flush can persist a partial state.
let excl = self.store.lock_flush_exclusion();
excl.flush()?;
let mut reorg_excl = Some(excl);
// Disconnect from current tip down to the fork point.
let disconnect_info = match self.perform_reorg(&fork_entry, current_tip) {
Ok(info) => info,
Err(e) => {
self.abort_reorg(reorg_excl.as_ref(), current_tip, tip_entry.height, &e, None);
return Err(e);
}
};This replaced an older rollback that replayed block bodies and could itself fail — the root cause of a silent mainnet UTXO loss. And when rollback can't be proven safe, the node chooses the nuclear option:
/// A reorg needs to roll back and the coin cache cannot be attributed to
/// it. Stop the process without writing anything.
///
/// There is no safe continuation. Discarding would destroy whatever the
/// other writer committed — that is issue #567, and the whole reason this
/// check exists. Not discarding leaves the cache holding a half-applied
/// reorg mixed with someone else's blocks, under a tip that belongs to
/// neither; continuing from there writes that to disk at the next flush.
///
/// So: abort. ... `abort()` rather than `exit()` is deliberate: no
/// destructor runs, no shutdown flush gets the chance to persist the
/// poisoned cache. The node restarts onto a consistent state and redoes
/// the work.
fn fail_stop_on_unsafe_discard(&self, reason: &str, ...) -> ! {DisconnectTip/ConnectTip, each step flushable, relying on cs_main exclusivity and per-block undo correctness; a mid-reorg crash recovers by replaying from disk state. satd's cache-only staging gives all-or-nothing semantics for the entire reorg — a stronger invariant, purchased with the flush-exclusion machinery and foreign-writer detection the previous slides showed.Part 3 · Chain & Validation
One trait, four implementations: ConsensusVerifier (Bitcoin Core's libbitcoinconsensus via FFI), RustVerifier (the in-tree consensus crate — a drop-in API-compatible replacement), NoopVerifier (tests), and ShadowVerifier — a decorator that runs the primary in the hot path and cross-checks every script on background threads:
/// Shadow verifier: runs the primary engine synchronously in the hot path,
/// dispatches shadow verification to a background thread pool asynchronously.
///
/// The connect thread never blocks on shadow results. Mismatches are logged
/// by the background workers. This makes shadow mode essentially free in
/// wall-clock terms — shadow uses spare CPU but doesn't slow block connection.
pub struct ShadowVerifier {
primary: Box<dyn ScriptVerifier>,
shadow_tx: crossbeam_channel::Sender<ShadowWork>,
queue_size: usize,
_workers: Vec<std::thread::JoinHandle<()>>,
/// Counts shadow txs dropped because the queue was full. Rate-limited
/// reporter (see `report_drop`) consumes this and logs an aggregated
/// WARN at most once per 5s — a per-drop WARN at IBD verify rates can
/// burn tens of percent of wall-clock on tracing+stdout alone.
dropped: std::sync::atomic::AtomicU64,Which engine is authoritative is an explicit, delegated property — because the prefetch pipeline's "script OK" lets the connect thread skip primary verification, and that verdict must come from the engine the user chose:
/// Identifies which concrete verifier backs the authoritative (primary)
/// decision path. Used so components like the prefetch pipeline can match
/// whichever engine the user's config selected as primary — otherwise a
/// prefetch worker's "script OK" say-so (which lets the connect thread
/// skip primary verify) would override the user's chosen authority.
pub enum PrimaryEngine { Cpp, Rust }The native engine is the 9.6k-line consensus crate: a pure-Rust script interpreter whose error variants mirror bitcoinconsensus::Error exactly, with one tx-wide SighashCache shared across inputs (BIP 143's hashPrevouts/hashSequence/hashOutputs are per-transaction, so per-input recomputation is pure waste). It has been shadow-validated against the C++ engine over every mainnet script from genesis to ~945k blocks with zero divergence.
Part 3 · Chain & Validation
Core validates a block in stages (CheckBlock, then ContextualCheckBlock) because it reaches them at different points. satd fuses the witness half into one function so no caller can accidentally get only one half — and makes the context explicit in the signature:
/// This is Bitcoin Core's `CheckBlock` (structure, merkle root, CVE-2012-2459
/// mutation, weight) plus the witness half of `ContextualCheckBlock`
/// ([`check_witness_rules`]). Core splits the two because it reaches them at
/// different points; satd keeps them together so that *every* caller which
/// validates a block gets both halves.
///
/// The witness rules are gated on segwit activation exactly as Core gates them
/// on `DEPLOYMENT_SEGWIT`, which is why this is not context-free: it needs
/// `network` and `height` to decide. Threading them through the signature
/// rather than reading them from an optional argument is deliberate — the
/// compiler makes a new caller supply the context instead of silently
/// inheriting the laxer, context-free behaviour.
pub fn check_block(block: &Block, network: Network, height: u32)
-> Result<(), ValidationError> {Where the check order differs from Core, the comment states the exact condition under which it would have to flip back — e.g. weight is checked before witness rules because satd never persists a verdict from this function, so a coinbase-witness-padded mutant can't poison the index against the honest block.
PoW validation handles four retarget regimes (mainnet, testnet3's 20-minute rule, BIP 94 testnet4 with the timewarp guard, signet) and is uniformly fail-closed:
/// Calculate expected difficulty bits for mainnet.
///
/// Fails closed (`BadDifficulty`) if the retarget-period seed block cannot be
/// found: a missing seed means we cannot compute the expected difficulty, so we
/// must reject rather than substitute `prev`'s bits (which would let an
/// under-difficulty block through at a retarget boundary on a damaged index).
fn calculate_next_bits<F>(Consensus error strings are wire-compatible with Core verbatim — bad-txns-inputs-missingorspent, bad-cb-height — because they travel in reject reasons and RPC errors that downstream tooling matches on.
Part 3 · Chain & Validation
During IBD, background workers pre-process blocks the connector hasn't reached yet — flat-file reads without the mutex, check_block, MTP, txid computation, batched UTXO warm-up, and speculative script verification. The connect thread skips primary verification for any transaction whose inputs still resolve (coins are immutable, so a found coin is provably the same data the worker used):
/// A block that has been pre-read, deserialized, and partially validated
/// by a background prefetch worker.
pub struct PreprocessedBlock {
pub height: u32,
pub hash: BlockHash,
pub block: Block,
pub mtp: u32,
/// Pre-computed txids (one per transaction in the block).
pub txids: Vec<Txid>,
/// Tx indices where all inputs were speculatively resolved AND scripts
/// were pre-verified successfully. Only populated in assumevalid mode.
pub script_verified_txs: HashSet<usize>,
/// `validation::block::check_block` already ran on `block`, off the connect
/// thread. Consumers must run it themselves when this is false — the
/// conservative default for anything not produced by a prefetch worker.
pub context_free_checked: bool,
}The buffer is a plain height-keyed HashMap — a comment records that an earlier coordinator design produced 0% hit rates. Speculative coins are hints: each is re-verified against the authoritative store at connect time.
loadtxoutset streams a Core-format UTXO snapshot in (validated against a hardcoded anchor table copied from Core's m_assumeutxo_data), then a background chainstate revalidates genesis→snapshot in a private RocksDB while the primary serves the tip. At the snapshot height, the background recomputes Core's hash_serialized_3 over its own UTXO set and compares it against the anchor — a mismatch flags the snapshot rather than panicking, and the handoff is pinned fail-closed by test.
CCheckQueue) but connects blocks strictly serially; satd pipelines across blocks. AssumeUTXO matches Core's design (Core invented it) including snapshot-file compatibility — satd reads Core-produced snapshots and writes snapshots Core can read, via the Core-wire codec in compressed_coin.rs.Part 3 · Chain & Validation
A distinctive property of this codebase: design rules live as dated postmortems at the exact decision point. Three that recur everywhere:
/// The single implementation behind every MTP the connect path uses. `plan` is
/// `Some` only during a chainstate reindex, where the store's height→hash index
/// must not be consulted: it is derived state that has been observed polluted
/// with a fork block (#322), it describes the *pre-reindex* chain rather than
/// the branch being replayed ... MTP gates BIP113 locktimes and BIP68 time-based
/// sequence locks, so resolving it against the wrong branch is a consensus
/// decision made about a chain the node is not building.//! That is not hypothetical. On a synced mainnet node the persisted tip was
//! eight blocks above the last block it had actually connected ... Every output
//! created in that eight-block window was absent from the UTXO set. The node
//! reported a healthy synced tip on a real canonical block,
//! `getblockchaininfo` was self-consistent, and `/healthz` stayed green for
//! five and a half hours. Nothing detected it ...
//!
//! This pass is the detector that was missing. It answers wrong rather than
//! not at all: a node that fails it is refusing to serve, not quietly serving
//! a truncated UTXO set.//! It exists because reasoning did not settle the mainnet incident behind #567
//! and reading the on-disk artifacts did. Four independent artifacts —
//! `coins`, `height_hash`, `undo`, `chain_tx` — each said something the others
//! did not, and the fourth reversed the conclusion drawn from the first three.
//! An assertion that "the tip is fine" is worth very little; an assertion that
//! names the outpoint, the height and the artifact is worth a great deal.Supporting machinery follows the same philosophy: height_index_repair.rs only adds rows for heights that have none (a wrong row would require adjudication it refuses to do); reorg_log.rs drops-and-counts webhook events rather than ever blocking consensus on external HTTP; disconnect.rs surfaces corrupt undo data as an error that lands the operator on -reindex-chainstate instead of a panic.
Part 4 · P2P Networking
manager.rs (7,519 lines) is the single owning actor for all peer state. Peer tasks never touch chain or mempool state directly — they push NetEvents onto one bounded mpsc channel, and one central run() loop drains them, so all mutation is serialized in one place:
/// Event sent from peer tasks to the central manager loop.
pub enum NetEvent {
PeerConnected {
id: PeerId,
addr: SocketAddr,
version: VersionMessage,
},
PeerDisconnected { id: PeerId },
MessageReceived {
id: PeerId,
msg: NetworkMessage,
},
}fn handle_message(&self, id: PeerId, msg: NetworkMessage) {
match msg {
NetworkMessage::Ping(nonce) => {
self.send_to_peer(id, NetworkMessage::Pong(nonce));
}
NetworkMessage::Inv(inventory) => self.handle_inv(id, inventory),
NetworkMessage::Headers(headers) => self.handle_headers(id, headers),
NetworkMessage::Block(block) => self.handle_block(id, block),
NetworkMessage::Tx(tx) => self.handle_tx(id, tx),
NetworkMessage::GetData(inv) => self.handle_getdata(id, inv),Each peer gets two tokio tasks — a never-cancelled reader plus a select!-driven write loop — and the reason is an async-Rust footgun worth knowing:
// Split connection into read/write halves to avoid cancel-safety issues.
// read_exact is not cancel-safe — if tokio::select! drops a recv() future
// mid-read, consumed bytes are lost and the stream becomes misaligned.
// By running the reader in a dedicated task, it is never cancelled.
let (mut reader, mut writer) = conn.split();CConnman runs a small fixed set of threads (socket handler, message handler, opencon) multiplexing all peers over select()/poll with shared state under cs_main + per-node locks. satd's task-per-peer model trades a few KB of stack per peer for the absence of cross-peer head-of-line blocking and the shared-lock discipline. Notable P2P scope choices: BIP 37 bloom filters intentionally absent (BIP 157/158 served instead), and peers.dat is satd-native (magic SADR) rather than Core's bucketed serialization.Part 4 · P2P Networking
satd wraps the rust-bitcoin bip324 crate (ElligatorSwift ECDH + ChaCha20-Poly1305) rather than reimplementing the crypto, and ports its synchronous reference driver to tokio. The whole peer pipeline speaks NetworkMessage above a Connection::{V1, V2} seam and never learns which transport carries it. Detection is responder-side byte-sniffing, exactly as the BIP prescribes:
/// BIP 324 leaves v1/v2 detection to the responder: read the first
/// bytes and, if they are the network magic, the peer is speaking
/// plaintext v1 (a `version` message starts with the magic); otherwise
/// treat the bytes as the front of the peer's ElligatorSwift key and
/// run the v2 handshake. The detection bytes are consumed off the
/// socket and replayed into whichever transport is built.
async fn accept_transport(self: &Arc<Self>, mut stream: TcpStream) -> Result<Connection, String> {
// ...
if first == expected {
if self.v2_only() {
return Err("v2only: rejecting inbound v1 peer".to_string());
}
Ok(Connection::v1_with_leading(stream, magic, first.to_vec()))
} else {Outbound falls back to v1 by re-dialing a fresh socket, and remembers the downgrade so reconnects skip the wasted v2 round trip:
/// satd-specific `-v2only`: refuse peers that don't speak BIP 324 v2.
/// Inbound v1 peers are dropped at detection; outbound v2 failures are
/// not downgraded. Implies `v2_transport`. Defaults to false.
v2_only: std::sync::atomic::AtomicBool,
/// Outbound destinations whose v2 handshake failed this session, so we
/// connect them straight as v1 instead of wasting a v2 round trip on
/// every reconnect. Keyed by socket address (direct peers only).
v2_downgraded: RwLock<HashSet<SocketAddr>>,-v2transport matches Core (on by default, graceful fallback). -v2only is satd-only: an anti-surveillance lever that drops every v1 peer. As of 2026 most surveillance and DoS nodes don't speak v2, so this sheds essentially all of that traffic without banlists — at the cost of also dropping honest un-upgraded peers, which is why it stays opt-in.Part 4 · P2P Networking
This is satd's biggest P2P divergence from Core. Instead of a sequential in-order download window, the whole remaining height range is shuffled into a pool and handed out randomly across peers — with a 256-block priority zone above the connect cursor so the sequential connector never starves:
/// BitTorrent-style download coordinator for parallel IBD.
///
/// Assigns random blocks from across the chain to many peers simultaneously,
/// maximizing download parallelism and avoiding the "slow peer bottleneck"
/// that sequential range assignment creates.
pub struct IbdScheduler {
phase: IbdPhase,
target_height: u32,
/// Global pool of block heights still needing download, shuffled randomly.
pending: VecDeque<u32>,
/// Heights currently assigned to a peer (in-flight).
in_flight: HashMap<u32, PeerId>,
/// Max blocks in-flight per peer.
per_peer_limit: u32,
/// Max total blocks downloaded ahead of connect cursor.
max_ahead: u32,The tuning constants read like a flight recorder. Where the swarm design conflicted with how stock Core peers behave, the evidence won:
// Match Bitcoin Core's MAX_BLOCKS_IN_TRANSIT_PER_PEER (16). The
// previous limit (128) packed our getdata to peers with batches
// 8x larger than what stock peers expect to process. On the
// mainnet 2026-05-13 wedge run, peers consistently disconnected
// us after ~60s with most of their assignment unfulfilled — the
// smaller batch lets peers cycle through requests faster and
// reduces wasted in-flight assignments when a peer drops.
per_peer_limit: 16,Stall handling is layered: coarse per-peer detection, per-height timeouts with a much shorter fuse near the cursor, and a per-(height, peer) anti-affinity cooldown — each traceable to a specific mainnet wedge (heights 783563, 785197, 945208 on one 2026-05-13 run):
/// Per-(height, peer) cooldown timestamp. When a height is released from
/// a peer (timeout or notfound), that peer is barred from being re-issued
/// the same height until `cooldown_until > now`. Prevents the "silent
/// peer keeps getting re-assigned the same height" wedge observed at
/// height 785197 on the 2026-05-13 mainnet IBD run, where the 15s
/// near-cursor timeout released a stuck height back to pending and the
/// same peer immediately re-claimed it via its priority-zone scan.
height_peer_cooldown: HashMap<u32, HashMap<PeerId, Instant>>,BLOCK_DOWNLOAD_WINDOW) with 16 in-flight per peer, disconnecting the peer that stalls the window's leading edge. satd's shuffle keeps every peer saturated regardless of which heights they're slow on; the priority zone plus the per-height release machinery is the price of keeping the sequential connector fed under that randomness.Part 4 · P2P Networking
New/tried tables and network-group bucketing for eclipse resistance are kept (max 64 new entries per /16 or ASN, ~50/50 tried/new dial selection, 16,384-entry cap), but as one HashMap with a cached group key rather than Core's fixed 1024/256 bucket arrays. The group function is pluggable so -asmap swaps /16 for ASN bucketing — and that interpreter is a bit-exact port of Core's util/asmap.cpp, because real Core-produced asmap files must decode to the same ASNs. A study in choosing where to diverge and where to be byte-exact.
// A short ID is only 6 bytes, so two distinct mempool transactions can
// hash to the same short ID (crafted, or ~1-in-2^48 by chance). Using
// either one to fill a slot would be a guess: if it is the wrong tx the
// reconstructed block fails its merkle check and the honest relayer is
// banned, with no `getblocktxn` fallback. So a short ID that matches more
// than one mempool tx is marked ambiguous (`None`) and treated as
// unavailable — the slot is requested instead. This mirrors Bitcoin Core's
// `PartiallyDownloadedBlock::InitData`, which resets a slot when a second
// mempool tx collides on its short ID.For the other collision case — a peer announcing the same short ID for two slots — Core hard-fails reconstruction and re-requests the block; satd routes just the ambiguous slots through getblocktxn: "the same outcome with less bandwidth."
A hand-rolled control-port client covering exactly what hidden-service bring-up needs: PROTOCOLINFO, AUTHENTICATE (SAFECOOKIE by default — HMAC-SHA256 challenge/response with mutual proof), ADD_ONION NEW:ED25519-V3, DEL_ONION. The lifecycle subtlety is documented up front: an ephemeral onion service dies with its control connection, so dropping the controller at exit is the teardown. Outbound SOCKS dials get a fresh random credential pair per peer, so Tor's stream isolation gives each peer its own circuit — matching Core's -proxyrandomize.
Part 5 · Mempool & Mining
The mempool is one physical pool — FxHashMap<Txid, MempoolEntry> plus a spends: FxHashMap<OutPoint, Txid> reverse index — split into two logical classes by a scope marker on every entry. Quarantined entries (held by the operator's policy) are withheld from relay and/or templates but never rejected; each class has its own byte budget and its own fee-rate eviction:
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct QuarantineScope {
pub relay: bool,
pub template: bool,
}
impl QuarantineScope {
/// Acting entry — withheld from nothing.
pub const fn acting() -> Self {
QuarantineScope { relay: false, template: false }
}
/// True for an acting (non-quarantined) entry.
pub fn is_acting(self) -> bool { !self.relay && !self.template }Ancestors and descendants are not materialized on the entry: parents come from tx.input, children from the spends reverse index, walked on demand. There is no incremental package bookkeeping to desync — at the cost of O(graph) walks on verbose RPC paths.
pub fn accept_transaction(
&self,
tx: Transaction,
chain_state: &ChainState,
script_verifier: &dyn ScriptVerifier,
source: TxSource,
allow_quarantined: bool,
) -> Result<Txid, MempoolError> {
let txid = tx.compute_txid();
// Snapshot the live policy once so the entire acceptance is judged
// against a single config version. A concurrent SIGHUP reload can swap
// `self.config` between calls but never mid-transaction.
let cfg = self.config.read().clone();The admission pipeline, in order: context-free checks → absolute finality at tip+1 (BIP 113 MTP) → BIP 68 → standardness (dust, datacarrier, standard scripts) → conflict detection → input resolution + CPFP ancestor limits → RBF → fee floor → policy DSL evaluation → two-class isolation → eviction/insert → events.
CTxMemPool maintains cached ancestor/descendant state per entry (and v31 replaces it with cluster mempool). satd matches Core's knobs (-maxmempool, -mempoolexpiry, ancestor/descendant counts, -mempoolfullrbf on) but implements simpler internals: RBF is BIP 125 rules 3+4 only (no "no new unconfirmed inputs", no 100-descendant eviction cap), eviction sorts by individual feerate rather than descendant-package feerate, and standardness is output-side only. Also unlike Core: mempool.dat shares the filename and re-validate-on-load behavior but is a satd-native format (magic SMPL) — the file is a hint, never a trusted input.Part 5 · Mempool & Mining
estimatesmartfee stays Core-shaped, backed by a percentile over recent confirmed-block feerates. The satd extension, estimatefees, simulates the next N block templates from the live mempool — ancestor-aggregate feerates so CPFP composes correctly — and never errors, always returning a confidence level (this closes Core issue #11500, open since 2017):
//! Mempool-based smart fee estimation.
//!
//! Simulates the next N block templates from the current mempool snapshot.
//! For each simulated block we record the lowest admitted *ancestor
//! feerate* — that is the fee level a new tx needs to land in block k.
//! Ancestor feerate aggregation handles CPFP correctly: a low-fee parent
//! + high-fee child are admitted together at the child's pull rate.
//!
//! This complements the historical-block `FeeEstimator` in `fee.rs`:
//! historical data reacts to what miners *did* (slow to respond to sudden
//! congestion), while mempool simulation reacts to what's queued *now*.
//!
//! The simulator is intentionally a pure function over an owned snapshot
//! of `MempoolEntry`s — it does not hold any mempool locks while running.The estimate ladder is forced monotone — and the comment explains why a naive per-block floor isn't:
/// Taking the running minimum is what makes the estimate **monotonically
/// non-increasing** in `target`: confirming within N+k blocks can never
/// cost more than confirming within N. A single per-block floor is *not*
/// monotone — greedy packing under the weight/sigop caps can defer a
/// high-feerate package past block 1, leaving a later block with a higher
/// admission floor than an earlier one. Reading one block per target then
/// produces a nonsensical ladder (e.g. next-block < ~30-min). The running
/// minimum collapses that to the honest answer: if a cheap tx made it into
/// an early block, every deeper target is at least as cheap.create_template selects from. Simulating the full pool would pack blocks with transactions this node quarantined on template (ones it will never mine) and quote inflated fees to wallets through the public surfaces (Esplora /fee-estimates, Electrum estimatefee). The result is memoized with a TTL because an unauthenticated per-request simulation is a DoS amplifier.CBlockPolicyEstimator is a decaying bucket tracker over confirmation history — sophisticated on the historical side, blind to the current queue, and errors out when it lacks data. satd's historical half is simpler (a percentile ring); its mempool half sees congestion the moment it forms.Part 5 · Mempool & Mining
Selection sorts once by individual effective feerate (including prioritisetransaction deltas), then loops: emit what's ready, defer what awaits a mempool parent, drop what's unminable. The deferral is what yields parent-before-child order — the comment records the invalid blocks produced before it existed. Finality and BIP 68 are re-checked here rather than trusted from admission, because a reorg or a persisted mempool can invalidate the earlier judgement:
// Resolve every input against the UTXO set, not mempool
// membership: a parent evicted after this child was admitted
// (expiry, RBF, block-connect conflict) is in neither the
// mempool nor the UTXO set, and treating "not in mempool" as
// "confirmed" would mine the orphaned child →
// bad-txns-inputs-missingorspent. An input creates one of
// four cases: already included in this template (the coin is
// born at `height`), a confirmed coin, a mempool parent that
// may still be included (defer to a later pass), or nothing
// anywhere (drop — unminable).
let bip68_enforced = (entry.tx.version.0 as u32) >= 2;
let mut awaits_parent = false;
let mut minable = true;
for input in &entry.tx.input {
let parent = input.previous_output.txid;
let prev_height = if included.contains(&parent) {
height
} else if let Some(coin) = chain_state.get_coin(&input.previous_output) {
coin.height
} else if in_mempool.contains(&parent) {
awaits_parent = true;
continue;
} else {
minable = false;
break;
};addPackageTxs) selects by ancestor score, so a low-fee parent bumped by a high-fee child is packed at the package rate. satd's template builder sorts by individual feerate and only defers children — a genuinely CPFP'd parent can sort to the bottom and miss the block, taking its child with it. Meanwhile the fee simulator on the previous slide does full ancestor-package packing: the estimator is currently smarter than the miner. The coinbase weight reserve (8,000 WU) matches Core v30's DEFAULT_BLOCK_RESERVED_WEIGHT exactly.Part 5 · Mempool & Mining
Where Core's relay policy is a fixed C++ decision tree behind flags, satd ships a small statically-typed expression language (policyfile=) with which operators write arbitrary shape-based rules. The defining choice: quarantine, not reject — there is no Reject verdict in the type, so a ruleset can withhold from relay/templates but can never make the node refuse a transaction baseline policy would accept. Consensus is untouched by construction.
# Exception first: my own submissions are never filtered.
allow own-submissions when tx.source == rpc or tx.source == mcp
# Ordinals / BRC-20 inscriptions.
quarantine ordinals on relay,template
when any inputs (in.leaf_script.contains_ops(script(OP_FALSE OP_IF push(0x6f7264))))
# Cheap, oversized generic OP_RETURN.
quarantine cheap-bulk-opreturn
when any outputs (out.script_type == op_return and out.op_return_size > 83)
and tx.fee_rate < node.min_relay_fee * 3
# Mine-neutral big-witness: relay it, just don't mine it.
quarantine no-mine-big-witness on template when tx.total_witness_size > 100kbThe pipeline is parse → typecheck → cost → eval, and the guarantees are named invariants: I4 totality — eval returns a value, never an error (saturating arithmetic, x/0 == 0, no indexing, no recursion); I5 cost bound — every compiled expression carries a worst-case static cost and the loader rejects rulesets over budget, backed by a runtime fuel meter (~100µs ceiling per tx) whose exhaustion fails safe-restrictive: quarantine, attributed to the implicit __fuel rule.
pub fn evaluate<'a>(&'a self, tx: &'a TxView<'a>, ctx: &Ctx) -> Verdict {
let mut fuel = DEFAULT_FUEL;
for rule in &self.rules {
let out = rule.cond.eval_metered(tx, ctx, fuel);
if out.fuel_exhausted {
return Verdict::fuel();
}
fuel = out.fuel_remaining;
if out.value.as_bool() {
return match rule.action {
Action::Quarantine => Verdict::Quarantine {
rule: rule.name.clone(),
scope: rule.scope,
},
Action::Allow => Verdict::Allow { rule: rule.name.clone() },
};
}
}
Verdict::Pass
}Held transactions inherit their quarantined ancestors' scopes (mining a child whose parent you withhold would be incoherent), and a held submission can never RBF-evict an acting-class transaction. Every standard surface — getrawmempool, Electrum, Esplora — stays acting-class-only and byte-identical whether or not anything is quarantined; observability is a disjoint additive surface (getpolicyinfo, listquarantine, policytest).
Part 5 · Mempool & Mining
A relay-withholding rule that catches Lightning justice or HTLC transactions degrades L2 enforcement network-wide. Instead of a syntactic lint, satd runs the whole ruleset, under its real first-match-wins semantics, against synthetic transactions shaped like actual enforcement traffic — and blocks the load if any of them would be withheld from relay:
//! Detectability splits exactly as the protocols do:
//! - **BOLT-3 (legacy / anchor)** enforcement scripts are spec-mandated, so a
//! faithful vector is an exact probe: commitment (force-close) with its anchor
//! outputs, the breach-remedy **justice** transaction (the time-critical E1
//! case) spending a revoked `to_local`, and second-stage HTLC timeout/success.
//! - **Taproot-channel key-path force-closes** are *indistinguishable* from any
//! other P2TR key-path spend, so they cannot be detected directly. The probe
//! is instead a generic, healthy P2TR key-path spend: a rule that quarantines
//! it is structurally over-broad and necessarily sweeps TR force-closes with
//! it.
//! - **Taproot-channel script-path enforcement** (justice / HTLC) reveals a
//! tapleaf and is partially recognizable.
//!
//! `allow` rules are never dangerous: they only widen relay, never withhold.Err(format!(
"refusing policy: rule(s) [{}] would withhold relay for Lightning \
enforcement transactions, degrading L2 enforcement network-wide (E1). \
Narrow them, scope them `on template`, or set allowdangerousfilters=1 \
to override.", relay_rules.join(", ")))Eight probe shapes cover BOLT-3 commitment/anchor/justice/HTLC and the taproot cases. Template-only withholding logs a warning but loads — declining to mine enforcement traffic is the operator's call; declining to relay it needs the explicit override. sat-cli policylint runs the same analysis offline (exit 3) so a dangerous rule is caught before it's ever loaded. The manual is honest about the limit: the gate can't catch a rule that snags enforcement through an angle the probes don't model — it removes the most common operator mistake, not the critique.
Part 6 · RPC & Surfaces
98 methods, registered in one place, served by jsonrpsee over a tower middleware chain — each cross-cutting concern an independently testable layer instead of inlined code in an HTTP callback. The ordering is deliberate and documented at the build site:
// AdmissionLayer is outermost so an over-budget request is shed (429)
// before any auth/compat work. AuthLayer is next so an unauthenticated
// request is rejected before the compat layer buffers its body;
// JsonRpcCompatLayer then normalizes Core-style (`jsonrpc` 1.0/1.1/
// absent) requests to 2.0 so jsonrpsee accepts them (see `compat.rs`).
let capability_filter = bearer.as_ref().map(|_| CapabilityLayer::new());
let tls_middleware = tower::ServiceBuilder::new()
.layer(AdmissionLayer::new(admission))
.layer(AuthLayer::new(auth, bearer))
.layer(JsonRpcCompatLayer::new());
let rpc_svc = ServerBuilder::new()
.set_config(server_cfg)
.set_http_middleware(tls_middleware)
.set_rpc_middleware(
RpcServiceBuilder::new()
.option_layer(rpc_filter)
.option_layer(capability_filter),
)
.to_service_builder()
.build(methods, stop_handle.clone());Admission control reimplements Core's -rpcthreads/-rpcworkqueue contract on an async runtime — a semaphore bounds in-flight calls, a counter bounds the backlog, and everything past the budget is shed before auth or body buffering does any work. One documented divergence: satd sheds with HTTP 429 + Retry-After where Core returns 503.
//! The read-only listener runs on the same separate, bounded tokio runtime
//! as the high-volume read surfaces (Esplora/Electrum/gRPC) so a flood of
//! consumer RPC traffic can never starve block connection or mempool
//! acceptance. But that isolation comes with a hard correctness constraint
//! discovered while moving surfaces off the core runtime:
//!
//! > A surface that **connects blocks** must not run on the API runtime.
//! > `connect_block` writes the address index inline and *then* broadcasts
//! > `ChainEvent::BlockConnected`; the address-index status notifier is an
//! > independent core-runtime consumer of that broadcast. When the
//! > broadcast originates from the API runtime, the cross-runtime wakeup
//! > can reorder the notifier ahead of the inline index write becoming
//! > visible, delivering a stale all-zeros status to SSE/Electrum
//! > subscribers.tls-config crate, and splits read traffic onto an isolated runtime (--api-threads) so consumer load can't contend with the threads that connect blocks.Part 6 · RPC & Surfaces
//! Every Core-ecosystem client built on the canonical libraries sends
//! the 1.0 form. NBitcoin (and therefore NBXplorer and BTCPayServer)
//! sends `"jsonrpc":"1.0"`; `python-bitcoinrpc`, many shell scripts, and
//! older tooling omit the member entirely. Against an unpatched
//! jsonrpsee, *every* call from those clients fails — which is exactly
//! the failure the NBXplorer compatibility canary surfaced (the indexer
//! could open a P2P connection but every `getblockchaininfo` RPC came
//! back `Invalid request`, so it never synced).
//!
//! This HTTP-level tower layer runs *before* jsonrpsee parses the body.Core emits BTC amounts as %d.%08d — always 0.00001000, never 0.00001. Strict parsers (Core Lightning's bcli) reject the shortest form, so serde_json's arbitrary_precision feature is on workspace-wide and amounts are formatted from integer satoshis:
/// Format an integer satoshi amount as a Bitcoin-Core-exact fixed
/// 8-decimal BTC value (`<whole>.<8-digit frac>`) emitted as a JSON number
/// literal. Integer arithmetic only — no `f64`, so exact for all amounts.
fn btc_fixed_8dp(sats: u64) -> Value {
let whole = sats / 100_000_000;
let frac = sats % 100_000_000;
let literal = format!("{whole}.{frac:08}");
// Under `arbitrary_precision`, `Number: FromStr` preserves the exact
// source text, so this serializes as `0.00001000`, not `0.00001`.
Value::Number(literal.parse()
.expect("fixed 8-decimal literal is always a valid JSON number"))
}The opt-in flip side: any request can ask for "amounts": "sats" and get exact integers — closing Core issue #3249, open since 2013 — verified by a "units": "sats" field in the response.
/// A structured RPC error. When rendered to a jsonrpsee `ErrorObjectOwned`:
/// - With extended-errors disabled (default): emits only `code` + `message`
/// (byte-identical to Bitcoin Core).
/// - With extended-errors enabled: also populates `data` with
/// `category`, `suggestion`, and `debug` when present.
#[derive(Debug, Clone)]
pub struct RpcError {
pub code: i32,
pub message: String,
/// Stable dashboard-friendly taxonomy. Examples:
/// - `mempool.policy.feerate`, `mempool.conflicts`
/// - `validation.consensus`, `rpc.input.parse`
/// - `storage.not_found`, `node.shutting_down`
pub category: &'static str,
pub suggestion: Option<String>,
pub debug: Option<Value>,
}The shared pattern across all three: the Core-shape default is byte-preserved, extensions are opt-in per request or per flag, and matching Core's leniency is a named Tier 1 obligation in STABILITY_POLICY.md.
Part 6 · RPC & Surfaces
Cookie, userpass, and -rpcauth HMAC stay Core-compatible and always resolve first (legacy clients see zero change, and the operator principal holds all capabilities so the filter is a no-op for them). The satd extension is opt-in bearer tokens — SHA-256-hashed in a reloadable TOML store, each carrying a capability set — factored into a transport-agnostic crate shared by JSON-RPC, Esplora, Electrum, MCP, and the events gRPC:
pub enum Capability {
/// Read-only JSON-RPC methods.
RpcRead,
/// Mutating JSON-RPC (`sendrawtransaction`,
/// node/index control, mining).
RpcWrite,
/// Esplora REST / SSE.
EsploraRead,
/// Open a streaming subscription (gRPC events).
StreamSubscribe,
/// Register outpoint/script/descriptor watches
/// (gated by `watch_quota`).
StreamWatch,
/// MCP tool access (wildcard `mcp:*`).
McpAll,
}// `CapabilitySet` is a `u16`, so the vocabulary must
// stay ≤ 16 entries — a 17th would make `1u16 << 16`
// over-shift (debug panic / release wrap → two
// capabilities aliasing one bit → silent privilege
// grant). This fails the build the moment that
// ceiling is crossed; widen `CapabilitySet` to `u32`
// then.
const _: () = assert!(
ALL_CAPS.len() <= 16,
"CapabilitySet is u16: widen it before adding \
a 17th capability"
);The per-method gate is fail-closed — an unclassified method requires the write capability, so a read-only token can never reach a method someone forgot to classify:
/// The capability a method requires. Read-classified methods need `rpc:read`;
/// everything else — mempool-submit, control, block-connecting, AND unclassified
/// (unknown) methods — needs `rpc:write`. Fail-closed: an unknown method can
/// never be reached by a read-only token.
fn required_capability(method: &str) -> Capability {
match classify(method) {
Some(RpcAccess::Read) => Capability::RpcRead,
_ => Capability::RpcWrite,
}
}Small touches with the same posture: the cookie file's permission mode is applied at open(2) time so the secret is never briefly world-readable; token comparison is constant-time; per-principal rate limits shed with 429 inside the same layer.
-rpcwhitelist per-user method list — no scoped tokens, no capability model, no shared auth across surfaces (each side-car brings its own).Part 6 · RPC & Surfaces
Bitcoin Core operators run electrs or Fulcrum as a separate process that re-indexes the whole chain, plus an Esplora deployment for REST. satd serves both wire formats in-process, over the same AddressIndex trait against the shared RocksDB — no second copy, no parallel rescan, no reorg races between processes.
/// `/blocks/tip/hash`. Plain-text big-endian hex
/// (matches upstream).
pub async fn tip_hash(
State(state): State<EsploraState>,
) -> String {
state.chain.tip_hash().to_string()
}Wire-shape parity with blockstream.info / mempool.space so BDK and the mempool.space SDK deserialize unchanged. SSE streams embed their concurrency permit in the response stream — tower's limit layer bounds request handling, not stream lifetime, so without it an attacker could pin sockets indefinitely.
pub fn scripthash_get_balance(
state: &ElectrumState, params: Value,
) -> Result<Value, JsonRpcError> {
let sh = parse_scripthash(¶ms,
"blockchain.scripthash.get_balance")?;
let (confirmed, unconfirmed) = state
.address_index
.balance(&sh.0)
.map_err(JsonRpcError::from_index)?;
Ok(serde_json::to_value(BalanceResponse {
confirmed, unconfirmed,
}).unwrap())
}28 methods; the protocol shape (method names, status hashes, merkle encoding) follows romanz/electrs with MIT attribution vendored in-tree. The implementation is satd's, one process away from the consensus core.
get_history asks the index for at most cap + 1 distinct (height, txid) entries: the index emits one row per matching output and input, so a fixed raw-row scan factor could truncate before reaching the cap and silently return partial history. The helper streams funding and spending in lockstep and dedupes inline — the kind of bug the review process caught before a wallet did.Part 7 · Indexes
Address history, outpoint-spend, tx index, BIP 158 filters, and BIP 352 silent-payment tweaks all ride the same StoreBatch as the chainstate. There is no index-writer thread and no post-commit hook — emission happens inline in connect_block's loop, right next to the coin mutation it describes:
batch.coin_removes.push((outpoint, coin.amount, coin.height));
// Address-history index: spending row, atomic with the
// chainstate update via the same StoreBatch. No-op when
// the index is disabled.
crate::index::address::emit_spending(
&mut batch, address_index, height, txid,
in_idx as u32, &coin, outpoint,
);
// outpoint_spend index: keyed by the consumed outpoint
// so Esplora outspend / gettxspendingprevout can answer
// in O(1). Same flag, same atomic batch.
crate::index::outpoint_spend::emit::emit_spend(
&mut batch, address_index, height, txid,
in_idx as u32, outpoint,
);The commit side states the guarantee — and even runs it in reverse: disabling an index while blocks connect clears its completeness marker inside the same batch, so the "this index is trustworthy" bit can't be lost to a crash window:
// BIP 158 filter index. Filter blob and chained filter header
// ride the same atomic batch as the chainstate update, so a
// crash mid-write rolls everything back together — protocol
// handlers can never observe a filter row whose chain segment
// is partially committed. Empty-batch fast-path skips both CFs.BaseIndex framework runs each index as a background thread consuming validation events into its own LevelDB, with its own "best block" pointer — an index can lag the tip, and callers must handle "index not synced". That design is why Core deliberately stays out of address indexing at scale, and why electrs exists. satd's single-batch design makes tip-consistency structural — the trade is that the connect path pays index write costs inline, and a fully-indexed node spends serious disk (the address index is ~120–180 GB compressed at mainnet tip, documented and opt-out via --addressindex=0).Part 7 · Indexes
Two column families, one 56-byte key shape. Every field is big-endian and the scripthash prefix leads, so RocksDB's own byte ordering is (height, txid, vout) ascending for a fixed script — get_history is a straight prefix iteration with no sort:
addr_funding_v2 key: scripthash_prefix[16] || height_be[4] || txid[32] || vout_be[4] (56 bytes)
value: amount_sat_be[8] (8 bytes)
addr_spending_v2 key: scripthash_prefix[16] || height_be[4] || txid[32] || vin_be[4] (56 bytes)
value: prev_outpoint_txid[32] || prev_outpoint_vout_be[4] (36 bytes)Truncating the scripthash to 16 bytes saves ~16 bytes per row — "the bulk of the disk-size delta against Bitcoin Core + electrs" — and the module doc defends the collision posture rather than hand-waving it:
//! ## Collision posture
//!
//! Scripthashes are `sha256(scriptPubKey)`. A 16-byte prefix gives
//! 2^128 codomain; birthday collision probability at 2^32 entries is
//! ~2^-64 — vanishingly small for honest workloads. A deliberate
//! collision is feasible at ~2^64 hashing work, but the attack outcome
//! is "querying scripthash X also returns events for scripthash Y" —
//! both X and Y are public on-chain data, so no privacy or correctness
//! violation results for the address-index use case.The portable half (trait, key codec, cursor types) lives in the leaf crate node-index so Electrum, Esplora, and SDK consumers depend on it without pulling in ChainState. The companion outpoint_spend CF answers the inverse question the UTXO set can't — "this output is gone, what spent it?" — keyed txid-first so /tx/:txid/outspends fans out in one seek. Its read adapter is fail-closed: a miss is only reported as "unspent" when the completeness marker covers the whole active chain; otherwise it returns Incomplete rather than a plausible lie.
Part 7 · Indexes
satd doesn't reimplement Golomb-coded sets — it feeds bitcoin::bip158::BlockFilter a resolver backed by the prev-output script map that connect_block already built for script verification, so the filter's element set ("this block's output scripts ∪ the scripts it spent") is computed from work the validator was doing anyway. The BIP 157 header chain is one hash step off the previous height's stored header, committed in the same batch — so a getcfheaders responder can never see a half-committed segment. With --peerblockfilters=1 the node advertises NODE_COMPACT_FILTERS and serves the modern light-client path (Zeus-embedded, Blixt, Mutiny).
One 33-byte public tweak per eligible transaction — T = input_hash · A — which is exactly what a scanning wallet needs and nothing it shouldn't have. The kernel is shared verbatim between the node's writer and the SDK's client-side scanner, so stored and recomputed tweaks are identical by construction:
/// Compute the public tweak `T = input_hash · A` for a transaction, or
/// `None` if it is not silent-payment eligible / must be skipped.
///
/// Skip conditions (facts 1–5): coinbase; no taproot output; spends a
/// future-SegWit output; no contributing inputs; input pubkey sum is the
/// point at infinity; `input_hash` is zero or ≥ the curve order.
pub fn compute_tweak(tx: &Transaction, prevout_spks: &[ScriptBuf]) -> Option<TweakEntry> {Each row is keyed by height but embeds the hash of the block it describes — self-authentication born directly from this codebase's height-index scars:
//! The embedded `block_hash` makes each row **self-authenticating**: any
//! reader (the D4 rescan fast path, streaming replay, the fallback RPC)
//! verifies the row describes the block it expects without consulting the
//! height→hash index — which this codebase has learned never to treat as
//! truth (the #322 accept_header clobber, the testnet4 MTP wedge). A
//! mid-read reorg surfaces as a hash mismatch and the reader falls back
//! or rejects rather than trusting height alone.Part 7 · Indexes
backfillindex <address|blockfilter|silentpayment> walks history while the node keeps validating new blocks. The safety argument is a height partition, stated identically in all three runners:
//! ## Concurrency with live `connect_block`
//!
//! - Backfill writes addr-CF rows for heights ≤ snapshot
//! - Live writes addr-CF rows for heights > snapshot
//! - Disjoint height prefixes → no key collisions; an exact-key
//! duplicate via reorg-disconnect→reconnect at h ≤ snapshot is
//! caught by the per-batch reorg check above and aborts the run.The one way the partition breaks is a reorg dragging the anchor off the active chain — so the runner re-verifies the anchor by walking back from the live tip before and after every single-block batch, and the cursor advance rides the same batch as the rows it describes, making resume-after-kill -9 exact. Pre-flight disk checks refuse to start under known mainnet footprints: 80 GB free for address (the temp CF peaks ~56 GB), 10 GB for filters (~6 GB of blobs), 6 GB for SP tweaks (~3.9 GB).
The shared rate meter behind getindexinfo's time-remaining estimate is a bug story in module form — the old formula divided wall-clock-since-first-start by progress, so a 48-hour pause was extrapolated as if the walk had been grinding through it:
//! ## Why nothing is persisted
//!
//! #546 suggested persisting an `elapsed_working_seconds` (or a resume
//! height) in the cursor. This deliberately does not: an accumulator has
//! to be flushed periodically or a `kill -9` loses it, and a persisted
//! anchor goes stale across exactly the crash-and-restart case the fix
//! exists for ... An in-memory anchor cannot survive the downtime by
//! construction. The cost is a few seconds of "no estimate" after a
//! restart while the new stint accumulates./// `#[must_use]` because binding it to `_` instead of a named `_stint`
/// drops it immediately, clearing the anchor before the first block is
/// walked and silently disabling every ETA. That is a one-character
/// mistake with no other symptom.
#[must_use = "the stint ends the moment this guard is dropped; bind it for the \
duration of the walk (`let _stint = ...`), not to `_`"]
pub struct StintGuard(Arc<StintMeter>);Part 8 · Streaming & Ops
The spec (docs/api/streaming.md) names the three gaps every existing node API leaves — descriptor lifecycle, outpoint-level subscription, cursor-based replay — and builds one surface to close them, with outpoint subscription as the base primitive. Every event is a versioned, edge-stamped envelope carried identically over gRPC, WebSocket/SSE, and ZMQ:
message Cursor {
uint32 height = 1; // block height of the last delivered confirmed item
uint32 tx_index = 2; // index within that block of the last delivered tx
uint64 mempool_seq = 3; // best-effort mempool high-water (advisory)
// Per-process epoch nonce of the issuing publisher. node_id is stable
// across restarts, but mempool_seq resets to 0 each daemon start; on a
// from_cursor resume the server discards mempool_seq when this differs
// from the live instance (daemon restarted since the cursor was issued).
// Confirmed (height) replay is instance-independent and unaffected.
uint64 instance_id = 4;
}
// Per-event identity stamp applied at the publisher's bridge layer.
message EdgeStamp {
bytes node_id = 1; // 16 raw bytes (UUIDv4)
string region = 2; // up to 8 ASCII bytes; empty = unset
uint64 edge_seen_at_ns = 3; // monotonic since publisher start
uint64 edge_wall_ns = 4; // wall-clock ns since Unix epoch
uint64 seq = 5; // monotonic per publisher, starts at 1
}The categories bitmask encodes a compatibility promise: two categories are excluded from the categories = 0 "all" default, so a pre-existing subscriber never starts receiving a new event type after a node upgrade:
/// Categories excluded from the `categories = 0` ("all") default: opt-in,
/// high-volume, or custody-adjacent streams a legacy subscriber must not begin
/// receiving after a node upgrade.
pub const EXPLICIT_ONLY_CATEGORIES: u32 = CATEGORY_TWEAKS | CATEGORY_STATUS;Replay reads the durable block store (there is no separate event log), resolves heights by walking prev_blockhash from the tip rather than the pollutable height index, caps spans at 10,000 blocks, and de-dups the replay→live seam by (height, hash) — keying on the hash is what makes a reorg at the seam correct.
-zmqpub*: one raw-bytes ZMQ topic per event type, no provenance, no ordering guarantee across topics, no replay — a missed message is simply gone, so every consumer rebuilds state by polling RPC. satd deliberately does not implement Core's topic scheme; the structured envelope replaces it, with a documented migration path.Part 8 · Streaming & Ops
Each connection registers a watch-set — outpoints, scripthashes (with per-script value floors), txid lifecycles, depth alarms, privacy-preserving k-bit script prefixes, BIP 352 scan keys. Matching is O(1) per transaction via inverted indexes, and lock-free counters gate every expensive step so an unwatched node does no extra work at all:
/// Registry of per-subscriber outpoint/script watch-sets with O(1)
/// matching. Cheap to consult when empty (a single atomic load).
pub struct WatchRegistry {
inner: RwLock<Inner>,
next_id: AtomicU64,
/// Lock-free count of registered watch *items* (outpoints + scripts)
/// across all subscribers. The matcher checks this before re-reading a
/// block, so a node with no watchers does zero extra work.
watch_items: AtomicUsize,
/// Lock-free count of registered *script* items only. Gates the
/// input-side script match: the matcher fetches a block's undo data (to
/// recover spent prevout scriptPubKeys) only when some script is watched,
/// so an outpoint-only watch-set pays nothing extra.
script_items: AtomicUsize,Spend-side script matching is the subtle half: a spending transaction doesn't carry the prevout's script, so confirmed spend matches are recovered from the block's undo data — and mempool entries capture sha256(scriptPubKey) per input at admission so the matcher can match mempool spends without a UTXO lookup. Scan-key secrets are zeroize-on-drop, never persisted, never logged.
Part 8 · Streaming & Ops
The Rust SDK (satd-events-client, 9.3k lines) and Go SDK (clients/go, package satdevents, 24k lines) are full peers: same resilient-client design, same twelve mirrored examples, and a differential parity harness (clients/go/cmd/paritydump) that drives both against one node and diffs rendered events. The Go E2E suite gates every satd PR.
//! [`ResilientSubscription`](crate::ResilientSubscription) wraps the one-way
//! `Subscribe` firehose; [`ResilientWatch`] is its twin for the bidirectional
//! `Watch` stream. It exists because the two recovery stories differ:
//!
//! - **The watch-set is per-connection.** The server holds no principal-keyed
//! state — when a `Watch` stream drops, its server-side watch-set and quota
//! leases are torn down with it. A reconnect therefore starts blank, so the
//! SDK has to **mirror** every add/remove the caller makes and **re-register**
//! the whole set on the new stream before anything matches again.
//! - **Re-anchor is in-band and deterministic (#439/#441).** Once the watch-set
//! is back, a single `set_cursor` replays confirmed history; the server
//! answers with exactly one [`Event::CursorAccepted`] ... or
//! [`Event::CursorRejected`]. [`ResilientWatch`] drives its catch-up off those
//! deterministic results instead of inferring a gap from the event flow.The SDKs earn their keep in the details — here's the Go client closing a race grpc-go leaves open:
// Wait for the server's response headers before handing the stream back.
//
// grpc-go starts a server-streaming call without waiting for the server to
// accept it, so without this Subscribe returns before the node has
// registered the subscription - and anything the caller triggers in that
// window (mining a block, broadcasting a transaction) is simply not on the
// stream. That silent race is a bad default for an event API: "subscribe,
// then do the thing" has to work.
if _, err := sc.Header(); err != nil {Part 8 · Streaming & Ops
Six conditions the daemon detects about itself — tip stall, low disk, mempool congestion, peer starvation, IBD completion, deep reorg — each raised simultaneously as a streaming status event, a getwarnings entry (which drives Core's -alertnotify), and a Prometheus gauge, all from a single transition so the three surfaces can never disagree. Detectors are level-triggered with hysteresis gaps, because clearing at the raise threshold turns a hovering metric into a pager storm. One detector carries a sharp design note:
/// No block connected for longer than the configured window. Clears when
/// the next block connects.
///
/// Deliberately *not* suppressed during initial block download:
/// `is_initial_block_download()` compares the tip header's timestamp
/// against the wall clock rather than tracking sync progress, so a node
/// that was caught up and then wedged re-enters it exactly when the
/// operator most needs paging. A node that is genuinely syncing connects
/// blocks continuously and never crosses the threshold on its own.
TipStall,Outbound webhooks come from alertfile= — a TOML file because a hook has a secret and a filter, and Core's flat first-wins config can't express several of those. Every delivery is HMAC-signed over a versioned signing string (timestamp, delivery ID, hook ID, body) with idempotency and attempt headers; a misconfigured receiver's 404 is dropped after retries rather than pinning the queue — "skipping the event loses one delivery; retrying it forever loses all of them."
/// One configured webhook.
///
/// `Debug` is hand-written: a derived one renders `secret` in full, and a
/// single `tracing::debug!(?hook)` added later would put a signing key in the
/// log. The crate already takes this posture for watch-set scan keys.
#[derive(Clone, PartialEq, Eq)]
pub struct Hook {-alertnotify but raises almost nothing through it in practice — most operators have never seen it run. There is no native health model, no webhook surface, no metrics endpoint; the ecosystem answer is external exporters with inconsistent metric names.Part 8 · Streaming & Ops
--metricsbind serves Prometheus text format plus /healthz and /readyz probes. The metric schema (satd_*, Prometheus suffix conventions) is a declared stability commitment, and the rendering layer encodes a parser landmine: a repeated # HELP makes strict parsers discard the entire page, so headers and samples are separate functions and a test parses the rendered page to assert exactly one header per family.
//! Two dedicated `std::thread`s — deliberately not tokio tasks. During the
//! 2026-05 mainnet IBD wedge the entire tokio runtime parked itself (every
//! worker in `futex_do_wait`, no progress for nine hours, RPC accept queue
//! piled up to 3084 backlogged connections). A watchdog scheduled on the
//! same runtime that froze would have frozen with it, so this module spawns
//! its own OS threads.On stall it dumps per-thread kernel state from /proc/self/task/* for the post-mortem, then SIGTERMs itself — abort() only if the graceful path is also stuck.
The TUI adapts to what the node is doing — IBD bitmap view, steady-state, mempool, chain — plus panes for reorgs, warnings, and RPC failure. sat-cli is a two-level noun verb tree (chain info, mempool top, debug blockfile-audit) with pretty output by default and -o json for scripts; the legacy raw form (sat-cli getblockchaininfo) still works. PSBT signing is local-only: keys read from stdin, never sent over RPC.
The Model Context Protocol listener re-exports the node's own RPC layer as typed tools for AI agents — each #[tool] is a one-line delegation into the same functions JSON-RPC calls, with schemas generated from the same Rust parameter structs the handlers destructure. No HTTP hop, no second implementation, and its metrics flags mirror the Prometheus context so both surfaces report identical state by construction.
bitcoin-cli plus debug.log. No TUI, no MCP, no native metrics; the watchdog role falls to systemd and the operator's own alerting.Part 9 · satd vs Core
Both behaviors sit inside the compatibility envelope; only the default differs. Operators who need Core's default set the corresponding flag.
| Default | Bitcoin Core | satd | Reasoning |
|---|---|---|---|
| Esplora REST listener | not present | on (loopback, unauth) | Native surface; loopback default keeps exposure safe. --esplora=0 to disable. |
| Address index | not present | on | Required by Esplora and Electrum. --addressindex=0 on storage-constrained boxes. |
| /metrics server | not present | off | Enable with --metricsbind. |
| Electrum server | not present | off | Enable with --electrum=1. |
| Block-filter index | off | off | Matches Core; --blockfilterindex=basic. |
-mempoolfullrbf | on (v28+) | on | Matches Core post-v28. |
-v2transport | on | on | Matches; satd adds opt-in -v2only. |
-listenonion | on (silent no-op without Tor) | off; on if -torcontrol set | Avoids dialing the Tor control port on every boot; explicit -torcontrol implies it on. |
| New blocks dir obfuscation | XOR on (v28+) | plaintext unless blocksxor=1 | Either way the key lives in blocks/xor.dat, so each can read the other's directory. |
| Log destination | debug.log + rotation | stdout only | Rotation delegated to journald/container runtime — which frees SIGHUP for config reload. |
Part 9 · satd vs Core
Each exclusion is a scope decision, not a gap. The pattern: drop surfaces that are deprecated, privacy-hostile, or better served by the modern replacement satd ships natively.
| Excluded surface | Why | The satd answer |
|---|---|---|
| Wallet (BDB legacy, WIF-keyed RPCs, descriptor GUI) | Keyless daemon by charter — no private keys in the process, ever. | Full PSBT construction/decode/analyze/combine/finalize over RPC; sat-cli signpsbtwithkey signs client-side, key read from stdin, never traverses RPC. |
| BIP 37 bloom filters | Deprecated in Core since v0.19; privacy leak and DoS vector. | BIP 157/158 compact filters, indexed and served natively. |
MemPool P2P message | Rarely used outside bloom-filter clients. | The mempool subscription stream over RPC/WS. |
Core-style -zmqpub* raw topics | One raw-bytes topic per event type; no provenance, no replay. | Structured event envelope (gRPC + ZMQ frames) with node identity, heartbeat, and cursor replay. |
| GPG release signing | Project-wide signing-stack decision. | minisign (artifacts) + cosign keyless (containers) + SSH signatures (git tags). |
Part 9 · satd vs Core
Compatibility claims are backed by machinery, running in CI on every PR:
libbitcoinconsensus, asserting identical accept/reject. Zero divergence.Part 9 · satd vs Core
A Core datadir is not byte-compatible (LevelDB vs RocksDB), but the flat block files are — including Core v28's XOR obfuscation, whose key is read automatically from blocks/xor.dat.
bitcoind; keep blocks/ to skip re-downloading the chain. Core's rev*.dat undo files and blocks/index are ignored — satd keeps undo data and the block index in RocksDB.chainstate/, indexes/, wallets/ aside; satd doesn't read them.bitcoin.conf. -reindex-chainstate replays the flat files into RocksDB.backfillindex address / blockfilter / silentpayment — backfills run concurrently with live validation, so the node serves correctly with partial history while they progress.One caveat: peers.dat shares Core's filename but uses a satd-native versioned format (magic SADR). An unrecognized file is silently discarded and the address set rebuilt from DNS seeds.
Closing
CORE_DIFFERENCES.md — the authoritative catalog of every intentional deviation from Bitcoin Core.STABILITY_POLICY.md — the Tier 1/2/3 compatibility contract and deprecation staging.docs/api/streaming.md — the wire-level streaming API specification.MANIFESTO.md — why a second implementation, in the project's own words.Generated from satd master 4874b537 (2026-08-19) · every snippet verbatim from source
Press t or esc to close.