Ledger implementation

These pages are for a reader who wants to open the code of the ledger or to build on it. The concepts, what the ledger records, how a block is agreed and why it needs more than two thirds of the validators, are on the Ledger page. This page gives the module, its classes and its API. The detail is on these pages:

Page What it covers
Ledger encoding The bytes of every record and message, what each signature covers, hashes, roots and references
Ledger state and rules The state, the order of every check, fees, rounds, the waiting transactions and heartbeats
Ledger consensus engine The voting engine, the sign state and the node that ties them to a ledger
Ledger storage The block log, the sign state file and the replay at start
Ledger networking The driver: intake, limits, gossip, sync, promotion and halting, and the transport over QUIC
Ledger in the daemon The service, the genesis and key files, the snapshot, the authority over the sporks

The module

The ledger is the Maven module ledger, artifact hedgehog-ledger, with one package, org.unigrid.hedgehog.ledger, exported by its module-info.java. It has no socket code, no files of its own choosing, no options and no threads. The application module depends on it (requires org.unigrid.hedgehog.ledger) and supplies all of those, in the package org.unigrid.hedgehog.service.ledger.

   
Dependencies BouncyCastle bcprov-jdk18on 1.85, SLF4J 2.0.20, Lombok 1.18.48
Test dependencies jqwik 1.10.2, Hamcrest 2.2, Jimfs 1.3.2
Compiler release 25, set in the parent pom.xml
Language features Records, sealed Transaction and Address, pattern-matching switch, switch expressions
Run its tests mvn -pl ledger test from the repository root, -Dtest=BlockLogTest for one class

The classes

Rules and state, see Ledger state and rules

Class Job
Ledger Implements ConsensusApp: ties the state, the waiting transactions and the block log together
LedgerState Balances, references, votes, validators, supply and the tip. Holds every validation rule and computes the state root
LedgerAuthority What the ledger asks the node: which mints are due, and which mints, claims and transfers it accepts
LedgerGenesis, Snapshot The start of the chain with the hash and total of the legacy snapshot, and the balances behind it
MintSchedule Turns a height of the legacy chain into a time
Mempool Waiting transactions and the newest heartbeat of each validator
ValidatorSet The ordered keys with the proposer and quorum rules
VerifiedSignatures Remembers the claims and transfers whose signatures were checked

Data, see Ledger encoding

Class Job
Transaction, Mint, Vote, Claim, Transfer The four kinds of transaction. Transaction is sealed, so there are no others
Address, AccountKey, LegacyAddress An Ed25519 public key or a 20-byte legacy address, and the sealed type over both
LegacySignature Verifies a signature of the legacy wallet, and recovers the public key that made it, which the wallet does not print
Reference A 32-byte single-use mint reference, and the two ways to derive one
Block, Certificate, Endorsement, Precommit A header with its certificates and transactions, a view with its signatures, one signature, and what it covers
Heartbeat, Status A validator’s signed statement that it sees a tip, and an unsigned claim of where a chain stands
BlockCodec, TransactionCodec, ConsensusCodec, WireCodec Byte encoding of blocks, transactions, consensus messages and the rest
MerkleRoot, Digests, Ed25519, Hashing Hashing and signing, and a salted ranking of peer ids
BlockLog The append-only file of blocks and its crash recovery, see Ledger storage
BlockRequest, BlockBatch A request for blocks, and up to 16 blocks in answer

Consensus and node, see Ledger consensus engine

Class Job
Consensus, ConsensusApp The voting engine for one validator, and the interface it uses to ask the ledger
Message, Proposal, Ballot, Decision, BallotBox The three consensus messages and the box that counts the ballots of a height
Action, Timeout, Timeouts What the engine asks its caller to do, a timer id, and the waiting times
SignState, SignStore, SignStateFile What a validator has signed, and the file that keeps it
LedgerNode One ledger, one engine and one sign state tied together
ChainGuard Refuses blocks and statuses that contradict the chain held or a trusted checkpoint

Driver and network, see Ledger networking

Class Job
LedgerDriver The only caller of a node: intake, gossip, sync, promotion and halting
NodeSupervisor, NodeCall, Promotion, Recency Replaces a node that failed, decides when an observer may vote, and remembers recent trouble
Syncer, SyncState, PeerView Fetches blocks from a peer that is ahead, keeps the claims and doubts about peers, lists the peers
Gossip Passes on what is new and charges the sender for what is forged
Door, DoorKeeper, InboundLanes, QueueBudget, PeerLimiter Sorting, queueing and rate limiting of what arrives
LedgerNetwork, LedgerPeer, Serial, Scheduler What the application provides: peers, a serial executor and timers
NodeFactory, DriverOptions, Checkpoint Inside LedgerDriver: how a node is built, the settings, and the block a synced ledger must have

Errors are InvalidBlockException, CorruptLogException, CorruptSignStateException, FileLockedException and BlockLogBehindException. Bytes holds package-private helpers.

In the application, package org.unigrid.hedgehog.service.ledger, see Ledger in the daemon and Ledger networking

Class Job
LedgerService The ledger as a part of the daemon: starts, reads and stops it
LedgerNodes, LedgerRuntime The NodeFactory over the files of the node, and the Serial and Scheduler
LedgerSetup, LedgerKeyFile, GenesisFile The checks of the options at start, the validator’s seed, and the genesis as JSON
LedgerSnapshot, SnapshotCheck The bootstrap snapshot as the ledger’s Snapshot, and whether it is the one of the genesis
SporkLedgerAuthority The LedgerAuthority over the sporks
LedgerGateway, LedgerProtocol, NettyLedgerNetwork, NettyLedgerPeer The transport: the gate, the protocol, the peers and one stream
OutboundQueue What waits to be written to one stream, in two lanes with the votes first

Beside them are MintEntry and LedgerText in model.bootstrap, the sporks LedgerBlacklist and LedgerFee and the tracker SporkLag in model.spork, and LedgerOptions in command.option.

Reading and using the ledger from outside, see REST interface and Ledger command line

Class Job
LedgerResource The REST resource under /ledger: state, account, block and the submit of a transaction. It reads the newest ledger through LedgerService.read and submits through LedgerService.submit
LedgerInfo, LedgerAccount, LedgerBlock, LedgerTransactionView The JSON records of the answers, in server.rest.entity. A field that does not apply is left out
LedgerSubmission, LedgerSubmitted The body of a submit, and the answer to it
LedgerText How an address and an amount of coins are written and read, shared by the resource, the commands and the mint spork
LedgerCommand and the six commands beside it hedgehog cli ledger info, balance, block, transfer, claim and vote, in command.cli.ledger
LedgerClient, ChainOption The requests of the commands to a node and their exit codes, and the --chain option 
flowchart TD
    Service[LedgerService] --> Driver[LedgerDriver]
    Service --> Nodes[LedgerNodes]
    Gateway[LedgerGateway] --> Driver
    Driver --> Supervisor[NodeSupervisor]
    Driver --> Network[LedgerNetwork]
    Supervisor -.builds through.-> Nodes
    Supervisor --> Node[LedgerNode]
    Node --> Consensus
    Node --> Ledger
    Node --> SignState[SignStateFile]
    Consensus --> BallotBox
    Consensus -.asks.-> Ledger
    Ledger --> LedgerState
    Ledger --> Mempool
    Ledger --> BlockLog
    LedgerState --> Authority[LedgerAuthority]
    LedgerState --> ValidatorSet
    LedgerState --> MerkleRoot
    Authority -.implemented by.-> Spork[SporkLedgerAuthority]
    BlockLog --> BlockCodec
    BlockCodec --> TransactionCodec

Driving the ledger

A ledger is put together by the caller, in three steps:

  1. new LedgerState(genesis, authority), or with a Snapshot as a third argument;
  2. BlockLog.open(path), which creates the file and its directory if they are missing;
  3. new Ledger(state, log), with an optional clock as a third argument, which replays every block of the log into the state with all checks. A log that was tampered with makes the constructor throw InvalidBlockException. A replay is vouched, so it ignores what the authority says now.

The caller owns the log and closes it. The waiting transactions are kept by a Mempool that the ledger creates itself, with room for 10,000 of them. A node is built on top of that, see Ledger consensus engine, and in a running system the driver builds the nodes through its NodeFactory. In the daemon that is LedgerNodes, and LedgerService builds the driver.

Ledger implements ConsensusApp, see Ledger consensus engine, and adds:

Method What it does
submit(Transaction), submit(Heartbeat) Empty when accepted, the reason otherwise
blocksFrom(from, max), hashAt(height) Read the log for peers, without waiting for a commit
block(height) One block of the log, empty below 1 and above the tip
status() The chain id, height and tip hash, as a Status
validators(), height(), tipHash(), chainId(), round(), roundEnds(), roundValidators() Where the chain stands: the set that signs blocks, the height and tip, the round of this node’s clock and when it ends, and the set that the round’s votes are judged by
supply(), balanceOf(Address), stateRoot(), pending() What it holds
nonceOf(AccountKey), claimedBy(LegacyAddress), claimable(LegacyAddress), mintedTo(Address) What the outside needs to build a transaction and show an account: the next nonce, who claimed an address, what a claim would move, and the sum of the mints made to an address
isProgressing(now, window) Whether more than two thirds of the validators sent a recent heartbeat

commit(block) is package-private: only LedgerNode commits. Every public method except blocksFrom and hashAt is synchronized; LedgerState itself is not thread-safe and relies on that.

The authority has five methods, all of them the node’s policy:

Method Asked
due(time) When a block is proposed or work is looked for. The mints this node knows to be due, in a fixed order; the ledger leaves out the ones already made
authorizes(Mint, time, supply, minted) When a mint is submitted, proposed or checked before a vote. The time is the node’s clock on submit and the block’s time otherwise. The supply, and the sum of the mints made to the recipient so far as the mints named it, are those of the state the mint would be made on
authorizes(Claim) The same, for a claim
authorizes(Transfer) The same, for the fee a transfer pays
leastNewAccountAmount() After authorizes(Transfer) said yes: the least amount that a transfer to an address the ledger holds no account for must carry. 0 means no least amount

authorizes(Claim) and authorizes(Transfer) have no default, so an authority that forgot one would not let anything through by accident; leastNewAccountAmount() defaults to 0. None of them is asked when a block that already carries a certificate is checked: the state is then vouched for, because more than two thirds of the validators asked their own authorities before they voted. The authority of the daemon is SporkLedgerAuthority, see Ledger in the daemon.

Errors and logging

Rejections that are expected, such as a bad transaction or block, are returned as a readable reason in an Optional<String>. An exception is thrown where the caller made a mistake or the data cannot be trusted: InvalidBlockException for a refused block in apply and commit; IllegalArgumentException for a malformed value or a second endorsement; CorruptLogException and CorruptSignStateException for a log or sign state that cannot be trusted; FileLockedException for a sign state held by another node; BlockLogBehindException for a log that lacks blocks the sign state has passed; and ArithmeticException where a sum would overflow, which a check turns into Could not be checked.

Logging is a warning when a torn record is dropped, one when a check could not be made, and what the driver reports when it halts or replaces a node. In the daemon, LedgerService logs the chain id at start, the wait for the snapshot, and once why the ledger is not running or has halted; SporkLedgerAuthority logs each cause of a refusal once, at debug level. A seed is never logged.

Tests

The tests use jqwik properties and examples, with Hamcrest assertions and a small Expect.assertThrows. Every test of the ledger module that touches a file uses a Jimfs file system.

Test Covers
AccountKeyTest, AddressTest, ReferenceTest Size, equality, order, both address kinds, copies, derived references
MintTest, VoteTest, ClaimTest, TransferTest, HeartbeatTest Encodings, and that every field is under the signature or the id
LegacySignatureTest Both key encodings, low s, a header that must match the key, ASCII only, a vector made by an independent implementation
TransactionCodecTest, BlockCodecTest, CertificateTest Round trips including the largest block, every truncation, trailing bytes, count limits
DigestsTest, MerkleRootTest, Ed25519Test Hashing, tree shape for odd and empty inputs, a published Ed25519 test vector
BlockTest, ValidatorSetTest, LedgerGenesisTest Header size, a hash that ignores the certificate, quorum for every set size, rotation by view, a chain id that commits to every field
MintScheduleTest The height-to-time rule, overflow and negative values
LedgerStateTest, LedgerClaimTest, LedgerTransferTest, LedgerFeeTest, LedgerConservationTest Mints, claims, transfers, fee payout, and that no money is made or lost
LedgerCertificateTest, LedgerLastCommitTest, LedgerVoteTest, LedgerRoundTest Exact quorum, views, the last commit, every vote rule, the end of a round, genesis validators never lost
LedgerClockTest, LedgerWorkTest, LedgerValidateTest, LedgerAuthorityTest, LedgerBlacklistTest Due mints, the 30-second limit, what counts as work, the supply passed to the authority, certified blocks that policy would refuse, a throwing authority
MempoolTest, MempoolTransferTest Slots, limits, the nonce window, replacing by fee, heartbeats and progress
LedgerTest, LedgerServeTest Propose, validate and commit end to end, restart from the log, serving blocks
BlockLogTest A torn tail at every cut, a zero tail, damage in the middle, an impossible length, a rolled-back write
ConsensusViewTest, ConsensusViewsTest, ConsensusRestoreTest One view, view changes, timers, locks, proofs, resting, restoring a saved state
ConsensusSimulationTest, LedgerNetworkTest Many nodes on a simulated network with delays, loss, partitions, crashes and restarts from disk
SignStateTest, SignStateFileTest, LedgerNodeTest, ChainGuardTest The format, every damaged byte, atomic save, the lock, save before send, a log behind the sign state
LedgerDriverTest, LedgerDriverSyncTest, LedgerDriverDoorTest, LedgerDriverNetworkTest, PeerLimiterTest, WireCodecTest Events, forwarding, scoring, halting, sync and promotion, queue limits, drivers on a virtual network

LedgerFixtures and CodecFixtures build keys, blocks and random valid values. FlakyChannel wraps a real file channel and fails a write after a chosen number of bytes; it is handed to the log through the package-private BlockLog.over. A simulated network, a virtual-time memory network and a manual scheduler drive the engine and the driver.

In the application, LedgerSyncNetworkTest runs real QUIC servers on one host and syncs, gossips and halts on a wrong checkpoint; NettyLedgerNetworkTest covers peer naming, ordering and limits; further tests cover the protocol gate, the pipeline and the options. LedgerFleetTest runs several daemons with their LedgerService in one process, see Ledger in the daemon.

The REST resource is tested by LedgerResourceTest on real ledgers, with every status of every endpoint, the limit of 16 waiting submits and the body limit. The commands are tested by LedgerReadCommandsTest, LedgerTransferCommandTest, LedgerClaimCommandTest, LedgerVoteCommandTest and LedgerClientTest against a fake node, and LedgerTextTest covers how addresses and amounts are read. LedgerCommandsAcceptanceTest runs the commands against the REST servers of a fleet of daemons, with a transfer, a claim signed as the legacy wallet signs it, a vote and a node without a ledger.

What the implementation does not do yet

  • No bundled genesis. A node runs a ledger only with --ledger-genesis.
  • A node whose ledger did not start still offers the ledger protocol. It drops what its peers send, and answers a request for blocks with an empty batch at once.
  • Heartbeats go unused. isProgressing has no caller.
  • No penalty for equivocation. The evidence is recorded and lost at the next start.
  • Waiting transactions are not stored. A restart of the process drops them. A node that replaces another in the same process takes over its waiting transactions, so a promotion keeps them.
  • Signed heights depend on the file. A sign state that is deleted or replaced by an older copy is not noticed, and the directory is not flushed on Windows or an in-memory file system.
  • A fork among an earlier validator set goes unseen. ChainGuard checks certificates against the current set only.
  • Everything is kept in memory. The log holds every block, the state every used reference, and a start replays every block and verifies every signature, twice for a validator.
  • Work is repeated. A block is checked with its certificate when the engine decides, again when the ledger commits it and a third time when the state applies it, and every waiting transaction is checked again afterwards.
  • No lock on the log file. Two processes could open the same block log.

In the source

  • ledger/src/main/java/org/unigrid/hedgehog/ledger/: the module, one package
  • ledger/src/main/java/org/unigrid/hedgehog/ledger/Ledger.java: the state, the waiting transactions and the log together
  • ledger/src/main/java/org/unigrid/hedgehog/ledger/ConsensusApp.java: the interface the engine asks the ledger through
  • ledger/src/main/java/org/unigrid/hedgehog/ledger/LedgerAuthority.java: the questions the node answers
  • ledger/src/main/java/org/unigrid/hedgehog/ledger/LedgerDriver.java: a node run against a network
  • application/src/main/java/org/unigrid/hedgehog/service/ledger/: the ledger in the daemon
  • application/src/main/java/org/unigrid/hedgehog/server/rest/LedgerResource.java: the /ledger endpoints, with their records in server/rest/entity/
  • application/src/main/java/org/unigrid/hedgehog/command/cli/ledger/: the commands of hedgehog cli ledger
  • application/src/main/java/org/unigrid/hedgehog/model/bootstrap/LedgerText.java: addresses and coin amounts as people write them
  • ledger/src/test/java/org/unigrid/hedgehog/ledger/ and application/src/test/java/org/unigrid/hedgehog/service/ledger/: the tests and their fixtures

Each subpage ends with the files of its part.

Table of contents