Grid sporks
After 0.0.8. The storage spork, type 1030, with its REST resource and
gridspork-getandgridspork-setsupport, is on themasterbranch and not part of a release yet. So are theblock,cliffandpercentfields of a vesting schedule, the ledger blacklist and ledger fee sporks, the communication spork, the mint storage entries that name a ledger account, and the board keys spork, type 3000, with the chain of changes to the board keys. Release 0.0.8 has the mint storage, mint supply, vesting storage and statistics key sporks, and all of the co-signing. Release 0.0.9 adds the board keys spork and the new vesting fields, and takes the storage, ledger and communication sporks only with the hidden--previewswitch, see Sporks a release node takes.
A grid spork is a small, signed settings record that every node on the Unigrid network holds and passes on to its peers. Sporks let the network’s key holders change shared parameters, such as the maximum coin supply or the rules for network storage, on every node at once, without a software release and without a consensus round. A node accepts a change only if two different trusted keys signed it, so no single key can alter the network on its own.
What a spork holds
Each spork has a type, a current value, the previous value, a timestamp for each and the signatures that authorise it. The previous value is always a full copy of the value it replaced, never a difference. A node keeps at most one spork of each type. A mint storage spork holds at most 16,384 entries, a vesting storage spork at most 4,096 schedules, and a signature history at most 4,096 entries.
| Type | Id | What it carries | A change is |
|---|---|---|---|
| Mint storage | 1000 | Amounts credited to an address at a chain height | A grow: adds or replaces entries |
| Mint supply | 1010 | The network’s maximum supply | A set: replaces the value |
| Vesting storage | 1020 | Vesting schedules per address | A grow: adds or replaces an address |
| Storage | 1030 | The parameters of Network storage | A set: replaces the value |
| Communication | 1060 | Switches on the SOCKS proxy and sets how its chains are built | A set: replaces the value |
| Statistics public key | 2001 | A public key held as text | Cannot be changed from Hedgehog |
| Board keys | 3000 | The chain of changes to the keys that sign sporks | A change: appends one change to the chain |
The Ledger adds two more, the ledger blacklist (1040) and the ledger fee (1050), and gives the mint sporks a use, see Ledger sporks.
A set replaces the whole value. A grow starts from the value already stored, or from the value of a proposal still waiting for its second signature, so several changes can go into one co-signature. No change can remove an entry. A spork of a type the node does not know is ignored and logged at debug level.
Sporks a release node takes
The storage (1030), ledger blacklist (1040), ledger fee (1050) and communication (1060) sporks set parts that
release 0.0.9 leaves off, and a node takes them only when it runs with the hidden --preview switch. Without
it a node drops such a spork from a peer once it has read its frame, before it notes any lag, and never
stores, holds, relays or sends one. POST /gridspork/wire answers 400 with This node takes no <TYPE> spork,
GET /gridspork/wire/{type} 404, and the REST resource of each type 503 with
This release does not take the <TYPE> spork. The listing of GET /gridspork and gridspork-log leave
those types out. A spork of them that spork.db still holds from a run with --preview stays in the file,
and nothing on the node acts on it; util spork-db treats every type alike. The gridspork-get and
gridspork-set subcommands of these four types are hidden from --help.
Mint storage (1000)
Holds amounts to be credited later: a map from an address and a block height of the legacy chain to an
amount of coins. One entry credits that amount to the address at that height. The legacy chain does not
mint these amounts, so Hedgehog adds those whose height lies above the current chain height to the
balance that /bootstrap/address reports, see Legacy chain snapshot.
The Ledger has no block heights, so it turns each height into a time: the time of the last block of the snapshot plus one minute for every height after it. It mints the amount when that time comes. Entries at or below the snapshot’s last height are already in the snapshot balances and are never minted again. An entry may also name a ledger account, with a time in seconds in place of the height, see Ledger sporks.
An entry with the same address and height as an existing one replaces its amount.
Read it with gridspork-get mint-storage. Propose an entry with
gridspork-grow mint-storage --address <address> --height <height> -D <amount>, with a board key in -k,
--key-file or $HEDGEHOG_BOARD_KEY; the command signs the next version on the machine and hands it to
the node in wire form. GET /gridspork/mint-storage/{address}/{height} reads one entry back. No JSON
request changes a spork.
Mint supply (1010)
Holds one number, the maximum coin supply, which is zero until a value is set. Hedgehog stores and
distributes it and shows it in GET /gridspork, and the Ledger mints nothing beyond it. Release
0.0.8 enforces nothing of it. It carries the governed flag, as do the ledger, communication and board keys
sporks, which is meant to send a change to a vote of the network. Nothing checks that flag yet.
Read it with gridspork-get mint-supply. Propose a value with
gridspork-set mint-supply -D <number> and a board key; the command signs it on the machine and hands it to
the node in wire form. No JSON request changes a spork.
Vesting storage (1020)
Holds one vesting schedule per address. A schedule has these fields:
| Field | Meaning |
|---|---|
amount |
The amount that vests; can be empty |
start |
When the schedule starts |
duration |
How long it runs |
parts |
The number of parts it is divided into |
cliff, percent, block |
Stored with the schedule; Hedgehog gives them no meaning (after 0.0.8) |
Hedgehog stores and distributes the schedules and calculates no releases from them. A node of 0.0.8 reads
each schedule without the three fields of 0.0.9, so the spork goes only to a peer that reads them, see
How proposals and sporks spread. A node of 0.0.8 sends its own vesting
spork in that layout, signed over it. A node skips the frame of a vesting spork whole when it comes over a
connection that did not settle on hedgehog/0.0.9, hedgehog-proxy/0.0.2 or the ledger protocol: it takes
nothing from it and keeps the stream open. A vesting spork that comes over no stream, such as one handed to
POST /gridspork/wire, is read in the layout of this release. Proposing a schedule for an address replaces
the schedule that address already has. A new address is an old-style Unigrid
address or a ledger account of 64 lower-case hex characters; the node refuses another at
/gridspork/wire with The vesting of <address> is to an address that is neither an old-style Unigrid
address nor a ledger account of 64 lower-case hexadecimal characters. No command sets this spork: it can
be submitted only in wire form, by a program that builds and signs the spork itself. Read it with
GET /gridspork/vesting-storage or GET /gridspork/vesting-storage/{address}.
Storage (1030)
Holds the parameters of Network storage and Erasure coding. Storage stays switched off on a node until a valid storage spork has been accepted. The values below are the defaults of a new proposal.
| Parameter | Default | Allowed | What it sets |
|---|---|---|---|
maxBytesPerNode |
10 GiB | 0 or more | Disk quota of a gridnode |
chunkSize |
1 MiB | A multiple of fragmentSize, large enough for its tag and manifest |
Size of an encrypted chunk |
fragmentSize |
64 KiB | Above 0 | Size of a fragment |
maxOuterDataChunks |
32 | 1 to 255 | Data chunks in a stripe |
outerParityPercent |
50 | 0 to 200 | Parity chunks of a stripe, as a percentage of its data chunks |
innerParityPercent |
50 | 0 to 200 | Parity fragments of a chunk; 16 data and 8 parity make the 24 guaranteed fragments |
maxParityPercent |
100 | innerParityPercent to 25500 |
Upper bound on the fragments of a chunk |
repairIntervalMinutes |
60 | 1 or more | Length of a repair epoch |
repairThresholdPercent |
50 | 1 to 100 | Share of the parity fragments that must be missing before repair starts |
tombstoneDays |
30 | 1 to 65535 | How long a delete is remembered |
manifestCopies |
3 | 1 to 16 | Copies of the manifest written per upload |
placementSlack |
8 | 0 to 255 | Extra gridnodes ranked beyond the fragments of a chunk |
extraPoolPercent |
20 | 0 to 90 | Share of the quota that the non-guaranteed fragments may use |
The stripe and chunk limits also require that data and parity together stay within 255 chunks, and the fragments of a chunk within 255 fragments. The REST interface answers 400 with the reason when a proposal breaks a rule. A node does not validate a value that arrives from a peer, but it keeps storage switched off while its stored value is invalid.
Read it with gridspork-get storage, from a node run with --preview. Propose a value with
gridspork-set storage -D '<json>' and a board key; the command prints the stored and proposed data, signs
on the machine and prints the answer, such as Accepted and the proposal with its digest. No JSON request
changes a spork.
Statistics public key (2001)
Holds a public key as text, empty by default. A value is hexadecimal digits and nothing else; the node
refuses another at /gridspork/wire with The public key of the statistics is hexadecimal digits and
nothing else. Nothing in Hedgehog reads it. It has no REST resource or CLI command, and Hedgehog never
creates a new version of it, so it exists on a node only when it arrives from a peer or is already in
spork.db. It is left out of the periodic publish described below.
Board keys (3000)
The board keys sign every other spork. Type 3000 carries the chain of changes to them: each change lists the keys that are active after it, and a node that holds no spork of this type goes by the keys built into the release. The chain is how the board replaces a key without a release.
The chain. The first change follows on the root, the SHA-512 digest of the tag hedgehog/board-keys/1,
the epoch number 1 and the built-in keys in ascending order. A change holds these fields:
| Field | Content |
|---|---|
sequence |
Its place in the chain, from 0 |
madeAt |
When it was made |
previous |
The digest of the change before it, or the root |
active |
The keys active after it, 2 to 16, in ascending order |
approvals |
A signature of the change by each key that approved it, at most 32, in ascending order of their keys |
A change holds when it follows on the change before it or the root, lists 2 to 16 keys of P-521 in
lower-case hex, in ascending order without repeats and not the same set as the keys before it, and carries
approvals in ascending order of their keys: a quorum of the keys active before it and every key it adds.
The quorum is max(2, n/2+1) of the n keys before it, so two keys approve a board of two or three and
three approve a board of four. No other key approves a change. An approval signs the tag
hedgehog/board-keys/change/1 and what the change decides, which is its sequence, its time, the digest
before it and its keys. The digest a change follows on is the SHA-512 of the tag
hedgehog/board-keys/link/1 and the same bytes without the approvals, as whoever holds a change can drop
an approval beyond the quorum.
A chain holds at most 64 changes, a change at most 16 keys and at most 32 approvals. A chain of 64 changes
has no next: util board-keys propose refuses another with Nothing was proposed: The chain of board keys
holds 64 changes, the most it can; a release with a new root starts a new one.
The head of the spork is signed and co-signed by two different keys of the active set that its own chain
ends in, whatever keys the node goes by at the time. Its signature log may name any key that was ever
active, the built-in keys included, so the history stays readable after a key is switched off. A node
replaces a version only with one that holds the stored chain and adds to it, or that holds the same chain
and is newer. No other chain takes its place, however new, and a change that follows on another version is
refused. The 60-minute window that bounds a data-changing sibling, and the first-version rule of the other
types, do not apply. An approval of a change and the head of a version of the board keys take signatures in
one form only: DER of the shortest length, with the lower of the two values of s.
Reading the chain. gridspork-get board-keys prints what GET /gridspork/board-keys answers:
| Field | Content |
|---|---|
epoch |
1 |
builtInKeys |
The board keys built into the release, or the private network’s --network-keys |
chain |
The changes, each with sequence, madeAt, previous, active and approvals, each approval with key and signature |
active |
The keys active now |
inactive |
The keys that were ever active and are not now |
A node that holds no spork of the type answers an empty chain, and active and inactive as the keys it
goes by.
On the network. A node publishes the board keys first, before the other sporks, to each peer every
three minutes and whenever a connection comes up. The spork travels only over a connection that settled on
hedgehog/0.0.9, hedgehog-proxy/0.0.2 or the ledger protocol, as a node of 0.0.8 cannot read the type, see
Peer-to-peer network protocol. A node of
0.0.8 refuses every spork signed by a key it does not know, so it must be upgraded before the board keys
change. It cannot read a spork.db that holds a board keys spork either, and moves the file aside as
spork.db.unreadable-<epoch seconds>.
How the board makes, approves and applies a change, the record of the longest chain, releases and forks, and a known limit of publishing over P2P are on Changing the board keys.
Who may change a spork
Trust comes from the board keys. The default of --network-keys is the four board keys of Stiftelsen The
Unigrid Foundation, and an operator can override it; a change of the board keys replaces them without a
release, see Board keys. An ordinary spork’s head must be signed by two different keys
active now, and every entry of its signature log by a key that was active at some time; the board keys spork
is signed by two keys of its own chain. A key that the board switched off signs nothing new, but the history
it signed stays valid.
The cli signing commands read the chain of the node they talk to and go by the keys it makes active;
--network-keys gives them the built-in keys that the chain is rooted in. A private network passes its own
list to its nodes and to every signing command.
A change needs two signatures, from different active board keys, and both cover exactly the same bytes: the type, flags, both timestamps, the current and previous values and the signature history described below. Those bytes are the spork’s own wire encoding, so a node that receives a spork from a peer checks exactly what the signer signed.
An incoming spork is compared with the stored one first, so a spork that is not newer costs no signature check. Two are checked although they are not newer: a next version stamped at most 60 minutes before a stored renewal made over the same version, and a copy of the stored version, with the same time stamp and digest, that arrives while the stored copy no longer counts under the active keys, see When a replacement is accepted. The checks are paid from the budget under Key facts, and a co-signed spork that is accepted gives its checks back, so a peer that sends several genuine sporks in a row has each of them checked. A proposal gives nothing back.
How a board member proposes and co-signs a change is on Changing a spork.
How proposals and sporks spread
Peers pass on only what they have not seen before, so a spork spreads through the network and then stops. Nodes also publish their stored sporks, except the statistics key, to each peer every three minutes and whenever a connection comes up, as far as the peer reads them, which is how a node that was offline, or is new, catches up.
What a peer reads follows from the protocol its connection settled on. The board keys, vesting storage and
storage sporks go only to a peer that settled on hedgehog/0.0.9, hedgehog-proxy/0.0.2 or the ledger
protocol, none of which release 0.0.8 offers: that release does not know the types of the board keys and
storage sporks, and reads each vesting schedule without cliff, percent and block. The communication
spork goes only over the proxy or the ledger protocol, and the ledger blacklist and fee only over the ledger
protocol. This holds for the publish, the relay of a spork that came in and what the REST interface hands
on.
The transport is described in Peer-to-peer network protocol.
What a node keeps. One background thread saves spork.db every three minutes, and the node saves it once
more when it stops. A save writes the file only when the database differs from what the node last wrote to it
since it started, or when the file is gone. On start, before it reads the file, a node removes the files
that an unfinished save left beside it, named spork.db.<number>.tmp. It keeps from the file only an
ordinary spork that two different board keys signed and co-signed, though not necessarily keys active now,
with a signature log that board keys signed; an ordinary spork whose signers were switched off since stays,
and counts for nothing until it is signed again or a copy of it that two different active board keys signed
arrives. It leaves out a spork in the slot of another type, one stamped further ahead than clocks drift
apart, one with an amount beyond the bounds of the wire, and a board keys spork whose chain does not hold.
Before it drops anything it copies the file to spork.db.unsigned-<epoch seconds> and logs one warning per
dropped type naming the copy, such as
The stored MINT_STORAGE spork is left out: two different board keys did not sign it. The file as it was is
kept as <path>. A stored spork stamped further ahead is logged at error level: The stored <TYPE> spork is
stamped at <time>, further ahead of the clock of this machine, <time>, than clocks drift apart: the clock of
this machine may be behind. The spork is left out, and the file as it was is kept as <path>. A spork.db
that cannot be read is moved aside as spork.db.unreadable-<epoch seconds> and the node starts without it.
A running node holds
spork.db.lock in its data directory; a second node does not start while that lock is held, and
util spork-db import refuses. A data directory in which the lock cannot be taken stops the daemon with
Cannot lock the spork database in <data directory>: <reason>, such as permission denied.
When a replacement is accepted
Every version carries a signature history (log). When a node makes a new version over its stored one,
the stored version’s own signatures and digest become the next entry of the log. The entries are
chained, so removing or rewriting an earlier entry breaks the current signatures. The history can be
read with gridspork-log.
Apart from the copies described below, a node replaces a stored spork that an active key signed only when the incoming version is signed by two different active board keys, its log begins with the stored log unchanged, every new history entry is validly signed by a key that was active at some time, and one of these holds:
- the log is longer and goes on from the stored version: the entry right after the stored log bears the stored version’s time stamp, to the millisecond, and a digest equal to the SHA-512 of its signed bytes, and it is co-signed where the stored version is;
- the log is the same length and equal to the stored log, so the version is a sibling made over the same version as the stored one, and it settles over the stored one as described below. Only when the stored log is empty, as for a first version, are siblings judged by having the same previous value and previous timestamp.
A node that missed several versions still takes the newest, since its log goes on from the node’s version. A version built on another version than the one the node holds is refused, however long its log.
A stored ordinary spork that no active key signed holds nothing back. A version that two different active board keys signed takes its place when it is newer than the stored spork and than every version in its log, and is none of them, whether or not its log goes on from the stored one; the rules for siblings below do not apply to it.
The signatures are not part of the signed bytes, so one version can exist as copies that differ only in their signatures, for example when two members co-sign the same proposal or a signature is written in its other form. A node tells the version it stores by its digest, so it takes a next version, a proposal or a renewal made over any copy of it. A copy of the stored version is not newer and does not replace it, with one exception: when the stored copy no longer counts under the active keys, a copy with the same time stamp and digest that two different active board keys signed replaces it, and is sent on. The board keys spork takes no such copy.
A version of the board keys spork is judged by its chain instead, see Board keys. Its signature log may name any key that was ever active, and its head is signed by keys that its own chain makes active.
Of two siblings the newer one wins, with these exceptions:
- A next version takes the time stamp of the version it was made over as its previous time stamp; a renewal keeps the previous time stamp and previous value of the version it renews. A renewal never replaces a next version made over the same version, whatever their time stamps. A next version replaces a newer renewal made over the same version when the renewal is stamped at most 60 minutes after it. So when the two are stamped at most 60 minutes apart, every node ends up with the next version; when they are further apart and their data differs, each node keeps the one it holds.
- A sibling whose data differs from the stored version replaces it only while its time stamp is at most 60 minutes after the stored version’s. A sibling with the same data, such as a second renewal, replaces it whenever it is newer.
- Two first versions, with a previous time stamp of EPOCH and an empty signature log, settle by their time stamps alone, however far apart they were made.
None of this applies to the board keys.
How a spork is renewed after a key change, including one a switched-off key signed, is on Renewing sporks.
How a spork database is checked, signed again and installed on a node is on Signing a spork database.
Key facts
| Item | Value |
|---|---|
| Signatures needed | Two, from different active board keys; the board keys spork by two keys of its own chain |
| Default network keys | Four built-in board keys; a chain replaces them without a release |
| Board keys spork | Type 3000, the chain of changes to the board keys, at most 64 changes |
| A change of the board keys | 2 to 16 keys, different from the keys before it; max(2, n/2+1) of the n keys before it approve it, and every key it adds |
| Board keys record | board-keys-<16 hex of the root>.seen and board-keys-<16 hex of the root>.seen.lock in the user data directory |
| Chain shipped by a release | /board-keys/release.chain; none in this release |
| Algorithm | ECDSA, SHA-512, P-521 (secp521r1) |
| Board key | From exactly one of -k <hex>, --key-file <path> or $HEDGEHOG_BOARD_KEY; it never leaves the machine |
| Signed spork submission | POST /gridspork/wire with application/octet-stream, at most 16,777,216 bytes |
| Proposal digest | SHA-512 of the signed bytes, 128 lower-case hex characters |
| Proposals held | One per spork type, in memory only |
| Proposal lifetime | 60 minutes from its timestamp; a stamp more than 2 minutes ahead is refused |
| Change window | A sibling with other data has a timestamp at most 60 minutes after the stored version’s; a next version and a renewal made over one version, stamped at most 60 minutes apart, settle on the next version, and a renewal never replaces a next version; two first versions settle by time stamp alone; a version of the board keys must hold the stored chain, and be longer or, for the same chain, newer |
| First version | A previous time stamp of EPOCH and an empty signature log; --first-version on gridspork-set and gridspork-grow; --first-version <TYPE> on gridspork-cosign, once for each type |
| Switched-off sporks | Stay in spork.db, count for nothing and are not published until two active board keys sign them again, or until a copy of the same version (same time stamp and digest) that two different active board keys signed takes their place |
| Mint storage size | At most 16,384 entries |
| Vesting storage size | At most 4,096 schedules |
| Signature history | At most 4,096 entries |
| Signature checks | For what peers send, at most 256 a second for the node and 128 for a type; the connections this node dialed take three quarters of each (192 and 96), inbound connections the quarter left (64 and 32); a source is its network group, IPv4 /16 or IPv6 /32, or stands alone when it is loopback, private, or a configured peer this node dialed; a source pays 12 a second of a type on a dialed connection, 8 on an inbound one, and has one bucket for all types together of 24 a second on a dialed connection, 8 on an inbound one, so a pool’s node ceiling takes at least 8 sources to spend; a signature that verified is not checked again while the node remembers it, up to 8,192 signatures; at most 12 proposals a minute for a type from one signer over P2P; what arrives over REST is not counted against the checks |
| Publish interval | Every 3 minutes; all but the statistics key are published, each only if it counts, the board keys first |
| Save interval | Every 3 minutes on one background thread, and once more when the node stops; the file is written only when the database differs from what the node last wrote since it started, or the file is missing; every type is saved, counting or not |
| Unfinished saves | spork.db.<number>.tmp beside spork.db, removed on start |
Unreadable spork.db |
Moved aside as spork.db.unreadable-<epoch seconds>; the node starts without it |
| Dropped sporks | spork.db copied to spork.db.unsigned-<epoch seconds>, and one warning per dropped type |
| Lock | spork.db.lock in the data directory, held while the node runs; a lock that cannot be taken stops the daemon with Cannot lock the spork database in <data directory>: <reason> |
| Stored in | spork.db in the user data directory |
In the source
application/src/main/java/org/unigrid/hedgehog/model/spork/GridSpork.java: the base typeapplication/src/main/java/org/unigrid/hedgehog/model/spork/SporkSuccession.java: which version may follow on the stored one, how siblings settle, and the copy that replaces a stored copy that does not countapplication/src/main/java/org/unigrid/hedgehog/model/spork/MintStorage.java,MintSupply.java,VestingStorage.java,StorageSpork.java,StatisticsPubKey.java,LedgerBlacklist.java,LedgerFee.javaandCommunicationSpork.java: the spork typesapplication/src/main/java/org/unigrid/hedgehog/model/spork/BoardKeys.java,KeyChange.java,BoardKeyChain.java,ProposedKeyChange.javaandmodel/crypto/ActiveKeys.java: the chain of board keys and the rules a change holds byapplication/src/main/java/org/unigrid/hedgehog/model/spork/BoardKeysEncoder.javaandBoardKeysDecoder.java: the chain in wire formapplication/src/main/java/org/unigrid/hedgehog/model/spork/SignatureLog.java: the signature historyapplication/src/main/java/org/unigrid/hedgehog/model/crypto/NetworkKey.java: the keys active now and every key that was active, with the chain that made themapplication/src/main/java/org/unigrid/hedgehog/model/spork/VerifySlots.javaandSporkLag.java: when an incoming spork’s signatures are checkedapplication/src/main/java/org/unigrid/hedgehog/transport/handler/PublishSporkChannelHandler.java: receiving sporksapplication/src/main/java/org/unigrid/hedgehog/model/spork/SporkRelease.java: the types a node takes, all of them with--previewapplication/src/main/java/org/unigrid/hedgehog/transport/handler/SporkRecipients.java: which peers a spork may be sent toapplication/src/main/java/org/unigrid/hedgehog/transport/schedule/PublishSporkSchedule.java: the periodic publishapplication/src/main/java/org/unigrid/hedgehog/producer/SporkDatabaseProducer.java: loads, prunes and savesspork.dbapplication/src/main/java/org/unigrid/hedgehog/model/spork/SporkDatabaseLock.java: the lock a running node holds onspork.dbapplication/src/main/java/org/unigrid/hedgehog/model/spork/content/SporkContent.java: the content rules a spork is checked by before it is signed or takenapplication/src/main/java/org/unigrid/hedgehog/model/network/codec/SporkWire.java: the wire form that the commands and the node’s wire endpoints read and writeapplication/src/main/java/org/unigrid/hedgehog/server/rest/GridSporkWireResource.java: handing out a stored spork or a proposal in wire form, and taking a signed spork backapplication/src/main/java/org/unigrid/hedgehog/server/rest/GridSporkResource.javaand the other spork resources: reading sporks and proposals as JSON, for displayapplication/src/main/java/org/unigrid/hedgehog/command/cli/spork/GridSporkGet.java:gridspork-getfor every type, withBoardKeysSpork.javabeside it