REST interface

After 0.0.8. The gridnode endpoints, file storage through /storage with its 64 GiB limit, the storage settings spork and the /ledger endpoints are on the master branch and not in release 0.0.8.

A running Hedgehog node can be controlled and queried over HTTPS, on a port separate from the one it uses to talk to other nodes. Operators use it to read the network settings the board maintains, to inspect and edit the node’s list of peers, to start or stop a gridnode, and to store and retrieve files on the network. The hedgehog cli command is simply a client of this interface, so everything it can do, any other program can do as well.

What it offers

The interface is a set of web resources, grouped by purpose:

  • Grid sporks. Read the network-wide settings described in Grid sporks, see the log of who signed each change and list proposed changes. A program fetches a stored spork or a proposal in wire form and hands a signed spork back; the node checks the signatures and never signs. A change takes effect only after two different board keys have signed it.
  • Peers. List the nodes this node knows about, and add or remove one. See Peer-to-peer network protocol for how nodes talk to each other.
  • Gridnode. Start or stop announcing this node as a gridnode, list the known gridnodes and read the collateral required for the current number of active ones.
  • File storage. Store a file and get back a fingerprint, then read or delete it with that fingerprint. The data is spread over gridnodes with erasure coding, as described in Network storage and Erasure coding.
  • Legacy chain data. Look up the balance and transactions of an address in the legacy chain snapshot, including mints still waiting to be paid out. See Legacy chain snapshot.
  • Ledger. Read the state of the node’s ledger, the balance of an account or a legacy address and any block, and send in a signed transaction. See The ledger endpoints.
  • Node control. Report the version, the chain height and the download status of the snapshot, and shut the node down.
  • S3-style buckets. A small subset of the Amazon S3 object-store operations (create, list and delete buckets; put, get, copy, list and delete objects) kept in the node’s own data directory. These files stay on the one node and are not replicated. On the s3 branch they are gone, replaced by the S3 endpoint on a port of its own.

How it works

The node runs an HTTPS server next to its peer-to-peer listener. Each request passes a token check first and is then handed to the matching resource, which reads or changes the same state the rest of the daemon uses. For how this listener fits with the other parts of the daemon, see Architecture overview.

flowchart LR
    A[HTTPS request] --> B[Bearer token check]
    B --> C[Resource class]
    C --> D[Daemon state<br/>sporks, peers, storage]
    D --> E[JSON response]

Authentication. Every request must carry the header Authorization: Bearer <token>, including requests for paths that do not exist. The token is taken from --resttoken or the environment variable HEDGEHOG_REST_TOKEN. When neither is set, the node generates a random token at each start and writes it to a file named rest.token in its data directory, readable only by its owner, so a hedgehog cli on the same machine works without configuration. A client on another machine passes the token explicitly. The token is checked as soon as the head of a request arrives: a request without it is answered with 401 before any of its body is read, and its connection is closed.

Encryption. The server uses a self-signed certificate created at each start. The bundled client accepts any certificate, so the connection is encrypted but the server is not authenticated. For this reason the interface listens on localhost by default and should be exposed beyond it only with that in mind.

Spork changes are signed by the board, not the node. The wire endpoints hand out a stored spork or a proposal in the form peers exchange, and take a signed spork back. A board member’s command fetches the wire form, checks it, shows what the change sets, signs on its own machine with a key from -k, --key-file or $HEDGEHOG_BOARD_KEY, and posts the signed spork. The node checks the signatures and the signature history as it checks a spork from a peer. The board key never reaches the node. A change is stored once a second board key has co-signed it and held as a proposal until then, and only then is it published to the network. The same signing rule is described in Grid sporks.

Uploads. File storage through /storage accepts files up to the limit in the table below and streams them, so a large file does not need to fit in memory. The fingerprint returned on upload is the only key to the file and is sent in a header rather than in the URL.

Body limits. The server bounds every request body before a resource reads it: 1 MiB on every path but a signed spork and the storage uploads. A signed spork posted to /gridspork/wire may be up to the 16 MiB a peer may send. The storage uploads (/storage and the S3-style buckets) take up to 64 GiB, or --restmaxupload if that is larger. A body above its limit is answered with 413 and the connection is closed. A body of any other path that is not complete within 30 seconds is answered with 408. On the s3 branch the S3-style buckets and --restmaxupload are gone.

The spork endpoints

A spork is signed on the machine of a board member, never on the node. The wire endpoints carry the form peers exchange, so the bytes a signature covers are the bytes every node reads:

Endpoint Answers
GET /gridspork/wire/{TYPE} 200 with the stored spork in wire form, 204 when the node holds none, 404 for a type it does not know
GET /gridspork/pending/{digest} 200 with the proposal in wire form, 404 when no proposal has that digest
POST /gridspork/wire The signed spork as the body, application/octet-stream, at most 16 MiB (16,777,216 bytes): 200 when the node stores it, 202 when it holds it as a proposal, with PendingSporkInfo as JSON

TYPE is one of MINT_STORAGE, MINT_SUPPLY, VESTING_STORAGE, STORAGE, LEDGER_BLACKLIST, LEDGER_FEE and STATISTICS_PUBKEY. A body that is not a spork, or that holds data no node can act on, is answered 400 with the reason in plain text. One that no current board key signed, or that no second current board key co-signed, is answered 401 with No current board key signed the spork, or no second one co-signed it. One that does not follow on the stored spork is answered 409 with The spork does not follow on the stored spork and the proposal held. The 202 answer holds type, timeStamp, expires, signer, digest and data.

The command line is a client of these endpoints: it fetches, checks and signs on its own machine, then posts the result, see Grid sporks. A board key never reaches the node.

The JSON endpoints show what the node holds and change nothing:

Endpoint Shows 204 when unset
GET /gridspork Every stored spork  
GET /gridspork/log The signature history of every stored spork yes
GET /gridspork/pending The proposals awaiting a co-signature, each with its digest yes
GET /gridspork/mint-storage The mint storage spork yes
GET /gridspork/mint-storage/{address}/{height} The amount of one entry; 404 when that key has none, 400 when the address or the number does not fit its kind yes
GET /gridspork/mint-supply The mint supply spork yes
GET /gridspork/vesting-storage The vesting storage spork yes
GET /gridspork/vesting-storage/{address} The vesting schedule of one address; 404 when it has none yes
GET /gridspork/storage The storage spork yes
GET /gridspork/ledger-blacklist The ledger blacklist spork yes
GET /gridspork/ledger-fee The ledger fee spork yes

The ledger endpoints

Four endpoints under /ledger let a program read the ledger of the node and send a transaction to it. They need the bearer token like every other endpoint. A node that was not started with --ledger, or whose ledger has not started yet, answers them as a node without a ledger: GET /ledger says running: false and the others answer 503. The node never signs for a client: the client signs the transaction and sends its bytes, so a key never reaches the node. Ledger command line is a client of these endpoints.

Endpoint Returns
GET /ledger The state of the ledger: chain, tip, supply, validators and minimum fee
GET /ledger/account/{address} The balance of an account or a legacy address
GET /ledger/block/{height} The block at a height, with its transactions
POST /ledger/transaction Hands a signed transaction to the ledger

How values are written. An amount is a string of coins with exactly eight decimals, such as "12.50000000", never a number, so nothing is rounded on the way. A hash is 128 lower-case hex characters. An account is a public key of 64 lower-case hex characters. A legacy address is base58. A time is in Unix milliseconds. A field that does not apply is left out, not set to zero. An error is {"error": "<reason>"}.

Reading the state

GET /ledger answers 200 with these fields:

Field Meaning
running The ledger has started and has neither stopped nor halted
halted, haltReason Whether the ledger halted, and why; the reason is empty otherwise
signing The node runs, its newest node was built to sign, and its key is one of the validators of the newest ledger
chainId The id of the chain, 128 hex characters
height, tipHash The height and hash of the newest block; 0 and empty before there is a ledger
supply The coins that exist
round The validator round of the node’s clock, never before the tip’s, which the next block it would make is in and a vote is signed for
roundEnds When that round ends and the next starts, in Unix milliseconds
validators The keys of the validators that sign blocks now, in the order that decides who proposes
roundValidators The set that a vote of round is judged by; it differs from validators between the end of a round and the block that closes it
minimumFee The lowest fee of a transfer that this node takes; left out while the fee spork is not set
droppedInbound How many items the node has dropped from its queue of incoming ledger messages
unsetSporks, laggingSporks The ledger sporks that are not set, and those of which a newer version is known that this node does not hold yet, by name: MINT_STORAGE, MINT_SUPPLY, LEDGER_BLACKLIST and LEDGER_FEE

A node answers no for a spork that is unset or lagging, so anything that depends on it is not minted, claimed or taken until it is set. Before the ledger exists the answer holds only running, halted, haltReason, signing, height, tipHash, droppedInbound and the two lists. A halted ledger is still read from the blocks it holds.

GET /ledger/account/{address} takes an account in lower-case hex or a legacy address, and answers 200 also for an address that holds nothing:

Field Account Legacy address Meaning
address yes yes The address as the ledger writes it
kind yes yes account or legacy
balance yes yes What it holds. For a legacy address that is what mints have put on it, and nothing once it is claimed
nonce yes   The number of transfers the account has made, which the next one must carry
mintedBySporks yes yes The sum of the mints made to the address so far, as the mints named it
claimable   yes What a claim would move now: the unclaimed snapshot balance and the mints; 0 once claimed
claimedBy   yes The account that claimed it, once claimed 
{
  "address": "3f2a…c9",
  "kind": "account",
  "balance": "250.00000000",
  "nonce": 3,
  "mintedBySporks": "0.00000000"
}

An address in upper-case hex, or one that is neither an account nor a valid legacy address, is answered with

  1. A 503 means the ledger is not running.

GET /ledger/block/{height} answers with the block as it was committed:

Field Meaning
height, view, time Where it sits, the view it was decided in, and its time
hash, previousHash The hash of the block and of the one before it, 128 hex characters
stateRoot, transactionRoot The roots it commits to
proposer The key of the validator that proposed it
certifiedBy, endorsements The validators whose precommits make its certificate, and how many
transactions The transactions, each with the fields of its kind

A transaction always has id, the hash of its encoding, and kind. The rest depends on the kind:

Kind Other fields
mint recipient, amount
vote voter, candidate, action (add or remove), round
claim recipient, the account that receives the coins, and claimedAddress, the legacy address that was moved
transfer sender, recipient, amount, fee, nonce

The height is plain digits. A height that is not, or that does not fit a 64-bit number, is answered with 400. A height of 0, or above the newest block, is answered with 404, and 503 means the ledger is not running. The fees of a block are paid to the validators that signed the block before it, so they are not those in certifiedBy, see Ledger.

Sending a transaction

POST /ledger/transaction takes a body of {"transaction": "<hex>"}, where the hex is the encoding of a claim, a transfer or a vote, see Ledger encoding. The client signs it first. A mint is refused: mints come only from the sporks. The call waits for the ledger to say whether it takes the transaction in, which is a check against the state, not a wait for a block. Find the transaction in a block later by its id.

Status Meaning Body
202 The ledger took it in; it waits for a block {"accepted": true, "id": "<hex>"}
409 The ledger refused it, with its own reason, such as Transfer nonce is already used, Balance does not cover the transfer or Fee is below what this node demands {"accepted": false, "id": "<hex>", "reason": "<reason>"}
400 The body is not that JSON, the hex is not valid or is longer than any transaction, the transaction does not decode, or it is a mint {"error": "<reason>"}
413 The body is above 4096 bytes {"error": "<reason>"}
429 16 submits already wait for an answer {"error": "<reason>"}
503 The ledger is not running, or failed to answer {"error": "<reason>"}
504 No answer within 10 seconds; the ledger may still take the transaction in {"error": "<reason>"}

A status of 202 means the transaction is waiting, not that a block holds it. A transaction that waits is sent on to the other nodes, and a validator puts it into a block. A transfer whose nonce is above the next one, by less than 16, is taken in too and waits for the transfers before it.

The limit of 16. Every submit that waits holds a thread of the server, and the server has no other limit on them, so at most 16 may wait at once and the next gets 429. A place is given back when the ledger has answered, not when the response is sent, so a client that gives up and tries again cannot queue more than the limit. A place is also given back after 60 seconds at the latest, so a ledger that never answers cannot fill all 16 for good.

Key facts

Item Value
Protocol HTTPS with a self-signed certificate
Default address localhost, port 52884 (options --resthost, --restport)
Peer-to-peer port, for comparison 52883, listening on all addresses by default
Authentication Bearer token on every request (--resttoken, HEDGEHOG_REST_TOKEN or rest.token)
Largest file through /storage 64 GiB
Largest upload to the S3-style buckets 1 GiB by default (--restmaxupload); gone on the s3 branch, see S3 endpoint
Largest request body 1 MiB, 16 MiB on POST /gridspork/wire, 64 GiB on the storage uploads; 413 above it
Largest signed spork 16,777,216 bytes (16 MiB)
Spork signing On the machine of a board member; the node checks both signatures and the key never reaches it
Time to send a body other than an upload 30 seconds, then 408
Largest body of POST /ledger/transaction 4096 bytes, answered with 413 above it
Submits that may wait at once 16, and a place is freed after 60 seconds at the latest
Wait for the ledger’s answer to a submit 10 seconds, then 504
Data format JSON; the wire form of a spork in application/octet-stream; XML for the S3-style buckets, which the s3 branch removes
Protocol versions reported by /version hedgehog/0.0.4, gridspork/0.0.4

In the source

  • application/src/main/java/org/unigrid/hedgehog/server/rest/RestServer.java: starts the HTTPS listener and lists every resource class.
  • application/src/main/java/org/unigrid/hedgehog/server/rest/BearerTokenFilter.java: the token check.
  • application/src/main/java/org/unigrid/hedgehog/server/rest/GridSporkWireResource.java: the wire endpoints that hand out a stored spork or a proposal and take a signed spork back.
  • application/src/main/java/org/unigrid/hedgehog/server/rest/GridSporkResource.java: sporks, proposals and signature histories as JSON, for display; the other spork resources sit in the same package.
  • application/src/main/java/org/unigrid/hedgehog/model/network/codec/SporkWire.java: the wire form.
  • application/src/main/java/org/unigrid/hedgehog/client/rest/SporkClient.java: the client of the wire endpoints that the command line uses.
  • application/src/main/java/org/unigrid/hedgehog/command/option/BoardKeyOptions.java: the three sources of the board key and their messages.
  • application/src/main/java/org/unigrid/hedgehog/server/rest/RequestBodyLimit.java: the body limits and the 413 answer.
  • application/src/main/java/org/unigrid/hedgehog/server/rest/StorageResource.java: file storage by fingerprint.
  • application/src/main/java/org/unigrid/hedgehog/server/rest/LedgerResource.java: the four /ledger endpoints; the JSON records they answer with are in server/rest/entity/ (LedgerInfo, LedgerAccount, LedgerBlock, LedgerTransactionView, LedgerSubmission and LedgerSubmitted).
  • application/src/main/java/org/unigrid/hedgehog/model/bootstrap/LedgerText.java: how addresses and coin amounts are read and written, shared with the command line and the mint spork.
  • application/src/main/java/org/unigrid/hedgehog/command/option/RestOptions.java: the command-line options.
  • application/src/main/java/org/unigrid/hedgehog/client/rest/RestClient.java: the client the CLI uses.
  • application/src/main/java/org/unigrid/hedgehog/command/CLI.java: the hedgehog cli subcommands.