BitcoinCash/specification
1
Fork 0
Code Issues Pull Requests Releases Activity

Merge remote-tracking branch 'verde/master'

Browse Source
This commit is contained in:
TomZ
2021-12-26 12:43:31 +01:00
parent fbf72937cb cdeaaa9b98
commit 385054336f
44 changed files with 496 additions and 103 deletions
Download Patch File Download Diff File Expand all files Collapse all files
home.md
+18 -33
View File
@@ -1,7 +1,6 @@
# Bitcoin Cash Protocol
### About
[Style Guide](/style-guide) — [Contributors](/contributors) — [Target Audience](/target-audience) — [Project History](/project-history)
### Basics
@@ -24,54 +23,40 @@
### Proof of Work (PoW)
[Proof of Work](/protocol/blockchain/proof-of-work) — [Difficulty Adjustment Algorithm](/protocol/blockchain/proof-of-work/difficulty-adjustment-algorithm) — [Mining](/protocol/blockchain/proof-of-work/mining) — [Stratum Protocol](/mining/stratum-protocol) — [Mining Pools](/mining/mining-pools)
### Addresses
[Address Types](/protocol/blockchain/addresses) — [Base58Check Encoding (legacy)](/protocol/blockchain/encoding/base58check) — [Cashaddr Encoding](/protocol/blockchain/encoding/cashaddr)
### Cryptography
[Bitcoin Keys (Public/Private)](/protocol/blockchain/cryptography/keys) — [Signatures (ECDSA/Schnorr)](/protocol/blockchain/cryptography/signatures) — [Multisignature (M-of-N multisig)](/protocol/blockchain/cryptography/multisignature)
### Network upgrades
**Pre-BCH:** [Bip-9](/protocol/forks/bip-0009) — [Bip-16](/protocol/forks/bip-0016) — [Bip-34](/protocol/forks/bip-0034) — [Bip-37](/protocol/forks/bip-0037) — [Bip-65](/protocol/forks/bip-0065) — [Bip-66](/protocol/forks/bip-0066) — [Bip-68](/protocol/forks/bip-0068) — [Bip-112](/protocol/forks/bip-0112) — [Bip-113](/protocol/forks/bip-0113) — [Bip-133](/protocol/forks/bip-0133) — [Bip-157](/protocol/forks/bip-0157) — [Bip-158](/protocol/forks/bip-0158) — [Bip-159](/protocol/forks/bip-0159)
#### Pre-BCH
**2017:** [BCH-UAHF (BUIP-55)](/protocol/forks/bch-uahf) — [HF-20171113](/protocol/forks/hf-20171113)
| Year (Creation) | BIPs |
|------|-------|
| 2012 | [Bip-16](/protocol/forks/bip-0016) — [Bip-34](/protocol/forks/bip-0034) — [BIP-35](/protocol/forks/bip-0035) — [Bip-37](/protocol/forks/bip-0037) |
| 2014 | [Bip-65](/protocol/forks/bip-0065) |
| 2015 | [Bip-66](/protocol/forks/bip-0066) — [Bip-68](/protocol/forks/bip-0068) — [Bip-112](/protocol/forks/bip-0112) — [Bip-113](/protocol/forks/bip-0113) — [Bip-9](/protocol/forks/bip-0009) |
| 2016 | [Bip-133](/protocol/forks/bip-0133) — [Bip-152](/protocol/forks/bip-0152) |
| 2017 | [Bip-157](/protocol/forks/bip-0157) — [Bip-158](/protocol/forks/bip-0158) — [Bip-159](/protocol/forks/bip-0159) |
**2018:** [HF-20180515](/protocol/forks/hf-20180515) — [HF-20181115](/protocol/forks/hf-20181115)
**2019:** [HF-20190515](/protocol/forks/hf-20190515) — [HF-20191115](/protocol/forks/hf-20191115)
**2020:** [HF-20200515](/protocol/forks/hf-20200515) — [HF-20201115](/protocol/forks/hf-20201115)
**2021:** [HF-20210515](/protocol/forks/hf-20210515)
#### BCH
| Year (Activation) | Hard Forks |
|------|---------------|
| 2017 | [BCH-UAHF (BUIP-55)](/protocol/forks/bch-uahf) — [HF-20171113](/protocol/forks/hf-20171113) |
| 2018 | [HF-20180515](/protocol/forks/hf-20180515) — [HF-20181115](/protocol/forks/hf-20181115) |
| 2019 | [HF-20190515](/protocol/forks/hf-20190515) — [HF-20191115](/protocol/forks/hf-20191115) |
| 2020 | [HF-20200515](/protocol/forks/hf-20200515) — [HF-20201115](/protocol/forks/hf-20201115) |
| 2021 | [HF-20210515](/protocol/forks/hf-20210515) |
### Network protocol
[Network Messages](/protocol/network/messages) — [Handshake](/protocol/network/node-handshake)
#### Announcement messages
[filteradd](/protocol/network/messages/filteradd) — [filterclear](/protocol/network/messages/filterclear) — [filterload](/protocol/network/messages/filterload) — [inv](/protocol/network/messages/inv)
#### Request messages
[feefilter](/protocol/network/messages/feefilter) — [getaddr](/protocol/network/messages/getaddr) — [getblocks](/protocol/network/messages/getblocks) — [getdata](/protocol/network/messages/getdata) — [getheaders](/protocol/network/messages/getheaders) — [ping](/protocol/network/messages/ping) —
[sendheaders](/protocol/network/messages/sendheaders) — [version](/protocol/network/messages/version)
#### Response messages
[addr](/protocol/network/messages/addr) — [block](/protocol/network/messages/block) — [headers](/protocol/network/messages/headers) — [notfound](/protocol/network/messages/notfound) — [merkleblock](/protocol/network/messages/merkleblock) — [pong](/protocol/network/messages/pong) —
[reject](/protocol/network/messages/reject) — [tx](/protocol/network/messages/tx) — [verack](/protocol/network/messages/verack)
#### Other messages (extensions)
[sendcmpct](/protocol/network/messages/sendcmpct) — [get_xthin](/protocol/network/messages/get_xthin) — [xthinblock](/protocol/network/messages/xthinblock) — [thinblock](/protocol/network/messages/thinblock) — [get_xblocktx](/protocol/network/messages/get_xblocktx) — [xblocktx](/protocol/network/messages/xblocktx) — [xupdate](/protocol/network/messages/xupdate) — [xversion](/protocol/network/messages/xversion) — [xverack](/protocol/network/messages/xverack)
### Addresses
[Address Types](/protocol/blockchain/addresses) — [Base58Check Encoding (legacy)](/protocol/blockchain/encoding/base58check) — [Cashaddr Encoding](/protocol/blockchain/encoding/cashaddr)
### Simple Payment Verification (SPV)
[SPV](/protocol/spv) — [Bloom Filters](/protocol/spv/bloom-filter)
### Simple Ledger Protocol
[SLP Overview](/protocol/slp) — [GENESIS](/protocol/slp/genesis) — [MINT](/protocol/slp/mint) — [SEND](/protocol/slp/send) — [COMMIT](/protocol/slp/commit)
### Miscellaneous
protocol/forks/bch-uahf.md
+1 -1
View File
@@ -1,4 +1,4 @@
# UAHF
# BCH-UAHF: Bitcoin Cash User-Activated Hard Fork
layout: specification
title: UAHF Technical Specification
protocol/forks/bip-0009.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0009
# BIP-9
BIP: 9
Title: Version bits with timeout and delay
Author: Pieter Wuille <pieter.wuille@gmail.com>
protocol/forks/bip-0016.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0016
# BIP-16
BIP: 16
Layer: Consensus (soft fork)
protocol/forks/bip-0034.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0034
# BIP-34
BIP: 34
Layer: Consensus (soft fork)
protocol/forks/bip-0035.md
+41
View File
@@ -0,0 +1,41 @@
# BIP-35
BIP: 35
Layer: Peer Services
Title: mempool message
Author: Jeff Garzik <jgarzik@exmulti.com>
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0035
Status: Final
Type: Standards Track
Created: 2012-08-16
## Abstract
Make a network node's transaction memory pool accessible via a new "mempool" message. Extend the existing "getdata" message behavior to permit accessing the transaction memory pool.
## Motivation
Several use cases make it desireable to expose a network node's transaction memory pool:
1. SPV clients, wishing to obtain zero-confirmation transactions sent or received.
2. Miners, to avoid missing lucrative fees, downloading existing network transactions after a restart.
3. Remote network diagnostics.
## Specification
1. The mempool message is defined as an empty message where pchCommand == "mempool"
2. Upon receipt of a "mempool" message, the node will respond with an "inv" message containing MSG_TX hashes of all the transactions in the node's transaction memory pool, if any.
3. The typical node behavior in response to an "inv" is "getdata". However, the reference Satoshi implementation ignores requests for transaction hashes outside that which is recently relayed. To support "mempool", an implementation must extend its "getdata" message support to querying the memory pool.
4. Feature discovery is enabled by checking two "version" message attributes:
- Protocol version >= 60002
- NODE_NETWORK bit set in nServices
Note that existing implementations drop "inv" messages with a vector size > 50000.
## Backward compatibility
Older clients remain 100% compatible and interoperable after this change.
## Implementation
https://github.com/bitcoin/bitcoin/pull/1641
protocol/forks/bip-0037.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0037
# BIP-37
BIP: 37
Layer: Peer Services
protocol/forks/bip-0065.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0065
# BIP-65
BIP: 65
Layer: Consensus (soft fork)
protocol/forks/bip-0066.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0066
# BIP-66
BIP: 66
Layer: Consensus (soft fork)
protocol/forks/bip-0068.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0068
# BIP-68
BIP: 68
Layer: Consensus (soft fork)
protocol/forks/bip-0112.md
+1 -1
View File
@@ -1,4 +1,4 @@
BIP-0112
# BIP-112
BIP: 112
Layer: Consensus (soft fork)
protocol/forks/bip-0113.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0113
# BIP-113
BIP: 113
Layer: Consensus (soft fork)
protocol/forks/bip-0133.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0133
# BIP-133
BIP: 133
Layer: Peer Services
protocol/forks/bip-0152-protocol-flow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

protocol/forks/bip-0152.md
+249
View File
@@ -0,0 +1,249 @@
# BIP-152
BIP: 152
Layer: Peer Services
Title: Compact Block Relay
Author: Matt Corallo <bip152@bluematt.me>
Comments-Summary: Unanimously Recommended for implementation
Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0152
Status: Final
Type: Standards Track
Created: 2016-04-27
License: PD
## Abstract
Compact blocks on the wire as a way to save bandwidth for nodes on the P2P network.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.
## Motivation
Historically, the Bitcoin P2P protocol has not been very bandwidth efficient for block relay. Every transaction in a block is included when relayed, even though a large number of the transactions in a given block are already available to nodes before the block is relayed. This causes moderate inbound bandwidth spikes for nodes when receiving blocks, but can cause very significant outbound bandwidth spikes for some nodes which receive a block before their peers. When such spikes occur, buffer bloat can make consumer-grade internet connections temporarily unusable, and can delay the relay of blocks to remote peers who may choose to wait instead of redundantly requesting the same block from other, less congested, peers.
Thus, decreasing the bandwidth used during block relay is very useful for many individuals running nodes.
While the goal of this work is explicitly not to reduce block transfer latency, it does, as a side effect reduce block transfer latencies in some rather significant ways. Additionally, this work forms a foundation for future work explicitly targeting low-latency block transfer.
## Specification for version 1
### Intended Protocol Flow
![Protocol flow](/protocol/forks/bip-0152-protocol-flow.png)
The protocol is intended to be used in two ways, depending on the peers and bandwidth available, as discussed [later](#implementation-notes). The "high-bandwidth" mode, which nodes may only enable for a few of their peers, is enabled by setting the first boolean to 1 in a <code>sendcmpct</code> message. In this mode, peers send new block announcements with the short transaction IDs already (via a <code>cmpctblock</code> message), possibly even before fully validating the block (as indicated by the grey box in the image above). In some cases no further round-trip is needed, and the receiver can reconstruct the block and process it as usual immediately. When some transactions were not available from local sources (ie mempool), a <code>getblocktxn</code>/<code>blocktxn</code> roundtrip is necessary, bringing the best-case latency to the same 1.5*RTT minimum time that nodes take today, though with significantly less bandwidth usage.
The "low-bandwidth" mode is enabled by setting the first boolean to 0 in a <code>sendcmpct</code> message. In this mode, peers send new block announcements with the usual inv/headers announcements (as per BIP130, and after fully validating the block). The receiving peer may then request the block using a MSG_CMPCT_BLOCK <code>getdata</code> request, which will receive a response of the header and short transaction IDs. In some cases no further round-trip is needed, and the receiver can reconstruct the block and process it as usual, taking the same 1.5*RTT minimum time that nodes take today, though with significantly less bandwidth usage. When some transactions were not available from local sources (ie mempool), a <code>getblocktxn</code>/<code>blocktxn</code> roundtrip is necessary, bringing the latency to at least 2.5*RTT in this case, again with significantly less bandwidth usage than today. Because TCP often exhibits worse transfer latency for larger data sizes (as a multiple of RTT), total latency is expected to be reduced even when the full 2.5*RTT transfer mechanism is used.
### New data structures
Several new data structures are added to the P2P network to relay compact blocks: PrefilledTransaction, HeaderAndShortIDs, BlockTransactionsRequest, and BlockTransactions.
For the purposes of this section, CompactSize refers to the variable-length integer encoding used across the existing P2P protocol to encode array lengths, among other things, in 1, 3, 5 or 9 bytes. Only CompactSize encodings which are minimally-encoded (ie the shortest length possible) are used by this spec. Any other CompactSize encodings are left with undefined behavior.
Several uses of CompactSize below are "differentially encoded". For these, instead of using raw indexes, the number encoded is the difference between the current index and the previous index, minus one. For example, a first index of 0 implies a real index of 0, a second index of 0 thereafter refers to a real index of 1, etc.
#### PrefilledTransaction
A PrefilledTransaction structure is used in HeaderAndShortIDs to provide a list of a few transactions explicitly.
| Field Name | Type | Size | Encoding | Purpose |
|--|--|--|--|--|
| index | CompactSize |1, 3 bytes | Compact Size, differentially encoded since the last PrefilledTransaction in a list | The index into the block at which this transaction is |
| tx | Transaction |variable | As encoded in "tx" messages sent in response to getdata MSG_TX | The transaction which is in the block at index index. |
#### HeaderAndShortIDs
A HeaderAndShortIDs structure is used to relay a block header, the short transactions IDs used for matching already-available transactions, and a select few transactions which we expect a peer may be missing.
| Field Name | Type | Size | Encoding | Purpose |
|--|--|--|--|--|
| header | Block header | 80 bytes | First 80 bytes of the block as defined by the encoding used by "block" messages | The header of the block being provided |
|nonce | uint64_t | 8 bytes | Little Endian | A nonce for use in short transaction ID calculations |
|shortids_length | CompactSize | 1 or 3 bytes | As used to encode array lengths elsewhere | The number of short transaction IDs in shortids (ie block tx count - prefilledtxn_length) |
|shortids | List of 6-byte integers | 6\*shortids_length bytes | Little Endian | The short transaction IDs calculated from the transactions which were not provided explicitly in prefilledtxn |
|prefilledtxn_length | CompactSize | 1 or 3 bytes | As used to encode array lengths elsewhere | The number of prefilled transactions in prefilledtxn (ie block tx count - shortids_length) |
|prefilledtxn | List of PrefilledTransactions | variable size\*prefilledtxn_length | As defined by PrefilledTransaction definition, above | Used to provide the coinbase transaction and a select few which we expect a peer may be missing |
#### BlockTransactionsRequest
A BlockTransactionsRequest structure is used to list transaction indexes in a block being requested.
| Field Name | Type | Size | Encoding | Purpose |
|--|--|--|--|--|
|blockhash | Binary blob | 32 bytes | The output from a double-SHA256 of the block header, as used elsewhere | The blockhash of the block which the transactions being requested are in |
|indexes_length | CompactSize | 1 or 3 bytes | As used to encode array lengths elsewhere | The number of transactions being requested |
|indexes | List of CompactSizes | 1 or 3 bytes\*indexes_length | Differentially encoded | The indexes of the transactions being requested in the block |
#### BlockTransactions
A BlockTransactions structure is used to provide some of the transactions in a block, as requested.
| Field Name | Type | Size | Encoding | Purpose |
|--|--|--|--|--|
| blockhash | Binary blob | 32 bytes | The output from a double-SHA256 of the block header, as used elsewhere | The blockhash of the block which the transactions being provided are in |
| transactions_length | CompactSize | 1 or 3 bytes | As used to encode array lengths elsewhere | The number of transactions provided |
| transactions | List of Transactions | variable | As encoded in "tx" messages in response to getdata MSG_TX | The transactions provided |
#### Short transaction IDs
Short transaction IDs are used to represent a transaction without sending a full 256-bit hash. They are calculated by:
1. single-SHA256 hashing the block header with the nonce appended (in little-endian)
2. Running SipHash-2-4 with the input being the transaction ID and the keys (k0/k1) set to the first two little-endian 64-bit integers from the above hash, respectively.
3. Dropping the 2 most significant bytes from the SipHash output to make it 6 bytes.
### New messages
A new inv type (MSG_CMPCT_BLOCK == 4) and several new protocol messages are added: sendcmpct, cmpctblock, getblocktxn, and blocktxn.
#### sendcmpct
1. The sendcmpct message is defined as a message containing a 1-byte integer followed by a 8-byte integer where pchCommand == "sendcmpct".
2. The first integer SHALL be interpreted as a boolean (and MUST have a value of either 1 or 0)
3. The second integer SHALL be interpreted as a little-endian version number. Nodes sending a sendcmpct message MUST currently set this value to 1.
4. Upon receipt of a "sendcmpct" message with the first and second integers set to 1, the node SHOULD announce new blocks by sending a cmpctblock message.
5. Upon receipt of a "sendcmpct" message with the first integer set to 0, the node SHOULD NOT announce new blocks by sending a cmpctblock message, but SHOULD announce new blocks by sending invs or headers, as defined by BIP130.
6. Upon receipt of a "sendcmpct" message with the second integer set to something other than 1, nodes MUST treat the peer as if they had not received the message (as it indicates the peer will provide an unexpected encoding in cmpctblock, and/or other, messages). This allows future versions to send duplicate sendcmpct messages with different versions as a part of a version handshake for future versions. See Protocol Versioning section, below, for more info on the specifics of the version-negotiation mechanics.
7. Nodes SHOULD check for a protocol version of >= 70014 before sending sendcmpct messages.
8. Nodes MUST NOT send a request for a MSG_CMPCT_BLOCK object to a peer before having received a sendcmpct message from that peer.
9. Nodes MUST NOT request a MSG_CMPCT_BLOCK object before having sent all sendcmpct messages to that peer which they intend to send, as the peer cannot know what version protocol to use in the response.
#### MSG_CMPCT_BLOCK
1. getdata messages may now contain requests for MSG_CMPCT_BLOCK objects.
2. Upon receipt of a getdata containing a request for a MSG_CMPCT_BLOCK object with the hash of a block which was recently announced and is close to the tip of the best chain of the receiver and after having sent the requesting peer a sendcmpct message, nodes MUST respond with a cmpctblock message containing appropriate data representing the block being requested.
3. Upon receipt of a getdata containing a request for a MSG_CMPCT_BLOCK object for which a cmpctblock message is not sent in response, a block message containing the requested block in non-compact form MUST be sent.
4. MSG_CMPCT_BLOCK inv objects MUST NOT appear anywhere except for in getdata messages.
#### cmpctblock
1. The cmpctblock message is defined as a message containing a serialized HeaderAndShortIDs message and pchCommand == "cmpctblock".
2. Upon receipt of a cmpctblock message after sending a sendcmpct message, nodes SHOULD calculate the short transaction ID for each unconfirmed transaction they have available (ie in their mempool) and compare each to each short transaction ID in the cmpctblock message.
3. After finding already-available transactions, nodes which do not have all transactions available to reconstruct the full block SHOULD request the missing transactions using a getblocktxn message.
4. A node MUST NOT send a cmpctblock message unless they are able to respond to a getblocktxn message which requests every transaction in the block.
5. A node MUST NOT send a cmpctblock message without having validated that the header properly commits to each transaction in the block, and properly builds on top of the existing, fully-validated chain with a valid proof-of-work either as a part of the current most-work valid chain, or building directly on top of it. A node MAY send a cmpctblock before validating that each transaction in the block validly spends existing UTXO set entries.
#### getblocktxn
1. The getblocktxn message is defined as a message containing a serialized BlockTransactionsRequest message and pchCommand == "getblocktxn".
2. Upon receipt of a properly-formatted getblocktxn message, nodes which recently provided the sender of such a message a cmpctblock for the block hash identified in this message MUST respond with either an appropriate blocktxn message, or a full block message. A blocktxn response MUST contain exactly and only each transaction which is present in the appropriate block at the index specified in the getblocktxn indexes list, in the order requested.
#### blocktxn
1. The blocktxn message is defined as a message containing a serialized BlockTransactions message and pchCommand == "blocktxn".
2. Upon receipt of a properly-formatted requested blocktxn message, nodes SHOULD attempt to reconstruct the full block by:
1. Taking the prefilledtxn transactions from the original cmpctblock and placing them in the marked positions.
2. For each short transaction ID from the original cmpctblock, in order, find the corresponding transaction either from the blocktxn message or from other sources and place it in the first available position in the block.
3. Once the block has been reconstructed, it shall be processed as normal, keeping in mind that short transaction IDs are expected to occasionally collide, and that nodes MUST NOT be penalized for such collisions, wherever they appear.
## Protocol Versioning
1. The protocol version negotiation allows two nodes to agree on the versions of compact blocks which they will exchange. As it is only in a single field, it does not allow a node to support a specific version in only one direction (sending or receiving).
2. Upon connection establishment, a node SHOULD send a burst of sendcmpct messages containing every version of compact block encodings for which they are willing to support sending cmpctblock and blocktxn messages, and receiving getblocktxn messages. These messages SHOULD be ordered in the order of the priority which the node wishes to receive cmpctblock/blocktxn messages, with the highest-priority version sendcmpct message sent first.
3. The encoding version used to send a cmpctblock or blocktxn message or to receive a getblocktxn message MUST be the second integer (version number) in the first sendcmpct message received for which a sendcmpct message with the same version number was sent.
4. Nodes MUST NOT send a sendcmpct message which contains a version number other than the version number which has been negotiated for receiving cmpctblock/blocktxn messages after sending a request for a MSG_CMPCT_BLOCK object, sending a cmpctblock, getblocktxn, blocktxn, or pong message.
5. As a node must send all sendcmpct messages which contain a novel version announcement before any other compact block-related messages, it is possible to determine which version of compact blocks will be used for each object received. It is, however, not possible to know which version will be used to encode the response to a request for a compact block object before any MSG_CMPCT_BLOCK-containing getdata, cmpctblock, getblocktxn, blocktxn, or ping/pong messages have been exchanged.
6. Thus, if a node wishes to determine exactly which version of compact blocks will be used before requesting a compact block object, it must send all of its sendcmpct version announcements, followed by a ping, and wait for the pong response to ensure it has received all sendcmpctblock version announcement messages from the remote peer. Nodes can, obviously, however, determine that the version used will be at least a certain version (in their priority order) after having received a sendcmpct message from the remote peer containing that version as the second integer.
### Sample Version Implementation
1. By way of example, an implementation of the above protocol might look like the following.
2. Upon exchanging version/verack messages, a node immediately sends its list of sendcmpct announcements to the other side, with the version which it wants to receive sent first.
3. Upon receiving the first sendcmpct announcement with a protocol version which is understood from the remote peer, a node will "lock in" the compact block encoding version which will be used to encode compact blocks to that peer.
4. The node then sets the current receive-protocol-version in use on the connection to that version, and uses it to decode new compact block messages.
5. Upon receiving subsequent sendcmpct announcements with a protocol version which is understood from the remote peer (ie a version which has been announced using a sendcmpct in the other direction), a node will check if that protocol version is higher-receive-priority than the current receive-protocol-version in use on the connection, and switch to that version for decoding new compact block messages received.
6. A node might wish to keep a flag for each peer which indicates compact block version negotiation is complete, which can be set upon receiving any compact block-related, or pong message.
7. The above implementation requires only a compile-time list of supported versions in some static priority order, two version fields per peer, and an optional negotiation-complete boolean per-peer.
## Specification for version 2
***NOTE: Segregated witness is not supported in Bitcoin Cash and therefore does not support this definition of Compact Blocks version 2.
The following is left solely for historical purposes.***
Compact blocks version 2 is almost identical to version 1, but supports segregated witness transactions (BIP 141 and BIP 144). The changes are:
1. The second integer (version number) inside sendcmpct is 2 instead of 1 (see Protocol Versioning section, above).
2. Transactions inside cmpctblock messages (both those used as direct announcement and those in response to getdata) and in blocktxn should include witness data, using the same format as responses to getdata MSG_WITNESS_TX, specified in BIP144.
3. Short transaction IDs sent to us in cmpctblock messages, and sent by us in getblocktxn messages, are computed using the same process as in version 1, but using the wtxid as defined in BIP 141 instead of the txid. Note that, though a node normally SHOULD, if a node does not include (ie must then include the short ID for) the coinbase transaction, it must be computed by encoding the transaction in witness format as defined by BIP 141.
4. Upon receipt of a getdata containing a request for a MSG_CMPCT_BLOCK object for which a cmpctblock message is not sent in response, the block message containing the requested block in non-compact form MUST be encoded with witnesses (as is sent in reply to a MSG_WITNESS_BLOCK getdata) if the protocol version used to encode the cmpctblock message would have been 2, and encoded without witnesses (as is sent in response to a MSG_BLOCK getdata) if the protocol version used to encode the cmpctblock message would have been 1.
## Implementation Notes
1. For nodes which have sufficient inbound bandwidth, sending a sendcmpct message with the first integer set to 1 to up to 3 peers is RECOMMENDED. If possible, it is RECOMMENDED that those peers be selected based on their past performance in providing blocks quickly (eg the three peers which provided the highest number of the recent N blocks the quickest), allowing nodes to receive blocks which come from those peers in only 0.5*RTT.
2. Nodes MUST NOT send such sendcmpct messages to more than three peers, as it encourages wasting outbound bandwidth across the network.
3. All nodes SHOULD send a sendcmpct message to all appropriate peers. This will reduce their outbound bandwidth usage by allowing their peers to request compact blocks instead of full blocks.
4. Nodes with limited inbound bandwidth SHOULD request blocks using MSG_CMPCT_BLOCK/getblocktxn requests, when possible. While this increases worst-case message round-trips, it is expected to reduce overall transfer latency as TCP is more likely to exhibit poor throughput on low-bandwidth nodes.
5. Nodes sending cmpctblock messages SHOULD limit prefilledtxn to 10KB of transactions. When in doubt, nodes SHOULD only include the coinbase transaction in prefilledtxn.
6. Nodes MAY pick one nonce per block they wish to send, and only build a cmpctblock message once for all peers which they wish to send a given block to. Nodes SHOULD NOT use the same nonce across multiple different blocks.
7. Nodes MAY impose additional requirements on when they announce new blocks by sending cmpctblock messages. For example, nodes with limited outbound bandwidth MAY choose to announce new blocks using inv/header messages (as per BIP130) to conserve outbound bandwidth.
8. Note that the MSG_CMPCT_BLOCK section does not require that nodes respond to MSG_CMPCT_BLOCK getdata requests for blocks which they did not recently announce. This allows nodes to calculate cmpctblock messages at announce-time instead of at request-time. Blocks which are requested with a MSG_CMPCT_BLOCK getdata, but which are not responded to with a cmpctblock message MUST be responded to with a block message, allowing nodes to request all blocks using MSG_CMPCT_BLOCK getdatas and rely on their peer to pick an appropriate response.
9. While the current version sends transactions with the same encodings as are used in tx messages and elsewhere in the protocol, the version field in sendcmpct is intended to allow this to change in the future. For this reason, it is recommended that the code used to decode PrefilledTransaction and BlockTransactions messages be prepared to take a different transaction encoding, if and when the version field in sendcmpct changes in a future BIP.
10. Any undefined behavior in this spec may cause failure to transfer block to, peer disconnection by, or self-destruction by the receiving node. A node receiving non-minimally-encoded CompactSize encodings should make a best-effort to eat the sender's cat.
### Pre-Validation Relay and Consistency Considerations
1. As high-bandwidth mode permits relaying of CMPCTBLOCK messages prior to full validation (requiring only that the block header is valid before relay), nodes SHOULD NOT ban a peer for announcing a new block with a CMPCTBLOCK message that is invalid, but has a valid header. For avoidance of doubt, nodes SHOULD bump their peer-to-peer protocol version to 70015 or higher to signal that they will not ban or punish a peer for announcing compact blocks prior to full validation, and nodes SHOULD NOT announce a CMPCTBLOCK to a peer with a version number below 70015 before fully validating the block.
2. SPV nodes which implement this spec must consider the implications of accepting blocks which were not validated by the node which provided them. Especially SPV nodes which allow users to select a "trusted full node" to sync from may wish to avoid implementing this spec in high-bandwidth mode.
3. Note that this spec does not change the requirement that nodes only relay information about blocks which they have fully validated in response to GETDATA/GETHEADERS/GETBLOCKS/etc requests. Nodes which announce using CMPCTBLOCK message and then receive a request for associated block data SHOULD ensure that messages do not go unresponded to, and that the appropriate data is provided after the block has been validated, subject to standard message-response ordering requirements. Note that no requirement is added that the node respond to the request with the new block included in eg GETHEADERS or GETBLOCKS messages, but the node SHOULD re-announce the block using the associated announcement methods after validation has completed if it is not included in the original response. On the other hand, nodes SHOULD delay responding to GETDATA requests for the block until validation has completed, stalling all message processing for the associated peer. REJECT messages are not considered "responses" for the purpose of this section.
4. As a result of the above requirements, implementors may wish to consider the potential for the introduction of delays in responses while remote peers validate blocks, avoiding delay-causing requests where possible.
## Justification
#### Protocol design
There have been many proposals to save wire bytes when relaying blocks. Many of them have a two-fold goal of reducing block relay time and thus rely on the use of significant processing power in order to avoid introducing additional worst-case RTTs. Because this work is not focused primarily on reducing block relay time, its design is much simpler (ie does not rely on set reconciliation protocols). Still, in testing at the time of writing, nodes are able to relay blocks without the extra getblocktxn/blocktxn RTT around 90% of the time. With a smart compact-block-announcement policy, it is thus expected that this work might allow blocks to be relayed between nodes in 0.5*RTT instead of 1.5*RTT at least 75% of the time.
#### Short transaction ID calculation
There are several design goals for the Short ID calculation:
* **Performance** The sender needs to compute short IDs for all block transactions, and the receiver for all mempool transactions they are being compared to. As we're easily talking about several thousand transactions, sub-microsecond processing per-transactions is needed.
* **Space** cmpctblock messages are never optional in this protocol, and contain a short ID for each non-prefilled transaction in the block. Thus, the size of short IDs is directly proportional to the maximum bandwidth savings possible.
* **Collision resistance** It should be hard for network participants to create transactions that cause collisions. If an attacker were able to cause such collisions, filling mempools (and, thus, blocks) with them would cause poor network propagation of new (or non-attacker, in the case of a miner) blocks.
SipHash is a secure, fast, and simple 64-bit MAC designed for network traffic authentication and collision-resistant hash tables. We truncate the output from SipHash-2-4 to 48 bits (see next section) in order to minimize space. The resulting 48-bit hash is certainly not large enough to avoid intentionally created individual collisons, but by using the block hash as a key to SipHash, an attacker cannot predict what keys will be used once their transactions are actually included in a relayed block. We mix in a per-connection 64-bit nonce to obtain independent short IDs on every connection, so that even block creators cannot control where collisions occur, and random collisions only ever affect a small number of connections at any given time. The mixing is done using SHA256(block_header || nonce), which is slow compared to SipHash, but only done once per block. It also adds the ability for nodes to choose the nonce in a better than random way to minimize collisions, though that is not necessary for correct behaviour. Conversely, nodes can also abuse this ability to increase their ability to introduce collisions in the blocks they relay themselves. However, they can already cause more problems by simply refusing to relay blocks. That is inevitable, and this design only seeks to prevent network-wide misbehavior.
#### Random collision probabilty
Thanks to the block-header-based SipHash keys, we can assume that the only collisions on links between honest nodes are random ones.
For each of the `t` block transactions, the receiver will compare its received short ID with that of a set of `m` mempool transactions. We assume that each of those `t` has a chance `r` to be included in that set of `m`. If we use `B` bits short IDs, for each comparison between a received short ID and a mempool transaction, there is a chance of `P = 1 - 1 / 2^B` that a mismatch is detected as such.
When comparing a given block transaction to the whole set of mempool transactions, there are 5 cases to distinguish:
1. The receiver has exactly one match, which is the correct one. This has chance `r * P^(m - 1)`.
2. The receiver has no matches. This has chance `(1 - r) * P^m`.
3. The receiver has at least two matches, one of which is correct. This has chance `r * (1 - P^(m - 1))`.
4. The receiver has at least two matches, both of which are incorrect. This has chance `(1 - r) * (1 - P^m - m * (1 - P) * P^(m - 1))`.
5. The receiver has exactly one match, but an incorrect one. This has chance `(1 - r) * m * (1 - P) * P^(m - 1)`.
(note that these 5 numbers always add up to 100%)
In case 1, we're good. In cases 2, 3, or 4, we request the full transaction because we know we're uncertain. Only in case 5, we fail to reconstruct. The chance that case 5 does not occur in any of the `t` transactions in a block is `(1 - (1 - r) * m * (1 - P) * P^(m - 1))^t`. This expression is well approximated by `1 - (1 - r) * m * (1 - P) * t` = `1 - (1 - r) * m * t / 2^B`. Thus, if we want only one in F block transmissions between honest nodes to fail under the conservative `r = 0` assumption, we need `log2(F * m * t)` bits hash functions.
This means that `B = 48` bits short IDs suffice for blocks with up to `t = 10000` transactions, mempools up to `m = 100000` transactions, with failure to reconstruct at most one in `F = 281474` blocks. Since failure to reconstruct just means we fall back to normal inv/header based relay, it isn't necessary to avoid such failure completely. It just needs to be sufficiently rare they have a lower impact than random transmission failures (for example, network disconnection, node overloaded, ...).
#### Separate version for segregated witness
The changes to transaction and block relay in BIP 144 introduce separate MSG_FILTERED_ versions of messages in getdata,
allowing a receiver to choose individually where witness data is wanted.
This method is not useful for compact blocks because `cmpctblock` blocks can be sent unsolicitedly in high-bandwidth
mode, so we need to negotiate at least whether those should include witness data up front. There is little use for a
validating node that only sometimes processes witness data, so we may as well use that negotiation for everything and
turn it into a separate protocol version. We also need a means to distinguish different versions of the same transaction
with different witnesses for correct reconstruction, so this also forces us to use wtxids instead of txids for short IDs
everywhere in that case.
## Backward compatibility
Older clients remain fully compatible and interoperable after this change.
## Implementation
https://github.com/bitcoin/bitcoin/pull/8068 for version 1.
https://github.com/bitcoin/bitcoin/pull/8393 for version 2.
## Acknowledgements
Thanks to Gregory Maxwell for the initial suggestion as well as a lot of back-and-forth design and significant testing.
Thanks to Nicolas Dorier for the protocol flow diagram.
## Copyright
This document is placed in the public domain.
protocol/forks/bip-0157.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0157
# BIP-157
BIP: 157
Layer: Peer Services
protocol/forks/bip-0158.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0158
# BIP-158
BIP: 158
Layer: Peer Services
protocol/forks/bip-0159.md
+1 -1
View File
@@ -1,4 +1,4 @@
# BIP-0159
# BIP-159
BIP: 159
Layer: Peer Services
protocol/network/messages.md
+27 -3
View File
@@ -71,11 +71,11 @@ Messages with an unrecognized `command string` are ignored by most implementatio
| [getheaders](/protocol/network/messages/getheaders) | *Requests block headers from a peer* |
| [ping](/protocol/network/messages/ping) | *Requests a confirmation (pong) that the peer is still active* |
| [sendheaders](/protocol/network/messages/sendheaders) | *Requests that new blocks are sent as headers instead of hashes* |
| [version](/protocol/network/messages/version) | *Describes peer capabilities* |
| mempool | *Request mempool contents* |
| [version](/protocol/network/messages/version) | *Describes peer capabilities, particularly through the [Services Bitfield](/protocol/network/messages/version#services-bitfield)* |
| [mempool](/protocol/network/messages/mempool) | *Request mempool contents* |
#### Responses
| Command String | Synopsis |
| -- | -- |
| [addr](/protocol/network/messages/addr) | *Provides a peer with the addresses of other peers* |
@@ -88,6 +88,30 @@ Messages with an unrecognized `command string` are ignored by most implementatio
| [tx](/protocol/network/messages/tx) | *Provides a transaction* |
| [verack](/protocol/network/messages/verack) | *Response to a [version](/protocol/network/messages/version) message* |
#### Compact Blocks
Compact blocks, defined in [BIP-152](/protocol/forks/bip-0152), seek to minimize the amount of data transferred when a block is mined by taking advantage of the fact that peers often already have most, if not all, of the transactions in a new block.
| Command String | Synopsis |
| -- | -- |
| [sendcmpct](/protocol/network/messages/sendcmpct) | *Indicates that this node supports the Compact Block protocol.* |
| [cmpctblock](/protocol/network/messages/cmpctblock) | *Announces and provides abbreviated contents of a block.* |
| [getblocktxn](/protocol/network/messages/getblocktxn) | *Requests additional transactions from a given block.* |
| [blocktxn](/protocol/network/messages/blocktxn) | *Returns requests transactions contained within a block (in response to [getblocktxn](/protocol/network/messages/getblocktxn).* |
#### Other Message Types (Extensions)
| Command String | Synopsis | Supported Implementations
| -- | -- | -- |
| **XVersion:** [xupdate](/protocol/network/messages/xupdate) | *Communicates a change in peer capabilities.* | BCHUnlimited |
| **XVersion:** [xversion](/protocol/network/messages/xversion) | *Describes peer capabilities in an extensible manner.* | BCHUnlimited |
| **XVersion:** [xverack](/protocol/network/messages/xverack) | *Response to an [xversion](/protocol/network/messages/xversion) message.* | BCHUnlimited |
| **XThin:** [get_xblocktx](/protocol/network/messages/get_xblocktx) | *Request unknown transactions from a block.* | BCHUnlimited |
| **XThin:** [get_xthin](/protocol/network/messages/get_xthin) | *Request a previously announced xthin block from the announcing peer.* | BCHUnlimited |
| **XThin:** [thinblock](/protocol/network/messages/thinblock) | *A description of a block including full transactions only when it is known that the peer does not have them.* | BCHUnlimited |
| **XThin:** [xthinblock](/protocol/network/messages/xthinblock) | *A description of a block including full transactions only when it is known that the peer does not have them. Uses truncated hashes to minimize data transfer* | BCHUnlimited |
| **XThin:** [xblocktx](/protocol/network/messages/xblocktx) | *Provides a set of a transactions contained within a block (in response to [get_xblocktx](/protocol/network/messages/get_xblocktx)).* | BCHUnlimited |
## Example message
The below segments, when concatenated in order, create a sample [verack](/protocol/network/messages/verack) message.
protocol/network/messages/addr.md
+1 -1
View File
@@ -5,7 +5,7 @@
# Response: Addresses ("addr")
Provide information about other prospective P2P protocol peers. Peers SHOULD not send this message unsolicited (see [getaddr](/protocol/network/messages/getaddr)), and nodes that receive an unsolicited addr message MUST ignore it. This behavior helps prevent eclipse and partitioning attacks by not allowing an attacker to aggressively seed peer connection tables with its own nodes.
Provide information about other prospective P2P protocol peers. Peers SHOULD not send this message unsolicited (see [`getaddr`](/protocol/network/messages/getaddr)), and nodes that receive an unsolicited `addr` message MUST ignore it. This behavior helps prevent eclipse and partitioning attacks by not allowing an attacker to aggressively seed peer connection tables with its own nodes.
## Message Format
protocol/network/messages/blocktxn.md
+13
View File
@@ -0,0 +1,13 @@
# Response: Block Transactions ("blocktxn")
Transmits a group of transactions from a block to a peer.
A `blocktxn` message is sent in response to a [`getblocktxn`](/protocol/network/messages/getblocktxn) message, and must include exactly the set of transactions requested, in the order requested (i.e. in the order they appear in the block).
## Format
| Field | Length | Format | Description |
|--|--|--|--|
| block hash | 32 bytes | bytes | The hash of the block that contains the transactions to follow. |
| transaction count | variable | [variable length integer](/protocol/formats/variable-length-integer) | The number of transactions being provided. |
| transactions | variable | `transaction count` [transactions](/protocol/blockchain/transaction) | The transactions, in order, serialized as though in a [`tx`](/protocol/network/messages/tx) message. |
protocol/network/messages/cmpctblock.md
+56
View File
@@ -0,0 +1,56 @@
# Announcement: Compact Block ("cmpctblock")
Transmits a compact block to a peer.
This message is automatically sent to "high bandwidth relaying" peers, or in response to a [`getdata`](/protocol/network/messages/getdata) request specifying the [compact block inventory type](/protocol/network/messages/inv#inventory-types) and block hash.
## Format
The below format is referred to in [BIP-152](/protocol/forks/bip-0152) as `HeaderAndShortIDs`.
| Field | Length | Format | Description |
|--|--|--|--|
| header | 80 bytes | [block header](/protocol/blockchain/block/block-header) | The header of the block being sent. |
| nonce | 4 bytes | unsigned integer<sup>[(LE)](/protocol/misc/endian/little)</sup> | A nonce used in the calculation of the short transaction IDs to follow. This is generated by the sender and must be unique per block but not necessary per peer. |
| short id count | variable | [variable length integer](/protocol/formats/variable-length-integer) | The number of short transaction IDs to follow. This will be the number of transaction in the block minus the number of "prefilled" transactions provided at the end of this message. |
| short ids | `short_id_count* 6` bytes | `short_id_count` 6-byte unsigned integers<sup>[(LE)](/protocol/misc/endian/little)</sup> | The list of transactions in the block, referenced by [short transaction IDs](#short-transaction-ids). This includes every transaction in the block *except* the "prefilled" transactions to follow. |
| prefilled transaction count | variable | [variable length integer](/protocol/formats/variable-length-integer) | The number of prefilled transactions to follow. |
| prefilled transactions | variable | `prefilled_transaction_count` [prefilled transactions](#prefilled-transactions) | The coinbase transaction and any other transactions in the block that the sender believes the peer may be missing. |
### Short Transaction IDs
Short transaction IDs are generated using the following steps:
1. Generate a key, `k`, as the little-endian single-SHA-256 hash of the block header concatenated with the little-endian compact block nonce generated by the sender (i.e. either a new random value or the one received from a peer).
2. Calculate the [SipHash-2-4](https://en.wikipedia.org/wiki/SipHash) of the full transaction ID using `k` as the key. For implementations that expect two keys, use the first 64-bits of the little-endian hash as `k<sub>0</sub>` and the second 64-bits as `k<sub>1</sub>`.
3. Drop the 2 most-significant bytes of the SipHash output to get the 6-byte short transaction ID.
For more information about the design of these short IDs, see [BIP-152:Short transaction ID calculation](/protocol/forks/bip-0152#short-transaction-id-calculation).
For additional details on how the recipient should handle this message, see [reconstructing the block](#reconstructing-the-block).
### Prefilled Transactions
Prefilled transactions specify the full transaction data for transactions that are not expected to already be known by the recipient.
The coinbase transaction is always such a transaction, while others may be included at the sender's discretion.
The format is as follows:
| Field | Length | Format | Description |
|--|--|--|--|
| index | variable | [variable length integer](/protocol/formats/variable-length-integer) | The ["differentially encoded"](#differentially-encoded-indexes) position of the transaction with in the block. |
| transaction | variable | [transaction](/protocol/blockchain/transaction#format) | The full transaction contents, as in a [`tx`](/protocol/network/messages/tx) message. |
## Differentially Encoded Indexes
Where compact-block-related messages reference the indexes of transactions within a block, they use a differential encoding to further minimize the amount of data used.
In such a list of transactions with indexes, each index is interpreted as a relative index from the previous transaction in the list.
That is, if the first transaction has an index of `0`, it is the first transaction in the block (true index `0`).
If the second transaction also has an index of `0`, it is the second transaction in the block (true index `1`).
Generally, if `d<sub>n</sub>` is the differentially encoded index for the `n`-th transaction in a given list, and `t<sub>n</sub>` is that transactions true index within the block, `t<sub>n</sub> = t<sub>n-1</sub> + d<sub>n</sub> + 1`.
Conversely, `d<sub>n</sub> = t<sub>n</sub> - t<sub>n-1</sub> - 1`.
## Reconstructing the Block
Upon receipt of a `cmpctblock` message, the recipient must first determine whether it now has all the transactions needed to reconstruct the block. First, all prefilled transactions should be processed. If some transactions are still unknown, the recipient may request then using a [`getblocktxn`](/protocol/network/messages/getblocktxn) message. Once the recipient has all of the necessary transactions, the block's [merkle tree](/protocol/blockchain/block/merkle-tree) can be re-built by adding the transactions in the order specified by the indexes. NOTE: since [HF-20181115](/protocol/forks/hf-20181115), [CTOR](/protocol/forks/hf-20181115#canonical-transaction-order) means that this the same order can also be achieved by sorting the transactions by their hashes.
For more information on when `cmpctblock` messages should be sent and how they should be validated, see [BIP-152](/protocol/forks/bip-0152).
protocol/network/messages/filterload.md
+1 -1
View File
@@ -9,7 +9,7 @@ Inserts a transaction and merkle block filter into the receiving peer, overwriti
### Effect on Transactions
This message installs a [bloom filter](/protocol/spv/bloom-filter) into the peer. Subsequent [inv](/protocol/network/messages/inv) and [merkleblock](/protocol/network/messages/merkleblock) messages only provide transactions that in match this bloom filter in some manner. The following items in a transaction are checked against the bloom filter:
This message installs a [bloom filter](/protocol/spv/bloom-filter) into the peer. Subsequent [`inv`](/protocol/network/messages/inv) and [`merkleblock`](/protocol/network/messages/merkleblock) messages only provide transactions that in match this bloom filter in some manner. The following items in a transaction are checked against the bloom filter:
- The transaction hash
- Each data field in every [locking script](/protocol/blockchain/transaction/locking-script) in the transaction
protocol/network/messages/get_xblocktx.md
+2 -2
View File
@@ -1,6 +1,6 @@
# Request: GET_XBLOCKTX
# Request: Get Xthin Block Transactions ("get_xblocktx")
This message is sent to remote peer to request the transations still needed to reconstruct the block from the mempool after receipt of the previous [`XTHINBLOCK`](xthinblock) or [`THINBLOCK`](thinblock) message.
This message is sent to remote peer to request the transactions still needed to reconstruct the block from the mempool after receipt of the previous [`xthinblock`](xthinblock) or [`thinblock`](thinblock) message.
This message should start with the hash of the block that is intended to be reconstructed, followed by a list of short hashes of the transactions still needed.
## Format
protocol/network/messages/get_xthin.md
+5 -5
View File
@@ -1,9 +1,9 @@
# Request: GET_XTHIN
# Request: Get Xthin Block ("get_xthin")
Request the [`XTHINBLOCK`](xthinblock) from the peer that was anoounced via the previous [`INV`](..\network\messages\inv) message.
The message should start with an inventory message that indicates the `XTHINBLOCK` requested, followed by the Bloom filter generated from the mempool.
Upon receipt of this request, the peer should loads the attached Bloom filter, creates and responds with `XTHINBLOCK`.
If the `XTHINBLOCK` cannot be created without hash collisions, the peer may repond with `THINBLOCK` or full blocks.
Request the [`xthinblock`](xthinblock) from the peer that was anoounced via the previous [`inv`](/protocol/network/messages/inv) message.
The message should start with an inventory message that indicates the `xthinblock` requested, followed by the Bloom filter generated from the mempool.
Upon receipt of this request, the peer should loads the attached Bloom filter, creates and responds with `xthinblock`.
If the `xthinblock` cannot be created without hash collisions, the peer may respond with `thinblock` or full blocks.
## Format
protocol/network/messages/getaddr.md
+1 -1
View File
@@ -1,6 +1,6 @@
# Request: Get Addresses (“getaddr”)
Request the information about active peers on the network. The recipient may reply with peer IP addresses using an [addr](/protocol/network/messages/addr) message.
Request the information about active peers on the network. The recipient may reply with peer IP addresses using an [`addr`](/protocol/network/messages/addr) message.
## Message Format
This message has no contents.
protocol/network/messages/getblocks.md
+4 -4
View File
@@ -5,7 +5,7 @@
# Request: Get Blocks (“getblocks”)
Request the sequence of blocks that occur after a specific block. If the specified block is on the server's most-work chain, the server responds with a set of up to 500 [INV](/protocol/network/messages/inv.md) messages identifying the next blocks on that chain. If the specified block is not on the most-work chain, the server uses block information in the *locator* structure to determine the fork point and provides [INV](/protocol/network/messages/inv.md) messages from that point.
Request the sequence of blocks that occur after a specific block. If the specified block is on the server's most-work chain, the server responds with a set of up to 500 [`inv`](/protocol/network/messages/inv.md) messages identifying the next blocks on that chain. If the specified block is not on the most-work chain, the server uses block information in the *locator* structure to determine the fork point and provides [`inv`](/protocol/network/messages/inv.md) messages from that point.
## Message Format
@@ -18,12 +18,12 @@ Request the sequence of blocks that occur after a specific block. If the specif
### Locator
See [GETHEADERS](/protocol/network/messages/getheaders.md) for a detailed description of the Locator object.
See [`getheaders`](/protocol/network/messages/getheaders) for a detailed description of the Locator object.
The response will begin at **the child of** the first hash in the locator list that matches a block hash identifier held by the responder. If no hashes match, there will be no INV messages sent.
*Use an empty locator to get an INV for block 1, and GETHEADERS that block to discover the genesis block hash in the prevBlock field*
*Use an empty locator to get an `inv` for block 1, and `getheaders` that block to discover the genesis block hash in the prevBlock field*
### Stop At Hash
The sender will stop sending INVs if it encounters this hash. If the hash is never encountered, the sender will stop after 500 INV messages or when it hits the blockchain tip.
The sender will stop sending inventory messages if it encounters this hash. If the hash is never encountered, the sender will stop after 500 inventory messages or when it hits the blockchain tip.
protocol/network/messages/getblocktxn.md
+14
View File
@@ -0,0 +1,14 @@
# Request: Get Block Transactions ("getblocktxn")
Requests unknown transactions in a block from a peer.
Upon receiving a [`cmpctblock`](/protocol/network/messages/cmpctblock) message, a node may determine it is still missing some transactions from the recent block, which it can request using a `getblocktxn` message.
A node receiving a `getblocktxn` message, that has recently sent a `cmpctblock` message for the specified block to this peer, must respond with either a [`blocktxn`](/protocol/network/messages/blocktxn) message or a [`block`](/protocol/network/messages/block) message.
## Format
| Field | Length | Format | Description |
|--|--|--|--|
| block hash | 32 bytes | bytes | The hash of the block containing the desired transactions. |
| indexes count | variable | [variable length integer](/protocol/formats/variable-length-integer) | The number of transactions being requested. |
| indexes | variable | `indexes_count` [variable length integers](/protocol/formats/variable-length-integer) | The [differentially encoded indexes](/protocol/network/messages/cmpctblock#differentially-encoded-indexes) of the transactions within the block being requested. |
protocol/network/messages/getdata.md
+1 -1
View File
@@ -5,7 +5,7 @@
# Request: Get Data (“getdata”)
Requests information (generally previously announced via an [inv](/protocol/network/messages/inv) message) from a peer.
Requests information (generally previously announced via an [`inv`](/protocol/network/messages/inv) message) from a peer.
As such, a getdata request carries the same general format as an inventory message and is used to request any items that the node was previously unaware.
## Message Format
protocol/network/messages/inv.md
+4 -4
View File
@@ -7,9 +7,9 @@
Notifies peers about the existence of some information (block or transaction).
Based on selected services in the [version](/protocol/network/messages/version) message, inventory messages may not be sent.
Based on selected services in the [`version`](/protocol/network/messages/version) message, inventory messages may not be sent.
If a bloom filter has been sent to this node via [filterload](/protocol/network/messages/filterload), a transaction inventory will only be sent for transactions that match the bloom filter.
If a bloom filter has been sent to this node via [`filterload`](/protocol/network/messages/filterload), a transaction inventory will only be sent for transactions that match the bloom filter.
## Message Format
@@ -18,7 +18,7 @@ If a bloom filter has been sent to this node via [filterload](/protocol/network/
| inventory count | variable | [variable length integer](/protocol/formats/variable-length-integer) | The number of inventory items in this message. |
| inventory items | `inventory_count` * 36 bytes | `inventory_count` [inventory items](#inventory-item-format) | The set of inventory items being transmitted. Up to 50,000 inventory items can be sent in a single inventory message.|
NOTE: Since a block header is a relatively small data structure, and block propagation speed is an important network metric, a peer may send [headers](/protocol/network/messages/headers) messages in place of inventory messages when a block arrives. This behavior can be requested using the [sendheaders](/protocol/network/messages/sendheaders) message.
NOTE: Since a block header is a relatively small data structure, and block propagation speed is an important network metric, a peer may send [`headers`](/protocol/network/messages/headers) messages in place of inventory messages when a block arrives. This behavior can be requested using the [`sendheaders`](/protocol/network/messages/sendheaders) message.
#### Inventory Item Format
@@ -38,5 +38,5 @@ The type of the object that is available.
| 0x0004 | Compact block
| 0x0005 | Xthin block (Bitcoin Unlimited)
| 0x0006 | Graphene Block (Bitcoin Unlimited)
| 0x94A0 | [Double Spend Proof](/protocol/network/messages/dsproof-beta) |
| 0x94A0 | [Double Spend Proof - Beta](/protocol/network/messages/dsproof-beta) |
protocol/network/messages/mempool.md
+10
View File
@@ -0,0 +1,10 @@
# Request: Mempool ("mempool")
Requests that the recipient notify the sender of transactions that are currently in its [mempool](/protocol/blockchain/memory-pool).
Recipients of a `mempool` message MAY respond with a set of transaction hashes currently in their mempool via an [`inv`](/protocol/network/messages/inv) message.
The mempool message was defined in [BIP-35](/protocol/forks/bip-0035).
## Message Format
This message has no contents.
protocol/network/messages/merkleblock.md
+3 -3
View File
@@ -7,9 +7,9 @@
Provides a block header and partial merkle proof tree to show that the selected transaction [hashes](/protocol/blockchain/hash) exist in the block.
Transactions are selected by honest servers if they match the bloom filter installed by the client. However, note that servers can omit transactions and this cannot be detected except by receiving a merkleblock message from an honest server.
Transactions are selected by honest servers if they match the bloom filter installed by the client. However, note that servers can omit transactions and this cannot be detected except by receiving a `merkleblock` message from an honest server.
All selected transactions are subsequently sent as separate [tx](/protocol/network/messages/tx) messages. Due to multi-threading on the server, clients should not assume that these transaction messages directly follow the merkleblock message.
All selected transactions are subsequently sent as separate [`tx`](/protocol/network/messages/tx) messages. Due to multi-threading on the server, clients should not assume that these transaction messages directly follow the `merkleblock` message.
## Message Format
@@ -24,7 +24,7 @@ All selected transactions are subsequently sent as separate [tx](/protocol/netwo
#### Partial Merkle Tree Flag Bits
The partial merkle tree flag bits effectively create a variable-width bitfield, indicating which transaction hashes from the block the merkleblock message pertains to actually matched the client's bloom filter.
The partial merkle tree flag bits effectively create a variable-width bitfield, indicating which transaction hashes from the block the `merkleblock` message pertains to actually matched the client's bloom filter.
If you picture the merkle tree root at the top, the flag bits represent a top-down yes/no approach to whether the a given branch (or leaf node) of the tree includes any matching transactions.
For full details, see [BIP-37: Partial Merkle Branch Format](/protocol/forks/bip-0037#partial-merkle-branch-format).
protocol/network/messages/ping.md
+2 -2
View File
@@ -8,11 +8,11 @@
Connection keep-alive, "aliveness" and latency discovery.
If a node receives a PING message, it replies as quickly as possible with a [PONG](/protocol/network/messages/pong.md) message with the provided *nonce*.
If a node receives a `ping` message, it replies as quickly as possible with a [`pong`](/protocol/network/messages/pong.md) message with the provided *nonce*.
## Message Format
| Field | Length | Format | Description |
|--|--|--|--|
| nonce | 8 bytes | unsigned 64 bit integer<sup>[(LE)](/protocol/misc/endian/little.md)</sup> | An arbitrary value provided to connect the ping message with the pong reply
| nonce | 8 bytes | unsigned 64 bit integer<sup>[(LE)](/protocol/misc/endian/little.md)</sup> | An arbitrary value provided to connect the ping message with the `pong` reply. |
protocol/network/messages/pong.md
+2 -2
View File
@@ -6,10 +6,10 @@
# Response: Pong ("pong")
Connection keep-alive, "aliveness" and latency discovery. This message is sent in response to a [PING](/protocol/network/messages/ping.md) message.
Connection keep-alive, "aliveness" and latency discovery. This message is sent in response to a [`ping`](/protocol/network/messages/ping) message.
## Message Format
| Field | Length | Format | Description |
|--|--|--|--|
| nonce | 8 bytes | unsigned 64 bit integer<sup>[(LE)](/protocol/misc/endian/little)</sup> | The value provided by the [PING](/protocol/network/messages/ping.md) message.
| nonce | 8 bytes | unsigned 64 bit integer<sup>[(LE)](/protocol/misc/endian/little)</sup> | The value provided by the `ping` message.
protocol/network/messages/sendcmpct.md
+3 -3
View File
@@ -1,13 +1,13 @@
# Request: SENDCMPCT
# Request: Send Compact Blocks ("sendcmpct")
This message notifies the peer that this node support Compact Block Relay.
Node should only send this message after protocol version >= 70014.
This message can indicate the High Bandwidth Relaying mode or Low Bandwidth Relaying mode by setting the `Mode` field.
Upon recepit of this message from the peer, node should announce new block via `cmpctblock` message if `Mode = 1`, or node should announce new block via `inv` or `header` message as defined in BIP-0130.
Upon receipt of this message from the peer, node should announce new block via `cmpctblock` message if `Mode = 1`, or node should announce new block via `inv` or `header` message as defined in BIP-0130.
## Format
| Field | Length | Format | Description |
|--|--|--|--|
| Mode | 1 byte | Integer | 1: High Bandwidth Relaying mode;</br> 0: Low Bandwidth Relaying mode;</br> Other: Invalid |
| Version | 8 bytes | Integer <sup>[LE](..\misc\endian\little)</sup> | Must set to 1, otherwise invalid.|
| Version | 8 bytes | Integer <sup>[(LE)](/protocol/misc/endian/little)</sup> | Must set to 1, otherwise invalid.|
protocol/network/messages/sendheaders.md
+1 -1
View File
@@ -1,6 +1,6 @@
# Request: Send Headers (“sendheaders”)
Requests that new blocks are sent to the sender as a [header](/protocol/network/messages/headers) message instead of as a block hash [inv](/protocol/network/messages/inv) message.
Requests that new blocks are sent to the sender as a [`header`](/protocol/network/messages/headers) message instead of as a block hash [`inv`](/protocol/network/messages/inv) message.
## Message Format
This message has no contents.
protocol/network/messages/thinblock.md
+7 -7
View File
@@ -1,14 +1,14 @@
# Response: THINBLOCK
# Response: Thin Block ("thinblock")
This message delivers an `THINBLOCK` to the remote peer.
The `THINBLOCK` starts with the block header, followed by the hashes of all the transactions in the block.
This message delivers an `thinblock` to the remote peer.
The `thinblock` starts with the block header, followed by the hashes of all the transactions in the block.
The transaction hashes are in the same order as they are in the actual block.
All the transactions that are in the block, but not matched by the Bloom filter inlcuded in the previous [`GET_XTHIN`](get_xthin) mesaage should be appended, which should include at least the coinbase transaction.
All the transactions that are in the block, but not matched by the Bloom filter included in the previous [`get_xthin`](get_xthin) message should be appended, which should include at least the coinbase transaction.
## Format
| Field | Length | Format | Description |
|--|--|--|--|
| Block header | 80 bytes | [block header](..\blockchain\block\block-header) | The header of the block transmitted.|
| Transaction hashes | # of tx * 32 bytes | vector | The truncated hashes of all the transactions included in the block.|
| Missing transactions | variable | vector | Transactions that are included in the blocks but are not matched by the Bloom filter.|
| Block header | 80 bytes | [block header](/protocol/blockchain/block/block-header) | The header of the block transmitted. |
| Transaction hashes | # of tx * 32 bytes | vector | The hashes of all the transactions included in the block. |
| Missing transactions | variable | vector | Transactions that are included in the blocks but are not matched by the Bloom filter. |
protocol/network/messages/verack.md
+1 -3
View File
@@ -1,12 +1,10 @@
# Handshake: Version Acknowledgement ("verack")
The `verack` message is sent in reply to a [version](/protocol/network/messages/version) message.
The `verack` message is sent in reply to a [`version`](/protocol/network/messages/version) message.
Sending a `verack` in response to a `version` message indicates to the remote that its connection and version has been accepted.
There is no version negotiation functionality between nodes; therefore if the node does not accept the version supplied by the remote then the node disconnects instead of responding with a `verack`.
This `verack` message consists of only a message header; the command string is "verack".
## Message Format
This message has no contents.
protocol/network/messages/version.md
+1 -1
View File
@@ -3,7 +3,7 @@
The version message is a part of the node connection [handshake](/protocol/network/node-handshake) and indicates various connection settings, networking information, and the services provided by the sending node (see Services Bitfield [below](#services-bitfield)).
The node connection is not considered established until both nodes have sent and received both a `version` and [verack](/protocol/network/messages/verack) message.
The node connection is not considered established until both nodes have sent and received both a `version` and [`verack`](/protocol/network/messages/verack) message.
## Message Format
protocol/network/messages/xblocktx.md
+2 -2
View File
@@ -1,6 +1,6 @@
# Response: XBLOCKTX
# Response: Xthin Block Transactions ("xblocktx")
This message delivers the transactions specified in the previous [`GET_XBLOCKTX`](get_xblocktx) message to the remote peer.
This message delivers the transactions specified in the previous [`get_xblocktx`](get_xblocktx) message to the remote peer.
This message starts with the hash of the block being reconstructed, followed by a list of transactions requested.
## Format
protocol/network/messages/xthinblock.md
+5 -5
View File
@@ -1,14 +1,14 @@
# Response: XTHINBLOCK
# Response: Xthin Block ("xthinblock")
This message delivers an `XTHINBLOCK` to the remote peer.
The `XTHINBLOCK` starts with the block header, followed by the truncated hashes of all the transactions in the block.
This message delivers an `xthinblock` to the remote peer.
The `xthinblock` starts with the block header, followed by the truncated hashes of all the transactions in the block.
The transaction hashes are truncated to the first 64 bits and are in the same order as they are in the actual block.
All the transactions that are in the block, but not matched by the Bloom filter inlcuded in the previous [`GET_XTHIN`](get_xthin) mesaage should be appended, which should include at least the coinbase transaction.
All the transactions that are in the block, but not matched by the Bloom filter included in the previous [`get_xthin`](get_xthin) message should be appended, which should include at least the coinbase transaction.
## Format
| Field | Length | Format | Description |
|--|--|--|--|
| Block header | 80 bytes | [block header](..\blockchain\block\block-header) | The header of the block transmitted.|
| Block header | 80 bytes | [block header](/protocol/blockchain/block/block-header) | The header of the block transmitted.|
| Transaction hashes | # of tx * 8 bytes | vector | The truncated hashes of all the transactions included in the block.|
| Missing transactions | variable | vector | Transactions that are included in the blocks but are not matched by the Bloom filter.|
protocol/network/messages/xupdate.md
+4 -1
View File
@@ -5,7 +5,10 @@
# Handshake Extension: XVersion (“xversion”)
This message notifies a peer about changes to protocol parameters. It follows the same format as [xversion](/protocol/network/messages/xversion) protocol parameters. Implementations **SHOULD** only send changed parameters, rather than every parameter. Note that some XVERSION parameters are not changeable and therefore will be ignored if they appear in this message.
This message notifies a peer about changes to protocol parameters.
It follows the same format as [`xversion`](/protocol/network/messages/xversion) protocol parameters.
Implementations SHOULD only send changed parameters, rather than every parameter.
Note that some `xversion` parameters are not changeable and therefore will be ignored if they appear in this message.
See the [xversion fields](/protocol/network/messages/xversion#xversion-fields) for detailed information about each parameter.
protocol/network/messages/xverack.md
+2 -2
View File
@@ -1,7 +1,7 @@
# Handshake Extension: XVersion Acknowledgement (“xverack”)
The `xverack` message is sent in reply to an [xversion](/protocol/network/messages/xversion) message.
Sending an `xverack` in response to a `version` message indicates to the remote that its XVersion values have been accepted.
The `xverack` message is sent in reply to an [`xversion`](/protocol/network/messages/xversion) message.
Sending an `xverack` in response to an `xversion` message indicates to the remote that its XVersion values have been accepted.
## Message Format
protocol/network/messages/xversion.md
+2 -2
View File
@@ -5,7 +5,7 @@
# Handshake Extension: XVersion (“xversion”)
This message notifies a peer about extended protocol parameters. This message MAY be sent during connection initialization. If sent, it MUST be sent immediately subsequent to the receipt of the [verack](/protocol/network/messages/verack.md) message, and before other non-initialization messages are sent.
This message notifies a peer about extended protocol parameters. This message MAY be sent during connection initialization. If sent, it MUST be sent immediately subsequent to the receipt of the [`verack`](/protocol/network/messages/verack) message, and before other non-initialization messages are sent.
## Message Format
@@ -38,7 +38,7 @@ XVersion field identifiers are 32 bits and split into a 16 bit prefix and 16 bit
See [xversionkeys.dat](https://github.com/BitcoinUnlimited/BitcoinUnlimited/blob/bucash1.7.0.0/src/xversionkeys.dat) for the most up-to-date field definitions defined by the BitcoinUnlimited full node.
Note that:
* *u64c* refers to a [variable length integer](/protocol/formats/variable-length-integer).
* *Changeable* fields MAY be changed during the course of a connection via the [xupdate](/protocol/network/messages/xupdate) message.
* *Changeable* fields MAY be changed during the course of a connection via the [`xupdate`](/protocol/network/messages/xupdate) message.
### Support