From 63d3a0eba3096efaaf1e83224f91ef19ee9b467e Mon Sep 17 00:00:00 2001 From: Jonathan Silverblood Date: Mon, 27 Dec 2021 14:47:44 +0200 Subject: [PATCH 01/22] Address size of the input sequence number According to https://github.com/bitcoincashorg/bitcoincash.org/blob/master/spec/replay-protected-sighash.md, the nSequence field should be 4-byte little endian. --- protocol/blockchain/transaction/transaction-signing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/protocol/blockchain/transaction/transaction-signing.md b/protocol/blockchain/transaction/transaction-signing.md index fb9792b..c310d06 100644 --- a/protocol/blockchain/transaction/transaction-signing.md +++ b/protocol/blockchain/transaction/transaction-signing.md @@ -81,7 +81,7 @@ The following table specifies, in detail, the preimage format for a signature wi | modified locking script length | variable | [variable length integer](/protocol/format/variable-length-integer) | The number of bytes for `modified_locking_script`. | | modified locking script | `modified_locking_script_length` bytes | bytes[(BE)](/protocol/misc/endian/big) | The subset of the locking script used for signing. See [Modified Locking Script](#modified-locking-script) | | previous output value | 8 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The value of the transaction output being spent. | -| input sequence number | 8 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The sequence number of the input this signature is for. | +| input sequence number | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The sequence number of the input this signature is for. | | transaction outputs hash | 32 bytes | hash[(BE)](/protocol/misc/endian/big) | A double SHA-256 hash of the outputs of the transaction. See [Transaction Outputs](#transaction-outputs-hash) for the hash preimage format. | | transaction lock time | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The lock time of the transaction. | | hash type | 4 bytes | [Hash Type](#hash-type)[(LE)](/protocol/misc/endian/little) | Flags indicating the rules for how this signature was generated. | From add7c46f1a5e5fc901ed700e6677750f1e4c203b Mon Sep 17 00:00:00 2001 From: Jonathan Silverblood Date: Mon, 27 Dec 2021 14:50:19 +0200 Subject: [PATCH 02/22] Link to may2022 update --- home.md | 1 + 1 file changed, 1 insertion(+) diff --git a/home.md b/home.md index 2f38296..62f49cc 100644 --- a/home.md +++ b/home.md @@ -46,6 +46,7 @@ | 2019 | [HF-20190515](/protocol/forks/hf-20190515) — [HF-20191115](/protocol/forks/hf-20191115) | | 2020 | [HF-20200515](/protocol/forks/hf-20200515) — [HF-20201115](/protocol/forks/hf-20201115) | | 2021 | [HF-20210515](/protocol/forks/hf-20210515) | +| 2022 | [HF-20220515](/protocol/forks/hf-20220515) | ### Network protocol [Network Messages](/protocol/network/messages) — [Handshake](/protocol/network/node-handshake) From 0492a1ab10ad8e6cc39fe24fa5c435aeeee29130 Mon Sep 17 00:00:00 2001 From: Jonathan Silverblood Date: Mon, 27 Dec 2021 14:54:22 +0200 Subject: [PATCH 03/22] Create hf-20220515.md --- protocol/forks/hf-20220515.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 protocol/forks/hf-20220515.md diff --git a/protocol/forks/hf-20220515.md b/protocol/forks/hf-20220515.md new file mode 100644 index 0000000..2dac700 --- /dev/null +++ b/protocol/forks/hf-20220515.md @@ -0,0 +1,10 @@ +# HF-20220515 + +The May 2022 hard fork is the second upgrade to be comprised of [CHIPs](#chips). + +## Contents + +The May 2022 hard fork is comprised the following CHIPs: + +1. [Native Introspection Opcodes](/protocol/forks/chips/2022-05-native-introspection-opcodes) +2. [Bigger Script Integers](/protocol/forks/chips/2022-05-bigger-script-integers) From 3cbb253900a9f9c336177261503f014b9ef31716 Mon Sep 17 00:00:00 2001 From: Jonathan Silverblood Date: Mon, 27 Dec 2021 14:55:42 +0200 Subject: [PATCH 04/22] Create 2022-05-bigger-script-integers --- .../chips/2022-05-bigger-script-integers | 359 ++++++++++++++++++ 1 file changed, 359 insertions(+) create mode 100644 protocol/forks/chips/2022-05-bigger-script-integers diff --git a/protocol/forks/chips/2022-05-bigger-script-integers b/protocol/forks/chips/2022-05-bigger-script-integers new file mode 100644 index 0000000..a72c0ac --- /dev/null +++ b/protocol/forks/chips/2022-05-bigger-script-integers @@ -0,0 +1,359 @@ +# CHIP-2021-03: Bigger Script Integers + +> OWNERS: [Jason Dreyzehner](https://bitjson.com/), [Rosco Kalis](https://github.com/rkalis), [Jonathan Silverblood](https://gitlab.com/monsterbitar) +> +> DISCUSSION: [Bitcoin Cash Research](https://bitcoincashresearch.org/t/improve-utility-of-script-calculations-larger-numbers-op-mul/39), [Telegram](https://t.me/joinchat/SeRFuFmba7tWOcKh) +> +> VERSION: 1.0 +> +> MILESTONES: **[Published](https://gitlab.com/GeneralProtocols/research/chips/-/blob/master/CHIP-2021-02-Bigger-Script-Integers.md)**, **[Specification](#technical-specification)**, **[Testnet](https://read.cash/@bitcoincashnode/bch-testnet4-for-may-2022-network-upgrade-updated-2021-11-23-95a21d51)**, **Accepted ([BU](https://twitter.com/BitcoinUnlimit/status/1460394221240786956) - [BCHN](https://twitter.com/bitcoincashnode/status/1460424339816284161) - [Knuth](https://twitter.com/KnuthNode/status/1460622308607938579) - [Verde](https://twitter.com/joshmgreen/status/1460592944516317185) - [Bitauth](https://twitter.com/Bitauth/status/1460371045207232512))**, Deployed (May 15th, 2022). + +## Summary + +This proposal expands the integer range allowed in BCH contracts (from 32-bit to 64-bit numbers) and re-enables the multiplication opcode (`OP_MUL`). + +## Deployment + +Deployment of this specification is proposed for the May 2022 upgrade. + +## Motivation + +BCH virtual machine (VM) math operations are currently limited to signed 32-bit integers, preventing contracts from operating on values larger than `2147483647` – representing satoshis, this is ~21 BCH. Workarounds which allow contracts to emulate higher-precision math are often impractical, difficult to secure, and significantly increase transaction sizes. + +This unusually-low limit on arithmetic operations has been present since the earliest Bitcoin release to avoid standardizing a strategy for overflow handling ([though 64-bit math was always used internally](#arithmetic-operation-overflows)). Since the development of covenants, this unusual overflow-handling strategy has become a source of contract vulnerabilities and a barrier to real-world applications. Few remaining computing environments operate using 32-bit integers, and this limit has never been relevant to [worst-case transaction validation performance](https://github.com/bitjson/bch-vm-limits#pre-deployment-hashing-limit-benchmark). + +## Benefits + +By allowing contracts to efficiently operate on larger numbers, this proposal enables new use cases, improves contract security, and reduces transaction sizes. + +### Larger Contract Values + +By expanding the upper bound of arithmetic inputs to `9223372036854775807`, contracts can efficiently operate on values larger than the total possible satoshi value of any transaction output ([approx. `2100000000000000`](#limiting-arithmetic-operations-to-8-byte-script-numbers)). This enables contracts to manage balances of any size, clearing the way for large, public decentralized applications. + +Expanded arithmetic capabilities also **enable greater practical use of new payment and financial systems** including: scheduled and recurring payments, risk-hedging contracts, synthetic assets, decentralized exchanges, inheritance and treasury management systems, crowdfunding and crowdmatching applications, loyalty point and token systems, delayed-withdrawal vaults (and other contract security strategies), and more. + +### Safer Contracts + +This proposal obviates the need for higher-precision math emulation. As a result, existing applications can be simplified, making them easier to develop and review. Additionally, by making arithmetic overflows less common (many common operations overflow 32 bits, but few approach 64 bits), contracts are less likely to include vulnerabilities or faults which can expose users to attacks and losses. + +### Reduced Transaction Sizes + +Because this proposal allows existing contracts to remove higher-precision math emulation, transactions employing these contracts are reduced in size. This reduces transaction fees for contract users, and it reduces storage and bandwidth costs for validators. + +## Costs & Risk Mitigation + +The following costs and risks have been assessed. + +### Modification to Transaction Validation + +Modifications to VM limits have the potential to increase worst-case transaction validation costs and expose VM implementations to Denial of Service (DOS) attacks. + +**Mitigations**: migration from 32-bit to 64-bit arithmetic has no impact on [worst-case validation performance](https://github.com/bitjson/bch-vm-limits#pre-deployment-hashing-limit-benchmark). (Notably, most implementations already use 64-bit or larger number representations internally, and overflow-checked 64-bit math is also available natively in most programming languages and computing environments.) Even at a significantly higher [practical limit approaching 10,000 operations](https://github.com/bitjson/bch-vm-limits), 64-bit arithmetic operations are thousands to millions of times less expensive than [existing scenarios](https://github.com/bitjson/bch-vm-limits#benchmarks) (varies by environment and implementation). + +### Node Upgrade Costs + +This proposal affects consensus – all fully-validating node software must implement these VM changes to remain in consensus. + +These VM changes are backwards-compatible: all past and currently-possible transactions remain valid under these new rules. + +### Ecosystem Upgrade Costs + +Because this proposal only affects internals of the VM, most wallets, block explorers, and other services will not require software modifications for these changes. Only software which offers VM evaluation (e.g. [Bitauth IDE](https://github.com/bitauth/bitauth-ide)) will be required to upgrade. + +Wallets and other services may also upgrade to add support for new contracts which will become possible after deployment of this proposal. + +### Protocol Implementation Costs + +By requiring support for 64-bit math, this proposal could increase the cost of new protocol implementations in unusual programming languages which lack native support for overflow-checked, signed, 64-bit math. + +**Mitigations**: nearly all modern platforms and languages include native support for 64-bit or larger integers. Additionally, transaction output values are already encoded using unsigned, 64-bit integers, so many BCH software libraries already include support for 64-bit integers. + +> For example, until recent years, JavaScript supported only 64-bit floating point numbers (IEEE-754). While JavaScript now widely supports [`BigInt`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/BigInt), many older BCH JavaScript libraries still include big integer polyfills to support encoding of transaction output values without using `BigInt`. These polyfills typically support 64-bit math, making implementation easier. + +## Technical Specification + +All BCH VM operations which operate on `Script Numbers` (A.K.A. `CSCriptNum`) are modified to support values within the expanded range of 8 bytes (64 bits), and `OP_MUL` (`0x95`/`149`) is re-enabled. + +### Script Number Range + +The `Script Number` format (A.K.A. `CSCriptNum`) is a consensus-standardized, variable-length, signed integer format used by all VM operations which consume or produce numeric values. In practice, the allowable range of the Script Number format is limited by the parsing of Script Number values within all VM operations which consume Script Numbers. + +Prior to activation of this proposal, Script Number parsing is limited to `4` byte stack values. After activation, Script Number parsing must be limited to `8` byte stack values. This expands the available range from a minimum of `0xffffffff` (`-2147483647`) and maximum of `0xffffff7f` (`2147483647`) to a minimum of `0xffffffffffffffff` (`-9223372036854775807`) and maximum of `0xffffffffffffff7f` (`9223372036854775807`). + +> Note: an unusual property of the existing Script Number format reduces its negative range by `1`: the Script Number format can hypothetically represent both "positive" `0` (`0x`, the empty stack item) and "negative" `0` (`0x80`) (despite [minimal-encoding requirements preventing this in practice](https://reference.cash/protocol/forks/2019-11-15-minimaldata)). As such, the minimum Script Number which can be represented in 8 bytes is `-9223372036854775807` rather than `-9223372036854775808` (the minimum signed 64-bit integer in C-like programming languages). + +All operations which consume Script Numbers must immediately fail evaluation if an input is received which exceeds the allowed range. (Note: since 2019-11-15, Script Numbers are also required by consensus to be [minimally encoded in most cases](https://reference.cash/protocol/forks/2019-11-15-minimaldata); this rule remains in effect.) + +#### Notice of Possible Future Expansion + +While unusual, it is possible to design contracts which rely on the rejection of otherwise-valid Script Numbers which are larger than 8 bytes. Contract authors are advised that future upgrades may further expand the supported range of BCH VM Script Numbers beyond 8 bytes. + +**This proposal interprets such failure-reliant constructions as intentional** – they are designed to fail unless/until a possible future network upgrade in which larger Script Numbers are enabled (i.e. a contract branch which can only be successfully evaluated in the event of such an upgrade). + +> As always, the security of a contract is the responsibility of the entity locking funds in that contract; funds can always be locked in insecure contracts (e.g. `OP_DROP OP_1`). This notice is provided to warn contract authors and explicitly codify a network policy: the possible existence of poorly-designed contracts will not preclude future upgrades from further expanding the range of Script Numbers. +> +> To ensure a contract will always fail when arithmetic results overflow or underflow 8-byte Script Numbers (in the rare case that such a behavior is desirable), that behavior must be either 1) explicitly validated or 2) introduced to the contract prior to the activation of any future upgrade which expands the range of Script Numbers. + +#### `Script Number` Test Vectors + +
+Valid Script Numbers + +| Hex | Value | +| -------------------- | ------------------------------ | +| `0x` (empty) | 0 | +| `0x01` | 1 | +| `0x02` | 2 | +| `0x03` | 3 | +| `0x7e` | 126 | +| `0x7f` | 127 | +| `0x8000` | 128 | +| `0x8100` | 129 | +| `0x8200` | 130 | +| `0xff00` | 255 | +| `0xfe7f` | 32766 | +| `0xff7f` | 32767 | +| `0x008000` | 32768 | +| `0x018000` | 32769 | +| `0x028000` | 32770 | +| `0xffff00` | 65535 | +| `0xffffff00` | 16777215 | +| `0xfeff7f` | 8388606 | +| `0xffff7f` | 8388607 | +| `0x00008000` | 8388608 | +| `0x01008000` | 8388609 | +| `0x02008000` | 8388610 | +| `0xfeffff7f` | 2147483646 | +| `0xffffff7f` | 2147483647 | +| `0x0000008000` | 2147483648 | +| `0x0100008000` | 2147483649 | +| `0xffffffff7f` | 549755813887 | +| `0x000000008000` | 549755813888 | +| `0xffffffffff7f` | 140737488355327 | +| `0x00000000008000` | 140737488355328 | +| `0xffffffffffff7f` | 36028797018963967 | +| `0x0000000000008000` | 36028797018963968 | +| `0xffffffffffffff7f` | 9223372036854775807 (maximum) | +| `0xffffffffffffffff` | -9223372036854775807 (minimum) | +| `0xfeffffffffffffff` | -9223372036854775806 | +| `0xffffffffffffff` | -36028797018963967 | +| `0xffffffffffff` | -140737488355327 | +| `0xffffffffff` | -549755813887 | +| `0xffffffff` | -2147483647 | +| `0xfeffffff` | -2147483646 | +| `0xfdffffff` | -2147483645 | +| `0xffffff80` | -16777215 | +| `0x01008080` | -8388609 | +| `0x00008080` | -8388608 | +| `0xffffff` | -8388607 | +| `0xfeffff` | -8388606 | +| `0xfdffff` | -8388605 | +| `0xffff80` | -65535 | +| `0x018080` | -32769 | +| `0x008080` | -32768 | +| `0xffff` | -32767 | +| `0xfeff` | -32766 | +| `0xfdff` | -32765 | +| `0xff80` | -255 | +| `0x8180` | -129 | +| `0x8080` | -128 | +| `0xff` | -127 | +| `0xfe` | -126 | +| `0xfd` | -125 | +| `0x82` | -2 | +| `0x81` | -1 | + +
+ +
+Invalid Script Numbers + +| Hex | Error | +| ---------------------- | ---------------------------------------------------------------- | +| `0x000000000000008000` | 9223372036854775808 exceeds the maximum Script Number. | +| `0x000000000000008080` | -9223372036854775808 is less than the minimum Script Number. | +| `0x00` | Non-minimal encoding (for `0x`/`0`) | +| `0x0000` | Non-minimal encoding (for `0x`/`0`) | +| `0x80` | Non-minimal encoding (for `0x`/`0`) | +| `0x0080` | Non-minimal encoding (for `0x`/`0`) | +| `0x0180` | Non-minimal encoding (for `0x81`/`-1`) | +| `0x010080` | Non-minimal encoding (for `0x81`/`-1`) | +| `0x01000080` | Non-minimal encoding (for `0x81`/`-1`) | +| `0x0100000080` | Non-minimal encoding (for `0x81`/`-1`) | +| `0xffffffffffff0080` | Non-minimal encoding (for `0xffffffffffff80`/`-281474976710655`) | + +
+ +### Arithmetic Operation Overflows + +All arithmetic VM operations must use (C-like) signed, 64-bit integer operations with overflow detection (e.g. using the [X86-64 GNU C Integer Overflow Builtins](https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html), `__builtin_ssubll_overflow`, `__builtin_saddll_overflow`, and `__builtin_smulll_overflow`, for `OP_SUB`, `OP_ADD`, and `OP_MUL`, respectively). If an operation overflows or underflows, the operation must immediately fail evaluation. + +Additionally, operations which produce precisely the minimum value (`-9223372036854775808`) – requiring 9 bytes to be encoded as a Script Number – **must [immediately fail evaluation](#disallowance-of-9-byte-script-numbers-from-arithmetic-results)**. (Implementation note: this error can be enforced during Script Number re-encoding.) + +### Re-Enable Multiplication (`OP_MUL`) + +The `OP_MUL` multiplication operation is re-enabled (at `0x95`/`149`, its original codepoint). `OP_MUL` performs C-style, overflow-checked, integer multiplication (e.g. using the [X86-64 GNU C Integer Overflow Builtins](https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html), `__builtin_smulll_overflow`). + +## Rationale + +This section documents design decisions made in this specification. + +### Inclusion of `OP_MUL` + +The `OP_MUL` operation was excluded from the [upgrade restoring disabled opcodes (May 2018)](https://reference.cash/protocol/forks/may-2018-reenabled-opcodes) because a solution for handling overflows was not yet decided; it was expected that `OP_MUL` would be re-enabled during another upgrade which expanded the accepted range of Script Number arithmetic inputs. + +Because this proposal offers a solution for arithmetic underflows and overflows, `OP_MUL` is no longer blocked. Re-activation is included directly in this proposal because the two changes are strongly connected and will benefit from a combined review. + +### Alternative Overflow Behavior + +Until this proposal, overflows have only been prevented indirectly: VM implementations typically employ signed 64-bit integers internally, and because numeric inputs to all operations have been limited to 4-byte Script Numbers, no operations are capable of producing Script Number results larger than 5 bytes. (While inputs are constrained to 4 bytes, 5-byte results are allowed.) With this proposal, overflows would be handled explicitly: they cause an evaluation to immediately fail. + +Alternatively, this proposal could attempt to maintain the previous "undefined" overflow behavior, where overflows aren't explicitly handled by the VM. However, that behavior would require a much less efficient implementation: to support 64-bit multiplication, VM implementations would be required to use at least 128-bit arithmetic internally (while still preventing contracts from using inputs larger than 64 bits). + +> To demonstrate, the maximum 64-bit/8-byte input `0xffffffffffffffff` (`18446744073709551615`), multiplied by itself is `0xfffffffffffffffe0000000000000001` (`340282366920938463426481119284349108225`), which requires 128-bits/16 bytes to represent. + +The overflow handling behavior implemented by this proposal is both more common (among popular programming languages and computing environments) and more efficient than the existing undefined overflow handling strategy. Additionally, this proposal's explicit overflow handling strategy also enables potential future operations (e.g. exponentiation) to be enabled using simple, common implementations. + +### Limiting Arithmetic Operations to 8-byte Script Numbers + +This proposal limits all inputs and outputs of arithmetic operations to the range which can be encoded in 8-byte Script Numbers. This range is nearly equivalent to the range of signed, 64-bit integers ([excluding only one value](#disallowance-of-9-byte-script-numbers-from-arithmetic-results)), and in the positive range (`9223372036854775807` maximum) significantly exceeds the largest possible value of any transaction output: `~2100000000000000` (`21 million BCH * 100 million satoshis`). + +Because signed, 64-bit integer arithmetic is natively implemented in most computing environments, this limit also offers practically-equivalent worst-case performance vs. the existing 4-byte Script Number limitation. (Notably, derivatives of the Satoshi implementation already use 64-bit numbers internally, but enforce the lower 4-byte limit to prevent overflows.) As such, an 8-byte limit significantly expands the functionality of the VM without impacting [worst-case transaction validation costs](https://github.com/bitjson/bch-vm-limits#pre-deployment-hashing-limit-benchmark) or VM implementation complexity. + +A future upgrade which adds support for significant subdivision of satoshi values could create demand for Script Numbers with a greater range than 8 bytes. However, given a maximum possible satoshi supply of `~2100000000000000`, the `9223372036854775807` maximum provides ample room for 1/1000th satoshi subunits before representing even the largest balances might require arithmetic workarounds. And even in these cases, many contracts will be able to either 1) emulate support for larger arithmetic operations using multi-step computations, or 2) operate on rounded values for very large numbers. Given this additional flexibility, the 8-byte limit is likely sufficient until a distant future upgrade. + +> Note: a popular BCH token protocol, [Simple Ledger Protocol (SLP)](https://slp.dev/), technically allows tokens to be created with much greater divisibility than BCH – BCH supports 8 decimal places (satoshis), while SLP tokens can support up to 18. This proposal does not consider these unusual cases to currently warrant a greater arithmetic range: divisibility beyond that of BCH is unlikely to be practically useful in commerce, and if `satoshis` become insufficiently divisible in the distant future, arithmetic range can be increased at the same time as divisibility. +> +> **Users of higher-level protocols like SLP who intend to employ VM arithmetic are advised to target an arithmetic range less than or equal to satoshis for maximum contracting flexibility.** + +### Continued Separation of Cryptographic and Arithmetic Operations + +Past proposals have suggested larger arithmetic limits in an effort to support number sizes useful to cryptosystems. While [deeper analysis](https://bitcoincashresearch.org/t/improve-utility-of-script-calculations-larger-numbers-op-mul/39/15) indicates that larger arithmetic limits are unlikely to be useful in implementing new cryptosystems, such larger limits could negatively impact transaction validation costs. + +In short, BCH VM cryptographic operations [do not operate on numbers](https://bitcoincashresearch.org/t/improve-utility-of-script-calculations-larger-numbers-op-mul/39/18): they are high-level APIs which operate on data structures representing (sets of) numbers. Compatibility between arithmetic and cryptographic operations would be complex and likely introduce performance bottlenecks. + +> Note, limiting arithmetic inputs to 8-byte Script Numbers **does not prevent larger numbers from being represented and used elsewhere in the BCH VM instruction set**. (In fact, larger numbers are already in use within signatures and public keys.) Future proposals could introduce new operations specifically designed to perform mathematical operations on cryptographic data structures (including greater than 8-byte Script Numbers). + +### Inclusion of Future Expansion Notice + +The [Notice of Possible Future Expansion](#notice-of-possible-future-expansion) is included in this specification to avert future controversy by documenting the proposal's intent with respect to future (not-yet specified) upgrades: **the BCH VM is not guaranteed to forever limit Script Numbers to 8 bytes**. + +If this were not clarified, any future Script Number upgrade proposals could be more easily mischaracterize by publicizing deposits made to contracts that are [intentionally designed to rely on the 8-byte overflow behavior](https://bitcoincashresearch.org/t/improve-utility-of-script-calculations-larger-numbers-op-mul/39/12). With this notice, such misdirection might be more easily identified as disingenuous. + +### Disallowance of 9-byte `Script Numbers` from Arithmetic Results + +Only one 9-byte Script Number can be represented within the signed 64-bit integer range to be used by VM arithmetic operations after activation of this proposal: `-9223372036854775808`. This precise value is disallowed (by limiting all Script Number arithmetic inputs and outputs to 8 bytes) to simplify both VM implementation and contract security analysis. + +If this 9-byte value were allowed in arithmetic results, it would break the assumption that all valid arithmetic results are also valid arithmetic inputs. In some covenants, this could present a subtle exploit: if an attacker can force the contract to somehow retain this precise 9-byte result, the attacker could place the covenant in an unintended state, preventing the 9-byte result from being successfully passed into other arithmetic operations. Furthermore, analyzing contracts for this vulnerability requires detailed information about the possible numeric ranges of arithmetic inputs and outputs, creating an unnecessary burden for static contract analysis. + +For example, if the 9-byte value were allowed, the script `<-9223372036854775807> OP_1SUB OP_1ADD` would successfully produce the 9-byte value after `OP_1SUB`, but the resulting 9-byte Script Number would be rejected by `OP_1ADD`. Implementations could add a special case for handling this particular signed 64-bit integer, 9-byte Script Number, but the corresponding positive number (`9223372036854775808`) is also not representable as a signed 64-bit integer (in most computing environments), so an operation like `<-9223372036854775808> OP_NEGATE` would also overflow. + +## Implementations + +_(in progress)_ + +### Test Cases + +_(in progress)_ + +## Evaluation of Alternatives + +Alternative designs for several components of this proposal have been documented in the [Rationale](#rationale) section, and several past proposals have also informed the design of this proposal: + +### 128-bit Integers + +An [earlier proposal for 128-bit integers](https://gist.github.com/rkalis/eabdbba283f3807c0e38bd677672b6ae) would also enable up to 128-bit arithmetic operations. The larger 128-bit range [may impact worst-case validation performance](https://gist.github.com/bitjson/78967c2affddaf3b7d1ab4dee71a940f#prototype--benchmark-dos-viability), and implementation is likely to be more complex in many computing environments. + +While future proposals could further expand the range of VM arithmetic operations, [64-bit math is likely sufficient even for operation on 1/1000th "fractional satoshis"](#limiting-arithmetic-operations-to-8-byte-script-numbers), so further expansion requires additional research. + +### NextChain BigNum + +[NextChain BigNum](http://nextchain.cash/bignum) would enable up to 4096-bit integer arithmetic, add the `OP_SETBMD`, `OP_BIN2BIGNUM`, and `OP_BIGNUM2BIN` operations, and introduce "type" information to all stack items (with a new `BigNum` type). + +Implementation of NextChain BigNum is notably more complex than other proposals, and it is unclear whether support for larger arithmetic inputs would have practical applications (see [Continued Separation of Cryptographic and Arithmetic Operations](#continued-separation-of-cryptographic-and-arithmetic-operations)). + +## Primary Stakeholders + +At least five primary stakeholder groups exist: + +### Node Developers + +At least six node implementations must be upgraded: + +- [Bitcoin Cash Node](https://bitcoincashnode.org/) +- [Bitcoin Unlimited](https://www.bitcoinunlimited.info/) +- [BCHD](https://bchd.cash/) +- [Flowee](https://flowee.org/) +- [Bitcoin Verde](https://bitcoinverde.org/) +- [Knuth](https://kth.cash/) + +### Library Developers + +At least five libraries must be upgraded: + +- [BitPay](https://bitpay.com/) developed the [bitcore-lib-cash](https://github.com/bitpay/bitcore/tree/master/packages/bitcore-lib-cash) library, which supports Script execution. +- [Jason Dreyzehner](https://github.com/bitjson) developed the [libauth](https://github.com/bitauth/libauth) library, which supports Script execution. +- [Pokkst](https://github.com/pokkst) developed the [bitcoincashj](https://github.com/pokkst/bitcoincashj) library, which supports Script execution. +- [Dagur Valberg Johansson](https://github.com/dagurval/) developed the [python-bitcoincash](https://github.com/dagurval/python-bitcoincash) library, which supports Script execution. +- [Tobias Ruck](https://github.com/eyeofpython) developed the [Iguana](https://github.com/be-cash/iguana) library, which supports Script execution. + +### Contract Developers + +Contract developers affected by existing limits include: + +- [General Protocols](https://generalprotocols.com/) created [AnyHedge](https://anyhedge.com/), a volatility risk-trading contract. AnyHedge contracts are limited to ~$15k and have a slight math error due to workarounds. +- [Licho](https://github.com/KarolTrzeszczkowski) created a [Last Will](https://github.com/KarolTrzeszczkowski/Electron-Cash-Last-Will-Plugin) contract to manage inheritance and the [Mecenas](https://github.com/KarolTrzeszczkowski/Mecenas-recurring-payment-EC-plugin) contract for recurring payments. These contracts are limited to ~21 BCH (~$10k). +- [Shomari Prince](https://github.com/nyusternie) created [Causes Cash](https://causes.cash/), which includes a modified Mecenas to support recurring payments in USD. These contracts are limited to ~21 BCH (~$10k). +- [James Cramer](https://github.com/jcramer/) created experimental [SLP Mint Guard](https://github.com/simpleledger/Electron-Cash-SLP/blob/cashscript-dev/lib/cashscript/slp_mint_guard.cash) contracts and [tokens with minting schedules](https://github.com/simpleledgerinc/slp-mint-contracts). These contracts are currently not limited, but to execute the proposed roadmap they will be very limited as they will need to perform arithmetic on SLP token amounts (which can have more decimals and lower USD values than BCH). +- [p0oker](https://github.com/p0o/) created an [SLP vending contract](https://github.com/p0o/yield-farming-bch-smart-contract) that mints tokens on-demand and is building a BCH staking contract that mints tokens over time and an SLP exchange contract to sell NFTs. Similar to James Cramer's contracts, these contracts will be very limited as they need to work with SLP amounts. +- [Jason Dreyzehner](https://github.com/bitjson) created [CashChannels](https://blog.bitjson.com/cashchannels-recurring-payments-for-bitcoin-cash-3b274fbfa6e2), recurring payments for Bitcoin Cash. These channels are limited to ~21 BCH (~$10k). + +### Node Operators & Miners + +_(in progress)_ + +### Investors + +These individuals and organizations have invested in the BCH currency and ecosystem on the premise that it can become peer to peer electronic cash for the world. These stakeholders expect the token to serve effectively as money, including in the innovative financial services which could be enabled by expanded arithmetic support. + +## Statements + +### Node Developers + +_(in progress)_ + +### Library Developers + +_(in progress)_ + +### Contract Developers + +_(in progress)_ + +#### General Protocols + +Developing workarounds to this limitation has cost General Protocols a large amount of time and money. This is still ongoing as the added complexity makes further smart contract changes more difficult. Because of the required code to address this, there are also other contract features that do not fit within the contract bytecode size limits. + +### Node Operators & Miners + +_(in progress)_ + +### Investors + +_(in progress)_ + +## Changelog + +This section summarizes the evolution of this document. + +- **v1.0 – 2021-6-9** (current) + - Completed technical specification + - Added `Rationale` section, revised supporting material (`Summary`, `Deployment`, `Motivation`, `Benefits`, `Evaluation of Alternatives`, etc.) +- **v0 – 2021-2-21** ([`32e9d5ed`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/32e9d5ed0ebffd13b351f26d4c4f9370598c9933/CHIP-2021-02-Bigger-Script-Integers.md)) + - Initial draft + +## Copyright Notice + +Copyright (c) 2021 GeneralProtocols / Research + +Permission is granted to copy, distribute and/or modify this document under the terms of the [MIT license](https://gitlab.com/GeneralProtocols/research/chips/-/blob/master/LICENSE). From 312386ebdd4cf196ee658a01c9a8289fb3ed917e Mon Sep 17 00:00:00 2001 From: Jonathan Silverblood Date: Mon, 27 Dec 2021 14:56:19 +0200 Subject: [PATCH 05/22] Create 2022-native-introspection-opcodes.md --- .../2022-native-introspection-opcodes.md | 386 ++++++++++++++++++ 1 file changed, 386 insertions(+) create mode 100644 protocol/forks/chips/2022-native-introspection-opcodes.md diff --git a/protocol/forks/chips/2022-native-introspection-opcodes.md b/protocol/forks/chips/2022-native-introspection-opcodes.md new file mode 100644 index 0000000..4c72614 --- /dev/null +++ b/protocol/forks/chips/2022-native-introspection-opcodes.md @@ -0,0 +1,386 @@ +# CHIP-2021-02: Native Introspection Opcodes + +> OWNERS: [Jason Dreyzehner](https://bitjson.com/), [Jonathan Silverblood](https://gitlab.com/monsterbitar) +> +> DISCUSSION: [Bitcoin Cash Research](https://bitcoincashresearch.org/t/native-introspection-chip-discussion/307), [Telegram](https://t.me/transactionintrospection) +> +> VERSION: 1.1.4 +> +> MILESTONES: **[Published](https://gitlab.com/GeneralProtocols/research/-/blob/master/CHIPs/May%202022,%20Native%20Introspection.md)**, **[Specification](#technical-specification)**, **[Testnet](https://read.cash/@bitcoincashnode/bch-testnet4-for-may-2022-network-upgrade-updated-2021-11-23-95a21d51)**, **Accepted ([BU](https://twitter.com/BitcoinUnlimit/status/1460394221240786956) - [BCHN](https://twitter.com/bitcoincashnode/status/1460424339816284161) - [Knuth](https://twitter.com/KnuthNode/status/1460622308607938579) - [Verde](https://twitter.com/joshmgreen/status/1460592944516317185) - [Bitauth](https://twitter.com/Bitauth/status/1460371045207232512))**, Deployed (May 15th, 2022). + +## Summary + +This proposal adds a set of new virtual machine (VM) operations which enable BCH contracts to efficiently access details about the current transaction like output values, recipients, and more – without increasing transaction validation costs. + +## Deployment + +Deployment of this specification is proposed for the May 2022 upgrade. + +This proposal does not depend on [`CHIP: Bigger Script Integers`](./CHIP-2021-02-Bigger-Script-Integers.md), but support for larger script numbers is required for all possible results of `OP_UTXOVALUE` and `OP_OUTPUTVALUE` to be used in arithmetic operations. It is therefore recommended that both CHIPs be deployed together. + +This proposal does not depend on [`CHIP: Version 3 Transaction Format (PMv3)`](https://github.com/bitjson/pmv3), but it includes an [integration specification](#chip-integration-version-3-transaction-format-pmv3) to add introspection support for the v3 transaction format. See also: [Interaction with PMv3](#interaction-with-pmv3). + +## Motivation + +Since the introduction of OP_CHECKDATASIG (2018-11-15), it has been possible to implement practical Bitcoin Cash **covenants – contracts which validate properties of the transaction in which the contract funds are spent**. + +Covenants enable a wide range of new innovation, but the strategy by which they are currently implemented is extremely inefficient: most of a transactions contents must be duplicated for each input which utilizes a covenant. In practice, this doubles or triples the size of even the simplest covenant transactions1, and advanced covenant applications quickly exceed VM limits such as the maximum standard unlocking bytecode length (A.K.A. `MAX_TX_IN_SCRIPT_SIG_SIZE`) of 1,650 bytes. This severely limits the extent of current BCH covenant development and usage. + +> Covenants are currently implemented using a workaround in which the same signature is checked by both `OP_CHECKSIG` and `OP_CHECKDATASIG`. The `OP_CHECKSIG` confirms the signature is valid for the current transaction, and the `OP_CHECKDATASIG` allows the contract to validate that the signature covers a particular preimage provided in the unlocking bytecode. If both checks are successful, the contract can trust that the preimage provided to `OP_CHECKDATASIG` is the genuine signing serialization of the current transaction. By further inspecting this preimage, the contract can validate other properties of the transaction. + +Because nodes are already required to have full access to a transaction's contents and UTXO information during transaction validation, native introspection opcodes can be introduced without impacting existing transaction validation costs. + +
+Calculations + +1. An optimized implementation of the `OP_CHECKSIG` + `OP_CHECKDATASIG` workaround (` OP_3DUP OP_SHA256 OP_ROT OP_CHECKDATASIGVERIFY OP_DROP OP_CAT OP_SWAP OP_CHECKSIGVERIFY`) requires at least 12 bytes of operations, a 33-byte public key, a 65 to 73-byte signature, and a [signing serialization preimage which is typically at least 188 bytes](https://github.com/bitcoincashorg/bitcoincash.org/blob/3e2e6da8c38dab7ba12149d327bc4b259aaad684/spec/replay-protected-sighash.md#digest-algorithm). In total, this construction is expected to require at least 298 bytes per covenant input. For some contracts, the preimage can be partially generated using `SIGHASH_ANYONECANPAY` and `OP_NUM2BIN`, theoretically reducing the preimage size to ~90 bytes and bringing the minimum possible overhead to ~200 bytes. + +
+ +## Benefits + +Adding native support for introspection operations would make covenants simpler, safer, and more efficient, enabling innovative new use cases. + +### Simpler, Safer Contracts + +While contracts using the current covenant workaround must carefully validate all provided transaction information, contracts using native introspection operations could safely rely on information returned by the VM without further validation. This significantly reduces contract size and complexity, making contract development and security review easier. + +### Reduced Transaction Sizes + +By eliminating the need for a wasteful double-signature checking workaround, covenant transaction sizes can be reduced by hundreds of bytes, making them as efficient as other common transaction types. This would reduce wasted network bandwidth and lower transaction fees for all covenant applications. + +Because covenant applications cover a large variety of common transaction types – including scheduled and recurring payments – improved efficiency in covenant transactions would significantly improve overall throughput of the BCH network. + +### Enable Innovation + +By eliminating waste in covenant contracts, native introspection operations can extend the scope of possible applications which can be developed within current VM limits. This would enable far more innovation on Bitcoin Cash, without increasing node validation costs. + +## Costs & Risk Mitigation + +The following costs and risks have been assessed. + +### Node Upgrade Costs + +This proposal affects consensus – all fully-validating node software must implement these VM changes to remain in consensus. + +> These VM changes are backwards-compatible: all past and currently-possible transactions remain valid under these new rules. + +### Other Software Upgrade Costs + +While most wallets, indexers, and other software will not require updates to support this proposal (beyond upgrades to any supporting node infrastructure), updates will be required to some libraries, transaction compilers, and other software which implements the BCH VM (e.g. [Bitauth IDE](https://github.com/bitauth/bitauth-ide)). + +### Assignment of Opcodes + +The specified operations would occupy 15 codepoints within the BCH VM instruction set. This represents a notable portion of the remaining 81 unused codepoints (80, 98, 101, 102, 137, 138, 176, 179-185, 189-255). However, we consider this to be a [prudent use of these codepoints](#use-of-single-byte-opcodes). + +## Technical Specification + +A range of Bitcoin Cash VM codepoints is reserved for transaction introspection operations – `0xc0` (192) through `0xcf` (207) – and a new set of 15 operations allows contracts to introspect all available transaction and evaluation state. + +### Nullary Operations + +The following 6 operations consume no items and push a single result to the stack. If the item to be pushed to the stack is longer than the stack item length limit (A.K.A. `MAX_SCRIPT_ELEMENT_SIZE`), immediately fail evaluation. If the additional stack item would cause the stack to exceed the maximum stack size (A.K.A. `MAX_STACK_SIZE`), immediately fail evaluation. + +| Operation | Codepoint | Description | +| ----------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| OP_INPUTINDEX | `0xc0` (192) | Push the index of the input being evaluated to the stack as a Script Number. | +| OP_ACTIVEBYTECODE | `0xc1` (193) | Push the bytecode currently being evaluated, beginning after the last executed `OP_CODESEPARATOR`, to the stack1. For Pay-to-Script-Hash (P2SH) evaluations, this is the redeem bytecode of the Unspent Transaction Output (UTXO) being spent; for all other evaluations, this is the locking bytecode of the UTXO being spent.2 | +| OP_TXVERSION | `0xc2` (194) | Push the version of the current transaction to the stack as a Script Number. | +| OP_TXINPUTCOUNT | `0xc3` (195) | Push the count of inputs in the current transaction to the stack as a Script Number. | +| OP_TXOUTPUTCOUNT | `0xc4` (196) | Push the count of outputs in the current transaction to the stack as a Script Number. | +| OP_TXLOCKTIME | `0xc5` (197) | Push the locktime of the current transaction to the stack as a Script Number. | + +1. `OP_ACTIVEBYTECODE` pushes the serialized bytecode for the instructions currently under evaluation beginning after the most recently executed `OP_CODESEPARATOR` and continuing through the final instruction. If no `OP_CODESEPARATOR` has been executed, `OP_ACTIVEBYTECODE` pushes the full, serialized bytecode for the instructions currently under evaluation. (In the Satoshi implementation, this is simply the `CScript` contents passed to the current `EvalScript`.) +2. This behavior matches the existing behavior of the BCH VM during P2SH evaluation – once the P2SH pattern is matched, the remaining stack is copied, and the VM begins evaluation again with the P2SH redeem bytecode set as the new `active bytecode`. (In the Satoshi implementation, this P2SH redeem bytecode is passed as a `CScript` to a new execution of `EvalScript`.) + +### Unary Operations + +The following 9 operations pop the top item from the stack as an index (Script Number) and push a single result to the stack. If the consumed value is not a valid, minimally-encoded index for the operation, an error is produced. If the item to be pushed to the stack is longer than the stack item length limit (A.K.A. `MAX_SCRIPT_ELEMENT_SIZE`), immediately fail evaluation. + +| Operation | Codepoint | Description | +| ---------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| OP_UTXOVALUE | `0xc6` (198) | Pop the top item from the stack as an input index (Script Number). Push the value (in satoshis) of the Unspent Transaction Output (UTXO) spent by that input to the stack as a Script Number. | +| OP_UTXOBYTECODE | `0xc7` (199) | Pop the top item from the stack as an input index (Script Number). Push the full locking bytecode of the Unspent Transaction Output (UTXO) spent by that input to the stack. | +| OP_OUTPOINTTXHASH | `0xc8` (200) | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint transaction hash – the hash of the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack in `OP_HASH256` byte order1. | +| OP_OUTPOINTINDEX | `0xc9` (201) | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint index – the index of the output in the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack as a Script Number. | +| OP_INPUTBYTECODE | `0xca` (202) | Pop the top item from the stack as an input index (Script Number). Push the unlocking bytecode of the input at that index to the stack. | +| OP_INPUTSEQUENCENUMBER | `0xcb` (203) | Pop the top item from the stack as an input index (Script Number). Push the sequence number of the input at that index to the stack as a Script Number. | +| OP_OUTPUTVALUE | `0xcc` (204) | Pop the top item from the stack as an output index (Script Number). Push the value (in satoshis) of the output at that index to the stack as a Script Number. | +| OP_OUTPUTBYTECODE | `0xcd` (205) | Pop the top item from the stack as an output index (Script Number). Push the locking bytecode of the output at that index to the stack. | + +1. This is the byte order produced/required by all BCH VM operations which employ SHA-256 (including `OP_SHA256` and `OP_HASH256`), the byte order used for outpoint transaction hashes in the P2P transaction format, and the byte order produced by most SHA-256 libraries. For reference, the genesis block header in this byte order is little-endian – `6fe28c0ab6f1b372c1a6a246ae63f74f931e8365e15a089c68d6190000000000` – and can be produced by this script: `<0x0100000000000000000000000000000000000000000000000000000000000000000000003ba3edfd7a7b12b27ac72c3e67768f617fc81bc3888a51323a9fb8aa4b1e5e4a29ab5f49ffff001d1dac2b7c> OP_HASH256`. (Note, this is the opposite byte order as is commonly used in user interfaces like block explorers.) + +--- + +### CHIP Integration: Bigger Script Integers + +Because VM arithmetic operations currently limit `Script Number` inputs to signed 32-bit integers, the maximum results of `OP_UTXOVALUE` and `OP_OUTPUTVALUE` which can be used in practical contracts is `2147483647` satoshis (`~21.47` BCH). For this reason, **it is recommended that [`CHIP: Bigger Script Integers`](./CHIP-2021-02-Bigger-Script-Integers.md) be deployed before or with this CHIP**. + +This CHIP can be deployed without `CHIP: Bigger Script Integers`, and no direct integration is required between the two CHIPs. If `CHIP: Bigger Script Integers` is not deployed with or before this proposal, the above specified operations must still behave as specified. (`OP_UTXOVALUE` and `OP_OUTPUTVALUE` may push Script Numbers to the stack which are up to 8 bytes in length and therefore could not be used as inputs to arithmetic operations. Until some arithmetic range upgrade, these operations would likely only be used in practice for values which can be encoded in 4 bytes or less.) + +### CHIP Integration: Version 3 Transaction Format (PMv3) + +If this proposal is adopted with or after [`CHIP: Version 3 Transaction Format (PMv3)`](https://github.com/bitjson/pmv3), the `OP_INPUTDETACHED` [unary introspection operation](#unary-operations) must also be activated: + +| Operation | Codepoint | Description | +| ---------------------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| OP_INPUTDETACHED3 | `0xce` (206) | Pop the top item from the stack as an input index (Script Number). If the input at that index uses a detached proof, push `1` (`0x01`), otherwise push `0` (`0x`, the empty item). | + +> See [Interaction with PMv3](#interaction-with-pmv3) for rationale. + +Additionally, the description of `OP_INPUTBYTECODE` can be clarified to confirm that its result is always the input's unlocking bytecode (regardless of whether or not the proof is detached): + +| Operation | Codepoint | Description | +| ---------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| OP_INPUTBYTECODE | `0xca` (202) | Pop the top item from the stack as an input index (Script Number). Push the unlocking bytecode of the input at that index to the stack. (If the selected input uses a detached proof, the pushed value is the unlocking bytecode content of the detached proof.) | + +## Rationale + +This section documents design decisions made in this specification. + +### Ordering Operations by Arity + +This proposal groups new introspection operations by [arity](https://en.wikipedia.org/wiki/Arity): all operations which consume no stack values precede all operations which consume an index value from the stack. + +This simplifies some implementations, allowing operations of like arities to more easily reuse logic. For example, in the Satoshi implementation, all unary numeric operations are [handled in the same fallthrough switch case](https://github.com/bitcoin-cash-node/bitcoin-cash-node/blob/8d6a597bb4d12824b2eb9a8a42708979a5395099/src/script/interpreter.cpp#L716-L727), allowing each operation to reuse the same stack pop and error handling logic. + +Beyond arity, operations are arranged within the instruction set to match the order in which their results appear within the P2P transaction format. + +### Operation Set Completeness + +The operations specified in this proposal expose the most raw form of every static property of transaction and evaluation state within the current VM model. + +This strategy necessarily incorporates operations which may initially seem to duplicate existing functionality, notably `OP_TXLOCKTIME` and `OP_INPUTSEQUENCENUMBER` (many use cases for these operations can also be accomplished using the existing `OP_CHECKLOCKTIMEVERIFY` and `OP_CHECKSEQUENCEVERIFY`, respectively). However, contract authors are not always able to reverse the order of a construction to make use of these operations. This limitation is particularly acute in covenant use cases: where other limitations prevent a contract from inverting a construction (or even providing the expected result of that operation via unlocking bytecode), some use cases are likely impossible without these functionally-similar introspection operations. + +> Notably, the existing instruction set includes `OP_SUB` even though `OP_ADD` can be indirectly used (backwards) to accomplish many of the same validations. The existence of inverse and functionally-similar operations is important for both accommodating new use cases and optimizing contracts. + +By implementing introspection in a single deployment as a complete, logical set of operations – rather than by piecemeal activation as contract authors lobby for operations which are important for their use cases – we allow new innovation to take place primarily in "userspace" and limit future technical debt. + +> Note: introspection of dynamic VM state – e.g. current instruction pointer index, the current state of various cumulative limits, etc. – is intentionally excluded from this specification. +> +> This approach avoids consensus standardization of various implementation details which could complicate some implementations and requires further research and justification. Instead, this proposal incorporates only introspection operations for previously consensus-standardized, static properties of transaction and evaluation state which are already available to contracts via the `OP_CHECKSIG` + `OP_CHECKDATASIG` hack. + +### Differences Between `OP_ACTIVEBYTECODE` and `OP_INPUTINDEX OP_UTXOBYTECODE` + +The operations specified in this proposal include partially-overlapping functionality in one rare case: the results of `OP_ACTIVEBYTECODE` and `OP_INPUTINDEX OP_UTXOBYTECODE` are equal in non-P2SH contracts (i.e. spending the output of a non-standard transaction). However, in the more common (standard) P2SH case, `OP_ACTIVEBYTECODE` produces the raw redeem bytecode, while `OP_INPUTINDEX OP_UTXOBYTECODE` will produce only the already-hashed result in the P2SH-template locking bytecode (`OP_HASH160 OP_PUSHBYTES_20 OP_EQUAL`). This distinction is very important in covenant applications: even in simple cases, it can save many hundreds of bytes by avoiding the need to push a duplicate copy of the covenant's redeem bytecode (to construct the covenant's next output). While `OP_ACTIVEBYTECODE` is very cheap for the VM (typically a simple buffer copy), this single optimization reduces many covenant sizes by 50% or more. + +> To demonstrate, the following script produces a `1` for P2SH contracts, but a `0` for non-P2SH contracts (spending a non-standard transaction): ` OP_ACTIVEBYTECODE OP_HASH160 OP_CAT OP_CAT OP_INPUTINDEX OP_UTXOBYTECODE OP_EQUAL`. + +Alternatively, this same functionality could be provided by modifying `OP_UTXOBYTECODE` to only return the "active" bytecode of each UTXO (i.e. for P2SH contracts, only the raw redeem bytecode), but this would require a significantly more complex implementation: a transaction's inputs would need to be evaluated in two phases, caching each P2SH input's redeem bytecode for access by other inputs where needed. Additionally, information would still be lost unless another introspection operation were made available for contracts to judge which inputs spend a P2SH UTXO (e.g. a unary `OP_ISP2SH`). + +An advantage of that more complex solution over the proposed `OP_ACTIVEBYTECODE` solution could be more efficient access to the raw redeem bytecode of sibling inputs (other than the "active" one). While this might be useful in optimizing some highly-advanced covenants, the proposed solution offers a far simpler consensus implementation and slightly more efficient contracts in the common case (1-byte `OP_ACTIVEBYTECODE` vs. 2-byte `OP_INPUTINDEX OP_UTXOBYTECODE`). + +> The complexity of this issue ultimately stems from the implementation chosen for [BIP16/P2SH](https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki), which bifurcated VM evaluation into two modes: standard and P2SH. This complicated the operational model of the VM and is particularly disruptive to the logical consistency of introspection operations between the two modes. The chosen solution focuses on simplicity and optimizing the most common cases, rather than a more complex solution which might optimize more cases. If usage of sibling redeem bytecode becomes common in real world applications, future upgrades could offer more targeted optimizations for those cases. + +### Introspection Operation Range + +This proposal leaves a 3 codepoint gap between the last defined operation in the current BCH instruction set (`OP_REVERSEBYTES`, `0xbc`/`188`) and the beginning of the proposed introspection operation range (`OP_INPUTINDEX`, `0xc0`/`192`). + +This breaks from the recent precedent (since Nov. 2018) of adding each new opcode at the next-available codepoint. (After `OP_NOP10`/`0xb9`, `OP_CHECKDATASIG`/`0xba`, `OP_CHECKDATASIGVERIFY`/`0xbb`, and `OP_REVERSEBYTES`/`0xbc` were appended to the ISA in activation order, rather than being grouped logically with similar operations). While this may appear to complicate logic for detecting invalid codepoints (from `op > FIRST_UNDEFINED_OP_VALUE` to `isUndefined(op)`), it is equivalent in practice, as validators are optimized for evaluating valid bytecode (the common case). For example, in the Satoshi implementation, a switch statement is used to switch between valid operations – invalid opcodes only return their error after the VM iterates through the full list of possible valid opcodes. + +By leaving gaps in the ISA, future upgrades may group similar operations nearer to each other. **In the long term, this may produce a more coherent ISA than one ordered only by opcode activation time.** + +In the context of this proposal, the proposed gap offers up to 3 codepoints for use either by formatting operations (logically grouped with `OP_REVERSEBYTES`) or for use by additional nullary introspection operations (if the VM model is extended to incorporate new transaction-level primitives or metadata). Likewise, the 2 remaining codepoints within the introspection range (`0xce` and `0xcf`) provide space for additional unary introspection operations. + +### Exclusion of Aggregation Operations + +With [the partial exception of `OP_ACTIVEBYTECODE`](#differences-between-op_activebytecode-and-op_inputindex-op_utxobytecode), this proposal includes only operations which expose raw state – derived state must be computed using other VM operations. While this simplifies the proposal to a clearly defined set of "primitive" operations, certain useful, derived state elements are not yet possible to compute in all cases. For example, since the VM does not include any construction for looping or iteration, aggregate values like "total input value" and "total output value" are not always possible for contracts to compute. + +While it would be possible to define aggregate operations like `OP_TXINPUTVALUE` and `OP_TXOUTPUTVALUE` to make this derived state accessible to all contracts, this proposal considers standardization of any derived state to be premature. Even with piecemeal implementation of some derived state operations, some important types of aggregated/derived state cannot be reasonably standardized, e.g. "public keys referenced by all P2PKH outputs". For contracts to support extracting this information, it's necessary to add support for generalized loops/iteration to the VM. + +For this reason, this proposal does not include any aggregate operations; contract authors should instead look to proposals like [`CHIP: Bounded Looping Operations`](https://github.com/bitjson/bch-loops) for this support. + +> With [`CHIP: Bounded Looping Operations`](https://github.com/bitjson/bch-loops), a `OP_TXINPUTVALUE` equivalent would require only 14 bytes: +> +> ``` +> <0> <0> +> OP_BEGIN // loop (re)starts with index, value +> OP_OVER OP_UTXOVALUE OP_ADD // Add UTXO value to total +> OP_SWAP OP_1ADD OP_SWAP // increment index +> OP_OVER OP_TXINPUTCOUNT OP_EQUAL +> OP_UNTIL // loop until next index would exceed final index +> OP_NIP // drop index, leaving sum of UTXO values +> ``` + +While not included in this proposal, future proposals may introduce aggregate operations either as a partial alternative to looping operations or as an optimization strategy for common aggregations. + +### Interaction with PMv3 + +The [PMv3 version 3 transaction format proposal](https://github.com/bitjson/pmv3) would add two fields which don't yet exist in the v1/v2 transaction formats: [`detached signatures`](https://github.com/bitjson/pmv3#detached-signatures) and [`detached proofs`](https://github.com/bitjson/pmv3#detached-proofs). If adopted with PMv3, this proposal recommends only one additional introspection operation (`OP_INPUTDETACHED`). + +Detached signatures are themselves inherently reusable across multiple inputs – rather than a specialized introspection operation, many use cases can reference a particular detached signature directly. Also, because all detached signature-covered transaction elements are themselves natively introspectable, there's little value in an operation which makes the `OP_CHECKSIG` + `OP_CHECKDATASIG` workaround more efficient for detached signatures. As such, **this proposal does not recommend any new introspection operation(s) for detached signatures**. + +Detached proofs add only one new bit of information: whether or not each input is detached (`true` or `false`). Access to this information is valuable to many public covenants; if a covenant requires proofs to be detached for transaction sizes to be kept in check, the covenant needs to prevent malicious users from interacting with the covenant using transactions with non-detached proofs (leading to increased covenant transaction costs and even disabling of poorly-designed public covenants). + +**To allow inspection of this information, the `OP_INPUTDETACHED` unary operation is proposed**, and the `OP_INPUTBYTECODE` operation is clarified: `(If the selected input uses a detached proof, the pushed value is the unlocking bytecode content of the detached proof.)`. + +This strategy makes validation of transaction shapes trivial: `OP_INPUTINDEX OP_INPUTDETACHED OP_VERIFY` would ensure that the current contract is using a detached proof. It also maintains a consistent behavior for `OP_INPUTBYTECODE` (for both detached and standard proofs) – the pushed results of `OP_INPUTBYTECODE` would always be the unlocking bytecode regardless of whether or not the proof is detached. + +> Note, the un-hashed result of `OP_INPUTBYTECODE` is most useful to practical contracts, but it's also trivial to reproduce any `detached proof hash` under this scheme: ` OP_INPUTBYTECODE OP_HASH256`. + +
+No-operation alternative + +Alternatively, it could be possible to avoid introducing a new `OP_INPUTDETACHED` operation at the cost of significantly increased contract complexity. Under this scheme, the `OP_INPUTBYTECODE` operation could instead be modified to push the `detached proof hash` for inputs which use detached proofs. + +While this configuration would technically expose all transaction information, verifying the detached-ness of any particular input would not be trivial, and reviewing contracts for security would become more difficult. + +In some cases, the size of the unlocking bytecode could simply be validated: `OP_INPUTBYTECODE OP_SIZE <32> OP_EQUAL`. If it can be guaranteed that no raw unlocking bytecode values of exactly 32-bytes exist for a particular input, this may be sufficient to guarantee that said input is detached. Otherwise, the contract will also be required to validate that a double SHA-256 preimage exists which equals the value returned by `OP_INPUTBYTECODE`. This comprehensively-secure solution would require duplicating the contents of the unlocking bytecode for each validated input, significantly increasing transaction sizes. + +Finally, this solution would break the logical consistency of `OP_INPUTBYTECODE` – contracts which depend on the ability to introspect unlocking bytecode would require careful logic to identify whether or not the value returned by `OP_INPUTBYTECODE` is actually unlocking bytecode rather than the hash of that input's unlocking bytecode. In some cases, this may open contracts to vulnerabilities in which carefully "mined" detached proof hashes could deceive the contract into believing a poorly-validated detached proof hash is raw unlocking bytecode. + +While this alternative strategy would save a codepoint in the BCH instruction set, it would significantly increase contract complexity, security review costs, and transaction costs. Therefore, this proposal recommends implementation of the simpler `OP_INPUTDETACHED` operation. + +
+ +### Use of Single-Byte Opcodes + +Earlier proposals attempted to use fewer opcodes by either [introducing all introspection operations as a single opcode](https://github.com/bitjson/op-pushstate) or by [introducing multi-byte opcodes](https://github.com/bitjson/op-pushstate/pull/1). However, [deeper analysis](https://bitcoincashresearch.org/t/bitcoin-script-multi-byte-opcodes/347/5) indicates that the BCH instruction set [might never reach 255 operations](https://bitcoincashresearch.org/t/bitcoin-script-multi-byte-opcodes/347/9) within the current VM model (future VM models are likely to have their own independent set of operations, rather than naively extending the current instruction set). + +While it is likely wise to reserve both the remaining `OP_NOP4`-`OP_NOP10` range and the range from `0xf0` to `0xff` for future multi-byte opcodes (leaving room for ~6000 2-byte opcodes), jumping to multi-byte opcodes would be a premature optimization, with uncertain future benefit and immediate negative consequences. + +As such – and because introspection operations are critical to all covenant use cases – this proposal specifies the proposed operations as standard, single-byte opcodes. + +### Push-Only Limitation of Unlocking Bytecode + +For clarity, this proposal does not modify the existing limitation (present since [BCH_2018-11-15](https://documentation.cash/protocol/forks/hf-20181115#push-only)) preventing transactions from including "non-push" operations – all operations at codepoints greater than 96 (`0x60`) – in unlocking bytecode. This limitation also prevents all operations specified in this proposal from appearing in unlocking bytecode. + +This limitation is maintained to prevent a third-party malleability vector: any non-push operation can be replaced in a malleated transaction with an equivalent push operation pushing the computed result of the non-push operation. While all third-party malleability vectors would be eliminated by [PMv3's detached signatures](hhttps://github.com/bitjson/pmv3#comprehensive-malleability-protection), this CHIP leaves any loosening of the non-push limitation to future proposals. + +### `OP_ACTIVEBYTECODE` Support for `OP_CODESEPARATOR` + +In earlier versions of this proposal, the result of `OP_ACTIVEBYTECODE` did not account for the last executed `OP_CODESEPARATOR`. However, the current behavior is considered more conservative (`OP_ACTIVEBYTECODE` returns the active bytecode beginning after the last evaluated `OP_CODESEPARATOR`). Contracts which don't utilize `OP_CODESEPARATOR` are unaffected by this change, contracts which utilize `OP_CODESEPARATOR` can access either behavior (an earlier result of `OP_ACTIVEBYTECODE` can always be saved for use after `OP_CODESEPARATOR` is called), and some unusual contracts may be able to utilize the new behavior to optimize their bytecode size or runtime requirements. + +Notably, the index of the last executed `OP_CODESEPARATOR` is available using the [`OP_CHECKSIG` + `OP_CHECKDATASIG` workaround](#motivation), so providing access to this state via `OP_ACTIVEBYTECODE` is also valuable for [operation set completeness](#operation-set-completeness). + +## Implementations + +- [Bitcoin Cash Node (BCHN) Merge Request 1208](https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node/-/merge_requests/1208) + +### Test Cases + +- [Bitcoin Cash Node (BCHN) Native Introspection Tests](https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node/-/blob/6fff8c761fda0ad15c9752d02db31aa65d58170f/src/test/native_introspection_tests.cpp) + +## Evaluation of Alternatives + +No alternative native introspection proposals are currently public, but alternative designs for several components of this proposal have been documented in the [Rationale](#rationale) section. + +Several past proposals have informed the design of this proposal: + +- The [initial `OP_PUSHSTATE`](https://github.com/bitjson/op-pushstate) proposal used a single opcode and templated approach. (The proposal [was withdrawn](https://bitcoincashresearch.org/t/chip-2021-02-add-native-introspection-opcodes/307/15) in favor of this CHIP.) +- An [implementation of a closely-related `OP_PUSH_TX_STATE`](http://nextchain.cash/op_push_tx_state) exists for the Bitcoin Unlimited node software in the NextChain project. +- A [multi-byte fork of the `OP_PUSHSTATE` proposal](https://github.com/bitjson/op-pushstate/pull/1) introduced the idea of [unary operations which select information by input/output index](https://github.com/bitjson/op-pushstate/pull/1#issuecomment-564072859). + +Delay or rejection of this proposal may incur the following costs: + +- Continued use of the inefficient workaround increases the size of the blockchain could negatively impact initial block download times and storage requirements. +- The complexity of the current workaround increases development costs and raises the barrier to entry for new developers. +- Because the current workaround is difficult to implement securely, implementation problems could result in loss of profits and reputational damage, both for the implementing company and the Bitcoin Cash ecosystem. +- Some applications and use cases will be impossible without native introspection, and competitors may outcompete BCH for these users. + +## Primary Stakeholders + +At least three primary stakeholder groups exist: + +### Companies and Organizations + +Companies and organizations currently building products utilizing introspection include: + +- [General Protocols](https://generalprotocols.com/ 'General Protocols') made [AnyHedge](https://anyhedge.com/ 'AnyHedge'), a volatility risk-trading contract. They plan to build additional, non-custodial services and can support additional functionality with native introspection. + +- [Tobias Ruck](https://twitter.com/TobiasRuck/ 'Tobias Ruck') created [be.cash](https://be.cash/ 'be.cash'), a refillable, offline wallet in the form of a credit card. + +- [Casues Cash](https://causes.cash/ 'Casues Cash') built a modified Mecenas to support recurring payments in USD. + +- [Mistcoin](https://mistcoin.org/ 'Mistcoin') has produced minable SLP tokens. + +- [Flipstarter](https://flipstarter.cash/ 'Flipstarter') funding contracts utilizing native introspection might improve usability. + +- [SmartBCH](https://smartbch.org/ 'SmartBCH') is building an EVM compatible Bitcoin Cash sidechain, and might use introspection as part of a bridge between the main and side chain. + +### Independent Developers + +A number of independent developers are developing tools and services that rely on introspection: + +- [bitjson](https://gist.github.com/bitjson 'bitjson') created [CashChannels](https://blog.bitjson.com/cashchannels-recurring-payments-for-bitcoin-cash-3b274fbfa6e2 'CashChannels'), reccuring payments for Bitcoin Cash. + +- [haryu703](https://github.com/haryu703 'haryu703') created [Hamingja](https://github.com/SLPVH/hamingja 'Hamingja'), a loyalty points system using non-tradable SLP token and is working on an SLP swap/trading contract. + +- [Licho](https://github.com/KarolTrzeszczkowski 'Licho') created the [Last Will](https://github.com/KarolTrzeszczkowski/Electron-Cash-Last-Will-Plugin 'Last Will') contract to manage inheritance, the [Mecenas](https://github.com/KarolTrzeszczkowski/Mecenas-recurring-payment-EC-plugin 'Mecenas') contract for recurring payments, and is interested in implementing traditional games, like [NIM](https://en.m.wikipedia.org/wiki/Nim 'NIM'). + +- [p0oker](https://twitter.com/p0oker 'p0oker') created an [SLP Vending contract](https://github.com/p0o/yield-farming-bch-smart-contract 'SLP Vending contract') that mints tokens on-demand and is building 1) a BCH staking contract that mints tokens over time and 2) an SLP exchange contract to sell NFTs. + +- [Tobias Ruck](https://twitter.com/TobiasRuck/ 'Tobias Ruck') is investigating a non-custodial on-chain gambling product. + +- [James Cramer](https://twitter.com/James_Cramer 'James Cramer') is experimenting with [SLP Mint Guard](https://github.com/simpleledger/Electron-Cash-SLP/blob/cashscript-dev/lib/cashscript/slp_mint_guard.cash 'SLP Mint Guard') to protect minting batons, [SLP Vault](https://github.com/simpleledger/Electron-Cash-SLP/blob/cashscript-dev/lib/cashscript/slp_vault.cash 'SLP Vault') to help reclaim unclaimed tokens, making [tokens with minting schedules](https://github.com/simpleledgerinc/slp-mint-contracts 'tokens with minting schedules'), and building [SLP Dollars](https://github.com/simpleledgerinc/cashscript/blob/master/examples/slp_dollar.cash 'SLP Dollars') which are freezable by the issuer. + +### Investors + +These individuals and organizations have invested in the BCH currency and ecosystem on the premise that it can become peer to peer electronic cash for the world. These stakeholders expect the token to serve effectively as money, including in the innovative financial services which could be enabled by native introspection. + +## Statements + +... + +### [General Protocols](https://generalprotocols.com/ 'General Protocols') + +> The CHIP requires further analysis and specification before it is possible to take a strong position. GP commits to supporting this CHIP with reasonable resources. GP predicts that the benefits to BCH value and network effect will greatly outweigh the costs. Regarding the multiple implementation options available, GP sees multiple good options and encourages the active CHIP participants to choose one with reasonable logic and zero polarization. + +### [p0oker](https://twitter.com/p0oker 'p0oker') + +> I believe removing the current work arounds with the addition of native introspection OP Codes can improve the reliability of the smart contracts on Bitcoin Cash. Covenants are important and useful in many business use cases and it’s important to make them bulletproof! + +### [Licho](https://github.com/KarolTrzeszczkowski 'Licho') + +> The covenant technology have proven to be a vibrant area of BCH development. It allows us to replicate more and more of traditional banking services in a non-custodial way. The native introspection seems to be the key ingredient of the future of permissionless money. + +### Tom Zander — founder [Flowee](https://flowee.org/ 'Flowee') + +> This CHIP is super valuable and I fully support the direction this is going in and the ideas behind this chip. I will continue to monitor the progress with enthusiasm. + +### Jimtendo - [recurr.app](https://recurr.app/ 'Recurr') + +> Introspection is a useful and welcome addition to Bitcoin Cash's Smart Contract capabilities. This CHIP describes a well thought-out and efficient way of achieving it and therefore has my full support and endorsement. + +### [Rosco Kalis](https://kalis.me/) - [cashscript](https://cashscript.org/) + +> If this CHIP is implemented, I will commit to supporting the entire range of implemented introspection opcodes in CashScript. This greatly simplifies the compiler code that deals with introspection and will result in smaller and more efficient covenant contracts. + +## Changelog + +This section summarizes the evolution of this document. + +- **v1.1.4 - 2021-12-1** (current) + - Link to BCHN test cases +- **v1.1.3 - 2021-8-25** ([`29054bb8`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/29054bb8215d059790edd03130b973f7626159c6/CHIP-2021-02-Add-Native-Introspection-Opcodes.md)) + - Add support for `OP_CODESEPARATOR` in `OP_ACTIVEBYTECODE` ([#27](https://gitlab.com/GeneralProtocols/research/chips/-/issues/27)) +- **v1.1.2 - 2021-6-22** ([`e9e8a67d`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/e9e8a67debc9c24aad1129b2df1bb0fc23cd6edc/CHIP-2021-02-Add-Native-Introspection-Opcodes.md)) + - Clarify minimal-encoding requirement for unary operation indexes + - Clarify interaction with `CHIP: Bigger Script Integers` + - Clarify (lack of) effect on transaction validation costs +- **v1.1.1 - 2021-6-10** ([`f501154f`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/f501154fce8bec0f8572ddfef0a689f72384a35d/CHIP-2021-02-Add-Native-Introspection-Opcodes.md)) + - Added statement from stakeholder +- **v1.1 – 2021-6-8** ([`d67eb98b`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/d67eb98be9db0f009859b7b945f1ac7fa4a24be9/CHIP-2021-02-Add-Native-Introspection-Opcodes.md)) + - Added version and `Changelog` + - Extracted integration specifications into independent `CHIP Integration` sections + - Correct and clarify resulting byte order of `OP_OUTPOINTTXHASH` + - Clarify failure of introspection operations which attempt to exceed stack limits +- **v1.0 – 2021-6-3** ([`163f74be`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/163f74bea8f526d268adf0ab6cb578b9f0bde426/CHIP-2021-02-Add-Native-Introspection-Opcodes.md)) + - Revised technical specification: + - Convert `OP_OUTPOINTTXHASH`, `OP_OUTPOINTINDEX`, `OP_UTXOBYTECODE`, and `OP_UTXOVALUE` to unary operations + - Group operations by arity (nullary, unary), then by P2P transaction format order + - Distinguish `OP_ACTIVEBYTECODE` from unary `OP_UTXOBYTECODE` ([rationale](#differences-between-op_activebytecode-and-op_inputindex-op_utxobytecode)) + - Removed formatting operations (`OP_NUM2VARINT` and `OP_VARINT2NUM`) and aggregated/hashed/templated operation placeholders + - Specified integration strategy for `CHIP: Version 3 Transaction Format (PMv3)` and `CHIP: Bigger Script Integers` + - Added `Rationale` section, revised supporting material (`Summary`, `Deployment`, `Motivation`, `Benefits`, etc.) +- **v0.1 – 2021-3-22** ([`95a99e22`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/95a99e2223671ba6b4801a6a86193b299418a7a3/CHIP-2021-02-Add-Native-Introspection-Opcodes.md)) + - Expanded `Motivation and Benefits`, `Costs and Risks`, and `List of major stakeholders` + - Initial technical specification +- **v0 – 2021-2-21** ([`53989f7d`](https://gitlab.com/GeneralProtocols/research/chips/-/blob/53989f7d67a3afbc97ff87d6e94ef8e5c98fddf8/May%202022,%20Native%20Introspection.md)) + - Initial draft + +## Copyright Notice + +Copyright (c) 2021 GeneralProtocols / Research + +Permission is granted to copy, distribute and/or modify this document under the terms of the [MIT license](https://gitlab.com/GeneralProtocols/research/chips/-/blob/master/LICENSE). From 7f651bea3b99022f4665902c573ec20641f65cb7 Mon Sep 17 00:00:00 2001 From: Jonathan Silverblood Date: Mon, 27 Dec 2021 14:56:59 +0200 Subject: [PATCH 06/22] Rename 2022-05-bigger-script-integers to 2022-05-bigger-script-integers.md --- ...5-bigger-script-integers => 2022-05-bigger-script-integers.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename protocol/forks/chips/{2022-05-bigger-script-integers => 2022-05-bigger-script-integers.md} (100%) diff --git a/protocol/forks/chips/2022-05-bigger-script-integers b/protocol/forks/chips/2022-05-bigger-script-integers.md similarity index 100% rename from protocol/forks/chips/2022-05-bigger-script-integers rename to protocol/forks/chips/2022-05-bigger-script-integers.md From 8a59c51281c43c3f353af3194ec679fc77361f11 Mon Sep 17 00:00:00 2001 From: 2qx <57997077+2qx@users.noreply.github.com> Date: Mon, 21 Mar 2022 13:27:11 -0400 Subject: [PATCH 07/22] Rename 2022-native-introspection-opcodes.md to 2022-05-native-introspection-opcodes.md --- ...pection-opcodes.md => 2022-05-native-introspection-opcodes.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename protocol/forks/chips/{2022-native-introspection-opcodes.md => 2022-05-native-introspection-opcodes.md} (100%) diff --git a/protocol/forks/chips/2022-native-introspection-opcodes.md b/protocol/forks/chips/2022-05-native-introspection-opcodes.md similarity index 100% rename from protocol/forks/chips/2022-native-introspection-opcodes.md rename to protocol/forks/chips/2022-05-native-introspection-opcodes.md From 7f5ff930739c57ff542da0b9bfd5dfa68a5572d0 Mon Sep 17 00:00:00 2001 From: Calin Culianu Date: Mon, 6 Jun 2022 15:23:04 -0500 Subject: [PATCH 08/22] Update to create a note about VARINT/ vs Compact size potential ambiguity --- protocol/formats/variable-length-integer.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/protocol/formats/variable-length-integer.md b/protocol/formats/variable-length-integer.md index 2064f4b..c60d2ad 100644 --- a/protocol/formats/variable-length-integer.md +++ b/protocol/formats/variable-length-integer.md @@ -1,6 +1,8 @@ # Compact Variable Length Integer -A variable-width data format for unsigned integers allowing a more compact representation for smaller values. This is sometimes referred to as "compact size" or "var int". +A variable-width data format for unsigned integers allowing a more compact representation for smaller values. This is sometimes referred to as "compact size" or "var int"\*. + +\* - Caution should be used when refering to this integer format as "var int" (or "VarInt"), however, in that it may create an ambiguity. The "Core" family of software (BCHN, BU, and Flowee) all have an internal "VARINT" format that is distinct from this integer format (this format is referred to as COMPACTSIZE or CompactSize in that lineage of software). Perhaps one should refer to this as either "compact int" or "compact size" in order to avoid the potential ambiguity with respect to that VARINT format known to nodes that are descendants of Core. ## Format From 33dc863ee57134dd258d4a444dec315fc67110e8 Mon Sep 17 00:00:00 2001 From: Calin Culianu Date: Mon, 6 Jun 2022 15:43:31 -0500 Subject: [PATCH 09/22] Updated to incorporate reviewer note --- protocol/formats/variable-length-integer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/protocol/formats/variable-length-integer.md b/protocol/formats/variable-length-integer.md index c60d2ad..018a360 100644 --- a/protocol/formats/variable-length-integer.md +++ b/protocol/formats/variable-length-integer.md @@ -2,7 +2,7 @@ A variable-width data format for unsigned integers allowing a more compact representation for smaller values. This is sometimes referred to as "compact size" or "var int"\*. -\* - Caution should be used when refering to this integer format as "var int" (or "VarInt"), however, in that it may create an ambiguity. The "Core" family of software (BCHN, BU, and Flowee) all have an internal "VARINT" format that is distinct from this integer format (this format is referred to as COMPACTSIZE or CompactSize in that lineage of software). Perhaps one should refer to this as either "compact int" or "compact size" in order to avoid the potential ambiguity with respect to that VARINT format known to nodes that are descendants of Core. +\* - Caution should be used when refering to this integer format as "var int" (or "VarInt"), however, in that it may create an ambiguity. The "Core" family of software (BCHN, BU, and Flowee) all have an internal "VARINT" format that is distinct from this integer format. In that "Core" lineage of software, COMPACTSIZE or CompactSize is used to refer to the protocol's variable length integer format, while "VarInt" refers specifically to their internal variable length integer format and is *not* a part of the Bitcoin Cash P2P protocol. ## Format From 201c728d0688c32b9cdf313b7f44099c7b853021 Mon Sep 17 00:00:00 2001 From: Jonathan Silverblood Date: Tue, 7 Jun 2022 11:16:01 +0300 Subject: [PATCH 10/22] Update hf-20201115.md Remove coinbase rule --- protocol/forks/hf-20201115.md | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) diff --git a/protocol/forks/hf-20201115.md b/protocol/forks/hf-20201115.md index f697410..4685ffb 100644 --- a/protocol/forks/hf-20201115.md +++ b/protocol/forks/hf-20201115.md @@ -26,17 +26,6 @@ Bitcoin Cash's Difficulty Adjustment Algorithm (DAA) is replaced with a new algo The specific implementation is called aserti3-2d. Details can be found in the [full specification: ASERT](/protocol/forks/2020-11-15-asert). -## Coinbase Rule - -The purpose of the new coinbase rule is to provide funding to development projects working on common Bitcoin Cash infrastructure. - -The coinbase rule enforces that at least 8% of the block reward must be spent as a single output to the following Bitcoin Cash address: -`bitcoincash:pqnqv9lt7e5vjyp0w88zf2af0l92l8rxdgnlxww9j9`. - -The amount of the output must be equal to or greater than the integer `required`, calculated as follows using integer math: -``` -required = (8 * blockReward) / 100 -``` ## Automatic Replay Protection @@ -64,4 +53,4 @@ should not have to change anything. [1] Median Time Past is described [here](/protocol/blockchain/transaction#median-time-past). It is guaranteed by consensus rules to be monotonically increasing. -[2] The `forkId` is defined as per the [replay protected sighash](/protocol/forks/replay-protected-sighash) specification. \ No newline at end of file +[2] The `forkId` is defined as per the [replay protected sighash](/protocol/forks/replay-protected-sighash) specification. From 06001d4794277f2fd1bdffd3558095d7abf58296 Mon Sep 17 00:00:00 2001 From: TomZ Date: Fri, 10 Jun 2022 22:41:31 +0200 Subject: [PATCH 11/22] Fix table --- protocol/network/messages/dsproof-beta.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/protocol/network/messages/dsproof-beta.md b/protocol/network/messages/dsproof-beta.md index 61b1191..39b532d 100644 --- a/protocol/network/messages/dsproof-beta.md +++ b/protocol/network/messages/dsproof-beta.md @@ -40,7 +40,7 @@ The `spender` format is as follows. Each field in the (below) table's `description` column loosely corresponds to the name of the preimage component used when generating transaction signatures per the [transaction signing algorithm](/protocol/blockchain/transaction/transaction-signing). | Field | Length | Format | [Signature Preimage](/protocol/blockchain/transaction/transaction-signing#preimage-format) Component | Description | -| -----------|:-----------:| ----------:|---------:| +| -----------|:-----------:| ----------:|---------:|---------:| | tx version | 4 | unsigned integer[(LE)](/protocol/misc/endian/little) | #1 | Copy of the transactions version field | | sequence | 4 | unsigned integer[(LE)](/protocol/misc/endian/little) | #7 | Copy of the sequence field of the input | | locktime | 4 | unsigned integer[(LE)](/protocol/misc/endian/little) | #9 | Copy of the transactions locktime field | From 85828eb13d5f30d2cbc65bf85be8c6701d47605e Mon Sep 17 00:00:00 2001 From: TomZ Date: Tue, 26 Jul 2022 00:21:15 +0200 Subject: [PATCH 12/22] fix encoding of emdash to be utf8 --- protocol/blockchain/script.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/protocol/blockchain/script.md b/protocol/blockchain/script.md index d40c539..35a1a3d 100644 --- a/protocol/blockchain/script.md +++ b/protocol/blockchain/script.md @@ -196,8 +196,8 @@ Numeric opcodes (OP_1ADD, etc) are restricted to operating on 4-byte integers. T | OP_TXLOCKTIME | 197 | 0xc5 | Nothing | number | Push the locktime of the current transaction to the stack as a Script Number. | | OP_UTXOVALUE | 198 | 0xc6 | index | number | Pop the top item from the stack as an input index (Script Number). Push the value (in satoshis) of the Unspent Transaction Output (UTXO) spent by that input to the stack as a Script Number. | | OP_UTXOBYTECODE | 199 | 0xc7 | index | script | Pop the top item from the stack as an input index (Script Number). Push the full locking bytecode of the Unspent Transaction Output (UTXO) spent by that input to the stack. | -| OP_OUTPOINTTXHASH | 200 | 0xc8 | index | hash | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint transaction hash – the hash of the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack in OP_HASH256 byte order. | -| OP_OUTPOINTINDEX | 201 | 0xc9 | index | number | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint index – the index of the output in the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack as a Script Number. | +| OP_OUTPOINTTXHASH | 200 | 0xc8 | index | hash | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint transaction hash – the hash of the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack in OP_HASH256 byte order. | +| OP_OUTPOINTINDEX | 201 | 0xc9 | index | number | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint index – the index of the output in the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack as a Script Number. | | OP_INPUTBYTECODE | 202 | 0xca | index | script | Pop the top item from the stack as an input index (Script Number). Push the unlocking bytecode of the input at that index to the stack. | | OP_INPUTSEQUENCENUMBER | 203 | 0xcb | index | number | Pop the top item from the stack as an input index (Script Number). Push the sequence number of the input at that index to the stack as a Script Number. | | OP_OUTPUTVALUE | 204 | 0xcc | index | number | Pop the top item from the stack as an output index (Script Number). Push the value (in satoshis) of the output at that index to the stack as a Script Number. | From 454d84cb58bf118317174550317b081027438b7d Mon Sep 17 00:00:00 2001 From: TomZ Date: Mon, 1 Aug 2022 23:39:23 +0200 Subject: [PATCH 13/22] fix links --- protocol/forks/bch-uahf.md | 2 +- protocol/forks/replay-protected-sighash.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/protocol/forks/bch-uahf.md b/protocol/forks/bch-uahf.md index d947495..bf4e9c8 100644 --- a/protocol/forks/bch-uahf.md +++ b/protocol/forks/bch-uahf.md @@ -186,6 +186,6 @@ NOTE 2: This bit is currently referred to as NODE_BITCOIN_CASH and displayed as [2] https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#Motivation -[3] [Digest for replay protected signature verification accross hard forks](/protocol/network/messages/dsproof-beta) +[3] [Digest for replay protected signature verification accross hard forks](/protocol/forks/replay-protected-sighash) [4] https://github.com/bitcoincashorg/bitcoincash.org/blob/master/spec/uahf-test-plan.md diff --git a/protocol/forks/replay-protected-sighash.md b/protocol/forks/replay-protected-sighash.md index 959e7a8..27be651 100644 --- a/protocol/forks/replay-protected-sighash.md +++ b/protocol/forks/replay-protected-sighash.md @@ -34,7 +34,7 @@ It is defined as follows: In presence of the `SIGHASH_FORKID` flag in the signature's sighash type, the proposed algorithm is used. -Signatures using the `SIGHASH_FORKID` digest method must be rejected before [UAHF](/protocol/forks/bch-uahf.md) is activated. +Signatures using the `SIGHASH_FORKID` digest method must be rejected before [UAHF](/protocol/forks/bch-uahf) is activated. In order to ensure proper activation, the reference implementation uses the `SCRIPT_ENABLE_SIGHASH_FORKID` flag when executing `EvalScript` . From c63701b6f2aa12d972f0bf4a8f162bee0fff8f66 Mon Sep 17 00:00:00 2001 From: Andrew Groot Date: Wed, 2 Nov 2022 15:24:06 -0400 Subject: [PATCH 14/22] Removing an additional reference to the coinbase rule. --- protocol/forks/hf-20201115.md | 1 - 1 file changed, 1 deletion(-) diff --git a/protocol/forks/hf-20201115.md b/protocol/forks/hf-20201115.md index 4685ffb..ea7c1bc 100644 --- a/protocol/forks/hf-20201115.md +++ b/protocol/forks/hf-20201115.md @@ -14,7 +14,6 @@ Bitcoin Cash will execute an upgrade of the network consensus rules according to Starting from the next block these consensus rules changes will take effect: * Bitcoin Cash's Difficulty Adjustment Algorithm (DAA) is replaced with a new system, referred to as aserti3-2d. -* The addition of a new coinbase rule. The following are not consensus changes, but are recommended policy changes for Bitcoin Cash implementations: From 9dcb7d0e5e3e81b38458715257c46265a89e62ce Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Wed, 29 Mar 2023 21:01:16 +0200 Subject: [PATCH 15/22] Update hf-20200515.md add a note about IFP --- protocol/forks/hf-20200515.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/protocol/forks/hf-20200515.md b/protocol/forks/hf-20200515.md index 3dd9196..84badf6 100644 --- a/protocol/forks/hf-20200515.md +++ b/protocol/forks/hf-20200515.md @@ -16,6 +16,7 @@ Starting from the next block these consensus rules changes will take effect: * Bitcoin Cash's SigOps counting and limiting system is replaced with a new system, referred to as SigChecks. * A new opcode called OP_REVERSEBYTES has been added to the script system. * Enforcement of the Infrastructure Funding Plan, subject to activation by [BIP 9](https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki) miner signalling. +(Post-activation note: the IFP was rejected and never activated on Bitcoin Cash network.) The following are not consensus changes, but are recommended policy changes for Bitcoin Cash implementations: @@ -37,6 +38,8 @@ Details can be found in the [full specification: OP_REVERSEBYTES](/protocol/fork ## Infrastructure Funding Plan +(Post-activation note: the IFP was rejected and never activated on Bitcoin Cash network.) + The purpose of the Infrastructure Funding Plan (IFP) is to provide funding to development projects working on common Bitcoin Cash infrastructure. If activated, it enforces that 5% of the block reward is spent to one of a set of specified addresses. Activation is triggered via [BIP 9](https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki) version bits signalling prior to the May 15 upgrade. @@ -69,4 +72,4 @@ should not have to change anything. [1] Median Time Past is described [here](/protocol/blockchain/transaction#median-time-past). It is guaranteed by consensus rules to be monotonically increasing. -[2] The `forkId` is defined as per the [replay protected sighash](/protocol/forks/replay-protected-sighash) specification. \ No newline at end of file +[2] The `forkId` is defined as per the [replay protected sighash](/protocol/forks/replay-protected-sighash) specification. From d1c43ded996a1ad95bb12fd15f70ef41dbdf5f94 Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Sun, 21 May 2023 11:16:16 +0200 Subject: [PATCH 16/22] Update to reflect BigInt and OP_MUL upgrade in HF20220515 --- protocol/blockchain/script.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/protocol/blockchain/script.md b/protocol/blockchain/script.md index 35a1a3d..e7a09f8 100644 --- a/protocol/blockchain/script.md +++ b/protocol/blockchain/script.md @@ -128,7 +128,10 @@ Op codes marked with **(do not use)** are disallowed and will make a transaction ### Arithmetic -Numeric opcodes (OP_1ADD, etc) are restricted to operating on 4-byte integers. The semantics are subtle, though: operands must be in the range [-2^31 +1...2^31 -1], but results may overflow (and are valid as long as they are not used in a subsequent numeric operation). +Numeric opcodes (OP_1ADD, etc.) are restricted to operating on 8-byte signed "Script Number" integers, enabled in [HF-20220515](/protocol/forks/hf-20220515). +This excludes the value `-9223372036854775808` that fits in 8-byte two's complement encoding, but does not fit in an 8-byte Script Number encoding used by the Script VM. +If an operation [overflows or underflows](protocol/forks/chips/2022-05-bigger-script-integers#arithmetic-operation-overflows), the operation must immediately fail evaluation. + |Word |Value |Hex |Input |Output | Description | |---------------------|-------|----|--------------|-----------------|------------------------------------------------------| @@ -142,7 +145,7 @@ Numeric opcodes (OP_1ADD, etc) are restricted to operating on 4-byte integers. T |OP_0NOTEQUAL | 146 |0x92|in |true / false | Returns 0 if the input is 0. 1 otherwise. | |OP_ADD | 147 |0x93|a b |out | *a* is added to *b*. | |OP_SUB | 148 |0x94|a b |out | *b* is subtracted from *a*. | -|OP_MUL | 149 |0x95|a b |out | *a* is multiplied by *b*. **DISABLED** | +|OP_MUL | 149 |0x95|a b |out | *a* is multiplied by *b*. Enabled in [HF-20220515](/protocol/forks/hf-20220515). | |OP_DIV | 150 |0x96|a b |out | *a* is [divided](/protocol/blockchain/script/integer-division) by *b*. | |OP_MOD | 151 |0x97|a b |out | Returns the remainder after *a* is [divided](/protocol/blockchain/script/integer-division) by *b*. | |OP_LSHIFT | 152 |0x98|a b |out | Shifts *a* left *b* bits, preserving sign. **DISABLED** | From f1c61ec8ce1fbb6e8dfda8bad4103595444755b1 Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Sun, 21 May 2023 11:35:51 +0200 Subject: [PATCH 17/22] Add CashTokens Introspection Opcodes --- protocol/blockchain/script.md | 38 ++++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/protocol/blockchain/script.md b/protocol/blockchain/script.md index e7a09f8..966e46e 100644 --- a/protocol/blockchain/script.md +++ b/protocol/blockchain/script.md @@ -189,22 +189,28 @@ If an operation [overflows or underflows](protocol/forks/chips/2022-05-bigger-sc ### Introspection -|Word |Value | Hex |Input |Output | Description | -|------------------------|------|------|----------------|---------|--------------------------------------------------------| -| OP_INPUTINDEX | 192 | 0xc0 | Nothing | number | Push the index of the input being evaluated to the stack as a Script Number. | -| OP_ACTIVEBYTECODE | 193 | 0xc1 | Nothing | script | Push the bytecode currently being evaluated, beginning after the last executed OP_CODESEPARATOR, to the stack1. For Pay-to-Script-Hash (P2SH) evaluations, this is the redeem bytecode of the Unspent Transaction Output (UTXO) being spent; for all other evaluations, this is the locking bytecode of the UTXO being spent. | -| OP_TXVERSION | 194 | 0xc2 | Nothing | number | Push the version of the current transaction to the stack as a Script Number. | -| OP_TXINPUTCOUNT | 195 | 0xc3 | Nothing | number | Push the count of inputs in the current transaction to the stack as a Script Number. | -| OP_TXOUTPUTCOUNT | 196 | 0xc4 | Nothing | number | Push the count of outputs in the current transaction to the stack as a Script Number. | -| OP_TXLOCKTIME | 197 | 0xc5 | Nothing | number | Push the locktime of the current transaction to the stack as a Script Number. | -| OP_UTXOVALUE | 198 | 0xc6 | index | number | Pop the top item from the stack as an input index (Script Number). Push the value (in satoshis) of the Unspent Transaction Output (UTXO) spent by that input to the stack as a Script Number. | -| OP_UTXOBYTECODE | 199 | 0xc7 | index | script | Pop the top item from the stack as an input index (Script Number). Push the full locking bytecode of the Unspent Transaction Output (UTXO) spent by that input to the stack. | -| OP_OUTPOINTTXHASH | 200 | 0xc8 | index | hash | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint transaction hash – the hash of the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack in OP_HASH256 byte order. | -| OP_OUTPOINTINDEX | 201 | 0xc9 | index | number | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint index – the index of the output in the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack as a Script Number. | -| OP_INPUTBYTECODE | 202 | 0xca | index | script | Pop the top item from the stack as an input index (Script Number). Push the unlocking bytecode of the input at that index to the stack. | -| OP_INPUTSEQUENCENUMBER | 203 | 0xcb | index | number | Pop the top item from the stack as an input index (Script Number). Push the sequence number of the input at that index to the stack as a Script Number. | -| OP_OUTPUTVALUE | 204 | 0xcc | index | number | Pop the top item from the stack as an output index (Script Number). Push the value (in satoshis) of the output at that index to the stack as a Script Number. | -| OP_OUTPUTBYTECODE | 205 | 0xcd | index | script | Pop the top item from the stack as an output index (Script Number). Push the locking bytecode of the output at that index to the stack. | +|Word |Value | Hex |Input |Output | Description | +|--------------------------|------|------|----------------|---------|--------------------------------------------------------| +| OP_INPUTINDEX | 192 | 0xc0 | Nothing | number | Push the index of the input being evaluated to the stack as a Script Number. | +| OP_ACTIVEBYTECODE | 193 | 0xc1 | Nothing | script | Push the bytecode currently being evaluated, beginning after the last executed OP_CODESEPARATOR, to the stack1. For Pay-to-Script-Hash (P2SH) evaluations, this is the redeem bytecode of the Unspent Transaction Output (UTXO) being spent; for all other evaluations, this is the locking bytecode of the UTXO being spent. | +| OP_TXVERSION | 194 | 0xc2 | Nothing | number | Push the version of the current transaction to the stack as a Script Number. | +| OP_TXINPUTCOUNT | 195 | 0xc3 | Nothing | number | Push the count of inputs in the current transaction to the stack as a Script Number. | +| OP_TXOUTPUTCOUNT | 196 | 0xc4 | Nothing | number | Push the count of outputs in the current transaction to the stack as a Script Number. | +| OP_TXLOCKTIME | 197 | 0xc5 | Nothing | number | Push the locktime of the current transaction to the stack as a Script Number. | +| OP_UTXOVALUE | 198 | 0xc6 | index | number | Pop the top item from the stack as an input index (Script Number). Push the value (in satoshis) of the Unspent Transaction Output (UTXO) spent by that input to the stack as a Script Number. | +| OP_UTXOBYTECODE | 199 | 0xc7 | index | script | Pop the top item from the stack as an input index (Script Number). Push the full locking bytecode of the Unspent Transaction Output (UTXO) spent by that input to the stack. | +| OP_OUTPOINTTXHASH | 200 | 0xc8 | index | hash | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint transaction hash – the hash of the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack in OP_HASH256 byte order. | +| OP_OUTPOINTINDEX | 201 | 0xc9 | index | number | Pop the top item from the stack as an input index (Script Number). From that input, push the outpoint index – the index of the output in the transaction which created the Unspent Transaction Output (UTXO) which is being spent – to the stack as a Script Number. | +| OP_INPUTBYTECODE | 202 | 0xca | index | script | Pop the top item from the stack as an input index (Script Number). Push the unlocking bytecode of the input at that index to the stack. | +| OP_INPUTSEQUENCENUMBER | 203 | 0xcb | index | number | Pop the top item from the stack as an input index (Script Number). Push the sequence number of the input at that index to the stack as a Script Number. | +| OP_OUTPUTVALUE | 204 | 0xcc | index | number | Pop the top item from the stack as an output index (Script Number). Push the value (in satoshis) of the output at that index to the stack as a Script Number. | +| OP_OUTPUTBYTECODE | 205 | 0xcd | index | script | Pop the top item from the stack as an output index (Script Number). Push the locking bytecode of the output at that index to the stack. | +| OP_UTXOTOKENCATEGORY | 206 | 0xce | index | script | Pop the top item from the stack as an input index (VM Number). If the Unspent Transaction Output (UTXO) spent by that input includes no tokens, push a 0 (VM Number) to the stack. If the UTXO does not include a non-fungible token with a capability, push the UTXO's token category, otherwise, push the concatenation of the token category and capability, where the mutable capability is represented by 1 (VM Number) and the minting capability is represented by 2 (VM Number). | +| OP_UTXOTOKENCOMMITMENT | 207 | 0xcf | index | script | Pop the top item from the stack as an input index (VM Number). Push the token commitment of the Unspent Transaction Output (UTXO) spent by that input to the stack. If the UTXO does not include a non-fungible token, or if it includes a non-fungible token with a zero-length commitment, push a 0 (VM Number). | +| OP_UTXOTOKENAMOUNT | 208 | 0xd0 | index | number | Pop the top item from the stack as an input index (VM Number). Push the fungible token amount of the Unspent Transaction Output (UTXO) spent by that input to the stack as a VM Number. If the UTXO includes no fungible tokens, push a 0 (VM Number). | +| OP_OUTPUTTOKENCATEGORY | 209 | 0xd1 | index | script | Pop the top item from the stack as an output index (VM Number). If the output at that index includes no tokens, push a 0 (VM Number) to the stack. If the output does not include a non-fungible token with a capability, push the output's token category, otherwise, push the concatenation of the token category and capability, where the mutable capability is represented by 1 (VM Number) and the minting capability is represented by 2 (VM Number). | +| OP_OUTPUTTOKENCOMMITMENT | 210 | 0xd2 | index | script | Pop the top item from the stack as an output index (VM Number). Push the token commitment of the output at that index to the stack. If the output does not include a non-fungible token, or if it includes a non-fungible token with a zero-length commitment, push a 0 (VM Number). | +| OP_OUTPUTTOKENAMOUNT | 211 | 0xd3 | index | number | Pop the top item from the stack as an output index (VM Number). Push the fungible token amount of the output at that index to the stack as a VM Number. If the output includes no fungible tokens, push a 0 (VM Number). | ### Reserved From 0d56e673e5a243c2ff7c5140f700fb5c4cbdd85b Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Sun, 21 May 2023 12:18:25 +0200 Subject: [PATCH 18/22] Add HF-20230515 --- protocol/forks/hf-20230515.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 protocol/forks/hf-20230515.md diff --git a/protocol/forks/hf-20230515.md b/protocol/forks/hf-20230515.md new file mode 100644 index 0000000..508e85e --- /dev/null +++ b/protocol/forks/hf-20230515.md @@ -0,0 +1,12 @@ +# HF-20230515 + +The May 2023 hard fork is the second upgrade to be comprised of [CHIPs](#chips). + +## Contents + +The May 2023 hard fork is comprised the following CHIPs: + +1. [CHIP 2021-01 Allow Transactions to be smaller in size](https://gitlab.com/bitcoin.cash/chips/-/blob/master/CHIP-2021-01-Restrict%20Transaction%20Versions.md) +2. [CHIP 2021-01 Restrict Transaction Version](https://gitlab.com/bitcoin.cash/chips/-/blob/master/CHIP-2021-01-Restrict%20Transaction%20Versions.md) +3. [CHIP-2022-02-CashTokens: Token Primitives for Bitcoin Cash](https://github.com/bitjson/cashtokens) +4. [CHIP-2022-05 Pay-to-Script-Hash-32 (P2SH32) for Bitcoin Cash](https://gitlab.com/0353F40E/p2sh32/-/tree/main) From 89f3a2fba11f414a8141560e609692753a787dbe Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Mon, 29 May 2023 18:30:27 +0200 Subject: [PATCH 19/22] Add CashTokens information to transaction.md --- protocol/blockchain/transaction.md | 46 ++++++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 2 deletions(-) diff --git a/protocol/blockchain/transaction.md b/protocol/blockchain/transaction.md index c085c95..23b03d9 100644 --- a/protocol/blockchain/transaction.md +++ b/protocol/blockchain/transaction.md @@ -64,10 +64,11 @@ As with lock-time, when the relative lock-time is interpreted as a time, it is c ## Transaction Output Transaction outputs are the "credits" of Bitcoin Cash. -Each transaction output denotes a number of satoshis and the requirements are spending them. +Each transaction output denotes a number of satoshis and the requirements for spending them. These requirements take the form of a [locking script](/protocol/blockchain/transaction/locking-script) and can equate to anything from the satoshis only being spendable by the owner of a specific private key, to anyone being able to spend them, to no one being able to spend them. While a transaction output has not been spent by another transaction (i.e. had its locking script "unlocked" by an input of a valid transaction) it is referred to as an unspent transaction output, or UTXO. A Transaction Output that is being spent by a Transaction Input is often referred to as the "PrevOut", short for the "Previous Output". +In addition to number of satoshis, transaction outputs may optionally encode token state using the [token output format](#token-output-format) enabled in [HF-20230515](/protocol/forks/hf-20230515). ### Format @@ -77,6 +78,47 @@ A Transaction Output that is being spent by a Transaction Input is often referre | locking script length | variable | [variable length integer](/protocol/formats/variable-length-integer) | The size of the locking script in bytes. | | locking script | variable | bytes[(BE)](/protocol/misc/endian/big) | The contents of the locking script. | +#### Token Output Format + +Tokens may be encoded in outputs in addition to satoshi value using a "token prefix", a data structure that can encode a token category, zero or one non-fungible token (NFT), and an amount of fungible tokens (FTs). + +For backwards-compatibility with existing transaction decoding implementations, a transaction output's token prefix (if present) is encoded before byte 0 of its locking bytecode, and the locking script length preceding the two fields is increased to cover both fields (such that the length could be renamed to "token prefix and locking script length"). +The token prefix is not part of the locking bytecode and must not be included in bytecode evaluation. +If the output contains a token, then the serialized output format becomes: + +| Field | Length | Format | Description | +|--|--|--|--| +| value | 8 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The number of satoshis to be transferred. | +| token prefix and locking script length | variable | [variable length integer](/protocol/formats/variable-length-integer) | The combined size of full token prefix and the locking script in bytes. | +| PREFIX_TOKEN | 1 byte | constant | Magic byte defined at codepoint 0xef (239) and indicates the presence of a token prefix. | +| token category ID | 32 bytes | bytes | After the PREFIX_TOKEN byte, a 32-byte "token category ID" is required, encoded in OP_HASH256 byte order. | +| token bitfield | 1 byte | bitfield | A bitfield encoding two 4-bit fields is required. | +| \[NFT commitment length\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | The size of the NFT commitment in bytes. Present only if token bitfield bit 0x40 is set. | +| \[NFT commitment\] | variable | bytes | The contents of the NFT commitment. Present only if token bitfield bit 0x40 is set. | +| \[FT amount\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | An amount of fungible tokens, present only if token bitfield bit `0x10` is set. | +| locking script | variable | bytes[(BE)](/protocol/misc/endian/big) | The contents of the locking script. | + +**Token Bitfield** + +The token bitfield encodes part of the NFT state, and bit flags that specify whether optional fields are present. +The two 4-bit fields are specified as: + 1. `prefix_structure` (`token_bitfield & 0xf0`) - 4 bitflags, defined at the higher half of the bitfield, indicating the structure of the token prefix: + 1. `0x80` (`0b10000000`) - `RESERVED_BIT`, must be unset. + 2. `0x40` (`0b01000000`) - `HAS_COMMITMENT_LENGTH`, the prefix encodes a commitment length and commitment. + 3. `0x20` (`0b00100000`) - `HAS_NFT`, the prefix encodes a non-fungible token. + 4. `0x10` (`0b00010000`) - `HAS_AMOUNT`, the prefix encodes an amount of fungible tokens. + 2. `nft_capability` (`token_bitfield & 0x0f`) – A 4-bit value, defined at the lower half of the bitfield, indicating the non-fungible token capability, if present. + 1. If not `HAS_NFT`: must be `0x00`. + 2. If `HAS_NFT`: + 1. `0x00` – No capability – the encoded non-fungible token is an **immutable token**. + 2. `0x01` – The **`mutable` capability** – the encoded non-fungible token is a **mutable token**. + 3. `0x02` – The **`minting` capability** – the encoded non-fungible token is a **minting token**. + 4. Values greater than `0x02` are reserved and must not be used. + ## Transaction Fee -Extra satoshis from the Transaction Inputs that are not accounted for in the Transaction Outputs may be collected by the miner as the transaction fee. \ No newline at end of file +Extra satoshis from the Transaction Inputs that are not accounted for in the Transaction Outputs may be collected by the miner as the transaction fee. + +## Implicit Burning of Tokens + +Extra satoshis from the Transaction Inputs that are not accounted for in the Transaction Outputs will be burned in such a transaction. From 660f20ca2639b7298ef38cbc756629f9198d84ac Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Tue, 12 Mar 2024 13:30:05 +0100 Subject: [PATCH 20/22] add hf-20230515 to main page and fix nit in description --- home.md | 2 +- protocol/forks/hf-20230515.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/home.md b/home.md index 62f49cc..c60e79c 100644 --- a/home.md +++ b/home.md @@ -29,7 +29,6 @@ ### Network upgrades #### Pre-BCH - | Year (Creation) | BIPs | |------|-------| | 2012 | [Bip-16](/protocol/forks/bip-0016) — [Bip-34](/protocol/forks/bip-0034) — [BIP-35](/protocol/forks/bip-0035) — [Bip-37](/protocol/forks/bip-0037) | @@ -47,6 +46,7 @@ | 2020 | [HF-20200515](/protocol/forks/hf-20200515) — [HF-20201115](/protocol/forks/hf-20201115) | | 2021 | [HF-20210515](/protocol/forks/hf-20210515) | | 2022 | [HF-20220515](/protocol/forks/hf-20220515) | +| 2023 | [HF-20230515](/protocol/forks/hf-20230515) | ### Network protocol [Network Messages](/protocol/network/messages) — [Handshake](/protocol/network/node-handshake) diff --git a/protocol/forks/hf-20230515.md b/protocol/forks/hf-20230515.md index 508e85e..01a7da4 100644 --- a/protocol/forks/hf-20230515.md +++ b/protocol/forks/hf-20230515.md @@ -1,6 +1,6 @@ # HF-20230515 -The May 2023 hard fork is the second upgrade to be comprised of [CHIPs](#chips). +The May 2023 hard fork is the third upgrade to be comprised of [CHIPs](#chips). ## Contents From e5153806e656c836b5ac1b9f2dce1e7c31243499 Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Tue, 12 Mar 2024 15:37:33 +0100 Subject: [PATCH 21/22] Update transaction signing with HF-20230515 - Include SIGHASH_UTXOS - Include token contents --- .../transaction/transaction-signing.md | 87 ++++++++++++++----- 1 file changed, 67 insertions(+), 20 deletions(-) diff --git a/protocol/blockchain/transaction/transaction-signing.md b/protocol/blockchain/transaction/transaction-signing.md index c310d06..8815052 100644 --- a/protocol/blockchain/transaction/transaction-signing.md +++ b/protocol/blockchain/transaction/transaction-signing.md @@ -13,13 +13,15 @@ However, there are a number of issues with signing a transaction that must be ad Points (1) and (2) mean that if the signature is changed, the transaction's hash will change. Points (2) and (3) mean that the data that the signature hash preimage (i.e. the data that is hashed and signed) must not be the full transaction data. -In addition, because signatures relate only to a single input to a transaction (i.e. spending an unspent transaction output or UTXO) the may be multiple signatures in a transaction potentially created by different private keys, or even different people. +In addition, because signatures relate only to a single input to a transaction (i.e. spending an unspent transaction output or UTXO) there may be multiple signatures in a transaction potentially created by different private keys, or even different people. As a consequence of these factors, signatures have more parameters than may be immediately obvious, and the details of how signatures are generated can be, and have been, changed in a number of ways. These parameters are encoded in the [Hash Type](#hash-type). In addition, as a part of [BCH-UAHF](/protocol/forks/bch-uahf) (activated in block 478,559), the transaction signed format changed from the legacy [Bitcoin Core (BTC) method](#bitcoin-core-signatures) to the [Bitcoin Cash (BCH) Signatures](#bitcoin-cash-signatures). In both cases, there is a signature preimage format (input) and a signature format (output). +With [HF-20230515](/protocol/forks/hf-20230515) protocol upgrade, the signature preimage format was extended to include spent and new outputs's token data, and a new hash type `SIGHASH_UTXOS` was introduced so signatures can commit to whole input's prevouts. + ### Hash Type Parameters that change the way a signature hash is generated are encoded in the hash type field. @@ -36,19 +38,23 @@ In conjunction with the above values, the higher-order bits act as a bitmask wit | Bit | Meaning | |--|--| +| `0x00000020` | `SIGHASH_UTXOS`. If set, the hash of whole UTXOs is inserted in the signing serialization algorithm immediately following hash of the previous outputs references. The hash is a 32-byte double SHA256 of the serialization of all referenced UTXOs, concatenated in input order, excluding output count. | | `0x00000040` | `SIGHASH_FORKID`. If set, indicates that this signature is for a Bitcoin Cash transaction. Required following BCH-UAHF, to prevent transactions from being valid on both the BTC and BCH chains. | | `0x00000080` | `SIGHASH_ANYONECANPAY`. Indicates that only information about the input the signature is for will be included, allowing other inputs to be added without impacting the signature for the current input. | Combining these, there are 6 valid signature hash types in Bitcoin Cash. Only the least significant byte (LSB) is shown in binary, since the rest of the bits are zero. -| Signature hash type | Value (hex) | LSB (bin) | Description | -| -------------------------------------------------------- | ----------- | ----------- | --------------------------------------------------------------------- | -| SIGHASH_ALL \| SIGHASH_FORKID | 0x00000041 | 0b01000001 | Signature applies to all inputs and outputs. | -| SIGHASH_NONE \| SIGHASH_FORKID | 0x00000042 | 0b01000010 | Signature applies to all inputs and none of the outputs. | -| SIGHASH_SINGLE \| SIGHASH_FORKID | 0x00000043 | 0b01000011 | Signature applies to all inputs and the output with the same index. | -| SIGHASH_ALL \| SIGHASH_ANYONECANPAY \| SIGHASH_FORKID | 0x000000C1 | 0b11000001 | Signature applies to its own input and all outputs. | -| SIGHASH_NONE \| SIGHASH_ANYONECANPAY \| SIGHASH_FORKID | 0x000000C2 | 0b11000010 | Signature applies to its own input and none of the outputs. | -| SIGHASH_SINGLE \| SIGHASH_ANYONECANPAY \| SIGHASH_FORKID | 0x000000C3 | 0b11000011 | Signature applies to its own input and the output with the same index.| +| Signature hash type | Value (hex) | LSB (bin) | Description | +| -------------------------------------------------------- | ----------- | ---------- | ------------ | +| SIGHASH_ALL \| SIGHASH_FORKID | 0x00000041 | 0b01000001 | Signature applies to all inputs and outputs. | +| SIGHASH_NONE \| SIGHASH_FORKID | 0x00000042 | 0b01000010 | Signature applies to all inputs and none of the outputs. | +| SIGHASH_SINGLE \| SIGHASH_FORKID | 0x00000043 | 0b01000011 | Signature applies to all inputs and the output with the same index. | +| SIGHASH_ALL \| SIGHASH_UTXOS \| SIGHASH_FORKID | 0x00000061 | 0b01100001 | Signature applies to all inputs and outputs. Signature preimage includes a hash of all referenced UTXOs. | +| SIGHASH_NONE \| SIGHASH_UTXOS \| SIGHASH_FORKID | 0x00000062 | 0b01100010 | Signature applies to all inputs and none of the outputs. Signature preimage includes a hash of all referenced UTXOs. | +| SIGHASH_SINGLE \| SIGHASH_UTXOS \| SIGHASH_FORKID | 0x00000063 | 0b01100011 | Signature applies to all inputs and the output with the same index. Signature preimage includes a hash of all referenced UTXOs. | +| SIGHASH_ALL \| SIGHASH_ANYONECANPAY \| SIGHASH_FORKID | 0x000000C1 | 0b11000001 | Signature applies to its own input and all outputs. | +| SIGHASH_NONE \| SIGHASH_ANYONECANPAY \| SIGHASH_FORKID | 0x000000C2 | 0b11000010 | Signature applies to its own input and none of the outputs. | +| SIGHASH_SINGLE \| SIGHASH_ANYONECANPAY \| SIGHASH_FORKID | 0x000000C3 | 0b11000011 | Signature applies to its own input and the output with the same index. | ## Bitcoin Cash Signatures @@ -60,24 +66,28 @@ At a high level, the preimage format for a signature within a single input is a 1. Transaction version 2. Previous transaction outputs identifiers -3. Transaction input sequence numbers -4. The identifier of the output being spent -5. The locking script of the output being spent -6. The value of the output being spent -7. The sequence number of the transaction input -8. The created transaction outputs -9. Transaction locktime -10. The signature hash type +3. If `SIGHASH_UTXOS` is set, whole content of all previous transaction outputs +4. Transaction input sequence numbers +5. The identifier of the output being spent +6. Token content of the output being spent +7. The locking script of the output being spent +8. The value of the output being spent +9. The sequence number of the transaction input being spent +10. The created transaction outputs +11. Transaction locktime +12. The signature hash type The following table specifies, in detail, the preimage format for a signature within a single input: | Field | Length | Format | Description | |--|--|--|--| | transaction version | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The value of transaction's version field. | -| previous outputs hash | 32 bytes | hash[(BE)](/protocol/misc/endian/big) | A double SHA-256 hash of the set of previous outputs spent by the inputs of the transaction. See [Previous Outputs](#previous-outputs-hash) for the hash preimage format.

If hash type is "ANYONECANPAY" then this is all `0x00` bytes. | +| previous outputs identifiers hash | 32 bytes | hash[(BE)](/protocol/misc/endian/big) | A double SHA-256 hash of the set of previous outputs identifiers (source transaction hash and output index) spent by the inputs of the transaction. See [Previous Outputs Identifiers](#previous-outputs-identifiers-hash) for the hash preimage format.

If hash type is "ANYONECANPAY" then this is all `0x00` bytes. | +| \[previous outputs hash\] | \[32 bytes\] | hash[(BE)](/protocol/misc/endian/big) | A double SHA-256 hash of the set of previous outputs spent by the inputs of the transaction. [Whole contents](../transaction#transaction-output) of the referenced outputs are concantenated and hashed. See [Previous Outputs](#previous-outputs-hash) for the hash preimage format.

If hash type doesn't have the "UTXOS" bit set, then this hash is omitted. | | sequence numbers hash | 32 bytes | hash[(BE)](/protocol/misc/endian/big) | A double SHA-256 hash of the set of sequence numbers of the inputs of the transaction. See [Sequence Numbers](#sequence-numbers-hash) for the hash preimage format.

If hash type is "ANYONECANPAY", "SINGLE", or "NONE" then this is all `0x00` bytes. | | previous output hash | 32 bytes | hash[(LE)](/protocol/misc/endian/little) | The transaction ID of the previous output being spent. | | previous output index | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The index of the output to be spent. | +| previous output token contents | variable | [token serialization](../transaction#token-output-format) | Previous output's token contents, including the PREFIX_TOKEN byte as the start. | | modified locking script length | variable | [variable length integer](/protocol/format/variable-length-integer) | The number of bytes for `modified_locking_script`. | | modified locking script | `modified_locking_script_length` bytes | bytes[(BE)](/protocol/misc/endian/big) | The subset of the locking script used for signing. See [Modified Locking Script](#modified-locking-script) | | previous output value | 8 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The value of the transaction output being spent. | @@ -86,7 +96,7 @@ The following table specifies, in detail, the preimage format for a signature wi | transaction lock time | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The lock time of the transaction. | | hash type | 4 bytes | [Hash Type](#hash-type)[(LE)](/protocol/misc/endian/little) | Flags indicating the rules for how this signature was generated. | -#### Previous Outputs Hash +#### Previous Outputs Identifiers Hash The double-SHA256-hash of the following data is used. @@ -97,6 +107,24 @@ For each transaction input in the transaction, append the following information: | previous transaction hash | 32 bytes | bytes[(LE)](/protocol/misc/endian/little) | The hash of the transaction that generated the output to be spent. | | output index | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The index of the output to be spent from the specified transaction. | +#### Previous Outputs Hash + +The double-SHA256-hash of the following data is used. + +For each transaction input in the transaction, append the following information: + +| Field | Length | Format | Description | +|--|--|--|--| +| value | 8 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The number of satoshis to be transferred. | +| \[token prefix and\] locking script length | variable | [variable length integer](/protocol/formats/variable-length-integer) | If the output contains tokens then the combined size of full token prefix and the locking script in bytes, else just the locking script size in bytes. | +| \[PREFIX_TOKEN\] | 1 byte | constant | Magic byte defined at codepoint 0xef (239) and indicates the presence of a token prefix. | +| \[token category ID\] | 32 bytes | bytes | After the PREFIX_TOKEN byte, a 32-byte "token category ID" is required, encoded in OP_HASH256 byte order. | +| \[token bitfield\] | 1 byte | bitfield | A bitfield encoding two 4-bit fields is required. | +| \[NFT commitment length\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | The size of the NFT commitment in bytes. Present only if token bitfield bit 0x40 is set. | +| \[NFT commitment\] | variable | bytes | The contents of the NFT commitment. Present only if token bitfield bit 0x40 is set. | +| \[FT amount\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | An amount of fungible tokens, present only if token bitfield bit `0x10` is set. | +| locking script | variable | bytes[(BE)](/protocol/misc/endian/big) | The contents of the locking script. | + #### Sequence Numbers Hash The double-SHA256-hash of the following data is used. @@ -107,6 +135,19 @@ For each transaction input in the transaction, append the following information: |--|--|--|--| | sequence number | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The sequence number field of the transaction input. | +#### Token Contents + +If the output being spent has tokens, append the following information: + +| Field | Length | Format | Description | +|--|--|--|--| +| PREFIX_TOKEN | 1 byte | constant | Magic byte defined at codepoint 0xef (239) and indicates the presence of a token prefix. | +| token category ID | 32 bytes | bytes | After the PREFIX_TOKEN byte, a 32-byte "token category ID" is required, encoded in OP_HASH256 byte order. | +| token bitfield | 1 byte | bitfield | A bitfield encoding two 4-bit fields is required. | +| \[NFT commitment length\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | The size of the NFT commitment in bytes. Present only if token bitfield bit 0x40 is set. | +| \[NFT commitment\] | variable | bytes | The contents of the NFT commitment. Present only if token bitfield bit 0x40 is set. | +| \[FT amount\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | An amount of fungible tokens, present only if token bitfield bit `0x10` is set. | + #### Modified Locking Script The locking script included in the signature preimage is, first, dependent on the type of locking script included in the previous output. @@ -136,7 +177,13 @@ For each transaction output to be signed (per the hash mode), append the followi | Field | Length | Format | Description | |--|--|--|--| | value | 8 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The number of satoshis to be transferred. | -| locking script length | variable | [variable length integer](/protocol/formats/variable-length-integer) | The size of the locking script in bytes. | +| \[token prefix and\] locking script length | variable | [variable length integer](/protocol/formats/variable-length-integer) | If the output contains tokens then the combined size of full token prefix and the locking script in bytes, else just the locking script size in bytes. | +| \[PREFIX_TOKEN\] | 1 byte | constant | Magic byte defined at codepoint 0xef (239) and indicates the presence of a token prefix. | +| \[token category ID\] | 32 bytes | bytes | After the PREFIX_TOKEN byte, a 32-byte "token category ID" is required, encoded in OP_HASH256 byte order. | +| \[token bitfield\] | 1 byte | bitfield | A bitfield encoding two 4-bit fields is required. | +| \[NFT commitment length\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | The size of the NFT commitment in bytes. Present only if token bitfield bit 0x40 is set. | +| \[NFT commitment\] | variable | bytes | The contents of the NFT commitment. Present only if token bitfield bit 0x40 is set. | +| \[FT amount\] | variable | [variable length integer](/protocol/formats/variable-length-integer) | An amount of fungible tokens, present only if token bitfield bit `0x10` is set. | | locking script | variable | bytes[(BE)](/protocol/misc/endian/big) | The contents of the locking script. | ### Signature Format From b8adc8291ef116c1518be03808fc7d288e988717 Mon Sep 17 00:00:00 2001 From: bitcoincashautist <80100588+A60AB5450353F40E@users.noreply.github.com> Date: Tue, 12 Mar 2024 15:40:57 +0100 Subject: [PATCH 22/22] indicate token content is omitted if utxo doesn't have tokens --- protocol/blockchain/transaction/transaction-signing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/protocol/blockchain/transaction/transaction-signing.md b/protocol/blockchain/transaction/transaction-signing.md index 8815052..4b27c92 100644 --- a/protocol/blockchain/transaction/transaction-signing.md +++ b/protocol/blockchain/transaction/transaction-signing.md @@ -87,7 +87,7 @@ The following table specifies, in detail, the preimage format for a signature wi | sequence numbers hash | 32 bytes | hash[(BE)](/protocol/misc/endian/big) | A double SHA-256 hash of the set of sequence numbers of the inputs of the transaction. See [Sequence Numbers](#sequence-numbers-hash) for the hash preimage format.

If hash type is "ANYONECANPAY", "SINGLE", or "NONE" then this is all `0x00` bytes. | | previous output hash | 32 bytes | hash[(LE)](/protocol/misc/endian/little) | The transaction ID of the previous output being spent. | | previous output index | 4 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The index of the output to be spent. | -| previous output token contents | variable | [token serialization](../transaction#token-output-format) | Previous output's token contents, including the PREFIX_TOKEN byte as the start. | +| \[previous output token contents\] | \[variable\] | [token serialization](../transaction#token-output-format) | Previous output's token contents, including the PREFIX_TOKEN byte as the start. If the previous output doesn't have tokens then this is omitted. | | modified locking script length | variable | [variable length integer](/protocol/format/variable-length-integer) | The number of bytes for `modified_locking_script`. | | modified locking script | `modified_locking_script_length` bytes | bytes[(BE)](/protocol/misc/endian/big) | The subset of the locking script used for signing. See [Modified Locking Script](#modified-locking-script) | | previous output value | 8 bytes | unsigned integer[(LE)](/protocol/misc/endian/little) | The value of the transaction output being spent. |