From 6b87d517081f2fdba6a50b8e7fb9147321def609 Mon Sep 17 00:00:00 2001 From: Jason Dreyzehner Date: Tue, 1 Oct 2024 20:30:46 -0400 Subject: [PATCH] Expand risk assessment, improve intro --- rationale.md | 154 +++++++++++++++++++++------- readme.md | 217 ++++++++++++++++++++++------------------ risk-assessment.md | 135 +++++++++++++++++++++++-- tests-and-benchmarks.md | 23 ++++- 4 files changed, 383 insertions(+), 146 deletions(-) diff --git a/rationale.md b/rationale.md index 8287d07..15ebb5d 100644 --- a/rationale.md +++ b/rationale.md @@ -1,14 +1,50 @@ -## Rationale +# Rationale This section documents design decisions made in this specification. -### Use of Explicitly-Defined Density Limits +
+ +Table of Contents + +- [Use of Explicitly-Defined Density Limits](#use-of-explicitly-defined-density-limits) +- [Exclusion of "Gas System" Behaviors](#exclusion-of-gas-system-behaviors) + - [Global-State Validation Architectures](#global-state-validation-architectures) + - [Stateless Validation Architecture](#stateless-validation-architecture) + - [Minimal Impact of Compute on Node Operation Costs](#minimal-impact-of-compute-on-node-operation-costs) +- [Non-Impact on "Data Storage" Costs and Incentives](#non-impact-on-data-storage-costs-and-incentives) + - [`OP_RETURN` Data Carrier Outputs](#op_return-data-carrier-outputs) + - [Data-Carrying Transactions](#data-carrying-transactions) + - [Indirect, Transaction "Padding" Incentives](#indirect-transaction-padding-incentives) +- [Retention of Control Stack Limit](#retention-of-control-stack-limit) +- [Use of Input Length-Based Densities](#use-of-input-length-based-densities) + - [Alternative: Transaction Length-Based Densities](#alternative-transaction-length-based-densities) + - [Alternative: UTXO-Length Increased Densities](#alternative-utxo-length-increased-densities) + - [Alternative: Transaction Fee-Increased Densities](#alternative-transaction-fee-increased-densities) +- [Selection of Input Length Formula](#selection-of-input-length-formula) +- [Hashing Limit by Digest Iterations](#hashing-limit-by-digest-iterations) +- [Selection of Hashing Limit](#selection-of-hashing-limit) +- [Exclusion of Signing Serialization Components from Hashing Limit](#exclusion-of-signing-serialization-components-from-hashing-limit) + - [Ongoing Value of OP_CODESEPARATOR Operation](#ongoing-value-of-op_codeseparator-operation) +- [Increased Usability of Multisig Stack Clearing](#increased-usability-of-multisig-stack-clearing) +- [Limitation of Pushed Bytes](#limitation-of-pushed-bytes) +- [Unification of Limits into Operation Cost](#unification-of-limits-into-operation-cost) +- [Selection of Operation Cost Limit](#selection-of-operation-cost-limit) +- [Selection of Base Instruction Cost](#selection-of-base-instruction-cost) +- [Inclusion of Numeric Encoding in Operation Costs](#inclusion-of-numeric-encoding-in-operation-costs) +- [Selection of Signature Verification Operation Cost](#selection-of-signature-verification-operation-cost) +- [Continued Availability of Deferred Signature Validation](#continued-availability-of-deferred-signature-validation) +- [Selection of Hash Digest Iteration Cost](#selection-of-hash-digest-iteration-cost) +- [Inclusion of "Notice of Possible Future Expansion"](#inclusion-of-notice-of-possible-future-expansion) + +
+ +## Use of Explicitly-Defined Density Limits This proposal defines explicit limits on the overall density of computation required to validate both transactions and blocks, particularly of [arithmetic](./readme.md#arithmetic-operation-cost), [hashing](./readme.md#hashing-limit), and [signature checking](./readme.md#signature-checking-operation-cost) computations. These explicit limits predictably cap the worst-case cost of validating transactions and blocks across all possible contract constructions and transaction-packing schemes (see [Rationale: Use of Input Length-Based Densities](#use-of-input-length-based-densities) for details). Alternatively, this proposal could continue to indirectly limit the worst-case validation costs of transactions and blocks by specifying a system of one or more constant limits applied at various stages of VM evaluation. At best, such **statically-defined, per-input limit schemes** obfuscate the true limits: determining the real-world, worst-case validation performance of such schemes requires exhaustive enumeration of optimally-packed attack cases. Worse, the precise construction of worst-case attacks becomes significant to real world node performance requirements; any contract construction discovery or network upgrade which reduces contract lengths can magnify an unpredictable set of worst case attacks: e.g. more abusive transactions may be packed into abusive blocks, more abusive inputs may be packed into abusive transactions, fewer abusive transactions are required to exhaust a node's compute capacity, etc. -Finally, because statically-defined, per-input limit schemes fundamentally apply the same limits to 41-byte inputs as they apply to 10,000-byte inputs, the baseline variability of compute requirements ranges from `1` to `243.90` times the worst-case result of any statically-defined system (again, determinable only via exhaustive enumeration)1. In practice, this means that some non-malicious contracts (real users) will be limited – ostensibly to protect the network from abusive use of computation during transaction validation – after using less than 1% of the net computation afforded to intentionally-malicious contracts2. +Finally, because statically-defined, per-input limit schemes fundamentally apply the same limits to 41-byte inputs as they apply to 10,000-byte inputs, the baseline variability of compute requirements ranges from `1` to `243.90` times the worst-case validation (again, determinable only via exhaustive enumeration)1. In practice, this means that some non-malicious contracts (real users) will be limited – ostensibly to protect the network from abusive use of computation during transaction validation – after using less than 1% of the net computation afforded to intentionally-malicious contracts2.
@@ -19,15 +55,15 @@ Finally, because statically-defined, per-input limit schemes fundamentally apply
-### Exclusion of "Gas System" Behaviors +## Exclusion of "Gas System" Behaviors This proposal preserves the Bitcoin Cash network's highly-scalable, **stateless validation architecture**. Bitcoin Cash contract validation remains stateless and cacheable: 1. Contracts can be verified using only data derived from spent UTXOs and the transaction itself, -2. A successful contract validation result can be safely cached without later re-validation, and +2. A successful contract validation result can be cached without later re-validation, and 3. All invalid transactions can be quickly rejected/discarded, with minimal computation requirements for network nodes. -#### Global-State Validation Architectures +### Global-State Validation Architectures The **global-state transaction validation architecture** of other, similarly-capable virtual machines – like the Ethereum Virtual Machine (EVM) – architecturally-prevent those systems from achieving the scalability of Bitcoin Cash: @@ -40,7 +76,7 @@ In these less efficient validation architectures, maintaining acceptable transac 1. **Poor user experiences** – users are forced to accept real-world financial losses for the unpredictability of gas fee markets and/or the technical failures of the network, nodes, contracts, and/or wallet systems. 2. **Financial limitation of contract capabilities** – because the per-computation impact on the entire network of contracts is thousands of higher than on scalable architectures like Bitcoin Cash, global-state systems are forced to charge a meaningful price for each additional computation. In many cases, this regime is so inefficient that it guides contract development away from algorithms which are more correct or provide better user experiences, and toward algorithms which are inexpensive to execute in the networks' virtual-compute pricing ("gas") system. Notably, the connection between virtual-compute pricing and real world performance is tenuous, with the real-world performance differences between various computations sometimes deviating from their charged "gas" prices by greater than an order of magnitude. The dominance of constant-product market makers (CPMMs) are an observable result of this phenomenon, despite other known constructions being superior for certain use cases (e.g. logarithmic market scoring rules). -#### Stateless Validation Architecture +### Stateless Validation Architecture In contrast, this proposal preserves the superior user experience and scalability of Bitcoin Cash: @@ -48,7 +84,7 @@ In contrast, this proposal preserves the superior user experience and scalabilit 2. **No-fee contract deployment and superior real-world privacy** – User wallets both “deploy” and “destroy” contracts incrementally as part of their usage, rather than calling monolithic, expensive-to-deploy, "global" contracts – improving both overall network throughput and real-world privacy. 3. **Reliably-low fees, even for complex contracts** – The per-computation contract validation costs for Bitcoin Cash node operators are thousands of times lower than such costs for node operators in global-state architectures (like Ethereum); existing limits already offer significantly more computation to most Bitcoin Cash contracts than are available to contracts on global-state systems. -#### Minimal Impact of Compute on Node Operation Costs +### Minimal Impact of Compute on Node Operation Costs In practice, computation costs of node operators are typically bounded for a given capacity, and actual utilization of computation has a relatively smaller impact on overall costs (e.g. electricity usage for a single system moving from 5% to 25% utilization). On the other hand, bandwidth and storage more commonly incur direct usage-based costs, with bandwidth commonly cited as the leading infrastructure cost of operation, followed by storage costs of increasing blockchain sizes (e.g. for archival nodes, indexers, and other Bitcoin Cash businesses and infrastructure operators). @@ -56,7 +92,47 @@ Beyond preventing abusive contract constructions from creating denial-of-service By design, this proposal reduces the overall worst-case computation costs of node operation, while significantly extending the universe of relatively inexpensive-to-validate contract constructions available to Bitcoin Cash contract developers. Further changes which might increase worst-case validation costs – like pay-for-compute schemes – are considered out-of-scope for this proposal (see [Use of Input Length-Based Densities](#use-of-input-length-based-densities) for details). -### Retention of Control Stack Limit +## Non-Impact on "Data Storage" Costs and Incentives + +This proposal intentionally avoids impacting the costs and incentives surrounding usage of transactions for storage of arbitrary (often non-financial) data with respect to 1) standard validation rules regarding `OP_RETURN` data carrier outputs, 2) standard validation rules regarding all other data-carrying transactions, and 3) computation-related "data storage" incentives indirectly resulting from all other transaction validation limits. + +### `OP_RETURN` Data Carrier Outputs + +Following [CHIP-2021-03-12 Multiple OP_RETURNs](https://github.com/ActorForth/Auction-Protocol/blob/main/CHIP-2021-03-12_Multiple_OP_RETURN_for_Bitcoin_Cash.md), data carrier outputs are currently limited to 223 cumulative bytes (including encoding overhead), with a maximum content of 220 bytes. While this constant [appears to have been selected by mistake](https://bitcoincashresearch.org/t/raising-the-520-byte-push-limit-201-operation-limit/282/8), it remains the primary limit to accessible, on-chain data storage with multiple existing indexers and other well-supported infrastructure. As increasing this limit could increase indexing and maintenance costs across a variety of ecosystem software and infrastructure, this proposal considers such increases to be out of scope. + +### Data-Carrying Transactions + +Beyond the easily-accessible, well-indexed storage available via [`OP_RETURN` data carrier outputs](#op_return-data-carrier-outputs), the next meaningful limit on (less-accessible) data storage is the limit on maximum standard transaction length (A.K.A. `MAX_STANDARD_TX_SIZE`), currently `100,000` bytes. Up to this length, a variety of custom data storage protocols can be designed to pack data into transactions using a variety of constructions, with limits on individual inputs or push sizes exerting a minimal impact (generally less than 1%) on overall storage efficiency. Beyond `100,000` bytes, real-world costs [discourage usage of on-chain storage](https://bitcoincashresearch.org/t/raising-the-520-byte-push-limit-201-operation-limit/282/15) for most kinds of applications. (Note also that Bitcoin Cash's existing practical limits to data-carrying transactions are either equivalent or more conservative than most other networks resulting from bitcoin splits, i.e. BTC, BSV, and XEC.) + +This proposal has no impact on the potential usage or efficiency of data-carrying transactions. In practice, most applications would remain best served by storing data off-chain and committing only the hash of such data into on-chain transactions (particularly in common, infrastructure-supported [`OP_RETURN` Data Carrier Outputs](#op_return-data-carrier-outputs)). + +### Indirect, Transaction "Padding" Incentives + +This proposal reduces indirect incentives for existing contracts to increase overall blockchain storage usage. + +Because the existing statically-defined limits create arbitrary (i.e. inconsistent with respect to performance requirements) partitions in evaluation of existing contracts – requiring computations to be split across multiple inputs – the net effect of this proposal on all existing contract usage is to eliminate overhead (e.g. additional transactions and/or per-input encoding overhead) which is currently required to work around the existing, per-contract evaluation limits. For all currently-possible contracts, this **maintains or reduces resulting transaction sizes and overall blockchain data usage**. + +However, while this proposal's re-targeted limits allow sufficient room for most contract developers to work without any consideration for VM limits, a subset of currently-possible-but-impractical contracts warrants further review: + +- Within the existing limits, evaluations requiring high-density and/or high-precision arithmetic – e.g. emulation of post-quantum cryptography, zero-knowledge proofs, homomorphic encryption, etc. – must be split into multiple inputs, with intermediate results passed across evaluations to allow computation to continue (working around the existing limits). In practice, the excessive cost of contract development and security review limits production deployment of such multi-input systems. Worse, transactions resulting from such systems would require significant per-input and per-output overhead, reducing their efficiency and increasing their overall impact on blockchain storage usage. + +- It could be argued that this proposal (coupled with [`CHIP-2024-07-BigInt`](https://github.com/bitjson/bch-bigint)) creates an incentive for these currently-theoretical, arithmetic-dense contracts to "pad" their [density control length](./readme.md#density-control-length)s with otherwise-unnecessary data in order to increase the maximum operation cost allocated to the input's evaluation. However, this argument reverses causation; in fact, these proposals so significantly reduce the overhead of arithmetic-dense contracts that many such contracts become efficient enough for realistic production development and deployment. (Note also that these are "economic use cases" related to securing or transacting BCH and/or CashTokens.1) In practice, these proposals produce an unequivocal improvement in the status quo with respect to overall blockchain storage efficiency, and they produce no meaningful change in the incentives surrounding "wasted" data usage. While further improvements are possible (see [Rationale: Use of Input Length-Based Densities](#use-of-input-length-based-densities)), this proposal considers further optimizations to be out of scope. + +
+ +Notes + +1. Even presuming a value judgement that "economic use cases" ought to be prioritized above equally-fee-paying "data storage use cases" (a statement on which this proposal remains neutral), the contracts in question can only reasonably be considered "economic use cases": + + - The purpose of the Bitcoin Cash VM is to adjudicate asset transfer rights without a central authority (for both BCH and CashTokens). + + - In this context, "data storage use cases" might be defined as any use cases in which this adjudication is not used. I.e. instead of relying on the VM for rights adjudication (and by extension, on-chain control of assets), off-chain systems simply use the network as a censorship-resistant messaging layer. Though data storage use cases can technically use the VM in purely-ceremonial ways (e.g. pretending to perform the same on-chain computations as have taken place in the off-chain system), if a system's consensus is reliant on an off-chain protocol, there remains little difference between committing inputs-plus-computation or fully-computed results to the chain (primarily transaction fee efficiency, where committing computed results will be less expensive than committing inputs and ceremonially re-performing the computation on-chain). + + - In contrast, any use case which _does_ require VM evaluation for adjudication of on-chain asset transfer rights (BCH and/or CashTokens), can only reasonably be described as an "economic use case", e.g. single-signature addresses, multi-party vaults, decentralized application treasuries, zero-knowledge proof covenants, sidechain bridges, etc. Within reasonable limits defined to protect network bandwidth, storage, or computation from abusive usage, further distinction between the qualifications of various VM-evaluated contracts as "economic" vs. "non-economic" becomes subjective/arbitrary. + +
+ +## Retention of Control Stack Limit This proposal avoids modifying the existing practical limit on control stack depth by introducing a specific `Control Stack Limit` in place of the current effective limit1. @@ -72,21 +148,27 @@ The existing control stack depth of `100` is already far in excess of all known -### Use of Input Length-Based Densities +## Use of Input Length-Based Densities This proposal limits both [hashing](readme.md#hashing-limit) and [operation cost](readme.md#operation-cost-limit) to maximum densities based on the approximate byte length of the input under evaluation (see [Rationale: Selection of Input Length Formula](#selection-of-input-length-formula)). +As is currently required within the existing system of limits, contract systems requiring additional computation following activation of this proposal may continue to rearrange or [break up expensive computations into multiple inputs](https://bitcoincashresearch.org/t/chip-2021-05-targeted-virtual-machine-limits/437/6). As this proposal attempts to avoid any impact to worst-case validation time – and future upgrades can deploy increases in operation cost limits – additional solutions for increasing per-input operation cost limits are considered out of this proposal's scope. + +### Alternative: Transaction Length-Based Densities + Alternatively, this proposal could measure densities relative to the byte length of the full containing transaction, sharing a budget across all of the transaction's inputs. Such a transaction-based approach would provide contracts with the most generous possible computation limits given the transaction fees paid, allowing computationally-expensive inputs to also claim the computing resources purchased via the bytes of transaction overhead, outputs, and of other inputs. Additionally, if a future upgrade were to [relax output standardness](https://bitcoincashresearch.org/t/p2sh32-a-long-term-solution-for-80-bit-p2sh-collision-attacks/750/13#relaxing-output-standardness-1), transaction-based budgets would also offer non-P2SH (Pay to Script Hash) contracts a greater portion of the computing resources purchased via the transaction's other bytes, particularly for contracts which rely mainly on introspection for validation and include little or no unlocking bytecode (e.g. the ["Merge Threads" script in Jedex](https://github.com/bitjson/jedex)). -However, this proposal's input-based approach is superior in that it: 1) allows contract authors to reliably predict a contract's available limits regardless of the size and other contents of the spending transaction, 2) ensures that contract evaluations which do not exceed VM limits can be composed together in multi-input transactions without further regard for VM limits, 3) preserves the simplicity of parallelizing input validation without cross-thread communication, 4) more conservatively limits the worst-case validation performance of maliciously-invalid transactions (by failing the malicious transaction earlier), and 5) could be safely expanded into a transaction-based approach by a future upgrade if necessary. +However, this proposal's input-based approach is superior in that it: 1) allows contract authors to reliably predict a contract's available limits regardless of the size and other contents of the spending transaction, 2) ensures that contract evaluations which do not exceed VM limits can be composed together in multi-input transactions without further regard for VM limits, 3) preserves the simplicity of parallelizing input validation without cross-thread communication, 4) more conservatively limits the worst-case validation performance of maliciously-invalid transactions (by failing the malicious transaction earlier), and 5) could be expanded into a transaction-based approach by a future upgrade if necessary. + +### Alternative: UTXO-Length Increased Densities Another alternative to offer greater limits to non-P2SH contracts would be to base densities on the byte length of both locking and unlocking bytecode, i.e. including the Unspent Transaction Output (UTXO) in the input's budget. However, this alternative approach would increase the volatility of worst-case transaction validation performance: the price of the UTXO's past bandwidth is paid by the previous transaction's mining fee, while the price of the UTXO's storage is paid only by the time-value of associated dust; if compute resources aren't strictly tied to costs in the present (like the current transaction's mining fee), the instantaneous computation requirements of transaction validation are also not bounded by limits in the present (e.g. slow-to-validate UTXOs may be stored up and evaluated in a smaller number of attack transactions). Finally, depending on the precise economics of particular attacks, limits enhanced by UTXO length may incentivize would-be attackers to inflate the UTXO set in advance of an attempted denial-of-service attack, increasing its cost-effectiveness. Instead, this proposal bases computation limits only on costs paid in the present, with the 41-byte minimum input length providing a reasonable minimum computation budget. +### Alternative: Transaction Fee-Increased Densities + Finally, this proposal could also increase the operation cost limit proportionally to the per-byte mining fee paid or based on the [declaration of "virtual" bytes](https://bitcoincashresearch.org/t/chip-2021-05-targeted-virtual-machine-limits/437/108), e.g. for fee rates of `2` satoshis-per-real-byte, the VM could allow a per-byte budget of `2000` (a `2.5` multiple, incentivizing contracts to pay the higher fee rate rather than simply padding the unlocking bytecode length). However, any allowed increase in per-real-byte operation cost also equivalently changes the variability of per-byte worst-case transaction validation time; such flexibility would need to be conservatively capped and/or interact with the block size limit to ensure predictability of transaction and block validation requirements. Additionally, future research based on real-world usage may support simpler alternatives, e.g. a one-time increase in the per-byte operation cost budget. -As is currently required within the existing system of limits, contract systems requiring additional computation following activation of this proposal may continue to rearrange or [break up expensive computations into multiple inputs](https://bitcoincashresearch.org/t/chip-2021-05-targeted-virtual-machine-limits/437/6). As this proposal attempts to avoid any impact to worst-case validation time – and future upgrades can safely deploy increases in operation cost limits – additional solutions for increasing per-input operation cost limits are considered out of this proposal's scope. - -### Selection of Input Length Formula +## Selection of Input Length Formula The `Density Control Length` used by the [Hashing](readme.md#hashing-limit) and [Operation Cost](readme.md#operation-cost-limit) limits in this proposal approximates the encoded length of inputs by adding the length of the unlocking bytecode (A.K.A. `scriptSig`) to the constant `41`: the minimum possible per-input overhead of version `1` and `2` transactions1. This approach both simplifies implementations – by avoiding the need to either check the length of the serialized input or compute its expected length – and simplifies some contract development and deployment concerns by avoiding the 2-byte change in available budget as contract length increases from `252` bytes (encoded as `0xfc`) to `253` bytes (encoded as `0xfdfd00`). @@ -100,15 +182,15 @@ Additionally, by avoiding direct use of encoded input length in limit calculatio -### Hashing Limit by Digest Iterations +## Hashing Limit by Digest Iterations One possible alternative to the proposed [Hashing Limit](readme.md#hashing-limit) design is to limit evaluations to a fixed count of "bytes hashed" (rather than digest iterations). While this seems simpler, it performs poorly in an adversarial environment: an attacker can choose message sizes to minimize bytes hashed while maximizing digest iterations required (e.g. 1-byte messages). -In practice, because the cost of hashing a 55-byte message is equivalent to the cost of hashing a single-byte message, any hashing limit which fails to account for the internal block-digest behavior of the [Merkle–Damgård](https://en.wikipedia.org/wiki/Merkle%E2%80%93Damg%C3%A5rd_construction) hashing algorithm will be vulnerable to a 55x magnification in worst-case validation costs (see [Digest Iteration Count](./readme.md#digest-iteration-count)). +In practice, because the performance cost of hashing a 55-byte message is equivalent to the performance cost of hashing a single-byte message, any hashing limit which fails to account for the internal block-digest behavior of the [Merkle–Damgård](https://en.wikipedia.org/wiki/Merkle%E2%80%93Damg%C3%A5rd_construction) hashing algorithm will be vulnerable to a 55x magnification in worst-case validation costs (see [Digest Iteration Count](./readme.md#digest-iteration-count)). By directly measuring digest iterations, non-malicious contracts can be allowed to evaluate a higher density of hashing operations without increasing the worst-case validation requirements of the VM as it existed prior to this proposal. -### Selection of Hashing Limit +## Selection of Hashing Limit This proposal sets the standard (transaction relay policy) hashing density limit at `0.5` digest iterations per spending input byte. This value is the asymptotic maximum density of hashing operations in plausibly non-malicious, standard transactions within the current VM limits and instruction set: assuming no transaction encoding overhead and that all hashed material is maximally-padded (1-byte segments), the most efficient methods for reducing hashed material into a single result (without any further processing) requires 1 additional operation (e.g. `OP_CAT`) per hashing operation1. @@ -129,13 +211,13 @@ Alternatively, this proposal could set a consensus hashing limit without a lower -### Exclusion of Signing Serialization Components from Hashing Limit +## Exclusion of Signing Serialization Components from Hashing Limit This proposal includes [the cost of hashing all signing serializations](readme.md#transaction-signature-checking-operations) in the proposed `Hashing Limit`, but it excludes the cost of any hashing required to produce the internal components of signing serializations (i.e. `hashPrevouts`, `hashUtxos`, `hashSequence`, and `hashOutputs`). This configuration reduces protocol complexity while correctly accounting for the real-world hashing cost of `coveredBytecode` (A.K.A. `scriptCode`), particularly in [preventing abuse of `OP_CODESEPARATOR`](https://gist.github.com/markblundeberg/c2c88d25d5f34213830e48d459cbfb44), future increases to maximum standard unlocking bytecode length (A.K.A. `MAX_TX_IN_SCRIPT_SIG_SIZE` – 1,650 bytes), and/or increases to consensus-maximum VM bytecode length (A.K.A. `MAX_SCRIPT_SIZE` – 10,000 bytes). -Alternatively, this proposal could also require internally accounting for the cost of hashing within signing serialization components. However, because components can be easily cached across signature checks – and for `hashPrevouts`, `hashUtxos`, `hashSequence`, across the entire transaction validation – such costs would need to be amortized across all of a transaction's signature checks to approximate their fixed, real-world cost. As signature checking is already sufficiently limited by `SigChecks` and operation cost, omitting hash digest iteration counting of signing serialization components reduces unnecessary protocol complexity. +Alternatively, this proposal could also require internally accounting for the cost of hashing within signing serialization components. However, because components can be cached across signature checks – and for `hashPrevouts`, `hashUtxos`, `hashSequence`, across the entire transaction validation – such costs would need to be amortized across all of a transaction's signature checks to approximate their fixed, real-world cost. As signature checking is already sufficiently limited by `SigChecks` and operation cost, omitting hash digest iteration counting of signing serialization components reduces unnecessary protocol complexity. -#### Ongoing Value of OP_CODESEPARATOR Operation +### Ongoing Value of OP_CODESEPARATOR Operation The `OP_CODESEPARATOR` operation modifies the signing serialization (A.K.A. "`SIGHASH`" preimage) used in later-executed signature operations by truncating the `coveredBytecode` component at the index of the last executed `OP_CODESEPARATOR` operation. @@ -145,13 +227,13 @@ Notably, the behavior of `OP_CODESEPARATOR` can be achieved more directly with ` While past proposals (for both Bitcoin Cash and similar networks) have called for deprecating or disabling `OP_CODESEPARATOR`, this proposal retains the feature while preventing excessive resource consumption, as `OP_CODESEPARATOR` is already available in the Bitcoin Cash instruction set, may have active users, and remains useful in some scenarios. -### Increased Usability of Multisig Stack Clearing +## Increased Usability of Multisig Stack Clearing Prior to this proposal, the multisig operations (`OP_CHECKMULTISIG` and `OP_CHECKMULTISIGVERIFY`) were limited in part via incrementing the operation count by the number of stack items provided as public keys. However, because public keys are only validated prior to their use in a signature check, it is also possible to use these operations to more efficiently drop items from the stack than via repeated calls to `OP_DROP`, `OP_2DROP`, or `OP_NIP`. While this eccentric behavior was likely unintentional, it has been available to contract authors since Bitcoin Cash's 2009 launch, and it is generally the most byte-efficient method of dropping between 9 and 20 stack items (reducing transaction sizes for some contracts). Because this existing stack-clearing behavior is a useful feature and does not meaningfully impact transaction validation costs, this proposal treats zero-signature multisig operations equivalently to all other constant and linear time operations. -### Limitation of Pushed Bytes +## Limitation of Pushed Bytes This proposal limits the density of both memory usage and computation by limiting bytes pushed to the stack to approximately `700` per spending input byte (the per-byte budget of `800` minus the base instruction cost of `100`). @@ -159,7 +241,7 @@ Because stack-pushed bytes become inputs to other operations, limiting the overa Additionally, this proposal’s density-based limit caps the maximum memory and memory bandwidth requirements of validating a large stream of transactions, regardless of the number of parallel validations being performed1. -Alternatively, this proposal could limit memory usage by continuously tracking total stack usage and enforcing some maximum limit. In addition to increasing implementation complexity (e.g. performant implementations would require a usage-tracking stack), this approach would 1) only implicitly limit memory bandwidth usage and 2) require additional limitations on linear-time operations. +Alternatively, this proposal could limit memory usage by continuously tracking total stack usage and enforcing some maximum limit. In addition to increasing implementation complexity (e.g. performant implementations would require a usage-tracking stack), this approach would 1) _implicitly_ limit memory bandwidth usage and 2) require additional limitations on linear-time operations.
@@ -169,11 +251,11 @@ Alternatively, this proposal could limit memory usage by continuously tracking t
-### Unification of Limits into Operation Cost +## Unification of Limits into Operation Cost While this proposal maintains independent limits for both signature checking and hashing (via `SigChecks` and hash digest iterations, respectively), the measurements used to enforce each limit also contribute to the unified `Operation Cost Limit`. This configuration prevents maliciously designed transactions from maximizing validation cost across multiple disparate limits, reducing the potential disparity in worst case performance between honest and malicious usage. -### Selection of Operation Cost Limit +## Selection of Operation Cost Limit This proposal sets the operation cost density at `800` per spending input byte. Given a base instruction cost of `100`, this density ensures a net budget of `700` per spending input byte, a constant rounded up from the maximum effective limit prior to this proposal1. @@ -185,13 +267,13 @@ This proposal sets the operation cost density at `800` per spending input byte. -### Selection of Base Instruction Cost +## Selection of Base Instruction Cost To retain a conservative limit on contract operation density, this proposal sets a base operation cost of `100` per evaluated instruction, including unexecuted and push operations. Given the [operation cost density limit of `800`](readme.md#operation-cost-limit), a base instruction cost of `100` ensures that the maximum operation density within spending inputs (8 per byte) remains within an order of magnitude of the current effective limit (approximately 1 per byte) resulting from the 201 operation limit. -Because base instruction cost can be safely reduced - but not increased - in future upgrades without invalidating contracts1, a base cost of `100` is more conservative than lower values (like `10`, `1`, or `0`). Additionally, a relatively-high base instruction cost leaves room for future upgrades to differentiate the costs of existing low and fixed-cost instructions, if necessary. +Because base instruction cost can be reduced - but not increased - in future upgrades without invalidating contracts1, a base cost of `100` is more conservative than lower values (like `10`, `1`, or `0`). Additionally, a relatively-high base instruction cost leaves room for future upgrades to differentiate the costs of existing low and fixed-cost instructions, if necessary. Alternatively, this proposal could omit any base instruction cost, limiting the cost of all instructions to their impact on the stack. However, instruction evaluation is not costless – a nonzero base cost properly accounts for the real world overhead of evaluating an instruction and verifying non-violation of applicable limits. @@ -205,13 +287,13 @@ Finally, setting an explicit base instruction cost reduces the VM’s implicit r -### Inclusion of Numeric Encoding in Operation Costs +## Inclusion of Numeric Encoding in Operation Costs The VM number format is designed to allow efficient manipulation of arbitrary-precision integers (Satoshi's initial VM implementation used the OpenSSL library's Multiple-Precision Integer format and operations), so sufficiently-optimized VM implementations can theoretically avoid the overhead of decoding and (re)encoding VM numbers. However, if this proposal were to assume zero operation cost for encoding/decoding, this optimization would be required of all performance-critical VM implementations to avoid divergence of real performance from measured operation cost. -Instead, this proposal increments the operation cost of all operations dealing with potentially-large numbers (greater than `2**32`) by the byte length of their numeric output (in effect, doubling the cost of pushing the output). This approach fully accounts for the cost of re-encoding numerical results from another internal representation as VM numbers – regardless of the underlying arithmetic implementation. (Note that the cost of decoding VM number inputs is already accounted for (in advance) by the comprehensive limiting of pushed bytes. See [Rationale: Limitation of Pushed Bytes](#limitation-of-pushed-bytes).) +Instead, this proposal increments the operation cost of all operations dealing with potentially-large numbers (greater than `2**32`) by the byte length of their numeric output (in effect, doubling the cost of pushing the output). This approach accounts for the cost of re-encoding numerical results from another internal representation as VM numbers – regardless of the underlying arithmetic implementation. (Note that the cost of decoding VM number inputs is already accounted for (in advance) by the comprehensive limiting of pushed bytes. See [Rationale: Limitation of Pushed Bytes](#limitation-of-pushed-bytes).) -Because operation cost can be safely reduced - but not increased - in future upgrades without invalidating contracts1, this proposal's approach is considered more conservative than omitting a cost for re-encoding. +Because operation cost can be reduced - but not increased - in future upgrades without invalidating contracts1, this proposal's approach is considered more conservative than omitting a cost for re-encoding.
@@ -221,7 +303,7 @@ Because operation cost can be safely reduced - but not increased - in future upg
-### Selection of Signature Verification Operation Cost +## Selection of Signature Verification Operation Cost In addition to the operation cost of hash digest iterations performed within signature checking operations, this proposal sets the operation cost of signature verification to `26000` per check. This constant is rounded down from the per-check budget available within current limits1. @@ -241,13 +323,13 @@ Given these trade-offs, this proposal takes the more conservative approach of ap -### Continued Availability of Deferred Signature Validation +## Continued Availability of Deferred Signature Validation By using the existing `SigCheck` metric in computing operation cost, this proposal retains the ability to defer signature validation (as described in the [2020-05 SigChecks specification](https://gitlab.com/bitcoin-cash-node/bchn-sw/bitcoincash-upgrade-specifications/-/blob/master/spec/2020-05-15-sigchecks.md)) until the rest of a contract has been evaluated. Notably, enforcement of the proposed hashing limit within signature checking operations can also be deferred until immediately prior to signature validation, though because other hashing operations must be evaluated during program execution, deferring either hashing or digest iteration estimation will not improve worst-case validation performance. -### Selection of Hash Digest Iteration Cost +## Selection of Hash Digest Iteration Cost -For block validation (consensus), this proposal sets the operation cost of hash digest iterations to `64` (the hashing algorithm block size); for transaction relay (standardness policy) this proposal sets the operation cost of hash digest iterations to `192` (`64*3=192`). These values ensure that all possible standard transactions remain valid by consensus while correctly accounting for the cost of hashing during standard validation. +For block validation (consensus), this proposal sets the operation cost of hash digest iterations to `64` (the hashing algorithm block size); for transaction relay (standardness policy) this proposal sets the operation cost of hash digest iterations to `192` (`64*3=192`). These values ensure that all possible standard transactions remain valid by consensus while accounting for the cost of hashing during standard validation. Each VM-supported hashing algorithm – RIPEMD-160, SHA-1, and SHA-256 – uses a [Merkle–Damgård construction](https://en.wikipedia.org/wiki/Merkle%E2%80%93Damg%C3%A5rd_construction) with a `block size` of 512 bits (64 bytes); operation cost is essentially a count of bytes processed by any VM operation, so the hashing algorithm block size (`64`) is the most appropriate multiple upon which to base hashing operation cost calculations. @@ -264,6 +346,6 @@ To determine the standard operation cost of hash digest iterations, the results -### Inclusion of "Notice of Possible Future Expansion" +## Inclusion of "Notice of Possible Future Expansion" -This proposal includes a [Notice of Possible Future Expansion](readme.md#notice-of-possible-future-expansion) to clarify established precedent in Bitcoin (Cash) protocol development: pre-signed transactions, contract systems, and protocols which rely on the non-existence of VM features must be interpreted as intentional. This precedent was established by Satoshi Nakamoto in [`reverted makefile.unix wx-config -- version 0.3.6`](https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node/-/commit/757f0769d8360ea043f469f3a35f6ec204740446) (July 29, 2010) with the addition of new opcodes `OP_NOP1` to `OP_NOP10`, and the precedent was reaffirmed by numerous upgrades to Bitcoin (Cash), e.g. [`OP_CHECKLOCKTIMEVERIFY`](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki), [Relative locktime](https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki), [`OP_CHECKSEQUENCEVERIFY`](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki), the [2018 opcode restoration](https://upgradespecs.bitcoincashnode.org/may-2018-reenabled-opcodes/), [CHIP-2021-03: Bigger Script Integers](https://gitlab.com/GeneralProtocols/research/chips/-/blob/master/CHIP-2021-02-Bigger-Script-Integers.md#notice-of-possible-future-expansion), and later upgrades. +This proposal includes a [Notice of Possible Future Expansion](readme.md#notice-of-possible-future-expansion) to clarify established precedent in Bitcoin (Cash) protocol development: pre-signed transactions, contract systems, and protocols which rely on the non-existence of VM features must be interpreted as intentional. This precedent was established by Satoshi Nakamoto in [`reverted makefile.unix wx-config -- version 0.3.6`](https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node/-/commit/757f0769d8360ea043f469f3a35f6ec204740446) (July 29, 2010) with the addition of new opcodes `OP_NOP1` to `OP_NOP10`, and the precedent was reaffirmed by numerous upgrades to Bitcoin (Cash), e.g. [`OP_CHECKLOCKTIMEVERIFY`](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki), [Relative locktime](https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki), [`OP_CHECKSEQUENCEVERIFY`](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki), the [2018 opcode restoration](https://upgradespecs.bitcoincashnode.org/may-2018-reenabled-opcodes/), [CHIP-2021-03: Bigger Script Integers](https://gitlab.com/GeneralProtocols/research/chips/-/blob/master/CHIP-2021-02-Bigger-Script-Integers.md#notice-of-possible-future-expansion), and other upgrades. diff --git a/readme.md b/readme.md index f1341b8..63f236a 100644 --- a/readme.md +++ b/readme.md @@ -9,9 +9,53 @@ Latest Revision Date: 2024-09-24 Version: 3.1.1 +
+ +Table of Contents + +- [Summary](#summary) +- [Motivation](#motivation) +- [Benefits](#benefits) +- [Deployment](#deployment) +- [Technical Summary](#technical-summary) +- [Technical Specification](#technical-specification) +- [Rationale](#rationale) +- [Tests \& Benchmarks](#tests--benchmarks) +- [Implementations](#implementations) +- [Evaluations of Alternatives](#evaluations-of-alternatives) +- [Risk Assessment](#risk-assessment) +- [Stakeholders \& Statements](#stakeholders--statements) +- [Feedback \& Reviews](#feedback--reviews) +- [Acknowledgements](#acknowledgements) +- [Changelog](#changelog) +- [Copyright](#copyright) + +
+ ## Summary -This proposal replaces several poorly-targeted virtual machine (VM) limits with alternatives that protect against the same malicious cases, while significantly increasing the power of the Bitcoin Cash contract system: +This proposal re-targets virtual machine (VM) limits to enable more advanced Bitcoin Cash contracts, reduce transaction sizes, and reduce full node compute requirements. + +## Motivation & Benefits + +This proposal enables: + +- **More advanced contracts** – The 201 opcode limit and 520-byte, Pay-to-Script-Hash (P2SH) contract length limit each raise the cost of developing Bitcoin Cash products by requiring contract authors to remove important features or otherwise complicate products with harder-to-audit, multi-input systems. Replacing these limits reduces the cost of contract development and security audits. + +- **Larger stack items** – Re-targeted limits enable new post-quantum cryptographic applications, stronger escrow and settlement strategies, larger hash preimages, more practical zero-knowledge proofs, homomorphic encryption, and other important developments for the future security and competitiveness of Bitcoin Cash. + +Additionally, this proposal renders the number length limit (A.K.A. `nMaxNumSize`) unnecessary. Following [cross-implementation verification work](risk-assessment.md#consideration-of-possible-future-changes), an [additional proposal](https://github.com/bitjson/bch-bigint) was created to enable: + +- **Simpler, easier-to-audit, high-precision math** – Lowers the cost of developing, maintaining, and auditing contract systems relying on high-precision math: automated market makers, decentralized exchanges, decentralized stablecoins, collateralized loan protocols, cross-chain and sidechain bridges, and other decentralized financial applications. By allowing Bitcoin Cash to offer contracts maximum-efficiency, native math operations, this proposal would significantly reduce transaction sizes, block space usage, and node CPU utilization vs. existing emulated-math solutions. + +## Deployment + +Deployment of this specification is proposed for the May 2025 upgrade. + +- Activation is proposed for `1731672000` MTP, (`2024-11-15T12:00:00.000Z`) on `chipnet`. +- Activation is proposed for `1747310400` MTP, (`2025-05-15T12:00:00.000Z`) on the BCH network (`mainnet`), `testnet3`, `testnet4`, and `scalenet`. + +## Technical Summary - The 201 operation limit is removed. - The 520-byte stack element length limit is raised to 10,000 bytes, a constant equal to the consensus-maximum VM bytecode length (A.K.A. `MAX_SCRIPT_SIZE`) prior to this proposal. @@ -23,23 +67,22 @@ This proposal **intentionally avoids modifying other existing properties of the - Other existing limits are not modified: - Signature operation count (A.K.A. `SigChecks`) density, - - Maximum cumulative stack and altstack depth (A.K.A. `MAX_STACK_SIZE` – `1000`), - - Maximum standard input bytecode length (A.K.A. `MAX_TX_IN_SCRIPT_SIG_SIZE` – 1,650 bytes), - - Consensus-maximum VM bytecode length (A.K.A. `MAX_SCRIPT_SIZE` – 10,000 bytes), - - Maximum standard transaction byte-length (A.K.A. `MAX_STANDARD_TX_SIZE` – 100,000 bytes), and - - Consensus-maximum transaction byte-length (A.K.A. `MAX_TX_SIZE` – 1,000,000 bytes). -- The cost and incentives around blockchain “data storage” are not measurably affected. -- The worst-case processing and memory requirements of the VM are not measurably affected. + - Maximum cumulative stack and altstack depth (A.K.A. `MAX_STACK_SIZE`; 1000 items), + - Maximum standard input bytecode length (A.K.A. `MAX_TX_IN_SCRIPT_SIG_SIZE`; 1,650 bytes), + - Consensus-maximum VM bytecode length (A.K.A. `MAX_SCRIPT_SIZE`; 10,000 bytes), + - Maximum standard transaction byte-length (A.K.A. `MAX_STANDARD_TX_SIZE`; 100,000 bytes), and + - Consensus-maximum transaction byte-length (A.K.A. `MAX_TX_SIZE`; 1,000,000 bytes). +- The cost and incentives around blockchain “data storage” are [not measurably affected](./rationale.md#non-impact-on-data-storage-costs-and-incentives). +- The worst-case processing and memory requirements of the VM are [not measurably affected](./tests-and-benchmarks.md). -
-Technical Overview of Changes to C++ Clients +### Technical Overview of Changes to C++ Clients The following overview summarizes all changes proposed by this document to C++ implementations following the general structure of the original Satoshi client, e.g. [Bitcoin Cash Node](https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node/-/blob/185f2d64143e807352c0a18f92d8f3ac14bf6840/src/script/interpreter.cpp): 1. `MAX_SCRIPT_ELEMENT_SIZE` increases from `520` to `10000`. 2. `nOpCount` becomes `nOpCost` (used to measure stack-pushed bytes and operation costs) 3. A new `static inline pushstack` is added to match `popstack`; `pushstack` increments `nOpCost` by the item length. -4. `if (opcode > OP_16 && ++nOpCount > MAX_OPS_PER_SCRIPT) { ... }` becomes `nOpCost += 100;` (no conditional, so also added for unexecuted and push operations). +4. `if (opcode > OP_16 && ++nOpCount > MAX_OPS_PER_SCRIPT) { ... }` becomes `nOpCost += 100;` (not conditional, so also added for unexecuted and push operations). 5. `case OP_ROLL:` adds `nOpCost += depth;` 6. `case OP_AND/OP_OR/OP_XOR:` adds `nOpCost += result.size();` 7. `case OP_1ADD...OP_0NOTEQUAL:` adds `nOpCost += bn.size();`, `case OP_ADD...OP_MAX:` adds `nOpCost += bn1.size() + bn2.size();`, `case OP_WITHIN:` adds `nOpCost += bn1.size() + bn2.size() + bn3.size();` @@ -49,70 +92,18 @@ The following overview summarizes all changes proposed by this document to C++ i 11. `SigChecks` limits remain unchanged; similar density checks apply to `nHashDigestIterations` and `nOpCost`. 12. Adds `if (vfExec.size() > 100) { return set_error(...` -
- -## Deployment - -Deployment of this specification is proposed for the May 2025 upgrade. - -- Activation is proposed for `1731672000` MTP, (`2024-11-15T12:00:00.000Z`) on `chipnet`. -- Activation is proposed for `1747310400` MTP, (`2025-05-15T12:00:00.000Z`) on the BCH network (`mainnet`), `testnet3`, `testnet4`, and `scalenet`. - -## Motivation - -Bitcoin Cash contracts are strictly limited to prevent maliciously-designed transactions from requiring excessive resources during transaction validation. Two of these limits are poorly-targeted, unintentionally preventing valuable use cases. - -- The **520-byte stack element length limit** (A.K.A. `MAX_SCRIPT_ELEMENT_SIZE`) currently prevents items longer than 520 bytes from being pushed, `OP_NUM2BIN`ed, or `OP_CAT`ed on to the stack. This also limits the length of Pay-To-Script-Hash (P2SH) redeem bytecode to 520 bytes. -- The **201 operation limit** prevents contracts with more than 201 non-push operations from being relayed on the network or mined in blocks. - -Collectively, existing limits: - -- Cap the maximum density of both hashing and elliptic curve math required for validation of signature checking operations. (See the [2020-05 SigChecks specification](https://gitlab.com/bitcoin-cash-node/bchn-sw/bitcoincash-upgrade-specifications/-/blob/master/spec/2020-05-15-sigchecks.md).) -- Cap the maximum density of hashing required in transaction input validation to approximately `3.44` hash digest iterations per standard spending input byte1. -- Cap the maximum stack usage of transaction input validation to approximately `628` bytes per spending input byte2. -- Cap the depth of the control stack to `100`3. - -While these limits have been sufficient to prevent Denial of Service (DOS) attacks by capping the maximum cost of transaction validation, their current design prevents valuable contract use cases and wastefully increases the size of certain transactions. - -
-Notes - -1. Benchmark `lcennk`. -2. Benchmark `d0kxwp`. -3. The existing `201` operation limit prevents any currently-valid contract from requiring a control stack of depth greater than `100`, e.g. `<1> OP_IF <1> OP_IF <1> OP_IF ... OP_ENDIF OP_ENDIF OP_ENDIF`. - -
- -## Benefits - -By replacing these limits with better-tuned alternatives, the Bitcoin Cash contract system can be made more powerful and efficient, without sacrificing node validation performance. - -### More Advanced Contracts - -The 520-byte stack element length limit prevents additional use cases by limiting the length of Pay-To-Script-Hash (P2SH) contracts, as P2SH redeem bytecode is pushed to the stack prior to evaluation. Additionally, the element length limit also prevents the use of larger hash preimages, e.g. in contracts which inspect parent transactions or utilize `OP_CHECKDATASIG`. - -By raising this limit, more advanced contracts can be supported. - -### More Efficient Contracts - -The 520-byte stack element length limit sometimes requires contract authors to design less byte-efficient contracts in order to fit contract code into 520 bytes. For example, rather than embedding data elements directly in P2SH redeem bytecode, authors may be required to pick and validate data from the unlocking bytecode, wasting transaction space. - -Likewise, both the stack element length limit and the 201 operation limit sometimes require contract authors to offload validation to additional outputs, increasing transaction sizes with otherwise-unnecessary overhead. - -With better-targeted VM limits, many contracts can be made more efficient. - ## Technical Specification -The existing **`Stack Element Size Limit`** is raised; two new limits are introduced: a **`Control Stack Limit`** and a **`Hashing Limit`**; and the **`201 Operation Limit`** is replaced by an **`Operation Cost Limit`**. +The existing **`Stack Element Length Limit`** is raised; two new limits are introduced: a **`Control Stack Limit`** and a **`Hashing Limit`**; and the **`201 Operation Limit`** is replaced by an **`Operation Cost Limit`**. ### Increased Stack Element Length Limit -The existing 520-byte stack element size limit (A.K.A. `MAX_SCRIPT_ELEMENT_SIZE`) is raised to 10,000 bytes. +The existing 520-byte stack element length limit (A.K.A. `MAX_SCRIPT_ELEMENT_SIZE`) is raised to 10,000 bytes.
Note on Maximum Standard Input Bytecode Length -This increases the maximum size of stack elements to be equal to the maximum allowed VM bytecode length (A.K.A. `MAX_SCRIPT_SIZE` – 10,000 bytes). To narrow the scope of this proposal, the maximum standard input bytecode length (A.K.A. `MAX_TX_IN_SCRIPT_SIG_SIZE` – 1,650 bytes) is unchanged. +This increases the maximum length of stack elements to be equal to the maximum allowed VM bytecode length (A.K.A. `MAX_SCRIPT_SIZE` – 10,000 bytes). To narrow the scope of this proposal, the maximum standard input bytecode length (A.K.A. `MAX_TX_IN_SCRIPT_SIG_SIZE` – 1,650 bytes) is unchanged.
@@ -120,15 +111,15 @@ This increases the maximum size of stack elements to be equal to the maximum all To retain the existing limit of `100` on control stack depth, a direct limit is placed on operations which push to the control stack (A.K.A. `vfExec` or `ConditionStack`). See [Rationale: Retention of Control Stack Limit](rationale.md#retention-of-control-stack-limit). -Currently, this limit impacts only the `OP_IF` and `OP_NOTIF` operations. For the purpose of enforcing this limit, `OP_IFDUP` is not considered to internally push to the control stack, i.e. an `OP_IFDUP` executed at a depth of `100` does not violate the `Control Stack Limit`. +This limit impacts only the `OP_IF` and `OP_NOTIF` operations. For the purpose of enforcing this limit, `OP_IFDUP` is not considered to internally push to the control stack, i.e. an `OP_IFDUP` executed at a depth of `100` does not violate the `Control Stack Limit`. ### Hashing Limit -A new limit is placed on `OP_RIPEMD160` (`0xa6`), `OP_SHA1` (`0xa7`), `OP_SHA256` (`0xa8`), `OP_HASH160` (`0xa9`), `OP_HASH256` (`0xaa`), `OP_CHECKSIG` (`0xac`), `OP_CHECKSIGVERIFY` (`0xad`), `OP_CHECKMULTISIG` (`0xae`), `OP_CHECKMULTISIGVERIFY` (`0xaf`), `OP_CHECKDATASIG` (`0xba`), and `OP_CHECKDATASIGVERIFY` (`0xbb`) to prevent excessive hashing function usage. +To prevent excessive hashing function usage, a direct limit is placed on `OP_RIPEMD160` (`0xa6`), `OP_SHA1` (`0xa7`), `OP_SHA256` (`0xa8`), `OP_HASH160` (`0xa9`), `OP_HASH256` (`0xaa`), `OP_CHECKSIG` (`0xac`), `OP_CHECKSIGVERIFY` (`0xad`), `OP_CHECKMULTISIG` (`0xae`), `OP_CHECKMULTISIGVERIFY` (`0xaf`), `OP_CHECKDATASIG` (`0xba`), and `OP_CHECKDATASIGVERIFY` (`0xbb`). -Before a hashing function is performed, its expected cost – in terms of digest iterations – is added to a cumulative total for the transaction input. If the cumulative digest iterations required to validate the input exceed the maximum allowed density, the operation produces an error. See [Rationale: Hashing Limit by Digest Iterations](rationale.md#hashing-limit-by-digest-iterations). +Before a hashing function is performed, its expected cost – in terms of digest iterations – is added to a cumulative total for the transaction input. If the cumulative total for the transaction input exceeds the maximum allowed density, the operation produces an error. See [Rationale: Hashing Limit by Digest Iterations](rationale.md#hashing-limit-by-digest-iterations). -Note that hash digest iterations are cumulative across all evaluation stages: unlocking bytecode, locking bytecode, and redeem bytecode (of P2SH evaluations). This differs from the behavior of the existing operation limit (A.K.A. `nOpCount`), which resets its count to `0` prior to each evaluation stage. +Note that hash digest iterations are cumulative across all evaluation stages of a transaction input: unlocking bytecode, locking bytecode, and redeem bytecode (of P2SH evaluations). This differs from the behavior of the existing operation limit (A.K.A. `nOpCount`), which resets its count to `0` prior to each evaluation stage. #### Density Control Length @@ -136,7 +127,7 @@ The `Density Control Length` is computed by adding the unlocking bytecode length #### Maximum Hashing Density -For standard transactions, the maximum density is `0.5` hash digest iterations per [Density Control Length](#density-control-length) byte; for block validation, the maximum density is `3.5` hash digest iterations per [Density Control Length](#density-control-length) byte. See [Rationale: Selection of Hashing Limit](rationale.md#selection-of-hashing-limit) and [Rationale: Use of Input-Length Based Densities](rationale.md#use-of-input-length-based-densities). +For standard transactions, the maximum density is `0.5` hash digest iterations per [Density Control Length](#density-control-length) byte. For block validation, the maximum density is `3.5` hash digest iterations per [Density Control Length](#density-control-length) byte. See [Rationale: Selection of Hashing Limit](rationale.md#selection-of-hashing-limit) and [Rationale: Use of Input-Length Based Densities](rationale.md#use-of-input-length-based-densities). Given the spending input's `unlocking_bytecode_length` (A.K.A. `scriptSig` length), hash digest iteration limits (`0.5` and `3.5`, respectively) may be calculated with the following C functions: @@ -161,7 +152,7 @@ const maxConsensusIterations = (unlockingBytecodeLength) => -Note that this formula relies on the [Density Control Length](#density-control-length) rather than the precise encoded length of the input. See [Rationale: Selection of Input Length Formula](rationale.md#selection-of-input-length-formula). +Note that this formula relies on the transaction input's [Density Control Length](#density-control-length) rather than the precise encoded length of the input. See [Rationale: Selection of Input Length Formula](rationale.md#selection-of-input-length-formula). #### Digest Iteration Count @@ -190,7 +181,7 @@ const digestIterations = (messageLength, isDouble) =>
Digest Iteration Count Test Vectors -These test vectors reflect the required hash digest iterations for a variety of message sizes, **without accounting for double hashing**. For double-hashed messages, the `Digest Iterations` shown must be further incremented by one. +These test vectors reflect the required hash digest iterations for a variety of message lengths, **without accounting for double hashing**. For double-hashed messages, the `Digest Iterations` shown must be further incremented by one. | Message Length (Bytes) | Digest Iterations | | ---------------------- | ----------------- | @@ -226,7 +217,7 @@ Each VM-supported hashing algorithm – RIPEMD-160, SHA-1, and SHA-256 – uses #### Hashing Operations -The `OP_RIPEMD160` (`0xa6`), `OP_SHA1` (`0xa7`), `OP_SHA256` (`0xa8`), `OP_HASH160` (`0xa9`), and `OP_HASH256` (`0xaa`) operations must compute the expected digest iterations for the length of the message to be hashed, adding the result to the spending transaction input's cumulative count. If the new total exceeds the limit, validation fails. +The `OP_RIPEMD160` (`0xa6`), `OP_SHA1` (`0xa7`), `OP_SHA256` (`0xa8`), `OP_HASH160` (`0xa9`), and `OP_HASH256` (`0xaa`) operations must compute the expected digest iterations for the length of the message to be hashed, adding the result to the spending transaction input's cumulative count. If the new total exceeds the hashing limit, validation fails. Note that evaluations triggering `P2SH20` and `P2SH32` evaluations must also account for the (`2` or more) hash digest iterations required to test validity of the redeem bytecode (`OP_HASH160 <20_bytes> OP_EQUAL` or `OP_HASH256 <32_bytes> OP_EQUAL`, respectively). @@ -239,7 +230,7 @@ The two-round hashing operations – `OP_HASH160` (`0xa9`) and `OP_HASH256` (`0x #### Transaction Signature Checking Operations -Following the assembly of the signing serialization, the `OP_CHECKSIG` (`0xac`), `OP_CHECKSIGVERIFY` (`0xad`), `OP_CHECKMULTISIG` (`0xae`), and `OP_CHECKMULTISIGVERIFY` (`0xaf`) operations must compute the expected digest iterations for the length of the signing serialization (including the iteration in which the final result is double-hashed), adding the result to the spending transaction input's cumulative count. If the new total exceeds the limit, validation fails. +Following the assembly of the signing serialization, the `OP_CHECKSIG` (`0xac`), `OP_CHECKSIGVERIFY` (`0xad`), `OP_CHECKMULTISIG` (`0xae`), and `OP_CHECKMULTISIGVERIFY` (`0xaf`) operations must compute the expected digest iterations for the length of the signing serialization (including the iteration in which the final result is double-hashed), adding the result to the spending transaction input's cumulative count. If the new total exceeds the hashing limit, validation fails. Note that hash digest iterations required to produce components of the signing serialization (i.e. `hashPrevouts`, `hashUtxos`, `hashSequence`, and `hashOutputs`) are excluded from the hashing limit, as implementations should cache these components across signature checks. See [Rationale: Exclusion of Signing Serialization Components from Hashing Limit](rationale.md#exclusion-of-signing-serialization-components-from-hashing-limit). @@ -249,19 +240,6 @@ The `OP_CHECKDATASIG` (`0xba`) and `OP_CHECKDATASIGVERIFY` (`0xbb`) operations m In counting digest iterations, note that these operations perform only a single round of hashing. -### Replacement of Operation Limit - -The existing 201-operation limit per evaluation (A.K.A. `MAX_OPS_PER_SCRIPT`) is removed and replaced by the `Operation Cost Limit`. - -
-Note on Zero-Signature, Multisignature Operations - -Note that prior to this proposal, the `OP_CHECKMULTISIG` (`0xae`) and `OP_CHECKMULTISIGVERIFY` (`0xaf`) operations incremented the operation count by the count of public keys, even if zero signatures are checked; because the total operation count of any contract evaluation was limited to 201 operations, the overall density of `OP_CHECKMULTISIG` and `OP_CHECKMULTISIGVERIFY` was very limited. - -Following the removal of the operation limit, both `OP_CHECKMULTISIG` and `OP_CHECKMULTISIGVERIFY` are limited only by the `SigChecks` and hashing limits; implementations should ensure that evaluations of `OP_CHECKMULTISIG` and `OP_CHECKMULTISIGVERIFY` requiring zero signature checks are sufficiently performant. See [Rationale: Increased Usability of Multisig Stack Clearing](rationale.md#increased-usability-of-multisig-stack-clearing). - -
- ### Operation Cost Limit An `Operation Cost Limit` is introduced, limiting transaction inputs to a cumulative operation cost of `800` per spending [Density Control Length](#density-control-length) byte. See [Rationale: Selection of Operation Cost Limit](rationale.md#selection-of-operation-cost-limit) and [Rationale: Use of Input-Length Based Densities](rationale.md#use-of-input-length-based-densities). @@ -286,8 +264,25 @@ const maxOperationCost = (unlockingBytecodeLength) => Note that this formula relies on the [Density Control Length](#density-control-length) rather than the precise encoded length of the input. See [Rationale: Selection of Input Length Formula](rationale.md#selection-of-input-length-formula). +### Replacement of 201-Operation Limit with Operation Cost Limit + +The existing 201-operation limit per evaluation (A.K.A. `MAX_OPS_PER_SCRIPT`) is removed and replaced by the [`Operation Cost Limit`](#operation-cost-limit). + +
+Note on Zero-Signature, Multisignature Operations + +Note that prior to this proposal, the `OP_CHECKMULTISIG` (`0xae`) and `OP_CHECKMULTISIGVERIFY` (`0xaf`) operations incremented the operation count by the count of public keys, even if zero signatures were checked. Since the total operation count of any contract evaluation was limited to 201 operations, the overall density of `OP_CHECKMULTISIG` and `OP_CHECKMULTISIGVERIFY` was restricted by the remaining operation count in applicable contracts. + +Following the removal of the operation limit, both `OP_CHECKMULTISIG` and `OP_CHECKMULTISIGVERIFY` are limited only by the `SigChecks` and hashing limits; implementations should ensure that evaluations of `OP_CHECKMULTISIG` and `OP_CHECKMULTISIGVERIFY` requiring zero signature checks are sufficiently performant. See [Rationale: Increased Usability of Multisig Stack Clearing](rationale.md#increased-usability-of-multisig-stack-clearing). + +
+ +#### Base Instruction Cost + For each evaluated instruction (including unexecuted and push operations), operation cost is incremented by `100`. See [Rationale: Selection of Base Instruction Cost](rationale.md#selection-of-base-instruction-cost). +#### Cumulative Cost Across Evaluation Stages + Note that operation costs are cumulative across all evaluation stages: unlocking bytecode, locking bytecode, and redeem bytecode (of P2SH evaluations). This differs from the behavior of the existing operation limit (A.K.A. `nOpCount`), which resets its count to `0` prior to each evaluation stage. #### Measurement of Stack-Pushed Bytes @@ -313,7 +308,7 @@ For example, `<'a'> <'b'> <'c'> <2> OP_ROLL` (producing `<'b'> <'c'> <'a'>`) is #### Arithmetic Operation Cost -To conservatively account for the cost of encoding VM numbers, the sum of all numeric output lengths with the potential to exceed `2**32` are added to the operation cost of all operations with such outputs: `OP_1ADD` (`0x8b`), `OP_1SUB` (`0x8c`), `OP_NEGATE` (`0x8f`), `OP_ABS` (`0x90`), `OP_NOT` (`0x91`), `OP_0NOTEQUAL` (`0x92`), `OP_ADD` (`0x93`), `OP_SUB` (`0x94`), `OP_MUL` (`0x95`), `OP_DIV` (`0x96`), `OP_MOD` (`0x97`), `OP_BOOLAND` (`0x9a`), `OP_BOOLOR` (`0x9b`), `OP_NUMEQUAL` (`0x9c`), `OP_NUMEQUALVERIFY` (`0x9d`), `OP_NUMNOTEQUAL` (`0x9e`), `OP_LESSTHAN` (`0x9f`), `OP_GREATERTHAN` (`0xa0`), `OP_LESSTHANOREQUAL` (`0xa1`), `OP_GREATERTHANOREQUAL` (`0xa2`), `OP_MIN` (`0xa3`), `OP_MAX` (`0xa4`), and `OP_WITHIN` (`0xa5`); e.g. given terms `a b -> c` (such as in ` OP_ADD`), the operation cost is: the base cost (`100`), plus the cost of re-encoding the output (`c.length`), plus the byte length of the result (`c.length`), for a final formula of `100 + (2 * c.length)`. See [Rationale: Inclusion of Numeric Encoding in Operation Costs](rationale.md#inclusion-of-numeric-encoding-in-operation-costs). +To account for the cost of encoding VM numbers, the sum of all numeric output lengths with the potential to exceed `2**32` are added to the operation cost of all operations with such outputs: `OP_1ADD` (`0x8b`), `OP_1SUB` (`0x8c`), `OP_NEGATE` (`0x8f`), `OP_ABS` (`0x90`), `OP_NOT` (`0x91`), `OP_0NOTEQUAL` (`0x92`), `OP_ADD` (`0x93`), `OP_SUB` (`0x94`), `OP_MUL` (`0x95`), `OP_DIV` (`0x96`), `OP_MOD` (`0x97`), `OP_BOOLAND` (`0x9a`), `OP_BOOLOR` (`0x9b`), `OP_NUMEQUAL` (`0x9c`), `OP_NUMEQUALVERIFY` (`0x9d`), `OP_NUMNOTEQUAL` (`0x9e`), `OP_LESSTHAN` (`0x9f`), `OP_GREATERTHAN` (`0xa0`), `OP_LESSTHANOREQUAL` (`0xa1`), `OP_GREATERTHANOREQUAL` (`0xa2`), `OP_MIN` (`0xa3`), `OP_MAX` (`0xa4`), and `OP_WITHIN` (`0xa5`); e.g. given terms `a b -> c` (such as in ` OP_ADD`), the operation cost is: the base cost (`100`), plus the cost of re-encoding the output (`c.length`), plus the byte length of the result (`c.length`), for a final formula of `100 + (2 * c.length)`. See [Rationale: Inclusion of Numeric Encoding in Operation Costs](rationale.md#inclusion-of-numeric-encoding-in-operation-costs). To account for O(n2) worst-case performance, the operation cost of `OP_MUL` (`0x95`), `OP_DIV` (`0x96`), and `OP_MOD` (`0x97`) are also increased by the product of their input lengths, i.e. given terms `a b -> c`, the operation cost is: the base cost (`100`) plus the cost of re-encoding the output (`c.length`), plus the byte length of the result (`c.length`), plus the product of the input lengths (`a.length * b.length`), for a final formula of `100 + (2 * c.length) + (a.length * b.length)`. @@ -331,13 +326,13 @@ All operations which increase the cumulative total of `SigChecks` (as defined by While unusual, it is possible to design pre-signed transactions, contract systems, and protocols which rely on the rejection of otherwise-valid transactions that exceed current VM limits. Contract authors are advised that future upgrades may further expand VM limits by increasing allowable operation cost density, reducing the accounted cost of particular operations, or otherwise. -**This proposal interprets such failure-reliant constructions as intentional** – they are designed to fail unless/until a possible future network upgrade in which such limits are increased, e.g. upgrade-activation futures contracts. See [Rationale: Inclusion of "Notice of Possible Future Expansion"](rationale.md#inclusion-of-notice-of-possible-future-expansion). +**This proposal interprets such failure-reliant constructions as intentional** – the constructions are designed to fail unless/until a possible future network upgrade in which such limits are increased, e.g. upgrade-activation futures contracts. See [Rationale: Inclusion of "Notice of Possible Future Expansion"](rationale.md#inclusion-of-notice-of-possible-future-expansion).
Notes -As always, the security of a contract is the responsibility of the entity locking funds in that contract; funds can always be locked in insecure contracts (e.g. "Anyone-Can-Spend addresses"). This notice is provided to warn contract authors and explicitly codify a network policy: the possible existence of poorly-designed contracts will not preclude future upgrades from further expanding VM limits. +The security of a contract is the responsibility of the entity locking funds in that contract; funds can always be locked in insecure contracts (e.g. "Anyone-Can-Spend addresses"). This notice is provided to warn contract authors and explicitly codify a network policy: the possible existence of poorly-designed contracts will not preclude future upgrades from further expanding VM limits. To ensure an otherwise-valid transaction will always fail when some limit is exceeded (in the rare case that such a behavior is desirable), that behavior must be either 1) explicitly validated or 2) introduced to the protocol or contract system in question prior to the activation of any future upgrade which expands the requisite limit. @@ -345,7 +340,7 @@ A similar notice also appeared in [CHIP-2021-03: Bigger Script Integers](https:/
-### Rationale +## Rationale - [Appendix: Rationale →](rationale.md#rationale) - [Use of Explicitly-Defined Density Limits](rationale.md#use-of-explicitly-defined-density-limits) @@ -353,11 +348,18 @@ A similar notice also appeared in [CHIP-2021-03: Bigger Script Integers](https:/ - [Global-State Validation Architectures](rationale.md#global-state-validation-architectures) - [Stateless Validation Architecture](rationale.md#stateless-validation-architecture) - [Minimal Impact of Compute on Node Operation Costs](rationale.md#minimal-impact-of-compute-on-node-operation-costs) - - [Retention of Control Stack Limit](#retention-of-control-stack-limit) + - [Non-Impact on "Data Storage" Costs and Incentives](rationale.md#non-impact-on-data-storage-costs-and-incentives) + - [`OP_RETURN` Data Carrier Outputs](rationale.md#op_return-data-carrier-outputs) + - [Data-Carrying Transactions](rationale.md#data-carrying-transactions) + - [Indirect, Transaction "Padding" Incentives](rationale.md#indirect-transaction-padding-incentives) + - [Retention of Control Stack Limit](rationale.md#retention-of-control-stack-limit) - [Use of Input Length-Based Densities](rationale.md#use-of-input-length-based-densities) + - [Alternative: Transaction Length-Based Densities](rationale.md#alternative-transaction-length-based-densities) + - [Alternative: UTXO-Length Increased Densities](rationale.md#alternative-utxo-length-increased-densities) + - [Alternative: Transaction Fee-Increased Densities](rationale.md#alternative-transaction-fee-increased-densities) - [Selection of Input Length Formula](rationale.md#selection-of-input-length-formula) - [Hashing Limit by Digest Iterations](rationale.md#hashing-limit-by-digest-iterations) - - [Selection of Hashing Limit](#selection-of-hashing-limit) + - [Selection of Hashing Limit](rationale.md#selection-of-hashing-limit) - [Exclusion of Signing Serialization Components from Hashing Limit](rationale.md#exclusion-of-signing-serialization-components-from-hashing-limit) - [Ongoing Value of OP_CODESEPARATOR Operation](rationale.md#ongoing-value-of-op_codeseparator-operation) - [Increased Usability of Multisig Stack Clearing](rationale.md#increased-usability-of-multisig-stack-clearing) @@ -373,7 +375,12 @@ A similar notice also appeared in [CHIP-2021-03: Bigger Script Integers](https:/ ## Tests & Benchmarks -This proposal includes a suite of functional tests and benchmarks to verify the performance of all operations within virtual machine implementations. See [Tests & Benchmarks](./tests-and-benchmarks.md) for details. +This proposal includes a suite of functional tests and benchmarks to verify the performance of all operations within virtual machine implementations. + +- [Appendix: Tests & Benchmarks →](tests-and-benchmarks.md#rationale) + - [Testing](tests-and-benchmarks.md#testing) + - [Benchmarks](tests-and-benchmarks.md#benchmarks) + - [Evaluation of Results](tests-and-benchmarks.md#evaluation-of-results) ## Implementations @@ -386,10 +393,12 @@ Please see the following reference implementations for additional examples and t - [Bitauth IDE](https://github.com/bitauth/bitauth-ide) – An online IDE for bitcoin (cash) contracts. [Pull Request #101](https://github.com/bitauth/bitauth-ide/pull/101). - Go: - [BCHD](https://bchd.cash/) – An alternative full node bitcoin cash implementation written in Go (golang). [OPReturnCode/bchd PR #1](https://github.com/OPReturnCode/bchd/pull/1) +- Java: + - [Bitcoin Verde](https://bitcoinverde.org/) – Bitcoin Verde is a Java full-node implementation of the Bitcoin Cash protocol. Bitcoin Verde provides a block explorer, development library, and network implementation diversification. [Issue #24](https://github.com/SoftwareVerde/bitcoin-verde/issues/24) ## Evaluations of Alternatives -This proposal comprehensively evaluates notable alternatives for each design decision made in the technical specification. Reviewed alternatives are enumerated here for ease of review: +This proposal evaluates notable alternatives for each design decision made in the technical specification. Reviewed alternatives are enumerated here for ease of review: - **Statically-Defined, Per-Input Limits** – Rather than this proposal's explicitly-specified limits, an alternative limits system could attempt to indirectly limit worst-case validation costs by specifying fixed per-input constant limits. This alternative would 1) create a minimum of ~244x variability in worst-case validation costs and 2) obfuscate the true limits such that worst-case costs are not predictable without exhaustive enumeration of optimally-packed attack cases. See [Rationale: Use of Explicitly-Defined Density Limits](./rationale.md#use-of-explicitly-defined-density-limits) and [Rationale: Use of Density-Based Limits](./rationale.md#use-of-density-based-limits) for details. @@ -432,6 +441,16 @@ This proposal comprehensively evaluates notable alternatives for each design dec - [`Chipnet` Preview Activation](risk-assessment.md#chipnet-preview-activation) - [Denial-of-Service (DoS) Risks](risk-assessment.md#denial-of-service-dos-risks) - [Expanded Node Performance Safety Margin](risk-assessment.md#expanded-node-performance-safety-margin) + - [Protocol Complexity Risks](risk-assessment.md#protocol-complexity-risks) + - [Support for Post-Activation Simplification](risk-assessment.md#support-for-post-activation-simplification) + - [Continuation of Nonstandard Invalidation Precedent](risk-assessment.md#continuation-of-nonstandard-invalidation-precedent) + - [Consideration of Possible Future Changes](risk-assessment.md#consideration-of-possible-future-changes) + - [Upgrade Costs](risk-assessment.md#upgrade-costs) + - [Node Upgrade Costs](risk-assessment.md#node-upgrade-costs) + - [Ecosystem Upgrade Costs](risk-assessment.md#ecosystem-upgrade-costs) + - [Maintenance Costs](risk-assessment.md#maintenance-costs) + - [Node Maintenance Costs](risk-assessment.md#node-maintenance-costs) + - [Ecosystem Maintenance Costs](risk-assessment.md#ecosystem-maintenance-costs) ## Stakeholders & Statements @@ -441,6 +460,11 @@ This proposal comprehensively evaluates notable alternatives for each design dec - [`Raising the 520 byte push limit & 201 operation limit` – Feb 8, 2021 | bitcoincashresearch.org](https://bitcoincashresearch.org/t/raising-the-520-byte-push-limit-201-operation-limit/282) - [`CHIP: Targeted Virtual Machine Limits` – May 12, 2021 | bitcoincashresearch.org](https://bitcoincashresearch.org/t/chip-targeted-virtual-machine-limits/437) +- [The BCH Podcast #122: VM Limits feat. Jason Dreyzehner](https://www.youtube.com/watch?v=Gy7uxpcyIl8) +- [Jason Dreyzehner on X: I proposed the Limits & BigInt CHIPs, Ask Me Anything](https://x.com/bitjson/status/1839361677759811611) +- [On r/btc: I proposed the Limits & BigInt CHIPs, Ask Me Anything –bitjson](https://old.reddit.com/r/btc/comments/1fq2hab/i_proposed_the_limits_bigint_chips_for_the_may/) +- [The Bitcoin Cash Podcast #130: Limits & BigInt CHIPs feat. Jason Dreyzehner](https://www.youtube.com/watch?v=dTMcgW1iHrU) +- [General Protocol Spaces (34): General Bull 34 of n - Jason Dreyzehner AMA](https://www.youtube.com/watch?v=xNQK3Ula2Ao) ## Changelog @@ -452,6 +476,7 @@ This section summarizes the evolution of this document. - Add latest test vectors and benchmarks ([#7](https://github.com/bitjson/bch-vm-limits/issues/7)) - Add [Rationale: Use of Explicitly-Defined Density Limits](./rationale.md#use-of-explicitly-defined-density-limits) ([#28](https://github.com/bitjson/bch-vm-limits/issues/28)) - Clarify explanation of hash digest iteration formula ([#29](https://github.com/bitjson/bch-vm-limits/issues/29)) + - Add [Rationale: Non-Impact on "Data Storage" Costs and Incentives](rationale.md#non-impact-on-data-storage-costs-and-incentives) ([#18](https://github.com/bitjson/bch-vm-limits/issues/18)) - **v3.1.0** ([`35dc2c52`](https://github.com/bitjson/bch-vm-limits/commit/35dc2c5210bb34dc5255c0613a6665edff07d6c0)) - Base densities on input length rather than transaction length ([#21](https://github.com/bitjson/bch-vm-limits/issues/21)) - Include numeric encoding in operation costs ([#20](https://github.com/bitjson/bch-vm-limits/issues/20)) diff --git a/risk-assessment.md b/risk-assessment.md index c33c324..2725104 100644 --- a/risk-assessment.md +++ b/risk-assessment.md @@ -2,31 +2,58 @@ The following security considerations, potential risks, and costs have been reviewed to verify the safety and advisability of this proposal. +
+ +Table of Contents + +- [Risks \& Security Considerations](#risks--security-considerations) + - [User Impact Risks](#user-impact-risks) + - [Reduced or Equivalent Node Validation Costs](#reduced-or-equivalent-node-validation-costs) + - [Increased or Equivalent Contract Capabilities](#increased-or-equivalent-contract-capabilities) + - [Consensus Risks](#consensus-risks) + - [Full-Transaction Test Vectors](#full-transaction-test-vectors) + - [New Performance Testing Methodology](#new-performance-testing-methodology) + - [`Chipnet` Preview Activation](#chipnet-preview-activation) + - [Denial-of-Service (DoS) Risks](#denial-of-service-dos-risks) + - [Expanded Node Performance Safety Margin](#expanded-node-performance-safety-margin) + - [Protocol Complexity Risks](#protocol-complexity-risks) + - [Support for Post-Activation Simplification](#support-for-post-activation-simplification) + - [Continuation of Nonstandard Invalidation Precedent](#continuation-of-nonstandard-invalidation-precedent) + - [Consideration of Possible Future Changes](#consideration-of-possible-future-changes) +- [Upgrade Costs](#upgrade-costs) + - [Node Upgrade Costs](#node-upgrade-costs) + - [Ecosystem Upgrade Costs](#ecosystem-upgrade-costs) +- [Maintenance Costs](#maintenance-costs) + - [Node Maintenance Costs](#node-maintenance-costs) + - [Ecosystem Maintenance Costs](#ecosystem-maintenance-costs) + +
+ ## Risks & Security Considerations -This section reviews the foreseeable security implications of the proposed changes to the Bitcoin Cash network. Key technical considerations include user impact risks, consensus risks, denial-of-service (DoS) risks, and risks to protocol complexity and ongoing maintenance burden of newly introduced behavior. +This section reviews the foreseeable security implications of the proposed changes to the Bitcoin Cash network. Key technical considerations include user impact risks, consensus risks, denial-of-service (DoS) risks, and risks to protocol complexity or maintenance burden of newly introduced behavior. ### User Impact Risks All upgrade proposals must carefully analyze proposed changes for potential impacts to existing Bitcoin Cash users and use cases. Virtual Machine (VM) upgrades can impact node operators and blockchain indexers (and therefore payment processors, exchanges, and other businesses), software development libraries, wallets, decentralized applications, and a wide range of pre-signed transactions, contract systems, and transaction-settled protocols. -This proposal is carefully designed to preserve backwards-compatibility along a variety of dimensions, minimizing user impact risk. +This proposal is designed to preserve backwards-compatibility along a variety of dimensions, minimizing user impact risks: #### Reduced or Equivalent Node Validation Costs -By carefully establishing all re-targeted limits at levels "rounded up" from their existing practical limits, this proposal minimizes the risk of increase to worst-case validation performance, preventing any increase in node operation costs. See [Selection of Operation Cost Limit](./rationale.md#selection-of-operation-cost-limit), [Selection of Base Instruction Cost](./rationale.md#selection-of-base-instruction-cost), [Selection of Hash Digest Iteration Cost](./rationale.md#selection-of-hash-digest-iteration-cost), and [Selection of Signature Verification Operation Cost](./rationale.md#selection-of-signature-verification-operation-cost). Further, this risk mitigation strategy has been empirically verified across multiple implementations using a wide range of [cross-implementation functional tests and performance benchmarks](./tests-and-benchmarks.md). +By establishing all re-targeted limits at levels "rounded up" from their existing practical limits, this proposal minimizes the risk of increase to worst-case validation performance, preventing any increase in node operation costs. See [Selection of Operation Cost Limit](./rationale.md#selection-of-operation-cost-limit), [Selection of Base Instruction Cost](./rationale.md#selection-of-base-instruction-cost), [Selection of Hash Digest Iteration Cost](./rationale.md#selection-of-hash-digest-iteration-cost), and [Selection of Signature Verification Operation Cost](./rationale.md#selection-of-signature-verification-operation-cost). Further, this risk mitigation strategy has been empirically verified across multiple implementations using a wide range of [cross-implementation functional tests and performance benchmarks](./tests-and-benchmarks.md). -Finally, even if these precautions were to fail in preventing or revealing a significant performance issue in a widely-used VM implementation, by design, this proposal expands an already [10x to 100x safety margin in performance-critical implementations](#expanded-node-performance-safety-margin). +Finally, even if these precautions were to fail in preventing a performance issue or revealing an existing performance issue in a widely-used VM implementation, by design, this proposal expands an already [10x to 100x safety margin in performance-critical implementations](#expanded-node-performance-safety-margin). #### Increased or Equivalent Contract Capabilities Because the proposed, re-targeted limits are equal to or greater than their current practical equivalents, existing contract systems, decentralized applications, pre-signed transactions, and transaction-settled protocols have minimal risk of negative impact (in many cases, these users are the primary beneficiaries of increased Bitcoin Cash VM capabilities). By design, all currently-standard transactions remain valid following activation of this proposal. -Finally, this proposal highlights and evaluates the risk of impact to a smaller, hypothetical set of users who might have intentionally designed limit-failure-reliant constructions, see [Notice of Possible Future Expansion](/readme.md#notice-of-possible-future-expansion) and [Rationale: Inclusion of "Notice of Possible Future Expansion"](./rationale.md). +Finally, this proposal highlights and evaluates the risk of impact to a smaller, hypothetical set of users who might have intentionally designed limit-failure-reliant constructions, see [Notice of Possible Future Expansion](/readme.md#notice-of-possible-future-expansion) and [Rationale: Inclusion of "Notice of Possible Future Expansion"](./rationale.md#inclusion-of-notice-of-possible-future-expansion). ### Consensus Risks -All network consensus upgrade proposals must account for consensus risks arising from incorrect or inconsistent implementation of consensus-critical changes. For VM upgrades, consensus risks primarily apply to node implementations and other software which performs VM evaluation as part of transaction validation. +All network consensus upgrade proposals must account for consensus risks arising from incorrect or inconsistent implementation of consensus-critical changes. For Virtual Machine (VM) upgrades, consensus risks primarily apply to node implementations and other software which performs VM evaluation as part of transaction validation. This proposal mitigates consensus risks via 1) an extensive set of full-transaction test vectors, 2) a new cross-implementation performance testing methodology, and 3) a 6-month early activation on `chipnet`. @@ -38,7 +65,7 @@ In developing these test vectors, a workflow for test vector development has bee #### New Performance Testing Methodology -Though this proposal is carefully designed to both [reduce worst-case node validation costs](#reduced-or-equivalent-node-validation-costs) and [expand the node validation performance safety margin](#expanded-node-performance-safety-margin), this proposal also standardizes a new performance benchmarking methodology to help detect performance irregularities and regressions in specific node implementations and VM-evaluating software. +Though this proposal is designed to both [reduce worst-case node validation costs](#reduced-or-equivalent-node-validation-costs) and [expand the node validation performance safety margin](#expanded-node-performance-safety-margin), this proposal also standardizes a new performance benchmarking methodology to help detect performance irregularities and regressions in specific node implementations and VM-evaluating software. In addition to purely functional tests, hundreds of carefully-designed worst-case performance benchmarks are integrated in the unified set of [tests and benchmarks](./tests-and-benchmarks.md), and a rigorous, cross-implementation performance testing strategy has been specified, with carefully selected metrics, file formats/structures, and an established methodology for [sharing results](./tests-and-benchmarks.md#evaluation-of-results) across implementations. @@ -58,16 +85,102 @@ All network consensus upgrade proposals which alter system limits carry risks re Beyond other precautions taken by this proposal ([maximally-conservative limits](#reduced-or-equivalent-node-validation-costs) and [extensive performance benchmarking](./tests-and-benchmarks.md)) – denial-of-service risks are further reduced by an expanded margin of safety for transaction and block validation performance in hashing-expensive scenarios. -This proposal sets a standard hashing limit at `0.5` iterations per byte: the asymptotic maximum density of hashing operations in [plausibly non-malicious, standard transactions](/rationale.md#selection-of-hashing-limit). This limit is `7x` lower than the currently-standard `3.5` iterations per byte practical limit; as such, the standard-validation margin of safety in implementations with relatively-slow hashing performance (or a hashing-related performance regression) is increased `~7x` by this proposal. Additionally, though worst case nonstandard performance is [already limited by mining economics](./tests-and-benchmarks.md#standard-vs-nonstandard-vms), the similarly-improved margin of safety on nonstandard hashing reduces risks for some node operators and increases the predictability of worst-case block validation performance. +This proposal sets a standard hashing limit at `0.5` iterations per byte: the asymptotic maximum density of hashing operations in [plausibly non-malicious, standard transactions](/rationale.md#selection-of-hashing-limit). This limit is ~`7x` lower than the currently-standard ~`3.5` iterations per byte practical limit; as such, the standard-validation margin of safety in implementations with relatively-slow hashing performance (or a hashing-related performance regression) is increased `~7x` by this proposal. Additionally, though worst case nonstandard performance is [already limited by mining economics](./tests-and-benchmarks.md#standard-vs-nonstandard-vms), the similarly-improved margin of safety on nonstandard hashing reduces risks for some node operators and increases the predictability of worst-case block validation performance. -Finally, note that this proposal also maintains Bitcoin Cash's existing margin of safety for all other transaction and block validation scenarios. At a maximum block size of `32MB` (increasing by no more than 2x per year), and a worst-case validation performance ([`packed 1-of-3 ECDSA BMS, bottom slot`](vmb_tests/bch_2023_standard/core.benchmarks.signature-checking.bms-ecdsa.standard_stats.csv)) of `4` to `5` times the compute-density of the [baseline benchmark](./tests-and-benchmarks.md#baseline-benchmark) for performance-optimized implementations like [Bitcoin Cash Node](./tests-and-benchmarks.md#bitcoin-cash-node-c), even relatively low-powered **consumer hardware has excess compute capacity of `10x` to `100x` the worst-case computation requirements** of fully verifying all Bitcoin Cash transactions and blocks1. +Finally, note that this proposal also maintains Bitcoin Cash's existing margin of safety for all other transaction and block validation scenarios. At a maximum block size of `32MB` (increasing by no more than 2x per year), and a worst-case standard validation performance ([`packed 1-of-3 ECDSA BMS, bottom slot`](vmb_tests/bch_2023_standard/core.benchmarks.signature-checking.bms-ecdsa.standard_stats.csv)) of `4` to `5` times the compute-density of the [baseline benchmark](./tests-and-benchmarks.md#baseline-benchmark) for performance-optimized implementations like [Bitcoin Cash Node](./tests-and-benchmarks.md#bitcoin-cash-node-c), even relatively low-powered **consumer hardware has excess compute capacity of `10x` to `100x` the worst-case computation requirements** of fully verifying all Bitcoin Cash transactions and blocks1. -Due to this node performance safety margin, in practice, any DOS attack attempting to create unusually-high CPU utilization across the network would likely be indistinguishable by most network participants from any other burst of low CPU-utilization transactions (e.g. transactions dominated by `OP_RETURN` data carrier outputs). In this scenario, the more notable costs for honest network nodes remain bandwidth and storage costs (and likewise from the attackers' perspective, transaction mining fees). See [Exclusion of "Gas System" Behaviors](./rationale.md#exclusion-of-gas-system-behaviors). +Due to this node performance safety margin, in practice, any DOS attack attempting to create unusually-high CPU utilization across the network would likely be indistinguishable by most network participants from any other burst of low CPU-utilization transactions (e.g. transactions dominated by [`OP_RETURN` data carrier outputs](./rationale.md#op_return-data-carrier-outputs)). In this scenario, the more notable costs for honest network nodes remain bandwidth and storage costs (and likewise from the attackers' perspective, transaction mining fees). See [Rationale: Minimal Impact of Compute on Node Operation Costs](./rationale.md#minimal-impact-of-compute-on-node-operation-costs).
Notes -1. [Bitcoin Cash Node](./tests-and-benchmarks.md#bitcoin-cash-node-c) on typical consumer CPUs can verify the baseline benchmark at approximately `10k` transactions per second, per core, caching disabled (see [benchmarks](./tests-and-benchmarks.md)). Given a maximum block size of `32MB`, worst-case baseline-relative validation performance of `4` to `5`, and the baseline transaction length (`366` bytes), worst-case required performance is approximately `350k` to `440k` baseline validations (`(32,000,000 * 5)/366 = 349,726`; `(32,000,000 * 5)/366 = 437,158`). Given a typical single-core validation speed of approximately `10k tx/core/second`, worst-case block times of `5` to `10` minutes (`300` to `600` seconds), and parallelism between `2` and `8`, a very conservative safety margin for VM performance is on the order of `10` to `100` (worst case: `300 seconds per block / (~440k / (10k/s * 2 cores)) ~= 13.7x`; more typical case: `600 seconds per block / (~350k / (10k/s * 8 cores) ~= 137x`). +1. [Bitcoin Cash Node](./tests-and-benchmarks.md#bitcoin-cash-node-c) on typical consumer CPUs can verify the baseline benchmark at approximately `10k` transactions per second, per core, caching disabled (see [benchmarks](./tests-and-benchmarks.md)). Given a maximum block size of `32MB`, worst-case baseline-relative standard validation performance of `4` to `5`, and the baseline transaction length (`366` bytes), worst-case required performance is approximately `350k` to `440k` baseline validations (`(32,000,000 * 5)/366 = 349,726`; `(32,000,000 * 5)/366 = 437,158`). Given a typical single-core validation speed of approximately `10k tx/core/second`, worst-case block times of `5` to `10` minutes (`300` to `600` seconds), and parallelism between `2` and `8`, a very conservative safety margin for VM performance is on the order of `10` to `100` (worst case: `300 seconds per block / (~440k / (10k/s * 2 cores)) ~= 13.7x`; more typical case: `600 seconds per block / (~350k / (10k/s * 8 cores) ~= 137x`).
+ +### Protocol Complexity Risks + +All upgrade proposals must carefully analyze proposed changes for both immediate and potential future impacts on overall protocol complexity. This proposal has been reviewed to ensure that all changes are 1) minimal, 2) necessary, and 3) avoid creating technical debt, even if future upgrades were to further expand limits or expand the VM's capabilities in a number of analyzed scenarios. + +#### Support for Post-Activation Simplification + +This proposal carefully re-targets limits in backwards-compatible ways. By design, all transactions which were previously standard (acceptable for transaction relay) remain valid in this proposal's re-targeted system. As a result, significant portions of activation code and historical code used to enforce the old limits can be safely removed following activation. + +#### Continuation of Nonstandard Invalidation Precedent + +To minimize technical debt and worst-case nonstandard validation performance, this proposal invalidates two sets of currently-nonstandard transactions (transactions which are rejected by transaction relay validation, but accepted by block validation): 1) nonstandard transactions which violate the nonstandard hashing limit, and 2) nonstandard transactions with one or more input(s) which exceed their operation cost limit. + +Nonstandard transactions of the first type are [demonstrably malicious within the current limit system](./rationale.md#selection-of-hashing-limit). However, a space of plausibly non-malicious contracts exists within the second type: contracts with spending code paths in which unlocking bytecode length is less than approximately 201 bytes and operation cost exceeds the input's limit (based on its [density control length](./readme.md#density-control-length)). + +While this proposal could attempt to create limit exemption(s) for code paths in this plausibly non-malicious space, such exemptions would mislead users concerning longstanding precedent: nonstandard transaction validation is used to safely deprecate categories of previously-standard transactions (e.g. to remedy contract security or resource exhaustion risks) – only standard transactions should ever be expected to remain valid across network upgrades. E.g. [Pay-to-Script-Hash (P2SH)](https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki) (2012), [`OP_CHECKLOCKTIMEVERIFY`](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki) (2014), [Strict DER Signatures](https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki) (2015), [`OP_CHECKSEQUENCEVERIFY`](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki) (2015), [`LOW_S` and `NULLFAIL`](https://github.com/bitcoin/bips/blob/master/bip-0146.mediawiki) (2016), [`OP_CHECKMULTISIG[VERIFY] NULLDUMMY`](https://github.com/bitcoin/bips/blob/master/bip-0147.mediawiki) (2016), [User Activated Hard Fork](https://reference.cash/protocol/forks/bch-uahf) (2017), [`BCH_2018_11`](https://github.com/bitcoincashorg/bitcoincash.org/blob/master/spec/2018-nov-upgrade.md), [`BCH_2019_05`](https://github.com/bitcoincashorg/bitcoincash.org/blob/master/spec/2019-05-15-schnorr.md), [`MINIMALDATA`](https://github.com/bitcoincashorg/bitcoincash.org/blob/master/spec/2019-11-15-minimaldata.md) (2019), and [`SigChecks`](https://github.com/bitcoincashorg/bitcoincash.org/blob/master/spec/2020-05-15-sigchecks.md) (2020), [CHIP 2021-01 Restrict Transaction Version](https://gitlab.com/bitcoin.cash/chips/-/blob/master/CHIP-2021-01-Restrict%20Transaction%20Versions.md), [CHIP-2022-02 CashTokens](https://github.com/cashtokens/cashtokens), and [CHIP-2022-05 P2SH32](https://gitlab.com/0353F40E/p2sh32/). + +In practice, the nonstandard invalidation precedent has served a critical role in 1) reducing the impact of many contract security, consensus, and node resource exhaustion risks, 2) protecting real users across upgrades by ensuring a safe deprecation path for behavior that unintentionally relies on contract system or network vulnerabilities, and 3) enabling protocol enhancements that may have otherwise been impractical without significantly increased protocol complexity if previously-nonstandard validation behaviors were recast as "real-user" usage rather than as a deprecation and safety feature. + +To preserve this important precedent for future upgrades, this proposal intentionally omits special considerations for any previously-nonstandard usage. By design, in specifying nonstandard validation behavior following the upgrade, only previously-standard validation can safely be considered within scope. By extension, it should be noted that any abusive behavior made nonstandard by this proposal is a candidate for full invalidation in future upgrades. + +#### Consideration of Possible Future Changes + +Finally, to minimize future protocol complexity, this proposal carefully reviews the potential impact of all re-targeted limits on a variety of possible future upgrades: + +- **Increased stack item length limit** – The re-targeted limits prevent significant increases in worst-case performance following an upgrade in which the `10,000`-byte stack item length limit (A.K.A. `MAX_SCRIPT_ELEMENT_SIZE`) is further increased. See [Rationale: Limitation of Pushed Bytes](./rationale.md#limitation-of-pushed-bytes). + +- **Increased contract length limits** – The re-targeted limits prevent significant increases in worst-case performance following an upgrade in which the `10,000`-byte contract length limit (A.K.A. `MAX_SCRIPT_SIZE`) is increased or eliminated in favor of cumulative transaction size limits. See [Rationale: Use of Explicitly-Defined Density Limits](./rationale.md#use-of-explicitly-defined-density-limits), [Rationale: Selection of Base Instruction Cost](./rationale.md#selection-of-base-instruction-cost) + +- **Increased transaction length limits** – The re-targeted limits prevent significant increases in worst-case performance following an upgrade in which the `100,000`-byte standard transaction length limit (A.K.A. `MAX_STANDARD_TX_SIZE`) or `1,000,000`-byte nonstandard transaction length limit (A.K.A. `MAX_TX_SIZE`) are increased. See [Rationale: Use of Explicitly-Defined Density Limits](./rationale.md#use-of-explicitly-defined-density-limits). + +- **Increased number length limit** – The re-targeted limits immediately eliminate the need for a number length limit (A.K.A. `nMaxNumSize`). Following review by multiple VM implementations, a companion proposal was created to simultaneously remove the number length limit: [CHIP-2024-07-BigInt: High-Precision Arithmetic for Bitcoin Cash](https://github.com/bitjson/bch-bigint). + +- **Addition of new opcodes** – Because the re-targeted limits create a unified "cost" metric for operations of significantly varying performance, new opcodes can be safely added to the VM by specifying only the new operation's cost formula, minimizing future upgrades' potential performance and protocol complexity impact. See [Rationale: Use of Explicitly-Defined Density Limits](./rationale.md#use-of-explicitly-defined-density-limits), [Rationale: Unification of Limits into Operation Cost](./rationale.md#unification-of-limits-into-operation-cost), + +- **Compatibility with additional control flow structures** – By limiting the density of pushed bytes and the density of evaluated operations, this proposal's re-targeted limits remain effective even if future upgrades significantly expand the VMs control flow capabilities: definite or indefinite loops, pushed bytecode evaluation, word/function definition and evaluation, or other structures. Further, the worst-case performance of contracts using such control flow structures can be safely decoupled from the contract length limit, allowing greater flexibility, e.g. loops which safely evaluate more instructions than would otherwise be possible to fit within contract length limits. See [Rationale: Retention of Control Stack Limit](./rationale.md#retention-of-control-stack-limit), [Rationale: Limitation of Pushed Bytes](./rationale.md#limitation-of-pushed-bytes), and [Rationale: Selection of Base Instruction Cost](./rationale.md#selection-of-base-instruction-cost). + +- **Increased operation density** – By establishing a relatively-high [Base Instruction Cost](./readme.md#base-instruction-cost), this proposal retains a similar ([within one order of magnitude](./rationale.md#selection-of-base-instruction-cost)) limit on operation density as under the existing limits. However, future upgrades may easily raise this limit by reducing the base instruction cost, requiring more careful optimization of instruction evaluation in all VM implementations while enabling a reduction in transaction sizes for operation-dense contracts. See [Rationale: Limitation of Pushed Bytes](./rationale.md#limitation-of-pushed-bytes) and [Rationale: Selection of Base Instruction Cost](./rationale.md#selection-of-base-instruction-cost). + +- **Increased per-operation overhead** (selectively-decreased operation density) - Additionally, by establishing a relatively-high [Base Instruction Cost](./readme.md#base-instruction-cost), this proposal retains an ability for future upgrades to differentiate the overhead cost of existing "low-cost" operations (costs nearly equal to the base instruction cost) by up to two orders of magnitude, without increasing worst-case validation performance, by selectively lowering only the base instruction cost of operations to be sorted into the new, "faster" set of operations. See [Rationale: Selection of Base Instruction Cost](./rationale.md#selection-of-base-instruction-cost). + +- **Relaxation of output standardness** – The re-targeted limits eliminate differences in worst-case performance between Pay-to-Script-Hash (P2SH) and non-P2SH contracts, simplifying potential future upgrades in which [output standardness is relaxed](https://bitcoincashresearch.org/t/relaxing-output-standardness/1391) to allow some custom, non-P2SH contracts within standard validation (e.g. such that output standardness could be determined only by UTXO length rather than by parsed UTXO contents). See [Rationale: Use of Explicitly-Defined Density Limits](./rationale.md#use-of-explicitly-defined-density-limits) and [Rationale: Use of Input Length-Based Densities](./rationale.md#use-of-input-length-based-densities), + +## Upgrade Costs + +This section reviews the costs of implementing the proposed changes. + +### Node Upgrade Costs + +This proposal affects consensus – all fully-validating node software must implement these Virtual Machine (VM) changes to remain in consensus. To minimize the cost of validating node implementation upgrades, this proposal includes a wide range of [cross-implementation functional tests and performance benchmarks](./tests-and-benchmarks.md). + +Additionally, to minimize implementation costs, this proposal has been carefully designed to avoid requiring several potential optimizations in performance-critical VM implementations: + +- **Optimized control stack** – This proposal allows – but does not require – VM implementations to implement an O(1) Control Stack. See [Rationale: Retention of Control Stack Limit](./rationale.md#retention-of-control-stack-limit). +- **Minimal-cost VM number encoding/decoding** – This proposal allows – but does not require – VM implementations to optimize the encoding or decoding of VM numbers between the consensus-enforced number encoding and whichever internal format is used by the implementation's high-precision arithmetic operations. This ensures that a wide variety of high-precision arithmetic libraries and built-in language features can be easily used in the development of performance-critical VM implementations. See [Rationale: Inclusion of Numeric Encoding in Operation Costs](./rationale.md#inclusion-of-numeric-encoding-in-operation-costs). +- **Batching of Schnorr signature verification** – This proposal allows – but does not require – VM implementations to optimize performance of Schnorr signature verification by deferring signature validation and batching the verification of Schnorr signatures. See [Rationale: Selection of Signature Verification Operation Cost](./rationale.md#selection-of-signature-verification-operation-cost) and [Rationale: Continued Availability of Deferred Signature Validation](./rationale.md#continued-availability-of-deferred-signature-validation). + +### Ecosystem Upgrade Costs + +Upgrade costs for the wider ecosystem – wallets, blockchain indexers, mining, mining-monitoring, and mining-administrative software, etc. are limited: + +- This proposal **does not create new user-facing primitives** (e.g. [CashTokens, 2023](https://cashtokens.org/)) which might demand significant changes or additional features in wallets, block explorers, or other ecosystem software. + +- This proposal **does not notably impact user or miner-visible metrics** (e.g. [Adaptive BlockSize Limit Algorithm, 2024](https://gitlab.com/0353F40E/ebaa#chip-2023-04-adaptive-blocksize-limit-algorithm-for-bitcoin-cash)) which might demand significant changes in software like block explorers, mining calculators, and proprietary mining software and tooling, as well as user-facing documentation and marketing materials referencing the well-known, Blocksize Limit metric. + +While all Bitcoin Cash software systems must inevitably be upgraded to follow all consensus upgrades (e.g. a company must annually upgrade to the latest version of their preferred full node implementation), ecosystem upgrade costs for this proposal are essentially confined to specialized software, tooling, and documentation for contract developers. Further, as the fundamental aim of this proposal is to expand the power and flexibility of Bitcoin Cash's VM for these stakeholders, significant portions of these costs were already speculatively paid, in hopes of later activation, during the creation and review of this proposal. + +## Maintenance Costs + +All network upgrade proposals must evaluate their foreseeable impact on maintenance costs. Virtual Machine (VM) upgrades can increase the risks of [consensus divergence](#consensus-risks), [performance issues](#denial-of-service-dos-risks), and increased [implementation complexity](#protocol-complexity-risks). This section reviews foreseeable ongoing costs following activation of the proposed changes. + +### Node Maintenance Costs + +This proposal minimizes any negative impact on node maintenance costs: + +- **Decreased or equivalent worst-case validation cost** – By design, this proposal establishes all re-targeted limits at levels which avoid any notable increase in worst-case validation cost, and in some cases, significantly decrease worst-case cost. This reduces ongoing node maintenance costs by reducing the potential impact of implementation-specific performance issues. + +- **New functional tests and performance benchmarks** – This proposal also significantly improves the long-term maintainability of most node implementations by contributing both an enhanced set of [full transaction test vectors](#full-transaction-test-vectors) a new [cross-implementation performance testing methodology](#new-performance-testing-methodology) to help root out existing or future performance issues in specific implementations. + +- **Potential increase in project maintenance resources** – By increasing the power and flexibility of Bitcoin Cash's VM for contract development, this proposal has the potential to attract further resources to node and VM implementations, both to 1) enhance performance of implementations primarily aimed at mining and/or business infrastructure, and 2) enhance the feature sets of alternative implementations aimed at contract development, decentralized application design, and/or specialized wallet management. + +### Ecosystem Maintenance Costs + +As with [ecosystem upgrade costs](#ecosystem-upgrade-costs), maintenance costs for the wider ecosystem – wallets, blockchain indexers, mining, mining-monitoring, and mining-administrative software, etc. are limited: this proposal **does not create new user-facing primitives**, and it **does not notably impact user or miner-visible metrics**. See [Ecosystem Upgrade Costs](#ecosystem-upgrade-costs). + +Features of this proposal which are relevant to [node maintenance costs](#node-maintenance-costs) are also relevant to the rest of the ecosystem: 1) decreased or equivalent worst-case validation cost, 2) new functional tests and performance benchmarks, and a 3) potential increase in project maintenance resources. diff --git a/tests-and-benchmarks.md b/tests-and-benchmarks.md index a1e7375..58c453e 100644 --- a/tests-and-benchmarks.md +++ b/tests-and-benchmarks.md @@ -5,6 +5,23 @@ 1. a full, encoded test transaction, and 2. An encoded set of UTXOs with which to validate the transaction. +
+ +Table of Contents + +- [Testing](#testing) + - [Standard Vs. Nonstandard VMs](#standard-vs-nonstandard-vms) + - [VMB Test Contents](#vmb-test-contents) +- [Benchmarks](#benchmarks) + - [Baseline Benchmark](#baseline-benchmark) +- [Evaluation of Results](#evaluation-of-results) + - [Bitcoin Cash Node (C++)](#bitcoin-cash-node-c) + - [Verification (BCHN)](#verification-bchn) + - [Libauth (JavaScript)](#libauth-javascript) + - [Verification (Libauth)](#verification-libauth) + +
+ ## Testing For ease of use, VMB tests are divided into directories by annual Virtual Machine (VM) version and expected evaluation result. All tests in each directory must be evaluated against both VM modes for that annual VM version, and validation performance measured relative to the [baseline test]() @@ -233,14 +250,14 @@ The following worst-case results for each category have been extracted from the
-### Verification +### Verification (BCHN) To independently reproduce the [provided data sets](./benchmarks/) on a new target system, execute the following commands on the system to be tested: ```sh git clone https://gitlab.com/cculianu/bitcoin-cash-node.git cd bitcoin-cash-node -git checkout wip_new_vmb_bench_vmlimits_bigint +git checkout merged_vmlimits_bigint_with_updated_benches mkdir build cd build cmake -GNinja .. -DBUILD_BITCOIN_WALLET=OFF -DBUILD_BITCOIN_QT=OFF -DENABLE_NATPMP=OFF -DENABLE_MAN=OFF -DCMAKE_BUILD_TYPE=Release @@ -264,7 +281,7 @@ Libauth duplicates significant quantities of data for each evaluated contract op ![Copy-on-Write, Contract-Debugging VM Performance (for Web Browser-Based Developer Tooling)](./benchmarks/libauth-results.png) -### Verification +### Verification (Libauth) To independently reproduce the [provided data sets](./benchmarks/) on a new target system, execute the following commands on the system to be tested: