For the txid/fee index we do a bit more than we can do for shallow block
validation.
Now we don't actually fully validate all transactions when this index is
enabled, we just fetch the fees.
The code that connects blocks together uses the block-hash (blockId) for
that, which we always just created on the fly.
Instead store the blockId on the state and in that way make adding
headers not entirely in-sequence be 10 times as fast.
Additionally, we wait with actual validation after a reindex until the
files are all found because otherwise we can't save the meta blocks.
this allows the user to ensure the metadata blocks fetch the previous
transaction in order to calculate the transaction-fee.
Normally this is skipped for transactions we don't need to do the script
validation for (old blocks).
[code-quality]
We got a warning about a dereference of a nullptr. But that implied we
have an impossible combination of member variables.
So, lets assert on that being impossible so;
a) we get a fail-fast if this ever happens.
b) the static code checker will stop warning us.
Notice that asserts are compiled out of release executables. So this is
effectively a no-op for releases.
The CAmount name is not helpful as its just an int64_t and not a
class, like the name implies. There were a handful of places where
it was passed in as const-ref, as a good example of this actually
creating sub-par code.
This adds a unit test to see if the combination of invalidateBlock and
reconsiderBlock do what we expect to do.
The main issue was that we store an invalidated block in the UTXO and we
forgot to re-validate that on reconsider.
Additionally I avoid writing a lot of unchanged data to the block-index.
and update the definitions of the Mainnet, Testnet (v3), and Regtest chain params accordingly. The heights for all 3 chains were taken by verifying the values in both BU and BCHN code bases.
NOTE: For TestNet v3, there is a 1 block discrepancy between BU (1378461) and BCHN (1378460) for the height of the May 2020 hardfork. I arbitrarily selected the value from BCHN, I'm not sure which is correct.
This is part of the protocol upgrade for 2020-05-15, and in general it
seems to go the direction of "we did this before, lets do this again".
The spec is clear enough, but there is still a lack of questioning and
testing. The problem this attempts to fix has been neutered for years[1].
The spec states:
> The essential idea of SigChecks is to perform counting solely in the
> spending transaction, and count actual executed signature check
> operations.
This, however nobel and logical, ignores that the
check-for-being-too-costly just pulled in a UTXO lookup and the loading
of the output script from the historical chain.
The goal that we protect against CPU over-use may be reached, but the
price is a total system slowdown. You can have multiple CPUs, but the
bus to permanent storage has one, max 2 parallel pipes.
To ensure theHub stays the number one scalable node, I didn't blindly
follow the spec, while making sure that the Hub is correctly going to
follow/reject consensus violations of newly mined blocks.
As a result the implementation in Flowee the Hub:
* does not check sigcheck-counts on historical blocks (more than 1000
blocks in the past).
This may increase the risk of chain-splits ever so slightly, but the cost
of disk-IO would be too high.
* No longer stores the value in the mempool, nor uses it for the
CPU-miner.
* Ties the sigcheck-limits to the user-set block-size-accept-limit.
This is contrary to the spec which mistakenly thinks that BCH has a
max block-size in the consensus rules. The effect is the same, though.
* The per-intput standardness suggestion is not implemented because
standardness checks don't currently fetch the previous outputs and
that would be too expensive to add.
* Standardness rules for the whole transaction are moved to the
mempool-acceptance logic instead. The cost would be too great
otherwise, similar to the previous point.
Again, the effect is the same as likely intented.
---
1) since the intro of the CachingTransactionSignatureChecker
It was only called twice, and in very close proximity. The class didn't
add anything.
This improves readability and with the new state its easier to write
too.
Merkle-block and merkle-tree classes and methods are pretty much stand-
alone and can be moved with no efforts.
Also move the relevent unit test to qtestlib.
The API changes in boost between 1.66 and later was the need
for the boost_compat.h header file.
Its been a long time since Flowee started demanding 1.67 minimum
for boost, making this compat obsolete.
When the UTXO saved checkpoints this change makes sure we also store the
index-db changes.
Since we stopped saving simple state changes fromt the index-db, the
only real data we save is the 'undo-block-index', as such this will be
relatively cheap to save.
Without an undo block position we will currently fail to verify those
blocks and as such it is useful to save all at the same time to actually
have a state we can start from.
In some unit tests I noticed that we write a block that was just loaded
from disk, this check avoids this overhead.
Not sure how relevant this is for normal operations.
The recovering of orphans was recursive and that meant there was a max
length of headers we could process with a gap in the chain due to normal
stack-depth for recursivity (approx 50k).
As headers are being provided to us from external peers this could be a
DOS vector.
This implementation avoids this problem by not being recursive.
To allow the UTXO to actually use the power of checkpoints we need to
make sure that the block-validation state is not stored separately from
it.
The goal is that when we have some curruption we can just go back to an
earlier state of the UTXO and re-validate the blocks to get to the
current tip. The often seen problem is that corruption will instead
leave the block-index (leveldb) with an incorrect state so the replay
fails.
This change solves that by no longer reading the block-validation-state
and no longer writing it on a state change.
This improves the double spend proof orphans code.
Also add a DSProof log-category and lots of log lines to make
looking at a debug build much more fun.
tomFlowee
TomZ