Small refactor; move the partition check away from main and the obsolete
global variables we used to use, instead now just add it to the DB which
owns the data it works on.
This fixes the bug that in various cases we'd incorrectly get a warning
about no blocks being found in the last N hours.
This is part of the protocol upgrade for 2020-05-15, and in general it
seems to go the direction of "we did this before, lets do this again".
The spec is clear enough, but there is still a lack of questioning and
testing. The problem this attempts to fix has been neutered for years[1].
The spec states:
> The essential idea of SigChecks is to perform counting solely in the
> spending transaction, and count actual executed signature check
> operations.
This, however nobel and logical, ignores that the
check-for-being-too-costly just pulled in a UTXO lookup and the loading
of the output script from the historical chain.
The goal that we protect against CPU over-use may be reached, but the
price is a total system slowdown. You can have multiple CPUs, but the
bus to permanent storage has one, max 2 parallel pipes.
To ensure theHub stays the number one scalable node, I didn't blindly
follow the spec, while making sure that the Hub is correctly going to
follow/reject consensus violations of newly mined blocks.
As a result the implementation in Flowee the Hub:
* does not check sigcheck-counts on historical blocks (more than 1000
blocks in the past).
This may increase the risk of chain-splits ever so slightly, but the cost
of disk-IO would be too high.
* No longer stores the value in the mempool, nor uses it for the
CPU-miner.
* Ties the sigcheck-limits to the user-set block-size-accept-limit.
This is contrary to the spec which mistakenly thinks that BCH has a
max block-size in the consensus rules. The effect is the same, though.
* The per-intput standardness suggestion is not implemented because
standardness checks don't currently fetch the previous outputs and
that would be too expensive to add.
* Standardness rules for the whole transaction are moved to the
mempool-acceptance logic instead. The cost would be too great
otherwise, similar to the previous point.
Again, the effect is the same as likely intented.
---
1) since the intro of the CachingTransactionSignatureChecker
When, on loading, the blockindex and the UTXO don't agree then try to find an older UTXO
state where they do agree.
The most typical state issue is where a block stored in the blocksdb is not available in
the index due to corruption or similar.
To allow the UTXO to actually use the power of checkpoints we need to
make sure that the block-validation state is not stored separately from
it.
The goal is that when we have some curruption we can just go back to an
earlier state of the UTXO and re-validate the blocks to get to the
current tip. The often seen problem is that corruption will instead
leave the block-index (leveldb) with an incorrect state so the replay
fails.
This change solves that by no longer reading the block-validation-state
and no longer writing it on a state change.
Move some globals and all stuff pulling in crypto to the
server/serverutil.cpp file
Remove dead code.
Move several items that were used in only one place to the respective
files using them.
Move the class WaitUntilFinishedHelper into its own file.
Made sure no header includes utils.h
remove a handfull of files including utils.h for nothing.
This removes the attempt at magic (predicting future) component
called the fee-estimator.
The direct effect is that all transactions have as a mempool acceptance
the same minimum fee. Regardless of how full the mempool or blocks were.
This mempool-acceptance minimum fee is 0 sat/byte. (aka free)
Notice the node-wide free-tx accept-limiter is going to avoid us getting
overloaded.
The RPC / HTTP servers were logging about being shutdown even if
they never were started in the first place. This confused people.
Also removed some empty methods.
Remove txIndex option, its move to Flowee Indexer
Also remove the GUI option for setting the amount
of "script validator threads". :D
Remove DBCash command line option too, equally obsolete.
During the BCH fork-off time we allowed the client to be started
as either BCH or BTC and as such there are several code-paths
that behave different based on this user setting.
We remove those as we no longer allow starting as BTC client.
This also removes the check for the min-block-size (rollback protection)
as this is accomplished by the checkpoints on the BCH chain.
The current system writes based on the amount of changes, which is a bit
risky as long as we have small blocks as the amount of changes may for
hours be below the "lets write" limit.
So also write every 5 minutes, just to make sure we won't keep data in
memory longer than needed. Also this allows pruning to happen for people
that don't run their node all the time.
This change makes the limits be static variables, which means they are
only settable once for a process. The only usecase so far is to use much
smaller limits in testing situations.
Instead of having a magic 10 open files, we now have a scheduled
run that closes all mapped files 30 seconds after last usage.
This allows many files to be opened with extremely low overhead and
cost (they are not actually loaded until needed, and even then only
per 4K page).
Actually delete the asio io_service when we are done with it since that
will delete any left over tasks in the queue.
This makes deletion order more predictable.
The new UnspentOutputDatabase classes are only very loosly a database, we
purely register and store unspent outputs there. But unline a DB we don't
allow modification (just insert and delete).
This replaces the coin-db (which was based on leveldb) and as first goal
this gives us a higher level of stability. The level-database was known
to give corruption issues.
Notice that users will need to do a manual reindex on first update.
* logs.conf now also found directly in the homedir if there is no
$HOME/.config dir
* default filename is now 'hub.log' instead of 'debug.log'
* logs.conf now also detects ALL as a keyword for section
* logs.conf now also supports 'option path [path]'
So, now we install the example config file
Then we follow XDG for retrieving it.
We first check XDG_CONFIG (~/.config/flowee) and if we can't find a confg
file there fall back to the data dir (~/.local/share/flowee).
Get rid of the weird boost signals setup
Move it to a new library 'interfaces' which is
for stuff that has no dependencies and can avoid
creating inter-library or cicular dependencies.
TomZ