Replace the m_buckets unsorted map with a lock-free version
based on atomic pointers. (BucketMap)
remove the m_leafs and move those into the bucket struct.
Make the access to the jumptable transactional to avoid one big lock
over all datastructures.
On my threadripper 2990WX the entire 150GB BCH blockchain was
parsed and imported in under 3 hours.
As the miners have started using the block-version bits for their
own purposes this means that BIP9 checks are rather irrelevant.
Signalling using version bits is rather silly anyway.
In Flowee there will no longer be warnings generated based on
these now outdated assumptions.
And this means less code :)
During the BCH fork-off time we allowed the client to be started
as either BCH or BTC and as such there are several code-paths
that behave different based on this user setting.
We remove those as we no longer allow starting as BTC client.
This also removes the check for the min-block-size (rollback protection)
as this is accomplished by the checkpoints on the BCH chain.
At least 50% of the UTXO entries reuse the posOnDisk
because they are outputs on the same tx. This changes
such entries from using 2 bytes to 1 byte only when saved
in a bucket.
On low core-count machines the request to save could end up being
delayed since all threads are busy doing things like validating
transactions.
So when a really big block came in I ended up throttling while there
was no save method running in parallel at all.
This fixes this and also makes throtteling be a per-thread thing
instead of doing it inside the mutex.
When reorgs greater than 6 are detected we refrain from taking
action (rolling back the chain) and wait for either user interaction
or till the chain extends the chain-tip again.
The current system writes based on the amount of changes, which is a bit
risky as long as we have small blocks as the amount of changes may for
hours be below the "lets write" limit.
So also write every 5 minutes, just to make sure we won't keep data in
memory longer than needed. Also this allows pruning to happen for people
that don't run their node all the time.
we only save a subsection of stuff in memory on each round of saving,
but I set the counter to zero on how long to wait for the next save.
This makes us save more often till we actually saved most stuff, avoiding
buildups of caches until we do the periodic big flush.
TomZ