Makes the change from yesterday complete: we now
compress all the chunks into one periodically.
This, again, is done with a timeout instead of on-close.
We have 45 seconds as a timeout to balance between need to save and
waiting long enough to get additional headers should it
have been a while since last run.
This change makes it so that we rely less on the downloaded
blockheader data being only in memory and us saving it when
the client shuts down.
Instead, we save more regularly and we save smaller files
instead of always overwriting the complete downloaded headers.
Having less to save at shutdown is then a happy side-effect.
What we did previously is load a static chain which was to avoid lots of
work by simply memory-mapping the data instead of inserting it in a
vector.
This improves upon that by also having a metadata file which holds the
block-hashes directly readable. This avoids us having to iterate over
the blockheaders and hashing them.
Additionally we now have a single chain-work field that is the total
amount added work for the headers. This saves us from doing 256-bit
divisions and work for each blockheader as well.
Result is that we make the statup mostly about memory moving and remove
all CPU intensive stuff. Going from 1450ms to 400ms on desktop.
I expect an even greater gain on mobile CPUs.
A peer that is behind is marked as "on a different chain", which is
ultimately correct, but the response to punish or ban them is too much.
We could very well reconnect later when the situation is resolved.
So, detect when a peer is simply behind and respond by disconnecting
without punishment.
Instead of assigning a privacy segment at connect, wait until we have
determined if this peer is on the same chain as us before we allocate a
peer slot.
This also helps us limit the amount of peers we send our bloom filter to.
This is done in several steps:
1. Separate my height from the remote peer heights.
Instead of assuming we have one height, recognize that a peer may
not be at the tip at the same time we are. We monitor headers/invs
to update the 'peerHeight' variable.
2. Ask for merkle blocks from a peer to the maximum height of that
peer (but not later than what we validated to be true).
This avoids us asking past the remotes tip which they didn't like.
3. Redo the SyncSPVAction to use all this and make it much more
reliable in finding peers to download from and getting all the
changes as fast as possible.
The NetworkManager usage was mostly for low connection counts and this
made defaults selection easy.
With more usages it is important to allow the NWM-connection to be more
configurable about memory usage and leaner in general.
This changes the headers-buffers (used to create envelopes) to not be per
connection anymore but per thread using the tread_local keyword.
This changes the ring-buffers to become configurable using
NetworkConnections::setMessageQueueSizes().
Also removing some include statements where they were not really needed
in the P2PNet lib.
We reuse the NetworkManager lower level code in order to connect
to the Bitcoin P2P network.
This implements the basics for anyone wanting to be a player on
this network.
tomFlowee
Your Name