The download from a peer is now limited to just 4000 blocks, at which
point the download proceeds from another peer.
Naturally it can go back to the first afterwards too.
The idea is that we download a month worth of blocks at a time and then
fetch the same blocks from a backup peer in order to be certain we
actually see the real state.
The problem we've seen is that after the first downloads we end up
starting the second and if the initial sync or a similar large download
happens, many people won't leave it running until the backup actually
completes since the UI looks like its finished.
This is also a good starting point for rating peers at download speed by
having predictable chunks and that means we could time and aim to use
the faster peers.
This is becoming relevant as we can see a huge difference where peers
give you 10x performance compared to some of the slower ones.
If between the first peer finishing downloading and the second
finishing downloading a block was mined, we need to pick a different
peer for the block the first one didn't download.
With the expansion of the database more agressive, checking a
larger set of peers for one that works becomes more important, as
such open more connections ever cycle if we can't find good ones
after half a minute.
With the expansion of the database more agressive, checking a
larger set of items for the best one becomes more important, as
such do a 10x for finding the best sccoring item.
The observed effect is finding previously useful peers within seconds
instead of a minute.
When our addresses database is filled with old or wrong addresses, we
end up trying to connect to a large number of IPs before we find actual
peers.
When we see that is the case, let's try to find more IP addresses from
our peers.
When we didn't personally verify the services, don't judge the IP based
on them and connect anyway.
In addition to not trusting the services we receive from the net, be
more careful with the usage of the 'lastConnected' field.
Notice that we don't actually use the result of that variable in the end
if the everConnected is likewise incorrect.
this introduces a new version of the address-db, as stored in the db
file itself. Causing a one-time 'upgrade'.
Main issue solved is that the 'everSeen' bool was restored incorrectly
and from then on out saved wrong too.
This may have caused bad selection of peers to connect to as the DB
grew.
Together with the various bugs in the last months update we also reset
the punishment to not avoid connecting to possibly perfectly fine peers.
This allows us to use a shared pointer while at the same time not having
the problem that a peer de-registration hits an already deleted
PrivacySegment.
The AddressDB stores separately the ipv4 and ipv6 addresses, ensuring
that the caller only receives IP addresess compatible with what they
asked.
Until now the booleans to define this were simply private members of the
DB and ipv6 was off.
This exposes those boolean to the outside world.
When a peer sends us an 'addr' message it includes the time they report
they last connected to it. This is untrusted data and we should not act
on it for our our connectivity stats.
Last year we started wrapping the Peer object in a shared pointer,
which is used now in the managers.
This continues the idea by making the PrivacySegment use smart
pointers too and the P2PNetInterface is changed to do the same
for downstream applications.
As we moved most of the creation of a BufferPool to be via the
Streaming::pool() method, which uses a thread-local, it makes sense
to start cleaning up the design and make it more modern C++.
The above mentioned method would return a reference and you'd see
loads of places use `auto &pool =` which is less than ideal.
As the number of places where we actually instantiate a BufferPool
goes down, the usage of some sort of smart pointer makes more sense.
This now makes all APIs use BufferPool be wrapped in a shared_ptr.
This follows the coding style guideline that the file that contains a
class should follow the exact name of this (main) class.
pubkey.{h|cpp} -> PublicKey.{h|cpp}
pubkey_utils.{h|cpp} -> PublicKeyUtils.{h|cpp}
The system writes a new file every single run (provided new headers were
received) and we sometimes compact them into a big file again.
The code forgot to remove the newly introduced info files of the old
files it compacted. Leaving confusing things happening after.
This solves that by making the first run remove all info files and re-
build them, adding a version byte to allow us freedom to do that in the
future again.
The bchd seed owner says that since there are no bchd nodes on main-chain
due to the code being outdated, the seed is empty.
We hope this will change in the future, but for now it makes no sense to
query it since it will just generate a fail.
This improves several corner cases on detecting if a peer we
are requesting headers from is actually giving them to us.
Specifically, the height could be zero for a genesis-only view, which
caused the detection to always give the higest score (height zero is
seen as special).
After a node is disconnected we now also reset the history in order to
let the new node get measured from only its own performance.
We optimistically create a new info file but as we start we might
instantly realize the file is useless and give up before having written
a single byte.
We now remove that file to avoid stale state.
Also be more verbose on warnings.
tomFlowee