This removes a premature optimization where we avoid hashing the header
in the small case that the interface doesn't actually find any useful
transactions in the block.
It hurts others calling this API, though, as they suddenly need access
to the full header instead of just the blockId.
Every peer that we connect to should get the bloom filter set and it
should also get the 'mempool' call sent once which will make that peer
respond with all matches of the bloom filters which are in the mempool.
The tricky part is that we should have the latest bloom filter set on
each of those peers before the mempool is sent, since they work
together.
Also if we are behind, the mempool should not be sent. (It can cause
problems in the wallet if we receive unconfirmed transactions before
mined transactions)
So, when a peer starts downloading merkle blocks, the bloom filter of all
the other peers becomes invalid the moment a match is found by an actual
wallet.
This is Ok on one peer because the merkleblock automatically updates the
filter on match on the server side, but obviously not on the other peers
we have for that wallet.
The approach we follow is that as soon as a sync-run is done on a single
peer (we do a main and also a backup sync), we tell the wallet to re-
build its bloom filter for _all_ peers and if the sync that peer did
leads us to be at the chain-tip, we also send each peer the mempool
command.
When the SyncChainAction was written, various interactions we do
today in the ConnectionManager did not happen yet. Features in
Peer didn't exist yet.
This updates the SyncChainAction to take those items into account
and be more responsive and conclue we are 'up to date' faster,
while also leaving behind a better state.
To enable this means the buildsystem will build all the applications
as well as the libraries.
Applications are 'hub' / 'txVulcano' / 'indexer' etc.
Default this one is turned off.
When we receive a known message (currently just avahello) we now
intantly ban and disconnect the peer, no need to
test compliance when they openly greet us saying they are not following
the same chain.
Tuesdays idea of adding some code into the SyncSPVAction didn't feel
right.
A second look made clear that bloom filter updates make much more sense
to go hand in hand with sending a mempool message. Especially since they
depend on each other on the server side.
To-rehash:
the wallet may decide at any time that a new bloom filter is needed. It
then uses the superclass (code in p2plib) PrivacySegment, to build that
filter. As part of that we get a lock object which, when going out of
scope, makes the peers that are subscribed to the privacySegment send
out the filter.
This separation of concerns means that the subclass wallet in the app
doens't know about peers or messages, only its superclass PrivacySegment
does.
What we did in this change is make the PrivacySegment class decide to
combine a bloom update with a mempool call. Typicall only once per
connection.
This means I can remove hacks in the SyncSPVAction which forced the
sending of the mempool message separately.
The job description of this component is to sync a wallet. Its means to
that end is to connect to peers.
Problem is that when we startup and the whole system is already in-sync,
then we forget to connect to peers. (that wasn't in the job description,
it just happened to be done as a sideeffect).
This changes the SyncSPVAction's job description to also make sure that
each wallet has the required number of peers.
We send a 'mempool' p2p command to a peer after we did a series of
merkleblock downloads.
This code adds the mempool p2p command to be sent to peers that didn't
participate in the merkleblock download and thus get the best out of all
our connected peers.
Now we have 3 more logically divided interfaces for the listener
pattern (callbacks).
A P2PNetInterface for peers maintainance.
A DataListenerInterface for the sections getting new data.
And last the new HeaderSyncInterface about the state of the
header-chain (application wide).
Avoid massive connect storm.
Using the network manager based logger we got into a fun situation where
logging to a remote host caused us to connect and fail, which caused us
to log this failed which made us try to connect and .. etc.
The unit tests testing base58 encoding were using univalue JSON
parsing, since we deprecatd univalue to be a hub-private lib
this has now been ported to Qts JSON parser.
Which also makes the clunky cmake hack nice to replace with the
QRC files concept.
As I was in there anyway, the base58 methods being global scope
C-style methods has now been fixed by putting them in a namespace.
The leveldb and univalue 3rd party libraries are not installed and
not needed by anyone outside of the Hub.
So move them there, making it easier for 3rd party usage.
This sanity check might avoid issues in future code by not assuming a
log channel (or anything it depends on) will avoid using logging at the
moment it is shutting down.
Makes the change from yesterday complete: we now
compress all the chunks into one periodically.
This, again, is done with a timeout instead of on-close.
We have 45 seconds as a timeout to balance between need to save and
waiting long enough to get additional headers should it
have been a while since last run.
This change makes it so that we rely less on the downloaded
blockheader data being only in memory and us saving it when
the client shuts down.
Instead, we save more regularly and we save smaller files
instead of always overwriting the complete downloaded headers.
Having less to save at shutdown is then a happy side-effect.
tomFlowee
Your Name