Commit Graph

922 Commits

Author SHA1 Message Date
TomZ 52f40acb93 Speed up loading 2020-05-19 08:27:53 +02:00
TomZ a5e6bea027 Merge branch '2020.03' 2020-05-18 22:52:19 +02:00
TomZ e9a2437c2d Add unit tests for UTXO and fixes
This fixes that the .info file is sometimes not written when nothing
changed in the db file.

This fixes that the info file with .1.info extension is reused again
and again, effectively reducing the usefulness of the snapshots.
This only really affects the indexer as pruning avoids this most
of the time.

Closes issue flowee-issues#6
2020-05-18 22:51:41 +02:00
TomZ 7ba11b9880 Unify usage of random number generator. 2020-05-18 19:53:11 +02:00
TomZ 45d9ac3bd2 Make Peer listen to Bloom filter changes 2020-05-18 14:32:34 +02:00
TomZ cde1372b2e Make PrivacySegment emit signals
this adds a listener interface and a way to emit the callbacks on
changing of the filter.

This also adds a mutex since we expect the Peer to use the filter which
will likely live in its own thread.
This makes the class thread safe and re-entrant.

Notice that we use a recursive mutex to allow various usecase on
altering the bloom filter.
The most involved one is a complete replacement which calls clear and
then various calls to add() style methods.
Second is a single 'add' which can be done without the clear first.

The second needs an explicit lock in the add() methods, which would
deadlock in the first usecase if I didn't pick the recursive mutex.
2020-05-18 14:25:14 +02:00
TomZ f6bf22e9ad Remember 'reset' time.
This allows the seeder to re-investigate a peer that has high
punishment, but has not been tried for a long time.
2020-05-17 22:31:30 +02:00
TomZ 158a70ca12 Increase requirement for DB size now we persist them 2020-05-16 12:17:24 +02:00
TomZ 01e8bcfb09 Announce all transactions per block at once.
Instead of forwarding one transaction at a time as the peer sends them
to us, bundle them in a group of transactions known to be merkle-checked
and all belonging together in one block.

Since the peer has no obligation (and with CTOR even less) to send the
transactions in natural order, we should get them per block so we know
all transactions forwarded have parents.
2020-05-16 11:01:27 +02:00
TomZ 10cc4443ab Add a mutex to the peer-address-db
As this is a multi-threaded enabled class, even though 99% of it is
reads, we still need to lock.
2020-05-16 11:01:27 +02:00
TomZ cafa0ede08 For wallets, start a sync at startup.
This helps when SPV wallets are still behind on merkle-blocks while the
wallet shut down.
We start instantly again, instead of waiting for the first block to be
mined.
2020-05-16 10:27:00 +02:00
TomZ c09aed6692 Add comment. 2020-05-15 17:41:52 +02:00
TomZ c2d53d75e4 Add some extra methods on the privacySegment
this allows more mature usage of the bloom filter.
2020-05-15 17:18:42 +02:00
TomZ b89d117475 Add checkpoints after latest protcol upgrade 2020-05-15 17:14:46 +02:00
TomZ 3c01d382d0 Merge branch '2020.03' 2020-05-15 15:45:06 +02:00
TomZ 21f519190d Fix copy/paste issue 2020-05-15 15:13:52 +02:00
TomZ c14aa3c1f7 Forgot to copy this. 2020-05-14 22:50:51 +02:00
TomZ ef69014fd2 Upgrade log line 2020-05-13 22:29:23 +02:00
TomZ 589932e876 Make dependent projects able to link to secp256k1
As the compile may or may not link to GMP, using projects
need to link to this too. Dynamic libs handle this, we ship
static libs and thus we need to handle this ourselves.
2020-05-13 22:29:23 +02:00
TomZ ba7e1d6b50 fix off-by-one 2020-05-11 23:15:30 +02:00
TomZ 2341e4723e P2PNet Make logs easier to read 2020-05-11 20:01:07 +02:00
TomZ 148f9c0e93 Improve tracking of 'good' peers.
This avoids a peer once sending acceptable headers and never
being bothered again. Instead we now check regularly and keep
track of when the peer was known to follow the same chain as us.
2020-05-11 18:49:16 +02:00
TomZ 7bfe5e0e2c Provide a getter for blockHeight on Blockchain 2020-05-11 18:07:10 +02:00
TomZ d0bc12ea2d API docs and minor fixes 2020-05-11 17:42:44 +02:00
TomZ 43d7626d46 P2PNet make the blockchain class save its state. 2020-05-11 15:16:59 +02:00
TomZ 93f9758ca5 Add load/save to PeerAddressDB 2020-05-11 13:24:06 +02:00
TomZ bd2a06a9ff Add a save to match the load Addr. 2020-05-11 13:24:06 +02:00
TomZ 79a92cf2ce NWM: work better with large concurrency
The asio stuff is quite broken when it comes to, ironically,
concurrency.
Connecting to peers lead to lots of 'canceled' (sic) errors being
generated internally in asio and other such issues.
Even though they were called on separate objects.

Adding a mutex around the async_connect calls made the amount of
failures go down dramatically.

This patch also includes upgrading boost::mutex to the std-lib
versions.
2020-05-11 13:19:10 +02:00
TomZ ed5b100ca5 Delete ununsed connections on shutdown. 2020-05-10 22:03:22 +02:00
TomZ 010df674e4 P2PNet: Use exceptions and avoid deadlock.
This avoids still having the blockchain lock while calling into the
connectionmanager which may do the reverse, causing deadlock.
2020-05-10 16:02:14 +02:00
TomZ 0b3dc7a8f5 NWM: fix reconnects/reuse of connection 2020-05-10 16:02:14 +02:00
TomZ ebbbdcd3a2 P2PNet: log improvement.
Also fix cron disconnecting peers before they connect.
2020-05-10 16:02:14 +02:00
TomZ 36cd1d4f56 Just disconnect a slow peer, don't ban. 2020-05-10 16:02:14 +02:00
TomZ 00453d52c4 NWM: Reuse connection objects
Avoid deleting and creating connections, which is really a rather rare
event anyway, and simply remember the connection object and reuse it
when a new one is requested.
2020-05-10 16:01:08 +02:00
TomZ d49b1ae5e2 Make NetworkManager connections leaner
The NetworkManager usage was mostly for low connection counts and this
made defaults selection easy.
With more usages it is important to allow the NWM-connection to be more
configurable about memory usage and leaner in general.

This changes the headers-buffers (used to create envelopes) to not be per
connection anymore but per thread using the tread_local keyword.

This changes the ring-buffers to become configurable using
NetworkConnections::setMessageQueueSizes().

Also removing some include statements where they were not really needed
in the P2PNet lib.
2020-05-09 20:23:05 +02:00
TomZ 5bb095f8a2 Minor fixes in usage of shared pointers. 2020-05-07 18:26:41 +02:00
TomZ 9b9be73a5d Detect already deleted peer. 2020-05-07 17:01:03 +02:00
TomZ 9a78e770ac P2PNet: handle errors better
This changes handling to also remove peers that fail with:
  "Cannot assign requested address"
An error seen in a Docker container on a system supporting IPv6.

Additionally we now always assign a error level so an unknown error will
still cause the node to be banned after 5 attempts.
2020-05-07 16:07:12 +02:00
TomZ dc9a92c57e Also install the new header. 2020-05-07 15:38:32 +02:00
TomZ 488e9b4322 Add missing dependency 2020-05-07 14:04:57 +02:00
TomZ 6a5ccf30cf Move 'sendheaders' msg to after version handshake 2020-05-06 23:59:02 +02:00
TomZ 39fb4723ef Lost this in the refactor. 2020-05-06 18:38:37 +02:00
TomZ 46e1f966e2 Re-enable actions. 2020-05-06 17:59:36 +02:00
TomZ 7ad2592d40 P2PNet: API cleanups and review + docs. 2020-05-06 10:42:58 +02:00
TomZ 991da54d3c List our own seeder too 2020-05-06 10:39:47 +02:00
TomZ fb6f91bf79 Simplify p2pnet ownership
Make the Peer object be owned by a shared pointer.
2020-05-05 22:53:25 +02:00
TomZ 3f60c0a48f remember 'good headers' flag for a peerAddress 2020-05-05 17:02:10 +02:00
TomZ 16168bc541 slight fixes in logging 2020-05-05 10:56:57 +02:00
TomZ bf76c5ab36 Write all bytes after command to zero.
Instead of just a single trailing zero, write all bytes of the
command-section. This is required by the protocol.
2020-05-05 10:54:26 +02:00
TomZ c2dd23d89b Merge branch '2020.03' 2020-05-03 21:02:50 +02:00