Commit Graph

70 Commits

Author SHA1 Message Date
TomZ e042fb90f5 Cancel timer on shutdown 2020-01-12 18:05:09 +01:00
TomZ aef124e523 Merge branch '2019.10' 2019-12-17 17:58:34 +01:00
TomZ 1143a71a92 Fix assignment typo 2019-12-17 17:57:08 +01:00
TomZ f4d8870001 NWM: tweaks to logs. 2019-12-17 17:36:15 +01:00
TomZ 8c842fa108 Catch exceptions on usage of socket
This should fix the issue where the NWM stopped listening due to us not
calling the proper setupCallback()

Also add some logging.
2019-12-11 23:47:21 +01:00
TomZ 152efb2c5d Print more details on making NWM connection. 2019-11-29 20:09:28 +01:00
TomZ a471d919eb Improve logging of not-acceptable connections
Log IP of remote that tries to talk to us but doesn't know the protocol.
2019-11-29 10:59:58 +01:00
TomZ e7ad998d57 Add -fPIC to installed libs 2019-11-04 14:20:50 +01:00
TomZ 83d455a5d4 Help devops with more info. 2019-08-29 22:33:57 +02:00
TomZ 191bc3758d Add a simpliefied NetworkManager::bind() method
One that doesn't require a callback to be passed in.
2019-08-24 22:25:10 +02:00
TomZ df7de2ecd2 Clean up the hairy ball of util.cpp
Move some globals and all stuff pulling in crypto to the
server/serverutil.cpp file

Remove dead code.

Move several items that were used in only one place to the respective
files using them.

Move the class WaitUntilFinishedHelper into its own file.

Made sure no header includes utils.h
remove a handfull of files including utils.h for nothing.
2019-08-24 22:25:09 +02:00
TomZ 9faf50029e Fix typo 2019-08-24 22:25:09 +02:00
TomZ 2bb06afa9e Cleanup the link-lines 2019-08-24 22:25:09 +02:00
TomZ f311073312 Avoid DNS lookups and better logging
When users type an IP address as hostname we optimistically try
to parse them in order to avoid asking the DNS anyway.
2019-08-19 21:43:08 +02:00
TomZ a56b67caf2 NWM; Create a new socket before reconnect 2019-06-22 10:59:47 +02:00
TomZ fca434b1a6 Make Indexer connect to Hub after loading the conf
Doing it before could lead to incoming data being
processed before the SQL database connection was setup.
2019-06-13 17:12:28 +02:00
TomZ e5233fba32 On close, reset the chunked internal data. 2019-06-13 12:21:38 +02:00
TomZ 0a916c21e9 Add UTXO support to APIs
Generically this looks like this;

IsUnspent
[
  { "txid": "0x111111", "output": 1 },
  { "txid": "0x111111", "output": 11 }
]
IsUnspentReply
[
  { "blockHeight": 1111, "offsetInBlock": 1111, "unspent": true },
  { "unspent": false }
]

GetUnspentOutput
[
  { "txid": "0x111111", "output": 1 },
  { "txid": "0x111111", "output": 0 }
]
GetUnspentOutputReply
[
  { "blockHeight": 1111, "offsetInBlock": 1111, "unspent": true,
    "amount": 1111, "outputscript": "bytearray" },
  { "unspent": false }
]

note: txids are naturally always sha256s.
2019-06-10 21:02:15 +02:00
TomZ e168e3904d Respond to error. 2019-06-07 09:53:05 +02:00
TomZ 99e7f8656c Make headers go to nice subdirs 2019-06-07 09:53:05 +02:00
TomZ ec2610139e Allow setting indexer connections in conf file
This adds the ability to read from the config file the network
connections. Both the listening address and port as well as the upstream
'hub' address.
2019-06-06 19:44:16 +02:00
TomZ 0c9f415eb8 Several NetworkManager fixes
This adds all the header ints to chunked messages, solving the problem
of losing RequestId on roundtrips if the answer was too big.

Added a unit test for this "new" features.

This makes callbacks all use shared_from_this() in order to avoid
callbacks being done on deleted instances (thanks boost!).

Last, special case when the user doesn't connect but just sends messages
which caused a send of data, only to realize the connection wasn't open,
and then a connect.
This makes sure we immediately start a connect on queue of a message.
2019-06-04 21:40:02 +02:00
TomZ 57b0b58aa7 Add code to stream a NetworkEndPoint to Logger. 2019-06-03 21:41:04 +02:00
TomZ ca2d6b041a fix connection() not finding an existing connection 2019-06-03 14:37:08 +02:00
TomZ a00938a118 Fix race condition in shutdown of NetworkConnection 2019-06-03 14:37:08 +02:00
TomZ 50870520df Fix NetworkConnection::isConnected() behaviour
This brings it closer to what we expect.
2019-06-03 14:36:59 +02:00
TomZ a1dfd1cde9 Work on installable headers 2019-06-03 10:59:33 +02:00
TomZ db9eb1adfb Install some more files 2019-05-30 13:36:57 +02:00
TomZ e6a23d5f4f Merge branch '2019.05' 2019-05-27 09:28:25 +02:00
TomZ 702addadbb Remove race-conditions in connecting 2019-05-27 09:27:26 +02:00
TomZ 8ed6d6a3e2 NWM: Fix reconnecting a dead connection
A disconnect would cause the socket to close which caused a
callback which triggered a timer to reconnect after half a minute...

This makes sure that after we manually disconnect we honor that.
2019-05-27 09:23:05 +02:00
TomZ a20118c5fb Add LiveService unit test
The bugreport claimed that sending an existing coinbase would give
inconsistent results. This test can't find any unexpected behaviour.
2019-05-27 09:23:05 +02:00
TomZ 18035c2e66 Start new testing setup for API testing
This is a blackbox type testing for specific API calls.
2019-05-25 17:06:23 +02:00
TomZ 38b4e8b189 Allow hubs -apibind to use 'localhost' string
The command really only accepts IP addresses, but we use 'localhost' as
an exception to bind to loopback for ipv4 and ipv6 at the same time.

Also improve logging of errors and avoid failing to start the API
service if a bind address wasn't an IP.
2019-05-24 23:14:15 +02:00
TomZ a54107c168 Make NetworkService thread-safe
This makes the access to the per-peer datastructure lock-free and
thread-safe.
2019-05-18 16:03:22 +02:00
TomZ ba2216d582 Add to CLI clients verbose/quiet options
All client apps (=connecting to a server only) now gained the  verbose
and quiet command line arguments to select a different level of
verbosity.

Servers don't need this as they use a config file.

This also makes the --debug option only for debug builds as the logging
system already doesn't compile in debug statements otherwise, it makes
little sense to try to print them.
2019-05-13 18:27:09 +02:00
TomZ 7fbb6772de NWM: Promote some debug level to warn/info 2019-05-13 12:20:57 +02:00
TomZ aa2c1a6f60 Vastly simplify the NWM callback after sent 2019-04-11 16:20:05 +02:00
TomZ 6cc433c589 Make usage of NetworkConnection::accept() clear
I added some docs to bind() that handlers there should call
accept().
Also made sure that when nobody accepts a connection it actually
gets disconnected instead of just being ignored.
2019-04-10 14:28:33 +02:00
TomZ ccd66a49b2 Document and refactor NetworkService
The main goal of this is to make sure that network services
implementations don't reuse one bufferpool for all their remotes as that
can cause threading issues.
One bufferpool per remote is now made easy.
2019-04-09 17:24:38 +02:00
TomZ 0c97e3261f Cleanup close/disconnect in NWM
Specifically avoid clearing the messages queue when the other side
disconnects.
2019-04-06 12:28:41 +02:00
TomZ cec1950650 Make connection wait on destructor
Make sure that a NetworkConnection object (which owns the callbacks)
doesn't return from its destructor until the already scheduled callbacks
in another thread have been completed.

This avoids shutdown scenarios where we execute in a deleted
object.
2019-04-06 11:59:49 +02:00
TomZ 9846b467e3 Move code and avoid work.
Move the creation of the ping message to the constructor (from the
connection-established method) so we are certain it gets done exactly
once.

Also avoid putting default (invalid) values in the hashmap.
2019-04-04 22:46:39 +02:00
TomZ 519cd1b6fc Work on stability of the NetworkManager
In extreme cases we might end up going over the bounds of the internal
buffers, as such myConnection.send() may now throw.

I also cleaned up the ping design and made it a bit more strict.
Last I increased the amount of incoming data I can receive in one go,
this will make throughput higher.
2019-04-01 10:57:42 +02:00
TomZ b8d329ef9f Further moves; NetworkEnums and netbase.{cpp|h}
NetworkEnums.h moves from utils to interfaces.
netbase moves from server to utils/
2019-03-29 21:39:08 +01:00
TomZ 5939de8ef0 Move API protocol definition to interfaces dir.
It makes no sense to include the API "library" just to use
the header file.
2019-03-29 21:07:28 +01:00
TomZ 57542dda91 Fix regression in NWM (after RingBuffers intro) 2019-03-28 10:23:32 +01:00
TomZ 6475d6060a Fix connecting to named hosts (as opposed to IPs) 2019-03-28 10:23:28 +01:00
TomZ cd31c199ee Add some more asserts to NWM 2019-03-27 20:25:07 +01:00
TomZ c8108f387c Drop API login requirement and api-cookie argument 2019-03-27 18:55:13 +01:00