Commit Graph

318 Commits

Author SHA1 Message Date
TomZ ab1b2e94d9 Make link on some systems 2019-06-19 19:28:02 +02:00
TomZ 4a4a61aeed Add unit test for GetTransaction
This also includes a little blockchain of 114 blocks (the first 100 just
coinbase to reach coin-maturity) with transactions generated by the
txVulcano. So lots of outputs.

I adjusted the BlackBoxTest to load this data into any hub with ease.
2019-06-16 23:04:14 +02:00
TomZ 412f8b2ecf Fix possible corruption on reorgs 2019-06-12 21:02:56 +02:00
TomZ 8e7e3b24ae Merge branch 'master' into httpengine 2019-06-10 21:27:46 +02:00
TomZ 0a916c21e9 Add UTXO support to APIs
Generically this looks like this;

IsUnspent
[
  { "txid": "0x111111", "output": 1 },
  { "txid": "0x111111", "output": 11 }
]
IsUnspentReply
[
  { "blockHeight": 1111, "offsetInBlock": 1111, "unspent": true },
  { "unspent": false }
]

GetUnspentOutput
[
  { "txid": "0x111111", "output": 1 },
  { "txid": "0x111111", "output": 0 }
]
GetUnspentOutputReply
[
  { "blockHeight": 1111, "offsetInBlock": 1111, "unspent": true,
    "amount": 1111, "outputscript": "bytearray" },
  { "unspent": false }
]

note: txids are naturally always sha256s.
2019-06-10 21:02:15 +02:00
TomZ ae6fb56654 Merge branch 'master' into httpengine 2019-06-07 09:57:11 +02:00
TomZ 449f91f46d Move solver method to utils lib
The code that detects standard script-types is now available as a
library function, renamed to live in a namespace;
Script::solver()
2019-06-06 22:21:46 +02:00
TomZ 01641773e1 Import httpengine library
This C++ library is a small framework which creates a full
http(s) server.

This is imported code from the qhttpengine open source project.
I modified some core components in order to make this framework
multi-threading and changed some APIs for cleanlyness. As such
it is not source compatible and getting it accepted upstream is
not realistic.
2019-06-04 21:41:26 +02:00
TomZ 0c9f415eb8 Several NetworkManager fixes
This adds all the header ints to chunked messages, solving the problem
of losing RequestId on roundtrips if the answer was too big.

Added a unit test for this "new" features.

This makes callbacks all use shared_from_this() in order to avoid
callbacks being done on deleted instances (thanks boost!).

Last, special case when the user doesn't connect but just sends messages
which caused a send of data, only to realize the connection wasn't open,
and then a connect.
This makes sure we immediately start a connect on queue of a message.
2019-06-04 21:40:02 +02:00
TomZ a1dfd1cde9 Work on installable headers 2019-06-03 10:59:33 +02:00
TomZ fbfc0f2015 update test to follow API change 2019-06-03 10:59:33 +02:00
TomZ 7203a058a3 Change API call SplitHostPort slightly
Instead of taking an int for the port, take a short.
This matches better with the networkmanager EndPoint struct.
2019-06-02 20:16:49 +02:00
TomZ f868a8274e Make all protocol upgrades equal.
The "UAHF" one used to be "more equal", mostly just because it happened
to be the first.
This makes them all equally equal.

Specifically this removes the special casing and the enum for the 201708
HF (aka BCH fork-point).
We select the right branch now purely based on the historical check-
points.
2019-06-02 13:57:47 +02:00
TomZ 0d72460471 Add another API test 2019-05-30 13:25:58 +02:00
TomZ d68b5e9416 Read and print hub log for CI usage.
When a blackbox test fails add the logs from the nodes
into the output of the log itself so they are all in one
place.

Especially handy for CI or docker usage where getting files
is a lot harder than getting standard-out.
2019-05-27 09:23:05 +02:00
TomZ a20118c5fb Add LiveService unit test
The bugreport claimed that sending an existing coinbase would give
inconsistent results. This test can't find any unexpected behaviour.
2019-05-27 09:23:05 +02:00
TomZ 18035c2e66 Start new testing setup for API testing
This is a blackbox type testing for specific API calls.
2019-05-25 17:06:23 +02:00
TomZ 5fae218cb6 Fix test being unstable on CI
The test assumed at least 2 cores, if you go below that (as the CI
dockers did) then we need to be aware of race-conditions when checking
on the status of the validation.
2019-04-29 10:25:29 +02:00
TomZ 31c0b25fb6 Fix synchronisation issues in test 2019-04-23 13:11:51 +02:00
TomZ a7f9b9fed5 Remove obsolete code.
Remove txIndex option, its move to Flowee Indexer

Also remove the GUI option for setting the amount
of "script validator threads". :D

Remove DBCash command line option too, equally obsolete.
2019-04-22 21:16:10 +02:00
TomZ 2577b6f7ef Merge branch 'may19PU' 2019-04-22 12:51:07 +02:00
TomZ eca66d8734 Add 'rollback' unit test and make it pass 2019-04-22 12:50:09 +02:00
TomZ 5dcc687273 Promote BlockValidation test
This includes a little refactor to make the test init and cleanup
after each individual unit.
2019-04-22 12:49:53 +02:00
TomZ b41e2795e7 Add Schnorr support 2019-04-19 17:30:59 +02:00
TomZ 707d2212de Move encoding (base58) out of server lib
Also write some new accessor methods on the baseclass.

The longer term idea is to make the server itself only use the uint160
raw version for bitcoin addresses.
For now move the encodings (between that uint160 and human-readable) to
the utils library.
2019-04-11 16:20:04 +02:00
TomZ df83dc8701 Upgrade HashStorage design phase2
Instead of merging a small list into a bigger and bigger list,
write lots of small lists (20MB each or so) and then when we hit
the wanted final size, merge all of those into one.
2019-04-09 11:19:03 +02:00
TomZ 1ce6a93984 Redo setup of HashStorage, phase 1 2019-04-08 13:19:54 +02:00
TomZ dfb28874ca Adjust design by inverting hash
Additionally I split the sql data over multiple tables
to avoid a forever-growing one.
2019-04-05 23:11:37 +02:00
TomZ 69ab1133f6 Remove obsolete addressingtype 2019-04-04 22:46:39 +02:00
TomZ 5ed3419d9b Add beginnings of the AddressIndexer as well 2019-04-04 22:46:38 +02:00
TomZ 285b27d3b8 Add multiple databases concept to HashStorage 2019-04-04 22:46:38 +02:00
TomZ 88dd81d090 Make find() work properly 2019-04-03 18:55:33 +02:00
TomZ dc1f721ce5 Rename some old code to use the product name 'Hub' 2019-04-02 17:45:19 +02:00
TomZ 3afd867454 Add binary search to the HashStorage 2019-04-02 12:07:08 +02:00
TomZ 5767012a9b Make second finalize() work 2019-04-01 22:53:39 +02:00
TomZ 836a558bd2 Make hashstorage level 2 work a little 2019-04-01 21:15:24 +02:00
TomZ 4155557c2f Start working on the HashStorage class 2019-04-01 18:34:39 +02:00
TomZ 519cd1b6fc Work on stability of the NetworkManager
In extreme cases we might end up going over the bounds of the internal
buffers, as such myConnection.send() may now throw.

I also cleaned up the ping design and made it a bit more strict.
Last I increased the amount of incoming data I can receive in one go,
this will make throughput higher.
2019-04-01 10:57:42 +02:00
TomZ 3d33bdad99 [cmake] Cleanup include paths
as the number of applications grows it makes more sense to separate
the 'server' library from the actually reusable libraries.
To recap, the 'server' library is what we started with when importing
all the code from the hub. Slowly we are moving good code out that
is stand-alone and reusable.
2019-03-29 21:52:07 +01:00
TomZ a8dd567006 Add unit test for the ring buffer class. 2019-03-24 11:43:43 +01:00
TomZ 6b29387801 Move NetworkManager unit test to own executable 2019-03-24 10:23:52 +01:00
TomZ 557c2000f2 Move CKey to live in utils as well 2019-03-13 22:43:25 +01:00
TomZ cfe690320d Move various standalone simple classes into utils
The 'server' library has always been a catch-all and
ideally only the hub links it in (far future goal).
In line with this I move a list of files out of server
into the utils lib.
I choose 'utils' because all these are plain old data
objects that many crypto apps will find useful.

now in utils/primitives/
* CScript
* CPubKey
* CTransaction
* CBlock
* FastTransaction
* FastBlock
* CScript

streams.h is now in utils/streaming/
hash.h is now in utils/
2019-03-11 15:40:42 +01:00
TomZ a09339dc05 Applying lessons learned for lock-free programming
The reason there are no standard library versions of lock-free
containers is because you want to always take full advantage of
the details in question.
In this case (read millions of times for each modification) it makes
no sense to use anything other than a standard container, but put in
a copy-on-write block. Simple and easy.
2019-03-05 18:52:49 +01:00
TomZ 31650b7340 Squashed commit of the utxo-lockfree-map branch
Replace the m_buckets unsorted map with a lock-free version
based on atomic pointers. (BucketMap)
remove the m_leafs and move those into the bucket struct.
Make the access to the jumptable transactional to avoid one big lock
over all datastructures.

On my threadripper 2990WX the entire 150GB BCH blockchain was
parsed and imported in under 3 hours.
2019-02-23 15:33:22 +01:00
TomZ c367278a1d Cleanup resizing code
We no longer resize blk / rev files, which was a premature optimization.
This means this code is obsolete.
2019-02-06 13:40:40 +01:00
TomZ 135bbb9a5f Cleanup BOOST_FOREACH -> for(:) 2018-12-30 15:33:11 +01:00
TomZ 4623a07968 Remove some usages of the multi-chain feature
During the BCH fork-off time we allowed the client to be started
as either BCH or BTC and as such there are several code-paths
that behave different based on this user setting.
We remove those as we no longer allow starting as BTC client.

This also removes the check for the min-block-size (rollback protection)
as this is accomplished by the checkpoints on the BCH chain.
2018-12-30 13:33:25 +01:00
TomZ 339f9c557c Make test easier to debug and fix porting bug 2018-12-30 13:07:22 +01:00
TomZ d36d99426b Migrate more tests out of boost to qtestlib 2018-12-28 22:26:33 +01:00