Commit Graph

193 Commits

Author SHA1 Message Date
TomZ df83dc8701 Upgrade HashStorage design phase2
Instead of merging a small list into a bigger and bigger list,
write lots of small lists (20MB each or so) and then when we hit
the wanted final size, merge all of those into one.
2019-04-09 11:19:03 +02:00
TomZ 1ce6a93984 Redo setup of HashStorage, phase 1 2019-04-08 13:19:54 +02:00
TomZ dfb28874ca Adjust design by inverting hash
Additionally I split the sql data over multiple tables
to avoid a forever-growing one.
2019-04-05 23:11:37 +02:00
TomZ 69ab1133f6 Remove obsolete addressingtype 2019-04-04 22:46:39 +02:00
TomZ 5ed3419d9b Add beginnings of the AddressIndexer as well 2019-04-04 22:46:38 +02:00
TomZ 285b27d3b8 Add multiple databases concept to HashStorage 2019-04-04 22:46:38 +02:00
TomZ 88dd81d090 Make find() work properly 2019-04-03 18:55:33 +02:00
TomZ dc1f721ce5 Rename some old code to use the product name 'Hub' 2019-04-02 17:45:19 +02:00
TomZ 3afd867454 Add binary search to the HashStorage 2019-04-02 12:07:08 +02:00
TomZ 5767012a9b Make second finalize() work 2019-04-01 22:53:39 +02:00
TomZ 836a558bd2 Make hashstorage level 2 work a little 2019-04-01 21:15:24 +02:00
TomZ 4155557c2f Start working on the HashStorage class 2019-04-01 18:34:39 +02:00
TomZ 519cd1b6fc Work on stability of the NetworkManager
In extreme cases we might end up going over the bounds of the internal
buffers, as such myConnection.send() may now throw.

I also cleaned up the ping design and made it a bit more strict.
Last I increased the amount of incoming data I can receive in one go,
this will make throughput higher.
2019-04-01 10:57:42 +02:00
TomZ 3d33bdad99 [cmake] Cleanup include paths
as the number of applications grows it makes more sense to separate
the 'server' library from the actually reusable libraries.
To recap, the 'server' library is what we started with when importing
all the code from the hub. Slowly we are moving good code out that
is stand-alone and reusable.
2019-03-29 21:52:07 +01:00
TomZ a8dd567006 Add unit test for the ring buffer class. 2019-03-24 11:43:43 +01:00
TomZ 6b29387801 Move NetworkManager unit test to own executable 2019-03-24 10:23:52 +01:00
TomZ 557c2000f2 Move CKey to live in utils as well 2019-03-13 22:43:25 +01:00
TomZ cfe690320d Move various standalone simple classes into utils
The 'server' library has always been a catch-all and
ideally only the hub links it in (far future goal).
In line with this I move a list of files out of server
into the utils lib.
I choose 'utils' because all these are plain old data
objects that many crypto apps will find useful.

now in utils/primitives/
* CScript
* CPubKey
* CTransaction
* CBlock
* FastTransaction
* FastBlock
* CScript

streams.h is now in utils/streaming/
hash.h is now in utils/
2019-03-11 15:40:42 +01:00
TomZ a09339dc05 Applying lessons learned for lock-free programming
The reason there are no standard library versions of lock-free
containers is because you want to always take full advantage of
the details in question.
In this case (read millions of times for each modification) it makes
no sense to use anything other than a standard container, but put in
a copy-on-write block. Simple and easy.
2019-03-05 18:52:49 +01:00
TomZ 31650b7340 Squashed commit of the utxo-lockfree-map branch
Replace the m_buckets unsorted map with a lock-free version
based on atomic pointers. (BucketMap)
remove the m_leafs and move those into the bucket struct.
Make the access to the jumptable transactional to avoid one big lock
over all datastructures.

On my threadripper 2990WX the entire 150GB BCH blockchain was
parsed and imported in under 3 hours.
2019-02-23 15:33:22 +01:00
TomZ c367278a1d Cleanup resizing code
We no longer resize blk / rev files, which was a premature optimization.
This means this code is obsolete.
2019-02-06 13:40:40 +01:00
TomZ 135bbb9a5f Cleanup BOOST_FOREACH -> for(:) 2018-12-30 15:33:11 +01:00
TomZ 4623a07968 Remove some usages of the multi-chain feature
During the BCH fork-off time we allowed the client to be started
as either BCH or BTC and as such there are several code-paths
that behave different based on this user setting.
We remove those as we no longer allow starting as BTC client.

This also removes the check for the min-block-size (rollback protection)
as this is accomplished by the checkpoints on the BCH chain.
2018-12-30 13:33:25 +01:00
TomZ 339f9c557c Make test easier to debug and fix porting bug 2018-12-30 13:07:22 +01:00
TomZ d36d99426b Migrate more tests out of boost to qtestlib 2018-12-28 22:26:33 +01:00
TomZ 18d032a556 Transfer more to QTestLib
As boost testlib is extremely IDE unfriendly, as well as human
unfriendly with lots of macros for basic C++ functions (like methods!!)
this is better for me.

But the real reason is that its just unstable. I get double deletes
on some releases of boost and I'm missing plain features that all
other test frameworks have.
For instance a QCOMPARE shows what is expected vs what we got. Boost
just fails.
In QTestLib I can mark a test as "expect fail" an idea that boost
tried and failed (can easily create false positives).
2018-12-28 16:21:49 +01:00
TomZ 3e3f8843f3 Remove the optional replay protection feature
Its sundown has passed, so the code is now effectively dead.
2018-11-29 23:13:48 +01:00
TomZ 951bdb68bf remove last usages of -chain= option
this is from back in the day when we supported switching between
the BTC and the BCH chain.
2018-11-21 12:20:20 +01:00
TomZ b256676044 Rename debug.log -> hub.log 2018-11-21 12:18:32 +01:00
TomZ 830726ef88 Make mining work again.
* Make mining software honor CTOR resorting.
* Fix bugs
2018-11-20 20:44:06 +01:00
TomZ 71b45cd38c Merge branch 'dev/HFNov2018' 2018-11-15 23:40:56 +01:00
TomZ c2412a0e59 Add testing for CTOR 2018-11-15 23:35:43 +01:00
TomZ 3b04ba0305 remove this, not useful if already running in gdb 2018-11-15 23:35:43 +01:00
TomZ 9aab378b0a Add checks for 100 bytes min tx size 2018-11-15 23:34:05 +01:00
TomZ 66b610adac Fix compile of tests without wallet 2018-11-13 23:36:35 +01:00
TomZ 35225f1400 Update API to make keys by default compressed 2018-11-13 14:30:17 +01:00
TomZ e0b3a14daa Import unit tests 2018-11-08 10:53:16 +01:00
TomZ 7b76d177e8 Remove chain obfuscation misfeature. 2018-11-02 20:42:45 +01:00
TomZ d8dfec2c26 Make test a bit more stable 2018-10-22 13:08:35 +02:00
TomZ e50384f421 Make invalidateblock work a bit better 2018-10-22 00:08:11 +02:00
TomZ 90f635e8ec Add a test for a cornercase of appendHeader and fix it. 2018-10-21 20:38:34 +02:00
TomZ fdd693541e Make the UTXO have some limits "configurable"
This change makes the limits be static variables, which means they are
only settable once for a process. The only usecase so far is to use much
smaller limits in testing situations.
2018-10-21 13:21:24 +02:00
TomZ 82d2651d4f Fix copy/paste error and add some more sanity checks
Also fix insert() usage in unit test to not hit that new assert
2018-10-13 21:33:17 +02:00
TomZ 6b20196529 Fix regression in py tests, caches work again 2018-10-12 23:13:42 +02:00
TomZ 5e91b276d3 Fix off-by one that made unit tests fail on 1-core dockers. 2018-10-11 21:45:40 +02:00
TomZ ab08bfa914 Fixlets in unit test setup 2018-10-11 17:36:36 +02:00
TomZ 91f9c5128a Make sure we specify type to avoid regressions 2018-10-11 16:27:04 +02:00
TomZ 145a67cd92 Update rpc tests to flowee changes 2018-10-11 16:26:50 +02:00
TomZ ef89fbc2ef Follow change in parsing of -regtest
Flowee changed in that regtest has to be given on the commandline
allowing different config files for different chains.
So pass the regtest on the commandline.
2018-10-11 16:26:42 +02:00
TomZ 1b8c1a586f Allow API server to be disabled via option 2018-10-11 16:26:36 +02:00