Flowee/thehub
1
Fork 0
Code Issues Pull Requests Releases 4 Wiki Activity
13,054 Commits 1 Branch 53 Tags
master
Commit Graph

1543 Commits

Author SHA1 Message Date
TomZ a28ffd18bb Write less txid's to the DB file and write shorter ones. 
When pruning we sort leafs by txid / output and refrain from
writing the txid repeatedly for outputs of the same tx.
Additionally, use the fact that we only ever get to a leaf via
a bucket and since the first 64 bits of a txid is there, skip
repeating them when writing to disk.
Last, make pruning have different strategies.

This should shrink the utxo by about 40%
 2018-09-24 21:24:28 +02:00
TomZ 1e9d9b3ea2 Fix system call, its in bytes not in ints 2018-09-24 21:23:27 +02:00
TomZ 343b0345eb Remove dead code 2018-09-24 21:23:27 +02:00
TomZ 39154aa625 Make startup/shutdown a bit more logical 
Actually delete the asio io_service when we are done with it since that
will delete any left over tasks in the queue.
This makes deletion order more predictable.
 2018-09-24 21:23:27 +02:00
TomZ 571d84ae0e Remove dead code. 2018-09-24 21:23:27 +02:00
TomZ c6acb41cc1 Fix validation engine sometimes hanging on shutdown. 2018-09-24 21:23:27 +02:00
TomZ 4d2f5130d4 Minor logging fix 
show users that have configured the block size too low that this why
they get no new data.
 2018-09-24 21:23:27 +02:00
TomZ 371ac9849c Fix case where reindex thought it was done 
After we extract all block-headers from the files and we then shut down
the node this makes sure that we avoid thinking we finished because
there are no more jobs.
This so that a restarting node has the opportunity to continue indexing.
 2018-09-24 21:23:27 +02:00
TomZ 16be66df67 Make logging a bit better (block validation) 
when far behind don't print every single new block / header found,
only print every 1000th or so.
 2018-09-24 21:23:27 +02:00
TomZ e7e4a43084 Make UTXO auto-prune 2018-09-20 22:03:44 +02:00
TomZ 02a7c1d2f6 Refactor pruner to lib from DBA tools 2018-09-20 22:03:44 +02:00
TomZ 13c2a64f56 Identify another usecase of utxo rollback and fix 
Including unit test
 2018-09-20 21:55:07 +02:00
TomZ 88e5e829a5 Re-do the UTXO rollback functionality 
this is a much less memory-intensive version as it doesn't copy
entire buckets anymore.
 2018-09-20 21:55:07 +02:00
TomZ 59916e1d01 Fixes in new utxo DB. 
Fix loading old checkpoints, add fflush etc.
 2018-09-20 21:55:07 +02:00
TomZ bf370f121d Make pruning work 2018-09-20 21:55:07 +02:00
TomZ 6c0f39317f Copy some data in the prune command 2018-09-20 21:55:07 +02:00
TomZ 4072951b00 Log more 2018-09-20 21:55:07 +02:00
TomZ 5aebd68835 Make faster. 2018-09-05 11:16:29 +02:00
TomZ 6c68f9df12 Updates to UnspentOutputDatabase classes 
* remove unused code
* reorg code to make the access to disk be outside the mutex
* add detection of slow disk-writes and slow down data coming in
* Update and fix the rmHint
 2018-09-05 11:16:29 +02:00
TomZ c17be105cf Rename variables for clarity of what they do 2018-08-30 18:44:12 +02:00
TomZ d062dbca39 remove dead code 2018-08-17 23:22:13 +02:00
TomZ f9f128b1cf Use signatures lookup cache 2018-08-17 23:13:02 +02:00
TomZ 82a670787e Merge branch 'master' into UnspentOutputDB 
Conflicts:
	testing/prevector/test_prevector.cpp
	testing/test/test_bitcoin.cpp
	testing/utxo/test_utxo.cpp
 2018-08-15 20:30:45 +02:00
TomZ 62bab30e36 Fix compiler warning 2018-08-15 20:23:36 +02:00
Tom Zander 69e7cd89f8 Merge branch 'master' into 'master' 
Add CuckooCache

See merge request FloweeTheHub/thehub!6
 2018-08-15 18:22:22 +00:00
TomZ c182d809fb Make logging now be tuned for qtestlib 
as the boost testlib is too erretic and gives us bugs, make the common
framework support qtestlib instead
 2018-08-14 15:53:00 +02:00
Tom Zander 4245b721b0 Merge branch 'testsuite' into 'master' 
Add optimized double SHA256

See merge request FloweeTheHub/thehub!8
 2018-08-13 09:42:41 +00:00
DesWurstes c9e9e19698 Add Sha256D64 to Merkle root calculation 2018-08-12 10:27:40 +03:00
DesWurstes 5dff3ac72d Add SHA256D64 2018-08-12 10:27:40 +03:00
DesWurstes 20ea0c1156 Add tests 
https://github.com/bitcoin/bitcoin/pull/9480 and https://github.com/bitcoin/bitcoin/pull/10128 but doesn't have the newest tests
 2018-08-11 12:57:33 +03:00
DesWurstes fffc707c74 Remove unneeded void 
https://github.com/bitcoin/bitcoin/pull/13774/files
 2018-08-10 19:16:35 +03:00
DesWurstes effe0227d3 Update CuckooCache to its newest version 2018-08-10 11:05:22 +03:00
DesWurstes f61e8f698e Add CuckooCache 
100% of code from https://github.com/bitcoin/bitcoin/pull/8895/commits/c9e69fb
 2018-08-10 11:05:22 +03:00
DesWurstes da075d73be Add SHA-NI and SSE4.1 SHA256 2018-08-09 16:01:13 +03:00
DesWurstes 87991845ee Superfluous space 2018-08-08 17:07:27 +03:00
DesWurstes b1ef38f886 TX search bug and improvements 2018-08-08 13:58:40 +03:00
DesWurstes dae8b90054 Define MSG_NOSIGNAL on Apple 
Solves another build error
 2018-08-07 12:56:15 +00:00
DesWurstes 2a5d464e73 Replace -r with -R 2018-08-07 12:10:14 +00:00
TomZ 854544f33a Make scanned but not validated files also save to blockindex 2018-08-06 13:57:09 +02:00
TomZ fd02e45f7f Fix off-by-one 2018-08-06 12:01:12 +02:00
TomZ eee7140368 Fix the loading and saving of the info files in case of corruption 
If there is corruption in one file, we need to rollback all of them
to a stable point where the entire DB agrees
 2018-08-06 12:00:14 +02:00
TomZ ebde1fd457 When restarting a reindex, avoid doing double work. 2018-08-06 12:00:14 +02:00
TomZ 8cca04ff1d Fix oops 2018-08-06 12:00:14 +02:00
TomZ 2fde96e54b Merge branch 'master' into UnspentOutputDB 2018-08-05 10:59:35 +02:00
TomZ c2cfea6b50 Notice exit faster. 
This also makes the exit more stable and avoids hangs.
 2018-08-05 10:58:19 +02:00
TomZ 6ab9e96071 log UTXO by text 2018-08-04 18:44:14 +02:00
TomZ f1705e33ad Be more agressive in writing 
the savings are questionable and the memory usage is a bit steep.
 2018-08-04 18:30:57 +02:00
TomZ 7e954a7f8c Let people know they need to reindex 2018-08-04 18:30:24 +02:00
TomZ d58352080e Add another UODB rollback unit test and fix 2018-08-04 17:28:08 +02:00
TomZ d5d21d214d Make the saving more fool-proof 2018-07-30 22:12:50 +02:00