Commit Graph

127 Commits

Author SHA1 Message Date
TomZ dad38b66c6 Flush indexer to SQL database when we reach Tip() 2019-04-11 00:01:54 +02:00
TomZ cec0c28be3 Make indexer subscribe to block updates
When an indexer is in-sync it will stop actively asking for
new blocks, so the block updates (Hub saying they found a new block)
will then allow the indexer to keep up-to-date
2019-04-10 22:17:13 +02:00
TomZ 78219d5bba Make the indexer-cli and indexer interact well
This adds lots of little things;
* Add GetTransaction API call
* I refactored the GetBlock API a little to reuse code.
* a new 'Version' API call for the hub
* API for the logging manager, so we can set a default
  setup with just C++-APIs
* various (usability) fixes in the FloweeServiceApplication
* Binding to localhost attempts to bind to both IPv4 and v6
* Print the actual transaction hex from indexer-cli (which really
  is just a testing app)
2019-04-10 20:52:04 +02:00
TomZ aea9c25c15 Make indexer-cli have more suitable output 2019-04-10 15:38:37 +02:00
TomZ 3e58679108 Add a new app indexer-cli
This is frankly just a testing application at the moment, but you
can connect to a Indexer server and get the information out, so its
at least useful.
2019-04-10 14:32:46 +02:00
TomZ 76fdd25b51 Avoid exit without save
Only throw at startup when SQL problems occur.
When they occur later ask for a clean shutdown so we can allow other
parts to shutdown cleanly.
2019-04-09 20:22:35 +02:00
TomZ 20694d4f33 Make indexer be a server too 2019-04-09 19:34:55 +02:00
TomZ 50b65d054b Cleanups and fixlets 2019-04-09 14:49:08 +02:00
TomZ 07f3a57f73 Fix misuse of std::sort API
Its not clear from my reading of the docs, but the std::sort version
that takes a struct actually makes a deep copy of the struct for each
sort. Which means that this usage is a really bad idea if you were
actually storing the data on this struct.
2019-04-09 12:02:45 +02:00
TomZ df83dc8701 Upgrade HashStorage design phase2
Instead of merging a small list into a bigger and bigger list,
write lots of small lists (20MB each or so) and then when we hit
the wanted final size, merge all of those into one.
2019-04-09 11:19:03 +02:00
TomZ 1ce6a93984 Redo setup of HashStorage, phase 1 2019-04-08 13:19:54 +02:00
TomZ a3a456cdcd Try to get more speed out of the SQL DB
we chunk the data and make sure we create a transaction so autocommit
is turned off.
2019-04-06 19:00:55 +02:00
TomZ e14254f33d Make indexer load a config file for setup
Using standard ini file format.
This is the easiest way to configure it to connect to an external SQL
database.
2019-04-06 15:16:37 +02:00
TomZ ef5dafe6e0 Add private header warnings 2019-04-06 12:28:34 +02:00
TomZ dfb28874ca Adjust design by inverting hash
Additionally I split the sql data over multiple tables
to avoid a forever-growing one.
2019-04-05 23:11:37 +02:00
TomZ 5ed3419d9b Add beginnings of the AddressIndexer as well 2019-04-04 22:46:38 +02:00
TomZ 285b27d3b8 Add multiple databases concept to HashStorage 2019-04-04 22:46:38 +02:00
TomZ 88dd81d090 Make find() work properly 2019-04-03 18:55:33 +02:00
TomZ e5d5b9ec91 Merge branch 'master' into addressdb 2019-04-03 18:09:56 +02:00
TomZ 0447d3c9f6 Correctly detect and print finishing of indexer 2019-04-03 16:16:44 +02:00
TomZ 3afd867454 Add binary search to the HashStorage 2019-04-02 12:07:08 +02:00
TomZ 5767012a9b Make second finalize() work 2019-04-01 22:53:39 +02:00
TomZ 836a558bd2 Make hashstorage level 2 work a little 2019-04-01 21:15:24 +02:00
TomZ 4155557c2f Start working on the HashStorage class 2019-04-01 18:34:39 +02:00
TomZ 4b7f6c7c86 Minor cleanups of indexer 2019-04-01 13:27:01 +02:00
TomZ 30d6cc9c72 Separate out a new FloweeServerApplication 2019-04-01 13:27:01 +02:00
TomZ 5c9373eeac Introduce new app 'indexer' 2019-04-01 10:57:03 +02:00