07f3a57f73
Fix misuse of std::sort API
...
Its not clear from my reading of the docs, but the std::sort version
that takes a struct actually makes a deep copy of the struct for each
sort. Which means that this usage is a really bad idea if you were
actually storing the data on this struct.
2019-04-09 12:02:45 +02:00
df83dc8701
Upgrade HashStorage design phase2
...
Instead of merging a small list into a bigger and bigger list,
write lots of small lists (20MB each or so) and then when we hit
the wanted final size, merge all of those into one.
2019-04-09 11:19:03 +02:00
1ce6a93984
Redo setup of HashStorage, phase 1
2019-04-08 13:19:54 +02:00
a3a456cdcd
Try to get more speed out of the SQL DB
...
we chunk the data and make sure we create a transaction so autocommit
is turned off.
2019-04-06 19:00:55 +02:00
e14254f33d
Make indexer load a config file for setup
...
Using standard ini file format.
This is the easiest way to configure it to connect to an external SQL
database.
2019-04-06 15:16:37 +02:00
ef5dafe6e0
Add private header warnings
2019-04-06 12:28:34 +02:00
dfb28874ca
Adjust design by inverting hash
...
Additionally I split the sql data over multiple tables
to avoid a forever-growing one.
2019-04-05 23:11:37 +02:00
5ed3419d9b
Add beginnings of the AddressIndexer as well
2019-04-04 22:46:38 +02:00
285b27d3b8
Add multiple databases concept to HashStorage
2019-04-04 22:46:38 +02:00
88dd81d090
Make find() work properly
2019-04-03 18:55:33 +02:00
e5d5b9ec91
Merge branch 'master' into addressdb
2019-04-03 18:09:56 +02:00
0447d3c9f6
Correctly detect and print finishing of indexer
2019-04-03 16:16:44 +02:00
3afd867454
Add binary search to the HashStorage
2019-04-02 12:07:08 +02:00
5767012a9b
Make second finalize() work
2019-04-01 22:53:39 +02:00
836a558bd2
Make hashstorage level 2 work a little
2019-04-01 21:15:24 +02:00
4155557c2f
Start working on the HashStorage class
2019-04-01 18:34:39 +02:00
4b7f6c7c86
Minor cleanups of indexer
2019-04-01 13:27:01 +02:00
30d6cc9c72
Separate out a new FloweeServerApplication
2019-04-01 13:27:01 +02:00
5c9373eeac
Introduce new app 'indexer'
2019-04-01 10:57:03 +02:00
TomZ