Flowee/thehub
1
Fork 0
Code Issues Pull Requests Releases 4 Wiki Activity
11,775 Commits 1 Branch 53 Tags
2020.03.3
Commit Graph

95 Commits

Author SHA1 Message Date
TomZ 9744e01343 Make compile 2020-02-19 16:55:46 +01:00
TomZ 11d16001c6 Indexer: add get for current sync-height 2020-01-20 12:17:55 +01:00
TomZ ce55457c33 Avoid asking for a block the hub won't have 2020-01-15 12:18:01 +01:00
TomZ 987081a229 Make the SQL server sort by block-height for us 2019-12-26 00:00:03 +01:00
TomZ 901860e406 Fix more edge-cases where indexer might stop updating. 2019-12-17 17:57:08 +01:00
TomZ 5796de30ba Be smarter about getting new block on connect 2019-12-12 23:19:26 +01:00
TomZ a85a9e1207 Indexer now uses blockCount 
Instead of asking for a block and reacting when we get a failure, this
just uses the current height instead.

This also makes 'flush' happen in each indexers' own thread, which should
be good for performance.
 2019-12-12 16:02:08 +01:00
TomZ 7b5e922658 Flush address DB faster. 
Ask the hub for the chain-length and when we processed the last block,
immediately flush the data to the SQL DB.
This makes use represent the whole world faster.
 2019-12-12 14:49:46 +01:00
TomZ 7d445b445b Add address indexer index. 2019-11-29 17:56:24 +01:00
TomZ 9e3b145b4e AddressIndexer: Wait for SQL DB 
Instead of exiting when no SQL DB is found, sleep for 5 seconds and try
again.
 2019-11-29 14:47:36 +01:00
TomZ b157ffba5f Update to also list new indexer 2019-11-29 11:07:54 +01:00
TomZ d573e307c3 Refactor; move SplitHostPort to utilstrencodings 2019-11-12 15:35:54 +01:00
TomZ 6cc749627d Merge branch '2019.09' 2019-11-11 21:12:18 +01:00
TomZ 0e8a4da185 Find address from newest to oldest. 2019-11-11 19:37:04 +01:00
TomZ 8dde1c86ed Avoid unneeded copying 2019-11-06 23:20:17 +01:00
TomZ 748cd54a8e Fix log line limiter 
Avoid logging being the slow part.
 2019-11-06 23:20:17 +01:00
TomZ 3c25232751 Exit when nothing is found. 2019-10-20 22:12:09 +02:00
TomZ 79e4974d19 Port API getblock::filter to use script-hash 
The API call for GetBlock has a filter-on-address functionality which is
now ported to no longer use ripe160 addresses but instead uses output-
script hashes.
This avoids problems for transactions not using p2pkh and generally is a
cleaner solution.

This also adds a unit test to test this feature.
 2019-10-20 22:12:09 +02:00
TomZ c4e255560b Port indexer client to use script-hash 
This allows us to support p2sh properly.
 2019-10-20 12:19:00 +02:00
TomZ 2396938cfb New version and only connect to new versions 2019-10-20 11:57:01 +02:00
TomZ 00814e24f9 API: Use hash of outscript instead of address 
The usage of a ripe160 for bitcoin addresses in the API and in the
Indexer loses some info, specifically what kind of script it is.
Additionally not all types of scripts fit this mold. At best that means
its not future-proof.

This adds a method to the API in order to select from a Tx the hashed
outscript (thats singlehashed sha256) and refactor the address indexer
to use that instead of the ripe160 address.

The API enums broke a little, so I used the opportunity to break it a
lot and clean up the enums in order to make them more future-proof.
But, yeah, software from before this commit is protocol incompatible
with software after this commit.
 2019-10-16 22:57:50 +02:00
TomZ e2209abcbb Make the slowness of the SQL DB show up in everyday logs. 2019-10-16 22:32:47 +02:00
TomZ 680437f16f Merge branch '2019.07' 2019-10-05 17:12:15 +02:00
TomZ 0e62640ba9 SQL DISTINCT is a keyword, make it DB agnostic 2019-10-02 17:41:36 +02:00
TomZ 5c5967c9cf Use Message::reply() and remove some boilerplate code. 2019-10-02 17:39:11 +02:00
TomZ 89c0f15955 Use Message::reply() and remove some boilerplate code. 2019-10-02 16:44:20 +02:00
TomZ 6b50a75053 Fix typo in comments 2019-09-13 15:12:54 +02:00
TomZ 425ee250cb Fixlets from static code inspector 2019-09-12 15:21:48 +02:00
TomZ 83d455a5d4 Help devops with more info. 2019-08-29 22:33:57 +02:00
TomZ b6c4ac6f88 Fixes in SQL speed 
This explicitly detects the initial sync and refuses to create
any indexes on the SQL databases during this massive insert stage.

Some other minor issues got fixed here too.
 2019-08-29 22:33:57 +02:00
TomZ 191bc3758d Add a simpliefied NetworkManager::bind() method 
One that doesn't require a callback to be passed in.
 2019-08-24 22:25:10 +02:00
TomZ c3ddab9cd4 Make Indexer bind to localhost as default 2019-08-24 22:25:10 +02:00
TomZ 2bb06afa9e Cleanup the link-lines 2019-08-24 22:25:09 +02:00
TomZ da272d7d7f Fix SQL usage for postgres & better logging 2019-08-19 21:42:43 +02:00
TomZ d26386965a Fix defaults 
This solves the issue that unless you explicitly turn off the txdb you
still get it.
 2019-08-18 16:23:32 +02:00
TomZ 30bee6bf08 Make connecting a higher prio log 2019-08-16 23:39:40 +02:00
TomZ 691ff90014 Avoid uninit data from entering DB 
Also fix typo in variable name
 2019-08-15 10:55:41 +02:00
TomZ d89f38e4fb More fixlets in indexer after to the rewrite 2019-08-13 23:19:08 +02:00
TomZ 0fdba5a404 Add special postgres SQL statements 
When the user selected postgres, use specific SQL statements that are
optimised for it.
 2019-08-13 22:21:22 +02:00
TomZ d03be86166 Make indexer follow the tip again 
The rewrite of the synching code broke this, so fixing.
 2019-08-13 16:19:31 +02:00
TomZ 53c9badc51 Use atomics in the proper way. 2019-08-09 19:53:13 +02:00
TomZ a1ee211968 make compile with older Qt 2019-06-26 22:50:20 +02:00
TomZ 896f5e7b68 Merge branch 'master' into MessageBuilder::Reply 2019-06-26 22:20:44 +02:00
TomZ 98b3e7cece use new MessageBuilder::reply() 2019-06-26 22:19:28 +02:00
TomZ cb5ff404d3 Refactor the Indexer application 
This moves each indexer to live in its own thread, making blocking
operations allowed and Ok and separate from incoming requests.
 2019-06-26 21:39:07 +02:00
TomZ 6337b1b93b Create indexes 2019-06-19 16:55:51 +02:00
TomZ 9f041cf926 Move handling of address-db requests to another thread 
A slow SQL database means we should not do this on the networkmanager
thread as that causes issues.
 2019-06-19 16:55:47 +02:00
TomZ a6888b74d8 Need 2 connections to do multithreading 
Lets open 2 SQL connections to allow the bulk inserts
to happen in a different thread from the selects we do
resulting from incoming questions.
 2019-06-19 16:55:39 +02:00
TomZ fca434b1a6 Make Indexer connect to Hub after loading the conf 
Doing it before could lead to incoming data being
processed before the SQL database connection was setup.
 2019-06-13 17:12:28 +02:00
TomZ 00963dd31b Make parsing indexer.conf a bit more verbose 
And init our new variables.
 2019-06-12 23:53:26 +02:00