62 Commits

Author SHA1 Message Date
tomFlowee 9e165d07ef Allow basedir to be given in the config file.
A general cleanup of how we start indexer.
2026-05-21 23:51:09 +02:00
tomFlowee 008eb35f95 Make compile faster
The IDE include checker got to the point where it is actually useful and
this removes a lot of unneeded includes.
Naturally, especially for headers like util.h, this may mean we need to
re-add includes in consuming cpp files that bloats the diff a bit.
2026-05-14 13:27:17 +02:00
tomFlowee 1dc2656bef Follow ASIO porting for apps too. 2025-02-11 19:41:22 +01:00
tomFlowee bb7275466b Stop using deprecated asio io_service
This ports the io-service to the source compatible io-context
class, with the most work going to the WorkerThreads which owns
that one.
2025-02-08 19:05:26 +01:00
tomFlowee bc47a700a4 Refactor; wrap BufferPool in shared_ptr
As we moved most of the creation of a BufferPool to be via the
Streaming::pool() method, which uses a thread-local, it makes sense
to start cleaning up the design and make it more modern C++.
The above mentioned method would return a reference and you'd see
loads of places use `auto &pool =` which is less than ideal.

As the number of places where we actually instantiate a BufferPool
goes down, the usage of some sort of smart pointer makes more sense.

This now makes all APIs use BufferPool be wrapped in a shared_ptr.
2023-12-21 15:23:23 +01:00
tomFlowee fce11e8d4f Rename file to match class it is in.
This follows the coding style guideline that the file that contains a
class should follow the exact name of this (main) class.

pubkey.{h|cpp} -> PublicKey.{h|cpp}
pubkey_utils.{h|cpp} -> PublicKeyUtils.{h|cpp}
2023-11-24 22:20:40 +01:00
tomFlowee 9fa877f425 Rename hashing methods to start with lowercase 2022-05-17 00:46:54 +02:00
tomFlowee 1d25e45a1c Reconcile more places to use pools API
This removes 2 more custom solutions to converge on one
  Streaming::pool()
namespaced method.
2022-01-25 23:20:12 +01:00
tomFlowee c71b6fa9c0 Fix off-by one and be more consistent
Lets default to using the plain and simple assert(), the Q_ASSERT macro
doesn't really add anything for us.
2021-02-06 23:18:21 +01:00
tomFlowee 3fe98afce9 Minor cleanup
This removes some unused includes from common headers and dead code
from some C++ files.
2021-01-05 22:05:25 +01:00
tomFlowee 9748d3988f Cleanups and improvements. 2020-12-25 23:51:06 +01:00
TomZ c244c94634 Add new Indexer API message: 'version'.
This mirrors the one in the Hub in that it returns the version of the
Indexer running on the server.
2020-07-27 21:27:26 +02:00
TomZ 11d16001c6 Indexer: add get for current sync-height 2020-01-20 12:17:55 +01:00
TomZ ce55457c33 Avoid asking for a block the hub won't have 2020-01-15 12:18:01 +01:00
TomZ 901860e406 Fix more edge-cases where indexer might stop updating. 2019-12-17 17:57:08 +01:00
TomZ 5796de30ba Be smarter about getting new block on connect 2019-12-12 23:19:26 +01:00
TomZ a85a9e1207 Indexer now uses blockCount
Instead of asking for a block and reacting when we get a failure, this
just uses the current height instead.

This also makes 'flush' happen in each indexers' own thread, which should
be good for performance.
2019-12-12 16:02:08 +01:00
TomZ 7b5e922658 Flush address DB faster.
Ask the hub for the chain-length and when we processed the last block,
immediately flush the data to the SQL DB.
This makes use represent the whole world faster.
2019-12-12 14:49:46 +01:00
TomZ d573e307c3 Refactor; move SplitHostPort to utilstrencodings 2019-11-12 15:35:54 +01:00
TomZ 748cd54a8e Fix log line limiter
Avoid logging being the slow part.
2019-11-06 23:20:17 +01:00
TomZ 79e4974d19 Port API getblock::filter to use script-hash
The API call for GetBlock has a filter-on-address functionality which is
now ported to no longer use ripe160 addresses but instead uses output-
script hashes.
This avoids problems for transactions not using p2pkh and generally is a
cleaner solution.

This also adds a unit test to test this feature.
2019-10-20 22:12:09 +02:00
TomZ 2396938cfb New version and only connect to new versions 2019-10-20 11:57:01 +02:00
TomZ 00814e24f9 API: Use hash of outscript instead of address
The usage of a ripe160 for bitcoin addresses in the API and in the
Indexer loses some info, specifically what kind of script it is.
Additionally not all types of scripts fit this mold. At best that means
its not future-proof.

This adds a method to the API in order to select from a Tx the hashed
outscript (thats singlehashed sha256) and refactor the address indexer
to use that instead of the ripe160 address.

The API enums broke a little, so I used the opportunity to break it a
lot and clean up the enums in order to make them more future-proof.
But, yeah, software from before this commit is protocol incompatible
with software after this commit.
2019-10-16 22:57:50 +02:00
TomZ 89c0f15955 Use Message::reply() and remove some boilerplate code. 2019-10-02 16:44:20 +02:00
TomZ 6b50a75053 Fix typo in comments 2019-09-13 15:12:54 +02:00
TomZ 425ee250cb Fixlets from static code inspector 2019-09-12 15:21:48 +02:00
TomZ 83d455a5d4 Help devops with more info. 2019-08-29 22:33:57 +02:00
TomZ b6c4ac6f88 Fixes in SQL speed
This explicitly detects the initial sync and refuses to create
any indexes on the SQL databases during this massive insert stage.

Some other minor issues got fixed here too.
2019-08-29 22:33:57 +02:00
TomZ 191bc3758d Add a simpliefied NetworkManager::bind() method
One that doesn't require a callback to be passed in.
2019-08-24 22:25:10 +02:00
TomZ da272d7d7f Fix SQL usage for postgres & better logging 2019-08-19 21:42:43 +02:00
TomZ d26386965a Fix defaults
This solves the issue that unless you explicitly turn off the txdb you
still get it.
2019-08-18 16:23:32 +02:00
TomZ 30bee6bf08 Make connecting a higher prio log 2019-08-16 23:39:40 +02:00
TomZ d89f38e4fb More fixlets in indexer after to the rewrite 2019-08-13 23:19:08 +02:00
TomZ 0fdba5a404 Add special postgres SQL statements
When the user selected postgres, use specific SQL statements that are
optimised for it.
2019-08-13 22:21:22 +02:00
TomZ d03be86166 Make indexer follow the tip again
The rewrite of the synching code broke this, so fixing.
2019-08-13 16:19:31 +02:00
TomZ 53c9badc51 Use atomics in the proper way. 2019-08-09 19:53:13 +02:00
TomZ a1ee211968 make compile with older Qt 2019-06-26 22:50:20 +02:00
TomZ 896f5e7b68 Merge branch 'master' into MessageBuilder::Reply 2019-06-26 22:20:44 +02:00
TomZ 98b3e7cece use new MessageBuilder::reply() 2019-06-26 22:19:28 +02:00
TomZ cb5ff404d3 Refactor the Indexer application
This moves each indexer to live in its own thread, making blocking
operations allowed and Ok and separate from incoming requests.
2019-06-26 21:39:07 +02:00
TomZ 9f041cf926 Move handling of address-db requests to another thread
A slow SQL database means we should not do this on the networkmanager
thread as that causes issues.
2019-06-19 16:55:47 +02:00
TomZ fca434b1a6 Make Indexer connect to Hub after loading the conf
Doing it before could lead to incoming data being
processed before the SQL database connection was setup.
2019-06-13 17:12:28 +02:00
TomZ 00963dd31b Make parsing indexer.conf a bit more verbose
And init our new variables.
2019-06-12 23:53:26 +02:00
TomZ d172dc4063 Don't fetch data when the node has an older chain.
For instance when its reindexing.
2019-06-11 21:43:50 +02:00
TomZ 9e69a3fef9 Add new indexer; spentoutput
where the UTXO allows you to find an output based on txid+output, the
spentoutput DB turns that around and allows you to find an input that
spends a certain output.

This also fits perfectly find into the UTXO database class, so this
creates a new dir "spent" which is filled as expected.
2019-06-11 17:38:30 +02:00
TomZ ec2610139e Allow setting indexer connections in conf file
This adds the ability to read from the config file the network
connections. Both the listening address and port as well as the upstream
'hub' address.
2019-06-06 19:44:16 +02:00
TomZ 0c9f415eb8 Several NetworkManager fixes
This adds all the header ints to chunked messages, solving the problem
of losing RequestId on roundtrips if the answer was too big.

Added a unit test for this "new" features.

This makes callbacks all use shared_from_this() in order to avoid
callbacks being done on deleted instances (thanks boost!).

Last, special case when the user doesn't connect but just sends messages
which caused a send of data, only to realize the connection wasn't open,
and then a connect.
This makes sure we immediately start a connect on queue of a message.
2019-06-04 21:40:02 +02:00
TomZ feb73bd1a8 Improve loglevel and naming. 2019-06-03 21:38:21 +02:00
TomZ edd4cada30 Make compile on older Qt
The method I choose was introduced in Qt 5.8.
This change makes it work on any Qt5 based release.
2019-04-11 19:59:54 +02:00
TomZ ed7ca33d6a Add decoding and printing addresses work.
This now supports cashaddress, legacy addresses and it prints
basic info about the usages of said address.
2019-04-11 18:45:28 +02:00