Commit Graph

348 Commits

Author SHA1 Message Date
TomZ fdaa6e4a81 Import some improvements from Calin.
* API docs
* Fix possible race condition in addOrphan()
* Reject proofs that violate simple constraints
* clear up possible confusing API (first/double spender getters).
2020-09-29 17:33:20 +02:00
TomZ 7afefefa41 Improve NWM flow-control
The NetworkManager now has more mature flow-control which means that we
send our buffers more regularly and we delay parsing received data if the
sending takes too long.

The direct effect should be that creating of a large number of messages
in response to incoming messages is no longer a problem in many cases.
2020-09-17 21:04:28 +02:00
TomZ 3cf670497d minor; remove comment 2020-09-11 18:54:21 +02:00
TomZ c7aee5efa9 Prevent incorrect usage and double delete
This only applies to the case where the MessageBuilder was used without
an external pool, but lets make sure the code is robust for even that
case.
2020-09-02 13:54:45 +02:00
TomZ dcf40f9161 Upgrade test to use qtestlib 2020-08-16 12:35:33 +02:00
TomZ 84ebae427e Import ASERT DAA
This includes renaming some methods which makes the import look larger
than it really is.
2020-08-12 21:42:54 +02:00
TomZ 297a22dfa8 Move and fix partitionCheck()
Small refactor; move the partition check away from main and the obsolete
global variables we used to use, instead now just add it to the DB which
owns the data it works on.

This fixes the bug that in various cases we'd incorrectly get a warning
about no blocks being found in the last N hours.
2020-08-10 10:13:02 +02:00
TomZ 6cd6c54140 Avoid duplication of numbers over sourcefiles
This simply moves the place where the number is changed to the settings
defaults instead of having various places copy/paste the value.
2020-07-23 14:45:42 +02:00
TomZ 436a061ddc Port merkle tests to qtestlib 2020-07-05 19:07:02 +02:00
TomZ 7e648699e7 Port some more test to qtestlib 2020-07-05 18:59:09 +02:00
TomZ 3c0367b091 Fix test
The test assumed a timing which naturally failed on some machines,
as such the test now checks for either of the two messages.
2020-07-05 16:31:49 +02:00
TomZ 3ca3311ee9 port crypt tests to qtestlib 2020-07-05 16:07:53 +02:00
TomZ 190394f0d7 Move bswap test to use qtestlib 2020-07-05 14:03:27 +02:00
TomZ 6fdbd2dfa9 Move arith_uint256_test to qtestlib 2020-07-05 14:01:54 +02:00
TomZ f716847705 Move univalue tests to qtestlib 2020-07-05 12:50:50 +02:00
TomZ 648eb56c9d Drop cmake add_dependencies() calls
When we made the 3rdparty libs be build by cmake the need to manually
add dependencies became irrelevant.
2020-07-05 12:50:09 +02:00
TomZ 34db927e0a Move unit test to qtestlib 2020-07-05 12:34:11 +02:00
TomZ 9429baf3dc Increase Tx init version
We don't actually use any new features, but users of our libraries might,
so just default to the new version.
2020-06-19 16:39:59 +02:00
TomZ c92b722c0f Fix test that broke on new Qt version
The usage of byte-array in json is not well defined, and as such Qt just
no longer handles it as it could end up causing out-of-bounds issues.

This misuse is limited to our unit test, so this is a cheap update.

Ref; https://bugreports.qt.io/browse/QTBUG-84739
2020-06-05 18:04:56 +02:00
TomZ ef90ed79d6 Avoid overloading virtual method
Instead of using open(), which makes unavailable the normal method,
use a new methodname instead.

This issue is a bit of an anti-pattern and it would likely be beneficial
to avoid having this as a subclass at all.
2020-06-05 16:44:39 +02:00
TomZ bab87a9900 Sync usage of random engine 2020-06-05 15:32:30 +02:00
TomZ dd251ba242 Merge branch '2020.04' 2020-06-01 20:03:15 +02:00
TomZ bd2ea54ca8 Increase timeout
Debug runs seem to take longer than the 5 seconds default too often that
this is an issue.
Make the timeout 30s
2020-06-01 20:02:01 +02:00
TomZ 72f1f3733c make clang happy; implicit coversion 2020-05-27 12:14:27 +02:00
TomZ 69a7db91ec Introduce TransactionMonitorService
This supplies a push notification on txid becoming known to the Hub.

Additionally this changes its close relative the AddressMonitorService
to use a different tag for the transaction data and the
double-spend-proof data so as to make it obvious which one is being
sent.
2020-05-19 20:57:51 +02:00
TomZ a5e6bea027 Merge branch '2020.03' 2020-05-18 22:52:19 +02:00
TomZ e9a2437c2d Add unit tests for UTXO and fixes
This fixes that the .info file is sometimes not written when nothing
changed in the db file.

This fixes that the info file with .1.info extension is reused again
and again, effectively reducing the usefulness of the snapshots.
This only really affects the indexer as pruning avoids this most
of the time.

Closes issue flowee-issues#6
2020-05-18 22:51:41 +02:00
TomZ d49b1ae5e2 Make NetworkManager connections leaner
The NetworkManager usage was mostly for low connection counts and this
made defaults selection easy.
With more usages it is important to allow the NWM-connection to be more
configurable about memory usage and leaner in general.

This changes the headers-buffers (used to create envelopes) to not be per
connection anymore but per thread using the tread_local keyword.

This changes the ring-buffers to become configurable using
NetworkConnections::setMessageQueueSizes().

Also removing some include statements where they were not really needed
in the P2PNet lib.
2020-05-09 20:23:05 +02:00
TomZ 2aa462f8bb Replace SigOps with SigChecks
This is part of the protocol upgrade for 2020-05-15, and in general it
seems to go the direction of "we did this before, lets do this again".

The spec is clear enough, but there is still a lack of questioning and
testing. The problem this attempts to fix has been neutered for years[1].

The spec states:
> The essential idea of SigChecks is to perform counting solely in the
> spending transaction, and count actual executed signature check
> operations.

This, however nobel and logical, ignores that the
check-for-being-too-costly just pulled in a UTXO lookup and the loading
of the output script from the historical chain.
The goal that we protect against CPU over-use may be reached, but the
price is a total system slowdown. You can have multiple CPUs, but the
bus to permanent storage has one, max 2 parallel pipes.

To ensure theHub stays the number one scalable node, I didn't blindly
follow the spec, while making sure that the Hub is correctly going to
follow/reject consensus violations of newly mined blocks.

As a result the implementation in Flowee the Hub:

* does not check sigcheck-counts on historical blocks (more than 1000
  blocks in the past).

  This may increase the risk of chain-splits ever so slightly, but the cost
  of disk-IO would be too high.

* No longer stores the value in the mempool, nor uses it for the
  CPU-miner.

* Ties the sigcheck-limits to the user-set block-size-accept-limit.

  This is contrary to the spec which mistakenly thinks that BCH has a
  max block-size in the consensus rules. The effect is the same, though.

* The per-intput standardness suggestion is not implemented because
  standardness checks don't currently fetch the previous outputs and
  that would be too expensive to add.

* Standardness rules for the whole transaction are moved to the
  mempool-acceptance logic instead. The cost would be too great
  otherwise, similar to the previous point.
  Again, the effect is the same as likely intented.

---
1) since the intro of the CachingTransactionSignatureChecker
2020-04-13 15:57:42 +02:00
TomZ 0386f38cc1 Inline CScriptCheck
It was only called twice, and in very close proximity. The class didn't
add anything.
This improves readability and with the new state its easier to write
too.
2020-04-13 13:44:18 +02:00
TomZ e101591f1a Refactor ScriptEval/ScriptVerify calls
Feeling cute, may update this API later.

namespace Script {
struct State {
    State() = default;
    State(uint32_t flags) : flags(flags) {}
    uint32_t flags = SCRIPT_VERIFY_NONE; // validation flags
    ScriptError error = SCRIPT_ERR_OK;

    const char* errorString() const;
};

bool eval(std::vector<std::vector<unsigned char> > &stack, const CScript
    &script, const BaseSignatureChecker checker, Script::State &state);
bool verify(const CScript& scriptSig, const CScript& scriptPubKey, const
    BaseSignatureChecker& checker, Script::State &state);
bool checkTransactionSignatureEncoding(const std::vector<unsigned char>
    &vchSig, State &state);
}

More of the same.
2020-04-12 18:08:09 +02:00
TomZ 337598a8bb Remove unused files 2020-04-11 17:04:14 +02:00
TomZ 414bee9e9f Implementation of OP_REVERSEBYTES
Previously named OP_REVERSE, then OP_BSWAP, then OP_ENDIAN_REVERSE.
2020-04-10 15:17:18 +02:00
TomZ 4733554817 Make bloom class more modern
This follows the coding style (method naming) better and it adds
an insert method which uses a Streaming::ConstBuffer argument.
2020-04-10 12:21:49 +02:00
TomZ fcdb52d5aa Make leveldb build with cmake
As this now means we completely use cmake, remove some legacy
stuff as well.
2020-04-08 22:48:29 +02:00
TomZ 1144f013a4 Build secp256k1 with cmake 2020-04-08 22:48:29 +02:00
TomZ ba76c35a7d Move merkle to flowee_utils libs
Merkle-block and merkle-tree classes and methods are pretty much stand-
alone and can be moved with no efforts.

Also move the relevent unit test to qtestlib.
2020-04-01 22:53:07 +02:00
TomZ 9276a45faf Move bloom filters to flowee_utils lib
Also move its unit test from the boost to qtestlib
2020-04-01 22:24:26 +02:00
TomZ 899a6e0f18 Make the API test a bit more stable.
The case where multiple hubs were started which cross-connected works
better if we have a small wait in-between the starts.
2020-03-19 18:53:35 +01:00
TomZ 09b4e5050e Make link under strictness rules
This re-adds the flowee_utils lib which was implied and should have
been auto-added by cmake, but it failed to link for AUR strictness
settings.
2020-03-19 16:08:56 +01:00
TomZ ac030e328c Fail fast. 2020-03-19 14:59:31 +01:00
TomZ 6e6f43a0d4 Remember peerID in DSP-orphanage
So when we adopt one proof, and it turns out to be faulty, we can
punish the peer.
2020-01-12 18:06:23 +01:00
TomZ d6a2488b5c Cleanup some DSP issues
remove the need for 'flags' as this code block doesn't use the
current flags at all.

Also validate proofs that were restored from orphan cache.
2020-01-12 17:10:29 +01:00
TomZ f7f4ea0093 make testing/streaming not link to flowee_server
Also split the testlib into two static libraries and avoid
recompile of the base test file in each test that used it.
2019-12-30 17:55:48 +01:00
TomZ 3315aec93f Add ConstBuffer::operator==() 2019-12-30 17:37:23 +01:00
TomZ f2191ad380 Make UTXO lookup report also the outindex
We have that info, it should be send.
2019-11-30 18:56:25 +01:00
TomZ d86c5a837f Make orphancache follow coding style-guide.
member methods start with a lowercase character.
2019-11-21 20:03:26 +01:00
TomZ 175096b2bd Refactor: move files
Move some files back to the server "library".
Merge the 'console' lib with server, as it doesn't really make sense with
just one file and nobody exclusively linking to it.

The server "libary" is not really a library, its the place we put all
the files shared by hub-qt hub-cli and hub.
We no longer depend on these files from other places (mostly due to
moving to the new logging framework) and as such we can move the files
back.
2019-11-13 19:09:24 +01:00
TomZ d573e307c3 Refactor; move SplitHostPort to utilstrencodings 2019-11-12 15:35:54 +01:00
TomZ 8e9808edaf Test: Follow changes in the API 2019-10-21 14:18:19 +02:00