Merge bitcoin#29775: Testnet4 including PoW difficulty adjustment fix

6bfa260 testnet: Add timewarp attack prevention for Testnet4 (Fabian Jahr)
0100907 testnet: Add Testnet4 difficulty adjustment rules fix (Fabian Jahr)
74a04f9 testnet: Introduce Testnet4 (Fabian Jahr)

Pull request description:

  To supplement the [ongoing conceptual discussion about a testnet reset](https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU/m/9yCPo3uUBwAJ) I have drafted a move to v4 including a fix to the difficulty adjustment mechanism, which was part of the motivation that started the discussion.

  Conceptual considerations:
  - The conceptual discussion about doing a testnet4 or softforking the fix into testnet3 is outside of the scope of this PR and I would ask reviewers to contribute their opinions on this on the ML instead. However, I am happy to adapt this PR to a softfork change on testnet3 if there is consensus for that instead.
  - The difficulty adjustment fix suggested here touches the `CalculateNextWorkRequired` function and uses the same logic used in `GetNextWorkRequired` to find the last previous block that was not mined with difficulty 1 under the exceptionf. An alternative fix briefly mentioned on the mailing list by Jameson Lopp would be to "restrict the special testnet minimum difficulty rule so that it can't be triggered on the block right before a difficulty retarget". That would also fix the issue but I find my suggestion here a bit more elegant.

ACKs for top commit:
  jsarenik:
    tACK 6bfa260
  achow101:
    ACK 6bfa260
  murchandamus:
    tACK 6bfa260

Tree-SHA512: 0b8b69a621406a944da5be551b863d065358ba94d85dd3b80d83c412660e230ee93b27316081fbee9b4851cc4ff8585db64c7dfa26cb5148ac835663f2712c3d

Author: achow101

contrib/completions/bash/bitcoin-cli.bash

@@ -9,7 +9,7 @@ _bitcoin_rpc() {
     local rpcargs=()
     for i in ${COMP_LINE}; do
         case "$i" in
-            -conf=*|-datadir=*|-regtest|-rpc*|-testnet)
+            -conf=*|-datadir=*|-regtest|-rpc*|-testnet|-testnet4)
                 rpcargs=( "${rpcargs[@]}" "$i" )
                 ;;
         esac

contrib/seeds/generate-seeds.py

@@ -5,11 +5,12 @@
 '''
 Script to generate list of seed nodes for kernel/chainparams.cpp.
 
-This script expects two text files in the directory that is passed as an
+This script expects three text files in the directory that is passed as an
 argument:
 
     nodes_main.txt
     nodes_test.txt
+    nodes_testnet4.txt
 
 These files must consist of lines in the format
 
@@ -171,6 +172,9 @@ def main():
     g.write('\n')
     with open(os.path.join(indir,'nodes_test.txt'), 'r', encoding="utf8") as f:
         process_nodes(g, f, 'chainparams_seed_test')
+    g.write('\n')
+    with open(os.path.join(indir,'nodes_testnet4.txt'), 'r', encoding="utf8") as f:
+        process_nodes(g, f, 'chainparams_seed_testnet4')
     g.write('#endif // BITCOIN_CHAINPARAMSSEEDS_H\n')
 
 if __name__ == '__main__':

doc/REST-interface.md

@@ -4,7 +4,7 @@ Unauthenticated REST Interface
 The REST API can be enabled with the `-rest` option.
 
 The interface runs on the same port as the JSON-RPC interface, by default port 8332 for mainnet, port 18332 for testnet,
-port 38332 for signet, and port 18443 for regtest.
+port 48332 for testnet4, port 38332 for signet, and port 18443 for regtest.
 
 REST Interface consistency guarantees
 -------------------------------------

doc/files.md

@@ -34,12 +34,13 @@ Windows  | `%LOCALAPPDATA%\Bitcoin\` <sup>[\[1\]](#note1)</sup>
 
 3. All content of the data directory, except for `bitcoin.conf` file, is chain-specific. This means the actual data directory paths for non-mainnet cases differ:
 
-Chain option                   | Data directory path
--------------------------------|------------------------------
-`-chain=main` (default)        | *path_to_datadir*`/`
-`-chain=test` or `-testnet`    | *path_to_datadir*`/testnet3/`
-`-chain=signet` or `-signet`   | *path_to_datadir*`/signet/`
-`-chain=regtest` or `-regtest` | *path_to_datadir*`/regtest/`
+Chain option                     | Data directory path
+---------------------------------|------------------------------
+`-chain=main` (default)          | *path_to_datadir*`/`
+`-chain=test` or `-testnet`      | *path_to_datadir*`/testnet3/`
+`-chain=testnet4` or `-testnet4` | *path_to_datadir*`/testnet4/`
+`-chain=signet` or `-signet`     | *path_to_datadir*`/signet/`
+`-chain=regtest` or `-regtest`   | *path_to_datadir*`/regtest/`
 
 ## Data directory layout
 

doc/release-process.md

@@ -311,13 +311,15 @@ Both variables are used as a guideline for how much space the user needs on thei
 Note that all values should be taken from a **fully synced** node and have an overhead of 5-10% added on top of its base value.
 
 To calculate `m_assumed_blockchain_size`, take the size in GiB of these directories:
-- For `mainnet` -> the data directory, excluding the `/testnet3`, `/signet`, and `/regtest` directories and any overly large files, e.g. a huge `debug.log`
+- For `mainnet` -> the data directory, excluding the `/testnet3`, `/testnet4`, `/signet`, and `/regtest` directories and any overly large files, e.g. a huge `debug.log`
 - For `testnet` -> `/testnet3`
+- For `testnet4` -> `/testnet4`
 - For `signet` -> `/signet`
 
 To calculate `m_assumed_chain_state_size`, take the size in GiB of these directories:
 - For `mainnet` -> `/chainstate`
 - For `testnet` -> `/testnet3/chainstate`
+- For `testnet4` -> `/testnet4/chainstate`
 - For `signet` -> `/signet/chainstate`
 
 Notes:

src/bitcoin-cli.cpp

@@ -75,6 +75,7 @@ static void SetupCliArgs(ArgsManager& argsman)
 
     const auto defaultBaseParams = CreateBaseChainParams(ChainType::MAIN);
     const auto testnetBaseParams = CreateBaseChainParams(ChainType::TESTNET);
+    const auto testnet4BaseParams = CreateBaseChainParams(ChainType::TESTNET4);
     const auto signetBaseParams = CreateBaseChainParams(ChainType::SIGNET);
     const auto regtestBaseParams = CreateBaseChainParams(ChainType::REGTEST);
 
@@ -98,7 +99,7 @@ static void SetupCliArgs(ArgsManager& argsman)
     argsman.AddArg("-rpcconnect=<ip>", strprintf("Send commands to node running on <ip> (default: %s)", DEFAULT_RPCCONNECT), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-rpccookiefile=<loc>", "Location of the auth cookie. Relative paths will be prefixed by a net-specific datadir location. (default: data dir)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-rpcpassword=<pw>", "Password for JSON-RPC connections", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
-    argsman.AddArg("-rpcport=<port>", strprintf("Connect to JSON-RPC on <port> (default: %u, testnet: %u, signet: %u, regtest: %u)", defaultBaseParams->RPCPort(), testnetBaseParams->RPCPort(), signetBaseParams->RPCPort(), regtestBaseParams->RPCPort()), ArgsManager::ALLOW_ANY | ArgsManager::NETWORK_ONLY, OptionsCategory::OPTIONS);
+    argsman.AddArg("-rpcport=<port>", strprintf("Connect to JSON-RPC on <port> (default: %u, testnet: %u, testnet4: %u, signet: %u, regtest: %u)", defaultBaseParams->RPCPort(), testnetBaseParams->RPCPort(), testnet4BaseParams->RPCPort(), signetBaseParams->RPCPort(), regtestBaseParams->RPCPort()), ArgsManager::ALLOW_ANY | ArgsManager::NETWORK_ONLY, OptionsCategory::OPTIONS);
     argsman.AddArg("-rpcuser=<user>", "Username for JSON-RPC connections", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-rpcwait", "Wait for RPC server to start", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
     argsman.AddArg("-rpcwaittimeout=<n>", strprintf("Timeout in seconds to wait for the RPC server to start, or 0 for no timeout. (default: %d)", DEFAULT_WAIT_CLIENT_TIMEOUT), ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::OPTIONS);
@@ -428,6 +429,8 @@ class NetinfoRequestHandler : public BaseRequestHandler
     std::string ChainToString() const
     {
         switch (gArgs.GetChainType()) {
+        case ChainType::TESTNET4:
+            return " testnet4";
         case ChainType::TESTNET:
             return " testnet";
         case ChainType::SIGNET:

src/bitcoin-wallet.cpp

@@ -69,7 +69,7 @@ static std::optional<int> WalletAppInit(ArgsManager& args, int argc, char* argv[
             strUsage += "\n"
                         "bitcoin-wallet is an offline tool for creating and interacting with " PACKAGE_NAME " wallet files.\n"
                         "By default bitcoin-wallet will act on wallets in the default mainnet wallet directory in the datadir.\n"
-                        "To change the target wallet, use the -datadir, -wallet and -regtest/-signet/-testnet arguments.\n\n"
+                        "To change the target wallet, use the -datadir, -wallet and -regtest/-signet/-testnet/-testnet4 arguments.\n\n"
                         "Usage:\n"
                         "  bitcoin-wallet [options] <command>\n";
             strUsage += "\n" + args.GetHelpMessage();

src/chainparams.cpp

@@ -115,6 +115,8 @@ std::unique_ptr<const CChainParams> CreateChainParams(const ArgsManager& args, c
         return CChainParams::Main();
     case ChainType::TESTNET:
         return CChainParams::TestNet();
+    case ChainType::TESTNET4:
+        return CChainParams::TestNet4();
     case ChainType::SIGNET: {
         auto opts = CChainParams::SigNetOptions{};
         ReadSigNetArgs(args, opts);

src/chainparamsbase.cpp

@@ -17,7 +17,8 @@ void SetupChainParamsBaseOptions(ArgsManager& argsman)
     argsman.AddArg("-regtest", "Enter regression test mode, which uses a special chain in which blocks can be solved instantly. "
                  "This is intended for regression testing tools and app development. Equivalent to -chain=regtest.", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CHAINPARAMS);
     argsman.AddArg("-testactivationheight=name@height.", "Set the activation height of 'name' (segwit, bip34, dersig, cltv, csv). (regtest-only)", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
-    argsman.AddArg("-testnet", "Use the test chain. Equivalent to -chain=test.", ArgsManager::ALLOW_ANY, OptionsCategory::CHAINPARAMS);
+    argsman.AddArg("-testnet", "Use the testnet3 chain. Equivalent to -chain=test. Support for testnet3 is deprecated and will be removed with the next release. Consider moving to testnet4 now by using -testnet4.", ArgsManager::ALLOW_ANY, OptionsCategory::CHAINPARAMS);
+    argsman.AddArg("-testnet4", "Use the testnet4 chain. Equivalent to -chain=testnet4.", ArgsManager::ALLOW_ANY, OptionsCategory::CHAINPARAMS);
     argsman.AddArg("-vbparams=deployment:start:end[:min_activation_height]", "Use given start/end times and min_activation_height for specified version bits deployment (regtest-only)", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::CHAINPARAMS);
     argsman.AddArg("-signet", "Use the signet chain. Equivalent to -chain=signet. Note that the network is defined by the -signetchallenge parameter", ArgsManager::ALLOW_ANY, OptionsCategory::CHAINPARAMS);
     argsman.AddArg("-signetchallenge", "Blocks must satisfy the given script to be considered valid (only for signet networks; defaults to the global default signet test network challenge)", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::CHAINPARAMS);
@@ -33,7 +34,7 @@ const CBaseChainParams& BaseParams()
 }
 
 /**
- * Port numbers for incoming Tor connections (8334, 18334, 38334, 18445) have
+ * Port numbers for incoming Tor connections (8334, 18334, 38334, 48334, 18445) have
  * been chosen arbitrarily to keep ranges of used ports tight.
  */
 std::unique_ptr<CBaseChainParams> CreateBaseChainParams(const ChainType chain)
@@ -43,6 +44,8 @@ std::unique_ptr<CBaseChainParams> CreateBaseChainParams(const ChainType chain)
         return std::make_unique<CBaseChainParams>("", 8332, 8334);
     case ChainType::TESTNET:
         return std::make_unique<CBaseChainParams>("testnet3", 18332, 18334);
+    case ChainType::TESTNET4:
+        return std::make_unique<CBaseChainParams>("testnet4", 48332, 48334);
     case ChainType::SIGNET:
         return std::make_unique<CBaseChainParams>("signet", 38332, 38334);
     case ChainType::REGTEST:

src/chainparamsseeds.h

@@ -1786,4 +1786,16 @@ static const uint8_t chainparams_seed_test[] = {
     0x04,0x20,0xbd,0x0e,0xc8,0x73,0x43,0xa7,0xc6,0x25,0x15,0xcf,0x3e,0x23,0xa8,0xb0,0xbf,0xe8,0x20,0xa7,0xec,0x2a,0xf6,0x37,0x6c,0x60,0x5e,0x4d,0xed,0xf4,0xb1,0xef,0xf7,0xb2,0x47,0x9d,
     0x04,0x20,0xc8,0x88,0xfe,0x71,0x5f,0xa3,0x6c,0x96,0x6a,0xd7,0x9e,0x38,0x84,0x9f,0x44,0xe1,0x6b,0xdc,0x98,0x31,0xad,0x96,0x29,0xe7,0x00,0x83,0x63,0x03,0xae,0x69,0x2e,0x63,0x47,0x9d,
 };
+
+static const uint8_t chainparams_seed_testnet4[] = {
+    0x01,0x04,0x39,0x80,0xb0,0xa3,0xbc,0xcd,
+    0x01,0x04,0x33,0x9e,0xf8,0x08,0xbc,0xcd,
+    0x01,0x04,0x5f,0xd9,0x49,0xa2,0xbc,0xcd,
+    0x01,0x04,0x12,0xbd,0x9c,0x66,0xbc,0xcd,
+    0x01,0x04,0x67,0x63,0xab,0xd4,0xbc,0xcd,
+    0x01,0x04,0x52,0x43,0x66,0x0f,0xbc,0xcd,
+    0x01,0x04,0x58,0x63,0xf8,0x32,0xbc,0xcd,
+    0x01,0x04,0x67,0xa5,0xc0,0xd2,0xbc,0xcd,
+    0x01,0x04,0x12,0xc9,0xcf,0x37,0xbc,0xcd,
+};
 #endif // BITCOIN_CHAINPARAMSSEEDS_H

src/common/args.cpp

@@ -159,6 +159,7 @@ std::list<SectionInfo> ArgsManager::GetUnrecognizedSections() const
         ChainTypeToString(ChainType::REGTEST),
         ChainTypeToString(ChainType::SIGNET),
         ChainTypeToString(ChainType::TESTNET),
+        ChainTypeToString(ChainType::TESTNET4),
         ChainTypeToString(ChainType::MAIN),
     };
 
@@ -773,10 +774,11 @@ std::variant<ChainType, std::string> ArgsManager::GetChainArg() const
     const bool fRegTest = get_net("-regtest");
     const bool fSigNet  = get_net("-signet");
     const bool fTestNet = get_net("-testnet");
+    const bool fTestNet4 = get_net("-testnet4");
     const auto chain_arg = GetArg("-chain");
 
-    if ((int)chain_arg.has_value() + (int)fRegTest + (int)fSigNet + (int)fTestNet > 1) {
-        throw std::runtime_error("Invalid combination of -regtest, -signet, -testnet and -chain. Can use at most one.");
+    if ((int)chain_arg.has_value() + (int)fRegTest + (int)fSigNet + (int)fTestNet + (int)fTestNet4 > 1) {
+        throw std::runtime_error("Invalid combination of -regtest, -signet, -testnet, -testnet4 and -chain. Can use at most one.");
     }
     if (chain_arg) {
         if (auto parsed = ChainTypeFromString(*chain_arg)) return *parsed;
@@ -786,6 +788,7 @@ std::variant<ChainType, std::string> ArgsManager::GetChainArg() const
     if (fRegTest) return ChainType::REGTEST;
     if (fSigNet) return ChainType::SIGNET;
     if (fTestNet) return ChainType::TESTNET;
+    if (fTestNet4) return ChainType::TESTNET4;
     return ChainType::MAIN;
 }
 

src/common/args.h

@@ -423,7 +423,7 @@ class ArgsManager
     fs::path GetDataDir(bool net_specific) const;
 
     /**
-     * Return -regtest/-signet/-testnet/-chain= setting as a ChainType enum if a
+     * Return -regtest/-signet/-testnet/-testnet4/-chain= setting as a ChainType enum if a
      * recognized chain type was set, or as a string if an unrecognized chain
      * name was set. Raise an exception if an invalid combination of flags was
      * provided.

src/consensus/params.h

@@ -108,6 +108,7 @@ struct Params {
     /** Proof of work parameters */
     uint256 powLimit;
     bool fPowAllowMinDifficultyBlocks;
+    bool enforce_BIP94;
     bool fPowNoRetargeting;
     int64_t nPowTargetSpacing;
     int64_t nPowTargetTimespan;