Merge branch 'dev' into new_zaino-ci_docker

Author: nachog00

Diff for: .dockerignore

@@ -5,4 +5,4 @@ target/
 .dockerignore
 Dockerfile
 *.md
-.env* 
+.env*

Diff for: CONTRIBUTING.md

@@ -17,12 +17,18 @@ To get started using Zaino, please see our [use cases document](./docs/use_cases
 We welcome and appreciate contributions in the form of code, documentation, bug reports and feature requests. We also generally enjoy feedback and outreach efforts.
 
 ## How to Contribute
-Code and documentation are very helpful and the lifeblood of Free Software. To merge in code to this repo, one will have to have a [GitHub account](https://docs.github.com/en/account-and-profile), and the ability to cryptographically verify commits against this identity. 
+Code and documentation are very helpful and the lifeblood of Free Software. To merge in code to this repo, one will have to have a [GitHub account](https://docs.github.com/en/account-and-profile), and the ability to cryptographically verify commits against this identity.
+
 The best way to verify is using a GPG signature. See [this document about commit signature verification.](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
+
 Code, being Rust, should be formatted using `rustfmt` and applying the `clippy` suggestions.
+
 Code should be as complex as it needs to be, but no more.
-All code will be reviewed in public, as conversations on the pull request. It is very possible there will be requested changed or questions. This is not a sign of disrespect, but needed to keep code quality high in an important piece of software in the Zcash ecosystem.
+
+All code will be reviewed in public, as conversations on the pull request. It is very possible there will be requested changes or questions. This is not a sign of disrespect, but is necessary to keep code quality high in an important piece of software in the Zcash ecosystem.
+
 Documentation should be clear and accurate to the latest commit on `dev`.
+
 These contributions must be [GitHub pull requests](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests) opened _from a personal fork_ of the project, _to this repo, zingolabs/zaino_. Generally pull requests will be against `dev`, the development branch.
 
 ## Bug Reports and Feature Requests
@@ -75,5 +81,5 @@ Building upon the work of others,
 Better quality control,
 
 Reduced maintenance costs.
- 
+
 To read more, see [this document on wikibooks](https://en.wikibooks.org/wiki/FOSS_A_General_Introduction/Preface).

Diff for: Cargo.lock

@@ -65,7 +65,7 @@ tonic-build = "0.12"
 prost = "0.13"
 serde = "1.0"
 serde_json = "1.0"
-jsonrpc-core = "18.0"
+jsonrpsee-core = "0.24"
 jsonrpsee-types = "0.24"
 
 # Hashmaps, channels, DBs

Diff for: Cargo.toml

@@ -32,13 +32,13 @@ The Zaino repo consists of several crates that collectively provide an indexing
 
 **Runtime:**
 - tokio
-- tokio-stream 
+- tokio-stream
 
 **CLI:**
 - clap
 
 **Tracing:**
-- tracing 
+- tracing
 - tracing-subscriber
 - tracing-futures
 
@@ -50,33 +50,33 @@ The Zaino repo consists of several crates that collectively provide an indexing
 - tonic
 - tonic-build
 - prost
-- serde 
+- serde
 - serde_json
-- jsonrpc-core 
+- jsonrpsee-core
 - jsonrpsee-types
 
 **Hashmaps, channels, DBs:**
-- indexmap 
+- indexmap
 - crossbeam-channel
-- dashmap 
-- lmdb 
+- dashmap
+- lmdb
 
 **Async:**
-- async-stream 
-- async-trait 
-- futures 
+- async-stream
+- async-trait
+- futures
 
 **Utility:**
 - thiserror
 - lazy-regex
 - once_cell
-- ctrlc 
+- ctrlc
 - chrono
-- which 
-- whoami 
+- which
+- whoami
 
 **Formats:**
-- base64 
+- base64
 - byteorder
 - sha2
 - hex

Diff for: docs/internal_spec.md

@@ -26,4 +26,4 @@ Zaino Currently Serves the following gRPC services as defined in the [LightWalle
 Zaino has also committed to taking over responsibility for serving all [Zcash RPC Services](https://zcash.github.io/rpc/) required by non-validator (miner) clients from Zcashd.
 A full specification of the Zcash RPC services served by Zaino, and their current state of development, can be seen [here](./Zaino-zcash-rpcs.pdf).
 
-  
+

Diff for: docs/rpc_api.md

@@ -5,32 +5,37 @@
 3) [Zcashd, Zcash-Cli](https://github.com/zcash/zcash)
 
 ### Tests
-1) Simlink or copy compiled `zebrad`, zcashd` and `zcash-cli` binaries to `$ zaino/test_binaries/bins/*`
+1) Symlink or copy compiled `zebrad`, `zcashd` and `zcash-cli` binaries to `zaino/test_binaries/bins/*`
+
+**Chain Cache** _Several tests rely on a cached chain to run, for these tests to pass the chain must first be generated:_
 
-Chain Cache: Several tests rely on a cached chain to run, for these tests to pass the chain must first be generated:
 2) Generate the zcashd chain cache `cargo nextest run generate_zcashd_chain_cache --run-ignored ignored-only`
 3) Generate the zebrad chain cache `cargo nextest run generate_zebrad_large_chain_cache --run-ignored ignored-only`
 
-Client Rpc Tests: For the client rpc tests to pass a Zaino release binary must be built and added to PATH:
-4) Build release binary `cargo build --release` and add to PATH. WARNING: these tests do not use the binary built by cargo nextest
+**Client Rpc Tests** _For the client rpc tests to pass a Zaino release binary must be built and added to PATH.
+WARNING: these tests do not use the binary built by cargo nextest._
+
+4) Build release binary `cargo build --release` and add to PATH. For example, `export PATH=./target/release:$PATH`
 
-5) Run `$ cargo nextest run`
+5) Run `cargo nextest run`
 
 NOTE: The client rpc get_subtree_roots tests are currently ignored, to run them testnet and mainnet chains must first be generated.
-- To run client rpc test `get_subtree_roots_sapling`:
+
+To run client rpc test `get_subtree_roots_sapling`:
 1) sync Zebrad testnet to at least 2 sapling shards
 2) copy the Zebrad testnet `state` cache to `zaino/integration-tests/chain_cache/get_subtree_roots_sapling` directory.
+
 See the `get_subtree_roots_sapling` test fixture doc comments in infrastructure for more details.
 
-- To run client rpc test `get_subtree_roots_orchard`:
+To run client rpc test `get_subtree_roots_orchard`:
 1) sync Zebrad mainnet to at least 2 orchard shards
 2) copy the Zebrad mainnet `state` cache to `zaino/integration-tests/chain_cache/get_subtree_roots_orchard` directory.
+
 See the `get_subtree_roots_orchard` test fixture doc comments in infrastructure for more details.
 
 - TESTNET TESTS:
-the testnet tests are temporary and will be replaced with regtest as soon as (https://github.com/zingolabs/zaino/issues/231) is resolved
-In the mean time, these tests can be ran, but it is a fiddly process. First, it needs a zebrad fully synced to testnet (depending
-on internet speed, etc., this could take 10+ hours). Then, the tests must be run 1 at a time (passing `--no-capture` will enforce this).
-Furthermore, due to https://github.com/zingolabs/infrastructure/issues/43, sometimes a zebrad will persist past the end of the test and
-hold a lock on the testnet cache, causing all remaining tests to fail. This process must be stopped manually, in order to allow testnet
-tests to work again.
+The testnet tests are temporary and will be replaced with regtest as soon as (https://github.com/zingolabs/zaino/issues/231) is resolved.
+In the mean time, these tests can be ran, but it is a fiddly process. First, it needs a zebrad fully synced to testnet (depending on internet speed, etc., this could take 10+ hours).
+To build the zebra testnet cache, the best way is to use zebra directly. With `zebrad` already in `$PATH` and from the `zaino/` directory run `zebrad --config ./zainod/zebrad.toml start`.
+Then, the tests must be run 1 at a time (passing `--no-capture` will enforce this).
+Furthermore, due to https://github.com/zingolabs/infrastructure/issues/43, sometimes a zebrad may persist past the end of the test and hold a lock on the testnet cache, causing remaining tests to fail. This process can be stopped manually, in order to allow testnet tests to work again.

Diff for: docs/testing.md

@@ -1,5 +1,5 @@
 use futures::StreamExt as _;
-use zaino_fetch::jsonrpc::connector::{test_node_and_return_url, JsonRpcConnector};
+use zaino_fetch::jsonrpsee::connector::{test_node_and_return_url, JsonRpSeeConnector};
 use zaino_proto::proto::service::{
     AddressList, BlockId, BlockRange, Exclude, GetAddressUtxosArg, GetSubtreeRootsArg,
     TransparentAddressBlockFilter, TxFilter,
@@ -221,7 +221,7 @@ async fn fetch_service_get_raw_mempool(validator: &str) {
         .as_ref()
         .expect("Clients are not initialized");
 
-    let json_service = JsonRpcConnector::new_with_basic_auth(
+    let json_service = JsonRpSeeConnector::new_with_basic_auth(
         test_node_and_return_url(
             test_manager.zebrad_rpc_listen_address,
             false,
@@ -586,7 +586,7 @@ async fn fetch_service_get_latest_block(validator: &str) {
     test_manager.local_net.generate_blocks(1).await.unwrap();
     tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;
 
-    let json_service = JsonRpcConnector::new_with_basic_auth(
+    let json_service = JsonRpSeeConnector::new_with_basic_auth(
         test_node_and_return_url(
             test_manager.zebrad_rpc_listen_address,
             false,

Diff for: integration-tests/tests/fetch_service.rs

@@ -1,5 +1,5 @@
 use core::panic;
-use zaino_fetch::jsonrpc::connector::{test_node_and_return_url, JsonRpcConnector};
+use zaino_fetch::jsonrpsee::connector::{test_node_and_return_url, JsonRpSeeConnector};
 use zaino_state::{
     config::BlockCacheConfig,
     local_cache::{BlockCache, BlockCacheSubscriber},
@@ -18,7 +18,7 @@ async fn create_test_manager_and_block_cache(
     enable_clients: bool,
 ) -> (
     TestManager,
-    JsonRpcConnector,
+    JsonRpSeeConnector,
     BlockCache,
     BlockCacheSubscriber,
 ) {
@@ -34,7 +34,7 @@ async fn create_test_manager_and_block_cache(
     .await
     .unwrap();
 
-    let json_service = JsonRpcConnector::new_with_basic_auth(
+    let json_service = JsonRpSeeConnector::new_with_basic_auth(
         test_node_and_return_url(
             test_manager.zebrad_rpc_listen_address,
             false,

Diff for: integration-tests/tests/local_cache.rs

@@ -8,7 +8,7 @@ use zingo_infra_testutils::services::validator::Validator;
 use zingolib::testutils::lightclient::from_inputs;
 
 mod wallet_basic {
-    use zaino_fetch::jsonrpc::connector::test_node_and_return_url;
+    use zaino_fetch::jsonrpsee::connector::test_node_and_return_url;
 
     use super::*;
 
@@ -199,20 +199,21 @@ mod wallet_basic {
         test_manager.local_net.generate_blocks(1).await.unwrap();
         tokio::time::sleep(std::time::Duration::from_millis(500)).await;
 
-        let fetch_service = zaino_fetch::jsonrpc::connector::JsonRpcConnector::new_with_basic_auth(
-            test_node_and_return_url(
-                test_manager.zebrad_rpc_listen_address,
-                false,
-                None,
-                Some("xxxxxx".to_string()),
-                Some("xxxxxx".to_string()),
+        let fetch_service =
+            zaino_fetch::jsonrpsee::connector::JsonRpSeeConnector::new_with_basic_auth(
+                test_node_and_return_url(
+                    test_manager.zebrad_rpc_listen_address,
+                    false,
+                    None,
+                    Some("xxxxxx".to_string()),
+                    Some("xxxxxx".to_string()),
+                )
+                .await
+                .unwrap(),
+                "xxxxxx".to_string(),
+                "xxxxxx".to_string(),
             )
-            .await
-            .unwrap(),
-            "xxxxxx".to_string(),
-            "xxxxxx".to_string(),
-        )
-        .unwrap();
+            .unwrap();
 
         println!("\n\nFetching Chain Height!\n");
 
@@ -543,20 +544,21 @@ mod wallet_basic {
 
         // test_manager.local_net.print_stdout();
 
-        let fetch_service = zaino_fetch::jsonrpc::connector::JsonRpcConnector::new_with_basic_auth(
-            test_node_and_return_url(
-                test_manager.zebrad_rpc_listen_address,
-                false,
-                None,
-                Some("xxxxxx".to_string()),
-                Some("xxxxxx".to_string()),
+        let fetch_service =
+            zaino_fetch::jsonrpsee::connector::JsonRpSeeConnector::new_with_basic_auth(
+                test_node_and_return_url(
+                    test_manager.zebrad_rpc_listen_address,
+                    false,
+                    None,
+                    Some("xxxxxx".to_string()),
+                    Some("xxxxxx".to_string()),
+                )
+                .await
+                .unwrap(),
+                "xxxxxx".to_string(),
+                "xxxxxx".to_string(),
             )
-            .await
-            .unwrap(),
-            "xxxxxx".to_string(),
-            "xxxxxx".to_string(),
-        )
-        .unwrap();
+            .unwrap();
 
         println!("\n\nFetching Raw Mempool!\n");
 

Diff for: integration-tests/tests/wallet_to_validator.rs

@@ -40,4 +40,3 @@ base64 = { workspace = true }
 byteorder = { workspace = true }
 sha2 = { workspace = true }
 jsonrpsee-types = { workspace = true }
-