Migrate from lmdb-zero to heed

[ardocrat]

Migrate from non-maintained lmdb-zero to heed.

• Speed up of sync as result
• No more stuck at parallel commands while syncing
• Single environment for peers and chain data, migrate existing dbs outside default lmdb environment (solves Use multiple LMDB dbs (tables) #3461)
• Ability to use multiple shared environments (per path).
• Use separate databases instead of prefixes, use default database for values without prefixes, migrate old environment (Use multiple LMDB dbs (tables) #3320)

[aglkm]

Compiled master branch with this PR and got the following core dump:

(gdb) bt
#0  __memcpy_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:265
#1  0x000055b7e12b2d8c in mdb_txn_renew0 (txn=0x55b7f38018f0)
    at /home/user/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/lmdb-master-sys-0.2.6/lmdb/libraries/liblmdb/mdb.c:3176
#2  0x000055b7e12b341e in mdb_txn_begin (env=0x55b7f36f4520, parent=0x0, flags=131072, ret=0x7ffe09ade890)
    at /home/user/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/lmdb-master-sys-0.2.6/lmdb/libraries/liblmdb/mdb.c:3349
#3  0x000055b7e128df1d in heed::txn::RoTxn<heed::txn::WithoutTls>::new<heed::txn::WithoutTls> (env=0x55b7f36f2270)
    at /home/user/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/heed-0.22.1/src/txn.rs:67
#4  0x000055b7e128f047 in heed::envs::env::Env<heed::txn::WithoutTls>::read_txn<heed::txn::WithoutTls> (self=0x55b7f36f2270)
    at /home/user/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/heed-0.22.1/src/envs/env.rs:427
#5  0x000055b7e1213e17 in grin_store::lmdb::Store::get_ser<grin_chain::types::Tip> (self=0x55b7f36f2240, key=..., deser_mode=...)
    at store/src/lmdb.rs:309
#6  0x000055b7e116d270 in grin_chain::store::ChainStore::head (self=0x55b7f36f2240) at chain/src/store.rs:65
#7  0x000055b7e1257799 in grin_chain::chain::Chain::head (self=0x55b7f36f9ce0) at chain/src/chain.rs:1457
#8  0x000055b7e124ae78 in grin_chain::chain::Chain::difficulty_iter (self=0x55b7f36f9ce0) at chain/src/chain.rs:1669
#9  0x000055b7e096ec81 in grin_servers::grin::server::Server::get_server_stats (self=0x7ffe09ae2b80) at servers/src/grin/server.rs:433
#10 0x000055b7e05f1b48 in grin::tui::ui::Controller::run (self=0x7ffe09ae25a0, server=...) at src/bin/tui/ui.rs:212
#11 0x000055b7e062cb62 in grin::cmd::server::start_server_tui::{closure#0} (serv=..., logs_rx=...) at src/bin/cmd/server.rs:60
#12 0x000055b7e06295e0 in grin_servers::grin::server::Server::start<grin::cmd::server::start_server_tui::{closure_env#0}> (
    config=..., logs_rx=..., info_callback=..., stop_state=..., api_chan=0x55b7f36f2180) at servers/src/grin/server.rs:121
#13 0x000055b7e064748a in grin::cmd::server::start_server_tui (config=..., logs_rx=..., api_chan=0x55b7f36f2180)
    at src/bin/cmd/server.rs:53
#14 0x000055b7e0645d31 in grin::cmd::server::start_server (config=..., logs_rx=..., api_chan=0x55b7f36f2180)
    at src/bin/cmd/server.rs:40
#15 0x000055b7e064670b in grin::cmd::server::server_command (server_args=..., global_config=..., logs_rx=..., api_chan=0x55b7f36f2180)
    at src/bin/cmd/server.rs:146
#16 0x000055b7e056f83e in grin::real_main () at src/bin/grin.rs:247
#17 0x000055b7e056d206 in grin::main () at src/bin/grin.rs:81

Stopped at: Sync step 1/7: Downloading headers: 13%

Not sure if the above stack is the place where the crash occurred, or just Linux defaulted to show thread 1 for bt command.
So here is the output from thread apply full bt command:
thread_apply_full_bt.txt

[wiesche89]

I think there is still a small race here, right? We check that open_txs_count is 0 first, and only then set resizing to true. In between, a new transaction could start because wait_for_resize() only blocks once resizing is true. Would it be safer to set resizing = true`before waiting for the open transactions to drain, then resize once the count reaches 0?

[ardocrat]

we also set resize_checking to true to avoid blocking of existing transactions so its must be safe to resize only when there is really 0 txs count and only 1 check is active. if we'll put resizing before checking txs count it will stuck at the loop of iterator with single opened tx since we are waiting resize to finish. If env.resize method will fail, it will be called again at next check.