Skip to content

Latest commit

 

History

History
168 lines (134 loc) · 5.86 KB

lnddatabasecompaction.md

File metadata and controls

168 lines (134 loc) · 5.86 KB

Compact the LND database (channel.db)

An over 1GB channel.db file does not work on 32bit systems: lightningnetwork/lnd#4811

# check the size of channel.db
sudo du -h /mnt/hdd/lnd/data/graph/mainnet/channel.db
# example output
# 1.0G    /mnt/hdd/lnd/data/graph/mainnet/channel.db

Auto-compact on restart

Since LND v0.12.0 can set db.bolt.auto-compact=true in the lnd.conf.

  • To edit:

    sudo nano /mnt/hdd/lnd/lnd.conf

  • insert the following (can leave out the comments):

     [bolt]
 # Whether the databases used within lnd should automatically be compacted on
 # every startup (and if the database has the configured minimum age). This is
 # disabled by default because it requires additional disk space to be available
 # during the compaction that is freed afterwards. In general compaction leads to
 # smaller database files.
 db.bolt.auto-compact=true
 # How long ago the last compaction of a database file must be for it to be
 # considered for auto compaction again. Can be set to 0 to compact on every
 # startup. (default: 168h)
 # db.bolt.auto-compact-min-age=0

  • restart lnd:

    sudo systemctl restart lnd

  • monitor the process (can take several minutes):

    sudo tail -fn 30 /mnt/hdd/lnd/logs/bitcoin/mainnet/lnd.log

  • might want to disable the auto-compact in lnd.conf and only activate on-demand to avoid long startup times:

     db.bolt.auto-compact=false

Compacting with Channels Tools

https://github.com/guggero/chantools#compactdb

  • Run the following commands in the RaspiBlitz terminal

    See the comments for what each command does.

# install chantools
# download, inspect and run the install script
wget https://raw.githubusercontent.com/openoms/lightning-node-management/master/lnd.updates/bonus.chantools.sh
cat bonus.chantools.sh
bash bonus.chantools.sh on

# stop lnd
sudo systemctl stop lnd

# change to the home directory of the bitcoin user
sudo su - bitcoin

# run the compacting
chantools compactdb --sourcedb /mnt/hdd/lnd/data/graph/mainnet/channel.db \
                --destdb /mnt/hdd/lnd/data/graph/mainnet/compacted.db

# check the size of the compacted.db
# (the first compacting will have the biggest effect)
du -h /mnt/hdd/lnd/data/graph/mainnet/compacted.db
# example output:
# 730M /mnt/hdd/lnd/data/graph/mainnet/compacted.db

# make sure lnd is not running (needs sudo)
exit
sudo systemctl stop lnd
sudo su - bitcoin

# backup the original database
mv /mnt/hdd/lnd/data/graph/mainnet/channel.db \
   /mnt/hdd/lnd/data/graph/mainnet/uncompacted.db

# move the compacted database in place of the old
mv /mnt/hdd/lnd/data/graph/mainnet/compacted.db \
   /mnt/hdd/lnd/data/graph/mainnet/channel.db

# exit the bitcoin user to admin
exit

# start lnd
sudo systemctl start lnd

# unlock the wallet
lncli unlock

Prune the revocation logs

  • available since LND v0.15.1

  • Does not replace the database compacting and needs to be done only once after updating to LND v0.15.1 and above

    # check the size of channel.db
sudo du -h /mnt/hdd/lnd/data/graph/mainnet/channel.db
# example output
# 1.0G    /mnt/hdd/lnd/data/graph/mainnet/channel.db

On a Raspiblitz or compatible system:

  • Edit the systemd service:

    sudo systemctl edit --full lnd

  • Edit the line starting with ExecStart= so it looks like:

    ExecStart=/usr/local/bin/lnd --configfile=/home/bitcoin/.lnd/lnd.conf --db.prune-revocation

  • CTRL+o, ENTER and CTRL+x to save then restart LND:

    sudo systemctl restart lnd

  • monitor the process in the logs:

    sudo tail -n 30 -f /mnt/hdd/lnd/logs/bitcoin/mainnet/lnd.log

  • can take 30-60 mins typically similar to compacting after a long time.

    Example output when pruning the revocation logs:
[INF] LTND: Version: 0.15.1-beta commit=v0.15.1-beta, build=production, logging=default, debuglevel=info
[INF] LTND: Active chain: Bitcoin (network=mainnet)
...
[INF] LTND: Opening the main database, this might take a few minutes...
[INF] LTND: Opening bbolt database, sync_freelist=false, auto_compact=false
[INF] LTND: Creating local graph and channel state DB instances
[INF] CHDB: Checking for schema update: latest_version=29, db_version=27
[INF] CHDB: Performing database schema migration
[INF] CHDB: Applying migration #28
[INF] CHDB: Creating top-level bucket: "chan-id-bucket" ...
[INF] CHDB: Created top-level bucket: "chan-id-bucket"
[INF] CHDB: Applying migration #29
[INF] CHDB: Checking for optional update: prune_revocation_log=true, db_version=empty
[INF] CHDB: Performing database optional migration: prune revocation log
[INF] CHDB: Migrating revocation logs, might take a while...
[INF] CHDB: Total logs=1358156, migrated=0
...
[INF] CHDB: Migration progress: 76.366%, still have: 320989
[INF] CHDB: Migration progress: 89.584%, still have: 141464
[INF] CHDB: Migration progress: 97.048%, still have: 40095
[INF] CHDB: Migrating old revocation logs finished, now checking the migration results...
[INF] CHDB: Migration check passed, now deleting the old logs...
[INF] CHDB: Old revocation log buckets removed!
[INF] LTND: Database(s) now open (time_to_open=7m1.713662454s)!

  • after the pruning has finished the node comes back online

  • will need to compact once again to remove the pruned logs and reduce the size of the channel.db

  • example output of the compaction after pruning the revocation logs:

    [INF] CHDB: DB compaction of /home/bitcoin/.lnd/data/graph/mainnet/channel.db successful, 3726725120 -> 1219481600 bytes (gain=3.06x)
[INF] CHDB: Swapping old DB file from /home/bitcoin/.lnd/data/graph/mainnet/temp-dont-use.db to /home/bitcoin/.lnd/data/graph/mainnet/channel.db