Skip to content

Enable compression option -c during ZFS Send #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

hcpss-gdelrosario
Copy link

This change will allow PVE to utilize ZFS Send -c compression option to speed up intra-cluster migration and replication.

The current implementation does not compress streams which significantly slows down migration on VMs/CTs with large data sets (ie, databases, filestorage ...). For example, if a VM is assigned a 512GB ZFS dataset and is migrated into another host, the ZFS Send will transmit 512GB of data regardless if the VM internally used 1GB of disk space or +500GB of space. Using the -c option, upon migrating the VM, due to compression only used datablocks are sent significantly increasing speed.

Note that even if the underlying ZFS source and destination datasets have the setting compression=on (the default setting in PVE), ZFS Send without the -c option will "unpack" the blocks and send the blocks uncompressed to the receiver.

I've tested this on a production cluster on PVE 8.3.3 with many instances of VMs and CTs with ZFS datasets ranging from 16GB all the way up to 1TB+. Migration and replication works as they did before without any issues.

utilize ZFS Send compression "-c" option to speed up intra-cluster migration
@Fantu
Copy link

Fantu commented Feb 26, 2025

Hi, if you didn't see it on the main page:

How-To Contribute
Proxmox uses a mailing list approach for development and won't accept GitHub based pull-requests. Please check our Developer Documentation for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants