Skip to content

extract: do not abort on corrupted chunks, replace them by all-zero data with a warning, fixes #840 - #10357

Open
ThomasWaldmann wants to merge 1 commit into
borgbackup:masterfrom
ThomasWaldmann:extract-corrupt-chunks-840
Open

extract: do not abort on corrupted chunks, replace them by all-zero data with a warning, fixes #840#10357
ThomasWaldmann wants to merge 1 commit into
borgbackup:masterfrom
ThomasWaldmann:extract-corrupt-chunks-840

Conversation

@ThomasWaldmann

@ThomasWaldmann ThomasWaldmann commented Sep 11, 2026

Copy link
Copy Markdown
Member

Problem (#840)

borg extract aborted with an IntegrityError (rc 90) on the first chunk that does not authenticate / decrypt / decompress, leaving that file at 0 bytes and all following files unextracted. --continue failed the same way. The only way out was borg check --repair --verify-data, which modifies the repository and needs a full verification pass.

Missing chunks, on the other hand, were already replaced by all-zero data of the correct size, but only with an error log line: no warning exit code and no file name (see the old TODO in test_extract_file_with_missing_chunk).

Change

  • DownloadPipeline.fetch_many gets a replace_corrupted flag: with it, a corrupted chunk is handled like a missing one, i.e. an error naming the chunk is logged and an all-zero replacement of the correct size is yielded. A damaged list parameter lets the caller learn which chunks were replaced.
  • extract (incl. --dry-run and --stdout) sets the flag. A damaged file is extracted completely (correct size and metadata, all-zero content where the damaged chunks were), the extraction continues with the next file, and each damaged file is reported with a per-file warning (new BackupDamagedChunksError, rc 113). The missing-chunk case gets the same warning and exit code now instead of silently exiting with rc 0. --dry-run thus can be used to find unreadable files without writing anything.
  • mount -o allow_damaged_files covers corrupted chunks too. Without the option, the FUSE read handlers now map a corrupted chunk to EIO (with an error log line) instead of an unhandled exception.
  • recreate and transfer keep raising IntegrityError for corrupted chunks: they would otherwise store all-zero data as if it were the content (the *_wrong_chunk_content tests still assert that).
  • Docs: extract epilog, mount option help, frontends.rst rc list.

Note: after such an extraction, extract --continue considers the damaged file done (size and mtime match), same as before for missing chunks.

Tests

  • test_extract_file_with_damaged_chunk / test_extract_dry_run_with_damaged_chunk, parametrized over missing and corrupted (byte flipped in the pack via corrupt_chunk_on_disk), check the rc, the warning with the file name, the correct size, that only the damaged chunk's region is zero and that the other files are intact.
  • test_fuse_allow_damaged_files parametrized the same way.
  • test_verify_data_wrong_chunk_content adapted: with BORG_ASSERT_ID=read, extract now reports the file (rc 113) instead of aborting.

Verified manually as well: 4 MB random file, 4 bytes flipped in the data pack, extract --dry-run and extract both report the file with rc 113, the file has the right size, the small file after it is extracted.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.95%. Comparing base (11d7d10) to head (57a1605).
⚠️ Report is 3 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10357      +/-   ##
==========================================
+ Coverage   87.93%   87.95%   +0.02%     
==========================================
  Files         103      103              
  Lines       18893    18926      +33     
  Branches     2917     2921       +4     
==========================================
+ Hits        16613    16647      +34     
+ Misses       1583     1581       -2     
- Partials      697      698       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread src/borg/archive.py Outdated
…ata with a warning, fixes borgbackup#840

Missing chunks were already replaced by all-zero data of the correct size during
extraction (with an error log line, but no warning exit code and no file name),
while a corrupted chunk (one that does not authenticate, decrypt or decompress)
aborted the whole extraction with an IntegrityError, leaving the file at 0 bytes
and all following files unextracted.

Now DownloadPipeline.fetch_many treats a corrupted chunk like a missing one if
the new replace_corrupted flag is set: it logs an error naming the chunk and
yields an all-zero replacement of the correct size. extract (incl. --dry-run
and --stdout) and mount -o allow_damaged_files set that flag, so a damaged file
is still extracted completely (correct size and metadata, all-zero content
where the damaged chunks were), the extraction continues with the next file,
and each damaged file is reported with a warning (new BackupDamagedChunksError,
rc 113). This also fixes the missing-chunk case, which now gets the same
per-file warning and exit code instead of silently exiting with rc 0.

recreate and transfer keep raising IntegrityError for corrupted chunks, as they
would otherwise store all-zero data as if it were the content (see the
*_wrong_chunk_content tests). The FUSE read handlers map a corrupted chunk to
EIO (with an error log line) instead of an unhandled exception.

Supersedes the borg 1.x-era PR borgbackup#7481 (extract --skip-errors).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ThomasWaldmann
ThomasWaldmann force-pushed the extract-corrupt-chunks-840 branch from 666da6c to 57a1605 Compare September 11, 2026 16:10
@ThomasWaldmann ThomasWaldmann modified the milestone: 2.0.0b25 Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant