Skip to content

Conversation

@RyuX51
Copy link

@RyuX51 RyuX51 commented Oct 27, 2025

Add --no-inc-recursive-skip-unchanged for accurate progress on resumed transfers

Problem

When using --info=progress2 with --no-i-r, interrupted transfers show incorrect progress percentages after resuming. For example, if a transfer is interrupted at 20%, restarting it will show progress from 1% to 80% instead of 0% to 100%.

Root cause: stats.total_size includes already-transferred unchanged files. rsync's normal operation will skip these files (they won't actually transfer), but they're already counted in the progress denominator, making the percentage calculation wrong.

Solution

This PR introduces --no-inc-recursive-skip-unchanged (with aliases --no-i-r-skip-unchanged and --no-i-r-s-u), which pre-scans destination files during generator initialization to identify unchanged files and adjusts stats.total_size before progress reporting begins.

Implementation

Generator-side pre-scanning:

  • Runs before the main processing loop in generate_files()
  • Uses existing quick_check_ok() logic to identify unchanged files
  • Marks unchanged files inactive with clear_file() to exclude them from processing
  • Adjusts stats.total_size by subtracting unchanged file sizes

Inter-process communication:

  • Introduces MSG_FLIST_COUNT message to sync statistics between generator and sender
  • Ensures accurate progress display regardless of which process shows progress
  • Sender receives both skipped count and adjusted stats.total_size

Universal compatibility:

  • Works for all transfer types: local-to-local, local-to-remote, remote-to-local, and daemon
  • Generator always has destination access, making this approach viable in all scenarios
  • Implies --no-i-r (requires full file list upfront for pre-scan)

Benefits

Accurate progress reporting: Shows genuine 0-100% progress on resumed transfers
Correct file counts: to-chk display reflects only files needing transfer
No significant overhead: Uses same stat operations rsync would perform anyway, just earlier in the pipeline

Testing

# Initial transfer (interrupted mid-way)
rsync -av --info=progress2 --no-i-r-s-u src/ host:dest/
# Transfer shows 0-50%, then interrupts

# Resume transfer
rsync -av --info=progress2 --no-i-r-s-u src/ host:dest/
# Progress correctly shows 0-100% (only remaining files counted)

Naming Convention

Following rsync's established patterns (like --no-inc-recursive / --no-i-r):

  • Full name: --no-inc-recursive-skip-unchanged (documented in man page only)
  • Medium alias: --no-i-r-skip-unchanged (hidden from --help)
  • Short alias: --no-i-r-s-u (hidden from --help)

All variants function identically; only the full documentation appears in the man page, maintaining consistency with rsync's help output philosophy.

Files Changed

  • generator.c: Pre-scan logic in prescan_for_unchanged()
  • io.c: IPC handler for MSG_FLIST_COUNT
  • options.c: Option definitions and validation
  • progress.c: Adjusted progress calculation
  • rsync.h: New message type and stats field
  • rsync.1.md: Comprehensive documentation
  • NEWS.md: Enhancement entry

Example Use Cases

Resumed large backups with accurate progress:

rsync -av --no-i-r-s-u --info=progress2 /data/ /backup/

Remote synchronization with meaningful progress tracking:

rsync -av --no-i-r-skip-unchanged --info=progress2 host:src/ dest/

Monitoring actual transfer progress (not inflated by unchanged files):

rsync -av --no-inc-recursive-skip-unchanged /src/ /build/

This enhancement addresses a long-standing UX issue that may confuse users, particularly those relying on progress reporting for automated backup systems and large data transfers where most files are typically unchanged.

@RyuX51 RyuX51 changed the title Add --no-i-r-skip-unchanged for accurate progress on resumed transfers Add --no-i-r-skip-unchanged for accurate progress on resumed transfers Oct 27, 2025
@RyuX51 RyuX51 changed the title Add --no-i-r-skip-unchanged for accurate progress on resumed transfers Add --no-i-r-s-u/--no-inc-recursive-skip-unchanged for accurate progress on resumed transfers Oct 27, 2025
@RyuX51 RyuX51 force-pushed the pr-no-i-r-skip-unchanged branch 2 times, most recently from 53febd6 to ad39ab0 Compare October 27, 2025 21:47
@RyuX51 RyuX51 force-pushed the pr-no-i-r-skip-unchanged branch 3 times, most recently from 7db3934 to 7cf1285 Compare October 29, 2025 21:44
Fixes progress reporting for local→remote transfers by sending adjusted
stats from receiver to sender via MSG_FLIST_COUNT IPC message.
@RyuX51 RyuX51 force-pushed the pr-no-i-r-skip-unchanged branch from 7cf1285 to 03c8fd5 Compare October 29, 2025 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant