Skip to content

LZW feature branch#4

Closed
itsfuad wants to merge 1 commit intomainfrom
jules_wip_16152481801220120628
Closed

LZW feature branch#4
itsfuad wants to merge 1 commit intomainfrom
jules_wip_16152481801220120628

Conversation

@itsfuad
Copy link
Owner

@itsfuad itsfuad commented Jun 16, 2025

This pull request introduces the LZW compression algorithm to the project, along with enhancements to progress tracking and utility functionality. The most significant changes include the implementation of LZW compression and decompression, updates to the CLI and helper files to support the new algorithm, and improvements to the progress reporting system.

LZW Compression and Decompression Implementation:

  • Added compressor/lzw/io.go with Zip and Unzip functions to handle file compression and decompression using the LZW algorithm. These functions include detailed progress reporting and safety checks for file paths during decompression.
  • Added compressor/lzw/lzw.go with core compressData and decompressData functions implementing the LZW algorithm, including dictionary management and bit-level operations.

CLI and Helper Updates:

  • Updated utils/cli.go to support the LZW algorithm in the CLI by adding it as a valid option and improving error messages for unsupported algorithms.
  • Added LZW to the Algorithm type in utils/helper.go to define it as a supported compression algorithm.

Progress Tracking Enhancements:

  • Enhanced ProgressInfo in utils/progress.go to include more granular details, such as file-specific and batch-level progress metrics.
  • Improved UpdateProgress in utils/progress.go to calculate progress based on bytes processed and construct detailed progress messages, including file names and sizes.

Utility Improvements:

  • Added utils/buffer.go with the ResizableBuffer type to provide a dynamic buffer for handling compressed data efficiently.

… done so far and provide feedback for Jules to continue.
@itsfuad itsfuad requested a review from Copilot June 16, 2025 13:48
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds LZW compression/decompression, integrates it into the CLI, enhances progress reporting with more detailed metrics, and introduces a reusable buffer utility.

  • Implements LZW algorithm in compressor/lzw with I/O wrappers supporting progress callbacks
  • Extends CLI and helper types to recognize the new lzw algorithm
  • Improves progress tracking (ProgressInfo, UpdateProgress, and ProgressTrackingReader)
  • Adds ResizableBuffer for dynamic in-memory buffering

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
utils/reader_progress.go New ProgressTrackingReader to wrap an io.Reader and report byte-level progress
utils/progress.go Expanded ProgressInfo and revamped UpdateProgress for batch and file-level metrics
utils/helper.go Added LZW constant to the Algorithm enum
utils/cli.go CLI parsing updated to accept lzw and improved error message
utils/buffer.go New ResizableBuffer type with read/write/reset functionality
compressor/lzw/lzw.go Core LZW encode/decode routines with bit-level writers/readers
compressor/lzw/io.go File-level Zip/Unzip I/O functions, metadata framing, path safety checks, progress hooks
Comments suppressed due to low confidence (4)

compressor/lzw/lzw.go:56

  • The core compressData function is complex and untested; consider adding unit tests covering small and large inputs, empty streams, and dictionary boundary conditions.
func compressData(input io.Reader, output io.Writer) error {

utils/buffer.go:19

  • Public methods on ResizableBuffer lack Go doc comments; adding brief descriptions for Write, Len, Bytes, Reader, and Reset will improve API clarity.
func (rb *ResizableBuffer) Write(p []byte) (n int, err error) {

compressor/lzw/lzw.go:30

  • [nitpick] The write method on bitWriter could be renamed to WriteBits or WriteCode to avoid confusion with io.Writer.Write.
func (bw *bitWriter) write(code uint32, numBits uint) error {

utils/reader_progress.go:17

  • [nitpick] Function NewProgressReader wraps a ProgressTrackingReader; renaming it to NewProgressTrackingReader would better reflect the underlying type.
func NewProgressReader(r io.Reader, totalSize int64, onProgress func(readBytes int64, totalBytes int64)) io.Reader {

Comment on lines +148 to +149
// TODO: Report progress for skipped file if possible, e.g.
// if progressCallback != nil { utils.UpdateProgress(utils.ProgressInfo{ ... CurrentFileName: fileName, OriginalMessage: "Skipped unsafe file" ... }, progressCallback) }
Copy link

Copilot AI Jun 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There’s a TODO left in Unzip for reporting skipped-file progress; consider implementing or removing this stub to avoid incomplete code paths.

Suggested change
// TODO: Report progress for skipped file if possible, e.g.
// if progressCallback != nil { utils.UpdateProgress(utils.ProgressInfo{ ... CurrentFileName: fileName, OriginalMessage: "Skipped unsafe file" ... }, progressCallback) }
// Report progress for skipped file if progressCallback is provided.
if progressCallback != nil {
utils.UpdateProgress(utils.ProgressInfo{
CurrentFileName: fileName,
OriginalMessage: "Skipped unsafe file",
PercentComplete: 0, // Skipped files do not contribute to progress percentage.
}, progressCallback)
}

Copilot uses AI. Check for mistakes.
fileName := string(fileNameBytes)
cleanFileName := filepath.Clean(fileName)

if strings.HasPrefix(cleanFileName, ".."+string(filepath.Separator)) || cleanFileName == ".." || strings.HasPrefix(cleanFileName, string(filepath.Separator)) || strings.Contains(fileName, "\\") || strings.Contains(fileName, ":") {
Copy link

Copilot AI Jun 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The path sanitization rejects any backslash or colon, which might block valid Unix filenames; consider using filepath.Rel or filepath.IsAbs for a more robust check against directory traversal.

Suggested change
if strings.HasPrefix(cleanFileName, ".."+string(filepath.Separator)) || cleanFileName == ".." || strings.HasPrefix(cleanFileName, string(filepath.Separator)) || strings.Contains(fileName, "\\") || strings.Contains(fileName, ":") {
relPath, err := filepath.Rel(outputDir, filepath.Join(outputDir, cleanFileName))
if err != nil || filepath.IsAbs(relPath) || strings.HasPrefix(relPath, ".."+string(filepath.Separator)) {

Copilot uses AI. Check for mistakes.
@itsfuad itsfuad closed this Jun 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants