Skip to content

Enable Git LFS to Manage Binary Files #830

Open
@rvveber

Description

@rvveber

Proposal: Enable Git LFS to Manage Binary Files

Some files in this repository are stored in binary format. Since Git cannot store differences for binary files, any change results in the full file being stored in history. This can lead to:

  • Repository bloat and degraded performance over time
  • Slower clone operations
  • Increased CI times due to larger downloads

To address these issues, Git LFS (Large File Storage) was created. Git LFS replaces large binary files with lightweight pointers in the repository while storing the actual contents externally.

See the official guide for setup instructions:
https://docs.github.com/en/repositories/working-with-files/managing-large-files/configuring-git-large-file-storage

Below is a handy script that scans the repository for binary files and reports their sizes in a human-readable format (requires ripgrep):

#!/usr/bin/env bash

# This script finds and lists all binary files recursively in the current directory.
total_bytes=0
while read -r file; do
    # Exclude files ignored by git
    if git check-ignore -q "$file"; then
        continue
    fi
    # Determine the file size
    size=$(stat -c%s "$file")
    # Skip empty files
    if [ "$size" -eq 0 ]; then
        continue
    fi
    # Use file -i to check if the file is binary (has charset=binary)
    if file -i "$file" | grep -q "charset=binary"; then
        # Report file and its size in human-readable format
        echo "$file - $(numfmt --to=iec $size)"
        total_bytes=$((total_bytes + size))
    fi
done < <(rg --files -g '!.git')
# Report the total size in human-readable format
echo "Total binary size: $(numfmt --to=iec $total_bytes)"

Enabling Git LFS will help keep the repository lean and maintain better performance over time.

Metadata

Metadata

Assignees

Labels

enhancementimprove an existing feature

Type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions