Skip to content

Add safe high-performance decoding and CLI - #155

Open
cjrolo wants to merge 17 commits into
mainfrom
modernize/decompression-first-core
Open

Add safe high-performance decoding and CLI#155
cjrolo wants to merge 17 commits into
mainfrom
modernize/decompression-first-core

Conversation

@cjrolo

@cjrolo cjrolo commented Jul 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • preserve BRO v1 bytes while adding typed, bounded parsing and transactional full/frame/range decoder APIs
  • reuse FFT plans and codec/output scratch; all eight comparable decompression benchmarks improve by 7.9%–96.8% with 0% growth in checked compressed outputs
  • modernize the CLI, WBRO/CSV input safety, Rust 1.81 build policy, and pinned CI/release workflows

Test plan

  • cargo fmt --all -- --check on Rust 1.81
  • cargo clippy --workspace --all-targets --locked -- -D warnings on Rust 1.81
  • cargo test --workspace --all-targets --locked on Rust 1.81 (225 tests)
  • cargo bench -p atsc --bench decompression_bench --no-run --locked
  • six BRO v1 byte-compatibility fixtures
  • three compressed-size regression gates (0% growth)
  • fresh same-toolchain Criterion comparison against the pre-change baseline
  • final whole-branch review with no remaining findings

Made with Cursor

cjrolo and others added 16 commits July 23, 2026 16:38
Define byte-compatibility, performance, API, and CLI gates before changing the codec.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Enforce bounded transactional encode, decode, and input handling while preserving BRO v1 wire compatibility.
Preflight v1 and frame point counts, then widen integer payloads directly into checked storage to prevent unaccounted allocations.
Comment on lines +20 to +38
name: Ubuntu 24.04
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v3
- uses: Swatinem/rust-cache@v2
with:
# rust-cache already handles all the sane defaults for caching rust builds.
# However, because we are running separate debug/release builds in parallel,
# we also need to add the runner and cargo_flags to the key so that a separate cache is used.
# Otherwise, only the last build to finish would get saved to the cache.
key: ${{ matrix.runner }} - ${{ matrix.cargo_flags }}
- name: Install external deps for the prom-remote-api crate
run: sudo apt-get install protobuf-compiler
- name: Install cargo-hack
run: cargo install cargo-hack --version 0.5.8
- name: Check `cargo fmt` was run
run: cargo fmt --all -- --check
- name: Ensure that all crates compile and have no warnings under every possible combination of features
# some things to explicitly point out:
# * clippy also reports rustc warnings and errors
# * clippy --all-targets causes clippy to run against tests and examples which it doesn't do by default.
run: |
# Display all clippy lint failures as warnings
cargo hack --feature-powerset clippy --all-targets --locked ${{ matrix.cargo_flags }}
# Fail CI on the first crate to fail clippy lints
cargo hack --feature-powerset clippy --all-targets --locked ${{ matrix.cargo_flags }} -- -D warnings
- name: Ensure that tests pass
run: |
cargo test --doc ${{ matrix.cargo_flags }} --all-features -- --show-output --nocapture
cargo test ${{ matrix.cargo_flags }} --all-features --all-targets -- --nocapture
- name: Ensure that tests did not create or modify any files that arent .gitignore'd
run: |
if [ -n "$(git status --porcelain)" ]; then
git status
exit 1
fi
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2
- name: Check `cargo fmt` was run
run: cargo fmt --all -- --check
- name: Ensure that all targets compile without warnings
run: cargo clippy --workspace --all-targets --locked -- -D warnings
- name: Ensure that all tests pass
run: cargo test --workspace --all-targets --locked
- name: Ensure that the decompression benchmark compiles
run: cargo bench -p atsc --bench decompression_bench --no-run
- name: Ensure that tests did not create or modify any files that arent .gitignore'd
run: |
if [ -n "$(git status --porcelain)" ]; then
git status
exit 1
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants