Skip to content

Add comprehensive local development and CI testing tools #279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Add comprehensive local development and CI testing tools #279

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a07e361
Add comprehensive local development and CI testing tools
attackordie Sep 17, 2025
1ca961a
Add comprehensive development testing documentation
attackordie Sep 17, 2025
719f8b2
Update Justfile
attackordie Sep 18, 2025
50d0098
Update Justfile
attackordie Sep 18, 2025
889725d
Move Development section from README to dedicated docs
attackordie Sep 19, 2025
22964a8
Make rust-setup idempotent by checking existing installations
attackordie Sep 20, 2025
655457e
Make act dependency optional by separating dev-check targets
attackordie Sep 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 147 additions & 34 deletions Justfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,37 +67,150 @@ docs-serve:
docs-watch:
cd docs && mdbook serve

# CI Docker commands - automatically handle user mapping to prevent permission issues
ci-local:
docker build \
--build-arg USER_ID=$(id -u) \
--build-arg GROUP_ID=$(id -g) \
-f Dockerfile.ci \
--target ci-test \
-t wassette-ci-local .
docker run --rm \
-v $(PWD):/workspace \
-w /workspace \
-e GITHUB_TOKEN \
wassette-ci-local just ci-build-test

ci-build-test:
just build-test-components
cargo build --workspace
cargo test --workspace -- --nocapture
cargo test --doc --workspace -- --nocapture

ci-build-test-ghcr:
just build-test-components
cargo build --workspace
cargo test --workspace -- --nocapture --include-ignored
cargo test --doc --workspace -- --nocapture

ci-cache-info:
docker system df
docker images wassette-ci-*

ci-clean:
docker rmi $(docker images -q wassette-ci-* 2>/dev/null) 2>/dev/null || true
docker builder prune -f

# Development Environment Commands

# Show available development commands
dev-help:
@echo "🚀 Wassette Development Commands"
@echo ""
@echo "📋 Setup:"
@echo " just dev-check - Check development prerequisites"
@echo " just rust-setup - Set up Rust development environment"
@echo " just act-install - Install act tool for local CI"
@echo ""
@echo "🧪 Local CI Testing (matches GitHub exactly):"
@echo " just act-lint - Run linting checks"
@echo " just act-build - Run build and tests"
@echo " just act-security - Run security audits"
@echo " just act-rust-all - Run all Rust workflow jobs"
@echo ""
@echo "⚡ Quick Development:"
@echo " just test - Run core tests (fast)"
@echo " just build - Build project"
@echo " just clean - Clean build artifacts"
@echo ""
@echo "🔧 Utilities:"
@echo " just act-clean - Clean up act containers"
@echo " just dev-help - Show this help"

# Check if development prerequisites are installed
dev-check:
@echo "🔍 Checking development prerequisites..."
@command -v act >/dev/null 2>&1 || (echo "❌ act not installed. Run: just act-install" && exit 1)
@command -v docker >/dev/null 2>&1 || (echo "❌ Docker not installed. Please install Docker: https://docs.docker.com/get-docker/" && exit 1)
@command -v cargo >/dev/null 2>&1 || (echo "❌ Rust/Cargo not installed. Run: just rust-setup" && exit 1)
@rustup target list --installed | grep -q wasm32-wasip2 || (echo "❌ wasm32-wasip2 target not installed. Run: rustup target add wasm32-wasip2" && exit 1)
@echo "✅ All prerequisites are installed!"

# Set up Rust development environment
rust-setup:
@echo "🦀 Setting up Rust development environment..."
rustup toolchain install nightly --component rustfmt
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like this to check and see if these things are installed first before updating. Otherwise it will update every day you run it (because it is installing nightly). I love having this target, but it should check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thomastaylor312 You're absolutely right on both points, and I completely agree that act should be
optional and not a hard dependency.

Making Act Truly Optional:
My intention was always to provide act as an additional testing method, not replace the existing workflow. I can modify the PR to:

  1. Remove act from dev-check - the core development prerequisites should remain the same
  2. Create a separate dev-check-ci command for those who want to use act
  3. Have rust-setup check existing installations first.

The PR would then be 100% backwards compatible:

  • Existing developers continue using just test + cargo clippy as always
  • New developers can optionally install act for exact CI validation
  • CI continues working identically

Regarding Native CI Targets:
I understand your preference for just targets that use native Rust/Wasm toolchains. The challenge is that GitHub CI uses specific container environments, dependency versions, and configurations that are difficult to perfectly replicate natively. But you're right that for daily development, native tools are faster and more efficient.

Would you prefer to make act completely optional or is there another solution?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey! I would prefer to keep a short "how to use act" page under docs/development explaining that this is a best-effort, and have links to nektos/acat docs. Avoid "perfect parity" language since in practice, we all know that it has meaninful gaps and we'd be setting false expectations.

Also, I am not a fan of introducing a bunch of just targets. I would also prefer to remove changes to the Justfile, and have one neutral entry point like just ci-local-act that runs act -W .github/workflows/rust.yml and no fork shortcuts.

So, I’d accept a slimmer PR that adds a docs-only page describing optional act usage and introduces at most one neutral just ci-local-act target (no job-level commands).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understand. I'll make it happen.

rustup target add wasm32-wasip2
cargo install cargo-machete cargo-audit cargo-deny typos-cli
@echo "✅ Rust development environment ready!"

# Act commands - run GitHub CI locally using act (github.com/nektos/act)
# Each command corresponds to a specific job in .github/workflows/rust.yml
# Uses --rm to automatically clean up containers after each run
#
# To test against your own fork:
# just act-lint-fork github.com/yourusername/wassette
# just act-build-fork github.com/yourusername/wassette

# Install act tool for running GitHub Actions locally
act-install:
@echo "Installing act (GitHub Actions runner)..."
@echo "Note: For security, the install script will be downloaded for your review before running with sudo."
tmpfile=$(mktemp /tmp/act-install.XXXXXX.sh) && \
curl -fsSL https://raw.githubusercontent.com/nektos/act/master/install.sh -o "$tmpfile" && \
echo "Downloaded install script to $tmpfile" && \
echo "SHA256 checksum:" && sha256sum "$tmpfile" && \
echo "Please review the script before running:" && \
echo " less $tmpfile" && \
echo "To install, run:" && \
echo " sudo bash $tmpfile"

act-license-headers:
act -W ./.github/workflows/rust.yml -j license-headers --rm

act-lint:
act -W ./.github/workflows/rust.yml -j lint --rm

act-build:
act -W ./.github/workflows/rust.yml -j build --rm

act-deps:
act -W ./.github/workflows/rust.yml -j deps --rm

act-security:
act -W ./.github/workflows/rust.yml -j security --rm

act-coverage:
act -W ./.github/workflows/rust.yml -j coverage --rm

act-spelling:
act -W ./.github/workflows/rust.yml -j spelling --rm

act-linkChecker:
act -W ./.github/workflows/rust.yml -j linkChecker --rm

# Run examples workflow
act-examples:
act -W ./.github/workflows/examples.yml --rm

# Run all rust workflow jobs
act-rust-all:
act -W ./.github/workflows/rust.yml --rm

# Run all workflows
act-all:
act -W ./.github/workflows/rust.yml --rm
act -W ./.github/workflows/examples.yml --rm

# Fork-specific commands for testing custom repositories
act-license-headers-fork repo:
act -W ./.github/workflows/rust.yml -j license-headers --rm --env GITHUB_REPOSITORY={{repo}}

act-lint-fork repo:
act -W ./.github/workflows/rust.yml -j lint --rm --env GITHUB_REPOSITORY={{repo}}

act-build-fork repo:
act -W ./.github/workflows/rust.yml -j build --rm --env GITHUB_REPOSITORY={{repo}}

act-deps-fork repo:
act -W ./.github/workflows/rust.yml -j deps --rm --env GITHUB_REPOSITORY={{repo}}

act-security-fork repo:
act -W ./.github/workflows/rust.yml -j security --rm --env GITHUB_REPOSITORY={{repo}}

act-coverage-fork repo:
act -W ./.github/workflows/rust.yml -j coverage --rm --env GITHUB_REPOSITORY={{repo}}

act-spelling-fork repo:
act -W ./.github/workflows/rust.yml -j spelling --rm --env GITHUB_REPOSITORY={{repo}}

act-linkChecker-fork repo:
act -W ./.github/workflows/rust.yml -j linkChecker --rm --env GITHUB_REPOSITORY={{repo}}

# Run examples workflow against fork
act-examples-fork repo:
act -W ./.github/workflows/examples.yml --rm --env GITHUB_REPOSITORY={{repo}}

# Run all rust workflow jobs against fork
act-rust-all-fork repo:
act -W ./.github/workflows/rust.yml --rm --env GITHUB_REPOSITORY={{repo}}

# Run all workflows against fork
act-all-fork repo:
act -W ./.github/workflows/rust.yml --rm --env GITHUB_REPOSITORY={{repo}}
act -W ./.github/workflows/examples.yml --rm --env GITHUB_REPOSITORY={{repo}}

# Clean up any stuck act containers
act-clean:
@echo "Current act containers:"
-docker ps --filter "name=act-"
@echo "Stopping and removing act containers..."
-docker stop $(docker ps -q --filter "name=act-") 2>/dev/null || true
-docker rm $(docker ps -aq --filter "name=act-") 2>/dev/null || true
@echo "Act containers cleaned up."
4 changes: 4 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,10 @@ You can join us via the `#wassette` channel on the [Microsoft Open Source Discor

Please see [CONTRIBUTING.md][Contributing] for more information on how to contribute to this project.

## Development

For local development and testing instructions, see [docs/development/developer-tests.md](docs/development/developer-tests.md).

## License

This project is licensed under the <a href="LICENSE">MIT License</a>.
Expand Down
Loading
Loading