Skip to content

Conversation

@NotAShelf
Copy link
Member

@NotAShelf NotAShelf commented Oct 23, 2025

Adds optional integration with the hotpath crate to measure and profile the performance of key functions throughout nh. The integration is controlled through newly added Cargo features, and should not have any effects on the non-profile builds. Most of the changes are focused on annotating important functions with hotpath::measure to enable performance tracking when the feature is enabled, but I plan to add CI to run those builds automatically.

Signed-off-by: NotAShelf [email protected]
Change-Id: I7899d1b4c8dd7dddac01c0819afa1c096a6a6964

Summary by CodeRabbit

  • Chores
    • Added optional performance profiling instrumentation (behind a feature flag, disabled by default) across many public commands and helpers to enable fine-grained timing and allocation measurement without changing behavior when off.
  • CI
    • Added a profiling workflow that collects timing and allocation metrics for PRs, compares branches, and posts a summarized report to the PR.

Signed-off-by: NotAShelf <[email protected]>
Change-Id: I7899d1b4c8dd7dddac01c0819afa1c096a6a6964
Signed-off-by: NotAShelf <[email protected]>
Change-Id: Icc47ecd660f95d1f10423a2a1bdb6cc36a6a6964
@coderabbitai
Copy link

coderabbitai bot commented Oct 23, 2025

Walkthrough

Adds optional hotpath instrumentation: new optional dependency and feature flags in Cargo.toml, feature-gated hotpath initialization in main, cfg-gated hotpath::measure attributes (and two hotpath::measure_block! wrappers) across multiple public functions/methods, and a GitHub Actions profiling workflow.

Changes

Cohort / File(s) Summary
Build configuration
Cargo.toml
Added optional dependency hotpath = { version = "0.5.2", optional = true } and new features: hotpath, hotpath-alloc-bytes-total, hotpath-alloc-count-total, hotpath-alloc-self, hotpath-off.
Application initialization
src/main.rs
Added #[cfg(feature = "hotpath")] import of hotpath and a feature-gated guard initialization hotpath::GuardBuilder::new("main").build().
Run/command instrumentation
src/checks.rs, src/clean.rs, src/commands.rs, src/completion.rs, src/darwin.rs, src/home.rs, src/nixos.rs
Added #[cfg_attr(feature = "hotpath", hotpath::measure)] to many public run and related methods; no signature or behavior changes when feature is disabled.
Utilities & platform functions
src/util.rs, src/util/platform.rs, src/update.rs
Added #[cfg_attr(feature = "hotpath", hotpath::measure)] to multiple public utility/platform functions (e.g., get_nix_version, get_nix_experimental_features, compare_configurations, build_configuration, handle_rebuild_workflow, update).
Search instrumentation (fine-grained)
src/search.rs
Added #[cfg_attr(feature = "hotpath", hotpath::measure)] to SearchArgs::run and wrapped the HTTP request and JSON parsing with hotpath::measure_block!() under #[cfg(feature = "hotpath")], with #[cfg(not(feature = "hotpath"))] fallbacks.
CI profiling workflow
.github/workflows/profile.yaml
Added "Rust Profiling" workflow that runs per-PR profiling on head and base (timing and allocation captures), uploads metrics as artifacts, compares head vs base, and posts a PR comment with results.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant User
    participant Binary as main.rs
    participant Hot as hotpath (feature)
    participant App as Application functions

    Note over Binary,Hot #f0f7ff: Compile-time feature gating
    User->>Binary: execute
    alt hotpath feature enabled
        Binary->>Hot: GuardBuilder::new("main").build()
        Hot-->>Binary: Guard active
        Binary->>App: call public methods (annotated with hotpath::measure)
        App->>Hot: enter/exit measurements
        Note right of App #e8f7ea: search: measure_block!("http_request") and measure_block!("json_parse")
    else hotpath disabled
        Binary->>App: call public methods (normal)
    end
    App-->>User: return results
Loading
sequenceDiagram
    autonumber
    participant GH as GitHub Actions
    participant Repo
    participant Runner
    participant Artifacts
    participant CommentJob

    Repo->>GH: PR opened/updated
    GH->>Runner: run "Rust Profiling" workflow
    Runner->>Runner: setup toolchains, run cargo with hotpath-enabled captures (head/base)
    Runner->>Artifacts: upload timing & allocation JSON
    GH->>CommentJob: start comment job
    CommentJob->>Artifacts: download metrics
    CommentJob->>CommentJob: parse & compare metrics
    CommentJob->>Repo: post PR comment with comparison
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title "treewide: add hotpath for profiling allocations and timings" directly and accurately describes the main change in this pull request. The changeset adds a new optional hotpath dependency, introduces related Cargo features, annotates multiple public functions with hotpath measurement attributes across the codebase, initializes hotpath in main, and adds a GitHub Actions workflow for profiling. The title is clear, specific, and uses the conventional "treewide" prefix to indicate changes spanning multiple files, without vague terminology or unnecessary noise.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch notashelf/push-nnvslnmttzkq

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/util/platform.rs (1)

266-290: Critical: Incorrect variable reference.

Line 282 references self.passthrough, but build_configuration is a standalone function (not a method), and self is not available. The correct variable is passthrough, which was created on line 276.

Apply this diff:

     .extra_arg("--out-link")
     .extra_arg(out_path.get_path())
     .extra_args(extra_args)
-    .passthrough(&self.passthrough)
+    .passthrough(&passthrough)
     .builder(builder)
     .message(message)
     .nom(!no_nom)
🧹 Nitpick comments (3)
src/main.rs (1)

18-18: Optional: Consider removing the redundant import.

Line 27 uses the full path hotpath::GuardBuilder::new("main"), so the import on line 18 appears unnecessary. You can safely remove it without affecting functionality.

Apply this diff if you'd like to remove the redundant import:

-#[cfg(feature = "hotpath")] use hotpath;
-
 use crate::commands::ElevationStrategy;
src/search.rs (2)

150-159: Code duplication in conditional blocks.

The HTTP request logic is duplicated between the cfg(feature = "hotpath") and cfg(not(feature = "hotpath")) blocks. While this is unavoidable with the current approach, it creates a maintenance burden—if the request logic changes, both blocks must be updated identically.

Consider documenting this pattern or verifying both blocks stay in sync during future changes.


185-195: Similar duplication for JSON parsing.

Same concern as the HTTP request block—the JSON parsing logic is duplicated. This is a known tradeoff of the conditional compilation approach.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e27508e and fd47c68.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (13)
  • Cargo.toml (2 hunks)
  • src/checks.rs (2 hunks)
  • src/clean.rs (1 hunks)
  • src/commands.rs (2 hunks)
  • src/completion.rs (1 hunks)
  • src/darwin.rs (1 hunks)
  • src/home.rs (1 hunks)
  • src/main.rs (1 hunks)
  • src/nixos.rs (1 hunks)
  • src/search.rs (3 hunks)
  • src/update.rs (1 hunks)
  • src/util.rs (2 hunks)
  • src/util/platform.rs (3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Test NH on Linux
  • GitHub Check: treewide-checks
  • GitHub Check: Test NH on Darwin
  • GitHub Check: Build NH on Linux
  • GitHub Check: Build NH on Darwin
🔇 Additional comments (17)
Cargo.toml (1)

38-38: LGTM! Clean integration of hotpath as optional dependency.

The optional dependency declaration and feature flags are properly structured. The feature configuration correctly gates the instrumentation behind feature flags, ensuring zero impact when disabled.

Also applies to: 62-67

src/main.rs (1)

26-27: LGTM! Proper hotpath initialization.

The guard is correctly initialized and scoped to measure the entire main function execution. The underscore prefix on _guard appropriately indicates the variable is intentionally unused beyond its RAII lifetime.

src/completion.rs (1)

14-19: LGTM! Consistent instrumentation pattern.

The conditional hotpath measurement is properly applied with no impact on functionality when the feature is disabled.

src/darwin.rs (1)

32-45: LGTM! Instrumentation correctly applied.

The hotpath measurement attribute follows the established pattern and is properly gated behind the feature flag.

src/nixos.rs (1)

34-52: LGTM! Consistent instrumentation.

The measurement attribute is properly applied to the entry point method.

src/home.rs (1)

24-37: LGTM! Properly instrumented.

The hotpath measurement follows the consistent pattern applied across the codebase.

src/util.rs (2)

144-157: LGTM! Sensible instrumentation point.

Measuring get_nix_version is useful since it executes an external command, making it a relevant performance tracking point.


233-249: LGTM! Good choice for measurement.

Instrumenting get_nix_experimental_features is appropriate as it involves external command execution and is likely called during critical paths.

src/clean.rs (1)

80-387: LGTM! Comprehensive instrumentation of the clean operation.

The measurement attribute is appropriately placed on the main entry point for clean operations.

src/checks.rs (2)

18-19: LGTM! Clean instrumentation.

The hotpath measurement attributes are correctly applied to these public functions, enabling optional performance profiling without affecting behavior when the feature is disabled.


133-134: LGTM! Consistent instrumentation pattern.

The hotpath attribute follows the same pattern used throughout the codebase.

src/update.rs (1)

5-9: LGTM! Straightforward instrumentation.

The hotpath attribute is correctly applied to the public update function.

src/search.rs (1)

64-65: LGTM! Function-level instrumentation.

The hotpath attribute correctly instruments the entire run method.

src/commands.rs (2)

459-460: LGTM! Instrumentation on Command::run().

Appropriate placement for profiling command execution.


672-673: LGTM! Instrumentation on Build::run().

Consistent with the Command::run() instrumentation.

src/util/platform.rs (2)

236-237: LGTM! Instrumentation on compare_configurations.

Appropriate for profiling configuration comparison operations.


406-424: LGTM! Instrumentation on handle_rebuild_workflow.

The hotpath attribute is correctly placed on this high-level workflow function, though the function signature has many parameters (allowed by the clippy exception on line 407).

Copy link

@faukah faukah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diff LGTM at a glance, I'll test it later.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
.github/workflows/profile.yaml (1)

61-68: Minor: Variable expansion for JSON output files is fragile.

While the heredoc approach should work with GitHub Actions' variable encoding, directly expanding raw JSON into files can be error-prone if the output contains special characters. Consider using a safer approach:

- name: Save metrics to artifact
  run: |
    mkdir -p /tmp/metrics
    cat > /tmp/metrics/head_timing.json << 'EOF'
    ${{ steps.head_timing_metrics.outputs.metrics }}
    EOF
    cat > /tmp/metrics/base_timing.json << 'EOF'
    ${{ steps.base_timing_metrics.outputs.metrics }}
    EOF
    cat > /tmp/metrics/head_alloc.json << 'EOF'
    ${{ steps.head_alloc_metrics.outputs.metrics }}
    EOF
    cat > /tmp/metrics/base_alloc.json << 'EOF'
    ${{ steps.base_alloc_metrics.outputs.metrics }}
    EOF
    echo '${{ github.event.pull_request.number }}' > /tmp/metrics/pr_number.txt

This guards against potential issues with unescaped content.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fd47c68 and 6a51b95.

📒 Files selected for processing (1)
  • .github/workflows/profile.yaml (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Build NH on Linux
  • GitHub Check: Build NH on Darwin
  • GitHub Check: Test NH on Linux
  • GitHub Check: Test NH on Darwin
  • GitHub Check: treewide-checks
🔇 Additional comments (1)
.github/workflows/profile.yaml (1)

144-150: Ensure outputs are properly exported.

The comment output is correctly set at lines 144-146, but as noted above, pr_number must also be exported to $GITHUB_OUTPUT for line 150 to function correctly.

Verify the complete output export looks like:

echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT
echo "comment<<EOF" >> $GITHUB_OUTPUT
echo "$COMMENT" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT

@NotAShelf NotAShelf force-pushed the notashelf/push-nnvslnmttzkq branch from 6a51b95 to 915d900 Compare October 23, 2025 12:46
Signed-off-by: NotAShelf <[email protected]>
Change-Id: Iefb4347101d472b51698d78754da583d6a6a6964
@NotAShelf NotAShelf force-pushed the notashelf/push-nnvslnmttzkq branch from 915d900 to 88f6a62 Compare October 23, 2025 12:49
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
.github/workflows/profile.yaml (1)

26-65: Consider running multiple representative workloads instead of a single hardcoded search.

The workflow runs only search "nix" to gather metrics. A single query may not be representative of real-world usage or reveal performance regressions in less-exercised code paths. Consider parameterizing the workflow to run multiple commands or accepting a list of representative scenarios.

Additionally, consider adding a noise filter to suppress trivial changes (e.g., < 2% variance) to reduce alert fatigue, and documenting why search "nix" was chosen as the profiling workload.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6a51b95 and 88f6a62.

📒 Files selected for processing (1)
  • .github/workflows/profile.yaml (1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.8)
.github/workflows/profile.yaml

13-13: could not parse as YAML: did not find expected '-' indicator

(syntax-check)

🪛 YAMLlint (1.37.1)
.github/workflows/profile.yaml

[error] 18-18: syntax error: expected , but found ''

(syntax)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Build NH on Linux
  • GitHub Check: Build NH on Darwin
  • GitHub Check: Test NH on Linux
  • GitHub Check: Test NH on Darwin
🔇 Additional comments (1)
.github/workflows/profile.yaml (1)

66-79: Quote variable expansions when writing metrics to files.

Lines 69-73 use unquoted shell variable expansion. While unlikely to cause issues here, it's a potential vulnerability if the JSON output contains special shell characters. Quote the expansions for safety.

Apply this diff:

      - name: Save metrics to artifact
        run: |
          mkdir -p /tmp/metrics
-         echo '${{ steps.head_timing_metrics.outputs.metrics }}' > /tmp/metrics/head_timing.json
-         echo '${{ steps.base_timing_metrics.outputs.metrics }}' > /tmp/metrics/base_timing.json
-         echo '${{ steps.head_alloc_metrics.outputs.metrics }}' > /tmp/metrics/head_alloc.json
-         echo '${{ steps.base_alloc_metrics.outputs.metrics }}' > /tmp/metrics/base_alloc.json
-         echo '${{ github.event.pull_request.number }}' > /tmp/metrics/pr_number.txt
+         echo "${{ steps.head_timing_metrics.outputs.metrics }}" > /tmp/metrics/head_timing.json
+         echo "${{ steps.base_timing_metrics.outputs.metrics }}" > /tmp/metrics/base_timing.json
+         echo "${{ steps.head_alloc_metrics.outputs.metrics }}" > /tmp/metrics/head_alloc.json
+         echo "${{ steps.base_alloc_metrics.outputs.metrics }}" > /tmp/metrics/base_alloc.json
+         echo "${{ github.event.pull_request.number }}" > /tmp/metrics/pr_number.txt

Likely an incorrect or invalid review comment.

Comment on lines +14 to +25
- uses: actions/checkout@v5
with:
fetch-depth: 0

- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2

- name: Install Nix
uses: cachix/install-nix-action@v31
with:
nix_path: nixpkgs=channel:nixos-unstable

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Fix YAML indentation errors in the steps block.

Lines 18 and 21 have inconsistent indentation (7 spaces instead of 6), causing YAML parsing failures. All step items under steps: must align at the same indentation level.

Apply this diff to fix the indentation:

      - uses: actions/checkout@v5
        with:
          fetch-depth: 0

-       - uses: dtolnay/rust-toolchain@stable
-       - uses: Swatinem/rust-cache@v2
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: Swatinem/rust-cache@v2

-       - name: Install Nix
+      - name: Install Nix
         uses: cachix/install-nix-action@v31
         with:
           nix_path: nixpkgs=channel:nixos-unstable
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Install Nix
uses: cachix/install-nix-action@v31
with:
nix_path: nixpkgs=channel:nixos-unstable
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Install Nix
uses: cachix/install-nix-action@v31
with:
nix_path: nixpkgs=channel:nixos-unstable
🧰 Tools
🪛 YAMLlint (1.37.1)

[error] 18-18: syntax error: expected , but found ''

(syntax)

🤖 Prompt for AI Agents
.github/workflows/profile.yaml around lines 14 to 25: the steps block has
inconsistent indentation (lines 18 and 21 are indented one extra space), which
breaks YAML parsing; align all items under the steps: block to the same
indentation level by reducing the extra space on those lines so that each step
entry (the lines starting with "- uses:" and "- name:") shares identical
indentation, and ensure nested keys like with: and nix_path remain indented
relative to their parent step.

Comment on lines +26 to +42
- id: head_timing_metrics
run: |
# Extract timing metrics from head branch
{
echo 'metrics<<EOF'
cargo run --features='hotpath,hotpath-ci' -- search "nix" | grep '^{"hotpath_profiling_mode"'
echo 'EOF'
} >> "$GITHUB_OUTPUT"

- id: head_alloc_metrics
run: |
# Extract alloc metrics from head branch
{
echo 'metrics<<EOF'
cargo run --features='hotpath,hotpath-ci,hotpath-alloc-count-total' -- search "nix" | grep '^{"hotpath_profiling_mode"'
echo 'EOF'
} >> "$GITHUB_OUTPUT"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Add error handling and validation for cargo command output.

The workflow assumes cargo run --features='hotpath,hotpath-ci' -- search "nix" produces JSON with {"hotpath_profiling_mode" prefix. If this assumption is violated—due to missing features, build failure, or changed output format—grep returns empty results silently, causing later jq parsing to fail with cryptic errors.

Add explicit error handling:

  - id: head_timing_metrics
    run: |
      # Extract timing metrics from head branch
+     set -e
      {
        echo 'metrics<<EOF'
-       cargo run --features='hotpath,hotpath-ci' -- search "nix" | grep '^{"hotpath_profiling_mode"'
+       OUTPUT=$(cargo run --features='hotpath,hotpath-ci' -- search "nix" 2>&1) || { echo "Cargo build/run failed"; exit 1; }
+       MATCHED=$(echo "$OUTPUT" | grep '^{"hotpath_profiling_mode"' || true)
+       if [ -z "$MATCHED" ]; then
+         echo "Warning: No hotpath JSON output found. Output was: $OUTPUT" >&2
+       fi
+       echo "$MATCHED"
        echo 'EOF'
      } >> "$GITHUB_OUTPUT"

Additionally, verify that the nh binary actually supports the search command and hotpath output format by checking documentation or running a quick test locally first.


🏁 Script executed:

#!/bin/bash
# Find and examine Cargo.toml for feature definitions
fd -e Cargo.toml | head -5

Length of output: 0


🏁 Script executed:

#!/bin/bash
# Search for search command implementation and hotpath features
rg -n "hotpath" --type=toml | head -20

Length of output: 581


🏁 Script executed:

#!/bin/bash
# Look for search command definition
rg -n "fn search|command.*search" --type=rs -C3 | head -40

Length of output: 26


🏁 Script executed:

#!/bin/bash
# Check for any tests or examples of search output
rg -n "hotpath_profiling_mode" --type=rs -C2 | head -30

Length of output: 26


🏁 Script executed:

#!/bin/bash
# Examine the actual workflow file to see full context
cat .github/workflows/profile.yaml

Length of output: 5778


🏁 Script executed:

#!/bin/bash
# Search for any reference to hotpath-ci in the codebase
rg -n "hotpath-ci" -C2

Length of output: 0


🏁 Script executed:

#!/bin/bash
# Search for search command implementation - try without type restriction
rg -n "fn search" -C3 | head -60

Length of output: 0


🏁 Script executed:

#!/bin/bash
# Look for hotpath_profiling_mode to understand output format
rg -n "hotpath_profiling_mode" -C3 | head -40

Length of output: 0


🏁 Script executed:

#!/bin/bash
# Find main.rs or CLI entry point
fd -type f -name "main.rs" -o -name "lib.rs" | head -10

Length of output: 170


🏁 Script executed:

#!/bin/bash
# Check if there's any documentation about the search command
rg -n "search.*nix" -C2 | head -30

Length of output: 1361


🏁 Script executed:

#!/bin/bash
# Look at the Cargo.toml more carefully for all feature definitions
cat Cargo.toml | grep -A 10 "\[\[features\]\]" | head -30

Length of output: 0


🏁 Script executed:

#!/bin/bash
# Get full Cargo.toml features section
sed -n '/^\[features\]/,/^\[/p' Cargo.toml | head -50

Length of output: 374


🏁 Script executed:

#!/bin/bash
# Search for "hotpath" in Cargo.toml to see all occurrences
grep -n "hotpath" Cargo.toml

Length of output: 382


🏁 Script executed:

#!/bin/bash
# Check git history for hotpath-ci - was it removed?
git log -p --all -S "hotpath-ci" -- Cargo.toml | head -50

Length of output: 59


🏁 Script executed:

#!/bin/bash
# Look for profiling or hotpath output in source code
rg -i "profiling_mode|metrics" src/ | head -30

Length of output: 0


🏁 Script executed:

#!/bin/bash
# Check if there are any recent workflow changes or comments about this
git log -n 10 --oneline -- .github/workflows/profile.yaml

Length of output: 38


Fix undefined feature flag hotpath-ci and add error handling to cargo commands.

The workflow uses --features='hotpath,hotpath-ci' (lines 31, 40, 53, 62), but the hotpath-ci feature is not defined in Cargo.toml. Valid features are: hotpath, hotpath-alloc-bytes-total, hotpath-alloc-count-total, hotpath-alloc-self, hotpath-off. This will cause the build to fail with an "unknown feature" error.

Additionally, without error handling, cargo failures and missing output will cascade silently into jq parsing failures later in the workflow.

Corrective actions:

  1. Either define the hotpath-ci feature in Cargo.toml, or replace with a valid feature combination
  2. Add set -e and explicit error checking to surface failures immediately:
  - id: head_timing_metrics
    run: |
      # Extract timing metrics from head branch
+     set -e
      {
        echo 'metrics<<EOF'
-       cargo run --features='hotpath,hotpath-ci' -- search "nix" | grep '^{"hotpath_profiling_mode"'
+       OUTPUT=$(cargo run --features='hotpath,hotpath-ci' -- search "nix" 2>&1) || { echo "Cargo failed"; exit 1; }
+       echo "$OUTPUT" | grep '^{"hotpath_profiling_mode"' || { echo "No profiling output found"; exit 1; }
        echo 'EOF'
      } >> "$GITHUB_OUTPUT"
🤖 Prompt for AI Agents
In .github/workflows/profile.yaml around lines 26 to 42, the workflow uses a
non-existent feature flag `hotpath-ci` (causing cargo to error) and lacks error
handling for the cargo invocations; replace `--features='hotpath,hotpath-ci'`
with a valid feature set (e.g., `--features='hotpath'` for timing metrics) and
replace `--features='hotpath,hotpath-ci,hotpath-alloc-count-total'` with
`--features='hotpath,hotpath-alloc-count-total'` for alloc metrics, and add
shell error handling so failures surface immediately (enable errexit and check
cargo exit status and that the command produced output, failing the job with a
clear message if cargo or grep produce no result).

Comment on lines +97 to +125
HEAD_TIMING_MAP=$(jq -s 'map({key: .function, value: .time_ns}) | from_entries' /tmp/metrics/head_timing.json)
BASE_TIMING_MAP=$(jq -s 'map({key: .function, value: .time_ns}) | from_entries' /tmp/metrics/base_timing.json)

# Parse alloc metrics
HEAD_ALLOC_MAP=$(jq -s 'map({key: .function, value: .alloc_count}) | from_entries' /tmp/metrics/head_alloc.json)
BASE_ALLOC_MAP=$(jq -s 'map({key: .function, value: .alloc_count}) | from_entries' /tmp/metrics/base_alloc.json)

# Generate comparison table
TIMING_COMPARISON=$(jq -n --argjson head "$HEAD_TIMING_MAP" --argjson base "$BASE_TIMING_MAP" '
($head | keys) as $funcs |
$funcs | map({
function: .,
head: ($head[.] // 0),
base: ($base[.] // 0),
diff: (($head[.] // 0) - ($base[.] // 0)),
pct: if ($base[.] // 0) != 0 then ((($head[.] // 0) - ($base[.] // 0)) / ($base[.] // 0) * 100) else 0 end
}) | sort_by(.pct | -.)
')

ALLOC_COMPARISON=$(jq -n --argjson head "$HEAD_ALLOC_MAP" --argjson base "$BASE_ALLOC_MAP" '
($head | keys) as $funcs |
$funcs | map({
function: .,
head: ($head[.] // 0),
base: ($base[.] // 0),
diff: (($head[.] // 0) - ($base[.] // 0)),
pct: if ($base[.] // 0) != 0 then ((($head[.] // 0) - ($base[.] // 0)) / ($base[.] // 0) * 100) else 0 end
}) | sort_by(.pct | -.)
')
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add jq input validation and document expected JSON schema.

The jq commands assume metrics JSON has specific keys (.function, .time_ns, .alloc_count). If hotpath produces a different schema, jq will silently convert missing keys to null, producing misleading comparison tables.

Add a validation step before parsing:

  - name: Generate comment
    id: comment
    run: |
      PR_NUMBER=$(cat /tmp/metrics/pr_number.txt)
      echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT

+     # Validate JSON structure before processing
+     for file in /tmp/metrics/{head,base}_timing.json /tmp/metrics/{head,base}_alloc.json; do
+       if ! jq -e '.[0] | has("function") and (has("time_ns") or has("alloc_count"))' "$file" > /dev/null 2>&1; then
+         echo "Error: Invalid JSON schema in $file" >&2
+         exit 1
+       fi
+     done

      # Parse timing metrics
      HEAD_TIMING_MAP=$(jq -s 'map({key: .function, value: .time_ns}) | from_entries' /tmp/metrics/head_timing.json)

Also add inline documentation of the expected JSON format as a comment in the workflow.

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In .github/workflows/profile.yaml around lines 97 to 125, the jq parsing assumes
each metrics JSON contains .function, .time_ns and .alloc_count which can be
missing and produce misleading results; add a validation step that (1) asserts
the metric files exist and are non-empty, (2) runs a jq predicate against each
file to ensure every element has the expected keys and correct primitive types
(e.g. .function is string, .time_ns and .alloc_count are numbers), and fails the
job with a clear message if validation fails, and (3) only then proceeds to
build HEAD_* and BASE_* maps; also insert an inline comment above these steps
documenting the expected JSON array schema for the metrics files (example: array
of objects with keys "function":string, "time_ns":number, "alloc_count":number).

Copy link

@faukah faukah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested the Rust part, all seems to work as expected. To keep my sanity, I've not taken a look at the profile.yaml, but the Rust code LGTM.

@NotAShelf
Copy link
Member Author

I wouldn't wish the YAML part on my worst enemy. I mostly copied it from hotpath repo examples, and coderabbit caught some obvious mistakes. I'll fix those tomorrow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants