-
Notifications
You must be signed in to change notification settings - Fork 47
Feat/add new metric checker tool #4334
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ChristianHuehn
wants to merge
32
commits into
main
Choose a base branch
from
feat/add-new-metric-checker-tool
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
5a0fea8
feat(analysis): add MetricThresholdChecker tool for CI/CD
ChristianHuehn d1d8520
docs(analysis): add MetricThresholdChecker to CHANGELOG
ChristianHuehn d6b29de
docs(gh-pages): add MetricThresholdChecker documentation
ChristianHuehn e97fe2f
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn f7ab085
test(analysis): add unit tests for ThresholdConfiguration
christian-huehn-mw 4b086fc
test(analysis): add unit tests for ThresholdConfigurationLoader
christian-huehn-mw 6b1e77f
test(analysis): add unit tests for ThresholdValidator
christian-huehn-mw fecd60b
test(analysis): add unit tests for ViolationFormatter
christian-huehn-mw cf9f9f0
test(analysis): add integration tests for MetricThresholdChecker
christian-huehn-mw 5c772d8
style(analysis): fix ktlint violations in test files
christian-huehn-mw e7f9ab2
refactor(analysis): organize MetricThresholdChecker test resources
christian-huehn-mw 00cf1f0
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn 24de841
refactor(analysis): restructure MetricThresholdChecker folder organiz…
christian-huehn-mw b6bfaeb
fix(analysis): fix table alignment in MetricThresholdChecker output
christian-huehn-mw a1bfda2
refactor(analysis): use UnifiedParser in MetricThresholdChecker
christian-huehn-mw 594dc7d
refactor(analysis): address PR review comments in MetricThresholdChecker
christian-huehn-mw bec3306
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn 695bb62
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn bb7fe9b
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn 298cfe6
docs(analysis): add issue number to MetricThresholdChecker changelog …
christian-huehn-mw 3c6fc35
chore(analysis): remove unused javax-activation dependency (#4334)
christian-huehn-mw 47b46ca
test(analysis): improve MetricThresholdCheckerTest structure (#4334)
christian-huehn-mw c16b84a
test(analysis): use first() instead of array indexing in ThresholdVal…
christian-huehn-mw 68b3bb2
test(analysis): clarify NumberFormatterTest rounding expectations (#4…
christian-huehn-mw 1af0a30
feat(analysis): add validation for max < min in MetricThreshold (#4334)
christian-huehn-mw 761e8f3
refactor(analysis): remove side effects from ThresholdValidator (#4334)
christian-huehn-mw 559eacf
refactor(analysis): improve TextWrapper code clarity (#4334)
christian-huehn-mw 7cc21f7
fix(analysis): implement explicit round-half-up in NumberFormatter (#…
christian-huehn-mw 914aca2
refactor(analysis): remove suppressions and casts from ThresholdConfi…
christian-huehn-mw 6e7852a
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn 4a48633
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn a0d9c2f
Merge branch 'main' into feat/add-new-metric-checker-tool
ChristianHuehn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
295 changes: 295 additions & 0 deletions
295
analysis/analysers/tools/MetricThresholdChecker/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,295 @@ | ||
| # Metric Threshold Checker | ||
|
|
||
| The Metric Threshold Checker is a CLI tool that validates code metrics against configured thresholds for CI/CD pipelines. It uses the UnifiedParser internally to analyze code and reports violations when metrics exceed the specified limits. | ||
|
|
||
| ## Usage | ||
|
|
||
| ```bash | ||
| ccsh metricthresholdchecker <input-path> --config <config-file> [options] | ||
| ``` | ||
|
|
||
| ### Parameters | ||
|
|
||
| - `<input-path>` - File or folder to analyze | ||
| - `-c, --config <config-file>` - Threshold configuration file (JSON or YAML) **(required)** | ||
| - `-e, --exclude <patterns>` - Comma-separated list of regex patterns to exclude files/folders | ||
| - `-fe, --file-extensions <extensions>` - Comma-separated list of file extensions to parse only those files | ||
| - `--bypass-gitignore` - Bypass .gitignore files and use regex-based exclusion instead | ||
| - `--verbose` - Enable verbose mode for detailed output | ||
|
|
||
| ## Configuration Format | ||
|
|
||
| The threshold configuration file can be in either YAML or JSON format. All thresholds are defined under `file_metrics` since the UnifiedParser stores all metrics at the file level (including aggregated function statistics). | ||
|
|
||
| Each metric can have: | ||
| - `min`: Minimum acceptable value (violation if below) | ||
| - `max`: Maximum acceptable value (violation if above) | ||
| - Both `min` and `max` can be specified for the same metric | ||
|
|
||
| ### Example Configuration (YAML) | ||
|
|
||
| ```yaml | ||
| file_metrics: | ||
| # Lines of code metrics | ||
| rloc: | ||
| max: 500 # Real lines of code per file | ||
| loc: | ||
| max: 600 # Total lines of code per file | ||
|
|
||
| # Complexity metrics | ||
| complexity: | ||
| max: 100 # Total file complexity | ||
| max_complexity_per_function: | ||
| max: 10 # No function should be too complex | ||
|
|
||
| # Function count and size | ||
| number_of_functions: | ||
| max: 20 # Not too many functions per file | ||
| max_rloc_per_function: | ||
| max: 50 # No function should be too long | ||
| mean_rloc_per_function: | ||
| max: 20 # Average function length | ||
| ``` | ||
|
|
||
| ### Example Configuration (JSON) | ||
|
|
||
| ```json | ||
| { | ||
| "file_metrics": { | ||
| "rloc": { | ||
| "max": 500 | ||
| }, | ||
| "complexity": { | ||
| "max": 100 | ||
| }, | ||
| "max_complexity_per_function": { | ||
| "max": 10 | ||
| }, | ||
| "number_of_functions": { | ||
| "max": 20 | ||
| }, | ||
| "max_rloc_per_function": { | ||
| "max": 50 | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| **Note:** The UnifiedParser stores all metrics at the file level. Function-level data is aggregated as max/min/mean/median statistics within each file. | ||
|
|
||
| ## Available Metrics | ||
|
|
||
| All metrics are file-level metrics from the UnifiedParser. Function-level data is aggregated into statistics. | ||
|
|
||
| ### Lines of Code Metrics | ||
| - `rloc` - Real lines of code (excluding comments and empty lines) | ||
| - `loc` - Total lines of code (including everything) | ||
| - `comment_lines` - Number of comment lines | ||
|
|
||
| ### Complexity Metrics | ||
| - `complexity` - Total cyclomatic complexity of the file | ||
| - `logic_complexity` - Logic complexity | ||
| - `max_complexity_per_function` - Highest complexity of any function in the file | ||
| - `min_complexity_per_function` - Lowest complexity of any function | ||
| - `mean_complexity_per_function` - Average complexity across all functions | ||
| - `median_complexity_per_function` - Median complexity | ||
|
|
||
| ### Function Count | ||
| - `number_of_functions` - Total number of functions in the file | ||
|
|
||
| ### Function Size Metrics (Aggregated) | ||
| - `max_rloc_per_function` - Length of the longest function | ||
| - `min_rloc_per_function` - Length of the shortest function | ||
| - `mean_rloc_per_function` - Average function length | ||
| - `median_rloc_per_function` - Median function length | ||
|
|
||
| ### Function Parameter Metrics (Aggregated) | ||
| - `max_parameters_per_function` - Most parameters any function has | ||
| - `min_parameters_per_function` - Fewest parameters any function has | ||
| - `mean_parameters_per_function` - Average parameters per function | ||
| - `median_parameters_per_function` - Median parameters per function | ||
|
|
||
| **Note:** All metrics are stored at the file level. There is no separate `function_metrics` section in the configuration. | ||
|
|
||
| ## Exit Codes | ||
|
|
||
| - `0` - All thresholds passed | ||
| - `1` - One or more threshold violations found | ||
| - `2` - Configuration or parsing errors | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Basic Usage | ||
|
|
||
| ```bash | ||
| ccsh metricthresholdchecker ./src --config thresholds.yml | ||
| ``` | ||
|
|
||
| ### Exclude Test Files | ||
|
|
||
| ```bash | ||
| ccsh metricthresholdchecker ./src --config thresholds.yml --exclude ".*test.*,.*spec.*" | ||
| ``` | ||
|
|
||
| ### Analyze Specific File Extensions | ||
|
|
||
| ```bash | ||
| ccsh metricthresholdchecker ./src --config thresholds.yml --file-extensions kt,java | ||
| ``` | ||
|
|
||
| ### Verbose Output | ||
|
|
||
| ```bash | ||
| ccsh metricthresholdchecker ./src --config thresholds.yml --verbose | ||
| ``` | ||
|
|
||
| ## CI/CD Integration | ||
|
|
||
| ### GitHub Actions | ||
|
|
||
| ```yaml | ||
| - name: Check Code Metrics | ||
| run: | | ||
| ./gradlew installDist | ||
| ./build/install/codecharta-analysis/bin/ccsh metricthresholdchecker src/ -c .codecharta-thresholds.yml | ||
| ``` | ||
|
|
||
| ### GitLab CI | ||
|
|
||
| ```yaml | ||
| code-quality: | ||
| script: | ||
| - ./gradlew installDist | ||
| - ./build/install/codecharta-analysis/bin/ccsh metricthresholdchecker src/ -c thresholds.yml | ||
| ``` | ||
|
|
||
| ### Jenkins | ||
|
|
||
| ```groovy | ||
| stage('Code Quality') { | ||
| steps { | ||
| sh './gradlew installDist' | ||
| sh './build/install/codecharta-analysis/bin/ccsh metricthresholdchecker src/ -c thresholds.yml' | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Output Format | ||
|
|
||
| When violations are found, the tool displays a formatted table showing: | ||
| - File path | ||
| - Metric name | ||
| - Actual value | ||
| - Threshold (min/max) | ||
| - How much the value exceeds the threshold | ||
|
|
||
| **Violations are sorted by how much they exceed the threshold (worst first) within each metric group**, making it easy to identify the most problematic files that need attention. | ||
|
|
||
| Example output: | ||
|
|
||
| ``` | ||
| Metric Threshold Check Results | ||
| ════════════════════════════════════════════════════════════ | ||
| ✗ 3 violation(s) found! | ||
|
|
||
| Violations by type: | ||
| - File metric violations: 3 | ||
| ════════════════════════════════════════════════════════════ | ||
|
|
||
| Violations: | ||
|
|
||
| Metric: rloc (2 violations) | ||
|
|
||
| Path Actual Value Threshold Exceeds By | ||
| ─────────────────────────────────────────────────────────────────────────── | ||
| src/HugeFile.kt 750 max: 500 +250 ← Worst violation first | ||
| src/LargeFile.kt 550 max: 500 +50 | ||
|
|
||
| Metric: mcc (1 violations) | ||
|
|
||
| Path Actual Value Threshold Exceeds By | ||
| ─────────────────────────────────────────────────────────────────────────── | ||
| src/Complex.kt 120 max: 100 +20 | ||
| ``` | ||
|
|
||
| ## Tips for Using the Tool | ||
|
|
||
| ### Prioritizing Fixes | ||
|
|
||
| Since violations are sorted by severity (how much they exceed the threshold), focus on the files at the top of each metric list first. These represent the most significant violations and will have the biggest impact when fixed. | ||
|
|
||
| ### Iterative Improvement | ||
|
|
||
| Start with lenient thresholds and gradually tighten them: | ||
|
|
||
| ```yaml | ||
| # Phase 1: Set thresholds above current worst violations | ||
| file_metrics: | ||
| rloc: | ||
| max: 1000 # Start high | ||
|
|
||
| # Phase 2: After fixing worst offenders, tighten | ||
| file_metrics: | ||
| rloc: | ||
| max: 500 # Reduce gradually | ||
|
|
||
| # Phase 3: Aim for best practices | ||
| file_metrics: | ||
| rloc: | ||
| max: 200 # Target healthy file size | ||
| ``` | ||
|
|
||
| ### CI Integration Best Practice | ||
|
|
||
| Run the checker on every pull request to prevent new violations from being introduced, even if existing violations remain: | ||
|
|
||
| ```yaml | ||
| # .github/workflows/code-quality.yml | ||
| - name: Check New Code | ||
| run: | | ||
| ccsh metricthresholdchecker src/ --config .codecharta-thresholds.yml | ||
| ``` | ||
|
|
||
| ## Common Use Cases | ||
|
|
||
| ### Prevent Large Files | ||
| ```yaml | ||
| file_metrics: | ||
| rloc: | ||
| max: 300 # Files shouldn't be too long | ||
| ``` | ||
|
|
||
| ### Control Complexity | ||
| ```yaml | ||
| file_metrics: | ||
| max_complexity_per_function: | ||
| max: 10 # No function should be too complex | ||
| complexity: | ||
| max: 50 # Total file complexity | ||
| ``` | ||
|
|
||
| ### Ensure Test Coverage Balance | ||
| ```yaml | ||
| file_metrics: | ||
| number_of_functions: | ||
| max: 30 # Keep files focused | ||
| min: 1 # Ensure files have at least one function | ||
| ``` | ||
|
|
||
| ### Monitor Function Size | ||
| ```yaml | ||
| file_metrics: | ||
| max_rloc_per_function: | ||
| max: 50 # No giant functions | ||
| mean_rloc_per_function: | ||
| max: 20 # Keep average function size small | ||
| ``` | ||
|
|
||
| ## Notes | ||
|
|
||
| - The tool uses Jackson for YAML/JSON parsing (no CVEs in current version) | ||
| - Thresholds are applied only to files that contain the specified metrics | ||
| - All metrics are file-level; function data is aggregated as max/min/mean/median statistics | ||
| - The tool respects `.gitignore` files by default (use `--bypass-gitignore` to disable) | ||
| - Violations are grouped by metric type and sorted by severity within each group | ||
| - Only use `file_metrics` in your configuration; there is no `function_metrics` section |
17 changes: 17 additions & 0 deletions
17
analysis/analysers/tools/MetricThresholdChecker/build.gradle.kts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| dependencies { | ||
| implementation(project(":model")) | ||
| implementation(project(":dialogProvider")) | ||
| implementation(project(":analysers:AnalyserInterface")) | ||
| implementation(project(":analysers:parsers:UnifiedParser")) | ||
|
|
||
| implementation(libs.picocli) | ||
| implementation(libs.kotter) | ||
| implementation(libs.kotter.test) | ||
| implementation(libs.jackson.databind) | ||
| implementation(libs.jackson.dataformat.yaml) | ||
| implementation(libs.jackson.module.kotlin) | ||
| } | ||
|
|
||
| tasks.test { | ||
| useJUnitPlatform() | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function mirrors a lot of the functionality of the private "scanInputProject" function without:
This should be refactored to avoid duplicated code