MCP tool usability improvements: validation errors, parameter naming, and SARIF enhancements

[data-douser]

Summary

Several usability issues were observed during an extended CodeQL analysis session using ql-mcp tools via an LLM agent (VS Code Copilot). These fall into two categories: tool input validation bugs that waste invocations, and missing capabilities that forced manual workarounds.

Related: #207 (the planned gh-ql-mcp-client Go rewrite addresses some of the SARIF workflow gaps noted here)

---

1. Validation errors revealed one-at-a-time

Tools affected: audit_store_findings, annotation_create (likely all tools using JSON Schema validation)

Current state: When multiple required parameters are missing, the tool returns an error for only the first missing field. After supplying that field and re-invoking, a second missing field is reported, and so on. This forces iterative trial-and-error to discover the full parameter contract.

Example with audit_store_findings:

1. Invoked without owner → must have required property 'owner'
2. Added owner, re-invoked → must have required property 'sourceLocation'

Example with annotation_create:

1. Invoked without category → must have required property 'category'
2. Added category, re-invoked → must have required property 'entityKey'
3. Added entityKey → success

Desired state: A single validation error listing ALL missing required properties (e.g. must have required properties: 'owner', 'sourceLocation').

Requirements

• Enable allErrors: true (or equivalent) in the JSON Schema validator so all violations are reported in one response
• Aggregate multiple validation errors into a single error message

---

2. database_create uses kebab-case params with unhelpful rejection message

Tool: codeql_database_create

Current state: The tool accepts source-root (kebab-case) but rejects sourceRoot (camelCase) with:

Your input to the tool was invalid (must NOT have additional properties)

The error does not identify which property was rejected. LLM agents commonly default to camelCase JSON keys, making this a frequent first-invocation failure.

Desired state: Either accept both naming conventions, or include the rejected property name in the error message.

Requirements

• Include the name of the unrecognized property in the error (e.g. unknown property 'sourceRoot' — did you mean 'source-root'?)
• Consider accepting camelCase aliases for kebab-case parameters

---

3. Python database extracts node_modules/ template files

Tool: codeql_database_create

Current state: The Python extractor picks up template .py files from node_modules/ (e.g. aws-cdk init templates with placeholder syntax). These produce py/syntax-error results in the analysis that are always false positives.

Desired state: node_modules/ is excluded by default for interpreted language extractors (matching CodeQL's own LGTM.com behavior), or an --exclude-paths parameter is surfaced.

Requirements

• Auto-exclude node_modules/ from Python and JavaScript database creation by default, or
• Expose an exclude-paths parameter on codeql_database_create

---

4. sarif_list_rules lacks per-rule result counts

Tool: sarif_list_rules

Current state: Returns full rule metadata (description, help, tags, severity) for every rule in the SARIF file but does not include the count of results per rule. To determine which rules actually produced findings and how many, the SARIF file must be parsed separately.

Desired state: Each rule entry includes a resultCount field.

Requirements

• Add a resultCount field to each rule in the sarif_list_rules response

---

5. No SARIF-to-git-diff correlation tool

Tool: (missing — feature request)

Current state: There is no MCP tool for determining whether a given SARIF alert is associated with code locations changed in a set of git commits. To triage SARIF results by branch, we had to:

1. Run git diff --name-only <merge-base>..HEAD to get changed files
2. Parse each SARIF file's results[].locations[].physicalLocation.artifactLocation.uri
3. Cross-reference manually to classify findings as "new on this branch" vs "pre-existing"

Desired state: A tool like sarif_diff_by_commits that accepts a SARIF path and a git ref range (or commit list), and returns results partitioned into {new: [...], preexisting: [...]} based on whether the finding's file (and optionally line range) overlaps with the diff. GitHub's Code Scanning API already performs this logic server-side for PR checks.

Requirements

• New tool that accepts a SARIF file path and a git ref range
• Partitions SARIF results into "new" vs "pre-existing" based on file-level (and optionally line-level) overlap with the git diff
• Returns structured output suitable for triage workflows

[data-douser]

Note that the codeql database analyze CLI command does not have specific arguments for excluding or ignoring specific paths. The right approach would probably be for the ql-mcp server to support managing and consistently referencing its own, custom Code Scanning (CodeQL) config file:

From codeql database analyze -h -vv output:

      --codescanning-config=<file>
                             [Advanced] Read a Code Scanning configuration file
                               specifying options on how to create the CodeQL
                               databases and what queries to run in later
                               steps. For more details on the format of this
                               configuration file, refer to https://aka.
                               ms/code-scanning-docs/config-file. To run
                               queries from this file in a later step, invoke
                               codeql database analyze without any other
                               queries specified.

The https://aka.ms/code-scanning-docs/config-file link just redirects to https://docs.github.com/en/code-security/reference/code-scanning/workflow-configuration-options#using-a-custom-configuration-file. This custom config file could then support a custom paths-ignore list of (file and/or directory) paths to be ignored during CodeQL dataset extraction (which is part of CodeQL database creation), where the wrapping vscode-codeql-development-mcp-server extension (for VSIX installations of ql-mcp server) needs extension config settings to allow the user to customize the (1) path of the custom CodeQL config file used by the ql-mcp server, and (2) the value(s) of commonly used custom CodeQL config fields like paths-ignore.