Add secure scanning of MCP Servers (#184)

* change v0 ->  v0.1

* v0 -> v0.1 doc and test changes

* using a constant variable

* api version to the improt script

* removed version from readme.md

* replace v0 to ANTHROPIC_API_VERSION in md

* fixed the syntax error

* changed name back to v0

* reverting release notes change

* changed v0.1 to {ANTHROPIC_API_VERSION}

* move v0.1 -> v0

* v0.1 -> v0

* more updates for uniformity

* docker changes

* updated readme.md

* updated docs/anthropic-registry-import.md

* updated anthropic_registry_spi.md

* updated design docs

* changed {REGISTRY_CONSTANTS.ANTHROPIC_API_VERSION} to v0.1 in design docs

* removed date and issue number

* renamed v0_routes -> registry_routes

* renamed test_v0_routes to test_registry_routes

* refactored

* changed import name from v0_router to registry_router

* updates to md file

* changed {{API_VERSION}} to {{{ANTHROPIC_API_VERSION}}

* cisco security scanner

* added llm to default analyzer

* setting MCP_SCANNER_LLM_API_KEY to env variable

* reverting llm change

* both yara and llm

* added #Security scans to gitignore

* checking for placeholder value

* Add security scanning enhancements and workflow improvements

- Add scan_all_servers.py CLI tool for bulk security scanning
  - Supports --token and --token-file parameters with priority handling
  - Generates comprehensive markdown reports with detailed findings
  - Reports saved to security_scans/scan_report.md (latest) and security_scans/reports/ (timestamped archives)
  - Masks tokens in logs for security (shows first 20 and last 10 chars)

- Enhance service_mgmt.sh security scan workflow
  - Auto-append /mcp to proxy_pass_url if not ending with /mcp or /sse
  - Load ADMIN_PASSWORD from .env file for auto-disabling unsafe servers
  - Fix authentication header forwarding in auth-server

- Update auth_server/server.py
  - Add fallback to check Authorization header if X-Authorization not present
  - Explicit priority: X-Authorization > Authorization

- Add mcp_security_scanner.py header support
  - Parse --headers argument and extract Bearer token
  - Pass token to mcp-scanner via --bearer-token

- Add example configs
  - shawndurrani-ai-server-config.json for external MCP server

- Update .gitignore
  - Add .roo/ for Roo IDE files

* Update security scanner documentation

Changes:
- Renamed docs/cisco-security-scanner-setup.md to docs/security-scanner.md
- Rewrote documentation to be generic (not Cisco-specific)
- Added MCP Supply Chain Security introduction
- Documented integration with Cisco AI Defence MCP Scanner
- Section 1: Security scanning during server addition
  - Command format and examples
  - Real config example (cloudflare-docs-server-config.json)
  - Real scan output example (docs.mcp.cloudflare.com_mcp.json)
  - Explained disabled state and security-pending tag
  - Added placeholder for screenshot
- Section 2: Periodic registry scans
  - Command examples for scan_all_servers.py
  - Report location and structure (security_scans/scan_report.md)
  - Reference to scan_report_example.md
- Updated README.md:
  - Added security scanning to "What's New" section
  - Added "Security Scanning" subsection to Enterprise Features
- Removed unnecessary prerequisites:
  - MCP Scanner install (already in pyproject.toml)
  - Registry admin credentials (handled by .env)
- Removed redundant troubleshooting section

Files changed:
- docs/cisco-security-scanner-setup.md → docs/security-scanner.md
- README.md (What's New + Enterprise Features sections)
- cli/examples/cloudflare-docs-server-config.json (new example)
- docs/scan_report_example.md (new reference report)

* Fix health checks and tool fetching for Cloudflare and streamable-http servers

This commit fixes multiple issues with health checks and automatic tool discovery:

1. Health Check - Proper MCP Session Management
   - Add proper MCP initialize flow to get session ID from server
   - Use server-generated session ID for subsequent ping requests
   - Skip URL pattern shortcut when supported_transports contains streamable-http
   - Handle auth failures during initialize by falling back to ping without auth

2. Tool Fetching - Header and URL Fixes
   - Add required Accept header: application/json, text/event-stream
   - Remove trailing slash from MCP URLs (Cloudflare rejects it)
   - Fix MCP client to properly handle Cloudflare's requirements

3. Tool Auto-Discovery - Enhanced Logic
   - Always fetch tools on first health check (previous_status == UNKNOWN)
   - Fetch tools when server transitions to healthy
   - Fetch tools if server is healthy but has empty tool_list
   - Ensures tools populate automatically on startup and registration

4. Import Script - Preserve Transport Type
   - Stop removing supported_transports field during import
   - Allows SSE servers to be registered with correct transport type
   - Fixes health checks for servers like ai.shawndurrani-mcp-merchant

Fixes Cloudflare Documentation MCP Server health checks and tool discovery.
Fixes sre-gateway showing unhealthy when auth token expires.

* Update security scanner documentation with screenshot reference

Replace placeholder text with actual screenshot reference for failed security scan.
Shows how servers that fail security scans are added in disabled state with
security-pending tag.

* Replace specific domain with example.com in security scanner docs

Use mcpgateway.example.com instead of mcpgateway.ddns.net for better
documentation practices with a generic example domain.

* Render example report summary as markdown instead of code block

Remove markdown code block wrapper from example report summary to display
it as rendered markdown for better readability.

* Remove redundant installation note from prerequisites

The statement about MCP Scanner being included in pyproject.toml is
unnecessary in the Prerequisites section.

---------

Co-authored-by: Nisha Deborah Philips <[email protected]>
Co-authored-by: Amit Arora <[email protected]>

Author: 3 people

.env.example

@@ -130,6 +130,12 @@ SMITHERY_API_KEY=your_smithery_api_key_here
 # Get this from https://console.anthropic.com/
 ANTHROPIC_API_KEY=your_anthropic_api_key_here
 
+# MCP Security Scanner LLM API Key (optional - only needed for LLM-based security analysis)
+# Default analyzer is YARA (no API key required)
+# To use LLM analyzer: ./cli/service_mgmt.sh add config.json yara,llm
+# Get OpenAI API key from https://platform.openai.com/api-keys
+MCP_SCANNER_LLM_API_KEY=your_openai_api_key_here
+
 # =============================================================================
 # CONTAINER REGISTRY CREDENTIALS (for CI/CD and local builds)
 # =============================================================================

.gitignore

@@ -191,6 +191,9 @@ cookies.txt
 # Scratchpad for temporary notes and planning
 .scratchpad/
 
+# Roo IDE files
+.roo/
+
 # OAuth tokens and credentials - never commit these!
 .oauth-tokens/
 .agentcore-params
@@ -281,4 +284,7 @@ coverage/
 
 # Anthropic registry temporary files
 anthropic_servers_*.json
-curated_import_list.txt
+curated_import_list.txt
+
+#Security scans
+security_scans/

README.md

@@ -82,6 +82,7 @@ The **MCP Gateway & Registry** is an enterprise-ready platform that centralizes
 
 ## What's New
 
+- **🔒 MCP Server Security Scanning** - Integrated vulnerability scanning with Cisco AI Defence MCP Scanner. Automatic security scans during server registration, periodic registry-wide scans with detailed markdown reports, and automatic disabling of servers with security issues.
 - **📥 Import Servers from Anthropic MCP Registry** - Import curated MCP servers from Anthropic's registry with a single command. [Import Guide](docs/anthropic-registry-import.md)
 - **🔌 Anthropic MCP Registry REST API Compatibility** - Full compatibility with Anthropic's MCP Registry REST API specification. [API Documentation](docs/anthropic_registry_api.md)
 - **🚀 Pre-built Images** - Deploy instantly with pre-built Docker images. [Get Started](#option-a-pre-built-images-instant-setup) | [macOS Guide](docs/macos-setup-guide.md)
@@ -373,6 +374,14 @@ Seamlessly integrate with Anthropic's official MCP Registry to import and access
 
 [Import Guide](docs/anthropic-registry-import.md) | [Registry API Documentation](docs/anthropic_registry_api.md)
 
+### Security Scanning
+
+**Integrated Vulnerability Detection:**
+- **Automated Security Scanning** - Integrated vulnerability scanning for MCP servers using Cisco AI Defence MCP Scanner, with automatic scans during registration and support for periodic registry-wide scans
+- **Detailed Security Reports** - Comprehensive markdown reports with vulnerability details, severity assessments, and remediation recommendations
+- **Automatic Protection** - Servers with security issues are automatically disabled with security-pending status to protect your infrastructure
+- **Compliance Ready** - Security audit trails and vulnerability tracking for enterprise compliance requirements
+
 ### Authentication & Authorization
 
 **Multiple Identity Modes:**

auth_server/server.py

@@ -846,7 +846,11 @@ async def validate_request(request: Request):
 
     try:
         # Extract headers
+        # Check for X-Authorization first (custom header used by this gateway)
+        # Only if X-Authorization is not present, check standard Authorization header
         authorization = request.headers.get("X-Authorization")
+        if not authorization:
+            authorization = request.headers.get("Authorization")
         cookie_header = request.headers.get("Cookie", "")
         user_pool_id = request.headers.get("X-User-Pool-Id")
         client_id = request.headers.get("X-Client-Id")

cli/examples/cloudflare-docs-server-config.json

@@ -0,0 +1,7 @@
+{
+  "server_name": "Cloudflare Documentation MCP Server",
+  "description": "Search Cloudflare documentation and get migration guides",
+  "path": "/cloudflare-docs",
+  "proxy_pass_url": "https://docs.mcp.cloudflare.com/mcp",
+  "supported_transports": ["streamable-http"]
+}

cli/examples/minimal-server-config.json

@@ -2,5 +2,6 @@
   "server_name": "Minimal MCP Server",
   "description": "A minimal server configuration with only required fields",
   "path": "/minimal-server",
-  "proxy_pass_url": "http://minimal-server:9001/"
+  "proxy_pass_url": "http://minimal-server:9001/",
+  "supported_transports": ["streamable-http"]
 }

cli/import_from_anthropic_registry.sh

@@ -6,11 +6,12 @@
 # and registers them with the local MCP Gateway Registry.
 #
 # Usage:
-#   ./import_from_anthropic_registry.sh [--dry-run] [--import-list <file>]
+#   ./import_from_anthropic_registry.sh [--dry-run] [--import-list <file>] [--analyzers <analyzers>]
 #
 # Environment Variables:
 #   GATEWAY_URL - Gateway URL (default: http://localhost)
 #                 Example: export GATEWAY_URL=https://mcpgateway.ddns.net
+#   MCP_SCANNER_LLM_API_KEY - API key for LLM-based security analysis (required if using llm analyzer)
 #
 
 set -e
@@ -87,17 +88,37 @@ validate_package() {
 # Parse arguments
 DRY_RUN=false
 IMPORT_LIST="$SCRIPT_DIR/import_server_list.txt"
+ANALYZERS="yara"
 
 while [[ $# -gt 0 ]]; do
     case $1 in
         --dry-run) DRY_RUN=true; shift ;;
         --import-list) IMPORT_LIST="$2"; shift 2 ;;
+        --analyzers) ANALYZERS="$2"; shift 2 ;;
         --help)
-            echo "Usage: $0 [--dry-run] [--import-list <file>]"
+            echo "Usage: $0 [--dry-run] [--import-list <file>] [--analyzers <analyzers>]"
+            echo ""
+            echo "Options:"
+            echo "  --dry-run              Dry run mode (don't register servers)"
+            echo "  --import-list <file>   Server list file (default: import_server_list.txt)"
+            echo "  --analyzers <list>     Security analyzers: yara, llm, or yara,llm (default: yara)"
             echo ""
             echo "Environment Variables:"
             echo "  GATEWAY_URL - Gateway URL (default: http://localhost)"
             echo "                Example: export GATEWAY_URL=https://mcpgateway.ddns.net"
+            echo "  MCP_SCANNER_LLM_API_KEY - API key for LLM analyzer (required if using llm)"
+            echo ""
+            echo "Examples:"
+            echo "  # Import with default YARA analyzer"
+            echo "  $0"
+            echo ""
+            echo "  # Import with both YARA and LLM analyzers"
+            echo "  export MCP_SCANNER_LLM_API_KEY=sk-..."
+            echo "  $0 --analyzers yara,llm"
+            echo ""
+            echo "  # Import with only LLM analyzer"
+            echo "  export MCP_SCANNER_LLM_API_KEY=sk-..."
+            echo "  $0 --analyzers llm"
             exit 0 ;;
         *) echo "Unknown option: $1"; exit 1 ;;
     esac
@@ -108,6 +129,21 @@ command -v jq >/dev/null || { print_error "jq required"; exit 1; }
 command -v curl >/dev/null || { print_error "curl required"; exit 1; }
 [ -f "$IMPORT_LIST" ] || { print_error "Import list not found: $IMPORT_LIST"; exit 1; }
 
+# Check if LLM analyzer is requested and API key is available
+if [[ "$ANALYZERS" == *"llm"* ]]; then
+    if [ -z "$MCP_SCANNER_LLM_API_KEY" ] || [[ "$MCP_SCANNER_LLM_API_KEY" == *"your_"* ]] || [[ "$MCP_SCANNER_LLM_API_KEY" == *"placeholder"* ]]; then
+        echo ""
+        print_error "LLM analyzer requested but MCP_SCANNER_LLM_API_KEY is not configured"
+        print_info "Current value: ${MCP_SCANNER_LLM_API_KEY:-<not set>}"
+        print_info ""
+        print_info "Options:"
+        print_info "  1. Add real API key to .env file: MCP_SCANNER_LLM_API_KEY=sk-..."
+        print_info "  2. Set environment variable: export MCP_SCANNER_LLM_API_KEY=sk-..."
+        print_info "  3. Use only YARA analyzer: $0 --analyzers yara"
+        exit 1
+    fi
+fi
+
 mkdir -p "$TEMP_DIR"
 
 # Read server list
@@ -119,6 +155,7 @@ while IFS= read -r line; do
 done < "$IMPORT_LIST"
 
 print_info "Found ${#servers[@]} servers to import"
+print_info "Security analyzers: $ANALYZERS"
 
 # Process each server
 success_count=0
@@ -192,10 +229,9 @@ result['path'] = '/$safe_path'
 
 # Remove unsupported fields for register_service tool
 # The user-facing register_service tool only supports basic fields
-# Note: auth_type, auth_provider, and headers are now kept for proper auth handling
+# Note: auth_type, auth_provider, headers, supported_transports, and tool_list are kept
 unsupported_fields = [
-    'repository_url', 'website_url', 'package_npm', 'remote_url',
-    'supported_transports', 'tool_list'
+    'repository_url', 'website_url', 'package_npm', 'remote_url'
 ]
 for field in unsupported_fields:
     result.pop(field, None)
@@ -209,14 +245,14 @@ with open('$config_file', 'w') as f:
 
     # Register with service_mgmt.sh (if not dry run)
     if [ "$DRY_RUN" = false ]; then
-        if GATEWAY_URL="$GATEWAY_URL" "$SCRIPT_DIR/service_mgmt.sh" add "$config_file"; then
+        if GATEWAY_URL="$GATEWAY_URL" "$SCRIPT_DIR/service_mgmt.sh" add "$config_file" "$ANALYZERS"; then
             print_success "Registered $server_name"
             success_count=$((success_count + 1))
         else
             print_error "Failed to register $server_name"
         fi
     else
-        print_info "[DRY RUN] Would register $server_name"
+        print_info "[DRY RUN] Would register $server_name with analyzers: $ANALYZERS"
         success_count=$((success_count + 1))
     fi