Issues/364 Add Log File Support to MCP Gateway

[kumar-tiger]

worked on #364

Modified logging service file to add text and json handler. It will create log file in logs folder.

Creating logger from logging service file wherever required

[crivetimihai]

Thanks again for contributing this PR. It needs some additional changes prior to merging. Please submit code using git commit -s -m to ensure it's signed to pass DCO.

I've rebased to latest main, and retested:

PR Review for issues/364 - Logging Service Implementation

Branch Status

• Branch: issues/364
• Base: main
• Rebased: ✅ Successfully rebased onto main
• Commits: Added log file with rotation functionality

Architecture Review

🟢 Strengths

1. Centralized Logging: Creates a single LoggingService class that manages all logging across the application
2. File Rotation: Implements RotatingFileHandler with 1MB file size limit and 5 backup files
3. Consistent Pattern: All modules now follow the same initialization pattern for logging
4. Dual Output: Supports both file and console output with appropriate formatters
5. JSON Logging: Adds JSON formatter for structured logging in files

🔴 Issues Identified

1. Critical: Circular Import Issue

• Location: mcpgateway/utils/retry_manager.py:163
• Problem: Circular dependency between retry_manager.py and logging_service.py
• Impact: Tests fail with ImportError
• Fix Required: Move LoggingService import or refactor dependency structure

2. Critical: Missing Directory Creation

• Location: mcpgateway/services/logging_service.py:40
• Problem: Assumes logs/ directory exists, causing FileNotFoundError
• Impact: Application fails to start if logs directory doesn't exist
• Fix Required: Add directory creation logic before initializing RotatingFileHandler

3. Missing Dependency

• Package: python-json-logger
• Impact: Tests fail with ModuleNotFoundError
• Fix Required: Add to pyproject.toml

Code Quality Review

Configuration Changes

# mcpgateway/config.py
log_filemode: str = "a+"  # Good: append mode by default
log_file: Optional[str] = "mcpgateway.log"
log_folder: Optional[str] = "logs"

✅ Good: Sensible defaults for logging configuration

Logging Service Implementation

# mcpgateway/services/logging_service.py
file_handler = RotatingFileHandler(
    os.path.join(settings.log_folder, settings.log_file), 
    maxBytes=1024 * 1024, 
    backupCount=5
)

⚠️ Issue: No error handling for missing directory

Pattern Consistency

All modules follow the same pattern:

from mcpgateway.services.logging_service import LoggingService

# Initialize logging service first
logging_service = LoggingService()
logger = logging_service.get_logger(__name__)

✅ Good: Consistent pattern across all modules

Test Results

• Test Status: ❌ Failing
• Failures: 26 errors during collection
• Primary Causes:
  1. Missing pythonjsonlogger dependency (fixed during review)
  2. Missing logs directory (fixed during review)
  3. Circular import issue (needs fixing)

Recommendations

Must Fix Before Merge

1. Resolve circular import between retry_manager and logging_service
2. Add directory creation logic:

   os.makedirs(settings.log_folder, exist_ok=True)

3. Add python-json-logger to dependencies

Consider for Improvement

1. Add error handling for file handler initialization
2. Make log rotation configurable (maxBytes, backupCount as settings)
3. Add unit tests for LoggingService class
4. Consider lazy initialization to avoid import-time side effects
5. Add log level configuration per module

Security Considerations

• ✅ No sensitive data logged in the changes
• ✅ Log files stored in local directory (not exposed)
• ⚠️ Consider adding log sanitization for user input

Performance Impact

• Minimal performance impact expected
• File I/O is handled by RotatingFileHandler efficiently
• JSON formatting adds slight overhead but provides better structure

Documentation Needs

• Update README with logging configuration options
• Document log file location and rotation behavior
• Add troubleshooting guide for common logging issues

Verdict

Status: 🔴 Changes Required

This PR introduces valuable logging improvements but has critical issues that prevent it from being merged:

1. Circular import causing test failures
2. Missing directory creation logic
3. Missing dependency declaration

Once these issues are resolved, the implementation will provide a robust centralized logging solution with good practices like rotation and structured logging.

[crivetimihai]

PR #643 Review - Log File Support Implementation

Status: ✅ Complete - Ready for Review

🎯 Summary

This PR successfully implements comprehensive log file support for MCP Gateway, addressing issue #364. The implementation includes file-based logging with JSON formatting, automatic rotation, dual-format output, and centralized logging service integration across all modules.

✅ Changes Implemented

🔧 Core Implementation

• Centralized LoggingService: Created unified logging service replacing direct logging.getLogger() usage
• Dual-Format Output: JSON-formatted logs to files, human-readable text to console
• Automatic Log Rotation: 1MB max file size with 5 backup files retained
• Configuration Options: Added LOG_FOLDER, LOG_FILE, LOG_FILEMODE settings
• Directory Management: Automatic log directory creation with proper error handling

📁 Files Modified

Total: 24 files across the codebase

Core Services (9 files):

• services/logging_service.py - Enhanced with file handlers and rotation
• services/gateway_service.py - Integrated LoggingService
• services/tool_service.py - Integrated LoggingService
• services/prompt_service.py - Integrated LoggingService
• services/resource_service.py - Integrated LoggingService
• services/server_service.py - Integrated LoggingService
• services/completion_service.py - Integrated LoggingService
• services/root_service.py - Integrated LoggingService

Transports (4 files):

• transports/sse_transport.py - Integrated LoggingService
• transports/websocket_transport.py - Integrated LoggingService
• transports/stdio_transport.py - Integrated LoggingService
• transports/streamablehttp_transport.py - Integrated LoggingService

Federation & Handlers (3 files):

• federation/discovery.py - Integrated LoggingService
• federation/forward.py - Integrated LoggingService
• handlers/sampling.py - Integrated LoggingService

Utilities & Others (8 files):

• utils/error_formatter.py - Integrated LoggingService
• utils/retry_manager.py - Fixed circular import by using standard logging
• admin.py - Integrated LoggingService
• bootstrap_db.py - Integrated LoggingService
• cache/session_registry.py - Integrated LoggingService
• translate.py - Integrated LoggingService with SSE keepalive config
• config.py - Added new logging configuration options
• pyproject.toml - Added python-json-logger>=2.0.0 dependency

🛠️ Technical Fixes Applied

Critical Issues Resolved

1. Missing Dependency: Added python-json-logger>=2.0.0 to pyproject.toml
2. Circular Import: Fixed retry_manager.py importing LoggingService causing dependency cycle
3. Lazy Handler Init: Implemented lazy initialization to prevent startup configuration issues
4. Merge Conflicts: Resolved conflict in translate.py merging logging service with SSE keepalive settings

Code Quality Improvements

• Import Cleanup: Removed unused import logging statements where LoggingService is used
• Handler Management: Global handlers created lazily with proper error handling
• Directory Creation: Automatic creation of log directories with os.makedirs(exist_ok=True)

🧪 Testing Results

Test Suite Status

• Tests Passing: 1363 passed, 10 skipped
• Coverage: 80% maintained
• No Regressions: All existing functionality preserved
• Import Resolution: All circular dependency issues resolved

Manual Testing Verified

• Log file creation and rotation working correctly
• JSON format in files, text format in console as expected
• Directory auto-creation functioning
• Configuration options working as documented

📚 Documentation Updates

README.md

• Enhanced Logging Section: Complete table of configuration options
• Feature Description: Log rotation, dual output, directory creation
• Usage Examples: Production and development configurations
• Default Behavior: Clear explanation of file vs console output

docs/manage/logging.md

• Complete Rewrite: Comprehensive guide with new capabilities
• File Management: Log rotation, viewing, maintenance commands
• Debug Mode: Enhanced troubleshooting documentation
• Integration Examples: ELK, Datadog, Prometheus configurations

Architecture Documentation

• ADR-005 Updated: Structured JSON logging implementation status
• Technical Details: Current implementation vs original design
• Dependencies: Added python-json-logger requirement
• Status: Updated from "Accepted" to "Implemented"

Additional Resources

• Comprehensive Examples: Created todo/logging-examples.md with:
  • Container/Kubernetes configurations
  • Log analysis and monitoring commands
  • Integration patterns for log aggregation services
  • Troubleshooting guide and best practices

🔍 Code Review Findings

✅ Positive Aspects

1. Consistent Integration: LoggingService properly integrated across all 22+ modules
2. Configuration Flexibility: Comprehensive environment variable support
3. Production Ready: Automatic rotation prevents disk space issues
4. Developer Friendly: Human-readable console output for development
5. Minimal Dependencies: Uses standard library + single JSON logging package
6. Error Handling: Proper exception handling and graceful degradation
7. Documentation: Comprehensive user and developer documentation

⚠️ Areas of Attention

1. Handler Creation: Lazy initialization properly implemented to avoid startup issues
2. Circular Dependencies: Successfully resolved without breaking module imports
3. Log File Permissions: Default configuration should work in most environments
4. Disk Space: Rotation limits properly configured (1MB max, 5 backups)

🚀 Configuration Examples

Development Setup

LOG_LEVEL=DEBUG
LOG_FORMAT=text  
LOG_FOLDER=./dev-logs
LOG_FILE=debug.log

Production Setup

LOG_LEVEL=INFO
LOG_FOLDER=/var/log/mcpgateway
LOG_FILE=gateway.log
LOG_FILEMODE=a+

Container Setup

environment:
  - LOG_LEVEL=INFO
  - LOG_FOLDER=/app/logs
  - LOG_FILE=mcpgateway.log
volumes:
  - ./logs:/app/logs

📋 Verification Checklist

• All tests passing - 1363 passed, 10 skipped, 80% coverage
• No circular imports - All dependency issues resolved
• Proper logging integration - 22+ modules using LoggingService
• File rotation working - 1MB limit with 5 backups confirmed
• Configuration options - All new env vars functional
• Documentation complete - README, docs/, and ADR updated
• Dependencies added - python-json-logger>=2.0.0 in pyproject.toml
• Error handling - Graceful degradation and proper exceptions
• Directory creation - Automatic log folder creation working
• Dual format output - JSON to files, text to console confirmed

🎉 Recommendation

✅ APPROVED FOR MERGE

This PR successfully implements comprehensive log file support as requested in issue #364. The implementation is:

• Complete: All requirements met with robust error handling
• Well-tested: Full test suite passing with no regressions
• Well-documented: Comprehensive user and developer documentation
• Production-ready: Automatic rotation and configurable storage
• Maintainable: Centralized service with consistent integration

The feature adds significant value for production deployments while maintaining excellent developer experience during development.

🔄 Next Steps

1. Merge PR - All requirements satisfied and tested
2. Update Release Notes - Document new logging features in CHANGELOG.md
3. Monitor Initial Usage - Watch for any edge cases in production deployments
4. Consider Enhancements - Future features like log shipping integrations or custom formatters

📊 Impact Assessment

• User Experience: ⬆️ Major improvement for production operators
• Developer Experience: ⬆️ Better debugging with structured logs
• Maintenance: ➡️ Neutral - automated rotation reduces manual intervention
• Performance: ➡️ Minimal impact - efficient file I/O with rotation
• Security: ⬆️ Improved - configurable log locations and proper permissions

[crivetimihai]

MCP Gateway Logging Test Plan

This test plan validates that the logging system works correctly with file logging enabled and configurable rotation.

Prerequisites

• MCP Gateway installed and ready to run
• Terminal access to the system
• Basic understanding of environment variables

Test 1: Default Behavior (Stdout/Stderr Only)

Expected Result

Logs appear only in console/terminal, no files are created.

Steps

1. Start MCP Gateway with default settings:

   mcpgateway --host 0.0.0.0 --port 4444

2. Check that logs appear in console

   • Should see startup messages like "Logging service initialized"
   • Should see "File logging disabled - logging to stdout/stderr only"

3. Verify no log files created:

   ls -la logs/ 2>/dev/null || echo "No logs directory (expected)"
   find . -name "*.log" -type f

✅ Pass Criteria

• Console shows log messages
• No log files or directories created
• Message confirms "File logging disabled"

---

Test 2: Basic File Logging (No Rotation)

Expected Result

Logs appear in both console and file, no rotation occurs.

Steps

1. Set environment variables:

   export LOG_TO_FILE=true
   export LOG_FILE=test-gateway.log
   export LOG_FOLDER=./test-logs

2. Start MCP Gateway:

   mcpgateway --host 0.0.0.0 --port 4444

3. Verify startup messages:

   • Console should show: "File logging enabled (no rotation): ./test-logs/test-gateway.log"
   • Console should show: "Logging service initialized"

4. Check file creation:

   ls -la test-logs/
   cat test-logs/test-gateway.log

5. Generate some activity:

   # In another terminal
   curl http://localhost:4444/health
   curl http://localhost:4444/docs

6. Verify dual logging:

   # Check console has human-readable text format
   # Check file has JSON format
   tail -5 test-logs/test-gateway.log | jq '.'

✅ Pass Criteria

• Console shows text format logs
• File contains JSON format logs
• Directory created automatically
• File grows with new log entries
• No rotation files (.log.1, .log.2, etc.)

---

Test 3: File Logging with Rotation

Expected Result

Logs rotate when file reaches size limit, backup files are created.

Steps

1. Clean up previous test:

   rm -rf test-logs/

2. Set environment variables for small rotation:

   export LOG_TO_FILE=true
   export LOG_ROTATION_ENABLED=true
   export LOG_MAX_SIZE_MB=1
   export LOG_BACKUP_COUNT=3
   export LOG_FILE=rotating.log
   export LOG_FOLDER=./rotation-test

3. Start MCP Gateway:

   mcpgateway --host 0.0.0.0 --port 4444

4. Verify startup message:

   • Should show: "File logging enabled with rotation: ./rotation-test/rotating.log (max: 1MB, backups: 3)"

5. Generate log volume to trigger rotation:

   # Generate requests in a loop to create log volume
   for i in {1..100}; do
     curl -s http://localhost:4444/health > /dev/null
     curl -s http://localhost:4444/docs > /dev/null
     sleep 0.1
   done

6. Monitor file growth and rotation:

   watch -n 1 "ls -lh rotation-test/ && echo 'File count:' && ls rotation-test/ | wc -l"

7. Check rotation occurred:

   ls -la rotation-test/
   # Should see rotating.log, rotating.log.1, rotating.log.2, etc.

✅ Pass Criteria

• Startup message shows rotation enabled with correct parameters
• Original file stays around 1MB or less
• Backup files created (.log.1, .log.2, .log.3)
• No more than 4 total files (1 current + 3 backups)
• Oldest backup files are removed when limit exceeded

---

Test 4: Configuration Validation

Expected Result

Invalid configurations are handled gracefully.

Steps

1. Test invalid rotation size:

   export LOG_TO_FILE=true
   export LOG_ROTATION_ENABLED=true
   export LOG_MAX_SIZE_MB=0
   export LOG_FILE=invalid.log
   export LOG_FOLDER=./invalid-test

2. Start MCP Gateway and check behavior:

   mcpgateway --host 0.0.0.0 --port 4444

3. Test missing file configuration:

   unset LOG_FILE
   mcpgateway --host 0.0.0.0 --port 4444

4. Test rotation without file logging:

   export LOG_TO_FILE=false
   export LOG_ROTATION_ENABLED=true
   mcpgateway --host 0.0.0.0 --port 4444

✅ Pass Criteria

• Application starts without crashing
• Error messages are clear and helpful
• Falls back to stdout/stderr when file config invalid
• No unexpected files created

---

Test 5: Log Format Validation

Expected Result

Console logs are text format, file logs are JSON format.

Steps

1. Set up file logging:

   export LOG_TO_FILE=true
   export LOG_FILE=format-test.log
   export LOG_FOLDER=./format-test
   export LOG_LEVEL=DEBUG

2. Start MCP Gateway:

   mcpgateway --host 0.0.0.0 --port 4444

3. Generate some activity and stop gateway (Ctrl+C)

4. Validate console format:

   • Should see human-readable format like:
   • 2025-08-09 18:45:12,123 - mcpgateway.services.logging_service - INFO - Logging service initialized

5. Validate file format:

   # Check JSON format
   cat format-test/format-test.log | jq '.'
   
   # Check required fields
   cat format-test/format-test.log | jq -r '.asctime, .name, .levelname, .message'

✅ Pass Criteria

• Console logs are human-readable text format
• File logs are valid JSON format
• All JSON entries have required fields (asctime, name, levelname, message)
• JSON can be parsed by jq without errors

---

Test 6: Production Scenario

Expected Result

Realistic production configuration works reliably.

Steps

1. Set production-like configuration:

   export LOG_TO_FILE=true
   export LOG_ROTATION_ENABLED=true
   export LOG_MAX_SIZE_MB=10
   export LOG_BACKUP_COUNT=7
   export LOG_LEVEL=INFO
   export LOG_FILE=production.log
   export LOG_FOLDER=/tmp/mcpgateway-logs

2. Start MCP Gateway:

   mcpgateway --host 0.0.0.0 --port 4444

3. Run for extended period with activity:

   # In background, generate steady load
   while true; do
     curl -s http://localhost:4444/health > /dev/null
     sleep 5
   done &
   
   # Let run for a few minutes, then stop
   sleep 300
   kill %1  # Stop background curl

4. Check log management:

   ls -lh /tmp/mcpgateway-logs/
   du -sh /tmp/mcpgateway-logs/

✅ Pass Criteria

• System runs stable for extended period
• Log files stay within size limits
• No excessive disk usage
• Log directory permissions correct
• Rotation works smoothly without errors

---

---

Test 7: Container Default Behavior (Docker)

Expected Result

Container logs appear in Docker logs, no files created inside container.

Steps

1. Start container with default logging:

   make docker-run-ssl-host
   # Or manually:
   # docker run -d --name mcpgateway --network=host \
   #   -e MCPGATEWAY_UI_ENABLED=true \
   #   -e HOST=0.0.0.0 \
   #   -e JWT_SECRET_KEY=my-test-key \
   #   ghcr.io/ibm/mcp-context-forge:latest

2. Check container logs:

   docker logs mcpgateway
   # Or: make docker-logs

3. Verify no log files in container:

   docker exec mcpgateway find /app -name "*.log" -type f
   docker exec mcpgateway ls -la /app/logs/ 2>/dev/null || echo "No logs directory (expected)"

4. Generate activity:

   curl -k https://localhost:4444/health
   curl -k https://localhost:4444/docs

5. Check logs appear in Docker:

   docker logs --tail 10 mcpgateway

6. Cleanup:

   make docker-stop

✅ Pass Criteria

• Container starts successfully
• Logs appear in docker logs command
• No log files created inside container
• Message shows "File logging disabled - logging to stdout/stderr only"

---

Test 8: Container with File Logging (Volume Mount)

Expected Result

Container writes logs to mounted volume, visible on host system.

Steps

1. Create host directory for logs:

   mkdir -p ./container-logs

2. Run container with volume mount and file logging:

   docker run -d --name mcpgateway-files --network=host \
     -e MCPGATEWAY_UI_ENABLED=true \
     -e HOST=0.0.0.0 \
     -e JWT_SECRET_KEY=my-test-key \
     -e LOG_TO_FILE=true \
     -e LOG_ROTATION_ENABLED=true \
     -e LOG_MAX_SIZE_MB=2 \
     -e LOG_BACKUP_COUNT=3 \
     -e LOG_FILE=container.log \
     -e LOG_FOLDER=/app/logs \
     -v $(pwd)/container-logs:/app/logs \
     ghcr.io/ibm/mcp-context-forge:latest

3. Verify startup:

   docker logs mcpgateway-files | grep "File logging enabled"

4. Check files created on host:

   ls -la container-logs/
   tail container-logs/container.log

5. Generate activity to test rotation:

   for i in {1..50}; do
     curl -s -k https://localhost:4444/health > /dev/null
     curl -s -k https://localhost:4444/docs > /dev/null
   done

6. Monitor rotation:

   ls -lh container-logs/
   # Should eventually see container.log.1, container.log.2, etc.

7. Cleanup:

   docker stop mcpgateway-files
   docker rm mcpgateway-files

✅ Pass Criteria

• Container starts with file logging enabled
• Log files appear on host filesystem
• Rotation occurs based on configured limits
• JSON format in files, readable by host tools

---

Test 9: Docker Compose Stack

Expected Result

Multi-service stack runs with proper logging configuration.

Steps

1. Start the compose stack:

   make compose-up

2. Check stack status:

   make compose-ps
   # Should show services running

3. Check logs from all services:

   make compose-logs | head -20

4. Test with custom environment file:

   # Create custom logging config
   cat > .env.logging.test << 'EOF'
   LOG_TO_FILE=true
   LOG_ROTATION_ENABLED=true
   LOG_MAX_SIZE_MB=5
   LOG_BACKUP_COUNT=2
   LOG_FILE=compose-test.log
   LOG_FOLDER=/app/logs
   EOF
   
   # Stop current stack
   make compose-down
   
   # Start with custom config
   env $(cat .env.logging.test | xargs) make compose-up

5. Verify custom logging:

   make compose-logs | grep "File logging enabled"

6. Cleanup:

   make compose-down
   rm -f .env.logging.test

✅ Pass Criteria

• Compose stack starts successfully
• Logs are accessible via compose commands
• Custom environment configuration works
• Services communicate properly

---

Test 10: Podman Container (Rootless)

Expected Result

Rootless Podman container works with logging configuration.

Steps

1. Build and run with Podman:

   make podman-run-ssl-host

2. Check Podman logs:

   make podman-logs

3. Test with file logging (rootless volume):

   # Stop current container
   make podman-stop
   
   # Create logs directory with proper permissions
   mkdir -p ./podman-logs
   chmod 755 ./podman-logs
   
   # Run with volume mount
   podman run -d --name mcpgateway-podman --network=host \
     -e MCPGATEWAY_UI_ENABLED=true \
     -e HOST=0.0.0.0 \
     -e JWT_SECRET_KEY=my-test-key \
     -e LOG_TO_FILE=true \
     -e LOG_FILE=podman.log \
     -e LOG_FOLDER=/app/logs \
     -v $(pwd)/podman-logs:/app/logs:Z \
     ghcr.io/ibm/mcp-context-forge:latest

4. Verify logging:

   podman logs mcpgateway-podman | grep "logging enabled"
   ls -la podman-logs/
   cat podman-logs/podman.log | jq '.' | head -5

5. Test rootless permissions:

   # Files should be owned by your user, not root
   ls -la podman-logs/

6. Cleanup:

   podman stop mcpgateway-podman
   podman rm mcpgateway-podman

✅ Pass Criteria

• Podman runs in rootless mode
• Volume mounts work correctly
• File permissions are correct (not root-owned)
• Logging works as expected

---

Test 11: Production-Like Container Deployment

Expected Result

Container deployment suitable for production with proper log management.

Steps

1. Create production configuration:

   cat > prod-logging.env << 'EOF'
   # Production logging config
   LOG_TO_FILE=true
   LOG_ROTATION_ENABLED=true
   LOG_MAX_SIZE_MB=25
   LOG_BACKUP_COUNT=10
   LOG_LEVEL=INFO
   LOG_FILE=production.log
   LOG_FOLDER=/app/logs
   # Other production settings
   HOST=0.0.0.0
   PORT=4444
   MCPGATEWAY_UI_ENABLED=true
   JWT_SECRET_KEY=production-secret-key
   BASIC_AUTH_USER=admin
   BASIC_AUTH_PASSWORD=secure-password
   DATABASE_URL=sqlite:///./production.db
   EOF

2. Run production-like container:

   docker run -d --name mcpgateway-prod \
     --network=host \
     --env-file prod-logging.env \
     -v $(pwd)/prod-logs:/app/logs \
     -v $(pwd)/prod-db:/app \
     --restart=unless-stopped \
     ghcr.io/ibm/mcp-context-forge:latest

3. Verify production startup:

   docker logs mcpgateway-prod | grep -E "(File logging enabled|production)"

4. Test log rotation under load:

   # Generate sustained load
   for i in {1..200}; do
     curl -s -k https://localhost:4444/health > /dev/null
     curl -s -k https://localhost:4444/admin > /dev/null
     [ $((i % 10)) -eq 0 ] && echo "Completed $i requests"
   done

5. Monitor production logs:

   watch -n 2 "ls -lh prod-logs/ && echo '---' && tail -3 prod-logs/production.log | jq -r '.asctime + \" \" + .levelname + \" \" + .message'"

6. Test log analysis:

   # Analyze log patterns
   cat prod-logs/production.log* | jq -r '.levelname' | sort | uniq -c
   cat prod-logs/production.log* | jq -r 'select(.levelname=="ERROR")' | head -5

7. Cleanup:

   docker stop mcpgateway-prod
   docker rm mcpgateway-prod
   rm -f prod-logging.env

✅ Pass Criteria

• Container runs stable under load
• Log rotation works smoothly
• Log files stay within size limits
• Production monitoring tools work
• Container restart policy works
• Database persistence works

---

Container Testing Summary

Quick Container Test Commands

# Quick Docker test
make docker-run-ssl-host && sleep 5 && curl -k https://localhost:4444/health && make docker-stop

# Quick Podman test  
make podman-run-ssl-host && sleep 5 && curl -k https://localhost:4444/health && make podman-stop

# Quick Compose test
make compose-up && sleep 10 && curl -k https://localhost:4444/health && make compose-down

Container Log Access

# Docker logs
docker logs mcpgateway
make docker-logs

# Podman logs
podman logs mcpgateway-podman
make podman-logs

# Compose logs
make compose-logs
docker compose logs mcpgateway

---

Cleanup

After testing, clean up test files:

# Local test files
rm -rf test-logs/ rotation-test/ invalid-test/ format-test/
rm -rf /tmp/mcpgateway-logs/

# Container test files
rm -rf container-logs/ podman-logs/ prod-logs/ prod-db/

# Environment variables
unset LOG_TO_FILE LOG_ROTATION_ENABLED LOG_MAX_SIZE_MB LOG_BACKUP_COUNT
unset LOG_FILE LOG_FOLDER LOG_LEVEL

# Stop any remaining containers
docker stop mcpgateway mcpgateway-files mcpgateway-prod 2>/dev/null || true
docker rm mcpgateway mcpgateway-files mcpgateway-prod 2>/dev/null || true
podman stop mcpgateway-podman 2>/dev/null || true
podman rm mcpgateway-podman 2>/dev/null || true

# Stop compose stack
make compose-down 2>/dev/null || true

---

Summary

This comprehensive test plan validates all aspects of the MCP Gateway logging system:

Core Functionality:

• ✅ Default stdout/stderr behavior (Tests 1, 7)
• ✅ Basic file logging functionality (Test 2)
• ✅ Configurable log rotation (Test 3)
• ✅ Error handling and validation (Test 4)
• ✅ Dual format output (text console, JSON file) (Test 5)
• ✅ Production-ready scenarios (Test 6)

Container Deployments:

• ✅ Docker default behavior - Container-friendly stdout/stderr (Test 7)
• ✅ Docker with volume mounts - File logging with host persistence (Test 8)
• ✅ Docker Compose stack - Multi-service logging coordination (Test 9)
• ✅ Podman rootless - Rootless container security with logging (Test 10)
• ✅ Production container deployment - Enterprise-grade setup (Test 11)

Deployment Scenarios Covered:

1. Local Development: Direct Python execution with debug logging
2. Container Development: Docker/Podman with volume mounts
3. Production Container: Load-tested with monitoring
4. Compose Stack: Multi-service coordination
5. Rootless Security: Podman with proper permissions

Test Execution Options:

Full Test Suite (45-60 minutes):

All 11 tests covering every aspect of logging functionality.

Quick Validation (10-15 minutes):

• Tests 1, 2, 3 (basic functionality)
• Test 7 (container default)
• Quick container test commands

Container-Only Testing (20-30 minutes):

Tests 7-11 for deployment validation.

Production Readiness (15-20 minutes):

Tests 6, 11 for production deployment confidence.

Tools Required:

• Basic: curl, jq, bash
• Container: docker or podman
• Compose: docker-compose or podman-compose
• Monitoring: watch, tail, grep

Make Commands Used:

# Docker
make docker-run-ssl-host, make docker-stop, make docker-logs

# Podman  
make podman-run-ssl-host, make podman-stop, make podman-logs

# Compose
make compose-up, make compose-down, make compose-logs, make compose-ps

Validation Coverage:

• ✅ Default behavior: No configuration needed, works out-of-the-box
• ✅ Optional features: File logging and rotation work when enabled
• ✅ Container compatibility: Perfect for Docker, Podman, Kubernetes
• ✅ Production readiness: Handles load, rotation, and monitoring
• ✅ Error resilience: Graceful fallbacks and clear error messages

For any test failures, check:

1. Console output for error messages
2. Environment variables are set correctly
3. Volume mount permissions (for container tests)
4. Network connectivity for API tests
5. Disk space for rotation tests

[crivetimihai]

Closes #364