Add DevOps REST API for dynamic WoT validation configuration

[hu-ahmed]

Currently, WoT validation configuration in Ditto is static and can only be modified through configuration files or ConfigMaps. This makes it challenging to handle model migrations and deployment transitions where temporary validation relaxation might be needed.

Proposed Solution:
Add a new DevOps REST API that provides CRUD operations for WoT validation settings, using Ditto's existing caching infrastructure and MongoDB for persistence.

Technical Details:

1. Storage Layer:

   • MongoDB for persistent storage using pekko persistence
   • Caffeine cache for in-memory caching
   • Ditto's pub/sub system for cluster-wide cache invalidation

2. Startup Process:

   • On service startup, load static configuration from things.conf
   • Initialize Caffeine cache with static configuration
   • Subscribe to pub/sub topic for dynamic updates
   • Load dynamic configurations from MongoDB
   • Merge static and dynamic configurations
   • Cache the merged result

3. Cache Invalidation Flow:

   • When config is updated via API:
     1. Update MongoDB
     2. Update local cache
     3. Publish update event via pub/sub
     4. Other instances receive event and update their caches
   • On instance startup:
     1. Load static config
     2. Load dynamic configs from MongoDB
     3. Subscribe to pub/sub updates
     4. Cache merged configuration

4. API Endpoints:

   GET    /api/2/devops/wot/validationConfigs
   GET    /api/2/devops/wot/validationConfigs/{scopeId}
   PUT    /api/2/devops/wot/validationConfigs/{scopeId}
   PATCH  /api/2/devops/wot/validationConfigs/{scopeId}
   DELETE /api/2/devops/wot/validationConfigs/{scopeId}
   GET    /api/2/devops/wot/validationConfigs/merged
   

5. Configuration Structure:

   {
     "validation-context": {
       "ditto-headers-patterns": [
         {
           "header-name": "pattern"
         }
       ],
       "thing-definition-patterns": [
         "pattern"
       ],
       "feature-definition-patterns": [
         "pattern"
       ]
     },
     "config-overrides": {
       "enabled": true,
       "log-warning-instead-of-failing-api-calls": false,
       "thing": {
         "enforce": {
   
         },
         "forbid": {
   
   
         }
       },
       "feature": {
         "enforce": {
   
         },
         "forbid": {
   
   
         }
       }
     }
   }

Benefits:

1. Real-time configuration updates across all Ditto instances
2. Temporary validation relaxation for migrations
3. No service restart required
4. Built-in persistence and caching
5. Automatic cache synchronization on startup and updates