Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: Clavata integration #1025

Open
1 task done
ilias-t opened this issue Mar 5, 2025 · 0 comments
Open
1 task done

feature: Clavata integration #1025

ilias-t opened this issue Mar 5, 2025 · 0 comments
Labels
enhancement New feature or request status: needs triage New issues that have not yet been reviewed or categorized.

Comments

@ilias-t
Copy link

ilias-t commented Mar 5, 2025

Did you check the docs?

  • I have read all the NeMo-Guardrails docs

Is your feature request related to a problem? Please describe.

Add support for Clavata that provides content moderation capabilities to detect and filter inappropriate content.

Describe the solution you'd like

A user can customize the content moderation behavior by:

  1. Configuring different policies for input and output flows
  2. Specifying which labels must match within a policy
  3. Setting the label match logic to either "ALL" (all specified labels must match) or "ANY" (at least one label must match)

Example config

rails:
  config:
    clavata:
      # Only provide this if you've been told to by Clavata.ai
      server_endpoint: "https://some-alt-endpoint.com"
      policies:
        - alias: "Violence"
          id: "00000000-0000-0000-0000-000000000000"
        - alias: "Weapons"
          id: "00000000-0000-0000-0000-000000000000"
      input:
        policy: "Violence"
        # Optional: Specify labels to require specific matches
        labels:
          - "Violence"
          - "Weapons"
          - "Drugs"
        label_match_logic: ALL  # Can be "ALL" or "ANY"
      output:
        policy: "Weapons"
  input:
    flows:
      - clavata check input
  output:
    flows:
      - clavata check output

Details

  • server_endpoint: The Clavata API endpoint (only if provided by Clavata.ai)
  • policies: List of policy configurations with aliases and IDs
  • input/output: Flow-specific configurations
    • policy: The policy alias to use for this flow
    • labels: (Optional) List of specific labels to check for
    • label_match_logic: (Optional) "ALL" requires all specified labels to match, "ANY" requires at least one match

Describe alternatives you've considered

N/A

Additional context

No response

@ilias-t ilias-t added enhancement New feature or request status: needs triage New issues that have not yet been reviewed or categorized. labels Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request status: needs triage New issues that have not yet been reviewed or categorized.
Projects
None yet
Development

No branches or pull requests

1 participant