Skip to content

[FEATURE] Use ML Commons validation methods in Workflow parsing #1152

@dbwiddis

Description

@dbwiddis

Is your feature request related to a problem?

While we currently use the API spec to validate existence of fields and do basic format validation, ML Commons performs some additional validation on the content of those fields. For example, opensearch-project/ml-commons#3805 added an allowlist of valid characters for connector and model names and descriptions.

However, this validation occurs at run time (assuming a user has called ML Commons API). If we have an invalid character in a name or description, we won't find out until we provision the workflow.

What solution would you like?

  1. Expose validation methods on ML Commons in methods that Flow Framework can use to independently validate the inputs.
  2. Add validation of these fields during Workflow validation. This will probably be in the WorkflowStepFactory class as part of either the WorkflowStepValidator (perhaps a new field for a validation method?) or a required validate() method on the WorkflowStep interface, or similar.

What alternatives have you considered?

Status quo: runtime errors on provisioning

Do you have any additional context?

Note as reported in opensearch-project/ml-commons#3860 there may be classpath issues resolving transitive dependencies in ML Commons that are not present in Flow Framework.

CC: @dhrubo-os, @jngz-es, @ylwu for thoughts and a heads-up on potential future ML Commons PRs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions