diff --git a/docs/configuration.rst b/docs/configuration.rst index 5e18b1da1..83dfe2a29 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -292,7 +292,7 @@ And use it like: }, } ] - + The same fields for the :ref:`supported_data_types` as in the :ref:`schema_validation` are accepted. If ``schema`` is given, ``type`` is required. All the other keys can also be defined via :ref:`needs_schema_definitions` or in the file passed via @@ -2414,31 +2414,6 @@ Default value: ``None`` The JSON file should contain the same structure as :ref:`needs_schema_definitions`: -.. _`needs_schema_severity`: - -needs_schema_severity -~~~~~~~~~~~~~~~~~~~~~ - -.. versionadded:: 6.0.0 - -Minimum severity level for schema validation reporting. -Extra option and extra link schema errors are always reported as violations. -For each entry in :ref:`needs_schema_definitions` the severity can be defined by the user. - -Available severity levels: - -- ``info``: Informational message (default) -- ``warning``: Warning message -- ``violation``: Violation message - -The levels align with how `SHACL `__ defines severity levels. - -Default value: ``"info"`` - -.. code-block:: python - - needs_schema_severity = "warning" - .. _`needs_schema_debug_active`: needs_schema_debug_active diff --git a/docs/schema/index.rst b/docs/schema/index.rst index fbc5f3ebe..25314f241 100644 --- a/docs/schema/index.rst +++ b/docs/schema/index.rst @@ -575,6 +575,25 @@ This validates that: For most use cases, 4 levels of nesting is sufficient. If you need deeper validation chains, consider restructuring your validation logic or link relationships. +.. _`severity_levels`: + +Severity levels +--------------- + +The schema validation supports three severity levels for reporting violations: + +- ``info``: A non-critical constraint violation indicating an informative message. +- ``warning``: A non-critical constraint violation indicating a warning. +- ``violation`` : A constraint violation. + +The specific severity value has no impact on the validation, but it can influence how violations +are reported to the user. See :ref:`understanding_errors` for more details how Sphinx +console output can be configured and how to :ref:`suppress errors `. + +Schema violations are categorised with :ref:`message types `. Each +message type is assigned a default severity of ``violation``. +The default can be overwritten in :ref:`schema rules `. + .. _`schema_components`: Schema components reference @@ -699,24 +718,62 @@ in via ``allOf`` to be considered evaluated: In this case, a need with a ``priority`` property would still trigger an unevaluated properties error, even though ``priority`` is in the ``required`` list. -Severity levels -~~~~~~~~~~~~~~~ +.. _`rule_severity`: + +Rule severity +~~~~~~~~~~~~~ -Each schema can specify a severity level: +Each schema rule can specify one of the severity levels ``info``, ``warning``, or +``violation`` as described in :ref:`severity_levels`. If not given, ``violation`` is used as +a default. -- ``violation`` (default): Violation message -- ``warning``: Warning message -- ``info``: Informational message +The severity influences how validation failures are displayed in the console. .. code-block:: json { "severity": "warning", - "message": "Approval required due to high efforts" + "select": { + "properties": { "type": { "const": "feat" } } + }, + "validate": { + "local": { + "properties": { "id": { "pattern": "^FEAT_[a-zA-Z0-9_-]*$" } } + } + } + }, + { + "select": { + "properties": { "type": { "const": "spec" } } + }, + "validate": { + "local": { + "properties": { "id": { "pattern": "^SPEC_[a-zA-Z0-9_-]*$" } } + } + } } -The config :ref:`needs_schema_severity` can be used to define a minimum severity level for a -warning to be reported. +**Console output for above example:** + +.. code-block:: text + + WARNING: Need 'FEAT' has schema warnings: + Severity: warning + Field: id + Need path: FEAT + Schema path: [0] > local > properties > id > pattern + Schema message: 'FEAT' does not match '^FEAT_[a-zA-Z0-9_-]*$' [sn_schema_warning.local_fail] + + ERROR: Need 'SPEC' has schema violations: + Severity: violation + Field: id + Need path: SPEC + Schema path: [1] > local > properties > id > pattern + Schema message: 'SPEC' does not match '^SPEC_[a-zA-Z0-9_-]*$' [sn_schema_violation.local_fail] + +Note how the ``FEAT`` report has a severity ``warning`` while ``SPEC`` is reported as ``violation``. +This is because the first schema explicitly set the severity to ``warning``, while the second +schema used the default ``violation``. .. _`schema_reuse`: @@ -986,6 +1043,8 @@ Array fields store lists of homogeneous typed values: - If ``minContains`` is not given, it defaults to 1 when ``contains`` is present. - If ``maxContains`` is not given, there is no upper limit. +.. _`understanding_errors`: + Running validation and understanding errors ------------------------------------------- @@ -997,11 +1056,22 @@ Error messages and output Validation errors include detailed information: -- **Severity**: The severity level of the violation +- **Severity**: The :ref:`severity level ` of the violation - **Field**: The specific field that failed validation -- **Need path**: The ID of the need that failed or the link chain for network validation -- **Schema path**: The JSON path within the schema that was violated -- **User message**: Custom message from the needs_schema.schemas list +- **Need path**: The ID of the need that failed or the link chain for network validation. + Link chains are represented as in ``IMPL_SAFE > links > SPEC_SAFE > specifies > FEAT_SAFE``, so + need IDs and link types form the chain, separated by ``>``. +- **Schema path**: The JSON path within the schema that was violated. The path starts with an + identifier built from the :ref:`schema rule id ` and the 0-based index + of the rule in the list. Examples:: + + [0] > local > properties > id > pattern + spec[1] > local > properties > id > pattern + + The first example has no schema rule ID defined, so only the index is used. + +- **User message**: Optional user-friendly :ref:`message ` set in the + schema rules. - **Schema message**: Detailed technical validation message from the validator Example error output:: @@ -1035,6 +1105,119 @@ need and the specific link that caused the issue:: Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > links > links > local > allOf > 0 > properties > asil > enum Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] +.. _`severity_handling`: + +Severity handling +~~~~~~~~~~~~~~~~~ + +The :ref:`severity_levels` affect how Sphinx emits validation messages. +Here is how they map to Sphinx logging levels and console output. + +Schema severity **info**: + +- Mapped to Sphinx logging level ``warning`` +- Displayed as ``WARNING:`` in red color in console +- Logged with type ``sn_schema_info`` + +Schema severity **warning**: + +- Mapped to Sphinx logging level ``warning`` +- Displayed as ``WARNING:`` in red color in console +- Logged with type ``sn_schema_warning`` + +Schema severity **violation**: + +- Mapped to Sphinx logging level ``error`` +- Displayed as ``ERROR:`` in dark red color in console +- Logged with type ``sn_schema_violation`` + +Besides the ``type`` information mentioned above, log messages also receive a +``subtype`` described in :ref:`message_types`. + +The output format is ``WARNING|ERROR: [type.subtype]``. + +The combination of ``type`` and ``subtype`` can be used to selectively +:ref:`suppress validation messages `. + +.. _`message_types`: + +Message types +~~~~~~~~~~~~~ + +The schema validation is done on :ref:`local ` or +:ref:`network ` level. Accordingly, the following message subtypes are used +to indicate which validation failed. + +Local validation: + +- ``extra_option_fail``: Extra option schema validation failed + (local validation, defined via ``schema`` key in :ref:`needs_extra_options`) +- ``extra_link_fail``: Extra link schema validation failed + (defined via ``schema`` key in :ref:`needs_extra_links`) +- ``local_fail``: Need-local validation failed + (defined via ``validate.local`` in schemas of :ref:`needs_schema_definitions`) + +Network validation: + +- ``network_missing_target``: Outgoing link target cannot be resolved +- ``network_contains_too_few``: Not enough valid links (minContains failed) +- ``network_contains_too_many``: Too many valid links (maxContains failed) +- ``network_items_fail``: Linked need's item validation failed + +If not overridden in the :ref:`schema rules `, a default severity of ``violation`` +is used for all message types. + +.. _`suppress_validation_messages`: + +Suppressing validation messages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Schema validation messages can be selectively suppressed using Sphinx's +`suppress_warnings `__ +configuration option. + +**Examples:** + +Suppress all schema violation errors: + +.. code-block:: python + + suppress_warnings = ["sn_schema_violation"] + +Suppress all schema warnings: + +.. code-block:: python + + suppress_warnings = ["sn_schema_warning"] + +Suppress specific validation rule failures: + +.. code-block:: python + + # Suppress only local validation failures for violations + suppress_warnings = ["sn_schema_violation.local_fail"] + + # Suppress only network link count issues for warnings + suppress_warnings = ["sn_schema_warning.network_contains_too_few"] + +Suppress multiple specific types: + +.. code-block:: python + + suppress_warnings = [ + "sn_schema_violation.local_fail", + "sn_schema_warning.extra_option_fail", + "sn_schema_info", # Suppress all info messages + ] + +.. note:: + + Suppressing validation messages does not prevent the validation from occurring; it only + hides the output. All violations will still be recorded in the schema violation report + (see :ref:`schema_violation_json`). + +.. _`schema_violation_json`: + Schema violation report JSON file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/sphinx_needs/config.py b/sphinx_needs/config.py index aae15a286..f903efbf7 100644 --- a/sphinx_needs/config.py +++ b/sphinx_needs/config.py @@ -19,7 +19,6 @@ ExtraOptionSchemaTypes, ExtraOptionStringSchemaType, SchemasFileRootType, - SeverityEnum, ) if TYPE_CHECKING: @@ -412,12 +411,6 @@ def get_default(cls, name: str) -> Any: ) """Path to a JSON file to load the schemas from.""" - schema_severity: str = field( - default=SeverityEnum.info.name, - metadata={"rebuild": "env", "types": (str,)}, - ) - """Severity level for the schema validation reporting.""" - schema_debug_active: bool = field( default=False, metadata={"rebuild": "env", "types": (bool,)}, diff --git a/sphinx_needs/logging.py b/sphinx_needs/logging.py index 100fd9a6c..3239ff889 100644 --- a/sphinx_needs/logging.py +++ b/sphinx_needs/logging.py @@ -126,3 +126,32 @@ def log_warning( color=color, once=once, ) + + +def log_error( + logger: SphinxLoggerAdapter, + message: str, + subtype: WarningSubTypes, + /, + location: str | tuple[str | None, int | None] | Node | None, + *, + color: str | None = None, + once: bool = False, + type: str = "needs", +) -> None: + # Since sphinx in v7.3, sphinx will show warning types if `show_warning_types=True` is set, + # and in v8.0 this was made the default. + if version_info < (8,): + if subtype: + message += f" [{type}.{subtype}]" + else: + message += f" [{type}]" + + logger.error( + message, + type=type, + subtype=subtype, + location=location, + color=color, + once=once, + ) diff --git a/sphinx_needs/schema/config.py b/sphinx_needs/schema/config.py index cae0455c2..d210c54e8 100644 --- a/sphinx_needs/schema/config.py +++ b/sphinx_needs/schema/config.py @@ -366,6 +366,8 @@ class SeverityEnum(IntEnum): Default severity for each rule. User provided schemas can overwrite the severity of a rule. +The rules ``extra_option_fail`` and ``extra_link_fail`` cannot be changed by the user, +but they can be suppressed specifically using suppress_warnings config. """ diff --git a/sphinx_needs/schema/core.py b/sphinx_needs/schema/core.py index 97ac02ce6..5ae23e07e 100644 --- a/sphinx_needs/schema/core.py +++ b/sphinx_needs/schema/core.py @@ -23,7 +23,6 @@ OntologyWarning, ValidateNeedMessageType, ValidateNeedType, - filter_warnings_severity, save_debug_files, ) from sphinx_needs.schema.utils import get_properties_from_schema @@ -114,7 +113,7 @@ def validate_need( need_path=[need["id"]], ) save_debug_files(config, new_warnings_options) - all_warnings.extend(filter_warnings_severity(config, new_warnings_options)) + all_warnings.extend(new_warnings_options) if _extra_link_schemas.get("properties"): new_warnings_links = get_ontology_warnings( @@ -127,7 +126,7 @@ def validate_need( need_path=[need["id"]], ) save_debug_files(config, new_warnings_links) - all_warnings.extend(filter_warnings_severity(config, new_warnings_links)) + all_warnings.extend(new_warnings_links) for type_schema in type_schemas: # maintain state for nested network validation @@ -231,10 +230,7 @@ def recurse_validate_schemas( user_severity=severity if recurse_level == 0 else None, ) save_debug_files(config, warnings_local) - warnings_local_filtered = filter_warnings_severity( - config, warnings_local, severity - ) - warnings.extend(warnings_local_filtered) + warnings.extend(warnings_local) if any_not_of_rule(warnings_local, rule_success): success = False if "network" in schema: @@ -369,9 +365,7 @@ def recurse_validate_schemas( if recurse_level == 0 and user_message is not None: # user message only added to the root validation warning["user_message"] = user_message - warnings.extend( - filter_warnings_severity(config, items_nok_warnings, severity) - ) + warnings.extend(items_nok_warnings) # Check contains validation results contains_success = True @@ -433,10 +427,7 @@ def recurse_validate_schemas( contains_warnings[-1]["user_message"] = user_message contains_success = False - filtered_contains_warnings = filter_warnings_severity( - config, contains_warnings, severity - ) - warnings.extend(filtered_contains_warnings) + warnings.extend(contains_warnings) # Overall success requires both items and minmax validation to pass if not (items_success and contains_success): @@ -629,8 +620,9 @@ def get_ontology_warnings( "final_schema": validation_report["final_schema"], "schema_path": [*schema_path, *msg["schema_path"]], "need_path": need_path, - "field": msg["field"], } + if msg.get("field"): + warning["field"] = msg["field"] if user_message is not None: warning["user_message"] = user_message warnings.append(warning) diff --git a/sphinx_needs/schema/process.py b/sphinx_needs/schema/process.py index fbf6f9085..340b24cb3 100644 --- a/sphinx_needs/schema/process.py +++ b/sphinx_needs/schema/process.py @@ -8,7 +8,7 @@ from sphinx_needs.api import get_needs_view from sphinx_needs.config import NeedsSphinxConfig from sphinx_needs.data import SphinxNeedsData -from sphinx_needs.logging import log_warning +from sphinx_needs.logging import log_error, log_warning from sphinx_needs.needsfile import generate_needs_schema from sphinx_needs.schema.config import SchemasRootType from sphinx_needs.schema.core import ( @@ -95,8 +95,12 @@ def process_schemas(app: Sphinx, builder: Builder) -> None: type=warning["type"], ) elif warning["log_lvl"] == "error": - logger.error( - warning["message"], type=warning["type"], subtype=warning["subtype"] + log_error( + logger, + warning["message"], + warning["subtype"], # type: ignore[arg-type] + None, + type=warning["type"], ) duration = end_time - start_time diff --git a/sphinx_needs/schema/reporting.py b/sphinx_needs/schema/reporting.py index e6dfdc5c2..92bc2c37f 100644 --- a/sphinx_needs/schema/reporting.py +++ b/sphinx_needs/schema/reporting.py @@ -6,10 +6,11 @@ from pathlib import Path from typing import TYPE_CHECKING, Any, Literal, TypedDict +from sphinx.errors import SphinxError + from sphinx_needs.config import NeedsSphinxConfig from sphinx_needs.need_item import NeedItem from sphinx_needs.schema.config import ( - MAP_RULE_DEFAULT_SEVERITY, MessageRuleEnum, NeedFieldsSchemaWithVersionType, SeverityEnum, @@ -72,45 +73,6 @@ class ValidateNeedType(TypedDict): """Separator between nested parts of debug output file names.""" -def filter_warnings_severity( - config: NeedsSphinxConfig, - warnings: list[OntologyWarning], - schema_root_severity: SeverityEnum | None = None, -) -> list[OntologyWarning]: - """ - Filter warnings by severity. - - There are multiple severity sources: - - needs_config.schema_severity: the minimum severity for warnings to be reported - - needs_config.schema_definitions: contains a severity field that determines the reported rule severity - - schemas.config.MAP_RULE_DEFAULT_SEVERITY: default if the schemas severities field is not set - - Precedence for reporting: - - if the rule has a default severity of none, it is ignored; those are never of interest - as reporting target, only for debugging purposes - - if the schema root severity is set, it overrides the rule severity - - if the schema root severity is not set, the rule default severity is used - """ - min_severity_for_report = SeverityEnum[config.schema_severity] - filtered_warnings = [] - for warning in warnings: - rule_default_severity = MAP_RULE_DEFAULT_SEVERITY[warning["rule"]] - if rule_default_severity is SeverityEnum.none: - # rule is not of interest for final reporting (e.g. unselected needs) - # it might be logged for schema debugging however - continue - # the warning severity is overriden by the schema root severity - # if it is unset, the rule mapping MAP_RULE_DEFAULT_SEVERITY is used - warning_severity = ( - schema_root_severity - if schema_root_severity is not None - else warning["severity"] - ) - if warning_severity >= min_severity_for_report: - filtered_warnings.append(warning) - return filtered_warnings - - def save_debug_files( config: NeedsSphinxConfig, warnings: list[OntologyWarning] ) -> None: @@ -192,9 +154,17 @@ class WarningDetails(TypedDict): user_msg: NotRequired[str] +class JSONFormattedWarningChild(TypedDict): + """JSON Formatted child warning (nested within parent warnings).""" + + subtype: str + details: WarningDetails + children: list[ChildWarning] + + class ChildWarning(TypedDict): need_id: str - warning: JSONFormattedWarning + warning: JSONFormattedWarningChild class JSONFormattedWarning(TypedDict): @@ -226,33 +196,50 @@ def get_formatted_warnings_recurse( formatted_warnings: list[FormattedWarning] = [] warning_msg = get_warning_msg(level, warning) has_title = level == 0 + log_lvl: Literal["warning", "error"] + + # Map severity to logger level and type if warning["severity"] == SeverityEnum.config_error: title = ( f"Need '{warning['need']['id']}' has configuration errors:" if has_title else "" ) - formatted_warning = FormattedWarning( - log_lvl="error", - type="sn_schema", - subtype=warning["rule"].value, - message=f"{title}{warning_msg}", - ) - else: + log_lvl = "error" + type_str = "sn_schema" + elif warning["severity"] == SeverityEnum.violation: title = ( - f"Need '{warning['need']['id']}' has validation errors:" + f"Need '{warning['need']['id']}' has schema violations:" if has_title else "" ) - formatted_warning = FormattedWarning( - log_lvl="warning", - type="sn_schema", - subtype=warning["rule"].value, - message=f"{title}{warning_msg}", + log_lvl = "error" + type_str = "sn_schema_violation" + elif warning["severity"] == SeverityEnum.warning: + title = ( + f"Need '{warning['need']['id']}' has schema warnings:" if has_title else "" ) + log_lvl = "warning" + type_str = "sn_schema_warning" + elif warning["severity"] == SeverityEnum.info: + title = f"Need '{warning['need']['id']}' has schema infos:" if has_title else "" + log_lvl = "warning" + type_str = "sn_schema_info" + else: + # SeverityEnum.none is filtered out earlier + raise SphinxError("Unexpected severity 'none' in console formatting.") + + formatted_warning = FormattedWarning( + log_lvl=log_lvl, + type=type_str, + subtype=warning["rule"].value, + message=f"{title}{warning_msg}", + ) formatted_warnings.append(formatted_warning) if "children" in warning: for child_warning in warning["children"]: + if child_warning["severity"] == SeverityEnum.none: + continue formatted_child_warnings = get_formatted_warnings_recurse( warning=child_warning, level=level + 1 ) @@ -262,7 +249,6 @@ def get_formatted_warnings_recurse( f"\n\n{indent}Details for {child_warning['need']['id']}" + formatted_child_warning["message"] ) - # formatted_warnings.extend(formatted_child_warnings) return formatted_warnings @@ -272,80 +258,21 @@ def get_formatted_warnings( """ Pretty format warnings from the ontology validation. + Warnings with severity 'none' are filtered out as they are only for debugging. + :returns: tuple [log level, type, sub-type, message] """ formatted_warnings: list[FormattedWarning] = [] for warnings in need_2_warnings.values(): for warning in warnings: + # Skip warnings with severity none (debug only) + if warning["severity"] == SeverityEnum.none: + continue new_warnings = get_formatted_warnings_recurse(warning=warning, level=0) formatted_warnings.extend(new_warnings) return formatted_warnings -def get_json_formatted_warnings_recurse( - warning: OntologyWarning, level: int -) -> list[JSONFormattedWarning]: - """ - Recursively format warning details. - - :param warning: The OntologyWarning to format. - :param level: The OntologyWarning level. - :return: A list of JSONFormattedWarning objects. - """ - formatted_warnings: list[JSONFormattedWarning] = [] - warning_details = get_json_warning_details(level, warning) - if warning["severity"] == SeverityEnum.config_error: - formatted_warning = JSONFormattedWarning( - log_lvl="error", - type="sn_schema", - subtype=warning["rule"].value, - details=warning_details, - children=[], - ) - else: - formatted_warning = JSONFormattedWarning( - log_lvl="warning", - type="sn_schema", - subtype=warning["rule"].value, - details=warning_details, - children=[], - ) - formatted_warnings.append(formatted_warning) - if "children" in warning: - for child_warning in warning["children"]: - formatted_child_warnings = get_json_formatted_warnings_recurse( - warning=child_warning, level=level + 1 - ) - for formatted_child_warning in formatted_child_warnings: - formatted_warning["children"].append( - { - "need_id": child_warning["need"]["id"], - "warning": formatted_child_warning, - } - ) - return formatted_warnings - - -def get_json_formatted_warnings( - need_2_warnings: dict[str, list[OntologyWarning]], -) -> dict[str, list[JSONFormattedWarning]]: - """ - JSON formatted warnings from the ontology validation for each need. - - :param need_2_warnings: A dictionary mapping need IDs to their warnings. - :return: A dictionary with list of JSONFormattedWarning objects. - """ - need_formatted_warnings: dict[str, list[JSONFormattedWarning]] = {} - for need_id, warnings in need_2_warnings.items(): - formatted_warnings: list[JSONFormattedWarning] = [] - for warning in warnings: - new_warnings = get_json_formatted_warnings_recurse(warning=warning, level=0) - formatted_warnings.extend(new_warnings) - - need_formatted_warnings[need_id] = formatted_warnings - return need_formatted_warnings - - def get_warning_msg(base_lvl: int, warning: OntologyWarning) -> str: """Craft a properly indented warning message.""" warning_msg = "" @@ -386,6 +313,108 @@ def nl_indent(level: int) -> str: return warning_msg +def _get_json_formatted_child_recurse( + warning: OntologyWarning, level: int +) -> JSONFormattedWarningChild: + """Helper function to format nested child warnings without log_lvl and type.""" + warning_details = get_json_warning_details(level, warning) + + formatted_child = JSONFormattedWarningChild( + subtype=warning["rule"].value, + details=warning_details, + children=[], + ) + if "children" in warning: + for child_warning in warning["children"]: + formatted_child["children"].append( + { + "need_id": child_warning["need"]["id"], + "warning": _get_json_formatted_child_recurse( + child_warning, level=level + 1 + ), + } + ) + return formatted_child + + +def get_json_formatted_warnings_recurse( + warning: OntologyWarning, level: int +) -> list[JSONFormattedWarning]: + """ + Recursively format warning details. + + :param warning: The OntologyWarning to format. + :param level: The OntologyWarning level (0 for root). + :return: A list of JSONFormattedWarning objects. + """ + warning_details = get_json_warning_details(level, warning) + + log_lvl: Literal["warning", "error"] + + # Root level warning includes log_lvl and type + # Map severity to logger level and type + if warning["severity"] == SeverityEnum.config_error: + log_lvl = "error" + type_str = "sn_schema" + elif warning["severity"] == SeverityEnum.violation: + log_lvl = "error" + type_str = "sn_schema_violation" + elif warning["severity"] == SeverityEnum.warning: + log_lvl = "warning" + type_str = "sn_schema_warning" + elif warning["severity"] == SeverityEnum.info: + log_lvl = "warning" + type_str = "sn_schema_info" + else: + # SeverityEnum.none is filtered out earlier + raise SphinxError("Unexpected severity 'none' in JSON formatting.") + + formatted_warning = JSONFormattedWarning( + log_lvl=log_lvl, + type=type_str, + subtype=warning["rule"].value, + details=warning_details, + children=[], + ) + if "children" in warning: + for child_warning in warning["children"]: + formatted_warning["children"].append( + { + "need_id": child_warning["need"]["id"], + "warning": _get_json_formatted_child_recurse( + child_warning, level=level + 1 + ), + } + ) + return [formatted_warning] + + +def get_json_formatted_warnings( + need_2_warnings: dict[str, list[OntologyWarning]], +) -> dict[str, list[JSONFormattedWarning]]: + """ + JSON formatted warnings from the ontology validation for each need. + + Warnings with severity 'none' are filtered out as they are only for debugging. + + :param need_2_warnings: A dictionary mapping need IDs to their warnings. + :return: A dictionary with list of JSONFormattedWarning objects. + """ + need_formatted_warnings: dict[str, list[JSONFormattedWarning]] = {} + for need_id, warnings in need_2_warnings.items(): + formatted_warnings: list[JSONFormattedWarning] = [] + for warning in warnings: + # Skip warnings with severity none (debug only) + if warning["severity"] == SeverityEnum.none: + continue + new_warnings = get_json_formatted_warnings_recurse(warning=warning, level=0) + formatted_warnings.extend(new_warnings) + + if formatted_warnings: + need_formatted_warnings[need_id] = formatted_warnings + return need_formatted_warnings + + def get_json_warning_details(base_lvl: int, warning: OntologyWarning) -> WarningDetails: """Get JSON formatted warning details.""" warning_details: WarningDetails = {} diff --git a/tests/doc_test/doc_schema_benchmark/ubproject.toml b/tests/doc_test/doc_schema_benchmark/ubproject.toml index 116b18dd7..a96e538d6 100644 --- a/tests/doc_test/doc_schema_benchmark/ubproject.toml +++ b/tests/doc_test/doc_schema_benchmark/ubproject.toml @@ -4,7 +4,6 @@ id_required = true a = ["a", 3, { "a" = "b" }] id_regex = "^[A-Z0-9_]{3,}" -schema_severity = 'info' [[needs.extra_options]] name = "efforts" diff --git a/tests/doc_test/doc_schema_e2e/ubproject.toml b/tests/doc_test/doc_schema_e2e/ubproject.toml index cb337060c..4d21be01a 100644 --- a/tests/doc_test/doc_schema_e2e/ubproject.toml +++ b/tests/doc_test/doc_schema_e2e/ubproject.toml @@ -3,7 +3,6 @@ [needs] id_required = true id_regex = "^[A-Z0-9_]{3,}" -schema_severity = 'warning' build_json = true schema_definitions_from_json = "schemas.json" schema_debug_active = true diff --git a/tests/doc_test/doc_schema_example/ubproject.toml b/tests/doc_test/doc_schema_example/ubproject.toml index 5bd212920..9de6dcfdc 100644 --- a/tests/doc_test/doc_schema_example/ubproject.toml +++ b/tests/doc_test/doc_schema_example/ubproject.toml @@ -3,7 +3,6 @@ [needs] id_required = true id_regex = "^[A-Z0-9_]{3,}" -schema_severity = 'warning' build_json = true schema_definitions_from_json = "schemas.json" schema_debug_active = true diff --git a/tests/schema/__snapshots__/test_schema.ambr b/tests/schema/__snapshots__/test_schema.ambr index c51b1287a..335521b77 100644 --- a/tests/schema/__snapshots__/test_schema.ambr +++ b/tests/schema/__snapshots__/test_schema.ambr @@ -169,39 +169,42 @@ # --- # name: test_schema_e2e[test_app0] ''' - WARNING: Need 'FEAt' has validation errors: + WARNING: Need 'FEAt' has schema warnings: Severity: warning Field: id Need path: FEAt Schema path: [0] > local > properties > id > pattern User message: id must be uppercase with numbers and underscores - Schema message: 'FEAt' does not match '^[A-Z0-9_]+$' [sn_schema.local_fail] - WARNING: Need 'SPEC_MISSING_APPROVAL' has validation errors: + Schema message: 'FEAt' does not match '^[A-Z0-9_]+$' [sn_schema_warning.local_fail] + ERROR: Need 'SPEC_MISSING_APPROVAL' has schema violations: Severity: violation - Field: Need path: SPEC_MISSING_APPROVAL Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('asil', 'priority' were unexpected) [sn_schema.local_fail] - WARNING: Need 'SPEC_MISSING_APPROVAL' has validation errors: + Schema message: Unevaluated properties are not allowed ('asil', 'priority' were unexpected) [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_MISSING_APPROVAL' has schema violations: Severity: violation - Field: Need path: SPEC_MISSING_APPROVAL Schema path: spec-approved-required[2] > local > required User message: Approval required due to high efforts - Schema message: 'approved' is a required property [sn_schema.local_fail] - WARNING: Need 'SPEC' has validation errors: + Schema message: 'approved' is a required property [sn_schema_violation.local_fail] + WARNING: Need 'SPEC_MISSING_APPROVAL' has schema infos: + Severity: info + Need path: SPEC_MISSING_APPROVAL + Schema path: spec-approved-not-given[3] > local > required + User message: Approval not given + Schema message: 'approved' is a required property [sn_schema_info.local_fail] + ERROR: Need 'SPEC' has schema violations: Severity: violation Field: id Need path: SPEC Schema path: spec[1] > local > properties > id > pattern - Schema message: 'SPEC' does not match '^SPEC_[a-zA-Z0-9_-]*$' [sn_schema.local_fail] - WARNING: Need 'SPEC_SAFE_UNSAFE_FEAT' has validation errors: + Schema message: 'SPEC' does not match '^SPEC_[a-zA-Z0-9_-]*$' [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_SAFE_UNSAFE_FEAT' has schema violations: Severity: violation - Field: Need path: SPEC_SAFE_UNSAFE_FEAT Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema.local_fail] - WARNING: Need 'SPEC_SAFE_UNSAFE_FEAT' has validation errors: + Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_SAFE_UNSAFE_FEAT' has schema violations: Severity: violation Need path: SPEC_SAFE_UNSAFE_FEAT > links Schema path: safe-spec-[links]->safe-feat[4] > validate > network > links > contains @@ -211,19 +214,17 @@ Field: asil Need path: SPEC_SAFE_UNSAFE_FEAT > links > FEAT Schema path: safe-spec-[links]->safe-feat[4] > validate > network > links > contains > local > allOf > 0 > properties > asil > enum - Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] - WARNING: Need 'SPEC_SAFE_ADD_UNSAFE_FEAT' has validation errors: + Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema_violation.network_contains_too_few] + ERROR: Need 'SPEC_SAFE_ADD_UNSAFE_FEAT' has schema violations: Severity: violation - Field: Need path: SPEC_SAFE_ADD_UNSAFE_FEAT Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema.local_fail] - WARNING: Need 'SPEC_SAFE' has validation errors: + Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_SAFE' has schema violations: Severity: violation - Field: Need path: SPEC_SAFE Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema.local_fail] + Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema_violation.local_fail] ''' # --- @@ -2240,349 +2241,1053 @@ # name: test_schemas[schema/fixtures/extra_links-contains] '' # --- +# name: test_schemas[schema/fixtures/extra_links-contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > contains - Schema message: ['SPEC_1'] does not contain items matching the given schema [sn_schema.extra_link_fail] + Schema message: ['SPEC_1'] does not contain items matching the given schema [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > contains', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] does not contain items matching the given schema", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-inject_array] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > minItems - Schema message: ['SPEC_1'] is too short [sn_schema.extra_link_fail] + Schema message: ['SPEC_1'] is too short [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-inject_array].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > minItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] is too short", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-items_pattern] '' # --- +# name: test_schemas[schema/fixtures/extra_links-items_pattern].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-items_pattern_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links.0 Need path: IMPL_1 Schema path: extra_links > schema > properties > links > items > pattern - Schema message: 'SPEC_1' does not match '^[A-Z0-9_]{10,}' [sn_schema.extra_link_fail] + Schema message: 'SPEC_1' does not match '^[A-Z0-9_]{10,}' [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-items_pattern_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links.0', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > items > pattern', + 'severity': 'violation', + 'validation_msg': "'SPEC_1' does not match '^[A-Z0-9_]{10,}'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_contains] '' # --- +# name: test_schemas[schema/fixtures/extra_links-max_contains].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > contains - Schema message: Too many items match the given schema (expected at most 1) [sn_schema.extra_link_fail] + Schema message: Too many items match the given schema (expected at most 1) [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-max_contains_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > contains', + 'severity': 'violation', + 'validation_msg': 'Too many items match the given schema (expected at most 1)', + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_items] '' # --- +# name: test_schemas[schema/fixtures/extra_links-max_items].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > maxItems - Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema.extra_link_fail] + Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-max_items_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > maxItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1', 'SPEC_2'] is too long", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_contains] '' # --- +# name: test_schemas[schema/fixtures/extra_links-min_contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > contains - Schema message: Too few items match the given schema (expected at least 2 but only 1 matched) [sn_schema.extra_link_fail] + Schema message: Too few items match the given schema (expected at least 2 but only 1 matched) [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-min_contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > contains', + 'severity': 'violation', + 'validation_msg': 'Too few items match the given schema (expected at least 2 but only 1 matched)', + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_items] '' # --- +# name: test_schemas[schema/fixtures/extra_links-min_items].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > minItems - Schema message: ['SPEC_1'] is too short [sn_schema.extra_link_fail] + Schema message: ['SPEC_1'] is too short [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-min_items_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > minItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] is too short", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-auto_inject_type] '' # --- +# name: test_schemas[schema/fixtures/extra_options-auto_inject_type].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-auto_inject_type_wrong_const] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'QM' was expected [sn_schema.local_fail] + Schema message: 'QM' was expected [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-auto_inject_type_wrong_const].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'QM' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean] '' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean].1 + dict({ + 'validated_needs_count': 14, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean_from_string_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'approved' is invalid: Cannot convert 'not-a-boolean' to boolean [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean_from_string_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_integer] '' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_integer].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_float_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'efforts' is invalid: Cannot convert '1.2' to integer [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_float_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_string_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'efforts' is invalid: Cannot convert 'QM' to integer [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_string_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_number] '' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_number].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_number_from_string_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'efforts' is invalid: Cannot convert 'QM' to float [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_number_from_string_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-const] '' # --- +# name: test_schemas[schema/fixtures/extra_options-const].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-const_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'QM' was expected [sn_schema.local_fail] + Schema message: 'QM' was expected [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-const_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'QM' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-enum] '' # --- +# name: test_schemas[schema/fixtures/extra_options-enum].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-enum_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > enum - Schema message: 'E' is not one of ['QM', 'A', 'B', 'C', 'D'] [sn_schema.local_fail] + Schema message: 'E' is not one of ['QM', 'A', 'B', 'C', 'D'] [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-enum_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > enum', + 'severity': 'violation', + 'validation_msg': "'E' is not one of ['QM', 'A', 'B', 'C', 'D']", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-integer_multiple_of] '' # --- +# name: test_schemas[schema/fixtures/extra_options-integer_multiple_of].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-integer_multiple_of_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: efforts Need path: IMPL_1 Schema path: extra_options > schema > properties > efforts > multipleOf - Schema message: 8 is not a multiple of 3 [sn_schema.extra_option_fail] + Schema message: 8 is not a multiple of 3 [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-integer_multiple_of_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'efforts', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > efforts > multipleOf', + 'severity': 'violation', + 'validation_msg': '8 is not a multiple of 3', + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-no_schema] '' # --- +# name: test_schemas[schema/fixtures/extra_options-no_schema].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-number_multiple_of] '' # --- +# name: test_schemas[schema/fixtures/extra_options-number_multiple_of].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-number_multiple_of_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: efforts Need path: IMPL_1 Schema path: extra_options > schema > properties > efforts > multipleOf - Schema message: 5.0 is not a multiple of 3.3 [sn_schema.extra_option_fail] + Schema message: 5.0 is not a multiple of 3.3 [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-number_multiple_of_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'efforts', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > efforts > multipleOf', + 'severity': 'violation', + 'validation_msg': '5.0 is not a multiple of 3.3', + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required] '' # --- +# name: test_schemas[schema/fixtures/extra_options-required].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_below_threshold] '' # --- +# name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_below_threshold].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_missing] '' # --- +# name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_missing].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_over_threshold] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > required User message: Required due to high efforts - Schema message: 'asil' is a required property [sn_schema.local_fail] + Schema message: 'asil' is a required property [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_over_threshold].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > required', + 'severity': 'violation', + 'user_msg': 'Required due to high efforts', + 'validation_msg': "'asil' is a required property", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_missing] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > required - Schema message: 'asil' is a required property [sn_schema.local_fail] + Schema message: 'asil' is a required property [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-required_missing].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > required', + 'severity': 'violation', + 'validation_msg': "'asil' is a required property", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-set_type_string] '' # --- +# name: test_schemas[schema/fixtures/extra_options-set_type_string].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: start_date Need path: IMPL_1 Schema path: extra_options > schema > properties > start_date > format - Schema message: 'not-a-date' is not a 'date' [sn_schema.extra_option_fail] + Schema message: 'not-a-date' is not a 'date' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'start_date', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > start_date > format', + 'severity': 'violation', + 'validation_msg': "'not-a-date' is not a 'date'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date_time] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date_time].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date_time_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: start_date Need path: IMPL_1 Schema path: extra_options > schema > properties > start_date > format - Schema message: '2025-07-1099:99:99Z' is not a 'date-time' [sn_schema.extra_option_fail] + Schema message: '2025-07-1099:99:99Z' is not a 'date-time' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date_time_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'start_date', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > start_date > format', + 'severity': 'violation', + 'validation_msg': "'2025-07-1099:99:99Z' is not a 'date-time'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_duration] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_duration].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_duration_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: _duration Need path: IMPL_1 Schema path: extra_options > schema > properties > _duration > format - Schema message: 'P1Q2Q10DT2H30M' is not a 'duration' [sn_schema.extra_option_fail] + Schema message: 'P1Q2Q10DT2H30M' is not a 'duration' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_duration_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': '_duration', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > _duration > format', + 'severity': 'violation', + 'validation_msg': "'P1Q2Q10DT2H30M' is not a 'duration'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_email] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_email].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_email_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: email Need path: IMPL_1 Schema path: extra_options > schema > properties > email > format - Schema message: 'not-a-mail' is not a 'email' [sn_schema.extra_option_fail] + Schema message: 'not-a-mail' is not a 'email' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_email_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'email', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > email > format', + 'severity': 'violation', + 'validation_msg': "'not-a-mail' is not a 'email'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_time] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_time].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_time_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: time Need path: IMPL_1 Schema path: extra_options > schema > properties > time > format - Schema message: '26:12:13' is not a 'time' [sn_schema.extra_option_fail] + Schema message: '26:12:13' is not a 'time' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_time_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'time', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > time > format', + 'severity': 'violation', + 'validation_msg': "'26:12:13' is not a 'time'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uri] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uri].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uri_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: uri Need path: IMPL_1 Schema path: extra_options > schema > properties > uri > format - Schema message: 'examplecom' is not a 'uri' [sn_schema.extra_option_fail] + Schema message: 'examplecom' is not a 'uri' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uri_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'uri', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > uri > format', + 'severity': 'violation', + 'validation_msg': "'examplecom' is not a 'uri'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uuid] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uuid].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uuid_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: uuid Need path: IMPL_1 Schema path: extra_options > schema > properties > uuid > format - Schema message: 'deadbeef-deadbeef-deadbeef-deadbeef' is not a 'uuid' [sn_schema.extra_option_fail] + Schema message: 'deadbeef-deadbeef-deadbeef-deadbeef' is not a 'uuid' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uuid_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'uuid', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > uuid > format', + 'severity': 'violation', + 'validation_msg': "'deadbeef-deadbeef-deadbeef-deadbeef' is not a 'uuid'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_type] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_type].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-wrong_type] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'asil' is invalid: Cannot convert 'QM' to integer [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-wrong_type].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-link_chain_hop_1_min_contains_error] ''' - WARNING: Need 'IMPL_SAFE' has validation errors: + ERROR: Need 'IMPL_SAFE' has schema violations: Severity: violation Need path: IMPL_SAFE > links Schema path: safe-impl-[links]->safe-spec[0] > validate > network > links > contains @@ -2592,13 +3297,49 @@ Field: asil Need path: IMPL_SAFE > links > SPEC_UNSAFE Schema path: safe-impl-[links]->safe-spec[0] > validate > network > links > contains > local > allOf > 0 > properties > asil > enum - Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] + Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-link_chain_hop_1_min_contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_SAFE': list([ + dict({ + 'children': list([ + dict({ + 'need_id': 'SPEC_UNSAFE', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_SAFE > links > SPEC_UNSAFE', + 'schema_path': 'safe-impl-[links]->safe-spec[0] > validate > network > links > contains > local > allOf > 0 > properties > asil > enum', + 'validation_msg': "'QM' is not one of ['A', 'B', 'C', 'D']", + }), + 'subtype': 'network_local_fail', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links', + 'schema_path': 'safe-impl-[links]->safe-spec[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: SPEC_UNSAFE", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-link_chain_hop_2_min_contains_error] ''' - WARNING: Need 'IMPL_SAFE' has validation errors: + ERROR: Need 'IMPL_SAFE' has schema violations: Severity: violation Need path: IMPL_SAFE > links Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains @@ -2614,93 +3355,331 @@ Field: asil Need path: IMPL_SAFE > links > SPEC_SAFE > links > REQ_UNSAFE Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > validate > network > links > contains > local > allOf > 0 > properties > asil > enum - Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] + Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-link_chain_hop_2_min_contains_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_SAFE': list([ + dict({ + 'children': list([ + dict({ + 'need_id': 'SPEC_SAFE', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links > SPEC_SAFE', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > local', + }), + 'subtype': 'network_local_success', + }), + }), + dict({ + 'need_id': 'SPEC_SAFE', + 'warning': dict({ + 'children': list([ + dict({ + 'need_id': 'REQ_UNSAFE', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_SAFE > links > SPEC_SAFE > links > REQ_UNSAFE', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > validate > network > links > contains > local > allOf > 0 > properties > asil > enum', + 'validation_msg': "'QM' is not one of ['A', 'B', 'C', 'D']", + }), + 'subtype': 'network_local_fail', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links > SPEC_SAFE > links', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > validate > network > links > contains', + 'user_msg': 'Safe impl links to safe spec links to safe req', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: REQ_UNSAFE", + }), + 'subtype': 'network_contains_too_few', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: SPEC_SAFE", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-link_chain_w_refs] '' # --- +# name: test_schemas[schema/fixtures/network-link_chain_w_refs].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_max_items] '' # --- +# name: test_schemas[schema/fixtures/network-local_max_items].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_max_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: [0] > local > properties > links > maxItems - Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema.local_fail] + Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/network-local_max_items_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > links > maxItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1', 'SPEC_2'] is too long", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_min_items] '' # --- +# name: test_schemas[schema/fixtures/network-local_min_items].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_min_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: [0] > local > properties > links > minItems - Schema message: ['SPEC_1'] is too short [sn_schema.local_fail] + Schema message: ['SPEC_1'] is too short [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/network-local_min_items_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > links > minItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] is too short", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-max_contains] '' # --- +# name: test_schemas[schema/fixtures/network-max_contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-max_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Need path: IMPL_1 > links Schema path: [0] > validate > network > links > contains - Schema message: Too many valid links of type 'links' (2 > 1) / ok: SPEC_1, SPEC_2 [sn_schema.network_contains_too_many] + Schema message: Too many valid links of type 'links' (2 > 1) / ok: SPEC_1, SPEC_2 [sn_schema_violation.network_contains_too_many] ''' # --- +# name: test_schemas[schema/fixtures/network-max_contains_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links', + 'schema_path': '[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too many valid links of type 'links' (2 > 1) / ok: SPEC_1, SPEC_2", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_many', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-max_network_levels] ''' - WARNING: Need 'SPEC_1' has validation errors: + ERROR: Need 'SPEC_1' has schema violations: Severity: violation Need path: IMPL_1 > links > SPEC_5 > links > SPEC_4 > links > SPEC_3 > links > SPEC_2 > links > SPEC_1 Schema path: [0] > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items - Schema message: Maximum network validation recursion level 4 reached. [sn_schema.network_max_nest_level] + Schema message: Maximum network validation recursion level 4 reached. [sn_schema_violation.network_max_nest_level] ''' # --- +# name: test_schemas[schema/fixtures/network-max_network_levels].1 + dict({ + 'validated_needs_count': 6, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links > SPEC_5 > links > SPEC_4 > links > SPEC_3 > links > SPEC_2 > links > SPEC_1', + 'schema_path': '[0] > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items', + 'severity': 'violation', + 'validation_msg': 'Maximum network validation recursion level 4 reached.', + }), + 'log_lvl': 'error', + 'subtype': 'network_max_nest_level', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-min_contains] '' # --- +# name: test_schemas[schema/fixtures/network-min_contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-min_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Need path: IMPL_1 > links Schema path: [0] > validate > network > links > contains - Schema message: Too few valid links of type 'links' (1 < 2) / ok: SPEC_1 [sn_schema.network_contains_too_few] + Schema message: Too few valid links of type 'links' (1 < 2) / ok: SPEC_1 [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-min_contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links', + 'schema_path': '[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (1 < 2) / ok: SPEC_1", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-no_error_empty_links] ''' - WARNING: Need 'IMPL_SAFE' has validation errors: + ERROR: Need 'IMPL_SAFE' has schema violations: Severity: violation Need path: IMPL_SAFE > links Schema path: impl-[links]->spec[0] > validate > network > links > contains - Schema message: Too few valid links of type 'links' (0 < 1) [sn_schema.network_contains_too_few] + Schema message: Too few valid links of type 'links' (0 < 1) [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-no_error_empty_links].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_SAFE': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links', + 'schema_path': 'impl-[links]->spec[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1)", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-schemas_in_conf] '' # --- +# name: test_schemas[schema/fixtures/network-schemas_in_conf].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-schemas_in_conf_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Need path: IMPL_1 > links Schema path: [0] > validate > network > links > contains @@ -2710,90 +3689,339 @@ Field: type Need path: IMPL_1 > links > SPEC_1 Schema path: [0] > validate > network > links > contains > local > properties > type > const - Schema message: 'req' was expected [sn_schema.network_contains_too_few] + Schema message: 'req' was expected [sn_schema_violation.network_contains_too_few] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_unset_is_reported] +# name: test_schemas[schema/fixtures/network-schemas_in_conf_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + dict({ + 'need_id': 'SPEC_1', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'type', + 'need_path': 'IMPL_1 > links > SPEC_1', + 'schema_path': '[0] > validate > network > links > contains > local > properties > type > const', + 'validation_msg': "'req' was expected", + }), + 'subtype': 'network_local_fail', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links', + 'schema_path': '[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: SPEC_1", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity] ''' - WARNING: Need 'IMPL_1' has validation errors: + WARNING: Need 'IMPL_1' has schema infos: Severity: info Field: asil Need path: IMPL_1 - Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema path: [1] > local > properties > asil > const + Schema message: 'B' was expected [sn_schema_info.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_warning_not_reported] - '' +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[1] > local > properties > asil > const', + 'severity': 'info', + 'validation_msg': "'B' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_info', + }), + ]), + }), + }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_unset_min_unset_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity_and_rule] + ''' + WARNING: Need 'IMPL_1' has schema infos: + Severity: info + Field: asil + Need path: IMPL_1 + Schema path: [1] > local > properties > asil > const + Schema message: 'B' was expected [sn_schema_info.local_fail] + ''' - WARNING: Need 'IMPL_1' has validation errors: +# --- +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity_and_rule].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[1] > local > properties > asil > const', + 'severity': 'info', + 'validation_msg': "'B' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_info', + }), + ]), + }), + }) +# --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_default] + ''' + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_violation.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_violation_min_unset_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_rule_default].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_info] ''' - WARNING: Need 'IMPL_1' has validation errors: - Severity: violation + WARNING: Need 'IMPL_1' has schema infos: + Severity: info Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_info.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_unset_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_rule_info].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'info', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_info', + }), + ]), + }), + }) +# --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_violation] ''' - WARNING: Need 'IMPL_1' has validation errors: - Severity: warning + ERROR: Need 'IMPL_1' has schema violations: + Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_violation.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_violation_not_reported] - '' +# name: test_schemas[schema/fixtures/reporting-severity_rule_violation].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_warning_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_rule_warning] ''' - WARNING: Need 'IMPL_1' has validation errors: + WARNING: Need 'IMPL_1' has schema warnings: Severity: warning Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_warning.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_warning].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'warning', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_warning', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/unevaluated-unevaluated_allof_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved' was unexpected) [sn_schema.local_fail] + Schema message: Unevaluated properties are not allowed ('approved' was unexpected) [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/unevaluated-unevaluated_allof_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > unevaluatedProperties', + 'severity': 'violation', + 'validation_msg': "Unevaluated properties are not allowed ('approved' was unexpected)", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/unevaluated-unevaluated_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('comment' was unexpected) [sn_schema.local_fail] + Schema message: Unevaluated properties are not allowed ('comment' was unexpected) [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/unevaluated-unevaluated_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > unevaluatedProperties', + 'severity': 'violation', + 'validation_msg': "Unevaluated properties are not allowed ('comment' was unexpected)", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- diff --git a/tests/schema/fixtures/reporting.yml b/tests/schema/fixtures/reporting.yml index f4f659c2b..a36dcc9f3 100644 --- a/tests/schema/fixtures/reporting.yml +++ b/tests/schema/fixtures/reporting.yml @@ -1,4 +1,4 @@ -severity_rule_unset_min_unset_is_reported: +severity_rule_default: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -19,7 +19,7 @@ severity_rule_unset_min_unset_is_reported: asil: const: A -severity_rule_info_min_unset_is_reported: +severity_rule_info: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -41,7 +41,7 @@ severity_rule_info_min_unset_is_reported: asil: const: A -severity_rule_warning_min_unset_is_reported: +severity_rule_warning: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -63,7 +63,7 @@ severity_rule_warning_min_unset_is_reported: asil: const: A -severity_rule_violation_min_unset_is_reported: +severity_rule_violation: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -85,15 +85,13 @@ severity_rule_violation_min_unset_is_reported: asil: const: A -severity_rule_info_min_warning_not_reported: +severity_default_suppress_severity: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" needs_schema_definitions_from_json = "schemas.json" + suppress_warnings = ["sn_schema_violation"] ubproject: | - [needs] - schema_severity = "warning" - [[needs.extra_options]] name = "asil" rst: | @@ -103,22 +101,25 @@ severity_rule_info_min_warning_not_reported: schemas: $defs: [] schemas: + - validate: + local: + properties: + asil: + const: A - severity: "info" validate: local: properties: asil: - const: A + const: B -severity_rule_warning_min_warning_is_reported: +severity_default_suppress_severity_and_rule: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" needs_schema_definitions_from_json = "schemas.json" + suppress_warnings = ["sn_schema_violation.local_fail"] ubproject: | - [needs] - schema_severity = "warning" - [[needs.extra_options]] name = "asil" rst: | @@ -128,34 +129,14 @@ severity_rule_warning_min_warning_is_reported: schemas: $defs: [] schemas: - - severity: "warning" - validate: + - validate: local: properties: asil: const: A - -severity_rule_warning_min_violation_not_reported: - conf: | - extensions = ["sphinx_needs"] - needs_from_toml = "ubproject.toml" - needs_schema_definitions_from_json = "schemas.json" - ubproject: | - [needs] - schema_severity = "violation" - - [[needs.extra_options]] - name = "asil" - rst: | - .. impl:: title - :id: IMPL_1 - :asil: QM - schemas: - $defs: [] - schemas: - - severity: "warning" + - severity: "info" validate: local: properties: asil: - const: A + const: B diff --git a/tests/schema/test_schema.py b/tests/schema/test_schema.py index dcdc4303a..d4e810795 100644 --- a/tests/schema/test_schema.py +++ b/tests/schema/test_schema.py @@ -63,6 +63,14 @@ def test_schemas( str(app.srcdir) + os.path.sep, "/" ) assert warnings == snapshot + + schema_violations: dict[str, Any] = json.loads( + Path(app.outdir, "schema_violations.json").read_text("utf8") + ) + exclude_keys = {"validated_needs_per_second", "validation_summary"} + for key in exclude_keys: + schema_violations.pop(key, None) + assert schema_violations == snapshot app.cleanup()