From 7bdfbd04b94b68d2405e31779cad6b97ddc21051 Mon Sep 17 00:00:00 2001 From: Marco Heinemann Date: Wed, 22 Oct 2025 16:17:59 +0200 Subject: [PATCH 1/5] =?UTF-8?q?=F0=9F=91=8C=20Improve=20schema=20severity?= =?UTF-8?q?=20handling?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/configuration.rst | 27 +--- docs/schema/index.rst | 179 +++++++++++++++++++-- sphinx_needs/config.py | 7 - sphinx_needs/schema/config.py | 2 + sphinx_needs/schema/core.py | 19 +-- sphinx_needs/schema/reporting.py | 263 +++++++++++++++++-------------- 6 files changed, 322 insertions(+), 175 deletions(-) diff --git a/docs/configuration.rst b/docs/configuration.rst index 5e18b1da1..83dfe2a29 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -292,7 +292,7 @@ And use it like: }, } ] - + The same fields for the :ref:`supported_data_types` as in the :ref:`schema_validation` are accepted. If ``schema`` is given, ``type`` is required. All the other keys can also be defined via :ref:`needs_schema_definitions` or in the file passed via @@ -2414,31 +2414,6 @@ Default value: ``None`` The JSON file should contain the same structure as :ref:`needs_schema_definitions`: -.. _`needs_schema_severity`: - -needs_schema_severity -~~~~~~~~~~~~~~~~~~~~~ - -.. versionadded:: 6.0.0 - -Minimum severity level for schema validation reporting. -Extra option and extra link schema errors are always reported as violations. -For each entry in :ref:`needs_schema_definitions` the severity can be defined by the user. - -Available severity levels: - -- ``info``: Informational message (default) -- ``warning``: Warning message -- ``violation``: Violation message - -The levels align with how `SHACL `__ defines severity levels. - -Default value: ``"info"`` - -.. code-block:: python - - needs_schema_severity = "warning" - .. _`needs_schema_debug_active`: needs_schema_debug_active diff --git a/docs/schema/index.rst b/docs/schema/index.rst index fbc5f3ebe..81f6f6ee8 100644 --- a/docs/schema/index.rst +++ b/docs/schema/index.rst @@ -575,6 +575,25 @@ This validates that: For most use cases, 4 levels of nesting is sufficient. If you need deeper validation chains, consider restructuring your validation logic or link relationships. +.. _`severity_levels`: + +Severity levels +--------------- + +The schema validation supports three severity levels for reporting violations: + +- ``info``: A non-critical constraint violation indicating an informative message. +- ``warning``: A non-critical constraint violation indicating a warning. +- ``violation`` : A constraint violation. + +The specific severity value has no impact on the validation, but it can influence how violations +are reported to the user. See :ref:`understanding_errors` for more details how Sphinx +console output can be configured and how to :ref:`suppress errors `. + +Schema violations are categorised with :ref:`message types `. Each +message type is assigned a default severity of ``violation``. +The default can be overwritten in :ref:`schema rules `. + .. _`schema_components`: Schema components reference @@ -699,24 +718,49 @@ in via ``allOf`` to be considered evaluated: In this case, a need with a ``priority`` property would still trigger an unevaluated properties error, even though ``priority`` is in the ``required`` list. -Severity levels -~~~~~~~~~~~~~~~ +.. _`rule_severity`: -Each schema can specify a severity level: +Rule severity +~~~~~~~~~~~~~ -- ``violation`` (default): Violation message -- ``warning``: Warning message -- ``info``: Informational message +Each schema rule can specify one of the following severity levels: -.. code-block:: json +- ``info``: A non-critical constraint violation indicating an informative message. +- ``warning``: A non-critical constraint violation indicating a warning. +- ``violation`` : A constraint violation. + + If that is not given, a violation + +rule specific default kicks in. The severity influences how validation failures are displayed +in the console: + +.. code-block:: python { "severity": "warning", "message": "Approval required due to high efforts" } -The config :ref:`needs_schema_severity` can be used to define a minimum severity level for a -warning to be reported. +All severities are reported during schema validation. + +**Console output example:** + +.. code-block:: text + + WARNING: Need 'FEAt' has validation warnings: + Severity: warning + Field: id + Need path: FEAt + Schema path: [0] > local > properties > id > pattern + User message: id must be uppercase with numbers and underscores + Schema message: 'FEAt' does not match '^[A-Z0-9_]+$' [sn_schema_warning.local_fail] + + ERROR: Need 'SPEC' has validation errors: + Severity: violation + Field: id + Need path: SPEC + Schema path: spec[1] > local > properties > id > pattern + Schema message: 'SPEC' does not match '^SPEC_[a-zA-Z0-9_-]*$' [sn_schema_violation.local_fail] .. _`schema_reuse`: @@ -986,12 +1030,76 @@ Array fields store lists of homogeneous typed values: - If ``minContains`` is not given, it defaults to 1 when ``contains`` is present. - If ``maxContains`` is not given, there is no upper limit. +.. _`understanding_errors`: + Running validation and understanding errors ------------------------------------------- After defining your schema configuration, running Sphinx will validate all needs against the defined schemas. This section explains the validation process and how to interpret errors. +.. _`severity_handling`: + +Severity handling +~~~~~~~~~~~~~~~~~ + +The :ref:`severity_levels` affect how Sphinx emits validation messages. +Here is how they map to Sphinx logging levels and console output. + +Schema severity **info**: + +- Mapped to Sphinx logging level ``warning`` +- Displayed as ``WARNING:`` in red color in console +- Logged with type ``sn_schema_info`` + +Schema severity **warning**: + +- Mapped to Sphinx logging level ``warning`` +- Displayed as ``WARNING:`` in red color in console +- Logged with type ``sn_schema_warning`` + +Schema severity **violation**: + +- Mapped to Sphinx logging level ``error`` +- Displayed as ``ERROR:`` in dark red color in console +- Logged with type ``sn_schema_violation`` + +Besides the ``type`` information mentioned above, log messages also receive a +``subtype`` described in :ref:`message_types`. + +The output format is ``WARNING|ERROR: [type.subtype]``. + +The combination of ``type`` and ``subtype`` can be used to selectively +:ref:`suppress validation messages `. + +.. _`message_types`: + +Message types +~~~~~~~~~~~~~ + +The schema validation is done on :ref:`local ` or +:ref:`network ` level. Accordingly, the following message subtypes are used +to indicate which validation failed. + +Local validation: + +- ``extra_option_fail``: Extra option schema validation failed + (local validation, defined via ``schema`` key in :ref:`needs_extra_options`) +- ``extra_link_fail``: Extra link schema validation failed + (defined via ``schema`` key in :ref:`needs_extra_links`) +- ``local_fail``: Need-local validation failed + (defined via ``validate.local`` in schemas of :ref:`needs_schema_definitions`) + +Network validation: + +- ``network_missing_target``: Outgoing link target cannot be resolved +- ``network_contains_too_few``: Not enough valid links (minContains failed) +- ``network_contains_too_many``: Too many valid links (maxContains failed) +- ``network_items_fail``: Linked need's item validation failed + +If not overridden in the :ref:`schema rules `, a default severity of ``violation`` +is used for all message types. + Error messages and output ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1001,7 +1109,7 @@ Validation errors include detailed information: - **Field**: The specific field that failed validation - **Need path**: The ID of the need that failed or the link chain for network validation - **Schema path**: The JSON path within the schema that was violated -- **User message**: Custom message from the needs_schema.schemas list +- **User message**: Custom message from each of the ``schemas`` in :ref:`needs_schema_definitions` - **Schema message**: Detailed technical validation message from the validator Example error output:: @@ -1035,6 +1143,57 @@ need and the specific link that caused the issue:: Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > links > links > local > allOf > 0 > properties > asil > enum Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] +.. _`suppress_validation_messages`: + +Suppressing validation messages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Schema validation messages can be selectively suppressed using Sphinx's +`suppress_warnings `__ +configuration option. + +**Examples:** + +Suppress all schema violation errors: + +.. code-block:: python + + suppress_warnings = ["sn_schema_violation"] + +Suppress all schema warnings: + +.. code-block:: python + + suppress_warnings = ["sn_schema_warning"] + +Suppress specific validation rule failures: + +.. code-block:: python + + # Suppress only local validation failures for violations + suppress_warnings = ["sn_schema_violation.local_fail"] + + # Suppress only network link count issues for warnings + suppress_warnings = ["sn_schema_warning.network_contains_too_few"] + +Suppress multiple specific types: + +.. code-block:: python + + suppress_warnings = [ + "sn_schema_violation.local_fail", + "sn_schema_warning.extra_option_fail", + "sn_schema_info", # Suppress all info messages + ] + +.. note:: + + Suppressing validation messages does not prevent the validation from occurring; it only + hides the output. All violations will still be recorded in the schema violation report + (see :ref:`schema_violation_json`). + +.. _`schema_violation_json`: + Schema violation report JSON file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/sphinx_needs/config.py b/sphinx_needs/config.py index aae15a286..f903efbf7 100644 --- a/sphinx_needs/config.py +++ b/sphinx_needs/config.py @@ -19,7 +19,6 @@ ExtraOptionSchemaTypes, ExtraOptionStringSchemaType, SchemasFileRootType, - SeverityEnum, ) if TYPE_CHECKING: @@ -412,12 +411,6 @@ def get_default(cls, name: str) -> Any: ) """Path to a JSON file to load the schemas from.""" - schema_severity: str = field( - default=SeverityEnum.info.name, - metadata={"rebuild": "env", "types": (str,)}, - ) - """Severity level for the schema validation reporting.""" - schema_debug_active: bool = field( default=False, metadata={"rebuild": "env", "types": (bool,)}, diff --git a/sphinx_needs/schema/config.py b/sphinx_needs/schema/config.py index cae0455c2..d210c54e8 100644 --- a/sphinx_needs/schema/config.py +++ b/sphinx_needs/schema/config.py @@ -366,6 +366,8 @@ class SeverityEnum(IntEnum): Default severity for each rule. User provided schemas can overwrite the severity of a rule. +The rules ``extra_option_fail`` and ``extra_link_fail`` cannot be changed by the user, +but they can be suppressed specifically using suppress_warnings config. """ diff --git a/sphinx_needs/schema/core.py b/sphinx_needs/schema/core.py index 97ac02ce6..21fc3d2eb 100644 --- a/sphinx_needs/schema/core.py +++ b/sphinx_needs/schema/core.py @@ -23,7 +23,6 @@ OntologyWarning, ValidateNeedMessageType, ValidateNeedType, - filter_warnings_severity, save_debug_files, ) from sphinx_needs.schema.utils import get_properties_from_schema @@ -114,7 +113,7 @@ def validate_need( need_path=[need["id"]], ) save_debug_files(config, new_warnings_options) - all_warnings.extend(filter_warnings_severity(config, new_warnings_options)) + all_warnings.extend(new_warnings_options) if _extra_link_schemas.get("properties"): new_warnings_links = get_ontology_warnings( @@ -127,7 +126,7 @@ def validate_need( need_path=[need["id"]], ) save_debug_files(config, new_warnings_links) - all_warnings.extend(filter_warnings_severity(config, new_warnings_links)) + all_warnings.extend(new_warnings_links) for type_schema in type_schemas: # maintain state for nested network validation @@ -231,10 +230,7 @@ def recurse_validate_schemas( user_severity=severity if recurse_level == 0 else None, ) save_debug_files(config, warnings_local) - warnings_local_filtered = filter_warnings_severity( - config, warnings_local, severity - ) - warnings.extend(warnings_local_filtered) + warnings.extend(warnings_local) if any_not_of_rule(warnings_local, rule_success): success = False if "network" in schema: @@ -369,9 +365,7 @@ def recurse_validate_schemas( if recurse_level == 0 and user_message is not None: # user message only added to the root validation warning["user_message"] = user_message - warnings.extend( - filter_warnings_severity(config, items_nok_warnings, severity) - ) + warnings.extend(items_nok_warnings) # Check contains validation results contains_success = True @@ -433,10 +427,7 @@ def recurse_validate_schemas( contains_warnings[-1]["user_message"] = user_message contains_success = False - filtered_contains_warnings = filter_warnings_severity( - config, contains_warnings, severity - ) - warnings.extend(filtered_contains_warnings) + warnings.extend(contains_warnings) # Overall success requires both items and minmax validation to pass if not (items_success and contains_success): diff --git a/sphinx_needs/schema/reporting.py b/sphinx_needs/schema/reporting.py index e6dfdc5c2..60cb54989 100644 --- a/sphinx_needs/schema/reporting.py +++ b/sphinx_needs/schema/reporting.py @@ -6,10 +6,11 @@ from pathlib import Path from typing import TYPE_CHECKING, Any, Literal, TypedDict +from sphinx.errors import SphinxError + from sphinx_needs.config import NeedsSphinxConfig from sphinx_needs.need_item import NeedItem from sphinx_needs.schema.config import ( - MAP_RULE_DEFAULT_SEVERITY, MessageRuleEnum, NeedFieldsSchemaWithVersionType, SeverityEnum, @@ -72,45 +73,6 @@ class ValidateNeedType(TypedDict): """Separator between nested parts of debug output file names.""" -def filter_warnings_severity( - config: NeedsSphinxConfig, - warnings: list[OntologyWarning], - schema_root_severity: SeverityEnum | None = None, -) -> list[OntologyWarning]: - """ - Filter warnings by severity. - - There are multiple severity sources: - - needs_config.schema_severity: the minimum severity for warnings to be reported - - needs_config.schema_definitions: contains a severity field that determines the reported rule severity - - schemas.config.MAP_RULE_DEFAULT_SEVERITY: default if the schemas severities field is not set - - Precedence for reporting: - - if the rule has a default severity of none, it is ignored; those are never of interest - as reporting target, only for debugging purposes - - if the schema root severity is set, it overrides the rule severity - - if the schema root severity is not set, the rule default severity is used - """ - min_severity_for_report = SeverityEnum[config.schema_severity] - filtered_warnings = [] - for warning in warnings: - rule_default_severity = MAP_RULE_DEFAULT_SEVERITY[warning["rule"]] - if rule_default_severity is SeverityEnum.none: - # rule is not of interest for final reporting (e.g. unselected needs) - # it might be logged for schema debugging however - continue - # the warning severity is overriden by the schema root severity - # if it is unset, the rule mapping MAP_RULE_DEFAULT_SEVERITY is used - warning_severity = ( - schema_root_severity - if schema_root_severity is not None - else warning["severity"] - ) - if warning_severity >= min_severity_for_report: - filtered_warnings.append(warning) - return filtered_warnings - - def save_debug_files( config: NeedsSphinxConfig, warnings: list[OntologyWarning] ) -> None: @@ -192,9 +154,17 @@ class WarningDetails(TypedDict): user_msg: NotRequired[str] +class JSONFormattedWarningChild(TypedDict): + """JSON Formatted child warning (nested within parent warnings).""" + + subtype: str + details: WarningDetails + children: list[ChildWarning] + + class ChildWarning(TypedDict): need_id: str - warning: JSONFormattedWarning + warning: JSONFormattedWarningChild class JSONFormattedWarning(TypedDict): @@ -226,30 +196,45 @@ def get_formatted_warnings_recurse( formatted_warnings: list[FormattedWarning] = [] warning_msg = get_warning_msg(level, warning) has_title = level == 0 + log_lvl: Literal["warning", "error"] + + # Map severity to logger level and type if warning["severity"] == SeverityEnum.config_error: title = ( f"Need '{warning['need']['id']}' has configuration errors:" if has_title else "" ) - formatted_warning = FormattedWarning( - log_lvl="error", - type="sn_schema", - subtype=warning["rule"].value, - message=f"{title}{warning_msg}", - ) - else: + log_lvl = "error" + type_str = "sn_schema" + elif warning["severity"] == SeverityEnum.violation: title = ( - f"Need '{warning['need']['id']}' has validation errors:" + f"Need '{warning['need']['id']}' has schema violations:" if has_title else "" ) - formatted_warning = FormattedWarning( - log_lvl="warning", - type="sn_schema", - subtype=warning["rule"].value, - message=f"{title}{warning_msg}", + log_lvl = "error" + type_str = "sn_schema_violation" + elif warning["severity"] == SeverityEnum.warning: + title = ( + f"Need '{warning['need']['id']}' has schema warnings:" if has_title else "" ) + log_lvl = "warning" + type_str = "sn_schema_warning" + elif warning["severity"] == SeverityEnum.info: + title = f"Need '{warning['need']['id']}' has schema infos:" if has_title else "" + log_lvl = "warning" + type_str = "sn_schema_info" + else: + # SeverityEnum.none is filtered out earlier + raise SphinxError("Unexpected severity 'none' in console formatting.") + + formatted_warning = FormattedWarning( + log_lvl=log_lvl, + type=type_str, + subtype=warning["rule"].value, + message=f"{title}{warning_msg}", + ) formatted_warnings.append(formatted_warning) if "children" in warning: for child_warning in warning["children"]: @@ -272,80 +257,21 @@ def get_formatted_warnings( """ Pretty format warnings from the ontology validation. + Warnings with severity 'none' are filtered out as they are only for debugging. + :returns: tuple [log level, type, sub-type, message] """ formatted_warnings: list[FormattedWarning] = [] for warnings in need_2_warnings.values(): for warning in warnings: + # Skip warnings with severity none (debug only) + if warning["severity"] == SeverityEnum.none: + continue new_warnings = get_formatted_warnings_recurse(warning=warning, level=0) formatted_warnings.extend(new_warnings) return formatted_warnings -def get_json_formatted_warnings_recurse( - warning: OntologyWarning, level: int -) -> list[JSONFormattedWarning]: - """ - Recursively format warning details. - - :param warning: The OntologyWarning to format. - :param level: The OntologyWarning level. - :return: A list of JSONFormattedWarning objects. - """ - formatted_warnings: list[JSONFormattedWarning] = [] - warning_details = get_json_warning_details(level, warning) - if warning["severity"] == SeverityEnum.config_error: - formatted_warning = JSONFormattedWarning( - log_lvl="error", - type="sn_schema", - subtype=warning["rule"].value, - details=warning_details, - children=[], - ) - else: - formatted_warning = JSONFormattedWarning( - log_lvl="warning", - type="sn_schema", - subtype=warning["rule"].value, - details=warning_details, - children=[], - ) - formatted_warnings.append(formatted_warning) - if "children" in warning: - for child_warning in warning["children"]: - formatted_child_warnings = get_json_formatted_warnings_recurse( - warning=child_warning, level=level + 1 - ) - for formatted_child_warning in formatted_child_warnings: - formatted_warning["children"].append( - { - "need_id": child_warning["need"]["id"], - "warning": formatted_child_warning, - } - ) - return formatted_warnings - - -def get_json_formatted_warnings( - need_2_warnings: dict[str, list[OntologyWarning]], -) -> dict[str, list[JSONFormattedWarning]]: - """ - JSON formatted warnings from the ontology validation for each need. - - :param need_2_warnings: A dictionary mapping need IDs to their warnings. - :return: A dictionary with list of JSONFormattedWarning objects. - """ - need_formatted_warnings: dict[str, list[JSONFormattedWarning]] = {} - for need_id, warnings in need_2_warnings.items(): - formatted_warnings: list[JSONFormattedWarning] = [] - for warning in warnings: - new_warnings = get_json_formatted_warnings_recurse(warning=warning, level=0) - formatted_warnings.extend(new_warnings) - - need_formatted_warnings[need_id] = formatted_warnings - return need_formatted_warnings - - def get_warning_msg(base_lvl: int, warning: OntologyWarning) -> str: """Craft a properly indented warning message.""" warning_msg = "" @@ -386,6 +312,107 @@ def nl_indent(level: int) -> str: return warning_msg +def _get_json_formatted_child_recurse( + warning: OntologyWarning, level: int +) -> JSONFormattedWarningChild: + """Helper function to format nested child warnings without log_lvl and type.""" + warning_details = get_json_warning_details(level, warning) + + formatted_child = JSONFormattedWarningChild( + subtype=warning["rule"].value, + details=warning_details, + children=[], + ) + if "children" in warning: + for child_warning in warning["children"]: + formatted_child["children"].append( + { + "need_id": child_warning["need"]["id"], + "warning": _get_json_formatted_child_recurse( + child_warning, level=level + 1 + ), + } + ) + return formatted_child + + +def get_json_formatted_warnings_recurse( + warning: OntologyWarning, level: int +) -> list[JSONFormattedWarning]: + """ + Recursively format warning details. + + :param warning: The OntologyWarning to format. + :param level: The OntologyWarning level (0 for root). + :return: A list of JSONFormattedWarning objects. + """ + warning_details = get_json_warning_details(level, warning) + + log_lvl: Literal["warning", "error"] + + # Root level warning includes log_lvl and type + # Map severity to logger level and type + if warning["severity"] == SeverityEnum.config_error: + log_lvl = "error" + type_str = "sn_schema" + elif warning["severity"] == SeverityEnum.violation: + log_lvl = "error" + type_str = "sn_schema_violation" + elif warning["severity"] == SeverityEnum.warning: + log_lvl = "warning" + type_str = "sn_schema_warning" + elif warning["severity"] == SeverityEnum.info: + log_lvl = "warning" + type_str = "sn_schema_info" + else: + # SeverityEnum.none is filtered out earlier + raise SphinxError("Unexpected severity 'none' in JSON formatting.") + + formatted_warning = JSONFormattedWarning( + log_lvl=log_lvl, + type=type_str, + subtype=warning["rule"].value, + details=warning_details, + children=[], + ) + if "children" in warning: + for child_warning in warning["children"]: + formatted_warning["children"].append( + { + "need_id": child_warning["need"]["id"], + "warning": _get_json_formatted_child_recurse( + child_warning, level=level + 1 + ), + } + ) + return [formatted_warning] + + +def get_json_formatted_warnings( + need_2_warnings: dict[str, list[OntologyWarning]], +) -> dict[str, list[JSONFormattedWarning]]: + """ + JSON formatted warnings from the ontology validation for each need. + + Warnings with severity 'none' are filtered out as they are only for debugging. + + :param need_2_warnings: A dictionary mapping need IDs to their warnings. + :return: A dictionary with list of JSONFormattedWarning objects. + """ + need_formatted_warnings: dict[str, list[JSONFormattedWarning]] = {} + for need_id, warnings in need_2_warnings.items(): + formatted_warnings: list[JSONFormattedWarning] = [] + for warning in warnings: + # Skip warnings with severity none (debug only) + if warning["severity"] == SeverityEnum.none: + continue + new_warnings = get_json_formatted_warnings_recurse(warning=warning, level=0) + formatted_warnings.extend(new_warnings) + + need_formatted_warnings[need_id] = formatted_warnings + return need_formatted_warnings + + def get_json_warning_details(base_lvl: int, warning: OntologyWarning) -> WarningDetails: """Get JSON formatted warning details.""" warning_details: WarningDetails = {} From fb17cd825102a1e5cecbe98752f58020d53e8364 Mon Sep 17 00:00:00 2001 From: Marco Heinemann Date: Wed, 22 Oct 2025 21:12:11 +0200 Subject: [PATCH 2/5] Fix tests --- sphinx_needs/schema/core.py | 3 +- sphinx_needs/schema/reporting.py | 6 +- .../doc_schema_benchmark/ubproject.toml | 1 - tests/doc_test/doc_schema_e2e/ubproject.toml | 1 - .../doc_schema_example/ubproject.toml | 1 - tests/schema/__snapshots__/test_schema.ambr | 1426 +++++++++++++++-- tests/schema/fixtures/reporting.yml | 4 +- tests/schema/test_schema.py | 8 + 8 files changed, 1341 insertions(+), 109 deletions(-) diff --git a/sphinx_needs/schema/core.py b/sphinx_needs/schema/core.py index 21fc3d2eb..5ae23e07e 100644 --- a/sphinx_needs/schema/core.py +++ b/sphinx_needs/schema/core.py @@ -620,8 +620,9 @@ def get_ontology_warnings( "final_schema": validation_report["final_schema"], "schema_path": [*schema_path, *msg["schema_path"]], "need_path": need_path, - "field": msg["field"], } + if msg.get("field"): + warning["field"] = msg["field"] if user_message is not None: warning["user_message"] = user_message warnings.append(warning) diff --git a/sphinx_needs/schema/reporting.py b/sphinx_needs/schema/reporting.py index 60cb54989..92bc2c37f 100644 --- a/sphinx_needs/schema/reporting.py +++ b/sphinx_needs/schema/reporting.py @@ -238,6 +238,8 @@ def get_formatted_warnings_recurse( formatted_warnings.append(formatted_warning) if "children" in warning: for child_warning in warning["children"]: + if child_warning["severity"] == SeverityEnum.none: + continue formatted_child_warnings = get_formatted_warnings_recurse( warning=child_warning, level=level + 1 ) @@ -247,7 +249,6 @@ def get_formatted_warnings_recurse( f"\n\n{indent}Details for {child_warning['need']['id']}" + formatted_child_warning["message"] ) - # formatted_warnings.extend(formatted_child_warnings) return formatted_warnings @@ -409,7 +410,8 @@ def get_json_formatted_warnings( new_warnings = get_json_formatted_warnings_recurse(warning=warning, level=0) formatted_warnings.extend(new_warnings) - need_formatted_warnings[need_id] = formatted_warnings + if formatted_warnings: + need_formatted_warnings[need_id] = formatted_warnings return need_formatted_warnings diff --git a/tests/doc_test/doc_schema_benchmark/ubproject.toml b/tests/doc_test/doc_schema_benchmark/ubproject.toml index 116b18dd7..a96e538d6 100644 --- a/tests/doc_test/doc_schema_benchmark/ubproject.toml +++ b/tests/doc_test/doc_schema_benchmark/ubproject.toml @@ -4,7 +4,6 @@ id_required = true a = ["a", 3, { "a" = "b" }] id_regex = "^[A-Z0-9_]{3,}" -schema_severity = 'info' [[needs.extra_options]] name = "efforts" diff --git a/tests/doc_test/doc_schema_e2e/ubproject.toml b/tests/doc_test/doc_schema_e2e/ubproject.toml index cb337060c..4d21be01a 100644 --- a/tests/doc_test/doc_schema_e2e/ubproject.toml +++ b/tests/doc_test/doc_schema_e2e/ubproject.toml @@ -3,7 +3,6 @@ [needs] id_required = true id_regex = "^[A-Z0-9_]{3,}" -schema_severity = 'warning' build_json = true schema_definitions_from_json = "schemas.json" schema_debug_active = true diff --git a/tests/doc_test/doc_schema_example/ubproject.toml b/tests/doc_test/doc_schema_example/ubproject.toml index 5bd212920..9de6dcfdc 100644 --- a/tests/doc_test/doc_schema_example/ubproject.toml +++ b/tests/doc_test/doc_schema_example/ubproject.toml @@ -3,7 +3,6 @@ [needs] id_required = true id_regex = "^[A-Z0-9_]{3,}" -schema_severity = 'warning' build_json = true schema_definitions_from_json = "schemas.json" schema_debug_active = true diff --git a/tests/schema/__snapshots__/test_schema.ambr b/tests/schema/__snapshots__/test_schema.ambr index c51b1287a..7351953dd 100644 --- a/tests/schema/__snapshots__/test_schema.ambr +++ b/tests/schema/__snapshots__/test_schema.ambr @@ -169,39 +169,42 @@ # --- # name: test_schema_e2e[test_app0] ''' - WARNING: Need 'FEAt' has validation errors: + WARNING: Need 'FEAt' has schema warnings: Severity: warning Field: id Need path: FEAt Schema path: [0] > local > properties > id > pattern User message: id must be uppercase with numbers and underscores - Schema message: 'FEAt' does not match '^[A-Z0-9_]+$' [sn_schema.local_fail] - WARNING: Need 'SPEC_MISSING_APPROVAL' has validation errors: + Schema message: 'FEAt' does not match '^[A-Z0-9_]+$' [sn_schema_warning.local_fail] + ERROR: Need 'SPEC_MISSING_APPROVAL' has schema violations: Severity: violation - Field: Need path: SPEC_MISSING_APPROVAL Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('asil', 'priority' were unexpected) [sn_schema.local_fail] - WARNING: Need 'SPEC_MISSING_APPROVAL' has validation errors: + Schema message: Unevaluated properties are not allowed ('asil', 'priority' were unexpected) [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_MISSING_APPROVAL' has schema violations: Severity: violation - Field: Need path: SPEC_MISSING_APPROVAL Schema path: spec-approved-required[2] > local > required User message: Approval required due to high efforts - Schema message: 'approved' is a required property [sn_schema.local_fail] - WARNING: Need 'SPEC' has validation errors: + Schema message: 'approved' is a required property [sn_schema_violation.local_fail] + WARNING: Need 'SPEC_MISSING_APPROVAL' has schema infos: + Severity: info + Need path: SPEC_MISSING_APPROVAL + Schema path: spec-approved-not-given[3] > local > required + User message: Approval not given + Schema message: 'approved' is a required property [sn_schema_info.local_fail] + ERROR: Need 'SPEC' has schema violations: Severity: violation Field: id Need path: SPEC Schema path: spec[1] > local > properties > id > pattern - Schema message: 'SPEC' does not match '^SPEC_[a-zA-Z0-9_-]*$' [sn_schema.local_fail] - WARNING: Need 'SPEC_SAFE_UNSAFE_FEAT' has validation errors: + Schema message: 'SPEC' does not match '^SPEC_[a-zA-Z0-9_-]*$' [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_SAFE_UNSAFE_FEAT' has schema violations: Severity: violation - Field: Need path: SPEC_SAFE_UNSAFE_FEAT Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema.local_fail] - WARNING: Need 'SPEC_SAFE_UNSAFE_FEAT' has validation errors: + Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_SAFE_UNSAFE_FEAT' has schema violations: Severity: violation Need path: SPEC_SAFE_UNSAFE_FEAT > links Schema path: safe-spec-[links]->safe-feat[4] > validate > network > links > contains @@ -211,19 +214,17 @@ Field: asil Need path: SPEC_SAFE_UNSAFE_FEAT > links > FEAT Schema path: safe-spec-[links]->safe-feat[4] > validate > network > links > contains > local > allOf > 0 > properties > asil > enum - Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] - WARNING: Need 'SPEC_SAFE_ADD_UNSAFE_FEAT' has validation errors: + Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema_violation.network_contains_too_few] + ERROR: Need 'SPEC_SAFE_ADD_UNSAFE_FEAT' has schema violations: Severity: violation - Field: Need path: SPEC_SAFE_ADD_UNSAFE_FEAT Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema.local_fail] - WARNING: Need 'SPEC_SAFE' has validation errors: + Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema_violation.local_fail] + ERROR: Need 'SPEC_SAFE' has schema violations: Severity: violation - Field: Need path: SPEC_SAFE Schema path: spec[1] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema.local_fail] + Schema message: Unevaluated properties are not allowed ('approved', 'asil', 'links', 'priority' were unexpected) [sn_schema_violation.local_fail] ''' # --- @@ -2240,349 +2241,1053 @@ # name: test_schemas[schema/fixtures/extra_links-contains] '' # --- +# name: test_schemas[schema/fixtures/extra_links-contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > contains - Schema message: ['SPEC_1'] does not contain items matching the given schema [sn_schema.extra_link_fail] + Schema message: ['SPEC_1'] does not contain items matching the given schema [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > contains', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] does not contain items matching the given schema", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-inject_array] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > minItems - Schema message: ['SPEC_1'] is too short [sn_schema.extra_link_fail] + Schema message: ['SPEC_1'] is too short [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-inject_array].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > minItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] is too short", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-items_pattern] '' # --- +# name: test_schemas[schema/fixtures/extra_links-items_pattern].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-items_pattern_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links.0 Need path: IMPL_1 Schema path: extra_links > schema > properties > links > items > pattern - Schema message: 'SPEC_1' does not match '^[A-Z0-9_]{10,}' [sn_schema.extra_link_fail] + Schema message: 'SPEC_1' does not match '^[A-Z0-9_]{10,}' [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-items_pattern_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links.0', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > items > pattern', + 'severity': 'violation', + 'validation_msg': "'SPEC_1' does not match '^[A-Z0-9_]{10,}'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_contains] '' # --- +# name: test_schemas[schema/fixtures/extra_links-max_contains].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > contains - Schema message: Too many items match the given schema (expected at most 1) [sn_schema.extra_link_fail] + Schema message: Too many items match the given schema (expected at most 1) [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-max_contains_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > contains', + 'severity': 'violation', + 'validation_msg': 'Too many items match the given schema (expected at most 1)', + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_items] '' # --- +# name: test_schemas[schema/fixtures/extra_links-max_items].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-max_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > maxItems - Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema.extra_link_fail] + Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-max_items_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > maxItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1', 'SPEC_2'] is too long", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_contains] '' # --- +# name: test_schemas[schema/fixtures/extra_links-min_contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > contains - Schema message: Too few items match the given schema (expected at least 2 but only 1 matched) [sn_schema.extra_link_fail] + Schema message: Too few items match the given schema (expected at least 2 but only 1 matched) [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-min_contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > contains', + 'severity': 'violation', + 'validation_msg': 'Too few items match the given schema (expected at least 2 but only 1 matched)', + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_items] '' # --- +# name: test_schemas[schema/fixtures/extra_links-min_items].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_links-min_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: extra_links > schema > properties > links > minItems - Schema message: ['SPEC_1'] is too short [sn_schema.extra_link_fail] + Schema message: ['SPEC_1'] is too short [sn_schema_violation.extra_link_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_links-min_items_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_links > schema > properties > links > minItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] is too short", + }), + 'log_lvl': 'error', + 'subtype': 'extra_link_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-auto_inject_type] '' # --- +# name: test_schemas[schema/fixtures/extra_options-auto_inject_type].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-auto_inject_type_wrong_const] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'QM' was expected [sn_schema.local_fail] + Schema message: 'QM' was expected [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-auto_inject_type_wrong_const].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'QM' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean] '' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean].1 + dict({ + 'validated_needs_count': 14, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean_from_string_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'approved' is invalid: Cannot convert 'not-a-boolean' to boolean [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_boolean_from_string_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_integer] '' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_integer].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_float_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'efforts' is invalid: Cannot convert '1.2' to integer [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_float_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_string_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'efforts' is invalid: Cannot convert 'QM' to integer [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_integer_from_string_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_number] '' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_number].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-coerce_to_number_from_string_error] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'efforts' is invalid: Cannot convert 'QM' to float [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-coerce_to_number_from_string_error].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-const] '' # --- +# name: test_schemas[schema/fixtures/extra_options-const].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-const_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'QM' was expected [sn_schema.local_fail] + Schema message: 'QM' was expected [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-const_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'QM' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-enum] '' # --- +# name: test_schemas[schema/fixtures/extra_options-enum].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-enum_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > enum - Schema message: 'E' is not one of ['QM', 'A', 'B', 'C', 'D'] [sn_schema.local_fail] + Schema message: 'E' is not one of ['QM', 'A', 'B', 'C', 'D'] [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-enum_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > enum', + 'severity': 'violation', + 'validation_msg': "'E' is not one of ['QM', 'A', 'B', 'C', 'D']", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-integer_multiple_of] '' # --- +# name: test_schemas[schema/fixtures/extra_options-integer_multiple_of].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-integer_multiple_of_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: efforts Need path: IMPL_1 Schema path: extra_options > schema > properties > efforts > multipleOf - Schema message: 8 is not a multiple of 3 [sn_schema.extra_option_fail] + Schema message: 8 is not a multiple of 3 [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-integer_multiple_of_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'efforts', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > efforts > multipleOf', + 'severity': 'violation', + 'validation_msg': '8 is not a multiple of 3', + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-no_schema] '' # --- +# name: test_schemas[schema/fixtures/extra_options-no_schema].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-number_multiple_of] '' # --- +# name: test_schemas[schema/fixtures/extra_options-number_multiple_of].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-number_multiple_of_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: efforts Need path: IMPL_1 Schema path: extra_options > schema > properties > efforts > multipleOf - Schema message: 5.0 is not a multiple of 3.3 [sn_schema.extra_option_fail] + Schema message: 5.0 is not a multiple of 3.3 [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-number_multiple_of_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'efforts', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > efforts > multipleOf', + 'severity': 'violation', + 'validation_msg': '5.0 is not a multiple of 3.3', + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required] '' # --- +# name: test_schemas[schema/fixtures/extra_options-required].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_below_threshold] '' # --- +# name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_below_threshold].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_missing] '' # --- +# name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_missing].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_over_threshold] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > required User message: Required due to high efforts - Schema message: 'asil' is a required property [sn_schema.local_fail] + Schema message: 'asil' is a required property [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-required_based_on_select_field_over_threshold].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > required', + 'severity': 'violation', + 'user_msg': 'Required due to high efforts', + 'validation_msg': "'asil' is a required property", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-required_missing] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > required - Schema message: 'asil' is a required property [sn_schema.local_fail] + Schema message: 'asil' is a required property [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-required_missing].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > required', + 'severity': 'violation', + 'validation_msg': "'asil' is a required property", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-set_type_string] '' # --- +# name: test_schemas[schema/fixtures/extra_options-set_type_string].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: start_date Need path: IMPL_1 Schema path: extra_options > schema > properties > start_date > format - Schema message: 'not-a-date' is not a 'date' [sn_schema.extra_option_fail] + Schema message: 'not-a-date' is not a 'date' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'start_date', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > start_date > format', + 'severity': 'violation', + 'validation_msg': "'not-a-date' is not a 'date'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date_time] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date_time].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_date_time_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: start_date Need path: IMPL_1 Schema path: extra_options > schema > properties > start_date > format - Schema message: '2025-07-1099:99:99Z' is not a 'date-time' [sn_schema.extra_option_fail] + Schema message: '2025-07-1099:99:99Z' is not a 'date-time' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_date_time_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'start_date', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > start_date > format', + 'severity': 'violation', + 'validation_msg': "'2025-07-1099:99:99Z' is not a 'date-time'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_duration] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_duration].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_duration_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: _duration Need path: IMPL_1 Schema path: extra_options > schema > properties > _duration > format - Schema message: 'P1Q2Q10DT2H30M' is not a 'duration' [sn_schema.extra_option_fail] + Schema message: 'P1Q2Q10DT2H30M' is not a 'duration' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_duration_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': '_duration', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > _duration > format', + 'severity': 'violation', + 'validation_msg': "'P1Q2Q10DT2H30M' is not a 'duration'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_email] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_email].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_email_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: email Need path: IMPL_1 Schema path: extra_options > schema > properties > email > format - Schema message: 'not-a-mail' is not a 'email' [sn_schema.extra_option_fail] + Schema message: 'not-a-mail' is not a 'email' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_email_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'email', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > email > format', + 'severity': 'violation', + 'validation_msg': "'not-a-mail' is not a 'email'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_time] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_time].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_time_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: time Need path: IMPL_1 Schema path: extra_options > schema > properties > time > format - Schema message: '26:12:13' is not a 'time' [sn_schema.extra_option_fail] + Schema message: '26:12:13' is not a 'time' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_time_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'time', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > time > format', + 'severity': 'violation', + 'validation_msg': "'26:12:13' is not a 'time'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uri] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uri].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uri_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: uri Need path: IMPL_1 Schema path: extra_options > schema > properties > uri > format - Schema message: 'examplecom' is not a 'uri' [sn_schema.extra_option_fail] + Schema message: 'examplecom' is not a 'uri' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uri_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'uri', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > uri > format', + 'severity': 'violation', + 'validation_msg': "'examplecom' is not a 'uri'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uuid] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uuid].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_format_uuid_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: uuid Need path: IMPL_1 Schema path: extra_options > schema > properties > uuid > format - Schema message: 'deadbeef-deadbeef-deadbeef-deadbeef' is not a 'uuid' [sn_schema.extra_option_fail] + Schema message: 'deadbeef-deadbeef-deadbeef-deadbeef' is not a 'uuid' [sn_schema_violation.extra_option_fail] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-string_format_uuid_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'uuid', + 'need_path': 'IMPL_1', + 'schema_path': 'extra_options > schema > properties > uuid > format', + 'severity': 'violation', + 'validation_msg': "'deadbeef-deadbeef-deadbeef-deadbeef' is not a 'uuid'", + }), + 'log_lvl': 'error', + 'subtype': 'extra_option_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-string_type] '' # --- +# name: test_schemas[schema/fixtures/extra_options-string_type].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/extra_options-wrong_type] ''' /index.rst:1: WARNING: Need could not be created: Extra option 'asil' is invalid: Cannot convert 'QM' to integer [needs.create_need] ''' # --- +# name: test_schemas[schema/fixtures/extra_options-wrong_type].1 + dict({ + 'validated_needs_count': 0, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-link_chain_hop_1_min_contains_error] ''' - WARNING: Need 'IMPL_SAFE' has validation errors: + ERROR: Need 'IMPL_SAFE' has schema violations: Severity: violation Need path: IMPL_SAFE > links Schema path: safe-impl-[links]->safe-spec[0] > validate > network > links > contains @@ -2592,13 +3297,49 @@ Field: asil Need path: IMPL_SAFE > links > SPEC_UNSAFE Schema path: safe-impl-[links]->safe-spec[0] > validate > network > links > contains > local > allOf > 0 > properties > asil > enum - Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] + Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-link_chain_hop_1_min_contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_SAFE': list([ + dict({ + 'children': list([ + dict({ + 'need_id': 'SPEC_UNSAFE', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_SAFE > links > SPEC_UNSAFE', + 'schema_path': 'safe-impl-[links]->safe-spec[0] > validate > network > links > contains > local > allOf > 0 > properties > asil > enum', + 'validation_msg': "'QM' is not one of ['A', 'B', 'C', 'D']", + }), + 'subtype': 'network_local_fail', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links', + 'schema_path': 'safe-impl-[links]->safe-spec[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: SPEC_UNSAFE", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-link_chain_hop_2_min_contains_error] ''' - WARNING: Need 'IMPL_SAFE' has validation errors: + ERROR: Need 'IMPL_SAFE' has schema violations: Severity: violation Need path: IMPL_SAFE > links Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains @@ -2614,93 +3355,331 @@ Field: asil Need path: IMPL_SAFE > links > SPEC_SAFE > links > REQ_UNSAFE Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > validate > network > links > contains > local > allOf > 0 > properties > asil > enum - Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] + Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-link_chain_hop_2_min_contains_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_SAFE': list([ + dict({ + 'children': list([ + dict({ + 'need_id': 'SPEC_SAFE', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links > SPEC_SAFE', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > local', + }), + 'subtype': 'network_local_success', + }), + }), + dict({ + 'need_id': 'SPEC_SAFE', + 'warning': dict({ + 'children': list([ + dict({ + 'need_id': 'REQ_UNSAFE', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_SAFE > links > SPEC_SAFE > links > REQ_UNSAFE', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > validate > network > links > contains > local > allOf > 0 > properties > asil > enum', + 'validation_msg': "'QM' is not one of ['A', 'B', 'C', 'D']", + }), + 'subtype': 'network_local_fail', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links > SPEC_SAFE > links', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains > validate > network > links > contains', + 'user_msg': 'Safe impl links to safe spec links to safe req', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: REQ_UNSAFE", + }), + 'subtype': 'network_contains_too_few', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links', + 'schema_path': 'safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: SPEC_SAFE", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-link_chain_w_refs] '' # --- +# name: test_schemas[schema/fixtures/network-link_chain_w_refs].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_max_items] '' # --- +# name: test_schemas[schema/fixtures/network-local_max_items].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_max_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: [0] > local > properties > links > maxItems - Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema.local_fail] + Schema message: ['SPEC_1', 'SPEC_2'] is too long [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/network-local_max_items_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > links > maxItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1', 'SPEC_2'] is too long", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_min_items] '' # --- +# name: test_schemas[schema/fixtures/network-local_min_items].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-local_min_items_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: links Need path: IMPL_1 Schema path: [0] > local > properties > links > minItems - Schema message: ['SPEC_1'] is too short [sn_schema.local_fail] + Schema message: ['SPEC_1'] is too short [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/network-local_min_items_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'links', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > links > minItems', + 'severity': 'violation', + 'validation_msg': "['SPEC_1'] is too short", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-max_contains] '' # --- +# name: test_schemas[schema/fixtures/network-max_contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-max_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Need path: IMPL_1 > links Schema path: [0] > validate > network > links > contains - Schema message: Too many valid links of type 'links' (2 > 1) / ok: SPEC_1, SPEC_2 [sn_schema.network_contains_too_many] + Schema message: Too many valid links of type 'links' (2 > 1) / ok: SPEC_1, SPEC_2 [sn_schema_violation.network_contains_too_many] ''' # --- +# name: test_schemas[schema/fixtures/network-max_contains_error].1 + dict({ + 'validated_needs_count': 3, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links', + 'schema_path': '[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too many valid links of type 'links' (2 > 1) / ok: SPEC_1, SPEC_2", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_many', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-max_network_levels] ''' - WARNING: Need 'SPEC_1' has validation errors: + ERROR: Need 'SPEC_1' has schema violations: Severity: violation Need path: IMPL_1 > links > SPEC_5 > links > SPEC_4 > links > SPEC_3 > links > SPEC_2 > links > SPEC_1 Schema path: [0] > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items - Schema message: Maximum network validation recursion level 4 reached. [sn_schema.network_max_nest_level] + Schema message: Maximum network validation recursion level 4 reached. [sn_schema_violation.network_max_nest_level] ''' # --- +# name: test_schemas[schema/fixtures/network-max_network_levels].1 + dict({ + 'validated_needs_count': 6, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links > SPEC_5 > links > SPEC_4 > links > SPEC_3 > links > SPEC_2 > links > SPEC_1', + 'schema_path': '[0] > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items > validate > network > links > items', + 'severity': 'violation', + 'validation_msg': 'Maximum network validation recursion level 4 reached.', + }), + 'log_lvl': 'error', + 'subtype': 'network_max_nest_level', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-min_contains] '' # --- +# name: test_schemas[schema/fixtures/network-min_contains].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-min_contains_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Need path: IMPL_1 > links Schema path: [0] > validate > network > links > contains - Schema message: Too few valid links of type 'links' (1 < 2) / ok: SPEC_1 [sn_schema.network_contains_too_few] + Schema message: Too few valid links of type 'links' (1 < 2) / ok: SPEC_1 [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-min_contains_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links', + 'schema_path': '[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (1 < 2) / ok: SPEC_1", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-no_error_empty_links] ''' - WARNING: Need 'IMPL_SAFE' has validation errors: + ERROR: Need 'IMPL_SAFE' has schema violations: Severity: violation Need path: IMPL_SAFE > links Schema path: impl-[links]->spec[0] > validate > network > links > contains - Schema message: Too few valid links of type 'links' (0 < 1) [sn_schema.network_contains_too_few] + Schema message: Too few valid links of type 'links' (0 < 1) [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-no_error_empty_links].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_SAFE': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_SAFE > links', + 'schema_path': 'impl-[links]->spec[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1)", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/network-schemas_in_conf] '' # --- +# name: test_schemas[schema/fixtures/network-schemas_in_conf].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + }), + }) +# --- # name: test_schemas[schema/fixtures/network-schemas_in_conf_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Need path: IMPL_1 > links Schema path: [0] > validate > network > links > contains @@ -2710,90 +3689,337 @@ Field: type Need path: IMPL_1 > links > SPEC_1 Schema path: [0] > validate > network > links > contains > local > properties > type > const - Schema message: 'req' was expected [sn_schema.network_contains_too_few] + Schema message: 'req' was expected [sn_schema_violation.network_contains_too_few] ''' # --- +# name: test_schemas[schema/fixtures/network-schemas_in_conf_error].1 + dict({ + 'validated_needs_count': 2, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + dict({ + 'need_id': 'SPEC_1', + 'warning': dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'type', + 'need_path': 'IMPL_1 > links > SPEC_1', + 'schema_path': '[0] > validate > network > links > contains > local > properties > type > const', + 'validation_msg': "'req' was expected", + }), + 'subtype': 'network_local_fail', + }), + }), + ]), + 'details': dict({ + 'need_path': 'IMPL_1 > links', + 'schema_path': '[0] > validate > network > links > contains', + 'severity': 'violation', + 'validation_msg': "Too few valid links of type 'links' (0 < 1) / nok: SPEC_1", + }), + 'log_lvl': 'error', + 'subtype': 'network_contains_too_few', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_unset_is_reported] ''' - WARNING: Need 'IMPL_1' has validation errors: + WARNING: Need 'IMPL_1' has schema infos: Severity: info Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_info.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_unset_is_reported].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'info', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_info', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_warning_not_reported] '' # --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_warning_not_reported].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'info', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_info', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/reporting-severity_rule_unset_min_unset_is_reported] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_unset_min_unset_is_reported].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/reporting-severity_rule_violation_min_unset_is_reported] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_violation_min_unset_is_reported].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'violation', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_unset_is_reported] ''' - WARNING: Need 'IMPL_1' has validation errors: + WARNING: Need 'IMPL_1' has schema warnings: Severity: warning Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_warning.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_unset_is_reported].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'warning', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_warning', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_violation_not_reported] - '' + ''' + WARNING: Need 'IMPL_1' has schema warnings: + Severity: warning + Field: asil + Need path: IMPL_1 + Schema path: [0] > local > properties > asil > const + Schema message: 'A' was expected [sn_schema_warning.local_fail] + + ''' +# --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_violation_not_reported].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'warning', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_warning', + }), + ]), + }), + }) # --- # name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_warning_is_reported] ''' - WARNING: Need 'IMPL_1' has validation errors: + WARNING: Need 'IMPL_1' has schema warnings: Severity: warning Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema.local_fail] + Schema message: 'A' was expected [sn_schema_warning.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_warning_is_reported].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > properties > asil > const', + 'severity': 'warning', + 'validation_msg': "'A' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_warning', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/unevaluated-unevaluated_allof_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('approved' was unexpected) [sn_schema.local_fail] + Schema message: Unevaluated properties are not allowed ('approved' was unexpected) [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/unevaluated-unevaluated_allof_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > unevaluatedProperties', + 'severity': 'violation', + 'validation_msg': "Unevaluated properties are not allowed ('approved' was unexpected)", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- # name: test_schemas[schema/fixtures/unevaluated-unevaluated_error] ''' - WARNING: Need 'IMPL_1' has validation errors: + ERROR: Need 'IMPL_1' has schema violations: Severity: violation - Field: Need path: IMPL_1 Schema path: [0] > local > unevaluatedProperties - Schema message: Unevaluated properties are not allowed ('comment' was unexpected) [sn_schema.local_fail] + Schema message: Unevaluated properties are not allowed ('comment' was unexpected) [sn_schema_violation.local_fail] ''' # --- +# name: test_schemas[schema/fixtures/unevaluated-unevaluated_error].1 + dict({ + 'validated_needs_count': 1, + 'validation_warnings': dict({ + 'IMPL_1': list([ + dict({ + 'children': list([ + ]), + 'details': dict({ + 'need_path': 'IMPL_1', + 'schema_path': '[0] > local > unevaluatedProperties', + 'severity': 'violation', + 'validation_msg': "Unevaluated properties are not allowed ('comment' was unexpected)", + }), + 'log_lvl': 'error', + 'subtype': 'local_fail', + 'type': 'sn_schema_violation', + }), + ]), + }), + }) +# --- diff --git a/tests/schema/fixtures/reporting.yml b/tests/schema/fixtures/reporting.yml index f4f659c2b..d7beafc58 100644 --- a/tests/schema/fixtures/reporting.yml +++ b/tests/schema/fixtures/reporting.yml @@ -90,10 +90,8 @@ severity_rule_info_min_warning_not_reported: extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" needs_schema_definitions_from_json = "schemas.json" + suppress_warnings = ["sn_schema_info"] ubproject: | - [needs] - schema_severity = "warning" - [[needs.extra_options]] name = "asil" rst: | diff --git a/tests/schema/test_schema.py b/tests/schema/test_schema.py index dcdc4303a..d4e810795 100644 --- a/tests/schema/test_schema.py +++ b/tests/schema/test_schema.py @@ -63,6 +63,14 @@ def test_schemas( str(app.srcdir) + os.path.sep, "/" ) assert warnings == snapshot + + schema_violations: dict[str, Any] = json.loads( + Path(app.outdir, "schema_violations.json").read_text("utf8") + ) + exclude_keys = {"validated_needs_per_second", "validation_summary"} + for key in exclude_keys: + schema_violations.pop(key, None) + assert schema_violations == snapshot app.cleanup() From ee2459e4bd177056905bdb01e20f7aad6c53760d Mon Sep 17 00:00:00 2001 From: Marco Heinemann Date: Wed, 22 Oct 2025 21:25:39 +0200 Subject: [PATCH 3/5] Fix reporting tests --- tests/schema/__snapshots__/test_schema.ambr | 94 +++++++++++---------- tests/schema/fixtures/reporting.yml | 51 ++++------- 2 files changed, 65 insertions(+), 80 deletions(-) diff --git a/tests/schema/__snapshots__/test_schema.ambr b/tests/schema/__snapshots__/test_schema.ambr index 7351953dd..335521b77 100644 --- a/tests/schema/__snapshots__/test_schema.ambr +++ b/tests/schema/__snapshots__/test_schema.ambr @@ -3729,18 +3729,18 @@ }), }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_unset_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity] ''' WARNING: Need 'IMPL_1' has schema infos: Severity: info Field: asil Need path: IMPL_1 - Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema_info.local_fail] + Schema path: [1] > local > properties > asil > const + Schema message: 'B' was expected [sn_schema_info.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_unset_is_reported].1 +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity].1 dict({ 'validated_needs_count': 1, 'validation_warnings': dict({ @@ -3752,34 +3752,22 @@ 'field': 'asil', 'need_path': 'IMPL_1', 'schema_path': '[0] > local > properties > asil > const', - 'severity': 'info', + 'severity': 'violation', 'validation_msg': "'A' was expected", }), - 'log_lvl': 'warning', + 'log_lvl': 'error', 'subtype': 'local_fail', - 'type': 'sn_schema_info', + 'type': 'sn_schema_violation', }), - ]), - }), - }) -# --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_warning_not_reported] - '' -# --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_info_min_warning_not_reported].1 - dict({ - 'validated_needs_count': 1, - 'validation_warnings': dict({ - 'IMPL_1': list([ dict({ 'children': list([ ]), 'details': dict({ 'field': 'asil', 'need_path': 'IMPL_1', - 'schema_path': '[0] > local > properties > asil > const', + 'schema_path': '[1] > local > properties > asil > const', 'severity': 'info', - 'validation_msg': "'A' was expected", + 'validation_msg': "'B' was expected", }), 'log_lvl': 'warning', 'subtype': 'local_fail', @@ -3789,18 +3777,18 @@ }), }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_unset_min_unset_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity_and_rule] ''' - ERROR: Need 'IMPL_1' has schema violations: - Severity: violation + WARNING: Need 'IMPL_1' has schema infos: + Severity: info Field: asil Need path: IMPL_1 - Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema_violation.local_fail] + Schema path: [1] > local > properties > asil > const + Schema message: 'B' was expected [sn_schema_info.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_unset_min_unset_is_reported].1 +# name: test_schemas[schema/fixtures/reporting-severity_default_suppress_severity_and_rule].1 dict({ 'validated_needs_count': 1, 'validation_warnings': dict({ @@ -3819,11 +3807,25 @@ 'subtype': 'local_fail', 'type': 'sn_schema_violation', }), + dict({ + 'children': list([ + ]), + 'details': dict({ + 'field': 'asil', + 'need_path': 'IMPL_1', + 'schema_path': '[1] > local > properties > asil > const', + 'severity': 'info', + 'validation_msg': "'B' was expected", + }), + 'log_lvl': 'warning', + 'subtype': 'local_fail', + 'type': 'sn_schema_info', + }), ]), }), }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_violation_min_unset_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_rule_default] ''' ERROR: Need 'IMPL_1' has schema violations: Severity: violation @@ -3834,7 +3836,7 @@ ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_violation_min_unset_is_reported].1 +# name: test_schemas[schema/fixtures/reporting-severity_rule_default].1 dict({ 'validated_needs_count': 1, 'validation_warnings': dict({ @@ -3857,18 +3859,18 @@ }), }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_unset_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_rule_info] ''' - WARNING: Need 'IMPL_1' has schema warnings: - Severity: warning + WARNING: Need 'IMPL_1' has schema infos: + Severity: info Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema_warning.local_fail] + Schema message: 'A' was expected [sn_schema_info.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_unset_is_reported].1 +# name: test_schemas[schema/fixtures/reporting-severity_rule_info].1 dict({ 'validated_needs_count': 1, 'validation_warnings': dict({ @@ -3880,29 +3882,29 @@ 'field': 'asil', 'need_path': 'IMPL_1', 'schema_path': '[0] > local > properties > asil > const', - 'severity': 'warning', + 'severity': 'info', 'validation_msg': "'A' was expected", }), 'log_lvl': 'warning', 'subtype': 'local_fail', - 'type': 'sn_schema_warning', + 'type': 'sn_schema_info', }), ]), }), }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_violation_not_reported] +# name: test_schemas[schema/fixtures/reporting-severity_rule_violation] ''' - WARNING: Need 'IMPL_1' has schema warnings: - Severity: warning + ERROR: Need 'IMPL_1' has schema violations: + Severity: violation Field: asil Need path: IMPL_1 Schema path: [0] > local > properties > asil > const - Schema message: 'A' was expected [sn_schema_warning.local_fail] + Schema message: 'A' was expected [sn_schema_violation.local_fail] ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_violation_not_reported].1 +# name: test_schemas[schema/fixtures/reporting-severity_rule_violation].1 dict({ 'validated_needs_count': 1, 'validation_warnings': dict({ @@ -3914,18 +3916,18 @@ 'field': 'asil', 'need_path': 'IMPL_1', 'schema_path': '[0] > local > properties > asil > const', - 'severity': 'warning', + 'severity': 'violation', 'validation_msg': "'A' was expected", }), - 'log_lvl': 'warning', + 'log_lvl': 'error', 'subtype': 'local_fail', - 'type': 'sn_schema_warning', + 'type': 'sn_schema_violation', }), ]), }), }) # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_warning_is_reported] +# name: test_schemas[schema/fixtures/reporting-severity_rule_warning] ''' WARNING: Need 'IMPL_1' has schema warnings: Severity: warning @@ -3936,7 +3938,7 @@ ''' # --- -# name: test_schemas[schema/fixtures/reporting-severity_rule_warning_min_warning_is_reported].1 +# name: test_schemas[schema/fixtures/reporting-severity_rule_warning].1 dict({ 'validated_needs_count': 1, 'validation_warnings': dict({ diff --git a/tests/schema/fixtures/reporting.yml b/tests/schema/fixtures/reporting.yml index d7beafc58..a36dcc9f3 100644 --- a/tests/schema/fixtures/reporting.yml +++ b/tests/schema/fixtures/reporting.yml @@ -1,4 +1,4 @@ -severity_rule_unset_min_unset_is_reported: +severity_rule_default: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -19,7 +19,7 @@ severity_rule_unset_min_unset_is_reported: asil: const: A -severity_rule_info_min_unset_is_reported: +severity_rule_info: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -41,7 +41,7 @@ severity_rule_info_min_unset_is_reported: asil: const: A -severity_rule_warning_min_unset_is_reported: +severity_rule_warning: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -63,7 +63,7 @@ severity_rule_warning_min_unset_is_reported: asil: const: A -severity_rule_violation_min_unset_is_reported: +severity_rule_violation: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" @@ -85,12 +85,12 @@ severity_rule_violation_min_unset_is_reported: asil: const: A -severity_rule_info_min_warning_not_reported: +severity_default_suppress_severity: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" needs_schema_definitions_from_json = "schemas.json" - suppress_warnings = ["sn_schema_info"] + suppress_warnings = ["sn_schema_violation"] ubproject: | [[needs.extra_options]] name = "asil" @@ -101,22 +101,25 @@ severity_rule_info_min_warning_not_reported: schemas: $defs: [] schemas: + - validate: + local: + properties: + asil: + const: A - severity: "info" validate: local: properties: asil: - const: A + const: B -severity_rule_warning_min_warning_is_reported: +severity_default_suppress_severity_and_rule: conf: | extensions = ["sphinx_needs"] needs_from_toml = "ubproject.toml" needs_schema_definitions_from_json = "schemas.json" + suppress_warnings = ["sn_schema_violation.local_fail"] ubproject: | - [needs] - schema_severity = "warning" - [[needs.extra_options]] name = "asil" rst: | @@ -126,34 +129,14 @@ severity_rule_warning_min_warning_is_reported: schemas: $defs: [] schemas: - - severity: "warning" - validate: + - validate: local: properties: asil: const: A - -severity_rule_warning_min_violation_not_reported: - conf: | - extensions = ["sphinx_needs"] - needs_from_toml = "ubproject.toml" - needs_schema_definitions_from_json = "schemas.json" - ubproject: | - [needs] - schema_severity = "violation" - - [[needs.extra_options]] - name = "asil" - rst: | - .. impl:: title - :id: IMPL_1 - :asil: QM - schemas: - $defs: [] - schemas: - - severity: "warning" + - severity: "info" validate: local: properties: asil: - const: A + const: B From 94f378843d4e07822e1752a3e2cd41b08822c2f6 Mon Sep 17 00:00:00 2001 From: Marco Heinemann Date: Wed, 22 Oct 2025 21:40:59 +0200 Subject: [PATCH 4/5] Fix Sphinx 7.4 --- sphinx_needs/logging.py | 29 +++++++++++++++++++++++++++++ sphinx_needs/schema/process.py | 10 +++++++--- 2 files changed, 36 insertions(+), 3 deletions(-) diff --git a/sphinx_needs/logging.py b/sphinx_needs/logging.py index 100fd9a6c..3239ff889 100644 --- a/sphinx_needs/logging.py +++ b/sphinx_needs/logging.py @@ -126,3 +126,32 @@ def log_warning( color=color, once=once, ) + + +def log_error( + logger: SphinxLoggerAdapter, + message: str, + subtype: WarningSubTypes, + /, + location: str | tuple[str | None, int | None] | Node | None, + *, + color: str | None = None, + once: bool = False, + type: str = "needs", +) -> None: + # Since sphinx in v7.3, sphinx will show warning types if `show_warning_types=True` is set, + # and in v8.0 this was made the default. + if version_info < (8,): + if subtype: + message += f" [{type}.{subtype}]" + else: + message += f" [{type}]" + + logger.error( + message, + type=type, + subtype=subtype, + location=location, + color=color, + once=once, + ) diff --git a/sphinx_needs/schema/process.py b/sphinx_needs/schema/process.py index fbf6f9085..340b24cb3 100644 --- a/sphinx_needs/schema/process.py +++ b/sphinx_needs/schema/process.py @@ -8,7 +8,7 @@ from sphinx_needs.api import get_needs_view from sphinx_needs.config import NeedsSphinxConfig from sphinx_needs.data import SphinxNeedsData -from sphinx_needs.logging import log_warning +from sphinx_needs.logging import log_error, log_warning from sphinx_needs.needsfile import generate_needs_schema from sphinx_needs.schema.config import SchemasRootType from sphinx_needs.schema.core import ( @@ -95,8 +95,12 @@ def process_schemas(app: Sphinx, builder: Builder) -> None: type=warning["type"], ) elif warning["log_lvl"] == "error": - logger.error( - warning["message"], type=warning["type"], subtype=warning["subtype"] + log_error( + logger, + warning["message"], + warning["subtype"], # type: ignore[arg-type] + None, + type=warning["type"], ) duration = end_time - start_time From 22a8d8cb35ea436731e260003e14755fc867daab Mon Sep 17 00:00:00 2001 From: Marco Heinemann Date: Fri, 24 Oct 2025 10:11:09 +0200 Subject: [PATCH 5/5] Improve docs --- docs/schema/index.rst | 150 ++++++++++++++++++++++++------------------ 1 file changed, 87 insertions(+), 63 deletions(-) diff --git a/docs/schema/index.rst b/docs/schema/index.rst index 81f6f6ee8..25314f241 100644 --- a/docs/schema/index.rst +++ b/docs/schema/index.rst @@ -723,45 +723,58 @@ error, even though ``priority`` is in the ``required`` list. Rule severity ~~~~~~~~~~~~~ -Each schema rule can specify one of the following severity levels: +Each schema rule can specify one of the severity levels ``info``, ``warning``, or +``violation`` as described in :ref:`severity_levels`. If not given, ``violation`` is used as +a default. -- ``info``: A non-critical constraint violation indicating an informative message. -- ``warning``: A non-critical constraint violation indicating a warning. -- ``violation`` : A constraint violation. - - If that is not given, a violation +The severity influences how validation failures are displayed in the console. -rule specific default kicks in. The severity influences how validation failures are displayed -in the console: - -.. code-block:: python +.. code-block:: json { "severity": "warning", - "message": "Approval required due to high efforts" + "select": { + "properties": { "type": { "const": "feat" } } + }, + "validate": { + "local": { + "properties": { "id": { "pattern": "^FEAT_[a-zA-Z0-9_-]*$" } } + } + } + }, + { + "select": { + "properties": { "type": { "const": "spec" } } + }, + "validate": { + "local": { + "properties": { "id": { "pattern": "^SPEC_[a-zA-Z0-9_-]*$" } } + } + } } -All severities are reported during schema validation. - -**Console output example:** +**Console output for above example:** .. code-block:: text - WARNING: Need 'FEAt' has validation warnings: + WARNING: Need 'FEAT' has schema warnings: Severity: warning Field: id - Need path: FEAt + Need path: FEAT Schema path: [0] > local > properties > id > pattern - User message: id must be uppercase with numbers and underscores - Schema message: 'FEAt' does not match '^[A-Z0-9_]+$' [sn_schema_warning.local_fail] + Schema message: 'FEAT' does not match '^FEAT_[a-zA-Z0-9_-]*$' [sn_schema_warning.local_fail] - ERROR: Need 'SPEC' has validation errors: + ERROR: Need 'SPEC' has schema violations: Severity: violation Field: id Need path: SPEC - Schema path: spec[1] > local > properties > id > pattern + Schema path: [1] > local > properties > id > pattern Schema message: 'SPEC' does not match '^SPEC_[a-zA-Z0-9_-]*$' [sn_schema_violation.local_fail] +Note how the ``FEAT`` report has a severity ``warning`` while ``SPEC`` is reported as ``violation``. +This is because the first schema explicitly set the severity to ``warning``, while the second +schema used the default ``violation``. + .. _`schema_reuse`: Schema definitions ($defs) @@ -1038,6 +1051,60 @@ Running validation and understanding errors After defining your schema configuration, running Sphinx will validate all needs against the defined schemas. This section explains the validation process and how to interpret errors. +Error messages and output +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Validation errors include detailed information: + +- **Severity**: The :ref:`severity level ` of the violation +- **Field**: The specific field that failed validation +- **Need path**: The ID of the need that failed or the link chain for network validation. + Link chains are represented as in ``IMPL_SAFE > links > SPEC_SAFE > specifies > FEAT_SAFE``, so + need IDs and link types form the chain, separated by ``>``. +- **Schema path**: The JSON path within the schema that was violated. The path starts with an + identifier built from the :ref:`schema rule id ` and the 0-based index + of the rule in the list. Examples:: + + [0] > local > properties > id > pattern + spec[1] > local > properties > id > pattern + + The first example has no schema rule ID defined, so only the index is used. + +- **User message**: Optional user-friendly :ref:`message ` set in the + schema rules. +- **Schema message**: Detailed technical validation message from the validator + +Example error output:: + + Need 'SPEC_P01' has validation errors: + Severity: violation + Field: id + Need path: SPEC_P01 + Schema path: spec[1] > local > properties > id > pattern + Schema message: 'SPEC_P01' does not match '^REQ[a-zA-Z0-9_-]*$' + +For nested network validation, it can be difficult to determine which constraint and need +caused the error in the chain. In such cases, the error will emit details about the failed +need and the specific link that caused the issue:: + + WARNING: Need 'IMPL_SAFE' has validation errors: + Severity: violation + Need path: IMPL_SAFE > links + Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links + User message: Safe impl links to safe spec links to safe req + Schema message: Too few valid links of type 'links' (0 < 1) / nok: SPEC_SAFE + + Details for SPEC_SAFE + Need path: IMPL_SAFE > links > SPEC_SAFE > links + Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > links > validate > network > links + Schema message: Too few valid links of type 'links' (0 < 1) / nok: REQ_UNSAFE + + Details for REQ_UNSAFE + Field: asil + Need path: IMPL_SAFE > links > SPEC_SAFE > links > REQ_UNSAFE + Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > links > links > local > allOf > 0 > properties > asil > enum + Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] + .. _`severity_handling`: Severity handling @@ -1100,49 +1167,6 @@ Network validation: If not overridden in the :ref:`schema rules `, a default severity of ``violation`` is used for all message types. -Error messages and output -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Validation errors include detailed information: - -- **Severity**: The severity level of the violation -- **Field**: The specific field that failed validation -- **Need path**: The ID of the need that failed or the link chain for network validation -- **Schema path**: The JSON path within the schema that was violated -- **User message**: Custom message from each of the ``schemas`` in :ref:`needs_schema_definitions` -- **Schema message**: Detailed technical validation message from the validator - -Example error output:: - - Need 'SPEC_P01' has validation errors: - Severity: violation - Field: id - Need path: SPEC_P01 - Schema path: spec[1] > local > properties > id > pattern - Schema message: 'SPEC_P01' does not match '^REQ[a-zA-Z0-9_-]*$' - -For nested network validation, it can be difficult to determine which constraint and need -caused the error in the chain. In such cases, the error will emit details about the failed -need and the specific link that caused the issue:: - - WARNING: Need 'IMPL_SAFE' has validation errors: - Severity: violation - Need path: IMPL_SAFE > links - Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > validate > network > links - User message: Safe impl links to safe spec links to safe req - Schema message: Too few valid links of type 'links' (0 < 1) / nok: SPEC_SAFE - - Details for SPEC_SAFE - Need path: IMPL_SAFE > links > SPEC_SAFE > links - Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > links > validate > network > links - Schema message: Too few valid links of type 'links' (0 < 1) / nok: REQ_UNSAFE - - Details for REQ_UNSAFE - Field: asil - Need path: IMPL_SAFE > links > SPEC_SAFE > links > REQ_UNSAFE - Schema path: safe-impl-[links]->safe-spec-[links]->safe-req[0] > links > links > local > allOf > 0 > properties > asil > enum - Schema message: 'QM' is not one of ['A', 'B', 'C', 'D'] [sn_schema.network_contains_too_few] - .. _`suppress_validation_messages`: Suppressing validation messages