[Bug] Integration Validation Missing Dataset Specific Schemas

[Mikaayenson]

Describe the Bug

When we're validating eql queries based on integrations, we:

1. Loop through each dataset schema
2. try / fail to see if the query successfully parsed with the schema
3. review failures to see if a field failed when it should have passed since the field exists in another schema required by the rule.

However, it did not completely account for nuances where some eql queries are sequences and may have different datasets per subquery.

To Reproduce

1. pull the branch from [New] Microsoft Entra ID Protection Alert and Device Registration #4688
2. run view-rule on the rule
3. see the validation fail

Expected Behavior

• Parsing queries like the one in the PR should pass (as the do on the SIEM EQL).
• I believe we need to modify how we store the package_schemas.

diff --git a/detection_rules/rule_validators.py b/detection_rules/rule_validators.py
index 5fe957ec1..f40e81d8d 100644
--- a/detection_rules/rule_validators.py
+++ b/detection_rules/rule_validators.py
@@ -464,6 +464,9 @@ class EQLValidator(QueryValidator):
 
             # add endpoint schema fields for multi-line fields
             integration_schema.update(**ecs.flatten(ecs.get_endpoint_schemas()))
+            if integration:
+                package_schemas[package][integration].update(**integration_schema)
             package_schemas[package].update(**integration_schema)
 
             eql_schema = ecs.KqlSchema2Eql(integration_schema)

• We then need to double check if we should conditionally set package_schemas[package].update(**integration_schema) or if it needs to be part of the else statement. This will most likely break other rules, and require further investigation to see if we need to tune those rules (or also set the root package_schemas[package] field.

Screenshots

No response

Desktop - OS

None

Desktop - Version

No response

Additional Context

We could consider larger refactors if it makes sense after digging deeper (to handle sequences vs flat queries different). For example, for sequences, we could validate the subquery fields against the schema associated to the event.dataset.