[google_secops] Add Alert v2 Data Stream and Deprecate Alert Data Stream by mohitjha-elastic · Pull Request #20258 · elastic/integrations

mohitjha-elastic · 2026-07-21T13:38:16Z

Proposed commit message

google_secops: add alert v2 data stream and deprecate existing alert data stream

Add a new `alerts_v2` data stream that uses the Chronicle `legacySearchDetections` 
API(`chronicle.googleapis.com`) for detection collection.
Deprecate the existing `alerts` data stream, which depended on the legacy Backstory 
API (`backstory.googleapis.com`) that is no longer available for new or recently updated
Google Cloud projects.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Clone integrations repo.
Install the elastic package locally.
Start the elastic stack using the elastic package.
Move to integrations/packages/google_secops directory.
Run the following command to run tests.

elastic-package test -v

Related Issues

Closes [google_secops] 403 Forbidden: Legacy Backstory API endpoint is deprecated #17640

infra-vault-gh-plugin-prod · 2026-07-21T13:38:20Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

github-actions · 2026-07-21T13:40:15Z

Elastic Docs Style Checker (Vale)

Summary: 52 warnings, 3 suggestions found

⚠️ Warnings (52): Fix when the suggestion improves clarity or correctness.

File	Line	Rule	Message
packages/google_secops/data_stream/alert_v2/fields/fields.yml	87	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	87	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	108	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	183	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	183	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	261	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	261	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	261	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	321	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	321	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	324	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	336	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	366	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	366	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	466	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	505	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	538	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	571	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	604	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	610	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	616	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	622	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	628	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	634	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	676	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	682	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	691	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	694	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	712	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	712	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	737	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	758	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	761	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	805	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	808	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	827	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	837	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	849	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	858	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	858	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	909	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	918	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	930	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	990	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	990	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1045	Elastic.DontUse	Don't use 'Please'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1051	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1054	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1057	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1100	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1103	Elastic.Latinisms	Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1115	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.

💡 Suggestions (3): Optional style improvements. Apply when helpful.

File	Line	Rule	Message
packages/google_secops/data_stream/alert_v2/fields/fields.yml	16	Elastic.WordChoice	Consider using 'deactivated, deselected, hidden, turned off, unavailable' instead of 'disabled', unless the term is in the UI.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	691	Elastic.Ellipses	In general, don't use an ellipsis.
packages/google_secops/data_stream/alert_v2/fields/fields.yml	1051	Elastic.Ellipses	In general, don't use an ellipsis.

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

elastic-vault-github-plugin-prod · 2026-07-21T13:43:48Z

✅ All changelog entries have the correct PR link.

elastic-vault-github-plugin-prod · 2026-07-21T13:59:35Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

infra-vault-gh-plugin-prod · 2026-07-21T13:59:39Z

💚 Build Succeeded

Buildkite Build
Commit: 80ea385

cc @mohitjha-elastic

vera-review-bot · 2026-07-21T14:11:06Z

+              description: Outcome variables from the detection rule, keyed by outcome name.
+            - name: risk_score
+              type: long
+            - name: risk_score


Severity: 🟠 High confidence: high path: packages/google_secops/data_stream/alert_v2/fields/fields.yml:28

detection.risk_score is defined twice with conflicting types (long then group) in fields.yml; remove the spurious group block so the field is only declared once as long.

Details

Within the detection group, risk_score is declared twice: first as a scalar type: long (line 26-27) and immediately again as an object type: group with int64_val/type/value sub-fields (line 28-36). A field cannot be both a scalar long leaf and an object, so this is a mapping conflict and a duplicate field definition that elastic-package check rejects. The group block is a copy of the detection.variables.risk_score structure defined later in the same file (line 55-66); nothing in the ingest pipeline writes detection.risk_score.int64_val/type/value — the pipeline converts the scalar detection.*.risk_score to long (tag convert_detection_risk_score_to_long) and later removes it, and the expected test output shows detection.risk_score only ever as a scalar. The object variant is therefore both invalid and unused.

Recommendation:

Keep the single scalar declaration and delete the duplicate group block:

- name: risk_score type: long - name: rule_id type: keyword description: "Identifier for the rule generating the detection."

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

vera-review-bot · 2026-07-21T14:11:08Z

Review summary

Issues found across the latest commits 80ea385 — 1 high

🟠 detection.risk_score is defined twice with conflicting types (long then group) in fields.yml (link) (Unresolved)

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

efd6 · 2026-07-22T23:23:02Z

There are a large number of uninformative fields in the test input, for example the large array of {"string": "string"} objects, but not limited to that; if the field is not the basis for logic, it does not need to be tested multiple times and if it's an array, unless the length is the subject of a logical decision, it does not need to be long. Can we reduce the inputs to just include test-informative fields.

efd6 · 2026-07-22T23:26:29Z

+            ?"next_page": has(body.nextPageToken) ?
+              optional.of({
+                "token": body.nextPageToken,
+              })
+            :
+              optional.none(),


Suggested change

?"next_page": has(body.nextPageToken) ?

optional.of({

"token": body.nextPageToken,

})

:

optional.none(),

?"next_page": body.?nextPageToken.optMap(t, {"token": t}),

efd6 · 2026-07-22T23:28:05Z

+        "startTime": [string(state.cursor.start_time)],
+        "endTime": [string(state.cursor.end_time)],


Suggested change

"startTime": [string(state.cursor.start_time)],

"endTime": [string(state.cursor.end_time)],

"startTime": [state.cursor.start_time],

"endTime": [state.cursor.end_time],

These are already strings.

efd6 · 2026-07-22T23:29:24Z

+                d.?collectionElements.orValue([]).map(ce,
+                  ce.?references.orValue([]).map(result, {


Suggested change

d.?collectionElements.orValue([]).map(ce,

ce.?references.orValue([]).map(result, {

d.?collectionElements.orValue([]).map(ce, has(ce.references),

ce.references.map(result, {

efd6 · 2026-07-22T23:29:42Z

+        "events": {
+          "error": {
+            "code": string(resp.StatusCode),
+            "id": string(resp.Status),


Suggested change

"id": string(resp.Status),

"id": resp.Status,

efd6 · 2026-07-22T23:29:50Z

+              size(resp.Body) != 0 ?
+                string(resp.Body)
+              :
+                string(resp.Status) + ' (' + string(resp.StatusCode) + ')'


Suggested change

string(resp.Status) + ' (' + string(resp.StatusCode) + ')'

resp.Status + ' (' + string(resp.StatusCode) + ')'

efd6 · 2026-07-22T23:35:25Z

+              if (list["severity"].toUpperCase() == 'CRITICAL') {
+                ctx.event.severity = 99
+              } else if (list["severity"].toUpperCase() == 'ERROR') {
+                ctx.event.severity = 99
+              } else if (list["severity"].toUpperCase() == 'HIGH') {
+                ctx.event.severity = 73
+              } else if (list["severity"].toUpperCase() == 'INFORMATIONAL') {
+                ctx.event.severity = 21
+              }  else if (list["severity"].toUpperCase() == 'LOW') {
+                ctx.event.severity = 21
+              } else if (list["severity"].toUpperCase() == 'MEDIUM') {
+                ctx.event.severity = 47
+              } else if (list["severity"].toUpperCase() == 'NONE') {
+                ctx.event.severity = 21
+              } else if (list["severity"].toUpperCase() == 'UNKNOWN_SEVERITY') {


Use case-insensitive comparison; equalsIgnoreCase.

Suggested change

if (list["severity"].toUpperCase() == 'CRITICAL') {

ctx.event.severity = 99

} else if (list["severity"].toUpperCase() == 'ERROR') {

ctx.event.severity = 99

} else if (list["severity"].toUpperCase() == 'HIGH') {

ctx.event.severity = 73

} else if (list["severity"].toUpperCase() == 'INFORMATIONAL') {

ctx.event.severity = 21

} else if (list["severity"].toUpperCase() == 'LOW') {

ctx.event.severity = 21

} else if (list["severity"].toUpperCase() == 'MEDIUM') {

ctx.event.severity = 47

} else if (list["severity"].toUpperCase() == 'NONE') {

ctx.event.severity = 21

} else if (list["severity"].toUpperCase() == 'UNKNOWN_SEVERITY') {

if (list["severity"].equalsIgnoreCase('critical')) {

ctx.event.severity = 99

} else if (list["severity"].equalsIgnoreCase('error')) {

ctx.event.severity = 99

} else if (list["severity"].equalsIgnoreCase('high')) {

ctx.event.severity = 73

} else if (list["severity"].equalsIgnoreCase('informational')) {

ctx.event.severity = 21

} else if (list["severity"].equalsIgnoreCase('low')) {

ctx.event.severity = 21

} else if (list["severity"].equalsIgnoreCase('medium')) {

ctx.event.severity = 47

} else if (list["severity"].equalsIgnoreCase('none')) {

ctx.event.severity = 21

} else if (list["severity"].equalsIgnoreCase('unknown_severity')) {

efd6 · 2026-07-22T23:44:17Z

+                    "y": 58
+                },
+                "panelIndex": "0c26cc51-158d-48f2-bb23-8f996391fb3d",
+                "title": "Top 10 Destination IP ",


Suggested change

"title": "Top 10 Destination IP ",

"title": "Top 10 Destination IP",

efd6 · 2026-07-22T23:45:24Z

+                },
+                "panelIndex": "22679e22-be09-4425-9b59-4f26d23fc230",
+                "panelRefName": "panel_22679e22-be09-4425-9b59-4f26d23fc230",
+                "title": "Rule Essential Details [Logs Google SecOps]",


Suggested change

"title": "Rule Essential Details [Logs Google SecOps]",

"title": "Rule Essential Details",

Maybe do a clean up of these throughout.

Add alert v2 data stream and deprecate alert data stream

be4308b

mohitjha-elastic self-assigned this Jul 21, 2026

mohitjha-elastic requested review from a team as code owners July 21, 2026 13:38

Update changelog eentry

80ea385

vera-review-bot Bot reviewed Jul 21, 2026

View reviewed changes

efd6 reviewed Jul 22, 2026

View reviewed changes

		"startTime": [string(state.cursor.start_time)],
		"endTime": [string(state.cursor.end_time)],

		d.?collectionElements.orValue([]).map(ce,
		ce.?references.orValue([]).map(result, {

	string(resp.Status) + ' (' + string(resp.StatusCode) + ')'
	resp.Status + ' (' + string(resp.StatusCode) + ')'

	"title": "Top 10 Destination IP ",
	"title": "Top 10 Destination IP",

	"title": "Rule Essential Details [Logs Google SecOps]",
	"title": "Rule Essential Details",

Uh oh!

Conversation

mohitjha-elastic commented Jul 21, 2026

Proposed commit message

Checklist

How to test this PR locally

Related Issues

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026

Elastic Docs Style Checker (Vale)

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jul 21, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Jul 21, 2026

🚀 Benchmarks report

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jul 21, 2026

💚 Build Succeeded

Uh oh!

vera-review-bot Bot Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

vera-review-bot Bot commented Jul 21, 2026

Review summary

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants