Skip to content

ref(lw-deletions): Add partition_column to DeletionSettings #7765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
onewland merged 3 commits into from
Feb 25, 2026
+14 −6
Merged

ref(lw-deletions): Add partition_column to DeletionSettings #7765

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
46f4f2b
ref(lw-deletions): Add partition_column to DeletionSettings
onewland Feb 24, 2026
b50f961
fix(config): Fix partition_format for eap_items and search_issues sto…
onewland Feb 24, 2026
aa2de2c
fix(config): Use client_timestamp for search_issues partition_column
onewland Feb 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...atasets/configuration/events_analytics_platform/storages/eap_item_co_occurring_attrs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,5 @@ schema:
]
local_table_name: eap_item_co_occurring_attrs_1_local
dist_table_name: eap_item_co_occurring_attrs_1_dist
partition_format: [date]
partition_format: [retention_days, date]
allocation_policies: []
3 changes: 2 additions & 1 deletion snuba/datasets/configuration/events_analytics_platform/storages/eap_items.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ schema:
]
local_table_name: eap_items_1_local
dist_table_name: eap_items_1_dist
partition_format: [date]
partition_format: [retention_days, date]

query_processors:
- processor: UniqInSelectAndHavingProcessor
Expand Down Expand Up @@ -138,6 +138,7 @@ deletion_settings:
is_enabled: 1
max_rows_to_delete: 2000000
bulk_delete_only: true
partition_column: timestamp
tables:
- eap_items_1_local
- eap_items_1_downsample_8_local
Expand Down
2 changes: 1 addition & 1 deletion ...a/datasets/configuration/events_analytics_platform/storages/eap_items_downsample_512.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ schema:
]
local_table_name: eap_items_1_downsample_512_local
dist_table_name: eap_items_1_downsample_512_dist
partition_format: [date]
partition_format: [retention_days, date]

query_processors:
- processor: UniqInSelectAndHavingProcessor
Expand Down
2 changes: 1 addition & 1 deletion snuba/datasets/configuration/events_analytics_platform/storages/eap_items_downsample_64.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ schema:
]
local_table_name: eap_items_1_downsample_64_local
dist_table_name: eap_items_1_downsample_64_dist
partition_format: [date]
partition_format: [retention_days, date]

query_processors:
- processor: UniqInSelectAndHavingProcessor
Expand Down
2 changes: 1 addition & 1 deletion snuba/datasets/configuration/events_analytics_platform/storages/eap_items_downsample_8.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ schema:
]
local_table_name: eap_items_1_downsample_8_local
dist_table_name: eap_items_1_downsample_8_dist
partition_format: [date]
partition_format: [retention_days, date]

query_processors:
- processor: UniqInSelectAndHavingProcessor
Expand Down
2 changes: 2 additions & 0 deletions snuba/datasets/configuration/issues/storages/search_issues.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ schema:
]
local_table_name: search_issues_local_v2
dist_table_name: search_issues_dist_v2
partition_format: [retention_days, date]

allocation_policies:
- name: ConcurrentRateLimitAllocationPolicy
Expand Down Expand Up @@ -143,6 +144,7 @@ query_processors:
deletion_settings:
is_enabled: 1
max_rows_to_delete: 2000000
partition_column: client_timestamp
tables:
- search_issues_local_v2
allowed_columns:
Expand Down
4 changes: 4 additions & 0 deletions snuba/datasets/configuration/json_schema.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -622,6 +622,10 @@ def registered_class_array_schema(
"items": {"type": "string"},
},
},
"partition_column": {
"type": "string",
"description": "Column used for partition splitting in lightweight deletes. When set, deletes can be split by toMonday(partition_column) to reduce per-mutation CPU.",
},
},
"required": ["is_enabled", "tables"],
"additionalProperties": False,
Expand Down
3 changes: 2 additions & 1 deletion snuba/datasets/deletion_settings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import annotations

from dataclasses import dataclass, field
from typing import Dict, List, Sequence
from typing import Dict, List, Optional, Sequence

from sentry_protos.snuba.v1.request_common_pb2 import TraceItemType

Expand All @@ -16,6 +16,7 @@ class DeletionSettings:
allowed_columns: Sequence[str] = field(default_factory=list)
max_rows_to_delete: int = MAX_ROWS_TO_DELETE_DEFAULT
allowed_attributes_by_item_type: Dict[str, List[str]] = field(default_factory=dict)
partition_column: Optional[str] = None


def get_trace_item_type_name(item_type: int) -> str:
Expand Down
Loading