Skip to content

feat: Reset of aggregate data exchange values on target server#22841

Closed
jason-p-pickering wants to merge 15 commits intomasterfrom
reset-adex
Closed

feat: Reset of aggregate data exchange values on target server#22841
jason-p-pickering wants to merge 15 commits intomasterfrom
reset-adex

Conversation

@jason-p-pickering
Copy link
Contributor

@jason-p-pickering jason-p-pickering commented Jan 26, 2026

Summary

Added a new API endpoint to let clients explicitly purge/reset target data for an aggregate data exchange. The endpoint builds a synthetic delete payload from the exchange definition (dx/pe/ou, output ID schemes, COC/AOC export metadata), sends it with ImportStrategy.NEW_UPDATE with data values being deleted on the target server. The payload consists of all possible combinations of data dimensions (DX/PE/OU) which have been defined by the requests of the exchange.

Reset is best‑effort based on the current exchange definition and source metadata. It does not "time travel" to delete values that were previously sent for dimension items that no longer exist on the source (e.g., an org unit that has since been deleted). If an org unit (or other dimension item) is no longer resolvable on the source, it won't appear in the reset payload and therefore won't be deleted on the target.

Usage

The reset endpoint is exposed as:

POST /api/aggregateDataExchanges/{uid}/reset

Where {uid} is the UID of the aggregate data exchange to reset.

A flag is also available on the exchange request definition to automatically reset before each exchange run:

"target": {
    "request": {
        "resetBeforeExchange": true
    }
}

When this flag is set to true, a reset operation will be performed automatically before each exchange run.

This feature is meant to address the common "stale data" scenario:

  1. On Monday the source sends a value for a given DE/OU/COC/PE/AOC via aggregate data exchange
  2. On Tuesday that value is deleted on the source
  3. On Wednesday, a new data exchange is performed. In order to remove the "stale" data on the target which no longer exists on the source, a reset is performed prior to sending new data values. The reset consists of a data value set payload of all possible values defined by the scope of the data exchange request(s). This has the effect of deleting all previous data which was sent by the exchange.
  4. A fresh exchange then re‑publishes the data to the target server, which now reflects the current state of the source system.

Note that a "reset" is not the same the "DELETE" strategy for data exchange. DELETE strategy only deletes values that are explicitly sent in the exchange payload. The DELETE payload is built based on data values which are present in the analytics of the source system. If a value has been deleted on the source system, it will not appear in analytics, and therefore will not be sent in the DELETE payload. Thus, DELETE strategy does not address the stale data scenario described above. A "reset" builds a synthetic payload of all possible combinations of DE/PE/OU/COC/AOC defined by the exchange, and deletes all those values on the target.

Limitations

Metadata changes on the source: Reset is based on the current exchange definition and resolvable metadata; if the exchange is modified to remove a dimension item (e.g., an OU/DE/COC/AOC is removed from the request, or deleted on the source), values previously sent for that item won't be in the reset payload and therefore won't be cleared on the target.

Data which has been updated on the target server
We assume that the source system is the sole source of data for a given block of DE/PE/OU/COC/AOC data items. If data has been subsequently updated on the server, and should not be deleted for some reason, other means (such as data approval) should be used to guard against updates from clients (including data exchange).

AI Disclaimer: Portions of this PR have been developed using AI agents.

@jason-p-pickering jason-p-pickering changed the title feat: Endpoint for reset of aggregate data exchange feat:Reset of aggregate data exchange values on target server Jan 27, 2026
@jbee jbee marked this pull request as draft January 27, 2026 07:35
@larshelge larshelge changed the title feat:Reset of aggregate data exchange values on target server feat: Reset of aggregate data exchange values on target server Jan 28, 2026
@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 4, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant