@@ -9284,6 +9284,15 @@ components:
92849284 that evaluates to a boolean.
92859285 example: ${true}
92869286 type: string
9287+ ComponentRecommendation:
9288+ description: Resource recommendation for a single Spark component (driver or
9289+ executor). Contains estimation data used to patch Spark job specs.
9290+ properties:
9291+ estimation:
9292+ $ref: '#/components/schemas/Estimation'
9293+ required:
9294+ - estimation
9295+ type: object
92879296 ComponentType:
92889297 description: The UI component type.
92899298 enum:
@@ -10480,6 +10489,28 @@ components:
1048010489 type: string
1048110490 x-enum-varnames:
1048210491 - COST_BY_ORG
10492+ Cpu:
10493+ description: CPU usage statistics derived from historical Spark job metrics.
10494+ Provides multiple estimates so users can choose between conservative and cost-saving
10495+ risk profiles.
10496+ properties:
10497+ max:
10498+ description: Maximum CPU usage observed for the job, expressed in millicores.
10499+ This represents the upper bound of usage.
10500+ format: int64
10501+ type: integer
10502+ p75:
10503+ description: 75th percentile of CPU usage (millicores). Represents a cost-saving
10504+ configuration while covering most workloads.
10505+ format: int64
10506+ type: integer
10507+ p95:
10508+ description: 95th percentile of CPU usage (millicores). Balances performance
10509+ and cost, providing a safer margin than p75.
10510+ format: int64
10511+ type: integer
10512+ type: object
10513+ x-model-simple-name: SpaCpu
1048310514 CreateActionConnectionRequest:
1048410515 description: Request used to create an action connection.
1048510516 properties:
@@ -15937,6 +15968,33 @@ components:
1593715968 type: string
1593815969 x-enum-varnames:
1593915970 - ESCALATION_POLICY_STEPS
15971+ Estimation:
15972+ description: Recommended resource values for a Spark driver or executor, derived
15973+ from recent real usage metrics. Used by SPA to propose more efficient pod
15974+ sizing.
15975+ properties:
15976+ cpu:
15977+ $ref: '#/components/schemas/Cpu'
15978+ ephemeral_storage:
15979+ description: Recommended ephemeral storage allocation (in MiB). Derived
15980+ from job temporary storage patterns.
15981+ format: int64
15982+ type: integer
15983+ heap:
15984+ description: Recommended JVM heap size (in MiB).
15985+ format: int64
15986+ type: integer
15987+ memory:
15988+ description: Recommended total memory allocation (in MiB). Includes both
15989+ heap and overhead.
15990+ format: int64
15991+ type: integer
15992+ overhead:
15993+ description: Recommended JVM overhead (in MiB). Computed as total memory
15994+ - heap.
15995+ format: int64
15996+ type: integer
15997+ type: object
1594015998 Event:
1594115999 description: The metadata associated with a request.
1594216000 properties:
@@ -32835,6 +32893,52 @@ components:
3283532893 x-enum-varnames:
3283632894 - ANY
3283732895 - ALL
32896+ RecommendationAttributes:
32897+ description: Attributes of the SPA Recommendation resource. Contains recommendations
32898+ for both driver and executor components.
32899+ properties:
32900+ driver:
32901+ $ref: '#/components/schemas/ComponentRecommendation'
32902+ executor:
32903+ $ref: '#/components/schemas/ComponentRecommendation'
32904+ required:
32905+ - driver
32906+ - executor
32907+ type: object
32908+ RecommendationData:
32909+ description: JSON:API resource object for SPA Recommendation. Includes type,
32910+ optional ID, and resource attributes with structured recommendations.
32911+ properties:
32912+ attributes:
32913+ $ref: '#/components/schemas/RecommendationAttributes'
32914+ id:
32915+ description: Resource identifier for the recommendation. Optional in responses.
32916+ type: string
32917+ type:
32918+ $ref: '#/components/schemas/RecommendationType'
32919+ required:
32920+ - type
32921+ - attributes
32922+ type: object
32923+ RecommendationDocument:
32924+ description: JSON:API document containing a single Recommendation resource.
32925+ Returned by SPA when the Spark Gateway requests recommendations.
32926+ properties:
32927+ data:
32928+ $ref: '#/components/schemas/RecommendationData'
32929+ required:
32930+ - data
32931+ type: object
32932+ RecommendationType:
32933+ default: recommendation
32934+ description: JSON:API resource type for Spark Pod Autosizing recommendations.
32935+ Identifies the Recommendation resource returned by SPA.
32936+ enum:
32937+ - recommendation
32938+ example: recommendation
32939+ type: string
32940+ x-enum-varnames:
32941+ - RECOMMENDATION
3283832942 RegisterAppKeyResponse:
3283932943 description: The response object after creating an app key registration.
3284032944 properties:
@@ -65710,6 +65814,69 @@ paths:
6571065814 x-unstable: '**Note**: This feature is in private beta. To request access, use
6571165815 the request access form in the [Service Level Objectives](https://docs.datadoghq.com/service_management/service_level_objectives/#slo-csv-export)
6571265816 docs.'
65817+ /api/v2/spa/recommendations/{service}/{shard}:
65818+ get:
65819+ description: Retrieve resource recommendations for a Spark job. The caller (Spark
65820+ Gateway or DJM UI) provides a service name and shard identifier, and SPA returns
65821+ structured recommendations for driver and executor resources.
65822+ operationId: GetSPARecommendations
65823+ parameters:
65824+ - description: The shard tag for a spark job, which differentiates jobs within
65825+ the same service that have different resource needs
65826+ in: path
65827+ name: shard
65828+ required: true
65829+ schema:
65830+ type: string
65831+ - description: The service name for a spark job
65832+ in: path
65833+ name: service
65834+ required: true
65835+ schema:
65836+ type: string
65837+ responses:
65838+ '200':
65839+ content:
65840+ application/json:
65841+ example:
65842+ data:
65843+ attributes:
65844+ driver:
65845+ estimation:
65846+ cpu:
65847+ max: 1500
65848+ p75: 1000
65849+ p95: 1200
65850+ ephemeral_storage: 896
65851+ heap: 6144
65852+ memory: 7168
65853+ overhead: 1024
65854+ executor:
65855+ estimation:
65856+ cpu:
65857+ max: 2000
65858+ p75: 1200
65859+ p95: 1500
65860+ ephemeral_storage: 512
65861+ heap: 3072
65862+ memory: 4096
65863+ overhead: 1024
65864+ id: dedupeactivecontexts:adp_dedupeactivecontexts_org2
65865+ type: recommendation
65866+ schema:
65867+ $ref: '#/components/schemas/RecommendationDocument'
65868+ description: OK
65869+ '400':
65870+ $ref: '#/components/responses/BadRequestResponse'
65871+ '403':
65872+ $ref: '#/components/responses/NotAuthorizedResponse'
65873+ '429':
65874+ $ref: '#/components/responses/TooManyRequestsResponse'
65875+ summary: Get SPA Recommendations
65876+ tags:
65877+ - Spa
65878+ x-unstable: '**Note**: This endpoint is in public beta and may change in the
65879+ future. It is not yet recommended for production use.'
6571365880 /api/v2/spans/analytics/aggregate:
6571465881 post:
6571565882 description: 'The API endpoint to aggregate spans into buckets and compute metrics
@@ -69127,6 +69294,9 @@ tags:
6912769294 externalDocs:
6912869295 url: https://docs.datadoghq.com/service_catalog/service_definitions#metadata-schema-v30-beta
6912969296 name: Software Catalog
69297+ - description: SPA (Spark Pod Autosizing) API. Provides resource recommendations and
69298+ cost insights to help optimize Spark job configurations.
69299+ name: Spa
6913069300- description: Search and aggregate your spans from your Datadog platform over HTTP.
6913169301 name: Spans
6913269302- description: Manage configuration of [span-based metrics](https://app.datadoghq.com/apm/traces/generate-metrics)
0 commit comments