[Feature request] Replacements value in the structured data.

[koba1t]

Is your feature request related to a problem? Please describe.

I feel difficult to Overlay when structured data is in k8s YAML's value. (ex. Structured data(yaml or json) format value in configMap data And structured data(almost json) format value in annotations(this way is used in many projects, I think)).

Therefore I propose to add options for replacing the value in structured data to replacements function.

I think this function can be an alternative vars function on almost use cases.

Example (json format value in configMap data)

apiVersion: v1
kind: ConfigMap
metadata:
  name: jsoned-configmap
data:
  config.json: |-
    {"config": {
      "id": "42",
      "hostname": "REPLACE_TARGET"
    }}

Example (json format value in annotations)

• https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-features#unique_backendconfig_per_service_port
• https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/#listen-ports

Describe the solution you'd like

FIrst, set format and formatPath parameters to replacements option.
And interpretation in k8s YAML value using formatPath with format.
Finally, execute replacements which place set by formatPath.

(Please watch Proposal config on the "Additional context" section.)

Describe alternatives you've considered

I tried to use delimiter: '"' options on replacement to parse json.
I think I will be able to resolve my problem with this solution, but It is very hard.

Additional context

Proposal config

source

apiVersion: v1
kind: ConfigMap
metadata:
  name: source-configmap
data:
  HOSTNAME: www.example.com
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: target-configmap
data:
  config.json: |-
    {"config": {
      "id": "42",
      "hostname": "REPLACE_TARGET_HOSTNAME"
    }}

replacement

replacements:
- source:
    kind: ConfigMap
    name: source-configmap
    fieldPath: data.HOSTNAME
  targets:
  - select:
      kind: ConfigMap
      name: target-configmap
    fieldPaths:
    - data.config\.json
    options:
      format: 'json'
      formatPath: '/config/hostname'

expected

apiVersion: v1
kind: ConfigMap
metadata:
  name: source-configmap
data:
  HOSTNAME: www.example.com
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: target-configmap
data:
  config.json: '{"config":{"hostname":"www.example.com","id":"42"}}'

[k8s-ci-robot]

@koba1t: This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[koba1t]

my implement is here(#4518)

[natasha41575]

Thank you @koba1t for the feature request and the implementation. We have discussed this in the kustomize bug scrub, and while we leaning towards accepting some version of this feature, we noted that similar issue was filed for patches #3787. This has a long discussion of what this could look like for patches, and if we want to do something similar for replacements we believe they should have a similar UX.

Because this is a major feature, this would need to be submitted as a mini in-repo KEP for further discussion, so that we can be very clear about what we are supporting. In that KEP, we should discuss the UX for both patches and replacements.

Please let us know if you have any questions about the process.

/triage under-consideration

[koba1t]

Hi @natasha41575

I tried to write a mini in-repo KEP, and I'm open PR #4558 now.

This is my first proposal, Could you give me any feedback?
I want to improve this proposal document.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[koba1t]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[george-angel]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[koba1t]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[koba1t]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[koba1t]

/remove-lifecycle stale

[natasha41575]

/triage accepted

@koba1t I think we are inclined to accept the feature, we just need to take some time to get your KEP through. I'll see if I can find time to look at it again next week.

[natasha41575]

Sending an update here that the KEP is merged, so please feel free to begin implementation.

[koba1t]

Thanks, Natasha!

[Martin4R]

This is a very cool feature. It would be really helpful for managing Kuberay configurations. Looking forward to it.

[undr-rowr]

@natasha41575, @koba1t any updates on this? This is much needed to fully embrace replacements.

[koba1t]

Hi @undr-rowr
I'm working now on the below PR.
#5679

[Benly-walter]

any updates?
this will greatly simplify our setup

[the-scott-hand]

@natasha41575 @koba1t Thanks for all the work you have done moving this feature along! Is there a projected timeline for when we will see #5679 implemented and released? Any blockers? This feature addresses a common issue faced by our team. We'd love to see this make its way into a release as soon as possible!

[srkn0]

also very excited about this feature!! :)

[alexrecuenco]

This can all be achieved through vars currently.

Why is that deprecated? I am having issues understanding the logic of removing that and then just moving the logic to something less powerful, and very limited in scope.

Although I would also love this feature, don't get me wrong.

[joebowbeer]

@alexrecuenco ICYMI, I opened a discussion of vars replacement/deprecation: #5046