🐛 Handle helm interrupt #1987
🐛 Handle helm interrupt #1987
Conversation
Introduces a workaround for 'interrupted' helm releases which enter into a 'pending' (-install/uninstall/rollback) state. If that happens, for example because of immediate application exit with one of those operations being in flight, helm is not able to resolve it automatically which means we end up in a permanent reconcile error state. One of the workarounds for this that has been repeated in the community is to remove metadata of the pending release, which is the solution chosen here. For full context see: #1776 helm/helm#5595
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1987 +/- ##
==========================================
+ Coverage 69.10% 69.25% +0.15%
==========================================
Files 79 79
Lines 7023 7045 +22
==========================================
+ Hits 4853 4879 +26
+ Misses 1888 1885 -3
+ Partials 282 281 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| } | ||
| // return an error to try to detect proper state (installation/upgrade) at next reconciliation | ||
| return nil, "", fmt.Errorf("removed pending release %q version %d metadata", rel.Name, rel.Version) | ||
| } |
There was a problem hiding this comment.
That is a reasonable way to start solving this one. But perhaps @thetechnick can let us know if the boxcutter initiative cannot solve it. If not, I think the latest questions over it were that we need to solve are:
- a) How can we ensure that we will not end up in an endless loop? We probably need a retry limit: [epic] Add retry limits and a new reason to the
Progressingstatus condition that is present when retry limit is reached. #1440 - b) If we fail, would Helm re-try alone?
- c) Assuming that the answer for (b) is no: Can we really do that for upgrades since we will delete the secret/history, OR should we do it only for a new install?
Description
rebase of #1776 - please check the original PR by @azych for description and reviewer comments. I'm reviving the work here as I couldn't push the rebase / fixes to the old PR.
Reviewer Checklist