feat(sharding): use non-blocking shard removal

[leon-ape]

What changed

Enable nonBlocking for the shardRemove lifecycle action in:

• Redis
• FalkorDB
• MongoDB

Set timeoutSeconds: 3600 for all three actions. For non-blocking actions, this is the total execution budget across all argument invocations, retries, and retry intervals.

Why

These shard-removal actions perform data-dependent work that can exceed the synchronous Action limit:

• Redis rebalances hash slots away from the shard and removes its cluster nodes.
• FalkorDB rebalances slots to zero and removes the shard nodes.
• MongoDB waits for chunk migration, moves database primaries when necessary, and waits for removeShard to complete.

Without an explicit timeout, MongoDB would use the 30-second default. The existing 600-second values for Redis and FalkorDB would become hard total limits after switching to non-blocking execution. A finite one-hour window matches the intended long-running shard-operation configuration while still bounding stuck actions.

KubeBlocks can persist the selected targets and poll these actions until they reach a terminal result.

User impact

Scale-in for these sharded databases no longer depends on shard removal finishing within one synchronous controller call and has up to one hour to finish migration and retries. This requires apecloud/kubeblocks#10725.

Validation

For each of redis, falkordb, and mongodb:

• helm dependency build --skip-refresh addons/<addon>
• helm lint addons/<addon>
• helm template <addon> addons/<addon> and verified spec.lifecycleActions.shardRemove.nonBlocking: true and timeoutSeconds: 3600

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (f84d4ad) to head (7df1eaa).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...cripts-ut-spec/sharding_lifecycle_contract_spec.sh	0.00%	11 Missing ⚠️
...-ut-spec/sharding_lifecycle_image_contract_spec.sh	0.00%	5 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #3285   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files        164     165    +1     
  Lines      25589   25605   +16     
=====================================
- Misses     25589   25605   +16     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[leon-ape]

P1: redis-cluster and falkordb-cluster keep their existing ShardingDefinition names while this PR changes their lifecycleActions.shardRemove specs, but both rendered resources omit apps.kubeblocks.io/skip-immutable-check. KubeBlocks hashes the full ShardingDefinition spec and marks a same-name update Unavailable when that hash changes, so upgrading either addon leaves its sharded lifecycle definition unavailable before the new removal mode can be used.

[weicao]

P1 fixed at exact head 7df1eaa9e492eb708ae923279bed98524a11c770.

Redis and FalkorDB ShardingDefinitions now use their existing full annotation helpers, which preserve kubeblocks.io/crd-api-version and add apps.kubeblocks.io/skip-immutable-check: "true". I added focused render-contract coverage for both charts.

Local evidence:

• base b91f17cd...: both rendered ShardingDefinitions omit the annotation
• candidate 7df1eaa9...: Redis/FalkorDB focused ShellSpec, Bash 3.2 + 5.3: 4/4 pass in each shell
• Redis + FalkorDB full ShellSpec: Bash 3.2 113 examples / 0 failures / 12 version skips; Bash 5.3 terminal green
• helm lint both charts, rendered manifests, ShellCheck warning gate, and git diff --check: pass
• independent Dev review: APPROVE/BLOCKER_LIST0

Core PR #10725 and runtime validation remain separate dependencies; this commit closes the addon-side same-name immutable-update blocker only.

[weicao]

Current dependency/runtime ledger (2026-08-09):

• addon source remains exact 7df1eaa9e492eb708ae923279bed98524a11c770; all eight required checks are green.
• Core dependency #10725 remains exact 3c1e30ec2372e9b9adc7936c10d7fa76465652fa, OPEN/REVIEW_REQUIRED. Its current source still serializes all selected shard/Pod targets into one Component annotation, so the supported 2,048-shard All/All case can exceed Kubernetes' 262,144-byte annotation limit before dispatch. Existing blocker: feat: support non-blocking shard actions kubeblocks#10725 (comment)
• The runtime candidate is pinned to Core 3c1e30ec..., addon 7df1eaa9..., and syncer d87e04cf52ea55e42bdd8b07ed0f9ac83116c3a3. No FalkorDB shardRemove runtime was started: the current project policy still prohibits FalkorDB non-general-runner create/START/retry, and no superseding approved common target is available.

This records exact dependencies and evidence boundaries only. It does not claim runtime acceptance.

[weicao]

Dependency exact-head update: Core PR #10725 moved from 3c1e30ec2372e9b9adc7936c10d7fa76465652fa to 631dd49b4b63610fd35b3e17c849548ce645bcae.

The new 441dbac commit isolates non-blocking sharding orchestration but leaves target-state storage unchanged: the complete selected shard/Pod JSON is still stored in one source Component annotation. The supported 2,048 shards x 3 Pods, All/All case therefore still exceeds Kubernetes' 262,144-byte annotation limit before dispatch. Exact-head Core evidence: apecloud/kubeblocks#10725 (comment)

Addon head remains 7df1eaa9e492eb708ae923279bed98524a11c770, source checks remain green, and the addon-side immutable-update blocker remains closed. This dependency head change invalidates prior Core-head runtime evidence; no exact shardRemove runtime PASS/N is claimed while the current FalkorDB runtime STOP policy remains in force.