IvanHunters 4844e1d99c fix(opensearch-operator): stop excluding a node whose drain cannot finish (#4323)
## What this PR does

The previous patch let a permanently yellow cluster past the restart
health gate. With three or more nodes `packages/apps/opensearch` renders
`drainDataNodes: true`, and the restart then reaches
`PreparePodForDelete`, which excludes the candidate node from allocation
before it can know whether the drain will finish.

On the cluster that made the gate wait it never finishes: a shard on
that node has nowhere else to go, which is the same condition holding
the cluster yellow. The exclusion is only withdrawn after a pod is
really deleted (`rollingRestart.go`, `upgrade.go`), so a data node stays
out of allocation for as long as the restart is stuck, shedding its
shards onto the rest of the cluster and buying nothing. Before the
yellow gate this path was unreachable with draining on, so nothing was
written at all.

`patches/skip-unusable-drain-exclusion.diff` asks the allocator before
excluding anything, and only while nothing is moving, so a drain making
progress costs nothing extra. A started shard answering
`can_move_to_other_node: no` means the drain cannot finish: no exclusion
is written, and one left by an earlier pass is withdrawn. A drain that
can still progress behaves exactly as before, and the two-data-node path
is untouched because it never waits for a full drain.

Checking after the exclusion instead would flap: the next reconcile ten
seconds later would apply it again.

The v2.8.0 base has no allocation-explain call of its own, so the patch
adds the client method and a response type carrying only the field the
decision reads.

The second commit makes the shipped tests real. The image build ran `go
build` alone, so a test file that stopped compiling, or stopped
asserting, still produced an image. `go test` over the two package trees
the patches touch now runs in the builder stage; both run without a
cluster.

Upstream as opensearch-project/opensearch-k8s-operator#1585, fixed there
by opensearch-project/opensearch-k8s-operator#1586. Both patches go away
with the first operator release carrying that change.

### Screenshots

Not a UI change.

### Downstream repositories

Walked the trigger map against the diff: two files under
`packages/system/opensearch-operator/images/opensearch-operator/`. No
package README or values schema, no API or CRD, no node prerequisites in
`hack/`, no vendored chart, no make target.

- [x] No downstream repository is affected by this change
- [ ] [cozystack/website](https://github.com/cozystack/website) -
follow-up:
- [ ]
[cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack)
- follow-up:
- [ ]
[cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack)
- follow-up:
- [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up:
- [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up:
- [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) -
follow-up:
- [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) -
follow-up:
- [ ]
[cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server)
- follow-up:
- [ ]
[cozystack/external-apps-example](https://github.com/cozystack/external-apps-example)
- follow-up:
- [ ] [cozystack/examples](https://github.com/cozystack/examples) -
follow-up:
- [ ] [cozystack/community](https://github.com/cozystack/community) -
follow-up:

### Release note

```release-note
fix(opensearch-operator): a rolling restart or upgrade that cannot drain its next node no longer leaves that node excluded from shard allocation for as long as the restart is stuck
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
  * Improved node draining during OpenSearch maintenance and restarts.
* The operator now checks whether shards can move before excluding a
node from allocation.
* If a shard cannot relocate, the existing exclusion is withdrawn and
draining pauses, helping prevent nodes from remaining unnecessarily
unavailable.
* Existing behavior is preserved for two-node clusters and while shard
relocation is already active.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-18 15:17:18 +03:00
2025-10-08 09:43:34 +05:00
2024-02-08 12:04:32 +01:00

Cozystack Cozystack

GitHub Release Apache-2.0 License Commit activity OpenSSF Best Practices OpenSSF Scorecard CLOMonitor Support

Cozystack

Cozystack is a free and open-source platform and framework for building clouds.

Cozystack is a CNCF Sandbox project that was originally built and sponsored by Ænix.

With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.

Use Cozystack to build your own cloud or provide a cost-effective development environment.

Cozystack user interface

Use-Cases

Documentation

The documentation is located on the cozystack.io website.

Read the Getting Started section for a quick start.

If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.

Versioning

Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.

Contributions

Contributions are highly appreciated and very welcomed!

In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.

You are welcome to take on the fix yourself. Commits are used to generate the changelog, and their author will be referenced in it.

If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.

Community

You are welcome to join us in #cozystack, on either CNCF Slack (get an invite) or Kubernetes Slack (get an invite), in our Telegram group, and at our community meetings. Add them to your Google Calendar or iCal for convenience.

Project documents

Document What it covers
Governance How decisions are made and who makes them
Contributor Ladder How to grow from contributor to maintainer
Maintainers Who maintains the project
Code of Conduct What we expect of each other
Security Policy Reporting a vulnerability, and which releases get fixes
Adopters Who runs Cozystack in production
Roadmap Where the project is going

License

Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.

Commercial Support

A list of companies providing commercial support for this project can be found on official site.

Languages
Go 47.2%
Shell 37.6%
TypeScript 7%
Go Template 5.4%
Makefile 1.4%
Other 1.2%