mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 22:13:18 +09:00
Andrei Kvapil
d3559d48ed
## What this PR does Adds tenant self-service import of virtual machines from VMware vSphere, as a pair of CRDs in a new `migration.cozystack.io` group reconciled by a controller in core. Konveyor Forklift is the transfer engine underneath; tenants never see it. A `VMImportSource` registers a long-lived connection to a provider — type, endpoint, credentials entered on the spec and materialized into a Secret by the controller, because tenants cannot create Secrets in Cozystack. A `VMImportTask` runs the one-shot operation: reference a source, name the machines, get `VMDisk`s and `VMInstance`s. The outputs carry no owner reference back to the task, so deleting a finished import removes the migration machinery and nothing else. This supersedes the chart-based tenant API in #1982 while keeping its Forklift packages unchanged, carried over here with attribution. The design is written up in cozystack/community#62, which records why a Helm chart lost to a controller: `apps.cozystack.io` is a projection over HelmReleases, so "the outputs survive deletion" would need `helm.sh/resource-policy: keep`; a completed migration is terminal while Flux reconciles toward a steady state forever; and the number and size of the disks an import produces is only known after the source VM has been inspected, which a template cannot express. Two things are worth calling out for review. **The controller ships no vSphere client.** Forklift already authenticates, validates VM references and reports unmapped networks and datastores, and the KubeVirt VirtualMachine it builds carries the source VM's CPU, memory and disk order — so the inventory is read from Forklift's own objects rather than from a second client that could disagree with the one doing the work. A tenant sees Forklift's verdicts verbatim rather than a paraphrase. **The volume handoff copies nothing.** The transferred PersistentVolume is retained and re-bound into the claim a `VMDisk` expects, with `cdi.kubevirt.io/storage.populatedFor` telling CDI the claim is already populated — the same primitive the Velero VM-restore path already depends on in production. That closes both the raw-PVC gap raised in review on #1982 and the duplicate-copy waste tracked in #3924. Everything is opt-in: three packages added to `bundles.enabledPackages`, and one platform value (`vmImport.vddkImage`) naming the VDDK image, which Cozystack cannot ship because it is built from a proprietary VMware SDK. Empty is a supported state and makes a vSphere source report itself unavailable at registration rather than mid-transfer. ### Validation Run on a live Cozystack v1.6.2 cluster and rolled back afterwards. Confirmed working end to end: CRD install and printer columns, the VDDK policy (including that no credentials Secret is projected for a source that cannot work), credential projection with the ownership guard, the Provider pair with the operator's image stamped on it, the local-cluster destination Provider reaching Ready/Connected/Inventory, Forklift's connection verdict mirrored onto the source verbatim, the diagnostic chain from Forklift through the source to the task, that nothing is created while a source is unready, and that deleting a source garbage-collects both Providers and the Secret. That testing found four defects, none of which were visible from reading code, and all are fixed here: the task controller could not start at all when Forklift is absent (a watch on an absent CRD never syncs, so controller-runtime refused to start it and the pod sat at 0/1); a vSphere secret needs a CA certificate rather than a thumbprint, so as first modelled every source would have sat at `SecretNotValid` forever with a reason that reads like a credentials problem; Forklift's absence was returned as a reconcile error, so a source kept a stale condition indefinitely while the controller hot-looped; and the controller's own Deployment violated the restricted Pod Security Standard, which is indefensible for a feature whose design argument is that an import stays inside it. A further four assumptions were corrected against operational experience from the original implementation's work on a real vSphere environment, which this session had no access to. Each failed silently: the VirtualMachine Forklift creates is labelled with the Plan's **UID**, not its name, so the disk would never have been adopted; the transfer arrives as a CDI-populated claim with no DataVolume, carrying an owner reference to that VirtualMachine, and clearing it was conditional on a DataVolume existing — on the original implementation garbage collection removed such a claim 357ms after the VM went, destroying a successfully migrated disk; the terminal signal is the phase the engine settles on, not only a completion timestamp; and progress read from a name-matched pipeline step pegs at zero if a release renames it. **What could not be tested:** everything past authenticating to vCenter — inventory resolution, the loop that learns network and datastore mappings from Forklift's validation conditions, the transfer itself, and this controller's own code path through the handoff. No vCenter was available. The handoff mechanics were separately proven by hand on the same cluster in earlier work; what remains unproven is this implementation of them. The mapping loop is the least-supported part of the change and is the thing to look at hardest. ### Screenshots https://github.com/user-attachments/assets/f313f6ba-8e72-42d9-b65a-033866ad39d5 ### Downstream repositories Walked the trigger map against the diff. `cozystack/website` is reached on two counts and the follow-up is open as [cozystack/website#673](https://github.com/cozystack/website/pull/673), a migration guide covering the pre-flight checks, the administrator's VDDK build, and the driver caveat the lab run surfaced: - `packages/core/platform/values.yaml` gains `vmImport.vddkImage`, and `content/en/docs/next/operations/configuration/platform-package.md` is a hand-written table of those keys. - Three platform components are added, which reaches `guides/platform-stack/_index.md` and `operations/configuration/licenses.md` — the latter is where the proprietary VDDK image belongs, since Cozystack references it without shipping it. The others were checked and are not affected: no package is added under `packages/apps/` or `packages/extra/` (terraform-provider, website app lists), no key the Ansible role sets changes, nothing under `hack/` is moved or renamed and no make target changes behaviour (ccp, external-apps-example), and no node prerequisite changes (talm). - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/pull/673 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(migration): import virtual machines from VMware vSphere into a tenant. A VMImportSource registers a connection to vCenter and a VMImportTask imports named machines, producing ordinary VMDisks and VMInstances that survive deletion of the import. Opt-in: add cozystack.forklift-operator, cozystack.forklift and cozystack.migration-controller to bundles.enabledPackages, and set vmImport.vddkImage to a VMware VDDK image you build yourself — Cozystack cannot redistribute it. Guest conversion and warm migration are not included in this version. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added vSphere VM migration support, including source connections, import tasks, inventory discovery, progress tracking, and conversion into Cozystack VMs and disks. * Added dashboard pages for managing migration sources and imports, with source-dependent VM selection and progress details. * Added optional VM firmware settings for BIOS/UEFI, Secure Boot, and persistent EFI storage. * Added opt-in platform packages for migration services and Forklift integration. * **Security** * Restricted access to the ingress admission webhook port with a network policy. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Cozystack
Cozystack is a free platform and framework for building clouds.
Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.
If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.
Community
You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.