Skip to content

fix(disks): restore disk from snapshot when storage rounds size up#2635

Draft
nevermarine wants to merge 16 commits into
mainfrom
fix/disks/restore-pvc-size-rounding
Draft

fix(disks): restore disk from snapshot when storage rounds size up#2635
nevermarine wants to merge 16 commits into
mainfrom
fix/disks/restore-pvc-size-rounding

Conversation

@nevermarine

Copy link
Copy Markdown
Collaborator

Description

Restoring a virtual disk (and therefore a virtual machine) from a snapshot fails on storage backends that round volume sizes up to a fixed granularity — for example Ceph RBD rounds up to whole GiB. The restored disk stays in the Failed phase with a ProvisioningFailed / "the specified pvc size is insufficient" error, and any VirtualMachineOperation restore that depends on it never reaches Completed.

It happens when the source disk was created without an explicit size: its volume is requested at the source image's virtual size, but the backend provisions a larger volume, so the snapshot's restore size ends up bigger than the size recorded in the snapshot metadata. On restore that smaller recorded size was rejected as insufficient. It is now treated as a lower bound and raised to the snapshot restore size, but only when the size was not explicitly set by the user (an explicit too-small user request is still rejected).

Why do we need it, and what problem does it solve?

On rounding storage backends (Ceph RBD is the common case) snapshot restore is effectively broken for any disk that was provisioned without an explicit size — the restored disk and the restore operation are stuck in Failed. This restores the ability to recover disks and VMs from snapshots on such storage.

What is the expected result?

Reproduction (on a Ceph RBD storage class):

  1. Create a VirtualDisk from a ClusterVirtualImage with spec.persistentVolumeClaim: {} (no size). The PVC is requested at the image virtual size but provisioned rounded up to the next GiB.
  2. Create a VirtualDiskSnapshot of it.
  3. Create a new VirtualDisk from that snapshot with no explicit size.

Before: the new disk goes to Failed (the specified pvc size is insufficient).
After: the new disk provisions at the snapshot restore size and becomes Ready; VMOP restore completes.

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: disks
type: fix
summary: "Restoring a virtual disk or virtual machine from a snapshot no longer fails on storage that rounds volume sizes up (e.g. Ceph RBD)."

Isteb4k and others added 16 commits July 7, 2026 13:46
Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Changelog v1.9.3

Signed-off-by: deckhouse-BOaTswain <89150800+deckhouse-boatswain@users.noreply.github.com>
Co-authored-by: universal-itengineer <universal-itengineer@users.noreply.github.com>
Signed-off-by: Valeriy Khorunzhin <valeriy.khorunzhin@flant.com>
During a restore the VM is stopped inside the maintenance window: its KVVMI and then the KVVM itself are deleted. Bringing the VM back up afterwards relied on an implicit side effect — the recreated KVVM getting RunStrategy=Always at creation time. That path is racy and only covers the always-on policies.

---------

Signed-off-by: Daniil Antoshin <daniil.antoshin@flant.com>
)

Signed-off-by: Maksim Fedotov <maksim.fedotov@flant.com>
Signed-off-by: Maksim Fedotov <maksim.fedotov@flant.com>
Add validation to getIPerfClientReport verifying that the iperf client session spans the virtual machine migration: start time before migration start, end time after migration end, and no more than one zero-byte interval during the process.

Signed-off-by: Dmitry Prytkov <dmitry.prytkov@flant.com>
…refix (#2594)

The d8 v ansible-inventory command read host variables from the bare
provisioning.virtualization.deckhouse.io/ prefix, while the
virtualization-provisioner module reads them from the separate
vars.provisioning.virtualization.deckhouse.io/ prefix. The same VM
annotations produced different inventories depending on the tool.

Read host variables from the vars. prefix to match the module. Groups
still come from provisioning.virtualization.deckhouse.io/groups. Update
the command help, usage examples and the ansible-inventory FAQ.

Signed-off-by: Pavel Tishkov <pavel.tishkov@flant.com>
…d manually (#2596)

fix(vm): fix VM stuck until the child KVVMI in Failed phase is deleted manually

Signed-off-by: Yaroslav Borbat <yaroslav.borbat@flant.com>
Virtualization has no primitive to manage a group of identical virtual machines whose count changes over time. Every "I need N identical VMs and the number varies" scenario — CI runner fleets, VDI desktop pools — has to be solved with orchestration outside the platform: users write their own controllers/scripts to create and delete VirtualMachines, watch their number, recreate lost ones and clean up their disks. This duplicates logic and is error-prone around races and node failures.

This PR introduces VirtualMachinePool (paid editions only, EE/SE+): a namespaced resource that declaratively keeps a requested number of identical VMs and integrates with kubectl scale, HPA and KEDA through the standard scale subresource. Its template is an ordinary VirtualMachineSpec, so a replica is no different from a manually created VM.

The feature is complete and covers:

Replica management — keeps the requested number of identical VMs, replaces lost ones, and reports state in status (replicas, readyReplicas, selector, conditions). It is cache-lag-safe (ReplicaSet-style expectations), so a lagging informer cache cannot double-create replicas.
Scale-down policy — NewestFirst / OldestFirst choose which replica leaves on anonymous scale-down via scale; Explicit forbids anonymous shrink through a webhook, so for "busy" workloads (CI runners, VDI) replicas can only be removed by address.
Addressed removal — the scaleDownWith subresource deletes named replicas and shrinks the pool by that count, instead of letting the controller pick victims.
In-place template propagation — editing the template rolls the change out to existing replicas; disruptive changes wait for a restart and are surfaced via status.restartPendingReplicas.
Reusable disks — per-replica disks are described in virtualDiskTemplates with a reclaim policy: Delete disks belong to the VM and are removed with it; Retain disks belong to the pool, outlive the replica and are reused on the next scale-up, with an optional warm buffer (keep) and TTL garbage-collection.
The resource is available only in paid editions, gated behind the VirtualMachinePool module feature gate (default off, locked off in CE). The API/CRD installs in every edition, but the controller self-gates on the feature gate, so the resource does nothing in CE.

---------

Signed-off-by: Pavel Tishkov <pavel.tishkov@flant.com>
Signed-off-by: Vladislav Panfilov <vladislav.panfilov@flant.com>
Co-authored-by: Vladislav Panfilov <vladislav.panfilov@flant.com>
…ate (#2621)

The VirtualMachinePool feature gate was an opt-in placeholder. As the feature
has not been released yet, drop the gate before release so the resource is
available in EE/SE+ out of the box and stays unavailable in CE — the same model
as VolumeMigration/TargetMigration. Nothing to migrate: the gate never shipped.

- gate becomes locked-on in EE/SE+, locked-off in CE (no user opt-in)
- dropped from the user-facing featureGates enum
- pool webhooks gated by edition in helm instead of the feature-gate list

Controller/webhooks keep self-gating on the now-locked gate.

Signed-off-by: Pavel Tishkov <pavel.tishkov@flant.com>
Adds:

- liveMigration.systemNetworkName ModuleConfig field. When set, VM live-migration traffic is routed over the per-node interfaces of a SystemNetwork CR from the SDN instead of the default node network.
- migrationiface controller resolves the per-node kernel interface name and writes it onto each Node as the annotation virtualization.deckhouse.io/migration-iface. virt-handler reads that annotation at startup and binds its migration proxy to the SystemNetwork's per-node IP.
- ModuleConfig validator to check if SystemNetwork added to liveMigration.systemNetworkName exists.
- New reason for VMOP if migration network is unavailable on nodes.

---------

Signed-off-by: Daniil Loktev <lokt.daniil@gmail.com>
Signed-off-by: Daniil Loktev <70405899+loktev-d@users.noreply.github.com>
Signed-off-by: Vladislav Panfilov <97229646+prismagod@users.noreply.github.com>
Co-authored-by: Vladislav Panfilov <97229646+prismagod@users.noreply.github.com>
Description
Log in to the private RU registry in the release channels workflow and check module versions against registry.deckhouse.ru/deckhouse.

Why do we need it, and what problem does it solve?
Release channel checks need access to artifacts in the RU registry. This keeps the workflow and version check pointed at the registry that contains the expected release artifacts.

---------

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…lose gaps (#2602)

Sizing policy validation errors were confusing and, in some cases, missing. When a VirtualMachineClass sets per-core memory limits, the error reported the allowed values per CPU core, while users configure the total memory — so the suggested numbers matched nothing they could actually set. Messages now report the total memory to set for the current number of cores, name the field to change, and combine multiple violations into a single readable list.

This also fixes cases where the policy was silently not enforced: the CPU cores step was never validated, and a memory / per-core minimum was ignored when no maximum was set.

Signed-off-by: Pavel Tishkov <pavel.tishkov@flant.com>
* docs: add VM launcher update notes to release notes

Signed-off-by: Vladislav Panfilov <vladislav.panfilov@flant.com>
When a VirtualDisk is created without an explicit size, its PVC is requested
at the data source's virtual size. Storage backends that round volume sizes up
(e.g. Ceph RBD rounds to whole GiB) provision a larger volume, so the
VolumeSnapshot restore size exceeds the originally requested size stored in the
snapshot metadata.

On restore that stored original size was validated against the snapshot restore
size and rejected as insufficient, leaving the restored disk Failed and the
VirtualMachineOperation restore stuck in Failed.

Use the restore size as a lower bound when the size comes from the snapshot
metadata rather than an explicit user request.

Signed-off-by: Maksim Fedotov <maksim.fedotov@flant.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.