Skip to content

Migrate all controllers to Server-Side Apply status updates#330

Draft
fwiesel wants to merge 19 commits into
mainfrom
ssa
Draft

Migrate all controllers to Server-Side Apply status updates#330
fwiesel wants to merge 19 commits into
mainfrom
ssa

Conversation

@fwiesel

@fwiesel fwiesel commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Migrates all controllers from PatchHypervisorStatusWithRetry to Server-Side Apply (SSA) Status().Apply() for all status writes.

Key design:

  • Each controller only sets fields it owns. With +listType=map / +listMapKey=type on the CRD, the API server merges conditions by type key — other controllers' conditions are untouched.
  • Each controller seeds only the conditions it previously wrote into the apply payload before upsetting them, preventing unintended pruning. ConditionFromStatus converts a single owned condition for this purpose.
  • Conditions are deleted by omission: if a controller intentionally omits a previously-set condition, SSA prunes it (used by HypervisorMaintenanceController to remove ConditionTypeEvicting on MaintenanceUnset).

Controllers migrated:

  • EvictionReconciler
  • HypervisorController (also guards against clobbering InternalIP with empty string)
  • HypervisorTaintController
  • HypervisorMaintenanceController (status + eviction creation via SSA)
  • OnboardingController (status + statusCfg threading through reconcile chain)
  • HypervisorOffboardingReconciler
  • AggregatesController
  • TraitsController
  • ready.Controller (removes status_patch.go)
  • NodeCertificateController (certificate creation via SSA)

Helpers in internal/utils/conditions.go:

  • ConditionFromStatus: converts a single owned condition to apply-config form
  • SetApplyConfigurationStatusCondition: upserts a condition entry, preserving LastTransitionTime when status is unchanged

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7cce2422-5621-431e-9e64-327bb78b1531

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ssa

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

fwiesel added 18 commits July 9, 2026 19:53
CI installs controller-gen@latest which is now v0.21.0. The new version
correctly omits the namespace parameter from apply configuration
constructors for cluster-scoped resources (Hypervisor, Eviction).

Without this regeneration, CI's generate step produces a 1-arg
Eviction() signature while the committed code has the stale 2-arg
version, causing build failures on any branch that calls
apiv1.Eviction(name, namespace).
Replace MergeFromWithOptimisticLock + retry with Status().Apply() for
all eviction status updates. Introduces ConditionsFromStatus and
SetApplyConfigurationStatusCondition helpers to utils — mirroring
meta.SetStatusCondition semantics — as the foundation for all SSA
condition management. The evictionStatusCfg helper seeds every apply
with the full set of owned fields so SSA never prunes values between
reconcile cycles.
Replace PatchHypervisorStatusWithRetry with Status().Apply() for the
InternalIP scalar and ConditionTypeTerminating condition. Re-fetch the
hypervisor after the status apply so the subsequent spec patch sees a
fresh resourceVersion.
Replace PatchHypervisorStatusWithRetry with Status().Apply() for the
ConditionTypeTainted condition.
Replace PatchHypervisorStatusWithRetry with Status().Apply() for
ConditionTypeHypervisorDisabled, ConditionTypeEvicting, and the Evicted
scalar. ConditionTypeEvicting is removed by omitting it from the apply
config — SSA prunes map-list entries no longer claimed by the sole owner.
Replace PatchHypervisorStatusWithRetry with Status().Apply(). The Nova
ID lookup is inlined into Reconcile and combined with the initial
condition in a single apply, avoiding two applies from the same field
manager in one reconcile cycle which would cause SSA to prune the IDs.
HypervisorID and ServiceID are included in every subsequent apply to
retain ownership.
Replace PatchHypervisorStatusWithRetry with Status().Apply() for the
ConditionTypeOffboarded condition.
Replace PatchHypervisorStatusWithRetry with Status().Apply() for
ConditionTypeAggregatesUpdated and the Aggregates slice. The error path
includes the current Aggregates in the apply to prevent SSA from pruning
previously-set values on a transient OpenStack error. Clearing aggregates
to an empty slice uses a targeted merge patch to work around the omitempty
limitation in the generated apply configuration type.
Replace PatchHypervisorStatusWithRetry with Status().Apply() for
ConditionTypeTraitsUpdated and the Traits slice. Error paths pass the
current hv.Status.Traits to retain ownership and avoid SSA releasing
it on transient placement API failures.
…rStatusWithRetry

Replace the last caller of PatchHypervisorStatusWithRetry with
Status().Apply() for ConditionTypeReady. With no remaining callers,
PatchHypervisorStatusWithRetry and StatusPatchBackoff are removed.
Replace Create + SetControllerReference with Apply for the Eviction CR.
The owner reference and labels are set in the apply configuration
metadata, and SSA handles the upsert. A Get is still performed after the
apply to read the current eviction status conditions.
Replace CreateOrUpdate + SetOwnerReference with Apply for the cert-manager
Certificate CR. The owner reference is set in the apply configuration
metadata. The retry-on-conflict loop is no longer needed since SSA handles
concurrent updates correctly. The controller now applies the full
certificate spec on every reconcile so IP/DNS changes are picked up
immediately.
…ctions

With SSA the status config is built once in Reconcile and passed to
reconcileComputeService and reconcileEviction which mutate it directly.
This eliminates the intermediate condition return values and makes the
single-apply-per-reconcile explicit. The early-return guard for
already-enabled/disabled state is inlined into each branch.
With SSA the status config is built once at the top of Reconcile and
threaded through to all sub-functions (abortOnboarding, initialOnboarding,
smokeTest, completeOnboarding) which mutate it directly. An apply closure
is also passed so each function calls apply() when it has determined the
desired condition, rather than building and applying the config itself.
This removes the applyOnboardingCondition helper and makes the flow fully
declarative: compute desired state, then apply once.
conditions.go (cherry-pick from rework-onboarding):
- ConditionsFromStatus: preserve ObservedGeneration; the taint controller's
  skip-check gates on it and it was silently dropped.
- SetApplyConfigurationStatusCondition: guard against nil/empty Type to
  prevent a panic when a caller omits the Type field.
- SetApplyConfigurationStatusCondition: propagate ObservedGeneration when
  updating an existing condition entry.

hypervisor_controller.go:
- Do not call WithInternalIP("") when the node has no NodeInternalIP address
  yet (e.g. during kubelet restart or bootstrap). WithInternalIP combined with
  ForceOwnership would overwrite a previously stored valid IP with an empty
  string.
- Remove unnecessary metav1.ConditionStatus() conversion (unconvert lint).

eviction_controller.go:
- Remove extra blank line flagged by gofmt.

Note: the 3 pre-existing test failures in hypervisor_maintenance_controller_test.go
and hypervisor_controller_test.go are unrelated to these changes.
With +listType=map / +listMapKey=type on HypervisorStatus.Conditions and
EvictionStatus.Conditions, the API server merges the conditions list by the
'type' key on Apply. This means:
- Conditions owned by other field managers are untouched (correct).
- Conditions previously written by this field manager that are absent from
  the Apply payload are pruned (also correct — used to delete conditions).

The previous ConditionsFromStatus helper round-tripped ALL conditions from
status into the apply payload. This was wrong: it caused each controller to
claim field ownership over conditions it does not own, interfering with other
controllers on the next Apply.

Replace ConditionsFromStatus with ConditionFromStatus (single condition),
and update every controller to seed only the condition(s) it owns before
calling SetApplyConfigurationStatusCondition. For controllers that need to
delete a condition by omission (HypervisorMaintenanceController removing
ConditionTypeEvicting on MaintenanceUnset), the owned conditions are seeded
selectively so the target condition is simply absent from the payload.

No behaviour change for the eviction_controller: it is the sole owner of
all conditions on the Eviction CR, so seeding all three types is correct.
Cluster-scoped resources (Hypervisor, Eviction) now take only a name
parameter in their apply configuration constructors. Remove the
previously required but always-empty namespace argument.
HypervisorController: the AgentPodsEvicted=False branch was returning
early with a requeue before running syncLabelsAndAnnotations, so node
label changes were not propagated to the Hypervisor while terminating.
Move the requeue after the spec patch so labels stay in sync.

HypervisorMaintenanceController: when enabling a compute service, also
clear the forced_down flag. Otherwise, an earlier HA event that set
forced_down=true would leave the service reachable but treated as down.
…rWithManager

SetupWithManager was inlining the same ctrl.NewControllerManagedBy call
that registerWithManager already provides, leaving registerWithManager
unreachable and triggering the unused lint error.
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Merging this branch changes the coverage (1 decrease, 2 increase)

Impacted Packages Coverage Δ 🤖
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller 68.90% (-0.01%) 👎
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/ready 73.21% (+1.28%) 👍
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/utils 80.77% (+7.44%) 👍

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/aggregates_controller.go 76.34% (-8.27%) 93 (+28) 71 (+16) 22 (+12) 👎
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/eviction_controller.go 61.05% (-0.17%) 172 (+7) 105 (+4) 67 (+3) 👎
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/hypervisor_controller.go 78.99% (-1.01%) 119 (-1) 94 (-2) 25 (+1) 👎
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/hypervisor_maintenance_controller.go 75.00% (-3.41%) 80 (-8) 60 (-9) 20 (+1) 👎
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/hypervisor_taint_controller.go 90.48% (-2.38%) 21 (+7) 19 (+6) 2 (+1) 👎
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/node_certificate_controller.go 80.95% (-0.68%) 42 (-7) 34 (-6) 8 (-1) 👎
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/offboarding_controller.go 73.97% (+1.12%) 73 (+3) 54 (+3) 19 👍
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/onboarding_controller.go 56.70% (+1.64%) 224 (-23) 127 (-9) 97 (-14) 👍
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/ready/controller.go 73.21% (+1.28%) 56 (-1) 41 15 (-1) 👍
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/traits_controller.go 75.41% (+1.97%) 61 (-3) 46 (-1) 15 (-2) 👍
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/utils/conditions.go 84.09% (+84.09%) 44 (+44) 37 (+37) 7 (+7) 🌟
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/utils/status_patch.go 0.00% (-85.71%) 0 (-7) 0 (-6) 0 (-1) 💀 💀 💀 💀 💀

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant