Skip to content

machineops: add provider lifecycle registration#459

Open
imiller31 wants to merge 1 commit into
Azure:mainfrom
imiller31:ismille/resumable-machine-ops
Open

machineops: add provider lifecycle registration#459
imiller31 wants to merge 1 commit into
Azure:mainfrom
imiller31:ismille/resumable-machine-ops

Conversation

@imiller31

@imiller31 imiller31 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace the public author-implemented provider interfaces and parallel provider lists with an immutable registration API
  • let one provider mix immediate and long-running operation strategies, with operation-local replay safety, replacement bootstrap, and cleanup
  • add controller.AddToManager so provider controllers register with controller-runtime without constructing the internal reconciler
  • persist long-running provider handles on status.targets[], use the current Machine provider ID for callbacks, and serialize host operations by Machine
  • migrate the Azure and OCI providers to registrations and narrow provider callbacks to identifiers instead of mutable Kubernetes objects

Provider authoring shape

provider, err := machineops.NewProvider(
    name,
    machineops.WithImmediateOperation(kind, execute, options...),
    machineops.WithLongRunningOperation(kind, begin, poll, options...),
)

err = machineopscontroller.AddToManager(manager, []*machineops.Provider{provider}, options)

Registered operations replace Supports and optional interface type assertions. The controller owns status transitions, credential resolution, retries, polling, per-Machine conflict serialization, and terminalization.

Long-running Begin callbacks must be idempotent for the stable OperationRequest.OperationUID. The controller may invoke Begin again until the returned operation handle has been persisted. Providers return ordinary errors for retryable failures and PermanentError when retrying cannot make progress.

Safety properties

  • accepted operations persist only the controller-owned provider name, operation ID, and optional resume token, then resume after controller restarts
  • the stable MachineOperation UID supplies correlation across both Begin retries and Poll calls
  • each provider callback receives the current Machine.spec.providerID
  • host operations targeting the same Machine pass one shared serialization gate before strategy dispatch
  • a failed handle status write safely replays the idempotent Begin callback with the same operation UID
  • ordinary begin errors retry Begin; PermanentError terminalizes the operation
  • ordinary poll errors retain the handle and retry; PermanentError terminalizes the operation
  • immediate operations resume only when explicitly registered with ReplaySafe()
  • provider ID handoff still precedes operation-local cleanup

Review follow-ups

  • prevent status-update watch events from bypassing the requested polling interval
  • serialize conflicts across both immediate and long-running operations
  • preserve InProgress when an immediate operation has an unknown outcome so the request is not replayed
  • stop declaring Azure host replacement replay-safe without a provider idempotency guarantee
  • construct replacement bootstrap data only for initial submission, not while polling
  • normalize and truncate provider-supplied condition reasons and messages before persisting them
  • measure stalled operations from provider acceptance time instead of object creation time

Validation

  • go generate ./api/machina/v1alpha3
  • go test ./api/machina/v1alpha3 ./pkg/machineops/... ./internal/machineops/... ./cmd/machine-ops-controller -count=1
  • go test -race ./pkg/machineops ./pkg/machineops/controller ./internal/machineops ./internal/machineops/providers/azurevm ./internal/machineops/providers/ociinstance ./cmd/machine-ops-controller -count=1
  • go vet ./pkg/machineops/... ./internal/machineops/... ./cmd/machine-ops-controller/...
  • golangci-lint run
  • ANS provider and client race suites using this branch as a workspace replacement
  • ANS emulator smoke test for reboot and reimage, reconstructing the provider between submission and polling

Unrelated repository test failure

go test ./... reaches the existing deterministic TestMaterializeEmbeddedFS failure: the test expects 01-namespace.yaml, which is not materialized in its temporary directory. The failure does not involve MachineOperation code.

@imiller31 imiller31 requested a review from a team July 10, 2026 22:09
@imiller31 imiller31 force-pushed the ismille/resumable-machine-ops branch from 7d45d02 to 77fb466 Compare July 13, 2026 17:59
@imiller31 imiller31 changed the title machineops: add resumable provider operations machineops: add provider lifecycle registration Jul 13, 2026
@imiller31 imiller31 force-pushed the ismille/resumable-machine-ops branch 4 times, most recently from f390bdc to e719e37 Compare July 14, 2026 20:23
@imiller31 imiller31 force-pushed the ismille/resumable-machine-ops branch from e719e37 to d62167c Compare July 14, 2026 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants