Azure stack poc 26 1 2404 - #1
Open
cadejacobson wants to merge 1399 commits into
Open
Conversation
String output changed in 7703634. Instance-id doesn't change on LXD / Focal.
The handle function of cc_mounts was hard to grok and had one of the highest cyclomatic complexity scores in the codebase. Functionally, the code should be unchanged.
Python interpreter initialization and module import time
contributes a significant amount of wall clock time to
cloud-init's runtime (and therefore to total boot time).
Cloud-init has four stages. Each stage starts its own Python
interpreter and loads the same libraries. To eliminate the
redundant work of starting an interpreter and loading libraries,
this changes cloud-init to run as a single process. Systemd
service ordering is retained by using the existing cloud-init
services as shims which use a synchronization protocol to start
each cloud-init stage and to communicate that each stage is
complete to the init system. Since multiple cloud-init processes
sit in the critical chain of starting the system, this reduces
boot time (including time to ssh login and time to cloud-init
completion).
Currently only systemd is supported, but the synchronization
protocol should be capable of supporting other init systems
as well with minor changes.
Note: This enables many additional follow-on improvements that
eliminate redundant work. However, these potential improvements
are temporarily ignored. This commit has been structured to
minimize the changes required to capture the majority of primary
performance savings while preserving correctness and the ability
to preserve backwards compatibility.
Since this changes the semantics of the existing cloud-init unit
files, this change takes the opportunity to rename one of its
systemd units which causes frequent user confusion. The unit named
cloud-init.service is often mistaken by users for being the only
cloud-init service, when it is simply one of four stages. This
stage is documented as the "network" stage, so this service will
be renamed to "cloud-init-network.service". A new notify service
is added as part of this implementation which contains the
cloud-init process. This unit is named "cloud-init-main.service".
Synchronization protocol
========================
- create one Unix socket for each systemd service stage
- send sd_notify()
- For each of the four stages (local, network, config, final):
- when init system sends "start" to the Unix socket, start the
stage
- when running stage is complete, send "done" to Unix socket
File changes
============
socket.py (new)
---------------
- define a systemd-notify helper function
- define a context manager which implements a multi-socket
synchronization protocol
cloud-init.service -> cloud-init-network.service (renamed)
----------------------------------------------------------
- renamed to cloud-network.service
cloud-{init-local,init-network,config,final}.services
-------------------------------------------
- change ExecStart to use netcat to connect to Unix socket and:
- send a start message
- wait for completion response
- note: a pure Python equivalent is possible for any downstreams
which do not package openbsd's netcat
cloud-init-main.service (new)
-----------------------------
- use service type to 'notify'
- invoke cloud-init in single process mode
- adopt systemd ordering requirements from cloud-init-local.service
- adopt KillMode from cloud-final.service
main.py
-------
- Add command line flag to indicate "all stages" mode
- In this mode run each stage followed by an IPC
synchronization protocol step
cloud-final.services
--------------------
- drop KillMode
cloud-init-local.services
-------------------------
- drop dependencies made redundant by ordering after
cloud-init-main.service
Performance Impact
==================
On Ubuntu 24.04, Python's wall clock start up time as measured with
`time python3 -c 'import cloudinit.cmd.main' on a few cloud types:
lxc container: 0.256s
QEMU machine: 0.300s
gce instance: 0.367s
ec2 instance: 0.491s
This change eliminates x1 this start up time from time to ssh.
This change eliminates x3 this start up time from cloud-init's total
completion. Total benefit varies based on the platform that the
instance is hosted by, but all platforms will measurably benefit from
this change.
BREAKING_CHANGE: Run all four cloud-init services as a single systemd service.
Commit 5322dca introduced an assumption to read_seeded that network-config must always be present for NoCloud datasource. Since it is still considered and optional supplemental configuration allow the read_seeed calls to succeed in the absence of network-config. Avoids failures seen in tests/integration-tests/datasources/test_nocloud.py:: test_nocloud_seedfrom_vendordata
patches: debian/patches/no-nocloud-network.patch debian/patches/no-single-process.patch
Bump the version in cloudinit/version.py to 24.3 and update ChangeLog.
Drop unnecessary environment variable. Fixes canonicalGH-5648
Bump the version in cloudinit/version.py to 24.3.1 and update ChangeLog.
patches: debian/patches/no-nocloud-network.patch
Revert remaning functional references to cloud-init-network service which will not exist on stable releases.
patches: - d/p/no-single-process.patch
- d/p/no-single-process.patch
patches: debian/patches/no-single-process.patch
cloud-init project now requires full SHA commits instead of tags for all workflows to utilize a more secure policy for CI runners. Any Github actions lacking full SHA pins in workflows/actions will error due to repo prevention policy. Additionally update SHA pins to latest known workflow releases.
…cal#6707) Fixes DNS queries for IP addresses that cause 2+ minute boot delays with systemd 259+. Moves IP detection earlier in is_resolvable() and removes legacy DNS-dependent metadata URL. Fixes canonicalGH-6641
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Commit Message
Additional Context
Test Steps
Merge type