add: container test plugin for pytest#17372
Open
realsdx wants to merge 3 commits into
Open
Conversation
realsdx
commented
May 20, 2026
| images with a given capability. | ||
| - **Image-specific:** add `cases/<image-family>/test_<topic>.py`. Tests | ||
| in such subdirectories are **automatically** restricted to that | ||
| - **Image-specific static:** add `cases/static/<image-family>/test_<topic>.py`. |
Contributor
Author
There was a problem hiding this comment.
I have odd feeling about these directory-based tests discovery, markers might be more useful. As you add more and more tests, directories and subdirectories just for categorization might become too complex.
12 tasks
reubeno
reviewed
May 22, 2026
| @pytest.mark.dockerfile() | ||
| def test_nginx_config_valid(container_exec) -> None: | ||
| """nginx configuration must pass validation.""" | ||
| result = container_exec("nginx -t 2>&1") |
Contributor
Author
There was a problem hiding this comment.
I assume agent did this, cause by default nginx -t outputs to stderr
| COPY nginx.conf /etc/nginx/nginx.conf | ||
| ``` | ||
|
|
||
| Mark tests with `@pytest.mark.dockerfile()` to trigger the build. The |
realsdx
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Azure Linux’s image-test framework to support runtime validation of OCI container images (in addition to existing static/offline filesystem checks) and wires the new runtime suite into azldev image test.
Changes:
- Added
--image-refsupport (mutually exclusive with--image-path) and new pytest markers for runtime/container and Dockerfile-based tests. - Introduced a
python-on-whales/Podman-based container runtime helper plus session/function fixtures to runpodman exec-style tests. - Restructured test layout into
cases/static/vscases/runtime/and added animages.tomlruntime test-suite forcontainer-basevariants.
Reviewed changes
Copilot reviewed 10 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| base/images/tests/utils/pytest_plugin.py | Adds --image-ref, new markers, and collection-time auto-marking for runtime/static test layout. |
| base/images/tests/utils/container_runtime.py | New Podman/python-on-whales orchestration helpers (load/pull/build/run/exec/cleanup). |
| base/images/tests/conftest.py | Adds runtime container fixtures (podman_client, running_container, container_exec, etc.) and supports --image-ref. |
| base/images/tests/README.md | Updates documentation to reflect static vs runtime suites and new CLI options/fixtures. |
| base/images/tests/pyproject.toml | Adds python-on-whales dependency for runtime container orchestration. |
| base/images/tests/cases/static/test_os_release.py | New shared static validation for /etc/os-release. |
| base/images/tests/cases/static/test_packages.py | New shared static RPM/package presence/absence checks (capability-gated). |
| base/images/tests/cases/static/vm-base/test_kernel.py | Adds VM kernel/modules checks and Fedora 43 CONFIG_LSM assertion. |
| base/images/tests/cases/static/vm-base/test_partitions.py | Adds VM partition layout checks (root + EFI). |
| base/images/tests/cases/static/container-base/test_container.py | Adds container static validation (no kernel modules). |
| base/images/tests/cases/runtime/container-base/test_basic.py | Adds basic runtime exec tests (shell + DNS). |
| base/images/tests/cases/runtime/container-base/test_nginx/* | Adds Dockerfile-based runtime example for nginx validation. |
| base/images/images.toml | Wires runtime-container-tests into container-base images and splits suite test-paths into cases/static/ and cases/runtime/. |
Comments suppressed due to low confidence (1)
base/images/tests/utils/pytest_plugin.py:156
pytest_configurederivesimage_typefrom--image-type,--capabilities, or--image-path, but it doesn’t consider--image-ref. When running with--image-refand no--capabilities/--image-type,image_typestaysNoneandcheck_tools(when=None)ends up requiring all tools (VM + container), which breaks the intended registry-ref workflow. Treat--image-refas implyingimage_type="container"during this early preflight derivation.
image_type = config.getoption("--image-type", default=None)
if image_type is None:
caps = parse_capabilities(config.getoption("--capabilities", default=None))
if caps:
image_type = derive_image_type_from_capabilities(caps)
Comment on lines
+14
to
+21
| from typing import NamedTuple | ||
|
|
||
| from python_on_whales import DockerClient | ||
| from python_on_whales.components.container.cli_wrapper import Container | ||
| from python_on_whales.exceptions import DockerException, NoSuchContainer, NoSuchImage | ||
|
|
||
| from .tools import NativeTool | ||
|
|
Comment on lines
+199
to
+203
| image_ref, | ||
| command=["sleep", "infinity"], | ||
| name=container_name, | ||
| detach=True, | ||
| ) |
Comment on lines
+99
to
+103
| # Load from archive file. | ||
| assert image_path is not None | ||
| logger.info("Loading image archive: %s", image_path) | ||
| image_refs = client.image.load(image_path) | ||
|
|
| @@ -0,0 +1,5 @@ | |||
| ARG BASE_IMAGE | |||
| FROM ${BASE_IMAGE} | |||
| RUN dnf install -y nginx && dnf clean all | |||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a
pytestbased validation framework AzureLinux's OCI container images, integrated withazldev image testTest it locally by (make sure to install and start podman first):