-
Notifications
You must be signed in to change notification settings - Fork 0
Package the documentation checkers and add a blocking translation gate #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
50aa666
3d222a3
9f9e033
9034e1f
0bb8ad0
abb1011
8fc2cff
8d54268
1c09268
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: [main] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| check: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| strategy: | ||
| matrix: | ||
| python-version: ["3.11", "3.12", "3.13"] | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: astral-sh/setup-uv@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
| - run: uv sync --locked | ||
| - run: uv run ruff check src/ tests/ | ||
| - run: uv run pytest |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # Byte-compiled / optimized / DLL files | ||
| __pycache__/ | ||
| *.py[cod] | ||
|
|
||
| # Distribution / packaging | ||
| build/ | ||
| dist/ | ||
| *.egg-info/ | ||
| .eggs/ | ||
|
|
||
| # Virtual environments | ||
| .venv/ | ||
| venv/ | ||
|
|
||
| # Test and coverage artifacts | ||
| .pytest_cache/ | ||
| .coverage | ||
| htmlcov/ | ||
|
|
||
| # Tooling caches | ||
| .ruff_cache/ | ||
|
|
||
| # Local workflow artifacts | ||
| .worktrees/ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| # halos-docs-tools - Agent Context | ||
|
|
||
| **Document Purpose**: Context for AI assistants working in this repository. | ||
|
|
||
| ## What this is | ||
|
|
||
| Six documentation checkers, packaged so that a MkDocs documentation repository | ||
| can pin them and get the identical code in CI and on a developer's machine. The | ||
| checkers were extracted from `hatlabs/halpi2`, where they lived as `scripts/`. | ||
|
|
||
| Intended consumers. Neither pins the package yet; both migrations are still | ||
| open, so `hatlabs/halpi2` still runs its own copies under `scripts/`: | ||
|
|
||
| - `hatlabs/halpi2` — nine locales, full translation gate | ||
| - `halos-org/docs.halos.fi` — no translations, anchor validation only | ||
|
|
||
| ## The six commands | ||
|
|
||
| Console entry points are declared in `pyproject.toml` under `[project.scripts]`. | ||
| **They are a public interface.** A second repository pins them, and the | ||
| `translate-page` skill in `hatlabs/halpi2` calls them by name. Renaming one is a | ||
| breaking change for every consumer. | ||
|
|
||
| Each module lives at `src/halos_docs_tools/<name>.py` and exposes `main()`. | ||
|
|
||
| ## Conventions | ||
|
|
||
| - Commands run from the root of a documentation repository. They read `docs/` | ||
| and `mkdocs.yml` relative to the working directory. Do not add hidden defaults | ||
| that make them work from elsewhere without saying so. | ||
| - Glossaries and language rules are repository content, not package content. | ||
| Where a checker needs them, the path is a CLI option with a default. | ||
| - The translation stamp format — `translated_from` in frontmatter, holding a git | ||
| blob hash — is fixed. Consumers have thousands of pages carrying it. | ||
|
|
||
| ## Distribution | ||
|
|
||
| This repository produces no `.deb`. There is no `VERSION` file, no | ||
| `debian/changelog`, and no APT dispatch — the workspace version-bump policy | ||
| governs `.deb`-producing repositories and does not apply here. | ||
|
|
||
| Releases are `pyproject.toml` version plus a `vX.Y.Z` git tag. Consumers pin the | ||
| tag. | ||
|
|
||
| ## Development | ||
|
|
||
| ``` | ||
| ./run deps install dependencies | ||
| ./run test run the test suite | ||
| ./run lint check with ruff | ||
| ./run check lint and test, as CI does | ||
| ``` | ||
|
|
||
| Install the pre-commit hooks after cloning with `./run install-hooks`. | ||
|
|
||
| ## Testing | ||
|
|
||
| Tests build a miniature documentation tree in a throwaway git repository — | ||
| `tests/conftest.py` — because the staleness check reads real git blob hashes. | ||
| Reuse those fixtures rather than mocking `git`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| @AGENTS.md |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| MIT License | ||
|
|
||
| Copyright (c) 2026 Hat Labs Oy | ||
|
|
||
| Permission is hereby granted, free of charge, to any person obtaining a copy | ||
| of this software and associated documentation files (the "Software"), to deal | ||
| in the Software without restriction, including without limitation the rights | ||
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
| copies of the Software, and to permit persons to whom the Software is | ||
| furnished to do so, subject to the following conditions: | ||
|
|
||
| The above copyright notice and this permission notice shall be included in all | ||
| copies or substantial portions of the Software. | ||
|
|
||
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
| SOFTWARE. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,133 @@ | ||
| # halos-docs-tools | ||
|
|
||
| Documentation checkers for HaLOS and Hat Labs MkDocs sites: translation status, | ||
| stamping, anchor validation, glossary and typography checks. | ||
|
|
||
| The same code runs in CI and on a laptop. Every check a pull request must pass is | ||
| runnable before you push. | ||
|
|
||
| ## Installing | ||
|
|
||
| Add it to a documentation repository's `pyproject.toml`, pinned to a tag: | ||
|
|
||
| ```toml | ||
| dependencies = [ | ||
| "halos-docs-tools @ git+https://github.com/halos-org/docs-tools@vX.Y.Z", | ||
| ] | ||
| ``` | ||
|
|
||
| Use a tag from the [releases page](https://github.com/halos-org/docs-tools/releases). | ||
| `uv sync` then puts all six commands on the path. Each repository pins its own | ||
| version; upgrading is a deliberate edit to that pin. | ||
|
|
||
| `git` must be on the path. `translation-status`, `stamp-translation` and | ||
| `check-glossary` shell out to it, and it is not something a Python dependency | ||
| can bring. | ||
|
|
||
| ## Commands | ||
|
|
||
| Run them from the root of a documentation repository — they expect `docs/` and | ||
| `mkdocs.yml` in the working directory. | ||
|
|
||
| | Command | Purpose | | ||
| |:---|:---| | ||
| | `translation-status` | report which translations are missing or out of date | | ||
| | `stamp-translation` | record the English source a translation was written against | | ||
| | `map-anchors` | rewrite English anchor fragments to the translated slugs | | ||
| | `check-glossary` | verify a translation uses the terms its glossary prescribes | | ||
| | `check-typography` | check quotation pairing and unit spacing per language | | ||
| | `check-anchors` | verify every internal anchor in a built site resolves | | ||
|
|
||
| ### Exit statuses | ||
|
|
||
| A workflow branches on these, so they are part of the interface. | ||
|
|
||
| | Status | Meaning | | ||
| |:---|:---| | ||
| | 0 | the check passed | | ||
| | 1 | the check found problems — broken anchors, unused glossary terms, typography faults, or (with `--check`) translations that are not current | | ||
| | 2 | the check could not run: `check-anchors` was given a site directory holding no built pages | | ||
|
|
||
| Glossaries and per-language rules stay in the documentation repository. This | ||
| package brings the checkers, not the terminology. | ||
|
|
||
| ### Generated pages and `check-anchors` | ||
|
|
||
| Some plugins generate a page whose internal fragments the checker cannot | ||
| resolve. `mkdocs-print-site-plugin` is one: on `docs.halos.fi` its single-page | ||
| export accounts for 690 broken fragments while the 36 content pages are clean. | ||
| Exclude such pages by path pattern: | ||
|
|
||
| ``` | ||
| check-anchors site --exclude 'print_page/*' | ||
| ``` | ||
|
|
||
| An excluded page contributes no links to the check. Its own headings stay | ||
| linkable, so other pages may still point into it. | ||
|
|
||
| `--base`, used to resolve root-absolute links, is read from `site_url` in | ||
| `mkdocs.yml`. A base that does not match the site makes the checker skip every | ||
| root-absolute link and report a pass it did not earn, so override it only when | ||
| you know the built site differs from the configuration. | ||
|
|
||
| ## How translation staleness is detected | ||
|
|
||
| A translation records the git blob hash of the English page it was written | ||
| against, in its own frontmatter: | ||
|
|
||
| ```yaml | ||
| --- | ||
| translated_from: <blob hash of docs/en/<path> at translation time> | ||
| --- | ||
| ``` | ||
|
|
||
| The English page carries nothing. Editing it changes its content, which changes | ||
| its hash, which makes every translation of it report as stale on its own. | ||
|
|
||
| `translation-status` classifies each page in each configured locale as `current`, | ||
| `stale`, `missing`, `unstamped` or `orphaned`. | ||
|
|
||
| ### Failing a build on it | ||
|
|
||
| ``` | ||
| translation-status --check | ||
| ``` | ||
|
|
||
| exits non-zero when any page in any configured locale is anything but | ||
| `current`, and names every entry responsible. Without `--check` the command | ||
| only reports, whatever it finds. | ||
|
|
||
| The gate is a property of the repository, not of a pull request's diff, so | ||
| `--only-pages` narrows the report and never the rule. | ||
|
|
||
| One consequence is worth knowing before you meet it: adding a locale to | ||
| `mkdocs.yml` makes every page `missing` in that locale immediately. A new | ||
| locale therefore arrives in a single pull request, together with its pages. | ||
|
|
||
| ### The pull request comment | ||
|
|
||
| ``` | ||
| translation-status --comment > body.md | ||
| ``` | ||
|
|
||
| writes a comment body describing every entry the gate fails on, with the | ||
| English changes since each translation was written, collapsed. The command | ||
| writes a body and nothing else; posting it belongs to whatever holds the token. | ||
|
|
||
| The body carries a `<!-- translation-status -->` marker so a workflow can find | ||
| and update its own previous comment rather than adding another one. If the body | ||
| would exceed GitHub's 65536-character limit, the diffs come out and the reader | ||
| is pointed at the job summary for them. | ||
|
Comment on lines
+117
to
+120
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win State the 60,000-character comment cap. The package requirement sets a 60,000-character limit. These lines instead state that diffs are removed only above GitHub's 65,536-character limit. Document the command's actual cap so workflow authors do not expect larger comment bodies. 🤖 Prompt for AI Agents |
||
|
|
||
| ## Development | ||
|
|
||
| ``` | ||
| ./run deps install dependencies | ||
| ./run test run the test suite | ||
| ./run lint check with ruff | ||
| ./run check lint and test, as CI does | ||
| ``` | ||
|
|
||
| ## License | ||
|
|
||
| MIT. Copyright Hat Labs Oy. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| pre-commit: | ||
| parallel: true | ||
| commands: | ||
| ruff-check: | ||
| glob: "*.py" | ||
| run: | | ||
| if command -v uv >/dev/null 2>&1; then | ||
| uv run ruff check src/ tests/ | ||
| elif command -v ruff >/dev/null 2>&1; then | ||
| ruff check src/ tests/ | ||
| else | ||
| echo "WARNING: ruff not found, skipping" | ||
| exit 0 | ||
| fi | ||
| fail_text: "Run 'ruff check src/ tests/' to see issues." | ||
|
|
||
| pytest: | ||
| glob: "*.py" | ||
| run: | | ||
| if command -v uv >/dev/null 2>&1; then | ||
| uv run pytest -q | ||
| else | ||
| echo "WARNING: uv not found, skipping" | ||
| exit 0 | ||
| fi | ||
| fail_text: "Run './run test' to see the failures." |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| [build-system] | ||
| requires = ["setuptools>=65.0", "wheel"] | ||
| build-backend = "setuptools.build_meta" | ||
|
|
||
| [project] | ||
| name = "halos-docs-tools" | ||
| version = "0.1.0" | ||
| description = "Documentation checkers for HaLOS and Hat Labs MkDocs sites" | ||
| readme = "README.md" | ||
| requires-python = ">=3.11" | ||
| license = {text = "MIT"} | ||
| authors = [ | ||
| {name = "Hat Labs Oy", email = "info@hatlabs.fi"} | ||
| ] | ||
| keywords = ["mkdocs", "documentation", "translation", "i18n"] | ||
| classifiers = [ | ||
| "Development Status :: 3 - Alpha", | ||
| "Intended Audience :: Developers", | ||
| "License :: OSI Approved :: MIT License", | ||
| "Programming Language :: Python :: 3", | ||
| "Programming Language :: Python :: 3.11", | ||
| "Programming Language :: Python :: 3.12", | ||
| "Programming Language :: Python :: 3.13", | ||
| "Topic :: Documentation", | ||
| "Topic :: Software Development :: Quality Assurance", | ||
| ] | ||
|
|
||
| dependencies = [ | ||
| "pyyaml>=6.0", | ||
| ] | ||
|
|
||
| [dependency-groups] | ||
| dev = [ | ||
| "pytest>=7.0", | ||
| "ruff>=0.1.0", | ||
| ] | ||
|
|
||
| [project.scripts] | ||
| translation-status = "halos_docs_tools.translation_status:main" | ||
| stamp-translation = "halos_docs_tools.stamp_translation:main" | ||
| map-anchors = "halos_docs_tools.map_anchors:main" | ||
| check-glossary = "halos_docs_tools.check_glossary:main" | ||
| check-typography = "halos_docs_tools.check_typography:main" | ||
| check-anchors = "halos_docs_tools.check_anchors:main" | ||
|
|
||
| [project.urls] | ||
| Homepage = "https://github.com/halos-org/docs-tools" | ||
| Repository = "https://github.com/halos-org/docs-tools" | ||
| Issues = "https://github.com/halos-org/docs-tools/issues" | ||
|
|
||
| [tool.setuptools] | ||
| package-dir = {"" = "src"} | ||
|
|
||
| [tool.setuptools.packages.find] | ||
| where = ["src"] | ||
|
|
||
| [tool.pytest.ini_options] | ||
| testpaths = ["tests"] | ||
| python_files = ["test_*.py"] | ||
| python_functions = ["test_*"] | ||
| addopts = [ | ||
| "-ra", | ||
| "--strict-markers", | ||
| "--strict-config", | ||
| ] | ||
|
|
||
| [tool.ruff] | ||
| line-length = 88 | ||
| target-version = "py311" | ||
|
|
||
| [tool.ruff.lint] | ||
| select = [ | ||
| "E", # pycodestyle errors | ||
| "W", # pycodestyle warnings | ||
| "F", # pyflakes | ||
| "I", # isort | ||
| "B", # flake8-bugbear | ||
| "C4", # flake8-comprehensions | ||
| "UP", # pyupgrade | ||
| ] | ||
| ignore = [ | ||
| "E501", # line too long (handled by formatter) | ||
| ] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Document status 2 for every checker.
Line 49 assigns status 2 only to an empty built site for
check-anchors.translation-status --check,check-glossary, andcheck-typographyalso return 2 when they cannot inspect required content. Define status 2 as a checker could not complete its required inspection, then list examples.🤖 Prompt for AI Agents