From ef851c638e806a6d9a252ffcdaa5830204c218d2 Mon Sep 17 00:00:00 2001 From: Olaf Monien Date: Sat, 25 Jul 2026 23:58:25 +0200 Subject: [PATCH 1/2] Docs: PR #90 split status index for fork packages A/B/C Record stack order, fork-only policy, upstream sequence, and links to per-package plans so later MHumm PRs do not rely on branch memory alone. Includes copies of the A/B/C design notes for a self-contained docs branch. --- Docs/plans/2026-07-25-aead-architecture.md | 64 ++++++++++++++ Docs/plans/2026-07-25-aes-ni.md | 83 ++++++++++++++++++ Docs/plans/2026-07-25-chacha-poly1305.md | 41 +++++++++ Docs/plans/2026-07-25-pr90-split-status.md | 97 ++++++++++++++++++++++ 4 files changed, 285 insertions(+) create mode 100644 Docs/plans/2026-07-25-aead-architecture.md create mode 100644 Docs/plans/2026-07-25-aes-ni.md create mode 100644 Docs/plans/2026-07-25-chacha-poly1305.md create mode 100644 Docs/plans/2026-07-25-pr90-split-status.md diff --git a/Docs/plans/2026-07-25-aead-architecture.md b/Docs/plans/2026-07-25-aead-architecture.md new file mode 100644 index 00000000..6aa361a7 --- /dev/null +++ b/Docs/plans/2026-07-25-aead-architecture.md @@ -0,0 +1,64 @@ +# AEAD Architecture Package (from PR #90 Concern A) + +**Branch:** `package/aead-architecture` +**Base:** `development` +**Donor:** [PR #90](https://github.com/MHumm/DelphiEncryptionCompendium/pull/90) (architecture only) +**GCM streaming reference:** PR #99 multi-call GCM semantics (absorbed) + +## Goal + +Separate AEAD architecture from ChaCha/Poly1305 (Cleanup-Roadmap Concern A vs B). + +## What landed + +| Area | Decision | +|------|----------| +| Mode object | Single `FAuthObj: TAuthenticatedCipherModesBase` instead of dual `FGCM`/`FCCM` | +| Public API | `IDECAuthenticatedCipher` unchanged | +| Protected API | `EncodeGCM`/`DecodeGCM`/`EncodeCCM`/`DecodeCCM` kept as dispatch wrappers (no rename break) | +| GCM streaming | PR #99 engine: GHASH partial + CTR keystream remainder + `FFinalized` | +| Tag timing | Tag is valid only after `Done` (multi-call GHASH). Callers must `Done` before reading the tag | +| CCM | Still one-shot; base `Done` is a no-op for the CCM object; ExpectedTag still verified in `TDECCipherModes.Done` | +| Poly1305 / ChaCha | **Out of scope** (package B) | +| `cmPoly1305` enum | **Not** added in this package | + +## What was rejected from PR #90 (as-is) + +1. **`fIsLastBlock` CTR model** — treats any non-16-aligned *call* as end of message; breaks multi-chunk streams (e.g. 7+25). +2. **Missing keystream remainder** — wrong ciphertext after partial blocks. +3. **Abstract hooks with empty `inherited`** — EAbstractError risk on CCM stubs. +4. **Burn-on-finalize of only H** without post-Done lock — unsafe continued Encode after Done. +5. **Shipping Poly1305/ChaCha/CPU units** with the architecture package. + +## Lifecycle (binding) + +``` +Init(Key, IV) + → set DataToAuthenticate / AuthenticationResultBitLength / ExpectedAuthenticationResult + → Encode* or Decode* (multi-call OK for GCM; one-shot for CCM) + → Done (finalizes GCM tag; verifies ExpectedTag if set) + → read CalculatedAuthenticationResult +``` + +After `Done`, further GCM `Encode`/`Decode` raise until `Init` again. `Done` is idempotent for the tag. + +## Files + +| File | Role | +|------|------| +| `Source/DECAuthenticatedCipherModesBase.pas` | Shared AEAD base + virtual `Done` | +| `Source/DECCipherModes.pas` | `FAuthObj` wiring, leak-safe `InitMode`, unified `Done` | +| `Source/DECCipherModesGCM.pas` | Multi-call GCM (PR #99 semantics) | +| `Unit Tests/Tests/TestDECCipherModesGCM.pas` | Multi-chunk + Done lifecycle tests | +| `Unit Tests/Data/gcmEncryptExtIV256_large.rsp` | Corrected large-vector tag | + +## Test results (Delphi 13, Win32 Console DUnit) + +- GCM suite: **19/19** green (including multi-chunk 16+16, 7+25, Done lifecycle) +- CCM suite: **16/16** green +- Non-AEAD cipher modes: green +- Residual reds: pre-existing Keccak vector issues only (separate PRs #98/#100) + +## Package B next + +ChaCha20 / XChaCha20 / Poly1305 AEAD on top of this base (`FAuthObj` + `Done` contract). diff --git a/Docs/plans/2026-07-25-aes-ni.md b/Docs/plans/2026-07-25-aes-ni.md new file mode 100644 index 00000000..73df168e --- /dev/null +++ b/Docs/plans/2026-07-25-aes-ni.md @@ -0,0 +1,83 @@ +# Package C — AES-NI acceleration with mandatory portable fallback + +**Branch:** `package/aes-ni` +**Base:** package B (`package/chacha-poly1305`) +**Donor:** git ref `pr-90` AES assembler in `Source/DECCiphers.pas` + `UseAESAsm` + +## Goal + +Accelerate AES/Rijndael on x86/x64 CPUs that support AES-NI, without ever +requiring AES-NI for correctness. Pure Pascal remains the default portable path +and always compiles. + +## Cross-platform policy + +| Environment | AES-NI machine code | Runtime path | +|-------------|---------------------|--------------| +| Delphi Win32/Win64 (X86ASM/X64ASM) + AES-NI CPU | Compiled | AES-NI when `UseAESAsm` | +| Same build, CPU without AES-NI | Compiled | Pure Pascal | +| PUREPASCAL / NO_ASM / FPC / non-x86 / ARM / mobile | **Not compiled** | Pure Pascal only | + +Correctness never depends on AES-NI. Hardware is an optional speedup. + +## Compile-time vs runtime gates + +1. **Compile-time:** all AES-NI procedures (`BuildAsmKey128/192/256`, + `BuildAsmDecodeKey`, `AESEncode`, `AESDecode`) are wrapped in + `{$IF defined(X86ASM) or defined(X64ASM)}`. +2. **Runtime (init):** AES-NI key schedule only if + `UseAESAsm and TDEC_CPUSupport.AES and (Size in [16,24,32])`. +3. **Runtime (encode/decode):** use AES-NI only when instance flag + `FAESAsmActive` was set by `DoInit` (avoids mismatch if key size was + non-standard or `UseAESAsm` changed after init). + +Default at unit init: + +```pascal +TCipher_Rijndael.UseAESAsm := + {$IF defined(X86ASM) or defined(X64ASM)} + TDEC_CPUSupport.AES + {$ELSE} + False + {$IFEND}; +``` + +## Force pure Pascal + +```pascal +TCipher_Rijndael.UseAESAsm := False; // before Init; also covers TCipher_AES* +``` + +Tests force PAS and AES-NI independently and require identical ciphertext. + +## Key schedule notes + +- Donor name `BuildAsmKey196` renamed to **`BuildAsmKey192`** (Intel 192-bit schedule). +- Decode schedule uses **AESIMC** (`BuildAsmDecodeKey`), reverse-ordered for sequential `AESDEC`. +- `AdditionalBufferSize` gains `+$20` so `AlignPtr32(FAdditionalBuffer)` plus the + max AES-NI schedule fits. PAS path still uses `FAdditionalBufferSize shr 1` + for encode/decode halves. +- Non-standard Rijndael key lengths (not 16/24/32) always use the PAS schedule. + +## Intel reference + +AES-NI instruction whitepaper: +https://www.intel.com/content/dam/develop/external/us/en/documents/aes-wp-2012-09-22-v01-165683.pdf + +## Files + +- `Source/DECCiphers.pas` — `UseAESAsm`, AES-NI kernels, DoInit/DoEncode/DoDecode dispatch +- `Unit Tests/Tests/TestDECAESNI.pas` — FIPS-197 ECB KATs + PAS vs AES-NI +- Relies on existing `DECCPUSupport.AES`, `AlignPtr32` (no change required) + +## Out of scope + +- ARM Crypto Extensions / ARMv8 AES (future) +- ChaCha / Poly1305 / GCM changes +- Auto-enable on non-Windows x86 unless DEC already defines X86ASM there + +## Residual risks + +- Non-16/24/32 key sizes never use AES-NI (intentional; PAS only). +- Changing `UseAESAsm` after `Init` does not switch path until re-`Init` + (`FAESAsmActive` is set at init time). diff --git a/Docs/plans/2026-07-25-chacha-poly1305.md b/Docs/plans/2026-07-25-chacha-poly1305.md new file mode 100644 index 00000000..f7757060 --- /dev/null +++ b/Docs/plans/2026-07-25-chacha-poly1305.md @@ -0,0 +1,41 @@ +# Package B — ChaCha20 / XChaCha20 / Poly1305 AEAD + +**Branch:** `package/chacha-poly1305` +**Base:** package A (`package/aead-architecture` @ ~47c7463) +**Donor:** git ref `pr-90` (d6eb393) — algorithm only + +## Goal + +Ship ChaCha20, XChaCha20, and Poly1305 AEAD without re-applying PR #90’s AEAD base / +GCM rewrite. Package A architecture stays authoritative. + +## Decisions + +| Topic | Decision | +|-------|----------| +| Base AEAD API | Keep package A: virtual `Done`, abstract `Encode`/`Decode` on `TAuthenticatedCipherModesBase`. No `InitAuth` / `UpdateWithEncDecBuf` / `FinalizeMAC` template. | +| Single auth object | `FAuthObj` only; `cmPoly1305` → `TPoly1305.Create` | +| Lifecycle | Init → AAD/tag props → Encode*/Decode* → Done → tag | +| Protected names | `EncodeGCM` / `DecodeGCM` dispatch retained for all auth modes | +| Poly1305 kernel default | **`pmPas`** (pure Pascal). AVX kept behind `CpuMode` / `{$IFNDEF PUREPASCAL}` but not auto-selected. | +| ChaCha kernel default | **`cmPas`**. SSE/AVX optional via `TCipher_ChaCha20.CpuMode`; no auto-upgrade on init. | +| AES-NI / UseAESAsm | **Not ported** from PR #90. | +| ChaCha default Mode | **Not** `cmPoly1305`. AEAD is opt-in (`Mode := cmPoly1305`). | +| Poly key for AEAD | ChaCha `OnAfterInitVectorInitialization`: counter 0 → 32-byte one-time key → `FAuthObj.Init`; payload continues at counter 1. | +| AVX FinalizePoly1305 bug | Fixed limb masking (`g[1]/`/`g[2]`/`g[3]` not repeated `g[0]`). | +| Empty PT / AAD-only | `DECCipherFormats` empty path uses `IsAuthenticatedBlockMode` (GCM, CCM, Poly1305). | +| Speed unit tests | Omitted (CI-flaky / non-functional). | + +## Files (summary) + +- New: `Source/DECCPUSupport.pas`, `Source/DECCipherModesPoly1305.pas` +- Port: `TCipher_ChaCha20`, `TCipher_XChaCha20` in `Source/DECCiphers.pas` +- Wire: `DECCipherBase`, `DECCipherModes`, `DECCipherFormats`, `DECUtil` (`AlignPtr32`) +- Tests: `Unit Tests/Tests/TestDECChaChaPoly1305.pas`, `Unit Tests/Data/chacha20_poly1305_test.json` + +## Out of scope + +- GCM/CCM algorithm changes +- PR #90 dual-object auth redesign +- AES-NI +- Default-enable AVX Poly1305 diff --git a/Docs/plans/2026-07-25-pr90-split-status.md b/Docs/plans/2026-07-25-pr90-split-status.md new file mode 100644 index 00000000..9875152e --- /dev/null +++ b/Docs/plans/2026-07-25-pr90-split-status.md @@ -0,0 +1,97 @@ +# PR #90 split — fork status (A / B / C) + +**Status date:** 2026-07-25 +**Audience:** maintainers working on the `omonien` fork +**Policy:** work stays on the **fork only** until earlier open PRs against `MHumm/development` are reviewed. **Do not open A/B/C against upstream yet** unless that policy changes. + +This document is the **index**. Per-package design lives in the linked plans. Code lives on the package branches (not necessarily on `development` yet). + +--- + +## Why this exists + +Open contribution [PR #90](https://github.com/MHumm/DelphiEncryptionCompendium/pull/90) mixed two concerns (AEAD architecture + ChaCha/Poly1305) plus AES-NI. Merging it as one unit was rejected; work was re-landed as reviewable packages on the fork. + +Branch ancestry alone shows *what* is stacked. This file records *intent*, *order*, and *what not to do*. + +--- + +## Package stack (fork) + +``` +origin/development + └── package/aead-architecture (A) AEAD FAuthObj + multi-call GCM + └── package/chacha-poly1305 (B) ChaCha20 / XChaCha20 / Poly1305 + └── package/aes-ni (C) AES-NI with mandatory pure-Pascal fallback +``` + +| Package | Branch | Fork PR (internal) | Detail plan | +|---------|--------|--------------------|-------------| +| **A** Architecture | `package/aead-architecture` | [omonien#5](https://github.com/omonien/DelphiEncryptionCompendium/pull/5) | [2026-07-25-aead-architecture.md](./2026-07-25-aead-architecture.md) | +| **B** ChaCha/Poly1305 | `package/chacha-poly1305` | [omonien#6](https://github.com/omonien/DelphiEncryptionCompendium/pull/6) | [2026-07-25-chacha-poly1305.md](./2026-07-25-chacha-poly1305.md) | +| **C** AES-NI | `package/aes-ni` | [omonien#7](https://github.com/omonien/DelphiEncryptionCompendium/pull/7) | [2026-07-25-aes-ni.md](./2026-07-25-aes-ni.md) | + +Fork PRs #5–#7 are for **tracking / internal review** on the fork. They are not a substitute for future PRs into `MHumm/DelphiEncryptionCompendium`. + +--- + +## Upstream strategy (when ready) + +Target base for each: **`MHumm/development`** (not `master`), after any prerequisite cleanup PRs Markus has accepted. + +| Step | Open against `MHumm/development` | Head (fork) | Depends on | +|------|----------------------------------|-------------|------------| +| 1 | Package A | `omonien:package/aead-architecture` | Current `development` | +| 2 | Package B | `omonien:package/chacha-poly1305` | **A merged** (else re-target / restack) | +| 3 | Package C | `omonien:package/aes-ni` | **B merged** (or A+B equivalent) | + +**Do not** open B or C against upstream before A is in (or carefully restacked onto post-A `development`). + +### Relation to other work + +| Item | Role | +|------|------| +| [PR #90](https://github.com/MHumm/DelphiEncryptionCompendium/pull/90) | **Donor / reference only** — do not merge as a single unit | +| GCM multi-chunk (fork `pr-fix-gcm-multichunk` / upstream PR #99 if still open) | Semantics **absorbed into A**; if #99 merges first, expect a small GCM-file conflict when landing A | +| Older fork cleanup branches (`pr-gitignore`, `pr-dunitx-migration`, Keccak/SHA3 fixes, `docs/style-guide`, …) | Independent of A/B/C; leave until Markus finishes those reviews | +| Residual Keccak unit failures on `development` | Pre-existing; not introduced by A/B/C | + +--- + +## One-line package summaries + +- **A:** One `FAuthObj` for authenticated modes; GCM multi-call + `Done` lifecycle; public `IDECAuthenticatedCipher` unchanged; no ChaCha. +- **B:** ChaCha20 / XChaCha20 / Poly1305 on A’s lifecycle; AEAD opt-in (`cmPoly1305`); portable PAS default for SIMD. +- **C:** AES-NI on x86/x64 only when compiled and CPU supports it; **pure Pascal always remains**; no ARM Crypto Extensions yet. + +--- + +## Verification snapshot (at package completion) + +Delphi 13, Win32 Console DUnit (`DECDUnitTestSuite`), run from `Compiled/BIN_IDE_Win32_Console`: + +| After | Notable green suites | +|-------|----------------------| +| A | GCM multi-chunk + Done lifecycle; CCM regression | +| B | + ChaCha20Poly1305 suite; GCM/CCM still green | +| C | + AES-NI FIPS KATs and PAS↔AES-NI match; ChaCha/GCM/CCM still green | + +Known reds on full suite at that time: pre-existing **Keccak** vector issues only (separate fix PRs). + +--- + +## Hygiene (not done yet — intentional) + +- Old `pr-*` / `Cleanup_OM` / `docs/style-guide` branches remain on the fork until earlier upstream reviews settle. +- Local `pr-90` ref (if present) is only a donor checkout; safe to delete anytime. +- After each package merges upstream, the corresponding `package/*` branch can be deleted or archived. + +--- + +## This docs branch + +**Branch:** `docs/pr90-split-status` +**Contents:** this index + copies of the three package plans (so the branch is readable without checking out A/B/C). +**Code:** none. Safe to open as a docs-only PR to the fork or to upstream later if useful. + +When a package plan is updated on `package/*`, refresh the copy here if this branch is still the status SSOT. From 436421278a58172554194638444b1f4986ae841f Mon Sep 17 00:00:00 2001 From: Olaf Monien Date: Mon, 27 Jul 2026 13:42:03 +0200 Subject: [PATCH 2/2] Docs: record 2026-07-27 stack restack and Bugbot/Augment fixes for A/B/C. Linear package ancestry restored; document addressed review findings and verification snapshot (GCM 21, ChaCha 12, AES-NI 7). --- Docs/plans/2026-07-25-pr90-split-status.md | 46 +++++++++++++++------- 1 file changed, 31 insertions(+), 15 deletions(-) diff --git a/Docs/plans/2026-07-25-pr90-split-status.md b/Docs/plans/2026-07-25-pr90-split-status.md index 9875152e..8fe34d06 100644 --- a/Docs/plans/2026-07-25-pr90-split-status.md +++ b/Docs/plans/2026-07-25-pr90-split-status.md @@ -1,6 +1,6 @@ # PR #90 split — fork status (A / B / C) -**Status date:** 2026-07-25 +**Status date:** 2026-07-27 **Audience:** maintainers working on the `omonien` fork **Policy:** work stays on the **fork only** until earlier open PRs against `MHumm/development` are reviewed. **Do not open A/B/C against upstream yet** unless that policy changes. @@ -19,12 +19,15 @@ Branch ancestry alone shows *what* is stacked. This file records *intent*, *orde ## Package stack (fork) ``` -origin/development +fork/development (== origin/development) └── package/aead-architecture (A) AEAD FAuthObj + multi-call GCM └── package/chacha-poly1305 (B) ChaCha20 / XChaCha20 / Poly1305 └── package/aes-ni (C) AES-NI with mandatory pure-Pascal fallback ``` +**Linear ancestry (2026-07-27):** restacked so B is a descendant of A and C of B +(previously three parallel cherry-picks with identical content but conflicting GitHub merges). + | Package | Branch | Fork PR (internal) | Detail plan | |---------|--------|--------------------|-------------| | **A** Architecture | `package/aead-architecture` | [omonien#5](https://github.com/omonien/DelphiEncryptionCompendium/pull/5) | [2026-07-25-aead-architecture.md](./2026-07-25-aead-architecture.md) | @@ -35,6 +38,20 @@ Fork PRs #5–#7 are for **tracking / internal review** on the fork. They are no --- +## Bugbot / Augment findings — addressed 2026-07-27 + +| Package | Finding | Resolution | +|---------|---------|------------| +| **A** | Mid-stream AAD change broke GCM tag | `SetDataToAuthenticate` rejects after AAD absorbed / after `Done` | +| **A** | Tag bit length > 128 over-read 16-byte GHASH tag | Cap at 1..128 bits (`EDECAuthLengthException`) + defensive `Move` | +| **A** | `EncodeCCM` forwarded to `EncodeGCM` (override coupling) | Both call `FAuthObj.Encode/Decode` independently | +| **B** | Poly1305 + block cipher decrypt used `DoEncode` only | **Stream-only:** `cmPoly1305` requires `BlockSize < 16` (ChaCha); AES rejected | +| **B** | Poly init left `FBufferSize = 0` | Restore `Context.BufferSize` after one-time Poly key derivation | +| **B** | Win32 AVX Poly clobbered XMM6/7 | `StoreXMM`/`RestoreXMM` on X86ASM as well as X64 | +| **C** | AES-256 NI encode key overwritten by decode schedule | Last encode key only at +224; decode at `(FRounds+1)*Blocks`; matching `AESDecode` | + +--- + ## Upstream strategy (when ready) Target base for each: **`MHumm/development`** (not `master`), after any prerequisite cleanup PRs Markus has accepted. @@ -52,31 +69,30 @@ Target base for each: **`MHumm/development`** (not `master`), after any prerequi | Item | Role | |------|------| | [PR #90](https://github.com/MHumm/DelphiEncryptionCompendium/pull/90) | **Donor / reference only** — do not merge as a single unit | -| GCM multi-chunk (fork `pr-fix-gcm-multichunk` / upstream PR #99 if still open) | Semantics **absorbed into A**; if #99 merges first, expect a small GCM-file conflict when landing A | -| Older fork cleanup branches (`pr-gitignore`, `pr-dunitx-migration`, Keccak/SHA3 fixes, `docs/style-guide`, …) | Independent of A/B/C; leave until Markus finishes those reviews | -| Residual Keccak unit failures on `development` | Pre-existing; not introduced by A/B/C | +| GCM multi-chunk (upstream PR #99 if still open) | Semantics **absorbed into A**; if #99 merges first, expect a small GCM-file conflict when landing A | +| Older fork cleanup branches / upstream #96–#103 | Independent of A/B/C; leave until Markus finishes those reviews | +| Residual Keccak unit failures | Pre-existing on `development`; not introduced by A/B/C | --- ## One-line package summaries - **A:** One `FAuthObj` for authenticated modes; GCM multi-call + `Done` lifecycle; public `IDECAuthenticatedCipher` unchanged; no ChaCha. -- **B:** ChaCha20 / XChaCha20 / Poly1305 on A’s lifecycle; AEAD opt-in (`cmPoly1305`); portable PAS default for SIMD. +- **B:** ChaCha20 / XChaCha20 / Poly1305 on A's lifecycle; AEAD opt-in (`cmPoly1305`); stream-cipher only; portable PAS default for SIMD. - **C:** AES-NI on x86/x64 only when compiled and CPU supports it; **pure Pascal always remains**; no ARM Crypto Extensions yet. --- -## Verification snapshot (at package completion) - -Delphi 13, Win32 Console DUnit (`DECDUnitTestSuite`), run from `Compiled/BIN_IDE_Win32_Console`: +## Verification snapshot (2026-07-27, after Bugbot fixes) -| After | Notable green suites | -|-------|----------------------| -| A | GCM multi-chunk + Done lifecycle; CCM regression | -| B | + ChaCha20Poly1305 suite; GCM/CCM still green | -| C | + AES-NI FIPS KATs and PAS↔AES-NI match; ChaCha/GCM/CCM still green | +Delphi 13, Win32 Console DUnit (`DECDUnitTestSuite`), from `Compiled/BIN_IDE_Win32_Console`: -Known reds on full suite at that time: pre-existing **Keccak** vector issues only (separate fix PRs). +| Suite | Result | +|-------|--------| +| TestTDECGCM | **21/21** green (incl. AAD lock + tag-length + multi-chunk + Done lifecycle) | +| TestChaCha20Poly1305 | **12/12** green | +| TestAESNI | **7/7** green (FIPS-197 PAS + PAS↔AES-NI) | +| Full suite known reds | pre-existing **Keccak** only (separate upstream fix PRs) | ---