Advance replay window only after CCM tag verifies

[balloob]

The problem

verifyNonceReplay() checked the replay window and advanced last_seen_counter/replay_window on the cleartext nonce, before the AES-CCM tag was verified. The replay check runs on the plaintext nonce prior to decrypt.

In an active session an attacker can exploit this to DoS the session:

1. Attacker sends a forged frame with a high counter (e.g. current + 20).
2. It passes the replay check (counter not yet seen, within the window).
3. The window advances: last_seen_counter jumps to that high counter.
4. The CCM tag fails, so the frame is rejected — but the window is now desynced.
5. Subsequent legitimate frames with the real (lower) counters now fall below the window / look like replays and are rejected.

A forged frame with a bad tag must never mutate the replay window.

The fix

Split the window advance out of verifyNonceReplay() into a new advanceNonceReplay(), and call it from decryptCommand() only after the AES-CCM tag has verified successfully.

The replay reject check stays where it is (before decrypt) — it's a cheap, correct early-out for genuine replays. The reorder is now:

1. Replay-reject check on the cleartext nonce → reject if already seen / below window (no mutation).
2. AES-CCM decrypt + verify tag → reject on failure without touching the window.
3. On tag success, advanceNonceReplay() records the counter and bumps the window exactly once.

Verification

No ARM SDK toolchain in this environment, so the reordered logic was extracted verbatim into a host-gcc harness (compiles clean under -Wall -Wextra):

legit ctr=10 (tag ok):            accepted=1  last_seen=10
FORGED ctr=30 (bad tag):          accepted=0  last_seen=10  <- window did NOT advance
legit ctr=11 (tag ok):            accepted=1  last_seen=11
replay ctr=10 (tag ok, seen):     accepted=0  <- rejected
RESULT: PASS - forged frame did not desync; legit lower-counter frame still accepted

Companion change

The Silabs firmware has the same issue — its replay window was just added in OpenDisplay/Firmware_Silabs#5 — and is being fixed there in lockstep.

🤖 Generated with Claude Code

https://claude.ai/code/session_012g2e8mr132vcizx92WsgiR

---

Companion PR: OpenDisplay/Firmware_Silabs#5 (same advance-after-auth reorder, Silabs — folded into its replay-protection PR). Both: reject-before-decrypt, advance-window-only-after-CCM-tag.