wc_rsa_verify: PUBKEY_ENC_UNKNOWN falls through to BAD_SIGNATURE — breaks all GnuPG/apt RSA signature verification

[MarkAtwood]

Summary

wc_rsa_verify() in cipher/rsa.c rejects every RSA signature whose libgcrypt
encoding context is PUBKEY_ENC_UNKNOWN. The switch (ctx.encoding) only
enumerates the named encodings (PSS / PKCS1 / PKCS1_RAW / RAW / OAEP) and routes
everything else — including PUBKEY_ENC_UNKNOWN (enum value 5) — to
default: rc = GPG_ERR_BAD_SIGNATURE.

GnuPG's OpenPGP signature verification supplies the full pre-encoded PKCS#1 v1.5
message and leaves ctx.encoding == PUBKEY_ENC_UNKNOWN. As a result all GnuPG
RSA signature verification fails ("1 bad signature", keys reported "Unusable"),
and because apt verifies repository signatures via gpg/libgcrypt, apt repo
signature verification breaks too.

Stock libgcrypt handles this: its rsa_verify() performs the raw
mpi_cmp(result, data) for all non-PSS encodings, not just an allowlist.

Location

• Branch: libgcrypt-1.11.0-wolfCrypt-fips-ready
• cipher/rsa.c, wc_rsa_verify(), the switch (ctx.encoding) (~L3592–L3640)

Reproduce

With this shim's libgcrypt installed (routing through wolfCrypt):

export GNUPGHOME=$(mktemp -d)
gpg --batch --pinentry-mode=loopback --passphrase "" \
    --quick-generate-key "test@example.com" rsa2048 sign 0
gpg --check-sigs      # -> "gpg: 1 bad signature"

Instrumentation shows wc_rsa_verify receiving ctx.encoding == 5
(PUBKEY_ENC_UNKNOWN) and taking the default branch.

The crypto itself is correct — this is purely the switch

An OpenSSL-produced PKCS#1 v1.5 RSA-SHA256 signature verifies correctly through
native wolfCrypt with the key loaded via wc_RsaPublicKeyDecodeRaw +
wc_RsaDirect(RSA_PUBLIC_DECRYPT) / wc_RsaSSL_Verify — the recovered encoded
message is byte-for-byte correct. So base wolfCrypt is fine; only the shim's
encoding dispatch is wrong.

Fix

Route PUBKEY_ENC_UNKNOWN into the existing raw public-op + mpi_cmp group (or,
more robustly, have default: do the raw compare, matching stock libgcrypt).
With the one-line change, gpg --check-sigs goes from "1 bad signature" to clean.
PR to follow.