Skip to content

fix: bound DKG pending message queue across NodeId reconnects - #7524

Draft
PastaPastaPasta wants to merge 1 commit into
dashpay:developfrom
PastaPastaPasta:sec/v017
Draft

fix: bound DKG pending message queue across NodeId reconnects#7524
PastaPastaPasta wants to merge 1 commit into
dashpay:developfrom
PastaPastaPasta:sec/v017

Conversation

@PastaPastaPasta

Copy link
Copy Markdown
Member

Issue being fixed or feature implemented

CDKGPendingMessages bounds retention only with messagesPerNode[from] >= maxMessagesPerNode. The key is NodeId, an ephemeral per-connection counter, and nothing ever erases the entry: there is no RemoveNode and no finalize hook, and Clear() runs only from the DKG round handler in active masternode mode.

A peer that disconnects and reconnects therefore gets a fresh NodeId and a fresh full quota, while its previously queued payloads and seenMessages hashes stay resident. pendingMessages and seenMessages grow linearly in the number of connections with no global ceiling.

For llmq_400_85 the per-connection quota is 800 QCONTRIB messages at up to ~70 KB each, roughly 56 MB per connection cycle, repeatable indefinitely.

The sender must have completed MNAuth with the operator key of a masternode in the current DMN list; quorum membership is not required. Importantly, victims are not limited to masternodes: in observer mode the handler still enqueues, never pops and never calls Clear(), so retention on ordinary full nodes is permanent.

What was done?

  • Add a global cap across all NodeIds, not just a per-node quota.
  • Add RemoveNode, wired to a new finalize hook, so a disconnecting peer's queued payloads and seen-hashes are released.
  • Evict from the greediest peer rather than rejecting new messages outright.

Known limitations - this is filed as a draft

Two issues are known and not yet resolved. Filing as a draft so the analysis is on the record and reviewers can weigh in on the direction before the work is finished.

  1. The ceiling is too high and measured in the wrong unit. maxMessagesPerNode * 8 is 6400 QCONTRIB messages at ~70 KB for a single queue of a single handler, roughly 450 MB; summed over LLMQ types and rotation indexes the "bounded" state is still on the order of a gigabyte. The bound should be byte-based rather than message-count-based.

  2. The eviction policy can be abused. EvictGreediestNode() runs before the seenMessages duplicate check, and a duplicate returns without incrementing the per-node count. A malicious masternode can therefore replay already-seen hashes to evict honest peers' queued DKG messages while never occupying a slot itself, and refresh its own quota by reconnecting. That inverts the intent of the policy and is DKG griefing.

Additionally, seenMessages entries for already-popped messages are still released only by Clear(), which runs once per round in active mode only.

How Has This Been Tested?

The first commit adds a regression test showing the pending queue grows across NodeId reconnects, ordered before the fix.

Full build and test validation is delegated to CI on this PR; the changes were not built locally.

Breaking Changes

None.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation
  • I have assigned this pull request to a milestone

CDKGPendingMessages bounded retention only per NodeId, an ephemeral per-connection counter, and never erased the entry. A peer that disconnects and reconnects therefore got a fresh quota while its previously queued payloads and seen-hashes stayed resident, so the queue grew linearly in the number of connections with no global ceiling. For llmq_400_85 that is roughly 56 MB per connection cycle, repeatable indefinitely. Victims are not limited to masternodes: in observer mode the handler still enqueues, never pops and never clears, so retention there is permanent.

Add a global cap across all NodeIds and release a disconnecting peer's payloads and seen-hashes via a finalize hook. Filed as a draft: the cap is still message-count-based rather than byte-based, and EvictGreediestNode runs before the duplicate check, so a malicious masternode can replay seen hashes to evict honest peers' queued messages without occupying a slot itself. Both are unresolved.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant