Skip to content

Fix pipe tablet memory self-lock during batching#18266

Open
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/pipe-tablet-resize-memory-deadlock
Open

Fix pipe tablet memory self-lock during batching#18266
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/pipe-tablet-resize-memory-deadlock

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Problem

A realtime Pipe with pattern.exclusion can convert InsertNode events into retained raw tablet events. These events create zero-sized PipeTabletMemoryBlocks and account for their memory later through forceResize.

Before this change, forceResize checked only the remaining non-floating memory. It did not apply the tablet/TsFile admission thresholds used by non-zero initial allocations. Raw tablets could therefore fill the whole non-floating pool. Once full, the sink batch could not allocate even the small buffer needed to transfer and release those tablets, causing a memory self-lock and a frozen RemainingEventCount.

Fix

Apply the existing tablet/TsFile hard admission thresholds when a typed data-structure block grows through resize. Normal blocks, including sink batch buffers, retain the existing total-memory check.

This backpressures raw-tablet production before it exhausts the pool and leaves memory for downstream transfer and release. It also avoids restoring a full static reservation for every sink batch.

A package-private constructor accepting IMemoryBlock was added so the memory policy can be tested with a deterministic, isolated memory pool.

Tests

Added PipeMemoryManagerResizeTest to verify that:

  • tablet growth is rejected after the tablet threshold is reached;
  • a normal sink batch can still allocate memory;
  • tablet growth resumes after retained tablet memory is released;
  • all accounted memory returns to zero.

Executed:

mvn -Ddevelocity.off=true -o test -pl iotdb-core/datanode -Dtest=PipeMemoryManagerResizeTest,PipeTabletEventPlainBatchTest,TsFileInsertionEventParserTest

Result: 25 tests run, 0 failures, 0 errors, 3 skipped.


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
  • added comments explaining the why and the intent of the code wherever it would not be obvious.
  • added unit tests to cover the new code path.

Key changed/added classes in this PR
  • PipeMemoryManager
  • PipeMemoryManagerResizeTest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant