Skip to content

Fix flaky SharedTsBlockQueue concurrency test#18244

Open
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix-shared-tsblock-queue-test-timeout
Open

Fix flaky SharedTsBlockQueue concurrency test#18244
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix-shared-tsblock-queue-test-timeout

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

SharedTsBlockQueueTest.concurrencyTest used a fixed JUnit timeout while polling counters, printing progress, and recursively scheduling listener tasks. Under CI load, the test could time out without surfacing worker failures, and its executor was never shut down.

This change makes the test deterministic by:

  • waiting for producer and consumer tasks through Future.get
  • propagating worker failures to the test thread
  • using the real memory manager and pool instead of unnecessary spies
  • asserting the final queue and memory state
  • always shutting down the executor

Testing

  • mvn -o -nsu -Ddevelocity.off=true spotless:check -pl iotdb-core/datanode
  • compiled and ran SharedTsBlockQueueTest directly: 2 tests passed
  • repeated the complete test class 20 times: 20/20 passed

A full reactor test attempt was blocked before reaching DataNode by an existing generated Thrift compilation error: TaskType.java could not resolve javax.annotation.

@JackieTien97 JackieTien97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One non-blocking suggestion to improve failure diagnostics.

return null;
});

sender.get(30, TimeUnit.SECONDS);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we wait for whichever worker completes first here? If the receiver fails before the sender finishes, it stops freeing queue memory; the sender can then block once the pool fills, so this Future.get call reports a 30-second TimeoutException and receiver.get is never reached. That hides the receiver's actual exception, even though this change aims to propagate worker failures. Using an ExecutorCompletionService with a shared deadline (and cancelling the sibling task on failure) would make this fail fast and preserve the root cause.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants