Skip to content

[Performance] Use dynamic bitmaps in AlignedTVList#18264

Open
jt2594838 wants to merge 1 commit into
masterfrom
use_long_backed_bitmap_for_memtable
Open

[Performance] Use dynamic bitmaps in AlignedTVList#18264
jt2594838 wants to merge 1 commit into
masterfrom
use_long_backed_bitmap_for_memtable

Conversation

@jt2594838

Copy link
Copy Markdown
Contributor

Description

Use dynamically selected bitmap implementations

Use BitMap.createBitMapDynamically for bitmap allocations created by AlignedTVList and LazyBitMap. With the default primitive_array_size=64, this selects the long-backed implementation and avoids a separate byte-array allocation.

Update bitmap memory accounting to use ramBytesUsed(), and grow the time-deletion bitmap with extend.

Preserve bitmap behavior

The long-backed implementation returns a copy from getByteArray(). Result-status bitmap updates therefore use merge instead of mutating the returned array. The public byte-array helper keeps its previous exact-length result, including byte-aligned sizes.

The all-value-deleted calculation reads each bitmap byte array once to avoid repeated temporary-array allocation.

Tests and performance

Added unit coverage for:

  • reduced bitmap retained size;
  • failed-status merging with non-byte-aligned offsets across a 64-row block boundary;
  • exact byte-array length for byte-aligned result bitmaps.

Added an opt-in local standalone performance IT. Workload: 256 aligned INT64 measurements, 64 rows per tablet, one null per measurement per tablet, 50 warmup batches, and 5 rounds of 100 batches.

Three independent JVM runs were compared using the median reported by each run:

Implementation Cross-run median Throughput
new BitMap(64) 123.145 ms/round 13.30M points/s
createBitMapDynamically(64) 110.376 ms/round 14.84M points/s

The dynamic bitmap reduced round time by about 10.4% and increased throughput by about 11.6%.


This PR has:

  • been self-reviewed.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes
  • AlignedTVList
  • LazyBitMap
  • AlignedTVListTest
  • IoTDBAlignedTVListBitMapPerformanceIT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant