Add metrics for HDFS checksum and no-checksum bytes read - #255
Draft
SaadASTheDev wants to merge 78 commits into
Draft
Add metrics for HDFS checksum and no-checksum bytes read#255SaadASTheDev wants to merge 78 commits into
SaadASTheDev wants to merge 78 commits into
Conversation
…sume shell implementation (not yet written upstream)
…tdCodec Co-authored-by: Charles Connell <cconnell@hubspot.com>
…rt discrete evaluations (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ta table isolation (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…stem table isolation (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ll pointer (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ions more likely to be approved (will be in 2.7) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…ent to the TableSkewCost (will be in 2.7) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…es. Can squash this, or delete in 2.7 (#167) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ostFunction epsilon when evaluating whether a move improved costs (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ldn't use "UnknownException" (not yet written upstream) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…eckAndMutate methods (not yet written upstream) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…n 2.7) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…kup failures (will be in 2.6.4) Signed-off-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…ationEndpoint when only one replica found (will be in 2.6.4)
…ottlingExceptions … (will be in 2.6.4) * HBASE-29469 Add metrics with more detail for RpcThrottlingExceptions (apache#7214) Co-authored-by: skhillon <skhillon@hubspot.com> Signed-off by: cconnell <cconnell@apache.org> Reviewed by: kgeisz <kevin.j.geiszler@gmail.com> * Removing unnecessary sanitization * Remove unnecessary tests --------- Co-authored-by: skhillon <skhillon@hubspot.com>
…te information (will be in 2.6.4) Signed-off by: Ray Mattingly <rmattingly@apache.org>
…ng individual rows on demand (will be in 2.6.4) Signed-off by: Ray Mattingly <rmattingly@apache.org>
…t2 (not yet merged upstream) (#197)
…er when reloading configuration (will be in 2.6.4) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Charles Connell <cconnell@apache.org>
…irdparty is upgraded)
…etween backups (#224) Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…#228) * Fix an issue with duplicated backups of inactive host WAL files * Skip meta region WAL files
… regionServer JMX Metric (apache#7481) (#229) * HBASE-29729 Add table descriptor hash
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…ta loss (#231) Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
… regions without moving (apache#7563) (apache#7611) (#232) (will be in 2.7) Signed-off-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Alex Hughes <ajkh88@users.noreply.github.com> Co-authored-by: Alex Hughes <ahughes@hubspot.com>
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…verridden by replication peers (apache#7578) Co-authored-by: skhillon <skhillon@hubspot.com> Signed-off by: <taklwu@apache.org>
This commit condenses several redundant APIs used to access the backup history. No functional changes. Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: DieterDP <90392398+DieterDP-ng@users.noreply.github.com>
Restores the ordering of BackupAdmin#getHistory that was accidentally reversed in HBASE-29808. Extends & refactors TestBackupShowHistory to verify correct behavior. Fixes a possible FileNotFoundException in BackupUtils#getHistory. Merges BackupSystemTable#getBackupHistory with BackupSystemTable#getBackupInfos, to further simplify backup info retrieval. Optimized various usages of history retrieval. Clarified some javadoc regarding backup history retrieval. (cherry picked from commit ed04e2e) Co-authored-by: Dieter De Paepe <dieterdp@ngdata.com>
…n backup roots have different host coverage (#237) Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…ression (#236) * Initial changes to allow partial cell read in WAL * Add tests * More tests * Add back comments * Remove redundant tests and simplify * Get tag value even if tag is out of bounds due to addition * Clean up a bit * Add explanatory comment * Initialize to empty * Resolve PR comment issue regarding LRU eviction for tags * Address PR review: two-phase rollback and remove unused dictType param Rollback in UndoableLRUDictionary previously restored nodes one at a time, doing remove/setContents/put on the content-based nodeToIndex HashMap. This could clobber entries when two nodes shared the same content during the restore (e.g., an evicted value re-added to a different slot). The fix restores all node state first, then rebuilds nodeToIndex from scratch. Also removes the unused dictType parameter from TagCompressionContext since every caller hardcodes LRUDictionary.class and we always need UndoableLRUDictionary for correctness. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: skhillon <skhillon@hubspot.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
… file switch detected in ReplicationSourceWALReader run loop (apache#7909) (#242) When ReplicationSourceWALReader.run() detects a WAL file switch via the switched() check, it enqueues an EOF batch but does not update currentPosition. If the outer loop restarts (e.g., due to WALEntryFilterRetryableException), the new WALEntryStream is created with the stale position from the old file, applied to the new file. This causes an infinite retry loop (EOFException: Cannot seek after EOF) and the corrupted position may be persisted to ZK, surviving restarts. The fix resets currentPosition to entryStream.getPosition() (which returns 0 after dequeueCurrentLog()) before enqueuing the EOF batch. Includes a regression test that reproduces the bug by using nb.capacity=1 to force EOF detection at line 153 (not inside readWALEntries), combined with a WALEntryFilterRetryableException on the first entry of the new file to trigger the outer loop restart. (cherry picked from commit e4f9c65) Signed-off-by: Duo Zhang <zhangduo@apache.org> Co-authored-by: skhillon <skhillon@hubspot.com>
…kups (#244) Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…processors in region replication (#247) * Filter cell. extended attributes and split oversized batches in region replica replication Region replica replication was forwarding all extended attributes including large "cell."-prefixed ones added by CDC coprocessors that have no use for replicas. Strip these in replicate() before buffering to reduce memory and network overhead. Also add RPC size limit enforcement (matching HBaseInterClusterReplicationEndpoint) to split batches that exceed hbase.ipc.max.request.size, preventing permanently stalled replication. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix test compilation: add replicationRpcLimit parameter to test call sites Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add unit tests for splitBatches and extract as package-private static method Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Remove controller.reset() between batch RPCs reset() clears priority, callTimeout, tableName, and regionInfo which were set by the superclass before call() was entered. Since setCellScanner is called at the top of each loop iteration with the new batch's scanner, no cleanup is needed between iterations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Revert batch-splitting changes, keep only cell. attribute filtering Scoping down to just the extended attribute filtering. The batch-splitting for oversized RPCs will be addressed separately. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: skhillon <skhillon@hubspot.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
charlesconnell
force-pushed
the
hubspot-2.6
branch
5 times, most recently
from
August 9, 2026 22:57
791eac7 to
e414974
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
12 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.