(Not upstreamed yet) Use async client in FullTableBackupClient - #223
(Not upstreamed yet) Use async client in FullTableBackupClient#223hgromer wants to merge 53 commits into
Conversation
…sume shell implementation (not yet written upstream)
…tdCodec Co-authored-by: Charles Connell <cconnell@hubspot.com>
…rt discrete evaluations (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ta table isolation (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…stem table isolation (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ll pointer (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ions more likely to be approved (will be in 2.7) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…ent to the TableSkewCost (will be in 2.7) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org>
…es. Can squash this, or delete in 2.7 (#167) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ostFunction epsilon when evaluating whether a move improved costs (will be in 2.7) Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…ldn't use "UnknownException" (not yet written upstream) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…eckAndMutate methods (not yet written upstream) Signed-off-by: Duo Zhang <zhangduo@apache.org> Signed-off-by: Nihal Jain <nihaljain@apache.org> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…n 2.7) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…kup failures (will be in 2.6.4) Signed-off-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Ray Mattingly <rmattingly@apache.org> Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
…ationEndpoint when only one replica found (will be in 2.6.4)
…ottlingExceptions … (will be in 2.6.4) * HBASE-29469 Add metrics with more detail for RpcThrottlingExceptions (apache#7214) Co-authored-by: skhillon <skhillon@hubspot.com> Signed-off by: cconnell <cconnell@apache.org> Reviewed by: kgeisz <kevin.j.geiszler@gmail.com> * Removing unnecessary sanitization * Remove unnecessary tests --------- Co-authored-by: skhillon <skhillon@hubspot.com>
…te information (will be in 2.6.4) Signed-off by: Ray Mattingly <rmattingly@apache.org>
…ng individual rows on demand (will be in 2.6.4) Signed-off by: Ray Mattingly <rmattingly@apache.org>
…t2 (not yet merged upstream) (#197)
…er when reloading configuration (will be in 2.6.4) Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org> Signed-off-by: Charles Connell <cconnell@apache.org>
…irdparty is upgraded)
#210) Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
…in 2.6.4) Signed-off by: Ray Mattingly <rmattingly@apache.org>
…n name changed, instead of failing RPC
…c configuration refresh (will be in 2.6.4) (#220) Signed-off-by: Charles Connell <cconnell@apache.org> Signed-off-by: Nick Dimiduk <ndimiduk@apache.org> Co-authored-by: Ray Mattingly <rmattingly@hubspot.com>
Co-authored-by: Hernan Gelaf-Romer <hgelafromer@hubspot.com>
|
This implementation doesn't attempt any unnecessary procedures |
| } | ||
|
|
||
| if (!admin.isSnapshotFinished(desc)) { | ||
| throw new IOException("Snapshot " + snapshotName + " not finished."); |
There was a problem hiding this comment.
Won't this still throw if the snapshot takes longer than waitMs?
There was a problem hiding this comment.
The initial problem I set out to solve was that the loop would re-try to take the snapshot on a later attempt and fail b/c the snapshot already exists. My initial fix is a bit clunky, and I thought this was cleaner
it's okay to throw if we timeout waiting for the snapshot to finish imo. we can up the config value if we want to. what do you think?
rmdmattingly
left a comment
rmdmattingly
left a comment
There was a problem hiding this comment.
Main change looks good (use snapshotAsync + bounded wait to avoid Admin RPC timeouts), but a couple correctness/compat notes before I can approve:
-
Snapshot already exists / timeout race: old code returned early if snapshotName already exists (to handle cases where snapshot is still running but the Admin call timed out). New code will throw on timeout or on “snapshot already exists”. Suggest: on TimeoutException/ExecutionException, check if a snapshot with this name exists (or use isSnapshotFinished when possible) and treat that as success if finished.
-
Exception type: snapshotTable() declares
throws IOExceptionbut wraps all Exceptions in RuntimeException. Prefer rethrow as IOException (or propagate IOException directly) to keep callers consistent. -
SnapshotType.FLUSH: previously used
admin.snapshot(name, table)which may default to a different type in some versions. If FLUSH is intended, maybe add a short comment noting why.
Config: BACKUP_WAIT_MS_KEY + DEFAULT_BACKUP_WAIT_MS matches prior 10*10s behavior; just make sure release notes mention the key rename.
charlesconnell
changed the title
791eac7 to
e414974
Compare
10 participants
Use an async call to take the snapshot so we don't need to do any weird looping. this is simpler