HBASE-29292 Automatically reassign regions when WALs and ZK go missing #6968
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Duo Zhang <[email protected]>
Signed-off-by: Nihal Jain <[email protected]>
Signed-off-by: Nick Dimiduk <[email protected]> Co-authored-by: Charles Connell <[email protected]>
…ache#6153) Signed-off-by: Duo Zhang <[email protected]> Signed-off-by: Pankaj <[email protected]> Signed-off-by: Istvan Toth <[email protected]>
…pache#6146) * Sync code as per irb 1.4.2 * Also provide option to try irb's new functionalities for colorize and autocomplete Signed-off-by: Duo Zhang <[email protected]> Signed-off-by: Nick Dimiduk <[email protected]> (cherry picked from commit 6788ff4)
… from output tarball (apache#6157) (apache#6159) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 98e7e1b)
…che#6162) (apache#6164) Signed-off-by: Pankaj <[email protected]> (cherry picked from commit 41dd87c)
…ache#6174) Signed-off-by: Duo Zhang <[email protected]>
…eTransition if procedure is initialised by next Active master (apache#6136) Added masterActiveTime as fencing token for remote procedures Signed-off-by: Duo Zhang <[email protected]> Reviewed-by: Aman Poonia <[email protected]>
…nalSslHandler (apache#6181) Signed-off-by: Balazs Meszaros <[email protected]>
…apache#6175) Signed-off-by: Pankaj Kumar<[email protected]> Reviewed-by: Vineet Kumar Maheshwari <[email protected]>
…rable state for the given backup (apache#6088) Co-authored-by: Ray Mattingly <[email protected]> Signed-off-by: Nick Dimiduk <[email protected]>
…lete (apache#6089) Co-authored-by: Ray Mattingly <[email protected]>
Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> Reviewed-by: Vineet Kumar Maheshwari <[email protected]>
…ks in a defined order (apache#6168)" to branch-2 (apache#6200) Signed-off-by: Nick Dimiduk <[email protected]> Co-authored-by: Charles Connell <[email protected]>
… callbac…" (apache#6203) This reverts commit 18f6734.
…fined order (apache#6168) Signed-off-by: Nick Dimiduk <[email protected]> Co-authored-by: Charles Connell <[email protected]>
…ks a connection to ZooKeeper (apache#6207) Signed-off-by: Duo Zhang <[email protected]> Co-authored-by: rodenli <[email protected]>
…procedure.jsp while Master is initializing (apache#6210) Signed-off-by: Duo Zhang <[email protected]>
…apache#6187) (apache#6214) Co-authored-by: WangXin <[email protected]> Co-authored-by: wangxin <[email protected]> Signed-off-by: Nihal Jain <[email protected]> (cherry picked from commit b161ad5)
…#6205) (apache#6211) Signed-off-by: Nick Dimiduk <[email protected]> (cherry picked from commit 35c7fc0)
… cache. (apache#6183) (apache#6198) Signed-off-by: Wellington Chevreuil <[email protected]>
…ile unassigning a region (apache#6197) (apache#6217) Signed-off-by: Wellington Chevreuil <[email protected]> (cherry picked from commit 888e4dd)
Co-authored-by: Ray Mattingly <[email protected]> Signed-off-by: Nick Dimiduk <[email protected]>
Co-authored-by: Ray Mattingly <[email protected]> Signed-off-by: Nick Dimiduk <[email protected]>
…pache#6218) (apache#5607) - Also handle changes required for hadoop-2 assembly Signed-off-by: Istvan Toth <[email protected]> Signed-off-by: Nick Dimiduk <[email protected]> (cherry picked from commit b6394b1)
…pache#6075) Signed-off-by: Duo Zhang <[email protected]>
…addendum: sync protobuf 3 version to hbase-thidparty)
…om persistence (apache#6182) (apache#6219) Signed-off-by: Wellington Chevreuil <[email protected]>
…apache#6226) (apache#6238) Co-authored-by: wangxin <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 3390195) Co-authored-by: WangXin <[email protected]>
Signed-off-by: Nick Dimiduk <[email protected]>
…StoreScanner (apache#6901) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit abc8b43)
…sisted (apache#6916) (apache#6910) Signed-off-by: Andrew Purtell <[email protected]> Signed-off-by: Duo Zhang <[email protected]> Signed-off-by: Reid Chan <[email protected]> Signed-off-by: gvprathyusha6 <[email protected]>
… as input and heap ByteBuffer as output, or vice versa (apache#6806) Signed-off-by: Nick Dimiduk <[email protected]>
Signed-off-by: Ray Mattingly <[email protected]> The HBase backup mechanism keeps track of which HFiles were bulk loaded, so they can be included in incremental backups. Before this ticket, these bulk load records were only deleted when an incremental backup is created. This commit adds 2 more locations: 1) after a full backup. Since a full backup already captures all data, this meant that unnecessary HFiles were being included in the next incremental backup. 2) after a table delete/truncate/CF-deletion. Previously, if an HFile was loaded before a table was cleared, the next incremental backup would effectively still include the HFile. This lead to incorrect data being restored. This commit also completely refactors & simplifies the test for this functionality. Co-authored-by: DieterDP <[email protected]>
) Co-authored-by: Hernan Gelaf-Romer <[email protected]> Signed-off-by: Nick Dimiduk <[email protected]> Signed-off-by: Ray Mattingly <[email protected]>
…t ClassLoader (apache#6918) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit bd30367)
apache#6857) Signed-off-by: Nick Dimiduk <[email protected]> Co-authored-by: Charles Connell <[email protected]>
…le (apache#6842) (apache#6929) Signed-off-by: Nick Dimiduk <[email protected]> Signed-off-by: Nihal Jain <[email protected]> Co-authored-by: Ray Mattingly <[email protected]>
…ributes field (apache#6823) Signed-off-by: Duo Zhang <[email protected]> Signed-off-by: Pankaj <[email protected]>
Signed-off-by: Nick Dimiduk <[email protected]>
…operties (apache#6948) Signed-off-by: Duo Zhang <[email protected]> Signed-off-by: Tak Lon (Stephen) Wu <[email protected]> (cherry picked from commit ae40069)
…Changed behavior (apache#6900) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 99bd5b5)
… support (apache#6953) (apache#6957) Signed-off-by: Nick Dimiduk <[email protected]> Co-authored-by: Ray Mattingly <[email protected]>
apache#6924) Signed-off-by: Nick Dimiduk <[email protected]> (cherry picked from commit 5dafa9e)
Co-authored-by: Josh Elser <[email protected]> Co-authored-by: Sergey Soldatov <[email protected]>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
let me reopen a new one against master #6981 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
sorry for the long delay from the comments of #2114 that peter asked changes for HBASE-24282.
This patch addresses the previous comments by introducing a new feature flag,
hbase.master.assign.regions.on.unknown.servers, to handle cases where an HBase cluster is restarted after WAL and ZooKeeper nodes have been removed. This is particularly useful when recreating an HBase cluster on cloud storage, where HDFS and ZooKeeper are always initialized fresh.