Skip to content

HDFS-17850. Reject append() when newGS equals current generation stamp. #8320

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
balodesecurity wants to merge 1 commit into
base: trunk
Choose a base branch
from
+49 −1
Open

HDFS-17850. Reject append() when newGS equals current generation stamp. #8320

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...fs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1410,7 +1410,7 @@ public ReplicaHandler append(ExtendedBlock b,
// The other reason is that an "append" is occurring to this block.

// check the validity of the parameter
if (newGS < b.getGenerationStamp()) {
if (newGS <= b.getGenerationStamp()) {
throw new IOException("The new generation stamp " + newGS +
" should be greater than the replica " + b + "'s generation stamp");
}
Expand Down
48 changes: 48 additions & 0 deletions ...rc/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2121,6 +2121,54 @@ public void delayGetMetaDataInputStream() {
}
}

/**
* HDFS-17850: Verify that append() rejects a newGS equal to the replica's
* current generation stamp. A client bug that reuses the same GS should
* fail immediately rather than silently corrupting the replica state.
*/
@Test
@Timeout(value = 60)
public void testAppendRejectsSameGenerationStamp() throws Exception {
MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf)
.numDataNodes(1).build();
try {
cluster.waitActive();
BlockReaderTestUtil util = new BlockReaderTestUtil(cluster,
new HdfsConfiguration(conf));
Path path = new Path("/testAppendSameGS");
util.writeFile(path, 1);
String bpid = cluster.getNameNode().getNamesystem().getBlockPoolId();
DataNode dn = cluster.getDataNodes().get(0);
FsDatasetImpl dnFSDataset = (FsDatasetImpl) dn.getFSDataset();
LocatedBlock blk = util.getFileBlocks(path, 512).get(0);
ExtendedBlock block = blk.getBlock();
long currentGS = block.getGenerationStamp();

// newGS == currentGS must be rejected (HDFS-17850 fix)
try {
dnFSDataset.append(block, currentGS, block.getNumBytes());
fail("Expected IOException when newGS equals current generation stamp");
} catch (IOException e) {
assertTrue(e.getMessage().contains("should be greater than"),
"Unexpected error message: " + e.getMessage());
}

// newGS < currentGS must also be rejected (pre-existing behaviour)
try {
dnFSDataset.append(block, currentGS - 1, block.getNumBytes());
fail("Expected IOException when newGS is less than current generation stamp");
} catch (IOException e) {
assertTrue(e.getMessage().contains("should be greater than"),
"Unexpected error message: " + e.getMessage());
}

// newGS > currentGS must succeed
dnFSDataset.append(block, currentGS + 1, block.getNumBytes());
} finally {
cluster.shutdown();
}
}

@Test
@Timeout(value = 30)
public void testAppend() {
Expand Down