HADOOP-19737: ABFS: Add metrics to identify improvements with read and write aggressiveness #8056

anmolanmol1234 · 2025-11-05T08:09:48Z

Introduces new performance metrics in the ABFS driver to monitor and evaluate the effectiveness of read and write aggressiveness tuning. These metrics help in understanding how thread pool behavior, CPU utilization, and heap availability impact overall I/O throughput and latency. By capturing detailed statistics such as active thread count, pool size, and system resource utilization, this enhancement enables data-driven analysis of optimizations made to improve ABFS read and write performance under varying workloads.

… HADOOP-19472

…234/hadoop into HADOOP-19472_metrics

hadoop-yetus · 2025-11-05T09:23:16Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 4 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	21m 49s		trunk passed
+1 💚	compile	0m 26s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 27s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	0m 18s		trunk passed
+1 💚	mvnsite	0m 27s		trunk passed
+1 💚	javadoc	0m 23s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 21s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌	spotbugs	0m 45s	/branch-spotbugs-hadoop-tools_hadoop-azure-warnings.html	hadoop-tools/hadoop-azure in trunk has 178 extant spotbugs warnings.
+1 💚	shadedclient	14m 42s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	14m 55s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 21s		the patch passed
+1 💚	compile	0m 21s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 21s		the patch passed
+1 💚	compile	0m 21s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 11s	/results-checkstyle-hadoop-tools_hadoop-azure.txt	hadoop-tools/hadoop-azure: The patch generated 26 new + 2 unchanged - 0 fixed = 28 total (was 2)
+1 💚	mvnsite	0m 21s		the patch passed
-1 ❌	javadoc	0m 17s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 21 new + 1544 unchanged - 0 fixed = 1565 total (was 1544)
-1 ❌	javadoc	0m 17s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 9 new + 1437 unchanged - 0 fixed = 1446 total (was 1437)
-1 ❌	spotbugs	0m 47s	/new-spotbugs-hadoop-tools_hadoop-azure.html	hadoop-tools/hadoop-azure generated 2 new + 178 unchanged - 0 fixed = 180 total (was 178)
+1 💚	shadedclient	14m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	2m 15s	/patch-unit-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 22s		The patch does not generate ASF License warnings.
		61m 13s

Reason	Tests
SpotBugs	module:hadoop-tools/hadoop-azure
	org.apache.hadoop.fs.azurebfs.services.AbfsReadThreadPoolMetrics.getUpdatedAtLeastOnce() may expose internal representation by returning AbfsReadThreadPoolMetrics.updatedAtLeastOnce At AbfsReadThreadPoolMetrics.java:by returning AbfsReadThreadPoolMetrics.updatedAtLeastOnce At AbfsReadThreadPoolMetrics.java:[line 132]
	org.apache.hadoop.fs.azurebfs.services.AbfsWriteThreadPoolMetrics.getUpdatedAtLeastOnce() may expose internal representation by returning AbfsWriteThreadPoolMetrics.updatedAtLeastOnce At AbfsWriteThreadPoolMetrics.java:by returning AbfsWriteThreadPoolMetrics.updatedAtLeastOnce At AbfsWriteThreadPoolMetrics.java:[line 136]
Failed junit tests	hadoop.fs.azurebfs.services.TestAbfsOutputStream

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/1/artifact/out/Dockerfile
GITHUB PR	#8056
JIRA Issue	HADOOP-19737
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 4f0cefa11539 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e1d1570`
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/1/testReport/
Max. process+thread count	634 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/1/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-11-05T14:37:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 24s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	22m 16s		trunk passed
+1 💚	compile	0m 24s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 24s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	0m 19s		trunk passed
+1 💚	mvnsite	0m 28s		trunk passed
+1 💚	javadoc	0m 26s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 20s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌	spotbugs	0m 46s	/branch-spotbugs-hadoop-tools_hadoop-azure-warnings.html	hadoop-tools/hadoop-azure in trunk has 178 extant spotbugs warnings.
+1 💚	shadedclient	15m 15s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	15m 37s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 19s		the patch passed
+1 💚	compile	0m 19s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 19s		the patch passed
+1 💚	compile	0m 19s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 19s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 11s		the patch passed
+1 💚	mvnsite	0m 22s		the patch passed
-1 ❌	javadoc	0m 19s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 20 new + 1544 unchanged - 0 fixed = 1564 total (was 1544)
-1 ❌	javadoc	0m 16s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 8 new + 1437 unchanged - 0 fixed = 1445 total (was 1437)
+1 💚	spotbugs	0m 46s		the patch passed
+1 💚	shadedclient	15m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 9s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 18s		The patch does not generate ASF License warnings.
		63m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/2/artifact/out/Dockerfile
GITHUB PR	#8056
JIRA Issue	HADOOP-19737
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 3be4c2201150 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `3198f84`
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/2/testReport/
Max. process+thread count	743 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/2/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-11-06T13:55:15Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	21m 11s		trunk passed
+1 💚	compile	0m 23s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 25s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	0m 19s		trunk passed
+1 💚	mvnsite	0m 27s		trunk passed
+1 💚	javadoc	0m 24s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 22s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌	spotbugs	0m 45s	/branch-spotbugs-hadoop-tools_hadoop-azure-warnings.html	hadoop-tools/hadoop-azure in trunk has 178 extant spotbugs warnings.
+1 💚	shadedclient	14m 6s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	14m 20s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 19s		the patch passed
+1 💚	compile	0m 19s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 19s		the patch passed
+1 💚	compile	0m 21s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 11s	/results-checkstyle-hadoop-tools_hadoop-azure.txt	hadoop-tools/hadoop-azure: The patch generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4)
+1 💚	mvnsite	0m 22s		the patch passed
-1 ❌	javadoc	0m 19s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 22 new + 1544 unchanged - 0 fixed = 1566 total (was 1544)
-1 ❌	javadoc	0m 17s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 10 new + 1437 unchanged - 0 fixed = 1447 total (was 1437)
+1 💚	spotbugs	0m 45s		the patch passed
+1 💚	shadedclient	14m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 9s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 19s		The patch does not generate ASF License warnings.
		59m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/3/artifact/out/Dockerfile
GITHUB PR	#8056
JIRA Issue	HADOOP-19737
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux a44e69c020b4 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aa15783`
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/3/testReport/
Max. process+thread count	635 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/3/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-11-06T14:43:20Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 26s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	21m 51s		trunk passed
+1 💚	compile	0m 26s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 26s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	0m 18s		trunk passed
+1 💚	mvnsite	0m 26s		trunk passed
+1 💚	javadoc	0m 24s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 21s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌	spotbugs	0m 47s	/branch-spotbugs-hadoop-tools_hadoop-azure-warnings.html	hadoop-tools/hadoop-azure in trunk has 178 extant spotbugs warnings.
+1 💚	shadedclient	14m 6s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	14m 19s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 19s		the patch passed
+1 💚	compile	0m 19s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 19s		the patch passed
+1 💚	compile	0m 21s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 11s	/results-checkstyle-hadoop-tools_hadoop-azure.txt	hadoop-tools/hadoop-azure: The patch generated 2 new + 4 unchanged - 0 fixed = 6 total (was 4)
+1 💚	mvnsite	0m 21s		the patch passed
-1 ❌	javadoc	0m 18s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 22 new + 1544 unchanged - 0 fixed = 1566 total (was 1544)
-1 ❌	javadoc	0m 16s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 10 new + 1437 unchanged - 0 fixed = 1447 total (was 1437)
+1 💚	spotbugs	0m 45s		the patch passed
+1 💚	shadedclient	14m 12s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 13s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 21s		The patch does not generate ASF License warnings.
		60m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/4/artifact/out/Dockerfile
GITHUB PR	#8056
JIRA Issue	HADOOP-19737
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 85504ab4b30d 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `bc3dd54`
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/4/testReport/
Max. process+thread count	612 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/4/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-11-11T09:25:09Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	20m 57s		trunk passed
+1 💚	compile	0m 25s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 23s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	0m 17s		trunk passed
+1 💚	mvnsite	0m 26s		trunk passed
+1 💚	javadoc	0m 23s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 22s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌	spotbugs	0m 44s	/branch-spotbugs-hadoop-tools_hadoop-azure-warnings.html	hadoop-tools/hadoop-azure in trunk has 178 extant spotbugs warnings.
+1 💚	shadedclient	14m 8s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	14m 20s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 19s		the patch passed
+1 💚	compile	0m 19s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 19s		the patch passed
+1 💚	compile	0m 22s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 22s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 11s		the patch passed
+1 💚	mvnsite	0m 20s		the patch passed
-1 ❌	javadoc	0m 17s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 22 new + 1544 unchanged - 0 fixed = 1566 total (was 1544)
-1 ❌	javadoc	0m 15s	/results-javadoc-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 10 new + 1437 unchanged - 0 fixed = 1447 total (was 1437)
+1 💚	spotbugs	0m 44s		the patch passed
+1 💚	shadedclient	14m 10s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 14s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 20s		The patch does not generate ASF License warnings.
		59m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/5/artifact/out/Dockerfile
GITHUB PR	#8056
JIRA Issue	HADOOP-19737
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 9be6cffb49bc 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1c444bc`
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/5/testReport/
Max. process+thread count	616 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8056/5/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

manika137 · 2025-11-12T10:40:16Z

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AbfsCountersImpl.java

+   */
+  @Override
+  public AbfsWriteThreadPoolMetrics getAbfsWriteThreadPoolMetrics() {
+    return abfsWriteThreadPoolMetrics != null ? abfsWriteThreadPoolMetrics : null;


we can have it as- return abfsWriteThreadPoolMetrics;
Same for getAbfsReadThreadPoolMetrics() below

anmolanmol1234 and others added 30 commits March 3, 2025 02:20

config changes

1193511

merge trunk

198fea8

fix for executor closing down

6bc388e

Merge branch 'trunk' of https://github.com/anmolanmol1234/hadoop into…

27cfcd8

… HADOOP-19472

changing total threads

8dd82e2

fix no of threads

e88c784

fix max thread pool size

fd699b1

Changes in pool size

a9b6803

changin minimum poll size

ab11da9

added test

108ad28

fix constants

f233943

fix merge conflicts

1ba12f6

spotbugs and checkstyle

8fe0842

fix spot bug

c56bdcb

Merge branch 'trunk' of https://github.com/anmolanmol1234/hadoop into…

ca7be88

… HADOOP-19472

fix variable and logging

5e79b10

Merge branch 'trunk' of https://github.com/anmolanmol1234/hadoop into…

35d293b

… HADOOP-19472

RBMV2 Implementation

37898b4

Test Improvements

28cb97f

Memory Downscale

3ee60df

Code Improv

7670846

Close Executor Service

6a68686

Close Fix

9552072

Close Fix

0c368d8

Merge branch 'trunk' into RBMV2_HADOOP-19622

a894d06

Compilation Error

ad8190b

Closed Fix

b4d8183

Code Improvements

f5883d0

Yetus Checks

d258d90

Merge conflicts with trunk

73a8012

anmolanmol1234 added 15 commits October 29, 2025 01:10

Merge branch 'HADOOP-19472_metrics' of https://github.com/anmolanmol1…

8e811e5

…234/hadoop into HADOOP-19472_metrics

fix time unit

30e66b4

fix time unit

032eca4

fix jvm

d29bf8f

remove unused imports

2f96875

Merge conflicts

9f677bb

metrics for read

1de8f68

Pus metrics

9f4fc51

Merge conflicts

d846599

Read and write metrcis

9bb9458

Push metrics

1340aab

Available memory

6905832

long changes

099cfab

Javadocs

902c981

add tests for metric validation

4ff809f

github-actions bot added trunk TOOLS ABFS labels Nov 5, 2025

Merge conflicts

e1d1570

spotbugs and ceckstyle

3198f84

anmolanmol1234 added 3 commits November 6, 2025 04:52

monitoring changes

9628ec4

fix tests

aa15783

Fix spaces

bc3dd54

Fix checkstyle

1c444bc

manika137 reviewed Nov 12, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-19737: ABFS: Add metrics to identify improvements with read and write aggressiveness #8056

HADOOP-19737: ABFS: Add metrics to identify improvements with read and write aggressiveness #8056

anmolanmol1234 commented Nov 5, 2025

Uh oh!

hadoop-yetus commented Nov 5, 2025

Uh oh!

hadoop-yetus commented Nov 5, 2025

Uh oh!

hadoop-yetus commented Nov 6, 2025

Uh oh!

hadoop-yetus commented Nov 6, 2025

Uh oh!

hadoop-yetus commented Nov 11, 2025

Uh oh!

manika137 Nov 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HADOOP-19737: ABFS: Add metrics to identify improvements with read and write aggressiveness #8056

Are you sure you want to change the base?

HADOOP-19737: ABFS: Add metrics to identify improvements with read and write aggressiveness #8056

Conversation

anmolanmol1234 commented Nov 5, 2025

Uh oh!

hadoop-yetus commented Nov 5, 2025

Uh oh!

hadoop-yetus commented Nov 5, 2025

Uh oh!

hadoop-yetus commented Nov 6, 2025

Uh oh!

hadoop-yetus commented Nov 6, 2025

Uh oh!

hadoop-yetus commented Nov 11, 2025

Uh oh!

manika137 Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants