SONARJAVA-6423 Implement S8715 No JUnit 4 assertions in JUnit 5 tests

[tomasz-tylenda-sonarsource]

---

Summary by Gitar

• New rule implementation:
  • Added JUnit4AssertionsCheck to flag usage of JUnit 4 assertions (org.junit.Assert) in JUnit Jupiter tests.
• Rule metadata:
  • Added documentation (S8715.html), configuration (S8715.json), and integrated the rule into Sonar agentic AI and Sonar way profiles.
• Testing:
  • Created JUnit4AssertionsCheckTest and sample code to verify detection, and updated JavaAgenticWayProfileTest to account for the new rule.
• Infrastructure:
  • Added diff_S8715.json to the autoscan test resources.

_{This will update automatically on new commits.}

[sonarqubecloud]

Agentic Analysis: Early Results

Agentic Analysis and Context Augmentation are available on your project. Here are some issues that could have been prevented. Follow the links to learn how to put them into action.

2 issue(s) found across 1 file(s):

Rule	File	Line	Message
java:S125	java-checks-test-sources/default/src/test/java/checks/tests/Junit4AssertionsCheckSampleTest.java	13	This block of commented-out lines of code should be removed.
java:S139	java-checks-test-sources/default/src/test/java/checks/tests/Junit4AssertionsCheckSampleTest.java	13	Move this trailing comment on the previous empty line.

_{Analyzed by SonarQube Agentic Analysis in 11.6 s}

[hashicorp-vault-sonar-prod]

SONARJAVA-6423

[gitar-bot]

⚠️ This PR is blocked due to unresolved code review findings.

Comment gitar unblock to override this block and allow merging.

_{Configure merge blocking · Maintainers can dismiss this review. Gitar never approves changes.}

[gitar-bot]

🚨 Bug: Test file name case mismatch will cause test failure on Linux

The test references checks/tests/JUnit4AssertionsCheckSampleTest.java (capital 'U' in JUnit) but the actual sample file is named Junit4AssertionsCheckSampleTest.java (lowercase 'u'). On case-sensitive file systems (Linux CI), this will cause the test to fail with a FileNotFoundException.

Fix 1: Fix the path in the test to match the actual file name (lowercase 'u' in Junit)

.onFile(testCodeSourcesPath("checks/tests/Junit4AssertionsCheckSampleTest.java"))

• Apply fix

Fix 2: Alternatively, rename the sample file to match the test's expected path (capital 'U' in JUnit), which is more consistent with the check class name JUnit4AssertionsCheck

Rename the sample file from Junit4AssertionsCheckSampleTest.java to JUnit4AssertionsCheckSampleTest.java

• Apply fix

_{Check a box to apply a fix or reply for a change | Was this helpful? React with 👍 / 👎}

[gitar-bot]

⚠️ Bug: Check only detects @test but misses other Jupiter annotations

The check only considers methods annotated with org.junit.jupiter.api.Test, but JUnit 5 test methods can also be annotated with @ParameterizedTest, @RepeatedTest, @TestFactory, or @TestTemplate. The codebase already provides UnitTestUtils.hasJUnit5TestAnnotation(MethodTree) which covers all five annotations. Using JUnit 4 assertions in a @ParameterizedTest method would not be flagged by this check.

Other checks in the codebase (e.g., AbstractJUnit5NotCompliantModifierChecker) use UnitTestUtils.hasJUnit5TestAnnotation() for this purpose.

Use the existing UnitTestUtils.hasJUnit5TestAnnotation() helper to cover all Jupiter test annotations (@test, @ParameterizedTest, @RepeatedTest, @testfactory, @testtemplate):

private static boolean isInJupiterTest(MethodInvocationTree mit) {
  MethodTree enclosingMethod = ExpressionUtils.getEnclosingMethod(mit);
  return enclosingMethod != null && UnitTestUtils.hasJUnit5TestAnnotation(enclosingMethod);
}

• Apply fix

_{Check the box to apply the fix or reply for a change | Was this helpful? React with 👍 / 👎}

[gitar-bot]

Code Review 🚫 Blocked 3 resolved / 5 findings

Implements S8715 to identify JUnit 4 assertions within JUnit 5 tests, but the check currently fails to account for all Jupiter annotations and contains a file naming case mismatch that will break Linux builds.

🚨 Bug: Test file name case mismatch will cause test failure on Linux

📄 java-checks/src/test/java/org/sonar/java/checks/tests/JUnit4AssertionsCheckTest.java:28 📄 java-checks-test-sources/default/src/test/java/checks/tests/Junit4AssertionsCheckSampleTest.java:1

The test references checks/tests/JUnit4AssertionsCheckSampleTest.java (capital 'U' in JUnit) but the actual sample file is named Junit4AssertionsCheckSampleTest.java (lowercase 'u'). On case-sensitive file systems (Linux CI), this will cause the test to fail with a FileNotFoundException.

Fix the path in the test to match the actual file name (lowercase 'u' in Junit)

.onFile(testCodeSourcesPath("checks/tests/Junit4AssertionsCheckSampleTest.java"))

Alternatively, rename the sample file to match the test's expected path (capital 'U' in JUnit), which is more consistent with the check class name JUnit4AssertionsCheck

Rename the sample file from Junit4AssertionsCheckSampleTest.java to JUnit4AssertionsCheckSampleTest.java

⚠️ Bug: Check only detects @Test but misses other Jupiter annotations

📄 java-checks/src/main/java/org/sonar/java/checks/tests/JUnit4AssertionsCheck.java:35 📄 java-checks/src/main/java/org/sonar/java/checks/tests/JUnit4AssertionsCheck.java:55-58

The check only considers methods annotated with org.junit.jupiter.api.Test, but JUnit 5 test methods can also be annotated with @ParameterizedTest, @RepeatedTest, @TestFactory, or @TestTemplate. The codebase already provides UnitTestUtils.hasJUnit5TestAnnotation(MethodTree) which covers all five annotations. Using JUnit 4 assertions in a @ParameterizedTest method would not be flagged by this check.

Other checks in the codebase (e.g., AbstractJUnit5NotCompliantModifierChecker) use UnitTestUtils.hasJUnit5TestAnnotation() for this purpose.

Use the existing UnitTestUtils.hasJUnit5TestAnnotation() helper to cover all Jupiter test annotations (@test, @ParameterizedTest, @RepeatedTest, @testfactory, @testtemplate)

private static boolean isInJupiterTest(MethodInvocationTree mit) {
  MethodTree enclosingMethod = ExpressionUtils.getEnclosingMethod(mit);
  return enclosingMethod != null && UnitTestUtils.hasJUnit5TestAnnotation(enclosingMethod);
}

✅ 3 resolved

✅ Bug: Check flags JUnit 4 assertions unconditionally, even in JUnit 4 tests

📄 java-checks/src/main/java/org/sonar/java/checks/tests/JUnit4AssertionsCheck.java:25-34
The rule is titled "No JUnit 4 assertions in JUnit 5 tests" but the implementation reports ANY call to org.junit.Assert.* regardless of whether the enclosing test method/class uses JUnit 5 annotations. This means the rule will produce false positives on legitimate JUnit 4 test classes that correctly use org.junit.Assert.

The codebase already provides UnitTestUtils.hasJUnit5TestAnnotation(MethodTree) which checks for @Test, @RepeatedTest, @TestFactory, @TestTemplate, and @ParameterizedTest from JUnit Jupiter. The check should walk up from the MethodInvocationTree to the enclosing method and verify it has a JUnit 5 annotation before reporting.

The test sample also lacks a negative case (a JUnit 4 test class using org.junit.Test with org.junit.Assert that should NOT be flagged).

✅ Bug: Check flags JUnit 4 assertions unconditionally, ignoring JUnit 5 context

📄 java-checks/src/main/java/org/sonar/java/checks/tests/JUnit4AssertionsCheck.java:46-49
The rule description and S8715.html clearly state it targets "tests written with the JUnit Jupiter API (JUnit 5 and later)", but the implementation in visitNode simply matches any call to org.junit.Assert methods without verifying the enclosing test method is annotated with a JUnit 5 annotation (e.g., @org.junit.jupiter.api.Test or @ParameterizedTest). This means the rule will produce false positives on pure JUnit 4 tests that legitimately use org.junit.Assert.

The codebase already provides UnitTestUtils.hasJUnit5TestAnnotation(MethodTree) for exactly this purpose (used in AbstractJUnit5NotCompliantModifierChecker and others). The check should traverse to the enclosing method, verify it carries a JUnit 5 test annotation, and only then report.

✅ Bug: Test sample file JUnit4AssertionsCheckSampleTest.java is missing

📄 java-checks/src/test/java/org/sonar/java/checks/tests/JUnit4AssertionsCheckTest.java:28
The test references testCodeSourcesPath("checks/tests/JUnit4AssertionsCheckSampleTest.java") but this file does not exist in the repository. The test will fail at runtime with a file-not-found error. A sample test file containing both compliant and noncompliant examples (with // Noncompliant markers) needs to be added to the test sources.

🤖 Prompt for agents

Code Review: Implements S8715 to identify JUnit 4 assertions within JUnit 5 tests, but the check currently fails to account for all Jupiter annotations and contains a file naming case mismatch that will break Linux builds.

1. 🚨 Bug: Test file name case mismatch will cause test failure on Linux
   Files: java-checks/src/test/java/org/sonar/java/checks/tests/JUnit4AssertionsCheckTest.java:28, java-checks-test-sources/default/src/test/java/checks/tests/Junit4AssertionsCheckSampleTest.java:1

   The test references `checks/tests/JUnit4AssertionsCheckSampleTest.java` (capital 'U' in JUnit) but the actual sample file is named `Junit4AssertionsCheckSampleTest.java` (lowercase 'u'). On case-sensitive file systems (Linux CI), this will cause the test to fail with a FileNotFoundException.

   Fix (Fix the path in the test to match the actual file name (lowercase 'u' in Junit)):
   .onFile(testCodeSourcesPath("checks/tests/Junit4AssertionsCheckSampleTest.java"))

   Fix (Alternatively, rename the sample file to match the test's expected path (capital 'U' in JUnit), which is more consistent with the check class name JUnit4AssertionsCheck):
   Rename the sample file from Junit4AssertionsCheckSampleTest.java to JUnit4AssertionsCheckSampleTest.java

2. ⚠️ Bug: Check only detects @Test but misses other Jupiter annotations
   Files: java-checks/src/main/java/org/sonar/java/checks/tests/JUnit4AssertionsCheck.java:35, java-checks/src/main/java/org/sonar/java/checks/tests/JUnit4AssertionsCheck.java:55-58

   The check only considers methods annotated with `org.junit.jupiter.api.Test`, but JUnit 5 test methods can also be annotated with `@ParameterizedTest`, `@RepeatedTest`, `@TestFactory`, or `@TestTemplate`. The codebase already provides `UnitTestUtils.hasJUnit5TestAnnotation(MethodTree)` which covers all five annotations. Using JUnit 4 assertions in a `@ParameterizedTest` method would not be flagged by this check.
   
   Other checks in the codebase (e.g., `AbstractJUnit5NotCompliantModifierChecker`) use `UnitTestUtils.hasJUnit5TestAnnotation()` for this purpose.

   Fix (Use the existing UnitTestUtils.hasJUnit5TestAnnotation() helper to cover all Jupiter test annotations (@Test, @ParameterizedTest, @RepeatedTest, @TestFactory, @TestTemplate)):
   private static boolean isInJupiterTest(MethodInvocationTree mit) {
     MethodTree enclosingMethod = ExpressionUtils.getEnclosingMethod(mit);
     return enclosingMethod != null && UnitTestUtils.hasJUnit5TestAnnotation(enclosingMethod);
   }

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.
Unblock → Override a blocking verdict and allow merging.

Comment with these commands to change:

Auto-apply	Compact	Unblock
gitar auto-apply:on	gitar display:verbose	gitar unblock

_{Was this helpful? React with 👍 / 👎 | Gitar}