KAFKA-18006: Add 3.9.0 to end-to-end test (streams)

[frankvicky]

JIRA: KAFKA-18006

Since 3.9 is released, we should add it to the e2e-test.
Follow-up of #17797

Committer Checklist (excluded from commit message)

• Verify design and implementation
• Verify test coverage and CI build status
• Verify documentation (including upgrade notes)

[frankvicky]

Test result of streams_upgrade_test.py:

17:49:07 SESSION REPORT (ALL TESTS)
17:49:07 ducktape version: 0.12.0
17:49:07 session_id:       2024-11-13--001
17:49:07 run time:         52 minutes 53.820 seconds
17:49:07 tests run:        20
17:49:07 passed:           20
17:49:07 flaky:            0
17:49:07 failed:           0
17:49:07 ignored:          0

Test result of streams_broker_compatibility_test.py:

10:34:01 SESSION REPORT (ALL TESTS)
10:34:01 ducktape version: 0.12.0
10:34:01 session_id:       2024-11-13--001
10:34:01 run time:         9 minutes 7.208 seconds
10:34:01 tests run:        20
10:34:01 passed:           20
10:34:01 flaky:            0
10:34:01 failed:           0
10:34:01 ignored:          0

Test result of streams_application_upgrade_test.py:

11:03:23 SESSION REPORT (ALL TESTS)
11:03:23 ducktape version: 0.12.0
11:03:23 session_id:       2024-11-13--001
11:03:23 run time:         23 minutes 25.706 seconds
11:03:23 tests run:        17
11:03:23 passed:           17
11:03:23 flaky:            0
11:03:23 failed:           0
11:03:23 ignored:          0

[cadonna]

Thanks for the PR, @frankvicky !

Here my comments.

[cadonna]

The following line is missing in task testAll property dependsOn:

':streams:upgrade-system-tests-38:test',

[frankvicky]

Thanks for pointing out.
Done 😺

[chia7712]

@frankvicky please rebase code, thanks!