Documentation regarding running/regenerating stability test plans

[EmilyMatt]

What is the problem the feature request solves?

I think the documentation is not clear enough on this.
Using a clean main branch, I tried running
./mvnw -pl spark -Dsuites="org.apache.spark.sql.comet.CometTPCDSV1_4_PlanStabilitySuite" -Pspark-3.5 -nsu test

It does not compile, I get an error in regards to

[ERROR] /home/emily/projects/datafusion-comet/spark/src/main/scala/org/apache/comet/CometExecIterator.scala:27: value COMET_METRICS_UPDATE_INTERVAL is not a member of object org.apache.comet.CometConf
[ERROR] /home/emily/projects/datafusion-comet/spark/src/main/scala/org/apache/comet/CometExecIterator.scala:75: not found: value COMET_METRICS_UPDATE_INTERVAL

I've tried compiling everything first, I've tried running all the tests first and only then running this, to no avail.
I've tried removing the -nsu, thinking snapshots might be interfering, as well as removing the -pl spark to see if maybe the general package is needed, the latter led to more general issues such as ByteBuffer not being found.

I'm compiling for Spark 3.5, using Ubuntu 24.04, on an x86 machine, with OpenJDK 8 installed, along with Scala 2.12
I've tried setting all the Profiles as well, this did not change anything.
I'm not sure which common snapshot does it use, as I've tried running make clean, mvn clean, as well as manually deleting all the target directories, tried running the tests directly afterwards, as well as after recompiling after the clean.
I think the documentation needs to be updated, as they present simple commands, which doesn't seem to be the case.

It's worth mentioning that compilation as well as all the tests work as expected, so this is not a general issue.
Leaving this as a feature request for updated documentation as I'm not sure whether there's a bug or I'm doing something wrong

Describe the potential solution

No response

Additional context

No response

[wForget]

You can try adding -am like ./mvnw -am -pl spark ..., or run your command after mvn install.

[andygrove]

I agree that the documentation is in need of improvement. I also found that the files get generated to $SPARK_HOME, so I usually set this env var to point to the root of the source repo.

[andygrove]

It is also necessary to use different Java versions for different Spark versions. I use JDK 11 for Spark 3.x.x and JDK 17 for Spark 4.x.x

[EmilyMatt]

@andygrove Is this mandatory? I have been running JDK 8 with Spark 3.5.1 for a while

[EmilyMatt]

also, I've been able to run the tests by running without "-pl spark" now, but then my code is not ran but a default/previous version of Comet somehow? I'm truly stumped by this, I think there is something I seriously don't understand about what Maven is doing

[andygrove]

also, I've been able to run the tests by running without "-pl spark" now, but then my code is not ran but a default/previous version of Comet somehow? I'm truly stumped by this, I think there is something I seriously don't understand about what Maven is doing

@EmilyMatt This may be because the diff files in the main branch currently have the wrong Comet version. This is fixed in #1404

[andygrove]

@EmilyMatt This may be because the diff files in the main branch currently have the wrong Comet version. This is fixed in #1404

Sorry, ignore this. I was conflating two different issues.