java.lang.ClassNotFoundException: org.apache.spark.shuffle.SplashShuffleManager

[Roy-Rupak]

Hello,
I have built and set up splash and spark according to the documentation. But when I try to run any spark application I am getting the following error message:
java.lang.ClassNotFoundException: org.apache.spark.shuffle.SplashShuffleManager

This is my spark-default.con file:

spark.files jars/splash-0.7.0.jar,jars/splash-0.7.0-sources.jar,jars/splash-0.7.0-shaded.jar,jars/splash-0.7.0-javadoc.jar
spark.driver.extraClassPath ./splash-0.7.0.jar:./splash-0.7.0-sources.jar:./splash-0.7.0-shaded.jar:./splash-0.7.0-javadoc.jar
spark.executor.extraClassPath jars/splash-0.7.0.jar,jars/splash-0.7.0-sources.jar,jars/splash-0.7.0-shaded.jar,jars/splash-0.7.0-javadoc.jar
#spark.shuffle.manager org.apache.spark.shuffle.SplashShuffleManager

set shuffle manager and storage plugin

spark.shuffle.manager org.apache.spark.shuffle.SplashShuffleManager
spark.shuffle.splash.storageFactory com.memverge.splash.shared.SharedFSFactory
spark.shuffle.splash.clearShuffleOutput false

# set the location of your shared folder

spark.shuffle.splash.folder /home/ruroy/spark/cache

If you could provide any hint to solve this issue that will be very helpful.

[Roy-Rupak]

@jealous Can you please have a look at the issue?

[sheperdh]

@Roy-Rupak It seems you are using Hibench to run spark workload. Can you provide us your spark.conf under Hibench/conf?

[Roy-Rupak]

Hello, @sheperdh thanks for the reply. Here are my spark.conf and hibench.conf files:
spark_conf.txt
hibench_conf.txt

[sheperdh]

I suspect the root cause is below configurations:
spark.driver.extraClassPath ./splash-0.7.0.jar:./splash-0.7.0-sources.jar:./splash-0.7.0-shaded.jar:./splash-0.7.0-javadoc.jar spark.executor.extraClassPath jars/splash-0.7.0.jar,jars/splash-0.7.0-sources.jar,jars/splash-0.7.0-shaded.jar,jars/splash-0.7.0-javadoc.jar
Can you try to use absolute file path?