Neo4j :errors during create-validation-parameters.sh #409
Description
I used ldbc_snb_datagen_hadoop main branch to generate the data. I did cp params-csv-merge-foreign.ini params.ini, and changed it as follows:
ldbc.snb.datagen.generator.scaleFactor:snb.interactive.100
ldbc.snb.datagen.serializer.dateFormatter:ldbc.snb.datagen.util.formatter.LongDateFormatter
ldbc.snb.datagen.serializer.dynamicActivitySerializer:ldbc.snb.datagen.serializer.snb.csv.dynamicserializer.activity.CsvCompositeDynamicActivitySerializer
ldbc.snb.datagen.serializer.dynamicPersonSerializer:ldbc.snb.datagen.serializer.snb.csv.dynamicserializer.person.CsvCompositeDynamicPersonSerializer
ldbc.snb.datagen.serializer.staticSerializer:ldbc.snb.datagen.serializer.snb.csv.staticserializer.CsvCompositeStaticSerializer
Then I'm in this repository. I am using tag 1.0.0 and ran scripts/load-in-one-step.sh successfully. I made the corresponding changes (ldbc.snb.interactive.updates_dir, ldbc.snb.interactive.parameters_dir, ldbc.snb.interactive.scale_factor)in cypher/driver/create-validation-parameters.properties, cypher/driver/validate.properties, and cypher/driver/benchmark.properties. And I replaced the files of ldbc_snb_interactive_v1_impls/cypher/test-data/vanilla/static and ldbc_snb_interactive_v1_impls/cypher/test-data/vanilla/dynamic with the files of social_network/static and social_network/dynamic generated by datagen。
When I run driver/create-validation-parameters.sh, I got the following error.So is there some kind of incompatibility with how I'm using the Hadoop-based data generation?
./create-validation-parameters.sh
CreateValidationParamsMode Loaded Workload: org.ldbcouncil.snb.driver.workloads.interactive.LdbcSnbInteractiveWorkload
Unable to load query from file: queries//interactive-complex-3-duration-as-function.cypherUnable to load query from file: queries//interactive-complex-4-duration-as-function.cypherUnable to load query from file: queries//interactive-complex-7-with-second.cypherUnable to load query from file: queries//interactive-update-1-add-person.cypherUnable to load query from file: queries//interactive-update-1-add-person-companies.cypherUnable to load query from file: queries//interactive-update-1-add-person-emails.cypherUnable to load query from file: queries//interactive-update-1-add-person-languages.cypherUnable to load query from file: queries//interactive-update-1-add-person-tags.cypherUnable to load query from file: queries//interactive-update-1-add-person-universities.cypherUnable to load query from file: queries//interactive-update-4-add-forum.cypherUnable to load query from file: queries//interactive-update-4-add-forum-tags.cypherUnable to load query from file: queries//interactive-update-6-add-post.cypherUnable to load query from file: queries//interactive-update-6-add-post-tags.cypherUnable to load query from file: queries//interactive-update-7-add-comment.cypherUnable to load query from file: queries//interactive-update-7-add-comment-tags.cypherUnable to load query from file: queries//interactive-update-6-add-post-content.cypherUnable to load query from file: queries//interactive-update-6-add-post-imagefile.cypherUnable to load query from file: queries//interactive-complex-3-duration-as-function.cypherUnable to load query from file: queries//interactive-complex-4-duration-as-function.cypherUnable to load query from file: queries//interactive-complex-7-with-second.cypherUnable to load query from file: queries//interactive-update-1-add-person.cypherUnable to load query from file: queries//interactive-update-1-add-person-companies.cypherUnable to load query from file: queries//interactive-update-1-add-person-emails.cypherUnable to load query from file: queries//interactive-update-1-add-person-languages.cypherUnable to load query from file: queries//interactive-update-1-add-person-tags.cypherUnable to load query from file: queries//interactive-update-1-add-person-universities.cypherUnable to load query from file: queries//interactive-update-4-add-forum.cypherUnable to load query from file: queries//interactive-update-4-add-forum-tags.cypherUnable to load query from file: queries//interactive-update-6-add-post.cypherUnable to load query from file: queries//interactive-update-6-add-post-tags.cypherUnable to load query from file: queries//interactive-update-7-add-comment.cypherUnable to load query from file: queries//interactive-update-7-add-comment-tags.cypherUnable to load query from file: queries//interactive-update-6-add-post-content.cypherUnable to load query from file: queries//interactive-update-6-add-post-imagefile.cypherCreateValidationParamsMode Loaded DB: org.ldbcouncil.snb.impls.workloads.cypher.interactive.CypherInteractiveDb
CreateValidationParamsMode Retrieving operation stream for workload: LdbcSnbInteractiveWorkload
WorkloadStreams Scanned 0 of 0 - OFFSET
WorkloadStreams Scanned 10,005 of 10,000 - RUN
CreateValidationParamsMode Driver Configuration
CreateValidationParamsMode Workload Start Time: 2025-03-13 - 02:22:52.828
Parameters:
Mode: create_validation
Name: LDBC-SNB
DB: org.ldbcouncil.snb.impls.workloads.cypher.interactive.CypherInteractiveDb
Workload: org.ldbcouncil.snb.driver.workloads.interactive.LdbcSnbInteractiveWorkload
Operation Count: 10,000
Warmup Count: 0
Skip Count: 0
Worker Threads: 1
Status Display Interval: 00:01.000 (m:s.ms)
Time Unit: MILLISECONDS
Results Directory: /home/test/Neo4j/ldbc_snb_interactive_v1_impls-1.0.0/cypher/results
Time Compression Ratio: 0.0010000
Validation Parameter Size: 50
Validation Serialization Check: true
Validation Database File: validation_params.csv
Spinner Sleep Duration: 00:00.000 (m:s.ms) / 0 (ms)
Print Help: false
Ignore Scheduled Start Times: true
User-defined parameters:
endpoint = bolt://localhost:7687
flush_log = false
ldbc.snb.interactive.LdbcQuery10_enable = true
ldbc.snb.interactive.LdbcQuery10_freq = 1
ldbc.snb.interactive.LdbcQuery10_interleave = 48
ldbc.snb.interactive.LdbcQuery11_enable = true
ldbc.snb.interactive.LdbcQuery11_freq = 1
ldbc.snb.interactive.LdbcQuery11_interleave = 48
ldbc.snb.interactive.LdbcQuery12_enable = true
ldbc.snb.interactive.LdbcQuery12_freq = 1
ldbc.snb.interactive.LdbcQuery12_interleave = 48
ldbc.snb.interactive.LdbcQuery13_enable = true
ldbc.snb.interactive.LdbcQuery13_freq = 1
ldbc.snb.interactive.LdbcQuery13_interleave = 48
ldbc.snb.interactive.LdbcQuery14_enable = true
ldbc.snb.interactive.LdbcQuery14_freq = 1
ldbc.snb.interactive.LdbcQuery14_interleave = 48
ldbc.snb.interactive.LdbcQuery1_enable = true
ldbc.snb.interactive.LdbcQuery1_freq = 1
ldbc.snb.interactive.LdbcQuery1_interleave = 48
ldbc.snb.interactive.LdbcQuery2_enable = true
ldbc.snb.interactive.LdbcQuery2_freq = 1
ldbc.snb.interactive.LdbcQuery2_interleave = 48
ldbc.snb.interactive.LdbcQuery3_enable = true
ldbc.snb.interactive.LdbcQuery3_freq = 1
ldbc.snb.interactive.LdbcQuery3_interleave = 48
ldbc.snb.interactive.LdbcQuery4_enable = true
ldbc.snb.interactive.LdbcQuery4_freq = 1
ldbc.snb.interactive.LdbcQuery4_interleave = 48
ldbc.snb.interactive.LdbcQuery5_enable = true
ldbc.snb.interactive.LdbcQuery5_freq = 1
ldbc.snb.interactive.LdbcQuery5_interleave = 48
ldbc.snb.interactive.LdbcQuery6_enable = true
ldbc.snb.interactive.LdbcQuery6_freq = 1
ldbc.snb.interactive.LdbcQuery6_interleave = 48
ldbc.snb.interactive.LdbcQuery7_enable = true
ldbc.snb.interactive.LdbcQuery7_freq = 1
ldbc.snb.interactive.LdbcQuery7_interleave = 48
ldbc.snb.interactive.LdbcQuery8_enable = true
ldbc.snb.interactive.LdbcQuery8_freq = 1
ldbc.snb.interactive.LdbcQuery8_interleave = 48
ldbc.snb.interactive.LdbcQuery9_enable = true
ldbc.snb.interactive.LdbcQuery9_freq = 1
ldbc.snb.interactive.LdbcQuery9_interleave = 48
ldbc.snb.interactive.LdbcShortQuery1PersonProfile_enable = true
ldbc.snb.interactive.LdbcShortQuery2PersonPosts_enable = true
ldbc.snb.interactive.LdbcShortQuery3PersonFriends_enable = true
ldbc.snb.interactive.LdbcShortQuery4MessageContent_enable = true
ldbc.snb.interactive.LdbcShortQuery5MessageCreator_enable = true
ldbc.snb.interactive.LdbcShortQuery6MessageForum_enable = true
ldbc.snb.interactive.LdbcShortQuery7MessageReplies_enable = true
ldbc.snb.interactive.LdbcUpdate1AddPerson_enable = true
ldbc.snb.interactive.LdbcUpdate2AddPostLike_enable = true
ldbc.snb.interactive.LdbcUpdate3AddCommentLike_enable = true
ldbc.snb.interactive.LdbcUpdate4AddForum_enable = true
ldbc.snb.interactive.LdbcUpdate5AddForumMembership_enable = true
ldbc.snb.interactive.LdbcUpdate6AddPost_enable = true
ldbc.snb.interactive.LdbcUpdate7AddComment_enable = true
ldbc.snb.interactive.LdbcUpdate8AddFriendship_enable = true
ldbc.snb.interactive.parameters_dir = /home/test/Neo4j/ldbc_snb_datagen_hadoop/substitution_parameters/
ldbc.snb.interactive.scale_factor = 100
ldbc.snb.interactive.short_read_dissipation = 0.2
ldbc.snb.interactive.update_interleave = 48
ldbc.snb.interactive.updates_dir = /home/test/Neo4j/ldbc_snb_datagen_hadoop/social_network/
log_delayed = true
password = admin
peer_identifiers =
printQueryNames = true
printQueryResults = false
printQueryStrings = false
queryDir = queries/
results_log = false
user = neo4j
workload_statistics = false
CreateValidationParamsMode Generating database validation file: /home/test/Neo4j/ldbc_snb_interactive_v1_impls-1.0.0/cypher/validation_params.csv
Client Client terminated unexpectedly
org.ldbcouncil.snb.driver.ClientException: Error encountered duration validation parameter creation
at org.ldbcouncil.snb.driver.client.CreateValidationParamsMode.startExecutionAndAwaitCompletion(CreateValidationParamsMode.java:193)
at org.ldbcouncil.snb.driver.Client.main(Client.java:65)
Caused by: org.ldbcouncil.snb.driver.ClientException: Error trying to write validation parameters to CSV file writer
at org.ldbcouncil.snb.driver.client.CreateValidationParamsMode.startExecutionAndAwaitCompletion(CreateValidationParamsMode.java:182)
... 1 more
Caused by: java.lang.NullPointerException
at org.ldbcouncil.snb.driver.validation.ValidationParamsToCsvRows.next(ValidationParamsToCsvRows.java:96)
at org.ldbcouncil.snb.driver.validation.ValidationParamsToCsvRows.next(ValidationParamsToCsvRows.java:18)
at org.ldbcouncil.snb.driver.client.CreateValidationParamsMode.startExecutionAndAwaitCompletion(CreateValidationParamsMode.java:165)
... 1 more