How To Run Expression Fuzzer Test with Presto as the Source of Truth?

[kagamiori]

Description

The Velox expression fuzzer test generates random expressions and evaluates them in Velox with random input data. It has been a powerful tool for detecting bugs in the Velox expression evaluator and scalar functions. There is an ongoing extension to this test to verify the Velox expression results against Presto’s.

Test Setup

Below are the steps to run the expression fuzzer test with verification against Presto on your Mac.

1. Check out the Velox repository from Github on your Mac and follow this instruction to build Velox.
2. Follow this instruction to start the Presto server in the background.
   • Download the latest version of Presto instead of 0.284 said in the instruction. E.g., 0.289 is the latest as of Oct 2024.
   • Follow the instructions till Step 14.
   • For Step 10, follow the instruction here: Setting up Presto to use as source of truth for Fuzzers #8111 (comment).
3. Build the target velox_expression_fuzzer_test in CLion or in your terminal.
4. While Presto server is running in the background, run velox_expression_fuzzer_test with the following arguments.

--enable_variadic_signatures
--velox_fuzzer_enable_complex_types
--lazy_vector_generation_ratio 0.2
--velox_fuzzer_enable_column_reuse
--velox_fuzzer_enable_expression_reuse
--max_expression_trees_per_step 2
--duration_sec=1800
--logtostderr=1
--minloglevel=0
--presto_url=http://127.0.0.1:8080
--batch_size=6
--special_forms="cast,coalesce,if,switch"
-v=1

The --duration_sec=1800 argument makes the expression fuzzer test run for 30 minutes. --batch_size=6 makes the input batches contain 6 rows. Please feel free to adjust them according to your needs.

5. After the expression fuzzer test finishes, find the pid of the Presto server and shut it down.

ps -x | grep presto
kill <pid of presto>

Run Expression Fuzzer with Specific Functions

If you would like to run expression fuzzer with only specific functions, add a command line argument like the following to velox_expression_fuzzer_test at the step 4 above. Names in the quoted list are Velox function names. E.g.,
--only="eq,neq,lt,gt,lte,gte".