-
Notifications
You must be signed in to change notification settings - Fork 5.5k
feat: Add max_split_preload_per_driver to java system config #26589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Reviewer's guide (collapsed on small PRs)Reviewer's GuideIntroduce a new Java session property Class diagram for updated SystemSessionProperties with max_split_preload_per_driverclassDiagram
class SystemSessionProperties {
+static final String MAX_SPLIT_PRELOAD_PER_DRIVER
+int getMaxSplitPreloadPerDriver(Session session)
}
SystemSessionProperties --> Session : uses
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey there - I've reviewed your changes and they look great!
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location> `presto-main-base/src/main/java/com/facebook/presto/SystemSessionProperties.java:194` </location>
<code_context>
public static final String IGNORE_STATS_CALCULATOR_FAILURES = "ignore_stats_calculator_failures";
public static final String PRINT_STATS_FOR_NON_JOIN_QUERY = "print_stats_for_non_join_query";
public static final String MAX_DRIVERS_PER_TASK = "max_drivers_per_task";
+ public static final String MAX_SPLIT_PRELOAD_PER_DRIVER = "max_split_preload_per_driver";
public static final String MAX_TASKS_PER_STAGE = "max_tasks_per_stage";
public static final String DEFAULT_FILTER_FACTOR_ENABLED = "default_filter_factor_enabled";
</code_context>
<issue_to_address>
**suggestion:** Consider validating the value for MAX_SPLIT_PRELOAD_PER_DRIVER.
Please add a validation step for MAX_SPLIT_PRELOAD_PER_DRIVER to ensure its value is non-negative and within reasonable limits, similar to the approach used for MAX_DRIVERS_PER_TASK.
Suggested implementation:
```java
integerProperty(
MAX_SPLIT_PRELOAD_PER_DRIVER,
"Maximum number of splits to preload per driver. Set to 0 to disable preloading",
0,
false,
value -> validateNullablePositiveIntegerValue(value, MAX_SPLIT_PRELOAD_PER_DRIVER),
object -> object),
```
If `validateNullablePositiveIntegerValue` is not already imported or available in this file, you will need to import or implement it. Also, ensure that the validation logic matches your requirements for "reasonable limits" (e.g., you may want to set an upper bound).
</issue_to_address>
### Comment 2
<location> `presto-main-base/src/main/java/com/facebook/presto/SystemSessionProperties.java:1030` </location>
<code_context>
object -> object),
+ integerProperty(
+ MAX_SPLIT_PRELOAD_PER_DRIVER,
+ "Maximum number of splits to preload per driver. Set to 0 to disable preloading",
+ 0,
+ false),
</code_context>
<issue_to_address>
**suggestion:** Clarify behavior for negative values in the property description.
If negative values are invalid, please update the description to explicitly state this.
```suggestion
"Maximum number of splits to preload per driver. Must be zero or positive. Set to 0 to disable preloading. Negative values are not allowed.",
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| object -> object), | ||
| integerProperty( | ||
| MAX_SPLIT_PRELOAD_PER_DRIVER, | ||
| "Maximum number of splits to preload per driver. Set to 0 to disable preloading", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: Clarify behavior for negative values in the property description.
If negative values are invalid, please update the description to explicitly state this.
| "Maximum number of splits to preload per driver. Set to 0 to disable preloading", | |
| "Maximum number of splits to preload per driver. Must be zero or positive. Set to 0 to disable preloading. Negative values are not allowed.", |
|
could you rename to session property? otherwise LGTM, thanks @yjyolandeyan ! |
|
Please add documentation for |
as titled
Description
add java session property max_split_preload_per_driver with default value 0
Motivation and Context
enable feature waitForPreloadSplitNanos
Impact
Test Plan
[e2e]
20251106_193157_00009_runst: waitForPreloadSplitNanos enabled when max_split_preload_per_driver = 2
20251105_230547_00023_gy8x9: waitForPreloadSplitNanos disabled when max_split_preload_per_driver = 0 (default)
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.