feat: Track individual expr's execution time in ProjectionExec metrics #18570
+199
−3
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
ProjectExecmetrics (inEXPLAIN ANALYZE) #18456Rationale for this change
Currently,
EXPLAIN ANALYZEforProjectionExeconly shows aggregated metrics like total elapsed compute time, but doesn't provide visibility into how much time individual expressions within the projection consume. This makes it difficult to identify performance bottlenecks when debugging slow queries with complex projection expressions.What changes are included in this PR?
1. Enhanced
ProjectionExecwith per-expression timing metrics:- Added per-expression Time metrics collection in
ProjectionStream- Each projection expression gets its own timer using the expression alias as the metric name
- Metrics are properly registered with the execution plan's metrics set
2. Extended metrics display in
EXPLAIN ANALYZE:- Modified
display.rsto detect ProjectionExec plans- Added
get_projection_expr_metrics()helper function to extract per-expression timing data- Enhanced metrics output to include per-expression times alongside standard metrics
3. Added intelligent time formatting:
- Implemented proper time unit formatting (ns, μs, ms, s) based on duration magnitude
- Maintains consistency with DataFusion's existing time formatting standards
Are these changes tested?
Yes - The tests have been added to check the proper functionality of latest functions.
Are there any user-facing changes?
Yes - This is an enhancement to the existing
EXPLAIN ANALYZEfunctionality