Add more metrics to DataSourceExec for vortex format in datafusion
#5851
haohuaijin
started this conversation in
Feature Requests
Replies: 3 comments 1 reply
|
Yeah this is definitely on my list, I'm planning on pushing some "completeness" datafusion stuff like this now that we finished merging the 51 upgrade which took longer than expected. |
0 replies
|
Its also related to some deeper changes in Vortex - I think the whole metrics things would be pretty nice if its a pluggable facade, but that's a bigger conversation that probably needs its own thread. |
1 reply
|
Starting to experiment with a potential Parquet -> Vortex migration, and indeed performance metrics would be very useful, notably pruning statistics / compute ressource consumption, which are core metrics we monitor in production |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
current only a few metrics show in DataSourceExec when search vortex format, for example

the parquet format have many useful metrics for debug the performance issue

All reactions