You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently if a task failed, the reducer stopped to read data and output to parquet. So the reducer data isn't completed. We need a way to read full data once the partition is enabled.
currently we filter sample by stage id and task id. we need to add filter by partition size as well. @marin-ma can we get the records number from driver before reducer read the data? I'd think so.
The text was updated successfully, but these errors were encountered:
Description
The text was updated successfully, but these errors were encountered: