-
Notifications
You must be signed in to change notification settings - Fork 1.6k
feat: add era import persistence commit #16059
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: add era import persistence commit #16059
Conversation
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should work, but I think this has side-effects if used from within a pipeline stage where we need to adhere to some rules when it comes to when we are allowed to open certain writers and commit,
but I can't fully remember what those are so we need to figure this out first.
@RomanHodulak we're definitely missing commit calls in the standalone import fn, so I think the challenge is now unifying this with the stage impl
Continuing #15982 that was wrongly open from
main
branch.Prep for #15909.
It adds file storage/db persistence per file,
and then a global db commit at the end of the import.cc @RomanHodulak, @mattsse.