You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Apr 20, 2020. It is now read-only.
The BlockwiseFileset class creates a lot of small or empty files that are used for bookkeeping purposes. On HPC storage, metadata performance on such files introduces a large overhead. Somehow these book-keeping items need to be kept somewhere else. Probably a simple database or sqlite store would be fine, provided it can handle a bit of concurrency.
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The
BlockwiseFileset
class creates a lot of small or empty files that are used for bookkeeping purposes. On HPC storage, metadata performance on such files introduces a large overhead. Somehow these book-keeping items need to be kept somewhere else. Probably a simple database or sqlite store would be fine, provided it can handle a bit of concurrency.The text was updated successfully, but these errors were encountered: