Use Velox fs for ssd cache evictlog file #11495
Conversation
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
|
@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| @@ -66,28 +69,6 @@ void disableCow(int32_t fd) { | |||
| #endif // linux | |||
| } | |||
|
|
|||
| // TODO: Remove this function once we migrate all files to velox fs. | |||
There was a problem hiding this comment.
We still have checkpoint file write though posix api? thanks!
| if (evictLogFd_ < 0) { | ||
| filesystems::FileOptions evictLogFileOptions; | ||
| evictLogFileOptions.shouldThrowOnFileAlreadyExists = false; | ||
| try { |
There was a problem hiding this comment.
Consider to provide a macro in followup
#define FILE_CALL(fileFunction, errorCallback);
errorCallback expects a std::exception and both fileFunction and errorCallback could be a lambda function to capture requires execution context.
| @@ -965,6 +948,9 @@ void SsdFile::disableFileCow() { | |||
| const std::unordered_map<std::string, std::string> attributes = { | |||
| {std::string(LocalWriteFile::Attributes::kNoCow), "true"}}; | |||
| writeFile_->setAttributes(attributes); | |||
| if (evictLogWriteFile_ != nullptr) { | |||
| evictLogWriteFile_->setAttributes(attributes); | |||
There was a problem hiding this comment.
We don't disable checkpoint file?
There was a problem hiding this comment.
It is disabled in a separate path as we haven't switched checkpoint files to velox fs.
velox/common/caching/SsdFile.cpp
Outdated
| try { | ||
| evictLogReadFile->pread(0, logSize, evicted.data()); | ||
| } catch (const std::exception& e) { | ||
| ++stats_.deleteCheckpointErrors; |
There was a problem hiding this comment.
We shall throw by VELOX_FAIL here? Can you use file fault injection to trigger the log eviction read failure?
zacw7
force-pushed
the
fs-evictlog
branch
2 times, most recently
from
2181e3b
to
69db055
Compare
|
@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: Switch the eviction log file to use Velox filesystem for file r/w operations, so that more advanced testing can be built by leveraging features like fault injections. Differential Revision: D65740612 Pulled By: zacw7
|
This pull request was exported from Phabricator. Differential Revision: D65740612 |
Summary: Switch the eviction log file to use Velox filesystem for file r/w operations, so that more advanced testing can be built by leveraging features like fault injections. Differential Revision: D65740612 Pulled By: zacw7
|
This pull request was exported from Phabricator. Differential Revision: D65740612 |
Switch the eviction log file to use Velox filesystem for file r/w operations, so that more advanced testing can be built by leveraging features like fault injections.