Modify is_remote_filesystem to return True for FUSE-mounted paths
#5885
Conversation
maddiedawson
changed the title
is_remote_filesystem to return True for FUSE-mounted paths
maddiedawson
changed the title
is_remote_filesystem to return True for FUSE-mounted pathsis_remote_filesystem to return True for FUSE-mounted paths
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
maddiedawson
force-pushed
the
dbfs
branch
3 times, most recently
from
4a4d3d3 to
8fd672b
Compare
| return False | ||
| if isinstance(fs, FuseFileSystem) or fs.protocol != "file": |
There was a problem hiding this comment.
Is it possible for a FUSE-mounted file system to not be a remote file system?
There was a problem hiding this comment.
Discussed offline, but I'm not sure if someone would use Fuse to mount a local fs
|
@lhoestq would you or another maintainer be able to review please? :) |
|
Why you do need to support FUSE mounted paths ?
|
|
Fuse is commonly used to mount remote file systems (e.g. S3, DBFS) as a local directory. Since it's slower than using an actual local device, it's better to treat it as remote to reduce latency. |
|
I think people would be confused if they don't have the same dataset behavior depending on the disk type. If they want to use a remote bucket they should use the remote URI instead, e.g. |
No description provided.