Deprecate pyfilesystem backend?

[timkpaine]

Such a deprecation plan would require:

• fsspec Localfilesystem replaces pyfilesystem osfs this appears to already be the case
• s3fs replaces pyfilesystem fs-s3fs - testing but its looking good
• fsspec smb replaces fs.smbfs - this has historically been the difficulty

[tharwan]

Is it already possible to configure jupyter-fs to use fsspec for smb? I could not quite tell from the readme but I am happy to try it in our setup and help stabilise it.

[vidartf]

A major improvement that can be made to the fsspec SMB code would be to have it rely on the smbclient.scandir call (at least when detail=True is passed to fsspec's .listdir method). Then it wouldn't need to do a stat call per directory entry in listdir, which can be very expensive in terms of number of network requests. But at least that can be patched in locally, and the current behavior matches that of PyFilesystem, so I'm mentioning it more as an FYI rather than something that should block this issue.

[vidartf]

Ah, I got distracted, I meant to ask what the issue is with the fsspec smb client implementation, as you mentioned that as a requirement.

[timkpaine]

Ah, I got distracted, I meant to ask what the issue is with the fsspec smb client implementation, as you mentioned that as a requirement.

I honestly dont remember at this point but i very vaguely recall @telamonian looking into it and there being some problem. I plan on testing shortly, I'm hoping to make our integration tests against smb and s3 parameterized by both pyfilesystem and fsspec.