Improve speed of file system

[ttlappalainen]

Related area

vfs_api

Hardware specification

ESP32

Is your feature request related to a problem?

espressif/esp-idf#3277

Describe the solution you'd like

In vfs_api.cpp FileImplPtr VFSImpl::open(...) calls stat, which is pretty slow with LittleFS or SPIFFS. After that succeeded open returns
return std::make_shared<VFSFileImpl>(this, path, mode);
VFSFileImpl constructor calls again stat. I made experimental new constructor, which gets stat structure and full path:
VFSFileImpl(VFSImpl* fs, const struct stat &st, const char* fpath, const char* path, const char* mode);
then instead of original constructor I call

        if (S_ISREG(st.st_mode) || S_ISDIR(st.st_mode)) {
            FileImplPtr ret=std::make_shared<VFSFileImpl>(this, st, temp, path, mode);
            free(temp);
            return ret;
        }
...

Since new constructor gets stat structure from caller, it does not need to call it again and so my open times are half of original. Currently I use that constructor only for existing files, but if new constructor would also get stat result as parameter, it could be used also for new files:
VFSFileImpl(VFSImpl* fs, const struct stat &st, int statResult, const char* fpath, const char* path, const char* mode);

Do you find any problem using open in this way?

Describe alternatives you've considered

No response

Additional context

No response

I have checked existing list of Feature requests and the Contribution Guide

• I confirm I have checked existing list of Feature requests and Contribution Guide.

[BlueAndi]

Additional performance improvements:

• Replace all sprintf(buffer, "%s%s", ...) with strcpy/strcat
• Reduce heap usage by using String, which really slows down. See this code fragment, as example for possible improvement:
  

  arduino-esp32/libraries/FS/src/vfs_api.cpp

  Lines 469 to 474 in bf53e2b

  	String fname = String(file->d_name);
  	String name = String(_path);
  	if(!fname.startsWith("/") && !name.endsWith("/")) {
  	name += "/";
  	}
  	name += fname;

See #7541

[Zhu-jiatong]

This fix could be extremely helpful for projects where the iteration of entire directories is needed. Hope it gets implemented soon.

[Parsaabasi]

Hello,

Due to the overwhelming volume of issues currently being addressed, we have decided to close the previously received tickets. If you still require assistance or if the issue persists, please don't hesitate to reopen the ticket.

Thanks.

[hitecSmartHome]

whats up with this?

[ttlappalainen]

I use the solution I orignally explained in system I make code.

[hitecSmartHome]

Could you reopen it pls? Fs is really slow

[hitecSmartHome]

I have tried to tweak fs in the menuconfig to make it faster. I played with cache sizes and things like that. I was able to optimise it for a lot of small files but it was unstable.

[ttlappalainen]

If there is any use for my changes, I can send them. I have them for Espressif 6.5 version.

[hitecSmartHome]

It would be awesome

[lbernstone]

@ttlappalainen Submit a PR. The devs will make any suggestions they have in the PR, and then it will get merged into the code.