Virtual File System and "directory listing"

[kolban]

Looking at the list of functions to be implemented by a virtual file system, I find myself surprised that there is nothing that exposes any kind of directory structure. We can open "files", read them, write them, close them, rename them, and delete them (unlink) ... but nothing obvious to "list" them? Was that by design?

[ESP_igrr]

From the application side, newlib 'opendir' and 'readdir' functions should be used. Internally they use open and read syscalls to open directory and read the list of files. These syscalls should be implemented by the fs driver.
Having said that, I haven't actually tested if 'opendir' and 'readdir' work as expected. Thanks for pointing this out. I'll write a test case and see if it works.

[kolban]

I think you answered my question. If we want to provide directory functions, then we must implement open and read to map to be cognizant of the data structures that opendir and readdir expect to receive. For example ... if I have a virtual file system mounted at /data and have the following:

/data/text.txt
/data/mydir
/data/mydir/text2.txt
/data/mydir/text3.txt

The our VFS handlers, when asked to open or stat "/data/mydir" would return the "data" that indicates that "/data/mydir" is a directory and also if one "reads" /data/mydir, it should return the data structure that would contain:

text2.txt
text3.txt

So the light bulb for me is that the VFS isn't as abstract a concept as I might have thought it was ... but rather our implementations must map from the "logical file system" we have available to us to what VFS presents ... which is an implementation of the Unix file system APIs and data structures on top of our "logical file system".

Please be sure and realize I'm NOT complaining in the least ... just trying to understand.

[ESP_igrr]

I think it may be possible to add provisions for directory listing functions which may be provided by the filesystem, but we will have to change the way 'readdir' and 'opendir' functions are implemented in newlib. The way the are implemented now relies on open and read, but I do realize this adds an extra level of indirection if FS already provides higher level functions.

In any case, you have given another point which should be mentioned in the docs. Regarding the placement of VFS, I realize I might have not chosen a proper name for this component. It works as a glue between newlib above and various FS drivers below, dispatching the calls from newlib to one of the drivers depending on the path prefix and file descriptor.

[kolban]

I had a search through the ESP-IDF github source and looked for opendir and readdir. Unfortunately I seem to find that opendir with dirent is not currently supported ... see:

https://github.com/espressif/esp-idf/bl ... s/dirent.h

Is this function still to come? Is there an alternate recipe I should be looking for?

[ESP_igrr]

You are correct. I have just checked on the same and it turns out that we don't have HAVE_OPENDIR and HAVE_READDIR enabled in our newlib config. Therefore newlib was built without these functions.

So i'm going to add them to VFS interface, so that a filesystem can implement them directly, without having to go via 'open' and 'read'.

[markwj]

You are correct. I have just checked on the same and it turns out that we don't have HAVE_OPENDIR and HAVE_READDIR enabled in our newlib config. Therefore newlib was built without these functions.

So i'm going to add them to VFS interface, so that a filesystem can implement them directly, without having to go via 'open' and 'read'.

I've just run into this. Getting an error back from opendir(), and it seems unimplemented for '/' root directory, but I can opendir() on '/sdcard' as a fatfs (for example).

Has there been any progress on putting these into the VFS interface, or what is the current implementation status?

[kolban]

There is an issue flagged as "in progress" that seems to prevent using opendir() on a mount point path ... see:

https://github.com/espressif/esp-idf/issues/664

There was also an issue on the notion of mounting at "/" itself ... so a file system listing of "/" downwards may be being addressed by:

https://github.com/espressif/esp-idf/issues/135

[ESP_igrr]

These functions are now exposed in VFS interface, and FATFS driver implements them.

However these functions don't do anything useful for "/", they only provide listing for paths which match one of the registered filesystems.

If you have a use case for listing root directories, please open a feature on Github and describe how you would like this to work.

[kolban]

@ESP_igrr,
Can you elaborate on the last post?

I can imagine that I want to use just one file system implementation in a solution (eg. FatFS). Rather than mount it at:

/myfs

I believe I would want to mount it at "/"

The thinking here is that since I know I have only one file system, why *not* mount it at root?

If I have a use case for only one file system and want to mount it at "/" then it would then seem to be desirable to be able to perform a directory listing at "/".