POSIX file operations implemented on ESP32

zliudr
Posts: 105
Joined: Thu Oct 03, 2019 5:15 am

POSIX file operations implemented on ESP32

Postby zliudr » Fri Jan 17, 2020 5:54 am

Question: if one task does something such as rename files in a directory (move them out of the directory), and another task does something else such as creating and writing files to the same folder, will there be any conflict? I am talking about the directory needs to be opened by the routine that renames/moves files (deletes entries from directory) and also by the routine that creates and writes files (add entries to directory).

I'm sure if it is Linux, the answer is NO conflicts but since this is an ESP32 (or RTOS) implementation, I am hoping for a NO too.

PeterR
Posts: 276
Joined: Mon Jun 04, 2018 2:47 pm

Re: POSIX file operations implemented on ESP32

Postby PeterR » Fri Jan 17, 2020 8:24 pm

e.g. https://stackoverflow.com/questions/424 ... ltaneously
So no, not all concurrent file access are safe. Even if declared as safe then I would take a 'I don't want to debug a potential RT issue' and use a C++11 mutex and sleep at night. Simple, quick, no more than a few 10's of nS performance hit etc.
A FLASH writes takes uS if not mS so your mutex use is trivial.

zliudr
Posts: 105
Joined: Thu Oct 03, 2019 5:15 am

Re: POSIX file operations implemented on ESP32

Postby zliudr » Mon Jan 20, 2020 5:52 pm

PeterR,

Thank you for your help! I just read about mutex (seen the term but never investigated). It seems that I should write a function that accesses the file directory and have mutex in it. Then different tasks shall call the same function when they want access to file directories, such as creating a file, moving a file.

If such thing as mutex already exists, why would a file operation library not do it internally to rid developers from having to do the same at application dev. level? Just curious. I've used MS DOS in the distant past. There is a place where DOS stores a status that says it's running. I think it's in case an interrupt handler also calls DOS when the main routine is interrupted while calling DOS. It should be implemented below the app. dev. level in my opinion.

PeterR
Posts: 276
Joined: Mon Jun 04, 2018 2:47 pm

Re: POSIX file operations implemented on ESP32

Postby PeterR » Mon Jan 20, 2020 7:17 pm

When would the filesystem release/set the mutex. On fopen()/fclose() or each fwrite()/fread()?
What happens if you fseek() and then the file is renamed/deleted?

Lots of documentation for the filesystem man. Guess he might just say that you should marshal access to the file system, (his) life is too short.

I cannot rename/delete a file that is open on Windows 10 BTW

Fundamentally a file is a shared resource and so I would always recommend marshelling access (because that works and otherwise I need to understand so much).

& yes wrap you're logging function, log.write(log *log) & log.tidy() can lock the open/close/delete of files, initiate tidying etc and launch and synchronise as many threads as needed to get the job done.

zliudr
Posts: 105
Joined: Thu Oct 03, 2019 5:15 am

Re: POSIX file operations implemented on ESP32

Postby zliudr » Mon Feb 10, 2020 3:15 am

Sorry I just got back to implementing mutex. Is it wise to just have one mutex for file system access or have multiple, one for each file that could be written/read by different tasks? Maybe the former for simplicity and memory saving?

In case of the single mutex for file system, I can just insert mutexTake() right before any file operation and mutexGive() right after. I'm not sure if file content is buffered beyond fclose() and hope not.

PeterR
Posts: 276
Joined: Mon Jun 04, 2018 2:47 pm

Re: POSIX file operations implemented on ESP32

Postby PeterR » Mon Feb 10, 2020 11:43 pm

I don't know the ESP specifics.
I started a responce with a long discussion on the 'C' standards.
Its best sumarised with 'don't, I cannot find a statement on multi threading write/rename' (no threads in C).

So sequence. The hardware will only support one erase/write at the same time so you will not loose much by doing so.
EDIT: One semaphore for the entire file system.

Who is online

Users browsing this forum: Baidu [Spider], Google [Bot], Palonso and 15 guests