SD Card (using FAT) file names randomly changing to uppercase with '~1'

[ThanhAuDuong]

Hello,
I'm currently using an ESP32-S3 to read/write files to an SD card using the FAT file system. I've noticed an issue where most of the file, folders names occasionally get converted to uppercase and appended with "~1".

For example:
- The original file name is "checkpoint-0001".
- After some time, it changes to "CHECKP~1".

My file names are typically between 5 to 32 characters long, and I don't believe the system have trouble handling long file names. The issue seems to occur randomly and are very hard to reproduce.
Does anyone have an idea why this is happening?
Or any direction for debugging would be greatly appreciated.
Thanks in advance!

[chegewara]

It is DOS convention, 8+3, naming.
Please check menucofig option for long names.

[ThanhAuDuong]

It is DOS convention, 8+3, naming.
Please check menucofig option for long names.

Hi,
I checked, and the "Long filename support" option was already set to "Long filename buffer on stack," so I assume the long filename option is enabled, max length is 255.
As I tested, the file names only changes after the system has been running for a few days (3-4 days, maybe; it’s not consistent). Prior to that, the system had no problem reading/writing files with names longer than 8 characters.

[chegewara]

It is odd, i never seen it before.
I am suggesting to create issue on github.

[adokitkat]

Hello. Could you please answer these questions?

• Which IDF version are you using please? Did this happen on an older version?

• What is your usual workflow with the files? Are you repeatably appending to them?

• Is your ESP32S3 chip 100% always running or is there a possibility it doesn't have a reliable power supply and could lose power during operation?

• Is data in those file intact/consistent after this happens or is it lost/corrupted?

As it was already said, this is name mangling to conform to 8.3 short file name performed by FATFS library when saved files have long names while long name support is not enabled - which it should be in your case, since this doesn't happen immediately...

[ThanhAuDuong]

Hello. Could you please answer these questions?

• Which IDF version are you using please? Did this happen on an older version?

• What is your usual workflow with the files? Are you repeatably appending to them?

• Is your ESP32S3 chip 100% always running or is there a possibility it doesn't have a reliable power supply and could lose power during operation?

• Is data in those file intact/consistent after this happens or is it lost/corrupted?

As it was already said, this is name mangling to conform to 8.3 short file name performed by FATFS library when saved files have long names while long name support is not enabled - which it should be in your case, since this doesn't happen immediately...

Hi,
1. I'm currently using IDF v5.1.2 with ADF v2.6 for audio playback. These are the only versions I've used from the beginning.
2. I mostly work with MP3 files and a log file. The log file is repeatedly appended, whereas the MP3 files are written to the SD card only once when I need to load them. However, the system repeatedly reads the MP3 files during audio playback.
3. I'm certain that the ESP32S3 is always running. I implemented an SD logging mechanism that continuously writes logs to the SD card. After the issue occurs, I review these logs, and there is no indication of a hard reset.
4. I noticed that a very small part of the log file appeared to be corrupted, but most of the data remained readable.

[adokitkat]

Are you sure you always close the file (or flush it) after writing/appending?

Are you sure you have enough memory? Maybe you could try to use LFN with heap instead of stack? I am sorry I am not giving you a concrete answer but FATFS code is not so easy to read and reason about...

There certainly is a corruption happening, but I am not sure where exactly and why. Filenames are always saved in SFN form (and mangled if necessary) even when LFN are enabled - LFN entries can store 13 characters each, are preceding and link to the SFN entry. There also is a checksum in each LFN entry. If something goes wrong there, the association is broken and LFN entries become ignored garbage. Here you can read more on how it works: http://elm-chan.org/docs/fat_e.html#lfn

Would it be possible for you to upgrade to IDF v5.3 and ADF v2.7? There have been a few changes and fixes in FATFS related code, though I do not guarantee it will 100% fix this issue. I just remember we have seen something similar to this and it was fixed when we updated the FATFS code (FATFS library is not made by us, it is 3rd party library we port and patch).