SDMMC Best Practices for Continual Writing

[coder.for.life]

I have a setup where I will continually write data to the SD card whenever there is an SD card inserted (187.5 KB/sec written in chunks of ~9600 every 50 milliseconds). The SD card can be removed at any point and reinserted later. The data between then is unimportant, and losing a few chunks when removed/inserted is not important either. But, we want to make sure that we don't damage the SD card itself or do anything to corrupt the filesystem making everything inaccessble.

I know that to get going for an SD card the following must be done:
1. `sd_pwr_ctrl_new_on_chip_ldo()`
2. `sdmmc_host_init()`
3. `sdmmc_host_init_slot()`
4. `esp_vfs_fat_register_cfg()`
5. `sdmmc_card_init()`
6. `ff_diskio_register_sdmmc()`
7. `f_mount()`

However, it seems that #1-3 definitely can happen any time. Do they have to be re-done if a card is removed and reinserted? What are the consequences of leaving the LDO, host, and slot setup? (More power drain? How much? Frying a card that is inserted?)

Additionally, #4 can also happen early and only once with no consequences. Is this true? Can the resulting FATFS object be shared between mounts (is it reset when re-mounted to a possibly different SD card)?

I know that 5-7 have to be done with the card inserted.

While I am writing data in chunks, can I keep a `FILE*` open the entire time, or should I close and re-open it every 50 ms? If I don't close it, I will at least `fflush()` it after each chunk written. If I keep it open, should I still call `fclose()` after I know the card is gone?

What happens if I don't call `f_mount(NULL, drv, 0);` to unmount? Is there anything else that must be done to unmount (e.g. do I need to use `esp_vfs_fat_unregister_path()` or can I leave it perpetually registered?)? Can/should I unmount after the card is removed? Or do I just mount again when I see another card?

Once I know a card is ejected, can I poll `sdmmc_get_status()` with the same card object (for a card that isn't there anymore) to see when another card is inserted? Or is there a better way to check for an inserted card (I do not have access to a CD pin)?

[pacucha42]

I have a setup where I will continually write data to the SD card whenever there is an SD card inserted (187.5 KB/sec written in chunks of ~9600 every 50 milliseconds). The SD card can be removed at any point and reinserted later. The data between then is unimportant, and losing a few chunks when removed/inserted is not important either. But, we want to make sure that we don't damage the SD card itself or do anything to corrupt the filesystem making everything inaccessble.

I know that to get going for an SD card the following must be done:
1. `sd_pwr_ctrl_new_on_chip_ldo()`
2. `sdmmc_host_init()`
3. `sdmmc_host_init_slot()`
4. `esp_vfs_fat_register_cfg()`
5. `sdmmc_card_init()`
6. `ff_diskio_register_sdmmc()`
7. `f_mount()`

There are certain discrepancies in your list - first: sd_pwr_ctrl_new_on_chip_ldo() is applicable only for chips with on-board LDO (like ESP32P4), second: all the rest is processed by convenience APIs provided for each FS/media supported (eg esp_vfs_fat_sdmmc_mount()). Your scheme is basically correct but the sequence is slightly improper - esp_vfs_fat_register_cfg() shouldn't be called before sdmmc_card_init() because it provides valid VFS function pointers not yet connected to real file-system. There are also few more details to handle during the mounting process. If you plan to do all the initialization manually, please refer to the internals of the convenience APIs. I understand you need to implement your own (re)mounting API functions, but I recommend staying as close as possible to the convenience functions mentioned.

However, it seems that #1-3 definitely can happen any time. Do they have to be re-done if a card is removed and reinserted?

I recommend to reinitialize everything from scratch since SD card removal and reinsert means a complete flow reset, as it is actually a power off/on cycle. The card’s status registers are reset, the initial state is set back to IDLE, etc. Theoretically, the SD host could stay powered up but ESP-IDF code deinitializes the host driver automatically when the last used slot is released.

What are the consequences of leaving the LDO, host, and slot setup? (More power drain? How much? Frying a card that is inserted?)

The power consumption remains pretty much the same when the card is unplugged, and the LDO and host controller do nothing. However, if you would keep trying to access the card (say due to detection polling of CMD0), there certainly will be some consumption penalty.

Additionally, #4 can also happen early and only once with no consequences. Is this true? Can the resulting FATFS object be shared between mounts (is it reset when re-mounted to a possibly different SD card)?

esp_vfs_fat_register_cfg() is called on every file-system mount attempt, usually only once during application startup. The FATFS object can't be shared or recycled, as it contains runtime information for given FATFS instance on one volume (work buffers, file-system pointers, op flags and many others). Usually, you don't deal directly with such constructs, ESP-IDF provides various code wrappers to let you build required file-system/device-driver stacks transparently. A common use case is to mount the file-system on your target partition/device via the convenience API (all-in-one) and then use standard C file interface functions like fopen() or fwrite() to handle your file operations. Once done, just close all the file pointers and unmount the file-system to avoid possible corruption.

You can use esp_littlefs file system to get better resilience against sudden power offs (card removal in your case), FatFS is not too robust in this regard. To resolve the issue, Espressif provides a file-system journaling component named esp_jrnl available for FatFS on SPI flash partitions, and we plan to port it to SD cards too. Unfortunately, I can't say exactly when it will happen.

Another option to mitigate power-off corruption is to use a raw SD access instead of deploying real file-system (some type of a simple log appender with CRC checks for each sector). This approach obviously requires extra efforts and such SD card would be readable only by your software tools.

I know that 5-7 have to be done with the card inserted.

Yes, all the APIs providing SD card operations require the card to be present (or you get an error). SD host controller APIs are independent on SD card being/not being available

While I am writing data in chunks, can I keep a FILE* open the entire time, or should I close and re-open it every 50 ms? If I don't close it, I will at least fflush() it after each chunk is written. If I keep it open, should I still call fclose() after I know the card is gone?

This depends on your application requirements – you can rely on default flushing implemented in FatFS (when its work buffer gets full), you can also call fsync(fileno()) after each write operation, or enable CONFIG_FATFS_IMMEDIATE_FSYNC Kconfig option to let this happen automatically. fsync() immediatelly transfers data in the FatFS work buffer into physical SD storage (automatically done by fclose()), still it doesn't prevent you from possible FAT corruption if the card is suddenly removed. Furthermore, in order to prevent memory leaks you should always call fclose() to reclaim the file pointer and other resources allocated by fopen().

What happens if I don't call f_mount(NULL, drv, 0); to unmount? Is there anything else that must be done to unmount (e.g. do I need to use esp_vfs_fat_unregister_path() or can I leave it perpetually registered?)? Can/should I unmount after the card is removed? Or do I just mount again when I see another card?

You would probably see huge memory leaks and take a risk of the application crashing. Definitely process all necessary unmount steps, see the code in esp_vfs_fat_sdcard_unmount()

Once I know a card is ejected, can I poll sdmmc_get_status() with the same card object (for a card that isn't there anymore) to see when another card is inserted? Or is there a better way to check for an inserted card (I do not have access to a CD pin)?

CD signal is definitely the only reliable way of detecting SD card plug/unplug. All software means are somewhat unreliable, you need to rely on specific error code sequences and always take the risk of false positive evaluation.

If you have no other options, you probably need your own variant of esp_vfs_fat_sdmmc_mount() which would initialize the host and slot and then do periodical check of sdmmc_send_cmd_go_idle_state() (part of sdmmc_card_init(), use polling period 1-3 sec) where ESP_OK indicates SD card inserted and ESP_ERR_TIMEOUT indicates no card in the slot. After successful detection all continues like in esp_vfs_fat_sdmmc_mount().

To detect SD card removal, try to deploy periodic check of sdmmc_get_status() (in dedicated thread, poll each few seconds). If you start seeing continuous timeouts (5+?), your card has been likely removed. Then you can stop your worker threads and reinitialize the whole SD stack.

NOTE: consequences of using 2 SD slots in parallel are omitted for simplicity reasons

This proposal is meant just as a fallback solution, eg when user incidentally unplugs SD card during the application run. Standard process of the card changing should employ some sort of “disconnect hardware” function (button, console/wifi command) to provide safe SD stack dismounting


Hope this helps, feel free to ask for further details.
Big thanks to adokitkat for providing good 50% of this answer!