USB Bulk reads at 60ksps

[Aussie Susan]

I'm using ESP-IDF V5.5.2 on a Seeed Studio Xiao ESP32S3 running at 160MHz with 8MB of octal PSRAM running at 40MHz.
What I am trying to do is to read 60ksps from a USB device using the USB Host software.
I have set up a freeRTOS task for the USB Host software and within that I set up 4 consecutive bulk transfers (each ultimately calling usb_host_transfer_submit()) with a callback to handle the results.
The way the USB device is set up with this arrangement I get 131,136 bytes from each transfer which means the callback is being called about every 2ms.
Also within the callback is another call to usb_host_transfer_submit() so the series of transfers should keep going.
All is (seemingly) working well as long as I do nothing in the callback (other than restart the next transfer).
But what I want to do is to trigger another task to process the bytes just received (and ultimately send them via a WiFI UDP transfer but I'm not there yet). I've tried using a freeRTOS queue or a semaphore but the 10ms task switching means that I always need to overwrite the USB buffer before the other task has even started.
I've therefore tried using task notifications but with the same (apparent) issue.
BTW, the USB task is pinned to one core and the 'processing' task is pinned to the other so they can work in parallel.
I've also noticed that, even if I create all the USB transfers and THEN submit them, the callback occurs for the first few before the later transfers are even submitted for the first time - and this really messes things up unless I have only one or 2 transfers.

Therefore my questions are:
1) Am I taking the wrong approach? - If so then what is the 'correct' way to have an app read 60ksps from a USB device and have the data processed
2) If my approach is correct, then what is 'correct' way of handling the issues I've outlined above

Susan

[MicroController]

I've tried using a freeRTOS queue or a semaphore

Should be the way to go...

but the 10ms task switching means that I always need to overwrite the USB buffer before the other task has even started.

Not sure what you mean by "the 10ms task switching". But yes, you generally need at least two buffers when you want to pass one to another task for (asynchronous) processing while already filling up the next one.

the USB task is pinned to one core and the 'processing' task is pinned to the other so they can work in parallel.

They will also run concurrently on any available cores when not pinned. I generally advise against restricting tasks to run only on a specific core, except when accessing core-specific functionality is required (interrupt allocation, "dedicated IO"), or to prevent implicit pinning to a 'random' core when using the FPU.

[nopnop2002]

BTW, the USB task is pinned to one core and the 'processing' task is pinned to the other so they can work in parallel.

We often see cases where people mistakenly believe that all system background tasks run on core 0, and therefore launch application tasks fixed on core 1.

As shown below, the operating core for several system background tasks is unpredictable (not pinned), so there is no point in launching application tasks fixed on core 1.

If you start a task with pinned to core 1, the task will be put on hold when core 0 is free and core 1 is busy.

Code: Select all

task         status   prio                     core
--------------------------------------------------- 
IDLE1           R       0       1036    6       1
IDLE0           R       0       912     5       0
tiT             B       18      2208    8       -1
Tmr Svc         B       1       1544    7       -1
ipc1            S       24      500     2       1
esp_timer       S       22      3240    3       0
wifi            B       23      3552    10      0
sys_evt         B       20      744     9       0
ipc0            S       24      508     1       0

If tasks have the same priority, each task will run equally in RoundRobin.
When a task is launched using xTaskCreate(), it determines which core to run it on according to the rules of FreeRTOS on the ESP32.
If another core becomes available during task execution, the core assignment is dynamically changed.
This indicates, for example, that a task started on core #1 might move to core #0 midway through its operation.
Pinned tasks will not have their cores moved while they are being processed.

[Aussie Susan]

Thank you both for your advice.
I'm 'guilty as charged' with regard to thinking that it is better to pin the USB and 'processing' tasks to different cores - I didn't realise that freeRTOS is smart enough to swap things around even in mid operation.
As for the "10ms task switching" comment, it is based on 1) the 100Hz 'tick' that freeRTOS has by default (and I've not changed that) and 2) the following timestamps:

I (4711) USB_Host: Starting the transfer: i: 0, t = 0x3c0c0d48
I (4711) USB_Host: Transfer 0 setup
I (4731) USB_Host: Buffer 0x3c100dbc: 0xaa, 0xaa... 0xaa
I (4731) USB_Host: Starting the transfer: i: 1, t = 0x3c100dbc
I (4731) USB_Host: Transfer 1 setup
I (4751) USB_Host: Buffer 0x3c140e1c: 0xaa, 0xaa... 0xaa
I (4751) USB_Host: Starting the transfer: i: 2, t = 0x3c140e1c
I (4751) USB_Host: Transfer 2 setup
I (4761) USB_Host: Buffer 0x3c180e7c: 0xaa, 0xaa... 0xaa
I (4761) USB_Host: Starting the transfer: i: 3, t = 0x3c180e7c
I (4761) USB_Host: Transfer 3 setup
I (4771) USB_Host: ---Starting transfers
I (4771) USB_Host: ---Transfers Started
I (4771) USB_Host: 0
I (4771) tle_control: Rx: 0
I (4771) USB_Host: 1
I (4781) tle_control: Rx: 1
I (4781) USB_Host: 2
I (4781) tle_control: Rx: 2
I (4781) USB_Host: 3
I (4791) tle_control: Rx: 3
I (4791) USB_Host: 0
I (4791) tle_control: Rx: 0
I (4791) USB_Host: 1
I (4801) tle_control: Rx: 1
I (4801) USB_Host: 2
I (4801) tle_control: Rx: 2
I (4801) USB_Host: 3

where these are all 'ESP_LOGI() calls and 'USB_Host" is the USB task and "tle_control' is the task that is receiving the 'task notifications sent by the USB task.
As you can see, the timestamps are all 10ms apart. (The numbers refer to the 4 bulk transfers that are created and the ones with the simple numbers are from the USB callback.) In the first case, the USB callback for both transfers 0 and 1 occur at the time time as does the processing of transfer 0. However the reception of the task notification for transfer 1 occurs 10ms later along with that for transfer 2. (This 10ms timestamp delta continues as long as the USB transfer continue.)
Hence my original concern that I'd really like to start processing each buffer as soon as possible after it has been made available and not wait for the next task switch.
Susan

[MicroController]

Code: Select all

...
I (4791) tle_control: Rx: 3
I (4791) USB_Host: 0
I (4791) tle_control: Rx: 0
I (4791) USB_Host: 1
...

...
As you can see, the timestamps are all 10ms apart.

Not really . And when they are, that's just an artifact of the logging routines using FreeRTOS's ticks for the timestamp, not a 'real' timing indication.

Cf.:

Code: Select all

unsigned i = 0;
const TickType_t start = xTaskGetTickCount();
while((xTaskGetTickCount() - start) <= 2) {
  i += 1;
  ESP_LOGI(TAG, "%u", i);
  esp_rom_delay_us(2500); // 2.5 ms
}

-->

Code: Select all

I (1502) main: 1
I (1502) main: 2
I (1502) main: 3
I (1502) main: 4
I (1512) main: 5
I (1512) main: 6
I (1512) main: 7
I (1512) main: 8
I (1522) main: 9
I (1522) main: 10
I (1522) main: 11
I (1522) main: 12

[Aussie Susan]

Thanks again - learning something new every day as I was not aware of the esp_rom_delay_us() function; also I thought that the timestamps reflected when the operation occurred and not just when it was printed.
Therefore I'm going to not worry about the timestamps.
I'll stop this thread here and see how far I get with the next steps.
Thanks again. everyone.
Susan