ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

samsam
Posts: 50
Joined: Sun Jun 04, 2017 5:26 pm

ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby samsam » Thu Jul 13, 2017 4:38 pm

From the ringbuf.h:
.... when the only way to place the next item would involve a wraparound from the end to the beginning of the ringbuffer. ...

- type = RINGBUF_TYPE_ALLOWSPLIT: The insertion code will split the item in two items; one which fits
in the space left at the end of the ringbuffer, one that contains the remaining data which is placed
in the beginning. Two xRingbufferReceive calls will be needed to retrieve the data.

How could detect if the item received is split or is not, and how will second xRingbufferReceive look like in such situation ?
Any example?
Thanks

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby kolban » Fri Jul 14, 2017 4:09 am

I'll take a guess ...

Because the programmer explicitly stated that RINGBUF_TYPE_ALLOWSPLIT was permitted then that is a tacit expression that it is ok to split the data into two "unrelated" chunks. So a call to xRingbufferReceive() will receive one chunk and a subsequent call to xRingbufferReceive() will return the second chunk. Nothing in the Ringbuffer architecture will ever indicate that they were created because of an automated split. If this is not acceptable, then specify RINGBUF_TYPE_NOSPLIT and you won't ever face the situation that there were two separate chunks created ... because in this case, there won't be.
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby ESP_Sprite » Fri Jul 14, 2017 1:52 pm

Kolban is absolutely right. However, I can imagine it's useful to detect a split and return that info to the retrieving end... If you want this, feel free to open up a feature request on Github and we'll see if we can implement this.

samsam
Posts: 50
Joined: Sun Jun 04, 2017 5:26 pm

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby samsam » Fri Jul 14, 2017 4:27 pm

Thanks for all that answers so far!
Some more weird story about the ring buffer:
if i create xRingbufferCreate(256, RINGBUF_TYPE_NOSPLIT); then xRingbufferGetMaxItemSize(..) returns 116,
but if i create xRingbufferCreate(356, RINGBUF_TYPE_NOSPLIT); then xRingbufferGetMaxItemSize(..) returns 166

So 100 bytes increase the buffer size give me only half gain on the max item size possible to enter?

And the second weird thing - doesnt matter when I call xRingbufferGetMaxItemSize(): as soon as I create the buffer, or when put 1,2,3... items, or remove (with vRingbufferReturnItem) 1,2,...all from the buffer - the result of xRingbufferGetMaxItemSize is always the same? Of course it might be no straight correlation between the Max item size available and available space in the ring buffer, but if cant use xRingbufferGetMaxItemSize to get available space in the ring buffer - is there any other way?

And btw the only info so far on the ring buffer I found in the Kolban's book (thanks Neil) - any other source ? If I rely only on what I see in ringbuf.h, I would never imagine that vRingbufferReturnItem would release the memory occupied in the ring buffer for the item given in the parameter :(

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby ESP_Sprite » Sat Jul 15, 2017 3:29 am

Good point - we may have overlooked it when documenting esp-idf because it was 'snuck in' in the FreeRTOS sources and these are mostly documented elsewhere - whereas the ring buffer isn't because we developed it ourselves. I'll add a ToDo for that.

Wrt the GetMaxItemSize: it's supposed to be the maximum item size that is guaranteed to be stored in the empty ringbuffer. Because the write pointer may be somewhere around the middle of the buffer when you try to store it, it's only half of the ringbuffer size when splits are not allowed. We could have made it twice as big with some logic that entirely resets the ring buffer when empty, but we did not need that at the time and this is a simpler solution.

samsam
Posts: 50
Joined: Sun Jun 04, 2017 5:26 pm

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby samsam » Sat Jul 15, 2017 5:42 pm

Ok, I left "Feature request" plus some proposal how could be done in Github. Not quite sure I've made it the right way as is my first try there ;)

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby kolban » Sun Nov 19, 2017 12:27 am

I'm reading the docs for the ringbuffer here:

https://github.com/espressif/esp-idf/bl ... /ringbuf.h

When splitting is not allowed, it states:

"... the maximum size is (buffer_size/2)-8 bytes."

Does this mean the value returned by "GetMaxItemSize()" or the "actual" size that can be inserted?

Imagine I have an empty buffer of size 1000 bytes and want to write 900 bytes .. will it be allowed?
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby ESP_Sprite » Sun Nov 19, 2017 10:23 am

The correct answer would be 'it depends'. The ring buffer, as ring buffers tend to do, has a read and a write pointer. If the write pointer is e.g. at 500 bytes from the start of the buffer, there's no way to write 900 bytes without a split. (A 'split' essentially is what happens when the ringbuffer wraps around; because the ringbuffer returns a pointer to its own memory, there's no way to return an item that 'wraps around' in a nice way.) If the write pointer is somewhere within 100 bytes of the start of the ring buffer, it's obviously no issue to write 900 bytes without splitting.

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby kolban » Sun Nov 19, 2017 3:29 pm

I'm imagining a scenario where I have a 1000 byte ring buffer. I write 500 bytes and then consume those 500 bytes. Now there is no data in the ring buffer.

Q: In an empty buffer, are the read/write pointers reset to the start of the buffer?

Now, after having written and then read 500 bytes .. and assuming that the read/write pointers are now pointing to offset 500, if I try and write 900 bytes ... there is obviously contiguous space in which to write it without splitting. In my mind, it would be to place the 900 bytes starting at offset 0 with the read pointer being at offset 0 and the write pointer being at offset 900.

Q: In this story, would the insertion of the data work? (assuming splitting is not desired).
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: ring buffer RINGBUF_TYPE_ALLOWSPLIT check for splitted item?

Postby ESP_Sprite » Sun Nov 19, 2017 9:22 pm

Theoretically, that could work... but at the moment, such pointer-resetting-when-empty is not implemented at the moment.

Who is online

Users browsing this forum: Majestic-12 [Bot] and 112 guests