Search found 3 matches
- Fri Dec 22, 2023 7:05 am
- Forum: ESP-IDF
- Topic: ESP-IDF precision of double limited to 7 decimal places?
- Replies: 6
- Views: 4707
Re: ESP-IDF precision of double limited to 7 decimal places?
I think the problem is not the decimal places but the total digits involved (integer part plus decimal part) that seams to be limited to 11 or 12 in total. I do not know about IDF but using Arduino IDE I had the following results:
float QI=17.0/3.0;
float RI=999999.999999/3.111;
void setup ...
- Fri Dec 22, 2023 7:04 am
- Forum: ESP-IDF
- Topic: ESP-IDF precision of double limited to 7 decimal places?
- Replies: 6
- Views: 4707
Re: ESP-IDF precision of double limited to 7 decimal places?
Check what the actual value of your double variable is. The base-10 string output may be truncated.
Any tips on how to do this? I am already printing long float, and I also sent this value in a http REST response to a debugging portal I built and it has the same result of truncated decimal ...
- Thu Dec 21, 2023 7:20 am
- Forum: ESP-IDF
- Topic: ESP-IDF precision of double limited to 7 decimal places?
- Replies: 6
- Views: 4707
ESP-IDF precision of double limited to 7 decimal places?
Dear all,
I am working on a project that involves handling latitude and longitude information, and require a precision of at least 8 decimal places.
It seems like doubles are strictly limited to 7 decimal places in ESP-IDF? Is there anyway to bypass this?
In the following code,
1: we take in a ...
I am working on a project that involves handling latitude and longitude information, and require a precision of at least 8 decimal places.
It seems like doubles are strictly limited to 7 decimal places in ESP-IDF? Is there anyway to bypass this?
In the following code,
1: we take in a ...