Search found 3 matches

by xwbxwb
Fri Dec 22, 2023 7:05 am
Forum: ESP-IDF
Topic: ESP-IDF precision of double limited to 7 decimal places?
Replies: 6
Views: 4707

Re: ESP-IDF precision of double limited to 7 decimal places?


I think the problem is not the decimal places but the total digits involved (integer part plus decimal part) that seams to be limited to 11 or 12 in total. I do not know about IDF but using Arduino IDE I had the following results:


float QI=17.0/3.0;
float RI=999999.999999/3.111;

void setup ...
by xwbxwb
Fri Dec 22, 2023 7:04 am
Forum: ESP-IDF
Topic: ESP-IDF precision of double limited to 7 decimal places?
Replies: 6
Views: 4707

Re: ESP-IDF precision of double limited to 7 decimal places?


Check what the actual value of your double variable is. The base-10 string output may be truncated.


Any tips on how to do this? I am already printing long float, and I also sent this value in a http REST response to a debugging portal I built and it has the same result of truncated decimal ...
by xwbxwb
Thu Dec 21, 2023 7:20 am
Forum: ESP-IDF
Topic: ESP-IDF precision of double limited to 7 decimal places?
Replies: 6
Views: 4707

ESP-IDF precision of double limited to 7 decimal places?

Dear all,

I am working on a project that involves handling latitude and longitude information, and require a precision of at least 8 decimal places.
It seems like doubles are strictly limited to 7 decimal places in ESP-IDF? Is there anyway to bypass this?
In the following code,
1: we take in a ...

Go to advanced search