Page 1 of 1

Building a new classifier for the ESP WHO

Posted: Sun Jul 28, 2019 1:48 pm
by caleb221
Hello!
I have been reading through the open sourced documentation on GitHub as well as the libraries that have been developed.
I understand that the MTMN face detection model is being used on the ESP32 and it is specially designed for facial recognition.

My question would be what were the steps taken in implementing this model in order to make it fit onto the ESP32?

I would like to create another model using mobilenets v2 and MTCNN, but I am unsure as to how to port this onto the ESP.

If anyone could help I would appreciate it a lot!
Thanks!

Re: Building a new classifier for the ESP WHO

Posted: Mon Jul 29, 2019 6:44 am
by ESP_GXC
When you have finished training the model with some popular frameworks like tensorflow, you can then implement the inference process to the ESP related devices. The model structure is easy to implement as we have provided some common operations and you can also rewrite what you need. The model coefficient conversion might be a hard work to do, but still can be done when understanding the format of data. The `dl_matrix3d_t` in the library is channel major layout, a.k.a NHWC. You can write a script to generate a header file that contains the coefficients in `dl_matrix3d_t` structure for GCC compiler.
In the first step, I suggest you to try float point solution to get the right functionality, then try to use fixed point solution to accelerate the process.

Re: Building a new classifier for the ESP WHO

Posted: Mon Jul 29, 2019 8:30 am
by caleb221
Ok Great! Thank you!
Also, i appreciate the quick reply :D
Thanks again!
-Caleb

Re: Building a new classifier for the ESP WHO

Posted: Fri Aug 09, 2019 11:11 am
by geekboood
Hi, I encounter a problem when I try to use the DL lib. Can you take a look at this issue?
https://github.com/espressif/esp-who/issues/106

Re: Building a new classifier for the ESP WHO

Posted: Thu Oct 24, 2019 11:32 am
by riccardo.brue
ESP_GXC wrote:
Mon Jul 29, 2019 6:44 am
When you have finished training the model with some popular frameworks like tensorflow, you can then implement the inference process to the ESP related devices. The model structure is easy to implement as we have provided some common operations and you can also rewrite what you need. The model coefficient conversion might be a hard work to do, but still can be done when understanding the format of data. The `dl_matrix3d_t` in the library is channel major layout, a.k.a NHWC. You can write a script to generate a header file that contains the coefficients in `dl_matrix3d_t` structure for GCC compiler.
In the first step, I suggest you to try float point solution to get the right functionality, then try to use fixed point solution to accelerate the process.
Can you please give more details about how to converte a trained tensorflow model in order to be used on the ESP-32? Thanks.

Re: Building a new classifier for the ESP WHO

Posted: Tue Dec 03, 2019 12:37 pm
by caleb221
If anyone is familiar with this framework, I am currently testing an implementation of MTCNN.
I am at this point with the code and am having a NaN issue.
more details explained here:
https://github.com/espressif/esp-face/issues/17
Thanks!
-Caleb