Building a new classifier for the ESP WHO

caleb221
Posts: 8
Joined: Sun Jul 28, 2019 12:38 pm
Contact:

Building a new classifier for the ESP WHO

Postby caleb221 » Sun Jul 28, 2019 1:48 pm

Hello!
I have been reading through the open sourced documentation on GitHub as well as the libraries that have been developed.
I understand that the MTMN face detection model is being used on the ESP32 and it is specially designed for facial recognition.

My question would be what were the steps taken in implementing this model in order to make it fit onto the ESP32?

I would like to create another model using mobilenets v2 and MTCNN, but I am unsure as to how to port this onto the ESP.

If anyone could help I would appreciate it a lot!
Thanks!

ESP_GXC
Posts: 17
Joined: Wed Nov 21, 2018 6:06 am

Re: Building a new classifier for the ESP WHO

Postby ESP_GXC » Mon Jul 29, 2019 6:44 am

When you have finished training the model with some popular frameworks like tensorflow, you can then implement the inference process to the ESP related devices. The model structure is easy to implement as we have provided some common operations and you can also rewrite what you need. The model coefficient conversion might be a hard work to do, but still can be done when understanding the format of data. The `dl_matrix3d_t` in the library is channel major layout, a.k.a NHWC. You can write a script to generate a header file that contains the coefficients in `dl_matrix3d_t` structure for GCC compiler.
In the first step, I suggest you to try float point solution to get the right functionality, then try to use fixed point solution to accelerate the process.

caleb221
Posts: 8
Joined: Sun Jul 28, 2019 12:38 pm
Contact:

Re: Building a new classifier for the ESP WHO

Postby caleb221 » Mon Jul 29, 2019 8:30 am

Ok Great! Thank you!
Also, i appreciate the quick reply :D
Thanks again!
-Caleb

geekboood
Posts: 2
Joined: Fri Aug 09, 2019 11:09 am

Re: Building a new classifier for the ESP WHO

Postby geekboood » Fri Aug 09, 2019 11:11 am

Hi, I encounter a problem when I try to use the DL lib. Can you take a look at this issue?
https://github.com/espressif/esp-who/issues/106

riccardo.brue
Posts: 10
Joined: Fri Jun 21, 2019 1:49 pm

Re: Building a new classifier for the ESP WHO

Postby riccardo.brue » Thu Oct 24, 2019 11:32 am

ESP_GXC wrote:
Mon Jul 29, 2019 6:44 am
When you have finished training the model with some popular frameworks like tensorflow, you can then implement the inference process to the ESP related devices. The model structure is easy to implement as we have provided some common operations and you can also rewrite what you need. The model coefficient conversion might be a hard work to do, but still can be done when understanding the format of data. The `dl_matrix3d_t` in the library is channel major layout, a.k.a NHWC. You can write a script to generate a header file that contains the coefficients in `dl_matrix3d_t` structure for GCC compiler.
In the first step, I suggest you to try float point solution to get the right functionality, then try to use fixed point solution to accelerate the process.
Can you please give more details about how to converte a trained tensorflow model in order to be used on the ESP-32? Thanks.

caleb221
Posts: 8
Joined: Sun Jul 28, 2019 12:38 pm
Contact:

Re: Building a new classifier for the ESP WHO

Postby caleb221 » Tue Dec 03, 2019 12:37 pm

If anyone is familiar with this framework, I am currently testing an implementation of MTCNN.
I am at this point with the code and am having a NaN issue.
more details explained here:
https://github.com/espressif/esp-face/issues/17
Thanks!
-Caleb

Who is online

Users browsing this forum: No registered users and 11 guests