Changed the library to "ESP32-HUB75-MatrixPanel-I2S-DMA"
And it works well with wifi and matrix.

After commenting the line in RGBmatrixPanel.h , the reboots stopped, but of course the matrix does not show anything
//timer_start(TIMER_GROUP_1, TIMER_0);

#if defined(ARDUINO_ARCH_ESP32)
timer_config_t tim_config;
tim_config.divider = 2; // Run Timer at 40 MHz
tim_config.counter_dir = TIMER ...

Hello.
Faced a strange problem with ESP32-WROOM-32U.
If I initialize the RGBMatrix panel (32*64, NOT connected to ESP32board, only initialization), then initialize the STA wifi, the ESP immediately gives the error
Guru Meditation Error: Core 1 panic'ed (Cache disabled but cached memory region ...