Seconding kugelblitz 's answer. Adding about 1mF of capacitance between the ESP32 power rail and ground significantly reduced the noise in my circuit.

I'd also recommend using surface-mount capacitors rather than through-hole ones, as you want your bypass caps to be as close to the ESP32 power ...