I had the same problem too and after reading a ton of posts have decided not to use wifi directly on the RMT host and instead made it receive commands via serial from another wifi enabled esp board.

this did increase the project costs by about 5 dollars but saved me hours and hours of trying ...