After failing the installation edit the file: ...\esp-idf\tools\idf_tools.py

In my case, it was the line 1183. Remove the argument '--no-site-packages' of the function.

The function looks like this before editing:
subprocess.check_call([sys.executable, '-m', 'virtualenv', '--no-site-packages ...