I have encountered the same issue and was able to resolve it.
Since device once enumerated will hold it's address one must keep the usb host lib installed at all times and also remove the call to usb_host_device_free_all from the usb lib task (handle_usb_events in this case). This is because after ...