European colonial power would have never left if they hadn't got into war of attrition with Hitlar.
British left their biggest colony India only when Indian soldiers revolted and they were too weak to crush it post WW2. It was simply not possible to rule after this incident.