Most people end up doing something or other their family disapproves of - it's not some unusual hardship of adult life that outright prevents you from doing things. In his case, being both a doofus and a bit of a Nazi cut off the possibility of future family reconciliation. I suppose there's a line even in royal families.