Keybase has end-to-end encryption and you can connect to people without publishing the fact. But unfortunately, it was recently purchased by Zoom and probably won't be headed in a good direction in the future.
Of course, Keybase's main idea is to have a verifiable public identity which might not be what you want.