Or you could do some kind of middle-ground like what Chrome is planning, where the user can only be prompted to enable notifications after they've reached some specific level of engagement with the site they're using:
https://github.com/WICG/interventions/issues/49#issuecomment...