1) Uptime (though this could be partially alleviated by retries)
and most of all:
2) "Trust"/"Spam score"
It's the main reason to use Sendgrid, AWS, Google, etc. Their "value" is not the email service, it's that their SMTP servers are trusted.
If tomorrow I can just send from localhost instead of going through Google it's fine for me, but in reality, my emails won't arrive due to these filters.
But yes, the “trust / spam score” is a legit challenge. If only device manufacturers were held liable for security flaws, but we sadly don’t live in that timeline.