Self hosting sounds so simple, but if you consider all the critical factors involved, in becomes a full time job. You own your server. In every regard.
And security is only one crucial aspect. How spam filters react to your IP is another story.
In the end I cherrish the dream but rely on third party server providers.