The problem is that there are 2 single points of failure in a marriage, and you only control one of them. I can be the best husband I can possibly be, work hard, provide for my family, love my wife with everything I have, and otherwise do what I can to try to make her happy. However, if she choses not to reciprocate this, there's literally nothing I can do about it. (You can change the genders here as appropriate, it's just that I'm a heterosexual male.)
The other problem is that people can change. Sometimes people go out of their way to be the very best version of themselves until they are married and then they just stop trying as hard or stop pretending to be the person the other person wants them to be. Also, people change as they get older, they have different priorities, different interests, new hobbies, changes in physical appearance, changes in libido, etc. After 10 or 20 years of marriage, it is very unlikely that you and your spouse haven't changed in some very significant ways.
Yo can look at it as a downside or as an upside.