This is similar to real life: if you vouch for someone (in business for example), and they scam them, your own reputation suffers. So vouching carries risk. Similarly, if you going around someone is unreliable, but people find out they actually aren't, your reputation also suffers. If vouching or denouncing become free, it will become too easy to weaponize.
Then again, if this is the case, why would you risk your own reputation to vouch for anyone anyway.
The same as when you vouch for your company to hire someone - because you will benefit from their help.
I think your suggestion is a good one.