They probably mean leaving ssh open to all ips. Take a look at your auth failure logs to see the thousands of daily attempts to compromise your server using default passwords. Most of those are low effort and low risk. Sometimes the bots will try password stuffing. Disabling password auth in sshd config is good practice.
Fail2ban also helps block repeated attempts like that.
There’s also the risk of a zero day RCE vulnerability in ssh (though I’ve not seen one in the 20 years I’ve been paying attention )
I tend to not expose ssh to the world and log in with some other method to pass the perimeter (VPN, IP whitelist, tailscale) and the ssh from inside.