The concern isn't that the website will know the IP, it's that every single entity on the network between Cloudflare and the authoritative DNS server (most or all of which will not be operated by the website) will know it.
It still may not be the right decision, but it's important to frame the trade-off correctly.