Sam’s statement partially contradicts this by saying they contracted the voice actor before ScarJo - but I believe there’s enough intent shown in ScarJos original tweet that Sam as a default disregarded the entire interaction as an inconvenience where he could be “naughty” and get away without consequences.
I'm sure lots of movies have tried to cast Scarlett Johanson for a role and she declined, so they went with another actress who looked similar. Should she sue them too?
If the job is being the first person you reached out to, then you accept that you can't get the job done...
Legally. He took the other option.