Well you hit on all the points that discuss the compromises that zip codes offer. Just because you have them in your data doesn't mean that they can produce anything useful. You are correct that no one knows their census unit is (if you are thinking from someone entering this on a website) but collecting location or address will be a lot better.
Fact is a lot of web data contains a zip but if you can collect something better it will usually render better results. Unless you are analyzing shipments then that is fine.