Slightly off-topic, but I've found that dialing into Zoom calls via phone sometimes alleviates the latency, at least for audio. I'd rather have bad A/V sync with less audio latency than perfect sync but more latency.
Just a tip that's helped me in similar situations!