You don't give it root on your desktop linux system you do all your sensitive stuff on of course. That makes zero sense. Home assistant really runs great even on a cheap raspberry pi if you don't have a VM- or dockerserver.
also, your faith in "VM" insulation appaears greater than mine. if i dont trust a VM i dont trust the host running the VM.
others have different opinions and that's ok. my systems run to my standards, however quirky they may be. im stating opinion here, not attempting to inscribe Sysadmin Commandments. them's written on the wall of the bathroom stall.
edit: just for reference, the last cpu i could say i trusted was before speculative execution was a feature. since then its more about risk mitigation. i'm not paranoid, there's people worse than me, and they're nuts. I'm just cautious and lazy.
Real virtualization is a bit more airtight, though. There have been some escape exploits but they all abused drivers that you wouldn't use heedless (shared folders, VGA, PCIe passthrough), not the virtualization layer. But that's a distinction without a different, really, so good on you for being careful!
This isn't some binary you downloaded from a Russian forum. VM isolation is more than enough.
You can extend the container image with your own Dockerfile.