something simpler I've did, in the same spirit: LXC containers (using Incus) in a VM. LXC containers look and feel like VMs, but are very lightweight. And the VM they all run in provide the hard sandbox.
and when I spin up a new LXC container cloud-init sets it up with the agents and my repos inside