Then I wrote a small tool[1] to streamline my sandboxing.
Now, I run agents inside it for keeping my non-working-directory files safe.
For some tools like markdown linter, I run them without network access as well.
1- https://github.com/ashishb/amazing-sandbox