I've been looking for something I could host for this kind of thing - for LLM agents. Ended up on
https://www.daytona.io/ as I couldn't find anything suitable to self host and realised it was a complex thing to manage. It seems Daytona is open source, including the server platform, but there is no documentation for the server element. Azure also seem to offer a service for this, it's a space that is growing rapidly.