zlacker

>>hugodu+(OP)
At CodeSandbox we use Firecracker for hosting development environments, and I agree with the points. Though I don't think that means you should not use Firecracker for running long-lived workloads.

We reclaim memory with a memory balloon device, for the disk trimming we discard (& compress) the disk, and for i/o speed we use io_uring (which we only use for scratch disks, the project disks are network disks).

It's a tradeoff. It's more work and does require custom implementations. For us that made sense, because in return we get a lightweight VMM that we can more easily extend with functionality like memory snapshotting and live VM cloning [1][2].

[1]: https://codesandbox.io/blog/how-we-clone-a-running-vm-in-2-s...

[2]: https://codesandbox.io/blog/cloning-microvms-using-userfault...

>>CompuI+IL
I don't know if this is relevant, but I've been intrigued by DragonflyBSD's "vkernel" [0] feature which (supposedly) allows for cloning the entire runtime state of the machine (established TCP connections, etc.) into a completely new userland memory space. I think they use it mostly for kernel debugging right now, but it's interesting to think about the possibilities of being able to just clone an entire running operating system to a new computer without interrupting even a single instruction.

[0] https://www.dragonflybsd.org/docs/handbook/vkernel/