Edit: when I say anything, I'm not talking user programs. I mean as in, before even the first instruction of the firmware -- before even the virtual disk file is zeroed out, in cases where it needs to be. You literally can't pause the VM during this interval because the window hasn't even popped up yet, and even when it has, you still can't for a while because it literally hasn't started running anything. So the kernel and even firmware initialization slowness are entirely irrelevant to my question.
Why is that?
If I let a VM use most of my hardware, it takes a few seconds from start to login prompt, which is the same time it takes for my Arch desktop to boot from pressing the button to seeing the login prompt.
You also need to start OS services, configure filesystems, prepare caches, configure networking, and so on. If you're not booting UKIs or similar tools, you'll also be loading a bootloader, then loading an initramfs into memory, then loading the main OS and starting the services you actually need, with eachsstep requiring certain daemons and hardware probes to work correctly.
There are tools to fix this problem. Amazon's Firecracker can start a Linux VM in a time similar to that of a container (milliseconds) by basically storing the initialized state of the VM and loading that into memory instead of actually performing a real boot. https://firecracker-microvm.github.io/
On Windows, I think it depends on the hypervisor you use. Hyper V has a pretty slow UEFI environment, its hard disk access always seems rather slow to me, and most Linux distro don't seem to package dedicated minimal kernels for it.
That's not what I'm asking.
I'm saying it takes a long time for it to even execute a single instruction, in the BIOS itself. Even for the window to pop up, before you can even pause the VM (because it hasn't even started yet). What you're describing comes after all that, which I already understand and am not asking about.
I'm saying it takes a long time for it to even execute a single instruction, in the BIOS itself. Even for the window to pop up, before you can even pause the VM (because it hasn't even started yet). What you're describing comes after all that, which I already understand and am not asking about.
I think task manager would tell you if there is a blip of memory usage and paging activity at the time. And I'm sure windows itself has profilers that can tell you what is happening when the VM is started..
On windows it was almost 10x faster. On the project where this change was released, my morning ritual was to come in, log on, run an svn pull command, lock my screen and go get coffee. I had at least ten minutes to kill after I got coffee, if the pot wasn’t empty when I got there.
Windows is hot garbage about fopen particularly when virus scanning is on.
I'm using Hyper-V and I can connect through XRDP to a GUI Ubuntu 22 in 10 seconds and I can SSH into a Ubuntu 22 server in 3 seconds after start.
In practice virtual machines are trying to emulate a lot of stuff that isn’t really needed but they’re doing it for compatibility.
If one builds a hypervisor which is optimized for startup speed and doesn’t need to support generalized legacy software then you can:
> Unlike traditional VMs that might take several seconds to start, Firecracker VMs can boot up in as little as 125ms.
For example: your VM starts up with the CPU in 16 bit mode because that’s just how things work in x86 and then it waits for the guest OS to set the CPU into 64 bit mode.
This is completely unnecessary if you just want to run x86-64 code in a virtualized environment and you control the guest kernel and can just assume things are in 64bit mode because it’s not the 70s or whatever
The guest OS would also need to probe few ports to get a bootable disk. If you control the kernel then you can just not do that and boot directly.
There’s a ton of stuff that isn’t needed
A virtual environment doesn’t even really need any BIOS or anything like that.
You can feel free to test with qemu direct kernel booting to see this skips a lot of delay without even having to use a specialized hypervisor like firecracker
Again, it was a few years ago, but we didn't solve the problem or identify an actual root cause. We stopped banging our heads against that particular wall and switched technologies.
Windows probably has an equivalent.
So this meant VMWare, VirtualBox, etc as they were would no longer work on Windows. Microsoft required all of them to switch to using Hyper-V libs behind the scenes to launch Hyper-V VMs and then present them as their own (while hiding them from the Hyper-V UI).
VirtualBox was slow, hot garbage on its own before this happened, but now it's even worse. They didn't optimize their Hyper-V integration as well as VMWare (eventually) did. VMWare is still worse off than it was though since it has to inherit all of Hyper-V's problems behind the scenes.
Hope this brings some clarity.
I’ve noticed that windows can only evict data from the page cache at about 5 GB/s. I do not know if this zeros the memory or that would need to be done in the allocation path.
A couple years ago I tracked down a long pause while starting qemu on Linux to it zeroing the 100s of GB of RAM given to the VM as 1 GB huge pages.
These may or may not be big contributors to what you are seeing, depending on the VM’s RAM size.
On the other hand, it could just as easily be something simple, like setting up hugepages or checksumming virtual hard disk image files.
Both are total guesses, though. Could be anything!
Management Engine .. actually I do not have the energy to deal with paranoid people. I never had that kind of energy. I never will. You’re all so efficient at drawing energy out of conversations and killing them. You’re like conversational vampires. It’s exhausting.
I don’t even care if you’re right or wrong about Intel ME. It is just so exhausting listening to you guys because of your word choices. It’s like you try to get ignored.
I respect your opinion and all, you just need to work on your messaging or something.