zlacker

[parent] [thread] 17 comments
1. arun-m+(OP)[view] [source] 2023-07-10 15:18:42
I think you could help me answer the question that has been in my mind for a month :)

Is there any article that tells the difference and relationship between KVM, QEMU, libvirt, virt-manager, Xen, Proxmox etc. with their typical use cases?

replies(3): >>reacto+w1 >>Izmaki+g6 >>theoss+y7
2. reacto+w1[view] [source] 2023-07-10 15:25:19
>>arun-m+(OP)
KVM is kernel-based virtual machine, with libvirt being its API abstraction over all of it. QEMU is a virtual machine host that leverages kvm or software virtualization to spin up machines on the host. virt-manager does the same. Xen is another virtual machine host, like KVM. Proxmox is a virtual machine manager (like QEMU, virt-manager) but is web based. Libvirt will provide abstraction for kvm,qemu,xen

Use cases: proxmox web interface exposed on your local network on a KVM Linux box that uses QEMU to manage VM’s. Proxmox will allow you to do that from the web. QEMU is great for single or small fleet of machines but should be automated for any heavy lifting. Proxmox will do that.

replies(1): >>bonzin+U8
3. Izmaki+g6[view] [source] 2023-07-10 15:45:42
>>arun-m+(OP)
I don't know if _one_ such article exists, but here is a piece of tech doc from oVirt (yet another tool) that shows how - or that - VDSM is used by oVirt to communicate with QEMU through libvirt: https://www.ovirt.org/develop/architecture/architecture.html...

In really simple terms, so simple that I'm not 100% sure they are correct:

* KVM is a hypervisor, or rather it lets you turn linux into a hypervisor [1], which will let you run VMs on your machine. I've heard KVM is rather hard to work with (steep learning curve). (Xen is also a hypervisor.)

* QEMU is a wrapper-of-a-sorts (a "machine emulator and virtualizer" [2]) which can be used on top of KVM (or Xen). "When used as a virtualizer, QEMU achieves near native performance by executing the guest code directly on the host CPU. QEMU supports virtualization when executing under the Xen hypervisor or using the KVM kernel module in Linux." [2]

* libvirt "is a toolkit to manage virtualization platforms" [3] and is used, e.g., by VDSM to communicate with QEMU.

* virt-manager is "a desktop user interface for managing virtual machines through libvirt" [4]. The screenshots on the project page should give an idea of what its typical use-case is - think VirtualBox and similar solutions.

* Proxmox is the above toolstack (-ish) but as one product.

---

[1] https://www.redhat.com/en/topics/virtualization/what-is-KVM

[2] https://wiki.qemu.org/Main_Page

[3] https://libvirt.org/

[4] https://virt-manager.org/

replies(1): >>Izmaki+E7
4. theoss+y7[view] [source] 2023-07-10 15:51:38
>>arun-m+(OP)
KVM is a Linux kernel implementation of the cpu extensions to accelerate vms to near bare metal speeds.

Qemu is a user space system emulator. It can emulate in software different architectures like ARM, x86, etc. It can also emulate drivers, networking, disks, etc. Is called via the command line.

The reason you'll see Qemu/KVM a lot is because Qemu is the emulator, the things actually running the VM. And it utilizes KVM (on linux, OSX has HVF, for example) to accelerate the VM when the host architecture matches the VM's.

Libvirt is an XML based API on top of Qemu (and others). It allows you to define networks, VMs (it calls them domains), and much more with a unified XML schema through libvirtd.

Virsh is a CLI tool to manage libvirtd. Virt-manager is a GUI to do the same.

Proxmox is Debian under the hood with Qemu/KVM running VMs. It provides a robust web UI and easy clustering capabilities. Along with nice to haves like easy management of disks, ceph, etc. You can also manage Ceph through an API with Terraform.

Xen is an alternative hypervisor (like esxi). Instead of running on top of Linux, Xen has it's own microkernel. This means less flexibility (there's no Linux body running things), but also simpler to manage and less attack surface. I haven't played much with xen though, KVM is kind of the defacto, but IIRC AWS used to use a modified Xen before KVM came along and ate Xen's lunch.

replies(3): >>adql+Vd >>monoca+Yv >>cthalu+OF
◧◩
5. Izmaki+E7[view] [source] [discussion] 2023-07-10 15:52:04
>>Izmaki+g6
Ps: typically if you want to run VMs you are faced with two paths only: 1) you want one or more VMs on your personal computer or 2) you want one or more VMs in an office environment.

On the first path you are likely going to be just fine with VirtualBox, VMWare Workstation or Hyper-V (Windows only) / Parallels (Mac intended). Which one you should pick depends on your desired use of the machines.

On the second path you would go with a solution that deals with the nitty-gritty details, such as Proxmox, oVirt, Hyper-V, ESXi, or any of the other many available options - granted you are not going full cloud-based, which opens up a whole lot of different options too.

You would generally never need to worry about which components are needed where and why. I've had to worry about it once or twice before, because I've had to debug why an oVirt solution was not behaving like I wanted it to behave. Knowing the inner workings helps in that case.

replies(1): >>crabbo+mW
◧◩
6. bonzin+U8[view] [source] [discussion] 2023-07-10 15:58:12
>>reacto+w1
This is almost entirely wrong especially as far as QEMU, Libvirt and virt-manager are concerned.

QEMU is a low level process that represents the virtual machine. It has no equivalent in Xen. Using QEMU directly is not a good idea unless your needs for VM configurations change all the time and you hardly reuse VMs.

Libvirt is at a higher level than QEMU. It manages the QEMU processes and gives them access to system resources (image files, network interfaces, pass-through PCI devices). It also makes it easy to manage the configuration of your virtual machines and the resources they use.

Higher still is virt-manager, which is a GUI interface for libvirt. Proxmox sits at roughly the same level as virt-manager.

replies(1): >>gabere+qc
◧◩◪
7. gabere+qc[view] [source] [discussion] 2023-07-10 16:14:43
>>bonzin+U8
How? KVM and Xen are kernel level. QEMU uses KVM but also has a software virtualization capability. Libvirt is an API abstraction over it all. virt-manager is a gui app to manage libvirt machines. Proxmox as well. Proxmox VE talks to VMHost via libvirt.
replies(1): >>bonzin+rd
◧◩◪◨
8. bonzin+rd[view] [source] [discussion] 2023-07-10 16:18:48
>>gabere+qc
Libvirt does not use KVM. Libvirt uses either QEMU (which in turn might or might not use KVM) or Xen or other hypervisors. So it's incorrect to say that Libvirt abstracts over KVM.

And virt-manager indeed manages Libvirt machines so it's not at the level of QEMU as you wrote in the parent comment:

> Proxmox is a virtual machine manager (like QEMU, virt-manager)

replies(1): >>gabere+Qi
◧◩
9. adql+Vd[view] [source] [discussion] 2023-07-10 16:20:26
>>theoss+y7
>Xen is an alternative hypervisor (like esxi). Instead of running on top of Linux, Xen has it's own microkernel. This means less flexibility (there's no Linux body running things), but also simpler to manage and less attack surface. I haven't played much with xen though, KVM is kind of the defacto, but iirc AWS uses a modified Xen.

If you actually played with Xen you'd know it's not actually easier to manage. And increased security claims are dubious at best, as same thing that would be attacked (dom0 managing the whole thing and running linux) have direct unfettered access to xen microkernel. There is reason many sites migrated away from Xen to KVM. Also many Xen drivers de facto run part Linux dom0 instance so you don't even get that isolation.

We ran Xen for few years, as KVM at first was still not as refined and Xen was first to mature market, and it was just million little annoying things.

KVM offers far simple and straightforward management. A VM is just a process. You can look at its CPU usage via normal tools. No magic. No driver problems.

◧◩◪◨⬒
10. gabere+Qi[view] [source] [discussion] 2023-07-10 16:38:31
>>bonzin+rd
Semantics, libvirt abstracts over KVM via QEMU because QEMU/KVM/HVT is all one driver.
replies(1): >>bonzin+wR
◧◩
11. monoca+Yv[view] [source] [discussion] 2023-07-10 17:27:05
>>theoss+y7
> I haven't played much with xen though, KVM is kind of the defacto, but iirc AWS uses a modified Xen.

AWS switched to KVM, and even a lot of AWS systems that report themselves as Xen are running as KVM with a compat shim to say it's Xen.

◧◩
12. cthalu+OF[view] [source] [discussion] 2023-07-10 17:56:33
>>theoss+y7
> Xen is an alternative hypervisor (like esxi). Instead of running on top of Linux, Xen has it's own microkernel. This means less flexibility (there's no Linux body running things), but also simpler to manage and less attack surface.

You're going to need dom0 (a "control domain") on any Xen host. Gotta have something running xl and the rest of the toolstack for managing it. dom0less technically exists but the drawbacks mean it's not really usable by most people in a production situation.

◧◩◪◨⬒⬓
13. bonzin+wR[view] [source] [discussion] 2023-07-10 18:38:20
>>gabere+Qi
KVM is not enough to create a virtual machine. KVM only virtualizes the processor, not a whole machine.
replies(1): >>reacto+Ec1
◧◩◪
14. crabbo+mW[view] [source] [discussion] 2023-07-10 19:00:28
>>Izmaki+E7
> On the first path you are likely going to be just fine with VirtualBox, VMWare Workstation or Hyper-V (Windows only) / Parallels (Mac intended).

As a Linux user, why would you want to use VirtualBox or VMWare Workstation? They are not so well integrated with the system, and, frankly, VirtualBox is more of a toy VM player... just go for virt-manager. It gives a conceptually similar interface to VirtualBox, but better integration with the rest of the system. Especially, when it comes to stuff like sending different key combinations.

I honestly cannot think of a single benefit to using VirtualBox (and I'm less familiar with VMWare player) compared to virt-manager. My guess is that it's more often used because it's also a common choice on MS Windows, so, you get more hits if you are going to search the Web for questions associated to VMs / you'd get tutorials for how to set up a VM that use VirtualBox. But, if you apply yourself to learning how either one of these works, you'd see no reason to choose it.

replies(1): >>mkespe+4n1
◧◩◪◨⬒⬓⬔
15. reacto+Ec1[view] [source] [discussion] 2023-07-10 20:16:19
>>bonzin+wR
>Using KVM, one can run multiple virtual machines running unmodified Linux or Windows images. Each virtual machine has private virtualized hardware: a network card, disk, graphics adapter, etc.

Straight from their site. QEMU is the user space interface, KVM the kernel space driver. It’s enough to run whatever OS. That’s the point.

For libvirt: https://libvirt.org/drivers.html

They support a bunch as well.

replies(2): >>bonzin+we1 >>cthalu+op1
◧◩◪◨⬒⬓⬔⧯
16. bonzin+we1[view] [source] [discussion] 2023-07-10 20:24:36
>>reacto+Ec1
Exactly, it's QEMU that abstracts over the processor virtualization APIs of KVM, Microsoft WHPX, Apple Hypervisor.framework etc. Not Libvirt.
◧◩◪◨
17. mkespe+4n1[view] [source] [discussion] 2023-07-10 21:07:47
>>crabbo+mW
The only reason for using it would be that the GUI is a little bit more capable. But feature crippling without the PUEL addons is horrible.
◧◩◪◨⬒⬓⬔⧯
18. cthalu+op1[view] [source] [discussion] 2023-07-10 21:20:39
>>reacto+Ec1
I don't want to necessarily make this an argument to/from authority, but for some context here - you are discussing this with Paolo Bonzini, maintainer of KVM, contributor to QEMU. In the list of people that best understand the difference and demarcation points between KVM and QEMU, he's pretty far up there.
[go to top]