Nested KVM virtualization is a powerful technology that allows you to run multiple virtual machines on one computer. This can be helpful if you need to run multiple applications on different machines, or if you want to use different operating systems on different machines. To enable nested KVM virtualization, you first need to install the appropriate software. Then, you need to create a new virtual machine and add the appropriate files to it. Finally, you need to enable the nested KVM virtualization feature in your BIOS or motherboard settings. If you have an AMD Ryzen processor or an Nvidia GeForce RTX 2070 graphics card, you can use the built-in virt-manager tool to manage your virtual machines. If you don’t have virt-manager, there are several other ways to manage your virtual machines. You can use a third-party software such as VMware Workstation or Hyper-V Manager. If you’re using Windows 10, there’s also a built-in feature called “KVM Switch” that lets you switch between different virtual machines without having to open each one individually. You can find this feature in the “System” menu under “Settings.”
There are several ways to create a virtual machine using KVM. QEMU can use KVM and libvirt and its Virtual Machine Manager UI provide a convenient interface around it.
KVM can be used to create nested virtual machines on compatible hardware. This lets you create VMs inside VMs to accommodate more complex use cases. Consider a virtualized development environment that runs on your host. You might need to run virtual device emulators within that environment, nested two levels deep from the bare metal hardware.
This article will show you how to set up nested KVM virtualization and test that it’s working. Before continuing, check you’ve got a functioning KVM installation available and that you’re familiar with creating new KVM virtual machines.
Checking Whether Nested Virtualization Is Enabled
Nested virtualization is supported by most modern processor families that offer hardware virtualization. You can check whether nesting is already enabled on your hypervisor by using the cat command to read one of the following paths, depending on whether you’ve got an Intel or AMD system:
The output should be either Y or N. Seeing Y means you’re good to go – nested virtualization is already turned on. You can skip down to the “Activating Nested Virtualization For a Guest” section below. If you see N in your terminal, it’s time to enable nesting in KVM’s kernel module.
Enabling Nested Virtualization
Nesting is controlled by a KVM kernel module parameter. You can change the parameter by editing /etc/modprobe.d/qemu-system-x86.conf. On some systems, this file could be called /etc/modprobe.d/kvm.conf.
You’ll probably see a single line similar to one of these:
Any one of these variants means KVM is active but nesting is disabled.
To enable nesting, simply add or change the nested parameter so it has 1 as its value:
Next you need to reload the KVM kernel module to apply your change. You should stop any running virtual machines before you do this.
Substitute kvm_amd instead of kvm_intel if you have an AMD processor.
Now repeat the command from earlier to check whether nesting is enabled. You should get Y as the output.
This method permanently enables nested virtualization. It will persist across reboots until you remove nested=1 from the KVM module’s parameters.
Activating Nested Virtualization For a Guest
Guest virtual machines can only use nested virtualization when they’re configured with a CPU mode that supports it. The guest needs a CPU definition that exactly matches the physical hardware on your host.
Most guests will work when the CPU mode is set to host-model, which is usually the default. This means the guest receives a CPU definition that’s similar to your host’s. In some cases you might need to use the host-passthrough mode that exactly passes through all the characteristics of the host CPU.
You can check and change a guest’s CPU type by retrieving its manifest with virsh. First run the virsh command to launch an interactive shell. Then type list –all to retrieve all your VMs:
Next run edit
Within the file, find the line that starts with <cpu mode=. Change it to one of these:
Save and close the file, then type exit into the virsh shell to close it. The guest should now be ready to start its own nested guests. Try changing modes if there seems to be a problem.
Checking a Guest Can Nest
Most operating systems can tell you whether they can create a VM. Run the following command within your VM to check whether a Linux guest has access to virtualization:
Virtualization is available if you get some output with svm or vmx highlighted in red. SVM will show up on AMD machines; VMX appears for Intel.
Now install a virtualization technology within the guest. You should find you can start a new nested VM. Here’s a screenshot showing an Ubuntu virtual machine that is itself running an Alpine guest using nested KVM:
Limitations
Nested guests come with a few limitations. Some KVM features become unavailable for guests that have started a nested VM. You won’t be able to migrate, save, or load these virtual machines, until the nested VM is stopped.
The actual effect of trying to perform one of these operations is undefined. Some systems could withstand it; others may cause a kernel panic. Always try to shutdown your nested guests before performing an operation on VMs above them in the chain.
Summary
Nested virtualization provides more power and flexibility. You can sandbox technologies that need their own virtualization to function, such as IDEs that launch device emulators.
Getting nesting to work with KVM is normally straightforward. Any troubleshooting should begin by checking the nested parameter is enabled for your KVM kernel module. After that, check the CPU model assigned to your guest and verify you’re using a compatible second-level hypervisor within the VM.
