《KVM实战：原理、进阶与性能调优》
目录
1. 硬件系统的配置
KVM 从诞生之初就需要硬件虚拟化扩展的支持，其最初的开发是基于x86和x86-64处理器架构上的 Linux 系统进行的。
目前，KVM 被移植到多种不同处理器架构上，但在x86-64架构上的支持是最完善的。
本文默认基于
x86-64的 Linux 系统进行相关操作
在x86-64架构的处理器中，KVM 需要的硬件虚拟化扩展分别为：
- Intel VT
- AMD-V
1.1 BIOS 开启 VT/VT-d
- 需要 CPU 在硬件上支持 VT 技术
- 在 BIOS 中启用
VT即Intel(R) Virtualization Technology - 在 BIOS 中启用
VT-d即Intel(R) VT for Directed I/O
1.2 重启后查看 CPU 特性标志
设置好VT和VT-d的相关选项，保存 BIOS 设置并退出，系统重启后生效。
可通过检查/proc/cpuinfo文件中的 CPU 特性标志（flags）来查看 CPU 目前是否支持硬件虚拟化：
Intel CPU 支持虚拟化的标志为
vmx
AMD CPU 支持虚拟化的标志为svm
> grep -E "vmx|svm" /proc/cpuinfo
flags : ... vmx ...
...
2. 安装宿主机操作系统
- CentOS 7 安装时不需要选择
Virtualization Host，因为后续会自行编译 KVM 及 QEMU 源码 - CentOS 7 推荐选择
Server with GUI、GNOME Desktop以及Development Tools
本文实验环境：OS
Ubuntu 18.04 LTS，CPUi7-6700 * 8，内存32G
3. 编译和安装 KVM
3.1 下载 KVM 源代码
> git clone git://git.kernel.org/pub/scm/virt/kvm/kvm.git
# or
> git clone https://git.kernel.org/pub/scm/virt/kvm/kvm.git
内核版本：本文使用的是
kvm.git的4.21.1版本，编译后显示的内核版本为4.20.0-rc6
源码地址： index: kvm/kvm.git。
3.2 配置 KVM
KVM 是作为 Linux 内核中的一个 module 而存在的，而kvm.git是一个包含了最新的 KVM 模块开发中代码的完整的 Linux 内核源码仓库。它的配置方式与普通的 Linux 内核配置完全一样，只是需要注意将 KVM 相关的配置选择为编译进内核或者编译为模块。
make 常用命令
在kvm.git目录下，运行make help查看关于如何配置和编译 kernel 的帮助说明：
> make help
Cleaning targets:
clean - Remove most generated files but keep the config and
enough build support to build external modules
mrproper - Remove all generated files + config + various backup files
distclean - mrproper + remove editor backup and patch files
Configuration targets:
config - Update current config utilising a line-oriented program
nconfig - Update current config utilising a ncurses menu based program
menuconfig - Update current config utilising a menu based program
xconfig - Update current config utilising a Qt based front-end
gconfig - Update current config utilising a GTK+ based front-end
oldconfig - Update current config utilising a provided .config as base
localmodconfig - Update current config disabling modules not loaded
localyesconfig - Update current config converting local mods to core
defconfig - New config with default from ARCH supplied defconfig
savedefconfig - Save current config as ./defconfig (minimal config)
allnoconfig - New config where all options are answered with no
allyesconfig - New config where all options are accepted with yes
allmodconfig - New config selecting modules when possible
alldefconfig - New config with all symbols set to default
randconfig - New config with random answer to all options
listnewconfig - List new options
olddefconfig - Same as oldconfig but sets new symbols to their
default value without prompting
kvmconfig - Enable additional options for kvm guest kernel support
xenconfig - Enable additional options for xen dom0 and guest kernel support
tinyconfig - Configure the tiniest possible kernel
testconfig - Run Kconfig unit tests (requires python3 and pytest)
Other generic targets:
all - Build all targets marked with [*]
* vmlinux - Build the bare kernel
* modules - Build all modules
modules_install - Install all modules to INSTALL_MOD_PATH (default: /)
dir/ - Build all files in dir and below
dir/file.[ois] - Build specified target only
dir/file.ll - Build the LLVM assembly file
(requires compiler support for LLVM assembly generation)
dir/file.lst - Build specified mixed source/assembly target only
(requires a recent binutils and recent build (System.map))
dir/file.ko - Build module including final link
modules_prepare - Set up for building external modules
tags/TAGS - Generate tags file for editors
cscope - Generate cscope index
gtags - Generate GNU GLOBAL index
kernelrelease - Output the release version string (use with make -s)
kernelversion - Output the version stored in Makefile (use with make -s)
image_name - Output the image name (use with make -s)
headers_install - Install sanitised kernel headers to INSTALL_HDR_PATH
(default: ./usr)
Static analysers:
checkstack - Generate a list of stack hogs
namespacecheck - Name space analysis on compiled kernel
versioncheck - Sanity check on version.h usage
includecheck - Check for duplicate included header files
export_report - List the usages of all exported symbols
headers_check - Sanity check on exported headers
headerdep - Detect inclusion cycles in headers
coccicheck - Check with Coccinelle
Kernel selftest:
kselftest - Build and run kernel selftest (run as root)
Build, install, and boot kernel before
running kselftest on it
kselftest-clean - Remove all generated kselftest files
kselftest-merge - Merge all the config dependencies of kselftest to existing
.config.
Userspace tools targets:
use "make tools/help"
or "cd tools; make help"
Kernel packaging:
rpm-pkg - Build both source and binary RPM kernel packages
binrpm-pkg - Build only the binary kernel RPM package
deb-pkg - Build both source and binary deb kernel packages
bindeb-pkg - Build only the binary kernel deb package
snap-pkg - Build only the binary kernel snap package (will connect to external hosts)
tar-pkg - Build the kernel as an uncompressed tarball
targz-pkg - Build the kernel as a gzip compressed tarball
tarbz2-pkg - Build the kernel as a bzip2 compressed tarball
tarxz-pkg - Build the kernel as a xz compressed tarball
perf-tar-src-pkg - Build perf-4.20.0-rc6.tar source tarball
perf-targz-src-pkg - Build perf-4.20.0-rc6.tar.gz source tarball
perf-tarbz2-src-pkg - Build perf-4.20.0-rc6.tar.bz2 source tarball
perf-tarxz-src-pkg - Build perf-4.20.0-rc6.tar.xz source tarball
Documentation targets:
Linux kernel internal documentation in different formats from ReST:
htmldocs - HTML
latexdocs - LaTeX
pdfdocs - PDF
epubdocs - EPUB
xmldocs - XML
linkcheckdocs - check for broken external links (will connect to external hosts)
refcheckdocs - check for references to non-existing files under Documentation
cleandocs - clean all generated files
make SPHINXDIRS="s1 s2" [target] Generate only docs of folder s1, s2
valid values for SPHINXDIRS are: driver-api networking input core-api userspace-api media gpu process sound crypto vm maintainer sh dev-tools doc-guide filesystems kernel-hacking admin-guide
make SPHINX_CONF={conf-file} [target] use *additional* sphinx-build
configuration. This is e.g. useful to build with nit-picking config.
Default location for the generated documents is Documentation/output
Architecture specific targets (x86):
* bzImage - Compressed kernel image (arch/x86/boot/bzImage)
install - Install kernel using
(your) ~/bin/installkernel or
(distribution) /sbin/installkernel or
install to $(INSTALL_PATH) and run lilo
fdimage - Create 1.4MB boot floppy image (arch/x86/boot/fdimage)
fdimage144 - Create 1.4MB boot floppy image (arch/x86/boot/fdimage)
fdimage288 - Create 2.8MB boot floppy image (arch/x86/boot/fdimage)
isoimage - Create a boot CD-ROM image (arch/x86/boot/image.iso)
bzdisk/fdimage*/isoimage also accept:
FDARGS="..." arguments for the booted kernel
FDINITRD=file initrd for the booted kernel
i386_defconfig - Build for i386
x86_64_defconfig - Build for x86_64
make V=0|1 [targets] 0 => quiet build (default), 1 => verbose build
make V=2 [targets] 2 => give reason for rebuild of target
make O=dir [targets] Locate all output files in "dir", including .config
make C=1 [targets] Check re-compiled c source with $CHECK (sparse by default)
make C=2 [targets] Force check of all c source with $CHECK
make RECORDMCOUNT_WARN=1 [targets] Warn about ignored mcount sections
make W=n [targets] Enable extra gcc checks, n=1,2,3 where
1: warnings which may be relevant and do not occur too often
2: warnings which occur quite often but may still be relevant
3: more obscure warnings, can most likely be ignored
Multiple levels can be combined with W=12 or W=123
Execute "make" or "make all" to build all targets marked with [*]
For further info see the ./README file
对 KVM 进行内核配置常用的一些配置命令如下：
make config：基于文本的最为传统也是最为枯燥的一种配置方式，适用于任何情况make oldconfig：在现有的内核设置文件基础上建立一个新的设置文件，只会向用户提供有关新内核特性的问题make silentoldconfig：和上面的make oldconfig一样，只是额外会静默更新选项的依赖关系make olddefconfig：和上面的make silentoldconfig一样，但不需要手动交互，而是对新选项以其默认值配置make menuconfig：基于终端的一种配置方式，提供了文本模式的图形用户界面，用户可以通过移动光标来浏览所支持的各种特性，要求系统中安装ncurses库make xconfig：基于 X Window 的一种配置方式，只能在 X Server 上运行 X 桌面应用程序时使用，并且依赖于 QT 库make gconfig：与make xconfig类似，不同的是它依赖于 GTK 库makedefconfig：按照内核代码中提供的默认配置文件对内核进行配置。例如在Intel x86_64平台上，默认配置为arch/x86/configs/x86_64_defconfig，生成.config文件可以用作初始化配置，然后再使用make menuconfig进行定制化配置make allmodconfig：尽可能多的使用y输入设置内核选项值，生成的配置中包含了全部的内核特性make allnoconfig：除必需的选项外，其他选项一律不选（常用于嵌入式系统的编译）make allmodconfig：尽可能多的使用m输入设置内核选项值来生成配置文件make localmodconfig：会执行lsmod命令查看当前系统中加载了哪些模块，并最终将原来的.config中不需要的模块去掉
make olddefconfig
为了确保生成的.config文件生成的 Kernel 是实际可以工作的（直接make defconfig生成的.config文件编译出来的 Kernel 常常是不能工作的），最佳实践是以你当前使用的 config 为基础，将它复制到当前编译目录下，重命名为.config，然后再通过make olddefconfig更新补充这个设置文件：
> cp /boot/config-3.10.0-862.14.4.el7.x86_64 .config
cp: overwrite ‘.config’? y
> make olddefconfig
HOSTCC scripts/basic/fixdep
HOSTCC scripts/kconfig/conf.o
HOSTCC scripts/kconfig/confdata.o
HOSTCC scripts/kconfig/expr.o
LEX scripts/kconfig/lexer.lex.c
YACC scripts/kconfig/parser.tab.h
HOSTCC scripts/kconfig/lexer.lex.o
YACC scripts/kconfig/parser.tab.c
HOSTCC scripts/kconfig/parser.tab.o
HOSTCC scripts/kconfig/preprocess.o
HOSTCC scripts/kconfig/symbol.o
HOSTLD scripts/kconfig/conf
scripts/kconfig/conf --olddefconfig Kconfig
.config:676:warning: symbol value 'm' invalid for CPU_FREQ_STAT
.config:755:warning: symbol value 'm' invalid for HOTPLUG_PCI_SHPC
.config:918:warning: symbol value 'm' invalid for NF_CT_PROTO_GRE
.config:946:warning: symbol value 'm' invalid for NF_NAT_REDIRECT
.config:949:warning: symbol value 'm' invalid for NF_TABLES_INET
.config:1111:warning: symbol value 'm' invalid for NF_TABLES_IPV4
.config:1115:warning: symbol value 'm' invalid for NF_TABLES_ARP
.config:1156:warning: symbol value 'm' invalid for NF_TABLES_IPV6
.config:1188:warning: symbol value 'm' invalid for NF_TABLES_BRIDGE
.config:1532:warning: symbol value 'm' invalid for NET_DEVLINK
.config:2958:warning: symbol value 'm' invalid for HW_RANDOM_TPM
.config:3554:warning: symbol value 'm' invalid for LIRC
.config:4104:warning: symbol value 'm' invalid for HSA_AMD
.config:4460:warning: symbol value 'm' invalid for SND_X86
#
# configuration written to .config
#
> cat .config | grep KVM
CONFIG_KVM_GUEST=y
CONFIG_KVM_DEBUG_FS=y
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_COMPAT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
CONFIG_KVM_AMD=m
CONFIG_KVM_AMD_SEV=y
CONFIG_KVM_MMU_AUDIT=y
CONFIG_PTP_1588_CLOCK_KVM=m
CONFIG_DRM_I915_GVT_KVMGT=m
make menuconfig
之后使用make menuconfig进行定制化配置，首先需要安装以下依赖项：
> yum install -y ncurses-devel flex bison
之后启动make menuconfig：
> make menuconfig
HOSTCC scripts/kconfig/mconf.o
HOSTCC scripts/kconfig/lxdialog/checklist.o
HOSTCC scripts/kconfig/lxdialog/inputbox.o
HOSTCC scripts/kconfig/lxdialog/menubox.o
HOSTCC scripts/kconfig/lxdialog/textbox.o
HOSTCC scripts/kconfig/lxdialog/util.o
HOSTCC scripts/kconfig/lxdialog/yesno.o
HOSTLD scripts/kconfig/mconf
scripts/kconfig/mconf Kconfig
*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.
make menuconfig 命令的选择界面
Virtualization 中的配置选项
修改完成后保存Save并退出Exit，可以看到.config中CONFIG_KVM_AMD已被修改：
> cat .config | grep KVM
CONFIG_KVM_GUEST=y
CONFIG_KVM_DEBUG_FS=y
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_COMPAT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
# CONFIG_KVM_AMD is not set
CONFIG_KVM_MMU_AUDIT=y
CONFIG_PTP_1588_CLOCK_KVM=m
CONFIG_DRM_I915_GVT_KVMGT=m
3.3 编译 KVM
在对 KVM 源代码进行配置之后，编译 KVM 就比较容易了。它的编译过程就是一个普通的 Linux 内核编译过程，包括以下三个步骤：
- 编译 kernel
- 编译 bzImage
- 编译 内核模块 modules
编译 kernel
> make vmlinux -j 10
error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
> yum install -y elfutils-libelf-devel
> make vmlinux -j 10
# 此处省略部分编译时的输出信息
GEN .version
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
AR init/built-in.a
LD vmlinux.o
MODPOST vmlinux.o
KSYM .tmp_kallsyms1.o
KSYM .tmp_kallsyms2.o
LD vmlinux # 这里就是编译、链接后生成的启动所需的 Linux Kernel 文件
SORTEX vmlinux
SYSMAP System.map
其中，编译命令中的-j参数不是必需的，他是允许 make 工具用多任务（job）来进行编译。例如上面的-j 10，表示 make 工具最多可以创建 20 个 GCC 进程，同时来进行编译任务。
在一个比较空闲的系统上，-j参数的推荐值大约为 2 倍于系统上的 CPU core。
如果
-j后面不跟数字，则 make 会根据现在系统中的 CPU core 的数量自动安排任务数，通常比 core 的数量略多一点
编译完成后，可看到当前目录下生成了我们所需的vmlinux内核文件：
> ls -hl vmlinux
-rwxr-xr-x 1 root root 446M Apr 21 18:14 vmlinux
编译 bzImage
> make bzImage
CALL scripts/checksyscalls.sh
CALL scripts/atomic/check-atomics.sh
DESCEND objtool
CHK include/generated/compile.h
HOSTCC arch/x86/tools/insn_decoder_test
HOSTCC arch/x86/tools/insn_sanity
TEST posttest
arch/x86/tools/insn_decoder_test: success: Decoded and checked 6299855 instructions
TEST posttest
arch/x86/tools/insn_sanity: Success: decoded and checked 1000000 random instructions with 0 errors (seed:0x7ffcb19)
CC arch/x86/boot/a20.o
AS arch/x86/boot/bioscall.o
CC arch/x86/boot/cmdline.o
AS arch/x86/boot/copy.o
HOSTCC arch/x86/boot/mkcpustr
CPUSTR arch/x86/boot/cpustr.h
CC arch/x86/boot/cpu.o
CC arch/x86/boot/cpuflags.o
CC arch/x86/boot/cpucheck.o
CC arch/x86/boot/early_serial_console.o
CC arch/x86/boot/edd.o
LDS arch/x86/boot/compressed/vmlinux.lds
AS arch/x86/boot/compressed/head_64.o
VOFFSET arch/x86/boot/compressed/../voffset.h
CC arch/x86/boot/compressed/misc.o
CC arch/x86/boot/compressed/string.o
CC arch/x86/boot/compressed/cmdline.o
CC arch/x86/boot/compressed/error.o
OBJCOPY arch/x86/boot/compressed/vmlinux.bin
RELOCS arch/x86/boot/compressed/vmlinux.relocs
GZIP arch/x86/boot/compressed/vmlinux.bin.gz
HOSTCC arch/x86/boot/compressed/mkpiggy
MKPIGGY arch/x86/boot/compressed/piggy.S
AS arch/x86/boot/compressed/piggy.o
CC arch/x86/boot/compressed/cpuflags.o
CC arch/x86/boot/compressed/early_serial_console.o
CC arch/x86/boot/compressed/kaslr.o
CC arch/x86/boot/compressed/kaslr_64.o
AS arch/x86/boot/compressed/mem_encrypt.o
CC arch/x86/boot/compressed/pgtable_64.o
CC arch/x86/boot/compressed/acpi.o
CC arch/x86/boot/compressed/eboot.o
AS arch/x86/boot/compressed/efi_stub_64.o
AS arch/x86/boot/compressed/efi_thunk_64.o
LD arch/x86/boot/compressed/vmlinux
ZOFFSET arch/x86/boot/zoffset.h
AS arch/x86/boot/header.o
CC arch/x86/boot/main.o
CC arch/x86/boot/memory.o
CC arch/x86/boot/pm.o
AS arch/x86/boot/pmjump.o
CC arch/x86/boot/printf.o
CC arch/x86/boot/regs.o
CC arch/x86/boot/string.o
CC arch/x86/boot/tty.o
CC arch/x86/boot/video.o
CC arch/x86/boot/video-mode.o
CC arch/x86/boot/version.o
CC arch/x86/boot/video-vga.o
CC arch/x86/boot/video-vesa.o
CC arch/x86/boot/video-bios.o
LD arch/x86/boot/setup.elf
OBJCOPY arch/x86/boot/setup.bin
OBJCOPY arch/x86/boot/vmlinux.bin
HOSTCC arch/x86/boot/tools/build
BUILD arch/x86/boot/bzImage
Setup is 17340 bytes (padded to 17408 bytes).
System is 7661 kB
CRC 93bcb672
Kernel: arch/x86/boot/bzImage is ready (#2)
可以看到arch/x86/boot/bzImage已经生成：
> ls -hl arch/x86/boot/bzImage
-rw-r--r-- 1 root root 7.5M Apr 21 19:03 arch/x86/boot/bzImage
> ls -hl arch/x86_64/boot/bzImage
lrwxrwxrwx 1 root root 22 Apr 21 19:03 arch/x86_64/boot/bzImage -> ../../x86/boot/bzImage
编译 module
编译 Kernel 和 bzImage 之后编译内核的模块：
> make modules -j 10
# 此处省略部分编译时的输出信息
LD [M] sound/soc/snd-soc-acpi.ko
LD [M] sound/soc/snd-soc-core.ko
LD [M] sound/soundcore.ko
LD [M] sound/synth/emux/snd-emux-synth.ko
LD [M] sound/synth/snd-util-mem.ko
LD [M] sound/usb/6fire/snd-usb-6fire.ko
LD [M] sound/usb/bcd2000/snd-bcd2000.ko
LD [M] sound/usb/caiaq/snd-usb-caiaq.ko
LD [M] sound/usb/hiface/snd-usb-hiface.ko
LD [M] sound/usb/line6/snd-usb-line6.ko
LD [M] sound/usb/line6/snd-usb-pod.ko
LD [M] sound/usb/line6/snd-usb-podhd.ko
LD [M] sound/usb/line6/snd-usb-toneport.ko
LD [M] sound/usb/line6/snd-usb-variax.ko
LD [M] sound/usb/misc/snd-ua101.ko
LD [M] sound/usb/snd-usb-audio.ko
LD [M] sound/usb/snd-usbmidi-lib.ko
LD [M] sound/usb/usx2y/snd-usb-us122l.ko
LD [M] sound/usb/usx2y/snd-usb-usx2y.ko
LD [M] sound/x86/snd-hdmi-lpe-audio.ko
LD [M] virt/lib/irqbypass.ko
3.4 安装 KVM
KVM 的安装包括两个步骤：
- 安装 module
- 安装 kernel 与 initramfs
安装 module
通过make modules_install命令可以将编译好的 module 安装到相应的目录中：
> make modules_install
# 此处省略部分编译时的输出信息
INSTALL sound/usb/snd-usbmidi-lib.ko
INSTALL sound/usb/usx2y/snd-usb-us122l.ko
INSTALL sound/usb/usx2y/snd-usb-usx2y.ko
INSTALL sound/x86/snd-hdmi-lpe-audio.ko
INSTALL virt/lib/irqbypass.ko
DEPMOD 4.20.0-rc6
默认情况下，module 会被安装到
/lib/modules/$kernel_version/kernel目录中
安装完成后可以查看对应的安装路径，且kvm.ko、kvm-intel.ko两个模块也已经安装：
> ls -hl /lib/modules/4.20.0-rc6/kernel
total 16K
drwxr-xr-x 3 root root 17 Apr 23 16:35 arch
drwxr-xr-x 3 root root 4.0K Apr 23 16:35 crypto
drwxr-xr-x 68 root root 4.0K Apr 23 16:36 drivers
drwxr-xr-x 25 root root 4.0K Apr 23 16:36 fs
drwxr-xr-x 3 root root 19 Apr 23 16:36 kernel
drwxr-xr-x 5 root root 207 Apr 23 16:36 lib
drwxr-xr-x 2 root root 32 Apr 23 16:36 mm
drwxr-xr-x 35 root root 4.0K Apr 23 16:37 net
drwxr-xr-x 12 root root 167 Apr 23 16:37 sound
drwxr-xr-x 3 root root 17 Apr 23 16:37 virt
> ls -hl /lib/modules/4.20.0-rc6/kernel/arch/x86/kvm
total 15M
-rw-r--r-- 1 root root 3.6M Apr 23 16:35 kvm-intel.ko
-rw-r--r-- 1 root root 11M Apr 23 16:35 kvm.ko
安装 kernel 和 initramfs
通过make install命令安装 kernel 和 initramfs，命令行输出如下：
> make install
sh ./arch/x86/boot/install.sh 4.20.0-rc6 arch/x86/boot/bzImage \
System.map "/boot"
> ls -hl /boot -t
total 300M
-rw------- 1 root root 108M Apr 23 21:37 initramfs-4.20.0-rc6.img
lrwxrwxrwx 1 root root 27 Apr 23 21:34 System.map -> /boot/System.map-4.20.0-rc6
lrwxrwxrwx 1 root root 24 Apr 23 21:34 vmlinuz -> /boot/vmlinuz-4.20.0-rc6
-rw-r--r-- 1 root root 3.5M Apr 23 21:34 System.map-4.20.0-rc6
-rw-r--r-- 1 root root 7.5M Apr 23 21:34 vmlinuz-4.20.0-rc6
drwx------. 2 root root 21 Jan 9 16:48 grub2
drwxr-xr-x. 2 root root 27 Nov 13 11:28 grub
drwx------ 3 root root 16K Jan 1 1970 efi
可以看到在/boot目录下生成了内核文件vmlinuz和initramfs等内核启动所需的文件。
另外在运行make install之后，/boot/efi/EFI/centos/grub.cfg配置文件中也自动添加了一个 grub 选项，如下所示：
注：在下面的
menuentry中还配置了KVMGT的相关选项，之后会另写一篇文章加以说明
menuentry 'Ubuntu，Linux 4.20.0-rc6 KVMGT' --class kvmgt --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option'gnulinux-4.20.0-rc6-advanced-26d36e85-367a-4200-87fb-0505c5837078' {
recordfail
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
set root='hd0,gpt8'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt8 --hint-efi=hd0,gpt8 --hint-baremetal=ahci0,gpt8 26d36e85-367a-4200-87fb-0505c5837078
else
search --no-floppy --fs-uuid --set=root 26d36e85-367a-4200-87fb-0505c5837078
fi
echo '载入 Linux 4.20.0-rc6 ...'
linux /boot/vmlinuz-4.20.0-rc6 root=UUID=26d36e85-367a-4200-87fb-0505c5837078 ro quiet splash $vt_handoff ignore_loglevel log_buf_len=128M console=ttyS0,115200,8n1 i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on drm.debug=0
echo '载入初始化内存盘...'
initrd /boot/initrd.img-4.20.0-rc6
}
联想扬天 T4900 开机进入 BIOS 快捷键 F12
检查grub.cfg配置无误后，重新启动系统，选择刚才为 KVM 而编译、安装的内核启动。
3.5 安装后的检查
进入系统后，使用uname -r查看内核版本：
> uname -r
4.20.0-rc6
通常情况下，系统启动时已经默认加载了kvm和kvm_intel这两个模块，如果没有加载，则需要手动使用modprobe命令依次加载这两个模块：
> modprobe kvm
> modprobe kvm_intel
> lsmod | grep kvm
kvm_intel 245760 0
kvmgt 28672 1
mdev 24576 2 kvmgt,vfio_mdev
vfio 32768 3 kvmgt,vfio_mdev,vfio_iommu_type1
kvm 634880 2 kvmgt,kvm_intel
irqbypass 16384 1 kvm
确认 KVM 相关模块加载成功后，检查/dev/kvm文件是否存在：
> ls -l /dev/kvm
crw-rw---- 1 root kvm 10, 232 Apr 29 09:36 /dev/kvm
/dev/kvm是 KVM 内核模块提供给用户空间 QEMU 程序使用的一个控制接口，提供了Guest OS运行所需要的模拟和实际的硬件设备环境。
crw-rw----以c为开头，表示/dev/kvm是一个字符设备
4. 编译和安装 QEMU
除了在内核空间的kvm.ko模块之外，在用户空间还需要 QEMU 来模拟 VM 所需要的 I/O 设备，并启动客户机进程。
在早期版本中，支持 KVM 的qemu-kvm是由 kernel 社区维护的专门用于 KVM 虚拟化的 QEMU 分支。2012 年末，这个分支并入了主流的 QEMU 仓库，从此就不再需要特殊的qemu-kvm，而只需在通用的 QEMU 命令后添加--enable-kvm选项，即可创建 KVM Guest。
4.1 下载 QEMU 源代码
直接下载源代码归档包：
~ > wget https://download.qemu.org/qemu-4.0.0.tar.xz
~ > tar -xvJf qemu-4.0.0.tar.xz
~ > cd qemu-4.0.0
qemu-4.0.0 > ls
accel capstone device_tree.c hmp-commands.hx MAINTAINERS pc-bios qemu-keymap.c qtest.c tpm.c
arch_init.c Changelog disas hmp-commands-info.hx Makefile po qemu-nbd.c README trace
audio chardev disas.c hmp.h Makefile.objs python qemu-nbd.texi replay trace-events
authz CODING_STYLE dma-helpers.c hw Makefile.target qapi qemu.nsi replication.c ui
backends config.log docs include memory.c qdev-monitor.c qemu-options.h replication.h util
balloon.c config-temp dtc io memory_ldst.inc.c qemu-bridge-helper.c qemu-options.hx roms VERSION
block configure dump.c ioport.c memory_mapping.c qemu-deprecated.texi qemu-options-wrapper.h rules.mak version.rc
block.c contrib exec.c iothread.c migration qemu-doc.texi qemu-option-trace.texi scripts vl.c
blockdev.c COPYING fpu job.c module-common.c qemu-edid.c qemu.sasl scsi win_dump.c
blockdev-nbd.c COPYING.LIB fsdev job-qmp.c monitor.c qemu-ga.texi qemu-seccomp.c slirp win_dump.h
blockjob.c cpus.c gdbstub.c Kconfig.host nbd qemu-img.c qemu-tech.texi stubs
bootdevice.c cpus-common.c gdb-xml libdecnumber net qemu-img-cmds.hx qga target
bsd-user crypto gitdm.config LICENSE numa.c qemu-img.texi qmp.c tcg
bt-host.c default-configs HACKING linux-headers os-posix.c qemu-io.c qobject tests
bt-vhci.c device-hotplug.c hmp.c linux-user os-win32.c qemu-io-cmds.c qom thunk.c
qemu-4.0.0 > cat VERSION
4.0.0
或者使用 Git 拉取 QEMU 源代码：
~ > git clone https://git.qemu.org/git/qemu.git
~ > cd qemu
qemu > git submodule init
qemu > git submodule update --recursive
4.2 配置和编译 QEMU
配置 QEMU
本文使用的 QEMU 版本为
4.0.50
首先运行./configure --help查看配置 QEMU 的选项及帮助信息：
qemu-4.0.50 > ./configure --help
Usage: configure [options]
Options: [defaults in brackets after descriptions]
Standard options:
--help print this message
--prefix=PREFIX install in PREFIX [/usr/local]
--interp-prefix=PREFIX where to find shared libraries, etc.
use %M for cpu name [/usr/gnemul/qemu-%M]
--target-list=LIST set target list (default: build everything)
Available targets: aarch64-softmmu alpha-softmmu
arm-softmmu cris-softmmu hppa-softmmu i386-softmmu
lm32-softmmu m68k-softmmu microblaze-softmmu
microblazeel-softmmu mips-softmmu mips64-softmmu
mips64el-softmmu mipsel-softmmu moxie-softmmu
nios2-softmmu or1k-softmmu ppc-softmmu ppc64-softmmu
riscv32-softmmu riscv64-softmmu s390x-softmmu
sh4-softmmu sh4eb-softmmu sparc-softmmu
sparc64-softmmu tricore-softmmu unicore32-softmmu
x86_64-softmmu xtensa-softmmu xtensaeb-softmmu
aarch64-linux-user aarch64_be-linux-user
alpha-linux-user arm-linux-user armeb-linux-user
cris-linux-user hppa-linux-user i386-linux-user
m68k-linux-user microblaze-linux-user
microblazeel-linux-user mips-linux-user
mips64-linux-user mips64el-linux-user
mipsel-linux-user mipsn32-linux-user
mipsn32el-linux-user nios2-linux-user
or1k-linux-user ppc-linux-user ppc64-linux-user
ppc64abi32-linux-user ppc64le-linux-user
riscv32-linux-user riscv64-linux-user
s390x-linux-user sh4-linux-user sh4eb-linux-user
sparc-linux-user sparc32plus-linux-user
sparc64-linux-user tilegx-linux-user
x86_64-linux-user xtensa-linux-user
xtensaeb-linux-user
--target-list-exclude=LIST exclude a set of targets from the default target-list
Advanced options (experts only):
--source-path=PATH path of source code [/kvm/qemu_src/qemu-4.0.0]
--cross-prefix=PREFIX use PREFIX for compile tools []
--cc=CC use C compiler CC [cc]
--iasl=IASL use ACPI compiler IASL [iasl]
--host-cc=CC use C compiler CC [cc] for code run at
build time
--cxx=CXX use C++ compiler CXX [c++]
--objcc=OBJCC use Objective-C compiler OBJCC [cc]
--extra-cflags=CFLAGS append extra C compiler flags QEMU_CFLAGS
--extra-cxxflags=CXXFLAGS append extra C++ compiler flags QEMU_CXXFLAGS
--extra-ldflags=LDFLAGS append extra linker flags LDFLAGS
--cross-cc-ARCH=CC use compiler when building ARCH guest test cases
--cross-cc-flags-ARCH= use compiler flags when building ARCH guest tests
--make=MAKE use specified make [make]
--install=INSTALL use specified install [install]
--python=PYTHON use specified python [python]
--smbd=SMBD use specified smbd [/usr/sbin/smbd]
--with-git=GIT use specified git [git]
--static enable static build [no]
--mandir=PATH install man pages in PATH
--datadir=PATH install firmware in PATH/qemu
--docdir=PATH install documentation in PATH/qemu
--bindir=PATH install binaries in PATH
--libdir=PATH install libraries in PATH
--libexecdir=PATH install helper binaries in PATH
--sysconfdir=PATH install config in PATH/qemu
--localstatedir=PATH install local state in PATH (set at runtime on win32)
--firmwarepath=PATH search PATH for firmware files
--with-confsuffix=SUFFIX suffix for QEMU data inside datadir/libdir/sysconfdir [/qemu]
--with-pkgversion=VERS use specified string as sub-version of the package
--enable-debug enable common debug build options
--enable-sanitizers enable default sanitizers
--disable-strip disable stripping binaries
--disable-werror disable compilation abort on warning
--disable-stack-protector disable compiler-provided stack protection
--audio-drv-list=LIST set audio drivers list:
Available drivers: oss alsa sdl pa
--block-drv-whitelist=L Same as --block-drv-rw-whitelist=L
--block-drv-rw-whitelist=L
set block driver read-write whitelist
(affects only QEMU, not qemu-img)
--block-drv-ro-whitelist=L
set block driver read-only whitelist
(affects only QEMU, not qemu-img)
--enable-trace-backends=B Set trace backend
Available backends: dtrace ftrace log simple syslog ust
--with-trace-file=NAME Full PATH,NAME of file to store traces
Default:trace-
--disable-slirp disable SLIRP userspace network connectivity
--enable-tcg-interpreter enable TCG with bytecode interpreter (TCI)
--enable-malloc-trim enable libc malloc_trim() for memory optimization
--oss-lib path to OSS library
--cpu=CPU Build for host CPU [x86_64]
--with-coroutine=BACKEND coroutine backend. Supported options:
ucontext, sigaltstack, windows
--enable-gcov enable test coverage analysis with gcov
--gcov=GCOV use specified gcov [gcov]
--disable-blobs disable installing provided firmware blobs
--with-vss-sdk=SDK-path enable Windows VSS support in QEMU Guest Agent
--with-win-sdk=SDK-path path to Windows Platform SDK (to build VSS .tlb)
--tls-priority default TLS protocol/cipher priority string
--enable-gprof QEMU profiling with gprof
--enable-profiler profiler support
--enable-debug-stack-usage
track the maximum stack usage of stacks created by qemu_alloc_stack
Optional features, enabled with --enable-FEATURE and
disabled with --disable-FEATURE, default is enabled if available:
system all system emulation targets
user supported user emulation targets
linux-user all linux usermode emulation targets
bsd-user all BSD usermode emulation targets
docs build documentation
guest-agent build the QEMU Guest Agent
guest-agent-msi build guest agent Windows MSI installation package
pie Position Independent Executables
modules modules support
debug-tcg TCG debugging (default is disabled)
debug-info debugging information
sparse sparse checker
gnutls GNUTLS cryptography support
nettle nettle cryptography support
gcrypt libgcrypt cryptography support
auth-pam PAM access control
sdl SDL UI
sdl_image SDL Image support for icons
gtk gtk UI
vte vte support for the gtk UI
curses curses UI
iconv font glyph conversion support
vnc VNC UI support
vnc-sasl SASL encryption for VNC server
vnc-jpeg JPEG lossy compression for VNC server
vnc-png PNG compression for VNC server
cocoa Cocoa UI (Mac OS X only)
virtfs VirtFS
mpath Multipath persistent reservation passthrough
xen xen backend driver support
xen-pci-passthrough PCI passthrough support for Xen
brlapi BrlAPI (Braile)
curl curl connectivity
membarrier membarrier system call (for Linux 4.14+ or Windows)
fdt fdt device tree
bluez bluez stack connectivity
kvm KVM acceleration support
hax HAX acceleration support
hvf Hypervisor.framework acceleration support
whpx Windows Hypervisor Platform acceleration support
rdma Enable RDMA-based migration
pvrdma Enable PVRDMA support
vde support for vde network
netmap support for netmap network
linux-aio Linux AIO support
cap-ng libcap-ng support
attr attr and xattr support
vhost-net vhost-net kernel acceleration support
vhost-vsock virtio sockets device support
vhost-scsi vhost-scsi kernel target support
vhost-crypto vhost-user-crypto backend support
vhost-kernel vhost kernel backend support
vhost-user vhost-user backend support
spice spice
rbd rados block device (rbd)
libiscsi iscsi support
libnfs nfs support
smartcard smartcard support (libcacard)
libusb libusb (for usb passthrough)
live-block-migration Block migration in the main migration stream
usb-redir usb network redirection support
lzo support of lzo compression library
snappy support of snappy compression library
bzip2 support of bzip2 compression library
(for reading bzip2-compressed dmg images)
lzfse support of lzfse compression library
(for reading lzfse-compressed dmg images)
seccomp seccomp support
coroutine-pool coroutine freelist (better performance)
glusterfs GlusterFS backend
tpm TPM support
libssh2 ssh block device support
numa libnuma support
libxml2 for Parallels image format
tcmalloc tcmalloc support
jemalloc jemalloc support
avx2 AVX2 optimization support
replication replication support
opengl opengl support
virglrenderer virgl rendering support
xfsctl xfsctl support
qom-cast-debug cast debugging support
tools build qemu-io, qemu-nbd and qemu-img tools
vxhs Veritas HyperScale vDisk backend support
bochs bochs image format support
cloop cloop image format support
dmg dmg image format support
qcow1 qcow v1 image format support
vdi vdi image format support
vvfat vvfat image format support
qed qed image format support
parallels parallels image format support
sheepdog sheepdog block driver support
crypto-afalg Linux AF_ALG crypto backend driver
capstone capstone disassembler support
debug-mutex mutex debugging support
libpmem libpmem support
NOTE: The object files are built at the place where configure is launched
根据实际需求启用或禁用相关配置项，配置命令如下：
./configure --prefix=/usr \
--enable-kvm \
--enable-libusb \
--enable-usb-redir \
--enable-debug \
--enable-debug-info \
--enable-curl \
--enable-sdl \
--enable-vhost-net \
--enable-spice \
--enable-vnc \
--enable-opengl \
--enable-gtk \
--target-list=x86_64-softmmu
可能需要根据提示信息安装相应的依赖包，参见 3.3.1 Build Qemu for KVMGT | gvt-linux
CentOS 7 的 yum 包：
yum install SDL2-devel libcurl-devel
Ubuntu 18.04 的 apt 包：
apt-get install libsdl2-dev libcurl4-openssl-dev libusbredirhost-dev
若相关依赖均已安装，则可成功配置，终端输出如下所示：
Install prefix /usr
BIOS directory /usr/share/qemu
firmware path /usr/share/qemu-firmware
binary directory /usr/bin
library directory /usr/lib
module directory /usr/lib/qemu
libexec directory /usr/libexec
include directory /usr/include
config directory /usr/etc
local state directory /usr/var
Manual directory /usr/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /kvm/qemu_src/qemu-4.0.50
GIT binary git
GIT submodules ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc capstone
C compiler cc
Host C compiler cc
C++ compiler c++
Objective-C compiler cc
ARFLAGS rv
CFLAGS -g
QEMU_CFLAGS -I/usr/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -Werror -pthread -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -std=gnu99 -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/include/libpng16 -I/usr/include/spice-server -I/usr/include/spice-1 -I$(SRC_PATH)/capstone/include
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
QEMU_LDFLAGS -L$(BUILD_DIR)/dtc/libfdt
make make
install install
python python -B (2.7.15rc1)
slirp support internal
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu
gprof enabled no
sparse enabled no
strip binaries no
profiler no
static build no
SDL support yes (2.0.8)
SDL image support no
GTK support yes (3.22.30)
GTK GL support yes
VTE support no
TLS priority NORMAL
GNUTLS support no
libgcrypt no
nettle no
libtasn1 no
PAM no
iconv support yes
curses support no
virgl support no
curl support yes
mingw32 support no
Audio drivers pa oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support yes
ATTR/XATTR support yes
Install blobs yes
KVM support yes
HAX support no
HVF support no
WHPX support no
TCG support yes
TCG debug enabled yes
TCG interpreter no
malloc trim support yes
RDMA support no
PVRDMA support no
fdt support git
membarrier no
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
posix_memalign yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends log
spice support yes (0.12.13/0.14.0)
rbd support no
xfsctl support no
smartcard support no
libusb yes
usb net redir yes
OpenGL support yes
OpenGL dmabufs yes
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
mutex debugging yes
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough
TPM emulator
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support no
lzfse support no
NUMA host support no
libxml2 yes
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
bochs support yes
cloop support yes
dmg support yes
qcow v1 support yes
vdi support yes
vvfat support yes
qed support yes
parallels support yes
sheepdog support yes
capstone git
docker yes
libpmem support no
libudev yes
default devices yes
NOTE: cross-compilers enabled: 'cc'
在配置完以后，当前目录
qemu-4.0.50下会生成config-host.mak和config.status文件：
config-host.mak：可查看上述./configure之后的配置结果，会在后续make中被引用config.status：便于后续要重新configure时，只要执行./config.status，就可以恢复上一次的配置
编译 QEMU
经过配置之后，编译就很简单了，直接执行make命令：
qemu-4.0.50 > make -j 16
qemu-4.0.50 > cd x86_64-softmmu
x86_64-softmmu > ls
accel config-devices.mak cpus.d dump.o gdbstub.o hmp-commands-info.h memory.d monitor.d qemu-system-x86_64 trace
arch_init.d config-devices.mak.old cpus.o exec.d gdbstub-xml.c hw memory_mapping.d monitor.o qtest.d win_dump.d
arch_init.o config-target.h disas.d exec.o gdbstub-xml.d ioport.d memory_mapping.o numa.d qtest.o win_dump.o
balloon.d config-target.h-timestamp disas.o fpu gdbstub-xml.o ioport.o memory.o numa.o target
balloon.o config-target.mak dump.d gdbstub.d hmp-commands.h Makefile migration qapi tcg
x86_64-softmmu > ./qemu-system-x86_64 --version
QEMU emulator version 4.0.50 (v4.0.0-142-ge0fb2c3d89-dirty)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
为了支持KVMGT，还需单独编译seabios，之后会在out目录下生成bios.bin文件：
qemu-4.0.50 > cd roms/seabios
seabios > make -j 16
seabios > ls ./out
asm-offsets.h bios.bin.prep ccode16.o.tmp.c code16.o code32seg.d include rom32seg.strip.o romlayout.d scripts vgasrc
autoconf.h bios.bin.raw ccode32flat.d code16.o.objdump code32seg.o rom16.o romlayout16.lds romlayout.o src
autoversion.h ccode16.d ccode32flat.o code32flat.o code32seg.o.objdump rom16.strip.o romlayout32flat.lds rom.o version.d
bios.bin ccode16.o ccode32flat.o.tmp.c code32flat.o.objdump code32seg.o.tmp.c rom32seg.o romlayout32seg.lds rom.o.objdump version.o
4.3 安装 QEMU
编译完成后，运行make install命令即可安装 QEMU：
qemu-4.0.50 > make install
qemu-4.0.50 > ls /usr/share/qemu
acpi-dsdt.aml edk2-x86_64-code.fd init ppc_rom.bin qemu-icon.bmp trace-events
bamboo.dtb edk2-x86_64-secure-code.fd keymaps pvh.bin qemu_logo_no_text.svg trace-events-all
bios-256k.bin efi-e1000e.rom kvmvapic.bin pxe-e1000.rom QEMU,tcx.bin u-boot.e500
bios.bin efi-e1000.rom linuxboot.bin pxe-eepro100.rom QEMU,VGA.bin u-boot-sam460-20100605.bin
canyonlands.dtb efi-eepro100.rom linuxboot_dma.bin pxe-ne2k_isa.rom qemu_vga.ndrv vgabios.bin
edk2-aarch64-code.fd efi-ne2k_pci.rom multiboot.bin pxe-ne2k_pci.rom s390-ccw.img vgabios-bochs-display.bin
edk2-arm-code.fd efi-pcnet.rom openbios-ppc pxe-pcnet32.rom s390-netboot.img vgabios-cirrus.bin
edk2-arm-vars.fd efi-rtl8139.rom openbios-sparc32 pxe-pcnet.rom s390-zipl.rom vgabios-qxl.bin
edk2-i386-code.fd efi-virtio.rom openbios-sparc64 pxe-rtl8139.rom sgabios.bin vgabios-ramfb.bin
edk2-i386-secure-code.fd efi-vmxnet3.rom palcode-clipper pxe-virtio.rom skiboot.lid vgabios-stdvga.bin
edk2-i386-vars.fd firmware petalogix-ml605.dtb q35-acpi-dsdt.aml slof.bin vgabios-virtio.bin
edk2-licenses.txt hppa-firmware.img petalogix-s3adsp1800.dtb QEMU,cgthree.bin spapr-rtas.bin vgabios-vmware.bin
qemu-4.0.50 > ls /usr/share/qemu/firmware
50-edk2-i386-secure.json 50-edk2-x86_64-secure.json 60-edk2-aarch64.json 60-edk2-arm.json 60-edk2-i386.json 60-edk2-x86_64.json
qemu-4.0.50 > ls /usr/share/qemu/keymaps
ar common da de-ch en-us et fo fr-be fr-ch hu it lt mk nl no pt ru sv tr
bepo cz de en-gb es fi fr fr-ca hr is ja lv modifiers nl-be pl pt-br sl th
另外还需将编译后的bios.bin拷贝至/usr/bin目录：
qemu-4.0.50 > cp roms/seabios/out/bios.bin /usr/bin/bios.bin
QEMU 的安装过程主要有以下几个任务：
- 创建 QEMU 的一些目录
- 复制一些配置文件到相应的目录下
- 复制一些固件文件（如
sgabios.bin、kvmvapic.bin）到目录下，以便 QEMU 命令行启动时可以找到对应的固件供客户机使用 - 复制
keymaps到相应的目录下，以便在客户机中支持各种所需的键盘类型 - 复制
qemu-system-x86_64、qemu-img等可执行程序到对应的目录下
> ls /usr/share/qemu
acpi-dsdt.aml edk2-x86_64-code.fd init ppc_rom.bin qemu-icon.bmp trace-events
bamboo.dtb edk2-x86_64-secure-code.fd keymaps pvh.bin qemu_logo_no_text.svg trace-events-all
bios-256k.bin efi-e1000e.rom kvmvapic.bin pxe-e1000.rom QEMU,tcx.bin u-boot.e500
bios.bin efi-e1000.rom linuxboot.bin pxe-eepro100.rom QEMU,VGA.bin u-boot-sam460-20100605.bin
canyonlands.dtb efi-eepro100.rom linuxboot_dma.bin pxe-ne2k_isa.rom qemu_vga.ndrv vgabios.bin
edk2-aarch64-code.fd efi-ne2k_pci.rom multiboot.bin pxe-ne2k_pci.rom s390-ccw.img vgabios-bochs-display.bin
edk2-arm-code.fd efi-pcnet.rom openbios-ppc pxe-pcnet32.rom s390-netboot.img vgabios-cirrus.bin
edk2-arm-vars.fd efi-rtl8139.rom openbios-sparc32 pxe-pcnet.rom s390-zipl.rom vgabios-qxl.bin
edk2-i386-code.fd efi-virtio.rom openbios-sparc64 pxe-rtl8139.rom sgabios.bin vgabios-ramfb.bin
edk2-i386-secure-code.fd efi-vmxnet3.rom palcode-clipper pxe-virtio.rom skiboot.lid vgabios-stdvga.bin
edk2-i386-vars.fd firmware petalogix-ml605.dtb q35-acpi-dsdt.aml slof.bin vgabios-virtio.bin
edk2-licenses.txt hppa-firmware.img petalogix-s3adsp1800.dtb QEMU,cgthree.bin spapr-rtas.bin vgabios-vmware.bin
> ls /usr/share/qemu/keymaps
ar common da de-ch en-us et fo fr-be fr-ch hu it lt mk nl no pt ru sv tr
bepo cz de en-gb es fi fr fr-ca hr is ja lv modifiers nl-be pl pt-br sl th
由于 QEMU 是用户空间的程序，所以安装之后不需要重启系统，直接使用
qemu-system-x86_64、qemu-img这样的命令行工具就可以了
5. 安装客户机
安装客户机前，需要先创建一个镜像文件来存储客户机中的系统和文件:
> qemu-img create -f raw fedora30.raw 20G # 指定为 raw 格式
Formatting 'fedora30.raw', fmt=raw size=21474836480
> qemu-img info fedora30.raw
image: fedora30.raw
file format: raw
virtual size: 20G (21474836480 bytes)
disk size: 0
可以看到目前fedora30.raw并不占用任何磁盘空间，这是因为qemu-img默认的方式是按需分配，镜像文件的大小会随着实际的使用而增大。
可在qemu-img命令中添加-o preallocation=full选项，使得镜像在创建时分配全部的空间，不过这样的话格式化过程就会比较耗时：
> qemu-img create -f raw fedora30-full.raw 10G -o preallocation=full
Formatting 'fedora30-full.raw', fmt=raw size=10737418240 preallocation='full'
> qemu-img info fedora30-full.raw
image: fedora30-full.raw
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: 10G
最后即可使用以下命令启动 KVM 客户机，并安装系统：
> qemu-system-x86_64 -accel kvm \
-m 2G -smp 2 -boot once=d \
-cdrom /kvm/iso/Fedora-Server-dvd-x86_64-30-1.2.iso \
-drive file=/kvm/image/fedora30.raw,format=raw
