QEMU 3.1.0 源码学习,更新中…

To be updated…

目录

1. QEMU 迁移

摘自qemu-3.1.0/docs/devel/migration.rst

1.1 Migration

QEMU 中关于保存/恢复正在运行客户机的状态的代码,有两个相对应的操作:

  • Saving the state
  • Restoring a guest

因此,QEMU 需要在目的宿主机上以相同的参数启动客户机,并且客户机所拥有的设备需要与迁移前保存时所拥有的设备保持一致

当我们可以保存/恢复客户机之后,还需要另一项功能,即迁移Migration

迁移意味着源宿主机上运行的 QEMU 可以被迁移至目标宿主机继续运行

KVM 虚拟机的迁移又可分为以下两种:

  • 静态迁移static migration,又称冷迁移cold migration
  • 动态迁移live migration,又称热迁移hot migration

其中动态迁移值得更多关注,因为运行中的客户机有很多的状态(例如RAM),而动态迁移可以保证客户机在保持运行的情况下,将这些状态一并迁移至目标宿主机

当然,客户机并不是真的一直处于运行态。当客户机所有的相关数据都已迁移至目标宿主机后,源宿主机上的客户机就会停止运行。而在目标宿主机上的客户机重新运行之前,还会有一段停机时间service down-time,通常情况下为几百毫秒以内

1.2 Transports

迁移的数据流一般都是字节流byte stream,可以通过常见的协议进行传递:

  • tcp migration:使用 TCP 套接字TCP Sockets完成迁移
  • unix migration:使用 UNIX 套接字UNIX Sockets完成迁移
  • exec migration:使用进程的标准输入/输出stdin/stdout完成迁移
  • fd migration:使用传递给 QEMU 的文件描述符fd完成迁移,并且 QEMU 不需要关心这个fd是如何打开的

In addition, support is included for migration using RDMA, which transports the page data using RDMA, where the hardware takes care of transporting the pages, and the load on the CPU is much lower. While the internals of RDMA migration are a bit different, this isn’t really visible outside the RAM migration code.

所有的迁移协议使用相同的infrastructure来保存/恢复虚拟机的设备。

1.3 Common infrastructure

持有迁移数据流文件、套接字sockets、文件描述符fd都抽象在migration/qemu-file.h中的QEMUFile结构体中。

结构体QEMUFileqemu-file.c中的定义如下:

#define IOV_MAX 1024 /* 定义在 include/qemu/osdep.h 中 */
...

#define IO_BUF_SIZE 32768
#define MAX_IOV_SIZE MIN(IOV_MAX, 64)

struct QEMUFile {
    const QEMUFileOps *ops;
    const QEMUFileHooks *hooks;
    void *opaque;

    int64_t bytes_xfer;
    int64_t xfer_limit;

    int64_t pos; /* start of buffer when writing, end of buffer
                    when reading */
    int buf_index;
    int buf_size; /* 0 when writing */
    uint8_t buf[IO_BUF_SIZE];

    DECLARE_BITMAP(may_free, MAX_IOV_SIZE);
    struct iovec iov[MAX_IOV_SIZE];
    unsigned int iovcnt;

    int last_error;
};

结构体QIOChannelinclude/io/channel.h中的定义如下:

/**
 * QIOChannel:
 *
 * The QIOChannel defines the core API for a generic I/O channel
 * class hierarchy. It is inspired by GIOChannel, but has the
 * following differences
 *
 *  - Use QOM to properly support arbitrary subclassing
 *  - Support use of iovecs for efficient I/O with multiple blocks
 *  - None of the character set translation, binary data exclusively
 *  - Direct support for QEMU Error object reporting
 *  - File descriptor passing
 *
 * This base class is abstract so cannot be instantiated. There
 * will be subclasses for dealing with sockets, files, and higher
 * level protocols such as TLS, WebSocket, etc.
 */

struct QIOChannel {
    Object parent;
    unsigned int features; /* bitmask of QIOChannelFeatures */
    char *name;
    AioContext *ctx;
    Coroutine *read_coroutine;
    Coroutine *write_coroutine;
#ifdef _WIN32
    HANDLE event; /* For use with GSource on Win32 */
#endif
};

大多数情况下,QEMUFile都和QIOChannelsubtype相互关联,例如QIOChannelTLSQIOChannelFileQIOChannelSocket

1.4 Saving the state of one device

对于大多数的设备,只需要调用一次common infrastructure即可,这些被称为non-iterative devices。这些设备的数据precopy migration预拷贝迁移阶段的最后被传送,此时虚拟机的 CPU 处于暂停状态

而对于iterative devices包含的数据量很大,例如内存RAMlarge tables

General advice for device developers

VMState

大部分的设备数据可以使用include/migration/vmstate.h中的VMSTATE宏定义来描述

结构体VMStateDescriptioninclude/migration/vmstate.h中定义如下:

struct VMStateDescription {
    const char *name;
    int unmigratable;
    int version_id;
    int minimum_version_id;
    int minimum_version_id_old;
    MigrationPriority priority;
    LoadStateHandler *load_state_old;
    int (*pre_load)(void *opaque);
    int (*post_load)(void *opaque, int version_id);
    int (*pre_save)(void *opaque);
    bool (*needed)(void *opaque);
    const VMStateField *fields;
    const VMStateDescription **subsections;
};

而在hw/input/pckbd.c中,vmstate_kdb定义如下:

static const VMStateDescription vmstate_kbd = {
    .name = "pckbd",
    .version_id = 3,
    .minimum_version_id = 3,
    .post_load = kbd_post_load,
    .fields = (VMStateField[]) {
        VMSTATE_UINT8(write_cmd, KBDState),
        VMSTATE_UINT8(status, KBDState),
        VMSTATE_UINT8(mode, KBDState),
        VMSTATE_UINT8(pending, KBDState),
        VMSTATE_END_OF_LIST()
    },
    .subsections = (const VMStateDescription*[]) {
        &vmstate_kbd_outport,
        NULL
    }
};
Legacy way

VMState相对应的是 QEMU 早期的实现方式:每个被迁移的设备需要注册两个函数,一个用来保存状态,另一个用来恢复状态

函数register_savevm_livemigration/savevm.c中的定义如下:

/* TODO: Individual devices generally have very little idea about the rest
   of the system, so instance_id should be removed/replaced.
   Meanwhile pass -1 as instance_id if you do not already have a clearly
   distinguishing id for all instances of your device class. */
int register_savevm_live(DeviceState *dev,
                         const char *idstr,
                         int instance_id,
                         int version_id,
                         SaveVMHandlers *ops,
                         void *opaque)
{
    SaveStateEntry *se;

    se = g_new0(SaveStateEntry, 1);
    se->version_id = version_id;
    se->section_id = savevm_state.global_section_id++;
    se->ops = ops;
    se->opaque = opaque;
    se->vmsd = NULL;
    /* if this is a live_savem then set is_ram */
    if (ops->save_setup != NULL) {
        se->is_ram = 1;
    }

    if (dev) {
        char *id = qdev_get_dev_path(dev);
        if (id) {
            if (snprintf(se->idstr, sizeof(se->idstr), "%s/", id) >=
                sizeof(se->idstr)) {
                error_report("Path too long for VMState (%s)", id);
                g_free(id);
                g_free(se);

                return -1;
            }
            g_free(id);

            se->compat = g_new0(CompatEntry, 1);
            pstrcpy(se->compat->idstr, sizeof(se->compat->idstr), idstr);
            se->compat->instance_id = instance_id == -1 ?
                         calculate_compat_instance_id(idstr) : instance_id;
            instance_id = -1;
        }
    }
    pstrcat(se->idstr, sizeof(se->idstr), idstr);

    if (instance_id == -1) {
        se->instance_id = calculate_new_instance_id(se->idstr);
    } else {
        se->instance_id = instance_id;
    }
    assert(!se->compat || se->instance_id == 0);
    savevm_state_handler_insert(se);
    return 0;
}

ops是一个指向SaveVMHanlers的指针对象,结构体SaveVMHandlersinclude/migration/register.h中的定义如下:

typedef struct SaveVMHandlers {
    /* This runs inside the iothread lock.  */
    SaveStateHandler *save_state;

    void (*save_cleanup)(void *opaque);
    int (*save_live_complete_postcopy)(QEMUFile *f, void *opaque);
    int (*save_live_complete_precopy)(QEMUFile *f, void *opaque);

    /* This runs both outside and inside the iothread lock.  */
    bool (*is_active)(void *opaque);
    bool (*has_postcopy)(void *opaque);

    /* is_active_iterate
     * If it is not NULL then qemu_savevm_state_iterate will skip iteration if
     * it returns false. For example, it is needed for only-postcopy-states,
     * which needs to be handled by qemu_savevm_state_setup and
     * qemu_savevm_state_pending, but do not need iterations until not in
     * postcopy stage.
     */
    bool (*is_active_iterate)(void *opaque);

    /* This runs outside the iothread lock in the migration case, and
     * within the lock in the savevm case.  The callback had better only
     * use data that is local to the migration thread or protected
     * by other locks.
     */
    int (*save_live_iterate)(QEMUFile *f, void *opaque);

    /* This runs outside the iothread lock!  */
    int (*save_setup)(QEMUFile *f, void *opaque);
    void (*save_live_pending)(QEMUFile *f, void *opaque,
                              uint64_t threshold_size,
                              uint64_t *res_precopy_only,
                              uint64_t *res_compatible,
                              uint64_t *res_postcopy_only);
    /* Note for save_live_pending:
     * - res_precopy_only is for data which must be migrated in precopy phase
     *     or in stopped state, in other words - before target vm start
     * - res_compatible is for data which may be migrated in any phase
     * - res_postcopy_only is for data which must be migrated in postcopy phase
     *     or in stopped state, in other words - after source vm stop
     *
     * Sum of res_postcopy_only, res_compatible and res_postcopy_only is the
     * whole amount of pending data.
     */


    LoadStateHandler *load_state;
    int (*load_setup)(QEMUFile *f, void *opaque);
    int (*load_cleanup)(void *opaque);
    /* Called when postcopy migration wants to resume from failure */
    int (*resume_prepare)(MigrationState *s, void *opaque);
} SaveVMHandlers;

可以看到有以下两个指针对象:

typedef struct SaveVMHandlers {
    /* This runs inside the iothread lock.  */
    SaveStateHandler *save_state;
    ...
    LoadStateHandler *load_state;
    ...
} SaveVMHandlers;

而在include/qemu/typedefs.h中:

typedef void SaveStateHandler(QEMUFile *f, void *opaque);
typedef int LoadStateHandler(QEMUFile *f, void *opaque, int version_id);

值得注意的是:load_state需要接收version_id作为参数,以便确认正在接收的状态数据的格式。而save_state不需要version_id参数,因为它总是会保存最新的状态

QEMU 之后应该会逐渐用VMState的方式来替代现有的VMState macros

Note that because the VMState macros still save the data in a raw format, in many cases it’s possible to replace legacy code with a carefully constructed VMState description that matches the byte layout of the existing code.

未完待续…

2. QEMU Detailed Study

摘自 QEMU Detailed Study | PDF

先看一张图:

QEMU 作为设备模拟器,可以模拟多种处理器架构。其中,待模拟的架构称为Target,而 QEMU 运行的系统环境称为Host

QEMU 中有一个模块叫做Tiny Code Generator,简称TCG,负责Target Code动态的翻译为Host Code,也即TCG Target

因此我们也可以将在模拟处理器上运行的代码 (OS + UserTools) 称为Guest CodeQEMU 的作用就是Guest Code提取出来,并将其转换为Host Specific Code

2.1 源码基本结构

启动过程

QEMU 的启动过程涉及以下几个重要的源文件:

  • vl.c
  • cpus.c
  • exec.c
  • cpu-exec.c

vl.c中定义了启动入口main函数,负责根据传入的参数例如RAMCPUdevices来建立虚拟机的运行环境。CPU 的执行也是从此处开始的。

硬件设备的模拟

所有与虚拟硬件设备相关的代码都在hw/目录下。

Guest (Target) 定义

target/目录下:

> pwd
/kvm/qemu-src/qemu-3.1.0/target

> ll
total 28K
drwxr-xr-x 2 ibm ibm  269 Dec 12  2018 alpha
drwxr-xr-x 2 ibm ibm 4.0K Dec 12  2018 arm
drwxr-xr-x 2 ibm ibm  296 Dec 12  2018 cris
drwxr-xr-x 2 ibm ibm  214 Dec 12  2018 hppa
drwxr-xr-x 3 ibm ibm 4.0K Dec 12  2018 i386
drwxr-xr-x 2 ibm ibm  219 Dec 12  2018 lm32
drwxr-xr-x 2 ibm ibm  299 Dec 12  2018 m68k
drwxr-xr-x 2 ibm ibm  210 Dec 12  2018 microblaze
drwxr-xr-x 2 ibm ibm 4.0K Dec 12  2018 mips
drwxr-xr-x 2 ibm ibm  164 Dec 12  2018 moxie
drwxr-xr-x 2 ibm ibm  166 Dec 12  2018 nios2
drwxr-xr-x 2 ibm ibm  319 Dec 12  2018 openrisc
drwxr-xr-x 3 ibm ibm 4.0K Dec 12  2018 ppc
drwxr-xr-x 2 ibm ibm  243 Dec 12  2018 riscv
drwxr-xr-x 2 ibm ibm 4.0K Dec 12  2018 s390x
drwxr-xr-x 2 ibm ibm  192 Dec 12  2018 sh4
drwxr-xr-x 2 ibm ibm 4.0K Dec 12  2018 sparc
drwxr-xr-x 2 ibm ibm  168 Dec 12  2018 tilegx
drwxr-xr-x 2 ibm ibm  223 Dec 12  2018 tricore
drwxr-xr-x 2 ibm ibm  179 Dec 12  2018 unicore32
drwxr-xr-x 8 ibm ibm 4.0K Dec 12  2018 xtensa
Host (TCG) 定义

tcg目录下:

> pwd
/kvm/qemu-src/qemu-3.1.0/tcg

> ll
total 576K
drwxr-xr-x 2 ibm ibm   74 Dec 12  2018 aarch64
drwxr-xr-x 2 ibm ibm   50 Dec 12  2018 arm
drwxr-xr-x 2 ibm ibm   74 Dec 12  2018 i386
-rw-r--r-- 1 ibm ibm  146 Dec 12  2018 LICENSE
drwxr-xr-x 2 ibm ibm   50 Dec 12  2018 mips
-rw-r--r-- 1 ibm ibm  48K Dec 12  2018 optimize.c
drwxr-xr-x 2 ibm ibm   50 Dec 12  2018 ppc
-rw-r--r-- 1 ibm ibm  22K Dec 12  2018 README
drwxr-xr-x 2 ibm ibm   50 Dec 12  2018 s390
drwxr-xr-x 2 ibm ibm   50 Dec 12  2018 sparc
-rw-r--r-- 1 ibm ibm 122K Dec 12  2018 tcg.c
-rw-r--r-- 1 ibm ibm 1.6K Dec 12  2018 tcg-common.c
-rw-r--r-- 1 ibm ibm 1.8K Dec 12  2018 tcg-gvec-desc.h
-rw-r--r-- 1 ibm ibm  47K Dec 12  2018 tcg.h
-rw-r--r-- 1 ibm ibm 3.0K Dec 12  2018 tcg-ldst.inc.c
-rw-r--r-- 1 ibm ibm 2.0K Dec 12  2018 tcg-mo.h
-rw-r--r-- 1 ibm ibm  94K Dec 12  2018 tcg-op.c
-rw-r--r-- 1 ibm ibm  11K Dec 12  2018 tcg-opc.h
-rw-r--r-- 1 ibm ibm  72K Dec 12  2018 tcg-op-gvec.c
-rw-r--r-- 1 ibm ibm  15K Dec 12  2018 tcg-op-gvec.h
-rw-r--r-- 1 ibm ibm  49K Dec 12  2018 tcg-op.h
-rw-r--r-- 1 ibm ibm  11K Dec 12  2018 tcg-op-vec.c
-rw-r--r-- 1 ibm ibm 5.2K Dec 12  2018 tcg-pool.inc.c
drwxr-xr-x 2 ibm ibm   64 Dec 12  2018 tci
-rw-r--r-- 1 ibm ibm  39K Dec 12  2018 tci.c
-rw-r--r-- 1 ibm ibm  394 Dec 12  2018 TODO
总结一下
  1. /vl.c:包含了main函数,负责启动虚拟机、运行 vCPU。main_loop()也存在于此文件中,虚拟机的转换是在这个循环中进行调用的
  2. /target/i386/translate.c:负责提取Guest Code并将其转换为平台无关的TCG ops。转换过程中的单位元是一个TBTranslation Block,只有当一个TB的转换执行结束后,才会轮到下一个TB。这里是 TCG 的前端
  3. /tcg/tcg.c:TCG 的主要实现代码,这里是 TCG 的后端
  4. /tcg/i386/tcg-target.inc.c:将TCG ops转换为Host Code
  5. /accel/tcg/cpu-exec.c:函数int cpu_exec(CPUState *cpu)会调用函数tb_find()在 Code Buffer 中查找下一个 TB,这里的 TB 指的是已经翻译成 Host 相关指令的 TB。如果找到了,就会调用cpu_loop_exec_tb()来在 Host 上执行

QEMU 的作用就是,提取Guest Code,并将其转换为Host Code

整个转换过程由两部分组成:

第一步前端完成,Target Code的代码块TB被转换成TCG-ops(独立于机器的中间代码)

第二步后端完成,利用 Host 架构对应的TCG,把TB生成的TCP-ops转换成Host Code

2.2 main 流程分析

main(…)

定义在/vl.c中,函数原型如下:

int main(int argc, char **argv, char **envp);

入口main函数,解析传入 QEMU 的命令行参数,并以此初始化 VM,例如内存大小、磁盘大小、启动盘等。

3. QEMU 2.12.1 迁移过程分析

3.1 迁移源端 QEMU

hmp-commands.hx中可以看到migrate命令对应的入口函数hmp_migrate

ETEXI

    {
        .name       = "migrate",
        .args_type  = "detach:-d,blk:-b,inc:-i,resume:-r,uri:s",
        .params     = "[-d] [-b] [-i] [-r] uri",
        .help       = "migrate to URI (using -d to not wait for completion)"
              "\n\t\t\t -b for migration without shared storage with"
              " full copy of disk\n\t\t\t -i for migration without "
              "shared storage with incremental copy of disk "
              "(base image shared between src and destination)"
                      "\n\t\t\t -r to resume a paused migration",
        .cmd        = hmp_migrate,
    },


STEXI
@item migrate [-d] [-b] [-i] @var{uri}
@findex migrate
Migrate to @var{uri} (using -d to not wait for completion).
    -b for migration with full copy of disk
    -i for migration with incremental copy of disk (base image is shared)
ETEXI

函数hmp_migratehmp.c中定义:

void hmp_migrate(Monitor *mon, const QDict *qdict)
{
    bool detach = qdict_get_try_bool(qdict, "detach", false);
    bool blk = qdict_get_try_bool(qdict, "blk", false);
    bool inc = qdict_get_try_bool(qdict, "inc", false);
    const char *uri = qdict_get_str(qdict, "uri");
    Error *err = NULL;

    qmp_migrate(uri, !!blk, blk, !!inc, inc, false, false, &err);
    if (err) {
        hmp_handle_error(mon, &err);
        return;
    }

    if (!detach) {
        HMPMigrationStatus *status;

        if (monitor_suspend(mon) < 0) {
            monitor_printf(mon, "terminal does not allow synchronous "
                           "migration, continuing detached\n");
            return;
        }

        status = g_malloc0(sizeof(*status));
        status->mon = mon;
        status->is_block_migration = blk || inc;
        status->timer = timer_new_ms(QEMU_CLOCK_REALTIME, hmp_migrate_status_cb,
                                          status);
        timer_mod(status->timer, qemu_clock_get_ms(QEMU_CLOCK_REALTIME));
    }
}

进行迁移逻辑处理的函数跳转到了qmp_migrate,在migration.c中定义:

void qmp_migrate(const char *uri, bool has_blk, bool blk,
                 bool has_inc, bool inc, bool has_detach, bool detach,
                 Error **errp)
{
    Error *local_err = NULL;
    MigrationState *s = migrate_get_current();
    const char *p;

    if (migration_is_setup_or_active(s->state) ||
        s->state == MIGRATION_STATUS_CANCELLING ||
        s->state == MIGRATION_STATUS_COLO) {
        error_setg(errp, QERR_MIGRATION_ACTIVE);
        return;
    }
    if (runstate_check(RUN_STATE_INMIGRATE)) {
        error_setg(errp, "Guest is waiting for an incoming migration");
        return;
    }

    if (migration_is_blocked(errp)) {
        return;
    }

    if ((has_blk && blk) || (has_inc && inc)) {
        if (migrate_use_block() || migrate_use_block_incremental()) {
            error_setg(errp, "Command options are incompatible with "
                       "current migration capabilities");
            return;
        }
        migrate_set_block_enabled(true, &local_err);
        if (local_err) {
            error_propagate(errp, local_err);
            return;
        }
        s->must_remove_block_options = true;
    }

    if (has_inc && inc) {
        migrate_set_block_incremental(s, true);
    }

    migrate_init(s);

    if (strstart(uri, "tcp:", &p)) {
        tcp_start_outgoing_migration(s, p, &local_err);
#ifdef CONFIG_RDMA
    } else if (strstart(uri, "rdma:", &p)) {
        rdma_start_outgoing_migration(s, p, &local_err);
#endif
    } else if (strstart(uri, "exec:", &p)) {
        exec_start_outgoing_migration(s, p, &local_err);
    } else if (strstart(uri, "unix:", &p)) {
        unix_start_outgoing_migration(s, p, &local_err);
    } else if (strstart(uri, "fd:", &p)) {
        fd_start_outgoing_migration(s, p, &local_err);
    } else {
        error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "uri",
                   "a valid migration protocol");
        migrate_set_state(&s->state, MIGRATION_STATUS_SETUP,
                          MIGRATION_STATUS_FAILED);
        block_cleanup_parameters(s);
        return;
    }

    if (local_err) {
        migrate_fd_error(s, local_err);
        error_propagate(errp, local_err);
        return;
    }
}

简单说下这个函数:首先通过migrate_get_current()获取当前的MigrationState指针对象,之后检查当前是否已经有迁移进程存在。之后下面的语句:

if (migration_is_blocked(errp)) {
    return;
}

...

/* migration.c 中定义 */
bool migration_is_blocked(Error **errp)
{
    if (qemu_savevm_state_blocked(errp)) {
        return true;
    }

    if (migration_blockers) {
        error_propagate(errp, error_copy(migration_blockers->data));
        return true;
    }

    return false;
}

这里通过qemu_savevm_state_blocked()来判断当前虚拟机状态适不适合进行迁移。


未完待续…

参考文章

  1. QEMU
  2. QEMU Wiki
  3. KVM
  4. kvm/kvm.git
  5. KVM Documents
  6. Virt Tools | Blogging about open source virtualization
  7. Migration | KVM
  8. QEMU Detailed Study | PDF
  9. QEMU vl.c 源码学习 | CSDN
  10. QEMU 参数解析 | CSDN
  11. qemu-kvm 部分流程/源代码分析 | CSDN
  12. qemu 学习(一)———— qemu 整体流程解读 | CSDN
  13. qemu 学习(二)———— qemu 中对处理器大小端的设置 | CSDN
  14. qemu 学习(三)———— qemu 中反汇编操作解析 | CSDN
  15. QEMU 源码架构 | ChinaUnix
  16. QEMU 源码分析系列(二) | ChinaUnix
  17. QEMU-KVM 虚机动态迁移原理 | 51CTO
  18. 虚拟化在线迁移优化实践(一):KVM虚拟化跨机迁移原理 - UCloud 云计算 | 知乎
  19. QEMU 热迁移简介 | 不忘初心,方得始终
  20. Live Migrating QEMU-KVM Virtual Machines | Red Hat Developer
  21. 虚拟机迁移之热迁移(live_migrate) | 随便写写
  22. 上面文章博主关于虚拟化的文章列表 | 随便写写
  23. KVM 虚拟机静态和动态迁移 | bbsmax
  24. 虚拟机活迁移揭秘 | 博客园
  25. qemu-kvm-1.1.0 源码中关于迁移的代码分析 | CSDN
  26. QEMU 源码分析系列(一)| CSDN
  27. qemu-kvm 虚拟机 live 迁移源代码解读 | CSDN
  28. QEMU live migration 代码简单剖析 | CSDN
  29. qemu-kvm savevm/loadvm 流程 | CSDN
  30. KVM/QEMU 2.3.0 虚拟机动态迁移分析(一)| CSDN
  31. KVM/QEMU 2.3.0 虚拟机动态迁移分析(二)| CSDN
  32. KVM/QEMU 2.3.0 虚拟机动态迁移分析(三)| CSDN
  33. Qemu-KVM 虚拟机初始化及创建过程源码简要分析(一)| CSDN
  34. Qemu-KVM 虚拟机初始化及创建过程源码简要分析(二)| CSDN
  35. qemu-kvm 虚拟机live迁移源代码解读 | CSDN
  36. 北方南方的文章列表 | CSDN
  37. qemu 迁移代码分析 | 随便写写
  38. QEMU live migration 代码简单剖析 | ChinaUnix
  39. qemu-kvm 虚拟机 live 迁移源代码解读 | ChinaUnix
  40. qemu-kvm 磁盘读写的缓冲(cache)的五种模式 | jusonalien
  41. 关于追踪 qemu 源码函数路径的一个方法 | jusonalien
  42. QEMU main 流程分析 | CSDN
  43. QEMU 翻译块(TB)分析 | CSDN
  44. 我见过最全的剖析 QEMU 原理的文章 | CSDN