History

virt: acrn: Introduce I/O request management

An I/O request of a User VM, which is constructed by the hypervisor, is distributed by the ACRN Hypervisor Service Module to an I/O client corresponding to the address range of the I/O request. For each User VM, there is a shared 4-KByte memory region used for I/O requests communication between the hypervisor and Service VM. An I/O request is a 256-byte structure buffer, which is 'struct acrn_io_request', that is filled by an I/O handler of the hypervisor when a trapped I/O access happens in a User VM. ACRN userspace in the Service VM first allocates a 4-KByte page and passes the GPA (Guest Physical Address) of the buffer to the hypervisor. The buffer is used as an array of 16 I/O request slots with each I/O request slot being 256 bytes. This array is indexed by vCPU ID. An I/O client, which is 'struct acrn_ioreq_client', is responsible for handling User VM I/O requests whose accessed GPA falls in a certain range. Multiple I/O clients can be associated with each User VM. There is a special client associated with each User VM, called the default client, that handles all I/O requests that do not fit into the range of any other I/O clients. The ACRN userspace acts as the default client for each User VM. The state transitions of a ACRN I/O request are as follows. FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ... FREE: this I/O request slot is empty PENDING: a valid I/O request is pending in this slot PROCESSING: the I/O request is being processed COMPLETE: the I/O request has been processed An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM and ACRN userspace are in charge of processing the others. The processing flow of I/O requests are listed as following: a) The I/O handler of the hypervisor will fill an I/O request with PENDING state when a trapped I/O access happens in a User VM. b) The hypervisor makes an upcall, which is a notification interrupt, to the Service VM. c) The upcall handler schedules a worker to dispatch I/O requests. d) The worker looks for the PENDING I/O requests, assigns them to different registered clients based on the address of the I/O accesses, updates their state to PROCESSING, and notifies the corresponding client to handle. e) The notified client handles the assigned I/O requests. f) The HSM updates I/O requests states to COMPLETE and notifies the hypervisor of the completion via hypercalls.

Shuo Liu committed 5 years ago

Commit 72f293d1 parent 88f537d7 files · +843 −10Browse files

Showing the changes to include/uapi (1 of 7 files). Show all

Unified · Split

include/uapi/linux/acrn.h+150−0
@@ -14,6 +14,145 @@ · 13 unmodified lines
1414 #include <linux/types.h>
1515 #include <linux/uuid.h>
1616
17+#define ACRN_IO_REQUEST_MAX 16
18+
19+#define ACRN_IOREQ_STATE_PENDING 0
20+#define ACRN_IOREQ_STATE_COMPLETE 1
21+#define ACRN_IOREQ_STATE_PROCESSING 2
22+#define ACRN_IOREQ_STATE_FREE 3
23+
24+#define ACRN_IOREQ_TYPE_PORTIO 0
25+#define ACRN_IOREQ_TYPE_MMIO 1
26+
27+#define ACRN_IOREQ_DIR_READ 0
28+#define ACRN_IOREQ_DIR_WRITE 1
29+
30+/**
31+ * struct acrn_mmio_request - Info of a MMIO I/O request
32+ * @direction: Access direction of this request (ACRN_IOREQ_DIR_*)
33+ * @reserved: Reserved for alignment and should be 0
34+ * @address: Access address of this MMIO I/O request
35+ * @size: Access size of this MMIO I/O request
36+ * @value: Read/write value of this MMIO I/O request
37+ */
38+struct acrn_mmio_request {
39+ __u32 direction;
40+ __u32 reserved;
41+ __u64 address;
42+ __u64 size;
43+ __u64 value;
44+};
45+
46+/**
47+ * struct acrn_pio_request - Info of a PIO I/O request
48+ * @direction: Access direction of this request (ACRN_IOREQ_DIR_*)
49+ * @reserved: Reserved for alignment and should be 0
50+ * @address: Access address of this PIO I/O request
51+ * @size: Access size of this PIO I/O request
52+ * @value: Read/write value of this PIO I/O request
53+ */
54+struct acrn_pio_request {
55+ __u32 direction;
56+ __u32 reserved;
57+ __u64 address;
58+ __u64 size;
59+ __u32 value;
60+};
61+
62+/**
63+ * struct acrn_io_request - 256-byte ACRN I/O request
64+ * @type: Type of this request (ACRN_IOREQ_TYPE_*).
65+ * @completion_polling: Polling flag. Hypervisor will poll completion of the
66+ * I/O request if this flag set.
67+ * @reserved0: Reserved fields.
68+ * @reqs: Union of different types of request. Byte offset: 64.
69+ * @reqs.pio_request: PIO request data of the I/O request.
70+ * @reqs.mmio_request: MMIO request data of the I/O request.
71+ * @reqs.data: Raw data of the I/O request.
72+ * @reserved1: Reserved fields.
73+ * @kernel_handled: Flag indicates this request need be handled in kernel.
74+ * @processed: The status of this request (ACRN_IOREQ_STATE_*).
75+ *
76+ * The state transitions of ACRN I/O request:
77+ *
78+ * FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ...
79+ *
80+ * An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM and
81+ * ACRN userspace are in charge of processing the others.
82+ *
83+ * On basis of the states illustrated above, a typical lifecycle of ACRN IO
84+ * request would look like:
85+ *
86+ * Flow (assume the initial state is FREE)
87+ * |
88+ * | Service VM vCPU 0 Service VM vCPU x User vCPU y
89+ * |
90+ * | hypervisor:
91+ * | fills in type, addr, etc.
92+ * | pauses the User VM vCPU y
93+ * | sets the state to PENDING (a)
94+ * | fires an upcall to Service VM
95+ * |
96+ * | HSM:
97+ * | scans for PENDING requests
98+ * | sets the states to PROCESSING (b)
99+ * | assigns the requests to clients (c)
100+ * V
101+ * | client:
102+ * | scans for the assigned requests
103+ * | handles the requests (d)
104+ * | HSM:
105+ * | sets states to COMPLETE
106+ * | notifies the hypervisor
107+ * |
108+ * | hypervisor:
109+ * | resumes User VM vCPU y (e)
110+ * |
111+ * | hypervisor:
112+ * | post handling (f)
113+ * V sets states to FREE
114+ *
115+ * Note that the procedures (a) to (f) in the illustration above require to be
116+ * strictly processed in the order. One vCPU cannot trigger another request of
117+ * I/O emulation before completing the previous one.
118+ *
119+ * Atomic and barriers are required when HSM and hypervisor accessing the state
120+ * of &struct acrn_io_request.
121+ *
122+ */
123+struct acrn_io_request {
124+ __u32 type;
125+ __u32 completion_polling;
126+ __u32 reserved0[14];
127+ union {
128+ struct acrn_pio_request pio_request;
129+ struct acrn_mmio_request mmio_request;
130+ __u64 data[8];
131+ } reqs;
132+ __u32 reserved1;
133+ __u32 kernel_handled;
134+ __u32 processed;
135+} __attribute__((aligned(256)));
136+
137+struct acrn_io_request_buffer {
138+ union {
139+ struct acrn_io_request req_slot[ACRN_IO_REQUEST_MAX];
140+ __u8 reserved[4096];
141+ };
142+};
143+
144+/**
145+ * struct acrn_ioreq_notify - The structure of ioreq completion notification
146+ * @vmid: User VM ID
147+ * @reserved: Reserved and should be 0
148+ * @vcpu: vCPU ID
149+ */
150+struct acrn_ioreq_notify {
151+ __u16 vmid;
152+ __u16 reserved;
153+ __u32 vcpu;
154+};
155+
17156 /**
18157 * struct acrn_vm_creation - Info to create a User VM
19158 * @vmid: User VM ID returned from the hypervisor
@@ -218,6 +357,17 @@ struct acrn_vm_memmap { · 198 unmodified lines
218357 #define ACRN_IOCTL_SET_VCPU_REGS \
219358 _IOW(ACRN_IOCTL_TYPE, 0x16, struct acrn_vcpu_regs)
220359
360+#define ACRN_IOCTL_NOTIFY_REQUEST_FINISH \
361+ _IOW(ACRN_IOCTL_TYPE, 0x31, struct acrn_ioreq_notify)
362+#define ACRN_IOCTL_CREATE_IOREQ_CLIENT \
363+ _IO(ACRN_IOCTL_TYPE, 0x32)
364+#define ACRN_IOCTL_ATTACH_IOREQ_CLIENT \
365+ _IO(ACRN_IOCTL_TYPE, 0x33)
366+#define ACRN_IOCTL_DESTROY_IOREQ_CLIENT \
367+ _IO(ACRN_IOCTL_TYPE, 0x34)
368+#define ACRN_IOCTL_CLEAR_VM_IOREQ \
369+ _IO(ACRN_IOCTL_TYPE, 0x35)
370+
221371 #define ACRN_IOCTL_SET_MEMSEG \
222372 _IOW(ACRN_IOCTL_TYPE, 0x41, struct acrn_vm_memmap)
223373 #define ACRN_IOCTL_UNSET_MEMSEG \