History

io_uring: support for user allocated memory for rings/sqes

Currently io_uring applications must call mmap(2) twice to map the rings themselves, and the sqes array. This works fine, but it does not support using huge pages to back the rings/sqes. Provide a way for the application to pass in pre-allocated memory for the rings/sqes, which can then suitably be allocated from shmfs or via mmap to get huge page support. Particularly for larger rings, this reduces the TLBs needed. If an application wishes to take advantage of that, it must pre-allocate the memory needed for the sq/cq ring, and the sqes. The former must be passed in via the io_uring_params->cq_off.user_data field, while the latter is passed in via the io_uring_params->sq_off.user_data field. Then it must set IORING_SETUP_NO_MMAP in the io_uring_params->flags field, and io_uring will then map the existing memory into the kernel for shared use. The application must not call mmap(2) to map rings as it otherwise would have, that will now fail with -EINVAL if this setup flag was used. The pages used for the rings and sqes must be contigious. The intent here is clearly that huge pages should be used, otherwise the normal setup procedure works fine as-is. The application may use one huge page for both the rings and sqes. Outside of those initialization changes, everything works like it did before.

Jens Axboe committed 5 years ago

Commit 03d89a21 parent 9c189ee3 files · +114 −11Browse files

Showing the changes to include/uapi (1 of 3 files). Show all

Unified · Split

include/uapi/linux/io_uring.h+7−2
@@ -173,6 +173,11 @@ enum { · 172 unmodified lines
173173 */
174174 #define IORING_SETUP_DEFER_TASKRUN (1U << 13)
175175
176+/*
177+ * Application provides the memory for the rings
178+ */
179+#define IORING_SETUP_NO_MMAP (1U << 14)
180+
176181 enum io_uring_op {
177182 IORING_OP_NOP,
178183 IORING_OP_READV,
@@ -406,7 +411,7 @@ struct io_sqring_offsets { · 227 unmodified lines
406411 __u32 dropped;
407412 __u32 array;
408413 __u32 resv1;
409− __u64 resv2;
414+ __u64 user_addr;
410415 };
411416
412417 /*
@@ -425,7 +430,7 @@ struct io_cqring_offsets { · 12 unmodified lines
425430 __u32 cqes;
426431 __u32 flags;
427432 __u32 resv1;
428− __u64 resv2;
433+ __u64 user_addr;
429434 };
430435
431436 /*

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t