Pull io_uring updates from Jens Axboe: "Here are the io_uring changes for this merge window. Light on new features this time around (just splice + buffer selection), lots of cleanups, fixes, and improvements to existing support. In particular, this contains: - Cleanup fixed file update handling for stack fallback (Hillf) - Re-work of how pollable async IO is handled, we no longer require thread offload to handle that. Instead we rely using poll to drive this, with task_work execution. - In conjunction with the above, allow expendable buffer selection, so that poll+recv (for example) no longer has to be a split operation. - Make sure we honor RLIMIT_FSIZE for buffered writes - Add support for splice (Pavel) - Linked work inheritance fixes and optimizations (Pavel) - Async work fixes and cleanups (Pavel) - Improve io-wq locking (Pavel) - Hashed link write improvements (Pavel) - SETUP_IOPOLL|SETUP_SQPOLL improvements (Xiaoguang)" * tag 'for-5.7/io_uring-2020-03-29' of git://git.kernel.dk/linux-block: (54 commits) io_uring: cleanup io_alloc_async_ctx() io_uring: fix missing 'return' in comment io-wq: handle hashed writes in chains io-uring: drop 'free_pfile' in struct io_file_put io-uring: drop completion when removing file io_uring: Fix ->data corruption on re-enqueue io-wq: close cancel gap for hashed linked work io_uring: make spdxcheck.py happy io_uring: honor original task RLIMIT_FSIZE io-wq: hash dependent work io-wq: split hashing and enqueueing io-wq: don't resched if there is no work io-wq: remove duplicated cancel code io_uring: fix truncated async read/readv and write/writev retry io_uring: dual license io_uring.h uapi header io_uring: io_uring_enter(2) don't poll while SETUP_IOPOLL|SETUP_SQPOLL enabled io_uring: Fix unused function warnings io_uring: add end-of-bits marker and build time verify it io_uring: provide means of removing buffers io_uring: add IOSQE_BUFFER_SELECT support for IORING_OP_RECVMSG ...
Showing the changes to kernel (1 of 12 files). Show all
Unified · Split
| @@ -97,16 +97,26 @@ void task_work_run(void) · 96 unmodified lines | ||
| 97 | 97 | * work->func() can do task_work_add(), do not set |
| 98 | 98 | * work_exited unless the list is empty. |
| 99 | 99 | */ |
| 100 | − raw_spin_lock_irq(&task->pi_lock); | |
| 101 | 100 | do { |
| 101 | + head = NULL; | |
| 102 | 102 | work = READ_ONCE(task->task_works); |
| 103 | − head = !work && (task->flags & PF_EXITING) ? | |
| 104 | − &work_exited : NULL; | |
| 103 | + if (!work) { | |
| 104 | + if (task->flags & PF_EXITING) | |
| 105 | + head = &work_exited; | |
| 106 | + else | |
| 107 | + break; | |
| 108 | + } | |
| 105 | 109 | } while (cmpxchg(&task->task_works, work, head) != work); |
| 106 | − raw_spin_unlock_irq(&task->pi_lock); | |
| 107 | 110 | |
| 108 | 111 | if (!work) |
| 109 | 112 | break; |
| 113 | + /* | |
| 114 | + * Synchronize with task_work_cancel(). It can not remove | |
| 115 | + * the first entry == work, cmpxchg(task_works) must fail. | |
| 116 | + * But it can remove another entry from the ->next list. | |
| 117 | + */ | |
| 118 | + raw_spin_lock_irq(&task->pi_lock); | |
| 119 | + raw_spin_unlock_irq(&task->pi_lock); | |
| 110 | 120 | |
| 111 | 121 | do { |
| 112 | 122 | next = work->next; |