Pull io_uring updates from Jens Axboe: "Here are the io_uring changes for this merge window. Light on new features this time around (just splice + buffer selection), lots of cleanups, fixes, and improvements to existing support. In particular, this contains: - Cleanup fixed file update handling for stack fallback (Hillf) - Re-work of how pollable async IO is handled, we no longer require thread offload to handle that. Instead we rely using poll to drive this, with task_work execution. - In conjunction with the above, allow expendable buffer selection, so that poll+recv (for example) no longer has to be a split operation. - Make sure we honor RLIMIT_FSIZE for buffered writes - Add support for splice (Pavel) - Linked work inheritance fixes and optimizations (Pavel) - Async work fixes and cleanups (Pavel) - Improve io-wq locking (Pavel) - Hashed link write improvements (Pavel) - SETUP_IOPOLL|SETUP_SQPOLL improvements (Xiaoguang)" * tag 'for-5.7/io_uring-2020-03-29' of git://git.kernel.dk/linux-block: (54 commits) io_uring: cleanup io_alloc_async_ctx() io_uring: fix missing 'return' in comment io-wq: handle hashed writes in chains io-uring: drop 'free_pfile' in struct io_file_put io-uring: drop completion when removing file io_uring: Fix ->data corruption on re-enqueue io-wq: close cancel gap for hashed linked work io_uring: make spdxcheck.py happy io_uring: honor original task RLIMIT_FSIZE io-wq: hash dependent work io-wq: split hashing and enqueueing io-wq: don't resched if there is no work io-wq: remove duplicated cancel code io_uring: fix truncated async read/readv and write/writev retry io_uring: dual license io_uring.h uapi header io_uring: io_uring_enter(2) don't poll while SETUP_IOPOLL|SETUP_SQPOLL enabled io_uring: Fix unused function warnings io_uring: add end-of-bits marker and build time verify it io_uring: provide means of removing buffers io_uring: add IOSQE_BUFFER_SELECT support for IORING_OP_RECVMSG ...
Showing the changes to kernel (1 of 12 files). Show all
Unified · Split
| @@ -97,16 +97,26 @@ void task_work_run(void) · 96 unmodified lines | |||
| 97 | * work->func() can do task_work_add(), do not set | 97 | * work->func() can do task_work_add(), do not set |
| 98 | * work_exited unless the list is empty. | 98 | * work_exited unless the list is empty. |
| 99 | */ | 99 | */ |
| 100 | raw_spin_lock_irq(&task->pi_lock); | ||
| 101 | do { | 100 | do { |
| 101 | head = NULL; | ||
| 102 | work = READ_ONCE(task->task_works); | 102 | work = READ_ONCE(task->task_works); |
| 103 | head = !work && (task->flags & PF_EXITING) ? | 103 | if (!work) { |
| 104 | &work_exited : NULL; | 104 | if (task->flags & PF_EXITING) |
| 105 | head = &work_exited; | ||
| 106 | else | ||
| 107 | break; | ||
| 108 | } | ||
| 105 | } while (cmpxchg(&task->task_works, work, head) != work); | 109 | } while (cmpxchg(&task->task_works, work, head) != work); |
| 106 | raw_spin_unlock_irq(&task->pi_lock); | ||
| 107 | 110 | ||
| 108 | if (!work) | 111 | if (!work) |
| 109 | break; | 112 | break; |
| 113 | /* | ||
| 114 | * Synchronize with task_work_cancel(). It can not remove | ||
| 115 | * the first entry == work, cmpxchg(task_works) must fail. | ||
| 116 | * But it can remove another entry from the ->next list. | ||
| 117 | */ | ||
| 118 | raw_spin_lock_irq(&task->pi_lock); | ||
| 119 | raw_spin_unlock_irq(&task->pi_lock); | ||
| 110 | 120 | ||
| 111 | do { | 121 | do { |
| 112 | next = work->next; | 122 | next = work->next; |