History

kprobes: consistent rcu api usage for kretprobe holder

It seems that the pointer-to-kretprobe "rp" within the kretprobe_holder is RCU-managed, based on the (non-rethook) implementation of get_kretprobe(). The thought behind this patch is to make use of the RCU API where possible when accessing this pointer so that the needed barriers are always in place and to self-document the code. The __rcu annotation to "rp" allows for sparse RCU checking. Plain writes done to the "rp" pointer are changed to make use of the RCU macro for assignment. For the single read, the implementation of get_kretprobe() is simplified by making use of an RCU macro which accomplishes the same, but note that the log warning text will be more generic. I did find that there is a difference in assembly generated between the usage of the RCU macros vs without. For example, on arm64, when using rcu_assign_pointer(), the corresponding store instruction is a store-release (STLR) which has an implicit barrier. When normal assignment is done, a regular store (STR) is found. In the macro case, this seems to be a result of rcu_assign_pointer() using smp_store_release() when the value to write is not NULL. Link: https://lore.kernel.org/all/20231122132058.3359-1-inwardvessel@gmail.com/

JP Kobryn committed 2 years ago

Commit d839a651 parent d67f39d2 files · +4 −7Browse files

Showing the changes to include/linux (1 of 2 files). Show all

Unified · Split

include/linux/kprobes.h+2−5
@@ -139,7 +139,7 @@ static inline bool kprobe_ftrace(struct kprobe *p) · 138 unmodified lines
139 *139 *
140 */140 */
141struct kretprobe_holder {141struct kretprobe_holder {
142 struct kretprobe *rp;142 struct kretprobe __rcu *rp;
143 struct objpool_head pool;143 struct objpool_head pool;
144};144};
145 145
@@ -245,10 +245,7 @@ unsigned long kretprobe_trampoline_handler(struct pt_regs *regs, · 99 unmodified lines
245 245
246static nokprobe_inline struct kretprobe *get_kretprobe(struct kretprobe_instance *ri)246static nokprobe_inline struct kretprobe *get_kretprobe(struct kretprobe_instance *ri)
247{247{
248 RCU_LOCKDEP_WARN(!rcu_read_lock_any_held(),248 return rcu_dereference_check(ri->rph->rp, rcu_read_lock_any_held());
249 "Kretprobe is accessed from instance under preemptive context");
250
251 return READ_ONCE(ri->rph->rp);
252}249}
253 250
254static nokprobe_inline unsigned long get_kretprobe_retaddr(struct kretprobe_instance *ri)251static nokprobe_inline unsigned long get_kretprobe_retaddr(struct kretprobe_instance *ri)