rcu: Support reclaim for head-less object

Update the kvfree_call_rcu() function with head-less support. This allows RCU to reclaim objects without an embedded rcu_head. tree-RCU: We introduce two chains of arrays to store SLAB-backed and vmalloc pointers, each. Storage in either of these arrays does not require embedding an rcu_head within the object. Maintaining the arrays may become impossible due to high memory pressure. For such cases there is an emergency path. Objects with rcu_head inside are just queued on a backup rcu_head list. Later on that list is drained. As for the head-less variant, as the current context can sleep, the following emergency measures are applied: a) Synchronously wait until a grace period has elapsed. b) Call kvfree(). tiny-RCU: For double argument calls, there are no new changes in behavior. For single argument call, kvfree() is directly inlined on the current stack after a synchronize_rcu() call. Note that for tiny-RCU, any call to synchronize_rcu() is actually a quiescent state, therefore it does nothing.

Uladzislau Rezki (Sony) committed 6 years ago

Commit 3042f831 parent ce4dce12 files · +60 −3Browse files

Showing the changes to include/linux (1 of 2 files). Show all

Unified · Split

include/linux/rcutiny.h+17−1

@@ -34,9 +34,25 @@ static inline void synchronize_rcu_expedited(void) · 33 unmodified lines
34	synchronize_rcu();	34	synchronize_rcu();
35	}	35	}
36		36
		37	/*
		38	* Add one more declaration of kvfree() here. It is
		39	* not so straight forward to just include <linux/mm.h>
		40	* where it is defined due to getting many compile
		41	* errors caused by that include.
		42	*/
		43	extern void kvfree(const void *addr);
		44
37	static inline void kvfree_call_rcu(struct rcu_head *head, rcu_callback_t func)	45	static inline void kvfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
38	{	46	{
39	call_rcu(head, func);	47	if (head) {
		48	call_rcu(head, func);
		49	return;
		50	}
		51
		52	// kvfree_rcu(one_arg) call.
		53	might_sleep();
		54	synchronize_rcu();
		55	kvfree((void *) func);
40	}	56	}
41		57
42	void rcu_qs(void);	58	void rcu_qs(void);