Michal Kubecek and Firo Yang did a very nice analysis of crashes happening in __inet_lookup_established(). Since a TCP socket can go from TCP_ESTABLISH to TCP_LISTEN (via a close()/socket()/listen() cycle) without a RCU grace period, I should not have changed listeners linkage in their hash table. They must use the nulls protocol (Documentation/RCU/rculist_nulls.txt), so that a lookup can detect a socket in a hash list was moved in another one. Since we added code in commit d296ba60d8e2 ("soreuseport: Resolve merge conflict for v4/v6 ordering fix"), we have to add hlist_nulls_add_tail_rcu() helper.
Showing the changes to include/linux (1 of 6 files). Show all
Unified · Split
| @@ -100,6 +100,43 @@ static inline void hlist_nulls_add_head_rcu(struct hlist_nulls_node *n, · 99 unmodified lines | ||
| 100 | 100 | first->pprev = &n->next; |
| 101 | 101 | } |
| 102 | 102 | |
| 103 | +/** | |
| 104 | + * hlist_nulls_add_tail_rcu | |
| 105 | + * @n: the element to add to the hash list. | |
| 106 | + * @h: the list to add to. | |
| 107 | + * | |
| 108 | + * Description: | |
| 109 | + * Adds the specified element to the specified hlist_nulls, | |
| 110 | + * while permitting racing traversals. | |
| 111 | + * | |
| 112 | + * The caller must take whatever precautions are necessary | |
| 113 | + * (such as holding appropriate locks) to avoid racing | |
| 114 | + * with another list-mutation primitive, such as hlist_nulls_add_head_rcu() | |
| 115 | + * or hlist_nulls_del_rcu(), running on this same list. | |
| 116 | + * However, it is perfectly legal to run concurrently with | |
| 117 | + * the _rcu list-traversal primitives, such as | |
| 118 | + * hlist_nulls_for_each_entry_rcu(), used to prevent memory-consistency | |
| 119 | + * problems on Alpha CPUs. Regardless of the type of CPU, the | |
| 120 | + * list-traversal primitive must be guarded by rcu_read_lock(). | |
| 121 | + */ | |
| 122 | +static inline void hlist_nulls_add_tail_rcu(struct hlist_nulls_node *n, | |
| 123 | + struct hlist_nulls_head *h) | |
| 124 | +{ | |
| 125 | + struct hlist_nulls_node *i, *last = NULL; | |
| 126 | + | |
| 127 | + /* Note: write side code, so rcu accessors are not needed. */ | |
| 128 | + for (i = h->first; !is_a_nulls(i); i = i->next) | |
| 129 | + last = i; | |
| 130 | + | |
| 131 | + if (last) { | |
| 132 | + n->next = last->next; | |
| 133 | + n->pprev = &last->next; | |
| 134 | + rcu_assign_pointer(hlist_next_rcu(last), n); | |
| 135 | + } else { | |
| 136 | + hlist_nulls_add_head_rcu(n, h); | |
| 137 | + } | |
| 138 | +} | |
| 139 | + | |
| 103 | 140 | /** |
| 104 | 141 | * hlist_nulls_for_each_entry_rcu - iterate over rcu list of given type |
| 105 | 142 | * @tpos: the type * to use as a loop cursor. |