History

Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next

Pablo Neira Ayuso says: ==================== Netfilter/IPVS updates for net-next The following patchset contains Netfilter/IPVS updates for net-next, more specifically: * Updates for ipset: 1) Coding style fix for ipset comment extension, from Jeremy Sowden. 2) De-inline many functions in ipset, from Jeremy Sowden. 3) Move ipset function definition from header to source file. 4) Move ip_set_put_flags() to source, export it as a symbol, remove inline. 5) Move range_to_mask() to the source file where this is used. 6) Move ip_set_get_ip_port() to the source file where this is used. * IPVS selftests and netns improvements: 7) Two patches to speedup ipvs netns dismantle, from Haishuang Yan. 8) Three patches to add selftest script for ipvs, also from Haishuang Yan. * Conntrack updates and new nf_hook_slow_list() function: 9) Document ct ecache extension, from Florian Westphal. 10) Skip ct extensions from ctnetlink dump, from Florian. 11) Free ct extension immediately, from Florian. 12) Skip access to ecache extension from nf_ct_deliver_cached_events() this is not correct as reported by Syzbot. 13) Add and use nf_hook_slow_list(), from Florian. * Flowtable infrastructure updates: 14) Move priority to nf_flowtable definition. 15) Dynamic allocation of per-device hooks in flowtables. 16) Allow to include netdevice only once in flowtable definitions. 17) Rise maximum number of devices per flowtable. * Netfilter hardware offload infrastructure updates: 18) Add nft_flow_block_chain() helper function. 19) Pass callback list to nft_setup_cb_call(). 20) Add nft_flow_cls_offload_setup() helper function. 21) Remove rules for the unregistered device via netdevice event. 22) Support for multiple devices in a basechain definition at the ingress hook. 22) Add nft_chain_offload_cmd() helper function. 23) Add nft_flow_block_offload_init() helper function. 24) Rewind in case of failing to bind multiple devices to hook. 25) Typo in IPv6 tproxy module description, from Norman Rasmussen. ====================

David S. Miller committed 7 years ago

Commit 4b1f5dd2 parents 64fe8e9 671312e43 files · +1346 −730Browse files

Unified · Split

43 files
  1. include/linux/netfilter.h +31 −10
  2. include/linux/netfilter/ipset/ip_set.h +8 −188
  3. include/linux/netfilter/ipset/ip_set_bitmap.h +0 −14
  4. include/linux/netfilter/ipset/ip_set_getport.h +0 −3
  5. include/net/ip_vs.h +1 −1
  6. include/net/netfilter/nf_conntrack_extend.h +0 −10
  7. include/net/netfilter/nf_flow_table.h +1 −0
  8. include/net/netfilter/nf_tables.h +10 −6
  9. include/uapi/linux/netfilter/nf_tables.h +2 −0
  10. net/ipv6/netfilter/nf_tproxy_ipv6.c +1 −1
  11. net/netfilter/core.c +20 −0
  12. net/netfilter/ipset/ip_set_bitmap_gen.h +1 −1
  13. net/netfilter/ipset/ip_set_bitmap_ip.c +19 −7
  14. net/netfilter/ipset/ip_set_bitmap_ipmac.c +9 −9
  15. net/netfilter/ipset/ip_set_bitmap_port.c +34 −7
  16. net/netfilter/ipset/ip_set_core.c +201 −11
  17. net/netfilter/ipset/ip_set_getport.c +0 −28
  18. net/netfilter/ipset/ip_set_hash_gen.h +2 −2
  19. net/netfilter/ipset/ip_set_hash_ip.c +5 −5
  20. net/netfilter/ipset/ip_set_hash_ipmac.c +4 −4
  21. net/netfilter/ipset/ip_set_hash_ipmark.c +4 −4
  22. net/netfilter/ipset/ip_set_hash_ipport.c +4 −4
  23. net/netfilter/ipset/ip_set_hash_ipportip.c +4 −4
  24. net/netfilter/ipset/ip_set_hash_ipportnet.c +12 −12
  25. net/netfilter/ipset/ip_set_hash_mac.c +3 −3
  26. net/netfilter/ipset/ip_set_hash_net.c +12 −12
  27. net/netfilter/ipset/ip_set_hash_netiface.c +12 −12
  28. net/netfilter/ipset/ip_set_hash_netnet.c +14 −14
  29. net/netfilter/ipset/ip_set_hash_netport.c +12 −12
  30. net/netfilter/ipset/ip_set_hash_netportnet.c +14 −14
  31. net/netfilter/ipset/ip_set_list_set.c +2 −2
  32. net/netfilter/ipvs/ip_vs_core.c +28 −19
  33. net/netfilter/ipvs/ip_vs_ctl.c +9 −3
  34. net/netfilter/ipvs/ip_vs_xmit.c +6 −12
  35. net/netfilter/nf_conntrack_core.c +0 −2
  36. net/netfilter/nf_conntrack_ecache.c +18 −5
  37. net/netfilter/nf_conntrack_extend.c +10 −11
  38. net/netfilter/nf_conntrack_netlink.c +50 −26
  39. net/netfilter/nf_tables_api.c +403 −169
  40. net/netfilter/nf_tables_offload.c +125 −63
  41. net/netfilter/nft_chain_filter.c +26 −19
  42. tools/testing/selftests/netfilter/Makefile +1 −1
  43. tools/testing/selftests/netfilter/ipvs.sh +228 −0
include/linux/netfilter.h+31−10
@@ -199,6 +199,8 @@ extern struct static_key nf_hooks_needed[NFPROTO_NUMPROTO][NF_MAX_HOOKS]; · 198 unmodified lines
199199 int nf_hook_slow(struct sk_buff *skb, struct nf_hook_state *state,
200200 const struct nf_hook_entries *e, unsigned int i);
201201
202+void nf_hook_slow_list(struct list_head *head, struct nf_hook_state *state,
203+ const struct nf_hook_entries *e);
202204 /**
203205 * nf_hook - call a netfilter hook
204206 *
@@ -311,17 +313,36 @@ NF_HOOK_LIST(uint8_t pf, unsigned int hook, struct net *net, struct sock *sk, · 106 unmodified lines
311313 struct list_head *head, struct net_device *in, struct net_device *out,
312314 int (*okfn)(struct net *, struct sock *, struct sk_buff *))
313315 {
314− struct sk_buff *skb, *next;
315− struct list_head sublist;
316−
317− INIT_LIST_HEAD(&sublist);
318− list_for_each_entry_safe(skb, next, head, list) {
319− list_del(&skb->list);
320− if (nf_hook(pf, hook, net, sk, skb, in, out, okfn) == 1)
321− list_add_tail(&skb->list, &sublist);
316+ struct nf_hook_entries *hook_head = NULL;
317+
318+#ifdef CONFIG_JUMP_LABEL
319+ if (__builtin_constant_p(pf) &&
320+ __builtin_constant_p(hook) &&
321+ !static_key_false(&nf_hooks_needed[pf][hook]))
322+ return;
323+#endif
324+
325+ rcu_read_lock();
326+ switch (pf) {
327+ case NFPROTO_IPV4:
328+ hook_head = rcu_dereference(net->nf.hooks_ipv4[hook]);
329+ break;
330+ case NFPROTO_IPV6:
331+ hook_head = rcu_dereference(net->nf.hooks_ipv6[hook]);
332+ break;
333+ default:
334+ WARN_ON_ONCE(1);
335+ break;
322336 }
323− /* Put passed packets back on main list */
324− list_splice(&sublist, head);
337+
338+ if (hook_head) {
339+ struct nf_hook_state state;
340+
341+ nf_hook_state_init(&state, hook, pf, in, out, sk, net, okfn);
342+
343+ nf_hook_slow_list(head, &state, hook_head);
344+ }
345+ rcu_read_unlock();
325346 }
326347
327348 /* Call setsockopt() */
include/linux/netfilter/ipset/ip_set.h+8−188
@@ -269,34 +269,15 @@ ip_set_ext_destroy(struct ip_set *set, void *data) · 268 unmodified lines
269269 /* Check that the extension is enabled for the set and
270270 * call it's destroy function for its extension part in data.
271271 */
272− if (SET_WITH_COMMENT(set))
273− ip_set_extensions[IPSET_EXT_ID_COMMENT].destroy(
274− set, ext_comment(data, set));
275−}
272+ if (SET_WITH_COMMENT(set)) {
273+ struct ip_set_comment *c = ext_comment(data, set);
276274
277−static inline int
278−ip_set_put_flags(struct sk_buff *skb, struct ip_set *set)
279−{
280− u32 cadt_flags = 0;
281−
282− if (SET_WITH_TIMEOUT(set))
283− if (unlikely(nla_put_net32(skb, IPSET_ATTR_TIMEOUT,
284− htonl(set->timeout))))
285− return -EMSGSIZE;
286− if (SET_WITH_COUNTER(set))
287− cadt_flags |= IPSET_FLAG_WITH_COUNTERS;
288− if (SET_WITH_COMMENT(set))
289− cadt_flags |= IPSET_FLAG_WITH_COMMENT;
290− if (SET_WITH_SKBINFO(set))
291− cadt_flags |= IPSET_FLAG_WITH_SKBINFO;
292− if (SET_WITH_FORCEADD(set))
293− cadt_flags |= IPSET_FLAG_WITH_FORCEADD;
294−
295− if (!cadt_flags)
296− return 0;
297− return nla_put_net32(skb, IPSET_ATTR_CADT_FLAGS, htonl(cadt_flags));
275+ ip_set_extensions[IPSET_EXT_ID_COMMENT].destroy(set, c);
276+ }
298277 }
299278
279+int ip_set_put_flags(struct sk_buff *skb, struct ip_set *set);
280+
300281 /* Netlink CB args */
301282 enum {
302283 IPSET_CB_NET = 0, /* net namespace */
@@ -506,144 +487,8 @@ ip_set_timeout_set(unsigned long *timeout, u32 value) · 203 unmodified lines
506487 *timeout = t;
507488 }
508489
509−static inline u32
510−ip_set_timeout_get(const unsigned long *timeout)
511−{
512− u32 t;
513−
514− if (*timeout == IPSET_ELEM_PERMANENT)
515− return 0;
516−
517− t = jiffies_to_msecs(*timeout - jiffies)/MSEC_PER_SEC;
518− /* Zero value in userspace means no timeout */
519− return t == 0 ? 1 : t;
520−}
521−
522−static inline char*
523−ip_set_comment_uget(struct nlattr *tb)
524−{
525− return nla_data(tb);
526−}
527−
528−/* Called from uadd only, protected by the set spinlock.
529− * The kadt functions don't use the comment extensions in any way.
530− */
531−static inline void
532−ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment,
533− const struct ip_set_ext *ext)
534−{
535− struct ip_set_comment_rcu *c = rcu_dereference_protected(comment->c, 1);
536− size_t len = ext->comment ? strlen(ext->comment) : 0;
537−
538− if (unlikely(c)) {
539− set->ext_size -= sizeof(*c) + strlen(c->str) + 1;
540− kfree_rcu(c, rcu);
541− rcu_assign_pointer(comment->c, NULL);
542− }
543− if (!len)
544− return;
545− if (unlikely(len > IPSET_MAX_COMMENT_SIZE))
546− len = IPSET_MAX_COMMENT_SIZE;
547− c = kmalloc(sizeof(*c) + len + 1, GFP_ATOMIC);
548− if (unlikely(!c))
549− return;
550− strlcpy(c->str, ext->comment, len + 1);
551− set->ext_size += sizeof(*c) + strlen(c->str) + 1;
552− rcu_assign_pointer(comment->c, c);
553−}
554−
555−/* Used only when dumping a set, protected by rcu_read_lock() */
556−static inline int
557−ip_set_put_comment(struct sk_buff *skb, const struct ip_set_comment *comment)
558−{
559− struct ip_set_comment_rcu *c = rcu_dereference(comment->c);
560−
561− if (!c)
562− return 0;
563− return nla_put_string(skb, IPSET_ATTR_COMMENT, c->str);
564−}
565−
566−/* Called from uadd/udel, flush or the garbage collectors protected
567− * by the set spinlock.
568− * Called when the set is destroyed and when there can't be any user
569− * of the set data anymore.
570− */
571−static inline void
572−ip_set_comment_free(struct ip_set *set, struct ip_set_comment *comment)
573−{
574− struct ip_set_comment_rcu *c;
575−
576− c = rcu_dereference_protected(comment->c, 1);
577− if (unlikely(!c))
578− return;
579− set->ext_size -= sizeof(*c) + strlen(c->str) + 1;
580− kfree_rcu(c, rcu);
581− rcu_assign_pointer(comment->c, NULL);
582−}
583−
584−static inline void
585−ip_set_add_bytes(u64 bytes, struct ip_set_counter *counter)
586−{
587− atomic64_add((long long)bytes, &(counter)->bytes);
588−}
589−
590−static inline void
591−ip_set_add_packets(u64 packets, struct ip_set_counter *counter)
592−{
593− atomic64_add((long long)packets, &(counter)->packets);
594−}
595−
596−static inline u64
597−ip_set_get_bytes(const struct ip_set_counter *counter)
598−{
599− return (u64)atomic64_read(&(counter)->bytes);
600−}
601−
602−static inline u64
603−ip_set_get_packets(const struct ip_set_counter *counter)
604−{
605− return (u64)atomic64_read(&(counter)->packets);
606−}
607−
608−static inline bool
609−ip_set_match_counter(u64 counter, u64 match, u8 op)
610−{
611− switch (op) {
612− case IPSET_COUNTER_NONE:
613− return true;
614− case IPSET_COUNTER_EQ:
615− return counter == match;
616− case IPSET_COUNTER_NE:
617− return counter != match;
618− case IPSET_COUNTER_LT:
619− return counter < match;
620− case IPSET_COUNTER_GT:
621− return counter > match;
622− }
623− return false;
624−}
625−
626−static inline void
627−ip_set_update_counter(struct ip_set_counter *counter,
628− const struct ip_set_ext *ext, u32 flags)
629−{
630− if (ext->packets != ULLONG_MAX &&
631− !(flags & IPSET_FLAG_SKIP_COUNTER_UPDATE)) {
632− ip_set_add_bytes(ext->bytes, counter);
633− ip_set_add_packets(ext->packets, counter);
634− }
635−}
636−
637−static inline bool
638−ip_set_put_counter(struct sk_buff *skb, const struct ip_set_counter *counter)
639−{
640− return nla_put_net64(skb, IPSET_ATTR_BYTES,
641− cpu_to_be64(ip_set_get_bytes(counter)),
642− IPSET_ATTR_PAD) ||
643− nla_put_net64(skb, IPSET_ATTR_PACKETS,
644− cpu_to_be64(ip_set_get_packets(counter)),
645− IPSET_ATTR_PAD);
646−}
490+void ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment,
491+ const struct ip_set_ext *ext);
647492
648493 static inline void
649494 ip_set_init_counter(struct ip_set_counter *counter,
@@ -655,31 +500,6 @@ ip_set_init_counter(struct ip_set_counter *counter, · 5 unmodified lines
655500 atomic64_set(&(counter)->packets, (long long)(ext->packets));
656501 }
657502
658−static inline void
659−ip_set_get_skbinfo(struct ip_set_skbinfo *skbinfo,
660− const struct ip_set_ext *ext,
661− struct ip_set_ext *mext, u32 flags)
662−{
663− mext->skbinfo = *skbinfo;
664−}
665−
666−static inline bool
667−ip_set_put_skbinfo(struct sk_buff *skb, const struct ip_set_skbinfo *skbinfo)
668−{
669− /* Send nonzero parameters only */
670− return ((skbinfo->skbmark || skbinfo->skbmarkmask) &&
671− nla_put_net64(skb, IPSET_ATTR_SKBMARK,
672− cpu_to_be64((u64)skbinfo->skbmark << 32 |
673− skbinfo->skbmarkmask),
674− IPSET_ATTR_PAD)) ||
675− (skbinfo->skbprio &&
676− nla_put_net32(skb, IPSET_ATTR_SKBPRIO,
677− cpu_to_be32(skbinfo->skbprio))) ||
678− (skbinfo->skbqueue &&
679− nla_put_net16(skb, IPSET_ATTR_SKBQUEUE,
680− cpu_to_be16(skbinfo->skbqueue)));
681−}
682−
683503 static inline void
684504 ip_set_init_skbinfo(struct ip_set_skbinfo *skbinfo,
685505 const struct ip_set_ext *ext)
include/linux/netfilter/ipset/ip_set_bitmap.h+0−14
@@ -12,18 +12,4 @@ enum { · 11 unmodified lines
1212 IPSET_ADD_START_STORED_TIMEOUT,
1313 };
1414
15−/* Common functions */
16−
17−static inline u32
18−range_to_mask(u32 from, u32 to, u8 *bits)
19−{
20− u32 mask = 0xFFFFFFFE;
21−
22− *bits = 32;
23− while (--(*bits) > 0 && mask && (to & mask) != from)
24− mask <<= 1;
25−
26− return mask;
27−}
28−
2915 #endif /* __IP_SET_BITMAP_H */
include/linux/netfilter/ipset/ip_set_getport.h+0−3
@@ -20,9 +20,6 @@ static inline bool ip_set_get_ip6_port(const struct sk_buff *skb, bool src, · 19 unmodified lines
2020 }
2121 #endif
2222
23−extern bool ip_set_get_ip_port(const struct sk_buff *skb, u8 pf, bool src,
24− __be16 *port);
25−
2623 static inline bool ip_set_proto_with_ports(u8 proto)
2724 {
2825 switch (proto) {
include/net/ip_vs.h+1−1
@@ -1324,7 +1324,7 @@ void ip_vs_protocol_net_cleanup(struct netns_ipvs *ipvs); · 1323 unmodified lines
13241324 void ip_vs_control_net_cleanup(struct netns_ipvs *ipvs);
13251325 void ip_vs_estimator_net_cleanup(struct netns_ipvs *ipvs);
13261326 void ip_vs_sync_net_cleanup(struct netns_ipvs *ipvs);
1327−void ip_vs_service_net_cleanup(struct netns_ipvs *ipvs);
1327+void ip_vs_service_nets_cleanup(struct list_head *net_list);
13281328
13291329 /* IPVS application functions
13301330 * (from ip_vs_app.c)
include/net/netfilter/nf_conntrack_extend.h+0−10
@@ -43,7 +43,6 @@ enum nf_ct_ext_id { · 42 unmodified lines
4343
4444 /* Extensions: optional stuff which isn't permanently in struct. */
4545 struct nf_ct_ext {
46− struct rcu_head rcu;
4746 u8 offset[NF_CT_EXT_NUM];
4847 u8 len;
4948 char data[0];
@@ -72,15 +71,6 @@ static inline void *__nf_ct_ext_find(const struct nf_conn *ct, u8 id) · 22 unmodified lines
7271 /* Destroy all relationships */
7372 void nf_ct_ext_destroy(struct nf_conn *ct);
7473
75−/* Free operation. If you want to free a object referred from private area,
76− * please implement __nf_ct_ext_free() and call it.
77− */
78−static inline void nf_ct_ext_free(struct nf_conn *ct)
79−{
80− if (ct->ext)
81− kfree_rcu(ct->ext, rcu);
82−}
83−
8474 /* Add this type, returns pointer to data or NULL. */
8575 void *nf_ct_ext_add(struct nf_conn *ct, enum nf_ct_ext_id id, gfp_t gfp);
8676
include/net/netfilter/nf_flow_table.h+1−0
@@ -24,6 +24,7 @@ struct nf_flowtable_type { · 23 unmodified lines
2424 struct nf_flowtable {
2525 struct list_head list;
2626 struct rhashtable rhashtable;
27+ int priority;
2728 const struct nf_flowtable_type *type;
2829 struct delayed_work gc_work;
2930 };
include/net/netfilter/nf_tables.h+10−6
@@ -963,25 +963,31 @@ struct nft_stats { · 962 unmodified lines
963963 struct u64_stats_sync syncp;
964964 };
965965
966+struct nft_hook {
967+ struct list_head list;
968+ struct nf_hook_ops ops;
969+ struct rcu_head rcu;
970+};
971+
966972 /**
967973 * struct nft_base_chain - nf_tables base chain
968974 *
969975 * @ops: netfilter hook ops
976+ * @hook_list: list of netfilter hooks (for NFPROTO_NETDEV family)
970977 * @type: chain type
971978 * @policy: default policy
972979 * @stats: per-cpu chain stats
973980 * @chain: the chain
974− * @dev_name: device name that this base chain is attached to (if any)
975981 * @flow_block: flow block (for hardware offload)
976982 */
977983 struct nft_base_chain {
978984 struct nf_hook_ops ops;
985+ struct list_head hook_list;
979986 const struct nft_chain_type *type;
980987 u8 policy;
981988 u8 flags;
982989 struct nft_stats __percpu *stats;
983990 struct nft_chain chain;
984− char dev_name[IFNAMSIZ];
985991 struct flow_block flow_block;
986992 };
987993
@@ -1146,7 +1152,7 @@ struct nft_object_ops { · 158 unmodified lines
11461152 int nft_register_obj(struct nft_object_type *obj_type);
11471153 void nft_unregister_obj(struct nft_object_type *obj_type);
11481154
1149−#define NFT_FLOWTABLE_DEVICE_MAX 8
1155+#define NFT_NETDEVICE_MAX 256
11501156
11511157 /**
11521158 * struct nft_flowtable - nf_tables flow table
@@ -1155,7 +1161,6 @@ void nft_unregister_obj(struct nft_object_type *obj_type); · 2 unmodified lines
11551161 * @table: the table the flow table is contained in
11561162 * @name: name of this flow table
11571163 * @hooknum: hook number
1158− * @priority: hook priority
11591164 * @ops_len: number of hooks in array
11601165 * @genmask: generation mask
11611166 * @use: number of references to this flow table
@@ -1169,13 +1174,12 @@ struct nft_flowtable { · 7 unmodified lines
11691174 struct nft_table *table;
11701175 char *name;
11711176 int hooknum;
1172− int priority;
11731177 int ops_len;
11741178 u32 genmask:2,
11751179 use:30;
11761180 u64 handle;
11771181 /* runtime data below here */
1178− struct nf_hook_ops *ops ____cacheline_aligned;
1182+ struct list_head hook_list ____cacheline_aligned;
11791183 struct nf_flowtable data;
11801184 };
11811185
include/uapi/linux/netfilter/nf_tables.h+2−0
@@ -144,12 +144,14 @@ enum nft_list_attributes { · 143 unmodified lines
144144 * @NFTA_HOOK_HOOKNUM: netfilter hook number (NLA_U32)
145145 * @NFTA_HOOK_PRIORITY: netfilter hook priority (NLA_U32)
146146 * @NFTA_HOOK_DEV: netdevice name (NLA_STRING)
147+ * @NFTA_HOOK_DEVS: list of netdevices (NLA_NESTED)
147148 */
148149 enum nft_hook_attributes {
149150 NFTA_HOOK_UNSPEC,
150151 NFTA_HOOK_HOOKNUM,
151152 NFTA_HOOK_PRIORITY,
152153 NFTA_HOOK_DEV,
154+ NFTA_HOOK_DEVS,
153155 __NFTA_HOOK_MAX
154156 };
155157 #define NFTA_HOOK_MAX (__NFTA_HOOK_MAX - 1)
net/ipv6/netfilter/nf_tproxy_ipv6.c+1−1
@@ -150,4 +150,4 @@ EXPORT_SYMBOL_GPL(nf_tproxy_get_sock_v6); · 149 unmodified lines
150150
151151 MODULE_LICENSE("GPL");
152152 MODULE_AUTHOR("Balazs Scheidler, Krisztian Kovacs");
153−MODULE_DESCRIPTION("Netfilter IPv4 transparent proxy support");
153+MODULE_DESCRIPTION("Netfilter IPv6 transparent proxy support");
net/netfilter/core.c+20−0
@@ -536,6 +536,26 @@ int nf_hook_slow(struct sk_buff *skb, struct nf_hook_state *state, · 535 unmodified lines
536536 }
537537 EXPORT_SYMBOL(nf_hook_slow);
538538
539+void nf_hook_slow_list(struct list_head *head, struct nf_hook_state *state,
540+ const struct nf_hook_entries *e)
541+{
542+ struct sk_buff *skb, *next;
543+ struct list_head sublist;
544+ int ret;
545+
546+ INIT_LIST_HEAD(&sublist);
547+
548+ list_for_each_entry_safe(skb, next, head, list) {
549+ skb_list_del_init(skb);
550+ ret = nf_hook_slow(skb, state, e, 0);
551+ if (ret == 1)
552+ list_add_tail(&skb->list, &sublist);
553+ }
554+ /* Put passed packets back on main list */
555+ list_splice(&sublist, head);
556+}
557+EXPORT_SYMBOL(nf_hook_slow_list);
558+
539559 /* This needs to be compiled in any case to avoid dependencies between the
540560 * nfnetlink_queue code and nf_conntrack.
541561 */
net/netfilter/ipset/ip_set_bitmap_gen.h+1−1
@@ -192,7 +192,7 @@ mtype_del(struct ip_set *set, void *value, const struct ip_set_ext *ext, · 191 unmodified lines
192192 }
193193
194194 #ifndef IP_SET_BITMAP_STORED_TIMEOUT
195−static inline bool
195+static bool
196196 mtype_is_filled(const struct mtype_elem *x)
197197 {
198198 return true;
net/netfilter/ipset/ip_set_bitmap_ip.c+19−7
@@ -55,7 +55,7 @@ struct bitmap_ip_adt_elem { · 54 unmodified lines
5555 u16 id;
5656 };
5757
58−static inline u32
58+static u32
5959 ip_to_id(const struct bitmap_ip *m, u32 ip)
6060 {
6161 return ((ip & ip_set_hostmask(m->netmask)) - m->first_ip) / m->hosts;
@@ -63,33 +63,33 @@ ip_to_id(const struct bitmap_ip *m, u32 ip) · 1 unmodified line
6363
6464 /* Common functions */
6565
66−static inline int
66+static int
6767 bitmap_ip_do_test(const struct bitmap_ip_adt_elem *e,
6868 struct bitmap_ip *map, size_t dsize)
6969 {
7070 return !!test_bit(e->id, map->members);
7171 }
7272
73−static inline int
73+static int
7474 bitmap_ip_gc_test(u16 id, const struct bitmap_ip *map, size_t dsize)
7575 {
7676 return !!test_bit(id, map->members);
7777 }
7878
79−static inline int
79+static int
8080 bitmap_ip_do_add(const struct bitmap_ip_adt_elem *e, struct bitmap_ip *map,
8181 u32 flags, size_t dsize)
8282 {
8383 return !!test_bit(e->id, map->members);
8484 }
8585
86−static inline int
86+static int
8787 bitmap_ip_do_del(const struct bitmap_ip_adt_elem *e, struct bitmap_ip *map)
8888 {
8989 return !test_and_clear_bit(e->id, map->members);
9090 }
9191
92−static inline int
92+static int
9393 bitmap_ip_do_list(struct sk_buff *skb, const struct bitmap_ip *map, u32 id,
9494 size_t dsize)
9595 {
@@ -97,7 +97,7 @@ bitmap_ip_do_list(struct sk_buff *skb, const struct bitmap_ip *map, u32 id, · 1 unmodified line
9797 htonl(map->first_ip + id * map->hosts));
9898 }
9999
100−static inline int
100+static int
101101 bitmap_ip_do_head(struct sk_buff *skb, const struct bitmap_ip *map)
102102 {
103103 return nla_put_ipaddr4(skb, IPSET_ATTR_IP, htonl(map->first_ip)) ||
@@ -237,6 +237,18 @@ init_map_ip(struct ip_set *set, struct bitmap_ip *map, · 133 unmodified lines
237237 return true;
238238 }
239239
240+static u32
241+range_to_mask(u32 from, u32 to, u8 *bits)
242+{
243+ u32 mask = 0xFFFFFFFE;
244+
245+ *bits = 32;
246+ while (--(*bits) > 0 && mask && (to & mask) != from)
247+ mask <<= 1;
248+
249+ return mask;
250+}
251+
240252 static int
241253 bitmap_ip_create(struct net *net, struct ip_set *set, struct nlattr *tb[],
242254 u32 flags)
net/netfilter/ipset/ip_set_bitmap_ipmac.c+9−9
@@ -65,7 +65,7 @@ struct bitmap_ipmac_elem { · 64 unmodified lines
6565 unsigned char filled;
6666 } __aligned(__alignof__(u64));
6767
68−static inline u32
68+static u32
6969 ip_to_id(const struct bitmap_ipmac *m, u32 ip)
7070 {
7171 return ip - m->first_ip;
@@ -79,7 +79,7 @@ ip_to_id(const struct bitmap_ipmac *m, u32 ip) · 7 unmodified lines
7979
8080 /* Common functions */
8181
82−static inline int
82+static int
8383 bitmap_ipmac_do_test(const struct bitmap_ipmac_adt_elem *e,
8484 const struct bitmap_ipmac *map, size_t dsize)
8585 {
@@ -94,7 +94,7 @@ bitmap_ipmac_do_test(const struct bitmap_ipmac_adt_elem *e, · 8 unmodified lines
9494 return -EAGAIN;
9595 }
9696
97−static inline int
97+static int
9898 bitmap_ipmac_gc_test(u16 id, const struct bitmap_ipmac *map, size_t dsize)
9999 {
100100 const struct bitmap_ipmac_elem *elem;
@@ -106,13 +106,13 @@ bitmap_ipmac_gc_test(u16 id, const struct bitmap_ipmac *map, size_t dsize) · 5 unmodified lines
106106 return elem->filled == MAC_FILLED;
107107 }
108108
109−static inline int
109+static int
110110 bitmap_ipmac_is_filled(const struct bitmap_ipmac_elem *elem)
111111 {
112112 return elem->filled == MAC_FILLED;
113113 }
114114
115−static inline int
115+static int
116116 bitmap_ipmac_add_timeout(unsigned long *timeout,
117117 const struct bitmap_ipmac_adt_elem *e,
118118 const struct ip_set_ext *ext, struct ip_set *set,
@@ -139,7 +139,7 @@ bitmap_ipmac_add_timeout(unsigned long *timeout, · 20 unmodified lines
139139 return 0;
140140 }
141141
142−static inline int
142+static int
143143 bitmap_ipmac_do_add(const struct bitmap_ipmac_adt_elem *e,
144144 struct bitmap_ipmac *map, u32 flags, size_t dsize)
145145 {
@@ -177,14 +177,14 @@ bitmap_ipmac_do_add(const struct bitmap_ipmac_adt_elem *e, · 31 unmodified lines
177177 return IPSET_ADD_STORE_PLAIN_TIMEOUT;
178178 }
179179
180−static inline int
180+static int
181181 bitmap_ipmac_do_del(const struct bitmap_ipmac_adt_elem *e,
182182 struct bitmap_ipmac *map)
183183 {
184184 return !test_and_clear_bit(e->id, map->members);
185185 }
186186
187−static inline int
187+static int
188188 bitmap_ipmac_do_list(struct sk_buff *skb, const struct bitmap_ipmac *map,
189189 u32 id, size_t dsize)
190190 {
@@ -197,7 +197,7 @@ bitmap_ipmac_do_list(struct sk_buff *skb, const struct bitmap_ipmac *map, · 6 unmodified lines
197197 nla_put(skb, IPSET_ATTR_ETHER, ETH_ALEN, elem->ether));
198198 }
199199
200−static inline int
200+static int
201201 bitmap_ipmac_do_head(struct sk_buff *skb, const struct bitmap_ipmac *map)
202202 {
203203 return nla_put_ipaddr4(skb, IPSET_ATTR_IP, htonl(map->first_ip)) ||
net/netfilter/ipset/ip_set_bitmap_port.c+34−7
@@ -46,7 +46,7 @@ struct bitmap_port_adt_elem { · 45 unmodified lines
4646 u16 id;
4747 };
4848
49−static inline u16
49+static u16
5050 port_to_id(const struct bitmap_port *m, u16 port)
5151 {
5252 return port - m->first_port;
@@ -54,34 +54,34 @@ port_to_id(const struct bitmap_port *m, u16 port) · 1 unmodified line
5454
5555 /* Common functions */
5656
57−static inline int
57+static int
5858 bitmap_port_do_test(const struct bitmap_port_adt_elem *e,
5959 const struct bitmap_port *map, size_t dsize)
6060 {
6161 return !!test_bit(e->id, map->members);
6262 }
6363
64−static inline int
64+static int
6565 bitmap_port_gc_test(u16 id, const struct bitmap_port *map, size_t dsize)
6666 {
6767 return !!test_bit(id, map->members);
6868 }
6969
70−static inline int
70+static int
7171 bitmap_port_do_add(const struct bitmap_port_adt_elem *e,
7272 struct bitmap_port *map, u32 flags, size_t dsize)
7373 {
7474 return !!test_bit(e->id, map->members);
7575 }
7676
77−static inline int
77+static int
7878 bitmap_port_do_del(const struct bitmap_port_adt_elem *e,
7979 struct bitmap_port *map)
8080 {
8181 return !test_and_clear_bit(e->id, map->members);
8282 }
8383
84−static inline int
84+static int
8585 bitmap_port_do_list(struct sk_buff *skb, const struct bitmap_port *map, u32 id,
8686 size_t dsize)
8787 {
@@ -89,13 +89,40 @@ bitmap_port_do_list(struct sk_buff *skb, const struct bitmap_port *map, u32 id, · 1 unmodified line
8989 htons(map->first_port + id));
9090 }
9191
92−static inline int
92+static int
9393 bitmap_port_do_head(struct sk_buff *skb, const struct bitmap_port *map)
9494 {
9595 return nla_put_net16(skb, IPSET_ATTR_PORT, htons(map->first_port)) ||
9696 nla_put_net16(skb, IPSET_ATTR_PORT_TO, htons(map->last_port));
9797 }
9898
99+static bool
100+ip_set_get_ip_port(const struct sk_buff *skb, u8 pf, bool src, __be16 *port)
101+{
102+ bool ret;
103+ u8 proto;
104+
105+ switch (pf) {
106+ case NFPROTO_IPV4:
107+ ret = ip_set_get_ip4_port(skb, src, port, &proto);
108+ break;
109+ case NFPROTO_IPV6:
110+ ret = ip_set_get_ip6_port(skb, src, port, &proto);
111+ break;
112+ default:
113+ return false;
114+ }
115+ if (!ret)
116+ return ret;
117+ switch (proto) {
118+ case IPPROTO_TCP:
119+ case IPPROTO_UDP:
120+ return true;
121+ default:
122+ return false;
123+ }
124+}
125+
99126 static int
100127 bitmap_port_kadt(struct ip_set *set, const struct sk_buff *skb,
101128 const struct xt_action_param *par,
net/netfilter/ipset/ip_set_core.c+201−11
@@ -35,7 +35,7 @@ struct ip_set_net { · 34 unmodified lines
3535
3636 static unsigned int ip_set_net_id __read_mostly;
3737
38−static inline struct ip_set_net *ip_set_pernet(struct net *net)
38+static struct ip_set_net *ip_set_pernet(struct net *net)
3939 {
4040 return net_generic(net, ip_set_net_id);
4141 }
@@ -67,13 +67,13 @@ MODULE_ALIAS_NFNL_SUBSYS(NFNL_SUBSYS_IPSET); · 25 unmodified lines
6767 * serialized by ip_set_type_mutex.
6868 */
6969
70−static inline void
70+static void
7171 ip_set_type_lock(void)
7272 {
7373 mutex_lock(&ip_set_type_mutex);
7474 }
7575
76−static inline void
76+static void
7777 ip_set_type_unlock(void)
7878 {
7979 mutex_unlock(&ip_set_type_mutex);
@@ -277,7 +277,7 @@ ip_set_free(void *members) · 197 unmodified lines
277277 }
278278 EXPORT_SYMBOL_GPL(ip_set_free);
279279
280−static inline bool
280+static bool
281281 flag_nested(const struct nlattr *nla)
282282 {
283283 return nla->nla_type & NLA_F_NESTED;
@@ -325,6 +325,83 @@ ip_set_get_ipaddr6(struct nlattr *nla, union nf_inet_addr *ipaddr) · 41 unmodified lines
325325 }
326326 EXPORT_SYMBOL_GPL(ip_set_get_ipaddr6);
327327
328+static u32
329+ip_set_timeout_get(const unsigned long *timeout)
330+{
331+ u32 t;
332+
333+ if (*timeout == IPSET_ELEM_PERMANENT)
334+ return 0;
335+
336+ t = jiffies_to_msecs(*timeout - jiffies) / MSEC_PER_SEC;
337+ /* Zero value in userspace means no timeout */
338+ return t == 0 ? 1 : t;
339+}
340+
341+static char *
342+ip_set_comment_uget(struct nlattr *tb)
343+{
344+ return nla_data(tb);
345+}
346+
347+/* Called from uadd only, protected by the set spinlock.
348+ * The kadt functions don't use the comment extensions in any way.
349+ */
350+void
351+ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment,
352+ const struct ip_set_ext *ext)
353+{
354+ struct ip_set_comment_rcu *c = rcu_dereference_protected(comment->c, 1);
355+ size_t len = ext->comment ? strlen(ext->comment) : 0;
356+
357+ if (unlikely(c)) {
358+ set->ext_size -= sizeof(*c) + strlen(c->str) + 1;
359+ kfree_rcu(c, rcu);
360+ rcu_assign_pointer(comment->c, NULL);
361+ }
362+ if (!len)
363+ return;
364+ if (unlikely(len > IPSET_MAX_COMMENT_SIZE))
365+ len = IPSET_MAX_COMMENT_SIZE;
366+ c = kmalloc(sizeof(*c) + len + 1, GFP_ATOMIC);
367+ if (unlikely(!c))
368+ return;
369+ strlcpy(c->str, ext->comment, len + 1);
370+ set->ext_size += sizeof(*c) + strlen(c->str) + 1;
371+ rcu_assign_pointer(comment->c, c);
372+}
373+EXPORT_SYMBOL_GPL(ip_set_init_comment);
374+
375+/* Used only when dumping a set, protected by rcu_read_lock() */
376+static int
377+ip_set_put_comment(struct sk_buff *skb, const struct ip_set_comment *comment)
378+{
379+ struct ip_set_comment_rcu *c = rcu_dereference(comment->c);
380+
381+ if (!c)
382+ return 0;
383+ return nla_put_string(skb, IPSET_ATTR_COMMENT, c->str);
384+}
385+
386+/* Called from uadd/udel, flush or the garbage collectors protected
387+ * by the set spinlock.
388+ * Called when the set is destroyed and when there can't be any user
389+ * of the set data anymore.
390+ */
391+static void
392+ip_set_comment_free(struct ip_set *set, void *ptr)
393+{
394+ struct ip_set_comment *comment = ptr;
395+ struct ip_set_comment_rcu *c;
396+
397+ c = rcu_dereference_protected(comment->c, 1);
398+ if (unlikely(!c))
399+ return;
400+ set->ext_size -= sizeof(*c) + strlen(c->str) + 1;
401+ kfree_rcu(c, rcu);
402+ rcu_assign_pointer(comment->c, NULL);
403+}
404+
328405 typedef void (*destroyer)(struct ip_set *, void *);
329406 /* ipset data extension types, in size order */
330407
@@ -351,12 +428,12 @@ const struct ip_set_ext_type ip_set_extensions[] = { · 20 unmodified lines
351428 .flag = IPSET_FLAG_WITH_COMMENT,
352429 .len = sizeof(struct ip_set_comment),
353430 .align = __alignof__(struct ip_set_comment),
354− .destroy = (destroyer) ip_set_comment_free,
431+ .destroy = ip_set_comment_free,
355432 },
356433 };
357434 EXPORT_SYMBOL_GPL(ip_set_extensions);
358435
359−static inline bool
436+static bool
360437 add_extension(enum ip_set_ext_id id, u32 flags, struct nlattr *tb[])
361438 {
362439 return ip_set_extensions[id].flag ?
@@ -446,6 +523,46 @@ ip_set_get_extensions(struct ip_set *set, struct nlattr *tb[], · 83 unmodified lines
446523 }
447524 EXPORT_SYMBOL_GPL(ip_set_get_extensions);
448525
526+static u64
527+ip_set_get_bytes(const struct ip_set_counter *counter)
528+{
529+ return (u64)atomic64_read(&(counter)->bytes);
530+}
531+
532+static u64
533+ip_set_get_packets(const struct ip_set_counter *counter)
534+{
535+ return (u64)atomic64_read(&(counter)->packets);
536+}
537+
538+static bool
539+ip_set_put_counter(struct sk_buff *skb, const struct ip_set_counter *counter)
540+{
541+ return nla_put_net64(skb, IPSET_ATTR_BYTES,
542+ cpu_to_be64(ip_set_get_bytes(counter)),
543+ IPSET_ATTR_PAD) ||
544+ nla_put_net64(skb, IPSET_ATTR_PACKETS,
545+ cpu_to_be64(ip_set_get_packets(counter)),
546+ IPSET_ATTR_PAD);
547+}
548+
549+static bool
550+ip_set_put_skbinfo(struct sk_buff *skb, const struct ip_set_skbinfo *skbinfo)
551+{
552+ /* Send nonzero parameters only */
553+ return ((skbinfo->skbmark || skbinfo->skbmarkmask) &&
554+ nla_put_net64(skb, IPSET_ATTR_SKBMARK,
555+ cpu_to_be64((u64)skbinfo->skbmark << 32 |
556+ skbinfo->skbmarkmask),
557+ IPSET_ATTR_PAD)) ||
558+ (skbinfo->skbprio &&
559+ nla_put_net32(skb, IPSET_ATTR_SKBPRIO,
560+ cpu_to_be32(skbinfo->skbprio))) ||
561+ (skbinfo->skbqueue &&
562+ nla_put_net16(skb, IPSET_ATTR_SKBQUEUE,
563+ cpu_to_be16(skbinfo->skbqueue)));
564+}
565+
449566 int
450567 ip_set_put_extensions(struct sk_buff *skb, const struct ip_set *set,
451568 const void *e, bool active)
@@ -471,6 +588,55 @@ ip_set_put_extensions(struct sk_buff *skb, const struct ip_set *set, · 19 unmodified lines
471588 }
472589 EXPORT_SYMBOL_GPL(ip_set_put_extensions);
473590
591+static bool
592+ip_set_match_counter(u64 counter, u64 match, u8 op)
593+{
594+ switch (op) {
595+ case IPSET_COUNTER_NONE:
596+ return true;
597+ case IPSET_COUNTER_EQ:
598+ return counter == match;
599+ case IPSET_COUNTER_NE:
600+ return counter != match;
601+ case IPSET_COUNTER_LT:
602+ return counter < match;
603+ case IPSET_COUNTER_GT:
604+ return counter > match;
605+ }
606+ return false;
607+}
608+
609+static void
610+ip_set_add_bytes(u64 bytes, struct ip_set_counter *counter)
611+{
612+ atomic64_add((long long)bytes, &(counter)->bytes);
613+}
614+
615+static void
616+ip_set_add_packets(u64 packets, struct ip_set_counter *counter)
617+{
618+ atomic64_add((long long)packets, &(counter)->packets);
619+}
620+
621+static void
622+ip_set_update_counter(struct ip_set_counter *counter,
623+ const struct ip_set_ext *ext, u32 flags)
624+{
625+ if (ext->packets != ULLONG_MAX &&
626+ !(flags & IPSET_FLAG_SKIP_COUNTER_UPDATE)) {
627+ ip_set_add_bytes(ext->bytes, counter);
628+ ip_set_add_packets(ext->packets, counter);
629+ }
630+}
631+
632+static void
633+ip_set_get_skbinfo(struct ip_set_skbinfo *skbinfo,
634+ const struct ip_set_ext *ext,
635+ struct ip_set_ext *mext, u32 flags)
636+{
637+ mext->skbinfo = *skbinfo;
638+}
639+
474640 bool
475641 ip_set_match_extensions(struct ip_set *set, const struct ip_set_ext *ext,
476642 struct ip_set_ext *mext, u32 flags, void *data)
@@ -506,7 +672,7 @@ EXPORT_SYMBOL_GPL(ip_set_match_extensions); · 29 unmodified lines
506672 * The set behind an index may change by swapping only, from userspace.
507673 */
508674
509−static inline void
675+static void
510676 __ip_set_get(struct ip_set *set)
511677 {
512678 write_lock_bh(&ip_set_ref_lock);
@@ -514,7 +680,7 @@ __ip_set_get(struct ip_set *set) · 1 unmodified line
514680 write_unlock_bh(&ip_set_ref_lock);
515681 }
516682
517−static inline void
683+static void
518684 __ip_set_put(struct ip_set *set)
519685 {
520686 write_lock_bh(&ip_set_ref_lock);
@@ -526,7 +692,7 @@ __ip_set_put(struct ip_set *set) · 5 unmodified lines
526692 /* set->ref can be swapped out by ip_set_swap, netlink events (like dump) need
527693 * a separate reference counter
528694 */
529−static inline void
695+static void
530696 __ip_set_put_netlink(struct ip_set *set)
531697 {
532698 write_lock_bh(&ip_set_ref_lock);
@@ -541,7 +707,7 @@ __ip_set_put_netlink(struct ip_set *set) · 8 unmodified lines
541707 * so it can't be destroyed (or changed) under our foot.
542708 */
543709
544−static inline struct ip_set *
710+static struct ip_set *
545711 ip_set_rcu_get(struct net *net, ip_set_id_t index)
546712 {
547713 struct ip_set *set;
@@ -670,7 +836,7 @@ EXPORT_SYMBOL_GPL(ip_set_get_byname); · 122 unmodified lines
670836 *
671837 */
672838
673−static inline void
839+static void
674840 __ip_set_put_byindex(struct ip_set_net *inst, ip_set_id_t index)
675841 {
676842 struct ip_set *set;
@@ -1252,6 +1418,30 @@ static int ip_set_swap(struct net *net, struct sock *ctnl, struct sk_buff *skb, · 575 unmodified lines
12521418 #define DUMP_TYPE(arg) (((u32)(arg)) & 0x0000FFFF)
12531419 #define DUMP_FLAGS(arg) (((u32)(arg)) >> 16)
12541420
1421+int
1422+ip_set_put_flags(struct sk_buff *skb, struct ip_set *set)
1423+{
1424+ u32 cadt_flags = 0;
1425+
1426+ if (SET_WITH_TIMEOUT(set))
1427+ if (unlikely(nla_put_net32(skb, IPSET_ATTR_TIMEOUT,
1428+ htonl(set->timeout))))
1429+ return -EMSGSIZE;
1430+ if (SET_WITH_COUNTER(set))
1431+ cadt_flags |= IPSET_FLAG_WITH_COUNTERS;
1432+ if (SET_WITH_COMMENT(set))
1433+ cadt_flags |= IPSET_FLAG_WITH_COMMENT;
1434+ if (SET_WITH_SKBINFO(set))
1435+ cadt_flags |= IPSET_FLAG_WITH_SKBINFO;
1436+ if (SET_WITH_FORCEADD(set))
1437+ cadt_flags |= IPSET_FLAG_WITH_FORCEADD;
1438+
1439+ if (!cadt_flags)
1440+ return 0;
1441+ return nla_put_net32(skb, IPSET_ATTR_CADT_FLAGS, htonl(cadt_flags));
1442+}
1443+EXPORT_SYMBOL_GPL(ip_set_put_flags);
1444+
12551445 static int
12561446 ip_set_dump_done(struct netlink_callback *cb)
12571447 {
net/netfilter/ipset/ip_set_getport.c+0−28
@@ -148,31 +148,3 @@ ip_set_get_ip6_port(const struct sk_buff *skb, bool src, · 147 unmodified lines
148148 }
149149 EXPORT_SYMBOL_GPL(ip_set_get_ip6_port);
150150 #endif
151−
152−bool
153−ip_set_get_ip_port(const struct sk_buff *skb, u8 pf, bool src, __be16 *port)
154−{
155− bool ret;
156− u8 proto;
157−
158− switch (pf) {
159− case NFPROTO_IPV4:
160− ret = ip_set_get_ip4_port(skb, src, port, &proto);
161− break;
162− case NFPROTO_IPV6:
163− ret = ip_set_get_ip6_port(skb, src, port, &proto);
164− break;
165− default:
166− return false;
167− }
168− if (!ret)
169− return ret;
170− switch (proto) {
171− case IPPROTO_TCP:
172− case IPPROTO_UDP:
173− return true;
174− default:
175− return false;
176− }
177−}
178−EXPORT_SYMBOL_GPL(ip_set_get_ip_port);
net/netfilter/ipset/ip_set_hash_gen.h+2−2
@@ -39,7 +39,7 @@ · 38 unmodified lines
3939 #ifdef IP_SET_HASH_WITH_MULTI
4040 #define AHASH_MAX(h) ((h)->ahash_max)
4141
42−static inline u8
42+static u8
4343 tune_ahash_max(u8 curr, u32 multi)
4444 {
4545 u32 n;
@@ -909,7 +909,7 @@ out: · 863 unmodified lines
909909 return ret;
910910 }
911911
912−static inline int
912+static int
913913 mtype_data_match(struct mtype_elem *data, const struct ip_set_ext *ext,
914914 struct ip_set_ext *mext, struct ip_set *set, u32 flags)
915915 {
net/netfilter/ipset/ip_set_hash_ip.c+5−5
@@ -44,7 +44,7 @@ struct hash_ip4_elem { · 43 unmodified lines
4444
4545 /* Common functions */
4646
47−static inline bool
47+static bool
4848 hash_ip4_data_equal(const struct hash_ip4_elem *e1,
4949 const struct hash_ip4_elem *e2,
5050 u32 *multi)
@@ -63,7 +63,7 @@ nla_put_failure: · 12 unmodified lines
6363 return true;
6464 }
6565
66−static inline void
66+static void
6767 hash_ip4_data_next(struct hash_ip4_elem *next, const struct hash_ip4_elem *e)
6868 {
6969 next->ip = e->ip;
@@ -171,7 +171,7 @@ struct hash_ip6_elem { · 101 unmodified lines
171171
172172 /* Common functions */
173173
174−static inline bool
174+static bool
175175 hash_ip6_data_equal(const struct hash_ip6_elem *ip1,
176176 const struct hash_ip6_elem *ip2,
177177 u32 *multi)
@@ -179,7 +179,7 @@ hash_ip6_data_equal(const struct hash_ip6_elem *ip1, · 1 unmodified line
179179 return ipv6_addr_equal(&ip1->ip.in6, &ip2->ip.in6);
180180 }
181181
182−static inline void
182+static void
183183 hash_ip6_netmask(union nf_inet_addr *ip, u8 prefix)
184184 {
185185 ip6_netmask(ip, prefix);
@@ -196,7 +196,7 @@ nla_put_failure: · 10 unmodified lines
196196 return true;
197197 }
198198
199−static inline void
199+static void
200200 hash_ip6_data_next(struct hash_ip6_elem *next, const struct hash_ip6_elem *e)
201201 {
202202 }
net/netfilter/ipset/ip_set_hash_ipmac.c+4−4
@@ -47,7 +47,7 @@ struct hash_ipmac4_elem { · 46 unmodified lines
4747
4848 /* Common functions */
4949
50−static inline bool
50+static bool
5151 hash_ipmac4_data_equal(const struct hash_ipmac4_elem *e1,
5252 const struct hash_ipmac4_elem *e2,
5353 u32 *multi)
@@ -67,7 +67,7 @@ nla_put_failure: · 13 unmodified lines
6767 return true;
6868 }
6969
70−static inline void
70+static void
7171 hash_ipmac4_data_next(struct hash_ipmac4_elem *next,
7272 const struct hash_ipmac4_elem *e)
7373 {
@@ -154,7 +154,7 @@ struct hash_ipmac6_elem { · 80 unmodified lines
154154
155155 /* Common functions */
156156
157−static inline bool
157+static bool
158158 hash_ipmac6_data_equal(const struct hash_ipmac6_elem *e1,
159159 const struct hash_ipmac6_elem *e2,
160160 u32 *multi)
@@ -175,7 +175,7 @@ nla_put_failure: · 14 unmodified lines
175175 return true;
176176 }
177177
178−static inline void
178+static void
179179 hash_ipmac6_data_next(struct hash_ipmac6_elem *next,
180180 const struct hash_ipmac6_elem *e)
181181 {
net/netfilter/ipset/ip_set_hash_ipmark.c+4−4
@@ -42,7 +42,7 @@ struct hash_ipmark4_elem { · 41 unmodified lines
4242
4343 /* Common functions */
4444
45−static inline bool
45+static bool
4646 hash_ipmark4_data_equal(const struct hash_ipmark4_elem *ip1,
4747 const struct hash_ipmark4_elem *ip2,
4848 u32 *multi)
@@ -64,7 +64,7 @@ nla_put_failure: · 15 unmodified lines
6464 return true;
6565 }
6666
67−static inline void
67+static void
6868 hash_ipmark4_data_next(struct hash_ipmark4_elem *next,
6969 const struct hash_ipmark4_elem *d)
7070 {
@@ -165,7 +165,7 @@ struct hash_ipmark6_elem { · 94 unmodified lines
165165
166166 /* Common functions */
167167
168−static inline bool
168+static bool
169169 hash_ipmark6_data_equal(const struct hash_ipmark6_elem *ip1,
170170 const struct hash_ipmark6_elem *ip2,
171171 u32 *multi)
@@ -187,7 +187,7 @@ nla_put_failure: · 15 unmodified lines
187187 return true;
188188 }
189189
190−static inline void
190+static void
191191 hash_ipmark6_data_next(struct hash_ipmark6_elem *next,
192192 const struct hash_ipmark6_elem *d)
193193 {
net/netfilter/ipset/ip_set_hash_ipport.c+4−4
@@ -47,7 +47,7 @@ struct hash_ipport4_elem { · 46 unmodified lines
4747
4848 /* Common functions */
4949
50−static inline bool
50+static bool
5151 hash_ipport4_data_equal(const struct hash_ipport4_elem *ip1,
5252 const struct hash_ipport4_elem *ip2,
5353 u32 *multi)
@@ -71,7 +71,7 @@ nla_put_failure: · 17 unmodified lines
7171 return true;
7272 }
7373
74−static inline void
74+static void
7575 hash_ipport4_data_next(struct hash_ipport4_elem *next,
7676 const struct hash_ipport4_elem *d)
7777 {
@@ -202,7 +202,7 @@ struct hash_ipport6_elem { · 124 unmodified lines
202202
203203 /* Common functions */
204204
205−static inline bool
205+static bool
206206 hash_ipport6_data_equal(const struct hash_ipport6_elem *ip1,
207207 const struct hash_ipport6_elem *ip2,
208208 u32 *multi)
@@ -226,7 +226,7 @@ nla_put_failure: · 17 unmodified lines
226226 return true;
227227 }
228228
229−static inline void
229+static void
230230 hash_ipport6_data_next(struct hash_ipport6_elem *next,
231231 const struct hash_ipport6_elem *d)
232232 {
net/netfilter/ipset/ip_set_hash_ipportip.c+4−4
@@ -46,7 +46,7 @@ struct hash_ipportip4_elem { · 45 unmodified lines
4646 u8 padding;
4747 };
4848
49−static inline bool
49+static bool
5050 hash_ipportip4_data_equal(const struct hash_ipportip4_elem *ip1,
5151 const struct hash_ipportip4_elem *ip2,
5252 u32 *multi)
@@ -72,7 +72,7 @@ nla_put_failure: · 19 unmodified lines
7272 return true;
7373 }
7474
75−static inline void
75+static void
7676 hash_ipportip4_data_next(struct hash_ipportip4_elem *next,
7777 const struct hash_ipportip4_elem *d)
7878 {
@@ -210,7 +210,7 @@ struct hash_ipportip6_elem { · 131 unmodified lines
210210
211211 /* Common functions */
212212
213−static inline bool
213+static bool
214214 hash_ipportip6_data_equal(const struct hash_ipportip6_elem *ip1,
215215 const struct hash_ipportip6_elem *ip2,
216216 u32 *multi)
@@ -236,7 +236,7 @@ nla_put_failure: · 19 unmodified lines
236236 return true;
237237 }
238238
239−static inline void
239+static void
240240 hash_ipportip6_data_next(struct hash_ipportip6_elem *next,
241241 const struct hash_ipportip6_elem *d)
242242 {
net/netfilter/ipset/ip_set_hash_ipportnet.c+12−12
@@ -59,7 +59,7 @@ struct hash_ipportnet4_elem { · 58 unmodified lines
5959
6060 /* Common functions */
6161
62−static inline bool
62+static bool
6363 hash_ipportnet4_data_equal(const struct hash_ipportnet4_elem *ip1,
6464 const struct hash_ipportnet4_elem *ip2,
6565 u32 *multi)
@@ -71,25 +71,25 @@ hash_ipportnet4_data_equal(const struct hash_ipportnet4_elem *ip1, · 5 unmodified lines
7171 ip1->proto == ip2->proto;
7272 }
7373
74−static inline int
74+static int
7575 hash_ipportnet4_do_data_match(const struct hash_ipportnet4_elem *elem)
7676 {
7777 return elem->nomatch ? -ENOTEMPTY : 1;
7878 }
7979
80−static inline void
80+static void
8181 hash_ipportnet4_data_set_flags(struct hash_ipportnet4_elem *elem, u32 flags)
8282 {
8383 elem->nomatch = !!((flags >> 16) & IPSET_FLAG_NOMATCH);
8484 }
8585
86−static inline void
86+static void
8787 hash_ipportnet4_data_reset_flags(struct hash_ipportnet4_elem *elem, u8 *flags)
8888 {
8989 swap(*flags, elem->nomatch);
9090 }
9191
92−static inline void
92+static void
9393 hash_ipportnet4_data_netmask(struct hash_ipportnet4_elem *elem, u8 cidr)
9494 {
9595 elem->ip2 &= ip_set_netmask(cidr);
@@ -116,7 +116,7 @@ nla_put_failure: · 20 unmodified lines
116116 return true;
117117 }
118118
119−static inline void
119+static void
120120 hash_ipportnet4_data_next(struct hash_ipportnet4_elem *next,
121121 const struct hash_ipportnet4_elem *d)
122122 {
@@ -308,7 +308,7 @@ struct hash_ipportnet6_elem { · 185 unmodified lines
308308
309309 /* Common functions */
310310
311−static inline bool
311+static bool
312312 hash_ipportnet6_data_equal(const struct hash_ipportnet6_elem *ip1,
313313 const struct hash_ipportnet6_elem *ip2,
314314 u32 *multi)
@@ -320,25 +320,25 @@ hash_ipportnet6_data_equal(const struct hash_ipportnet6_elem *ip1, · 5 unmodified lines
320320 ip1->proto == ip2->proto;
321321 }
322322
323−static inline int
323+static int
324324 hash_ipportnet6_do_data_match(const struct hash_ipportnet6_elem *elem)
325325 {
326326 return elem->nomatch ? -ENOTEMPTY : 1;
327327 }
328328
329−static inline void
329+static void
330330 hash_ipportnet6_data_set_flags(struct hash_ipportnet6_elem *elem, u32 flags)
331331 {
332332 elem->nomatch = !!((flags >> 16) & IPSET_FLAG_NOMATCH);
333333 }
334334
335−static inline void
335+static void
336336 hash_ipportnet6_data_reset_flags(struct hash_ipportnet6_elem *elem, u8 *flags)
337337 {
338338 swap(*flags, elem->nomatch);
339339 }
340340
341−static inline void
341+static void
342342 hash_ipportnet6_data_netmask(struct hash_ipportnet6_elem *elem, u8 cidr)
343343 {
344344 ip6_netmask(&elem->ip2, cidr);
@@ -365,7 +365,7 @@ nla_put_failure: · 20 unmodified lines
365365 return true;
366366 }
367367
368−static inline void
368+static void
369369 hash_ipportnet6_data_next(struct hash_ipportnet6_elem *next,
370370 const struct hash_ipportnet6_elem *d)
371371 {
net/netfilter/ipset/ip_set_hash_mac.c+3−3
@@ -37,7 +37,7 @@ struct hash_mac4_elem { · 36 unmodified lines
3737
3838 /* Common functions */
3939
40−static inline bool
40+static bool
4141 hash_mac4_data_equal(const struct hash_mac4_elem *e1,
4242 const struct hash_mac4_elem *e2,
4343 u32 *multi)
@@ -45,7 +45,7 @@ hash_mac4_data_equal(const struct hash_mac4_elem *e1, · 1 unmodified line
4545 return ether_addr_equal(e1->ether, e2->ether);
4646 }
4747
48−static inline bool
48+static bool
4949 hash_mac4_data_list(struct sk_buff *skb, const struct hash_mac4_elem *e)
5050 {
5151 if (nla_put(skb, IPSET_ATTR_ETHER, ETH_ALEN, e->ether))
@@ -56,7 +56,7 @@ nla_put_failure: · 4 unmodified lines
5656 return true;
5757 }
5858
59−static inline void
59+static void
6060 hash_mac4_data_next(struct hash_mac4_elem *next,
6161 const struct hash_mac4_elem *e)
6262 {
net/netfilter/ipset/ip_set_hash_net.c+12−12
@@ -47,7 +47,7 @@ struct hash_net4_elem { · 46 unmodified lines
4747
4848 /* Common functions */
4949
50−static inline bool
50+static bool
5151 hash_net4_data_equal(const struct hash_net4_elem *ip1,
5252 const struct hash_net4_elem *ip2,
5353 u32 *multi)
@@ -56,25 +56,25 @@ hash_net4_data_equal(const struct hash_net4_elem *ip1, · 2 unmodified lines
5656 ip1->cidr == ip2->cidr;
5757 }
5858
59−static inline int
59+static int
6060 hash_net4_do_data_match(const struct hash_net4_elem *elem)
6161 {
6262 return elem->nomatch ? -ENOTEMPTY : 1;
6363 }
6464
65−static inline void
65+static void
6666 hash_net4_data_set_flags(struct hash_net4_elem *elem, u32 flags)
6767 {
6868 elem->nomatch = (flags >> 16) & IPSET_FLAG_NOMATCH;
6969 }
7070
71−static inline void
71+static void
7272 hash_net4_data_reset_flags(struct hash_net4_elem *elem, u8 *flags)
7373 {
7474 swap(*flags, elem->nomatch);
7575 }
7676
77−static inline void
77+static void
7878 hash_net4_data_netmask(struct hash_net4_elem *elem, u8 cidr)
7979 {
8080 elem->ip &= ip_set_netmask(cidr);
@@ -97,7 +97,7 @@ nla_put_failure: · 16 unmodified lines
9797 return true;
9898 }
9999
100−static inline void
100+static void
101101 hash_net4_data_next(struct hash_net4_elem *next,
102102 const struct hash_net4_elem *d)
103103 {
@@ -212,7 +212,7 @@ struct hash_net6_elem { · 108 unmodified lines
212212
213213 /* Common functions */
214214
215−static inline bool
215+static bool
216216 hash_net6_data_equal(const struct hash_net6_elem *ip1,
217217 const struct hash_net6_elem *ip2,
218218 u32 *multi)
@@ -221,25 +221,25 @@ hash_net6_data_equal(const struct hash_net6_elem *ip1, · 2 unmodified lines
221221 ip1->cidr == ip2->cidr;
222222 }
223223
224−static inline int
224+static int
225225 hash_net6_do_data_match(const struct hash_net6_elem *elem)
226226 {
227227 return elem->nomatch ? -ENOTEMPTY : 1;
228228 }
229229
230−static inline void
230+static void
231231 hash_net6_data_set_flags(struct hash_net6_elem *elem, u32 flags)
232232 {
233233 elem->nomatch = (flags >> 16) & IPSET_FLAG_NOMATCH;
234234 }
235235
236−static inline void
236+static void
237237 hash_net6_data_reset_flags(struct hash_net6_elem *elem, u8 *flags)
238238 {
239239 swap(*flags, elem->nomatch);
240240 }
241241
242−static inline void
242+static void
243243 hash_net6_data_netmask(struct hash_net6_elem *elem, u8 cidr)
244244 {
245245 ip6_netmask(&elem->ip, cidr);
@@ -262,7 +262,7 @@ nla_put_failure: · 16 unmodified lines
262262 return true;
263263 }
264264
265−static inline void
265+static void
266266 hash_net6_data_next(struct hash_net6_elem *next,
267267 const struct hash_net6_elem *d)
268268 {
net/netfilter/ipset/ip_set_hash_netiface.c+12−12
@@ -62,7 +62,7 @@ struct hash_netiface4_elem { · 61 unmodified lines
6262
6363 /* Common functions */
6464
65−static inline bool
65+static bool
6666 hash_netiface4_data_equal(const struct hash_netiface4_elem *ip1,
6767 const struct hash_netiface4_elem *ip2,
6868 u32 *multi)
@@ -74,25 +74,25 @@ hash_netiface4_data_equal(const struct hash_netiface4_elem *ip1, · 5 unmodified lines
7474 strcmp(ip1->iface, ip2->iface) == 0;
7575 }
7676
77−static inline int
77+static int
7878 hash_netiface4_do_data_match(const struct hash_netiface4_elem *elem)
7979 {
8080 return elem->nomatch ? -ENOTEMPTY : 1;
8181 }
8282
83−static inline void
83+static void
8484 hash_netiface4_data_set_flags(struct hash_netiface4_elem *elem, u32 flags)
8585 {
8686 elem->nomatch = (flags >> 16) & IPSET_FLAG_NOMATCH;
8787 }
8888
89−static inline void
89+static void
9090 hash_netiface4_data_reset_flags(struct hash_netiface4_elem *elem, u8 *flags)
9191 {
9292 swap(*flags, elem->nomatch);
9393 }
9494
95−static inline void
95+static void
9696 hash_netiface4_data_netmask(struct hash_netiface4_elem *elem, u8 cidr)
9797 {
9898 elem->ip &= ip_set_netmask(cidr);
@@ -119,7 +119,7 @@ nla_put_failure: · 20 unmodified lines
119119 return true;
120120 }
121121
122−static inline void
122+static void
123123 hash_netiface4_data_next(struct hash_netiface4_elem *next,
124124 const struct hash_netiface4_elem *d)
125125 {
@@ -285,7 +285,7 @@ struct hash_netiface6_elem { · 159 unmodified lines
285285
286286 /* Common functions */
287287
288−static inline bool
288+static bool
289289 hash_netiface6_data_equal(const struct hash_netiface6_elem *ip1,
290290 const struct hash_netiface6_elem *ip2,
291291 u32 *multi)
@@ -297,25 +297,25 @@ hash_netiface6_data_equal(const struct hash_netiface6_elem *ip1, · 5 unmodified lines
297297 strcmp(ip1->iface, ip2->iface) == 0;
298298 }
299299
300−static inline int
300+static int
301301 hash_netiface6_do_data_match(const struct hash_netiface6_elem *elem)
302302 {
303303 return elem->nomatch ? -ENOTEMPTY : 1;
304304 }
305305
306−static inline void
306+static void
307307 hash_netiface6_data_set_flags(struct hash_netiface6_elem *elem, u32 flags)
308308 {
309309 elem->nomatch = (flags >> 16) & IPSET_FLAG_NOMATCH;
310310 }
311311
312−static inline void
312+static void
313313 hash_netiface6_data_reset_flags(struct hash_netiface6_elem *elem, u8 *flags)
314314 {
315315 swap(*flags, elem->nomatch);
316316 }
317317
318−static inline void
318+static void
319319 hash_netiface6_data_netmask(struct hash_netiface6_elem *elem, u8 cidr)
320320 {
321321 ip6_netmask(&elem->ip, cidr);
@@ -342,7 +342,7 @@ nla_put_failure: · 20 unmodified lines
342342 return true;
343343 }
344344
345−static inline void
345+static void
346346 hash_netiface6_data_next(struct hash_netiface6_elem *next,
347347 const struct hash_netiface6_elem *d)
348348 {
net/netfilter/ipset/ip_set_hash_netnet.c+14−14
@@ -52,7 +52,7 @@ struct hash_netnet4_elem { · 51 unmodified lines
5252
5353 /* Common functions */
5454
55−static inline bool
55+static bool
5656 hash_netnet4_data_equal(const struct hash_netnet4_elem *ip1,
5757 const struct hash_netnet4_elem *ip2,
5858 u32 *multi)
@@ -61,32 +61,32 @@ hash_netnet4_data_equal(const struct hash_netnet4_elem *ip1, · 2 unmodified lines
6161 ip1->ccmp == ip2->ccmp;
6262 }
6363
64−static inline int
64+static int
6565 hash_netnet4_do_data_match(const struct hash_netnet4_elem *elem)
6666 {
6767 return elem->nomatch ? -ENOTEMPTY : 1;
6868 }
6969
70−static inline void
70+static void
7171 hash_netnet4_data_set_flags(struct hash_netnet4_elem *elem, u32 flags)
7272 {
7373 elem->nomatch = (flags >> 16) & IPSET_FLAG_NOMATCH;
7474 }
7575
76−static inline void
76+static void
7777 hash_netnet4_data_reset_flags(struct hash_netnet4_elem *elem, u8 *flags)
7878 {
7979 swap(*flags, elem->nomatch);
8080 }
8181
82−static inline void
82+static void
8383 hash_netnet4_data_reset_elem(struct hash_netnet4_elem *elem,
8484 struct hash_netnet4_elem *orig)
8585 {
8686 elem->ip[1] = orig->ip[1];
8787 }
8888
89−static inline void
89+static void
9090 hash_netnet4_data_netmask(struct hash_netnet4_elem *elem, u8 cidr, bool inner)
9191 {
9292 if (inner) {
@@ -117,7 +117,7 @@ nla_put_failure: · 24 unmodified lines
117117 return true;
118118 }
119119
120−static inline void
120+static void
121121 hash_netnet4_data_next(struct hash_netnet4_elem *next,
122122 const struct hash_netnet4_elem *d)
123123 {
@@ -282,7 +282,7 @@ struct hash_netnet6_elem { · 158 unmodified lines
282282
283283 /* Common functions */
284284
285−static inline bool
285+static bool
286286 hash_netnet6_data_equal(const struct hash_netnet6_elem *ip1,
287287 const struct hash_netnet6_elem *ip2,
288288 u32 *multi)
@@ -292,32 +292,32 @@ hash_netnet6_data_equal(const struct hash_netnet6_elem *ip1, · 3 unmodified lines
292292 ip1->ccmp == ip2->ccmp;
293293 }
294294
295−static inline int
295+static int
296296 hash_netnet6_do_data_match(const struct hash_netnet6_elem *elem)
297297 {
298298 return elem->nomatch ? -ENOTEMPTY : 1;
299299 }
300300
301−static inline void
301+static void
302302 hash_netnet6_data_set_flags(struct hash_netnet6_elem *elem, u32 flags)
303303 {
304304 elem->nomatch = (flags >> 16) & IPSET_FLAG_NOMATCH;
305305 }
306306
307−static inline void
307+static void
308308 hash_netnet6_data_reset_flags(struct hash_netnet6_elem *elem, u8 *flags)
309309 {
310310 swap(*flags, elem->nomatch);
311311 }
312312
313−static inline void
313+static void
314314 hash_netnet6_data_reset_elem(struct hash_netnet6_elem *elem,
315315 struct hash_netnet6_elem *orig)
316316 {
317317 elem->ip[1] = orig->ip[1];
318318 }
319319
320−static inline void
320+static void
321321 hash_netnet6_data_netmask(struct hash_netnet6_elem *elem, u8 cidr, bool inner)
322322 {
323323 if (inner) {
@@ -348,7 +348,7 @@ nla_put_failure: · 24 unmodified lines
348348 return true;
349349 }
350350
351−static inline void
351+static void
352352 hash_netnet6_data_next(struct hash_netnet6_elem *next,
353353 const struct hash_netnet6_elem *d)
354354 {
net/netfilter/ipset/ip_set_hash_netport.c+12−12
@@ -57,7 +57,7 @@ struct hash_netport4_elem { · 56 unmodified lines
5757
5858 /* Common functions */
5959
60−static inline bool
60+static bool
6161 hash_netport4_data_equal(const struct hash_netport4_elem *ip1,
6262 const struct hash_netport4_elem *ip2,
6363 u32 *multi)
@@ -68,25 +68,25 @@ hash_netport4_data_equal(const struct hash_netport4_elem *ip1, · 4 unmodified lines
6868 ip1->cidr == ip2->cidr;
6969 }
7070
71−static inline int
71+static int
7272 hash_netport4_do_data_match(const struct hash_netport4_elem *elem)
7373 {
7474 return elem->nomatch ? -ENOTEMPTY : 1;
7575 }
7676
77−static inline void
77+static void
7878 hash_netport4_data_set_flags(struct hash_netport4_elem *elem, u32 flags)
7979 {
8080 elem->nomatch = !!((flags >> 16) & IPSET_FLAG_NOMATCH);
8181 }
8282
83−static inline void
83+static void
8484 hash_netport4_data_reset_flags(struct hash_netport4_elem *elem, u8 *flags)
8585 {
8686 swap(*flags, elem->nomatch);
8787 }
8888
89−static inline void
89+static void
9090 hash_netport4_data_netmask(struct hash_netport4_elem *elem, u8 cidr)
9191 {
9292 elem->ip &= ip_set_netmask(cidr);
@@ -112,7 +112,7 @@ nla_put_failure: · 19 unmodified lines
112112 return true;
113113 }
114114
115−static inline void
115+static void
116116 hash_netport4_data_next(struct hash_netport4_elem *next,
117117 const struct hash_netport4_elem *d)
118118 {
@@ -270,7 +270,7 @@ struct hash_netport6_elem { · 151 unmodified lines
270270
271271 /* Common functions */
272272
273−static inline bool
273+static bool
274274 hash_netport6_data_equal(const struct hash_netport6_elem *ip1,
275275 const struct hash_netport6_elem *ip2,
276276 u32 *multi)
@@ -281,25 +281,25 @@ hash_netport6_data_equal(const struct hash_netport6_elem *ip1, · 4 unmodified lines
281281 ip1->cidr == ip2->cidr;
282282 }
283283
284−static inline int
284+static int
285285 hash_netport6_do_data_match(const struct hash_netport6_elem *elem)
286286 {
287287 return elem->nomatch ? -ENOTEMPTY : 1;
288288 }
289289
290−static inline void
290+static void
291291 hash_netport6_data_set_flags(struct hash_netport6_elem *elem, u32 flags)
292292 {
293293 elem->nomatch = !!((flags >> 16) & IPSET_FLAG_NOMATCH);
294294 }
295295
296−static inline void
296+static void
297297 hash_netport6_data_reset_flags(struct hash_netport6_elem *elem, u8 *flags)
298298 {
299299 swap(*flags, elem->nomatch);
300300 }
301301
302−static inline void
302+static void
303303 hash_netport6_data_netmask(struct hash_netport6_elem *elem, u8 cidr)
304304 {
305305 ip6_netmask(&elem->ip, cidr);
@@ -325,7 +325,7 @@ nla_put_failure: · 19 unmodified lines
325325 return true;
326326 }
327327
328−static inline void
328+static void
329329 hash_netport6_data_next(struct hash_netport6_elem *next,
330330 const struct hash_netport6_elem *d)
331331 {
net/netfilter/ipset/ip_set_hash_netportnet.c+14−14
@@ -56,7 +56,7 @@ struct hash_netportnet4_elem { · 55 unmodified lines
5656
5757 /* Common functions */
5858
59−static inline bool
59+static bool
6060 hash_netportnet4_data_equal(const struct hash_netportnet4_elem *ip1,
6161 const struct hash_netportnet4_elem *ip2,
6262 u32 *multi)
@@ -67,32 +67,32 @@ hash_netportnet4_data_equal(const struct hash_netportnet4_elem *ip1, · 4 unmodified lines
6767 ip1->proto == ip2->proto;
6868 }
6969
70−static inline int
70+static int
7171 hash_netportnet4_do_data_match(const struct hash_netportnet4_elem *elem)
7272 {
7373 return elem->nomatch ? -ENOTEMPTY : 1;
7474 }
7575
76−static inline void
76+static void
7777 hash_netportnet4_data_set_flags(struct hash_netportnet4_elem *elem, u32 flags)
7878 {
7979 elem->nomatch = !!((flags >> 16) & IPSET_FLAG_NOMATCH);
8080 }
8181
82−static inline void
82+static void
8383 hash_netportnet4_data_reset_flags(struct hash_netportnet4_elem *elem, u8 *flags)
8484 {
8585 swap(*flags, elem->nomatch);
8686 }
8787
88−static inline void
88+static void
8989 hash_netportnet4_data_reset_elem(struct hash_netportnet4_elem *elem,
9090 struct hash_netportnet4_elem *orig)
9191 {
9292 elem->ip[1] = orig->ip[1];
9393 }
9494
95−static inline void
95+static void
9696 hash_netportnet4_data_netmask(struct hash_netportnet4_elem *elem,
9797 u8 cidr, bool inner)
9898 {
@@ -126,7 +126,7 @@ nla_put_failure: · 27 unmodified lines
126126 return true;
127127 }
128128
129−static inline void
129+static void
130130 hash_netportnet4_data_next(struct hash_netportnet4_elem *next,
131131 const struct hash_netportnet4_elem *d)
132132 {
@@ -331,7 +331,7 @@ struct hash_netportnet6_elem { · 198 unmodified lines
331331
332332 /* Common functions */
333333
334−static inline bool
334+static bool
335335 hash_netportnet6_data_equal(const struct hash_netportnet6_elem *ip1,
336336 const struct hash_netportnet6_elem *ip2,
337337 u32 *multi)
@@ -343,32 +343,32 @@ hash_netportnet6_data_equal(const struct hash_netportnet6_elem *ip1, · 5 unmodified lines
343343 ip1->proto == ip2->proto;
344344 }
345345
346−static inline int
346+static int
347347 hash_netportnet6_do_data_match(const struct hash_netportnet6_elem *elem)
348348 {
349349 return elem->nomatch ? -ENOTEMPTY : 1;
350350 }
351351
352−static inline void
352+static void
353353 hash_netportnet6_data_set_flags(struct hash_netportnet6_elem *elem, u32 flags)
354354 {
355355 elem->nomatch = !!((flags >> 16) & IPSET_FLAG_NOMATCH);
356356 }
357357
358−static inline void
358+static void
359359 hash_netportnet6_data_reset_flags(struct hash_netportnet6_elem *elem, u8 *flags)
360360 {
361361 swap(*flags, elem->nomatch);
362362 }
363363
364−static inline void
364+static void
365365 hash_netportnet6_data_reset_elem(struct hash_netportnet6_elem *elem,
366366 struct hash_netportnet6_elem *orig)
367367 {
368368 elem->ip[1] = orig->ip[1];
369369 }
370370
371−static inline void
371+static void
372372 hash_netportnet6_data_netmask(struct hash_netportnet6_elem *elem,
373373 u8 cidr, bool inner)
374374 {
@@ -402,7 +402,7 @@ nla_put_failure: · 27 unmodified lines
402402 return true;
403403 }
404404
405−static inline void
405+static void
406406 hash_netportnet6_data_next(struct hash_netportnet6_elem *next,
407407 const struct hash_netportnet6_elem *d)
408408 {
net/netfilter/ipset/ip_set_list_set.c+2−2
@@ -149,7 +149,7 @@ __list_set_del_rcu(struct rcu_head * rcu) · 148 unmodified lines
149149 kfree(e);
150150 }
151151
152−static inline void
152+static void
153153 list_set_del(struct ip_set *set, struct set_elem *e)
154154 {
155155 struct list_set *map = set->data;
@@ -160,7 +160,7 @@ list_set_del(struct ip_set *set, struct set_elem *e) · 4 unmodified lines
160160 call_rcu(&e->rcu, __list_set_del_rcu);
161161 }
162162
163−static inline void
163+static void
164164 list_set_replace(struct ip_set *set, struct set_elem *e, struct set_elem *old)
165165 {
166166 struct list_set *map = set->data;
net/netfilter/ipvs/ip_vs_core.c+28−19
@@ -2402,18 +2402,22 @@ estimator_fail: · 2401 unmodified lines
24022402 return -ENOMEM;
24032403 }
24042404
2405−static void __net_exit __ip_vs_cleanup(struct net *net)
2405+static void __net_exit __ip_vs_cleanup_batch(struct list_head *net_list)
24062406 {
2407− struct netns_ipvs *ipvs = net_ipvs(net);
2408−
2409− ip_vs_service_net_cleanup(ipvs); /* ip_vs_flush() with locks */
2410− ip_vs_conn_net_cleanup(ipvs);
2411− ip_vs_app_net_cleanup(ipvs);
2412− ip_vs_protocol_net_cleanup(ipvs);
2413− ip_vs_control_net_cleanup(ipvs);
2414− ip_vs_estimator_net_cleanup(ipvs);
2415− IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen);
2416− net->ipvs = NULL;
2407+ struct netns_ipvs *ipvs;
2408+ struct net *net;
2409+
2410+ ip_vs_service_nets_cleanup(net_list); /* ip_vs_flush() with locks */
2411+ list_for_each_entry(net, net_list, exit_list) {
2412+ ipvs = net_ipvs(net);
2413+ ip_vs_conn_net_cleanup(ipvs);
2414+ ip_vs_app_net_cleanup(ipvs);
2415+ ip_vs_protocol_net_cleanup(ipvs);
2416+ ip_vs_control_net_cleanup(ipvs);
2417+ ip_vs_estimator_net_cleanup(ipvs);
2418+ IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen);
2419+ net->ipvs = NULL;
2420+ }
24172421 }
24182422
24192423 static int __net_init __ip_vs_dev_init(struct net *net)
@@ -2429,27 +2433,32 @@ hook_fail: · 9 unmodified lines
24292433 return ret;
24302434 }
24312435
2432−static void __net_exit __ip_vs_dev_cleanup(struct net *net)
2436+static void __net_exit __ip_vs_dev_cleanup_batch(struct list_head *net_list)
24332437 {
2434− struct netns_ipvs *ipvs = net_ipvs(net);
2438+ struct netns_ipvs *ipvs;
2439+ struct net *net;
2440+
24352441 EnterFunction(2);
2436− nf_unregister_net_hooks(net, ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
2437− ipvs->enable = 0; /* Disable packet reception */
2438− smp_wmb();
2439− ip_vs_sync_net_cleanup(ipvs);
2442+ list_for_each_entry(net, net_list, exit_list) {
2443+ ipvs = net_ipvs(net);
2444+ nf_unregister_net_hooks(net, ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
2445+ ipvs->enable = 0; /* Disable packet reception */
2446+ smp_wmb();
2447+ ip_vs_sync_net_cleanup(ipvs);
2448+ }
24402449 LeaveFunction(2);
24412450 }
24422451
24432452 static struct pernet_operations ipvs_core_ops = {
24442453 .init = __ip_vs_init,
2445− .exit = __ip_vs_cleanup,
2454+ .exit_batch = __ip_vs_cleanup_batch,
24462455 .id = &ip_vs_net_id,
24472456 .size = sizeof(struct netns_ipvs),
24482457 };
24492458
24502459 static struct pernet_operations ipvs_core_dev_ops = {
24512460 .init = __ip_vs_dev_init,
2452− .exit = __ip_vs_dev_cleanup,
2461+ .exit_batch = __ip_vs_dev_cleanup_batch,
24532462 };
24542463
24552464 /*
net/netfilter/ipvs/ip_vs_ctl.c+9−3
@@ -1607,14 +1607,20 @@ static int ip_vs_flush(struct netns_ipvs *ipvs, bool cleanup) · 1606 unmodified lines
16071607
16081608 /*
16091609 * Delete service by {netns} in the service table.
1610− * Called by __ip_vs_cleanup()
1610+ * Called by __ip_vs_batch_cleanup()
16111611 */
1612−void ip_vs_service_net_cleanup(struct netns_ipvs *ipvs)
1612+void ip_vs_service_nets_cleanup(struct list_head *net_list)
16131613 {
1614+ struct netns_ipvs *ipvs;
1615+ struct net *net;
1616+
16141617 EnterFunction(2);
16151618 /* Check for "full" addressed entries */
16161619 mutex_lock(&__ip_vs_mutex);
1617− ip_vs_flush(ipvs, true);
1620+ list_for_each_entry(net, net_list, exit_list) {
1621+ ipvs = net_ipvs(net);
1622+ ip_vs_flush(ipvs, true);
1623+ }
16181624 mutex_unlock(&__ip_vs_mutex);
16191625 LeaveFunction(2);
16201626 }
net/netfilter/ipvs/ip_vs_xmit.c+6−12
@@ -407,12 +407,9 @@ __ip_vs_get_out_rt(struct netns_ipvs *ipvs, int skb_af, struct sk_buff *skb, · 406 unmodified lines
407407 goto err_put;
408408
409409 skb_dst_drop(skb);
410− if (noref) {
411− if (!local)
412− skb_dst_set_noref(skb, &rt->dst);
413− else
414− skb_dst_set(skb, dst_clone(&rt->dst));
415− } else
410+ if (noref)
411+ skb_dst_set_noref(skb, &rt->dst);
412+ else
416413 skb_dst_set(skb, &rt->dst);
417414
418415 return local;
@@ -574,12 +571,9 @@ __ip_vs_get_out_rt_v6(struct netns_ipvs *ipvs, int skb_af, struct sk_buff *skb, · 155 unmodified lines
574571 goto err_put;
575572
576573 skb_dst_drop(skb);
577− if (noref) {
578− if (!local)
579− skb_dst_set_noref(skb, &rt->dst);
580− else
581− skb_dst_set(skb, dst_clone(&rt->dst));
582− } else
574+ if (noref)
575+ skb_dst_set_noref(skb, &rt->dst);
576+ else
583577 skb_dst_set(skb, &rt->dst);
584578
585579 return local;
net/netfilter/nf_conntrack_core.c+0−2
@@ -573,7 +573,6 @@ EXPORT_SYMBOL_GPL(nf_ct_tmpl_alloc); · 572 unmodified lines
573573 void nf_ct_tmpl_free(struct nf_conn *tmpl)
574574 {
575575 nf_ct_ext_destroy(tmpl);
576− nf_ct_ext_free(tmpl);
577576
578577 if (ARCH_KMALLOC_MINALIGN <= NFCT_INFOMASK)
579578 kfree((char *)tmpl - tmpl->proto.tmpl_padto);
@@ -1417,7 +1416,6 @@ void nf_conntrack_free(struct nf_conn *ct) · 837 unmodified lines
14171416 WARN_ON(atomic_read(&ct->ct_general.use) != 0);
14181417
14191418 nf_ct_ext_destroy(ct);
1420− nf_ct_ext_free(ct);
14211419 kmem_cache_free(nf_conntrack_cachep, ct);
14221420 smp_mb__before_atomic();
14231421 atomic_dec(&net->ct.count);
net/netfilter/nf_conntrack_ecache.c+18−5
@@ -30,6 +30,7 @@ · 29 unmodified lines
3030 static DEFINE_MUTEX(nf_ct_ecache_mutex);
3131
3232 #define ECACHE_RETRY_WAIT (HZ/10)
33+#define ECACHE_STACK_ALLOC (256 / sizeof(void *))
3334
3435 enum retry_state {
3536 STATE_CONGESTED,
@@ -39,11 +40,11 @@ enum retry_state { · 3 unmodified lines
3940
4041 static enum retry_state ecache_work_evict_list(struct ct_pcpu *pcpu)
4142 {
42− struct nf_conn *refs[16];
43+ struct nf_conn *refs[ECACHE_STACK_ALLOC];
44+ enum retry_state ret = STATE_DONE;
4345 struct nf_conntrack_tuple_hash *h;
4446 struct hlist_nulls_node *n;
4547 unsigned int evicted = 0;
46− enum retry_state ret = STATE_DONE;
4748
4849 spin_lock(&pcpu->lock);
4950
@@ -54,10 +55,22 @@ static enum retry_state ecache_work_evict_list(struct ct_pcpu *pcpu) · 4 unmodified lines
5455 if (!nf_ct_is_confirmed(ct))
5556 continue;
5657
58+ /* This ecache access is safe because the ct is on the
59+ * pcpu dying list and we hold the spinlock -- the entry
60+ * cannot be free'd until after the lock is released.
61+ *
62+ * This is true even if ct has a refcount of 0: the
63+ * cpu that is about to free the entry must remove it
64+ * from the dying list and needs the lock to do so.
65+ */
5766 e = nf_ct_ecache_find(ct);
5867 if (!e || e->state != NFCT_ECACHE_DESTROY_FAIL)
5968 continue;
6069
70+ /* ct is in NFCT_ECACHE_DESTROY_FAIL state, this means
71+ * the worker owns this entry: the ct will remain valid
72+ * until the worker puts its ct reference.
73+ */
6174 if (nf_conntrack_event(IPCT_DESTROY, ct)) {
6275 ret = STATE_CONGESTED;
6376 break;
@@ -189,15 +202,15 @@ void nf_ct_deliver_cached_events(struct nf_conn *ct) · 125 unmodified lines
189202 if (notify == NULL)
190203 goto out_unlock;
191204
205+ if (!nf_ct_is_confirmed(ct) || nf_ct_is_dying(ct))
206+ goto out_unlock;
207+
192208 e = nf_ct_ecache_find(ct);
193209 if (e == NULL)
194210 goto out_unlock;
195211
196212 events = xchg(&e->cache, 0);
197213
198− if (!nf_ct_is_confirmed(ct) || nf_ct_is_dying(ct))
199− goto out_unlock;
200−
201214 /* We make a copy of the missed event cache without taking
202215 * the lock, thus we may send missed events twice. However,
203216 * this does not harm and it happens very rarely. */
net/netfilter/nf_conntrack_extend.c+10−11
@@ -34,21 +34,24 @@ void nf_ct_ext_destroy(struct nf_conn *ct) · 33 unmodified lines
3434 t->destroy(ct);
3535 rcu_read_unlock();
3636 }
37+
38+ kfree(ct->ext);
3739 }
3840 EXPORT_SYMBOL(nf_ct_ext_destroy);
3941
4042 void *nf_ct_ext_add(struct nf_conn *ct, enum nf_ct_ext_id id, gfp_t gfp)
4143 {
4244 unsigned int newlen, newoff, oldlen, alloc;
43− struct nf_ct_ext *old, *new;
4445 struct nf_ct_ext_type *t;
46+ struct nf_ct_ext *new;
4547
4648 /* Conntrack must not be confirmed to avoid races on reallocation. */
4749 WARN_ON(nf_ct_is_confirmed(ct));
4850
49− old = ct->ext;
5051
51− if (old) {
52+ if (ct->ext) {
53+ const struct nf_ct_ext *old = ct->ext;
54+
5255 if (__nf_ct_ext_exist(old, id))
5356 return NULL;
5457 oldlen = old->len;
@@ -68,22 +71,18 @@ void *nf_ct_ext_add(struct nf_conn *ct, enum nf_ct_ext_id id, gfp_t gfp) · 13 unmodified lines
6871 rcu_read_unlock();
6972
7073 alloc = max(newlen, NF_CT_EXT_PREALLOC);
71− kmemleak_not_leak(old);
72− new = __krealloc(old, alloc, gfp);
74+ new = krealloc(ct->ext, alloc, gfp);
7375 if (!new)
7476 return NULL;
7577
76− if (!old) {
78+ if (!ct->ext)
7779 memset(new->offset, 0, sizeof(new->offset));
78− ct->ext = new;
79− } else if (new != old) {
80− kfree_rcu(old, rcu);
81− rcu_assign_pointer(ct->ext, new);
82− }
8380
8481 new->offset[id] = newoff;
8582 new->len = newlen;
8683 memset((void *)new + newoff, 0, newlen - newoff);
84+
85+ ct->ext = new;
8786 return (void *)new + newoff;
8887 }
8988 EXPORT_SYMBOL(nf_ct_ext_add);
net/netfilter/nf_conntrack_netlink.c+50−26
@@ -506,9 +506,45 @@ nla_put_failure: · 505 unmodified lines
506506 return -1;
507507 }
508508
509+/* all these functions access ct->ext. Caller must either hold a reference
510+ * on ct or prevent its deletion by holding either the bucket spinlock or
511+ * pcpu dying list lock.
512+ */
513+static int ctnetlink_dump_extinfo(struct sk_buff *skb,
514+ struct nf_conn *ct, u32 type)
515+{
516+ if (ctnetlink_dump_acct(skb, ct, type) < 0 ||
517+ ctnetlink_dump_timestamp(skb, ct) < 0 ||
518+ ctnetlink_dump_helpinfo(skb, ct) < 0 ||
519+ ctnetlink_dump_labels(skb, ct) < 0 ||
520+ ctnetlink_dump_ct_seq_adj(skb, ct) < 0 ||
521+ ctnetlink_dump_ct_synproxy(skb, ct) < 0)
522+ return -1;
523+
524+ return 0;
525+}
526+
527+static int ctnetlink_dump_info(struct sk_buff *skb, struct nf_conn *ct)
528+{
529+ if (ctnetlink_dump_status(skb, ct) < 0 ||
530+ ctnetlink_dump_mark(skb, ct) < 0 ||
531+ ctnetlink_dump_secctx(skb, ct) < 0 ||
532+ ctnetlink_dump_id(skb, ct) < 0 ||
533+ ctnetlink_dump_use(skb, ct) < 0 ||
534+ ctnetlink_dump_master(skb, ct) < 0)
535+ return -1;
536+
537+ if (!test_bit(IPS_OFFLOAD_BIT, &ct->status) &&
538+ (ctnetlink_dump_timeout(skb, ct) < 0 ||
539+ ctnetlink_dump_protoinfo(skb, ct) < 0))
540+ return -1;
541+
542+ return 0;
543+}
544+
509545 static int
510546 ctnetlink_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type,
511− struct nf_conn *ct)
547+ struct nf_conn *ct, bool extinfo)
512548 {
513549 const struct nf_conntrack_zone *zone;
514550 struct nlmsghdr *nlh;
@@ -552,23 +588,9 @@ ctnetlink_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type, · 37 unmodified lines
552588 NF_CT_DEFAULT_ZONE_DIR) < 0)
553589 goto nla_put_failure;
554590
555− if (ctnetlink_dump_status(skb, ct) < 0 ||
556− ctnetlink_dump_acct(skb, ct, type) < 0 ||
557− ctnetlink_dump_timestamp(skb, ct) < 0 ||
558− ctnetlink_dump_helpinfo(skb, ct) < 0 ||
559− ctnetlink_dump_mark(skb, ct) < 0 ||
560− ctnetlink_dump_secctx(skb, ct) < 0 ||
561− ctnetlink_dump_labels(skb, ct) < 0 ||
562− ctnetlink_dump_id(skb, ct) < 0 ||
563− ctnetlink_dump_use(skb, ct) < 0 ||
564− ctnetlink_dump_master(skb, ct) < 0 ||
565− ctnetlink_dump_ct_seq_adj(skb, ct) < 0 ||
566− ctnetlink_dump_ct_synproxy(skb, ct) < 0)
591+ if (ctnetlink_dump_info(skb, ct) < 0)
567592 goto nla_put_failure;
568−
569− if (!test_bit(IPS_OFFLOAD_BIT, &ct->status) &&
570− (ctnetlink_dump_timeout(skb, ct) < 0 ||
571− ctnetlink_dump_protoinfo(skb, ct) < 0))
593+ if (extinfo && ctnetlink_dump_extinfo(skb, ct, type) < 0)
572594 goto nla_put_failure;
573595
574596 nlmsg_end(skb, nlh);
@@ -953,13 +975,11 @@ restart: · 378 unmodified lines
953975 if (!ctnetlink_filter_match(ct, cb->data))
954976 continue;
955977
956− rcu_read_lock();
957978 res =
958979 ctnetlink_fill_info(skb, NETLINK_CB(cb->skb).portid,
959980 cb->nlh->nlmsg_seq,
960981 NFNL_MSG_TYPE(cb->nlh->nlmsg_type),
961− ct);
962− rcu_read_unlock();
982+ ct, true);
963983 if (res < 0) {
964984 nf_conntrack_get(&ct->ct_general);
965985 cb->args[1] = (unsigned long)ct;
@@ -1364,10 +1384,8 @@ static int ctnetlink_get_conntrack(struct net *net, struct sock *ctnl, · 398 unmodified lines
13641384 return -ENOMEM;
13651385 }
13661386
1367− rcu_read_lock();
13681387 err = ctnetlink_fill_info(skb2, NETLINK_CB(skb).portid, nlh->nlmsg_seq,
1369− NFNL_MSG_TYPE(nlh->nlmsg_type), ct);
1370− rcu_read_unlock();
1388+ NFNL_MSG_TYPE(nlh->nlmsg_type), ct, true);
13711389 nf_ct_put(ct);
13721390 if (err <= 0)
13731391 goto free;
@@ -1429,12 +1447,18 @@ restart: · 55 unmodified lines
14291447 continue;
14301448 cb->args[1] = 0;
14311449 }
1432− rcu_read_lock();
1450+
1451+ /* We can't dump extension info for the unconfirmed
1452+ * list because unconfirmed conntracks can have
1453+ * ct->ext reallocated (and thus freed).
1454+ *
1455+ * In the dying list case ct->ext can't be free'd
1456+ * until after we drop pcpu->lock.
1457+ */
14331458 res = ctnetlink_fill_info(skb, NETLINK_CB(cb->skb).portid,
14341459 cb->nlh->nlmsg_seq,
14351460 NFNL_MSG_TYPE(cb->nlh->nlmsg_type),
1436− ct);
1437− rcu_read_unlock();
1461+ ct, dying ? true : false);
14381462 if (res < 0) {
14391463 if (!atomic_inc_not_zero(&ct->ct_general.use))
14401464 continue;
net/netfilter/nf_tables_api.c+403−169
@@ -151,11 +151,64 @@ static void nft_set_trans_bind(const struct nft_ctx *ctx, struct nft_set *set) · 150 unmodified lines
151151 }
152152 }
153153
154+static int nft_netdev_register_hooks(struct net *net,
155+ struct list_head *hook_list)
156+{
157+ struct nft_hook *hook;
158+ int err, j;
159+
160+ j = 0;
161+ list_for_each_entry(hook, hook_list, list) {
162+ err = nf_register_net_hook(net, &hook->ops);
163+ if (err < 0)
164+ goto err_register;
165+
166+ j++;
167+ }
168+ return 0;
169+
170+err_register:
171+ list_for_each_entry(hook, hook_list, list) {
172+ if (j-- <= 0)
173+ break;
174+
175+ nf_unregister_net_hook(net, &hook->ops);
176+ }
177+ return err;
178+}
179+
180+static void nft_netdev_unregister_hooks(struct net *net,
181+ struct list_head *hook_list)
182+{
183+ struct nft_hook *hook;
184+
185+ list_for_each_entry(hook, hook_list, list)
186+ nf_unregister_net_hook(net, &hook->ops);
187+}
188+
189+static int nft_register_basechain_hooks(struct net *net, int family,
190+ struct nft_base_chain *basechain)
191+{
192+ if (family == NFPROTO_NETDEV)
193+ return nft_netdev_register_hooks(net, &basechain->hook_list);
194+
195+ return nf_register_net_hook(net, &basechain->ops);
196+}
197+
198+static void nft_unregister_basechain_hooks(struct net *net, int family,
199+ struct nft_base_chain *basechain)
200+{
201+ if (family == NFPROTO_NETDEV)
202+ nft_netdev_unregister_hooks(net, &basechain->hook_list);
203+ else
204+ nf_unregister_net_hook(net, &basechain->ops);
205+}
206+
154207 static int nf_tables_register_hook(struct net *net,
155208 const struct nft_table *table,
156209 struct nft_chain *chain)
157210 {
158− const struct nft_base_chain *basechain;
211+ struct nft_base_chain *basechain;
159212 const struct nf_hook_ops *ops;
160213
161214 if (table->flags & NFT_TABLE_F_DORMANT ||
@@ -168,14 +221,14 @@ static int nf_tables_register_hook(struct net *net, · 6 unmodified lines
168221 if (basechain->type->ops_register)
169222 return basechain->type->ops_register(net, ops);
170223
171− return nf_register_net_hook(net, ops);
224+ return nft_register_basechain_hooks(net, table->family, basechain);
172225 }
173226
174227 static void nf_tables_unregister_hook(struct net *net,
175228 const struct nft_table *table,
176229 struct nft_chain *chain)
177230 {
178− const struct nft_base_chain *basechain;
231+ struct nft_base_chain *basechain;
179232 const struct nf_hook_ops *ops;
180233
181234 if (table->flags & NFT_TABLE_F_DORMANT ||
@@ -187,7 +240,7 @@ static void nf_tables_unregister_hook(struct net *net, · 5 unmodified lines
187240 if (basechain->type->ops_unregister)
188241 return basechain->type->ops_unregister(net, ops);
189242
190− nf_unregister_net_hook(net, ops);
243+ nft_unregister_basechain_hooks(net, table->family, basechain);
191244 }
192245
193246 static int nft_trans_table_add(struct nft_ctx *ctx, int msg_type)
@@ -742,7 +795,8 @@ static void nft_table_disable(struct net *net, struct nft_table *table, u32 cnt) · 548 unmodified lines
742795 if (cnt && i++ == cnt)
743796 break;
744797
745− nf_unregister_net_hook(net, &nft_base_chain(chain)->ops);
798+ nft_unregister_basechain_hooks(net, table->family,
799+ nft_base_chain(chain));
746800 }
747801 }
748802
@@ -757,14 +811,16 @@ static int nf_tables_table_enable(struct net *net, struct nft_table *table) · 8 unmodified lines
757811 if (!nft_is_base_chain(chain))
758812 continue;
759813
760− err = nf_register_net_hook(net, &nft_base_chain(chain)->ops);
814+ err = nft_register_basechain_hooks(net, table->family,
815+ nft_base_chain(chain));
761816 if (err < 0)
762− goto err;
817+ goto err_register_hooks;
763818
764819 i++;
765820 }
766821 return 0;
767−err:
822+
823+err_register_hooks:
768824 if (i)
769825 nft_table_disable(net, table, i);
770826 return err;
@@ -1225,6 +1281,46 @@ nla_put_failure: · 454 unmodified lines
12251281 return -ENOSPC;
12261282 }
12271283
1284+static int nft_dump_basechain_hook(struct sk_buff *skb, int family,
1285+ const struct nft_base_chain *basechain)
1286+{
1287+ const struct nf_hook_ops *ops = &basechain->ops;
1288+ struct nft_hook *hook, *first = NULL;
1289+ struct nlattr *nest, *nest_devs;
1290+ int n = 0;
1291+
1292+ nest = nla_nest_start_noflag(skb, NFTA_CHAIN_HOOK);
1293+ if (nest == NULL)
1294+ goto nla_put_failure;
1295+ if (nla_put_be32(skb, NFTA_HOOK_HOOKNUM, htonl(ops->hooknum)))
1296+ goto nla_put_failure;
1297+ if (nla_put_be32(skb, NFTA_HOOK_PRIORITY, htonl(ops->priority)))
1298+ goto nla_put_failure;
1299+
1300+ if (family == NFPROTO_NETDEV) {
1301+ nest_devs = nla_nest_start_noflag(skb, NFTA_HOOK_DEVS);
1302+ list_for_each_entry(hook, &basechain->hook_list, list) {
1303+ if (!first)
1304+ first = hook;
1305+
1306+ if (nla_put_string(skb, NFTA_DEVICE_NAME,
1307+ hook->ops.dev->name))
1308+ goto nla_put_failure;
1309+ n++;
1310+ }
1311+ nla_nest_end(skb, nest_devs);
1312+
1313+ if (n == 1 &&
1314+ nla_put_string(skb, NFTA_HOOK_DEV, first->ops.dev->name))
1315+ goto nla_put_failure;
1316+ }
1317+ nla_nest_end(skb, nest);
1318+
1319+ return 0;
1320+nla_put_failure:
1321+ return -1;
1322+}
1323+
12281324 static int nf_tables_fill_chain_info(struct sk_buff *skb, struct net *net,
12291325 u32 portid, u32 seq, int event, u32 flags,
12301326 int family, const struct nft_table *table,
@@ -1253,21 +1349,10 @@ static int nf_tables_fill_chain_info(struct sk_buff *skb, struct net *net, · 22 unmodified lines
12531349
12541350 if (nft_is_base_chain(chain)) {
12551351 const struct nft_base_chain *basechain = nft_base_chain(chain);
1256− const struct nf_hook_ops *ops = &basechain->ops;
12571352 struct nft_stats __percpu *stats;
1258− struct nlattr *nest;
12591353
1260− nest = nla_nest_start_noflag(skb, NFTA_CHAIN_HOOK);
1261− if (nest == NULL)
1262− goto nla_put_failure;
1263− if (nla_put_be32(skb, NFTA_HOOK_HOOKNUM, htonl(ops->hooknum)))
1264− goto nla_put_failure;
1265− if (nla_put_be32(skb, NFTA_HOOK_PRIORITY, htonl(ops->priority)))
1354+ if (nft_dump_basechain_hook(skb, family, basechain))
12661355 goto nla_put_failure;
1267− if (basechain->dev_name[0] &&
1268− nla_put_string(skb, NFTA_HOOK_DEV, basechain->dev_name))
1269− goto nla_put_failure;
1270− nla_nest_end(skb, nest);
12711356
12721357 if (nla_put_be32(skb, NFTA_CHAIN_POLICY,
12731358 htonl(basechain->policy)))
@@ -1485,6 +1570,7 @@ static void nf_tables_chain_free_chain_rules(struct nft_chain *chain) · 211 unmodified lines
14851570 static void nf_tables_chain_destroy(struct nft_ctx *ctx)
14861571 {
14871572 struct nft_chain *chain = ctx->chain;
1573+ struct nft_hook *hook, *next;
14881574
14891575 if (WARN_ON(chain->use > 0))
14901576 return;
@@ -1495,6 +1581,13 @@ static void nf_tables_chain_destroy(struct nft_ctx *ctx) · 4 unmodified lines
14951581 if (nft_is_base_chain(chain)) {
14961582 struct nft_base_chain *basechain = nft_base_chain(chain);
14971583
1584+ if (ctx->family == NFPROTO_NETDEV) {
1585+ list_for_each_entry_safe(hook, next,
1586+ &basechain->hook_list, list) {
1587+ list_del_rcu(&hook->list);
1588+ kfree_rcu(hook, rcu);
1589+ }
1590+ }
14981591 module_put(basechain->type->owner);
14991592 if (rcu_access_pointer(basechain->stats)) {
15001593 static_branch_dec(&nft_counters_enabled);
@@ -1508,13 +1601,125 @@ static void nf_tables_chain_destroy(struct nft_ctx *ctx) · 7 unmodified lines
15081601 }
15091602 }
15101603
1604+static struct nft_hook *nft_netdev_hook_alloc(struct net *net,
1605+ const struct nlattr *attr)
1606+{
1607+ struct net_device *dev;
1608+ char ifname[IFNAMSIZ];
1609+ struct nft_hook *hook;
1610+ int err;
1611+
1612+ hook = kmalloc(sizeof(struct nft_hook), GFP_KERNEL);
1613+ if (!hook) {
1614+ err = -ENOMEM;
1615+ goto err_hook_alloc;
1616+ }
1617+
1618+ nla_strlcpy(ifname, attr, IFNAMSIZ);
1619+ dev = __dev_get_by_name(net, ifname);
1620+ if (!dev) {
1621+ err = -ENOENT;
1622+ goto err_hook_dev;
1623+ }
1624+ hook->ops.dev = dev;
1625+
1626+ return hook;
1627+
1628+err_hook_dev:
1629+ kfree(hook);
1630+err_hook_alloc:
1631+ return ERR_PTR(err);
1632+}
1633+
1634+static bool nft_hook_list_find(struct list_head *hook_list,
1635+ const struct nft_hook *this)
1636+{
1637+ struct nft_hook *hook;
1638+
1639+ list_for_each_entry(hook, hook_list, list) {
1640+ if (this->ops.dev == hook->ops.dev)
1641+ return true;
1642+ }
1643+
1644+ return false;
1645+}
1646+
1647+static int nf_tables_parse_netdev_hooks(struct net *net,
1648+ const struct nlattr *attr,
1649+ struct list_head *hook_list)
1650+{
1651+ struct nft_hook *hook, *next;
1652+ const struct nlattr *tmp;
1653+ int rem, n = 0, err;
1654+
1655+ nla_for_each_nested(tmp, attr, rem) {
1656+ if (nla_type(tmp) != NFTA_DEVICE_NAME) {
1657+ err = -EINVAL;
1658+ goto err_hook;
1659+ }
1660+
1661+ hook = nft_netdev_hook_alloc(net, tmp);
1662+ if (IS_ERR(hook)) {
1663+ err = PTR_ERR(hook);
1664+ goto err_hook;
1665+ }
1666+ if (nft_hook_list_find(hook_list, hook)) {
1667+ err = -EEXIST;
1668+ goto err_hook;
1669+ }
1670+ list_add_tail(&hook->list, hook_list);
1671+ n++;
1672+
1673+ if (n == NFT_NETDEVICE_MAX) {
1674+ err = -EFBIG;
1675+ goto err_hook;
1676+ }
1677+ }
1678+ if (!n)
1679+ return -EINVAL;
1680+
1681+ return 0;
1682+
1683+err_hook:
1684+ list_for_each_entry_safe(hook, next, hook_list, list) {
1685+ list_del(&hook->list);
1686+ kfree(hook);
1687+ }
1688+ return err;
1689+}
1690+
15111691 struct nft_chain_hook {
15121692 u32 num;
15131693 s32 priority;
15141694 const struct nft_chain_type *type;
1515− struct net_device *dev;
1695+ struct list_head list;
15161696 };
15171697
1698+static int nft_chain_parse_netdev(struct net *net,
1699+ struct nlattr *tb[],
1700+ struct list_head *hook_list)
1701+{
1702+ struct nft_hook *hook;
1703+ int err;
1704+
1705+ if (tb[NFTA_HOOK_DEV]) {
1706+ hook = nft_netdev_hook_alloc(net, tb[NFTA_HOOK_DEV]);
1707+ if (IS_ERR(hook))
1708+ return PTR_ERR(hook);
1709+
1710+ list_add_tail(&hook->list, hook_list);
1711+ } else if (tb[NFTA_HOOK_DEVS]) {
1712+ err = nf_tables_parse_netdev_hooks(net, tb[NFTA_HOOK_DEVS],
1713+ hook_list);
1714+ if (err < 0)
1715+ return err;
1716+ } else {
1717+ return -EINVAL;
1718+ }
1719+
1720+ return 0;
1721+}
1722+
15181723 static int nft_chain_parse_hook(struct net *net,
15191724 const struct nlattr * const nla[],
15201725 struct nft_chain_hook *hook, u8 family,
@@ -1522,7 +1727,6 @@ static int nft_chain_parse_hook(struct net *net, · 1 unmodified line
15221727 {
15231728 struct nlattr *ha[NFTA_HOOK_MAX + 1];
15241729 const struct nft_chain_type *type;
1525− struct net_device *dev;
15261730 int err;
15271731
15281732 lockdep_assert_held(&net->nft.commit_mutex);
@@ -1560,23 +1764,14 @@ static int nft_chain_parse_hook(struct net *net, · 31 unmodified lines
15601764
15611765 hook->type = type;
15621766
1563− hook->dev = NULL;
1767+ INIT_LIST_HEAD(&hook->list);
15641768 if (family == NFPROTO_NETDEV) {
1565− char ifname[IFNAMSIZ];
1566−
1567− if (!ha[NFTA_HOOK_DEV]) {
1568− module_put(type->owner);
1569− return -EOPNOTSUPP;
1570− }
1571−
1572− nla_strlcpy(ifname, ha[NFTA_HOOK_DEV], IFNAMSIZ);
1573− dev = __dev_get_by_name(net, ifname);
1574− if (!dev) {
1769+ err = nft_chain_parse_netdev(net, ha, &hook->list);
1770+ if (err < 0) {
15751771 module_put(type->owner);
1576− return -ENOENT;
1772+ return err;
15771773 }
1578− hook->dev = dev;
1579− } else if (ha[NFTA_HOOK_DEV]) {
1774+ } else if (ha[NFTA_HOOK_DEV] || ha[NFTA_HOOK_DEVS]) {
15801775 module_put(type->owner);
15811776 return -EOPNOTSUPP;
15821777 }
@@ -1586,6 +1781,12 @@ static int nft_chain_parse_hook(struct net *net, · 3 unmodified lines
15861781
15871782 static void nft_chain_release_hook(struct nft_chain_hook *hook)
15881783 {
1784+ struct nft_hook *h, *next;
1785+
1786+ list_for_each_entry_safe(h, next, &hook->list, list) {
1787+ list_del(&h->list);
1788+ kfree(h);
1789+ }
15891790 module_put(hook->type->owner);
15901791 }
15911792
@@ -1610,6 +1811,49 @@ static struct nft_rule **nf_tables_chain_alloc_rules(const struct nft_chain *cha · 18 unmodified lines
16101811 return kvmalloc(alloc, GFP_KERNEL);
16111812 }
16121813
1814+static void nft_basechain_hook_init(struct nf_hook_ops *ops, u8 family,
1815+ const struct nft_chain_hook *hook,
1816+ struct nft_chain *chain)
1817+{
1818+ ops->pf = family;
1819+ ops->hooknum = hook->num;
1820+ ops->priority = hook->priority;
1821+ ops->priv = chain;
1822+ ops->hook = hook->type->hooks[ops->hooknum];
1823+}
1824+
1825+static int nft_basechain_init(struct nft_base_chain *basechain, u8 family,
1826+ struct nft_chain_hook *hook, u32 flags)
1827+{
1828+ struct nft_chain *chain;
1829+ struct nft_hook *h;
1830+
1831+ basechain->type = hook->type;
1832+ INIT_LIST_HEAD(&basechain->hook_list);
1833+ chain = &basechain->chain;
1834+
1835+ if (family == NFPROTO_NETDEV) {
1836+ list_splice_init(&hook->list, &basechain->hook_list);
1837+ list_for_each_entry(h, &basechain->hook_list, list)
1838+ nft_basechain_hook_init(&h->ops, family, hook, chain);
1839+
1840+ basechain->ops.hooknum = hook->num;
1841+ basechain->ops.priority = hook->priority;
1842+ } else {
1843+ nft_basechain_hook_init(&basechain->ops, family, hook, chain);
1844+ }
1845+
1846+ chain->flags |= NFT_BASE_CHAIN | flags;
1847+ basechain->policy = NF_ACCEPT;
1848+ if (chain->flags & NFT_CHAIN_HW_OFFLOAD &&
1849+ nft_chain_offload_priority(basechain) < 0)
1850+ return -EOPNOTSUPP;
1851+
1852+ flow_block_init(&basechain->flow_block);
1853+
1854+ return 0;
1855+}
1856+
16131857 static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
16141858 u8 policy, u32 flags)
16151859 {
@@ -1628,7 +1872,6 @@ static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask, · 12 unmodified lines
16281872
16291873 if (nla[NFTA_CHAIN_HOOK]) {
16301874 struct nft_chain_hook hook;
1631− struct nf_hook_ops *ops;
16321875
16331876 err = nft_chain_parse_hook(net, nla, &hook, family, true);
16341877 if (err < 0)
@@ -1639,9 +1882,7 @@ static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask, · 4 unmodified lines
16391882 nft_chain_release_hook(&hook);
16401883 return -ENOMEM;
16411884 }
1642−
1643− if (hook.dev != NULL)
1644− strncpy(basechain->dev_name, hook.dev->name, IFNAMSIZ);
1885+ chain = &basechain->chain;
16451886
16461887 if (nla[NFTA_CHAIN_COUNTERS]) {
16471888 stats = nft_stats_alloc(nla[NFTA_CHAIN_COUNTERS]);
@@ -1654,24 +1895,12 @@ static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask, · 6 unmodified lines
16541895 static_branch_inc(&nft_counters_enabled);
16551896 }
16561897
1657− basechain->type = hook.type;
1658− chain = &basechain->chain;
1659−
1660− ops = &basechain->ops;
1661− ops->pf = family;
1662− ops->hooknum = hook.num;
1663− ops->priority = hook.priority;
1664− ops->priv = chain;
1665− ops->hook = hook.type->hooks[ops->hooknum];
1666− ops->dev = hook.dev;
1667−
1668− chain->flags |= NFT_BASE_CHAIN | flags;
1669− basechain->policy = NF_ACCEPT;
1670− if (chain->flags & NFT_CHAIN_HW_OFFLOAD &&
1671− nft_chain_offload_priority(basechain) < 0)
1672− return -EOPNOTSUPP;
1673−
1674− flow_block_init(&basechain->flow_block);
1898+ err = nft_basechain_init(basechain, family, &hook, flags);
1899+ if (err < 0) {
1900+ nft_chain_release_hook(&hook);
1901+ kfree(basechain);
1902+ return err;
1903+ }
16751904 } else {
16761905 chain = kzalloc(sizeof(*chain), GFP_KERNEL);
16771906 if (chain == NULL)
@@ -1731,6 +1960,25 @@ err1: · 53 unmodified lines
17311960 return err;
17321961 }
17331962
1963+static bool nft_hook_list_equal(struct list_head *hook_list1,
1964+ struct list_head *hook_list2)
1965+{
1966+ struct nft_hook *hook;
1967+ int n = 0, m = 0;
1968+
1969+ n = 0;
1970+ list_for_each_entry(hook, hook_list2, list) {
1971+ if (!nft_hook_list_find(hook_list1, hook))
1972+ return false;
1973+
1974+ n++;
1975+ }
1976+ list_for_each_entry(hook, hook_list1, list)
1977+ m++;
1978+
1979+ return n == m;
1980+}
1981+
17341982 static int nf_tables_updchain(struct nft_ctx *ctx, u8 genmask, u8 policy,
17351983 u32 flags)
17361984 {
@@ -1762,12 +2010,19 @@ static int nf_tables_updchain(struct nft_ctx *ctx, u8 genmask, u8 policy, · 25 unmodified lines
17622010 return -EBUSY;
17632011 }
17642012
1765− ops = &basechain->ops;
1766− if (ops->hooknum != hook.num ||
1767− ops->priority != hook.priority ||
1768− ops->dev != hook.dev) {
1769− nft_chain_release_hook(&hook);
1770− return -EBUSY;
2013+ if (ctx->family == NFPROTO_NETDEV) {
2014+ if (!nft_hook_list_equal(&basechain->hook_list,
2015+ &hook.list)) {
2016+ nft_chain_release_hook(&hook);
2017+ return -EBUSY;
2018+ }
2019+ } else {
2020+ ops = &basechain->ops;
2021+ if (ops->hooknum != hook.num ||
2022+ ops->priority != hook.priority) {
2023+ nft_chain_release_hook(&hook);
2024+ return -EBUSY;
2025+ }
17712026 }
17722027 nft_chain_release_hook(&hook);
17732028 }
@@ -5628,43 +5883,6 @@ nft_flowtable_lookup_byhandle(const struct nft_table *table, · 3854 unmodified lines
56285883 return ERR_PTR(-ENOENT);
56295884 }
56305885
5631−static int nf_tables_parse_devices(const struct nft_ctx *ctx,
5632− const struct nlattr *attr,
5633− struct net_device *dev_array[], int *len)
5634−{
5635− const struct nlattr *tmp;
5636− struct net_device *dev;
5637− char ifname[IFNAMSIZ];
5638− int rem, n = 0, err;
5639−
5640− nla_for_each_nested(tmp, attr, rem) {
5641− if (nla_type(tmp) != NFTA_DEVICE_NAME) {
5642− err = -EINVAL;
5643− goto err1;
5644− }
5645−
5646− nla_strlcpy(ifname, tmp, IFNAMSIZ);
5647− dev = __dev_get_by_name(ctx->net, ifname);
5648− if (!dev) {
5649− err = -ENOENT;
5650− goto err1;
5651− }
5652−
5653− dev_array[n++] = dev;
5654− if (n == NFT_FLOWTABLE_DEVICE_MAX) {
5655− err = -EFBIG;
5656− goto err1;
5657− }
5658− }
5659− if (!len)
5660− return -EINVAL;
5661−
5662− err = 0;
5663−err1:
5664− *len = n;
5665− return err;
5666−}
5667−
56685886 static const struct nla_policy nft_flowtable_hook_policy[NFTA_FLOWTABLE_HOOK_MAX + 1] = {
56695887 [NFTA_FLOWTABLE_HOOK_NUM] = { .type = NLA_U32 },
56705888 [NFTA_FLOWTABLE_HOOK_PRIORITY] = { .type = NLA_U32 },
@@ -5675,11 +5893,10 @@ static int nf_tables_flowtable_parse_hook(const struct nft_ctx *ctx, · 4 unmodified lines
56755893 const struct nlattr *attr,
56765894 struct nft_flowtable *flowtable)
56775895 {
5678− struct net_device *dev_array[NFT_FLOWTABLE_DEVICE_MAX];
56795896 struct nlattr *tb[NFTA_FLOWTABLE_HOOK_MAX + 1];
5680− struct nf_hook_ops *ops;
5897+ struct nft_hook *hook;
56815898 int hooknum, priority;
5682− int err, n = 0, i;
5899+ int err;
56835900
56845901 err = nla_parse_nested_deprecated(tb, NFTA_FLOWTABLE_HOOK_MAX, attr,
56855902 nft_flowtable_hook_policy, NULL);
@@ -5697,27 +5914,21 @@ static int nf_tables_flowtable_parse_hook(const struct nft_ctx *ctx, · 11 unmodified lines
56975914
56985915 priority = ntohl(nla_get_be32(tb[NFTA_FLOWTABLE_HOOK_PRIORITY]));
56995916
5700− err = nf_tables_parse_devices(ctx, tb[NFTA_FLOWTABLE_HOOK_DEVS],
5701− dev_array, &n);
5917+ err = nf_tables_parse_netdev_hooks(ctx->net,
5918+ tb[NFTA_FLOWTABLE_HOOK_DEVS],
5919+ &flowtable->hook_list);
57025920 if (err < 0)
57035921 return err;
57045922
5705− ops = kcalloc(n, sizeof(struct nf_hook_ops), GFP_KERNEL);
5706− if (!ops)
5707− return -ENOMEM;
5708−
5709− flowtable->hooknum = hooknum;
5710− flowtable->priority = priority;
5711− flowtable->ops = ops;
5712− flowtable->ops_len = n;
5923+ flowtable->hooknum = hooknum;
5924+ flowtable->data.priority = priority;
57135925
5714− for (i = 0; i < n; i++) {
5715− flowtable->ops[i].pf = NFPROTO_NETDEV;
5716− flowtable->ops[i].hooknum = hooknum;
5717− flowtable->ops[i].priority = priority;
5718− flowtable->ops[i].priv = &flowtable->data;
5719− flowtable->ops[i].hook = flowtable->data.type->hook;
5720− flowtable->ops[i].dev = dev_array[i];
5926+ list_for_each_entry(hook, &flowtable->hook_list, list) {
5927+ hook->ops.pf = NFPROTO_NETDEV;
5928+ hook->ops.hooknum = hooknum;
5929+ hook->ops.priority = priority;
5930+ hook->ops.priv = &flowtable->data;
5931+ hook->ops.hook = flowtable->data.type->hook;
57215932 }
57225933
57235934 return err;
@@ -5757,14 +5968,51 @@ nft_flowtable_type_get(struct net *net, u8 family) · 33 unmodified lines
57575968 static void nft_unregister_flowtable_net_hooks(struct net *net,
57585969 struct nft_flowtable *flowtable)
57595970 {
5760− int i;
5971+ struct nft_hook *hook;
57615972
5762− for (i = 0; i < flowtable->ops_len; i++) {
5763− if (!flowtable->ops[i].dev)
5764− continue;
5973+ list_for_each_entry(hook, &flowtable->hook_list, list)
5974+ nf_unregister_net_hook(net, &hook->ops);
5975+}
57655976
5766− nf_unregister_net_hook(net, &flowtable->ops[i]);
5977+static int nft_register_flowtable_net_hooks(struct net *net,
5978+ struct nft_table *table,
5979+ struct nft_flowtable *flowtable)
5980+{
5981+ struct nft_hook *hook, *hook2, *next;
5982+ struct nft_flowtable *ft;
5983+ int err, i = 0;
5984+
5985+ list_for_each_entry(hook, &flowtable->hook_list, list) {
5986+ list_for_each_entry(ft, &table->flowtables, list) {
5987+ list_for_each_entry(hook2, &ft->hook_list, list) {
5988+ if (hook->ops.dev == hook2->ops.dev &&
5989+ hook->ops.pf == hook2->ops.pf) {
5990+ err = -EBUSY;
5991+ goto err_unregister_net_hooks;
5992+ }
5993+ }
5994+ }
5995+
5996+ err = nf_register_net_hook(net, &hook->ops);
5997+ if (err < 0)
5998+ goto err_unregister_net_hooks;
5999+
6000+ i++;
57676001 }
6002+
6003+ return 0;
6004+
6005+err_unregister_net_hooks:
6006+ list_for_each_entry_safe(hook, next, &flowtable->hook_list, list) {
6007+ if (i-- <= 0)
6008+ break;
6009+
6010+ nf_unregister_net_hook(net, &hook->ops);
6011+ list_del_rcu(&hook->list);
6012+ kfree_rcu(hook, rcu);
6013+ }
6014+
6015+ return err;
57686016 }
57696017
57706018 static int nf_tables_newflowtable(struct net *net, struct sock *nlsk,
@@ -5775,12 +6023,13 @@ static int nf_tables_newflowtable(struct net *net, struct sock *nlsk, · 4 unmodified lines
57756023 {
57766024 const struct nfgenmsg *nfmsg = nlmsg_data(nlh);
57776025 const struct nf_flowtable_type *type;
5778− struct nft_flowtable *flowtable, *ft;
57796026 u8 genmask = nft_genmask_next(net);
57806027 int family = nfmsg->nfgen_family;
6028+ struct nft_flowtable *flowtable;
6029+ struct nft_hook *hook, *next;
57816030 struct nft_table *table;
57826031 struct nft_ctx ctx;
5783− int err, i, k;
6032+ int err;
57846033
57856034 if (!nla[NFTA_FLOWTABLE_TABLE] ||
57866035 !nla[NFTA_FLOWTABLE_NAME] ||
@@ -5819,6 +6068,7 @@ static int nf_tables_newflowtable(struct net *net, struct sock *nlsk, · 32 unmodified lines
58196068
58206069 flowtable->table = table;
58216070 flowtable->handle = nf_tables_alloc_handle(table);
6071+ INIT_LIST_HEAD(&flowtable->hook_list);
58226072
58236073 flowtable->name = nla_strdup(nla[NFTA_FLOWTABLE_NAME], GFP_KERNEL);
58246074 if (!flowtable->name) {
@@ -5842,43 +6092,24 @@ static int nf_tables_newflowtable(struct net *net, struct sock *nlsk, · 17 unmodified lines
58426092 if (err < 0)
58436093 goto err4;
58446094
5845− for (i = 0; i < flowtable->ops_len; i++) {
5846− if (!flowtable->ops[i].dev)
5847− continue;
5848−
5849− list_for_each_entry(ft, &table->flowtables, list) {
5850− for (k = 0; k < ft->ops_len; k++) {
5851− if (!ft->ops[k].dev)
5852− continue;
5853−
5854− if (flowtable->ops[i].dev == ft->ops[k].dev &&
5855− flowtable->ops[i].pf == ft->ops[k].pf) {
5856− err = -EBUSY;
5857− goto err5;
5858− }
5859− }
5860− }
5861−
5862− err = nf_register_net_hook(net, &flowtable->ops[i]);
5863− if (err < 0)
5864− goto err5;
5865− }
6095+ err = nft_register_flowtable_net_hooks(ctx.net, table, flowtable);
6096+ if (err < 0)
6097+ goto err4;
58666098
58676099 err = nft_trans_flowtable_add(&ctx, NFT_MSG_NEWFLOWTABLE, flowtable);
58686100 if (err < 0)
5869− goto err6;
6101+ goto err5;
58706102
58716103 list_add_tail_rcu(&flowtable->list, &table->flowtables);
58726104 table->use++;
58736105
58746106 return 0;
5875−err6:
5876− i = flowtable->ops_len;
58776107 err5:
5878− for (k = i - 1; k >= 0; k--)
5879− nf_unregister_net_hook(net, &flowtable->ops[k]);
5880−
5881− kfree(flowtable->ops);
6108+ list_for_each_entry_safe(hook, next, &flowtable->hook_list, list) {
6109+ nf_unregister_net_hook(net, &hook->ops);
6110+ list_del_rcu(&hook->list);
6111+ kfree_rcu(hook, rcu);
6112+ }
58826113 err4:
58836114 flowtable->data.type->free(&flowtable->data);
58846115 err3:
@@ -5945,8 +6176,8 @@ static int nf_tables_fill_flowtable_info(struct sk_buff *skb, struct net *net, · 60 unmodified lines
59456176 {
59466177 struct nlattr *nest, *nest_devs;
59476178 struct nfgenmsg *nfmsg;
6179+ struct nft_hook *hook;
59486180 struct nlmsghdr *nlh;
5949− int i;
59506181
59516182 event = nfnl_msg_type(NFNL_SUBSYS_NFTABLES, event);
59526183 nlh = nlmsg_put(skb, portid, seq, event, sizeof(struct nfgenmsg), flags);
@@ -5969,18 +6200,15 @@ static int nf_tables_fill_flowtable_info(struct sk_buff *skb, struct net *net, · 16 unmodified lines
59696200 if (!nest)
59706201 goto nla_put_failure;
59716202 if (nla_put_be32(skb, NFTA_FLOWTABLE_HOOK_NUM, htonl(flowtable->hooknum)) ||
5972− nla_put_be32(skb, NFTA_FLOWTABLE_HOOK_PRIORITY, htonl(flowtable->priority)))
6203+ nla_put_be32(skb, NFTA_FLOWTABLE_HOOK_PRIORITY, htonl(flowtable->data.priority)))
59736204 goto nla_put_failure;
59746205
59756206 nest_devs = nla_nest_start_noflag(skb, NFTA_FLOWTABLE_HOOK_DEVS);
59766207 if (!nest_devs)
59776208 goto nla_put_failure;
59786209
5979− for (i = 0; i < flowtable->ops_len; i++) {
5980− const struct net_device *dev = READ_ONCE(flowtable->ops[i].dev);
5981−
5982− if (dev &&
5983− nla_put_string(skb, NFTA_DEVICE_NAME, dev->name))
6210+ list_for_each_entry_rcu(hook, &flowtable->hook_list, list) {
6211+ if (nla_put_string(skb, NFTA_DEVICE_NAME, hook->ops.dev->name))
59846212 goto nla_put_failure;
59856213 }
59866214 nla_nest_end(skb, nest_devs);
@@ -6171,7 +6399,12 @@ err: · 184 unmodified lines
61716399
61726400 static void nf_tables_flowtable_destroy(struct nft_flowtable *flowtable)
61736401 {
6174− kfree(flowtable->ops);
6402+ struct nft_hook *hook, *next;
6403+
6404+ list_for_each_entry_safe(hook, next, &flowtable->hook_list, list) {
6405+ list_del_rcu(&hook->list);
6406+ kfree(hook);
6407+ }
61756408 kfree(flowtable->name);
61766409 flowtable->data.type->free(&flowtable->data);
61776410 module_put(flowtable->data.type->owner);
@@ -6211,14 +6444,15 @@ nla_put_failure: · 33 unmodified lines
62116444 static void nft_flowtable_event(unsigned long event, struct net_device *dev,
62126445 struct nft_flowtable *flowtable)
62136446 {
6214− int i;
6447+ struct nft_hook *hook;
62156448
6216− for (i = 0; i < flowtable->ops_len; i++) {
6217− if (flowtable->ops[i].dev != dev)
6449+ list_for_each_entry(hook, &flowtable->hook_list, list) {
6450+ if (hook->ops.dev != dev)
62186451 continue;
62196452
6220− nf_unregister_net_hook(dev_net(dev), &flowtable->ops[i]);
6221− flowtable->ops[i].dev = NULL;
6453+ nf_unregister_net_hook(dev_net(dev), &hook->ops);
6454+ list_del_rcu(&hook->list);
6455+ kfree_rcu(hook, rcu);
62226456 break;
62236457 }
62246458 }
net/netfilter/nf_tables_offload.c+125−63
@@ -132,13 +132,13 @@ static void nft_flow_offload_common_init(struct flow_cls_common_offload *common, · 131 unmodified lines
132132 common->extack = extack;
133133 }
134134
135−static int nft_setup_cb_call(struct nft_base_chain *basechain,
136− enum tc_setup_type type, void *type_data)
135+static int nft_setup_cb_call(enum tc_setup_type type, void *type_data,
136+ struct list_head *cb_list)
137137 {
138138 struct flow_block_cb *block_cb;
139139 int err;
140140
141− list_for_each_entry(block_cb, &basechain->flow_block.cb_list, list) {
141+ list_for_each_entry(block_cb, cb_list, list) {
142142 err = block_cb->cb(type, type_data, block_cb->cb_priv);
143143 if (err < 0)
144144 return err;
@@ -155,32 +155,44 @@ int nft_chain_offload_priority(struct nft_base_chain *basechain) · 10 unmodified lines
155155 return 0;
156156 }
157157
158+static void nft_flow_cls_offload_setup(struct flow_cls_offload *cls_flow,
159+ const struct nft_base_chain *basechain,
160+ const struct nft_rule *rule,
161+ const struct nft_flow_rule *flow,
162+ enum flow_cls_command command)
163+{
164+ struct netlink_ext_ack extack;
165+ __be16 proto = ETH_P_ALL;
166+
167+ memset(cls_flow, 0, sizeof(*cls_flow));
168+
169+ if (flow)
170+ proto = flow->proto;
171+
172+ nft_flow_offload_common_init(&cls_flow->common, proto,
173+ basechain->ops.priority, &extack);
174+ cls_flow->command = command;
175+ cls_flow->cookie = (unsigned long) rule;
176+ if (flow)
177+ cls_flow->rule = flow->rule;
178+}
179+
158180 static int nft_flow_offload_rule(struct nft_chain *chain,
159181 struct nft_rule *rule,
160182 struct nft_flow_rule *flow,
161183 enum flow_cls_command command)
162184 {
163− struct flow_cls_offload cls_flow = {};
185+ struct flow_cls_offload cls_flow;
164186 struct nft_base_chain *basechain;
165− struct netlink_ext_ack extack;
166− __be16 proto = ETH_P_ALL;
167187
168188 if (!nft_is_base_chain(chain))
169189 return -EOPNOTSUPP;
170190
171191 basechain = nft_base_chain(chain);
192+ nft_flow_cls_offload_setup(&cls_flow, basechain, rule, flow, command);
172193
173− if (flow)
174− proto = flow->proto;
175−
176− nft_flow_offload_common_init(&cls_flow.common, proto,
177− basechain->ops.priority, &extack);
178− cls_flow.command = command;
179− cls_flow.cookie = (unsigned long) rule;
180− if (flow)
181− cls_flow.rule = flow->rule;
182−
183− return nft_setup_cb_call(basechain, TC_SETUP_CLSFLOWER, &cls_flow);
194+ return nft_setup_cb_call(TC_SETUP_CLSFLOWER, &cls_flow,
195+ &basechain->flow_block.cb_list);
184196 }
185197
186198 static int nft_flow_offload_bind(struct flow_block_offload *bo,
@@ -194,6 +206,16 @@ static int nft_flow_offload_unbind(struct flow_block_offload *bo, · 7 unmodified lines
194206 struct nft_base_chain *basechain)
195207 {
196208 struct flow_block_cb *block_cb, *next;
209+ struct flow_cls_offload cls_flow;
210+ struct nft_chain *chain;
211+ struct nft_rule *rule;
212+
213+ chain = &basechain->chain;
214+ list_for_each_entry(rule, &chain->rules, list) {
215+ nft_flow_cls_offload_setup(&cls_flow, basechain, rule, NULL,
216+ FLOW_CLS_DESTROY);
217+ nft_setup_cb_call(TC_SETUP_CLSFLOWER, &cls_flow, &bo->cb_list);
218+ }
197219
198220 list_for_each_entry_safe(block_cb, next, &bo->cb_list, list) {
199221 list_del(&block_cb->list);
@@ -224,20 +246,30 @@ static int nft_block_setup(struct nft_base_chain *basechain, · 24 unmodified lines
224246 return err;
225247 }
226248
249+static void nft_flow_block_offload_init(struct flow_block_offload *bo,
250+ struct net *net,
251+ enum flow_block_command cmd,
252+ struct nft_base_chain *basechain,
253+ struct netlink_ext_ack *extack)
254+{
255+ memset(bo, 0, sizeof(*bo));
256+ bo->net = net;
257+ bo->block = &basechain->flow_block;
258+ bo->command = cmd;
259+ bo->binder_type = FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS;
260+ bo->extack = extack;
261+ INIT_LIST_HEAD(&bo->cb_list);
262+}
263+
227264 static int nft_block_offload_cmd(struct nft_base_chain *chain,
228265 struct net_device *dev,
229266 enum flow_block_command cmd)
230267 {
231268 struct netlink_ext_ack extack = {};
232− struct flow_block_offload bo = {};
269+ struct flow_block_offload bo;
233270 int err;
234271
235− bo.net = dev_net(dev);
236− bo.block = &chain->flow_block;
237− bo.command = cmd;
238− bo.binder_type = FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS;
239− bo.extack = &extack;
240− INIT_LIST_HEAD(&bo.cb_list);
272+ nft_flow_block_offload_init(&bo, dev_net(dev), cmd, chain, &extack);
241273
242274 err = dev->netdev_ops->ndo_setup_tc(dev, TC_SETUP_BLOCK, &bo);
243275 if (err < 0)
@@ -253,17 +285,12 @@ static void nft_indr_block_ing_cmd(struct net_device *dev, · 9 unmodified lines
253285 enum flow_block_command cmd)
254286 {
255287 struct netlink_ext_ack extack = {};
256− struct flow_block_offload bo = {};
288+ struct flow_block_offload bo;
257289
258290 if (!chain)
259291 return;
260292
261− bo.net = dev_net(dev);
262− bo.block = &chain->flow_block;
263− bo.command = cmd;
264− bo.binder_type = FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS;
265− bo.extack = &extack;
266− INIT_LIST_HEAD(&bo.cb_list);
293+ nft_flow_block_offload_init(&bo, dev_net(dev), cmd, chain, &extack);
267294
268295 cb(dev, cb_priv, TC_SETUP_BLOCK, &bo);
269296
@@ -274,15 +301,10 @@ static int nft_indr_block_offload_cmd(struct nft_base_chain *chain, · 4 unmodified lines
274301 struct net_device *dev,
275302 enum flow_block_command cmd)
276303 {
277− struct flow_block_offload bo = {};
278304 struct netlink_ext_ack extack = {};
305+ struct flow_block_offload bo;
279306
280− bo.net = dev_net(dev);
281− bo.block = &chain->flow_block;
282− bo.command = cmd;
283− bo.binder_type = FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS;
284− bo.extack = &extack;
285− INIT_LIST_HEAD(&bo.cb_list);
307+ nft_flow_block_offload_init(&bo, dev_net(dev), cmd, chain, &extack);
286308
287309 flow_indr_block_call(dev, &bo, cmd);
288310
@@ -294,32 +316,73 @@ static int nft_indr_block_offload_cmd(struct nft_base_chain *chain, · 5 unmodified lines
294316
295317 #define FLOW_SETUP_BLOCK TC_SETUP_BLOCK
296318
297−static int nft_flow_offload_chain(struct nft_chain *chain,
298− u8 *ppolicy,
319+static int nft_chain_offload_cmd(struct nft_base_chain *basechain,
320+ struct net_device *dev,
321+ enum flow_block_command cmd)
322+{
323+ int err;
324+
325+ if (dev->netdev_ops->ndo_setup_tc)
326+ err = nft_block_offload_cmd(basechain, dev, cmd);
327+ else
328+ err = nft_indr_block_offload_cmd(basechain, dev, cmd);
329+
330+ return err;
331+}
332+
333+static int nft_flow_block_chain(struct nft_base_chain *basechain,
334+ const struct net_device *this_dev,
335+ enum flow_block_command cmd)
336+{
337+ struct net_device *dev;
338+ struct nft_hook *hook;
339+ int err, i = 0;
340+
341+ list_for_each_entry(hook, &basechain->hook_list, list) {
342+ dev = hook->ops.dev;
343+ if (this_dev && this_dev != dev)
344+ continue;
345+
346+ err = nft_chain_offload_cmd(basechain, dev, cmd);
347+ if (err < 0 && cmd == FLOW_BLOCK_BIND) {
348+ if (!this_dev)
349+ goto err_flow_block;
350+
351+ return err;
352+ }
353+ i++;
354+ }
355+
356+ return 0;
357+
358+err_flow_block:
359+ list_for_each_entry(hook, &basechain->hook_list, list) {
360+ if (i-- <= 0)
361+ break;
362+
363+ dev = hook->ops.dev;
364+ nft_chain_offload_cmd(basechain, dev, FLOW_BLOCK_UNBIND);
365+ }
366+ return err;
367+}
368+
369+static int nft_flow_offload_chain(struct nft_chain *chain, u8 *ppolicy,
299370 enum flow_block_command cmd)
300371 {
301372 struct nft_base_chain *basechain;
302− struct net_device *dev;
303373 u8 policy;
304374
305375 if (!nft_is_base_chain(chain))
306376 return -EOPNOTSUPP;
307377
308378 basechain = nft_base_chain(chain);
309− dev = basechain->ops.dev;
310− if (!dev)
311− return -EOPNOTSUPP;
312−
313379 policy = ppolicy ? *ppolicy : basechain->policy;
314380
315381 /* Only default policy to accept is supported for now. */
316382 if (cmd == FLOW_BLOCK_BIND && policy == NF_DROP)
317383 return -EOPNOTSUPP;
318384
319− if (dev->netdev_ops->ndo_setup_tc)
320− return nft_block_offload_cmd(basechain, dev, cmd);
321− else
322− return nft_indr_block_offload_cmd(basechain, dev, cmd);
385+ return nft_flow_block_chain(basechain, NULL, cmd);
323386 }
324387
325388 int nft_flow_rule_offload_commit(struct net *net)
@@ -385,6 +448,7 @@ static struct nft_chain *__nft_offload_get_chain(struct net_device *dev) · 59 unmodified lines
385448 {
386449 struct nft_base_chain *basechain;
387450 struct net *net = dev_net(dev);
451+ struct nft_hook *hook, *found;
388452 const struct nft_table *table;
389453 struct nft_chain *chain;
390454
@@ -397,8 +461,16 @@ static struct nft_chain *__nft_offload_get_chain(struct net_device *dev) · 6 unmodified lines
397461 !(chain->flags & NFT_CHAIN_HW_OFFLOAD))
398462 continue;
399463
464+ found = NULL;
400465 basechain = nft_base_chain(chain);
401− if (strncmp(basechain->dev_name, dev->name, IFNAMSIZ))
466+ list_for_each_entry(hook, &basechain->hook_list, list) {
467+ if (hook->ops.dev != dev)
468+ continue;
469+
470+ found = hook;
471+ break;
472+ }
473+ if (!found)
402474 continue;
403475
404476 return chain;
@@ -426,18 +498,6 @@ static void nft_indr_block_cb(struct net_device *dev, · 21 unmodified lines
426498 mutex_unlock(&net->nft.commit_mutex);
427499 }
428500
429−static void nft_offload_chain_clean(struct nft_chain *chain)
430−{
431− struct nft_rule *rule;
432−
433− list_for_each_entry(rule, &chain->rules, list) {
434− nft_flow_offload_rule(chain, rule,
435− NULL, FLOW_CLS_DESTROY);
436− }
437−
438− nft_flow_offload_chain(chain, NULL, FLOW_BLOCK_UNBIND);
439−}
440−
441501 static int nft_offload_netdev_event(struct notifier_block *this,
442502 unsigned long event, void *ptr)
443503 {
@@ -448,7 +508,9 @@ static int nft_offload_netdev_event(struct notifier_block *this, · 4 unmodified lines
448508 mutex_lock(&net->nft.commit_mutex);
449509 chain = __nft_offload_get_chain(dev);
450510 if (chain)
451− nft_offload_chain_clean(chain);
511+ nft_flow_block_chain(nft_base_chain(chain), dev,
512+ FLOW_BLOCK_UNBIND);
513+
452514 mutex_unlock(&net->nft.commit_mutex);
453515
454516 return NOTIFY_DONE;
net/netfilter/nft_chain_filter.c+26−19
@@ -287,28 +287,35 @@ static void nft_netdev_event(unsigned long event, struct net_device *dev, · 286 unmodified lines
287287 struct nft_ctx *ctx)
288288 {
289289 struct nft_base_chain *basechain = nft_base_chain(ctx->chain);
290+ struct nft_hook *hook, *found = NULL;
291+ int n = 0;
290292
291− switch (event) {
292− case NETDEV_UNREGISTER:
293− if (strcmp(basechain->dev_name, dev->name) != 0)
294− return;
295−
296− /* UNREGISTER events are also happpening on netns exit.
297− *
298− * Altough nf_tables core releases all tables/chains, only
299− * this event handler provides guarantee that
300− * basechain.ops->dev is still accessible, so we cannot
301− * skip exiting net namespaces.
302− */
303− __nft_release_basechain(ctx);
304− break;
305− case NETDEV_CHANGENAME:
306− if (dev->ifindex != basechain->ops.dev->ifindex)
307− return;
293+ if (event != NETDEV_UNREGISTER)
294+ return;
308295
309− strncpy(basechain->dev_name, dev->name, IFNAMSIZ);
310− break;
296+ list_for_each_entry(hook, &basechain->hook_list, list) {
297+ if (hook->ops.dev == dev)
298+ found = hook;
299+
300+ n++;
311301 }
302+ if (!found)
303+ return;
304+
305+ if (n > 1) {
306+ nf_unregister_net_hook(ctx->net, &found->ops);
307+ list_del_rcu(&found->list);
308+ kfree_rcu(found, rcu);
309+ return;
310+ }
311+
312+ /* UNREGISTER events are also happening on netns exit.
313+ *
314+ * Although nf_tables core releases all tables/chains, only this event
315+ * handler provides guarantee that hook->ops.dev is still accessible,
316+ * so we cannot skip exiting net namespaces.
317+ */
318+ __nft_release_basechain(ctx);
312319 }
313320
314321 static int nf_tables_netdev_event(struct notifier_block *this,
tools/testing/selftests/netfilter/Makefile+1−1
@@ -2,6 +2,6 @@ · 1 unmodified line
22 # Makefile for netfilter selftests
33
44 TEST_PROGS := nft_trans_stress.sh nft_nat.sh bridge_brouter.sh \
5− conntrack_icmp_related.sh nft_flowtable.sh
5+ conntrack_icmp_related.sh nft_flowtable.sh ipvs.sh
66
77 include ../lib.mk
tools/testing/selftests/netfilter/ipvs.sh+228−0
@@ -0,0 +1,228 @@
1+#!/bin/sh
2+# SPDX-License-Identifier: GPL-2.0
3+#
4+# End-to-end ipvs test suite
5+# Topology:
6+#--------------------------------------------------------------+
7+# | |
8+# ns0 | ns1 |
9+# ----------- | ----------- ----------- |
10+# | veth01 | --------- | veth10 | | veth12 | |
11+# ----------- peer ----------- ----------- |
12+# | | | |
13+# ----------- | | |
14+# | br0 | |----------------- peer |--------------|
15+# ----------- | | |
16+# | | | |
17+# ---------- peer ---------- ----------- |
18+# | veth02 | --------- | veth20 | | veth21 | |
19+# ---------- | ---------- ----------- |
20+# | ns2 |
21+# | |
22+#--------------------------------------------------------------+
23+#
24+# We assume that all network driver are loaded
25+#
26+
27+# Kselftest framework requirement - SKIP code is 4.
28+ksft_skip=4
29+ret=0
30+GREEN='\033[0;92m'
31+RED='\033[0;31m'
32+NC='\033[0m' # No Color
33+
34+readonly port=8080
35+
36+readonly vip_v4=207.175.44.110
37+readonly cip_v4=10.0.0.2
38+readonly gip_v4=10.0.0.1
39+readonly dip_v4=172.16.0.1
40+readonly rip_v4=172.16.0.2
41+readonly sip_v4=10.0.0.3
42+
43+readonly infile="$(mktemp)"
44+readonly outfile="$(mktemp)"
45+readonly datalen=32
46+
47+sysipvsnet="/proc/sys/net/ipv4/vs/"
48+if [ ! -d $sysipvsnet ]; then
49+ modprobe -q ip_vs
50+ if [ $? -ne 0 ]; then
51+ echo "skip: could not run test without ipvs module"
52+ exit $ksft_skip
53+ fi
54+fi
55+
56+ip -Version > /dev/null 2>&1
57+if [ $? -ne 0 ]; then
58+ echo "SKIP: Could not run test without ip tool"
59+ exit $ksft_skip
60+fi
61+
62+ipvsadm -v > /dev/null 2>&1
63+if [ $? -ne 0 ]; then
64+ echo "SKIP: Could not run test without ipvsadm"
65+ exit $ksft_skip
66+fi
67+
68+setup() {
69+ ip netns add ns0
70+ ip netns add ns1
71+ ip netns add ns2
72+
73+ ip link add veth01 netns ns0 type veth peer name veth10 netns ns1
74+ ip link add veth02 netns ns0 type veth peer name veth20 netns ns2
75+ ip link add veth12 netns ns1 type veth peer name veth21 netns ns2
76+
77+ ip netns exec ns0 ip link set veth01 up
78+ ip netns exec ns0 ip link set veth02 up
79+ ip netns exec ns0 ip link add br0 type bridge
80+ ip netns exec ns0 ip link set veth01 master br0
81+ ip netns exec ns0 ip link set veth02 master br0
82+ ip netns exec ns0 ip link set br0 up
83+ ip netns exec ns0 ip addr add ${cip_v4}/24 dev br0
84+
85+ ip netns exec ns1 ip link set lo up
86+ ip netns exec ns1 ip link set veth10 up
87+ ip netns exec ns1 ip addr add ${gip_v4}/24 dev veth10
88+ ip netns exec ns1 ip link set veth12 up
89+ ip netns exec ns1 ip addr add ${dip_v4}/24 dev veth12
90+
91+ ip netns exec ns2 ip link set lo up
92+ ip netns exec ns2 ip link set veth21 up
93+ ip netns exec ns2 ip addr add ${rip_v4}/24 dev veth21
94+ ip netns exec ns2 ip link set veth20 up
95+ ip netns exec ns2 ip addr add ${sip_v4}/24 dev veth20
96+
97+ sleep 1
98+
99+ dd if=/dev/urandom of="${infile}" bs="${datalen}" count=1 status=none
100+}
101+
102+cleanup() {
103+ for i in 0 1 2
104+ do
105+ ip netns del ns$i > /dev/null 2>&1
106+ done
107+
108+ if [ -f "${outfile}" ]; then
109+ rm "${outfile}"
110+ fi
111+ if [ -f "${infile}" ]; then
112+ rm "${infile}"
113+ fi
114+}
115+
116+server_listen() {
117+ ip netns exec ns2 nc -l -p 8080 > "${outfile}" &
118+ server_pid=$!
119+ sleep 0.2
120+}
121+
122+client_connect() {
123+ ip netns exec ns0 timeout 2 nc -w 1 ${vip_v4} ${port} < "${infile}"
124+}
125+
126+verify_data() {
127+ wait "${server_pid}"
128+ cmp "$infile" "$outfile" 2>/dev/null
129+}
130+
131+test_service() {
132+ server_listen
133+ client_connect
134+ verify_data
135+}
136+
137+
138+test_dr() {
139+ ip netns exec ns0 ip route add ${vip_v4} via ${gip_v4} dev br0
140+
141+ ip netns exec ns1 sysctl -qw net.ipv4.ip_forward=1
142+ ip netns exec ns1 ipvsadm -A -t ${vip_v4}:${port} -s rr
143+ ip netns exec ns1 ipvsadm -a -t ${vip_v4}:${port} -r ${rip_v4}:${port}
144+ ip netns exec ns1 ip addr add ${vip_v4}/32 dev lo:1
145+
146+ # avoid incorrect arp response
147+ ip netns exec ns2 sysctl -qw net.ipv4.conf.all.arp_ignore=1
148+ ip netns exec ns2 sysctl -qw net.ipv4.conf.all.arp_announce=2
149+ # avoid reverse route lookup
150+ ip netns exec ns2 sysctl -qw net.ipv4.conf.all.rp_filter=0
151+ ip netns exec ns2 sysctl -qw net.ipv4.conf.veth21.rp_filter=0
152+ ip netns exec ns2 ip addr add ${vip_v4}/32 dev lo:1
153+
154+ test_service
155+}
156+
157+test_nat() {
158+ ip netns exec ns0 ip route add ${vip_v4} via ${gip_v4} dev br0
159+
160+ ip netns exec ns1 sysctl -qw net.ipv4.ip_forward=1
161+ ip netns exec ns1 ipvsadm -A -t ${vip_v4}:${port} -s rr
162+ ip netns exec ns1 ipvsadm -a -m -t ${vip_v4}:${port} -r ${rip_v4}:${port}
163+ ip netns exec ns1 ip addr add ${vip_v4}/32 dev lo:1
164+
165+ ip netns exec ns2 ip link del veth20
166+ ip netns exec ns2 ip route add default via ${dip_v4} dev veth21
167+
168+ test_service
169+}
170+
171+test_tun() {
172+ ip netns exec ns0 ip route add ${vip_v4} via ${gip_v4} dev br0
173+
174+ ip netns exec ns1 modprobe ipip
175+ ip netns exec ns1 ip link set tunl0 up
176+ ip netns exec ns1 sysctl -qw net.ipv4.ip_forward=0
177+ ip netns exec ns1 sysctl -qw net.ipv4.conf.all.send_redirects=0
178+ ip netns exec ns1 sysctl -qw net.ipv4.conf.default.send_redirects=0
179+ ip netns exec ns1 ipvsadm -A -t ${vip_v4}:${port} -s rr
180+ ip netns exec ns1 ipvsadm -a -i -t ${vip_v4}:${port} -r ${rip_v4}:${port}
181+ ip netns exec ns1 ip addr add ${vip_v4}/32 dev lo:1
182+
183+ ip netns exec ns2 modprobe ipip
184+ ip netns exec ns2 ip link set tunl0 up
185+ ip netns exec ns2 sysctl -qw net.ipv4.conf.all.arp_ignore=1
186+ ip netns exec ns2 sysctl -qw net.ipv4.conf.all.arp_announce=2
187+ ip netns exec ns2 sysctl -qw net.ipv4.conf.all.rp_filter=0
188+ ip netns exec ns2 sysctl -qw net.ipv4.conf.tunl0.rp_filter=0
189+ ip netns exec ns2 sysctl -qw net.ipv4.conf.veth21.rp_filter=0
190+ ip netns exec ns2 ip addr add ${vip_v4}/32 dev lo:1
191+
192+ test_service
193+}
194+
195+run_tests() {
196+ local errors=
197+
198+ echo "Testing DR mode..."
199+ cleanup
200+ setup
201+ test_dr
202+ errors=$(( $errors + $? ))
203+
204+ echo "Testing NAT mode..."
205+ cleanup
206+ setup
207+ test_nat
208+ errors=$(( $errors + $? ))
209+
210+ echo "Testing Tunnel mode..."
211+ cleanup
212+ setup
213+ test_tun
214+ errors=$(( $errors + $? ))
215+
216+ return $errors
217+}
218+
219+trap cleanup EXIT
220+
221+run_tests
222+
223+if [ $? -ne 0 ]; then
224+ echo -e "$(basename $0): ${RED}FAIL${NC}"
225+ exit 1
226+fi
227+echo -e "$(basename $0): ${GREEN}PASS${NC}"
228+exit 0

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t