Pull networking fixes from Paolo Abeni: "This is again larger than usual: the backlog accumulated in the past weeks is not done yet. I'm not aware of any known pending regression. Including fixes from netfilter, Bluetooth, WiFi and CAN. Current release - regressions: - bluetooth: remove unnecessary hci_conn_get in create_conn_sync - can: isotp: fix timer drain order, wakeup handling and tx_gen ordering - eth: - tun/vhost: revert avoid ptr_ring tail-drop when a qdisc is present Previous releases - regressions: - core: do not send ICMP/NDISC Redirects when peer allocation fails - ipv6: take nexthop lock for f6i_list walks in replace check and notify - wifi: fix an ath12k MLO regression impacting WCN7850/QCC2072. - netfilter: nf_tables: make nft_object rhltable per table - af_unix: fix listen() succeeding on sockets in the wrong state - openvswitch: fix potential UAF on meter attach failure - bluetooth: - fix advertising data UAFs - avoid deadlocks in iso_sock_timeout - smc: fix socket use-after-free during link group termination - dpll: use pin owner's dpll ref for pin-level attribute reporting - eth: - veth: convert frag_list skbs before running XDP - ice: wait for reset completion in ice_resume() - igc: remove napi_synchronize() in igc_down() - vxlan: use pskb_network_may_pull() for transmit path header pulls Previous releases - always broken: - xsk: fix AF_XDP multi-buffer Tx descriptor reclaim - psp: fix NULL genl_sock deref race with concurrent netns teardown - netfilter: widen NAT rewrite delta to s32 in sip_help_tcp() - can: peak_usb: fix double free of transfer buffer on URB submit error - dibs: fix use-after-free of dmb_node in loopback attach/detach/unregister - sctp: prevent peer transport count overflow - dsa: mt7530: error out on failed reads in MT7531 PHY polling - eth: - idpf: bound interrupt-vector register fill to the allocated array" * tag 'net-7.2-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (156 commits) qede: sync udp_tunnel ports outside qede_lock in the recovery path net: openvswitch: fix potential UAF on meter attach failure octeontx2-pf: Set correct sequence for carrier off and tx queue stop net: libwx: fix FDIR ATR queue mismatch for software VLAN packets net: dsa: realtek: use devm_mutex_init for l2_lock net: dsa: realtek: use devm_mutex_init for vlan_lock net: dsa: realtek: use devm_mutex_init for regmap lock net: dsa: realtek: rtl8365mb: use devm_mutex_init for mib_lock ptp: netc: fix potential interrupt storm caused by incorrect unbind order net: mana: Return error code from mana_create_rxq() net: openvswitch: fix skb leak on flow key update failure during ct net: openvswitch: fix skb leak on flow key update failure during recirculation net: stmmac: Fix E2E delay mechanism net: dsa: mt7530: error out on failed reads in MT7531 PHY polling net: dsa: mt7530: error out on failed reads in ATC/VTCR command polling net: dsa: mt7530: check bus->read() errors in the MDIO regmap backend Revert "tun/tap: add ptr_ring consume helper with netdev queue wakeup" Revert "vhost-net: wake queue of tun/tap after ptr_ring consume" Revert "ptr_ring: move free-space check into separate helper" Revert "tun/tap & vhost-net: avoid ptr_ring tail-drop when a qdisc is present" ...
Unified · Split
| @@ -72,6 +72,7 @@ Alice Mikityanska <alice.kernel@fastmail.im> <maximmi@nvidia.com> · 71 unmodified lines | ||
| 72 | 72 | Alice Mikityanska <alice.kernel@fastmail.im> <maxim@isovalent.com> |
| 73 | 73 | Alice Mikityanska <alice.kernel@fastmail.im> <alice@isovalent.com> |
| 74 | 74 | Aloka Dixit <quic_alokad@quicinc.com> <alokad@codeaurora.org> |
| 75 | +Alvin Šipraga <alvin.sipraga@analog.com> <alsi@bang-olufsen.dk> | |
| 75 | 76 | Al Viro <viro@ftp.linux.org.uk> |
| 76 | 77 | Al Viro <viro@zenIV.linux.org.uk> |
| 77 | 78 | Amit Blay <quic_ablay@quicinc.com> <ablay@codeaurora.org> |
| @@ -43,12 +43,13 @@ UMEM also has two rings: the FILL ring and the COMPLETION ring. The · 42 unmodified lines | ||
| 43 | 43 | FILL ring is used by the application to send down addr for the kernel |
| 44 | 44 | to fill in with RX packet data. References to these frames will then |
| 45 | 45 | appear in the RX ring once each packet has been received. The |
| 46 | −COMPLETION ring, on the other hand, contains frame addr that the | |
| 47 | −kernel has transmitted completely and can now be used again by user | |
| 48 | −space, for either TX or RX. Thus, the frame addrs appearing in the | |
| 49 | −COMPLETION ring are addrs that were previously transmitted using the | |
| 50 | −TX ring. In summary, the RX and FILL rings are used for the RX path | |
| 51 | −and the TX and COMPLETION rings are used for the TX path. | |
| 46 | +COMPLETION ring, on the other hand, contains frame addresses from Tx | |
| 47 | +descriptors that the kernel has finished processing and that can now be | |
| 48 | +used again by user space, for either Tx or Rx. This includes frames whose | |
| 49 | +transmission has completed as well as frames referenced by invalid Tx | |
| 50 | +descriptors rejected by the kernel. A completion therefore returns | |
| 51 | +ownership of a frame to user space, but does not by itself guarantee that | |
| 52 | +the packet was successfully transmitted. | |
| 52 | 53 | |
| 53 | 54 | The socket is then finally bound with a bind() call to a device and a |
| 54 | 55 | specific queue id on that device, and it is not until bind is |
| @@ -169,14 +170,15 @@ chunks mode, then the incoming addr will be left untouched. · 114 unmodified lines | ||
| 169 | 170 | UMEM Completion Ring |
| 170 | 171 | ~~~~~~~~~~~~~~~~~~~~ |
| 171 | 172 | |
| 172 | −The COMPLETION Ring is used transfer ownership of UMEM frames from | |
| 173 | +The COMPLETION Ring is used to transfer ownership of UMEM frames from | |
| 173 | 174 | kernel-space to user-space. Just like the FILL ring, UMEM indices are |
| 174 | −used. | |
| 175 | − | |
| 176 | −Frames passed from the kernel to user-space are frames that has been | |
| 177 | −sent (TX ring) and can be used by user-space again. | |
| 178 | − | |
| 179 | −The user application consumes UMEM addrs from this ring. | |
| 175 | +used. Frames passed from the kernel to user-space are frames referenced | |
| 176 | +by Tx descriptors that the kernel has finished processing and can be | |
| 177 | +used by user-space again. This includes both frames whose transmission | |
| 178 | +has completed and frames referenced by invalid Tx descriptors that were | |
| 179 | +rejected and reclaimed by the kernel. A completion entry does not | |
| 180 | +guarantee successful packet transmission. The user application consumes | |
| 181 | +UMEM addrs from this ring. | |
| 180 | 182 | |
| 181 | 183 | |
| 182 | 184 | RX Ring |
| @@ -504,21 +506,25 @@ will be treated as an invalid descriptor. · 321 unmodified lines | ||
| 504 | 506 | These are the semantics for producing packets onto AF_XDP Tx ring |
| 505 | 507 | consisting of multiple frames: |
| 506 | 508 | |
| 507 | −* When an invalid descriptor is found, all the other | |
| 508 | − descriptors/frames of this packet are marked as invalid and not | |
| 509 | − completed. The next descriptor is treated as the start of a new | |
| 510 | − packet, even if this was not the intent (because we cannot guess | |
| 511 | − the intent). As before, if your program is producing invalid | |
| 512 | − descriptors you have a bug that must be fixed. | |
| 509 | +* When an invalid descriptor is found, the complete packet is treated as | |
| 510 | + invalid. The kernel consumes descriptors through the descriptor marking | |
| 511 | + the end of the packet and returns all their frame addresses through the | |
| 512 | + COMPLETION ring. A standalone invalid descriptor is treated as a | |
| 513 | + one-descriptor invalid packet. The descriptor following the end of the | |
| 514 | + invalid packet is treated as the start of a new packet. As before, if | |
| 515 | + your program is producing invalid descriptors you have a bug that must | |
| 516 | + be fixed. Rejected descriptors are reported in the ``tx_invalid_descs`` | |
| 517 | + statistic. | |
| 513 | 518 | |
| 514 | 519 | * Zero length descriptors are treated as invalid descriptors. |
| 515 | 520 | |
| 516 | 521 | * For copy mode, the maximum supported number of frames in a packet is |
| 517 | − equal to CONFIG_MAX_SKB_FRAGS + 1. If it is exceeded, all | |
| 518 | − descriptors accumulated so far are dropped and treated as | |
| 519 | − invalid. To produce an application that will work on any system | |
| 520 | − regardless of this config setting, limit the number of frags to 18, | |
| 521 | − as the minimum value of the config is 17. | |
| 522 | + equal to CONFIG_MAX_SKB_FRAGS + 1. If it is exceeded, all descriptors | |
| 523 | + through the end of the oversized packet are consumed, treated as invalid, | |
| 524 | + and their frame addresses are returned through the COMPLETION ring. To | |
| 525 | + produce an application that will work on any system regardless of this | |
| 526 | + config setting, limit the number of frags to 18, as the minimum value of | |
| 527 | + the config is 17. | |
| 522 | 528 | |
| 523 | 529 | * For zero-copy mode, the limit is up to what the NIC HW |
| 524 | 530 | supports. Usually at least five on the NICs we have checked. We |
| @@ -22736,7 +22736,8 @@ F: drivers/watchdog/realtek_otto_wdt.c · 22735 unmodified lines | ||
| 22736 | 22736 | |
| 22737 | 22737 | REALTEK RTL83xx SMI DSA ROUTER CHIPS |
| 22738 | 22738 | M: Linus Walleij <linusw@kernel.org> |
| 22739 | −M: Alvin Šipraga <alsi@bang-olufsen.dk> | |
| 22739 | +M: Luiz Angelo Daros de Luca <luizluca@gmail.com> | |
| 22740 | +R: Alvin Šipraga <alvin.sipraga@analog.com> | |
| 22740 | 22741 | S: Maintained |
| 22741 | 22742 | F: Documentation/devicetree/bindings/net/dsa/realtek.yaml |
| 22742 | 22743 | F: drivers/net/dsa/realtek/* |
| @@ -24633,8 +24634,7 @@ SHARED MEMORY COMMUNICATIONS (SMC) SOCKETS · 1890 unmodified lines | ||
| 24633 | 24634 | M: D. Wythe <alibuda@linux.alibaba.com> |
| 24634 | 24635 | M: Dust Li <dust.li@linux.alibaba.com> |
| 24635 | 24636 | M: Sidraya Jayagond <sidraya@linux.ibm.com> |
| 24636 | −M: Wenjia Zhang <wenjia@linux.ibm.com> | |
| 24637 | −R: Mahanta Jambigi <mjambigi@linux.ibm.com> | |
| 24637 | +M: Mahanta Jambigi <mjambigi@linux.ibm.com> | |
| 24638 | 24638 | R: Tony Lu <tonylu@linux.alibaba.com> |
| 24639 | 24639 | R: Wen Gu <guwen@linux.alibaba.com> |
| 24640 | 24640 | L: linux-rdma@vger.kernel.org |
| @@ -3771,6 +3771,9 @@ static int btintel_diagnostics(struct hci_dev *hdev, struct sk_buff *skb) · 3770 unmodified lines | ||
| 3771 | 3771 | { |
| 3772 | 3772 | struct intel_tlv *tlv = (void *)&skb->data[5]; |
| 3773 | 3773 | |
| 3774 | + if (skb->len < 5 + sizeof(*tlv) + sizeof(tlv->val[0])) | |
| 3775 | + goto recv_frame; | |
| 3776 | + | |
| 3774 | 3777 | /* The first event is always an event type TLV */ |
| 3775 | 3778 | if (tlv->type != INTEL_TLV_TYPE_ID) |
| 3776 | 3779 | goto recv_frame; |
| @@ -804,56 +804,44 @@ static int btmtk_usb_uhw_reg_write(struct hci_dev *hdev, u32 reg, u32 val) · 803 unmodified lines | ||
| 804 | 804 | static int btmtk_usb_uhw_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) |
| 805 | 805 | { |
| 806 | 806 | struct btmtk_data *data = hci_get_priv(hdev); |
| 807 | − int pipe, err; | |
| 808 | − void *buf; | |
| 809 | − | |
| 810 | − buf = kzalloc(4, GFP_KERNEL); | |
| 811 | − if (!buf) | |
| 812 | − return -ENOMEM; | |
| 807 | + u8 buf[sizeof(u32)]; | |
| 808 | + int err; | |
| 813 | 809 | |
| 814 | − pipe = usb_rcvctrlpipe(data->udev, 0); | |
| 815 | − err = usb_control_msg(data->udev, pipe, 0x01, | |
| 816 | − 0xDE, | |
| 817 | − reg >> 16, reg & 0xffff, | |
| 818 | − buf, 4, USB_CTRL_GET_TIMEOUT); | |
| 819 | − if (err < 0) { | |
| 810 | + *val = 0; | |
| 811 | + err = usb_control_msg_recv(data->udev, 0, 0x01, | |
| 812 | + 0xDE, | |
| 813 | + reg >> 16, reg & 0xffff, | |
| 814 | + buf, sizeof(buf), USB_CTRL_GET_TIMEOUT, | |
| 815 | + GFP_KERNEL); | |
| 816 | + if (err) { | |
| 820 | 817 | bt_dev_err(hdev, "Failed to read uhw reg(%d)", err); |
| 821 | − goto err_free_buf; | |
| 818 | + return err; | |
| 822 | 819 | } |
| 823 | 820 | |
| 824 | 821 | *val = get_unaligned_le32(buf); |
| 825 | 822 | bt_dev_dbg(hdev, "reg=%x, value=0x%08x", reg, *val); |
| 826 | 823 | |
| 827 | −err_free_buf: | |
| 828 | − kfree(buf); | |
| 829 | − | |
| 830 | − return err; | |
| 824 | + return 0; | |
| 831 | 825 | } |
| 832 | 826 | |
| 833 | 827 | static int btmtk_usb_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) |
| 834 | 828 | { |
| 835 | 829 | struct btmtk_data *data = hci_get_priv(hdev); |
| 836 | − int pipe, err, size = sizeof(u32); | |
| 837 | − void *buf; | |
| 838 | − | |
| 839 | − buf = kzalloc(size, GFP_KERNEL); | |
| 840 | − if (!buf) | |
| 841 | − return -ENOMEM; | |
| 830 | + u8 buf[sizeof(u32)]; | |
| 831 | + int err; | |
| 842 | 832 | |
| 843 | − pipe = usb_rcvctrlpipe(data->udev, 0); | |
| 844 | − err = usb_control_msg(data->udev, pipe, 0x63, | |
| 845 | − USB_TYPE_VENDOR | USB_DIR_IN, | |
| 846 | − reg >> 16, reg & 0xffff, | |
| 847 | − buf, size, USB_CTRL_GET_TIMEOUT); | |
| 833 | + *val = 0; | |
| 834 | + err = usb_control_msg_recv(data->udev, 0, 0x63, | |
| 835 | + USB_TYPE_VENDOR | USB_DIR_IN, | |
| 836 | + reg >> 16, reg & 0xffff, | |
| 837 | + buf, sizeof(buf), USB_CTRL_GET_TIMEOUT, | |
| 838 | + GFP_KERNEL); | |
| 848 | 839 | if (err < 0) |
| 849 | − goto err_free_buf; | |
| 840 | + return err; | |
| 850 | 841 | |
| 851 | 842 | *val = get_unaligned_le32(buf); |
| 852 | 843 | |
| 853 | −err_free_buf: | |
| 854 | − kfree(buf); | |
| 855 | − | |
| 856 | − return err; | |
| 844 | + return 0; | |
| 857 | 845 | } |
| 858 | 846 | |
| 859 | 847 | static int btmtk_usb_id_get(struct hci_dev *hdev, u32 reg, u32 *id) |
| @@ -877,7 +865,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 17 unmodified lines | ||
| 877 | 865 | |
| 878 | 866 | if (dev_id == 0x7922) { |
| 879 | 867 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 880 | − if (err < 0) | |
| 868 | + if (err) | |
| 881 | 869 | return err; |
| 882 | 870 | val |= 0x00002020; |
| 883 | 871 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); |
| @@ -887,7 +875,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 887 | 875 | if (err < 0) |
| 888 | 876 | return err; |
| 889 | 877 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 890 | − if (err < 0) | |
| 878 | + if (err) | |
| 891 | 879 | return err; |
| 892 | 880 | val |= BIT(0); |
| 893 | 881 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); |
| @@ -896,14 +884,14 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 2 unmodified lines | ||
| 896 | 884 | msleep(100); |
| 897 | 885 | } else if (dev_id == 0x7925 || dev_id == 0x6639) { |
| 898 | 886 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 899 | − if (err < 0) | |
| 887 | + if (err) | |
| 900 | 888 | return err; |
| 901 | 889 | val |= (1 << 5); |
| 902 | 890 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); |
| 903 | 891 | if (err < 0) |
| 904 | 892 | return err; |
| 905 | 893 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 906 | − if (err < 0) | |
| 894 | + if (err) | |
| 907 | 895 | return err; |
| 908 | 896 | val &= 0xFFFF00FF; |
| 909 | 897 | val |= (1 << 13); |
| @@ -914,7 +902,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 4 unmodified lines | ||
| 914 | 902 | if (err < 0) |
| 915 | 903 | return err; |
| 916 | 904 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 917 | − if (err < 0) | |
| 905 | + if (err) | |
| 918 | 906 | return err; |
| 919 | 907 | val |= (1 << 0); |
| 920 | 908 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); |
| @@ -924,13 +912,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 924 | 912 | if (err < 0) |
| 925 | 913 | return err; |
| 926 | 914 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); |
| 927 | − if (err < 0) | |
| 915 | + if (err) | |
| 928 | 916 | return err; |
| 929 | 917 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); |
| 930 | 918 | if (err < 0) |
| 931 | 919 | return err; |
| 932 | 920 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); |
| 933 | − if (err < 0) | |
| 921 | + if (err) | |
| 934 | 922 | return err; |
| 935 | 923 | msleep(100); |
| 936 | 924 | } else { |
| @@ -940,7 +928,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 940 | 928 | if (err < 0) |
| 941 | 929 | return err; |
| 942 | 930 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_WDT_STATUS, &val); |
| 943 | − if (err < 0) | |
| 931 | + if (err) | |
| 944 | 932 | return err; |
| 945 | 933 | /* Reset the bluetooth chip via USB interface. */ |
| 946 | 934 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, 1); |
| @@ -950,13 +938,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | ||
| 950 | 938 | if (err < 0) |
| 951 | 939 | return err; |
| 952 | 940 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); |
| 953 | − if (err < 0) | |
| 941 | + if (err) | |
| 954 | 942 | return err; |
| 955 | 943 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); |
| 956 | 944 | if (err < 0) |
| 957 | 945 | return err; |
| 958 | 946 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); |
| 959 | − if (err < 0) | |
| 947 | + if (err) | |
| 960 | 948 | return err; |
| 961 | 949 | /* MT7921 need to delay 20ms between toggle reset bit */ |
| 962 | 950 | msleep(20); |
| @@ -964,7 +952,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 1 unmodified line | ||
| 964 | 952 | if (err < 0) |
| 965 | 953 | return err; |
| 966 | 954 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 967 | − if (err < 0) | |
| 955 | + if (err) | |
| 968 | 956 | return err; |
| 969 | 957 | } |
| 970 | 958 | |
| @@ -980,7 +968,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 9 unmodified lines | ||
| 980 | 968 | } |
| 981 | 969 | |
| 982 | 970 | err = btmtk_usb_id_get(hdev, 0x70010200, &val); |
| 983 | − if (err < 0 || (!val && dev_id != 0x6639)) | |
| 971 | + if (err || (!val && dev_id != 0x6639)) | |
| 984 | 972 | bt_dev_err(hdev, "Can't get device id, subsys reset fail."); |
| 985 | 973 | |
| 986 | 974 | return err; |
| @@ -1324,24 +1312,24 @@ int btmtk_usb_setup(struct hci_dev *hdev) · 337 unmodified lines | ||
| 1324 | 1312 | calltime = ktime_get(); |
| 1325 | 1313 | |
| 1326 | 1314 | err = btmtk_usb_id_get(hdev, 0x80000008, &dev_id); |
| 1327 | − if (err < 0) { | |
| 1315 | + if (err) { | |
| 1328 | 1316 | bt_dev_err(hdev, "Failed to get device id (%d)", err); |
| 1329 | 1317 | return err; |
| 1330 | 1318 | } |
| 1331 | 1319 | |
| 1332 | 1320 | if (!dev_id || dev_id != 0x7663) { |
| 1333 | 1321 | err = btmtk_usb_id_get(hdev, 0x70010200, &dev_id); |
| 1334 | − if (err < 0) { | |
| 1322 | + if (err) { | |
| 1335 | 1323 | bt_dev_err(hdev, "Failed to get device id (%d)", err); |
| 1336 | 1324 | return err; |
| 1337 | 1325 | } |
| 1338 | 1326 | err = btmtk_usb_id_get(hdev, 0x80021004, &fw_version); |
| 1339 | − if (err < 0) { | |
| 1327 | + if (err) { | |
| 1340 | 1328 | bt_dev_err(hdev, "Failed to get fw version (%d)", err); |
| 1341 | 1329 | return err; |
| 1342 | 1330 | } |
| 1343 | 1331 | err = btmtk_usb_id_get(hdev, 0x70010020, &fw_flavor); |
| 1344 | − if (err < 0) { | |
| 1332 | + if (err) { | |
| 1345 | 1333 | bt_dev_err(hdev, "Failed to get fw flavor (%d)", err); |
| 1346 | 1334 | return err; |
| 1347 | 1335 | } |
| @@ -3424,28 +3424,16 @@ static const char *qca_get_fw_subdirectory(const struct qca_version *ver) · 3423 unmodified lines | ||
| 3424 | 3424 | static int btusb_qca_send_vendor_req(struct usb_device *udev, u8 request, |
| 3425 | 3425 | void *data, u16 size) |
| 3426 | 3426 | { |
| 3427 | − int pipe, err; | |
| 3428 | − u8 *buf; | |
| 3429 | − | |
| 3430 | − buf = kmalloc(size, GFP_KERNEL); | |
| 3431 | − if (!buf) | |
| 3432 | − return -ENOMEM; | |
| 3427 | + int err; | |
| 3433 | 3428 | |
| 3434 | 3429 | /* Found some of USB hosts have IOT issues with ours so that we should |
| 3435 | 3430 | * not wait until HCI layer is ready. |
| 3436 | 3431 | */ |
| 3437 | − pipe = usb_rcvctrlpipe(udev, 0); | |
| 3438 | − err = usb_control_msg(udev, pipe, request, USB_TYPE_VENDOR | USB_DIR_IN, | |
| 3439 | − 0, 0, buf, size, USB_CTRL_GET_TIMEOUT); | |
| 3440 | − if (err < 0) { | |
| 3432 | + err = usb_control_msg_recv(udev, 0, request, USB_TYPE_VENDOR | USB_DIR_IN, | |
| 3433 | + 0, 0, data, size, USB_CTRL_GET_TIMEOUT, | |
| 3434 | + GFP_KERNEL); | |
| 3435 | + if (err) | |
| 3441 | 3436 | dev_err(&udev->dev, "Failed to access otp area (%d)", err); |
| 3442 | − goto done; | |
| 3443 | − } | |
| 3444 | − | |
| 3445 | − memcpy(data, buf, size); | |
| 3446 | − | |
| 3447 | −done: | |
| 3448 | − kfree(buf); | |
| 3449 | 3437 | |
| 3450 | 3438 | return err; |
| 3451 | 3439 | } |
| @@ -3652,7 +3640,7 @@ static bool btusb_qca_need_patch(struct usb_device *udev) · 200 unmodified lines | ||
| 3652 | 3640 | struct qca_version ver; |
| 3653 | 3641 | |
| 3654 | 3642 | if (btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3655 | − sizeof(ver)) < 0) | |
| 3643 | + sizeof(ver))) | |
| 3656 | 3644 | return false; |
| 3657 | 3645 | /* only low ROM versions need patches */ |
| 3658 | 3646 | return !(le32_to_cpu(ver.rom_version) & ~0xffffU); |
| @@ -3670,7 +3658,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 11 unmodified lines | ||
| 3670 | 3658 | |
| 3671 | 3659 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3672 | 3660 | sizeof(ver)); |
| 3673 | − if (err < 0) | |
| 3661 | + if (err) | |
| 3674 | 3662 | return err; |
| 3675 | 3663 | |
| 3676 | 3664 | ver_rom = le32_to_cpu(ver.rom_version); |
| @@ -3693,7 +3681,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 16 unmodified lines | ||
| 3693 | 3681 | |
| 3694 | 3682 | err = btusb_qca_send_vendor_req(udev, QCA_CHECK_STATUS, &status, |
| 3695 | 3683 | sizeof(status)); |
| 3696 | − if (err < 0) | |
| 3684 | + if (err) | |
| 3697 | 3685 | return err; |
| 3698 | 3686 | |
| 3699 | 3687 | if (!(status & QCA_PATCH_UPDATED)) { |
| @@ -3704,7 +3692,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 4 unmodified lines | ||
| 3704 | 3692 | |
| 3705 | 3693 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3706 | 3694 | sizeof(ver)); |
| 3707 | − if (err < 0) | |
| 3695 | + if (err) | |
| 3708 | 3696 | return err; |
| 3709 | 3697 | |
| 3710 | 3698 | btdata->qca_dump.fw_version = le32_to_cpu(ver.patch_version); |
| @@ -118,14 +118,9 @@ err_bit: · 117 unmodified lines | ||
| 118 | 118 | return rc; |
| 119 | 119 | } |
| 120 | 120 | |
| 121 | −static void __dibs_lo_unregister_dmb(struct dibs_lo_dev *ldev, | |
| 122 | − struct dibs_lo_dmb_node *dmb_node) | |
| 121 | +static void dibs_lo_free_dmb(struct dibs_lo_dev *ldev, | |
| 122 | + struct dibs_lo_dmb_node *dmb_node) | |
| 123 | 123 | { |
| 124 | − /* remove dmb from hash table */ | |
| 125 | − write_lock_bh(&ldev->dmb_ht_lock); | |
| 126 | − hash_del(&dmb_node->list); | |
| 127 | − write_unlock_bh(&ldev->dmb_ht_lock); | |
| 128 | − | |
| 129 | 124 | clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); |
| 130 | 125 | folio_put(virt_to_folio(dmb_node->cpu_addr)); |
| 131 | 126 | kfree(dmb_node); |
| @@ -139,27 +134,33 @@ static int dibs_lo_unregister_dmb(struct dibs_dev *dibs, struct dibs_dmb *dmb) · 7 unmodified lines | ||
| 139 | 134 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; |
| 140 | 135 | struct dibs_lo_dev *ldev; |
| 141 | 136 | unsigned long flags; |
| 137 | + bool last; | |
| 142 | 138 | |
| 143 | 139 | ldev = dibs->drv_priv; |
| 144 | 140 | |
| 145 | 141 | /* find dmb from hash table */ |
| 146 | − read_lock_bh(&ldev->dmb_ht_lock); | |
| 142 | + write_lock_bh(&ldev->dmb_ht_lock); | |
| 147 | 143 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { |
| 148 | 144 | if (tmp_node->token == dmb->dmb_tok) { |
| 149 | 145 | dmb_node = tmp_node; |
| 150 | 146 | break; |
| 151 | 147 | } |
| 152 | 148 | } |
| 153 | − read_unlock_bh(&ldev->dmb_ht_lock); | |
| 154 | − if (!dmb_node) | |
| 149 | + if (!dmb_node) { | |
| 150 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 155 | 151 | return -EINVAL; |
| 152 | + } | |
| 153 | + last = refcount_dec_and_test(&dmb_node->refcnt); | |
| 154 | + if (last) | |
| 155 | + hash_del(&dmb_node->list); | |
| 156 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 156 | 157 | |
| 157 | − if (refcount_dec_and_test(&dmb_node->refcnt)) { | |
| 158 | + if (last) { | |
| 158 | 159 | spin_lock_irqsave(&dibs->lock, flags); |
| 159 | 160 | dibs->dmb_clientid_arr[dmb_node->sba_idx] = NO_DIBS_CLIENT; |
| 160 | 161 | spin_unlock_irqrestore(&dibs->lock, flags); |
| 161 | 162 | |
| 162 | − __dibs_lo_unregister_dmb(ldev, dmb_node); | |
| 163 | + dibs_lo_free_dmb(ldev, dmb_node); | |
| 163 | 164 | } |
| 164 | 165 | return 0; |
| 165 | 166 | } |
| @@ -188,14 +189,9 @@ static int dibs_lo_attach_dmb(struct dibs_dev *dibs, struct dibs_dmb *dmb) · 22 unmodified lines | ||
| 188 | 189 | read_unlock_bh(&ldev->dmb_ht_lock); |
| 189 | 190 | return -EINVAL; |
| 190 | 191 | } |
| 192 | + refcount_inc(&dmb_node->refcnt); | |
| 191 | 193 | read_unlock_bh(&ldev->dmb_ht_lock); |
| 192 | 194 | |
| 193 | − if (!refcount_inc_not_zero(&dmb_node->refcnt)) | |
| 194 | − /* the dmb is being unregistered, but has | |
| 195 | − * not been removed from the hash table. | |
| 196 | − */ | |
| 197 | − return -EINVAL; | |
| 198 | − | |
| 199 | 195 | /* provide dmb information */ |
| 200 | 196 | dmb->idx = dmb_node->sba_idx; |
| 201 | 197 | dmb->dmb_tok = dmb_node->token; |
| @@ -209,11 +205,12 @@ static int dibs_lo_detach_dmb(struct dibs_dev *dibs, u64 token) · 7 unmodified lines | ||
| 209 | 205 | { |
| 210 | 206 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; |
| 211 | 207 | struct dibs_lo_dev *ldev; |
| 208 | + bool last; | |
| 212 | 209 | |
| 213 | 210 | ldev = dibs->drv_priv; |
| 214 | 211 | |
| 215 | 212 | /* find dmb_node according to dmb->dmb_tok */ |
| 216 | − read_lock_bh(&ldev->dmb_ht_lock); | |
| 213 | + write_lock_bh(&ldev->dmb_ht_lock); | |
| 217 | 214 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { |
| 218 | 215 | if (tmp_node->token == token) { |
| 219 | 216 | dmb_node = tmp_node; |
| @@ -221,13 +218,17 @@ static int dibs_lo_detach_dmb(struct dibs_dev *dibs, u64 token) · 1 unmodified line | ||
| 221 | 218 | } |
| 222 | 219 | } |
| 223 | 220 | if (!dmb_node) { |
| 224 | − read_unlock_bh(&ldev->dmb_ht_lock); | |
| 221 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 225 | 222 | return -EINVAL; |
| 226 | 223 | } |
| 227 | − read_unlock_bh(&ldev->dmb_ht_lock); | |
| 224 | + last = refcount_dec_and_test(&dmb_node->refcnt); | |
| 225 | + if (last) | |
| 226 | + hash_del(&dmb_node->list); | |
| 227 | + write_unlock_bh(&ldev->dmb_ht_lock); | |
| 228 | + | |
| 229 | + if (last) | |
| 230 | + dibs_lo_free_dmb(ldev, dmb_node); | |
| 228 | 231 | |
| 229 | − if (refcount_dec_and_test(&dmb_node->refcnt)) | |
| 230 | − __dibs_lo_unregister_dmb(ldev, dmb_node); | |
| 231 | 232 | return 0; |
| 232 | 233 | } |
| 233 | 234 | |
| @@ -1142,6 +1142,33 @@ void *dpll_pin_on_pin_priv(struct dpll_pin *parent, · 1141 unmodified lines | ||
| 1142 | 1142 | return reg->priv; |
| 1143 | 1143 | } |
| 1144 | 1144 | |
| 1145 | +/** | |
| 1146 | + * dpll_pin_own_dpll_ref_first - find the first owner dpll ref of a pin | |
| 1147 | + * @pin: pointer to a dpll pin | |
| 1148 | + * | |
| 1149 | + * Search pin's dpll_refs for a ref whose dpll matches the pin's | |
| 1150 | + * (module, clock_id) tuple, i.e. the dpll registered by the driver | |
| 1151 | + * that created the pin. This ensures pin-level attributes are | |
| 1152 | + * reported and modified using the owner's ops even when the pin is | |
| 1153 | + * also registered with dplls from other drivers. | |
| 1154 | + * | |
| 1155 | + * Return: pointer to the owner's dpll_pin_ref, or NULL if no | |
| 1156 | + * owner ref is found. | |
| 1157 | + */ | |
| 1158 | +struct dpll_pin_ref *dpll_pin_own_dpll_ref_first(struct dpll_pin *pin) | |
| 1159 | +{ | |
| 1160 | + struct dpll_pin_ref *ref; | |
| 1161 | + unsigned long i; | |
| 1162 | + | |
| 1163 | + xa_for_each(&pin->dpll_refs, i, ref) { | |
| 1164 | + if (ref->dpll->module == pin->module && | |
| 1165 | + ref->dpll->clock_id == pin->clock_id) | |
| 1166 | + return ref; | |
| 1167 | + } | |
| 1168 | + | |
| 1169 | + return NULL; | |
| 1170 | +} | |
| 1171 | + | |
| 1145 | 1172 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref) |
| 1146 | 1173 | { |
| 1147 | 1174 | struct dpll_pin_registration *reg; |
| @@ -93,6 +93,7 @@ void *dpll_pin_on_pin_priv(struct dpll_pin *parent, struct dpll_pin *pin); · 92 unmodified lines | ||
| 93 | 93 | |
| 94 | 94 | const struct dpll_device_ops *dpll_device_ops(struct dpll_device *dpll); |
| 95 | 95 | struct dpll_device *dpll_device_get_by_id(int id); |
| 96 | +struct dpll_pin_ref *dpll_pin_own_dpll_ref_first(struct dpll_pin *pin); | |
| 96 | 97 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref); |
| 97 | 98 | struct dpll_pin_ref *dpll_xa_ref_dpll_first(struct xarray *xa_refs); |
| 98 | 99 | extern struct xarray dpll_device_xa; |
| @@ -699,7 +699,9 @@ dpll_cmd_pin_get_one(struct sk_buff *msg, struct dpll_pin *pin, · 698 unmodified lines | ||
| 699 | 699 | struct dpll_pin_ref *ref; |
| 700 | 700 | int ret; |
| 701 | 701 | |
| 702 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 702 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 703 | + if (!ref) | |
| 704 | + ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 703 | 705 | ASSERT_NOT_NULL(ref); |
| 704 | 706 | |
| 705 | 707 | ret = dpll_msg_add_pin_handle(msg, pin); |
| @@ -1090,12 +1092,19 @@ dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a, · 384 unmodified lines | ||
| 1090 | 1092 | |
| 1091 | 1093 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1092 | 1094 | ops = dpll_pin_ops(ref); |
| 1093 | − if (!ops->frequency_set || !ops->frequency_get) { | |
| 1094 | − NL_SET_ERR_MSG(extack, "frequency set not supported by the device"); | |
| 1095 | + if ((!ops->frequency_set || !ops->frequency_get) && | |
| 1096 | + ref->dpll->module == pin->module && | |
| 1097 | + ref->dpll->clock_id == pin->clock_id) { | |
| 1098 | + NL_SET_ERR_MSG(extack, | |
| 1099 | + "frequency set not supported by the device"); | |
| 1095 | 1100 | return -EOPNOTSUPP; |
| 1096 | 1101 | } |
| 1097 | 1102 | } |
| 1098 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1103 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1104 | + if (!ref) { | |
| 1105 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1106 | + return -ENODEV; | |
| 1107 | + } | |
| 1099 | 1108 | ops = dpll_pin_ops(ref); |
| 1100 | 1109 | dpll = ref->dpll; |
| 1101 | 1110 | ret = ops->frequency_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, |
| @@ -1109,6 +1118,8 @@ dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a, · 7 unmodified lines | ||
| 1109 | 1118 | |
| 1110 | 1119 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1111 | 1120 | ops = dpll_pin_ops(ref); |
| 1121 | + if (!ops->frequency_set) | |
| 1122 | + continue; | |
| 1112 | 1123 | dpll = ref->dpll; |
| 1113 | 1124 | ret = ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1114 | 1125 | dpll, dpll_priv(dpll), freq, extack); |
| @@ -1128,6 +1139,8 @@ rollback: · 13 unmodified lines | ||
| 1128 | 1139 | if (ref == failed) |
| 1129 | 1140 | break; |
| 1130 | 1141 | ops = dpll_pin_ops(ref); |
| 1142 | + if (!ops->frequency_set) | |
| 1143 | + continue; | |
| 1131 | 1144 | dpll = ref->dpll; |
| 1132 | 1145 | if (ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1133 | 1146 | dpll, dpll_priv(dpll), old_freq, extack)) |
| @@ -1151,13 +1164,19 @@ dpll_pin_esync_set(struct dpll_pin *pin, struct nlattr *a, · 17 unmodified lines | ||
| 1151 | 1164 | |
| 1152 | 1165 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1153 | 1166 | ops = dpll_pin_ops(ref); |
| 1154 | − if (!ops->esync_set || !ops->esync_get) { | |
| 1167 | + if ((!ops->esync_set || !ops->esync_get) && | |
| 1168 | + ref->dpll->module == pin->module && | |
| 1169 | + ref->dpll->clock_id == pin->clock_id) { | |
| 1155 | 1170 | NL_SET_ERR_MSG(extack, |
| 1156 | 1171 | "embedded sync feature is not supported by this device"); |
| 1157 | 1172 | return -EOPNOTSUPP; |
| 1158 | 1173 | } |
| 1159 | 1174 | } |
| 1160 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1175 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1176 | + if (!ref) { | |
| 1177 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1178 | + return -ENODEV; | |
| 1179 | + } | |
| 1161 | 1180 | ops = dpll_pin_ops(ref); |
| 1162 | 1181 | dpll = ref->dpll; |
| 1163 | 1182 | ret = ops->esync_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, |
| @@ -1181,6 +1200,8 @@ dpll_pin_esync_set(struct dpll_pin *pin, struct nlattr *a, · 17 unmodified lines | ||
| 1181 | 1200 | void *pin_dpll_priv; |
| 1182 | 1201 | |
| 1183 | 1202 | ops = dpll_pin_ops(ref); |
| 1203 | + if (!ops->esync_set) | |
| 1204 | + continue; | |
| 1184 | 1205 | dpll = ref->dpll; |
| 1185 | 1206 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); |
| 1186 | 1207 | ret = ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), |
| @@ -1204,6 +1225,8 @@ rollback: · 17 unmodified lines | ||
| 1204 | 1225 | if (ref == failed) |
| 1205 | 1226 | break; |
| 1206 | 1227 | ops = dpll_pin_ops(ref); |
| 1228 | + if (!ops->esync_set) | |
| 1229 | + continue; | |
| 1207 | 1230 | dpll = ref->dpll; |
| 1208 | 1231 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); |
| 1209 | 1232 | if (ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), |
| @@ -1238,8 +1261,11 @@ dpll_pin_ref_sync_state_set(struct dpll_pin *pin, · 28 unmodified lines | ||
| 1238 | 1261 | NL_SET_ERR_MSG(extack, "reference sync pin not available"); |
| 1239 | 1262 | return -EINVAL; |
| 1240 | 1263 | } |
| 1241 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1242 | − ASSERT_NOT_NULL(ref); | |
| 1264 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1265 | + if (!ref) { | |
| 1266 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1267 | + return -ENODEV; | |
| 1268 | + } | |
| 1243 | 1269 | ops = dpll_pin_ops(ref); |
| 1244 | 1270 | if (!ops->ref_sync_set || !ops->ref_sync_get) { |
| 1245 | 1271 | NL_SET_ERR_MSG(extack, "reference sync not supported by this pin"); |
| @@ -1258,6 +1284,8 @@ dpll_pin_ref_sync_state_set(struct dpll_pin *pin, · 12 unmodified lines | ||
| 1258 | 1284 | return 0; |
| 1259 | 1285 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1260 | 1286 | ops = dpll_pin_ops(ref); |
| 1287 | + if (!ops->ref_sync_set) | |
| 1288 | + continue; | |
| 1261 | 1289 | dpll = ref->dpll; |
| 1262 | 1290 | ret = ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1263 | 1291 | ref_sync_pin, |
| @@ -1280,6 +1308,8 @@ rollback: · 16 unmodified lines | ||
| 1280 | 1308 | if (ref == failed) |
| 1281 | 1309 | break; |
| 1282 | 1310 | ops = dpll_pin_ops(ref); |
| 1311 | + if (!ops->ref_sync_set) | |
| 1312 | + continue; | |
| 1283 | 1313 | dpll = ref->dpll; |
| 1284 | 1314 | if (ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1285 | 1315 | ref_sync_pin, |
| @@ -1471,12 +1501,18 @@ dpll_pin_phase_adj_set(struct dpll_pin *pin, struct nlattr *phase_adj_attr, · 185 unmodified lines | ||
| 1471 | 1501 | |
| 1472 | 1502 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1473 | 1503 | ops = dpll_pin_ops(ref); |
| 1474 | − if (!ops->phase_adjust_set || !ops->phase_adjust_get) { | |
| 1504 | + if ((!ops->phase_adjust_set || !ops->phase_adjust_get) && | |
| 1505 | + ref->dpll->module == pin->module && | |
| 1506 | + ref->dpll->clock_id == pin->clock_id) { | |
| 1475 | 1507 | NL_SET_ERR_MSG(extack, "phase adjust not supported"); |
| 1476 | 1508 | return -EOPNOTSUPP; |
| 1477 | 1509 | } |
| 1478 | 1510 | } |
| 1479 | − ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | |
| 1511 | + ref = dpll_pin_own_dpll_ref_first(pin); | |
| 1512 | + if (!ref) { | |
| 1513 | + NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | |
| 1514 | + return -ENODEV; | |
| 1515 | + } | |
| 1480 | 1516 | ops = dpll_pin_ops(ref); |
| 1481 | 1517 | dpll = ref->dpll; |
| 1482 | 1518 | ret = ops->phase_adjust_get(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| @@ -1491,6 +1527,8 @@ dpll_pin_phase_adj_set(struct dpll_pin *pin, struct nlattr *phase_adj_attr, · 8 unmodified lines | ||
| 1491 | 1527 | |
| 1492 | 1528 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1493 | 1529 | ops = dpll_pin_ops(ref); |
| 1530 | + if (!ops->phase_adjust_set) | |
| 1531 | + continue; | |
| 1494 | 1532 | dpll = ref->dpll; |
| 1495 | 1533 | ret = ops->phase_adjust_set(pin, |
| 1496 | 1534 | dpll_pin_on_dpll_priv(dpll, pin), |
| @@ -1513,6 +1551,8 @@ rollback: · 16 unmodified lines | ||
| 1513 | 1551 | if (ref == failed) |
| 1514 | 1552 | break; |
| 1515 | 1553 | ops = dpll_pin_ops(ref); |
| 1554 | + if (!ops->phase_adjust_set) | |
| 1555 | + continue; | |
| 1516 | 1556 | dpll = ref->dpll; |
| 1517 | 1557 | if (ops->phase_adjust_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1518 | 1558 | dpll, dpll_priv(dpll), old_phase_adj, |
| @@ -597,20 +597,20 @@ static int c_can_chip_config(struct net_device *dev) · 596 unmodified lines | ||
| 597 | 597 | return err; |
| 598 | 598 | |
| 599 | 599 | /* enable automatic retransmission */ |
| 600 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR); | |
| 600 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR | CONTROL_INIT); | |
| 601 | 601 | |
| 602 | 602 | if ((priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) && |
| 603 | 603 | (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)) { |
| 604 | 604 | /* loopback + silent mode : useful for hot self-test */ |
| 605 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | |
| 605 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); | |
| 606 | 606 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK | TEST_SILENT); |
| 607 | 607 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK) { |
| 608 | 608 | /* loopback mode : useful for self-test function */ |
| 609 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | |
| 609 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); | |
| 610 | 610 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK); |
| 611 | 611 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) { |
| 612 | 612 | /* silent mode : bus-monitoring mode */ |
| 613 | − priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | |
| 613 | + priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); | |
| 614 | 614 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_SILENT); |
| 615 | 615 | } |
| 616 | 616 | |
| @@ -340,8 +340,8 @@ static void ctucan_set_mode(struct ctucan_priv *priv, const struct can_ctrlmode · 339 unmodified lines | ||
| 340 | 340 | (mode_reg & ~REG_MODE_FDE); |
| 341 | 341 | |
| 342 | 342 | mode_reg = (mode->flags & CAN_CTRLMODE_PRESUME_ACK) ? |
| 343 | − (mode_reg | REG_MODE_ACF) : | |
| 344 | − (mode_reg & ~REG_MODE_ACF); | |
| 343 | + (mode_reg | REG_MODE_STM) : | |
| 344 | + (mode_reg & ~REG_MODE_STM); | |
| 345 | 345 | |
| 346 | 346 | mode_reg = (mode->flags & CAN_CTRLMODE_FD_NON_ISO) ? |
| 347 | 347 | (mode_reg | REG_MODE_NISOFD) : |
| @@ -869,7 +869,7 @@ static void ctucan_err_interrupt(struct net_device *ndev, u32 isr) · 521 unmodified lines | ||
| 869 | 869 | break; |
| 870 | 870 | case CAN_STATE_ERROR_ACTIVE: |
| 871 | 871 | if (skb) { |
| 872 | − cf->can_id |= CAN_ERR_CNT; | |
| 872 | + cf->can_id |= CAN_ERR_CRTL | CAN_ERR_CNT; | |
| 873 | 873 | cf->data[1] = CAN_ERR_CRTL_ACTIVE; |
| 874 | 874 | cf->data[6] = bec.txerr; |
| 875 | 875 | cf->data[7] = bec.rxerr; |
| @@ -1136,8 +1136,12 @@ static irqreturn_t ctucan_interrupt(int irq, void *dev_id) · 260 unmodified lines | ||
| 1136 | 1136 | /* Error interrupts */ |
| 1137 | 1137 | if (FIELD_GET(REG_INT_STAT_EWLI, isr) || |
| 1138 | 1138 | FIELD_GET(REG_INT_STAT_FCSI, isr) || |
| 1139 | − FIELD_GET(REG_INT_STAT_ALI, isr)) { | |
| 1140 | − icr = isr & (REG_INT_STAT_EWLI | REG_INT_STAT_FCSI | REG_INT_STAT_ALI); | |
| 1139 | + FIELD_GET(REG_INT_STAT_ALI, isr) || | |
| 1140 | + FIELD_GET(REG_INT_STAT_BEI, isr)) { | |
| 1141 | + icr = isr & (REG_INT_STAT_EWLI | | |
| 1142 | + REG_INT_STAT_FCSI | | |
| 1143 | + REG_INT_STAT_ALI | | |
| 1144 | + REG_INT_STAT_BEI); | |
| 1141 | 1145 | |
| 1142 | 1146 | ctucan_netdev_dbg(ndev, "some ERR interrupt: clearing 0x%08x\n", icr); |
| 1143 | 1147 | ctucan_write32(priv, CTUCANFD_INT_STAT, icr); |
| @@ -194,7 +194,7 @@ err_free_board: · 193 unmodified lines | ||
| 194 | 194 | pci_set_drvdata(pdev, NULL); |
| 195 | 195 | kfree(bdata); |
| 196 | 196 | err_pci_iounmap_bar0: |
| 197 | − pci_iounmap(pdev, cra_addr); | |
| 197 | + pci_iounmap(pdev, bar0_base); | |
| 198 | 198 | err_pci_iounmap_bar1: |
| 199 | 199 | pci_iounmap(pdev, addr); |
| 200 | 200 | err_release_regions: |
| @@ -266,6 +266,7 @@ static const struct pci_device_id ctucan_pci_tbl[] = { · 65 unmodified lines | ||
| 266 | 266 | CTUCAN_WITH_CTUCAN_ID)}, |
| 267 | 267 | {}, |
| 268 | 268 | }; |
| 269 | +MODULE_DEVICE_TABLE(pci, ctucan_pci_tbl); | |
| 269 | 270 | |
| 270 | 271 | static struct pci_driver ctucan_pci_driver = { |
| 271 | 272 | .name = KBUILD_MODNAME, |
| @@ -2003,20 +2003,12 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 2002 unmodified lines | ||
| 2003 | 2003 | u32 ch, sts; |
| 2004 | 2004 | int err; |
| 2005 | 2005 | |
| 2006 | − err = reset_control_reset(gpriv->rstc1); | |
| 2007 | − if (err) | |
| 2008 | − return err; | |
| 2009 | − | |
| 2010 | − err = reset_control_reset(gpriv->rstc2); | |
| 2011 | − if (err) | |
| 2012 | − goto fail_reset1; | |
| 2013 | − | |
| 2014 | 2006 | /* Enable peripheral clock for register access */ |
| 2015 | 2007 | err = clk_prepare_enable(gpriv->clkp); |
| 2016 | 2008 | if (err) { |
| 2017 | 2009 | dev_err(dev, "failed to enable peripheral clock: %pe\n", |
| 2018 | 2010 | ERR_PTR(err)); |
| 2019 | − goto fail_reset2; | |
| 2011 | + return err; | |
| 2020 | 2012 | } |
| 2021 | 2013 | |
| 2022 | 2014 | /* Enable RAM clock */ |
| @@ -2027,10 +2019,18 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 4 unmodified lines | ||
| 2027 | 2019 | goto fail_clk; |
| 2028 | 2020 | } |
| 2029 | 2021 | |
| 2022 | + err = reset_control_reset(gpriv->rstc1); | |
| 2023 | + if (err) | |
| 2024 | + goto fail_ram_clk; | |
| 2025 | + | |
| 2026 | + err = reset_control_reset(gpriv->rstc2); | |
| 2027 | + if (err) | |
| 2028 | + goto fail_reset1; | |
| 2029 | + | |
| 2030 | 2030 | err = rcar_canfd_reset_controller(gpriv); |
| 2031 | 2031 | if (err) { |
| 2032 | 2032 | dev_err(dev, "reset controller failed: %pe\n", ERR_PTR(err)); |
| 2033 | − goto fail_ram_clk; | |
| 2033 | + goto fail_reset2; | |
| 2034 | 2034 | } |
| 2035 | 2035 | |
| 2036 | 2036 | /* Controller in Global reset & Channel reset mode */ |
| @@ -2068,14 +2068,14 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 31 unmodified lines | ||
| 2068 | 2068 | |
| 2069 | 2069 | fail_mode: |
| 2070 | 2070 | rcar_canfd_disable_global_interrupts(gpriv); |
| 2071 | −fail_ram_clk: | |
| 2072 | − clk_disable_unprepare(gpriv->clk_ram); | |
| 2073 | −fail_clk: | |
| 2074 | − clk_disable_unprepare(gpriv->clkp); | |
| 2075 | 2071 | fail_reset2: |
| 2076 | 2072 | reset_control_assert(gpriv->rstc2); |
| 2077 | 2073 | fail_reset1: |
| 2078 | 2074 | reset_control_assert(gpriv->rstc1); |
| 2075 | +fail_ram_clk: | |
| 2076 | + clk_disable_unprepare(gpriv->clk_ram); | |
| 2077 | +fail_clk: | |
| 2078 | + clk_disable_unprepare(gpriv->clkp); | |
| 2079 | 2079 | return err; |
| 2080 | 2080 | } |
| 2081 | 2081 | |
| @@ -2090,10 +2090,10 @@ static void rcar_canfd_global_deinit(struct rcar_canfd_global *gpriv, bool full) · 8 unmodified lines | ||
| 2090 | 2090 | rcar_canfd_set_bit(gpriv->base, RCANFD_GCTR, RCANFD_GCTR_GSLPR); |
| 2091 | 2091 | } |
| 2092 | 2092 | |
| 2093 | − clk_disable_unprepare(gpriv->clk_ram); | |
| 2094 | − clk_disable_unprepare(gpriv->clkp); | |
| 2095 | 2093 | reset_control_assert(gpriv->rstc2); |
| 2096 | 2094 | reset_control_assert(gpriv->rstc1); |
| 2095 | + clk_disable_unprepare(gpriv->clk_ram); | |
| 2096 | + clk_disable_unprepare(gpriv->clkp); | |
| 2097 | 2097 | } |
| 2098 | 2098 | |
| 2099 | 2099 | static int rcar_canfd_probe(struct platform_device *pdev) |
| @@ -91,12 +91,12 @@ int softing_bootloader_command(struct softing *card, int16_t cmd, · 90 unmodified lines | ||
| 91 | 91 | return ret; |
| 92 | 92 | } |
| 93 | 93 | |
| 94 | −static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, | |
| 95 | − uint16_t *plen, const uint8_t **pdat) | |
| 94 | +static int fw_parse(const u8 **pmem, const u8 *limit, u16 *ptype, | |
| 95 | + u32 *paddr, u16 *plen, const u8 **pdat) | |
| 96 | 96 | { |
| 97 | 97 | uint16_t checksum[2]; |
| 98 | − const uint8_t *mem; | |
| 99 | − const uint8_t *end; | |
| 98 | + const u8 *mem; | |
| 99 | + const u8 *record_end; | |
| 100 | 100 | |
| 101 | 101 | /* |
| 102 | 102 | * firmware records are a binary, unaligned stream composed of: |
| @@ -114,14 +114,21 @@ static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, · 11 unmodified lines | ||
| 114 | 114 | * endianness & alignment. |
| 115 | 115 | */ |
| 116 | 116 | mem = *pmem; |
| 117 | + /* A record needs an 8-byte prefix and a 2-byte checksum. */ | |
| 118 | + if (mem > limit || limit - mem < 10) | |
| 119 | + return -EINVAL; | |
| 120 | + | |
| 117 | 121 | *ptype = le16_to_cpup((void *)&mem[0]); |
| 118 | 122 | *paddr = le32_to_cpup((void *)&mem[2]); |
| 119 | 123 | *plen = le16_to_cpup((void *)&mem[6]); |
| 124 | + if (*plen > limit - mem - 10) | |
| 125 | + return -EINVAL; | |
| 126 | + | |
| 120 | 127 | *pdat = &mem[8]; |
| 121 | 128 | /* verify checksum */ |
| 122 | − end = &mem[8 + *plen]; | |
| 123 | − checksum[0] = le16_to_cpup((void *)end); | |
| 124 | − for (checksum[1] = 0; mem < end; ++mem) | |
| 129 | + record_end = &mem[8 + *plen]; | |
| 130 | + checksum[0] = le16_to_cpup((void *)record_end); | |
| 131 | + for (checksum[1] = 0; mem < record_end; ++mem) | |
| 125 | 132 | checksum[1] += *mem; |
| 126 | 133 | if (checksum[0] != checksum[1]) |
| 127 | 134 | return -EINVAL; |
| @@ -139,6 +146,7 @@ int softing_load_fw(const char *file, struct softing *card, · 11 unmodified lines | ||
| 139 | 146 | uint16_t type, len; |
| 140 | 147 | uint32_t addr; |
| 141 | 148 | uint8_t *buf = NULL, *new_buf; |
| 149 | + s64 dpram_offset; | |
| 142 | 150 | int buflen = 0; |
| 143 | 151 | int8_t type_end = 0; |
| 144 | 152 | |
| @@ -153,7 +161,7 @@ int softing_load_fw(const char *file, struct softing *card, · 8 unmodified lines | ||
| 153 | 161 | mem = fw->data; |
| 154 | 162 | end = &mem[fw->size]; |
| 155 | 163 | /* look for header record */ |
| 156 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 164 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 157 | 165 | if (ret < 0) |
| 158 | 166 | goto failed; |
| 159 | 167 | if (type != 0xffff) |
| @@ -164,7 +172,7 @@ int softing_load_fw(const char *file, struct softing *card, · 4 unmodified lines | ||
| 164 | 172 | } |
| 165 | 173 | /* ok, we had a header */ |
| 166 | 174 | while (mem < end) { |
| 167 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 175 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 168 | 176 | if (ret < 0) |
| 169 | 177 | goto failed; |
| 170 | 178 | if (type == 3) { |
| @@ -179,9 +187,13 @@ int softing_load_fw(const char *file, struct softing *card, · 8 unmodified lines | ||
| 179 | 187 | goto failed; |
| 180 | 188 | } |
| 181 | 189 | |
| 182 | − if ((addr + len + offset) > size) | |
| 190 | + dpram_offset = (s64)addr + offset; | |
| 191 | + if (dpram_offset < 0 || dpram_offset > size || | |
| 192 | + len > size - dpram_offset) { | |
| 193 | + ret = -EINVAL; | |
| 183 | 194 | goto failed; |
| 184 | − memcpy_toio(&dpram[addr + offset], dat, len); | |
| 195 | + } | |
| 196 | + memcpy_toio(&dpram[dpram_offset], dat, len); | |
| 185 | 197 | /* be sure to flush caches from IO space */ |
| 186 | 198 | mb(); |
| 187 | 199 | if (len > buflen) { |
| @@ -195,7 +207,7 @@ int softing_load_fw(const char *file, struct softing *card, · 7 unmodified lines | ||
| 195 | 207 | buf = new_buf; |
| 196 | 208 | } |
| 197 | 209 | /* verify record data */ |
| 198 | − memcpy_fromio(buf, &dpram[addr + offset], len); | |
| 210 | + memcpy_fromio(buf, &dpram[dpram_offset], len); | |
| 199 | 211 | if (memcmp(buf, dat, len)) { |
| 200 | 212 | /* is not ok */ |
| 201 | 213 | dev_alert(&card->pdev->dev, "DPRAM readback failed\n"); |
| @@ -237,7 +249,7 @@ int softing_load_app_fw(const char *file, struct softing *card) · 35 unmodified lines | ||
| 237 | 249 | mem = fw->data; |
| 238 | 250 | end = &mem[fw->size]; |
| 239 | 251 | /* look for header record */ |
| 240 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 252 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 241 | 253 | if (ret) |
| 242 | 254 | goto failed; |
| 243 | 255 | ret = -EINVAL; |
| @@ -253,7 +265,7 @@ int softing_load_app_fw(const char *file, struct softing *card) · 9 unmodified lines | ||
| 253 | 265 | } |
| 254 | 266 | /* ok, we had a header */ |
| 255 | 267 | while (mem < end) { |
| 256 | − ret = fw_parse(&mem, &type, &addr, &len, &dat); | |
| 268 | + ret = fw_parse(&mem, end, &type, &addr, &len, &dat); | |
| 257 | 269 | if (ret) |
| 258 | 270 | goto failed; |
| 259 | 271 | |
| @@ -279,6 +291,12 @@ int softing_load_app_fw(const char *file, struct softing *card) · 19 unmodified lines | ||
| 279 | 291 | /* work in 16bit (target) */ |
| 280 | 292 | sum &= 0xffff; |
| 281 | 293 | |
| 294 | + if (card->pdat->app.offs > card->dpram_size || | |
| 295 | + len > card->dpram_size - card->pdat->app.offs) { | |
| 296 | + ret = -EINVAL; | |
| 297 | + goto failed; | |
| 298 | + } | |
| 299 | + | |
| 282 | 300 | memcpy_toio(&card->dpram[card->pdat->app.offs], dat, len); |
| 283 | 301 | iowrite32(card->pdat->app.offs + card->pdat->app.addr, |
| 284 | 302 | &card->dpram[DPRAM_COMMAND + 2]); |
| @@ -409,6 +409,40 @@ static void ems_usb_rx_err(struct ems_usb *dev, struct ems_cpc_msg *msg) · 408 unmodified lines | ||
| 409 | 409 | netif_rx(skb); |
| 410 | 410 | } |
| 411 | 411 | |
| 412 | +static bool ems_usb_rx_msg_len_valid(struct ems_cpc_msg *msg) | |
| 413 | +{ | |
| 414 | + size_t len = msg->length; | |
| 415 | + size_t can_len; | |
| 416 | + | |
| 417 | + switch (msg->type) { | |
| 418 | + case CPC_MSG_TYPE_CAN_STATE: | |
| 419 | + return len >= sizeof(msg->msg.can_state); | |
| 420 | + | |
| 421 | + case CPC_MSG_TYPE_CAN_FRAME: | |
| 422 | + case CPC_MSG_TYPE_EXT_CAN_FRAME: | |
| 423 | + case CPC_MSG_TYPE_RTR_FRAME: | |
| 424 | + case CPC_MSG_TYPE_EXT_RTR_FRAME: | |
| 425 | + if (len < CPC_CAN_MSG_MIN_SIZE) | |
| 426 | + return false; | |
| 427 | + | |
| 428 | + if (msg->type == CPC_MSG_TYPE_RTR_FRAME || | |
| 429 | + msg->type == CPC_MSG_TYPE_EXT_RTR_FRAME) | |
| 430 | + return true; | |
| 431 | + | |
| 432 | + can_len = can_cc_dlc2len(msg->msg.can_msg.length & 0xf); | |
| 433 | + return len >= CPC_CAN_MSG_MIN_SIZE + can_len; | |
| 434 | + | |
| 435 | + case CPC_MSG_TYPE_CAN_FRAME_ERROR: | |
| 436 | + return len >= sizeof(msg->msg.error); | |
| 437 | + | |
| 438 | + case CPC_MSG_TYPE_OVERRUN: | |
| 439 | + return len >= sizeof(msg->msg.overrun); | |
| 440 | + | |
| 441 | + default: | |
| 442 | + return true; | |
| 443 | + } | |
| 444 | +} | |
| 445 | + | |
| 412 | 446 | /* |
| 413 | 447 | * callback for bulk IN urb |
| 414 | 448 | */ |
| @@ -451,6 +485,15 @@ static void ems_usb_read_bulk_callback(struct urb *urb) · 36 unmodified lines | ||
| 451 | 485 | } |
| 452 | 486 | |
| 453 | 487 | msg = (struct ems_cpc_msg *)&ibuf[start]; |
| 488 | + if (msg->length > | |
| 489 | + urb->actual_length - start - CPC_MSG_HEADER_LEN) { | |
| 490 | + netdev_err(netdev, "format error\n"); | |
| 491 | + break; | |
| 492 | + } | |
| 493 | + if (!ems_usb_rx_msg_len_valid(msg)) { | |
| 494 | + netdev_err(netdev, "format error\n"); | |
| 495 | + break; | |
| 496 | + } | |
| 454 | 497 | |
| 455 | 498 | switch (msg->type) { |
| 456 | 499 | case CPC_MSG_TYPE_CAN_STATE: |
| @@ -1476,7 +1476,6 @@ static void es58x_read_bulk_callback(struct urb *urb) · 1475 unmodified lines | ||
| 1476 | 1476 | dev_err_ratelimited(dev, |
| 1477 | 1477 | "Failed resubmitting read bulk urb: %pe\n", |
| 1478 | 1478 | ERR_PTR(ret)); |
| 1479 | − return; | |
| 1480 | 1479 | |
| 1481 | 1480 | free_urb: |
| 1482 | 1481 | usb_free_coherent(urb->dev, urb->transfer_buffer_length, |
| @@ -674,7 +674,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) · 673 unmodified lines | ||
| 674 | 674 | if (hf->flags & GS_CAN_FLAG_FD) { |
| 675 | 675 | skb = alloc_canfd_skb(netdev, &cfd); |
| 676 | 676 | if (!skb) |
| 677 | − return; | |
| 677 | + goto resubmit_urb; | |
| 678 | 678 | |
| 679 | 679 | cfd->can_id = le32_to_cpu(hf->can_id); |
| 680 | 680 | cfd->len = data_length; |
| @@ -687,7 +687,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) · 6 unmodified lines | ||
| 687 | 687 | } else { |
| 688 | 688 | skb = alloc_can_skb(netdev, &cf); |
| 689 | 689 | if (!skb) |
| 690 | − return; | |
| 690 | + goto resubmit_urb; | |
| 691 | 691 | |
| 692 | 692 | cf->can_id = le32_to_cpu(hf->can_id); |
| 693 | 693 | can_frame_set_cc_len(cf, hf->can_dlc, dev->can.ctrlmode); |
| @@ -1626,6 +1626,7 @@ static int kvaser_usb_hydra_get_busparams(struct kvaser_usb_net_priv *priv, · 1625 unmodified lines | ||
| 1626 | 1626 | reinit_completion(&priv->get_busparams_comp); |
| 1627 | 1627 | |
| 1628 | 1628 | err = kvaser_usb_send_cmd(dev, cmd, cmd_len); |
| 1629 | + kfree(cmd); | |
| 1629 | 1630 | if (err) |
| 1630 | 1631 | return err; |
| 1631 | 1632 | |
| @@ -691,13 +691,22 @@ static int kvaser_usb_leaf_wait_cmd(const struct kvaser_usb *dev, u8 id, · 690 unmodified lines | ||
| 691 | 691 | continue; |
| 692 | 692 | } |
| 693 | 693 | |
| 694 | − if (pos + tmp->len > actual_len) { | |
| 694 | + if (tmp->len < CMD_HEADER_LEN || | |
| 695 | + tmp->len > actual_len - pos) { | |
| 695 | 696 | dev_err_ratelimited(&dev->intf->dev, |
| 696 | 697 | "Format error\n"); |
| 697 | 698 | break; |
| 698 | 699 | } |
| 699 | 700 | |
| 700 | 701 | if (tmp->id == id) { |
| 702 | + if (tmp->len > sizeof(*cmd)) { | |
| 703 | + dev_err_ratelimited(&dev->intf->dev, | |
| 704 | + "Received command %u too large (%u)\n", | |
| 705 | + tmp->id, tmp->len); | |
| 706 | + err = -EIO; | |
| 707 | + goto end; | |
| 708 | + } | |
| 709 | + | |
| 701 | 710 | memcpy(cmd, tmp, tmp->len); |
| 702 | 711 | goto end; |
| 703 | 712 | } |
| @@ -1737,7 +1746,7 @@ static void kvaser_usb_leaf_read_bulk_callback(struct kvaser_usb *dev, · 1033 unmodified lines | ||
| 1737 | 1746 | continue; |
| 1738 | 1747 | } |
| 1739 | 1748 | |
| 1740 | − if (pos + cmd->len > len) { | |
| 1749 | + if (cmd->len < CMD_HEADER_LEN || cmd->len > len - pos) { | |
| 1741 | 1750 | dev_err_ratelimited(&dev->intf->dev, "Format error\n"); |
| 1742 | 1751 | break; |
| 1743 | 1752 | } |
| @@ -470,7 +470,6 @@ static int peak_usb_start(struct peak_usb_device *dev) · 469 unmodified lines | ||
| 470 | 470 | netif_device_detach(dev->netdev); |
| 471 | 471 | |
| 472 | 472 | usb_unanchor_urb(urb); |
| 473 | − kfree(buf); | |
| 474 | 473 | usb_free_urb(urb); |
| 475 | 474 | break; |
| 476 | 475 | } |
| @@ -566,6 +566,13 @@ static int pcan_usb_fd_decode_canmsg(struct pcan_usb_fd_if *usb_if, · 565 unmodified lines | ||
| 566 | 566 | dev->can.ctrlmode); |
| 567 | 567 | } |
| 568 | 568 | |
| 569 | + if (!(rx_msg_flags & PUCAN_MSG_RTR) && | |
| 570 | + le16_to_cpu(rx_msg->size) - offsetof(struct pucan_rx_msg, d) < | |
| 571 | + cfd->len) { | |
| 572 | + kfree_skb(skb); | |
| 573 | + return -EBADMSG; | |
| 574 | + } | |
| 575 | + | |
| 569 | 576 | cfd->can_id = le32_to_cpu(rm->can_id); |
| 570 | 577 | |
| 571 | 578 | if (rx_msg_flags & PUCAN_MSG_EXT_ID) |
| @@ -714,6 +721,24 @@ static void pcan_usb_fd_decode_ts(struct pcan_usb_fd_if *usb_if, · 142 unmodified lines | ||
| 714 | 721 | peak_usb_set_ts_now(&usb_if->time_ref, le32_to_cpu(ts->ts_low)); |
| 715 | 722 | } |
| 716 | 723 | |
| 724 | +static size_t pcan_usb_fd_rx_msg_min_size(u16 rx_msg_type) | |
| 725 | +{ | |
| 726 | + switch (rx_msg_type) { | |
| 727 | + case PUCAN_MSG_CAN_RX: | |
| 728 | + return offsetof(struct pucan_rx_msg, d); | |
| 729 | + case PCAN_UFD_MSG_CALIBRATION: | |
| 730 | + return sizeof(struct pcan_ufd_ts_msg); | |
| 731 | + case PUCAN_MSG_ERROR: | |
| 732 | + return sizeof(struct pucan_error_msg); | |
| 733 | + case PUCAN_MSG_STATUS: | |
| 734 | + return sizeof(struct pucan_status_msg); | |
| 735 | + case PCAN_UFD_MSG_OVERRUN: | |
| 736 | + return sizeof(struct pcan_ufd_ovr_msg); | |
| 737 | + default: | |
| 738 | + return sizeof(struct pucan_msg); | |
| 739 | + } | |
| 740 | +} | |
| 741 | + | |
| 717 | 742 | /* callback for bulk IN urb */ |
| 718 | 743 | static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) |
| 719 | 744 | { |
| @@ -728,6 +753,12 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) · 8 unmodified lines | ||
| 728 | 753 | msg_end = urb->transfer_buffer + urb->actual_length; |
| 729 | 754 | for (; msg_ptr < msg_end;) { |
| 730 | 755 | u16 rx_msg_type, rx_msg_size; |
| 756 | + size_t rx_msg_min_size; | |
| 757 | + | |
| 758 | + if (msg_end - msg_ptr < sizeof(*rx_msg)) { | |
| 759 | + err = -EBADMSG; | |
| 760 | + break; | |
| 761 | + } | |
| 731 | 762 | |
| 732 | 763 | rx_msg = (struct pucan_msg *)msg_ptr; |
| 733 | 764 | if (!rx_msg->size) { |
| @@ -739,13 +770,20 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) · 5 unmodified lines | ||
| 739 | 770 | rx_msg_type = le16_to_cpu(rx_msg->type); |
| 740 | 771 | |
| 741 | 772 | /* check if the record goes out of current packet */ |
| 742 | − if (msg_ptr + rx_msg_size > msg_end) { | |
| 773 | + if (rx_msg_size > msg_end - msg_ptr) { | |
| 743 | 774 | netdev_err(netdev, |
| 744 | 775 | "got frag rec: should inc usb rx buf sze\n"); |
| 745 | 776 | err = -EBADMSG; |
| 746 | 777 | break; |
| 747 | 778 | } |
| 748 | 779 | |
| 780 | + rx_msg_min_size = pcan_usb_fd_rx_msg_min_size(rx_msg_type); | |
| 781 | + if (rx_msg_size < rx_msg_min_size) { | |
| 782 | + netdev_err(netdev, "got short rec\n"); | |
| 783 | + err = -EBADMSG; | |
| 784 | + break; | |
| 785 | + } | |
| 786 | + | |
| 749 | 787 | switch (rx_msg_type) { |
| 750 | 788 | case PUCAN_MSG_CAN_RX: |
| 751 | 789 | err = pcan_usb_fd_decode_canmsg(usb_if, rx_msg); |
| @@ -534,12 +534,18 @@ static int pcan_usb_pro_handle_canmsg(struct pcan_usb_pro_interface *usb_if, · 533 unmodified lines | ||
| 534 | 534 | struct pcan_usb_pro_rxmsg *rx) |
| 535 | 535 | { |
| 536 | 536 | const unsigned int ctrl_idx = (rx->len >> 4) & 0x0f; |
| 537 | − struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; | |
| 538 | − struct net_device *netdev = dev->netdev; | |
| 537 | + struct peak_usb_device *dev; | |
| 538 | + struct net_device *netdev; | |
| 539 | 539 | struct can_frame *can_frame; |
| 540 | 540 | struct sk_buff *skb; |
| 541 | 541 | struct skb_shared_hwtstamps *hwts; |
| 542 | 542 | |
| 543 | + if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) | |
| 544 | + return -EINVAL; | |
| 545 | + | |
| 546 | + dev = usb_if->dev[ctrl_idx]; | |
| 547 | + netdev = dev->netdev; | |
| 548 | + | |
| 543 | 549 | skb = alloc_can_skb(netdev, &can_frame); |
| 544 | 550 | if (!skb) |
| 545 | 551 | return -ENOMEM; |
| @@ -573,14 +579,20 @@ static int pcan_usb_pro_handle_error(struct pcan_usb_pro_interface *usb_if, · 27 unmodified lines | ||
| 573 | 579 | { |
| 574 | 580 | const u16 raw_status = le16_to_cpu(er->status); |
| 575 | 581 | const unsigned int ctrl_idx = (er->channel >> 4) & 0x0f; |
| 576 | − struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; | |
| 577 | − struct net_device *netdev = dev->netdev; | |
| 582 | + struct peak_usb_device *dev; | |
| 583 | + struct net_device *netdev; | |
| 578 | 584 | struct can_frame *can_frame; |
| 579 | 585 | enum can_state new_state = CAN_STATE_ERROR_ACTIVE; |
| 580 | 586 | u8 err_mask = 0; |
| 581 | 587 | struct sk_buff *skb; |
| 582 | 588 | struct skb_shared_hwtstamps *hwts; |
| 583 | 589 | |
| 590 | + if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) | |
| 591 | + return -EINVAL; | |
| 592 | + | |
| 593 | + dev = usb_if->dev[ctrl_idx]; | |
| 594 | + netdev = dev->netdev; | |
| 595 | + | |
| 584 | 596 | /* nothing should be sent while in BUS_OFF state */ |
| 585 | 597 | if (dev->can.state == CAN_STATE_BUS_OFF) |
| 586 | 598 | return 0; |
| @@ -55,8 +55,15 @@ mt7530_regmap_read(void *context, unsigned int reg, unsigned int *val) · 54 unmodified lines | ||
| 55 | 55 | if (ret < 0) |
| 56 | 56 | return ret; |
| 57 | 57 | |
| 58 | − lo = bus->read(bus, priv->mdiodev->addr, r); | |
| 59 | − hi = bus->read(bus, priv->mdiodev->addr, 0x10); | |
| 58 | + ret = bus->read(bus, priv->mdiodev->addr, r); | |
| 59 | + if (ret < 0) | |
| 60 | + return ret; | |
| 61 | + lo = ret; | |
| 62 | + | |
| 63 | + ret = bus->read(bus, priv->mdiodev->addr, 0x10); | |
| 64 | + if (ret < 0) | |
| 65 | + return ret; | |
| 66 | + hi = ret; | |
| 60 | 67 | |
| 61 | 68 | *val = (hi << 16) | (lo & 0xffff); |
| 62 | 69 | |
| @@ -191,12 +191,6 @@ mt7530_write(struct mt7530_priv *priv, u32 reg, u32 val) · 190 unmodified lines | ||
| 191 | 191 | mt7530_mutex_unlock(priv); |
| 192 | 192 | } |
| 193 | 193 | |
| 194 | −static u32 | |
| 195 | −_mt7530_unlocked_read(struct mt7530_dummy_poll *p) | |
| 196 | −{ | |
| 197 | − return mt7530_mii_read(p->priv, p->reg); | |
| 198 | −} | |
| 199 | − | |
| 200 | 194 | static u32 |
| 201 | 195 | _mt7530_read(struct mt7530_dummy_poll *p) |
| 202 | 196 | { |
| @@ -248,15 +242,20 @@ mt7530_fdb_cmd(struct mt7530_priv *priv, enum mt7530_fdb_cmd cmd, u32 *rsp) · 45 unmodified lines | ||
| 248 | 242 | { |
| 249 | 243 | u32 val; |
| 250 | 244 | int ret; |
| 251 | − struct mt7530_dummy_poll p; | |
| 252 | 245 | |
| 253 | 246 | /* Set the command operating upon the MAC address entries */ |
| 254 | 247 | val = ATC_BUSY | ATC_MAT(0) | cmd; |
| 255 | 248 | mt7530_write(priv, MT7530_ATC, val); |
| 256 | 249 | |
| 257 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7530_ATC); | |
| 258 | − ret = readx_poll_timeout(_mt7530_read, &p, val, | |
| 259 | − !(val & ATC_BUSY), 20, 20000); | |
| 250 | + mt7530_mutex_lock(priv); | |
| 251 | + | |
| 252 | + ret = regmap_read_poll_timeout(priv->regmap, MT7530_ATC, val, | |
| 253 | + !(val & ATC_BUSY), 20, 20000); | |
| 254 | + if (!ret) | |
| 255 | + ret = regmap_read(priv->regmap, MT7530_ATC, &val); | |
| 256 | + | |
| 257 | + mt7530_mutex_unlock(priv); | |
| 258 | + | |
| 260 | 259 | if (ret < 0) { |
| 261 | 260 | dev_err(priv->dev, "reset timeout\n"); |
| 262 | 261 | return ret; |
| @@ -265,7 +264,6 @@ mt7530_fdb_cmd(struct mt7530_priv *priv, enum mt7530_fdb_cmd cmd, u32 *rsp) · 2 unmodified lines | ||
| 265 | 264 | /* Additional sanity for read command if the specified |
| 266 | 265 | * entry is invalid |
| 267 | 266 | */ |
| 268 | − val = mt7530_read(priv, MT7530_ATC); | |
| 269 | 267 | if ((cmd == MT7530_FDB_READ) && (val & ATC_INVALID)) |
| 270 | 268 | return -EINVAL; |
| 271 | 269 | |
| @@ -549,16 +547,13 @@ static int · 277 unmodified lines | ||
| 549 | 547 | mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, |
| 550 | 548 | int regnum) |
| 551 | 549 | { |
| 552 | − struct mt7530_dummy_poll p; | |
| 553 | 550 | u32 reg, val; |
| 554 | 551 | int ret; |
| 555 | 552 | |
| 556 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 557 | − | |
| 558 | 553 | mt7530_mutex_lock(priv); |
| 559 | 554 | |
| 560 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 561 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 555 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 556 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 562 | 557 | if (ret < 0) { |
| 563 | 558 | dev_err(priv->dev, "poll timeout\n"); |
| 564 | 559 | goto out; |
| @@ -568,8 +563,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 568 | 563 | MT7531_MDIO_DEV_ADDR(devad) | regnum; |
| 569 | 564 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 570 | 565 | |
| 571 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 572 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 566 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 567 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 573 | 568 | if (ret < 0) { |
| 574 | 569 | dev_err(priv->dev, "poll timeout\n"); |
| 575 | 570 | goto out; |
| @@ -579,8 +574,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 579 | 574 | MT7531_MDIO_DEV_ADDR(devad); |
| 580 | 575 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 581 | 576 | |
| 582 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 583 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 577 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 578 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 584 | 579 | if (ret < 0) { |
| 585 | 580 | dev_err(priv->dev, "poll timeout\n"); |
| 586 | 581 | goto out; |
| @@ -597,16 +592,13 @@ static int · 10 unmodified lines | ||
| 597 | 592 | mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, |
| 598 | 593 | int regnum, u16 data) |
| 599 | 594 | { |
| 600 | − struct mt7530_dummy_poll p; | |
| 601 | 595 | u32 val, reg; |
| 602 | 596 | int ret; |
| 603 | 597 | |
| 604 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 605 | − | |
| 606 | 598 | mt7530_mutex_lock(priv); |
| 607 | 599 | |
| 608 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 609 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 600 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 601 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 610 | 602 | if (ret < 0) { |
| 611 | 603 | dev_err(priv->dev, "poll timeout\n"); |
| 612 | 604 | goto out; |
| @@ -616,8 +608,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 616 | 608 | MT7531_MDIO_DEV_ADDR(devad) | regnum; |
| 617 | 609 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 618 | 610 | |
| 619 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 620 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 611 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 612 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 621 | 613 | if (ret < 0) { |
| 622 | 614 | dev_err(priv->dev, "poll timeout\n"); |
| 623 | 615 | goto out; |
| @@ -627,8 +619,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | ||
| 627 | 619 | MT7531_MDIO_DEV_ADDR(devad) | data; |
| 628 | 620 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 629 | 621 | |
| 630 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 631 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 622 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 623 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 632 | 624 | if (ret < 0) { |
| 633 | 625 | dev_err(priv->dev, "poll timeout\n"); |
| 634 | 626 | goto out; |
| @@ -643,16 +635,13 @@ out: · 8 unmodified lines | ||
| 643 | 635 | static int |
| 644 | 636 | mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) |
| 645 | 637 | { |
| 646 | − struct mt7530_dummy_poll p; | |
| 647 | 638 | int ret; |
| 648 | 639 | u32 val; |
| 649 | 640 | |
| 650 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 651 | − | |
| 652 | 641 | mt7530_mutex_lock(priv); |
| 653 | 642 | |
| 654 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 655 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 643 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 644 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 656 | 645 | if (ret < 0) { |
| 657 | 646 | dev_err(priv->dev, "poll timeout\n"); |
| 658 | 647 | goto out; |
| @@ -663,8 +652,8 @@ mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) · 4 unmodified lines | ||
| 663 | 652 | |
| 664 | 653 | mt7530_mii_write(priv, MT7531_PHY_IAC, val | MT7531_PHY_ACS_ST); |
| 665 | 654 | |
| 666 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | |
| 667 | − !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 655 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, | |
| 656 | + !(val & MT7531_PHY_ACS_ST), 20, 100000); | |
| 668 | 657 | if (ret < 0) { |
| 669 | 658 | dev_err(priv->dev, "poll timeout\n"); |
| 670 | 659 | goto out; |
| @@ -681,16 +670,13 @@ static int · 10 unmodified lines | ||
| 681 | 670 | mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, |
| 682 | 671 | u16 data) |
| 683 | 672 | { |
| 684 | − struct mt7530_dummy_poll p; | |
| 685 | 673 | int ret; |
| 686 | 674 | u32 reg; |
| 687 | 675 | |
| 688 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | |
| 689 | − | |
| 690 | 676 | mt7530_mutex_lock(priv); |
| 691 | 677 | |
| 692 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, | |
| 693 | − !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 678 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, | |
| 679 | + !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 694 | 680 | if (ret < 0) { |
| 695 | 681 | dev_err(priv->dev, "poll timeout\n"); |
| 696 | 682 | goto out; |
| @@ -701,8 +687,8 @@ mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, · 4 unmodified lines | ||
| 701 | 687 | |
| 702 | 688 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 703 | 689 | |
| 704 | − ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, | |
| 705 | − !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 690 | + ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, | |
| 691 | + !(reg & MT7531_PHY_ACS_ST), 20, 100000); | |
| 706 | 692 | if (ret < 0) { |
| 707 | 693 | dev_err(priv->dev, "poll timeout\n"); |
| 708 | 694 | goto out; |
| @@ -1626,22 +1612,26 @@ mt7530_port_bridge_join(struct dsa_switch *ds, int port, · 917 unmodified lines | ||
| 1626 | 1612 | static int |
| 1627 | 1613 | mt7530_vlan_cmd(struct mt7530_priv *priv, enum mt7530_vlan_cmd cmd, u16 vid) |
| 1628 | 1614 | { |
| 1629 | − struct mt7530_dummy_poll p; | |
| 1630 | 1615 | u32 val; |
| 1631 | 1616 | int ret; |
| 1632 | 1617 | |
| 1633 | 1618 | val = VTCR_BUSY | VTCR_FUNC(cmd) | vid; |
| 1634 | 1619 | mt7530_write(priv, MT7530_VTCR, val); |
| 1635 | 1620 | |
| 1636 | − INIT_MT7530_DUMMY_POLL(&p, priv, MT7530_VTCR); | |
| 1637 | − ret = readx_poll_timeout(_mt7530_read, &p, val, | |
| 1638 | − !(val & VTCR_BUSY), 20, 20000); | |
| 1621 | + mt7530_mutex_lock(priv); | |
| 1622 | + | |
| 1623 | + ret = regmap_read_poll_timeout(priv->regmap, MT7530_VTCR, val, | |
| 1624 | + !(val & VTCR_BUSY), 20, 20000); | |
| 1625 | + if (!ret) | |
| 1626 | + ret = regmap_read(priv->regmap, MT7530_VTCR, &val); | |
| 1627 | + | |
| 1628 | + mt7530_mutex_unlock(priv); | |
| 1629 | + | |
| 1639 | 1630 | if (ret < 0) { |
| 1640 | 1631 | dev_err(priv->dev, "poll timeout\n"); |
| 1641 | 1632 | return ret; |
| 1642 | 1633 | } |
| 1643 | 1634 | |
| 1644 | − val = mt7530_read(priv, MT7530_VTCR); | |
| 1645 | 1635 | if (val & VTCR_INVALID) { |
| 1646 | 1636 | dev_err(priv->dev, "read VTCR invalid\n"); |
| 1647 | 1637 | return -EINVAL; |
| @@ -1988,16 +1988,19 @@ static void rtl8365mb_get_stats64(struct dsa_switch *ds, int port, · 1987 unmodified lines | ||
| 1988 | 1988 | spin_unlock(&p->stats_lock); |
| 1989 | 1989 | } |
| 1990 | 1990 | |
| 1991 | −static void rtl8365mb_stats_setup(struct realtek_priv *priv) | |
| 1991 | +static int rtl8365mb_stats_setup(struct realtek_priv *priv) | |
| 1992 | 1992 | { |
| 1993 | 1993 | struct rtl8365mb *mb = priv->chip_data; |
| 1994 | 1994 | struct dsa_switch *ds = &priv->ds; |
| 1995 | 1995 | struct dsa_port *dp; |
| 1996 | + int ret; | |
| 1996 | 1997 | |
| 1997 | 1998 | /* Per-chip global mutex to protect MIB counter access, since doing |
| 1998 | 1999 | * so requires accessing a series of registers in a particular order. |
| 1999 | 2000 | */ |
| 2000 | − mutex_init(&mb->mib_lock); | |
| 2001 | + ret = devm_mutex_init(priv->dev, &mb->mib_lock); | |
| 2002 | + if (ret) | |
| 2003 | + return ret; | |
| 2001 | 2004 | |
| 2002 | 2005 | dsa_switch_for_each_available_port(dp, ds) { |
| 2003 | 2006 | struct rtl8365mb_port *p = &mb->ports[dp->index]; |
| @@ -2010,6 +2013,8 @@ static void rtl8365mb_stats_setup(struct realtek_priv *priv) · 6 unmodified lines | ||
| 2010 | 2013 | */ |
| 2011 | 2014 | INIT_DELAYED_WORK(&p->mib_work, rtl8365mb_stats_poll); |
| 2012 | 2015 | } |
| 2016 | + | |
| 2017 | + return 0; | |
| 2013 | 2018 | } |
| 2014 | 2019 | |
| 2015 | 2020 | static void rtl8365mb_stats_teardown(struct realtek_priv *priv) |
| @@ -2567,7 +2572,12 @@ static int rtl8365mb_setup(struct dsa_switch *ds) · 551 unmodified lines | ||
| 2567 | 2572 | } |
| 2568 | 2573 | |
| 2569 | 2574 | /* Start statistics counter polling */ |
| 2570 | − rtl8365mb_stats_setup(priv); | |
| 2575 | + ret = rtl8365mb_stats_setup(priv); | |
| 2576 | + if (ret) { | |
| 2577 | + dev_err(priv->dev, "failed to setup stats: %pe\n", | |
| 2578 | + ERR_PTR(ret)); | |
| 2579 | + goto out_teardown_irq; | |
| 2580 | + } | |
| 2571 | 2581 | |
| 2572 | 2582 | return 0; |
| 2573 | 2583 | |
| @@ -156,9 +156,17 @@ rtl83xx_probe(struct device *dev, · 155 unmodified lines | ||
| 156 | 156 | if (!priv) |
| 157 | 157 | return ERR_PTR(-ENOMEM); |
| 158 | 158 | |
| 159 | − mutex_init(&priv->map_lock); | |
| 160 | − mutex_init(&priv->vlan_lock); | |
| 161 | − mutex_init(&priv->l2_lock); | |
| 159 | + ret = devm_mutex_init(dev, &priv->map_lock); | |
| 160 | + if (ret) | |
| 161 | + return ERR_PTR(ret); | |
| 162 | + | |
| 163 | + ret = devm_mutex_init(dev, &priv->vlan_lock); | |
| 164 | + if (ret) | |
| 165 | + return ERR_PTR(ret); | |
| 166 | + | |
| 167 | + ret = devm_mutex_init(dev, &priv->l2_lock); | |
| 168 | + if (ret) | |
| 169 | + return ERR_PTR(ret); | |
| 162 | 170 | |
| 163 | 171 | rc.lock_arg = priv; |
| 164 | 172 | priv->map = devm_regmap_init(dev, NULL, priv, &rc); |
| @@ -1222,11 +1222,11 @@ err_eeprom: · 1221 unmodified lines | ||
| 1222 | 1222 | |
| 1223 | 1223 | if (hw->flash_address) |
| 1224 | 1224 | iounmap(hw->flash_address); |
| 1225 | +err_mdio_ioremap: | |
| 1225 | 1226 | kfree(adapter->tx_ring); |
| 1226 | 1227 | kfree(adapter->rx_ring); |
| 1227 | 1228 | err_dma: |
| 1228 | 1229 | err_sw_init: |
| 1229 | −err_mdio_ioremap: | |
| 1230 | 1230 | iounmap(hw->ce4100_gbe_mdio_base_virt); |
| 1231 | 1231 | iounmap(hw->hw_addr); |
| 1232 | 1232 | err_ioremap: |
| @@ -793,7 +793,7 @@ err: · 792 unmodified lines | ||
| 793 | 793 | ret, |
| 794 | 794 | libie_aq_str(pf->hw.adminq.sq_last_status), |
| 795 | 795 | pin_type_name[pin_type], pin->idx); |
| 796 | − else | |
| 796 | + else if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) | |
| 797 | 797 | dev_err_ratelimited(ice_pf_to_dev(pf), |
| 798 | 798 | "err:%d %s failed to update %s pin:%u\n", |
| 799 | 799 | ret, |
| @@ -3024,7 +3024,8 @@ static int ice_dpll_pps_update_phase_offsets(struct ice_pf *pf, · 2224 unmodified lines | ||
| 3024 | 3024 | *phase_offset_pins_updated = 0; |
| 3025 | 3025 | ret = ice_aq_get_cgu_input_pin_measure(&pf->hw, DPLL_TYPE_PPS, meas, |
| 3026 | 3026 | ARRAY_SIZE(meas)); |
| 3027 | − if (ret && pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EAGAIN) { | |
| 3027 | + if (ret && (pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EAGAIN || | |
| 3028 | + pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EBUSY)) { | |
| 3028 | 3029 | return 0; |
| 3029 | 3030 | } else if (ret) { |
| 3030 | 3031 | dev_err(ice_pf_to_dev(pf), |
| @@ -3086,10 +3087,12 @@ ice_dpll_update_state(struct ice_pf *pf, struct ice_dpll *d, bool init) · 55 unmodified lines | ||
| 3086 | 3087 | d->dpll_idx, d->prev_input_idx, d->input_idx, |
| 3087 | 3088 | d->dpll_state, d->prev_dpll_state, d->mode); |
| 3088 | 3089 | if (ret) { |
| 3089 | − dev_err(ice_pf_to_dev(pf), | |
| 3090 | − "update dpll=%d state failed, ret=%d %s\n", | |
| 3091 | − d->dpll_idx, ret, | |
| 3092 | − libie_aq_str(pf->hw.adminq.sq_last_status)); | |
| 3090 | + /* EBUSY is expected during reset recovery, don't log error */ | |
| 3091 | + if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) | |
| 3092 | + dev_err(ice_pf_to_dev(pf), | |
| 3093 | + "update dpll=%d state failed, ret=%d %s\n", | |
| 3094 | + d->dpll_idx, ret, | |
| 3095 | + libie_aq_str(pf->hw.adminq.sq_last_status)); | |
| 3093 | 3096 | return ret; |
| 3094 | 3097 | } |
| 3095 | 3098 | if (init) { |
| @@ -3158,7 +3161,9 @@ static void ice_dpll_periodic_work(struct kthread_work *work) · 62 unmodified lines | ||
| 3158 | 3161 | d->periodic_counter % dp->phase_offset_monitor_period == 0) |
| 3159 | 3162 | ret = ice_dpll_pps_update_phase_offsets(pf, &phase_offset_ntf); |
| 3160 | 3163 | if (ret) { |
| 3161 | − d->cgu_state_acq_err_num++; | |
| 3164 | + /* EBUSY is expected during reset recovery */ | |
| 3165 | + if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) | |
| 3166 | + d->cgu_state_acq_err_num++; | |
| 3162 | 3167 | /* stop rescheduling this worker */ |
| 3163 | 3168 | if (d->cgu_state_acq_err_num > |
| 3164 | 3169 | ICE_CGU_STATE_ACQ_ERR_THRESHOLD) { |
| @@ -1069,18 +1069,18 @@ static int ice_lbtest_prepare_rings(struct ice_vsi *vsi) · 1068 unmodified lines | ||
| 1069 | 1069 | |
| 1070 | 1070 | status = ice_vsi_cfg_lan(vsi); |
| 1071 | 1071 | if (status) |
| 1072 | − goto err_setup_rx_ring; | |
| 1072 | + goto err_cfg_lan; | |
| 1073 | 1073 | |
| 1074 | 1074 | status = ice_vsi_start_all_rx_rings(vsi); |
| 1075 | 1075 | if (status) |
| 1076 | − goto err_start_rx_ring; | |
| 1076 | + goto err_cfg_lan; | |
| 1077 | 1077 | |
| 1078 | 1078 | return 0; |
| 1079 | 1079 | |
| 1080 | −err_start_rx_ring: | |
| 1081 | − ice_vsi_free_rx_rings(vsi); | |
| 1082 | −err_setup_rx_ring: | |
| 1080 | +err_cfg_lan: | |
| 1083 | 1081 | ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, 0); |
| 1082 | +err_setup_rx_ring: | |
| 1083 | + ice_vsi_free_rx_rings(vsi); | |
| 1084 | 1084 | err_setup_tx_ring: |
| 1085 | 1085 | ice_vsi_free_tx_rings(vsi); |
| 1086 | 1086 | |
| @@ -5637,6 +5637,16 @@ static int ice_resume(struct device *dev) · 5636 unmodified lines | ||
| 5637 | 5637 | /* Restart the service task */ |
| 5638 | 5638 | mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period)); |
| 5639 | 5639 | |
| 5640 | + /* Best-effort wait for the scheduled reset to finish so that the | |
| 5641 | + * device is operational before returning. Without this, userspace | |
| 5642 | + * (e.g. NetworkManager) may try to open the net device while the | |
| 5643 | + * asynchronous reset is still in progress, hitting -EBUSY. | |
| 5644 | + */ | |
| 5645 | + ret = ice_wait_for_reset(pf, secs_to_jiffies(10)); | |
| 5646 | + if (ret) | |
| 5647 | + dev_err(dev, "Wait for reset timed out (10s) during resume: %d\n", | |
| 5648 | + ret); | |
| 5649 | + | |
| 5640 | 5650 | return 0; |
| 5641 | 5651 | } |
| 5642 | 5652 | |
| @@ -848,6 +848,30 @@ static void ice_notify_vf_reset(struct ice_vf *vf) · 847 unmodified lines | ||
| 848 | 848 | NULL); |
| 849 | 849 | } |
| 850 | 850 | |
| 851 | +/** | |
| 852 | + * ice_reset_interrupts - clear all queue interrupt configuration for a VSI | |
| 853 | + * @vsi: the VSI whose interrupt registers should be cleared | |
| 854 | + * | |
| 855 | + * Zero the QINT_RQCTL and QINT_TQCTL registers for all allocated queues | |
| 856 | + * in the VSI. This clears the entire register including MSIX_INDX, ITR_INDX, | |
| 857 | + * CAUSE_ENA and NEXTQ fields, unlike ice_vf_dis_rxq_interrupt() which only | |
| 858 | + * clears the CAUSE_ENA bit. | |
| 859 | + */ | |
| 860 | +void ice_reset_interrupts(struct ice_vsi *vsi) | |
| 861 | +{ | |
| 862 | + struct ice_pf *pf = vsi->back; | |
| 863 | + struct ice_hw *hw = &pf->hw; | |
| 864 | + int i; | |
| 865 | + | |
| 866 | + ice_for_each_alloc_rxq(vsi, i) | |
| 867 | + wr32(hw, QINT_RQCTL(vsi->rxq_map[i]), 0); | |
| 868 | + | |
| 869 | + ice_for_each_alloc_txq(vsi, i) | |
| 870 | + wr32(hw, QINT_TQCTL(vsi->txq_map[i]), 0); | |
| 871 | + | |
| 872 | + ice_flush(hw); | |
| 873 | +} | |
| 874 | + | |
| 851 | 875 | /** |
| 852 | 876 | * ice_reset_vf - Reset a particular VF |
| 853 | 877 | * @vf: pointer to the VF structure |
| @@ -919,6 +943,9 @@ int ice_reset_vf(struct ice_vf *vf, u32 flags) · 65 unmodified lines | ||
| 919 | 943 | |
| 920 | 944 | ice_dis_vf_qs(vf); |
| 921 | 945 | |
| 946 | + /* cleanup interrupt registers */ | |
| 947 | + ice_reset_interrupts(vsi); | |
| 948 | + | |
| 922 | 949 | /* Call Disable LAN Tx queue AQ whether or not queues are |
| 923 | 950 | * enabled. This is needed for successful completion of VFR. |
| 924 | 951 | */ |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | void ice_initialize_vf_entry(struct ice_vf *vf); |
| 27 | 27 | void ice_deinitialize_vf_entry(struct ice_vf *vf); |
| 28 | 28 | void ice_dis_vf_qs(struct ice_vf *vf); |
| 29 | +void ice_reset_interrupts(struct ice_vsi *vsi); | |
| 29 | 30 | int ice_check_vf_init(struct ice_vf *vf); |
| 30 | 31 | enum virtchnl_status_code ice_err_to_virt_err(int err); |
| 31 | 32 | struct ice_port_info *ice_vf_get_port_info(struct ice_vf *vf); |
| @@ -224,6 +224,24 @@ void ice_vf_ena_rxq_interrupt(struct ice_vsi *vsi, u32 q_idx) · 223 unmodified lines | ||
| 224 | 224 | wr32(hw, QINT_RQCTL(pfq), reg | QINT_RQCTL_CAUSE_ENA_M); |
| 225 | 225 | } |
| 226 | 226 | |
| 227 | +/** | |
| 228 | + * ice_vf_dis_rxq_interrupt - disable Rx queue interrupt via QINT_RQCTL | |
| 229 | + * @vsi: VSI of the VF to configure | |
| 230 | + * @q_idx: VF queue index used to determine the queue in the PF's space | |
| 231 | + */ | |
| 232 | +static void ice_vf_dis_rxq_interrupt(struct ice_vsi *vsi, u32 q_idx) | |
| 233 | +{ | |
| 234 | + struct ice_hw *hw = &vsi->back->hw; | |
| 235 | + u32 pfq = vsi->rxq_map[q_idx]; | |
| 236 | + u32 reg; | |
| 237 | + | |
| 238 | + reg = rd32(hw, QINT_RQCTL(pfq)); | |
| 239 | + reg &= ~QINT_RQCTL_CAUSE_ENA_M; | |
| 240 | + wr32(hw, QINT_RQCTL(pfq), reg); | |
| 241 | + | |
| 242 | + ice_flush(hw); | |
| 243 | +} | |
| 244 | + | |
| 227 | 245 | /** |
| 228 | 246 | * ice_vc_ena_qs_msg |
| 229 | 247 | * @vf: pointer to the VF info |
| @@ -416,6 +434,8 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg) · 186 unmodified lines | ||
| 416 | 434 | goto error_param; |
| 417 | 435 | } |
| 418 | 436 | |
| 437 | + for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) | |
| 438 | + ice_vf_dis_rxq_interrupt(vsi, vf_q_id); | |
| 419 | 439 | bitmap_zero(vf->rxq_ena, ICE_MAX_RSS_QS_PER_VF); |
| 420 | 440 | } else if (q_map) { |
| 421 | 441 | for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) { |
| @@ -436,6 +456,7 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg) · 14 unmodified lines | ||
| 436 | 456 | goto error_param; |
| 437 | 457 | } |
| 438 | 458 | |
| 459 | + ice_vf_dis_rxq_interrupt(vsi, vf_q_id); | |
| 439 | 460 | /* Clear enabled queues flag */ |
| 440 | 461 | clear_bit(vf_q_id, vf->rxq_ena); |
| 441 | 462 | } |
| @@ -87,7 +87,7 @@ static int idpf_intr_reg_init(struct idpf_vport *vport, · 86 unmodified lines | ||
| 87 | 87 | if (!reg_vals) |
| 88 | 88 | return -ENOMEM; |
| 89 | 89 | |
| 90 | − num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals); | |
| 90 | + num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals, total_vecs); | |
| 91 | 91 | if (num_regs < num_vecs) { |
| 92 | 92 | err = -EINVAL; |
| 93 | 93 | goto free_reg_vals; |
| @@ -139,7 +139,7 @@ static int idpf_mb_intr_req_irq(struct idpf_adapter *adapter) · 138 unmodified lines | ||
| 139 | 139 | if (err) { |
| 140 | 140 | dev_err(&adapter->pdev->dev, |
| 141 | 141 | "IRQ request for mailbox failed, error: %d\n", err); |
| 142 | − | |
| 142 | + kfree(name); | |
| 143 | 143 | return err; |
| 144 | 144 | } |
| 145 | 145 | |
| @@ -3097,10 +3097,7 @@ static netdev_tx_t idpf_tx_splitq_frame(struct sk_buff *skb, · 3096 unmodified lines | ||
| 3097 | 3097 | |
| 3098 | 3098 | tx_params.dtype = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE; |
| 3099 | 3099 | tx_params.eop_cmd = IDPF_TXD_FLEX_FLOW_CMD_EOP; |
| 3100 | − /* Set the RE bit to periodically "clean" the descriptor ring. | |
| 3101 | − * MIN_GAP is set to MIN_RING size to ensure it will be set at | |
| 3102 | − * least once each time around the ring. | |
| 3103 | − */ | |
| 3100 | + /* Set the RE bit periodically to "clean" the descriptor ring */ | |
| 3104 | 3101 | if (idpf_tx_splitq_need_re(tx_q)) { |
| 3105 | 3102 | tx_params.eop_cmd |= IDPF_TXD_FLEX_FLOW_CMD_RE; |
| 3106 | 3103 | tx_q->txq_grp->num_completions_pending++; |
| @@ -21,7 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | /* Mailbox Queue */ |
| 22 | 22 | #define IDPF_MAX_MBXQ 1 |
| 23 | 23 | |
| 24 | −#define IDPF_MIN_TXQ_DESC 64 | |
| 24 | +#define IDPF_MIN_TXQ_DESC 128 | |
| 25 | 25 | #define IDPF_MIN_RXQ_DESC 64 |
| 26 | 26 | #define IDPF_MIN_TXQ_COMPLQ_DESC 256 |
| 27 | 27 | #define IDPF_MAX_QIDS 256 |
| @@ -86,7 +86,7 @@ static int idpf_vf_intr_reg_init(struct idpf_vport *vport, · 85 unmodified lines | ||
| 86 | 86 | if (!reg_vals) |
| 87 | 87 | return -ENOMEM; |
| 88 | 88 | |
| 89 | − num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals); | |
| 89 | + num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals, total_vecs); | |
| 90 | 90 | if (num_regs < num_vecs) { |
| 91 | 91 | err = -EINVAL; |
| 92 | 92 | goto free_reg_vals; |
| @@ -1318,11 +1318,12 @@ idpf_vport_init_queue_reg_chunks(struct idpf_vport_config *vport_config, · 1317 unmodified lines | ||
| 1318 | 1318 | * idpf_get_reg_intr_vecs - Get vector queue register offset |
| 1319 | 1319 | * @adapter: adapter structure to get the vector chunks |
| 1320 | 1320 | * @reg_vals: Register offsets to store in |
| 1321 | + * @num_vecs: number of entries the @reg_vals array can hold | |
| 1321 | 1322 | * |
| 1322 | 1323 | * Return: number of registers that got populated |
| 1323 | 1324 | */ |
| 1324 | 1325 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, |
| 1325 | − struct idpf_vec_regs *reg_vals) | |
| 1326 | + struct idpf_vec_regs *reg_vals, int num_vecs) | |
| 1326 | 1327 | { |
| 1327 | 1328 | struct virtchnl2_vector_chunks *chunks; |
| 1328 | 1329 | struct idpf_vec_regs reg_val; |
| @@ -1346,7 +1347,7 @@ int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, · 17 unmodified lines | ||
| 1346 | 1347 | dynctl_reg_spacing = le32_to_cpu(chunk->dynctl_reg_spacing); |
| 1347 | 1348 | itrn_reg_spacing = le32_to_cpu(chunk->itrn_reg_spacing); |
| 1348 | 1349 | |
| 1349 | − for (i = 0; i < num_vec; i++) { | |
| 1350 | + for (i = 0; i < num_vec && num_regs < num_vecs; i++) { | |
| 1350 | 1351 | reg_vals[num_regs].dyn_ctl_reg = reg_val.dyn_ctl_reg; |
| 1351 | 1352 | reg_vals[num_regs].itrn_reg = reg_val.itrn_reg; |
| 1352 | 1353 | reg_vals[num_regs].itrn_index_spacing = |
| @@ -104,7 +104,7 @@ int idpf_vc_core_init(struct idpf_adapter *adapter); · 103 unmodified lines | ||
| 104 | 104 | void idpf_vc_core_deinit(struct idpf_adapter *adapter); |
| 105 | 105 | |
| 106 | 106 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, |
| 107 | − struct idpf_vec_regs *reg_vals); | |
| 107 | + struct idpf_vec_regs *reg_vals, int num_vecs); | |
| 108 | 108 | int idpf_queue_reg_init(struct idpf_vport *vport, |
| 109 | 109 | struct idpf_q_vec_rsrc *rsrc, |
| 110 | 110 | struct idpf_queue_id_reg_info *chunks); |
| @@ -2190,8 +2190,6 @@ dma_error: · 2189 unmodified lines | ||
| 2190 | 2190 | buffer_info->time_stamp = 0; |
| 2191 | 2191 | buffer_info->length = 0; |
| 2192 | 2192 | buffer_info->mapped_as_page = false; |
| 2193 | − if (count) | |
| 2194 | − count--; | |
| 2195 | 2193 | |
| 2196 | 2194 | /* clear timestamp and dma mappings for remaining portion of packet */ |
| 2197 | 2195 | while (count--) { |
| @@ -5352,9 +5352,8 @@ void igc_down(struct igc_adapter *adapter) · 5351 unmodified lines | ||
| 5352 | 5352 | |
| 5353 | 5353 | for (i = 0; i < adapter->num_q_vectors; i++) { |
| 5354 | 5354 | if (adapter->q_vector[i]) { |
| 5355 | − napi_synchronize(&adapter->q_vector[i]->napi); | |
| 5356 | − igc_set_queue_napi(adapter, i, NULL); | |
| 5357 | 5355 | napi_disable(&adapter->q_vector[i]->napi); |
| 5356 | + igc_set_queue_napi(adapter, i, NULL); | |
| 5358 | 5357 | } |
| 5359 | 5358 | } |
| 5360 | 5359 | |
| @@ -518,6 +518,18 @@ int cgx_set_pkind(void *cgxd, u8 lmac_id, int pkind) · 517 unmodified lines | ||
| 518 | 518 | return 0; |
| 519 | 519 | } |
| 520 | 520 | |
| 521 | +int cgx_get_pkind(void *cgxd, u8 lmac_id, int *pkind) | |
| 522 | +{ | |
| 523 | + struct cgx *cgx = cgxd; | |
| 524 | + | |
| 525 | + if (!is_lmac_valid(cgx, lmac_id)) | |
| 526 | + return -ENODEV; | |
| 527 | + | |
| 528 | + *pkind = cgx_read(cgx, lmac_id, cgx->mac_ops->rxid_map_offset); | |
| 529 | + *pkind = *pkind & 0x3F; | |
| 530 | + return 0; | |
| 531 | +} | |
| 532 | + | |
| 521 | 533 | static u8 cgx_get_lmac_type(void *cgxd, int lmac_id) |
| 522 | 534 | { |
| 523 | 535 | struct cgx *cgx = cgxd; |
| @@ -141,6 +141,7 @@ int cgx_get_cgxid(void *cgxd); · 140 unmodified lines | ||
| 141 | 141 | int cgx_get_lmac_cnt(void *cgxd); |
| 142 | 142 | void *cgx_get_pdata(int cgx_id); |
| 143 | 143 | int cgx_set_pkind(void *cgxd, u8 lmac_id, int pkind); |
| 144 | +int cgx_get_pkind(void *cgxd, u8 lmac_id, int *pkind); | |
| 144 | 145 | int cgx_lmac_evh_register(struct cgx_event_cb *cb, void *cgxd, int lmac_id); |
| 145 | 146 | int cgx_lmac_evh_unregister(void *cgxd, int lmac_id); |
| 146 | 147 | int cgx_get_tx_stats(void *cgxd, int lmac_id, int idx, u64 *tx_stat); |
| @@ -1115,6 +1115,8 @@ void npc_read_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam, · 1114 unmodified lines | ||
| 1115 | 1115 | u8 *intf, u8 *ena); |
| 1116 | 1116 | int npc_config_cntr_default_entries(struct rvu *rvu, bool enable); |
| 1117 | 1117 | bool is_cgx_config_permitted(struct rvu *rvu, u16 pcifunc); |
| 1118 | +bool rvu_cgx_check_permission_and_set_pkind(struct rvu *rvu, u16 pcifunc, int pkind); | |
| 1119 | +bool rvu_cgx_is_pkind_config_permitted(struct rvu *rvu, u16 pcifunc); | |
| 1118 | 1120 | bool is_mac_feature_supported(struct rvu *rvu, int pf, int feature); |
| 1119 | 1121 | u32 rvu_cgx_get_fifolen(struct rvu *rvu); |
| 1120 | 1122 | void *rvu_first_cgx_pdata(struct rvu *rvu); |
| @@ -1355,3 +1355,82 @@ void rvu_mac_reset(struct rvu *rvu, u16 pcifunc) · 1354 unmodified lines | ||
| 1355 | 1355 | if (mac_ops->mac_reset(cgxd, lmac, !is_vf(pcifunc))) |
| 1356 | 1356 | dev_err(rvu->dev, "Failed to reset MAC\n"); |
| 1357 | 1357 | } |
| 1358 | + | |
| 1359 | +/* Do not allow CGX-mapped VFs to overwrite PKIND when special parse kinds | |
| 1360 | + * (HiGig, EDSA, etc.) are in use on the shared LMAC. VFs must not program | |
| 1361 | + * NPC_TX_DEF_PKIND on NIX_AF_LFX_TX_PARSE_CFG in that case: the PF owns | |
| 1362 | + * parse mode and no separate NPC_TX_HIGIG_PKIND is installed on the VF LF. | |
| 1363 | + * TX-parse callers skip the write when denied; rvu_lf_reset() clears each LF | |
| 1364 | + * before alloc so the next permitted owner programs NPC_TX_DEF_PKIND. | |
| 1365 | + */ | |
| 1366 | +bool rvu_cgx_is_pkind_config_permitted(struct rvu *rvu, u16 pcifunc) | |
| 1367 | +{ | |
| 1368 | + int pf, err, rxpkind; | |
| 1369 | + u8 cgx_id, lmac_id; | |
| 1370 | + void *cgxd; | |
| 1371 | + | |
| 1372 | + pf = rvu_get_pf(rvu->pdev, pcifunc); | |
| 1373 | + | |
| 1374 | + if (!(pcifunc & RVU_PFVF_FUNC_MASK)) | |
| 1375 | + return true; | |
| 1376 | + | |
| 1377 | + if (!is_pf_cgxmapped(rvu, pf)) | |
| 1378 | + return true; | |
| 1379 | + | |
| 1380 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 1381 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 1382 | + err = cgx_get_pkind(cgxd, lmac_id, &rxpkind); | |
| 1383 | + if (err) | |
| 1384 | + return false; | |
| 1385 | + | |
| 1386 | + switch (rxpkind) { | |
| 1387 | + case NPC_RX_HIGIG_PKIND: | |
| 1388 | + case NPC_RX_EDSA_PKIND: | |
| 1389 | + return false; | |
| 1390 | + default: | |
| 1391 | + return true; | |
| 1392 | + } | |
| 1393 | +} | |
| 1394 | + | |
| 1395 | +/* Do not allow CGX-mapped VFs to overwrite PKIND when special parse kinds | |
| 1396 | + * (HiGig, EDSA, etc.) are in use on the shared LMAC. | |
| 1397 | + */ | |
| 1398 | +bool rvu_cgx_check_permission_and_set_pkind(struct rvu *rvu, u16 pcifunc, int pkind) | |
| 1399 | +{ | |
| 1400 | + int pf, err, rxpkind; | |
| 1401 | + u8 cgx_id, lmac_id; | |
| 1402 | + struct cgx *cgxd; | |
| 1403 | + | |
| 1404 | + pf = rvu_get_pf(rvu->pdev, pcifunc); | |
| 1405 | + | |
| 1406 | + if (!is_pf_cgxmapped(rvu, pf)) | |
| 1407 | + return false; | |
| 1408 | + | |
| 1409 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 1410 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 1411 | + | |
| 1412 | + mutex_lock(&cgxd->lock); | |
| 1413 | + if (!is_vf(pcifunc)) | |
| 1414 | + goto set; | |
| 1415 | + | |
| 1416 | + err = cgx_get_pkind(cgxd, lmac_id, &rxpkind); | |
| 1417 | + if (err) | |
| 1418 | + goto err; | |
| 1419 | + | |
| 1420 | + switch (rxpkind) { | |
| 1421 | + case NPC_RX_HIGIG_PKIND: | |
| 1422 | + case NPC_RX_EDSA_PKIND: | |
| 1423 | + goto err; | |
| 1424 | + default: | |
| 1425 | + break; | |
| 1426 | + } | |
| 1427 | + | |
| 1428 | +set: | |
| 1429 | + cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, pkind); | |
| 1430 | + mutex_unlock(&cgxd->lock); | |
| 1431 | + return true; | |
| 1432 | + | |
| 1433 | +err: | |
| 1434 | + mutex_unlock(&cgxd->lock); | |
| 1435 | + return false; | |
| 1436 | +} | |
| @@ -363,8 +363,8 @@ static int nix_interface_init(struct rvu *rvu, u16 pcifunc, int type, int nixlf, · 362 unmodified lines | ||
| 363 | 363 | pfvf->tx_chan_cnt = 1; |
| 364 | 364 | rsp->tx_link = cgx_id * hw->lmac_per_cgx + lmac_id; |
| 365 | 365 | |
| 366 | − cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, pkind); | |
| 367 | − rvu_npc_set_pkind(rvu, pkind, pfvf); | |
| 366 | + if (rvu_cgx_check_permission_and_set_pkind(rvu, pcifunc, pkind)) | |
| 367 | + rvu_npc_set_pkind(rvu, pkind, pfvf); | |
| 368 | 368 | break; |
| 369 | 369 | case NIX_INTF_TYPE_LBK: |
| 370 | 370 | vf = (pcifunc & RVU_PFVF_FUNC_MASK) - 1; |
| @@ -1505,13 +1505,15 @@ int rvu_mbox_handler_nix_lf_alloc(struct rvu *rvu, · 1134 unmodified lines | ||
| 1505 | 1505 | struct nix_lf_alloc_req *req, |
| 1506 | 1506 | struct nix_lf_alloc_rsp *rsp) |
| 1507 | 1507 | { |
| 1508 | − int nixlf, qints, hwctx_size, intf, rc = 0; | |
| 1508 | + int nixlf, qints, hwctx_size, intf, rc = 0, pf; | |
| 1509 | 1509 | u16 bcast, mcast, promisc, ucast; |
| 1510 | 1510 | struct rvu_hwinfo *hw = rvu->hw; |
| 1511 | 1511 | u16 pcifunc = req->hdr.pcifunc; |
| 1512 | + u8 cgx_id = 0, lmac_id = 0; | |
| 1512 | 1513 | bool rules_created = false; |
| 1513 | 1514 | struct rvu_block *block; |
| 1514 | 1515 | struct rvu_pfvf *pfvf; |
| 1516 | + struct cgx *cgxd; | |
| 1515 | 1517 | u64 cfg, ctx_cfg; |
| 1516 | 1518 | int blkaddr; |
| 1517 | 1519 | |
| @@ -1685,8 +1687,20 @@ int rvu_mbox_handler_nix_lf_alloc(struct rvu *rvu, · 167 unmodified lines | ||
| 1685 | 1687 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_RX_CFG(nixlf), req->rx_cfg); |
| 1686 | 1688 | |
| 1687 | 1689 | /* Configure pkind for TX parse config */ |
| 1690 | + | |
| 1691 | + pf = rvu_get_pf(rvu->pdev, pcifunc); | |
| 1688 | 1692 | cfg = NPC_TX_DEF_PKIND; |
| 1689 | − rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | |
| 1693 | + | |
| 1694 | + if (is_pf_cgxmapped(rvu, pf) && is_vf(pcifunc)) { | |
| 1695 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 1696 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 1697 | + mutex_lock(&cgxd->lock); | |
| 1698 | + if (rvu_cgx_is_pkind_config_permitted(rvu, pcifunc)) | |
| 1699 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | |
| 1700 | + mutex_unlock(&cgxd->lock); | |
| 1701 | + } else { | |
| 1702 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | |
| 1703 | + } | |
| 1690 | 1704 | |
| 1691 | 1705 | if (is_rep_dev(rvu, pcifunc)) { |
| 1692 | 1706 | pfvf->tx_chan_base = RVU_SWITCH_LBK_CHAN; |
| @@ -19,6 +19,7 @@ · 18 unmodified lines | ||
| 19 | 19 | #include "cn20k/npc.h" |
| 20 | 20 | #include "rvu_npc.h" |
| 21 | 21 | #include "cn20k/reg.h" |
| 22 | +#include "lmac_common.h" | |
| 22 | 23 | |
| 23 | 24 | #define RSVD_MCAM_ENTRIES_PER_PF 3 /* Broadcast, Promisc and AllMulticast */ |
| 24 | 25 | #define RSVD_MCAM_ENTRIES_PER_NIXLF 1 /* Ucast for LFs */ |
| @@ -4200,10 +4201,11 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 4175 unmodified lines | ||
| 4200 | 4201 | |
| 4201 | 4202 | { |
| 4202 | 4203 | struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc); |
| 4203 | − int blkaddr, nixlf, rc, intf_mode; | |
| 4204 | 4204 | int pf = rvu_get_pf(rvu->pdev, pcifunc); |
| 4205 | + int blkaddr, nixlf, rc, intf_mode; | |
| 4206 | + u8 cgx_id = 0, lmac_id = 0; | |
| 4205 | 4207 | u64 rxpkind, txpkind; |
| 4206 | − u8 cgx_id, lmac_id; | |
| 4208 | + struct cgx *cgxd; | |
| 4207 | 4209 | |
| 4208 | 4210 | /* use default pkind to disable edsa/higig */ |
| 4209 | 4211 | rxpkind = rvu_npc_get_pkind(rvu, pf); |
| @@ -4227,12 +4229,8 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 17 unmodified lines | ||
| 4227 | 4229 | /* rx pkind set req valid only for cgx mapped PFs */ |
| 4228 | 4230 | if (!is_cgx_config_permitted(rvu, pcifunc)) |
| 4229 | 4231 | return 0; |
| 4230 | − rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | |
| 4231 | − | |
| 4232 | − rc = cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, | |
| 4233 | − rxpkind); | |
| 4234 | − if (rc) | |
| 4235 | − return rc; | |
| 4232 | + if (!rvu_cgx_check_permission_and_set_pkind(rvu, pcifunc, rxpkind)) | |
| 4233 | + return -EINVAL; | |
| 4236 | 4234 | } |
| 4237 | 4235 | |
| 4238 | 4236 | if (dir & PKIND_TX) { |
| @@ -4241,8 +4239,19 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 2 unmodified lines | ||
| 4241 | 4239 | if (rc) |
| 4242 | 4240 | return rc; |
| 4243 | 4241 | |
| 4244 | − rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | |
| 4245 | − txpkind); | |
| 4242 | + if (is_pf_cgxmapped(rvu, pf) && is_vf(pcifunc)) { | |
| 4243 | + rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, | |
| 4244 | + &lmac_id); | |
| 4245 | + cgxd = rvu_cgx_pdata(cgx_id, rvu); | |
| 4246 | + mutex_lock(&cgxd->lock); | |
| 4247 | + if (rvu_cgx_is_pkind_config_permitted(rvu, pcifunc)) | |
| 4248 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | |
| 4249 | + txpkind); | |
| 4250 | + mutex_unlock(&cgxd->lock); | |
| 4251 | + } else { | |
| 4252 | + rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | |
| 4253 | + txpkind); | |
| 4254 | + } | |
| 4246 | 4255 | } |
| 4247 | 4256 | |
| 4248 | 4257 | pfvf->intf_mode = intf_mode; |
| @@ -889,8 +889,8 @@ static void otx2_handle_link_event(struct otx2_nic *pf) · 888 unmodified lines | ||
| 889 | 889 | netif_carrier_on(netdev); |
| 890 | 890 | netif_tx_start_all_queues(netdev); |
| 891 | 891 | } else { |
| 892 | − netif_tx_stop_all_queues(netdev); | |
| 893 | 892 | netif_carrier_off(netdev); |
| 893 | + netif_tx_stop_all_queues(netdev); | |
| 894 | 894 | } |
| 895 | 895 | } |
| 896 | 896 | |
| @@ -3467,7 +3467,7 @@ static void mtk_poll_controller(struct net_device *dev) · 3466 unmodified lines | ||
| 3467 | 3467 | |
| 3468 | 3468 | mtk_tx_irq_disable(eth, MTK_TX_DONE_INT); |
| 3469 | 3469 | mtk_rx_irq_disable(eth, eth->soc->rx.irq_done_mask); |
| 3470 | − mtk_handle_irq_rx(eth->irq[MTK_FE_IRQ_RX], dev); | |
| 3470 | + mtk_handle_irq_rx(eth->irq[MTK_FE_IRQ_RX], eth); | |
| 3471 | 3471 | mtk_tx_irq_enable(eth, MTK_TX_DONE_INT); |
| 3472 | 3472 | mtk_rx_irq_enable(eth, eth->soc->rx.irq_done_mask); |
| 3473 | 3473 | } |
| @@ -984,7 +984,6 @@ struct mlx5e_priv { · 983 unmodified lines | ||
| 984 | 984 | struct mlx5e_mqprio_rl *mqprio_rl; |
| 985 | 985 | struct dentry *dfs_root; |
| 986 | 986 | struct mlx5_devcom_comp_dev *devcom; |
| 987 | − struct ethtool_fec_hist_range *fec_ranges; | |
| 988 | 987 | }; |
| 989 | 988 | |
| 990 | 989 | static inline u16 mlx5e_stats_nch_read(const struct mlx5e_priv *priv) |
| @@ -6415,14 +6415,8 @@ int mlx5e_priv_init(struct mlx5e_priv *priv, · 6414 unmodified lines | ||
| 6415 | 6415 | if (!priv->channel_stats) |
| 6416 | 6416 | goto err_free_tx_rates; |
| 6417 | 6417 | |
| 6418 | − priv->fec_ranges = kzalloc_objs(*priv->fec_ranges, ETHTOOL_FEC_HIST_MAX); | |
| 6419 | − if (!priv->fec_ranges) | |
| 6420 | − goto err_free_channel_stats; | |
| 6421 | − | |
| 6422 | 6418 | return 0; |
| 6423 | 6419 | |
| 6424 | −err_free_channel_stats: | |
| 6425 | − kfree(priv->channel_stats); | |
| 6426 | 6420 | err_free_tx_rates: |
| 6427 | 6421 | kfree(priv->tx_rates); |
| 6428 | 6422 | err_free_txq2sq_stats: |
| @@ -6447,7 +6441,6 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv) · 18 unmodified lines | ||
| 6447 | 6441 | if (!priv->mdev) |
| 6448 | 6442 | return; |
| 6449 | 6443 | |
| 6450 | − kfree(priv->fec_ranges); | |
| 6451 | 6444 | for (i = 0; i < priv->stats_nch; i++) |
| 6452 | 6445 | kvfree(priv->channel_stats[i]); |
| 6453 | 6446 | kfree(priv->channel_stats); |
| @@ -1550,7 +1550,7 @@ static bool fec_rs_validate_hist_type(int mode, int hist_type) · 1549 unmodified lines | ||
| 1550 | 1550 | |
| 1551 | 1551 | static u8 |
| 1552 | 1552 | fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, |
| 1553 | − const struct ethtool_fec_hist_range **ranges) | |
| 1553 | + struct ethtool_fec_hist_range *ranges) | |
| 1554 | 1554 | { |
| 1555 | 1555 | struct mlx5_core_dev *mdev = priv->mdev; |
| 1556 | 1556 | u32 out[MLX5_ST_SZ_DW(pphcr_reg)] = {0}; |
| @@ -1558,8 +1558,6 @@ fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, · 1 unmodified line | ||
| 1558 | 1558 | int sz = MLX5_ST_SZ_BYTES(pphcr_reg); |
| 1559 | 1559 | u8 hist_type, num_of_bins; |
| 1560 | 1560 | |
| 1561 | − memset(priv->fec_ranges, 0, | |
| 1562 | − ETHTOOL_FEC_HIST_MAX * sizeof(*priv->fec_ranges)); | |
| 1563 | 1561 | MLX5_SET(pphcr_reg, in, local_port, 1); |
| 1564 | 1562 | if (mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPHCR, 0, 0)) |
| 1565 | 1563 | return 0; |
| @@ -1575,12 +1573,11 @@ fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, · 9 unmodified lines | ||
| 1575 | 1573 | for (int i = 0; i < num_of_bins; i++) { |
| 1576 | 1574 | void *bin_range = MLX5_ADDR_OF(pphcr_reg, out, bin_range[i]); |
| 1577 | 1575 | |
| 1578 | − priv->fec_ranges[i].high = MLX5_GET(bin_range_layout, bin_range, | |
| 1579 | − high_val); | |
| 1580 | − priv->fec_ranges[i].low = MLX5_GET(bin_range_layout, bin_range, | |
| 1581 | − low_val); | |
| 1576 | + ranges[i].high = MLX5_GET(bin_range_layout, bin_range, | |
| 1577 | + high_val); | |
| 1578 | + ranges[i].low = MLX5_GET(bin_range_layout, bin_range, | |
| 1579 | + low_val); | |
| 1582 | 1580 | } |
| 1583 | − *ranges = priv->fec_ranges; | |
| 1584 | 1581 | |
| 1585 | 1582 | return num_of_bins; |
| 1586 | 1583 | } |
| @@ -1622,10 +1619,12 @@ static void fec_set_histograms_stats(struct mlx5e_priv *priv, int mode, · 35 unmodified lines | ||
| 1622 | 1619 | case MLX5E_FEC_LLRS_272_257_1: |
| 1623 | 1620 | case MLX5E_FEC_RS_544_514_INTERLEAVED_QUAD: |
| 1624 | 1621 | num_of_bins = |
| 1625 | − fec_rs_histogram_fill_ranges(priv, mode, &hist->ranges); | |
| 1626 | − if (num_of_bins) | |
| 1622 | + fec_rs_histogram_fill_ranges(priv, mode, hist->ranges_buf); | |
| 1623 | + if (num_of_bins) { | |
| 1624 | + hist->ranges = hist->ranges_buf; | |
| 1627 | 1625 | return fec_rs_histogram_fill_stats(priv, num_of_bins, |
| 1628 | 1626 | hist); |
| 1627 | + } | |
| 1629 | 1628 | break; |
| 1630 | 1629 | default: |
| 1631 | 1630 | return; |
| @@ -2829,7 +2829,7 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc, · 2828 unmodified lines | ||
| 2829 | 2829 | |
| 2830 | 2830 | rxq = kvzalloc_flex(*rxq, rx_oobs, apc->rx_queue_size); |
| 2831 | 2831 | if (!rxq) |
| 2832 | − return NULL; | |
| 2832 | + return ERR_PTR(-ENOMEM); | |
| 2833 | 2833 | |
| 2834 | 2834 | rxq->ndev = ndev; |
| 2835 | 2835 | rxq->num_rx_buf = apc->rx_queue_size; |
| @@ -2930,7 +2930,7 @@ out: · 94 unmodified lines | ||
| 2930 | 2930 | |
| 2931 | 2931 | mana_destroy_rxq(apc, rxq, false); |
| 2932 | 2932 | |
| 2933 | − return NULL; | |
| 2933 | + return ERR_PTR(err); | |
| 2934 | 2934 | } |
| 2935 | 2935 | |
| 2936 | 2936 | static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx) |
| @@ -2964,8 +2964,8 @@ static int mana_add_rx_queues(struct mana_port_context *apc, · 27 unmodified lines | ||
| 2964 | 2964 | |
| 2965 | 2965 | for (i = 0; i < apc->num_queues; i++) { |
| 2966 | 2966 | rxq = mana_create_rxq(apc, i, &apc->eqs[i], ndev); |
| 2967 | − if (!rxq) { | |
| 2968 | − err = -ENOMEM; | |
| 2967 | + if (IS_ERR(rxq)) { | |
| 2968 | + err = PTR_ERR(rxq); | |
| 2969 | 2969 | netdev_err(ndev, "Failed to create rxq %d : %d\n", i, err); |
| 2970 | 2970 | goto out; |
| 2971 | 2971 | } |
| @@ -6187,10 +6187,10 @@ static void nv_remove(struct pci_dev *pci_dev) · 6186 unmodified lines | ||
| 6187 | 6187 | struct net_device *dev = pci_get_drvdata(pci_dev); |
| 6188 | 6188 | struct fe_priv *np = netdev_priv(dev); |
| 6189 | 6189 | |
| 6190 | − free_percpu(np->txrx_stats); | |
| 6191 | − | |
| 6192 | 6190 | unregister_netdev(dev); |
| 6193 | 6191 | |
| 6192 | + free_percpu(np->txrx_stats); | |
| 6193 | + | |
| 6194 | 6194 | nv_restore_mac_addr(pci_dev); |
| 6195 | 6195 | |
| 6196 | 6196 | /* restore any phy related changes */ |
| @@ -107,7 +107,7 @@ static void qede_remove(struct pci_dev *pdev); · 106 unmodified lines | ||
| 107 | 107 | static void qede_shutdown(struct pci_dev *pdev); |
| 108 | 108 | static void qede_link_update(void *dev, struct qed_link_output *link); |
| 109 | 109 | static void qede_schedule_recovery_handler(void *dev); |
| 110 | −static void qede_recovery_handler(struct qede_dev *edev); | |
| 110 | +static bool qede_recovery_handler(struct qede_dev *edev); | |
| 111 | 111 | static void qede_schedule_hw_err_handler(void *dev, |
| 112 | 112 | enum qed_hw_err_type err_type); |
| 113 | 113 | static void qede_get_eth_tlv_data(void *edev, void *data); |
| @@ -1043,21 +1043,6 @@ void __qede_unlock(struct qede_dev *edev) · 929 unmodified lines | ||
| 1043 | 1043 | mutex_unlock(&edev->qede_lock); |
| 1044 | 1044 | } |
| 1045 | 1045 | |
| 1046 | −/* This version of the lock should be used when acquiring the RTNL lock is also | |
| 1047 | − * needed in addition to the internal qede lock. | |
| 1048 | − */ | |
| 1049 | −static void qede_lock(struct qede_dev *edev) | |
| 1050 | −{ | |
| 1051 | − rtnl_lock(); | |
| 1052 | − __qede_lock(edev); | |
| 1053 | −} | |
| 1054 | − | |
| 1055 | −static void qede_unlock(struct qede_dev *edev) | |
| 1056 | −{ | |
| 1057 | − __qede_unlock(edev); | |
| 1058 | − rtnl_unlock(); | |
| 1059 | −} | |
| 1060 | − | |
| 1061 | 1046 | static void qede_periodic_task(struct work_struct *work) |
| 1062 | 1047 | { |
| 1063 | 1048 | struct qede_dev *edev = container_of(work, struct qede_dev, |
| @@ -1094,6 +1079,8 @@ static void qede_sp_task(struct work_struct *work) · 30 unmodified lines | ||
| 1094 | 1079 | */ |
| 1095 | 1080 | |
| 1096 | 1081 | if (test_and_clear_bit(QEDE_SP_RECOVERY, &edev->sp_flags)) { |
| 1082 | + bool reloaded; | |
| 1083 | + | |
| 1097 | 1084 | cancel_delayed_work_sync(&edev->periodic_task); |
| 1098 | 1085 | #ifdef CONFIG_QED_SRIOV |
| 1099 | 1086 | /* SRIOV must be disabled outside the lock to avoid a deadlock. |
| @@ -1102,9 +1089,17 @@ static void qede_sp_task(struct work_struct *work) · 2 unmodified lines | ||
| 1102 | 1089 | if (pci_num_vf(edev->pdev)) |
| 1103 | 1090 | qede_sriov_configure(edev->pdev, 0); |
| 1104 | 1091 | #endif |
| 1105 | − qede_lock(edev); | |
| 1106 | − qede_recovery_handler(edev); | |
| 1107 | − qede_unlock(edev); | |
| 1092 | + rtnl_lock(); | |
| 1093 | + __qede_lock(edev); | |
| 1094 | + reloaded = qede_recovery_handler(edev); | |
| 1095 | + __qede_unlock(edev); | |
| 1096 | + | |
| 1097 | + /* The udp_tunnel core synchronously calls back into | |
| 1098 | + * qede_udp_tunnel_sync(), which takes the qede lock. | |
| 1099 | + */ | |
| 1100 | + if (reloaded) | |
| 1101 | + udp_tunnel_nic_reset_ntf(edev->ndev); | |
| 1102 | + rtnl_unlock(); | |
| 1108 | 1103 | } |
| 1109 | 1104 | |
| 1110 | 1105 | __qede_lock(edev); |
| @@ -2645,9 +2640,13 @@ static void qede_recovery_failed(struct qede_dev *edev) · 1534 unmodified lines | ||
| 2645 | 2640 | edev->ops->common->set_power_state(edev->cdev, PCI_D3hot); |
| 2646 | 2641 | } |
| 2647 | 2642 | |
| 2648 | −static void qede_recovery_handler(struct qede_dev *edev) | |
| 2643 | +/* Returns true if an open device was successfully reloaded and its | |
| 2644 | + * udp_tunnel ports need to be re-synced by the caller. | |
| 2645 | + */ | |
| 2646 | +static bool qede_recovery_handler(struct qede_dev *edev) | |
| 2649 | 2647 | { |
| 2650 | 2648 | u32 curr_state = edev->state; |
| 2649 | + bool reloaded = false; | |
| 2651 | 2650 | int rc; |
| 2652 | 2651 | |
| 2653 | 2652 | DP_NOTICE(edev, "Starting a recovery process\n"); |
| @@ -2677,17 +2676,18 @@ static void qede_recovery_handler(struct qede_dev *edev) · 23 unmodified lines | ||
| 2677 | 2676 | goto err; |
| 2678 | 2677 | |
| 2679 | 2678 | qede_config_rx_mode(edev->ndev); |
| 2680 | − udp_tunnel_nic_reset_ntf(edev->ndev); | |
| 2679 | + reloaded = true; | |
| 2681 | 2680 | } |
| 2682 | 2681 | |
| 2683 | 2682 | edev->state = curr_state; |
| 2684 | 2683 | |
| 2685 | 2684 | DP_NOTICE(edev, "Recovery handling is done\n"); |
| 2686 | 2685 | |
| 2687 | − return; | |
| 2686 | + return reloaded; | |
| 2688 | 2687 | |
| 2689 | 2688 | err: |
| 2690 | 2689 | qede_recovery_failed(edev); |
| 2690 | + return false; | |
| 2691 | 2691 | } |
| 2692 | 2692 | |
| 2693 | 2693 | static void qede_atomic_hw_err_handler(struct qede_dev *edev) |
| @@ -1623,6 +1623,9 @@ static netdev_tx_t rtase_start_xmit(struct sk_buff *skb, · 1622 unmodified lines | ||
| 1623 | 1623 | err_dma_1: |
| 1624 | 1624 | ring->skbuff[entry] = NULL; |
| 1625 | 1625 | rtase_tx_clear_range(ring, ring->cur_idx + 1, frags); |
| 1626 | + if (frags) | |
| 1627 | + /* the frags were cleared above, along with the skb */ | |
| 1628 | + return NETDEV_TX_OK; | |
| 1626 | 1629 | |
| 1627 | 1630 | err_dma_0: |
| 1628 | 1631 | tp->stats.tx_dropped++; |
| @@ -597,14 +597,13 @@ static int init_dma_desc_rings(struct net_device *netd) · 596 unmodified lines | ||
| 597 | 597 | |
| 598 | 598 | return 0; |
| 599 | 599 | |
| 600 | −txalloc_err: | |
| 601 | − while (queue_num--) | |
| 602 | − free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); | |
| 603 | − return ret; | |
| 604 | − | |
| 605 | 600 | rxalloc_err: |
| 606 | 601 | while (queue_num--) |
| 607 | 602 | free_rx_ring(priv->device, priv->rxq[queue_num], rx_rsize); |
| 603 | + queue_num = SXGBE_TX_QUEUES; | |
| 604 | +txalloc_err: | |
| 605 | + while (queue_num--) | |
| 606 | + free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); | |
| 608 | 607 | return ret; |
| 609 | 608 | } |
| 610 | 609 | |
| @@ -1079,7 +1078,9 @@ static int sxgbe_open(struct net_device *dev) · 468 unmodified lines | ||
| 1079 | 1078 | priv->dma_buf_sz = SXGBE_ALIGN(DMA_BUFFER_SIZE); |
| 1080 | 1079 | priv->tx_tc = TC_DEFAULT; |
| 1081 | 1080 | priv->rx_tc = TC_DEFAULT; |
| 1082 | − init_dma_desc_rings(dev); | |
| 1081 | + ret = init_dma_desc_rings(dev); | |
| 1082 | + if (ret) | |
| 1083 | + goto init_phy_error; | |
| 1083 | 1084 | |
| 1084 | 1085 | /* DMA initialization and SW reset */ |
| 1085 | 1086 | ret = sxgbe_init_dma_engine(priv); |
| @@ -1188,6 +1189,7 @@ static int sxgbe_open(struct net_device *dev) · 102 unmodified lines | ||
| 1188 | 1189 | |
| 1189 | 1190 | init_error: |
| 1190 | 1191 | free_dma_desc_resources(priv); |
| 1192 | +init_phy_error: | |
| 1191 | 1193 | if (dev->phydev) |
| 1192 | 1194 | phy_disconnect(dev->phydev); |
| 1193 | 1195 | phy_error: |
| @@ -755,7 +755,8 @@ static int stmmac_hwtstamp_set(struct net_device *dev, · 754 unmodified lines | ||
| 755 | 755 | config->rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; |
| 756 | 756 | ptp_v2 = PTP_TCR_TSVER2ENA; |
| 757 | 757 | snap_type_sel = PTP_TCR_SNAPTYPSEL_1; |
| 758 | − if (priv->synopsys_id < DWMAC_CORE_4_10) | |
| 758 | + if (priv->synopsys_id < DWMAC_CORE_3_70 && | |
| 759 | + priv->plat->core_type != DWMAC_CORE_XGMAC) | |
| 759 | 760 | ts_event_en = PTP_TCR_TSEVNTENA; |
| 760 | 761 | ptp_over_ipv4_udp = PTP_TCR_TSIPV4ENA; |
| 761 | 762 | ptp_over_ipv6_udp = PTP_TCR_TSIPV6ENA; |
| @@ -1606,6 +1606,8 @@ static netdev_tx_t wx_xmit_frame_ring(struct sk_buff *skb, · 1605 unmodified lines | ||
| 1606 | 1606 | if (skb_vlan_tag_present(skb)) { |
| 1607 | 1607 | tx_flags |= skb_vlan_tag_get(skb) << WX_TX_FLAGS_VLAN_SHIFT; |
| 1608 | 1608 | tx_flags |= WX_TX_FLAGS_HW_VLAN; |
| 1609 | + } else if (eth_type_vlan(skb->protocol)) { | |
| 1610 | + tx_flags |= WX_TX_FLAGS_SW_VLAN; | |
| 1609 | 1611 | } |
| 1610 | 1612 | |
| 1611 | 1613 | if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) && |
| @@ -1875,8 +1875,8 @@ struct phylink *phylink_create(struct phylink_config *config, · 1874 unmodified lines | ||
| 1875 | 1875 | } else if (config->type == PHYLINK_DEV) { |
| 1876 | 1876 | pl->dev = config->dev; |
| 1877 | 1877 | } else { |
| 1878 | − kfree(pl); | |
| 1879 | − return ERR_PTR(-EINVAL); | |
| 1878 | + ret = -EINVAL; | |
| 1879 | + goto free_pl; | |
| 1880 | 1880 | } |
| 1881 | 1881 | |
| 1882 | 1882 | pl->mac_supports_eee_ops = phylink_mac_implements_lpi(mac_ops); |
| @@ -1909,28 +1909,29 @@ struct phylink *phylink_create(struct phylink_config *config, · 26 unmodified lines | ||
| 1909 | 1909 | phylink_validate(pl, pl->supported, &pl->link_config); |
| 1910 | 1910 | |
| 1911 | 1911 | ret = phylink_parse_mode(pl, fwnode); |
| 1912 | − if (ret < 0) { | |
| 1913 | − kfree(pl); | |
| 1914 | − return ERR_PTR(ret); | |
| 1915 | − } | |
| 1912 | + if (ret < 0) | |
| 1913 | + goto free_pl; | |
| 1916 | 1914 | |
| 1917 | 1915 | if (pl->cfg_link_an_mode == MLO_AN_FIXED) { |
| 1918 | 1916 | ret = phylink_parse_fixedlink(pl, fwnode); |
| 1919 | − if (ret < 0) { | |
| 1920 | − kfree(pl); | |
| 1921 | − return ERR_PTR(ret); | |
| 1922 | − } | |
| 1917 | + if (ret < 0) | |
| 1918 | + goto release_link_gpio; | |
| 1923 | 1919 | } |
| 1924 | 1920 | |
| 1925 | 1921 | pl->req_link_an_mode = pl->cfg_link_an_mode; |
| 1926 | 1922 | |
| 1927 | 1923 | ret = phylink_register_sfp(pl, fwnode); |
| 1928 | − if (ret < 0) { | |
| 1929 | − kfree(pl); | |
| 1930 | − return ERR_PTR(ret); | |
| 1931 | − } | |
| 1924 | + if (ret < 0) | |
| 1925 | + goto release_link_gpio; | |
| 1932 | 1926 | |
| 1933 | 1927 | return pl; |
| 1928 | + | |
| 1929 | +release_link_gpio: | |
| 1930 | + if (pl->link_gpio) | |
| 1931 | + gpiod_put(pl->link_gpio); | |
| 1932 | +free_pl: | |
| 1933 | + kfree(pl); | |
| 1934 | + return ERR_PTR(ret); | |
| 1934 | 1935 | } |
| 1935 | 1936 | EXPORT_SYMBOL_GPL(phylink_create); |
| 1936 | 1937 | |
| @@ -145,8 +145,6 @@ struct tun_file { · 144 unmodified lines | ||
| 145 | 145 | struct list_head next; |
| 146 | 146 | struct tun_struct *detached; |
| 147 | 147 | struct ptr_ring tx_ring; |
| 148 | − /* Protected by tx_ring.consumer_lock */ | |
| 149 | − int cons_cnt; | |
| 150 | 148 | struct xdp_rxq_info xdp_rxq; |
| 151 | 149 | }; |
| 152 | 150 | |
| @@ -590,13 +588,8 @@ static void __tun_detach(struct tun_file *tfile, bool clean) · 437 unmodified lines | ||
| 590 | 588 | rcu_assign_pointer(tun->tfiles[index], |
| 591 | 589 | tun->tfiles[tun->numqueues - 1]); |
| 592 | 590 | ntfile = rtnl_dereference(tun->tfiles[index]); |
| 593 | − spin_lock(&ntfile->tx_ring.consumer_lock); | |
| 594 | 591 | ntfile->queue_index = index; |
| 595 | 592 | ntfile->xdp_rxq.queue_index = index; |
| 596 | − ntfile->cons_cnt = 0; | |
| 597 | − if (__ptr_ring_empty(&ntfile->tx_ring)) | |
| 598 | − netif_wake_subqueue(tun->dev, index); | |
| 599 | − spin_unlock(&ntfile->tx_ring.consumer_lock); | |
| 600 | 593 | rcu_assign_pointer(tun->tfiles[tun->numqueues - 1], |
| 601 | 594 | NULL); |
| 602 | 595 | |
| @@ -737,9 +730,6 @@ static int tun_attach(struct tun_struct *tun, struct file *file, · 134 unmodified lines | ||
| 737 | 730 | goto out; |
| 738 | 731 | } |
| 739 | 732 | |
| 740 | − spin_lock(&tfile->tx_ring.consumer_lock); | |
| 741 | − tfile->cons_cnt = 0; | |
| 742 | − spin_unlock(&tfile->tx_ring.consumer_lock); | |
| 743 | 733 | tfile->queue_index = tun->numqueues; |
| 744 | 734 | tfile->socket.sk->sk_shutdown &= ~RCV_SHUTDOWN; |
| 745 | 735 | |
| @@ -1018,7 +1008,6 @@ static netdev_tx_t tun_net_xmit(struct sk_buff *skb, struct net_device *dev) · 272 unmodified lines | ||
| 1018 | 1008 | struct netdev_queue *queue; |
| 1019 | 1009 | struct tun_file *tfile; |
| 1020 | 1010 | int len = skb->len; |
| 1021 | − int ret; | |
| 1022 | 1011 | |
| 1023 | 1012 | rcu_read_lock(); |
| 1024 | 1013 | tfile = rcu_dereference(tun->tfiles[txq]); |
| @@ -1073,33 +1062,13 @@ static netdev_tx_t tun_net_xmit(struct sk_buff *skb, struct net_device *dev) · 48 unmodified lines | ||
| 1073 | 1062 | |
| 1074 | 1063 | nf_reset_ct(skb); |
| 1075 | 1064 | |
| 1076 | − queue = netdev_get_tx_queue(dev, txq); | |
| 1077 | − | |
| 1078 | − spin_lock(&tfile->tx_ring.producer_lock); | |
| 1079 | − ret = __ptr_ring_produce(&tfile->tx_ring, skb); | |
| 1080 | − if (!qdisc_txq_has_no_queue(queue) && | |
| 1081 | − __ptr_ring_check_produce(&tfile->tx_ring) == -ENOSPC) { | |
| 1082 | − netif_tx_stop_queue(queue); | |
| 1083 | − /* Paired with smp_mb() in __tun_wake_queue() */ | |
| 1084 | − smp_mb__after_atomic(); | |
| 1085 | − if (!__ptr_ring_check_produce(&tfile->tx_ring)) | |
| 1086 | − netif_tx_wake_queue(queue); | |
| 1087 | − } | |
| 1088 | − spin_unlock(&tfile->tx_ring.producer_lock); | |
| 1089 | − | |
| 1090 | − if (ret) { | |
| 1091 | − /* This should be a rare case if a qdisc is present, but | |
| 1092 | − * can happen due to lltx. | |
| 1093 | − * Since skb_tx_timestamp(), skb_orphan(), | |
| 1094 | − * run_ebpf_filter() and pskb_trim() could have tinkered | |
| 1095 | − * with the SKB, returning NETDEV_TX_BUSY is unsafe and | |
| 1096 | − * we must drop instead. | |
| 1097 | − */ | |
| 1065 | + if (ptr_ring_produce(&tfile->tx_ring, skb)) { | |
| 1098 | 1066 | drop_reason = SKB_DROP_REASON_FULL_RING; |
| 1099 | 1067 | goto drop; |
| 1100 | 1068 | } |
| 1101 | 1069 | |
| 1102 | 1070 | /* dev->lltx requires to do our own update of trans_start */ |
| 1071 | + queue = netdev_get_tx_queue(dev, txq); | |
| 1103 | 1072 | txq_trans_cond_update(queue); |
| 1104 | 1073 | |
| 1105 | 1074 | /* Notify and wake up reader process */ |
| @@ -2147,46 +2116,13 @@ done: · 1041 unmodified lines | ||
| 2147 | 2116 | return total; |
| 2148 | 2117 | } |
| 2149 | 2118 | |
| 2150 | −/* Callers must hold ring.consumer_lock */ | |
| 2151 | −static void __tun_wake_queue(struct tun_struct *tun, | |
| 2152 | − struct tun_file *tfile, int consumed) | |
| 2153 | −{ | |
| 2154 | − struct netdev_queue *txq = netdev_get_tx_queue(tun->dev, | |
| 2155 | − tfile->queue_index); | |
| 2156 | − | |
| 2157 | − /* Paired with smp_mb__after_atomic() in tun_net_xmit() */ | |
| 2158 | − smp_mb(); | |
| 2159 | − if (netif_tx_queue_stopped(txq)) { | |
| 2160 | − tfile->cons_cnt += consumed; | |
| 2161 | − if (tfile->cons_cnt >= tfile->tx_ring.size / 2 || | |
| 2162 | − __ptr_ring_empty(&tfile->tx_ring)) { | |
| 2163 | − netif_tx_wake_queue(txq); | |
| 2164 | − tfile->cons_cnt = 0; | |
| 2165 | − } | |
| 2166 | − } | |
| 2167 | −} | |
| 2168 | − | |
| 2169 | −static void *tun_ring_consume(struct tun_struct *tun, struct tun_file *tfile) | |
| 2170 | −{ | |
| 2171 | − void *ptr; | |
| 2172 | − | |
| 2173 | − spin_lock(&tfile->tx_ring.consumer_lock); | |
| 2174 | − ptr = __ptr_ring_consume(&tfile->tx_ring); | |
| 2175 | − if (ptr) | |
| 2176 | − __tun_wake_queue(tun, tfile, 1); | |
| 2177 | − | |
| 2178 | − spin_unlock(&tfile->tx_ring.consumer_lock); | |
| 2179 | − return ptr; | |
| 2180 | −} | |
| 2181 | − | |
| 2182 | −static void *tun_ring_recv(struct tun_struct *tun, struct tun_file *tfile, | |
| 2183 | − int noblock, int *err) | |
| 2119 | +static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) | |
| 2184 | 2120 | { |
| 2185 | 2121 | DECLARE_WAITQUEUE(wait, current); |
| 2186 | 2122 | void *ptr = NULL; |
| 2187 | 2123 | int error = 0; |
| 2188 | 2124 | |
| 2189 | − ptr = tun_ring_consume(tun, tfile); | |
| 2125 | + ptr = ptr_ring_consume(&tfile->tx_ring); | |
| 2190 | 2126 | if (ptr) |
| 2191 | 2127 | goto out; |
| 2192 | 2128 | if (noblock) { |
| @@ -2198,7 +2134,7 @@ static void *tun_ring_recv(struct tun_struct *tun, struct tun_file *tfile, · 5 unmodified lines | ||
| 2198 | 2134 | |
| 2199 | 2135 | while (1) { |
| 2200 | 2136 | set_current_state(TASK_INTERRUPTIBLE); |
| 2201 | − ptr = tun_ring_consume(tun, tfile); | |
| 2137 | + ptr = ptr_ring_consume(&tfile->tx_ring); | |
| 2202 | 2138 | if (ptr) |
| 2203 | 2139 | break; |
| 2204 | 2140 | if (signal_pending(current)) { |
| @@ -2235,7 +2171,7 @@ static ssize_t tun_do_read(struct tun_struct *tun, struct tun_file *tfile, · 30 unmodified lines | ||
| 2235 | 2171 | |
| 2236 | 2172 | if (!ptr) { |
| 2237 | 2173 | /* Read frames from ring */ |
| 2238 | − ptr = tun_ring_recv(tun, tfile, noblock, &err); | |
| 2174 | + ptr = tun_ring_recv(tfile, noblock, &err); | |
| 2239 | 2175 | if (!ptr) |
| 2240 | 2176 | return err; |
| 2241 | 2177 | } |
| @@ -3690,16 +3626,6 @@ static int tun_queue_resize(struct tun_struct *tun) · 1448 unmodified lines | ||
| 3690 | 3626 | dev->tx_queue_len, GFP_KERNEL, |
| 3691 | 3627 | tun_ptr_free); |
| 3692 | 3628 | |
| 3693 | − if (!ret) { | |
| 3694 | − for (i = 0; i < tun->numqueues; i++) { | |
| 3695 | − tfile = rtnl_dereference(tun->tfiles[i]); | |
| 3696 | − spin_lock(&tfile->tx_ring.consumer_lock); | |
| 3697 | − netif_wake_subqueue(tun->dev, tfile->queue_index); | |
| 3698 | − tfile->cons_cnt = 0; | |
| 3699 | − spin_unlock(&tfile->tx_ring.consumer_lock); | |
| 3700 | − } | |
| 3701 | − } | |
| 3702 | − | |
| 3703 | 3629 | kfree(rings); |
| 3704 | 3630 | return ret; |
| 3705 | 3631 | } |
| @@ -3808,29 +3734,6 @@ struct ptr_ring *tun_get_tx_ring(struct file *file) · 102 unmodified lines | ||
| 3808 | 3734 | } |
| 3809 | 3735 | EXPORT_SYMBOL_GPL(tun_get_tx_ring); |
| 3810 | 3736 | |
| 3811 | −/* Callers must hold ring.consumer_lock */ | |
| 3812 | −void tun_wake_queue(struct file *file, int consumed) | |
| 3813 | −{ | |
| 3814 | − struct tun_file *tfile; | |
| 3815 | − struct tun_struct *tun; | |
| 3816 | − | |
| 3817 | − if (file->f_op != &tun_fops) | |
| 3818 | − return; | |
| 3819 | − | |
| 3820 | − tfile = file->private_data; | |
| 3821 | − if (!tfile) | |
| 3822 | − return; | |
| 3823 | − | |
| 3824 | − rcu_read_lock(); | |
| 3825 | − | |
| 3826 | − tun = rcu_dereference(tfile->tun); | |
| 3827 | − if (tun) | |
| 3828 | − __tun_wake_queue(tun, tfile, consumed); | |
| 3829 | − | |
| 3830 | − rcu_read_unlock(); | |
| 3831 | −} | |
| 3832 | −EXPORT_SYMBOL_GPL(tun_wake_queue); | |
| 3833 | − | |
| 3834 | 3737 | module_init(tun_init); |
| 3835 | 3738 | module_exit(tun_cleanup); |
| 3836 | 3739 | MODULE_DESCRIPTION(DRV_DESCRIPTION); |
| @@ -756,7 +756,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, · 755 unmodified lines | ||
| 756 | 756 | u32 frame_sz; |
| 757 | 757 | |
| 758 | 758 | if (skb_shared(skb) || skb_head_is_locked(skb) || |
| 759 | − skb_shinfo(skb)->nr_frags || | |
| 759 | + skb_is_nonlinear(skb) || | |
| 760 | 760 | skb_headroom(skb) < XDP_PACKET_HEADROOM) { |
| 761 | 761 | if (skb_pp_cow_data(rq->page_pool, pskb, XDP_PACKET_HEADROOM)) |
| 762 | 762 | goto drop; |
| @@ -771,7 +771,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, · 8 unmodified lines | ||
| 771 | 771 | xdp_prepare_buff(xdp, skb->head, skb_headroom(skb), |
| 772 | 772 | skb_headlen(skb), true); |
| 773 | 773 | |
| 774 | − if (skb_is_nonlinear(skb)) { | |
| 774 | + if (skb_shinfo(skb)->nr_frags) { | |
| 775 | 775 | skb_shinfo(skb)->xdp_frags_size = skb->data_len; |
| 776 | 776 | xdp_buff_set_frags_flag(xdp); |
| 777 | 777 | } else { |
| @@ -1850,7 +1850,7 @@ static int arp_reduce(struct net_device *dev, struct sk_buff *skb, __be32 vni) · 1849 unmodified lines | ||
| 1850 | 1850 | if (dev->flags & IFF_NOARP) |
| 1851 | 1851 | goto out; |
| 1852 | 1852 | |
| 1853 | − if (!pskb_may_pull(skb, arp_hdr_len(dev))) { | |
| 1853 | + if (!pskb_network_may_pull(skb, arp_hdr_len(dev))) { | |
| 1854 | 1854 | dev_dstats_tx_dropped(dev); |
| 1855 | 1855 | vxlan_vnifilter_count(vxlan, vni, NULL, |
| 1856 | 1856 | VXLAN_VNI_STATS_TX_DROPS, 0); |
| @@ -2111,7 +2111,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 254 unmodified lines | ||
| 2111 | 2111 | { |
| 2112 | 2112 | struct iphdr *pip; |
| 2113 | 2113 | |
| 2114 | − if (!pskb_may_pull(skb, sizeof(struct iphdr))) | |
| 2114 | + if (!pskb_network_may_pull(skb, sizeof(struct iphdr))) | |
| 2115 | 2115 | return false; |
| 2116 | 2116 | pip = ip_hdr(skb); |
| 2117 | 2117 | n = neigh_lookup(&arp_tbl, &pip->daddr, dev); |
| @@ -2137,7 +2137,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 19 unmodified lines | ||
| 2137 | 2137 | */ |
| 2138 | 2138 | if (!ipv6_mod_enabled()) |
| 2139 | 2139 | return false; |
| 2140 | − if (!pskb_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 2140 | + if (!pskb_network_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 2141 | 2141 | return false; |
| 2142 | 2142 | pip6 = ipv6_hdr(skb); |
| 2143 | 2143 | n = neigh_lookup(&nd_tbl, &pip6->daddr, dev); |
| @@ -2159,13 +2159,19 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 15 unmodified lines | ||
| 2159 | 2159 | } |
| 2160 | 2160 | |
| 2161 | 2161 | if (n) { |
| 2162 | + u8 haddr[ETH_ALEN]; | |
| 2162 | 2163 | bool diff; |
| 2163 | 2164 | |
| 2164 | − diff = !ether_addr_equal(eth_hdr(skb)->h_dest, n->ha); | |
| 2165 | + neigh_ha_snapshot(haddr, n, dev); | |
| 2166 | + diff = !ether_addr_equal_unaligned(eth_hdr(skb)->h_dest, haddr); | |
| 2165 | 2167 | if (diff) { |
| 2168 | + if (skb_cow_head(skb, 0)) { | |
| 2169 | + neigh_release(n); | |
| 2170 | + return false; | |
| 2171 | + } | |
| 2166 | 2172 | memcpy(eth_hdr(skb)->h_source, eth_hdr(skb)->h_dest, |
| 2167 | 2173 | dev->addr_len); |
| 2168 | − memcpy(eth_hdr(skb)->h_dest, n->ha, dev->addr_len); | |
| 2174 | + memcpy(eth_hdr(skb)->h_dest, haddr, dev->addr_len); | |
| 2169 | 2175 | } |
| 2170 | 2176 | neigh_release(n); |
| 2171 | 2177 | return diff; |
| @@ -2757,8 +2763,8 @@ static netdev_tx_t vxlan_xmit(struct sk_buff *skb, struct net_device *dev) · 585 unmodified lines | ||
| 2757 | 2763 | return arp_reduce(dev, skb, vni); |
| 2758 | 2764 | #if IS_ENABLED(CONFIG_IPV6) |
| 2759 | 2765 | else if (ntohs(eth->h_proto) == ETH_P_IPV6 && |
| 2760 | − pskb_may_pull(skb, sizeof(struct ipv6hdr) + | |
| 2761 | − sizeof(struct nd_msg)) && | |
| 2766 | + pskb_network_may_pull(skb, sizeof(struct ipv6hdr) + | |
| 2767 | + sizeof(struct nd_msg)) && | |
| 2762 | 2768 | ipv6_hdr(skb)->nexthdr == IPPROTO_ICMPV6) { |
| 2763 | 2769 | struct nd_msg *m = (struct nd_msg *)(ipv6_hdr(skb) + 1); |
| 2764 | 2770 | |
| @@ -2796,6 +2802,7 @@ static netdev_tx_t vxlan_xmit(struct sk_buff *skb, struct net_device *dev) · 31 unmodified lines | ||
| 2796 | 2802 | (ntohs(eth->h_proto) == ETH_P_IP || |
| 2797 | 2803 | ntohs(eth->h_proto) == ETH_P_IPV6)) { |
| 2798 | 2804 | did_rsc = route_shortcircuit(dev, skb); |
| 2805 | + eth = eth_hdr(skb); | |
| 2799 | 2806 | if (did_rsc) |
| 2800 | 2807 | f = vxlan_find_mac_tx(vxlan, eth->h_dest, vni); |
| 2801 | 2808 | } |
| @@ -1631,7 +1631,7 @@ struct vxlan_mdb_entry *vxlan_mdb_entry_skb_get(struct vxlan_dev *vxlan, · 1630 unmodified lines | ||
| 1631 | 1631 | |
| 1632 | 1632 | switch (skb->protocol) { |
| 1633 | 1633 | case htons(ETH_P_IP): |
| 1634 | − if (!pskb_may_pull(skb, sizeof(struct iphdr))) | |
| 1634 | + if (!pskb_network_may_pull(skb, sizeof(struct iphdr))) | |
| 1635 | 1635 | return NULL; |
| 1636 | 1636 | group.dst.sa.sa_family = AF_INET; |
| 1637 | 1637 | group.dst.sin.sin_addr.s_addr = ip_hdr(skb)->daddr; |
| @@ -1640,7 +1640,7 @@ struct vxlan_mdb_entry *vxlan_mdb_entry_skb_get(struct vxlan_dev *vxlan, · 2 unmodified lines | ||
| 1640 | 1640 | break; |
| 1641 | 1641 | #if IS_ENABLED(CONFIG_IPV6) |
| 1642 | 1642 | case htons(ETH_P_IPV6): |
| 1643 | − if (!pskb_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 1643 | + if (!pskb_network_may_pull(skb, sizeof(struct ipv6hdr))) | |
| 1644 | 1644 | return NULL; |
| 1645 | 1645 | group.dst.sa.sa_family = AF_INET6; |
| 1646 | 1646 | group.dst.sin6.sin6_addr = ipv6_hdr(skb)->daddr; |
| @@ -1544,6 +1544,8 @@ static void ath12k_core_pre_reconfigure_recovery(struct ath12k_base *ab) · 1543 unmodified lines | ||
| 1544 | 1544 | } |
| 1545 | 1545 | |
| 1546 | 1546 | wiphy_unlock(ah->hw->wiphy); |
| 1547 | + | |
| 1548 | + complete(&ah->peer_ml_id_done); | |
| 1547 | 1549 | } |
| 1548 | 1550 | |
| 1549 | 1551 | wake_up(&ab->wmi_ab.tx_credits_wq); |
| @@ -72,6 +72,7 @@ · 71 unmodified lines | ||
| 72 | 72 | |
| 73 | 73 | #define ATH12K_MAX_MLO_PEERS 256 |
| 74 | 74 | #define ATH12K_MLO_PEER_ID_INVALID 0xFFFF |
| 75 | +#define ATH12K_MLO_PEER_ID_PENDING 0xFFFE | |
| 75 | 76 | |
| 76 | 77 | #define ATH12K_INVALID_RSSI_FULL -1 |
| 77 | 78 | #define ATH12K_INVALID_RSSI_EMPTY -128 |
| @@ -793,6 +794,8 @@ struct ath12k_hw { · 715 unmodified lines | ||
| 793 | 794 | enum ath12k_hw_state state; |
| 794 | 795 | bool regd_updated; |
| 795 | 796 | bool use_6ghz_regd; |
| 797 | + bool host_alloc_ml_id; | |
| 798 | + struct completion peer_ml_id_done; | |
| 796 | 799 | |
| 797 | 800 | u8 num_radio; |
| 798 | 801 | |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | |
| 7 | 7 | #include "core.h" |
| 8 | 8 | #include "peer.h" |
| 9 | +#include "dp_peer.h" | |
| 9 | 10 | #include "htc.h" |
| 10 | 11 | #include "dp_htt.h" |
| 11 | 12 | #include "debugfs_htt_stats.h" |
| @@ -575,6 +576,51 @@ exit: · 563 unmodified lines | ||
| 575 | 576 | rcu_read_unlock(); |
| 576 | 577 | } |
| 577 | 578 | |
| 579 | +static void ath12k_dp_htt_mlo_peer_map_handler(struct ath12k_base *ab, | |
| 580 | + struct sk_buff *skb) | |
| 581 | +{ | |
| 582 | + struct htt_resp_msg *resp = (struct htt_resp_msg *)skb->data; | |
| 583 | + struct htt_t2h_mlo_peer_map_event *ev = &resp->mlo_peer_map_ev; | |
| 584 | + u16 raw_peer_id, peer_id, addr_h16; | |
| 585 | + u8 peer_addr[ETH_ALEN]; | |
| 586 | + int ret; | |
| 587 | + | |
| 588 | + if (skb->len < sizeof(*ev)) { | |
| 589 | + ath12k_warn(ab, "unexpected htt mlo peer map event len %u\n", | |
| 590 | + skb->len); | |
| 591 | + return; | |
| 592 | + } | |
| 593 | + | |
| 594 | + raw_peer_id = le32_get_bits(ev->info0, | |
| 595 | + HTT_T2H_MLO_PEER_MAP_INFO0_MLO_PEER_ID); | |
| 596 | + peer_id = raw_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 597 | + | |
| 598 | + addr_h16 = le32_get_bits(ev->info1, | |
| 599 | + HTT_T2H_MLO_PEER_MAP_INFO1_MAC_ADDR_H16); | |
| 600 | + ath12k_dp_get_mac_addr(le32_to_cpu(ev->mac_addr_l32), addr_h16, | |
| 601 | + peer_addr); | |
| 602 | + | |
| 603 | + ath12k_dbg(ab, ATH12K_DBG_DP_HTT, "htt mlo peer map peer %pM id %u\n", | |
| 604 | + peer_addr, peer_id); | |
| 605 | + | |
| 606 | + /* | |
| 607 | + * Fix up the dp_peer entry created with ATH12K_MLO_PEER_ID_PENDING | |
| 608 | + * earlier; on chips with host_alloc_ml_id == false this is the only | |
| 609 | + * point at which the host learns the firmware-assigned ID. Chips | |
| 610 | + * that allocate the ID on the host also receive this event but the | |
| 611 | + * firmware-reported ID matches the host-allocated one, so there is | |
| 612 | + * nothing to fix up. | |
| 613 | + */ | |
| 614 | + if (!ab->hw_params->host_alloc_ml_id) { | |
| 615 | + ret = ath12k_dp_peer_fixup_peer_id(ab, peer_addr, | |
| 616 | + peer_id); | |
| 617 | + if (ret) | |
| 618 | + ath12k_warn(ab, | |
| 619 | + "failed to fix up peer id %u for dp peer %pM: %d\n", | |
| 620 | + peer_id, peer_addr, ret); | |
| 621 | + } | |
| 622 | +} | |
| 623 | + | |
| 578 | 624 | void ath12k_dp_htt_htc_t2h_msg_handler(struct ath12k_base *ab, |
| 579 | 625 | struct sk_buff *skb) |
| 580 | 626 | { |
| @@ -659,6 +705,9 @@ void ath12k_dp_htt_htc_t2h_msg_handler(struct ath12k_base *ab, · 78 unmodified lines | ||
| 659 | 705 | case HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND: |
| 660 | 706 | ath12k_htt_mlo_offset_event_handler(ab, skb); |
| 661 | 707 | break; |
| 708 | + case HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP: | |
| 709 | + ath12k_dp_htt_mlo_peer_map_handler(ab, skb); | |
| 710 | + break; | |
| 662 | 711 | default: |
| 663 | 712 | ath12k_dbg(ab, ATH12K_DBG_DP_HTT, "dp_htt event %d not handled\n", |
| 664 | 713 | type); |
| @@ -930,6 +930,7 @@ enum htt_t2h_msg_type { · 929 unmodified lines | ||
| 930 | 930 | HTT_T2H_MSG_TYPE_EXT_STATS_CONF = 0x1c, |
| 931 | 931 | HTT_T2H_MSG_TYPE_BKPRESSURE_EVENT_IND = 0x24, |
| 932 | 932 | HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND = 0x28, |
| 933 | + HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP = 0x29, | |
| 933 | 934 | HTT_T2H_MSG_TYPE_PEER_MAP3 = 0x2b, |
| 934 | 935 | HTT_T2H_MSG_TYPE_VDEV_TXRX_STATS_PERIODIC_IND = 0x2c, |
| 935 | 936 | }; |
| @@ -974,11 +975,22 @@ struct htt_t2h_peer_unmap_event { · 38 unmodified lines | ||
| 974 | 975 | __le32 info1; |
| 975 | 976 | } __packed; |
| 976 | 977 | |
| 978 | +#define HTT_T2H_MLO_PEER_MAP_INFO0_MLO_PEER_ID GENMASK(23, 8) | |
| 979 | +#define HTT_T2H_MLO_PEER_MAP_INFO1_MAC_ADDR_H16 GENMASK(15, 0) | |
| 980 | + | |
| 981 | +struct htt_t2h_mlo_peer_map_event { | |
| 982 | + __le32 info0; | |
| 983 | + __le32 mac_addr_l32; | |
| 984 | + __le32 info1; | |
| 985 | + __le32 reserved[5]; | |
| 986 | +} __packed; | |
| 987 | + | |
| 977 | 988 | struct htt_resp_msg { |
| 978 | 989 | union { |
| 979 | 990 | struct htt_t2h_version_conf_msg version_msg; |
| 980 | 991 | struct htt_t2h_peer_map_event peer_map_ev; |
| 981 | 992 | struct htt_t2h_peer_unmap_event peer_unmap_ev; |
| 993 | + struct htt_t2h_mlo_peer_map_event mlo_peer_map_ev; | |
| 982 | 994 | }; |
| 983 | 995 | } __packed; |
| 984 | 996 | |
| @@ -475,7 +475,9 @@ int ath12k_dp_peer_create(struct ath12k_dp_hw *dp_hw, u8 *addr, · 474 unmodified lines | ||
| 475 | 475 | dp_peer->is_mlo = params->is_mlo; |
| 476 | 476 | |
| 477 | 477 | /* |
| 478 | − * For MLO client, the host assigns the ML peer ID, so set peer_id in dp_peer | |
| 478 | + * For MLO client, the ML peer ID, either known or PENDING, needs to be | |
| 479 | + * initialized here since the following logic depends on it. | |
| 480 | + * | |
| 479 | 481 | * For non-MLO client, host gets link peer ID from firmware and will be |
| 480 | 482 | * assigned at the time of link peer creation |
| 481 | 483 | */ |
| @@ -491,13 +493,17 @@ int ath12k_dp_peer_create(struct ath12k_dp_hw *dp_hw, u8 *addr, · 9 unmodified lines | ||
| 491 | 493 | list_add(&dp_peer->list, &dp_hw->dp_peers_list); |
| 492 | 494 | |
| 493 | 495 | /* |
| 494 | − * For MLO client, the peer_id for ath12k_dp_peer is allocated by host | |
| 495 | − * and that peer_id is known at this point, and hence this ath12k_dp_peer | |
| 496 | − * can be added to the RCU table using the peer_id. | |
| 497 | − * For non-MLO client, this addition to RCU table shall be done at the | |
| 498 | − * time of assignment of ath12k_dp_link_peer to ath12k_dp_peer. | |
| 496 | + * For an MLO client whose ML peer ID is allocated by the host, the | |
| 497 | + * peer_id is known here and the dp_peer can be added to the RCU | |
| 498 | + * table using it. For an MLO client on chips where the firmware | |
| 499 | + * allocates the ID, peer_id is ATH12K_MLO_PEER_ID_PENDING and the | |
| 500 | + * RCU table publish is deferred to the | |
| 501 | + * HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP handler. For a non-MLO client | |
| 502 | + * the publish happens later, at the time of assignment of | |
| 503 | + * ath12k_dp_link_peer to ath12k_dp_peer. | |
| 499 | 504 | */ |
| 500 | − if (dp_peer->is_mlo) | |
| 505 | + if (dp_peer->is_mlo && | |
| 506 | + dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | |
| 501 | 507 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], dp_peer); |
| 502 | 508 | |
| 503 | 509 | spin_unlock_bh(&dp_hw->peer_lock); |
| @@ -518,7 +524,8 @@ void ath12k_dp_peer_delete(struct ath12k_dp_hw *dp_hw, u8 *addr, · 14 unmodified lines | ||
| 518 | 524 | return; |
| 519 | 525 | } |
| 520 | 526 | |
| 521 | − if (dp_peer->is_mlo) | |
| 527 | + if (dp_peer->is_mlo && | |
| 528 | + dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | |
| 522 | 529 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], NULL); |
| 523 | 530 | |
| 524 | 531 | list_del(&dp_peer->list); |
| @@ -695,3 +702,55 @@ void ath12k_dp_link_peer_reset_rx_stats(struct ath12k_dp *dp, const u8 *addr) · 170 unmodified lines | ||
| 695 | 702 | if (rx_stats) |
| 696 | 703 | memset(rx_stats, 0, sizeof(*rx_stats)); |
| 697 | 704 | } |
| 705 | + | |
| 706 | +int ath12k_dp_peer_fixup_peer_id(struct ath12k_base *ab, | |
| 707 | + const u8 *peer_addr, u16 peer_id) | |
| 708 | +{ | |
| 709 | + struct ath12k_dp_link_peer *link_peer; | |
| 710 | + struct ath12k_dp_peer *dp_peer = NULL; | |
| 711 | + struct ath12k_hw_group *ag = ab->ag; | |
| 712 | + struct ath12k_dp_hw *dp_hw = NULL; | |
| 713 | + struct ath12k_hw *ah; | |
| 714 | + int i; | |
| 715 | + | |
| 716 | + if (peer_id >= (ATH12K_PEER_ML_ID_VALID | ATH12K_MAX_MLO_PEERS)) | |
| 717 | + return -EINVAL; | |
| 718 | + | |
| 719 | + for (i = 0; i < ag->num_hw; i++) { | |
| 720 | + ah = ag->ah[i]; | |
| 721 | + if (!ah) | |
| 722 | + continue; | |
| 723 | + | |
| 724 | + spin_lock_bh(&ah->dp_hw.peer_lock); | |
| 725 | + dp_peer = ath12k_dp_peer_find_by_addr(&ah->dp_hw, | |
| 726 | + (u8 *)peer_addr); | |
| 727 | + if (dp_peer) { | |
| 728 | + dp_hw = &ah->dp_hw; | |
| 729 | + break; | |
| 730 | + } | |
| 731 | + spin_unlock_bh(&ah->dp_hw.peer_lock); | |
| 732 | + } | |
| 733 | + | |
| 734 | + if (!dp_peer) | |
| 735 | + return -ENOENT; | |
| 736 | + | |
| 737 | + /* dp_hw->peer_lock is held */ | |
| 738 | + | |
| 739 | + dp_peer->peer_id = peer_id; | |
| 740 | + rcu_assign_pointer(dp_hw->dp_peers[peer_id], dp_peer); | |
| 741 | + | |
| 742 | + for (i = 0; i < ATH12K_NUM_MAX_LINKS; i++) { | |
| 743 | + link_peer = rcu_dereference_protected(dp_peer->link_peers[i], | |
| 744 | + lockdep_is_held(&dp_hw->peer_lock)); | |
| 745 | + if (link_peer) | |
| 746 | + link_peer->ml_id = peer_id; | |
| 747 | + } | |
| 748 | + | |
| 749 | + ath12k_sta_to_ahsta(dp_peer->sta)->ml_peer_id = peer_id; | |
| 750 | + | |
| 751 | + spin_unlock_bh(&dp_hw->peer_lock); | |
| 752 | + | |
| 753 | + complete(&ah->peer_ml_id_done); | |
| 754 | + | |
| 755 | + return 0; | |
| 756 | +} | |
| @@ -181,4 +181,6 @@ struct ath12k_dp_peer *ath12k_dp_peer_find_by_peerid(struct ath12k_pdev_dp *dp_p · 180 unmodified lines | ||
| 181 | 181 | struct ath12k_dp_link_peer * |
| 182 | 182 | ath12k_dp_link_peer_find_by_peerid(struct ath12k_pdev_dp *dp_pdev, u16 peer_id); |
| 183 | 183 | void ath12k_dp_link_peer_free(struct ath12k_dp_link_peer *peer); |
| 184 | +int ath12k_dp_peer_fixup_peer_id(struct ath12k_base *ab, const u8 *peer_addr, | |
| 185 | + u16 peer_id); | |
| 184 | 186 | #endif |
| @@ -236,6 +236,8 @@ struct ath12k_hw_params { · 235 unmodified lines | ||
| 236 | 236 | u32 max_client_dbs; |
| 237 | 237 | u32 max_client_dbs_sbs; |
| 238 | 238 | } client; |
| 239 | + | |
| 240 | + bool host_alloc_ml_id; | |
| 239 | 241 | }; |
| 240 | 242 | |
| 241 | 243 | struct ath12k_hw_ops { |
| @@ -1282,13 +1282,17 @@ void ath12k_mac_dp_peer_cleanup(struct ath12k_hw *ah) · 1281 unmodified lines | ||
| 1282 | 1282 | struct ath12k_dp_peer *dp_peer, *tmp; |
| 1283 | 1283 | struct ath12k_dp_hw *dp_hw = &ah->dp_hw; |
| 1284 | 1284 | |
| 1285 | + lockdep_assert_wiphy(ah->hw->wiphy); | |
| 1286 | + | |
| 1285 | 1287 | INIT_LIST_HEAD(&peers); |
| 1286 | 1288 | |
| 1287 | 1289 | spin_lock_bh(&dp_hw->peer_lock); |
| 1288 | 1290 | list_for_each_entry_safe(dp_peer, tmp, &dp_hw->dp_peers_list, list) { |
| 1289 | 1291 | if (dp_peer->is_mlo) { |
| 1290 | − rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], NULL); | |
| 1291 | − clear_bit(dp_peer->peer_id, ah->free_ml_peer_id_map); | |
| 1292 | + if (dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | |
| 1293 | + rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], | |
| 1294 | + NULL); | |
| 1295 | + ath12k_peer_ml_free(ah, ath12k_sta_to_ahsta(dp_peer->sta)); | |
| 1292 | 1296 | } |
| 1293 | 1297 | |
| 1294 | 1298 | list_move(&dp_peer->list, &peers); |
| @@ -3531,11 +3535,16 @@ static void ath12k_peer_assoc_h_mlo(struct ath12k_link_sta *arsta, · 2236 unmodified lines | ||
| 3531 | 3535 | struct ath12k_sta *ahsta = arsta->ahsta; |
| 3532 | 3536 | struct ath12k_link_sta *arsta_p; |
| 3533 | 3537 | struct ath12k_link_vif *arvif; |
| 3538 | + struct ath12k_hw *ah = arsta->arvif->ar->ah; | |
| 3534 | 3539 | unsigned long links; |
| 3535 | 3540 | u8 link_id; |
| 3536 | 3541 | int i; |
| 3537 | 3542 | |
| 3538 | − if (!sta->mlo || ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) | |
| 3543 | + if (!sta->mlo) | |
| 3544 | + return; | |
| 3545 | + | |
| 3546 | + if (ah->host_alloc_ml_id && | |
| 3547 | + ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) | |
| 3539 | 3548 | return; |
| 3540 | 3549 | |
| 3541 | 3550 | ml->enabled = true; |
| @@ -3543,12 +3552,25 @@ static void ath12k_peer_assoc_h_mlo(struct ath12k_link_sta *arsta, · 1 unmodified line | ||
| 3543 | 3552 | |
| 3544 | 3553 | /* For now considering the primary umac based on assoc link */ |
| 3545 | 3554 | ml->primary_umac = arsta->is_assoc_link; |
| 3546 | − ml->peer_id_valid = true; | |
| 3555 | + /* | |
| 3556 | + * Only chips that allocate the MLD peer ID on the host send a valid | |
| 3557 | + * ml_peer_id in WMI_PEER_ASSOC_CMDID. For chips where the firmware | |
| 3558 | + * picks the ID, leave peer_id_valid false to avoid unexpected issues. | |
| 3559 | + */ | |
| 3560 | + ml->peer_id_valid = ah->host_alloc_ml_id; | |
| 3547 | 3561 | ml->logical_link_idx_valid = true; |
| 3548 | 3562 | |
| 3549 | 3563 | ether_addr_copy(ml->mld_addr, sta->addr); |
| 3550 | 3564 | ml->logical_link_idx = arsta->link_idx; |
| 3551 | − ml->ml_peer_id = ahsta->ml_peer_id; | |
| 3565 | + /* | |
| 3566 | + * WMI_MLO_PEER_ASSOC_PARAMS expects the raw ML peer ID without | |
| 3567 | + * the host-side ATH12K_PEER_ML_ID_VALID bookkeeping bit. For chips | |
| 3568 | + * where the firmware allocates the ID, the field is unused (the | |
| 3569 | + * firmware always allocates regardless of the value here); send 0 | |
| 3570 | + * to make that intent explicit. | |
| 3571 | + */ | |
| 3572 | + ml->ml_peer_id = ah->host_alloc_ml_id ? | |
| 3573 | + (ahsta->ml_peer_id & ~ATH12K_PEER_ML_ID_VALID) : 0; | |
| 3552 | 3574 | ml->ieee_link_id = arsta->link_id; |
| 3553 | 3575 | ml->num_partner_links = 0; |
| 3554 | 3576 | ml->eml_cap = sta->eml_cap; |
| @@ -3595,8 +3617,6 @@ static void ath12k_peer_assoc_prepare(struct ath12k *ar, · 40 unmodified lines | ||
| 3595 | 3617 | |
| 3596 | 3618 | memset(arg, 0, sizeof(*arg)); |
| 3597 | 3619 | |
| 3598 | − reinit_completion(&ar->peer_assoc_done); | |
| 3599 | − | |
| 3600 | 3620 | arg->peer_new_assoc = !reassoc; |
| 3601 | 3621 | ath12k_peer_assoc_h_basic(ar, arvif, arsta, arg); |
| 3602 | 3622 | ath12k_peer_assoc_h_crypto(ar, arvif, arsta, arg); |
| @@ -3836,6 +3856,52 @@ static u32 ath12k_mac_ieee80211_sta_bw_to_wmi(struct ath12k *ar, · 233 unmodified lines | ||
| 3836 | 3856 | return bw; |
| 3837 | 3857 | } |
| 3838 | 3858 | |
| 3859 | +static int ath12k_mac_peer_assoc(struct ath12k *ar, | |
| 3860 | + struct ath12k_wmi_peer_assoc_arg *peer_arg) | |
| 3861 | +{ | |
| 3862 | + struct ath12k_hw *ah = ath12k_ar_to_ah(ar); | |
| 3863 | + int ret; | |
| 3864 | + | |
| 3865 | + reinit_completion(&ar->peer_assoc_done); | |
| 3866 | + reinit_completion(&ah->peer_ml_id_done); | |
| 3867 | + | |
| 3868 | + ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 3869 | + if (ret) { | |
| 3870 | + ath12k_warn(ar->ab, "failed to run peer assoc for %pM vdev %i: %d\n", | |
| 3871 | + peer_arg->peer_mac, peer_arg->vdev_id, ret); | |
| 3872 | + return ret; | |
| 3873 | + } | |
| 3874 | + | |
| 3875 | + if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | |
| 3876 | + ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 3877 | + peer_arg->peer_mac, peer_arg->vdev_id); | |
| 3878 | + return -ETIMEDOUT; | |
| 3879 | + } | |
| 3880 | + | |
| 3881 | + /* | |
| 3882 | + * For devices where the firmware allocates the MLD peer ID, the host | |
| 3883 | + * learns the real ID only from the MLO_RX_PEER_MAP HTT event, which is | |
| 3884 | + * handled in a softirq (BH workqueue) context that cannot take the | |
| 3885 | + * wiphy lock. Block here, while still holding the wiphy lock, until | |
| 3886 | + * that event has fixed up the ID. This serialises the fixup against | |
| 3887 | + * all other wiphy-locked ml_peer_id accesses. | |
| 3888 | + * | |
| 3889 | + * The firmware sends the event only once, in response to the assoc-link | |
| 3890 | + * peer assoc, so block only for that link. | |
| 3891 | + */ | |
| 3892 | + if (!ah->host_alloc_ml_id && | |
| 3893 | + peer_arg->is_assoc && | |
| 3894 | + peer_arg->ml.enabled && | |
| 3895 | + peer_arg->ml.assoc_link && | |
| 3896 | + !wait_for_completion_timeout(&ah->peer_ml_id_done, 1 * HZ)) { | |
| 3897 | + ath12k_warn(ar->ab, "failed to get MLO peer map event for %pM vdev %i\n", | |
| 3898 | + peer_arg->peer_mac, peer_arg->vdev_id); | |
| 3899 | + return -ETIMEDOUT; | |
| 3900 | + } | |
| 3901 | + | |
| 3902 | + return 0; | |
| 3903 | +} | |
| 3904 | + | |
| 3839 | 3905 | static void ath12k_bss_assoc(struct ath12k *ar, |
| 3840 | 3906 | struct ath12k_link_vif *arvif, |
| 3841 | 3907 | struct ieee80211_bss_conf *bss_conf) |
| @@ -3916,18 +3982,10 @@ static void ath12k_bss_assoc(struct ath12k *ar, · 74 unmodified lines | ||
| 3916 | 3982 | } |
| 3917 | 3983 | |
| 3918 | 3984 | peer_arg->is_assoc = true; |
| 3919 | − ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 3920 | − if (ret) { | |
| 3921 | − ath12k_warn(ar->ab, "failed to run peer assoc for %pM vdev %i: %d\n", | |
| 3922 | − bss_conf->bssid, arvif->vdev_id, ret); | |
| 3923 | − return; | |
| 3924 | − } | |
| 3925 | 3985 | |
| 3926 | − if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | |
| 3927 | − ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 3928 | − bss_conf->bssid, arvif->vdev_id); | |
| 3986 | + ret = ath12k_mac_peer_assoc(ar, peer_arg); | |
| 3987 | + if (ret) | |
| 3929 | 3988 | return; |
| 3930 | − } | |
| 3931 | 3989 | |
| 3932 | 3990 | ret = ath12k_setup_peer_smps(ar, arvif, bss_conf->bssid, |
| 3933 | 3991 | &link_sta->ht_cap, &link_sta->he_6ghz_capa); |
| @@ -6481,18 +6539,10 @@ static int ath12k_mac_station_assoc(struct ath12k *ar, · 2547 unmodified lines | ||
| 6481 | 6539 | } |
| 6482 | 6540 | |
| 6483 | 6541 | peer_arg->is_assoc = true; |
| 6484 | − ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 6485 | − if (ret) { | |
| 6486 | − ath12k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n", | |
| 6487 | − arsta->addr, arvif->vdev_id, ret); | |
| 6488 | − return ret; | |
| 6489 | − } | |
| 6490 | 6542 | |
| 6491 | − if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | |
| 6492 | − ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 6493 | − arsta->addr, arvif->vdev_id); | |
| 6494 | − return -ETIMEDOUT; | |
| 6495 | − } | |
| 6543 | + ret = ath12k_mac_peer_assoc(ar, peer_arg); | |
| 6544 | + if (ret) | |
| 6545 | + return ret; | |
| 6496 | 6546 | |
| 6497 | 6547 | num_vht_rates = ath12k_mac_bitrate_mask_num_vht_rates(ar, band, mask); |
| 6498 | 6548 | num_he_rates = ath12k_mac_bitrate_mask_num_he_rates(ar, band, mask); |
| @@ -6841,14 +6891,8 @@ static void ath12k_sta_rc_update_wk(struct wiphy *wiphy, struct wiphy_work *wk) · 342 unmodified lines | ||
| 6841 | 6891 | peer_arg, true); |
| 6842 | 6892 | |
| 6843 | 6893 | peer_arg->is_assoc = false; |
| 6844 | − err = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | |
| 6845 | − if (err) | |
| 6846 | − ath12k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n", | |
| 6847 | − arsta->addr, arvif->vdev_id, err); | |
| 6848 | 6894 | |
| 6849 | − if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) | |
| 6850 | − ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | |
| 6851 | − arsta->addr, arvif->vdev_id); | |
| 6895 | + ath12k_mac_peer_assoc(ar, peer_arg); | |
| 6852 | 6896 | } |
| 6853 | 6897 | } |
| 6854 | 6898 | } |
| @@ -7266,10 +7310,8 @@ static void ath12k_mac_ml_station_remove(struct ath12k_vif *ahvif, · 411 unmodified lines | ||
| 7266 | 7310 | ath12k_mac_free_unassign_link_sta(ah, ahsta, link_id); |
| 7267 | 7311 | } |
| 7268 | 7312 | |
| 7269 | − if (sta->mlo) { | |
| 7270 | − clear_bit(ahsta->ml_peer_id, ah->free_ml_peer_id_map); | |
| 7271 | − ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 7272 | − } | |
| 7313 | + if (sta->mlo) | |
| 7314 | + ath12k_peer_ml_free(ah, ahsta); | |
| 7273 | 7315 | } |
| 7274 | 7316 | |
| 7275 | 7317 | static int ath12k_mac_handle_link_sta_state(struct ieee80211_hw *hw, |
| @@ -7733,15 +7775,23 @@ int ath12k_mac_op_sta_state(struct ieee80211_hw *hw, · 457 unmodified lines | ||
| 7733 | 7775 | /* ML sta */ |
| 7734 | 7776 | if (sta->mlo && !ahsta->links_map && |
| 7735 | 7777 | (hweight16(sta->valid_links) == 1)) { |
| 7736 | − ahsta->ml_peer_id = ath12k_peer_ml_alloc(ah); | |
| 7737 | − if (ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) { | |
| 7738 | − ath12k_hw_warn(ah, "unable to allocate ML peer id for sta %pM", | |
| 7739 | − sta->addr); | |
| 7740 | − goto exit; | |
| 7778 | + if (ah->host_alloc_ml_id) { | |
| 7779 | + ahsta->ml_peer_id = ath12k_peer_ml_alloc(ah); | |
| 7780 | + if (ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) { | |
| 7781 | + ath12k_hw_warn(ah, "unable to allocate ML peer id for sta %pM", | |
| 7782 | + sta->addr); | |
| 7783 | + goto exit; | |
| 7784 | + } | |
| 7785 | + } else { | |
| 7786 | + /* | |
| 7787 | + * firmware allocates the ML peer ID and notifies | |
| 7788 | + * the host via HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP | |
| 7789 | + */ | |
| 7790 | + ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_PENDING; | |
| 7741 | 7791 | } |
| 7742 | 7792 | |
| 7743 | 7793 | dp_params.is_mlo = true; |
| 7744 | − dp_params.peer_id = ahsta->ml_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 7794 | + dp_params.peer_id = ahsta->ml_peer_id; | |
| 7745 | 7795 | } |
| 7746 | 7796 | |
| 7747 | 7797 | dp_params.sta = sta; |
| @@ -7878,10 +7928,8 @@ int ath12k_mac_op_sta_state(struct ieee80211_hw *hw, · 130 unmodified lines | ||
| 7878 | 7928 | peer_delete: |
| 7879 | 7929 | ath12k_dp_peer_delete(&ah->dp_hw, sta->addr, sta); |
| 7880 | 7930 | ml_peer_id_clear: |
| 7881 | − if (sta->mlo) { | |
| 7882 | − clear_bit(ahsta->ml_peer_id, ah->free_ml_peer_id_map); | |
| 7883 | − ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 7884 | − } | |
| 7931 | + if (sta->mlo) | |
| 7932 | + ath12k_peer_ml_free(ah, ahsta); | |
| 7885 | 7933 | exit: |
| 7886 | 7934 | /* update the state if everything went well */ |
| 7887 | 7935 | if (!ret) |
| @@ -15310,6 +15358,7 @@ static struct ath12k_hw *ath12k_mac_hw_allocate(struct ath12k_hw_group *ag, · 7422 unmodified lines | ||
| 15310 | 15358 | ah->num_radio = num_pdev_map; |
| 15311 | 15359 | |
| 15312 | 15360 | mutex_init(&ah->hw_mutex); |
| 15361 | + init_completion(&ah->peer_ml_id_done); | |
| 15313 | 15362 | |
| 15314 | 15363 | spin_lock_init(&ah->dp_hw.peer_lock); |
| 15315 | 15364 | INIT_LIST_HEAD(&ah->dp_hw.dp_peers_list); |
| @@ -15384,8 +15433,9 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 68 unmodified lines | ||
| 15384 | 15433 | int mac_id, device_id, total_radio, num_hw; |
| 15385 | 15434 | struct ath12k_base *ab; |
| 15386 | 15435 | struct ath12k_hw *ah; |
| 15387 | − int ret, i, j; | |
| 15436 | + bool conf = false; | |
| 15388 | 15437 | u8 radio_per_hw; |
| 15438 | + int ret, i, j; | |
| 15389 | 15439 | |
| 15390 | 15440 | total_radio = 0; |
| 15391 | 15441 | for (i = 0; i < ag->num_devices; i++) { |
| @@ -15425,6 +15475,20 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 33 unmodified lines | ||
| 15425 | 15475 | } |
| 15426 | 15476 | |
| 15427 | 15477 | ab = ag->ab[device_id]; |
| 15478 | + | |
| 15479 | + /* | |
| 15480 | + * the assumption is all devices within an ah | |
| 15481 | + * share the same host_alloc_ml_id configuration | |
| 15482 | + */ | |
| 15483 | + if (j == 0) { | |
| 15484 | + conf = ab->hw_params->host_alloc_ml_id; | |
| 15485 | + } else if (conf != ab->hw_params->host_alloc_ml_id) { | |
| 15486 | + ath12k_warn(ab, "inconsistent ML ID config within ah, device 0 uses %s allocated ID, while device %u doesn't\n", | |
| 15487 | + conf ? "host" : "firmware", device_id); | |
| 15488 | + ret = -EINVAL; | |
| 15489 | + goto err; | |
| 15490 | + } | |
| 15491 | + | |
| 15428 | 15492 | pdev_map[j].ab = ab; |
| 15429 | 15493 | pdev_map[j].pdev_idx = mac_id; |
| 15430 | 15494 | mac_id++; |
| @@ -15449,6 +15513,7 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 18 unmodified lines | ||
| 15449 | 15513 | } |
| 15450 | 15514 | |
| 15451 | 15515 | ah->dev = ab->dev; |
| 15516 | + ah->host_alloc_ml_id = conf; | |
| 15452 | 15517 | |
| 15453 | 15518 | ag->ah[i] = ah; |
| 15454 | 15519 | ag->num_hw++; |
| @@ -230,7 +230,16 @@ int ath12k_peer_create(struct ath12k *ar, struct ath12k_link_vif *arvif, · 229 unmodified lines | ||
| 230 | 230 | /* Fill ML info into created peer */ |
| 231 | 231 | if (sta->mlo) { |
| 232 | 232 | ml_peer_id = ahsta->ml_peer_id; |
| 233 | − peer->ml_id = ml_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 233 | + /* | |
| 234 | + * For chips where firmware allocates the ML peer ID, | |
| 235 | + * ml_peer_id is ATH12K_MLO_PEER_ID_PENDING here. The | |
| 236 | + * MLO_RX_PEER_MAP HTT event handler fixes up | |
| 237 | + * peer->ml_id once the ID is known. | |
| 238 | + */ | |
| 239 | + if (ml_peer_id == ATH12K_MLO_PEER_ID_PENDING) | |
| 240 | + peer->ml_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 241 | + else | |
| 242 | + peer->ml_id = ml_peer_id; | |
| 234 | 243 | ether_addr_copy(peer->ml_addr, sta->addr); |
| 235 | 244 | |
| 236 | 245 | /* the assoc link is considered primary for now */ |
| @@ -276,9 +285,25 @@ u16 ath12k_peer_ml_alloc(struct ath12k_hw *ah) · 39 unmodified lines | ||
| 276 | 285 | } |
| 277 | 286 | |
| 278 | 287 | if (ml_peer_id == ATH12K_MAX_MLO_PEERS) |
| 279 | − ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 288 | + return ATH12K_MLO_PEER_ID_INVALID; | |
| 280 | 289 | |
| 281 | − return ml_peer_id; | |
| 290 | + return ml_peer_id | ATH12K_PEER_ML_ID_VALID; | |
| 291 | +} | |
| 292 | + | |
| 293 | +void ath12k_peer_ml_free(struct ath12k_hw *ah, struct ath12k_sta *ahsta) | |
| 294 | +{ | |
| 295 | + lockdep_assert_wiphy(ah->hw->wiphy); | |
| 296 | + | |
| 297 | + /* | |
| 298 | + * Only devices that allocate the ID on the host own a slot in | |
| 299 | + * free_ml_peer_id_map. | |
| 300 | + */ | |
| 301 | + if (ah->host_alloc_ml_id && | |
| 302 | + (ahsta->ml_peer_id < | |
| 303 | + (ATH12K_MAX_MLO_PEERS | ATH12K_PEER_ML_ID_VALID))) | |
| 304 | + clear_bit(ahsta->ml_peer_id & ~ATH12K_PEER_ML_ID_VALID, | |
| 305 | + ah->free_ml_peer_id_map); | |
| 306 | + ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | |
| 282 | 307 | } |
| 283 | 308 | |
| 284 | 309 | int ath12k_peer_mlo_link_peers_delete(struct ath12k_vif *ahvif, struct ath12k_sta *ahsta) |
| @@ -26,4 +26,5 @@ int ath12k_link_sta_rhash_add(struct ath12k_base *ab, struct ath12k_link_sta *ar · 25 unmodified lines | ||
| 26 | 26 | struct ath12k_link_sta *ath12k_link_sta_find_by_addr(struct ath12k_base *ab, |
| 27 | 27 | const u8 *addr); |
| 28 | 28 | u16 ath12k_peer_ml_alloc(struct ath12k_hw *ah); |
| 29 | +void ath12k_peer_ml_free(struct ath12k_hw *ah, struct ath12k_sta *ahsta); | |
| 29 | 30 | #endif /* _PEER_H_ */ |
| @@ -442,6 +442,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 441 unmodified lines | ||
| 442 | 442 | .max_client_dbs = 128, |
| 443 | 443 | .max_client_dbs_sbs = 128, |
| 444 | 444 | }, |
| 445 | + | |
| 446 | + .host_alloc_ml_id = true, | |
| 445 | 447 | }, |
| 446 | 448 | { |
| 447 | 449 | .name = "wcn7850 hw2.0", |
| @@ -533,6 +535,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 85 unmodified lines | ||
| 533 | 535 | .max_client_dbs = 128, |
| 534 | 536 | .max_client_dbs_sbs = 128, |
| 535 | 537 | }, |
| 538 | + | |
| 539 | + .host_alloc_ml_id = false, | |
| 536 | 540 | }, |
| 537 | 541 | { |
| 538 | 542 | .name = "qcn9274 hw2.0", |
| @@ -620,6 +624,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 81 unmodified lines | ||
| 620 | 624 | .max_client_dbs = 128, |
| 621 | 625 | .max_client_dbs_sbs = 128, |
| 622 | 626 | }, |
| 627 | + | |
| 628 | + .host_alloc_ml_id = true, | |
| 623 | 629 | }, |
| 624 | 630 | { |
| 625 | 631 | .name = "ipq5332 hw1.0", |
| @@ -700,6 +706,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 74 unmodified lines | ||
| 700 | 706 | .max_client_dbs = 128, |
| 701 | 707 | .max_client_dbs_sbs = 128, |
| 702 | 708 | }, |
| 709 | + | |
| 710 | + .host_alloc_ml_id = true, | |
| 703 | 711 | }, |
| 704 | 712 | { |
| 705 | 713 | .name = "qcc2072 hw1.0", |
| @@ -792,6 +800,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 86 unmodified lines | ||
| 792 | 800 | .max_client_dbs = 128, |
| 793 | 801 | .max_client_dbs_sbs = 128, |
| 794 | 802 | }, |
| 803 | + | |
| 804 | + .host_alloc_ml_id = false, | |
| 795 | 805 | }, |
| 796 | 806 | { |
| 797 | 807 | .name = "ipq5424 hw1.0", |
| @@ -876,6 +886,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 78 unmodified lines | ||
| 876 | 886 | .max_client_dbs = 128, |
| 877 | 887 | .max_client_dbs_sbs = 128, |
| 878 | 888 | }, |
| 889 | + | |
| 890 | + .host_alloc_ml_id = true, | |
| 879 | 891 | }, |
| 880 | 892 | }; |
| 881 | 893 | |
| @@ -2179,8 +2179,8 @@ il_remove_station(struct il_priv *il, const u8 sta_id, const u8 * addr) · 2178 unmodified lines | ||
| 2179 | 2179 | il->stations[sta_id].used &= ~IL_STA_DRIVER_ACTIVE; |
| 2180 | 2180 | |
| 2181 | 2181 | il->num_stations--; |
| 2182 | − | |
| 2183 | − BUG_ON(il->num_stations < 0); | |
| 2182 | + if (WARN_ON(il->num_stations < 0)) | |
| 2183 | + il->num_stations = 0; | |
| 2184 | 2184 | |
| 2185 | 2185 | spin_unlock_irqrestore(&il->sta_lock, flags); |
| 2186 | 2186 | |
| @@ -2328,7 +2328,8 @@ il_dealloc_bcast_stations(struct il_priv *il) · 141 unmodified lines | ||
| 2328 | 2328 | |
| 2329 | 2329 | il->stations[i].used &= ~IL_STA_UCODE_ACTIVE; |
| 2330 | 2330 | il->num_stations--; |
| 2331 | − BUG_ON(il->num_stations < 0); | |
| 2331 | + if (WARN_ON(il->num_stations < 0)) | |
| 2332 | + il->num_stations = 0; | |
| 2332 | 2333 | kfree(il->stations[i].lq); |
| 2333 | 2334 | il->stations[i].lq = NULL; |
| 2334 | 2335 | } |
| @@ -396,7 +396,7 @@ il_dbgfs_stations_read(struct file *file, char __user *user_buf, size_t count, · 395 unmodified lines | ||
| 396 | 396 | int i, j, pos = 0; |
| 397 | 397 | ssize_t ret; |
| 398 | 398 | /* Add 30 for initial string */ |
| 399 | − const size_t bufsz = 30 + sizeof(char) * 500 * (il->num_stations); | |
| 399 | + const size_t bufsz = 30 + sizeof(char) * 500 * max_sta; | |
| 400 | 400 | |
| 401 | 401 | buf = kmalloc(bufsz, GFP_KERNEL); |
| 402 | 402 | if (!buf) |
| @@ -44,7 +44,7 @@ static int mwifiex_11n_dispatch_amsdu_pkt(struct mwifiex_private *priv, · 43 unmodified lines | ||
| 44 | 44 | ntohs(rx_hdr->eth803_hdr.h_proto) == ETH_P_TDLS) { |
| 45 | 45 | mwifiex_process_tdls_action_frame(priv, |
| 46 | 46 | (u8 *)rx_hdr, |
| 47 | − skb->len); | |
| 47 | + rx_skb->len); | |
| 48 | 48 | } |
| 49 | 49 | |
| 50 | 50 | if (priv->bss_role == MWIFIEX_BSS_ROLE_UAP) |
| @@ -769,6 +769,7 @@ static void netc_timer_init(struct netc_timer *priv) · 768 unmodified lines | ||
| 769 | 769 | TMR_CTRL_TE | TMR_CTRL_FS; |
| 770 | 770 | netc_timer_wr(priv, NETC_TMR_CTRL, tmr_ctrl); |
| 771 | 771 | netc_timer_wr(priv, NETC_TMR_PRSC, priv->oclk_prsc); |
| 772 | + netc_timer_wr(priv, NETC_TMR_TEMASK, 0); | |
| 772 | 773 | |
| 773 | 774 | /* Disable FIPER by default */ |
| 774 | 775 | fiper_ctrl = netc_timer_rd(priv, NETC_TMR_FIPER_CTRL); |
| @@ -901,6 +902,11 @@ static irqreturn_t netc_timer_isr(int irq, void *data) · 126 unmodified lines | ||
| 901 | 902 | /* Clear interrupts status */ |
| 902 | 903 | netc_timer_wr(priv, NETC_TMR_TEVENT, tmr_event); |
| 903 | 904 | |
| 905 | + if (!tmr_event) { | |
| 906 | + spin_unlock(&priv->lock); | |
| 907 | + return IRQ_NONE; | |
| 908 | + } | |
| 909 | + | |
| 904 | 910 | if (tmr_event & TMR_TEVENT_ALMEN(0)) |
| 905 | 911 | netc_timer_alarm_write(priv, NETC_TMR_DEFAULT_ALARM, 0); |
| 906 | 912 | |
| @@ -936,7 +942,8 @@ static int netc_timer_init_msix_irq(struct netc_timer *priv) · 29 unmodified lines | ||
| 936 | 942 | } |
| 937 | 943 | |
| 938 | 944 | priv->irq = pci_irq_vector(pdev, 0); |
| 939 | − err = request_irq(priv->irq, netc_timer_isr, 0, priv->irq_name, priv); | |
| 945 | + err = request_irq(priv->irq, netc_timer_isr, IRQF_NO_AUTOEN, | |
| 946 | + priv->irq_name, priv); | |
| 940 | 947 | if (err) { |
| 941 | 948 | dev_err(&pdev->dev, "request_irq() failed\n"); |
| 942 | 949 | pci_free_irq_vectors(pdev); |
| @@ -951,7 +958,6 @@ static void netc_timer_free_msix_irq(struct netc_timer *priv) · 8 unmodified lines | ||
| 951 | 958 | { |
| 952 | 959 | struct pci_dev *pdev = priv->pdev; |
| 953 | 960 | |
| 954 | − disable_irq(priv->irq); | |
| 955 | 961 | free_irq(priv->irq, priv); |
| 956 | 962 | pci_free_irq_vectors(pdev); |
| 957 | 963 | } |
| @@ -1005,6 +1011,8 @@ static int netc_timer_probe(struct pci_dev *pdev, · 47 unmodified lines | ||
| 1005 | 1011 | goto free_msix_irq; |
| 1006 | 1012 | } |
| 1007 | 1013 | |
| 1014 | + enable_irq(priv->irq); | |
| 1015 | + | |
| 1008 | 1016 | return 0; |
| 1009 | 1017 | |
| 1010 | 1018 | free_msix_irq: |
| @@ -1019,9 +1027,10 @@ static void netc_timer_remove(struct pci_dev *pdev) · 8 unmodified lines | ||
| 1019 | 1027 | { |
| 1020 | 1028 | struct netc_timer *priv = pci_get_drvdata(pdev); |
| 1021 | 1029 | |
| 1030 | + disable_irq(priv->irq); | |
| 1031 | + ptp_clock_unregister(priv->clock); | |
| 1022 | 1032 | netc_timer_wr(priv, NETC_TMR_TEMASK, 0); |
| 1023 | 1033 | netc_timer_wr(priv, NETC_TMR_CTRL, 0); |
| 1024 | − ptp_clock_unregister(priv->clock); | |
| 1025 | 1034 | netc_timer_free_msix_irq(priv); |
| 1026 | 1035 | netc_timer_pci_remove(pdev); |
| 1027 | 1036 | } |
| @@ -6525,6 +6525,9 @@ int qeth_siocdevprivate(struct net_device *dev, struct ifreq *rq, void __user *d · 6524 unmodified lines | ||
| 6525 | 6525 | struct qeth_card *card = dev->ml_priv; |
| 6526 | 6526 | int rc = 0; |
| 6527 | 6527 | |
| 6528 | + if (!capable(CAP_NET_ADMIN)) | |
| 6529 | + return -EPERM; | |
| 6530 | + | |
| 6528 | 6531 | switch (cmd) { |
| 6529 | 6532 | case SIOC_QETH_ADP_SET_SNMP_CONTROL: |
| 6530 | 6533 | rc = qeth_snmp_command(card, data); |
| @@ -176,21 +176,13 @@ static void *vhost_net_buf_consume(struct vhost_net_buf *rxq) · 175 unmodified lines | ||
| 176 | 176 | return ret; |
| 177 | 177 | } |
| 178 | 178 | |
| 179 | −static int vhost_net_buf_produce(struct sock *sk, | |
| 180 | − struct vhost_net_virtqueue *nvq) | |
| 179 | +static int vhost_net_buf_produce(struct vhost_net_virtqueue *nvq) | |
| 181 | 180 | { |
| 182 | − struct file *file = sk->sk_socket->file; | |
| 183 | 181 | struct vhost_net_buf *rxq = &nvq->rxq; |
| 184 | 182 | |
| 185 | 183 | rxq->head = 0; |
| 186 | − spin_lock(&nvq->rx_ring->consumer_lock); | |
| 187 | − rxq->tail = __ptr_ring_consume_batched(nvq->rx_ring, rxq->queue, | |
| 188 | − VHOST_NET_BATCH); | |
| 189 | − | |
| 190 | − if (rxq->tail) | |
| 191 | − tun_wake_queue(file, rxq->tail); | |
| 192 | − | |
| 193 | − spin_unlock(&nvq->rx_ring->consumer_lock); | |
| 184 | + rxq->tail = ptr_ring_consume_batched(nvq->rx_ring, rxq->queue, | |
| 185 | + VHOST_NET_BATCH); | |
| 194 | 186 | return rxq->tail; |
| 195 | 187 | } |
| 196 | 188 | |
| @@ -217,15 +209,14 @@ static int vhost_net_buf_peek_len(void *ptr) · 20 unmodified lines | ||
| 217 | 209 | return __skb_array_len_with_tag(ptr); |
| 218 | 210 | } |
| 219 | 211 | |
| 220 | −static int vhost_net_buf_peek(struct sock *sk, | |
| 221 | − struct vhost_net_virtqueue *nvq) | |
| 212 | +static int vhost_net_buf_peek(struct vhost_net_virtqueue *nvq) | |
| 222 | 213 | { |
| 223 | 214 | struct vhost_net_buf *rxq = &nvq->rxq; |
| 224 | 215 | |
| 225 | 216 | if (!vhost_net_buf_is_empty(rxq)) |
| 226 | 217 | goto out; |
| 227 | 218 | |
| 228 | − if (!vhost_net_buf_produce(sk, nvq)) | |
| 219 | + if (!vhost_net_buf_produce(nvq)) | |
| 229 | 220 | return 0; |
| 230 | 221 | |
| 231 | 222 | out: |
| @@ -1013,7 +1004,7 @@ static int peek_head_len(struct vhost_net_virtqueue *rvq, struct sock *sk) · 781 unmodified lines | ||
| 1013 | 1004 | unsigned long flags; |
| 1014 | 1005 | |
| 1015 | 1006 | if (rvq->rx_ring) |
| 1016 | − return vhost_net_buf_peek(sk, rvq); | |
| 1007 | + return vhost_net_buf_peek(rvq); | |
| 1017 | 1008 | |
| 1018 | 1009 | spin_lock_irqsave(&sk->sk_receive_queue.lock, flags); |
| 1019 | 1010 | head = skb_peek(&sk->sk_receive_queue); |
| @@ -562,6 +562,7 @@ struct ethtool_fec_hist { · 561 unmodified lines | ||
| 562 | 562 | u64 per_lane[ETHTOOL_MAX_LANES]; |
| 563 | 563 | } values[ETHTOOL_FEC_HIST_MAX]; |
| 564 | 564 | const struct ethtool_fec_hist_range *ranges; |
| 565 | + struct ethtool_fec_hist_range ranges_buf[ETHTOOL_FEC_HIST_MAX]; | |
| 565 | 566 | }; |
| 566 | 567 | /** |
| 567 | 568 | * struct ethtool_fec_stats - statistics for IEEE 802.3 FEC |
| @@ -22,7 +22,6 @@ struct tun_msg_ctl { · 21 unmodified lines | ||
| 22 | 22 | #if defined(CONFIG_TUN) || defined(CONFIG_TUN_MODULE) |
| 23 | 23 | struct socket *tun_get_socket(struct file *); |
| 24 | 24 | struct ptr_ring *tun_get_tx_ring(struct file *file); |
| 25 | −void tun_wake_queue(struct file *file, int consumed); | |
| 26 | 25 | |
| 27 | 26 | static inline bool tun_is_xdp_frame(void *ptr) |
| 28 | 27 | { |
| @@ -56,8 +55,6 @@ static inline struct ptr_ring *tun_get_tx_ring(struct file *f) · 27 unmodified lines | ||
| 56 | 55 | return ERR_PTR(-EINVAL); |
| 57 | 56 | } |
| 58 | 57 | |
| 59 | −static inline void tun_wake_queue(struct file *f, int consumed) {} | |
| 60 | − | |
| 61 | 58 | static inline bool tun_is_xdp_frame(void *ptr) |
| 62 | 59 | { |
| 63 | 60 | return false; |
| @@ -115,7 +115,7 @@ struct nf_nat_sip_hooks { · 114 unmodified lines | ||
| 115 | 115 | unsigned int *datalen); |
| 116 | 116 | |
| 117 | 117 | void (*seq_adjust)(struct sk_buff *skb, |
| 118 | − unsigned int protoff, s16 off); | |
| 118 | + unsigned int protoff, s32 off); | |
| 119 | 119 | |
| 120 | 120 | unsigned int (*expect)(struct sk_buff *skb, |
| 121 | 121 | unsigned int protoff, |
| @@ -96,20 +96,6 @@ static inline bool ptr_ring_full_bh(struct ptr_ring *r) · 95 unmodified lines | ||
| 96 | 96 | return ret; |
| 97 | 97 | } |
| 98 | 98 | |
| 99 | −/* Note: callers invoking this in a loop must use a compiler barrier, | |
| 100 | − * for example cpu_relax(). Callers must hold producer_lock. | |
| 101 | − */ | |
| 102 | −static inline int __ptr_ring_check_produce(struct ptr_ring *r) | |
| 103 | −{ | |
| 104 | − if (unlikely(!r->size)) | |
| 105 | − return -EINVAL; | |
| 106 | − | |
| 107 | − if (data_race(r->queue[r->producer])) | |
| 108 | − return -ENOSPC; | |
| 109 | − | |
| 110 | − return 0; | |
| 111 | −} | |
| 112 | − | |
| 113 | 99 | /* Note: callers invoking this in a loop must use a compiler barrier, |
| 114 | 100 | * for example cpu_relax(). Callers must hold producer_lock. |
| 115 | 101 | * Callers are responsible for making sure pointer that is being queued |
| @@ -117,10 +103,8 @@ static inline int __ptr_ring_check_produce(struct ptr_ring *r) · 1 unmodified line | ||
| 117 | 103 | */ |
| 118 | 104 | static inline int __ptr_ring_produce(struct ptr_ring *r, void *ptr) |
| 119 | 105 | { |
| 120 | − int p = __ptr_ring_check_produce(r); | |
| 121 | − | |
| 122 | − if (p) | |
| 123 | − return p; | |
| 106 | + if (unlikely(!r->size) || data_race(r->queue[r->producer])) | |
| 107 | + return -ENOSPC; | |
| 124 | 108 | |
| 125 | 109 | /* Make sure the pointer we are storing points to a valid data. */ |
| 126 | 110 | /* Pairs with the dependency ordering in __ptr_ring_consume. */ |
| @@ -767,9 +767,11 @@ struct hci_conn { · 766 unmodified lines | ||
| 767 | 767 | struct dentry *debugfs; |
| 768 | 768 | |
| 769 | 769 | struct hci_dev *hdev; |
| 770 | + | |
| 771 | + spinlock_t proto_lock; /* lock guarding protocol data */ | |
| 770 | 772 | void *l2cap_data; |
| 771 | 773 | void *sco_data; |
| 772 | − void *iso_data; | |
| 774 | + void *iso_data __guarded_by(&proto_lock); | |
| 773 | 775 | |
| 774 | 776 | struct list_head link_list; |
| 775 | 777 | struct hci_conn *parent; |
| @@ -25,7 +25,9 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <linux/netfilter.h> /* for union nf_inet_addr */ |
| 26 | 26 | #include <linux/ip.h> |
| 27 | 27 | #include <linux/ipv6.h> /* for struct ipv6hdr */ |
| 28 | +#include <net/route.h> | |
| 28 | 29 | #include <net/ipv6.h> |
| 30 | +#include <net/ip6_fib.h> | |
| 29 | 31 | #if IS_ENABLED(CONFIG_NF_CONNTRACK) |
| 30 | 32 | #include <net/netfilter/nf_conntrack.h> |
| 31 | 33 | #endif |
| @@ -1972,8 +1974,8 @@ int ip_vs_tunnel_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 1940 unmodified lines | ||
| 1972 | 1974 | int ip_vs_dr_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1973 | 1975 | struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); |
| 1974 | 1976 | int ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1975 | − struct ip_vs_protocol *pp, int offset, | |
| 1976 | − unsigned int hooknum, struct ip_vs_iphdr *iph); | |
| 1977 | + struct ip_vs_protocol *pp, unsigned int toff, | |
| 1978 | + unsigned int hooknum, struct ip_vs_iphdr *ciph); | |
| 1977 | 1979 | void ip_vs_dest_dst_rcu_free(struct rcu_head *head); |
| 1978 | 1980 | |
| 1979 | 1981 | #ifdef CONFIG_IP_VS_IPV6 |
| @@ -1986,8 +1988,8 @@ int ip_vs_tunnel_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 6 unmodified lines | ||
| 1986 | 1988 | int ip_vs_dr_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1987 | 1989 | struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); |
| 1988 | 1990 | int ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1989 | − struct ip_vs_protocol *pp, int offset, | |
| 1990 | − unsigned int hooknum, struct ip_vs_iphdr *iph); | |
| 1991 | + struct ip_vs_protocol *pp, unsigned int toff, | |
| 1992 | + unsigned int hooknum, struct ip_vs_iphdr *ciph); | |
| 1991 | 1993 | #endif |
| 1992 | 1994 | |
| 1993 | 1995 | #ifdef CONFIG_SYSCTL |
| @@ -2059,15 +2061,15 @@ static inline bool ip_vs_conn_use_hash2(struct ip_vs_conn *cp) · 65 unmodified lines | ||
| 2059 | 2061 | } |
| 2060 | 2062 | |
| 2061 | 2063 | void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 2062 | − struct ip_vs_conn *cp, int dir); | |
| 2064 | + struct ip_vs_conn *cp, int dir, unsigned int toff, | |
| 2065 | + bool has_ports); | |
| 2063 | 2066 | |
| 2064 | 2067 | #ifdef CONFIG_IP_VS_IPV6 |
| 2065 | 2068 | void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 2066 | − struct ip_vs_conn *cp, int dir); | |
| 2069 | + struct ip_vs_conn *cp, int dir, unsigned int toff, | |
| 2070 | + bool has_ports, struct ip_vs_iphdr *ciph); | |
| 2067 | 2071 | #endif |
| 2068 | 2072 | |
| 2069 | −__sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset); | |
| 2070 | − | |
| 2071 | 2073 | static inline __wsum ip_vs_check_diff4(__be32 old, __be32 new, __wsum oldsum) |
| 2072 | 2074 | { |
| 2073 | 2075 | __be32 diff[2] = { ~old, new }; |
| @@ -2093,6 +2095,33 @@ static inline __wsum ip_vs_check_diff2(__be16 old, __be16 new, __wsum oldsum) · 19 unmodified lines | ||
| 2093 | 2095 | return csum_partial(diff, sizeof(diff), oldsum); |
| 2094 | 2096 | } |
| 2095 | 2097 | |
| 2098 | +static inline bool ip_vs_checksum_needed(struct sk_buff *skb, int af) | |
| 2099 | +{ | |
| 2100 | + /* Checksum unnecessary or already validated? */ | |
| 2101 | + if (skb_csum_unnecessary(skb)) | |
| 2102 | + return false; | |
| 2103 | + /* LOCAL_OUT ? */ | |
| 2104 | + if (!skb->dev || skb->dev->flags & IFF_LOOPBACK) | |
| 2105 | + return false; | |
| 2106 | + /* !LOCAL_IN (FORWARD) ? */ | |
| 2107 | + if (af == AF_INET6) { | |
| 2108 | + if (!(dst_rt6_info(skb_dst(skb))->rt6i_flags & RTF_LOCAL)) | |
| 2109 | + return false; | |
| 2110 | + } else { | |
| 2111 | + if (!(skb_rtable(skb)->rt_flags & RTCF_LOCAL)) | |
| 2112 | + return false; | |
| 2113 | + } | |
| 2114 | + return true; | |
| 2115 | +} | |
| 2116 | + | |
| 2117 | +static inline bool ip_vs_checksum_common_check(struct sk_buff *skb, | |
| 2118 | + int offset, int proto, int af) | |
| 2119 | +{ | |
| 2120 | + if (!ip_vs_checksum_needed(skb, af)) | |
| 2121 | + return true; | |
| 2122 | + return !nf_checksum(skb, NF_INET_LOCAL_IN, offset, proto, af); | |
| 2123 | +} | |
| 2124 | + | |
| 2096 | 2125 | /* Forget current conntrack (unconfirmed) and attach notrack entry */ |
| 2097 | 2126 | static inline void ip_vs_notrack(struct sk_buff *skb) |
| 2098 | 2127 | { |
| @@ -161,6 +161,9 @@ static inline int nf_ct_expect_related(struct nf_conntrack_expect *expect, · 160 unmodified lines | ||
| 161 | 161 | return nf_ct_expect_related_report(expect, 0, 0, flags); |
| 162 | 162 | } |
| 163 | 163 | |
| 164 | +int nf_ct_expect_related_pair(struct nf_conntrack_expect *expect[], | |
| 165 | + unsigned int flag); | |
| 166 | + | |
| 164 | 167 | struct nf_conn_help; |
| 165 | 168 | void nf_ct_expectation_gc(struct nf_conn_help *master_help); |
| 166 | 169 | |
| @@ -1294,6 +1294,7 @@ static inline void nft_use_inc_restore(u32 *use) · 1293 unmodified lines | ||
| 1294 | 1294 | * @sets: sets in the table |
| 1295 | 1295 | * @objects: stateful objects in the table |
| 1296 | 1296 | * @flowtables: flow tables in the table |
| 1297 | + * @objname_ht: hashtable for objects lookup by name | |
| 1297 | 1298 | * @hgenerator: handle generator state |
| 1298 | 1299 | * @handle: table handle |
| 1299 | 1300 | * @use: number of chain references to this table |
| @@ -1313,6 +1314,7 @@ struct nft_table { · 13 unmodified lines | ||
| 1313 | 1314 | struct list_head sets; |
| 1314 | 1315 | struct list_head objects; |
| 1315 | 1316 | struct list_head flowtables; |
| 1317 | + struct rhltable objname_ht; | |
| 1316 | 1318 | u64 hgenerator; |
| 1317 | 1319 | u64 handle; |
| 1318 | 1320 | u32 use; |
| @@ -1400,7 +1402,7 @@ static inline void *nft_obj_data(const struct nft_object *obj) · 81 unmodified lines | ||
| 1400 | 1402 | #define nft_expr_obj(expr) *((struct nft_object **)nft_expr_priv(expr)) |
| 1401 | 1403 | |
| 1402 | 1404 | struct nft_object *nft_obj_lookup(const struct net *net, |
| 1403 | − const struct nft_table *table, | |
| 1405 | + struct nft_table *table, | |
| 1404 | 1406 | const struct nlattr *nla, u32 objtype, |
| 1405 | 1407 | u8 genmask); |
| 1406 | 1408 | |
| @@ -80,6 +80,7 @@ struct xdp_sock { · 79 unmodified lines | ||
| 80 | 80 | * call of __xsk_generic_xmit(). |
| 81 | 81 | */ |
| 82 | 82 | struct sk_buff *skb; |
| 83 | + bool drain_cont; | |
| 83 | 84 | |
| 84 | 85 | struct list_head map_list; |
| 85 | 86 | /* Protects map_list */ |
| @@ -78,6 +78,9 @@ struct xsk_buff_pool { · 77 unmodified lines | ||
| 78 | 78 | u32 chunk_size; |
| 79 | 79 | u32 chunk_shift; |
| 80 | 80 | u32 frame_len; |
| 81 | + u32 tx_descs_nentries; | |
| 82 | + u32 reclaim_descs; | |
| 83 | + u32 tx_zc_pending_descs; | |
| 81 | 84 | u32 xdp_zc_max_segs; |
| 82 | 85 | u8 tx_metadata_len; /* inherited from umem */ |
| 83 | 86 | u8 cached_need_wakeup; |
| @@ -102,12 +105,14 @@ struct xsk_buff_pool { · 18 unmodified lines | ||
| 102 | 105 | |
| 103 | 106 | /* AF_XDP core. */ |
| 104 | 107 | struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, |
| 105 | − struct xdp_umem *umem); | |
| 108 | + struct xdp_umem *umem, | |
| 109 | + u32 max_segs); | |
| 106 | 110 | int xp_assign_dev(struct xsk_buff_pool *pool, struct net_device *dev, |
| 107 | 111 | u16 queue_id, u16 flags); |
| 108 | 112 | int xp_assign_dev_shared(struct xsk_buff_pool *pool, struct xdp_sock *umem_xs, |
| 109 | 113 | struct net_device *dev, u16 queue_id); |
| 110 | −int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs); | |
| 114 | +int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs, | |
| 115 | + u32 max_segs); | |
| 111 | 116 | void xp_destroy(struct xsk_buff_pool *pool); |
| 112 | 117 | void xp_get_pool(struct xsk_buff_pool *pool); |
| 113 | 118 | bool xp_put_pool(struct xsk_buff_pool *pool); |
| @@ -1123,6 +1123,8 @@ static struct hci_conn *__hci_conn_add(struct hci_dev *hdev, int type, · 1122 unmodified lines | ||
| 1123 | 1123 | INIT_DELAYED_WORK(&conn->idle_work, hci_conn_idle); |
| 1124 | 1124 | INIT_DELAYED_WORK(&conn->le_conn_timeout, le_conn_timeout); |
| 1125 | 1125 | |
| 1126 | + spin_lock_init(&conn->proto_lock); | |
| 1127 | + | |
| 1126 | 1128 | atomic_set(&conn->refcnt, 0); |
| 1127 | 1129 | |
| 1128 | 1130 | hci_dev_hold(hdev); |
| @@ -3163,6 +3165,13 @@ static int abort_conn_sync(struct hci_dev *hdev, void *data) · 2034 unmodified lines | ||
| 3163 | 3165 | return hci_abort_conn_sync(hdev, conn, conn->abort_reason); |
| 3164 | 3166 | } |
| 3165 | 3167 | |
| 3168 | +static void abort_conn_destroy(struct hci_dev *hdev, void *data, int err) | |
| 3169 | +{ | |
| 3170 | + struct hci_conn *conn = data; | |
| 3171 | + | |
| 3172 | + hci_conn_put(conn); | |
| 3173 | +} | |
| 3174 | + | |
| 3166 | 3175 | int hci_abort_conn(struct hci_conn *conn, u8 reason) |
| 3167 | 3176 | { |
| 3168 | 3177 | struct hci_dev *hdev = conn->hdev; |
| @@ -3188,7 +3197,10 @@ int hci_abort_conn(struct hci_conn *conn, u8 reason) · 19 unmodified lines | ||
| 3188 | 3197 | * as a result to MGMT_OP_DISCONNECT/MGMT_OP_UNPAIR which does |
| 3189 | 3198 | * already queue its callback on cmd_sync_work. |
| 3190 | 3199 | */ |
| 3191 | − err = hci_cmd_sync_run_once(hdev, abort_conn_sync, conn, NULL); | |
| 3200 | + err = hci_cmd_sync_run_once(hdev, abort_conn_sync, hci_conn_get(conn), | |
| 3201 | + abort_conn_destroy); | |
| 3202 | + if (err) | |
| 3203 | + hci_conn_put(conn); | |
| 3192 | 3204 | return (err == -EEXIST) ? 0 : err; |
| 3193 | 3205 | } |
| 3194 | 3206 | |
| @@ -1233,10 +1233,11 @@ static int hci_set_adv_set_random_addr_sync(struct hci_dev *hdev, u8 instance, · 1232 unmodified lines | ||
| 1233 | 1233 | } |
| 1234 | 1234 | |
| 1235 | 1235 | static int |
| 1236 | −hci_set_ext_adv_params_sync(struct hci_dev *hdev, struct adv_info *adv, | |
| 1236 | +hci_set_ext_adv_params_sync(struct hci_dev *hdev, u8 instance, | |
| 1237 | 1237 | const struct hci_cp_le_set_ext_adv_params *cp, |
| 1238 | 1238 | struct hci_rp_le_set_ext_adv_params *rp) |
| 1239 | 1239 | { |
| 1240 | + struct adv_info *adv; | |
| 1240 | 1241 | struct sk_buff *skb; |
| 1241 | 1242 | |
| 1242 | 1243 | skb = __hci_cmd_sync(hdev, HCI_OP_LE_SET_EXT_ADV_PARAMS, sizeof(*cp), |
| @@ -1264,11 +1265,15 @@ hci_set_ext_adv_params_sync(struct hci_dev *hdev, struct adv_info *adv, · 21 unmodified lines | ||
| 1264 | 1265 | |
| 1265 | 1266 | if (!rp->status) { |
| 1266 | 1267 | hdev->adv_addr_type = cp->own_addr_type; |
| 1267 | − if (!cp->handle) { | |
| 1268 | + if (!instance) { | |
| 1268 | 1269 | /* Store in hdev for instance 0 */ |
| 1269 | 1270 | hdev->adv_tx_power = rp->tx_power; |
| 1270 | − } else if (adv) { | |
| 1271 | − adv->tx_power = rp->tx_power; | |
| 1271 | + } else { | |
| 1272 | + hci_dev_lock(hdev); | |
| 1273 | + adv = hci_find_adv_instance(hdev, instance); | |
| 1274 | + if (adv) | |
| 1275 | + adv->tx_power = rp->tx_power; | |
| 1276 | + hci_dev_unlock(hdev); | |
| 1272 | 1277 | } |
| 1273 | 1278 | } |
| 1274 | 1279 | |
| @@ -1284,9 +1289,13 @@ static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance) · 9 unmodified lines | ||
| 1284 | 1289 | int err; |
| 1285 | 1290 | |
| 1286 | 1291 | if (instance) { |
| 1292 | + hci_dev_lock(hdev); | |
| 1293 | + | |
| 1287 | 1294 | adv = hci_find_adv_instance(hdev, instance); |
| 1288 | − if (!adv || !adv->adv_data_changed) | |
| 1295 | + if (!adv || !adv->adv_data_changed) { | |
| 1296 | + hci_dev_unlock(hdev); | |
| 1289 | 1297 | return 0; |
| 1298 | + } | |
| 1290 | 1299 | } |
| 1291 | 1300 | |
| 1292 | 1301 | len = eir_create_adv_data(hdev, instance, pdu->data, |
| @@ -1297,16 +1306,27 @@ static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance) · 4 unmodified lines | ||
| 1297 | 1306 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; |
| 1298 | 1307 | pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; |
| 1299 | 1308 | |
| 1309 | + if (adv) { | |
| 1310 | + adv->adv_data_changed = false; | |
| 1311 | + hci_dev_unlock(hdev); | |
| 1312 | + } | |
| 1313 | + | |
| 1300 | 1314 | err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_DATA, |
| 1301 | 1315 | struct_size(pdu, data, len), pdu, |
| 1302 | 1316 | HCI_CMD_TIMEOUT); |
| 1303 | − if (err) | |
| 1317 | + if (err) { | |
| 1318 | + if (instance) { | |
| 1319 | + hci_dev_lock(hdev); | |
| 1320 | + adv = hci_find_adv_instance(hdev, instance); | |
| 1321 | + if (adv) | |
| 1322 | + adv->adv_data_changed = true; | |
| 1323 | + hci_dev_unlock(hdev); | |
| 1324 | + } | |
| 1325 | + | |
| 1304 | 1326 | return err; |
| 1327 | + } | |
| 1305 | 1328 | |
| 1306 | − /* Update data if the command succeed */ | |
| 1307 | − if (adv) { | |
| 1308 | − adv->adv_data_changed = false; | |
| 1309 | − } else { | |
| 1329 | + if (!instance) { | |
| 1310 | 1330 | memcpy(hdev->adv_data, pdu->data, len); |
| 1311 | 1331 | hdev->adv_data_len = len; |
| 1312 | 1332 | } |
| @@ -1360,22 +1380,22 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 47 unmodified lines | ||
| 1360 | 1380 | struct adv_info *adv; |
| 1361 | 1381 | bool secondary_adv; |
| 1362 | 1382 | |
| 1363 | − if (instance > 0) { | |
| 1364 | − adv = hci_find_adv_instance(hdev, instance); | |
| 1365 | − if (!adv) | |
| 1366 | − return -EINVAL; | |
| 1367 | − } else { | |
| 1368 | − adv = NULL; | |
| 1369 | − } | |
| 1370 | − | |
| 1371 | 1383 | /* Updating parameters of an active instance will return a |
| 1372 | − * Command Disallowed error, so we must first disable the | |
| 1373 | − * instance if it is active. | |
| 1384 | + * Command Disallowed error, so disable it before taking a snapshot. | |
| 1374 | 1385 | */ |
| 1375 | − if (adv) { | |
| 1386 | + if (instance > 0) { | |
| 1376 | 1387 | err = hci_disable_ext_adv_instance_sync(hdev, instance); |
| 1377 | 1388 | if (err) |
| 1378 | 1389 | return err; |
| 1390 | + | |
| 1391 | + hci_dev_lock(hdev); | |
| 1392 | + adv = hci_find_adv_instance(hdev, instance); | |
| 1393 | + if (!adv) { | |
| 1394 | + hci_dev_unlock(hdev); | |
| 1395 | + return -EINVAL; | |
| 1396 | + } | |
| 1397 | + } else { | |
| 1398 | + adv = NULL; | |
| 1379 | 1399 | } |
| 1380 | 1400 | |
| 1381 | 1401 | flags = hci_adv_instance_flags(hdev, instance); |
| @@ -1386,8 +1406,11 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 4 unmodified lines | ||
| 1386 | 1406 | connectable = (flags & MGMT_ADV_FLAG_CONNECTABLE) || |
| 1387 | 1407 | mgmt_get_connectable(hdev); |
| 1388 | 1408 | |
| 1389 | − if (!is_advertising_allowed(hdev, connectable)) | |
| 1409 | + if (!is_advertising_allowed(hdev, connectable)) { | |
| 1410 | + if (instance) | |
| 1411 | + hci_dev_unlock(hdev); | |
| 1390 | 1412 | return -EPERM; |
| 1413 | + } | |
| 1391 | 1414 | |
| 1392 | 1415 | /* Set require_privacy to true only when non-connectable |
| 1393 | 1416 | * advertising is used and it is not periodic. |
| @@ -1398,8 +1421,11 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 4 unmodified lines | ||
| 1398 | 1421 | err = hci_get_random_address(hdev, require_privacy, |
| 1399 | 1422 | adv_use_rpa(hdev, flags), adv, |
| 1400 | 1423 | &own_addr_type, &random_addr); |
| 1401 | − if (err < 0) | |
| 1424 | + if (err < 0) { | |
| 1425 | + if (instance) | |
| 1426 | + hci_dev_unlock(hdev); | |
| 1402 | 1427 | return err; |
| 1428 | + } | |
| 1403 | 1429 | |
| 1404 | 1430 | memset(&cp, 0, sizeof(cp)); |
| 1405 | 1431 | |
| @@ -1450,6 +1476,9 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 44 unmodified lines | ||
| 1450 | 1476 | cp.channel_map = hdev->le_adv_channel_map; |
| 1451 | 1477 | cp.handle = adv ? adv->handle : instance; |
| 1452 | 1478 | |
| 1479 | + if (instance) | |
| 1480 | + hci_dev_unlock(hdev); | |
| 1481 | + | |
| 1453 | 1482 | if (flags & MGMT_ADV_FLAG_SEC_2M) { |
| 1454 | 1483 | cp.primary_phy = HCI_ADV_PHY_1M; |
| 1455 | 1484 | cp.secondary_phy = HCI_ADV_PHY_2M; |
| @@ -1462,12 +1491,12 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 6 unmodified lines | ||
| 1462 | 1491 | cp.secondary_phy = HCI_ADV_PHY_1M; |
| 1463 | 1492 | } |
| 1464 | 1493 | |
| 1465 | − err = hci_set_ext_adv_params_sync(hdev, adv, &cp, &rp); | |
| 1494 | + err = hci_set_ext_adv_params_sync(hdev, instance, &cp, &rp); | |
| 1466 | 1495 | if (err) |
| 1467 | 1496 | return err; |
| 1468 | 1497 | |
| 1469 | 1498 | /* Update adv data as tx power is known now */ |
| 1470 | − err = hci_set_ext_adv_data_sync(hdev, cp.handle); | |
| 1499 | + err = hci_set_ext_adv_data_sync(hdev, instance); | |
| 1471 | 1500 | if (err) |
| 1472 | 1501 | return err; |
| 1473 | 1502 | |
| @@ -1475,9 +1504,14 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 1 unmodified line | ||
| 1475 | 1504 | own_addr_type == ADDR_LE_DEV_RANDOM_RESOLVED) && |
| 1476 | 1505 | bacmp(&random_addr, BDADDR_ANY)) { |
| 1477 | 1506 | /* Check if random address need to be updated */ |
| 1478 | − if (adv) { | |
| 1479 | − if (!bacmp(&random_addr, &adv->random_addr)) | |
| 1507 | + if (instance) { | |
| 1508 | + hci_dev_lock(hdev); | |
| 1509 | + adv = hci_find_adv_instance(hdev, instance); | |
| 1510 | + if (!adv || !bacmp(&random_addr, &adv->random_addr)) { | |
| 1511 | + hci_dev_unlock(hdev); | |
| 1480 | 1512 | return 0; |
| 1513 | + } | |
| 1514 | + hci_dev_unlock(hdev); | |
| 1481 | 1515 | } else { |
| 1482 | 1516 | if (!bacmp(&random_addr, &hdev->random_addr)) |
| 1483 | 1517 | return 0; |
| @@ -1499,9 +1533,13 @@ static int hci_set_ext_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) · 15 unmodified lines | ||
| 1499 | 1533 | int err; |
| 1500 | 1534 | |
| 1501 | 1535 | if (instance) { |
| 1536 | + hci_dev_lock(hdev); | |
| 1537 | + | |
| 1502 | 1538 | adv = hci_find_adv_instance(hdev, instance); |
| 1503 | − if (!adv || !adv->scan_rsp_changed) | |
| 1539 | + if (!adv || !adv->scan_rsp_changed) { | |
| 1540 | + hci_dev_unlock(hdev); | |
| 1504 | 1541 | return 0; |
| 1542 | + } | |
| 1505 | 1543 | } |
| 1506 | 1544 | |
| 1507 | 1545 | len = eir_create_scan_rsp(hdev, instance, pdu->data); |
| @@ -1511,15 +1549,27 @@ static int hci_set_ext_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) · 3 unmodified lines | ||
| 1511 | 1549 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; |
| 1512 | 1550 | pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; |
| 1513 | 1551 | |
| 1552 | + if (adv) { | |
| 1553 | + adv->scan_rsp_changed = false; | |
| 1554 | + hci_dev_unlock(hdev); | |
| 1555 | + } | |
| 1556 | + | |
| 1514 | 1557 | err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_RSP_DATA, |
| 1515 | 1558 | struct_size(pdu, data, len), pdu, |
| 1516 | 1559 | HCI_CMD_TIMEOUT); |
| 1517 | − if (err) | |
| 1560 | + if (err) { | |
| 1561 | + if (instance) { | |
| 1562 | + hci_dev_lock(hdev); | |
| 1563 | + adv = hci_find_adv_instance(hdev, instance); | |
| 1564 | + if (adv) | |
| 1565 | + adv->scan_rsp_changed = true; | |
| 1566 | + hci_dev_unlock(hdev); | |
| 1567 | + } | |
| 1568 | + | |
| 1518 | 1569 | return err; |
| 1570 | + } | |
| 1519 | 1571 | |
| 1520 | − if (adv) { | |
| 1521 | − adv->scan_rsp_changed = false; | |
| 1522 | − } else { | |
| 1572 | + if (!instance) { | |
| 1523 | 1573 | memcpy(hdev->scan_rsp_data, pdu->data, len); |
| 1524 | 1574 | hdev->scan_rsp_data_len = len; |
| 1525 | 1575 | } |
| @@ -1534,8 +1584,14 @@ static int __hci_set_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) · 8 unmodified lines | ||
| 1534 | 1584 | |
| 1535 | 1585 | memset(&cp, 0, sizeof(cp)); |
| 1536 | 1586 | |
| 1587 | + if (instance) | |
| 1588 | + hci_dev_lock(hdev); | |
| 1589 | + | |
| 1537 | 1590 | len = eir_create_scan_rsp(hdev, instance, cp.data); |
| 1538 | 1591 | |
| 1592 | + if (instance) | |
| 1593 | + hci_dev_unlock(hdev); | |
| 1594 | + | |
| 1539 | 1595 | if (hdev->scan_rsp_data_len == len && |
| 1540 | 1596 | !memcmp(cp.data, hdev->scan_rsp_data, len)) |
| 1541 | 1597 | return 0; |
| @@ -1670,9 +1726,13 @@ static int hci_set_per_adv_data_sync(struct hci_dev *hdev, u8 instance) · 128 unmodified lines | ||
| 1670 | 1726 | struct adv_info *adv = NULL; |
| 1671 | 1727 | |
| 1672 | 1728 | if (instance) { |
| 1729 | + hci_dev_lock(hdev); | |
| 1730 | + | |
| 1673 | 1731 | adv = hci_find_adv_instance(hdev, instance); |
| 1674 | − if (!adv || !adv->periodic) | |
| 1732 | + if (!adv || !adv->periodic) { | |
| 1733 | + hci_dev_unlock(hdev); | |
| 1675 | 1734 | return 0; |
| 1735 | + } | |
| 1676 | 1736 | } |
| 1677 | 1737 | |
| 1678 | 1738 | len = eir_create_per_adv_data(hdev, instance, pdu->data); |
| @@ -1681,6 +1741,9 @@ static int hci_set_per_adv_data_sync(struct hci_dev *hdev, u8 instance) · 2 unmodified lines | ||
| 1681 | 1741 | pdu->handle = adv ? adv->handle : instance; |
| 1682 | 1742 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; |
| 1683 | 1743 | |
| 1744 | + if (adv) | |
| 1745 | + hci_dev_unlock(hdev); | |
| 1746 | + | |
| 1684 | 1747 | return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_PER_ADV_DATA, |
| 1685 | 1748 | struct_size(pdu, data, len), pdu, |
| 1686 | 1749 | HCI_CMD_TIMEOUT); |
| @@ -6523,7 +6586,7 @@ static int hci_le_ext_directed_advertising_sync(struct hci_dev *hdev, · 4836 unmodified lines | ||
| 6523 | 6586 | if (err) |
| 6524 | 6587 | return err; |
| 6525 | 6588 | |
| 6526 | − err = hci_set_ext_adv_params_sync(hdev, NULL, &cp, &rp); | |
| 6589 | + err = hci_set_ext_adv_params_sync(hdev, 0, &cp, &rp); | |
| 6527 | 6590 | if (err) |
| 6528 | 6591 | return err; |
| 6529 | 6592 | |
| @@ -6678,11 +6741,6 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data) · 148 unmodified lines | ||
| 6678 | 6741 | |
| 6679 | 6742 | bt_dev_dbg(hdev, "conn %p", conn); |
| 6680 | 6743 | |
| 6681 | − /* Hold a reference so conn stays valid for the HCI_CONN_CREATE | |
| 6682 | − * clear_bit() at done. | |
| 6683 | − */ | |
| 6684 | − hci_conn_get(conn); | |
| 6685 | − | |
| 6686 | 6744 | clear_bit(HCI_CONN_SCANNING, &conn->flags); |
| 6687 | 6745 | conn->state = BT_CONNECT; |
| 6688 | 6746 | |
| @@ -6694,8 +6752,9 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data) · 5 unmodified lines | ||
| 6694 | 6752 | if (hci_dev_test_flag(hdev, HCI_LE_SCAN) && |
| 6695 | 6753 | hdev->le_scan_type == LE_SCAN_ACTIVE && |
| 6696 | 6754 | !hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES)) { |
| 6697 | − hci_conn_del(conn); | |
| 6698 | − hci_conn_put(conn); | |
| 6755 | + conn->state = BT_OPEN; | |
| 6756 | + hci_abort_conn_sync(hdev, conn, | |
| 6757 | + HCI_ERROR_REJ_LIMITED_RESOURCES); | |
| 6699 | 6758 | return -EBUSY; |
| 6700 | 6759 | } |
| 6701 | 6760 | |
| @@ -6793,7 +6852,6 @@ done: · 91 unmodified lines | ||
| 6793 | 6852 | |
| 6794 | 6853 | /* Re-enable advertising after the connection attempt is finished. */ |
| 6795 | 6854 | hci_resume_advertising_sync(hdev); |
| 6796 | − hci_conn_put(conn); | |
| 6797 | 6855 | return err; |
| 6798 | 6856 | } |
| 6799 | 6857 | |
| @@ -7068,11 +7126,6 @@ static int hci_acl_create_conn_sync(struct hci_dev *hdev, void *data) · 268 unmodified lines | ||
| 7068 | 7126 | else |
| 7069 | 7127 | cp.role_switch = 0x00; |
| 7070 | 7128 | |
| 7071 | − /* Hold a reference so conn stays valid for the HCI_CONN_CREATE | |
| 7072 | − * clear_bit() below. | |
| 7073 | − */ | |
| 7074 | − hci_conn_get(conn); | |
| 7075 | − | |
| 7076 | 7129 | /* Mark create connection in flight so hci_cancel_connect_sync() can |
| 7077 | 7130 | * cancel it while blocking on the connection complete event. |
| 7078 | 7131 | */ |
| @@ -7084,17 +7137,27 @@ static int hci_acl_create_conn_sync(struct hci_dev *hdev, void *data) · 5 unmodified lines | ||
| 7084 | 7137 | conn->conn_timeout, NULL); |
| 7085 | 7138 | |
| 7086 | 7139 | clear_bit(HCI_CONN_CREATE, &conn->flags); |
| 7087 | − hci_conn_put(conn); | |
| 7088 | 7140 | |
| 7089 | 7141 | return err; |
| 7090 | 7142 | } |
| 7091 | 7143 | |
| 7144 | +static void hci_acl_create_conn_sync_complete(struct hci_dev *hdev, void *data, | |
| 7145 | + int err) | |
| 7146 | +{ | |
| 7147 | + struct hci_conn *conn = data; | |
| 7148 | + | |
| 7149 | + hci_conn_put(conn); | |
| 7150 | +} | |
| 7151 | + | |
| 7092 | 7152 | int hci_connect_acl_sync(struct hci_dev *hdev, struct hci_conn *conn) |
| 7093 | 7153 | { |
| 7094 | 7154 | int err; |
| 7095 | 7155 | |
| 7096 | − err = hci_cmd_sync_queue_once(hdev, hci_acl_create_conn_sync, conn, | |
| 7097 | − NULL); | |
| 7156 | + err = hci_cmd_sync_queue_once(hdev, hci_acl_create_conn_sync, | |
| 7157 | + hci_conn_get(conn), | |
| 7158 | + hci_acl_create_conn_sync_complete); | |
| 7159 | + if (err) | |
| 7160 | + hci_conn_put(conn); | |
| 7098 | 7161 | return (err == -EEXIST) ? 0 : err; |
| 7099 | 7162 | } |
| 7100 | 7163 | |
| @@ -7105,36 +7168,41 @@ static void create_le_conn_complete(struct hci_dev *hdev, void *data, int err) · 4 unmodified lines | ||
| 7105 | 7168 | bt_dev_dbg(hdev, "err %d", err); |
| 7106 | 7169 | |
| 7107 | 7170 | if (err == -ECANCELED) |
| 7108 | − return; | |
| 7171 | + goto done; | |
| 7109 | 7172 | |
| 7110 | 7173 | hci_dev_lock(hdev); |
| 7111 | 7174 | |
| 7112 | 7175 | if (!hci_conn_valid(hdev, conn)) |
| 7113 | − goto done; | |
| 7176 | + goto unlock; | |
| 7114 | 7177 | |
| 7115 | 7178 | if (!err) { |
| 7116 | 7179 | hci_connect_le_scan_cleanup(conn, 0x00); |
| 7117 | − goto done; | |
| 7180 | + goto unlock; | |
| 7118 | 7181 | } |
| 7119 | 7182 | |
| 7120 | 7183 | /* Check if connection is still pending */ |
| 7121 | 7184 | if (conn != hci_lookup_le_connect(hdev)) |
| 7122 | − goto done; | |
| 7185 | + goto unlock; | |
| 7123 | 7186 | |
| 7124 | 7187 | /* Flush to make sure we send create conn cancel command if needed */ |
| 7125 | 7188 | flush_delayed_work(&conn->le_conn_timeout); |
| 7126 | 7189 | hci_conn_failed(conn, bt_status(err)); |
| 7127 | 7190 | |
| 7128 | −done: | |
| 7191 | +unlock: | |
| 7129 | 7192 | hci_dev_unlock(hdev); |
| 7193 | +done: | |
| 7194 | + hci_conn_put(conn); | |
| 7130 | 7195 | } |
| 7131 | 7196 | |
| 7132 | 7197 | int hci_connect_le_sync(struct hci_dev *hdev, struct hci_conn *conn) |
| 7133 | 7198 | { |
| 7134 | 7199 | int err; |
| 7135 | 7200 | |
| 7136 | − err = hci_cmd_sync_queue_once(hdev, hci_le_create_conn_sync, conn, | |
| 7201 | + err = hci_cmd_sync_queue_once(hdev, hci_le_create_conn_sync, | |
| 7202 | + hci_conn_get(conn), | |
| 7137 | 7203 | create_le_conn_complete); |
| 7204 | + if (err) | |
| 7205 | + hci_conn_put(conn); | |
| 7138 | 7206 | return (err == -EEXIST) ? 0 : err; |
| 7139 | 7207 | } |
| 7140 | 7208 | |
| @@ -7257,7 +7325,7 @@ static void create_pa_complete(struct hci_dev *hdev, void *data, int err) · 116 unmodified lines | ||
| 7257 | 7325 | bt_dev_dbg(hdev, "err %d", err); |
| 7258 | 7326 | |
| 7259 | 7327 | if (err == -ECANCELED) |
| 7260 | − return; | |
| 7328 | + goto done; | |
| 7261 | 7329 | |
| 7262 | 7330 | hci_dev_lock(hdev); |
| 7263 | 7331 | |
| @@ -7281,6 +7349,8 @@ static void create_pa_complete(struct hci_dev *hdev, void *data, int err) · 17 unmodified lines | ||
| 7281 | 7349 | |
| 7282 | 7350 | unlock: |
| 7283 | 7351 | hci_dev_unlock(hdev); |
| 7352 | +done: | |
| 7353 | + hci_conn_put(conn); | |
| 7284 | 7354 | } |
| 7285 | 7355 | |
| 7286 | 7356 | static int hci_le_past_params_sync(struct hci_dev *hdev, struct hci_conn *conn, |
| @@ -7431,8 +7501,11 @@ int hci_connect_pa_sync(struct hci_dev *hdev, struct hci_conn *conn) · 144 unmodified lines | ||
| 7431 | 7501 | { |
| 7432 | 7502 | int err; |
| 7433 | 7503 | |
| 7434 | − err = hci_cmd_sync_queue_once(hdev, hci_le_pa_create_sync, conn, | |
| 7504 | + err = hci_cmd_sync_queue_once(hdev, hci_le_pa_create_sync, | |
| 7505 | + hci_conn_get(conn), | |
| 7435 | 7506 | create_pa_complete); |
| 7507 | + if (err) | |
| 7508 | + hci_conn_put(conn); | |
| 7436 | 7509 | return (err == -EEXIST) ? 0 : err; |
| 7437 | 7510 | } |
| 7438 | 7511 | |
| @@ -7443,10 +7516,12 @@ static void create_big_complete(struct hci_dev *hdev, void *data, int err) · 4 unmodified lines | ||
| 7443 | 7516 | bt_dev_dbg(hdev, "err %d", err); |
| 7444 | 7517 | |
| 7445 | 7518 | if (err == -ECANCELED) |
| 7446 | − return; | |
| 7519 | + goto done; | |
| 7447 | 7520 | |
| 7448 | − if (hci_conn_valid(hdev, conn)) | |
| 7449 | − clear_bit(HCI_CONN_CREATE_BIG_SYNC, &conn->flags); | |
| 7521 | + clear_bit(HCI_CONN_CREATE_BIG_SYNC, &conn->flags); | |
| 7522 | + | |
| 7523 | +done: | |
| 7524 | + hci_conn_put(conn); | |
| 7450 | 7525 | } |
| 7451 | 7526 | |
| 7452 | 7527 | static int hci_le_big_create_sync(struct hci_dev *hdev, void *data) |
| @@ -7498,8 +7573,14 @@ int hci_connect_big_sync(struct hci_dev *hdev, struct hci_conn *conn) · 45 unmodified lines | ||
| 7498 | 7573 | { |
| 7499 | 7574 | int err; |
| 7500 | 7575 | |
| 7501 | − err = hci_cmd_sync_queue_once(hdev, hci_le_big_create_sync, conn, | |
| 7576 | + if (!conn) | |
| 7577 | + return 0; | |
| 7578 | + | |
| 7579 | + err = hci_cmd_sync_queue_once(hdev, hci_le_big_create_sync, | |
| 7580 | + hci_conn_get(conn), | |
| 7502 | 7581 | create_big_complete); |
| 7582 | + if (err) | |
| 7583 | + hci_conn_put(conn); | |
| 7503 | 7584 | return (err == -EEXIST) ? 0 : err; |
| 7504 | 7585 | } |
| 7505 | 7586 | |
| @@ -7514,6 +7595,8 @@ static void past_complete(struct hci_dev *hdev, void *data, int err) · 8 unmodified lines | ||
| 7514 | 7595 | |
| 7515 | 7596 | bt_dev_dbg(hdev, "err %d", err); |
| 7516 | 7597 | |
| 7598 | + hci_conn_put(past->conn); | |
| 7599 | + hci_conn_put(past->le); | |
| 7517 | 7600 | kfree(past); |
| 7518 | 7601 | } |
| 7519 | 7602 | |
| @@ -7578,8 +7661,8 @@ int hci_past_sync(struct hci_conn *conn, struct hci_conn *le) · 58 unmodified lines | ||
| 7578 | 7661 | if (!data) |
| 7579 | 7662 | return -ENOMEM; |
| 7580 | 7663 | |
| 7581 | − data->conn = conn; | |
| 7582 | − data->le = le; | |
| 7664 | + data->conn = hci_conn_get(conn); | |
| 7665 | + data->le = hci_conn_get(le); | |
| 7583 | 7666 | |
| 7584 | 7667 | if (conn->role == HCI_ROLE_MASTER) |
| 7585 | 7668 | err = hci_cmd_sync_queue_once(conn->hdev, |
| @@ -7589,8 +7672,11 @@ int hci_past_sync(struct hci_conn *conn, struct hci_conn *le) · 3 unmodified lines | ||
| 7589 | 7672 | err = hci_cmd_sync_queue_once(conn->hdev, hci_le_past_sync, |
| 7590 | 7673 | data, past_complete); |
| 7591 | 7674 | |
| 7592 | − if (err) | |
| 7675 | + if (err) { | |
| 7676 | + hci_conn_put(data->conn); | |
| 7677 | + hci_conn_put(data->le); | |
| 7593 | 7678 | kfree(data); |
| 7679 | + } | |
| 7594 | 7680 | |
| 7595 | 7681 | return (err == -EEXIST) ? 0 : err; |
| 7596 | 7682 | } |
| @@ -543,9 +543,10 @@ static int hidp_process_data(struct hidp_session *session, struct sk_buff *skb, · 542 unmodified lines | ||
| 543 | 543 | } |
| 544 | 544 | |
| 545 | 545 | if (test_bit(HIDP_WAITING_FOR_RETURN, &session->flags) && |
| 546 | − param == session->waiting_report_type) { | |
| 546 | + param == session->waiting_report_type) { | |
| 547 | 547 | if (session->waiting_report_number < 0 || |
| 548 | − session->waiting_report_number == skb->data[0]) { | |
| 548 | + (skb->len && | |
| 549 | + session->waiting_report_number == skb->data[0])) { | |
| 549 | 550 | /* hidp_get_raw_report() is waiting on this report. */ |
| 550 | 551 | session->report_return = skb; |
| 551 | 552 | done_with_skb = 0; |
| @@ -560,16 +561,18 @@ static int hidp_process_data(struct hidp_session *session, struct sk_buff *skb, · 8 unmodified lines | ||
| 560 | 561 | static void hidp_recv_ctrl_frame(struct hidp_session *session, |
| 561 | 562 | struct sk_buff *skb) |
| 562 | 563 | { |
| 563 | − unsigned char hdr, type, param; | |
| 564 | + unsigned char type, param; | |
| 565 | + u8 *hdr; | |
| 564 | 566 | int free_skb = 1; |
| 565 | 567 | |
| 566 | 568 | BT_DBG("session %p skb %p len %u", session, skb, skb->len); |
| 567 | 569 | |
| 568 | − hdr = skb->data[0]; | |
| 569 | − skb_pull(skb, 1); | |
| 570 | + hdr = skb_pull_data(skb, 1); | |
| 571 | + if (!hdr) | |
| 572 | + goto free; | |
| 570 | 573 | |
| 571 | − type = hdr & HIDP_HEADER_TRANS_MASK; | |
| 572 | − param = hdr & HIDP_HEADER_PARAM_MASK; | |
| 574 | + type = *hdr & HIDP_HEADER_TRANS_MASK; | |
| 575 | + param = *hdr & HIDP_HEADER_PARAM_MASK; | |
| 573 | 576 | |
| 574 | 577 | switch (type) { |
| 575 | 578 | case HIDP_TRANS_HANDSHAKE: |
| @@ -590,6 +593,7 @@ static void hidp_recv_ctrl_frame(struct hidp_session *session, · 14 unmodified lines | ||
| 590 | 593 | break; |
| 591 | 594 | } |
| 592 | 595 | |
| 596 | +free: | |
| 593 | 597 | if (free_skb) |
| 594 | 598 | kfree_skb(skb); |
| 595 | 599 | } |
| @@ -597,14 +601,15 @@ static void hidp_recv_ctrl_frame(struct hidp_session *session, · 1 unmodified line | ||
| 597 | 601 | static void hidp_recv_intr_frame(struct hidp_session *session, |
| 598 | 602 | struct sk_buff *skb) |
| 599 | 603 | { |
| 600 | − unsigned char hdr; | |
| 604 | + u8 *hdr; | |
| 601 | 605 | |
| 602 | 606 | BT_DBG("session %p skb %p len %u", session, skb, skb->len); |
| 603 | 607 | |
| 604 | − hdr = skb->data[0]; | |
| 605 | − skb_pull(skb, 1); | |
| 608 | + hdr = skb_pull_data(skb, 1); | |
| 609 | + if (!hdr) | |
| 610 | + goto free; | |
| 606 | 611 | |
| 607 | − if (hdr == (HIDP_TRANS_DATA | HIDP_DATA_RTYPE_INPUT)) { | |
| 612 | + if (*hdr == (HIDP_TRANS_DATA | HIDP_DATA_RTYPE_INPUT)) { | |
| 608 | 613 | hidp_set_timer(session); |
| 609 | 614 | |
| 610 | 615 | if (session->input) |
| @@ -616,9 +621,10 @@ static void hidp_recv_intr_frame(struct hidp_session *session, · 5 unmodified lines | ||
| 616 | 621 | BT_DBG("report len %d", skb->len); |
| 617 | 622 | } |
| 618 | 623 | } else { |
| 619 | − BT_DBG("Unsupported protocol header 0x%02x", hdr); | |
| 624 | + BT_DBG("Unsupported protocol header 0x%02x", *hdr); | |
| 620 | 625 | } |
| 621 | 626 | |
| 627 | +free: | |
| 622 | 628 | kfree_skb(skb); |
| 623 | 629 | } |
| 624 | 630 | |
| @@ -24,15 +24,19 @@ static struct bt_sock_list iso_sk_list = { · 23 unmodified lines | ||
| 24 | 24 | }; |
| 25 | 25 | |
| 26 | 26 | /* ---- ISO connections ---- */ |
| 27 | +enum { | |
| 28 | + ISO_CONN_DROPPED, | |
| 29 | + __ISO_CONN_NUM_FLAGS | |
| 30 | +}; | |
| 31 | + | |
| 27 | 32 | struct iso_conn { |
| 28 | 33 | struct hci_conn *hcon; |
| 34 | + DECLARE_BITMAP(flags, __ISO_CONN_NUM_FLAGS); | |
| 29 | 35 | |
| 30 | 36 | /* @lock: spinlock protecting changes to iso_conn fields */ |
| 31 | 37 | spinlock_t lock; |
| 32 | 38 | struct sock *sk; |
| 33 | 39 | |
| 34 | − struct delayed_work timeout_work; | |
| 35 | − | |
| 36 | 40 | struct sk_buff *rx_skb; |
| 37 | 41 | __u32 rx_len; |
| 38 | 42 | __u16 tx_sn; |
| @@ -56,6 +60,7 @@ static void iso_sock_kill(struct sock *sk); · 17 unmodified lines | ||
| 56 | 60 | enum { |
| 57 | 61 | BT_SK_BIG_SYNC, |
| 58 | 62 | BT_SK_PA_SYNC, |
| 63 | + BT_SK_KILLED, | |
| 59 | 64 | }; |
| 60 | 65 | |
| 61 | 66 | struct iso_pinfo { |
| @@ -74,6 +79,7 @@ struct iso_pinfo { · 12 unmodified lines | ||
| 74 | 79 | __u8 base_len; |
| 75 | 80 | __u8 base[BASE_MAX_LENGTH]; |
| 76 | 81 | struct iso_conn *conn; |
| 82 | + struct delayed_work timeout_work; | |
| 77 | 83 | }; |
| 78 | 84 | |
| 79 | 85 | static struct bt_iso_qos default_qos; |
| @@ -102,16 +108,18 @@ static void iso_conn_free(struct kref *ref) · 22 unmodified lines | ||
| 102 | 108 | |
| 103 | 109 | BT_DBG("conn %p", conn); |
| 104 | 110 | |
| 105 | − if (conn->sk) | |
| 106 | − iso_pi(conn->sk)->conn = NULL; | |
| 107 | − | |
| 108 | 111 | if (conn->hcon) { |
| 109 | − conn->hcon->iso_data = NULL; | |
| 110 | − hci_conn_drop(conn->hcon); | |
| 111 | − } | |
| 112 | + spin_lock(&conn->hcon->proto_lock); | |
| 112 | 113 | |
| 113 | − /* Ensure no more work items will run since hci_conn has been dropped */ | |
| 114 | − disable_delayed_work_sync(&conn->timeout_work); | |
| 114 | + /* Check we are not racing with iso_conn_add */ | |
| 115 | + if (conn->hcon->iso_data == conn) { | |
| 116 | + conn->hcon->iso_data = NULL; | |
| 117 | + if (!test_and_set_bit(ISO_CONN_DROPPED, conn->flags)) | |
| 118 | + hci_conn_drop(conn->hcon); | |
| 119 | + } | |
| 120 | + | |
| 121 | + spin_unlock(&conn->hcon->proto_lock); | |
| 122 | + } | |
| 115 | 123 | |
| 116 | 124 | kfree_skb(conn->rx_skb); |
| 117 | 125 | |
| @@ -125,7 +133,21 @@ static void iso_conn_put(struct iso_conn *conn) · 7 unmodified lines | ||
| 125 | 133 | |
| 126 | 134 | BT_DBG("conn %p refcnt %d", conn, kref_read(&conn->ref)); |
| 127 | 135 | |
| 136 | + /* The following race vs. iso_conn_del() is possible: | |
| 137 | + * | |
| 138 | + * 1. conn->hcon != NULL here | |
| 139 | + * 2. kref_put puts the last reference | |
| 140 | + * 3. concurrent iso_conn_del() gets iso_conn_hold_unless_zero() -> NULL | |
| 141 | + * and returns immediately, so conn->hcon is not cleared | |
| 142 | + * 4. iso_conn_free() dereferences conn->hcon | |
| 143 | + * | |
| 144 | + * To avoid UAF in step 4, take RCU before decrementing the refcount. | |
| 145 | + */ | |
| 146 | + rcu_read_lock(); | |
| 147 | + | |
| 128 | 148 | kref_put(&conn->ref, iso_conn_free); |
| 149 | + | |
| 150 | + rcu_read_unlock(); | |
| 129 | 151 | } |
| 130 | 152 | |
| 131 | 153 | static struct iso_conn *iso_conn_hold_unless_zero(struct iso_conn *conn) |
| @@ -141,6 +163,14 @@ static struct iso_conn *iso_conn_hold_unless_zero(struct iso_conn *conn) · 9 unmodified lines | ||
| 141 | 163 | return conn; |
| 142 | 164 | } |
| 143 | 165 | |
| 166 | +static struct iso_conn *iso_conn_hold(struct iso_conn *conn) | |
| 167 | +{ | |
| 168 | + BT_DBG("conn %p refcnt %u", conn, kref_read(&conn->ref)); | |
| 169 | + | |
| 170 | + kref_get(&conn->ref); | |
| 171 | + return conn; | |
| 172 | +} | |
| 173 | + | |
| 144 | 174 | static struct sock *iso_sock_hold(struct iso_conn *conn) |
| 145 | 175 | { |
| 146 | 176 | if (!conn || !bt_sock_linked(&iso_sk_list, conn->sk)) |
| @@ -153,78 +183,81 @@ static struct sock *iso_sock_hold(struct iso_conn *conn) · 6 unmodified lines | ||
| 153 | 183 | |
| 154 | 184 | static void iso_sock_timeout(struct work_struct *work) |
| 155 | 185 | { |
| 156 | − struct iso_conn *conn = container_of(work, struct iso_conn, | |
| 157 | − timeout_work.work); | |
| 158 | − struct sock *sk; | |
| 159 | − | |
| 160 | − conn = iso_conn_hold_unless_zero(conn); | |
| 161 | − if (!conn) | |
| 162 | − return; | |
| 163 | − | |
| 164 | − iso_conn_lock(conn); | |
| 165 | − sk = iso_sock_hold(conn); | |
| 166 | − iso_conn_unlock(conn); | |
| 167 | − iso_conn_put(conn); | |
| 168 | − | |
| 169 | − if (!sk) | |
| 170 | − return; | |
| 186 | + struct iso_pinfo *pi = container_of(work, struct iso_pinfo, | |
| 187 | + timeout_work.work); | |
| 188 | + struct sock *sk = &pi->bt.sk; | |
| 171 | 189 | |
| 172 | 190 | BT_DBG("sock %p state %d", sk, sk->sk_state); |
| 173 | 191 | |
| 174 | 192 | lock_sock(sk); |
| 175 | − sk->sk_err = ETIMEDOUT; | |
| 176 | − sk->sk_state_change(sk); | |
| 193 | + if (!sock_flag(sk, SOCK_ZAPPED)) { | |
| 194 | + sk->sk_err = ETIMEDOUT; | |
| 195 | + sk->sk_state_change(sk); | |
| 196 | + } | |
| 177 | 197 | release_sock(sk); |
| 178 | − sock_put(sk); | |
| 179 | 198 | } |
| 180 | 199 | |
| 181 | 200 | static void iso_sock_set_timer(struct sock *sk, long timeout) |
| 182 | 201 | { |
| 202 | + lockdep_assert(lockdep_sock_is_held(sk)); | |
| 203 | + | |
| 204 | + cancel_delayed_work(&iso_pi(sk)->timeout_work); | |
| 205 | + | |
| 183 | 206 | if (!iso_pi(sk)->conn) |
| 184 | 207 | return; |
| 185 | 208 | |
| 186 | 209 | BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout); |
| 187 | − cancel_delayed_work(&iso_pi(sk)->conn->timeout_work); | |
| 188 | − schedule_delayed_work(&iso_pi(sk)->conn->timeout_work, timeout); | |
| 210 | + schedule_delayed_work(&iso_pi(sk)->timeout_work, timeout); | |
| 189 | 211 | } |
| 190 | 212 | |
| 191 | 213 | static void iso_sock_clear_timer(struct sock *sk) |
| 192 | 214 | { |
| 193 | − if (!iso_pi(sk)->conn) | |
| 194 | − return; | |
| 215 | + BT_DBG("sock %p state %d", sk, sk->sk_state); | |
| 216 | + cancel_delayed_work(&iso_pi(sk)->timeout_work); | |
| 217 | +} | |
| 218 | + | |
| 219 | +static void iso_sock_disable_timer(struct sock *sk) | |
| 220 | +{ | |
| 221 | + lockdep_assert(!lockdep_sock_is_held(sk)); | |
| 195 | 222 | |
| 196 | 223 | BT_DBG("sock %p state %d", sk, sk->sk_state); |
| 197 | − cancel_delayed_work(&iso_pi(sk)->conn->timeout_work); | |
| 224 | + disable_delayed_work_sync(&iso_pi(sk)->timeout_work); | |
| 198 | 225 | } |
| 199 | 226 | |
| 200 | 227 | /* ---- ISO connections ---- */ |
| 201 | 228 | static struct iso_conn *iso_conn_add(struct hci_conn *hcon) |
| 229 | + __must_hold(&hcon->hdev->lock) | |
| 202 | 230 | { |
| 203 | − struct iso_conn *conn = hcon->iso_data; | |
| 231 | + struct iso_conn *conn; | |
| 204 | 232 | |
| 205 | − conn = iso_conn_hold_unless_zero(conn); | |
| 233 | + spin_lock(&hcon->proto_lock); | |
| 234 | + | |
| 235 | + conn = iso_conn_hold_unless_zero(hcon->iso_data); | |
| 206 | 236 | if (conn) { |
| 207 | 237 | if (!conn->hcon) { |
| 208 | 238 | iso_conn_lock(conn); |
| 209 | 239 | conn->hcon = hcon; |
| 210 | 240 | iso_conn_unlock(conn); |
| 211 | 241 | } |
| 212 | − iso_conn_put(conn); | |
| 242 | + spin_unlock(&hcon->proto_lock); | |
| 213 | 243 | return conn; |
| 214 | 244 | } |
| 215 | 245 | |
| 216 | − conn = kzalloc_obj(*conn); | |
| 217 | − if (!conn) | |
| 246 | + conn = kzalloc_obj(*conn, GFP_ATOMIC); | |
| 247 | + if (!conn) { | |
| 248 | + spin_unlock(&hcon->proto_lock); | |
| 218 | 249 | return NULL; |
| 250 | + } | |
| 219 | 251 | |
| 220 | 252 | kref_init(&conn->ref); |
| 221 | 253 | spin_lock_init(&conn->lock); |
| 222 | − INIT_DELAYED_WORK(&conn->timeout_work, iso_sock_timeout); | |
| 223 | 254 | |
| 224 | 255 | hcon->iso_data = conn; |
| 225 | 256 | conn->hcon = hcon; |
| 226 | 257 | conn->tx_sn = 0; |
| 227 | 258 | |
| 259 | + spin_unlock(&hcon->proto_lock); | |
| 260 | + | |
| 228 | 261 | BT_DBG("hcon %p conn %p", hcon, conn); |
| 229 | 262 | |
| 230 | 263 | return conn; |
| @@ -263,11 +296,14 @@ static void iso_chan_del(struct sock *sk, int err) · 32 unmodified lines | ||
| 263 | 296 | } |
| 264 | 297 | |
| 265 | 298 | static void iso_conn_del(struct hci_conn *hcon, int err) |
| 299 | + __must_hold(&hcon->hdev->lock) | |
| 266 | 300 | { |
| 267 | − struct iso_conn *conn = hcon->iso_data; | |
| 301 | + struct iso_conn *conn; | |
| 268 | 302 | struct sock *sk; |
| 269 | 303 | |
| 270 | − conn = iso_conn_hold_unless_zero(conn); | |
| 304 | + spin_lock(&hcon->proto_lock); | |
| 305 | + conn = iso_conn_hold_unless_zero(hcon->iso_data); | |
| 306 | + spin_unlock(&hcon->proto_lock); | |
| 271 | 307 | if (!conn) |
| 272 | 308 | return; |
| 273 | 309 | |
| @@ -277,18 +313,28 @@ static void iso_conn_del(struct hci_conn *hcon, int err) · 3 unmodified lines | ||
| 277 | 313 | iso_conn_lock(conn); |
| 278 | 314 | sk = iso_sock_hold(conn); |
| 279 | 315 | iso_conn_unlock(conn); |
| 280 | − iso_conn_put(conn); | |
| 281 | 316 | |
| 282 | − if (!sk) { | |
| 283 | − iso_conn_put(conn); | |
| 284 | − return; | |
| 285 | − } | |
| 317 | + if (!sk) | |
| 318 | + goto done; | |
| 319 | + | |
| 320 | + iso_sock_disable_timer(sk); | |
| 286 | 321 | |
| 287 | 322 | lock_sock(sk); |
| 288 | − iso_sock_clear_timer(sk); | |
| 289 | 323 | iso_chan_del(sk, err); |
| 290 | 324 | release_sock(sk); |
| 325 | + iso_sock_kill(sk); | |
| 291 | 326 | sock_put(sk); |
| 327 | + | |
| 328 | +done: | |
| 329 | + /* No sk access to conn->hcon any more (lock_sock + hdev->lock) */ | |
| 330 | + spin_lock(&hcon->proto_lock); | |
| 331 | + iso_conn_lock(conn); | |
| 332 | + conn->hcon = NULL; | |
| 333 | + hcon->iso_data = NULL; | |
| 334 | + iso_conn_unlock(conn); | |
| 335 | + spin_unlock(&hcon->proto_lock); | |
| 336 | + | |
| 337 | + iso_conn_put(conn); | |
| 292 | 338 | } |
| 293 | 339 | |
| 294 | 340 | static int __iso_chan_add(struct iso_conn *conn, struct sock *sk, |
| @@ -304,8 +350,14 @@ static int __iso_chan_add(struct iso_conn *conn, struct sock *sk, · 9 unmodified lines | ||
| 304 | 350 | return -EBUSY; |
| 305 | 351 | } |
| 306 | 352 | |
| 307 | − iso_pi(sk)->conn = conn; | |
| 353 | + if (!conn->hcon) { | |
| 354 | + BT_ERR("conn->hcon missing"); | |
| 355 | + return -EIO; | |
| 356 | + } | |
| 357 | + | |
| 358 | + iso_pi(sk)->conn = iso_conn_hold(conn); | |
| 308 | 359 | conn->sk = sk; |
| 360 | + clear_bit(ISO_CONN_DROPPED, conn->flags); | |
| 309 | 361 | |
| 310 | 362 | if (parent) |
| 311 | 363 | bt_accept_enqueue(parent, sk, true); |
| @@ -402,6 +454,8 @@ static int iso_connect_bis(struct sock *sk) · 90 unmodified lines | ||
| 402 | 454 | iso_pi(sk)->bc_sid = hcon->sid; |
| 403 | 455 | } |
| 404 | 456 | |
| 457 | + lockdep_assert_held(&hcon->hdev->lock); | |
| 458 | + | |
| 405 | 459 | conn = iso_conn_add(hcon); |
| 406 | 460 | if (!conn) { |
| 407 | 461 | hci_conn_drop(hcon); |
| @@ -410,6 +464,7 @@ static int iso_connect_bis(struct sock *sk) · 2 unmodified lines | ||
| 410 | 464 | } |
| 411 | 465 | |
| 412 | 466 | err = iso_chan_add(conn, sk, NULL); |
| 467 | + iso_conn_put(conn); | |
| 413 | 468 | if (err) |
| 414 | 469 | goto unlock; |
| 415 | 470 | |
| @@ -504,6 +559,8 @@ static int iso_connect_cis(struct sock *sk) · 88 unmodified lines | ||
| 504 | 559 | } |
| 505 | 560 | } |
| 506 | 561 | |
| 562 | + lockdep_assert_held(&hcon->hdev->lock); | |
| 563 | + | |
| 507 | 564 | conn = iso_conn_add(hcon); |
| 508 | 565 | if (!conn) { |
| 509 | 566 | hci_conn_drop(hcon); |
| @@ -512,6 +569,7 @@ static int iso_connect_cis(struct sock *sk) · 2 unmodified lines | ||
| 512 | 569 | } |
| 513 | 570 | |
| 514 | 571 | err = iso_chan_add(conn, sk, NULL); |
| 572 | + iso_conn_put(conn); | |
| 515 | 573 | if (err) |
| 516 | 574 | goto unlock; |
| 517 | 575 | |
| @@ -790,9 +848,15 @@ static void iso_sock_cleanup_listen(struct sock *parent) · 272 unmodified lines | ||
| 790 | 848 | */ |
| 791 | 849 | static void iso_sock_kill(struct sock *sk) |
| 792 | 850 | { |
| 851 | + iso_sock_disable_timer(sk); | |
| 852 | + | |
| 853 | + lock_sock(sk); | |
| 854 | + | |
| 793 | 855 | if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket || |
| 794 | − sock_flag(sk, SOCK_DEAD)) | |
| 856 | + test_bit(BT_SK_KILLED, &iso_pi(sk)->flags)) { | |
| 857 | + release_sock(sk); | |
| 795 | 858 | return; |
| 859 | + } | |
| 796 | 860 | |
| 797 | 861 | BT_DBG("sk %p state %d", sk, sk->sk_state); |
| 798 | 862 | |
| @@ -806,6 +870,9 @@ static void iso_sock_kill(struct sock *sk) · 7 unmodified lines | ||
| 806 | 870 | /* Kill poor orphan */ |
| 807 | 871 | bt_sock_unlink(&iso_sk_list, sk); |
| 808 | 872 | sock_set_flag(sk, SOCK_DEAD); |
| 873 | + set_bit(BT_SK_KILLED, &iso_pi(sk)->flags); | |
| 874 | + | |
| 875 | + release_sock(sk); | |
| 809 | 876 | sock_put(sk); |
| 810 | 877 | } |
| 811 | 878 | |
| @@ -825,8 +892,8 @@ static void iso_sock_disconn(struct sock *sk) · 13 unmodified lines | ||
| 825 | 892 | */ |
| 826 | 893 | if (bis_sk) { |
| 827 | 894 | hcon->state = BT_OPEN; |
| 828 | − hcon->iso_data = NULL; | |
| 829 | − iso_pi(sk)->conn->hcon = NULL; | |
| 895 | + set_bit(ISO_CONN_DROPPED, iso_pi(sk)->conn->flags); | |
| 896 | + | |
| 830 | 897 | iso_sock_clear_timer(sk); |
| 831 | 898 | iso_chan_del(sk, bt_to_errno(hcon->abort_reason)); |
| 832 | 899 | sock_put(bis_sk); |
| @@ -835,10 +902,8 @@ static void iso_sock_disconn(struct sock *sk) · 2 unmodified lines | ||
| 835 | 902 | } |
| 836 | 903 | |
| 837 | 904 | sk->sk_state = BT_DISCONN; |
| 838 | − iso_conn_lock(iso_pi(sk)->conn); | |
| 839 | − hci_conn_drop(iso_pi(sk)->conn->hcon); | |
| 840 | − iso_pi(sk)->conn->hcon = NULL; | |
| 841 | − iso_conn_unlock(iso_pi(sk)->conn); | |
| 905 | + if (!test_and_set_bit(ISO_CONN_DROPPED, iso_pi(sk)->conn->flags)) | |
| 906 | + hci_conn_drop(iso_pi(sk)->conn->hcon); | |
| 842 | 907 | } |
| 843 | 908 | |
| 844 | 909 | static void __iso_sock_close(struct sock *sk) |
| @@ -880,11 +945,11 @@ static void __iso_sock_close(struct sock *sk) · 35 unmodified lines | ||
| 880 | 945 | /* Must be called on unlocked socket. */ |
| 881 | 946 | static void iso_sock_close(struct sock *sk) |
| 882 | 947 | { |
| 948 | + iso_sock_disable_timer(sk); | |
| 949 | + | |
| 883 | 950 | lock_sock(sk); |
| 884 | − iso_sock_clear_timer(sk); | |
| 885 | 951 | __iso_sock_close(sk); |
| 886 | 952 | release_sock(sk); |
| 887 | − iso_sock_kill(sk); | |
| 888 | 953 | } |
| 889 | 954 | |
| 890 | 955 | static void iso_sock_init(struct sock *sk, struct sock *parent) |
| @@ -951,6 +1016,8 @@ static struct sock *iso_sock_alloc(struct net *net, struct socket *sock, · 60 unmodified lines | ||
| 951 | 1016 | iso_pi(sk)->qos = default_qos; |
| 952 | 1017 | iso_pi(sk)->sync_handle = -1; |
| 953 | 1018 | |
| 1019 | + INIT_DELAYED_WORK(&iso_pi(sk)->timeout_work, iso_sock_timeout); | |
| 1020 | + | |
| 954 | 1021 | bt_sock_link(&iso_sk_list, sk); |
| 955 | 1022 | return sk; |
| 956 | 1023 | } |
| @@ -1033,15 +1100,15 @@ static int iso_sock_rebind_bis(struct sock *sk, struct sockaddr_iso *sa, · 76 unmodified lines | ||
| 1033 | 1100 | goto done; |
| 1034 | 1101 | } |
| 1035 | 1102 | |
| 1036 | − iso_pi(sk)->bc_num_bis = sa->iso_bc->bc_num_bis; | |
| 1037 | − | |
| 1038 | − for (int i = 0; i < iso_pi(sk)->bc_num_bis; i++) | |
| 1103 | + for (int i = 0; i < sa->iso_bc->bc_num_bis; i++) | |
| 1039 | 1104 | if (sa->iso_bc->bc_bis[i] < 0x01 || |
| 1040 | 1105 | sa->iso_bc->bc_bis[i] > 0x1f) { |
| 1041 | 1106 | err = -EINVAL; |
| 1042 | 1107 | goto done; |
| 1043 | 1108 | } |
| 1044 | 1109 | |
| 1110 | + iso_pi(sk)->bc_num_bis = sa->iso_bc->bc_num_bis; | |
| 1111 | + | |
| 1045 | 1112 | memcpy(iso_pi(sk)->bc_bis, sa->iso_bc->bc_bis, |
| 1046 | 1113 | iso_pi(sk)->bc_num_bis); |
| 1047 | 1114 | |
| @@ -1276,6 +1343,8 @@ static int iso_listen_bis(struct sock *sk) · 228 unmodified lines | ||
| 1276 | 1343 | goto unlock; |
| 1277 | 1344 | } |
| 1278 | 1345 | |
| 1346 | + lockdep_assert_held(&hcon->hdev->lock); | |
| 1347 | + | |
| 1279 | 1348 | conn = iso_conn_add(hcon); |
| 1280 | 1349 | if (!conn) { |
| 1281 | 1350 | hci_conn_drop(hcon); |
| @@ -1284,10 +1353,9 @@ static int iso_listen_bis(struct sock *sk) · 2 unmodified lines | ||
| 1284 | 1353 | } |
| 1285 | 1354 | |
| 1286 | 1355 | err = iso_chan_add(conn, sk, NULL); |
| 1287 | − if (err) { | |
| 1288 | − hci_conn_drop(hcon); | |
| 1356 | + iso_conn_put(conn); | |
| 1357 | + if (err) | |
| 1289 | 1358 | goto unlock; |
| 1290 | − } | |
| 1291 | 1359 | |
| 1292 | 1360 | unlock: |
| 1293 | 1361 | release_sock(sk); |
| @@ -1466,6 +1534,8 @@ static int iso_sock_getname(struct socket *sock, struct sockaddr *addr, · 172 unmodified lines | ||
| 1466 | 1534 | |
| 1467 | 1535 | BT_DBG("sock %p, sk %p", sock, sk); |
| 1468 | 1536 | |
| 1537 | + lock_sock(sk); | |
| 1538 | + | |
| 1469 | 1539 | addr->sa_family = AF_BLUETOOTH; |
| 1470 | 1540 | |
| 1471 | 1541 | if (peer) { |
| @@ -1487,6 +1557,8 @@ static int iso_sock_getname(struct socket *sock, struct sockaddr *addr, · 15 unmodified lines | ||
| 1487 | 1557 | sa->iso_bdaddr_type = iso_pi(sk)->src_type; |
| 1488 | 1558 | } |
| 1489 | 1559 | |
| 1560 | + release_sock(sk); | |
| 1561 | + | |
| 1490 | 1562 | return len; |
| 1491 | 1563 | } |
| 1492 | 1564 | |
| @@ -1786,7 +1858,7 @@ static bool check_bcast_qos(struct bt_iso_qos *qos) · 293 unmodified lines | ||
| 1786 | 1858 | return false; |
| 1787 | 1859 | |
| 1788 | 1860 | if (!qos->bcast.timeout) |
| 1789 | − qos->bcast.sync_timeout = BT_ISO_SYNC_TIMEOUT; | |
| 1861 | + qos->bcast.timeout = BT_ISO_SYNC_TIMEOUT; | |
| 1790 | 1862 | |
| 1791 | 1863 | if (qos->bcast.timeout < 0x000a || qos->bcast.timeout > 0x4000) |
| 1792 | 1864 | return false; |
| @@ -2028,8 +2100,16 @@ static int iso_sock_release(struct socket *sock) · 235 unmodified lines | ||
| 2028 | 2100 | release_sock(sk); |
| 2029 | 2101 | } |
| 2030 | 2102 | |
| 2103 | + /* Make sure sk is valid even if iso_conn_del() is concurrent */ | |
| 2104 | + sock_hold(sk); | |
| 2105 | + | |
| 2106 | + lock_sock(sk); | |
| 2031 | 2107 | sock_orphan(sk); |
| 2108 | + release_sock(sk); | |
| 2109 | + | |
| 2032 | 2110 | iso_sock_kill(sk); |
| 2111 | + | |
| 2112 | + sock_put(sk); | |
| 2033 | 2113 | return err; |
| 2034 | 2114 | } |
| 2035 | 2115 | |
| @@ -2037,14 +2117,17 @@ static void iso_sock_ready(struct sock *sk) · 1 unmodified line | ||
| 2037 | 2117 | { |
| 2038 | 2118 | BT_DBG("sk %p", sk); |
| 2039 | 2119 | |
| 2040 | − if (!sk) | |
| 2120 | + lockdep_assert(lockdep_sock_is_held(sk)); | |
| 2121 | + | |
| 2122 | + switch (sk->sk_state) { | |
| 2123 | + case BT_DISCONN: | |
| 2124 | + case BT_CLOSED: | |
| 2041 | 2125 | return; |
| 2126 | + } | |
| 2042 | 2127 | |
| 2043 | − lock_sock(sk); | |
| 2044 | 2128 | iso_sock_clear_timer(sk); |
| 2045 | 2129 | sk->sk_state = BT_CONNECTED; |
| 2046 | 2130 | sk->sk_state_change(sk); |
| 2047 | − release_sock(sk); | |
| 2048 | 2131 | } |
| 2049 | 2132 | |
| 2050 | 2133 | static bool iso_match_big(struct sock *sk, void *data) |
| @@ -2074,7 +2157,7 @@ static bool iso_match_dst(struct sock *sk, void *data) · 23 unmodified lines | ||
| 2074 | 2157 | static void iso_conn_ready(struct iso_conn *conn) |
| 2075 | 2158 | { |
| 2076 | 2159 | struct sock *parent = NULL; |
| 2077 | − struct sock *sk = conn->sk; | |
| 2160 | + struct sock *sk; | |
| 2078 | 2161 | struct hci_ev_le_big_sync_established *ev = NULL; |
| 2079 | 2162 | struct hci_ev_le_pa_sync_established *ev2 = NULL; |
| 2080 | 2163 | struct hci_ev_le_per_adv_report *ev3 = NULL; |
| @@ -2083,7 +2166,22 @@ static void iso_conn_ready(struct iso_conn *conn) · 2 unmodified lines | ||
| 2083 | 2166 | |
| 2084 | 2167 | BT_DBG("conn %p", conn); |
| 2085 | 2168 | |
| 2169 | + iso_conn_lock(conn); | |
| 2170 | + sk = iso_sock_hold(conn); | |
| 2171 | + iso_conn_unlock(conn); | |
| 2172 | + | |
| 2086 | 2173 | if (sk) { |
| 2174 | + lock_sock(sk); | |
| 2175 | + | |
| 2176 | + /* conn->sk may have become NULL if racing with sk close, but | |
| 2177 | + * due to held hdev->lock, it can't become different sk. | |
| 2178 | + */ | |
| 2179 | + if (!conn->sk) { | |
| 2180 | + release_sock(sk); | |
| 2181 | + sock_put(sk); | |
| 2182 | + return; | |
| 2183 | + } | |
| 2184 | + | |
| 2087 | 2185 | /* Attempt to update source address in case of BIS Sender if |
| 2088 | 2186 | * the advertisement is using a random address. |
| 2089 | 2187 | */ |
| @@ -2096,14 +2194,15 @@ static void iso_conn_ready(struct iso_conn *conn) · 6 unmodified lines | ||
| 2096 | 2194 | adv = hci_find_adv_instance(bis->hdev, |
| 2097 | 2195 | bis->iso_qos.bcast.bis); |
| 2098 | 2196 | if (adv && bacmp(&adv->random_addr, BDADDR_ANY)) { |
| 2099 | − lock_sock(sk); | |
| 2100 | 2197 | iso_pi(sk)->src_type = BDADDR_LE_RANDOM; |
| 2101 | 2198 | bacpy(&iso_pi(sk)->src, &adv->random_addr); |
| 2102 | − release_sock(sk); | |
| 2103 | 2199 | } |
| 2104 | 2200 | } |
| 2105 | 2201 | |
| 2106 | − iso_sock_ready(conn->sk); | |
| 2202 | + iso_sock_ready(sk); | |
| 2203 | + | |
| 2204 | + release_sock(sk); | |
| 2205 | + sock_put(sk); | |
| 2107 | 2206 | } else { |
| 2108 | 2207 | hcon = conn->hcon; |
| 2109 | 2208 | if (!hcon) |
| @@ -2350,7 +2449,7 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 240 unmodified lines | ||
| 2350 | 2449 | |
| 2351 | 2450 | lock_sock(sk); |
| 2352 | 2451 | |
| 2353 | − hcon = iso_pi(sk)->conn->hcon; | |
| 2452 | + hcon = iso_pi(sk)->conn ? iso_pi(sk)->conn->hcon : NULL; | |
| 2354 | 2453 | iso_pi(sk)->qos.bcast.encryption = ev2->encryption; |
| 2355 | 2454 | |
| 2356 | 2455 | if (ev2->num_bis < iso_pi(sk)->bc_num_bis) |
| @@ -2390,9 +2489,11 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 33 unmodified lines | ||
| 2390 | 2489 | if (!sk) |
| 2391 | 2490 | goto done; |
| 2392 | 2491 | |
| 2393 | − hcon = iso_pi(sk)->conn->hcon; | |
| 2492 | + lock_sock(sk); | |
| 2493 | + | |
| 2494 | + hcon = iso_pi(sk)->conn ? iso_pi(sk)->conn->hcon : NULL; | |
| 2394 | 2495 | if (!hcon) |
| 2395 | − goto done; | |
| 2496 | + goto release3; | |
| 2396 | 2497 | |
| 2397 | 2498 | if (ev3->data_status == LE_PA_DATA_TRUNCATED) { |
| 2398 | 2499 | /* The controller was unable to retrieve PA data. */ |
| @@ -2400,12 +2501,12 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 1 unmodified line | ||
| 2400 | 2501 | HCI_MAX_PER_AD_TOT_LEN); |
| 2401 | 2502 | hcon->le_per_adv_data_len = 0; |
| 2402 | 2503 | hcon->le_per_adv_data_offset = 0; |
| 2403 | − goto done; | |
| 2504 | + goto release3; | |
| 2404 | 2505 | } |
| 2405 | 2506 | |
| 2406 | 2507 | if (hcon->le_per_adv_data_offset + ev3->length > |
| 2407 | 2508 | HCI_MAX_PER_AD_TOT_LEN) |
| 2408 | − goto done; | |
| 2509 | + goto release3; | |
| 2409 | 2510 | |
| 2410 | 2511 | memcpy(hcon->le_per_adv_data + hcon->le_per_adv_data_offset, |
| 2411 | 2512 | ev3->data, ev3->length); |
| @@ -2424,18 +2525,19 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 12 unmodified lines | ||
| 2424 | 2525 | &base_len); |
| 2425 | 2526 | |
| 2426 | 2527 | if (!base || base_len > BASE_MAX_LENGTH) |
| 2427 | − goto done; | |
| 2528 | + goto release3; | |
| 2428 | 2529 | |
| 2429 | − lock_sock(sk); | |
| 2430 | 2530 | memcpy(iso_pi(sk)->base, base, base_len); |
| 2431 | 2531 | iso_pi(sk)->base_len = base_len; |
| 2432 | − release_sock(sk); | |
| 2433 | 2532 | } else { |
| 2434 | 2533 | /* This is a PA data fragment. Keep pa_data_len set to 0 |
| 2435 | 2534 | * until all data has been reassembled. |
| 2436 | 2535 | */ |
| 2437 | 2536 | hcon->le_per_adv_data_len = 0; |
| 2438 | 2537 | } |
| 2538 | + | |
| 2539 | +release3: | |
| 2540 | + release_sock(sk); | |
| 2439 | 2541 | } else { |
| 2440 | 2542 | sk = iso_get_sock(hdev, &hdev->bdaddr, BDADDR_ANY, |
| 2441 | 2543 | BT_LISTEN, iso_match_dst, BDADDR_ANY); |
| @@ -2454,6 +2556,7 @@ done: · 12 unmodified lines | ||
| 2454 | 2556 | } |
| 2455 | 2557 | |
| 2456 | 2558 | static void iso_connect_cfm(struct hci_conn *hcon, __u8 status) |
| 2559 | + __must_hold(&hcon->hdev->lock) | |
| 2457 | 2560 | { |
| 2458 | 2561 | if (hcon->type != CIS_LINK && hcon->type != BIS_LINK && |
| 2459 | 2562 | hcon->type != PA_LINK) { |
| @@ -2465,8 +2568,10 @@ static void iso_connect_cfm(struct hci_conn *hcon, __u8 status) · 5 unmodified lines | ||
| 2465 | 2568 | struct hci_link *link, *t; |
| 2466 | 2569 | |
| 2467 | 2570 | list_for_each_entry_safe(link, t, &hcon->link_list, |
| 2468 | − list) | |
| 2571 | + list) { | |
| 2572 | + lockdep_assert_held(&link->conn->hdev->lock); | |
| 2469 | 2573 | iso_conn_del(link->conn, bt_to_errno(status)); |
| 2574 | + } | |
| 2470 | 2575 | |
| 2471 | 2576 | return; |
| 2472 | 2577 | } |
| @@ -2488,14 +2593,17 @@ static void iso_connect_cfm(struct hci_conn *hcon, __u8 status) · 15 unmodified lines | ||
| 2488 | 2593 | struct iso_conn *conn; |
| 2489 | 2594 | |
| 2490 | 2595 | conn = iso_conn_add(hcon); |
| 2491 | − if (conn) | |
| 2596 | + if (conn) { | |
| 2492 | 2597 | iso_conn_ready(conn); |
| 2598 | + iso_conn_put(conn); | |
| 2599 | + } | |
| 2493 | 2600 | } else { |
| 2494 | 2601 | iso_conn_del(hcon, bt_to_errno(status)); |
| 2495 | 2602 | } |
| 2496 | 2603 | } |
| 2497 | 2604 | |
| 2498 | 2605 | static void iso_disconn_cfm(struct hci_conn *hcon, __u8 reason) |
| 2606 | + __must_hold(&hcon->hdev->lock) | |
| 2499 | 2607 | { |
| 2500 | 2608 | if (hcon->type != CIS_LINK && hcon->type != BIS_LINK && |
| 2501 | 2609 | hcon->type != PA_LINK) |
| @@ -2522,7 +2630,10 @@ int iso_recv(struct hci_dev *hdev, u16 handle, struct sk_buff *skb, u16 flags) · 20 unmodified lines | ||
| 2522 | 2630 | return -ENOENT; |
| 2523 | 2631 | } |
| 2524 | 2632 | |
| 2633 | + spin_lock(&hcon->proto_lock); | |
| 2525 | 2634 | conn = iso_conn_hold_unless_zero(hcon->iso_data); |
| 2635 | + spin_unlock(&hcon->proto_lock); | |
| 2636 | + | |
| 2526 | 2637 | hcon = NULL; |
| 2527 | 2638 | |
| 2528 | 2639 | hci_dev_unlock(hdev); |
| @@ -4820,6 +4820,10 @@ static int l2cap_le_connect_rsp(struct l2cap_conn *conn, · 4819 unmodified lines | ||
| 4820 | 4820 | if (!chan) |
| 4821 | 4821 | return -EBADSLT; |
| 4822 | 4822 | |
| 4823 | + chan = l2cap_chan_hold_unless_zero(chan); | |
| 4824 | + if (!chan) | |
| 4825 | + return -EBADSLT; | |
| 4826 | + | |
| 4823 | 4827 | err = 0; |
| 4824 | 4828 | |
| 4825 | 4829 | l2cap_chan_lock(chan); |
| @@ -4865,6 +4869,7 @@ static int l2cap_le_connect_rsp(struct l2cap_conn *conn, · 39 unmodified lines | ||
| 4865 | 4869 | } |
| 4866 | 4870 | |
| 4867 | 4871 | l2cap_chan_unlock(chan); |
| 4872 | + l2cap_chan_put(chan); | |
| 4868 | 4873 | |
| 4869 | 4874 | return err; |
| 4870 | 4875 | } |
| @@ -2696,18 +2696,28 @@ static int mgmt_hci_cmd_sync(struct sock *sk, struct hci_dev *hdev, · 2695 unmodified lines | ||
| 2696 | 2696 | static bool pending_eir_or_class(struct hci_dev *hdev) |
| 2697 | 2697 | { |
| 2698 | 2698 | struct mgmt_pending_cmd *cmd; |
| 2699 | + bool pending = false; | |
| 2700 | + | |
| 2701 | + mutex_lock(&hdev->mgmt_pending_lock); | |
| 2699 | 2702 | |
| 2700 | 2703 | list_for_each_entry(cmd, &hdev->mgmt_pending, list) { |
| 2701 | 2704 | switch (cmd->opcode) { |
| 2702 | 2705 | case MGMT_OP_ADD_UUID: |
| 2703 | 2706 | case MGMT_OP_REMOVE_UUID: |
| 2704 | 2707 | case MGMT_OP_SET_DEV_CLASS: |
| 2708 | + case MGMT_OP_SET_LOCAL_NAME: | |
| 2705 | 2709 | case MGMT_OP_SET_POWERED: |
| 2706 | − return true; | |
| 2710 | + pending = true; | |
| 2711 | + break; | |
| 2707 | 2712 | } |
| 2713 | + | |
| 2714 | + if (pending) | |
| 2715 | + break; | |
| 2708 | 2716 | } |
| 2709 | 2717 | |
| 2710 | − return false; | |
| 2718 | + mutex_unlock(&hdev->mgmt_pending_lock); | |
| 2719 | + | |
| 2720 | + return pending; | |
| 2711 | 2721 | } |
| 2712 | 2722 | |
| 2713 | 2723 | static const u8 bluetooth_base_uuid[] = { |
| @@ -3514,11 +3524,13 @@ static int set_io_capability(struct sock *sk, struct hci_dev *hdev, void *data, · 800 unmodified lines | ||
| 3514 | 3524 | NULL, 0); |
| 3515 | 3525 | } |
| 3516 | 3526 | |
| 3517 | −static struct mgmt_pending_cmd *find_pairing(struct hci_conn *conn) | |
| 3527 | +static struct mgmt_pending_cmd *remove_pairing(struct hci_conn *conn) | |
| 3518 | 3528 | { |
| 3519 | 3529 | struct hci_dev *hdev = conn->hdev; |
| 3520 | 3530 | struct mgmt_pending_cmd *cmd; |
| 3521 | 3531 | |
| 3532 | + mutex_lock(&hdev->mgmt_pending_lock); | |
| 3533 | + | |
| 3522 | 3534 | list_for_each_entry(cmd, &hdev->mgmt_pending, list) { |
| 3523 | 3535 | if (cmd->opcode != MGMT_OP_PAIR_DEVICE) |
| 3524 | 3536 | continue; |
| @@ -3526,9 +3538,39 @@ static struct mgmt_pending_cmd *find_pairing(struct hci_conn *conn) · 1 unmodified line | ||
| 3526 | 3538 | if (cmd->user_data != conn) |
| 3527 | 3539 | continue; |
| 3528 | 3540 | |
| 3541 | + list_del(&cmd->list); | |
| 3542 | + mutex_unlock(&hdev->mgmt_pending_lock); | |
| 3529 | 3543 | return cmd; |
| 3530 | 3544 | } |
| 3531 | 3545 | |
| 3546 | + mutex_unlock(&hdev->mgmt_pending_lock); | |
| 3547 | + | |
| 3548 | + return NULL; | |
| 3549 | +} | |
| 3550 | + | |
| 3551 | +static struct mgmt_pending_cmd *remove_pairing_by_addr(struct hci_dev *hdev, | |
| 3552 | + bdaddr_t *bdaddr) | |
| 3553 | +{ | |
| 3554 | + struct mgmt_pending_cmd *cmd; | |
| 3555 | + struct hci_conn *conn; | |
| 3556 | + | |
| 3557 | + mutex_lock(&hdev->mgmt_pending_lock); | |
| 3558 | + | |
| 3559 | + list_for_each_entry(cmd, &hdev->mgmt_pending, list) { | |
| 3560 | + if (cmd->opcode != MGMT_OP_PAIR_DEVICE) | |
| 3561 | + continue; | |
| 3562 | + | |
| 3563 | + conn = cmd->user_data; | |
| 3564 | + if (bacmp(bdaddr, &conn->dst) != 0) | |
| 3565 | + continue; | |
| 3566 | + | |
| 3567 | + list_del(&cmd->list); | |
| 3568 | + mutex_unlock(&hdev->mgmt_pending_lock); | |
| 3569 | + return cmd; | |
| 3570 | + } | |
| 3571 | + | |
| 3572 | + mutex_unlock(&hdev->mgmt_pending_lock); | |
| 3573 | + | |
| 3532 | 3574 | return NULL; |
| 3533 | 3575 | } |
| 3534 | 3576 | |
| @@ -3566,10 +3608,10 @@ void mgmt_smp_complete(struct hci_conn *conn, bool complete) · 31 unmodified lines | ||
| 3566 | 3608 | u8 status = complete ? MGMT_STATUS_SUCCESS : MGMT_STATUS_FAILED; |
| 3567 | 3609 | struct mgmt_pending_cmd *cmd; |
| 3568 | 3610 | |
| 3569 | − cmd = find_pairing(conn); | |
| 3611 | + cmd = remove_pairing(conn); | |
| 3570 | 3612 | if (cmd) { |
| 3571 | 3613 | cmd->cmd_complete(cmd, status); |
| 3572 | − mgmt_pending_remove(cmd); | |
| 3614 | + mgmt_pending_free(cmd); | |
| 3573 | 3615 | } |
| 3574 | 3616 | } |
| 3575 | 3617 | |
| @@ -3579,14 +3621,14 @@ static void pairing_complete_cb(struct hci_conn *conn, u8 status) · 3 unmodified lines | ||
| 3579 | 3621 | |
| 3580 | 3622 | BT_DBG("status %u", status); |
| 3581 | 3623 | |
| 3582 | − cmd = find_pairing(conn); | |
| 3624 | + cmd = remove_pairing(conn); | |
| 3583 | 3625 | if (!cmd) { |
| 3584 | 3626 | BT_DBG("Unable to find a pending command"); |
| 3585 | 3627 | return; |
| 3586 | 3628 | } |
| 3587 | 3629 | |
| 3588 | 3630 | cmd->cmd_complete(cmd, mgmt_status(status)); |
| 3589 | − mgmt_pending_remove(cmd); | |
| 3631 | + mgmt_pending_free(cmd); | |
| 3590 | 3632 | } |
| 3591 | 3633 | |
| 3592 | 3634 | static void le_pairing_complete_cb(struct hci_conn *conn, u8 status) |
| @@ -3598,14 +3640,14 @@ static void le_pairing_complete_cb(struct hci_conn *conn, u8 status) · 5 unmodified lines | ||
| 3598 | 3640 | if (!status) |
| 3599 | 3641 | return; |
| 3600 | 3642 | |
| 3601 | − cmd = find_pairing(conn); | |
| 3643 | + cmd = remove_pairing(conn); | |
| 3602 | 3644 | if (!cmd) { |
| 3603 | 3645 | BT_DBG("Unable to find a pending command"); |
| 3604 | 3646 | return; |
| 3605 | 3647 | } |
| 3606 | 3648 | |
| 3607 | 3649 | cmd->cmd_complete(cmd, mgmt_status(status)); |
| 3608 | − mgmt_pending_remove(cmd); | |
| 3650 | + mgmt_pending_free(cmd); | |
| 3609 | 3651 | } |
| 3610 | 3652 | |
| 3611 | 3653 | static int pair_device(struct sock *sk, struct hci_dev *hdev, void *data, |
| @@ -3762,23 +3804,17 @@ static int cancel_pair_device(struct sock *sk, struct hci_dev *hdev, void *data, · 150 unmodified lines | ||
| 3762 | 3804 | goto unlock; |
| 3763 | 3805 | } |
| 3764 | 3806 | |
| 3765 | − cmd = pending_find(MGMT_OP_PAIR_DEVICE, hdev); | |
| 3807 | + cmd = remove_pairing_by_addr(hdev, &addr->bdaddr); | |
| 3766 | 3808 | if (!cmd) { |
| 3767 | 3809 | err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, |
| 3768 | 3810 | MGMT_STATUS_INVALID_PARAMS); |
| 3769 | 3811 | goto unlock; |
| 3770 | 3812 | } |
| 3771 | 3813 | |
| 3772 | − conn = cmd->user_data; | |
| 3773 | − | |
| 3774 | − if (bacmp(&addr->bdaddr, &conn->dst) != 0) { | |
| 3775 | − err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, | |
| 3776 | − MGMT_STATUS_INVALID_PARAMS); | |
| 3777 | − goto unlock; | |
| 3778 | − } | |
| 3814 | + conn = hci_conn_get(cmd->user_data); | |
| 3779 | 3815 | |
| 3780 | 3816 | cmd->cmd_complete(cmd, MGMT_STATUS_CANCELLED); |
| 3781 | − mgmt_pending_remove(cmd); | |
| 3817 | + mgmt_pending_free(cmd); | |
| 3782 | 3818 | |
| 3783 | 3819 | err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, 0, |
| 3784 | 3820 | addr, sizeof(*addr)); |
| @@ -3796,6 +3832,8 @@ static int cancel_pair_device(struct sock *sk, struct hci_dev *hdev, void *data, · 11 unmodified lines | ||
| 3796 | 3832 | if (conn->conn_reason == CONN_REASON_PAIR_DEVICE) |
| 3797 | 3833 | hci_abort_conn(conn, HCI_ERROR_REMOTE_USER_TERM); |
| 3798 | 3834 | |
| 3835 | + hci_conn_put(conn); | |
| 3836 | + | |
| 3799 | 3837 | unlock: |
| 3800 | 3838 | hci_dev_unlock(hdev); |
| 3801 | 3839 | return err; |
| @@ -4043,6 +4081,12 @@ static int set_local_name(struct sock *sk, struct hci_dev *hdev, void *data, · 241 unmodified lines | ||
| 4043 | 4081 | goto failed; |
| 4044 | 4082 | } |
| 4045 | 4083 | |
| 4084 | + if (hdev_is_powered(hdev) && pending_eir_or_class(hdev)) { | |
| 4085 | + err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_LOCAL_NAME, | |
| 4086 | + MGMT_STATUS_BUSY); | |
| 4087 | + goto failed; | |
| 4088 | + } | |
| 4089 | + | |
| 4046 | 4090 | memcpy(hdev->short_name, cp->short_name, sizeof(hdev->short_name)); |
| 4047 | 4091 | |
| 4048 | 4092 | if (!hdev_is_powered(hdev)) { |
| @@ -10137,14 +10181,14 @@ void mgmt_auth_failed(struct hci_conn *conn, u8 hci_status) · 6088 unmodified lines | ||
| 10137 | 10181 | ev.addr.type = link_to_bdaddr(conn->type, conn->dst_type); |
| 10138 | 10182 | ev.status = status; |
| 10139 | 10183 | |
| 10140 | − cmd = find_pairing(conn); | |
| 10184 | + cmd = remove_pairing(conn); | |
| 10141 | 10185 | |
| 10142 | 10186 | mgmt_event(MGMT_EV_AUTH_FAILED, conn->hdev, &ev, sizeof(ev), |
| 10143 | 10187 | cmd ? cmd->sk : NULL); |
| 10144 | 10188 | |
| 10145 | 10189 | if (cmd) { |
| 10146 | 10190 | cmd->cmd_complete(cmd, status); |
| 10147 | − mgmt_pending_remove(cmd); | |
| 10191 | + mgmt_pending_free(cmd); | |
| 10148 | 10192 | } |
| 10149 | 10193 | } |
| 10150 | 10194 | |
| @@ -1795,6 +1795,11 @@ static struct rfcomm_session *rfcomm_recv_frame(struct rfcomm_session *s, · 1794 unmodified lines | ||
| 1795 | 1795 | return s; |
| 1796 | 1796 | } |
| 1797 | 1797 | |
| 1798 | + if (skb->len < sizeof(*hdr) + 1) { | |
| 1799 | + kfree_skb(skb); | |
| 1800 | + return s; | |
| 1801 | + } | |
| 1802 | + | |
| 1798 | 1803 | dlci = __get_dlci(hdr->addr); |
| 1799 | 1804 | type = __get_type(hdr->ctrl); |
| 1800 | 1805 | |
| @@ -188,6 +188,9 @@ static void sco_sock_clear_timer(struct sock *sk) · 187 unmodified lines | ||
| 188 | 188 | } |
| 189 | 189 | |
| 190 | 190 | /* ---- SCO connections ---- */ |
| 191 | +/* Consumes a reference on @hcon, which the returned sco_conn owns until it is | |
| 192 | + * freed. On failure (NULL return) the reference is left for the caller to drop. | |
| 193 | + */ | |
| 191 | 194 | static struct sco_conn *sco_conn_add(struct hci_conn *hcon) |
| 192 | 195 | { |
| 193 | 196 | struct sco_conn *conn = hcon->sco_data; |
| @@ -198,6 +201,9 @@ static struct sco_conn *sco_conn_add(struct hci_conn *hcon) · 4 unmodified lines | ||
| 198 | 201 | sco_conn_lock(conn); |
| 199 | 202 | conn->hcon = hcon; |
| 200 | 203 | sco_conn_unlock(conn); |
| 204 | + } else { | |
| 205 | + /* conn already owns a reference on hcon */ | |
| 206 | + hci_conn_drop(hcon); | |
| 201 | 207 | } |
| 202 | 208 | return conn; |
| 203 | 209 | } |
| @@ -265,10 +271,8 @@ static void sco_conn_del(struct hci_conn *hcon, int err) · 61 unmodified lines | ||
| 265 | 271 | sco_conn_unlock(conn); |
| 266 | 272 | sco_conn_put(conn); |
| 267 | 273 | |
| 268 | − if (!sk) { | |
| 269 | − sco_conn_put(conn); | |
| 274 | + if (!sk) | |
| 270 | 275 | return; |
| 271 | − } | |
| 272 | 276 | |
| 273 | 277 | /* Kill socket */ |
| 274 | 278 | lock_sock(sk); |
| @@ -283,7 +287,7 @@ static void __sco_chan_add(struct sco_conn *conn, struct sock *sk, · 8 unmodified lines | ||
| 283 | 287 | { |
| 284 | 288 | BT_DBG("conn %p", conn); |
| 285 | 289 | |
| 286 | − sco_pi(sk)->conn = conn; | |
| 290 | + sco_pi(sk)->conn = sco_conn_hold(conn); | |
| 287 | 291 | conn->sk = sk; |
| 288 | 292 | |
| 289 | 293 | if (parent) |
| @@ -366,15 +370,15 @@ static int sco_connect(struct sock *sk) · 76 unmodified lines | ||
| 366 | 370 | */ |
| 367 | 371 | if (sk->sk_state != BT_OPEN && sk->sk_state != BT_BOUND) { |
| 368 | 372 | release_sock(sk); |
| 369 | − hci_conn_drop(hcon); | |
| 373 | + sco_conn_put(conn); | |
| 370 | 374 | err = -EBADFD; |
| 371 | 375 | goto unlock; |
| 372 | 376 | } |
| 373 | 377 | |
| 374 | 378 | err = sco_chan_add(conn, sk, NULL); |
| 379 | + sco_conn_put(conn); | |
| 375 | 380 | if (err) { |
| 376 | 381 | release_sock(sk); |
| 377 | − hci_conn_drop(hcon); | |
| 378 | 382 | goto unlock; |
| 379 | 383 | } |
| 380 | 384 | |
| @@ -1452,8 +1456,6 @@ static void sco_conn_ready(struct sco_conn *conn) · 1071 unmodified lines | ||
| 1452 | 1456 | bacpy(&sco_pi(sk)->src, &conn->hcon->src); |
| 1453 | 1457 | bacpy(&sco_pi(sk)->dst, &conn->hcon->dst); |
| 1454 | 1458 | |
| 1455 | − sco_conn_hold(conn); | |
| 1456 | − hci_conn_hold(conn->hcon); | |
| 1457 | 1459 | __sco_chan_add(conn, sk, parent); |
| 1458 | 1460 | |
| 1459 | 1461 | if (test_bit(BT_SK_DEFER_SETUP, &bt_sk(parent)->flags)) |
| @@ -1509,10 +1511,12 @@ static void sco_connect_cfm(struct hci_conn *hcon, __u8 status) · 49 unmodified lines | ||
| 1509 | 1511 | if (!status) { |
| 1510 | 1512 | struct sco_conn *conn; |
| 1511 | 1513 | |
| 1512 | − conn = sco_conn_add(hcon); | |
| 1514 | + conn = sco_conn_add(hci_conn_hold(hcon)); | |
| 1513 | 1515 | if (conn) { |
| 1514 | 1516 | sco_conn_ready(conn); |
| 1515 | 1517 | sco_conn_put(conn); |
| 1518 | + } else { | |
| 1519 | + hci_conn_drop(hcon); | |
| 1516 | 1520 | } |
| 1517 | 1521 | } else |
| 1518 | 1522 | sco_conn_del(hcon, bt_to_errno(status)); |
| @@ -215,7 +215,7 @@ static struct sk_buff *br_mrp_alloc_test_skb(struct br_mrp *mrp, · 214 unmodified lines | ||
| 215 | 215 | struct br_mrp_oui_hdr *oui = NULL; |
| 216 | 216 | u8 length; |
| 217 | 217 | |
| 218 | − length = sizeof(*sub_opt) + sizeof(*sub_tlv) + sizeof(oui) + | |
| 218 | + length = sizeof(*sub_opt) + sizeof(*sub_tlv) + sizeof(*oui) + | |
| 219 | 219 | MRP_OPT_PADDING; |
| 220 | 220 | br_mrp_skb_tlv(skb, BR_MRP_TLV_HEADER_OPTION, length); |
| 221 | 221 | |
| @@ -3687,6 +3687,7 @@ br_multicast_leave_group(struct net_bridge_mcast *brmctx, · 3686 unmodified lines | ||
| 3687 | 3687 | |
| 3688 | 3688 | p->flags |= MDB_PG_FLAGS_FAST_LEAVE; |
| 3689 | 3689 | br_multicast_del_pg(mp, p, pp); |
| 3690 | + break; | |
| 3690 | 3691 | } |
| 3691 | 3692 | goto out; |
| 3692 | 3693 | } |
| @@ -127,6 +127,15 @@ MODULE_PARM_DESC(max_pdu_size, "maximum isotp pdu size (default " · 126 unmodified lines | ||
| 127 | 127 | #define ISOTP_FC_TIMEOUT 1 /* 1 sec */ |
| 128 | 128 | #define ISOTP_ECHO_TIMEOUT 2 /* 2 secs */ |
| 129 | 129 | |
| 130 | +/* so->tx_result[so->tx_gen % ISOTP_TX_RESULT_SLOTS] holds the packed value | |
| 131 | + * (err << ISOTP_TX_RESULT_GEN_BITS | gen) for each tx generation slot, so it | |
| 132 | + * can be handled with a single READ_ONCE()/WRITE_ONCE() access. | |
| 133 | + */ | |
| 134 | +#define ISOTP_TX_RESULT_SLOTS 4 | |
| 135 | +#define ISOTP_TX_RESULT_GEN_BITS 24 | |
| 136 | +#define ISOTP_TX_RESULT_GEN_MASK ((1U << ISOTP_TX_RESULT_GEN_BITS) - 1) | |
| 137 | +#define ISOTP_TX_RESULT_ERR_MASK 0xFF | |
| 138 | + | |
| 130 | 139 | enum { |
| 131 | 140 | ISOTP_IDLE = 0, |
| 132 | 141 | ISOTP_WAIT_FIRST_FC, |
| @@ -166,7 +175,8 @@ struct isotp_sock { · 33 unmodified lines | ||
| 166 | 175 | u32 force_tx_stmin; |
| 167 | 176 | u32 force_rx_stmin; |
| 168 | 177 | u32 cfecho; /* consecutive frame echo tag */ |
| 169 | − u32 tx_gen; /* generation, bumped per new tx transfer */ | |
| 178 | + u32 tx_gen; /* transfer generation, increased per new tx transfer */ | |
| 179 | + u32 tx_result[ISOTP_TX_RESULT_SLOTS]; /* per-generation result slots */ | |
| 170 | 180 | struct tpcon rx, tx; |
| 171 | 181 | struct list_head notifier; |
| 172 | 182 | wait_queue_head_t wait; |
| @@ -177,6 +187,65 @@ static LIST_HEAD(isotp_notifier_list); · 4 unmodified lines | ||
| 177 | 187 | static DEFINE_SPINLOCK(isotp_notifier_lock); |
| 178 | 188 | static struct isotp_sock *isotp_busy_notifier; |
| 179 | 189 | |
| 190 | +/* increase (24 bit) tx generation value */ | |
| 191 | +static u32 isotp_inc_tx_gen(u32 gen) | |
| 192 | +{ | |
| 193 | + return (gen + 1) & ISOTP_TX_RESULT_GEN_MASK; | |
| 194 | +} | |
| 195 | + | |
| 196 | +/* store 8 bit error and 24 bit tx generation values in packed u32 element */ | |
| 197 | +static u32 isotp_pack_tx_result(u32 gen, int err) | |
| 198 | +{ | |
| 199 | + return gen | ((u32)err << ISOTP_TX_RESULT_GEN_BITS); | |
| 200 | +} | |
| 201 | + | |
| 202 | +/* get the 24 bit tx generation value from the tx result */ | |
| 203 | +static u32 isotp_get_tx_gen(u32 gen_err) | |
| 204 | +{ | |
| 205 | + return gen_err & ISOTP_TX_RESULT_GEN_MASK; | |
| 206 | +} | |
| 207 | + | |
| 208 | +/* get the 8 bit error value from the tx result */ | |
| 209 | +static u32 isotp_get_tx_err(u32 gen_err) | |
| 210 | +{ | |
| 211 | + return (gen_err >> ISOTP_TX_RESULT_GEN_BITS) & ISOTP_TX_RESULT_ERR_MASK; | |
| 212 | +} | |
| 213 | + | |
| 214 | +/* store transfer result in per-generation%4 so->tx_result[] slot */ | |
| 215 | +static void isotp_set_tx_result(struct isotp_sock *so, u32 gen, int err) | |
| 216 | +{ | |
| 217 | + WRITE_ONCE(so->tx_result[gen % ISOTP_TX_RESULT_SLOTS], | |
| 218 | + isotp_pack_tx_result(gen, err)); | |
| 219 | +} | |
| 220 | + | |
| 221 | +/* fetch the result recorded for 'gen', as a (negative) errno (0 for success) */ | |
| 222 | +static int isotp_get_tx_result(struct isotp_sock *so, u32 gen) | |
| 223 | +{ | |
| 224 | + u32 result = READ_ONCE(so->tx_result[gen % ISOTP_TX_RESULT_SLOTS]); | |
| 225 | + | |
| 226 | + if (isotp_get_tx_gen(result) != gen) { | |
| 227 | + pr_notice_once("can-isotp: tx_result[] slot reused before read\n"); | |
| 228 | + | |
| 229 | + /* report failure rather than risk a false success */ | |
| 230 | + return -ECOMM; | |
| 231 | + } | |
| 232 | + | |
| 233 | + return -(isotp_get_tx_err(result)); | |
| 234 | +} | |
| 235 | + | |
| 236 | +/* true if done, shut down or superseded ('gen' is no longer the active | |
| 237 | + * transfer). Reads tx.state first (acquire) so tx_gen/tx_result reads | |
| 238 | + * below see at least what that state write published (common sequence). | |
| 239 | + */ | |
| 240 | +static bool isotp_tx_gen_done(struct isotp_sock *so, u32 gen) | |
| 241 | +{ | |
| 242 | + /* read tx.state first for the common sequence */ | |
| 243 | + u32 state = smp_load_acquire(&so->tx.state); | |
| 244 | + | |
| 245 | + return state == ISOTP_IDLE || state == ISOTP_SHUTDOWN || | |
| 246 | + READ_ONCE(so->tx_gen) != gen; | |
| 247 | +} | |
| 248 | + | |
| 180 | 249 | static inline struct isotp_sock *isotp_sk(const struct sock *sk) |
| 181 | 250 | { |
| 182 | 251 | return (struct isotp_sock *)sk; |
| @@ -199,7 +268,7 @@ static enum hrtimer_restart isotp_rx_timer_handler(struct hrtimer *hrtimer) · 16 unmodified lines | ||
| 199 | 268 | rxtimer); |
| 200 | 269 | struct sock *sk = &so->sk; |
| 201 | 270 | |
| 202 | − if (so->rx.state == ISOTP_WAIT_DATA) { | |
| 271 | + if (READ_ONCE(so->rx.state) == ISOTP_WAIT_DATA) { | |
| 203 | 272 | /* we did not get new data frames in time */ |
| 204 | 273 | |
| 205 | 274 | /* report 'connection timed out' */ |
| @@ -208,7 +277,7 @@ static enum hrtimer_restart isotp_rx_timer_handler(struct hrtimer *hrtimer) · 2 unmodified lines | ||
| 208 | 277 | sk_error_report(sk); |
| 209 | 278 | |
| 210 | 279 | /* reset rx state */ |
| 211 | − so->rx.state = ISOTP_IDLE; | |
| 280 | + WRITE_ONCE(so->rx.state, ISOTP_IDLE); | |
| 212 | 281 | } |
| 213 | 282 | |
| 214 | 283 | return HRTIMER_NORESTART; |
| @@ -372,20 +441,19 @@ static void isotp_send_cframe(struct isotp_sock *so); · 157 unmodified lines | ||
| 372 | 441 | static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) |
| 373 | 442 | { |
| 374 | 443 | struct sock *sk = &so->sk; |
| 444 | + int tx_err = EBADMSG; /* default for unknown FC status */ | |
| 375 | 445 | |
| 376 | − if (so->tx.state != ISOTP_WAIT_FC && | |
| 377 | − so->tx.state != ISOTP_WAIT_FIRST_FC) | |
| 446 | + if (READ_ONCE(so->tx.state) != ISOTP_WAIT_FC && | |
| 447 | + READ_ONCE(so->tx.state) != ISOTP_WAIT_FIRST_FC) | |
| 378 | 448 | return 0; |
| 379 | 449 | |
| 380 | 450 | hrtimer_cancel(&so->txtimer); |
| 381 | 451 | |
| 382 | 452 | /* isotp_tx_timeout() may have given up on this job while |
| 383 | − * hrtimer_cancel() above waited for it to finish; so->rx_lock | |
| 384 | − * (held by our caller isotp_rcv()) rules out a concurrent claim, | |
| 385 | − * so a plain recheck is enough here. | |
| 453 | + * hrtimer_cancel() above waited for it to finish => recheck | |
| 386 | 454 | */ |
| 387 | − if (so->tx.state != ISOTP_WAIT_FC && | |
| 388 | − so->tx.state != ISOTP_WAIT_FIRST_FC) | |
| 455 | + if (READ_ONCE(so->tx.state) != ISOTP_WAIT_FC && | |
| 456 | + READ_ONCE(so->tx.state) != ISOTP_WAIT_FIRST_FC) | |
| 389 | 457 | return 1; |
| 390 | 458 | |
| 391 | 459 | if ((cf->len < ae + FC_CONTENT_SZ) || |
| @@ -396,13 +464,15 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) · 4 unmodified lines | ||
| 396 | 464 | if (!sock_flag(sk, SOCK_DEAD)) |
| 397 | 465 | sk_error_report(sk); |
| 398 | 466 | |
| 399 | − so->tx.state = ISOTP_IDLE; | |
| 467 | + isotp_set_tx_result(so, so->tx_gen, EBADMSG); | |
| 468 | + /* set to IDLE after publishing tx_result */ | |
| 469 | + smp_store_release(&so->tx.state, ISOTP_IDLE); | |
| 400 | 470 | wake_up_interruptible(&so->wait); |
| 401 | 471 | return 1; |
| 402 | 472 | } |
| 403 | 473 | |
| 404 | 474 | /* get static/dynamic communication params from first/every FC frame */ |
| 405 | − if (so->tx.state == ISOTP_WAIT_FIRST_FC || | |
| 475 | + if (READ_ONCE(so->tx.state) == ISOTP_WAIT_FIRST_FC || | |
| 406 | 476 | so->opt.flags & CAN_ISOTP_DYN_FC_PARMS) { |
| 407 | 477 | so->txfc.bs = cf->data[ae + 1]; |
| 408 | 478 | so->txfc.stmin = cf->data[ae + 2]; |
| @@ -426,13 +496,13 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) · 17 unmodified lines | ||
| 426 | 496 | so->tx_gap = ktime_add_ns(so->tx_gap, |
| 427 | 497 | (so->txfc.stmin - 0xF0) |
| 428 | 498 | * 100000); |
| 429 | − so->tx.state = ISOTP_WAIT_FC; | |
| 499 | + WRITE_ONCE(so->tx.state, ISOTP_WAIT_FC); | |
| 430 | 500 | } |
| 431 | 501 | |
| 432 | 502 | switch (cf->data[ae] & 0x0F) { |
| 433 | 503 | case ISOTP_FC_CTS: |
| 434 | 504 | so->tx.bs = 0; |
| 435 | − so->tx.state = ISOTP_SENDING; | |
| 505 | + WRITE_ONCE(so->tx.state, ISOTP_SENDING); | |
| 436 | 506 | /* send CF frame and enable echo timeout handling */ |
| 437 | 507 | hrtimer_start(&so->echotimer, ktime_set(ISOTP_ECHO_TIMEOUT, 0), |
| 438 | 508 | HRTIMER_MODE_REL_SOFT); |
| @@ -447,14 +517,19 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) · 8 unmodified lines | ||
| 447 | 517 | |
| 448 | 518 | case ISOTP_FC_OVFLW: |
| 449 | 519 | /* overflow on receiver side - report 'message too long' */ |
| 450 | − sk->sk_err = EMSGSIZE; | |
| 451 | − if (!sock_flag(sk, SOCK_DEAD)) | |
| 452 | − sk_error_report(sk); | |
| 520 | + tx_err = EMSGSIZE; | |
| 453 | 521 | fallthrough; |
| 454 | 522 | |
| 455 | 523 | default: |
| 456 | − /* stop this tx job */ | |
| 457 | − so->tx.state = ISOTP_IDLE; | |
| 524 | + /* reserved/unknown flow status (tx_err defaults to EBADMSG) */ | |
| 525 | + | |
| 526 | + sk->sk_err = tx_err; | |
| 527 | + if (!sock_flag(sk, SOCK_DEAD)) | |
| 528 | + sk_error_report(sk); | |
| 529 | + | |
| 530 | + isotp_set_tx_result(so, so->tx_gen, tx_err); | |
| 531 | + /* set to IDLE after publishing tx_result */ | |
| 532 | + smp_store_release(&so->tx.state, ISOTP_IDLE); | |
| 458 | 533 | wake_up_interruptible(&so->wait); |
| 459 | 534 | } |
| 460 | 535 | return 0; |
| @@ -467,7 +542,7 @@ static int isotp_rcv_sf(struct sock *sk, struct canfd_frame *cf, int pcilen, · 6 unmodified lines | ||
| 467 | 542 | struct sk_buff *nskb; |
| 468 | 543 | |
| 469 | 544 | hrtimer_cancel(&so->rxtimer); |
| 470 | − so->rx.state = ISOTP_IDLE; | |
| 545 | + WRITE_ONCE(so->rx.state, ISOTP_IDLE); | |
| 471 | 546 | |
| 472 | 547 | if (!len || len > cf->len - pcilen) |
| 473 | 548 | return 1; |
| @@ -501,7 +576,7 @@ static int isotp_rcv_ff(struct sock *sk, struct canfd_frame *cf, int ae) · 27 unmodified lines | ||
| 501 | 576 | int ff_pci_sz; |
| 502 | 577 | |
| 503 | 578 | hrtimer_cancel(&so->rxtimer); |
| 504 | − so->rx.state = ISOTP_IDLE; | |
| 579 | + WRITE_ONCE(so->rx.state, ISOTP_IDLE); | |
| 505 | 580 | |
| 506 | 581 | /* get the used sender LL_DL from the (first) CAN frame data length */ |
| 507 | 582 | so->rx.ll_dl = padlen(cf->len); |
| @@ -555,7 +630,7 @@ static int isotp_rcv_ff(struct sock *sk, struct canfd_frame *cf, int ae) · 47 unmodified lines | ||
| 555 | 630 | |
| 556 | 631 | /* initial setup for this pdu reception */ |
| 557 | 632 | so->rx.sn = 1; |
| 558 | − so->rx.state = ISOTP_WAIT_DATA; | |
| 633 | + WRITE_ONCE(so->rx.state, ISOTP_WAIT_DATA); | |
| 559 | 634 | |
| 560 | 635 | /* no creation of flow control frames */ |
| 561 | 636 | if (so->opt.flags & CAN_ISOTP_LISTEN_MODE) |
| @@ -573,7 +648,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 11 unmodified lines | ||
| 573 | 648 | struct sk_buff *nskb; |
| 574 | 649 | int i; |
| 575 | 650 | |
| 576 | − if (so->rx.state != ISOTP_WAIT_DATA) | |
| 651 | + if (READ_ONCE(so->rx.state) != ISOTP_WAIT_DATA) | |
| 577 | 652 | return 0; |
| 578 | 653 | |
| 579 | 654 | /* drop if timestamp gap is less than force_rx_stmin nano secs */ |
| @@ -588,11 +663,9 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 8 unmodified lines | ||
| 588 | 663 | hrtimer_cancel(&so->rxtimer); |
| 589 | 664 | |
| 590 | 665 | /* isotp_rx_timer_handler() may have raced us for so->rx.state |
| 591 | − * while hrtimer_cancel() above waited for it to finish, already | |
| 592 | − * reporting ETIMEDOUT and resetting the reception; don't process | |
| 593 | − * this CF into a reassembly that has already been given up on. | |
| 666 | + * while hrtimer_cancel() above waited for it to finish => recheck | |
| 594 | 667 | */ |
| 595 | − if (so->rx.state != ISOTP_WAIT_DATA) | |
| 668 | + if (READ_ONCE(so->rx.state) != ISOTP_WAIT_DATA) | |
| 596 | 669 | return 1; |
| 597 | 670 | |
| 598 | 671 | /* CFs are never longer than the FF */ |
| @@ -613,7 +686,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 14 unmodified lines | ||
| 613 | 686 | sk_error_report(sk); |
| 614 | 687 | |
| 615 | 688 | /* reset rx state */ |
| 616 | − so->rx.state = ISOTP_IDLE; | |
| 689 | + WRITE_ONCE(so->rx.state, ISOTP_IDLE); | |
| 617 | 690 | return 1; |
| 618 | 691 | } |
| 619 | 692 | so->rx.sn++; |
| @@ -627,7 +700,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 7 unmodified lines | ||
| 627 | 700 | |
| 628 | 701 | if (so->rx.idx >= so->rx.len) { |
| 629 | 702 | /* we are done */ |
| 630 | − so->rx.state = ISOTP_IDLE; | |
| 703 | + WRITE_ONCE(so->rx.state, ISOTP_IDLE); | |
| 631 | 704 | |
| 632 | 705 | if ((so->opt.flags & ISOTP_CHECK_PADDING) && |
| 633 | 706 | check_pad(so, cf, i + 1, so->opt.rxpad_content)) { |
| @@ -698,8 +771,10 @@ static void isotp_rcv(struct sk_buff *skb, void *data) · 64 unmodified lines | ||
| 698 | 771 | |
| 699 | 772 | if (so->opt.flags & CAN_ISOTP_HALF_DUPLEX) { |
| 700 | 773 | /* check rx/tx path half duplex expectations */ |
| 701 | − if ((so->tx.state != ISOTP_IDLE && n_pci_type != N_PCI_FC) || | |
| 702 | − (so->rx.state != ISOTP_IDLE && n_pci_type == N_PCI_FC)) | |
| 774 | + if ((READ_ONCE(so->tx.state) != ISOTP_IDLE && | |
| 775 | + n_pci_type != N_PCI_FC) || | |
| 776 | + (READ_ONCE(so->rx.state) != ISOTP_IDLE && | |
| 777 | + n_pci_type == N_PCI_FC)) | |
| 703 | 778 | goto out_unlock; |
| 704 | 779 | } |
| 705 | 780 | |
| @@ -794,6 +869,7 @@ static void isotp_send_cframe(struct isotp_sock *so) · 88 unmodified lines | ||
| 794 | 869 | struct canfd_frame *cf; |
| 795 | 870 | int can_send_ret; |
| 796 | 871 | int ae = (so->opt.flags & CAN_ISOTP_EXTEND_ADDR) ? 1 : 0; |
| 872 | + u32 old_cfecho; | |
| 797 | 873 | |
| 798 | 874 | dev = dev_get_by_index(sock_net(sk), so->ifindex); |
| 799 | 875 | if (!dev) |
| @@ -814,6 +890,9 @@ static void isotp_send_cframe(struct isotp_sock *so) · 14 unmodified lines | ||
| 814 | 890 | |
| 815 | 891 | csx->can_iif = dev->ifindex; |
| 816 | 892 | |
| 893 | + /* set uid in tx skb to identify CF echo frames */ | |
| 894 | + can_set_skb_uid(skb); | |
| 895 | + | |
| 817 | 896 | cf = (struct canfd_frame *)skb->data; |
| 818 | 897 | skb_put_zero(skb, so->ll.mtu); |
| 819 | 898 | |
| @@ -830,12 +909,15 @@ static void isotp_send_cframe(struct isotp_sock *so) · 10 unmodified lines | ||
| 830 | 909 | skb->dev = dev; |
| 831 | 910 | can_skb_set_owner(skb, sk); |
| 832 | 911 | |
| 833 | − /* cfecho should have been zero'ed by init/isotp_rcv_echo() */ | |
| 834 | − if (so->cfecho) | |
| 835 | − pr_notice_once("can-isotp: cfecho is %08X != 0\n", so->cfecho); | |
| 912 | + /* zero'ed by init/isotp_rcv_echo(); reached lock-free via | |
| 913 | + * isotp_txfr_timer_handler() too, so use READ_ONCE()/WRITE_ONCE() | |
| 914 | + */ | |
| 915 | + old_cfecho = READ_ONCE(so->cfecho); | |
| 916 | + if (old_cfecho) | |
| 917 | + pr_notice_once("can-isotp: cfecho is %08X != 0\n", old_cfecho); | |
| 836 | 918 | |
| 837 | 919 | /* set consecutive frame echo tag */ |
| 838 | − so->cfecho = *(u32 *)cf->data; | |
| 920 | + WRITE_ONCE(so->cfecho, skb->hash); | |
| 839 | 921 | |
| 840 | 922 | /* send frame with local echo enabled */ |
| 841 | 923 | can_send_ret = can_send(skb, 1); |
| @@ -887,7 +969,6 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) · 45 unmodified lines | ||
| 887 | 969 | { |
| 888 | 970 | struct sock *sk = (struct sock *)data; |
| 889 | 971 | struct isotp_sock *so = isotp_sk(sk); |
| 890 | − struct canfd_frame *cf = (struct canfd_frame *)skb->data; | |
| 891 | 972 | |
| 892 | 973 | /* only handle my own local echo CF/SF skb's (no FF!) */ |
| 893 | 974 | if (skb->sk != sk) |
| @@ -899,32 +980,35 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) · 5 unmodified lines | ||
| 899 | 980 | spin_lock(&so->rx_lock); |
| 900 | 981 | |
| 901 | 982 | /* so->cfecho may since belong to a new transfer; recheck under lock */ |
| 902 | − if (so->cfecho != *(u32 *)cf->data) | |
| 983 | + if (READ_ONCE(so->cfecho) != skb->hash) | |
| 903 | 984 | goto out_unlock; |
| 904 | 985 | |
| 905 | 986 | /* cancel local echo timeout */ |
| 906 | 987 | hrtimer_cancel(&so->echotimer); |
| 907 | 988 | |
| 908 | 989 | /* local echo skb with consecutive frame has been consumed */ |
| 909 | − so->cfecho = 0; | |
| 990 | + WRITE_ONCE(so->cfecho, 0); | |
| 910 | 991 | |
| 911 | 992 | /* claiming a transfer also takes so->rx_lock, so a plain recheck |
| 912 | 993 | * is enough: so->tx.state can't have flipped to ISOTP_SENDING for |
| 913 | 994 | * a new claim while we're still in here |
| 914 | 995 | */ |
| 915 | − if (so->tx.state != ISOTP_SENDING) | |
| 996 | + if (READ_ONCE(so->tx.state) != ISOTP_SENDING) | |
| 916 | 997 | goto out_unlock; |
| 917 | 998 | |
| 918 | 999 | if (so->tx.idx >= so->tx.len) { |
| 919 | 1000 | /* we are done */ |
| 920 | − so->tx.state = ISOTP_IDLE; | |
| 1001 | + | |
| 1002 | + isotp_set_tx_result(so, so->tx_gen, 0); | |
| 1003 | + /* set to IDLE after publishing tx_result */ | |
| 1004 | + smp_store_release(&so->tx.state, ISOTP_IDLE); | |
| 921 | 1005 | wake_up_interruptible(&so->wait); |
| 922 | 1006 | goto out_unlock; |
| 923 | 1007 | } |
| 924 | 1008 | |
| 925 | 1009 | if (so->txfc.bs && so->tx.bs >= so->txfc.bs) { |
| 926 | 1010 | /* stop and wait for FC with timeout */ |
| 927 | − so->tx.state = ISOTP_WAIT_FC; | |
| 1011 | + WRITE_ONCE(so->tx.state, ISOTP_WAIT_FC); | |
| 928 | 1012 | hrtimer_start(&so->txtimer, ktime_set(ISOTP_FC_TIMEOUT, 0), |
| 929 | 1013 | HRTIMER_MODE_REL_SOFT); |
| 930 | 1014 | goto out_unlock; |
| @@ -946,16 +1030,20 @@ out_unlock: · 15 unmodified lines | ||
| 946 | 1030 | spin_unlock(&so->rx_lock); |
| 947 | 1031 | } |
| 948 | 1032 | |
| 949 | −/* shared by so->txtimer's and so->echotimer's callbacks. Both timers get | |
| 950 | − * cancelled under so->rx_lock elsewhere, so this must stay lock-free to | |
| 951 | − * avoid deadlocking with that; uses so->tx_gen instead to avoid tainting | |
| 952 | − * a new transfer with an error from the one that just timed out. | |
| 1033 | +/* isotp_tx_timeout: we did not get any flow control or echo frame in time | |
| 1034 | + * | |
| 1035 | + * Shared by so->txtimer's and so->echotimer's callbacks. Both timers get | |
| 1036 | + * cancelled under so->rx_lock elsewhere, so this must stay lock-free. | |
| 1037 | + * | |
| 1038 | + * tx.state is acquired before tx_gen. Common sequence in isotp_tx_gen_done(). | |
| 1039 | + * cmpxchg() only orders itself, not the two preceding loads. | |
| 953 | 1040 | */ |
| 954 | 1041 | static enum hrtimer_restart isotp_tx_timeout(struct isotp_sock *so) |
| 955 | 1042 | { |
| 956 | 1043 | struct sock *sk = &so->sk; |
| 1044 | + /* read tx.state first for the common sequence */ | |
| 1045 | + u32 old_state = smp_load_acquire(&so->tx.state); | |
| 957 | 1046 | u32 gen = READ_ONCE(so->tx_gen); |
| 958 | − u32 old_state = READ_ONCE(so->tx.state); | |
| 959 | 1047 | |
| 960 | 1048 | /* don't handle timeouts in IDLE or SHUTDOWN state */ |
| 961 | 1049 | if (old_state == ISOTP_IDLE || old_state == ISOTP_SHUTDOWN) |
| @@ -965,14 +1053,14 @@ static enum hrtimer_restart isotp_tx_timeout(struct isotp_sock *so) · 3 unmodified lines | ||
| 965 | 1053 | if (cmpxchg(&so->tx.state, old_state, ISOTP_IDLE) != old_state) |
| 966 | 1054 | return HRTIMER_NORESTART; |
| 967 | 1055 | |
| 968 | − /* we did not get any flow control or echo frame in time */ | |
| 1056 | + /* detected timeout: report 'communication error on send' */ | |
| 969 | 1057 | |
| 970 | − if (READ_ONCE(so->tx_gen) == gen) { | |
| 971 | − /* report 'communication error on send' */ | |
| 972 | − sk->sk_err = ECOMM; | |
| 973 | − if (!sock_flag(sk, SOCK_DEAD)) | |
| 974 | − sk_error_report(sk); | |
| 975 | − } | |
| 1058 | + /* a stale read of this slot by a waiter still falls back to ECOMM */ | |
| 1059 | + isotp_set_tx_result(so, gen, ECOMM); | |
| 1060 | + | |
| 1061 | + sk->sk_err = ECOMM; | |
| 1062 | + if (!sock_flag(sk, SOCK_DEAD)) | |
| 1063 | + sk_error_report(sk); | |
| 976 | 1064 | |
| 977 | 1065 | wake_up_interruptible(&so->wait); |
| 978 | 1066 | |
| @@ -1007,7 +1095,7 @@ static enum hrtimer_restart isotp_txfr_timer_handler(struct hrtimer *hrtimer) · 28 unmodified lines | ||
| 1007 | 1095 | HRTIMER_MODE_REL_SOFT); |
| 1008 | 1096 | |
| 1009 | 1097 | /* cfecho should be consumed by isotp_rcv_echo() here */ |
| 1010 | − if (so->tx.state == ISOTP_SENDING && !so->cfecho) | |
| 1098 | + if (READ_ONCE(so->tx.state) == ISOTP_SENDING && !READ_ONCE(so->cfecho)) | |
| 1011 | 1099 | isotp_send_cframe(so); |
| 1012 | 1100 | |
| 1013 | 1101 | return HRTIMER_NORESTART; |
| @@ -1026,10 +1114,12 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 12 unmodified lines | ||
| 1026 | 1114 | s64 hrtimer_sec = ISOTP_ECHO_TIMEOUT; |
| 1027 | 1115 | struct hrtimer *tx_hrt = &so->echotimer; |
| 1028 | 1116 | u32 new_state = ISOTP_SENDING; |
| 1117 | + u32 my_gen; | |
| 1118 | + u32 old_cfecho; | |
| 1029 | 1119 | int off; |
| 1030 | 1120 | int err; |
| 1031 | 1121 | |
| 1032 | − if (!so->bound || so->tx.state == ISOTP_SHUTDOWN) | |
| 1122 | + if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) | |
| 1033 | 1123 | return -EADDRNOTAVAIL; |
| 1034 | 1124 | |
| 1035 | 1125 | /* claim the socket under so->rx_lock: this serializes the claim |
| @@ -1046,29 +1136,33 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 10 unmodified lines | ||
| 1046 | 1136 | if (msg->msg_flags & MSG_DONTWAIT) |
| 1047 | 1137 | return -EAGAIN; |
| 1048 | 1138 | |
| 1049 | − if (so->tx.state == ISOTP_SHUTDOWN) | |
| 1139 | + if (READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) | |
| 1050 | 1140 | return -EADDRNOTAVAIL; |
| 1051 | 1141 | |
| 1052 | 1142 | /* wait for complete transmission of current pdu */ |
| 1053 | 1143 | err = wait_event_interruptible(so->wait, |
| 1054 | − so->tx.state == ISOTP_IDLE); | |
| 1144 | + READ_ONCE(so->tx.state) == ISOTP_IDLE || | |
| 1145 | + READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN); | |
| 1055 | 1146 | if (err) |
| 1056 | 1147 | return err; |
| 1057 | 1148 | } |
| 1058 | 1149 | |
| 1059 | − /* new transfer: bump so->tx_gen and drain the old one's timers, | |
| 1060 | − * still under the so->rx_lock we just claimed the socket with | |
| 1061 | − */ | |
| 1062 | − WRITE_ONCE(so->tx.state, ISOTP_SENDING); | |
| 1063 | − WRITE_ONCE(so->tx_gen, READ_ONCE(so->tx_gen) + 1); | |
| 1150 | + /* txfrtimer's callback re-arms echotimer lock-free: drain it first */ | |
| 1151 | + hrtimer_cancel(&so->txfrtimer); | |
| 1064 | 1152 | hrtimer_cancel(&so->txtimer); |
| 1065 | 1153 | hrtimer_cancel(&so->echotimer); |
| 1066 | − hrtimer_cancel(&so->txfrtimer); | |
| 1067 | − so->cfecho = 0; | |
| 1154 | + | |
| 1155 | + /* new transfer: increment so->tx_gen and set tx.state after barrier */ | |
| 1156 | + my_gen = isotp_inc_tx_gen(READ_ONCE(so->tx_gen)); | |
| 1157 | + isotp_set_tx_result(so, my_gen, ECOMM); /* prevent stale slot matching */ | |
| 1158 | + WRITE_ONCE(so->tx_gen, my_gen); | |
| 1159 | + smp_wmb(); /* see smp_load_acquire() in isotp_tx_[timeout|gen_done] */ | |
| 1160 | + WRITE_ONCE(so->tx.state, ISOTP_SENDING); | |
| 1161 | + WRITE_ONCE(so->cfecho, 0); | |
| 1068 | 1162 | spin_unlock_bh(&so->rx_lock); |
| 1069 | 1163 | |
| 1070 | 1164 | /* so->bound is only checked once above - a wakeup may have |
| 1071 | − * unbound/rebound the socket meanwhile, so re-validate it | |
| 1165 | + * unbound/rebound the socket meanwhile => recheck | |
| 1072 | 1166 | */ |
| 1073 | 1167 | if (!so->bound) { |
| 1074 | 1168 | err = -EADDRNOTAVAIL; |
| @@ -1127,6 +1221,9 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 52 unmodified lines | ||
| 1127 | 1221 | |
| 1128 | 1222 | csx->can_iif = dev->ifindex; |
| 1129 | 1223 | |
| 1224 | + /* set uid in tx skb to identify CF echo frames */ | |
| 1225 | + can_set_skb_uid(skb); | |
| 1226 | + | |
| 1130 | 1227 | so->tx.len = size; |
| 1131 | 1228 | so->tx.idx = 0; |
| 1132 | 1229 | |
| @@ -1134,8 +1231,9 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 1 unmodified line | ||
| 1134 | 1231 | skb_put_zero(skb, so->ll.mtu); |
| 1135 | 1232 | |
| 1136 | 1233 | /* cfecho should have been zero'ed by init / former isotp_rcv_echo() */ |
| 1137 | − if (so->cfecho) | |
| 1138 | − pr_notice_once("can-isotp: uninit cfecho %08X\n", so->cfecho); | |
| 1234 | + old_cfecho = READ_ONCE(so->cfecho); | |
| 1235 | + if (old_cfecho) | |
| 1236 | + pr_notice_once("can-isotp: uninit cfecho %08X\n", old_cfecho); | |
| 1139 | 1237 | |
| 1140 | 1238 | /* check for single frame transmission depending on TX_DL */ |
| 1141 | 1239 | if (size <= so->tx.ll_dl - SF_PCI_SZ4 - ae - off) { |
| @@ -1163,7 +1261,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 21 unmodified lines | ||
| 1163 | 1261 | cf->data[ae] |= size; |
| 1164 | 1262 | |
| 1165 | 1263 | /* set CF echo tag for isotp_rcv_echo() (SF-mode) */ |
| 1166 | − so->cfecho = *(u32 *)cf->data; | |
| 1264 | + WRITE_ONCE(so->cfecho, skb->hash); | |
| 1167 | 1265 | } else { |
| 1168 | 1266 | /* send first frame */ |
| 1169 | 1267 | |
| @@ -1180,7 +1278,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 10 unmodified lines | ||
| 1180 | 1278 | so->txfc.bs = 0; |
| 1181 | 1279 | |
| 1182 | 1280 | /* set CF echo tag for isotp_rcv_echo() (CF-mode) */ |
| 1183 | − so->cfecho = *(u32 *)cf->data; | |
| 1281 | + WRITE_ONCE(so->cfecho, skb->hash); | |
| 1184 | 1282 | } else { |
| 1185 | 1283 | /* standard flow control check */ |
| 1186 | 1284 | new_state = ISOTP_WAIT_FIRST_FC; |
| @@ -1190,12 +1288,12 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 3 unmodified lines | ||
| 1190 | 1288 | tx_hrt = &so->txtimer; |
| 1191 | 1289 | |
| 1192 | 1290 | /* no CF echo tag for isotp_rcv_echo() (FF-mode) */ |
| 1193 | − so->cfecho = 0; | |
| 1291 | + WRITE_ONCE(so->cfecho, 0); | |
| 1194 | 1292 | } |
| 1195 | 1293 | } |
| 1196 | 1294 | |
| 1197 | 1295 | spin_lock_bh(&so->rx_lock); |
| 1198 | − if (so->tx.state == ISOTP_SHUTDOWN) { | |
| 1296 | + if (READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) { | |
| 1199 | 1297 | /* isotp_release() has since taken over and already drained |
| 1200 | 1298 | * our timers - don't send into a socket that's going away |
| 1201 | 1299 | */ |
| @@ -1206,7 +1304,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 4 unmodified lines | ||
| 1206 | 1304 | return -EADDRNOTAVAIL; |
| 1207 | 1305 | } |
| 1208 | 1306 | /* WAIT_FIRST_FC for standard FF, else stays ISOTP_SENDING */ |
| 1209 | − so->tx.state = new_state; | |
| 1307 | + WRITE_ONCE(so->tx.state, new_state); | |
| 1210 | 1308 | hrtimer_start(tx_hrt, ktime_set(hrtimer_sec, 0), |
| 1211 | 1309 | HRTIMER_MODE_REL_SOFT); |
| 1212 | 1310 | spin_unlock_bh(&so->rx_lock); |
| @@ -1223,20 +1321,49 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 10 unmodified lines | ||
| 1223 | 1321 | __func__, ERR_PTR(err)); |
| 1224 | 1322 | |
| 1225 | 1323 | spin_lock_bh(&so->rx_lock); |
| 1324 | + | |
| 1325 | + /* new transfer already claimed by a concurrent completion, | |
| 1326 | + * timeout or sendmsg() while we were stuck in can_send()? | |
| 1327 | + */ | |
| 1328 | + if (READ_ONCE(so->tx_gen) != my_gen) { | |
| 1329 | + /* don't touch timers and state of the new transfer */ | |
| 1330 | + spin_unlock_bh(&so->rx_lock); | |
| 1331 | + return err; | |
| 1332 | + } | |
| 1333 | + | |
| 1226 | 1334 | /* no transmission -> no timeout monitoring */ |
| 1227 | 1335 | hrtimer_cancel(tx_hrt); |
| 1228 | 1336 | goto err_out_drop_locked; |
| 1229 | 1337 | } |
| 1230 | 1338 | |
| 1231 | 1339 | if (wait_tx_done) { |
| 1232 | − /* wait for complete transmission of current pdu */ | |
| 1233 | − err = wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE); | |
| 1340 | + /* wake up for: | |
| 1341 | + * - concurrent sendmsg() claiming a new transfer | |
| 1342 | + * - complete transmission of current PDU | |
| 1343 | + * - shutdown state change in isotp_release() | |
| 1344 | + * isotp_tx_gen_done() uses common tx.state/tx_gen read sequence | |
| 1345 | + */ | |
| 1346 | + err = wait_event_interruptible(so->wait, | |
| 1347 | + isotp_tx_gen_done(so, my_gen)); | |
| 1234 | 1348 | if (err) |
| 1235 | 1349 | goto err_event_drop; |
| 1236 | 1350 | |
| 1237 | − err = sock_error(sk); | |
| 1238 | − if (err) | |
| 1239 | − return err; | |
| 1351 | + /* still our claim, but isotp_release() force-shut it down */ | |
| 1352 | + if (smp_load_acquire(&so->tx.state) == ISOTP_SHUTDOWN && | |
| 1353 | + READ_ONCE(so->tx_gen) == my_gen) { | |
| 1354 | + err = -EADDRNOTAVAIL; | |
| 1355 | + goto err_event_drop; | |
| 1356 | + } | |
| 1357 | + | |
| 1358 | + /* own completion, or tx_gen moved on - either way this is | |
| 1359 | + * what isotp_get_tx_result() recorded for my_gen | |
| 1360 | + */ | |
| 1361 | + err = isotp_get_tx_result(so, my_gen); | |
| 1362 | + | |
| 1363 | + /* drain to avoid stale error for a later poll()/SO_ERROR */ | |
| 1364 | + sock_error(sk); | |
| 1365 | + | |
| 1366 | + return err ? err : size; | |
| 1240 | 1367 | } |
| 1241 | 1368 | |
| 1242 | 1369 | return size; |
| @@ -1246,15 +1373,26 @@ err_out_drop: · 3 unmodified lines | ||
| 1246 | 1373 | spin_lock_bh(&so->rx_lock); |
| 1247 | 1374 | goto err_out_drop_locked; |
| 1248 | 1375 | err_event_drop: |
| 1249 | − /* interrupted waiting on our own transfer - drain its timers */ | |
| 1376 | + /* interrupted or shut down while waiting on our own transfer */ | |
| 1250 | 1377 | spin_lock_bh(&so->rx_lock); |
| 1378 | + | |
| 1379 | + /* new transfer already started by concurrent sendmsg()? */ | |
| 1380 | + if (READ_ONCE(so->tx_gen) != my_gen) { | |
| 1381 | + /* don't touch timers and states of the new transfer */ | |
| 1382 | + spin_unlock_bh(&so->rx_lock); | |
| 1383 | + return err; | |
| 1384 | + } | |
| 1385 | + | |
| 1251 | 1386 | hrtimer_cancel(&so->txfrtimer); |
| 1252 | 1387 | hrtimer_cancel(&so->txtimer); |
| 1253 | 1388 | hrtimer_cancel(&so->echotimer); |
| 1254 | 1389 | err_out_drop_locked: |
| 1255 | 1390 | /* release the claim; so->rx_lock still held from above */ |
| 1256 | − so->cfecho = 0; | |
| 1257 | − so->tx.state = ISOTP_IDLE; | |
| 1391 | + WRITE_ONCE(so->cfecho, 0); | |
| 1392 | + | |
| 1393 | + /* only claim to IDLE if isotp_release() has not taken over */ | |
| 1394 | + if (READ_ONCE(so->tx.state) != ISOTP_SHUTDOWN) | |
| 1395 | + WRITE_ONCE(so->tx.state, ISOTP_IDLE); | |
| 1258 | 1396 | spin_unlock_bh(&so->rx_lock); |
| 1259 | 1397 | wake_up_interruptible(&so->wait); |
| 1260 | 1398 | |
| @@ -1320,8 +1458,9 @@ static int isotp_release(struct socket *sock) · 59 unmodified lines | ||
| 1320 | 1458 | /* best-effort: wait for a running pdu to finish, but don't block on |
| 1321 | 1459 | * it forever - give up after the first signal |
| 1322 | 1460 | */ |
| 1323 | − while (so->tx.state != ISOTP_IDLE && | |
| 1324 | − wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE) == 0) | |
| 1461 | + while (READ_ONCE(so->tx.state) != ISOTP_IDLE && | |
| 1462 | + wait_event_interruptible(so->wait, | |
| 1463 | + READ_ONCE(so->tx.state) == ISOTP_IDLE) == 0) | |
| 1325 | 1464 | ; |
| 1326 | 1465 | |
| 1327 | 1466 | /* claim the socket under so->rx_lock like sendmsg() does, so its |
| @@ -1329,9 +1468,12 @@ static int isotp_release(struct socket *sock) · 1 unmodified line | ||
| 1329 | 1468 | * unconditionally, even when a signal cut the wait above short |
| 1330 | 1469 | */ |
| 1331 | 1470 | spin_lock_bh(&so->rx_lock); |
| 1332 | − so->tx.state = ISOTP_SHUTDOWN; | |
| 1471 | + WRITE_ONCE(so->tx.state, ISOTP_SHUTDOWN); | |
| 1333 | 1472 | spin_unlock_bh(&so->rx_lock); |
| 1334 | − so->rx.state = ISOTP_IDLE; | |
| 1473 | + WRITE_ONCE(so->rx.state, ISOTP_IDLE); | |
| 1474 | + | |
| 1475 | + /* forced SHUTDOWN may have skipped IDLE (gave up on a signal) */ | |
| 1476 | + wake_up_interruptible(&so->wait); | |
| 1335 | 1477 | |
| 1336 | 1478 | spin_lock(&isotp_notifier_lock); |
| 1337 | 1479 | while (isotp_busy_notifier == so) { |
| @@ -1447,7 +1589,8 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l · 109 unmodified lines | ||
| 1447 | 1589 | * with so->bound in the same lock_sock() section above, so there is |
| 1448 | 1590 | * no window in which a concurrent isotp_notify() could be missed. |
| 1449 | 1591 | */ |
| 1450 | − if (so->tx.state != ISOTP_IDLE || so->rx.state != ISOTP_IDLE) { | |
| 1592 | + if (READ_ONCE(so->tx.state) != ISOTP_IDLE || | |
| 1593 | + READ_ONCE(so->rx.state) != ISOTP_IDLE) { | |
| 1451 | 1594 | err = -EAGAIN; |
| 1452 | 1595 | goto out; |
| 1453 | 1596 | } |
| @@ -1481,7 +1624,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l · 27 unmodified lines | ||
| 1481 | 1624 | isotp_rcv, sk, "isotp", sk); |
| 1482 | 1625 | |
| 1483 | 1626 | /* no consecutive frame echo skb in flight */ |
| 1484 | − so->cfecho = 0; | |
| 1627 | + WRITE_ONCE(so->cfecho, 0); | |
| 1485 | 1628 | |
| 1486 | 1629 | /* register for echo skb's */ |
| 1487 | 1630 | can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), |
| @@ -1847,7 +1990,7 @@ static __poll_t isotp_poll(struct file *file, struct socket *sock, poll_table *w · 359 unmodified lines | ||
| 1847 | 1990 | poll_wait(file, &so->wait, wait); |
| 1848 | 1991 | |
| 1849 | 1992 | /* Check for false positives due to TX state */ |
| 1850 | − if ((mask & EPOLLWRNORM) && (so->tx.state != ISOTP_IDLE)) | |
| 1993 | + if ((mask & EPOLLWRNORM) && (READ_ONCE(so->tx.state) != ISOTP_IDLE)) | |
| 1851 | 1994 | mask &= ~(EPOLLOUT | EPOLLWRNORM); |
| 1852 | 1995 | |
| 1853 | 1996 | return mask; |
| @@ -1907,13 +2050,18 @@ static __init int isotp_module_init(void) · 53 unmodified lines | ||
| 1907 | 2050 | |
| 1908 | 2051 | pr_info("can: isotp protocol (max_pdu_size %d)\n", max_pdu_size); |
| 1909 | 2052 | |
| 2053 | + err = register_netdevice_notifier(&canisotp_notifier); | |
| 2054 | + if (err) | |
| 2055 | + return err; | |
| 2056 | + | |
| 1910 | 2057 | err = can_proto_register(&isotp_can_proto); |
| 1911 | − if (err < 0) | |
| 2058 | + if (err < 0) { | |
| 1912 | 2059 | pr_err("can: registration of isotp protocol failed %pe\n", ERR_PTR(err)); |
| 1913 | − else | |
| 1914 | − register_netdevice_notifier(&canisotp_notifier); | |
| 2060 | + unregister_netdevice_notifier(&canisotp_notifier); | |
| 2061 | + return err; | |
| 2062 | + } | |
| 1915 | 2063 | |
| 1916 | − return err; | |
| 2064 | + return 0; | |
| 1917 | 2065 | } |
| 1918 | 2066 | |
| 1919 | 2067 | static __exit void isotp_module_exit(void) |
| @@ -20,6 +20,7 @@ static void __j1939_ecu_release(struct kref *kref) · 19 unmodified lines | ||
| 20 | 20 | struct j1939_priv *priv = ecu->priv; |
| 21 | 21 | |
| 22 | 22 | list_del(&ecu->list); |
| 23 | + netdev_put(priv->ndev, &ecu->priv_dev_tracker); | |
| 23 | 24 | kfree(ecu); |
| 24 | 25 | j1939_priv_put(priv); |
| 25 | 26 | } |
| @@ -155,6 +156,7 @@ struct j1939_ecu *j1939_ecu_create_locked(struct j1939_priv *priv, name_t name) · 129 unmodified lines | ||
| 155 | 156 | if (!ecu) |
| 156 | 157 | return ERR_PTR(-ENOMEM); |
| 157 | 158 | kref_init(&ecu->kref); |
| 159 | + netdev_hold(priv->ndev, &ecu->priv_dev_tracker, gfp_any()); | |
| 158 | 160 | ecu->addr = J1939_IDLE_ADDR; |
| 159 | 161 | ecu->name = name; |
| 160 | 162 | |
| @@ -38,6 +38,7 @@ struct j1939_ecu { · 37 unmodified lines | ||
| 38 | 38 | struct hrtimer ac_timer; |
| 39 | 39 | struct kref kref; |
| 40 | 40 | struct j1939_priv *priv; |
| 41 | + netdevice_tracker priv_dev_tracker; | |
| 41 | 42 | |
| 42 | 43 | /* count users, to help transport protocol decide for interaction */ |
| 43 | 44 | int nusers; |
| @@ -60,6 +61,7 @@ struct j1939_priv { · 16 unmodified lines | ||
| 60 | 61 | rwlock_t lock; |
| 61 | 62 | |
| 62 | 63 | struct net_device *ndev; |
| 64 | + netdevice_tracker dev_tracker; | |
| 63 | 65 | |
| 64 | 66 | /* list of 256 ecu ptrs, that cache the claimed addresses. |
| 65 | 67 | * also protected by the above lock |
| @@ -230,6 +232,7 @@ enum j1939_session_state { · 164 unmodified lines | ||
| 230 | 232 | |
| 231 | 233 | struct j1939_session { |
| 232 | 234 | struct j1939_priv *priv; |
| 235 | + netdevice_tracker priv_dev_tracker; | |
| 233 | 236 | struct list_head active_session_list_entry; |
| 234 | 237 | struct list_head sk_session_queue_entry; |
| 235 | 238 | struct kref kref; |
| @@ -137,7 +137,7 @@ static struct j1939_priv *j1939_priv_create(struct net_device *ndev) · 136 unmodified lines | ||
| 137 | 137 | priv->ndev = ndev; |
| 138 | 138 | kref_init(&priv->kref); |
| 139 | 139 | kref_init(&priv->rx_kref); |
| 140 | − dev_hold(ndev); | |
| 140 | + netdev_hold(ndev, &priv->dev_tracker, GFP_KERNEL); | |
| 141 | 141 | |
| 142 | 142 | netdev_dbg(priv->ndev, "%s : 0x%p\n", __func__, priv); |
| 143 | 143 | |
| @@ -163,7 +163,7 @@ static void __j1939_priv_release(struct kref *kref) · 19 unmodified lines | ||
| 163 | 163 | WARN_ON_ONCE(!list_empty(&priv->ecus)); |
| 164 | 164 | WARN_ON_ONCE(!list_empty(&priv->j1939_socks)); |
| 165 | 165 | |
| 166 | − dev_put(ndev); | |
| 166 | + netdev_put(ndev, &priv->dev_tracker); | |
| 167 | 167 | kfree(priv); |
| 168 | 168 | } |
| 169 | 169 | |
| @@ -281,7 +281,7 @@ struct j1939_priv *j1939_netdev_start(struct net_device *ndev) · 111 unmodified lines | ||
| 281 | 281 | */ |
| 282 | 282 | kref_get(&priv_new->rx_kref); |
| 283 | 283 | mutex_unlock(&j1939_netdev_lock); |
| 284 | − dev_put(ndev); | |
| 284 | + netdev_put(ndev, &priv->dev_tracker); | |
| 285 | 285 | kfree(priv); |
| 286 | 286 | return priv_new; |
| 287 | 287 | } |
| @@ -298,7 +298,7 @@ struct j1939_priv *j1939_netdev_start(struct net_device *ndev) · 10 unmodified lines | ||
| 298 | 298 | j1939_priv_set(ndev, NULL); |
| 299 | 299 | mutex_unlock(&j1939_netdev_lock); |
| 300 | 300 | |
| 301 | − dev_put(ndev); | |
| 301 | + netdev_put(ndev, &priv->dev_tracker); | |
| 302 | 302 | kfree(priv); |
| 303 | 303 | |
| 304 | 304 | return ERR_PTR(ret); |
| @@ -283,6 +283,7 @@ static void j1939_session_destroy(struct j1939_session *session) · 282 unmodified lines | ||
| 283 | 283 | kfree_skb(skb); |
| 284 | 284 | } |
| 285 | 285 | __j1939_session_drop(session); |
| 286 | + netdev_put(session->priv->ndev, &session->priv_dev_tracker); | |
| 286 | 287 | j1939_priv_put(session->priv); |
| 287 | 288 | kfree(session); |
| 288 | 289 | } |
| @@ -1526,6 +1527,7 @@ static struct j1939_session *j1939_session_new(struct j1939_priv *priv, · 1237 unmodified lines | ||
| 1526 | 1527 | INIT_LIST_HEAD(&session->active_session_list_entry); |
| 1527 | 1528 | INIT_LIST_HEAD(&session->sk_session_queue_entry); |
| 1528 | 1529 | kref_init(&session->kref); |
| 1530 | + netdev_hold(priv->ndev, &session->priv_dev_tracker, gfp_any()); | |
| 1529 | 1531 | |
| 1530 | 1532 | j1939_priv_get(priv); |
| 1531 | 1533 | session->priv = priv; |
| @@ -1579,7 +1581,7 @@ j1939_session *j1939_session_fresh_new(struct j1939_priv *priv, · 47 unmodified lines | ||
| 1579 | 1581 | } |
| 1580 | 1582 | |
| 1581 | 1583 | /* alloc data area */ |
| 1582 | − skb_put(skb, size); | |
| 1584 | + skb_put_zero(skb, size); | |
| 1583 | 1585 | /* skb is recounted in j1939_session_new() */ |
| 1584 | 1586 | return session; |
| 1585 | 1587 | } |
| @@ -255,8 +255,10 @@ static int bpf_lwt_xmit_reroute(struct sk_buff *skb) · 254 unmodified lines | ||
| 255 | 255 | * if there is enough header space in skb. |
| 256 | 256 | */ |
| 257 | 257 | err = skb_cow_head(skb, LL_RESERVED_SPACE(dst->dev)); |
| 258 | − if (unlikely(err)) | |
| 258 | + if (unlikely(err)) { | |
| 259 | + dst_release(dst); | |
| 259 | 260 | goto err; |
| 261 | + } | |
| 260 | 262 | |
| 261 | 263 | skb_dst_drop(skb); |
| 262 | 264 | skb_dst_set(skb, dst); |
| @@ -3972,6 +3972,7 @@ static void _rem_dev_from_if_list(struct pktgen_thread *t, · 3971 unmodified lines | ||
| 3972 | 3972 | struct pktgen_dev *p; |
| 3973 | 3973 | |
| 3974 | 3974 | if_lock(t); |
| 3975 | + proc_remove(pkt_dev->entry); | |
| 3975 | 3976 | list_for_each_safe(q, n, &t->if_list) { |
| 3976 | 3977 | p = list_entry(q, struct pktgen_dev, list); |
| 3977 | 3978 | if (p == pkt_dev) |
| @@ -4001,9 +4002,6 @@ static int pktgen_remove_device(struct pktgen_thread *t, · 23 unmodified lines | ||
| 4001 | 4002 | * list to determine if interface already exist, avoid race |
| 4002 | 4003 | * with proc_create_data() |
| 4003 | 4004 | */ |
| 4004 | − proc_remove(pkt_dev->entry); | |
| 4005 | − | |
| 4006 | − /* And update the thread if_list */ | |
| 4007 | 4005 | _rem_dev_from_if_list(t, pkt_dev); |
| 4008 | 4006 | |
| 4009 | 4007 | #ifdef CONFIG_XFRM |
| @@ -927,6 +927,18 @@ static void skb_clone_fraglist(struct sk_buff *skb) · 926 unmodified lines | ||
| 927 | 927 | skb_get(list); |
| 928 | 928 | } |
| 929 | 929 | |
| 930 | +/** | |
| 931 | + * skb_pp_cow_data() - copy skb data into page-pool backed storage | |
| 932 | + * @pool: page pool to allocate from | |
| 933 | + * @pskb: pointer to skb pointer, replaced with the copied skb on success | |
| 934 | + * @headroom: headroom to reserve in the copied skb | |
| 935 | + * | |
| 936 | + * skb_copy_bits() handles both frags[] and frag_list input. If the copied | |
| 937 | + * skb remains non-linear, it uses frags[], which is the representation used | |
| 938 | + * by XDP multi-buffer. | |
| 939 | + * | |
| 940 | + * Return: 0 on success or a negative errno on failure. | |
| 941 | + */ | |
| 930 | 942 | int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, |
| 931 | 943 | unsigned int headroom) |
| 932 | 944 | { |
| @@ -936,12 +948,6 @@ int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, · 3 unmodified lines | ||
| 936 | 948 | int err, i, head_off; |
| 937 | 949 | void *data; |
| 938 | 950 | |
| 939 | − /* XDP does not support fraglist so we need to linearize | |
| 940 | − * the skb. | |
| 941 | − */ | |
| 942 | − if (skb_has_frag_list(skb)) | |
| 943 | − return -EOPNOTSUPP; | |
| 944 | − | |
| 945 | 951 | max_head_size = SKB_WITH_OVERHEAD(PAGE_SIZE - headroom); |
| 946 | 952 | if (skb->len > max_head_size + MAX_SKB_FRAGS * PAGE_SIZE) |
| 947 | 953 | return -ENOMEM; |
| @@ -629,9 +629,9 @@ static int fou_create(struct net *net, struct fou_cfg *cfg, · 628 unmodified lines | ||
| 629 | 629 | return 0; |
| 630 | 630 | |
| 631 | 631 | error: |
| 632 | − kfree(fou); | |
| 633 | 632 | if (sock) |
| 634 | 633 | udp_tunnel_sock_release(sock->sk); |
| 634 | + kfree_rcu(fou, rcu); | |
| 635 | 635 | |
| 636 | 636 | return err; |
| 637 | 637 | } |
| @@ -182,6 +182,7 @@ static int nat_rtp_rtcp(struct sk_buff *skb, struct nf_conn *ct, · 181 unmodified lines | ||
| 182 | 182 | struct nf_conntrack_expect *rtp_exp, |
| 183 | 183 | struct nf_conntrack_expect *rtcp_exp) |
| 184 | 184 | { |
| 185 | + struct nf_conntrack_expect *rtp_pair[2] = { rtp_exp, rtcp_exp }; | |
| 185 | 186 | struct nf_ct_h323_master *info = nfct_help_data(ct); |
| 186 | 187 | int dir = CTINFO2DIR(ctinfo); |
| 187 | 188 | int i; |
| @@ -227,22 +228,13 @@ static int nat_rtp_rtcp(struct sk_buff *skb, struct nf_conn *ct, · 39 unmodified lines | ||
| 227 | 228 | int ret; |
| 228 | 229 | |
| 229 | 230 | rtp_exp->tuple.dst.u.udp.port = htons(nated_port); |
| 230 | − ret = nf_ct_expect_related(rtp_exp, 0); | |
| 231 | + rtcp_exp->tuple.dst.u.udp.port = htons(nated_port + 1); | |
| 232 | + ret = nf_ct_expect_related_pair(rtp_pair, 0); | |
| 231 | 233 | if (ret == 0) { |
| 232 | − rtcp_exp->tuple.dst.u.udp.port = | |
| 233 | − htons(nated_port + 1); | |
| 234 | − ret = nf_ct_expect_related(rtcp_exp, 0); | |
| 235 | − if (ret == 0) | |
| 236 | − break; | |
| 237 | − else if (ret == -EBUSY) { | |
| 238 | − nf_ct_unexpect_related(rtp_exp); | |
| 239 | − continue; | |
| 240 | − } else if (ret < 0) { | |
| 241 | − nf_ct_unexpect_related(rtp_exp); | |
| 242 | − nated_port = 0; | |
| 243 | − break; | |
| 244 | − } | |
| 245 | − } else if (ret != -EBUSY) { | |
| 234 | + break; | |
| 235 | + } else if (ret == -EBUSY) { | |
| 236 | + continue; | |
| 237 | + } else if (ret < 0) { | |
| 246 | 238 | nated_port = 0; |
| 247 | 239 | break; |
| 248 | 240 | } |
| @@ -1597,14 +1597,21 @@ static int fib6_check_nh_list(struct nexthop *old, struct nexthop *new, · 1596 unmodified lines | ||
| 1597 | 1597 | struct netlink_ext_ack *extack) |
| 1598 | 1598 | { |
| 1599 | 1599 | struct fib6_info *f6i; |
| 1600 | + int err = 0; | |
| 1600 | 1601 | |
| 1601 | 1602 | if (list_empty(&old->f6i_list)) |
| 1602 | 1603 | return 0; |
| 1603 | 1604 | |
| 1605 | + spin_lock_bh(&old->lock); | |
| 1604 | 1606 | list_for_each_entry(f6i, &old->f6i_list, nh_list) { |
| 1605 | − if (check_src_addr(&f6i->fib6_src.addr, extack) < 0) | |
| 1606 | − return -EINVAL; | |
| 1607 | + err = check_src_addr(&f6i->fib6_src.addr, extack); | |
| 1608 | + if (err) | |
| 1609 | + break; | |
| 1607 | 1610 | } |
| 1611 | + spin_unlock_bh(&old->lock); | |
| 1612 | + | |
| 1613 | + if (err) | |
| 1614 | + return err; | |
| 1608 | 1615 | |
| 1609 | 1616 | return fib6_check_nexthop(new, NULL, extack); |
| 1610 | 1617 | } |
| @@ -2233,18 +2240,18 @@ static void remove_one_nexthop(struct net *net, struct nexthop *nh, · 622 unmodified lines | ||
| 2233 | 2240 | static void nh_rt_cache_flush(struct net *net, struct nexthop *nh, |
| 2234 | 2241 | struct nexthop *replaced_nh) |
| 2235 | 2242 | { |
| 2236 | − struct fib6_info *f6i; | |
| 2237 | 2243 | struct nh_group *nhg; |
| 2244 | + bool have_f6i; | |
| 2238 | 2245 | int i; |
| 2239 | 2246 | |
| 2240 | 2247 | if (!list_empty(&nh->fi_list)) |
| 2241 | 2248 | rt_cache_flush(net); |
| 2242 | 2249 | |
| 2243 | − list_for_each_entry(f6i, &nh->f6i_list, nh_list) { | |
| 2244 | − spin_lock_bh(&f6i->fib6_table->tb6_lock); | |
| 2245 | − fib6_update_sernum_upto_root(net, f6i); | |
| 2246 | − spin_unlock_bh(&f6i->fib6_table->tb6_lock); | |
| 2247 | − } | |
| 2250 | + spin_lock_bh(&nh->lock); | |
| 2251 | + have_f6i = !list_empty(&nh->f6i_list); | |
| 2252 | + spin_unlock_bh(&nh->lock); | |
| 2253 | + if (have_f6i) | |
| 2254 | + rt_genid_bump_ipv6(net); | |
| 2248 | 2255 | |
| 2249 | 2256 | /* if an IPv6 group was replaced, we have to release all old |
| 2250 | 2257 | * dsts to make sure all refcounts are released |
| @@ -2538,8 +2545,10 @@ static void __nexthop_replace_notify(struct net *net, struct nexthop *nh, · 287 unmodified lines | ||
| 2538 | 2545 | fi->nh_updated = false; |
| 2539 | 2546 | } |
| 2540 | 2547 | |
| 2548 | + spin_lock_bh(&nh->lock); | |
| 2541 | 2549 | list_for_each_entry(f6i, &nh->f6i_list, nh_list) |
| 2542 | 2550 | fib6_rt_update(net, f6i, info); |
| 2551 | + spin_unlock_bh(&nh->lock); | |
| 2543 | 2552 | } |
| 2544 | 2553 | |
| 2545 | 2554 | /* send RTM_NEWROUTE with REPLACE flag set for all FIB entries |
| @@ -892,8 +892,6 @@ void ip_rt_send_redirect(struct sk_buff *skb) · 891 unmodified lines | ||
| 892 | 892 | peer = inet_getpeer_v4(net->ipv4.peers, ip_hdr(skb)->saddr, vif); |
| 893 | 893 | if (!peer) { |
| 894 | 894 | rcu_read_unlock(); |
| 895 | − icmp_send(skb, ICMP_REDIRECT, ICMP_REDIR_HOST, | |
| 896 | − rt_nexthop(rt, ip_hdr(skb)->daddr)); | |
| 897 | 895 | return; |
| 898 | 896 | } |
| 899 | 897 | |
| @@ -32,13 +32,12 @@ struct udp_tunnel_nic_table_entry { · 31 unmodified lines | ||
| 32 | 32 | * @lock: protects all fields |
| 33 | 33 | * @need_sync: at least one port start changed |
| 34 | 34 | * @need_replay: space was freed, we need a replay of all ports |
| 35 | − * @work_pending: @work is currently scheduled | |
| 36 | 35 | * @n_tables: number of tables under @entries |
| 37 | 36 | * @missed: bitmap of tables which overflown |
| 38 | 37 | * @entries: table of tables of ports currently offloaded |
| 39 | 38 | */ |
| 40 | 39 | struct udp_tunnel_nic { |
| 41 | − struct work_struct work; | |
| 40 | + struct delayed_work work; | |
| 42 | 41 | |
| 43 | 42 | struct net_device *dev; |
| 44 | 43 | |
| @@ -46,7 +45,6 @@ struct udp_tunnel_nic { · 1 unmodified line | ||
| 46 | 45 | |
| 47 | 46 | u8 need_sync:1; |
| 48 | 47 | u8 need_replay:1; |
| 49 | − u8 work_pending:1; | |
| 50 | 48 | |
| 51 | 49 | unsigned int n_tables; |
| 52 | 50 | unsigned long missed; |
| @@ -301,11 +299,10 @@ __udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn) · 248 unmodified lines | ||
| 301 | 299 | static void |
| 302 | 300 | udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn) |
| 303 | 301 | { |
| 304 | − if (!utn->need_sync || utn->work_pending) | |
| 302 | + if (!utn->need_sync) | |
| 305 | 303 | return; |
| 306 | 304 | |
| 307 | − queue_work(udp_tunnel_nic_workqueue, &utn->work); | |
| 308 | − utn->work_pending = 1; | |
| 305 | + queue_delayed_work(udp_tunnel_nic_workqueue, &utn->work, 0); | |
| 309 | 306 | } |
| 310 | 307 | |
| 311 | 308 | static bool |
| @@ -731,12 +728,17 @@ udp_tunnel_nic_replay(struct net_device *dev, struct udp_tunnel_nic *utn) · 419 unmodified lines | ||
| 731 | 728 | static void udp_tunnel_nic_device_sync_work(struct work_struct *work) |
| 732 | 729 | { |
| 733 | 730 | struct udp_tunnel_nic *utn = |
| 734 | − container_of(work, struct udp_tunnel_nic, work); | |
| 731 | + container_of(work, struct udp_tunnel_nic, work.work); | |
| 735 | 732 | |
| 736 | − rtnl_lock(); | |
| 733 | + /* We cannot block on RTNL here, otherwise we would deadlock with | |
| 734 | + * udp_tunnel_nic_unregister() calling cancel_delayed_work_sync() | |
| 735 | + * while holding RTNL. Requeue with 1 jiffy delay if RTNL is contended. | |
| 736 | + */ | |
| 737 | + if (!rtnl_trylock()) { | |
| 738 | + queue_delayed_work(udp_tunnel_nic_workqueue, &utn->work, 1); | |
| 739 | + return; | |
| 740 | + } | |
| 737 | 741 | mutex_lock(&utn->lock); |
| 738 | − | |
| 739 | − utn->work_pending = 0; | |
| 740 | 742 | __udp_tunnel_nic_device_sync(utn->dev, utn); |
| 741 | 743 | |
| 742 | 744 | if (utn->need_replay) |
| @@ -757,7 +759,7 @@ udp_tunnel_nic_alloc(const struct udp_tunnel_nic_info *info, · 14 unmodified lines | ||
| 757 | 759 | if (!utn) |
| 758 | 760 | return NULL; |
| 759 | 761 | utn->n_tables = n_tables; |
| 760 | − INIT_WORK(&utn->work, udp_tunnel_nic_device_sync_work); | |
| 762 | + INIT_DELAYED_WORK(&utn->work, udp_tunnel_nic_device_sync_work); | |
| 761 | 763 | mutex_init(&utn->lock); |
| 762 | 764 | |
| 763 | 765 | for (i = 0; i < n_tables; i++) { |
| @@ -901,11 +903,11 @@ udp_tunnel_nic_unregister(struct net_device *dev, struct udp_tunnel_nic *utn) · 137 unmodified lines | ||
| 901 | 903 | udp_tunnel_nic_flush(dev, utn); |
| 902 | 904 | udp_tunnel_nic_unlock(dev); |
| 903 | 905 | |
| 904 | − /* Wait for the work to be done using the state, netdev core will | |
| 905 | − * retry unregister until we give up our reference on this device. | |
| 906 | + /* Make sure no work is running or queued before freeing @utn. | |
| 907 | + * The work handler uses rtnl_trylock(), so it will not deadlock | |
| 908 | + * against the RTNL we are holding here. | |
| 906 | 909 | */ |
| 907 | − if (utn->work_pending) | |
| 908 | − return; | |
| 910 | + cancel_delayed_work_sync(&utn->work); | |
| 909 | 911 | |
| 910 | 912 | udp_tunnel_nic_free(utn); |
| 911 | 913 | release_dev: |
| @@ -308,6 +308,7 @@ INDIRECT_CALLABLE_SCOPE bool fib6_rule_suppress(struct fib_rule *rule, · 307 unmodified lines | ||
| 308 | 308 | |
| 309 | 309 | suppress_route: |
| 310 | 310 | ip6_rt_put_flags(rt, flags); |
| 311 | + res->rt6 = NULL; | |
| 311 | 312 | return true; |
| 312 | 313 | } |
| 313 | 314 | |
| @@ -1494,6 +1494,7 @@ int fib6_add(struct fib6_node *root, struct fib6_info *rt, · 1493 unmodified lines | ||
| 1494 | 1494 | root, and then (in failure) stale node |
| 1495 | 1495 | in main tree. |
| 1496 | 1496 | */ |
| 1497 | + fib6_info_release(info->nl_net->ipv6.fib6_null_entry); | |
| 1497 | 1498 | node_free_immediate(info->nl_net, sfn); |
| 1498 | 1499 | err = PTR_ERR(sn); |
| 1499 | 1500 | goto failure; |
| @@ -641,7 +641,7 @@ int ip6_forward(struct sk_buff *skb) · 640 unmodified lines | ||
| 641 | 641 | /* Limit redirects both by destination (here) |
| 642 | 642 | and by source (inside ndisc_send_redirect) |
| 643 | 643 | */ |
| 644 | − if (inet_peer_xrlim_allow(peer, 1*HZ)) | |
| 644 | + if (peer && inet_peer_xrlim_allow(peer, 1*HZ)) | |
| 645 | 645 | ndisc_send_redirect(skb, target); |
| 646 | 646 | rcu_read_unlock(); |
| 647 | 647 | } else { |
| @@ -1707,6 +1707,8 @@ void ndisc_send_redirect(struct sk_buff *skb, const struct in6_addr *target) · 1706 unmodified lines | ||
| 1707 | 1707 | } |
| 1708 | 1708 | |
| 1709 | 1709 | peer = inet_getpeer_v6(net->ipv6.peers, &ipv6_hdr(skb)->saddr); |
| 1710 | + if (!peer) | |
| 1711 | + goto release; | |
| 1710 | 1712 | ret = inet_peer_xrlim_allow(peer, 1*HZ); |
| 1711 | 1713 | |
| 1712 | 1714 | if (!ret) |
| @@ -915,6 +915,7 @@ void ieee80211_stop_tx_ba_cb(struct sta_info *sta, int tid, · 914 unmodified lines | ||
| 915 | 915 | struct tid_ampdu_tx *tid_tx) |
| 916 | 916 | { |
| 917 | 917 | struct ieee80211_sub_if_data *sdata = sta->sdata; |
| 918 | + bool ndp = ndp = tid_tx->ndp; | |
| 918 | 919 | bool send_delba = false; |
| 919 | 920 | bool start_txq = false; |
| 920 | 921 | |
| @@ -934,6 +935,7 @@ void ieee80211_stop_tx_ba_cb(struct sta_info *sta, int tid, · 13 unmodified lines | ||
| 934 | 935 | send_delba = true; |
| 935 | 936 | |
| 936 | 937 | ieee80211_remove_tid_tx(sta, tid); |
| 938 | + /* tid_tx is now invalid since ieee80211_remove_tid_tx() frees it */ | |
| 937 | 939 | start_txq = true; |
| 938 | 940 | |
| 939 | 941 | unlock_sta: |
| @@ -946,7 +948,7 @@ void ieee80211_stop_tx_ba_cb(struct sta_info *sta, int tid, · 6 unmodified lines | ||
| 946 | 948 | ieee80211_send_delba(sdata, sta->sta.addr, tid, |
| 947 | 949 | WLAN_BACK_INITIATOR, |
| 948 | 950 | WLAN_REASON_QSTA_NOT_USE, |
| 949 | − tid_tx->ndp); | |
| 951 | + ndp); | |
| 950 | 952 | } |
| 951 | 953 | |
| 952 | 954 | void ieee80211_stop_tx_ba_cb_irqsafe(struct ieee80211_vif *vif, |
| @@ -101,6 +101,10 @@ ieee80211_s1g_rx_twt_setup(struct ieee80211_sub_if_data *sdata, · 100 unmodified lines | ||
| 101 | 101 | struct ieee80211_twt_setup *twt = (void *)mgmt->u.action.s1g.variable; |
| 102 | 102 | struct ieee80211_twt_params *twt_agrt = (void *)twt->params; |
| 103 | 103 | |
| 104 | + if (!(twt->control & IEEE80211_TWT_CONTROL_NEG_TYPE_BROADCAST) && | |
| 105 | + twt->length < sizeof(twt->control) + sizeof(*twt_agrt)) | |
| 106 | + return; | |
| 107 | + | |
| 104 | 108 | twt_agrt->req_type &= cpu_to_le16(~IEEE80211_TWT_REQTYPE_REQUEST); |
| 105 | 109 | |
| 106 | 110 | /* broadcast TWT not supported yet */ |
| @@ -2539,6 +2539,7 @@ static int mpls_getroute(struct sk_buff *in_skb, struct nlmsghdr *in_nlh, · 2538 unmodified lines | ||
| 2539 | 2539 | r->rtm_family = AF_MPLS; |
| 2540 | 2540 | r->rtm_dst_len = 20; |
| 2541 | 2541 | r->rtm_src_len = 0; |
| 2542 | + r->rtm_tos = 0; | |
| 2542 | 2543 | r->rtm_table = RT_TABLE_MAIN; |
| 2543 | 2544 | r->rtm_type = RTN_UNICAST; |
| 2544 | 2545 | r->rtm_scope = RT_SCOPE_UNIVERSE; |
| @@ -1005,7 +1005,7 @@ overwrite_extensions: · 1004 unmodified lines | ||
| 1005 | 1005 | #endif |
| 1006 | 1006 | if (SET_WITH_COUNTER(set)) |
| 1007 | 1007 | ip_set_init_counter(ext_counter(data, set), ext); |
| 1008 | − if (SET_WITH_COMMENT(set)) | |
| 1008 | + if (SET_WITH_COMMENT(set) && !ext->target) | |
| 1009 | 1009 | ip_set_init_comment(set, ext_comment(data, set), ext); |
| 1010 | 1010 | if (SET_WITH_SKBINFO(set)) |
| 1011 | 1011 | ip_set_init_skbinfo(ext_skbinfo(data, set), ext); |
| @@ -367,7 +367,7 @@ static inline int app_tcp_pkt_out(struct ip_vs_conn *cp, struct sk_buff *skb, · 366 unmodified lines | ||
| 367 | 367 | if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) |
| 368 | 368 | return 0; |
| 369 | 369 | |
| 370 | − th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len); | |
| 370 | + th = (struct tcphdr *)(skb->data + ipvsh->len); | |
| 371 | 371 | |
| 372 | 372 | /* |
| 373 | 373 | * Remember seq number in case this pkt gets resized |
| @@ -443,7 +443,7 @@ static inline int app_tcp_pkt_in(struct ip_vs_conn *cp, struct sk_buff *skb, · 69 unmodified lines | ||
| 443 | 443 | if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) |
| 444 | 444 | return 0; |
| 445 | 445 | |
| 446 | − th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len); | |
| 446 | + th = (struct tcphdr *)(skb->data + ipvsh->len); | |
| 447 | 447 | |
| 448 | 448 | /* |
| 449 | 449 | * Remember seq number in case this pkt gets resized |
| @@ -70,25 +70,45 @@ static struct kmem_cache *ip_vs_conn_cachep __read_mostly; · 69 unmodified lines | ||
| 70 | 70 | * bucket or hash table |
| 71 | 71 | * - hash table resize works like rehash but always rehashes into new table |
| 72 | 72 | * - bit lock on bucket serializes all operations that modify the chain |
| 73 | + * - on resize, bucket from the old table is locked before bucket from the | |
| 74 | + * new table | |
| 73 | 75 | * - cp->lock protects conn fields like cp->flags, cp->dest |
| 74 | 76 | */ |
| 75 | 77 | |
| 76 | −/* Lock conn_tab bucket for conn hash/unhash, not for rehash */ | |
| 78 | +/** | |
| 79 | + * conn_tab_lock - Lock conn_tab buckets for conn hash/unhash, not for rehash | |
| 80 | + * @t: hash table for hn0, new_tbl when new_hash=true | |
| 81 | + * @t2: hash table for hn1, new_tbl when new_hash2=true | |
| 82 | + * @cp: connection | |
| 83 | + * @hash_key: hash key for hn0 | |
| 84 | + * @hash_key2: hash key for hn1 | |
| 85 | + * @use2: using hn1 (double hashing) based on the forwarding method | |
| 86 | + * @new_hash: mode for hn0, hash node (true) or seek node (false) | |
| 87 | + * @new_hash2: mode for hn1, hash node (true) or seek node (false) | |
| 88 | + * @head_ret: returned head for hn0 | |
| 89 | + * @head2_ret: returned head for hn1 | |
| 90 | + * | |
| 91 | + * We support 3 modes: | |
| 92 | + * - seek mode for both nodes, used for unhashing | |
| 93 | + * - hash mode for both nodes, used for hashing | |
| 94 | + * - seek hn0 and hash hn1, used when forwarding method is changed | |
| 95 | + */ | |
| 77 | 96 | static __always_inline void |
| 78 | −conn_tab_lock(struct ip_vs_rht *t, struct ip_vs_conn *cp, u32 hash_key, | |
| 79 | − u32 hash_key2, bool use2, bool new_hash, | |
| 80 | − struct hlist_bl_head **head_ret, struct hlist_bl_head **head2_ret) | |
| 97 | +conn_tab_lock(struct ip_vs_rht *t, struct ip_vs_rht *t2, struct ip_vs_conn *cp, | |
| 98 | + u32 hash_key, u32 hash_key2, bool use2, bool new_hash, | |
| 99 | + bool new_hash2, struct hlist_bl_head **head_ret, | |
| 100 | + struct hlist_bl_head **head2_ret) | |
| 81 | 101 | { |
| 82 | 102 | struct hlist_bl_head *head, *head2; |
| 83 | 103 | u32 hash_key_new, hash_key_new2; |
| 84 | − struct ip_vs_rht *t2 = t; | |
| 85 | − u32 idx, idx2; | |
| 104 | + int idx = 0, idx2 = 0; | |
| 105 | + | |
| 106 | + /* Advance idx2 when new_hash is not set but hash_key2 | |
| 107 | + * is for new table | |
| 108 | + */ | |
| 109 | + if (new_hash2 && use2 && t != t2) | |
| 110 | + idx2++; | |
| 86 | 111 | |
| 87 | − idx = hash_key & t->mask; | |
| 88 | − if (use2) | |
| 89 | − idx2 = hash_key2 & t->mask; | |
| 90 | − else | |
| 91 | − idx2 = idx; | |
| 92 | 112 | if (!new_hash) { |
| 93 | 113 | /* We need to lock the bucket in the right table */ |
| 94 | 114 | |
| @@ -100,46 +120,45 @@ retry: · 5 unmodified lines | ||
| 100 | 120 | * both nodes in different tables, use idx/idx2 |
| 101 | 121 | * for proper lock ordering for heads. |
| 102 | 122 | */ |
| 103 | − idx = hash_key & t->mask; | |
| 104 | − idx |= IP_VS_RHT_TABLE_ID_MASK; | |
| 105 | − } | |
| 106 | − if (use2) { | |
| 107 | − if (!ip_vs_rht_same_table(t2, hash_key2)) { | |
| 108 | − /* It is already moved to new table */ | |
| 109 | − t2 = rcu_dereference(t2->new_tbl); | |
| 110 | − idx2 = hash_key2 & t2->mask; | |
| 111 | − idx2 |= IP_VS_RHT_TABLE_ID_MASK; | |
| 112 | − } | |
| 113 | − } else { | |
| 114 | − idx2 = idx; | |
| 123 | + idx++; | |
| 115 | 124 | } |
| 116 | 125 | } |
| 126 | + if (use2 && !new_hash2 && !ip_vs_rht_same_table(t2, hash_key2)) { | |
| 127 | + /* It is already moved to new table */ | |
| 128 | + t2 = rcu_dereference(t2->new_tbl); | |
| 129 | + idx2++; | |
| 130 | + } | |
| 117 | 131 | |
| 132 | + if (!use2) | |
| 133 | + idx2 = idx; | |
| 118 | 134 | head = t->buckets + (hash_key & t->mask); |
| 119 | 135 | head2 = use2 ? t2->buckets + (hash_key2 & t2->mask) : head; |
| 120 | 136 | |
| 121 | − local_bh_disable(); | |
| 122 | − /* Do not touch seqcount, this is a safe operation */ | |
| 123 | − | |
| 124 | − if (idx <= idx2) { | |
| 137 | + if (idx > idx2 || (head > head2 && idx == idx2)) { | |
| 138 | + hlist_bl_lock(head2); | |
| 125 | 139 | hlist_bl_lock(head); |
| 126 | − if (head != head2) | |
| 127 | − hlist_bl_lock(head2); | |
| 128 | 140 | } else { |
| 129 | − hlist_bl_lock(head2); | |
| 130 | 141 | hlist_bl_lock(head); |
| 142 | + if (head != head2) | |
| 143 | + hlist_bl_lock(head2); | |
| 131 | 144 | } |
| 132 | 145 | if (!new_hash) { |
| 146 | + bool changed; | |
| 147 | + | |
| 133 | 148 | /* Ensure hash_key is read under lock */ |
| 134 | 149 | hash_key_new = READ_ONCE(cp->hn0.hash_key); |
| 135 | − hash_key_new2 = READ_ONCE(cp->hn1.hash_key); | |
| 150 | + changed = hash_key != hash_key_new; | |
| 151 | + if (use2 && !new_hash2) { | |
| 152 | + hash_key_new2 = READ_ONCE(cp->hn1.hash_key); | |
| 153 | + changed |= hash_key2 != hash_key_new2; | |
| 154 | + } else { | |
| 155 | + hash_key_new2 = hash_key2; | |
| 156 | + } | |
| 136 | 157 | /* Hash changed ? */ |
| 137 | − if (hash_key != hash_key_new || | |
| 138 | − (hash_key2 != hash_key_new2 && use2)) { | |
| 158 | + if (changed) { | |
| 139 | 159 | if (head != head2) |
| 140 | 160 | hlist_bl_unlock(head2); |
| 141 | 161 | hlist_bl_unlock(head); |
| 142 | − local_bh_enable(); | |
| 143 | 162 | hash_key = hash_key_new; |
| 144 | 163 | hash_key2 = hash_key_new2; |
| 145 | 164 | goto retry; |
| @@ -155,7 +174,6 @@ static inline void conn_tab_unlock(struct hlist_bl_head *head, · 9 unmodified lines | ||
| 155 | 174 | if (head != head2) |
| 156 | 175 | hlist_bl_unlock(head2); |
| 157 | 176 | hlist_bl_unlock(head); |
| 158 | − local_bh_enable(); | |
| 159 | 177 | } |
| 160 | 178 | |
| 161 | 179 | static void ip_vs_conn_expire(struct timer_list *t); |
| @@ -268,8 +286,9 @@ static inline int ip_vs_conn_hash(struct ip_vs_conn *cp) · 106 unmodified lines | ||
| 268 | 286 | use2 = false; |
| 269 | 287 | } |
| 270 | 288 | |
| 271 | − conn_tab_lock(t, cp, hash_key, hash_key2, use2, true /* new_hash */, | |
| 272 | − &head, &head2); | |
| 289 | + local_bh_disable(); | |
| 290 | + conn_tab_lock(t, t, cp, hash_key, hash_key2, use2, true /* new_hash */, | |
| 291 | + true /* new_hash2 */, &head, &head2); | |
| 273 | 292 | |
| 274 | 293 | cp->flags |= IP_VS_CONN_F_HASHED; |
| 275 | 294 | WRITE_ONCE(cp->hn0.hash_key, hash_key); |
| @@ -280,6 +299,7 @@ static inline int ip_vs_conn_hash(struct ip_vs_conn *cp) · 4 unmodified lines | ||
| 280 | 299 | hlist_bl_add_head_rcu(&cp->hn1.node, head2); |
| 281 | 300 | |
| 282 | 301 | conn_tab_unlock(head, head2); |
| 302 | + local_bh_enable(); | |
| 283 | 303 | ret = 1; |
| 284 | 304 | |
| 285 | 305 | /* Schedule resizing if load increases */ |
| @@ -306,18 +326,20 @@ static inline bool ip_vs_conn_unlink(struct ip_vs_conn *cp) · 20 unmodified lines | ||
| 306 | 326 | return refcount_dec_if_one(&cp->refcnt); |
| 307 | 327 | |
| 308 | 328 | rcu_read_lock(); |
| 329 | + local_bh_disable(); | |
| 309 | 330 | |
| 310 | 331 | t = rcu_dereference(ipvs->conn_tab); |
| 311 | 332 | hash_key = READ_ONCE(cp->hn0.hash_key); |
| 312 | 333 | hash_key2 = READ_ONCE(cp->hn1.hash_key); |
| 313 | 334 | use2 = ip_vs_conn_use_hash2(cp); |
| 314 | 335 | |
| 315 | − conn_tab_lock(t, cp, hash_key, hash_key2, use2, false /* new_hash */, | |
| 316 | − &head, &head2); | |
| 336 | + conn_tab_lock(t, t, cp, hash_key, hash_key2, use2, false /* new_hash */, | |
| 337 | + false /* new_hash2 */, &head, &head2); | |
| 317 | 338 | |
| 318 | 339 | if (cp->flags & IP_VS_CONN_F_HASHED) { |
| 319 | 340 | /* Decrease refcnt and unlink conn only if we are last user */ |
| 320 | − if (refcount_dec_if_one(&cp->refcnt)) { | |
| 341 | + if (use2 == ip_vs_conn_use_hash2(cp) && | |
| 342 | + refcount_dec_if_one(&cp->refcnt)) { | |
| 321 | 343 | hlist_bl_del_rcu(&cp->hn0.node); |
| 322 | 344 | if (use2) |
| 323 | 345 | hlist_bl_del_rcu(&cp->hn1.node); |
| @@ -328,6 +350,7 @@ static inline bool ip_vs_conn_unlink(struct ip_vs_conn *cp) · 4 unmodified lines | ||
| 328 | 350 | |
| 329 | 351 | conn_tab_unlock(head, head2); |
| 330 | 352 | |
| 353 | + local_bh_enable(); | |
| 331 | 354 | rcu_read_unlock(); |
| 332 | 355 | |
| 333 | 356 | return ret; |
| @@ -632,6 +655,7 @@ void ip_vs_conn_fill_cport(struct ip_vs_conn *cp, __be16 cport) · 298 unmodified lines | ||
| 632 | 655 | int ntbl; |
| 633 | 656 | int dir; |
| 634 | 657 | |
| 658 | +restart: | |
| 635 | 659 | /* No packets from inside, so we can do it in 2 steps. */ |
| 636 | 660 | dir = use2 ? 1 : 0; |
| 637 | 661 | |
| @@ -686,6 +710,23 @@ retry: · 48 unmodified lines | ||
| 686 | 710 | /* Protect the cp->flags modification */ |
| 687 | 711 | spin_lock_bh(&cp->lock); |
| 688 | 712 | |
| 713 | + /* Recheck the forwarding method under lock */ | |
| 714 | + if (use2 != ip_vs_conn_use_hash2(cp)) { | |
| 715 | + use2 = !use2; | |
| 716 | + if (use2) { | |
| 717 | + spin_unlock_bh(&cp->lock); | |
| 718 | + /* Restart with new use2 value */ | |
| 719 | + goto restart; | |
| 720 | + } | |
| 721 | + if (dir) { | |
| 722 | + /* Not started yet, so just skip dir 1 */ | |
| 723 | + spin_unlock_bh(&cp->lock); | |
| 724 | + dir--; | |
| 725 | + goto next_dir; | |
| 726 | + } | |
| 727 | + /* Just finish dir 0 */ | |
| 728 | + } | |
| 729 | + | |
| 689 | 730 | /* Lock seqcount only for the old bucket, even if we are on new table |
| 690 | 731 | * because it affects the del operation, not the adding. |
| 691 | 732 | */ |
| @@ -752,6 +793,61 @@ retry: · 60 unmodified lines | ||
| 752 | 793 | goto next_dir; |
| 753 | 794 | } |
| 754 | 795 | |
| 796 | +/* Change forwarding method for hashed conn */ | |
| 797 | +static void ip_vs_conn_change_fwd_mask(struct ip_vs_conn *cp, u32 new_flags) | |
| 798 | +{ | |
| 799 | + struct netns_ipvs *ipvs = cp->ipvs; | |
| 800 | + struct hlist_bl_head *head, *head2; | |
| 801 | + u32 hash2, hash_key, hash_key2; | |
| 802 | + struct ip_vs_rht *t, *t2; | |
| 803 | + | |
| 804 | + /* See ip_vs_conn_use_hash2() for reference */ | |
| 805 | + if ((cp->flags & IP_VS_CONN_F_TEMPLATE) || | |
| 806 | + /* No change in double hashing ? */ | |
| 807 | + (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ) == | |
| 808 | + ((new_flags & IP_VS_CONN_F_FWD_MASK) == IP_VS_CONN_F_MASQ)) { | |
| 809 | + cp->flags = new_flags; | |
| 810 | + return; | |
| 811 | + } | |
| 812 | + t = rcu_dereference(ipvs->conn_tab); | |
| 813 | + if (ip_vs_conn_use_hash2(cp)) { | |
| 814 | + /* Stop double hashing */ | |
| 815 | + hash_key = READ_ONCE(cp->hn0.hash_key); | |
| 816 | + hash_key2 = READ_ONCE(cp->hn1.hash_key); | |
| 817 | + | |
| 818 | + conn_tab_lock(t, t, cp, hash_key, hash_key2, true /* use2 */, | |
| 819 | + false /* new_hash */, false /* new_hash2 */, | |
| 820 | + &head, &head2); | |
| 821 | + | |
| 822 | + /* Keep both hash keys in same table */ | |
| 823 | + hash_key = READ_ONCE(cp->hn0.hash_key); | |
| 824 | + WRITE_ONCE(cp->hn1.hash_key, hash_key); | |
| 825 | + hlist_bl_del_rcu(&cp->hn1.node); | |
| 826 | + cp->flags = new_flags; | |
| 827 | + | |
| 828 | + conn_tab_unlock(head, head2); | |
| 829 | + } else { | |
| 830 | + /* Start double hashing */ | |
| 831 | + | |
| 832 | + hash_key = READ_ONCE(cp->hn0.hash_key); | |
| 833 | + | |
| 834 | + t2 = rcu_dereference(t->new_tbl); | |
| 835 | + hash2 = ip_vs_conn_hashkey_conn(t2, cp, true); | |
| 836 | + hash_key2 = ip_vs_rht_build_hash_key(t2, hash2); | |
| 837 | + | |
| 838 | + /* Change the forwarding method under locked hn0 */ | |
| 839 | + conn_tab_lock(t, t2, cp, hash_key, hash_key2, true /* use2 */, | |
| 840 | + false /* new_hash */, true /* new_hash2 */, | |
| 841 | + &head, &head2); | |
| 842 | + | |
| 843 | + WRITE_ONCE(cp->hn1.hash_key, hash_key2); | |
| 844 | + cp->flags = new_flags; | |
| 845 | + hlist_bl_add_head_rcu(&cp->hn1.node, head2); | |
| 846 | + | |
| 847 | + conn_tab_unlock(head, head2); | |
| 848 | + } | |
| 849 | +} | |
| 850 | + | |
| 755 | 851 | /* Get default load factor to map conn_count/u_thresh to t->size */ |
| 756 | 852 | static int ip_vs_conn_default_load_factor(struct netns_ipvs *ipvs) |
| 757 | 853 | { |
| @@ -1014,6 +1110,9 @@ ip_vs_bind_dest(struct ip_vs_conn *cp, struct ip_vs_dest *dest) · 256 unmodified lines | ||
| 1014 | 1110 | flags = cp->flags; |
| 1015 | 1111 | /* Bind with the destination and its corresponding transmitter */ |
| 1016 | 1112 | if (flags & IP_VS_CONN_F_SYNC) { |
| 1113 | + /* Synced conns are hashed, so they can not get this flag */ | |
| 1114 | + conn_flags &= ~IP_VS_CONN_F_ONE_PACKET; | |
| 1115 | + | |
| 1017 | 1116 | /* if the connection is not template and is created |
| 1018 | 1117 | * by sync, preserve the activity flag. |
| 1019 | 1118 | */ |
| @@ -1021,9 +1120,18 @@ ip_vs_bind_dest(struct ip_vs_conn *cp, struct ip_vs_dest *dest) · 1 unmodified line | ||
| 1021 | 1120 | conn_flags &= ~IP_VS_CONN_F_INACTIVE; |
| 1022 | 1121 | /* connections inherit forwarding method from dest */ |
| 1023 | 1122 | flags &= ~(IP_VS_CONN_F_FWD_MASK | IP_VS_CONN_F_NOOUTPUT); |
| 1123 | + flags |= conn_flags; | |
| 1124 | + /* Changing forwarding method for hashed conn can | |
| 1125 | + * happen only under locks | |
| 1126 | + */ | |
| 1127 | + if (cp->flags & IP_VS_CONN_F_HASHED) | |
| 1128 | + ip_vs_conn_change_fwd_mask(cp, flags); | |
| 1129 | + else | |
| 1130 | + cp->flags = flags; | |
| 1131 | + } else { | |
| 1132 | + flags |= conn_flags; | |
| 1133 | + cp->flags = flags; | |
| 1024 | 1134 | } |
| 1025 | − flags |= conn_flags; | |
| 1026 | − cp->flags = flags; | |
| 1027 | 1135 | cp->dest = dest; |
| 1028 | 1136 | |
| 1029 | 1137 | IP_VS_DBG_BUF(7, "Bind-dest %s c:%s:%d v:%s:%d " |
| @@ -867,7 +867,7 @@ static int sysctl_nat_icmp_send(struct netns_ipvs *ipvs) { return 0; } · 866 unmodified lines | ||
| 867 | 867 | |
| 868 | 868 | #endif |
| 869 | 869 | |
| 870 | −__sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset) | |
| 870 | +static __sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset) | |
| 871 | 871 | { |
| 872 | 872 | return csum_fold(skb_checksum(skb, offset, skb->len - offset, 0)); |
| 873 | 873 | } |
| @@ -924,13 +924,13 @@ static int ip_vs_route_me_harder(struct netns_ipvs *ipvs, int af, · 50 unmodified lines | ||
| 924 | 924 | * - inout: 1=in->out, 0=out->in |
| 925 | 925 | */ |
| 926 | 926 | void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 927 | − struct ip_vs_conn *cp, int inout) | |
| 927 | + struct ip_vs_conn *cp, int inout, unsigned int toff, | |
| 928 | + bool has_ports) | |
| 928 | 929 | { |
| 929 | 930 | struct iphdr *iph = ip_hdr(skb); |
| 930 | − unsigned int icmp_offset = iph->ihl*4; | |
| 931 | − struct icmphdr *icmph = (struct icmphdr *)(skb_network_header(skb) + | |
| 932 | − icmp_offset); | |
| 931 | + struct icmphdr *icmph = (struct icmphdr *)(skb->data + toff); | |
| 933 | 932 | struct iphdr *ciph = (struct iphdr *)(icmph + 1); |
| 933 | + unsigned int coff __maybe_unused = toff + sizeof(struct icmphdr); | |
| 934 | 934 | |
| 935 | 935 | if (inout) { |
| 936 | 936 | iph->saddr = cp->vaddr.ip; |
| @@ -945,8 +945,7 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, · 8 unmodified lines | ||
| 945 | 945 | } |
| 946 | 946 | |
| 947 | 947 | /* the TCP/UDP/SCTP port */ |
| 948 | − if (IPPROTO_TCP == ciph->protocol || IPPROTO_UDP == ciph->protocol || | |
| 949 | − IPPROTO_SCTP == ciph->protocol) { | |
| 948 | + if (has_ports) { | |
| 950 | 949 | __be16 *ports = (void *)ciph + ciph->ihl*4; |
| 951 | 950 | |
| 952 | 951 | if (inout) |
| @@ -957,48 +956,40 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, · 4 unmodified lines | ||
| 957 | 956 | |
| 958 | 957 | /* And finally the ICMP checksum */ |
| 959 | 958 | icmph->checksum = 0; |
| 960 | − icmph->checksum = ip_vs_checksum_complete(skb, icmp_offset); | |
| 959 | + icmph->checksum = ip_vs_checksum_complete(skb, toff); | |
| 961 | 960 | skb->ip_summed = CHECKSUM_UNNECESSARY; |
| 962 | 961 | |
| 963 | 962 | if (inout) |
| 964 | − IP_VS_DBG_PKT(11, AF_INET, pp, skb, (void *)ciph - (void *)iph, | |
| 965 | − "Forwarding altered outgoing ICMP"); | |
| 963 | + IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, | |
| 964 | + "Forwarding altered outgoing ICMP"); | |
| 966 | 965 | else |
| 967 | − IP_VS_DBG_PKT(11, AF_INET, pp, skb, (void *)ciph - (void *)iph, | |
| 968 | − "Forwarding altered incoming ICMP"); | |
| 966 | + IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, | |
| 967 | + "Forwarding altered incoming ICMP"); | |
| 969 | 968 | } |
| 970 | 969 | |
| 971 | 970 | #ifdef CONFIG_IP_VS_IPV6 |
| 972 | 971 | void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 973 | − struct ip_vs_conn *cp, int inout) | |
| 972 | + struct ip_vs_conn *cp, int inout, unsigned int toff, | |
| 973 | + bool has_ports, struct ip_vs_iphdr *ciph) | |
| 974 | 974 | { |
| 975 | 975 | struct ipv6hdr *iph = ipv6_hdr(skb); |
| 976 | − unsigned int icmp_offset = 0; | |
| 977 | − unsigned int offs = 0; /* header offset*/ | |
| 978 | − int protocol; | |
| 979 | 976 | struct icmp6hdr *icmph; |
| 980 | − struct ipv6hdr *ciph; | |
| 981 | − unsigned short fragoffs; | |
| 977 | + struct ipv6hdr *cih; | |
| 982 | 978 | |
| 983 | − ipv6_find_hdr(skb, &icmp_offset, IPPROTO_ICMPV6, &fragoffs, NULL); | |
| 984 | − icmph = (struct icmp6hdr *)(skb_network_header(skb) + icmp_offset); | |
| 985 | − offs = icmp_offset + sizeof(struct icmp6hdr); | |
| 986 | − ciph = (struct ipv6hdr *)(skb_network_header(skb) + offs); | |
| 987 | − | |
| 988 | − protocol = ipv6_find_hdr(skb, &offs, -1, &fragoffs, NULL); | |
| 979 | + icmph = (struct icmp6hdr *)(skb->data + toff); | |
| 980 | + cih = (struct ipv6hdr *)(skb->data + ciph->off); | |
| 989 | 981 | |
| 990 | 982 | if (inout) { |
| 991 | 983 | iph->saddr = cp->vaddr.in6; |
| 992 | − ciph->daddr = cp->vaddr.in6; | |
| 984 | + cih->daddr = cp->vaddr.in6; | |
| 993 | 985 | } else { |
| 994 | 986 | iph->daddr = cp->daddr.in6; |
| 995 | − ciph->saddr = cp->daddr.in6; | |
| 987 | + cih->saddr = cp->daddr.in6; | |
| 996 | 988 | } |
| 997 | 989 | |
| 998 | 990 | /* the TCP/UDP/SCTP port */ |
| 999 | − if (!fragoffs && (IPPROTO_TCP == protocol || IPPROTO_UDP == protocol || | |
| 1000 | − IPPROTO_SCTP == protocol)) { | |
| 1001 | − __be16 *ports = (void *)(skb_network_header(skb) + offs); | |
| 991 | + if (has_ports) { | |
| 992 | + __be16 *ports = (void *)(skb->data + ciph->len); | |
| 1002 | 993 | |
| 1003 | 994 | IP_VS_DBG(11, "%s() changed port %d to %d\n", __func__, |
| 1004 | 995 | ntohs(inout ? ports[1] : ports[0]), |
| @@ -1011,19 +1002,17 @@ void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, · 6 unmodified lines | ||
| 1011 | 1002 | |
| 1012 | 1003 | /* And finally the ICMP checksum */ |
| 1013 | 1004 | icmph->icmp6_cksum = ~csum_ipv6_magic(&iph->saddr, &iph->daddr, |
| 1014 | − skb->len - icmp_offset, | |
| 1005 | + skb->len - toff, | |
| 1015 | 1006 | IPPROTO_ICMPV6, 0); |
| 1016 | − skb->csum_start = skb_network_header(skb) - skb->head + icmp_offset; | |
| 1007 | + skb->csum_start = skb_headroom(skb) + toff; | |
| 1017 | 1008 | skb->csum_offset = offsetof(struct icmp6hdr, icmp6_cksum); |
| 1018 | 1009 | skb->ip_summed = CHECKSUM_PARTIAL; |
| 1019 | 1010 | |
| 1020 | 1011 | if (inout) |
| 1021 | − IP_VS_DBG_PKT(11, AF_INET6, pp, skb, | |
| 1022 | − (void *)ciph - (void *)iph, | |
| 1012 | + IP_VS_DBG_PKT(11, AF_INET6, pp, skb, ciph->off, | |
| 1023 | 1013 | "Forwarding altered outgoing ICMPv6"); |
| 1024 | 1014 | else |
| 1025 | − IP_VS_DBG_PKT(11, AF_INET6, pp, skb, | |
| 1026 | − (void *)ciph - (void *)iph, | |
| 1015 | + IP_VS_DBG_PKT(11, AF_INET6, pp, skb, ciph->off, | |
| 1027 | 1016 | "Forwarding altered incoming ICMPv6"); |
| 1028 | 1017 | } |
| 1029 | 1018 | #endif |
| @@ -1033,36 +1022,41 @@ void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, · 3 unmodified lines | ||
| 1033 | 1022 | */ |
| 1034 | 1023 | static int handle_response_icmp(int af, struct sk_buff *skb, |
| 1035 | 1024 | union nf_inet_addr *snet, |
| 1036 | − __u8 protocol, struct ip_vs_conn *cp, | |
| 1025 | + struct ip_vs_conn *cp, | |
| 1037 | 1026 | struct ip_vs_protocol *pp, |
| 1038 | − unsigned int offset, unsigned int ihl, | |
| 1039 | − unsigned int hooknum) | |
| 1027 | + struct ip_vs_iphdr *ciph, | |
| 1028 | + unsigned int toff, unsigned int hooknum) | |
| 1040 | 1029 | { |
| 1030 | + int iproto = af == AF_INET6 ? IPPROTO_ICMPV6 : IPPROTO_ICMP; | |
| 1041 | 1031 | unsigned int verdict = NF_DROP; |
| 1032 | + unsigned int ctoff = ciph->len; | |
| 1033 | + bool has_ports = false; | |
| 1042 | 1034 | |
| 1043 | 1035 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) |
| 1044 | 1036 | goto after_nat; |
| 1045 | 1037 | |
| 1046 | 1038 | /* Ensure the checksum is correct */ |
| 1047 | − if (!skb_csum_unnecessary(skb) && ip_vs_checksum_complete(skb, ihl)) { | |
| 1039 | + if (!ip_vs_checksum_common_check(skb, toff, iproto, af)) { | |
| 1048 | 1040 | /* Failed checksum! */ |
| 1049 | 1041 | IP_VS_DBG_BUF(1, "Forward ICMP: failed checksum from %s!\n", |
| 1050 | 1042 | IP_VS_DBG_ADDR(af, snet)); |
| 1051 | 1043 | goto out; |
| 1052 | 1044 | } |
| 1053 | 1045 | |
| 1054 | − if (IPPROTO_TCP == protocol || IPPROTO_UDP == protocol || | |
| 1055 | − IPPROTO_SCTP == protocol) | |
| 1056 | − offset += 2 * sizeof(__u16); | |
| 1057 | − if (skb_ensure_writable(skb, offset)) | |
| 1046 | + if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || | |
| 1047 | + ciph->protocol == IPPROTO_SCTP) { | |
| 1048 | + ctoff += 2 * sizeof(__u16); | |
| 1049 | + has_ports = true; | |
| 1050 | + } | |
| 1051 | + if (skb_ensure_writable(skb, ctoff)) | |
| 1058 | 1052 | goto out; |
| 1059 | 1053 | |
| 1060 | 1054 | #ifdef CONFIG_IP_VS_IPV6 |
| 1061 | 1055 | if (af == AF_INET6) |
| 1062 | − ip_vs_nat_icmp_v6(skb, pp, cp, 1); | |
| 1056 | + ip_vs_nat_icmp_v6(skb, pp, cp, 1, toff, has_ports, ciph); | |
| 1063 | 1057 | else |
| 1064 | 1058 | #endif |
| 1065 | − ip_vs_nat_icmp(skb, pp, cp, 1); | |
| 1059 | + ip_vs_nat_icmp(skb, pp, cp, 1, toff, has_ports); | |
| 1066 | 1060 | |
| 1067 | 1061 | if (ip_vs_route_me_harder(cp->ipvs, af, skb, hooknum)) |
| 1068 | 1062 | goto out; |
| @@ -1090,9 +1084,9 @@ out: · 21 unmodified lines | ||
| 1090 | 1084 | * Currently handles error types - unreachable, quench, ttl exceeded. |
| 1091 | 1085 | */ |
| 1092 | 1086 | static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, |
| 1093 | − int *related, unsigned int hooknum) | |
| 1087 | + int *related, unsigned int hooknum, | |
| 1088 | + struct ip_vs_iphdr *ipvsh) | |
| 1094 | 1089 | { |
| 1095 | − struct iphdr *iph; | |
| 1096 | 1090 | struct icmphdr _icmph, *ic; |
| 1097 | 1091 | struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ |
| 1098 | 1092 | struct ip_vs_iphdr ciph; |
| @@ -1107,17 +1101,19 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 8 unmodified lines | ||
| 1107 | 1101 | if (ip_is_fragment(ip_hdr(skb))) { |
| 1108 | 1102 | if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) |
| 1109 | 1103 | return NF_STOLEN; |
| 1104 | + if (!ip_vs_fill_iph_skb(AF_INET, skb, false, ipvsh)) | |
| 1105 | + return NF_ACCEPT; | |
| 1110 | 1106 | } |
| 1111 | 1107 | |
| 1112 | − iph = ip_hdr(skb); | |
| 1113 | − offset = ihl = iph->ihl * 4; | |
| 1108 | + ihl = ipvsh->len; | |
| 1109 | + offset = ipvsh->len; | |
| 1114 | 1110 | ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); |
| 1115 | 1111 | if (ic == NULL) |
| 1116 | 1112 | return NF_DROP; |
| 1117 | 1113 | |
| 1118 | 1114 | IP_VS_DBG(12, "Outgoing ICMP (%d,%d) %pI4->%pI4\n", |
| 1119 | 1115 | ic->type, ntohs(icmp_id(ic)), |
| 1120 | − &iph->saddr, &iph->daddr); | |
| 1116 | + &ipvsh->saddr.ip, &ipvsh->daddr.ip); | |
| 1121 | 1117 | |
| 1122 | 1118 | /* |
| 1123 | 1119 | * Work through seeing if this is for us. |
| @@ -1136,7 +1132,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 12 unmodified lines | ||
| 1136 | 1132 | /* Now find the contained IP header */ |
| 1137 | 1133 | offset += sizeof(_icmph); |
| 1138 | 1134 | cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); |
| 1139 | − if (cih == NULL) | |
| 1135 | + if (!(cih && cih->version == 4 && cih->ihl >= 5)) | |
| 1140 | 1136 | return NF_ACCEPT; /* The packet looks wrong, ignore */ |
| 1141 | 1137 | |
| 1142 | 1138 | pp = ip_vs_proto_get(cih->protocol); |
| @@ -1144,8 +1140,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 1 unmodified line | ||
| 1144 | 1140 | return NF_ACCEPT; |
| 1145 | 1141 | |
| 1146 | 1142 | /* Is the embedded protocol header present? */ |
| 1147 | − if (unlikely(cih->frag_off & htons(IP_OFFSET) && | |
| 1148 | − pp->dont_defrag)) | |
| 1143 | + if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) | |
| 1149 | 1144 | return NF_ACCEPT; |
| 1150 | 1145 | |
| 1151 | 1146 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, |
| @@ -1159,9 +1154,9 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 7 unmodified lines | ||
| 1159 | 1154 | if (!cp) |
| 1160 | 1155 | return NF_ACCEPT; |
| 1161 | 1156 | |
| 1162 | − snet.ip = iph->saddr; | |
| 1163 | − return handle_response_icmp(AF_INET, skb, &snet, cih->protocol, cp, | |
| 1164 | − pp, ciph.len, ihl, hooknum); | |
| 1157 | + snet.ip = ipvsh->saddr.ip; | |
| 1158 | + return handle_response_icmp(AF_INET, skb, &snet, cp, pp, &ciph, ihl, | |
| 1159 | + hooknum); | |
| 1165 | 1160 | } |
| 1166 | 1161 | |
| 1167 | 1162 | #ifdef CONFIG_IP_VS_IPV6 |
| @@ -1174,7 +1169,6 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 6 unmodified lines | ||
| 1174 | 1169 | struct ip_vs_conn *cp; |
| 1175 | 1170 | struct ip_vs_protocol *pp; |
| 1176 | 1171 | union nf_inet_addr snet; |
| 1177 | − unsigned int offset; | |
| 1178 | 1172 | |
| 1179 | 1173 | *related = 1; |
| 1180 | 1174 | ic = frag_safe_skb_hp(skb, ipvsh->len, sizeof(_icmph), &_icmph); |
| @@ -1210,6 +1204,10 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 29 unmodified lines | ||
| 1210 | 1204 | if (!pp) |
| 1211 | 1205 | return NF_ACCEPT; |
| 1212 | 1206 | |
| 1207 | + /* Is the embedded protocol header present? */ | |
| 1208 | + if (unlikely(ciph.fragoffs && !pp->dont_defrag)) | |
| 1209 | + return NF_ACCEPT; | |
| 1210 | + | |
| 1213 | 1211 | /* The embedded headers contain source and dest in reverse order */ |
| 1214 | 1212 | cp = INDIRECT_CALL_1(pp->conn_out_get, ip_vs_conn_out_get_proto, |
| 1215 | 1213 | ipvs, AF_INET6, skb, &ciph); |
| @@ -1217,9 +1215,8 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 1 unmodified line | ||
| 1217 | 1215 | return NF_ACCEPT; |
| 1218 | 1216 | |
| 1219 | 1217 | snet.in6 = ciph.saddr.in6; |
| 1220 | − offset = ciph.len; | |
| 1221 | − return handle_response_icmp(AF_INET6, skb, &snet, ciph.protocol, cp, | |
| 1222 | − pp, offset, ipvsh->len, hooknum); | |
| 1218 | + return handle_response_icmp(AF_INET6, skb, &snet, cp, pp, &ciph, | |
| 1219 | + ipvsh->len, hooknum); | |
| 1223 | 1220 | } |
| 1224 | 1221 | #endif |
| 1225 | 1222 | |
| @@ -1545,7 +1542,8 @@ ip_vs_out_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *stat · 319 unmodified lines | ||
| 1545 | 1542 | #endif |
| 1546 | 1543 | if (unlikely(iph.protocol == IPPROTO_ICMP)) { |
| 1547 | 1544 | int related; |
| 1548 | − int verdict = ip_vs_out_icmp(ipvs, skb, &related, hooknum); | |
| 1545 | + int verdict = ip_vs_out_icmp(ipvs, skb, &related, | |
| 1546 | + hooknum, &iph); | |
| 1549 | 1547 | |
| 1550 | 1548 | if (related) |
| 1551 | 1549 | return verdict; |
| @@ -1753,9 +1751,8 @@ unk: · 201 unmodified lines | ||
| 1753 | 1751 | */ |
| 1754 | 1752 | static int |
| 1755 | 1753 | ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, |
| 1756 | − unsigned int hooknum) | |
| 1754 | + unsigned int hooknum, struct ip_vs_iphdr *iph) | |
| 1757 | 1755 | { |
| 1758 | − struct iphdr *iph; | |
| 1759 | 1756 | struct icmphdr _icmph, *ic; |
| 1760 | 1757 | struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ |
| 1761 | 1758 | struct ip_vs_iphdr ciph; |
| @@ -1765,7 +1762,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 3 unmodified lines | ||
| 1765 | 1762 | unsigned int offset, offset2, ihl, verdict; |
| 1766 | 1763 | bool tunnel, new_cp = false; |
| 1767 | 1764 | union nf_inet_addr *raddr; |
| 1768 | − char *outer_proto = "IPIP"; | |
| 1765 | + char *outer_proto __maybe_unused = "IPIP"; | |
| 1769 | 1766 | unsigned int hlen_ipip; |
| 1770 | 1767 | int ulen = 0; |
| 1771 | 1768 | |
| @@ -1775,17 +1772,19 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 3 unmodified lines | ||
| 1775 | 1772 | if (ip_is_fragment(ip_hdr(skb))) { |
| 1776 | 1773 | if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) |
| 1777 | 1774 | return NF_STOLEN; |
| 1775 | + if (!ip_vs_fill_iph_skb(AF_INET, skb, false, iph)) | |
| 1776 | + return NF_ACCEPT; | |
| 1778 | 1777 | } |
| 1779 | 1778 | |
| 1780 | − iph = ip_hdr(skb); | |
| 1781 | − offset = ihl = iph->ihl * 4; | |
| 1779 | + ihl = iph->len; | |
| 1780 | + offset = iph->len; | |
| 1782 | 1781 | ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); |
| 1783 | 1782 | if (ic == NULL) |
| 1784 | 1783 | return NF_DROP; |
| 1785 | 1784 | |
| 1786 | 1785 | IP_VS_DBG(12, "Incoming ICMP (%d,%d) %pI4->%pI4\n", |
| 1787 | 1786 | ic->type, ntohs(icmp_id(ic)), |
| 1788 | − &iph->saddr, &iph->daddr); | |
| 1787 | + &iph->saddr.ip, &iph->daddr.ip); | |
| 1789 | 1788 | |
| 1790 | 1789 | /* |
| 1791 | 1790 | * Work through seeing if this is for us. |
| @@ -1867,8 +1866,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 75 unmodified lines | ||
| 1867 | 1866 | pp = pd->pp; |
| 1868 | 1867 | |
| 1869 | 1868 | /* Is the embedded protocol header present? */ |
| 1870 | − if (unlikely(cih->frag_off & htons(IP_OFFSET) && | |
| 1871 | − pp->dont_defrag)) | |
| 1869 | + if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) | |
| 1872 | 1870 | return NF_ACCEPT; |
| 1873 | 1871 | |
| 1874 | 1872 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, |
| @@ -1876,7 +1874,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 1 unmodified line | ||
| 1876 | 1874 | |
| 1877 | 1875 | offset2 = offset; |
| 1878 | 1876 | ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph); |
| 1879 | − offset = ciph.len; | |
| 1880 | 1877 | |
| 1881 | 1878 | /* The embedded headers contain source and dest in reverse order. |
| 1882 | 1879 | * For IPIP/UDP/GRE tunnel this is error for request, not for reply. |
| @@ -1898,10 +1895,11 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 15 unmodified lines | ||
| 1898 | 1895 | verdict = NF_DROP; |
| 1899 | 1896 | |
| 1900 | 1897 | /* Ensure the checksum is correct */ |
| 1901 | − if (!skb_csum_unnecessary(skb) && ip_vs_checksum_complete(skb, ihl)) { | |
| 1898 | + if ((IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ || tunnel) && | |
| 1899 | + !ip_vs_checksum_common_check(skb, ihl, IPPROTO_ICMP, AF_INET)) { | |
| 1902 | 1900 | /* Failed checksum! */ |
| 1903 | 1901 | IP_VS_DBG(1, "Incoming ICMP: failed checksum from %pI4!\n", |
| 1904 | − &iph->saddr); | |
| 1902 | + &iph->saddr.ip); | |
| 1905 | 1903 | goto out; |
| 1906 | 1904 | } |
| 1907 | 1905 | |
| @@ -1969,10 +1967,7 @@ ignore_tunnel: · 61 unmodified lines | ||
| 1969 | 1967 | |
| 1970 | 1968 | /* do the statistics and put it back */ |
| 1971 | 1969 | ip_vs_in_stats(cp, skb); |
| 1972 | − if (IPPROTO_TCP == cih->protocol || IPPROTO_UDP == cih->protocol || | |
| 1973 | − IPPROTO_SCTP == cih->protocol) | |
| 1974 | − offset += 2 * sizeof(__u16); | |
| 1975 | − verdict = ip_vs_icmp_xmit(skb, cp, pp, offset, hooknum, &ciph); | |
| 1970 | + verdict = ip_vs_icmp_xmit(skb, cp, pp, iph->len, hooknum, &ciph); | |
| 1976 | 1971 | |
| 1977 | 1972 | out: |
| 1978 | 1973 | if (likely(!new_cp)) |
| @@ -2032,8 +2027,8 @@ static int ip_vs_in_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 53 unmodified lines | ||
| 2032 | 2027 | return NF_ACCEPT; |
| 2033 | 2028 | pp = pd->pp; |
| 2034 | 2029 | |
| 2035 | − /* Cannot handle fragmented embedded protocol */ | |
| 2036 | − if (ciph.fragoffs) | |
| 2030 | + /* Is the embedded protocol header present? */ | |
| 2031 | + if (ciph.fragoffs && !pp->dont_defrag) | |
| 2037 | 2032 | return NF_ACCEPT; |
| 2038 | 2033 | |
| 2039 | 2034 | IP_VS_DBG_PKT(11, AF_INET6, pp, skb, offset, |
| @@ -2057,23 +2052,22 @@ static int ip_vs_in_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 17 unmodified lines | ||
| 2057 | 2052 | new_cp = true; |
| 2058 | 2053 | } |
| 2059 | 2054 | |
| 2060 | − /* VS/TUN, VS/DR and LOCALNODE just let it go */ | |
| 2061 | − if ((hooknum == NF_INET_LOCAL_OUT) && | |
| 2062 | − (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ)) { | |
| 2063 | − verdict = NF_ACCEPT; | |
| 2055 | + verdict = NF_DROP; | |
| 2056 | + | |
| 2057 | + /* Ensure the checksum is correct */ | |
| 2058 | + if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ && | |
| 2059 | + !ip_vs_checksum_common_check(skb, iph->len, IPPROTO_ICMPV6, | |
| 2060 | + AF_INET6)) { | |
| 2061 | + /* Failed checksum! */ | |
| 2062 | + IP_VS_DBG(1, "Incoming ICMPv6: failed checksum from %pI6c!\n", | |
| 2063 | + &iph->saddr); | |
| 2064 | 2064 | goto out; |
| 2065 | 2065 | } |
| 2066 | 2066 | |
| 2067 | 2067 | /* do the statistics and put it back */ |
| 2068 | 2068 | ip_vs_in_stats(cp, skb); |
| 2069 | 2069 | |
| 2070 | − /* Need to mangle contained IPv6 header in ICMPv6 packet */ | |
| 2071 | − offset = ciph.len; | |
| 2072 | − if (IPPROTO_TCP == ciph.protocol || IPPROTO_UDP == ciph.protocol || | |
| 2073 | − IPPROTO_SCTP == ciph.protocol) | |
| 2074 | − offset += 2 * sizeof(__u16); /* Also mangle ports */ | |
| 2075 | − | |
| 2076 | − verdict = ip_vs_icmp_xmit_v6(skb, cp, pp, offset, hooknum, &ciph); | |
| 2070 | + verdict = ip_vs_icmp_xmit_v6(skb, cp, pp, iph->len, hooknum, &ciph); | |
| 2077 | 2071 | |
| 2078 | 2072 | out: |
| 2079 | 2073 | if (likely(!new_cp)) |
| @@ -2152,7 +2146,7 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state · 72 unmodified lines | ||
| 2152 | 2146 | if (unlikely(iph.protocol == IPPROTO_ICMP)) { |
| 2153 | 2147 | int related; |
| 2154 | 2148 | int verdict = ip_vs_in_icmp(ipvs, skb, &related, |
| 2155 | − hooknum); | |
| 2149 | + hooknum, &iph); | |
| 2156 | 2150 | |
| 2157 | 2151 | if (related) |
| 2158 | 2152 | return verdict; |
| @@ -2200,8 +2194,11 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state · 41 unmodified lines | ||
| 2200 | 2194 | } |
| 2201 | 2195 | |
| 2202 | 2196 | if (resched) { |
| 2203 | − if (!old_ct) | |
| 2197 | + if (!old_ct) { | |
| 2198 | + spin_lock_bh(&cp->lock); | |
| 2204 | 2199 | cp->flags &= ~IP_VS_CONN_F_NFCT; |
| 2200 | + spin_unlock_bh(&cp->lock); | |
| 2201 | + } | |
| 2205 | 2202 | if (!atomic_read(&cp->n_control)) |
| 2206 | 2203 | ip_vs_conn_expire_now(cp); |
| 2207 | 2204 | __ip_vs_conn_put(cp); |
| @@ -2217,8 +2214,11 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state · 9 unmodified lines | ||
| 2217 | 2214 | if (sysctl_expire_nodest_conn(ipvs)) { |
| 2218 | 2215 | bool old_ct = ip_vs_conn_uses_old_conntrack(cp, skb); |
| 2219 | 2216 | |
| 2220 | − if (!old_ct) | |
| 2217 | + if (!old_ct) { | |
| 2218 | + spin_lock_bh(&cp->lock); | |
| 2221 | 2219 | cp->flags &= ~IP_VS_CONN_F_NFCT; |
| 2220 | + spin_unlock_bh(&cp->lock); | |
| 2221 | + } | |
| 2222 | 2222 | |
| 2223 | 2223 | ip_vs_conn_expire_now(cp); |
| 2224 | 2224 | __ip_vs_conn_put(cp); |
| @@ -2288,6 +2288,7 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, · 63 unmodified lines | ||
| 2288 | 2288 | const struct nf_hook_state *state) |
| 2289 | 2289 | { |
| 2290 | 2290 | struct netns_ipvs *ipvs = net_ipvs(state->net); |
| 2291 | + struct ip_vs_iphdr iphdr; | |
| 2291 | 2292 | int r; |
| 2292 | 2293 | |
| 2293 | 2294 | /* ipvs enabled in this netns ? */ |
| @@ -2297,10 +2298,9 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, · 3 unmodified lines | ||
| 2297 | 2298 | if (state->pf == NFPROTO_IPV4) { |
| 2298 | 2299 | if (ip_hdr(skb)->protocol != IPPROTO_ICMP) |
| 2299 | 2300 | return NF_ACCEPT; |
| 2301 | + ip_vs_fill_iph_skb(AF_INET, skb, false, &iphdr); | |
| 2300 | 2302 | #ifdef CONFIG_IP_VS_IPV6 |
| 2301 | 2303 | } else { |
| 2302 | − struct ip_vs_iphdr iphdr; | |
| 2303 | − | |
| 2304 | 2304 | ip_vs_fill_iph_skb(AF_INET6, skb, false, &iphdr); |
| 2305 | 2305 | |
| 2306 | 2306 | if (iphdr.protocol != IPPROTO_ICMPV6) |
| @@ -2310,7 +2310,7 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, · 3 unmodified lines | ||
| 2310 | 2310 | #endif |
| 2311 | 2311 | } |
| 2312 | 2312 | |
| 2313 | − return ip_vs_in_icmp(ipvs, skb, &r, state->hook); | |
| 2313 | + return ip_vs_in_icmp(ipvs, skb, &r, state->hook, &iphdr); | |
| 2314 | 2314 | } |
| 2315 | 2315 | |
| 2316 | 2316 | static const struct nf_hook_ops ip_vs_ops4[] = { |
| @@ -11,7 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | |
| 12 | 12 | static int |
| 13 | 13 | sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 14 | − unsigned int sctphoff); | |
| 14 | + struct ip_vs_iphdr *iph); | |
| 15 | 15 | |
| 16 | 16 | static int |
| 17 | 17 | sctp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, |
| @@ -109,7 +109,7 @@ sctp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 91 unmodified lines | ||
| 109 | 109 | int ret; |
| 110 | 110 | |
| 111 | 111 | /* Some checks before mangling */ |
| 112 | − if (!sctp_csum_check(cp->af, skb, pp, sctphoff)) | |
| 112 | + if (!sctp_csum_check(cp->af, skb, pp, iph)) | |
| 113 | 113 | return 0; |
| 114 | 114 | |
| 115 | 115 | /* Call application helper if needed */ |
| @@ -121,7 +121,7 @@ sctp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 5 unmodified lines | ||
| 121 | 121 | payload_csum = true; |
| 122 | 122 | } |
| 123 | 123 | |
| 124 | − sctph = (void *) skb_network_header(skb) + sctphoff; | |
| 124 | + sctph = (void *)skb->data + sctphoff; | |
| 125 | 125 | |
| 126 | 126 | /* Only update csum if we really have to */ |
| 127 | 127 | if (sctph->source != cp->vport || payload_csum || |
| @@ -157,7 +157,7 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 29 unmodified lines | ||
| 157 | 157 | int ret; |
| 158 | 158 | |
| 159 | 159 | /* Some checks before mangling */ |
| 160 | − if (!sctp_csum_check(cp->af, skb, pp, sctphoff)) | |
| 160 | + if (!sctp_csum_check(cp->af, skb, pp, iph)) | |
| 161 | 161 | return 0; |
| 162 | 162 | |
| 163 | 163 | /* Call application helper if needed */ |
| @@ -169,7 +169,7 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 5 unmodified lines | ||
| 169 | 169 | payload_csum = true; |
| 170 | 170 | } |
| 171 | 171 | |
| 172 | − sctph = (void *) skb_network_header(skb) + sctphoff; | |
| 172 | + sctph = (void *)skb->data + sctphoff; | |
| 173 | 173 | |
| 174 | 174 | /* Only update csum if we really have to */ |
| 175 | 175 | if (sctph->dest != cp->dport || payload_csum || |
| @@ -187,19 +187,22 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 11 unmodified lines | ||
| 187 | 187 | |
| 188 | 188 | static int |
| 189 | 189 | sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 190 | − unsigned int sctphoff) | |
| 190 | + struct ip_vs_iphdr *iph) | |
| 191 | 191 | { |
| 192 | + unsigned int sctphoff = iph->len; | |
| 192 | 193 | struct sctphdr *sh; |
| 193 | 194 | __le32 cmp, val; |
| 194 | 195 | |
| 196 | + if (!ip_vs_checksum_needed(skb, af)) | |
| 197 | + return 1; | |
| 195 | 198 | sh = (struct sctphdr *)(skb->data + sctphoff); |
| 196 | 199 | cmp = sh->checksum; |
| 197 | 200 | val = sctp_compute_cksum(skb, sctphoff); |
| 198 | 201 | |
| 199 | 202 | if (val != cmp) { |
| 200 | 203 | /* CRC failure, dump it. */ |
| 201 | − IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | |
| 202 | − "Failed checksum for"); | |
| 204 | + IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, | |
| 205 | + "Failed checksum for"); | |
| 203 | 206 | return 0; |
| 204 | 207 | } |
| 205 | 208 | return 1; |
| @@ -29,7 +29,7 @@ · 28 unmodified lines | ||
| 29 | 29 | |
| 30 | 30 | static int |
| 31 | 31 | tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 32 | − unsigned int tcphoff); | |
| 32 | + struct ip_vs_iphdr *iph); | |
| 33 | 33 | |
| 34 | 34 | static int |
| 35 | 35 | tcp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, |
| @@ -166,7 +166,7 @@ tcp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 130 unmodified lines | ||
| 166 | 166 | int ret; |
| 167 | 167 | |
| 168 | 168 | /* Some checks before mangling */ |
| 169 | − if (!tcp_csum_check(cp->af, skb, pp, tcphoff)) | |
| 169 | + if (!tcp_csum_check(cp->af, skb, pp, iph)) | |
| 170 | 170 | return 0; |
| 171 | 171 | |
| 172 | 172 | /* Call application helper if needed */ |
| @@ -179,7 +179,7 @@ tcp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 6 unmodified lines | ||
| 179 | 179 | payload_csum = true; |
| 180 | 180 | } |
| 181 | 181 | |
| 182 | − tcph = (void *)skb_network_header(skb) + tcphoff; | |
| 182 | + tcph = (void *)skb->data + tcphoff; | |
| 183 | 183 | tcph->source = cp->vport; |
| 184 | 184 | |
| 185 | 185 | /* Adjust TCP checksums */ |
| @@ -244,7 +244,7 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 58 unmodified lines | ||
| 244 | 244 | int ret; |
| 245 | 245 | |
| 246 | 246 | /* Some checks before mangling */ |
| 247 | − if (!tcp_csum_check(cp->af, skb, pp, tcphoff)) | |
| 247 | + if (!tcp_csum_check(cp->af, skb, pp, iph)) | |
| 248 | 248 | return 0; |
| 249 | 249 | |
| 250 | 250 | /* |
| @@ -260,7 +260,7 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 9 unmodified lines | ||
| 260 | 260 | payload_csum = true; |
| 261 | 261 | } |
| 262 | 262 | |
| 263 | − tcph = (void *)skb_network_header(skb) + tcphoff; | |
| 263 | + tcph = (void *)skb->data + tcphoff; | |
| 264 | 264 | tcph->dest = cp->dport; |
| 265 | 265 | |
| 266 | 266 | /* |
| @@ -302,41 +302,13 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 35 unmodified lines | ||
| 302 | 302 | |
| 303 | 303 | static int |
| 304 | 304 | tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 305 | − unsigned int tcphoff) | |
| 305 | + struct ip_vs_iphdr *iph) | |
| 306 | 306 | { |
| 307 | − switch (skb->ip_summed) { | |
| 308 | − case CHECKSUM_NONE: | |
| 309 | − skb->csum = skb_checksum(skb, tcphoff, skb->len - tcphoff, 0); | |
| 310 | − fallthrough; | |
| 311 | − case CHECKSUM_COMPLETE: | |
| 312 | −#ifdef CONFIG_IP_VS_IPV6 | |
| 313 | − if (af == AF_INET6) { | |
| 314 | − if (csum_ipv6_magic(&ipv6_hdr(skb)->saddr, | |
| 315 | − &ipv6_hdr(skb)->daddr, | |
| 316 | − skb->len - tcphoff, | |
| 317 | − IPPROTO_TCP, | |
| 318 | − skb->csum)) { | |
| 319 | − IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | |
| 320 | − "Failed checksum for"); | |
| 321 | − return 0; | |
| 322 | − } | |
| 323 | − } else | |
| 324 | −#endif | |
| 325 | − if (csum_tcpudp_magic(ip_hdr(skb)->saddr, | |
| 326 | − ip_hdr(skb)->daddr, | |
| 327 | − skb->len - tcphoff, | |
| 328 | − ip_hdr(skb)->protocol, | |
| 329 | − skb->csum)) { | |
| 330 | − IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | |
| 331 | − "Failed checksum for"); | |
| 332 | − return 0; | |
| 333 | − } | |
| 334 | − break; | |
| 335 | − default: | |
| 336 | − /* No need to checksum. */ | |
| 337 | − break; | |
| 307 | + if (!ip_vs_checksum_common_check(skb, iph->len, IPPROTO_TCP, af)) { | |
| 308 | + IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, | |
| 309 | + "Failed checksum for"); | |
| 310 | + return 0; | |
| 338 | 311 | } |
| 339 | − | |
| 340 | 312 | return 1; |
| 341 | 313 | } |
| 342 | 314 | |
| @@ -25,7 +25,7 @@ · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | static int |
| 27 | 27 | udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 28 | − unsigned int udphoff); | |
| 28 | + struct ip_vs_iphdr *iph); | |
| 29 | 29 | |
| 30 | 30 | static int |
| 31 | 31 | udp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, |
| @@ -155,7 +155,7 @@ udp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 123 unmodified lines | ||
| 155 | 155 | int ret; |
| 156 | 156 | |
| 157 | 157 | /* Some checks before mangling */ |
| 158 | − if (!udp_csum_check(cp->af, skb, pp, udphoff)) | |
| 158 | + if (!udp_csum_check(cp->af, skb, pp, iph)) | |
| 159 | 159 | return 0; |
| 160 | 160 | |
| 161 | 161 | /* |
| @@ -170,7 +170,7 @@ udp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 8 unmodified lines | ||
| 170 | 170 | payload_csum = true; |
| 171 | 171 | } |
| 172 | 172 | |
| 173 | − udph = (void *)skb_network_header(skb) + udphoff; | |
| 173 | + udph = (void *)skb->data + udphoff; | |
| 174 | 174 | udph->source = cp->vport; |
| 175 | 175 | |
| 176 | 176 | /* |
| @@ -238,7 +238,7 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 61 unmodified lines | ||
| 238 | 238 | int ret; |
| 239 | 239 | |
| 240 | 240 | /* Some checks before mangling */ |
| 241 | − if (!udp_csum_check(cp->af, skb, pp, udphoff)) | |
| 241 | + if (!udp_csum_check(cp->af, skb, pp, iph)) | |
| 242 | 242 | return 0; |
| 243 | 243 | |
| 244 | 244 | /* |
| @@ -254,7 +254,7 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 9 unmodified lines | ||
| 254 | 254 | payload_csum = true; |
| 255 | 255 | } |
| 256 | 256 | |
| 257 | − udph = (void *)skb_network_header(skb) + udphoff; | |
| 257 | + udph = (void *)skb->data + udphoff; | |
| 258 | 258 | udph->dest = cp->dport; |
| 259 | 259 | |
| 260 | 260 | /* |
| @@ -298,48 +298,20 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 37 unmodified lines | ||
| 298 | 298 | |
| 299 | 299 | static int |
| 300 | 300 | udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 301 | − unsigned int udphoff) | |
| 301 | + struct ip_vs_iphdr *iph) | |
| 302 | 302 | { |
| 303 | 303 | struct udphdr _udph, *uh; |
| 304 | 304 | |
| 305 | − uh = skb_header_pointer(skb, udphoff, sizeof(_udph), &_udph); | |
| 305 | + uh = skb_header_pointer(skb, iph->len, sizeof(_udph), &_udph); | |
| 306 | 306 | if (uh == NULL) |
| 307 | 307 | return 0; |
| 308 | 308 | |
| 309 | − if (uh->check != 0) { | |
| 310 | − switch (skb->ip_summed) { | |
| 311 | − case CHECKSUM_NONE: | |
| 312 | − skb->csum = skb_checksum(skb, udphoff, | |
| 313 | − skb->len - udphoff, 0); | |
| 314 | − fallthrough; | |
| 315 | − case CHECKSUM_COMPLETE: | |
| 316 | −#ifdef CONFIG_IP_VS_IPV6 | |
| 317 | − if (af == AF_INET6) { | |
| 318 | − if (csum_ipv6_magic(&ipv6_hdr(skb)->saddr, | |
| 319 | − &ipv6_hdr(skb)->daddr, | |
| 320 | − skb->len - udphoff, | |
| 321 | − IPPROTO_UDP, | |
| 322 | − skb->csum)) { | |
| 323 | − IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | |
| 324 | − "Failed checksum for"); | |
| 325 | − return 0; | |
| 326 | − } | |
| 327 | − } else | |
| 328 | −#endif | |
| 329 | − if (csum_tcpudp_magic(ip_hdr(skb)->saddr, | |
| 330 | − ip_hdr(skb)->daddr, | |
| 331 | − skb->len - udphoff, | |
| 332 | − ip_hdr(skb)->protocol, | |
| 333 | − skb->csum)) { | |
| 334 | − IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | |
| 335 | − "Failed checksum for"); | |
| 336 | − return 0; | |
| 337 | − } | |
| 338 | − break; | |
| 339 | − default: | |
| 340 | − /* No need to checksum. */ | |
| 341 | − break; | |
| 342 | − } | |
| 309 | + if (!uh->check) | |
| 310 | + return 1; | |
| 311 | + if (!ip_vs_checksum_common_check(skb, iph->len, IPPROTO_UDP, af)) { | |
| 312 | + IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, | |
| 313 | + "Failed checksum for"); | |
| 314 | + return 0; | |
| 343 | 315 | } |
| 344 | 316 | return 1; |
| 345 | 317 | } |
| @@ -1502,20 +1502,22 @@ tx_error: · 1501 unmodified lines | ||
| 1502 | 1502 | */ |
| 1503 | 1503 | int |
| 1504 | 1504 | ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1505 | − struct ip_vs_protocol *pp, int offset, unsigned int hooknum, | |
| 1506 | − struct ip_vs_iphdr *iph) | |
| 1505 | + struct ip_vs_protocol *pp, unsigned int toff, | |
| 1506 | + unsigned int hooknum, struct ip_vs_iphdr *ciph) | |
| 1507 | 1507 | { |
| 1508 | 1508 | struct rtable *rt; /* Route to the other host */ |
| 1509 | 1509 | int rc; |
| 1510 | 1510 | int local; |
| 1511 | 1511 | int rt_mode, was_input; |
| 1512 | + bool has_ports = false; | |
| 1513 | + unsigned int wlen; | |
| 1512 | 1514 | |
| 1513 | 1515 | /* The ICMP packet for VS/TUN, VS/DR and LOCALNODE will be |
| 1514 | 1516 | forwarded directly here, because there is no need to |
| 1515 | 1517 | translate address/port back */ |
| 1516 | 1518 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { |
| 1517 | 1519 | if (cp->packet_xmit) |
| 1518 | − rc = cp->packet_xmit(skb, cp, pp, iph); | |
| 1520 | + rc = cp->packet_xmit(skb, cp, pp, ciph); | |
| 1519 | 1521 | else |
| 1520 | 1522 | rc = NF_ACCEPT; |
| 1521 | 1523 | /* do not touch skb anymore */ |
| @@ -1533,7 +1535,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 11 unmodified lines | ||
| 1533 | 1535 | IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | |
| 1534 | 1536 | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; |
| 1535 | 1537 | local = __ip_vs_get_out_rt(cp->ipvs, cp->af, skb, cp->dest, cp->daddr.ip, rt_mode, |
| 1536 | − NULL, iph); | |
| 1538 | + NULL, ciph); | |
| 1537 | 1539 | if (local < 0) |
| 1538 | 1540 | goto tx_error; |
| 1539 | 1541 | rt = skb_rtable(skb); |
| @@ -1564,14 +1566,21 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 24 unmodified lines | ||
| 1564 | 1566 | goto tx_error; |
| 1565 | 1567 | } |
| 1566 | 1568 | |
| 1569 | + wlen = ciph->len; | |
| 1570 | + if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || | |
| 1571 | + ciph->protocol == IPPROTO_SCTP) { | |
| 1572 | + wlen += 2 * sizeof(__u16); /* Also mangle ports */ | |
| 1573 | + has_ports = true; | |
| 1574 | + } | |
| 1575 | + | |
| 1567 | 1576 | /* copy-on-write the packet before mangling it */ |
| 1568 | − if (skb_ensure_writable(skb, offset)) | |
| 1577 | + if (skb_ensure_writable(skb, wlen)) | |
| 1569 | 1578 | goto tx_error; |
| 1570 | 1579 | |
| 1571 | 1580 | if (skb_cow(skb, rt->dst.dev->hard_header_len)) |
| 1572 | 1581 | goto tx_error; |
| 1573 | 1582 | |
| 1574 | − ip_vs_nat_icmp(skb, pp, cp, 0); | |
| 1583 | + ip_vs_nat_icmp(skb, pp, cp, 0, toff, has_ports); | |
| 1575 | 1584 | |
| 1576 | 1585 | /* Another hack: avoid icmp_send in ip_fragment */ |
| 1577 | 1586 | skb->ignore_df = 1; |
| @@ -1587,10 +1596,12 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 9 unmodified lines | ||
| 1587 | 1596 | #ifdef CONFIG_IP_VS_IPV6 |
| 1588 | 1597 | int |
| 1589 | 1598 | ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1590 | − struct ip_vs_protocol *pp, int offset, unsigned int hooknum, | |
| 1591 | − struct ip_vs_iphdr *ipvsh) | |
| 1599 | + struct ip_vs_protocol *pp, unsigned int toff, | |
| 1600 | + unsigned int hooknum, struct ip_vs_iphdr *ciph) | |
| 1592 | 1601 | { |
| 1602 | + bool has_ports = false; | |
| 1593 | 1603 | struct rt6_info *rt; /* Route to the other host */ |
| 1604 | + unsigned int wlen; | |
| 1594 | 1605 | int rc; |
| 1595 | 1606 | int local; |
| 1596 | 1607 | int rt_mode; |
| @@ -1600,7 +1611,7 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 3 unmodified lines | ||
| 1600 | 1611 | translate address/port back */ |
| 1601 | 1612 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { |
| 1602 | 1613 | if (cp->packet_xmit) |
| 1603 | − rc = cp->packet_xmit(skb, cp, pp, ipvsh); | |
| 1614 | + rc = cp->packet_xmit(skb, cp, pp, ciph); | |
| 1604 | 1615 | else |
| 1605 | 1616 | rc = NF_ACCEPT; |
| 1606 | 1617 | /* do not touch skb anymore */ |
| @@ -1617,7 +1628,7 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 10 unmodified lines | ||
| 1617 | 1628 | IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | |
| 1618 | 1629 | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; |
| 1619 | 1630 | local = __ip_vs_get_out_rt_v6(cp->ipvs, cp->af, skb, cp->dest, |
| 1620 | − &cp->daddr.in6, NULL, ipvsh, 0, rt_mode); | |
| 1631 | + &cp->daddr.in6, NULL, ciph, 0, rt_mode); | |
| 1621 | 1632 | if (local < 0) |
| 1622 | 1633 | goto tx_error; |
| 1623 | 1634 | rt = dst_rt6_info(skb_dst(skb)); |
| @@ -1648,14 +1659,21 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 24 unmodified lines | ||
| 1648 | 1659 | goto tx_error; |
| 1649 | 1660 | } |
| 1650 | 1661 | |
| 1662 | + wlen = ciph->len; | |
| 1663 | + if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || | |
| 1664 | + ciph->protocol == IPPROTO_SCTP) { | |
| 1665 | + wlen += 2 * sizeof(__u16); /* Also mangle ports */ | |
| 1666 | + has_ports = true; | |
| 1667 | + } | |
| 1668 | + | |
| 1651 | 1669 | /* copy-on-write the packet before mangling it */ |
| 1652 | − if (skb_ensure_writable(skb, offset)) | |
| 1670 | + if (skb_ensure_writable(skb, wlen)) | |
| 1653 | 1671 | goto tx_error; |
| 1654 | 1672 | |
| 1655 | 1673 | if (skb_cow(skb, rt->dst.dev->hard_header_len)) |
| 1656 | 1674 | goto tx_error; |
| 1657 | 1675 | |
| 1658 | − ip_vs_nat_icmp_v6(skb, pp, cp, 0); | |
| 1676 | + ip_vs_nat_icmp_v6(skb, pp, cp, 0, toff, has_ports, ciph); | |
| 1659 | 1677 | |
| 1660 | 1678 | /* Another hack: avoid icmp_send in ip_fragment */ |
| 1661 | 1679 | skb->ignore_df = 1; |
| @@ -427,7 +427,6 @@ static void nf_ct_expect_insert(struct nf_conntrack_expect *exp, · 426 unmodified lines | ||
| 427 | 427 | exp->timeout += helper->expect_policy[exp->class].timeout * HZ; |
| 428 | 428 | |
| 429 | 429 | hlist_add_head_rcu(&exp->lnode, &master_help->expectations); |
| 430 | − master_help->expecting[exp->class]++; | |
| 431 | 430 | |
| 432 | 431 | hlist_add_head_rcu(&exp->hnode, &nf_ct_expect_hash[h]); |
| 433 | 432 | cnet = nf_ct_pernet(net); |
| @@ -534,6 +533,7 @@ int nf_ct_expect_related_report(struct nf_conntrack_expect *expect, · 100 unmodified lines | ||
| 534 | 533 | if (ret < 0) |
| 535 | 534 | goto out; |
| 536 | 535 | |
| 536 | + master_help->expecting[expect->class]++; | |
| 537 | 537 | nf_ct_expect_insert(expect, master_help); |
| 538 | 538 | |
| 539 | 539 | nf_ct_expect_event_report(IPEXP_NEW, expect, portid, report); |
| @@ -546,6 +546,39 @@ out: · 6 unmodified lines | ||
| 546 | 546 | } |
| 547 | 547 | EXPORT_SYMBOL_GPL(nf_ct_expect_related_report); |
| 548 | 548 | |
| 549 | +int nf_ct_expect_related_pair(struct nf_conntrack_expect *expect[], | |
| 550 | + unsigned int flags) | |
| 551 | +{ | |
| 552 | + struct nf_conn_help *master_help; | |
| 553 | + int i, ret; | |
| 554 | + | |
| 555 | + spin_lock_bh(&nf_conntrack_expect_lock); | |
| 556 | + master_help = nfct_help(expect[0]->master); | |
| 557 | + if (!master_help || master_help != nfct_help(expect[1]->master)) { | |
| 558 | + ret = -EINVAL; | |
| 559 | + goto out; | |
| 560 | + } | |
| 561 | + | |
| 562 | + for (i = 0; i < 2; i++) { | |
| 563 | + ret = __nf_ct_expect_check(expect[i], master_help, flags); | |
| 564 | + if (ret < 0) { | |
| 565 | + if (i == 1) | |
| 566 | + master_help->expecting[expect[0]->class]--; | |
| 567 | + goto out; | |
| 568 | + } | |
| 569 | + master_help->expecting[expect[i]->class]++; | |
| 570 | + } | |
| 571 | + | |
| 572 | + for (i = 0; i < 2; i++) { | |
| 573 | + nf_ct_expect_insert(expect[i], master_help); | |
| 574 | + nf_ct_expect_event_report(IPEXP_NEW, expect[i], 0, 0); | |
| 575 | + } | |
| 576 | +out: | |
| 577 | + spin_unlock_bh(&nf_conntrack_expect_lock); | |
| 578 | + return ret; | |
| 579 | +} | |
| 580 | +EXPORT_SYMBOL_GPL(nf_ct_expect_related_pair); | |
| 581 | + | |
| 549 | 582 | void nf_ct_expect_iterate_destroy(bool (*iter)(struct nf_conntrack_expect *e, void *data), |
| 550 | 583 | void *data) |
| 551 | 584 | { |
| @@ -1663,7 +1663,7 @@ static int sip_help_tcp(struct sk_buff *skb, unsigned int protoff, · 1662 unmodified lines | ||
| 1663 | 1663 | unsigned int matchoff, matchlen; |
| 1664 | 1664 | unsigned int msglen, origlen; |
| 1665 | 1665 | const char *dptr, *end; |
| 1666 | − s16 diff, tdiff = 0; | |
| 1666 | + s32 diff, tdiff = 0; | |
| 1667 | 1667 | int ret = NF_ACCEPT; |
| 1668 | 1668 | unsigned long clen; |
| 1669 | 1669 | bool term; |
| @@ -321,7 +321,7 @@ next: · 320 unmodified lines | ||
| 321 | 321 | } |
| 322 | 322 | |
| 323 | 323 | static void nf_nat_sip_seq_adjust(struct sk_buff *skb, unsigned int protoff, |
| 324 | − s16 off) | |
| 324 | + s32 off) | |
| 325 | 325 | { |
| 326 | 326 | enum ip_conntrack_info ctinfo; |
| 327 | 327 | struct nf_conn *ct = nf_ct_get(skb, &ctinfo); |
| @@ -592,6 +592,7 @@ static unsigned int nf_nat_sdp_media(struct sk_buff *skb, unsigned int protoff, · 264 unmodified lines | ||
| 592 | 592 | unsigned int medialen, |
| 593 | 593 | union nf_inet_addr *rtp_addr) |
| 594 | 594 | { |
| 595 | + struct nf_conntrack_expect *rtp_pair[2] = { rtp_exp, rtcp_exp }; | |
| 595 | 596 | enum ip_conntrack_info ctinfo; |
| 596 | 597 | struct nf_conn *ct = nf_ct_get(skb, &ctinfo); |
| 597 | 598 | enum ip_conntrack_dir dir = CTINFO2DIR(ctinfo); |
| @@ -622,24 +623,15 @@ static unsigned int nf_nat_sdp_media(struct sk_buff *skb, unsigned int protoff, · 24 unmodified lines | ||
| 622 | 623 | int ret; |
| 623 | 624 | |
| 624 | 625 | rtp_exp->tuple.dst.u.udp.port = htons(port); |
| 625 | − ret = nf_ct_expect_related(rtp_exp, | |
| 626 | − NF_CT_EXP_F_SKIP_MASTER); | |
| 627 | − if (ret == -EBUSY) | |
| 628 | − continue; | |
| 629 | − else if (ret < 0) { | |
| 630 | − port = 0; | |
| 631 | − break; | |
| 632 | − } | |
| 633 | 626 | rtcp_exp->tuple.dst.u.udp.port = htons(port + 1); |
| 634 | − ret = nf_ct_expect_related(rtcp_exp, | |
| 635 | − NF_CT_EXP_F_SKIP_MASTER); | |
| 627 | + | |
| 628 | + ret = nf_ct_expect_related_pair(rtp_pair, | |
| 629 | + NF_CT_EXP_F_SKIP_MASTER); | |
| 636 | 630 | if (ret == 0) |
| 637 | 631 | break; |
| 638 | − else if (ret == -EBUSY) { | |
| 639 | − nf_ct_unexpect_related(rtp_exp); | |
| 632 | + else if (ret == -EBUSY) | |
| 640 | 633 | continue; |
| 641 | − } else if (ret < 0) { | |
| 642 | − nf_ct_unexpect_related(rtp_exp); | |
| 634 | + else if (ret < 0) { | |
| 643 | 635 | port = 0; |
| 644 | 636 | break; |
| 645 | 637 | } |
| @@ -45,8 +45,6 @@ enum { · 44 unmodified lines | ||
| 45 | 45 | NFT_VALIDATE_DO, |
| 46 | 46 | }; |
| 47 | 47 | |
| 48 | −static struct rhltable nft_objname_ht; | |
| 49 | − | |
| 50 | 48 | static u32 nft_chain_hash(const void *data, u32 len, u32 seed); |
| 51 | 49 | static u32 nft_chain_hash_obj(const void *data, u32 len, u32 seed); |
| 52 | 50 | static int nft_chain_hash_cmp(struct rhashtable_compare_arg *, const void *); |
| @@ -1635,6 +1633,10 @@ static int nf_tables_newtable(struct sk_buff *skb, const struct nfnl_info *info, · 1582 unmodified lines | ||
| 1635 | 1633 | if (err) |
| 1636 | 1634 | goto err_chain_ht; |
| 1637 | 1635 | |
| 1636 | + err = rhltable_init(&table->objname_ht, &nft_objname_ht_params); | |
| 1637 | + if (err < 0) | |
| 1638 | + goto err_obj_ht; | |
| 1639 | + | |
| 1638 | 1640 | INIT_LIST_HEAD(&table->chains); |
| 1639 | 1641 | INIT_LIST_HEAD(&table->sets); |
| 1640 | 1642 | INIT_LIST_HEAD(&table->objects); |
| @@ -1653,6 +1655,8 @@ static int nf_tables_newtable(struct sk_buff *skb, const struct nfnl_info *info, · 12 unmodified lines | ||
| 1653 | 1655 | list_add_tail_rcu(&table->list, &nft_net->tables); |
| 1654 | 1656 | return 0; |
| 1655 | 1657 | err_trans: |
| 1658 | + rhltable_destroy(&table->objname_ht); | |
| 1659 | +err_obj_ht: | |
| 1656 | 1660 | rhltable_destroy(&table->chains_ht); |
| 1657 | 1661 | err_chain_ht: |
| 1658 | 1662 | kfree(table->udata); |
| @@ -1819,6 +1823,7 @@ static void nf_tables_table_destroy(struct nft_table *table) · 160 unmodified lines | ||
| 1819 | 1823 | return; |
| 1820 | 1824 | |
| 1821 | 1825 | rhltable_destroy(&table->chains_ht); |
| 1826 | + rhltable_destroy(&table->objname_ht); | |
| 1822 | 1827 | kfree(table->name); |
| 1823 | 1828 | kfree(table->udata); |
| 1824 | 1829 | kfree(table); |
| @@ -8086,7 +8091,7 @@ void nft_unregister_obj(struct nft_object_type *obj_type) · 6261 unmodified lines | ||
| 8086 | 8091 | EXPORT_SYMBOL_GPL(nft_unregister_obj); |
| 8087 | 8092 | |
| 8088 | 8093 | struct nft_object *nft_obj_lookup(const struct net *net, |
| 8089 | − const struct nft_table *table, | |
| 8094 | + struct nft_table *table, | |
| 8090 | 8095 | const struct nlattr *nla, u32 objtype, |
| 8091 | 8096 | u8 genmask) |
| 8092 | 8097 | { |
| @@ -8102,7 +8107,7 @@ struct nft_object *nft_obj_lookup(const struct net *net, · 9 unmodified lines | ||
| 8102 | 8107 | !lockdep_commit_lock_is_held(net)); |
| 8103 | 8108 | |
| 8104 | 8109 | rcu_read_lock(); |
| 8105 | − list = rhltable_lookup(&nft_objname_ht, &k, nft_objname_ht_params); | |
| 8110 | + list = rhltable_lookup(&table->objname_ht, &k, nft_objname_ht_params); | |
| 8106 | 8111 | if (!list) |
| 8107 | 8112 | goto out; |
| 8108 | 8113 | |
| @@ -8382,7 +8387,7 @@ static int nf_tables_newobj(struct sk_buff *skb, const struct nfnl_info *info, · 273 unmodified lines | ||
| 8382 | 8387 | if (err < 0) |
| 8383 | 8388 | goto err_trans; |
| 8384 | 8389 | |
| 8385 | − err = rhltable_insert(&nft_objname_ht, &obj->rhlhead, | |
| 8390 | + err = rhltable_insert(&table->objname_ht, &obj->rhlhead, | |
| 8386 | 8391 | nft_objname_ht_params); |
| 8387 | 8392 | if (err < 0) |
| 8388 | 8393 | goto err_obj_ht; |
| @@ -8567,8 +8572,8 @@ nf_tables_getobj_single(u32 portid, const struct nfnl_info *info, · 178 unmodified lines | ||
| 8567 | 8572 | struct netlink_ext_ack *extack = info->extack; |
| 8568 | 8573 | u8 genmask = nft_genmask_cur(info->net); |
| 8569 | 8574 | u8 family = info->nfmsg->nfgen_family; |
| 8570 | − const struct nft_table *table; | |
| 8571 | 8575 | struct net *net = info->net; |
| 8576 | + struct nft_table *table; | |
| 8572 | 8577 | struct nft_object *obj; |
| 8573 | 8578 | struct sk_buff *skb2; |
| 8574 | 8579 | u32 objtype; |
| @@ -10437,9 +10442,9 @@ static void nf_tables_commit_chain(struct net *net, struct nft_chain *chain) · 1862 unmodified lines | ||
| 10437 | 10442 | nf_tables_commit_chain_free_rules_old(g0); |
| 10438 | 10443 | } |
| 10439 | 10444 | |
| 10440 | −static void nft_obj_del(struct nft_object *obj) | |
| 10445 | +static void nft_obj_del(struct nft_table *table, struct nft_object *obj) | |
| 10441 | 10446 | { |
| 10442 | − rhltable_remove(&nft_objname_ht, &obj->rhlhead, nft_objname_ht_params); | |
| 10447 | + rhltable_remove(&table->objname_ht, &obj->rhlhead, nft_objname_ht_params); | |
| 10443 | 10448 | list_del_rcu(&obj->list); |
| 10444 | 10449 | } |
| 10445 | 10450 | |
| @@ -11124,7 +11129,7 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb) · 678 unmodified lines | ||
| 11124 | 11129 | break; |
| 11125 | 11130 | case NFT_MSG_DELOBJ: |
| 11126 | 11131 | case NFT_MSG_DESTROYOBJ: |
| 11127 | − nft_obj_del(nft_trans_obj(trans)); | |
| 11132 | + nft_obj_del(table, nft_trans_obj(trans)); | |
| 11128 | 11133 | nf_tables_obj_notify(&ctx, nft_trans_obj(trans), |
| 11129 | 11134 | trans->msg_type); |
| 11130 | 11135 | break; |
| @@ -11416,7 +11421,7 @@ static int __nf_tables_abort(struct net *net, enum nfnl_abort_action action) · 285 unmodified lines | ||
| 11416 | 11421 | nft_trans_destroy(trans); |
| 11417 | 11422 | } else { |
| 11418 | 11423 | nft_use_dec_restore(&table->use); |
| 11419 | − nft_obj_del(nft_trans_obj(trans)); | |
| 11424 | + nft_obj_del(table, nft_trans_obj(trans)); | |
| 11420 | 11425 | } |
| 11421 | 11426 | break; |
| 11422 | 11427 | case NFT_MSG_DELOBJ: |
| @@ -12043,7 +12048,7 @@ static void __nft_release_table(struct net *net, struct nft_table *table) · 620 unmodified lines | ||
| 12043 | 12048 | nft_set_destroy(&ctx, set); |
| 12044 | 12049 | } |
| 12045 | 12050 | list_for_each_entry_safe(obj, ne, &table->objects, list) { |
| 12046 | − nft_obj_del(obj); | |
| 12051 | + nft_obj_del(table, obj); | |
| 12047 | 12052 | nft_use_dec(&table->use); |
| 12048 | 12053 | nft_obj_destroy(&ctx, obj); |
| 12049 | 12054 | } |
| @@ -12225,10 +12230,6 @@ static int __init nf_tables_module_init(void) · 175 unmodified lines | ||
| 12225 | 12230 | if (err < 0) |
| 12226 | 12231 | goto err_netdev_notifier; |
| 12227 | 12232 | |
| 12228 | − err = rhltable_init(&nft_objname_ht, &nft_objname_ht_params); | |
| 12229 | − if (err < 0) | |
| 12230 | − goto err_rht_objname; | |
| 12231 | − | |
| 12232 | 12233 | err = nft_offload_init(); |
| 12233 | 12234 | if (err < 0) |
| 12234 | 12235 | goto err_offload; |
| @@ -12251,8 +12252,6 @@ err_nfnl_subsys: · 16 unmodified lines | ||
| 12251 | 12252 | err_netlink_notifier: |
| 12252 | 12253 | nft_offload_exit(); |
| 12253 | 12254 | err_offload: |
| 12254 | − rhltable_destroy(&nft_objname_ht); | |
| 12255 | −err_rht_objname: | |
| 12256 | 12255 | unregister_netdevice_notifier(&nf_tables_flowtable_notifier); |
| 12257 | 12256 | err_netdev_notifier: |
| 12258 | 12257 | nf_tables_core_module_exit(); |
| @@ -12274,7 +12273,6 @@ static void __exit nf_tables_module_exit(void) · 15 unmodified lines | ||
| 12274 | 12273 | unregister_pernet_subsys(&nf_tables_net_ops); |
| 12275 | 12274 | cancel_work_sync(&trans_gc_work); |
| 12276 | 12275 | rcu_barrier(); |
| 12277 | − rhltable_destroy(&nft_objname_ht); | |
| 12278 | 12276 | nf_tables_core_module_exit(); |
| 12279 | 12277 | } |
| 12280 | 12278 | |
| @@ -259,9 +259,7 @@ nla_put_failure: · 258 unmodified lines | ||
| 259 | 259 | static bool nft_payload_offload_mask(struct nft_offload_reg *reg, |
| 260 | 260 | u32 priv_len, u32 field_len) |
| 261 | 261 | { |
| 262 | − unsigned int remainder, delta, k; | |
| 263 | 262 | struct nft_data mask = {}; |
| 264 | − __be32 remainder_mask; | |
| 265 | 263 | |
| 266 | 264 | if (priv_len == field_len) { |
| 267 | 265 | memset(®->mask, 0xff, priv_len); |
| @@ -270,15 +268,7 @@ static bool nft_payload_offload_mask(struct nft_offload_reg *reg, · 2 unmodified lines | ||
| 270 | 268 | return false; |
| 271 | 269 | } |
| 272 | 270 | |
| 273 | − memset(&mask, 0xff, field_len); | |
| 274 | − remainder = priv_len % sizeof(u32); | |
| 275 | − if (remainder) { | |
| 276 | − k = priv_len / sizeof(u32); | |
| 277 | − delta = field_len - priv_len; | |
| 278 | − remainder_mask = htonl(~((1 << (delta * BITS_PER_BYTE)) - 1)); | |
| 279 | − mask.data[k] = (__force u32)remainder_mask; | |
| 280 | − } | |
| 281 | − | |
| 271 | + memset(&mask, 0xff, priv_len); | |
| 282 | 272 | memcpy(®->mask, &mask, field_len); |
| 283 | 273 | |
| 284 | 274 | return true; |
| @@ -117,6 +117,7 @@ struct xt_hashlimit_htable { · 116 unmodified lines | ||
| 117 | 117 | refcount_t use; |
| 118 | 118 | u_int8_t family; |
| 119 | 119 | bool rnd_initialized; |
| 120 | + bool ratematch; | |
| 120 | 121 | |
| 121 | 122 | struct hashlimit_cfg3 cfg; /* config */ |
| 122 | 123 | |
| @@ -323,6 +324,7 @@ static int htable_create(struct net *net, struct hashlimit_cfg3 *cfg, · 200 unmodified lines | ||
| 323 | 324 | kvfree(hinfo); |
| 324 | 325 | return -ENOMEM; |
| 325 | 326 | } |
| 327 | + hinfo->ratematch = !!(cfg->mode & XT_HASHLIMIT_RATE_MATCH); | |
| 326 | 328 | spin_lock_init(&hinfo->lock); |
| 327 | 329 | |
| 328 | 330 | switch (revision) { |
| @@ -872,7 +874,10 @@ static int hashlimit_mt_check_common(const struct xt_mtchk_param *par, · 543 unmodified lines | ||
| 872 | 874 | } |
| 873 | 875 | |
| 874 | 876 | /* Check for overflow. */ |
| 875 | − if (revision >= 3 && cfg->mode & XT_HASHLIMIT_RATE_MATCH) { | |
| 877 | + if (cfg->mode & XT_HASHLIMIT_RATE_MATCH) { | |
| 878 | + if (revision < 3) | |
| 879 | + return -EINVAL; | |
| 880 | + | |
| 876 | 881 | if (cfg->avg == 0 || cfg->avg > U32_MAX) { |
| 877 | 882 | pr_info_ratelimited("invalid rate\n"); |
| 878 | 883 | return -ERANGE; |
| @@ -905,6 +910,15 @@ static int hashlimit_mt_check_common(const struct xt_mtchk_param *par, · 26 unmodified lines | ||
| 905 | 910 | mutex_unlock(&hashlimit_mutex); |
| 906 | 911 | return ret; |
| 907 | 912 | } |
| 913 | + } else { | |
| 914 | + if ((cfg->mode & XT_HASHLIMIT_RATE_MATCH && | |
| 915 | + !(*hinfo)->ratematch) || | |
| 916 | + (!(cfg->mode & XT_HASHLIMIT_RATE_MATCH) && | |
| 917 | + (*hinfo)->ratematch)) { | |
| 918 | + mutex_unlock(&hashlimit_mutex); | |
| 919 | + htable_put(*hinfo); | |
| 920 | + return -EINVAL; | |
| 921 | + } | |
| 908 | 922 | } |
| 909 | 923 | mutex_unlock(&hashlimit_mutex); |
| 910 | 924 | |
| @@ -1108,6 +1108,10 @@ static int execute_masked_set_action(struct sk_buff *skb, · 1107 unmodified lines | ||
| 1108 | 1108 | return err; |
| 1109 | 1109 | } |
| 1110 | 1110 | |
| 1111 | +/* When 'last' is true, recirc() should always consume the 'skb'. | |
| 1112 | + * Otherwise, recirc() should keep 'skb' intact regardless what | |
| 1113 | + * actions are executed on recirculation. | |
| 1114 | + */ | |
| 1111 | 1115 | static int execute_recirc(struct datapath *dp, struct sk_buff *skb, |
| 1112 | 1116 | struct sw_flow_key *key, |
| 1113 | 1117 | const struct nlattr *a, bool last) |
| @@ -1118,8 +1122,12 @@ static int execute_recirc(struct datapath *dp, struct sk_buff *skb, · 4 unmodified lines | ||
| 1118 | 1122 | int err; |
| 1119 | 1123 | |
| 1120 | 1124 | err = ovs_flow_key_update(skb, key); |
| 1121 | − if (err) | |
| 1125 | + if (err) { | |
| 1126 | + if (last) | |
| 1127 | + ovs_kfree_skb_reason(skb, | |
| 1128 | + OVS_DROP_ACTION_ERROR); | |
| 1122 | 1129 | return err; |
| 1130 | + } | |
| 1123 | 1131 | } |
| 1124 | 1132 | BUG_ON(!is_flow_key_valid(key)); |
| 1125 | 1133 | |
| @@ -1372,7 +1380,7 @@ static int do_execute_actions(struct datapath *dp, struct sk_buff *skb, · 246 unmodified lines | ||
| 1372 | 1380 | if (!is_flow_key_valid(key)) { |
| 1373 | 1381 | err = ovs_flow_key_update(skb, key); |
| 1374 | 1382 | if (err) |
| 1375 | − return err; | |
| 1383 | + break; | |
| 1376 | 1384 | } |
| 1377 | 1385 | |
| 1378 | 1386 | err = ovs_ct_execute(ovs_dp_get_net(dp), skb, key, |
| @@ -133,18 +133,10 @@ static void dp_meter_instance_remove(struct dp_meter_instance *ti, · 132 unmodified lines | ||
| 133 | 133 | |
| 134 | 134 | static int attach_meter(struct dp_meter_table *tbl, struct dp_meter *meter) |
| 135 | 135 | { |
| 136 | − struct dp_meter_instance *ti = rcu_dereference_ovsl(tbl->ti); | |
| 137 | − u32 hash = meter_hash(ti, meter->id); | |
| 136 | + struct dp_meter_instance *ti; | |
| 137 | + u32 hash; | |
| 138 | 138 | int err; |
| 139 | 139 | |
| 140 | − /* In generally, slots selected should be empty, because | |
| 141 | − * OvS uses id-pool to fetch a available id. | |
| 142 | − */ | |
| 143 | − if (unlikely(rcu_dereference_ovsl(ti->dp_meters[hash]))) | |
| 144 | − return -EBUSY; | |
| 145 | − | |
| 146 | − dp_meter_instance_insert(ti, meter); | |
| 147 | − | |
| 148 | 140 | /* That function is thread-safe. */ |
| 149 | 141 | tbl->count++; |
| 150 | 142 | if (tbl->count >= tbl->max_meters_allowed) { |
| @@ -152,16 +144,29 @@ static int attach_meter(struct dp_meter_table *tbl, struct dp_meter *meter) · 1 unmodified line | ||
| 152 | 144 | goto attach_err; |
| 153 | 145 | } |
| 154 | 146 | |
| 155 | − if (tbl->count >= ti->n_meters && | |
| 156 | − dp_meter_instance_realloc(tbl, ti->n_meters * 2)) { | |
| 157 | − err = -ENOMEM; | |
| 147 | + ti = rcu_dereference_ovsl(tbl->ti); | |
| 148 | + if (tbl->count >= ti->n_meters) { | |
| 149 | + err = dp_meter_instance_realloc(tbl, ti->n_meters * 2); | |
| 150 | + if (err) | |
| 151 | + goto attach_err; | |
| 152 | + | |
| 153 | + ti = rcu_dereference_ovsl(tbl->ti); | |
| 154 | + } | |
| 155 | + | |
| 156 | + hash = meter_hash(ti, meter->id); | |
| 157 | + | |
| 158 | + /* In general, selected slots should be empty, because | |
| 159 | + * OvS uses id-pool to fetch available ids. | |
| 160 | + */ | |
| 161 | + if (unlikely(rcu_dereference_ovsl(ti->dp_meters[hash]))) { | |
| 162 | + err = -EBUSY; | |
| 158 | 163 | goto attach_err; |
| 159 | 164 | } |
| 160 | 165 | |
| 166 | + dp_meter_instance_insert(ti, meter); | |
| 161 | 167 | return 0; |
| 162 | 168 | |
| 163 | 169 | attach_err: |
| 164 | − dp_meter_instance_remove(ti, meter); | |
| 165 | 170 | tbl->count--; |
| 166 | 171 | return err; |
| 167 | 172 | } |
| @@ -62,7 +62,14 @@ psp_nl_multicast_per_ns(struct psp_dev *psd, unsigned int group, · 61 unmodified lines | ||
| 62 | 62 | struct net *main_net; |
| 63 | 63 | struct sk_buff *ntf; |
| 64 | 64 | |
| 65 | − main_net = dev_net(psd->main_netdev); | |
| 65 | + /* device may be changing netns in parallel */ | |
| 66 | + rcu_read_lock(); | |
| 67 | + main_net = maybe_get_net(dev_net_rcu(psd->main_netdev)); | |
| 68 | + rcu_read_unlock(); | |
| 69 | + | |
| 70 | + if (!main_net) | |
| 71 | + return; | |
| 72 | + | |
| 66 | 73 | xa_init(&sent_nets); |
| 67 | 74 | |
| 68 | 75 | list_for_each_entry(entry, &psd->assoc_dev_list, dev_list) { |
| @@ -88,10 +95,10 @@ psp_nl_multicast_per_ns(struct psp_dev *psd, unsigned int group, · 19 unmodified lines | ||
| 88 | 95 | |
| 89 | 96 | /* Send to main device netns */ |
| 90 | 97 | ntf = build_ntf(psd, main_net, ctx); |
| 91 | − if (!ntf) | |
| 92 | − return; | |
| 93 | − genlmsg_multicast_netns(&psp_nl_family, main_net, ntf, 0, group, | |
| 94 | − GFP_KERNEL); | |
| 98 | + if (ntf) | |
| 99 | + genlmsg_multicast_netns(&psp_nl_family, main_net, ntf, 0, group, | |
| 100 | + GFP_KERNEL); | |
| 101 | + put_net(main_net); | |
| 95 | 102 | } |
| 96 | 103 | |
| 97 | 104 | static struct sk_buff *psp_nl_clone_ntf(struct psp_dev *psd, struct net *net, |
| @@ -355,23 +355,25 @@ int rds_tcp_laddr_check(struct net *net, const struct in6_addr *addr, · 354 unmodified lines | ||
| 355 | 355 | /* If the scope_id is specified, check only those addresses |
| 356 | 356 | * hosted on the specified interface. |
| 357 | 357 | */ |
| 358 | + rcu_read_lock(); | |
| 358 | 359 | if (scope_id != 0) { |
| 359 | − rcu_read_lock(); | |
| 360 | 360 | dev = dev_get_by_index_rcu(net, scope_id); |
| 361 | 361 | /* scope_id is not valid... */ |
| 362 | 362 | if (!dev) { |
| 363 | 363 | rcu_read_unlock(); |
| 364 | 364 | return -EADDRNOTAVAIL; |
| 365 | 365 | } |
| 366 | − rcu_read_unlock(); | |
| 367 | 366 | } |
| 368 | 367 | #if IS_ENABLED(CONFIG_IPV6) |
| 369 | 368 | if (ipv6_mod_enabled()) { |
| 370 | 369 | ret = ipv6_chk_addr(net, addr, dev, 0); |
| 371 | − if (ret) | |
| 370 | + if (ret) { | |
| 371 | + rcu_read_unlock(); | |
| 372 | 372 | return 0; |
| 373 | + } | |
| 373 | 374 | } |
| 374 | 375 | #endif |
| 376 | + rcu_read_unlock(); | |
| 375 | 377 | return -EADDRNOTAVAIL; |
| 376 | 378 | } |
| 377 | 379 | |
| @@ -1107,6 +1107,13 @@ static int u32_change(struct net *net, struct sk_buff *in_skb, · 1106 unmodified lines | ||
| 1107 | 1107 | goto erridr; |
| 1108 | 1108 | } |
| 1109 | 1109 | |
| 1110 | + if (s->offshift >= 16) { | |
| 1111 | + NL_SET_ERR_MSG_MOD(extack, | |
| 1112 | + "offshift must be less than 16"); | |
| 1113 | + err = -EINVAL; | |
| 1114 | + goto erridr; | |
| 1115 | + } | |
| 1116 | + | |
| 1110 | 1117 | n = kzalloc_flex(*n, sel.keys, s->nkeys); |
| 1111 | 1118 | if (n == NULL) { |
| 1112 | 1119 | err = -ENOBUFS; |
| @@ -2609,9 +2609,11 @@ static void cake_configure_rates(struct Qdisc *sch, u64 rate, bool rate_adjust) · 2608 unmodified lines | ||
| 2609 | 2609 | break; |
| 2610 | 2610 | } |
| 2611 | 2611 | |
| 2612 | − for (c = qd->tin_cnt; c < CAKE_MAX_TINS; c++) { | |
| 2613 | − cake_clear_tin(sch, c); | |
| 2614 | − qd->tins[c].cparams.mtu_time = qd->tins[ft].cparams.mtu_time; | |
| 2612 | + if (!rate_adjust) { | |
| 2613 | + for (c = qd->tin_cnt; c < CAKE_MAX_TINS; c++) { | |
| 2614 | + cake_clear_tin(sch, c); | |
| 2615 | + qd->tins[c].cparams.mtu_time = qd->tins[ft].cparams.mtu_time; | |
| 2616 | + } | |
| 2615 | 2617 | } |
| 2616 | 2618 | |
| 2617 | 2619 | qd->rate_ns = qd->tins[ft].tin_rate_ns; |
| @@ -614,6 +614,9 @@ struct sctp_transport *sctp_assoc_add_peer(struct sctp_association *asoc, · 613 unmodified lines | ||
| 614 | 614 | return peer; |
| 615 | 615 | } |
| 616 | 616 | |
| 617 | + if (asoc->peer.transport_count == U16_MAX) | |
| 618 | + return NULL; | |
| 619 | + | |
| 617 | 620 | peer = sctp_transport_new(asoc->base.net, addr, gfp); |
| 618 | 621 | if (!peer) |
| 619 | 622 | return NULL; |
| @@ -1802,9 +1802,9 @@ struct sctp_association *sctp_unpack_cookie( · 1801 unmodified lines | ||
| 1802 | 1802 | goto fail; |
| 1803 | 1803 | } |
| 1804 | 1804 | |
| 1805 | − /* Check to see if the cookie is stale. If there is already | |
| 1806 | − * an association, there is no need to check cookie's expiration | |
| 1807 | − * for init collision case of lost COOKIE ACK. | |
| 1805 | + /* Check to see if the cookie is stale. RFC 9260 Section 5.2.4 | |
| 1806 | + * exempts an expired cookie only when both Verification Tags match | |
| 1807 | + * the current association. | |
| 1808 | 1808 | * If skb has been timestamped, then use the stamp, otherwise |
| 1809 | 1809 | * use current time. This introduces a small possibility that |
| 1810 | 1810 | * a cookie may be considered expired, but this would only slow |
| @@ -1815,7 +1815,10 @@ struct sctp_association *sctp_unpack_cookie( · 4 unmodified lines | ||
| 1815 | 1815 | else |
| 1816 | 1816 | kt = ktime_get_real(); |
| 1817 | 1817 | |
| 1818 | − if (!asoc && ktime_before(bear_cookie->expiration, kt)) { | |
| 1818 | + if ((!asoc || | |
| 1819 | + asoc->c.my_vtag != bear_cookie->my_vtag || | |
| 1820 | + asoc->c.peer_vtag != bear_cookie->peer_vtag) && | |
| 1821 | + ktime_before(bear_cookie->expiration, kt)) { | |
| 1819 | 1822 | suseconds_t usecs = ktime_to_us(ktime_sub(kt, bear_cookie->expiration)); |
| 1820 | 1823 | __be32 n = htonl(usecs); |
| 1821 | 1824 | |
| @@ -2168,7 +2171,13 @@ static enum sctp_ierror sctp_verify_param(struct net *net, · 346 unmodified lines | ||
| 2168 | 2171 | case SCTP_PARAM_HEARTBEAT_INFO: |
| 2169 | 2172 | case SCTP_PARAM_UNRECOGNIZED_PARAMETERS: |
| 2170 | 2173 | case SCTP_PARAM_ECN_CAPABLE: |
| 2174 | + break; | |
| 2171 | 2175 | case SCTP_PARAM_ADAPTATION_LAYER_IND: |
| 2176 | + if (ntohs(param.p->length) != sizeof(*param.aind)) { | |
| 2177 | + sctp_process_inv_paramlength(asoc, param.p, | |
| 2178 | + chunk, err_chunk); | |
| 2179 | + retval = SCTP_IERROR_ABORT; | |
| 2180 | + } | |
| 2172 | 2181 | break; |
| 2173 | 2182 | |
| 2174 | 2183 | case SCTP_PARAM_SUPPORTED_EXT: |
| @@ -1572,10 +1572,10 @@ static void __smc_lgr_terminate(struct smc_link_group *lgr, bool soft) · 1571 unmodified lines | ||
| 1572 | 1572 | read_lock_bh(&lgr->conns_lock); |
| 1573 | 1573 | node = rb_first(&lgr->conns_all); |
| 1574 | 1574 | while (node) { |
| 1575 | − read_unlock_bh(&lgr->conns_lock); | |
| 1576 | 1575 | conn = rb_entry(node, struct smc_connection, alert_node); |
| 1577 | 1576 | smc = container_of(conn, struct smc_sock, conn); |
| 1578 | 1577 | sock_hold(&smc->sk); /* sock_put below */ |
| 1578 | + read_unlock_bh(&lgr->conns_lock); | |
| 1579 | 1579 | lock_sock(&smc->sk); |
| 1580 | 1580 | smc_conn_kill(conn, soft); |
| 1581 | 1581 | release_sock(&smc->sk); |
| @@ -796,7 +796,7 @@ static __poll_t tipc_poll(struct file *file, struct socket *sock, · 795 unmodified lines | ||
| 796 | 796 | __poll_t revents = 0; |
| 797 | 797 | |
| 798 | 798 | sock_poll_wait(file, sock, wait); |
| 799 | − trace_tipc_sk_poll(sk, NULL, TIPC_DUMP_ALL, " "); | |
| 799 | + trace_tipc_sk_poll(sk, NULL, TIPC_DUMP_NONE, " "); | |
| 800 | 800 | |
| 801 | 801 | if (sk->sk_shutdown & RCV_SHUTDOWN) |
| 802 | 802 | revents |= EPOLLRDHUP | EPOLLIN | EPOLLRDNORM; |
| @@ -823,6 +823,7 @@ static int unix_listen(struct socket *sock, int backlog) · 822 unmodified lines | ||
| 823 | 823 | if (err) |
| 824 | 824 | goto out; |
| 825 | 825 | unix_state_lock(sk); |
| 826 | + err = -EINVAL; | |
| 826 | 827 | if (sk->sk_state != TCP_CLOSE && sk->sk_state != TCP_LISTEN) |
| 827 | 828 | goto out_unlock; |
| 828 | 829 | if (backlog > sk->sk_max_ack_backlog) |
| @@ -420,6 +420,7 @@ int nl80211_pmsr_start(struct sk_buff *skb, struct genl_info *info) · 419 unmodified lines | ||
| 420 | 420 | const struct cfg80211_pmsr_capabilities *capa; |
| 421 | 421 | struct cfg80211_pmsr_request *req; |
| 422 | 422 | struct nlattr *peers, *peer; |
| 423 | + u64 cookie; | |
| 423 | 424 | |
| 424 | 425 | capa = rdev->wiphy.pmsr_capa; |
| 425 | 426 | |
| @@ -521,14 +522,27 @@ int nl80211_pmsr_start(struct sk_buff *skb, struct genl_info *info) · 95 unmodified lines | ||
| 521 | 522 | } |
| 522 | 523 | req->cookie = cfg80211_assign_cookie(rdev); |
| 523 | 524 | req->nl_portid = info->snd_portid; |
| 525 | + cookie = req->cookie; | |
| 526 | + | |
| 527 | + /* | |
| 528 | + * Add to the list before the driver call; under races or broken | |
| 529 | + * drivers, completion may free the request before rdev_start_pmsr() | |
| 530 | + * returns. Use the saved cookie below. | |
| 531 | + */ | |
| 532 | + spin_lock_bh(&wdev->pmsr_lock); | |
| 533 | + list_add_tail(&req->list, &wdev->pmsr_list); | |
| 534 | + spin_unlock_bh(&wdev->pmsr_lock); | |
| 524 | 535 | |
| 525 | 536 | err = rdev_start_pmsr(rdev, wdev, req); |
| 526 | − if (err) | |
| 537 | + if (err) { | |
| 538 | + /* An error return leaves the request owned by this path. */ | |
| 539 | + spin_lock_bh(&wdev->pmsr_lock); | |
| 540 | + list_del(&req->list); | |
| 541 | + spin_unlock_bh(&wdev->pmsr_lock); | |
| 527 | 542 | goto out_err; |
| 543 | + } | |
| 528 | 544 | |
| 529 | − list_add_tail(&req->list, &wdev->pmsr_list); | |
| 530 | − | |
| 531 | − nl_set_extack_cookie_u64(info->extack, req->cookie); | |
| 545 | + nl_set_extack_cookie_u64(info->extack, cookie); | |
| 532 | 546 | return 0; |
| 533 | 547 | out_err: |
| 534 | 548 | kfree(req); |
| @@ -319,6 +319,15 @@ int cfg80211_wext_siwgenie(struct net_device *dev, · 318 unmodified lines | ||
| 319 | 319 | return 0; |
| 320 | 320 | |
| 321 | 321 | if (ie_len) { |
| 322 | + const struct element *elem; | |
| 323 | + | |
| 324 | + for_each_element(elem, extra, ie_len) { | |
| 325 | + /* nothing */ | |
| 326 | + } | |
| 327 | + | |
| 328 | + if (!for_each_element_completed(elem, extra, ie_len)) | |
| 329 | + return -EINVAL; | |
| 330 | + | |
| 322 | 331 | ie = kmemdup(extra, ie_len, GFP_KERNEL); |
| 323 | 332 | if (!ie) |
| 324 | 333 | return -ENOMEM; |
| @@ -499,6 +499,23 @@ void __xsk_map_flush(struct list_head *flush_list) · 498 unmodified lines | ||
| 499 | 499 | |
| 500 | 500 | void xsk_tx_completed(struct xsk_buff_pool *pool, u32 nb_entries) |
| 501 | 501 | { |
| 502 | + u32 reclaim_descs = READ_ONCE(pool->reclaim_descs); | |
| 503 | + | |
| 504 | + if (unlikely(reclaim_descs)) { | |
| 505 | + u32 pending_descs = READ_ONCE(pool->tx_zc_pending_descs); | |
| 506 | + | |
| 507 | + if (nb_entries < pending_descs) { | |
| 508 | + WRITE_ONCE(pool->tx_zc_pending_descs, | |
| 509 | + pending_descs - nb_entries); | |
| 510 | + xskq_prod_submit_n(pool->cq, nb_entries); | |
| 511 | + return; | |
| 512 | + } | |
| 513 | + | |
| 514 | + WRITE_ONCE(pool->tx_zc_pending_descs, 0); | |
| 515 | + nb_entries += reclaim_descs; | |
| 516 | + WRITE_ONCE(pool->reclaim_descs, 0); | |
| 517 | + } | |
| 518 | + | |
| 502 | 519 | xskq_prod_submit_n(pool->cq, nb_entries); |
| 503 | 520 | } |
| 504 | 521 | EXPORT_SYMBOL(xsk_tx_completed); |
| @@ -574,24 +591,157 @@ static u32 xsk_tx_peek_release_fallback(struct xsk_buff_pool *pool, u32 max_entr · 69 unmodified lines | ||
| 574 | 591 | return nb_pkts; |
| 575 | 592 | } |
| 576 | 593 | |
| 594 | +static void xsk_tx_commit_batch(struct xsk_buff_pool *pool, | |
| 595 | + struct xsk_tx_batch *batch) | |
| 596 | +{ | |
| 597 | + u32 nb_descs = xsk_tx_batch_cq_descs(batch); | |
| 598 | + u32 cq_cached_prod; | |
| 599 | + | |
| 600 | + if (!nb_descs) | |
| 601 | + return; | |
| 602 | + | |
| 603 | + cq_cached_prod = pool->cq->cached_prod; | |
| 604 | + xskq_prod_write_addr_batch(pool->cq, pool->tx_descs, nb_descs); | |
| 605 | + | |
| 606 | + if (unlikely(batch->reclaim_descs)) { | |
| 607 | + u32 cq_pending_descs; | |
| 608 | + | |
| 609 | + /* CQ is positional. Descriptors already written but not | |
| 610 | + * submitted must complete before any reclaim-only descriptors | |
| 611 | + * appended below. | |
| 612 | + */ | |
| 613 | + cq_pending_descs = cq_cached_prod - xskq_get_prod(pool->cq); | |
| 614 | + | |
| 615 | + WRITE_ONCE(pool->tx_zc_pending_descs, | |
| 616 | + batch->tx_descs + cq_pending_descs); | |
| 617 | + WRITE_ONCE(pool->reclaim_descs, batch->reclaim_descs); | |
| 618 | + if (unlikely(!pool->tx_zc_pending_descs)) | |
| 619 | + xsk_tx_completed(pool, 0); | |
| 620 | + } | |
| 621 | +} | |
| 622 | + | |
| 623 | +static struct xsk_tx_batch | |
| 624 | +__xsk_tx_peek_release_desc_batch(struct xsk_buff_pool *pool, struct xdp_sock *xs, | |
| 625 | + struct xdp_desc *descs, u32 max_descs) | |
| 626 | +{ | |
| 627 | + struct xsk_tx_batch batch = {}; | |
| 628 | + u32 entries; | |
| 629 | + | |
| 630 | + entries = xskq_cons_nb_entries(xs->tx, max_descs); | |
| 631 | + if (!entries) | |
| 632 | + return batch; | |
| 633 | + | |
| 634 | + batch = xskq_cons_read_desc_batch(xs, pool, descs, max_descs); | |
| 635 | + if (!xsk_tx_batch_cq_descs(&batch)) { | |
| 636 | + xs->tx->queue_empty_descs++; | |
| 637 | + } else { | |
| 638 | + __xskq_cons_release(xs->tx); | |
| 639 | + xs->sk.sk_write_space(&xs->sk); | |
| 640 | + } | |
| 641 | + return batch; | |
| 642 | +} | |
| 643 | + | |
| 644 | +static struct xsk_tx_batch | |
| 645 | +xsk_tx_peek_release_shared_desc_batch(struct xsk_buff_pool *pool, u32 max_descs) | |
| 646 | +{ | |
| 647 | + u32 cq_descs_before, cq_descs_after; | |
| 648 | + struct xsk_tx_batch sum_batch = {}; | |
| 649 | + bool budget_exhausted; | |
| 650 | + u32 per_socket_budget; | |
| 651 | + struct xdp_sock *xs; | |
| 652 | + | |
| 653 | + /* The fairness quota must allow one maximum-sized valid packet. */ | |
| 654 | + per_socket_budget = max_t(u32, MAX_PER_SOCKET_BUDGET, | |
| 655 | + pool->xdp_zc_max_segs); | |
| 656 | + | |
| 657 | +again: | |
| 658 | + budget_exhausted = false; | |
| 659 | + cq_descs_before = xsk_tx_batch_cq_descs(&sum_batch); | |
| 660 | + list_for_each_entry_rcu(xs, &pool->xsk_tx_list, tx_list) { | |
| 661 | + u32 budget, budget_left, offset, remaining, used; | |
| 662 | + struct xsk_tx_batch curr_batch; | |
| 663 | + | |
| 664 | + /* Once reclaim-only descriptors have been appended to the CQ | |
| 665 | + * address area, do not append driver-visible Tx descriptors | |
| 666 | + * from another socket after them. xsk_tx_completed() relies on | |
| 667 | + * all driver-visible descriptors preceding all reclaim-only | |
| 668 | + * descriptors in CQ order. | |
| 669 | + */ | |
| 670 | + if (sum_batch.reclaim_descs) | |
| 671 | + break; | |
| 672 | + | |
| 673 | + /* be gentle when playing with pool->tx_descs */ | |
| 674 | + offset = xsk_tx_batch_cq_descs(&sum_batch); | |
| 675 | + if (offset >= max_descs) | |
| 676 | + break; | |
| 677 | + | |
| 678 | + if (xs->tx_budget_spent >= per_socket_budget) { | |
| 679 | + if (xskq_cons_nb_entries(xs->tx, 1)) | |
| 680 | + budget_exhausted = true; | |
| 681 | + continue; | |
| 682 | + } | |
| 683 | + | |
| 684 | + budget_left = per_socket_budget - xs->tx_budget_spent; | |
| 685 | + remaining = max_descs - offset; | |
| 686 | + budget = min(remaining, budget_left); | |
| 687 | + | |
| 688 | + curr_batch = __xsk_tx_peek_release_desc_batch(pool, xs, | |
| 689 | + pool->tx_descs + offset, | |
| 690 | + budget); | |
| 691 | + used = xsk_tx_batch_cq_descs(&curr_batch); | |
| 692 | + if (!used) { | |
| 693 | + if (curr_batch.budget_limited && budget_left < remaining) | |
| 694 | + budget_exhausted = true; | |
| 695 | + continue; | |
| 696 | + } | |
| 697 | + | |
| 698 | + xs->tx_budget_spent += used; | |
| 699 | + sum_batch.tx_descs += curr_batch.tx_descs; | |
| 700 | + sum_batch.reclaim_descs = curr_batch.reclaim_descs; | |
| 701 | + } | |
| 702 | + | |
| 703 | + cq_descs_after = xsk_tx_batch_cq_descs(&sum_batch); | |
| 704 | + | |
| 705 | + if (sum_batch.reclaim_descs || cq_descs_after >= max_descs) | |
| 706 | + return sum_batch; | |
| 707 | + | |
| 708 | + /* Continue filling the batch while this pass made progress */ | |
| 709 | + if (cq_descs_before != cq_descs_after) | |
| 710 | + goto again; | |
| 711 | + | |
| 712 | + if (!budget_exhausted) | |
| 713 | + return sum_batch; | |
| 714 | + | |
| 715 | + list_for_each_entry_rcu(xs, &pool->xsk_tx_list, tx_list) | |
| 716 | + xs->tx_budget_spent = 0; | |
| 717 | + goto again; | |
| 718 | +} | |
| 719 | + | |
| 577 | 720 | u32 xsk_tx_peek_release_desc_batch(struct xsk_buff_pool *pool, u32 nb_pkts) |
| 578 | 721 | { |
| 722 | + struct xsk_tx_batch batch = {}; | |
| 579 | 723 | struct xdp_sock *xs; |
| 724 | + bool umem_shared; | |
| 580 | 725 | |
| 581 | 726 | rcu_read_lock(); |
| 582 | − if (!list_is_singular(&pool->xsk_tx_list)) { | |
| 583 | − /* Fallback to the non-batched version */ | |
| 584 | − rcu_read_unlock(); | |
| 585 | − return xsk_tx_peek_release_fallback(pool, nb_pkts); | |
| 586 | − } | |
| 727 | + if (unlikely(READ_ONCE(pool->reclaim_descs))) | |
| 728 | + goto out; | |
| 587 | 729 | |
| 588 | − xs = list_first_or_null_rcu(&pool->xsk_tx_list, struct xdp_sock, tx_list); | |
| 589 | − if (!xs) { | |
| 590 | − nb_pkts = 0; | |
| 730 | + xs = list_first_or_null_rcu(&pool->xsk_tx_list, struct xdp_sock, | |
| 731 | + tx_list); | |
| 732 | + if (!xs) | |
| 591 | 733 | goto out; |
| 592 | − } | |
| 593 | 734 | |
| 594 | − nb_pkts = xskq_cons_nb_entries(xs->tx, nb_pkts); | |
| 735 | + nb_pkts = min(nb_pkts, pool->tx_descs_nentries); | |
| 736 | + if (!nb_pkts) | |
| 737 | + goto out; | |
| 738 | + | |
| 739 | + umem_shared = !list_is_singular(&pool->xsk_tx_list); | |
| 740 | + | |
| 741 | + if (umem_shared && !(pool->umem->flags & XDP_UMEM_SG_FLAG)) { | |
| 742 | + rcu_read_unlock(); | |
| 743 | + return xsk_tx_peek_release_fallback(pool, nb_pkts); | |
| 744 | + } | |
| 595 | 745 | |
| 596 | 746 | /* This is the backpressure mechanism for the Tx path. Try to |
| 597 | 747 | * reserve space in the completion queue for all packets, but |
| @@ -603,19 +753,16 @@ u32 xsk_tx_peek_release_desc_batch(struct xsk_buff_pool *pool, u32 nb_pkts) · 5 unmodified lines | ||
| 603 | 753 | if (!nb_pkts) |
| 604 | 754 | goto out; |
| 605 | 755 | |
| 606 | − nb_pkts = xskq_cons_read_desc_batch(xs->tx, pool, nb_pkts); | |
| 607 | − if (!nb_pkts) { | |
| 608 | − xs->tx->queue_empty_descs++; | |
| 609 | − goto out; | |
| 610 | − } | |
| 611 | − | |
| 612 | − __xskq_cons_release(xs->tx); | |
| 613 | − xskq_prod_write_addr_batch(pool->cq, pool->tx_descs, nb_pkts); | |
| 614 | − xs->sk.sk_write_space(&xs->sk); | |
| 756 | + batch = umem_shared ? | |
| 757 | + xsk_tx_peek_release_shared_desc_batch(pool, nb_pkts) : | |
| 758 | + __xsk_tx_peek_release_desc_batch(pool, xs, | |
| 759 | + pool->tx_descs, | |
| 760 | + nb_pkts); | |
| 761 | + xsk_tx_commit_batch(pool, &batch); | |
| 615 | 762 | |
| 616 | 763 | out: |
| 617 | 764 | rcu_read_unlock(); |
| 618 | − return nb_pkts; | |
| 765 | + return batch.tx_descs; | |
| 619 | 766 | } |
| 620 | 767 | EXPORT_SYMBOL(xsk_tx_peek_release_desc_batch); |
| 621 | 768 | |
| @@ -737,6 +884,19 @@ static void xsk_cq_submit_addr_locked(struct xsk_buff_pool *pool, · 115 unmodified lines | ||
| 737 | 884 | spin_unlock_irqrestore(&pool->cq_prod_lock, flags); |
| 738 | 885 | } |
| 739 | 886 | |
| 887 | +static void xsk_cq_submit_addr_single_locked(struct xsk_buff_pool *pool, | |
| 888 | + struct xdp_desc *desc) | |
| 889 | +{ | |
| 890 | + unsigned long flags; | |
| 891 | + u32 idx; | |
| 892 | + | |
| 893 | + spin_lock_irqsave(&pool->cq_prod_lock, flags); | |
| 894 | + idx = xskq_get_prod(pool->cq); | |
| 895 | + xskq_prod_write_addr(pool->cq, idx, desc->addr); | |
| 896 | + xskq_prod_submit_n(pool->cq, 1); | |
| 897 | + spin_unlock_irqrestore(&pool->cq_prod_lock, flags); | |
| 898 | +} | |
| 899 | + | |
| 740 | 900 | static void xsk_cq_cancel_locked(struct xsk_buff_pool *pool, u32 n) |
| 741 | 901 | { |
| 742 | 902 | spin_lock(&pool->cq->cq_cached_prod_lock); |
| @@ -794,8 +954,11 @@ static void xsk_consume_skb(struct sk_buff *skb) · 51 unmodified lines | ||
| 794 | 954 | |
| 795 | 955 | static void xsk_drop_skb(struct sk_buff *skb) |
| 796 | 956 | { |
| 797 | − xdp_sk(skb->sk)->tx->invalid_descs += xsk_get_num_desc(skb); | |
| 798 | − xsk_consume_skb(skb); | |
| 957 | + struct xdp_sock *xs = xdp_sk(skb->sk); | |
| 958 | + | |
| 959 | + xs->tx->invalid_descs += xsk_get_num_desc(skb); | |
| 960 | + consume_skb(skb); | |
| 961 | + xs->skb = NULL; | |
| 799 | 962 | } |
| 800 | 963 | |
| 801 | 964 | static int xsk_skb_metadata(struct sk_buff *skb, void *buffer, |
| @@ -877,7 +1040,7 @@ static struct sk_buff *xsk_build_skb_zerocopy(struct xdp_sock *xs, · 75 unmodified lines | ||
| 877 | 1040 | return ERR_PTR(-ENOMEM); |
| 878 | 1041 | |
| 879 | 1042 | /* in case of -EOVERFLOW that could happen below, |
| 880 | − * xsk_consume_skb() will release this node as whole skb | |
| 1043 | + * xsk_drop_skb() will release this node as whole skb | |
| 881 | 1044 | * would be dropped, which implies freeing all list elements |
| 882 | 1045 | */ |
| 883 | 1046 | xsk_addr->addrs[xsk_addr->num_descs] = desc->addr; |
| @@ -969,6 +1132,8 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs, · 85 unmodified lines | ||
| 969 | 1132 | goto free_err; |
| 970 | 1133 | } |
| 971 | 1134 | |
| 1135 | + xsk_addr->addrs[xsk_addr->num_descs] = desc->addr; | |
| 1136 | + | |
| 972 | 1137 | if (unlikely(nr_frags == (MAX_SKB_FRAGS - 1) && xp_mb_desc(desc))) { |
| 973 | 1138 | err = -EOVERFLOW; |
| 974 | 1139 | goto free_err; |
| @@ -986,8 +1151,6 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs, · 11 unmodified lines | ||
| 986 | 1151 | |
| 987 | 1152 | skb_add_rx_frag(skb, nr_frags, page, 0, len, PAGE_SIZE); |
| 988 | 1153 | refcount_add(PAGE_SIZE, &xs->sk.sk_wmem_alloc); |
| 989 | − | |
| 990 | − xsk_addr->addrs[xsk_addr->num_descs] = desc->addr; | |
| 991 | 1154 | } |
| 992 | 1155 | } |
| 993 | 1156 | |
| @@ -1025,13 +1188,14 @@ free_err: · 31 unmodified lines | ||
| 1025 | 1188 | static int __xsk_generic_xmit(struct sock *sk) |
| 1026 | 1189 | { |
| 1027 | 1190 | struct xdp_sock *xs = xdp_sk(sk); |
| 1028 | − bool sent_frame = false; | |
| 1029 | 1191 | struct xdp_desc desc; |
| 1030 | 1192 | struct sk_buff *skb; |
| 1193 | + u32 cached_cons; | |
| 1031 | 1194 | u32 max_batch; |
| 1032 | 1195 | int err = 0; |
| 1033 | 1196 | |
| 1034 | 1197 | mutex_lock(&xs->mutex); |
| 1198 | + cached_cons = xs->tx->cached_cons; | |
| 1035 | 1199 | |
| 1036 | 1200 | /* Since we dropped the RCU read lock, the socket state might have changed. */ |
| 1037 | 1201 | if (unlikely(!xsk_is_bound(xs))) { |
| @@ -1060,11 +1224,21 @@ static int __xsk_generic_xmit(struct sock *sk) · 22 unmodified lines | ||
| 1060 | 1224 | goto out; |
| 1061 | 1225 | } |
| 1062 | 1226 | |
| 1227 | + if (unlikely(xs->drain_cont)) { | |
| 1228 | + xsk_cq_submit_addr_single_locked(xs->pool, &desc); | |
| 1229 | + xs->tx->invalid_descs++; | |
| 1230 | + xskq_cons_release(xs->tx); | |
| 1231 | + xs->drain_cont = xp_mb_desc(&desc); | |
| 1232 | + continue; | |
| 1233 | + } | |
| 1234 | + | |
| 1063 | 1235 | skb = xsk_build_skb(xs, &desc); |
| 1064 | 1236 | if (IS_ERR(skb)) { |
| 1065 | 1237 | err = PTR_ERR(skb); |
| 1066 | 1238 | if (err != -EOVERFLOW) |
| 1067 | 1239 | goto out; |
| 1240 | + if (xp_mb_desc(&desc)) | |
| 1241 | + xs->drain_cont = true; | |
| 1068 | 1242 | err = 0; |
| 1069 | 1243 | continue; |
| 1070 | 1244 | } |
| @@ -1093,18 +1267,33 @@ static int __xsk_generic_xmit(struct sock *sk) · 22 unmodified lines | ||
| 1093 | 1267 | goto out; |
| 1094 | 1268 | } |
| 1095 | 1269 | |
| 1096 | − sent_frame = true; | |
| 1097 | 1270 | xs->skb = NULL; |
| 1098 | 1271 | } |
| 1099 | 1272 | |
| 1100 | 1273 | if (xskq_has_descs(xs->tx)) { |
| 1274 | + bool drain = xs->skb || xs->drain_cont || xp_mb_desc(&desc); | |
| 1275 | + | |
| 1276 | + err = xsk_cq_reserve_locked(xs->pool); | |
| 1277 | + if (err) { | |
| 1278 | + xs->tx->invalid_descs--; | |
| 1279 | + if (xs->skb) | |
| 1280 | + xsk_drop_skb(xs->skb); | |
| 1281 | + xs->drain_cont = drain; | |
| 1282 | + err = -EAGAIN; | |
| 1283 | + goto out; | |
| 1284 | + } | |
| 1285 | + | |
| 1101 | 1286 | if (xs->skb) |
| 1102 | 1287 | xsk_drop_skb(xs->skb); |
| 1288 | + | |
| 1289 | + xsk_cq_submit_addr_single_locked(xs->pool, &desc); | |
| 1290 | + | |
| 1103 | 1291 | xskq_cons_release(xs->tx); |
| 1292 | + xs->drain_cont = xp_mb_desc(&desc); | |
| 1104 | 1293 | } |
| 1105 | 1294 | |
| 1106 | 1295 | out: |
| 1107 | − if (sent_frame) | |
| 1296 | + if (xs->tx->cached_cons != cached_cons) | |
| 1108 | 1297 | __xsk_tx_release(xs); |
| 1109 | 1298 | |
| 1110 | 1299 | mutex_unlock(&xs->mutex); |
| @@ -1483,7 +1672,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr_unsized *addr, int addr · 372 unmodified lines | ||
| 1483 | 1672 | * and/or device. |
| 1484 | 1673 | */ |
| 1485 | 1674 | xs->pool = xp_create_and_assign_umem(xs, |
| 1486 | − umem_xs->umem); | |
| 1675 | + umem_xs->umem, | |
| 1676 | + dev->xdp_zc_max_segs); | |
| 1487 | 1677 | if (!xs->pool) { |
| 1488 | 1678 | err = -ENOMEM; |
| 1489 | 1679 | sockfd_put(sock); |
| @@ -1515,7 +1705,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr_unsized *addr, int addr · 25 unmodified lines | ||
| 1515 | 1705 | * utilizes |
| 1516 | 1706 | */ |
| 1517 | 1707 | if (xs->tx && !xs->pool->tx_descs) { |
| 1518 | − err = xp_alloc_tx_descs(xs->pool, xs); | |
| 1708 | + err = xp_alloc_tx_descs(xs->pool, xs, | |
| 1709 | + dev->xdp_zc_max_segs); | |
| 1519 | 1710 | if (err) { |
| 1520 | 1711 | xp_put_pool(xs->pool); |
| 1521 | 1712 | xs->pool = NULL; |
| @@ -1533,7 +1724,9 @@ static int xsk_bind(struct socket *sock, struct sockaddr_unsized *addr, int addr · 11 unmodified lines | ||
| 1533 | 1724 | goto out_unlock; |
| 1534 | 1725 | } else { |
| 1535 | 1726 | /* This xsk has its own umem. */ |
| 1536 | − xs->pool = xp_create_and_assign_umem(xs, xs->umem); | |
| 1727 | + xs->pool = xp_create_and_assign_umem(xs, xs->umem, | |
| 1728 | + dev->xdp_zc_max_segs); | |
| 1729 | + | |
| 1537 | 1730 | if (!xs->pool) { |
| 1538 | 1731 | err = -ENOMEM; |
| 1539 | 1732 | goto out_unlock; |
| @@ -42,17 +42,22 @@ void xp_destroy(struct xsk_buff_pool *pool) · 41 unmodified lines | ||
| 42 | 42 | kvfree(pool); |
| 43 | 43 | } |
| 44 | 44 | |
| 45 | −int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs) | |
| 45 | +int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs, | |
| 46 | + u32 max_segs) | |
| 46 | 47 | { |
| 47 | − pool->tx_descs = kvzalloc_objs(*pool->tx_descs, xs->tx->nentries); | |
| 48 | + u32 nentries = max(xs->tx->nentries, max_segs); | |
| 49 | + | |
| 50 | + pool->tx_descs = kvzalloc_objs(*pool->tx_descs, nentries); | |
| 48 | 51 | if (!pool->tx_descs) |
| 49 | 52 | return -ENOMEM; |
| 50 | 53 | |
| 54 | + pool->tx_descs_nentries = nentries; | |
| 51 | 55 | return 0; |
| 52 | 56 | } |
| 53 | 57 | |
| 54 | 58 | struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, |
| 55 | − struct xdp_umem *umem) | |
| 59 | + struct xdp_umem *umem, | |
| 60 | + u32 max_segs) | |
| 56 | 61 | { |
| 57 | 62 | bool unaligned = umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG; |
| 58 | 63 | struct xsk_buff_pool *pool; |
| @@ -69,7 +74,7 @@ struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, · 10 unmodified lines | ||
| 69 | 74 | goto out; |
| 70 | 75 | |
| 71 | 76 | if (xs->tx) |
| 72 | − if (xp_alloc_tx_descs(pool, xs)) | |
| 77 | + if (xp_alloc_tx_descs(pool, xs, max_segs)) | |
| 73 | 78 | goto out; |
| 74 | 79 | |
| 75 | 80 | pool->chunk_mask = ~((u64)umem->chunk_size - 1); |
| @@ -58,6 +58,17 @@ struct parsed_desc { · 57 unmodified lines | ||
| 58 | 58 | u32 valid; |
| 59 | 59 | }; |
| 60 | 60 | |
| 61 | +struct xsk_tx_batch { | |
| 62 | + u32 tx_descs; | |
| 63 | + u32 reclaim_descs; | |
| 64 | + bool budget_limited; | |
| 65 | +}; | |
| 66 | + | |
| 67 | +static inline u32 xsk_tx_batch_cq_descs(const struct xsk_tx_batch *batch) | |
| 68 | +{ | |
| 69 | + return batch->tx_descs + batch->reclaim_descs; | |
| 70 | +} | |
| 71 | + | |
| 61 | 72 | /* The structure of the shared state of the rings are a simple |
| 62 | 73 | * circular buffer, as outlined in |
| 63 | 74 | * Documentation/core-api/circular-buffers.rst. For the Rx and |
| @@ -263,17 +274,18 @@ static inline void parse_desc(struct xsk_queue *q, struct xsk_buff_pool *pool, · 199 unmodified lines | ||
| 263 | 274 | parsed->mb = xp_mb_desc(desc); |
| 264 | 275 | } |
| 265 | 276 | |
| 266 | −static inline | |
| 267 | −u32 xskq_cons_read_desc_batch(struct xsk_queue *q, struct xsk_buff_pool *pool, | |
| 268 | − u32 max) | |
| 277 | +static inline struct xsk_tx_batch | |
| 278 | +xskq_cons_read_desc_batch(struct xdp_sock *xs, struct xsk_buff_pool *pool, | |
| 279 | + struct xdp_desc *descs, u32 max) | |
| 269 | 280 | { |
| 270 | − u32 cached_cons = q->cached_cons, nb_entries = 0; | |
| 271 | − struct xdp_desc *descs = pool->tx_descs; | |
| 272 | − u32 total_descs = 0, nr_frags = 0; | |
| 281 | + bool drain = READ_ONCE(xs->drain_cont); | |
| 282 | + u32 cached_cons, nb_entries = 0; | |
| 283 | + struct xsk_tx_batch batch = {}; | |
| 284 | + struct xsk_queue *q = xs->tx; | |
| 285 | + u32 nr_frags = 0; | |
| 286 | + | |
| 287 | + cached_cons = q->cached_cons; | |
| 273 | 288 | |
| 274 | − /* track first entry, if stumble upon *any* invalid descriptor, rewind | |
| 275 | − * current packet that consists of frags and stop the processing | |
| 276 | − */ | |
| 277 | 289 | while (cached_cons != q->cached_prod && nb_entries < max) { |
| 278 | 290 | struct xdp_rxtx_ring *ring = (struct xdp_rxtx_ring *)q->ring; |
| 279 | 291 | u32 idx = cached_cons & q->ring_mask; |
| @@ -283,25 +295,42 @@ u32 xskq_cons_read_desc_batch(struct xsk_queue *q, struct xsk_buff_pool *pool, · 3 unmodified lines | ||
| 283 | 295 | cached_cons++; |
| 284 | 296 | parse_desc(q, pool, &descs[nb_entries], &parsed); |
| 285 | 297 | if (unlikely(!parsed.valid)) |
| 286 | − break; | |
| 298 | + drain = true; | |
| 299 | + | |
| 300 | + nr_frags++; | |
| 301 | + nb_entries++; | |
| 287 | 302 | |
| 288 | 303 | if (likely(!parsed.mb)) { |
| 289 | − total_descs += (nr_frags + 1); | |
| 290 | − nr_frags = 0; | |
| 291 | − } else { | |
| 292 | − nr_frags++; | |
| 293 | − if (nr_frags == pool->xdp_zc_max_segs) { | |
| 304 | + if (unlikely(drain)) { | |
| 305 | + batch.reclaim_descs = nr_frags; | |
| 306 | + WRITE_ONCE(xs->drain_cont, false); | |
| 294 | 307 | nr_frags = 0; |
| 295 | 308 | break; |
| 296 | 309 | } |
| 310 | + | |
| 311 | + batch.tx_descs += nr_frags; | |
| 312 | + nr_frags = 0; | |
| 313 | + continue; | |
| 314 | + } | |
| 315 | + | |
| 316 | + if (nr_frags == pool->xdp_zc_max_segs) | |
| 317 | + drain = true; | |
| 318 | + } | |
| 319 | + | |
| 320 | + if (nr_frags) { | |
| 321 | + if (drain) { | |
| 322 | + batch.reclaim_descs = nr_frags; | |
| 323 | + WRITE_ONCE(xs->drain_cont, true); | |
| 324 | + } else { | |
| 325 | + if (nb_entries == max) | |
| 326 | + batch.budget_limited = true; | |
| 327 | + cached_cons -= nr_frags; | |
| 297 | 328 | } |
| 298 | − nb_entries++; | |
| 299 | 329 | } |
| 300 | 330 | |
| 301 | − cached_cons -= nr_frags; | |
| 302 | 331 | /* Release valid plus any invalid entries */ |
| 303 | 332 | xskq_cons_release_n(q, cached_cons - q->cached_cons); |
| 304 | − return total_descs; | |
| 333 | + return batch; | |
| 305 | 334 | } |
| 306 | 335 | |
| 307 | 336 | /* Functions for consumers */ |
| @@ -427,14 +427,14 @@ static u32 pkt_nb_frags(u32 frame_size, struct pkt_stream *pkt_stream, struct pk · 426 unmodified lines | ||
| 427 | 427 | } |
| 428 | 428 | |
| 429 | 429 | /* Search for the end of the packet in verbatim mode */ |
| 430 | − if (!pkt_continues(pkt->options) || !pkt->valid) | |
| 430 | + if (!pkt_continues(pkt->options)) | |
| 431 | 431 | return nb_frags; |
| 432 | 432 | |
| 433 | 433 | next_frag = pkt_stream->current_pkt_nb; |
| 434 | 434 | pkt++; |
| 435 | 435 | while (next_frag++ < pkt_stream->nb_pkts) { |
| 436 | 436 | nb_frags++; |
| 437 | − if (!pkt_continues(pkt->options) || !pkt->valid) | |
| 437 | + if (!pkt_continues(pkt->options)) | |
| 438 | 438 | break; |
| 439 | 439 | pkt++; |
| 440 | 440 | } |
| @@ -665,11 +665,11 @@ static struct pkt_stream *__pkt_stream_generate_custom(struct ifobject *ifobj, s · 224 unmodified lines | ||
| 665 | 665 | if (!frame->valid || !pkt_continues(frame->options)) |
| 666 | 666 | payload++; |
| 667 | 667 | } else { |
| 668 | − if (frame->valid) | |
| 668 | + if (frame->valid) { | |
| 669 | 669 | len += frame->len; |
| 670 | − if (frame->valid && pkt_continues(frame->options)) | |
| 671 | − continue; | |
| 672 | − | |
| 670 | + if (pkt_continues(frame->options)) | |
| 671 | + continue; | |
| 672 | + } | |
| 673 | 673 | pkt->pkt_nb = pkt_nb; |
| 674 | 674 | pkt->len = len; |
| 675 | 675 | pkt->valid = frame->valid; |
| @@ -1250,10 +1250,9 @@ static int __send_pkts(struct ifobject *ifobject, struct xsk_socket_info *xsk, · 574 unmodified lines | ||
| 1250 | 1250 | } |
| 1251 | 1251 | } |
| 1252 | 1252 | |
| 1253 | − if (pkt && pkt->valid) { | |
| 1253 | + if (pkt && pkt->valid) | |
| 1254 | 1254 | valid_pkts++; |
| 1255 | − valid_frags += nb_frags; | |
| 1256 | − } | |
| 1255 | + valid_frags += nb_frags; | |
| 1257 | 1256 | } |
| 1258 | 1257 | |
| 1259 | 1258 | pthread_mutex_lock(&pacing_mutex); |
| @@ -2099,13 +2098,16 @@ int testapp_invalid_desc_mb(struct test_spec *test) · 839 unmodified lines | ||
| 2099 | 2098 | {0, 0, 0, false, 0}, |
| 2100 | 2099 | /* Invalid address in the second frame */ |
| 2101 | 2100 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2102 | − {umem_sz, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, | |
| 2101 | + {umem_sz * 2, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, | |
| 2102 | + {0, MIN_PKT_SIZE, 0, false, 0}, | |
| 2103 | 2103 | /* Invalid len in the middle */ |
| 2104 | 2104 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2105 | 2105 | {0, XSK_UMEM__INVALID_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2106 | + {0, MIN_PKT_SIZE, 0, false, 0}, | |
| 2106 | 2107 | /* Invalid options in the middle */ |
| 2107 | 2108 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2108 | 2109 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XSK_DESC__INVALID_OPTION}, |
| 2110 | + {0, MIN_PKT_SIZE, 0, false, 0}, | |
| 2109 | 2111 | /* Transmit 2 frags, receive 3 */ |
| 2110 | 2112 | {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, XDP_PKT_CONTD}, |
| 2111 | 2113 | {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, 0}, |
| @@ -2117,8 +2119,8 @@ int testapp_invalid_desc_mb(struct test_spec *test) · 5 unmodified lines | ||
| 2117 | 2119 | |
| 2118 | 2120 | if (umem->unaligned_mode) { |
| 2119 | 2121 | /* Crossing a chunk boundary allowed */ |
| 2120 | − pkts[12].valid = true; | |
| 2121 | − pkts[13].valid = true; | |
| 2122 | + pkts[15].valid = true; | |
| 2123 | + pkts[16].valid = true; | |
| 2122 | 2124 | } |
| 2123 | 2125 | |
| 2124 | 2126 | test->mtu = MAX_ETH_JUMBO_SIZE; |
| @@ -2270,7 +2272,7 @@ int testapp_too_many_frags(struct test_spec *test) · 145 unmodified lines | ||
| 2270 | 2272 | max_frags += 1; |
| 2271 | 2273 | } |
| 2272 | 2274 | |
| 2273 | − pkts = calloc(2 * max_frags + 2, sizeof(struct pkt)); | |
| 2275 | + pkts = calloc(2 * max_frags + 3, sizeof(struct pkt)); | |
| 2274 | 2276 | if (!pkts) |
| 2275 | 2277 | return TEST_FAILURE; |
| 2276 | 2278 | |
| @@ -2288,24 +2290,30 @@ int testapp_too_many_frags(struct test_spec *test) · 11 unmodified lines | ||
| 2288 | 2290 | } |
| 2289 | 2291 | pkts[max_frags].options = 0; |
| 2290 | 2292 | |
| 2291 | − /* An invalid packet with the max amount of frags but signals packet | |
| 2292 | − * continues on the last frag | |
| 2293 | − */ | |
| 2294 | − for (i = max_frags + 1; i < 2 * max_frags + 1; i++) { | |
| 2293 | + /* An invalid packet with the max + 1 amount of frags */ | |
| 2294 | + for (i = max_frags + 1; i < 2 * max_frags + 2; i++) { | |
| 2295 | 2295 | pkts[i].len = MIN_PKT_SIZE; |
| 2296 | 2296 | pkts[i].options = XDP_PKT_CONTD; |
| 2297 | − pkts[i].valid = false; | |
| 2297 | + pkts[i].valid = true; | |
| 2298 | 2298 | } |
| 2299 | + pkts[2 * max_frags + 1].options = 0; | |
| 2299 | 2300 | |
| 2300 | 2301 | /* Valid packet for synch */ |
| 2301 | − pkts[2 * max_frags + 1].len = MIN_PKT_SIZE; | |
| 2302 | − pkts[2 * max_frags + 1].valid = true; | |
| 2302 | + pkts[2 * max_frags + 2].len = MIN_PKT_SIZE; | |
| 2303 | + pkts[2 * max_frags + 2].valid = true; | |
| 2303 | 2304 | |
| 2304 | − if (pkt_stream_generate_custom(test, pkts, 2 * max_frags + 2)) { | |
| 2305 | + if (pkt_stream_generate_custom(test, pkts, 2 * max_frags + 3)) { | |
| 2305 | 2306 | free(pkts); |
| 2306 | 2307 | return TEST_FAILURE; |
| 2307 | 2308 | } |
| 2308 | 2309 | |
| 2310 | + /* The generated Tx stream must keep the too-big packet valid so that | |
| 2311 | + * __send_pkts() accounts its descriptors in outstanding_tx. The Rx | |
| 2312 | + * stream, however, must not expect this packet on the wire. | |
| 2313 | + */ | |
| 2314 | + test->ifobj_rx->xsk->pkt_stream->pkts[2].valid = false; | |
| 2315 | + test->ifobj_rx->xsk->pkt_stream->nb_valid_entries--; | |
| 2316 | + | |
| 2309 | 2317 | ret = testapp_validate_traffic(test); |
| 2310 | 2318 | free(pkts); |
| 2311 | 2319 | return ret; |
| @@ -6,3 +6,4 @@ scm_rights · 5 unmodified lines | ||
| 6 | 6 | so_peek_off |
| 7 | 7 | unix_connect |
| 8 | 8 | unix_connreset |
| 9 | +unix_listen | |
| @@ -14,6 +14,7 @@ TEST_GEN_PROGS := \ · 13 unmodified lines | ||
| 14 | 14 | so_peek_off \ |
| 15 | 15 | unix_connect \ |
| 16 | 16 | unix_connreset \ |
| 17 | + unix_listen \ | |
| 17 | 18 | # end of TEST_GEN_PROGS |
| 18 | 19 | |
| 19 | 20 | include ../../lib.mk |
| @@ -0,0 +1,187 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0 | |
| 2 | +/* | |
| 3 | + * Tests for the state checks in AF_UNIX listen(). | |
| 4 | + * | |
| 5 | + * The central case is a regression test: listen() on a bound socket that | |
| 6 | + * is already connected (i.e. not in TCP_CLOSE or TCP_LISTEN state) must | |
| 7 | + * fail with EINVAL. A prior change accidentally let it return success | |
| 8 | + * without doing anything, because a helper called in between reset the | |
| 9 | + * error code to 0. The neighbouring checks (unbound, already listening) | |
| 10 | + * are tested too so they cannot silently regress the same way. | |
| 11 | + * | |
| 12 | + * Every case runs for both listenable socket types (SOCK_STREAM and | |
| 13 | + * SOCK_SEQPACKET) and both pathname and abstract addresses. | |
| 14 | + */ | |
| 15 | +#define _GNU_SOURCE | |
| 16 | + | |
| 17 | +#include <errno.h> | |
| 18 | +#include <stddef.h> | |
| 19 | +#include <stdio.h> | |
| 20 | +#include <string.h> | |
| 21 | +#include <unistd.h> | |
| 22 | + | |
| 23 | +#include <sys/socket.h> | |
| 24 | +#include <sys/un.h> | |
| 25 | + | |
| 26 | +#include "kselftest_harness.h" | |
| 27 | + | |
| 28 | +#define SK_NAME "unix_listen_sk" | |
| 29 | +#define SRV_NAME "unix_listen_srv" | |
| 30 | + | |
| 31 | +FIXTURE(unix_listen) | |
| 32 | +{ | |
| 33 | + int sk; /* socket under test */ | |
| 34 | + int server; /* a listening peer, when a test needs one */ | |
| 35 | + struct sockaddr_un addr, srv_addr; | |
| 36 | + socklen_t addrlen, srv_addrlen; | |
| 37 | +}; | |
| 38 | + | |
| 39 | +FIXTURE_VARIANT(unix_listen) | |
| 40 | +{ | |
| 41 | + int type; | |
| 42 | + int abstract; | |
| 43 | +}; | |
| 44 | + | |
| 45 | +FIXTURE_VARIANT_ADD(unix_listen, stream_pathname) | |
| 46 | +{ | |
| 47 | + .type = SOCK_STREAM, | |
| 48 | + .abstract = 0, | |
| 49 | +}; | |
| 50 | + | |
| 51 | +FIXTURE_VARIANT_ADD(unix_listen, stream_abstract) | |
| 52 | +{ | |
| 53 | + .type = SOCK_STREAM, | |
| 54 | + .abstract = 1, | |
| 55 | +}; | |
| 56 | + | |
| 57 | +FIXTURE_VARIANT_ADD(unix_listen, seqpacket_pathname) | |
| 58 | +{ | |
| 59 | + .type = SOCK_SEQPACKET, | |
| 60 | + .abstract = 0, | |
| 61 | +}; | |
| 62 | + | |
| 63 | +FIXTURE_VARIANT_ADD(unix_listen, seqpacket_abstract) | |
| 64 | +{ | |
| 65 | + .type = SOCK_SEQPACKET, | |
| 66 | + .abstract = 1, | |
| 67 | +}; | |
| 68 | + | |
| 69 | +/* Fill @addr with a pathname or abstract address named @name. */ | |
| 70 | +static socklen_t unix_set_addr(struct sockaddr_un *addr, const char *name, | |
| 71 | + int abstract) | |
| 72 | +{ | |
| 73 | + size_t len = strlen(name); | |
| 74 | + | |
| 75 | + memset(addr, 0, sizeof(*addr)); | |
| 76 | + addr->sun_family = AF_UNIX; | |
| 77 | + /* An abstract address leads with a NUL and has no filesystem entry. */ | |
| 78 | + memcpy(addr->sun_path + (abstract ? 1 : 0), name, len); | |
| 79 | + | |
| 80 | + return offsetof(struct sockaddr_un, sun_path) + len + 1; | |
| 81 | +} | |
| 82 | + | |
| 83 | +FIXTURE_SETUP(unix_listen) | |
| 84 | +{ | |
| 85 | + self->sk = -1; | |
| 86 | + self->server = -1; | |
| 87 | + self->addrlen = unix_set_addr(&self->addr, SK_NAME, variant->abstract); | |
| 88 | + self->srv_addrlen = unix_set_addr(&self->srv_addr, SRV_NAME, | |
| 89 | + variant->abstract); | |
| 90 | +} | |
| 91 | + | |
| 92 | +FIXTURE_TEARDOWN(unix_listen) | |
| 93 | +{ | |
| 94 | + if (self->sk >= 0) | |
| 95 | + close(self->sk); | |
| 96 | + if (self->server >= 0) | |
| 97 | + close(self->server); | |
| 98 | + | |
| 99 | + /* Pathname sockets leave a filesystem entry behind; abstract ones do not. */ | |
| 100 | + if (!variant->abstract) { | |
| 101 | + remove(SK_NAME); | |
| 102 | + remove(SRV_NAME); | |
| 103 | + } | |
| 104 | +} | |
| 105 | + | |
| 106 | +/* A bound socket in TCP_CLOSE is the normal, allowed case. */ | |
| 107 | +TEST_F(unix_listen, bound_is_ok) | |
| 108 | +{ | |
| 109 | + int err; | |
| 110 | + | |
| 111 | + self->sk = socket(AF_UNIX, variant->type, 0); | |
| 112 | + ASSERT_LE(0, self->sk); | |
| 113 | + | |
| 114 | + err = bind(self->sk, (struct sockaddr *)&self->addr, self->addrlen); | |
| 115 | + ASSERT_EQ(0, err); | |
| 116 | + | |
| 117 | + err = listen(self->sk, 8); | |
| 118 | + EXPECT_EQ(0, err); | |
| 119 | +} | |
| 120 | + | |
| 121 | +/* Listening again on an already-listening socket (TCP_LISTEN) is allowed. */ | |
| 122 | +TEST_F(unix_listen, relisten_is_ok) | |
| 123 | +{ | |
| 124 | + int err; | |
| 125 | + | |
| 126 | + self->sk = socket(AF_UNIX, variant->type, 0); | |
| 127 | + ASSERT_LE(0, self->sk); | |
| 128 | + | |
| 129 | + err = bind(self->sk, (struct sockaddr *)&self->addr, self->addrlen); | |
| 130 | + ASSERT_EQ(0, err); | |
| 131 | + | |
| 132 | + err = listen(self->sk, 8); | |
| 133 | + ASSERT_EQ(0, err); | |
| 134 | + | |
| 135 | + err = listen(self->sk, 16); | |
| 136 | + EXPECT_EQ(0, err); | |
| 137 | +} | |
| 138 | + | |
| 139 | +/* listen() on an unbound socket fails: there is nothing to listen on. */ | |
| 140 | +TEST_F(unix_listen, unbound_is_einval) | |
| 141 | +{ | |
| 142 | + int err; | |
| 143 | + | |
| 144 | + self->sk = socket(AF_UNIX, variant->type, 0); | |
| 145 | + ASSERT_LE(0, self->sk); | |
| 146 | + | |
| 147 | + err = listen(self->sk, 8); | |
| 148 | + EXPECT_EQ(-1, err); | |
| 149 | + EXPECT_EQ(EINVAL, errno); | |
| 150 | +} | |
| 151 | + | |
| 152 | +/* | |
| 153 | + * The regression: a bound socket that has already been connected is not in | |
| 154 | + * TCP_CLOSE or TCP_LISTEN, so listen() must reject it with EINVAL rather | |
| 155 | + * than quietly succeeding. | |
| 156 | + */ | |
| 157 | +TEST_F(unix_listen, connected_is_einval) | |
| 158 | +{ | |
| 159 | + int err; | |
| 160 | + | |
| 161 | + self->server = socket(AF_UNIX, variant->type, 0); | |
| 162 | + ASSERT_LE(0, self->server); | |
| 163 | + | |
| 164 | + err = bind(self->server, (struct sockaddr *)&self->srv_addr, | |
| 165 | + self->srv_addrlen); | |
| 166 | + ASSERT_EQ(0, err); | |
| 167 | + | |
| 168 | + err = listen(self->server, 8); | |
| 169 | + ASSERT_EQ(0, err); | |
| 170 | + | |
| 171 | + self->sk = socket(AF_UNIX, variant->type, 0); | |
| 172 | + ASSERT_LE(0, self->sk); | |
| 173 | + | |
| 174 | + /* Bind first so the unbound check does not mask the state check. */ | |
| 175 | + err = bind(self->sk, (struct sockaddr *)&self->addr, self->addrlen); | |
| 176 | + ASSERT_EQ(0, err); | |
| 177 | + | |
| 178 | + err = connect(self->sk, (struct sockaddr *)&self->srv_addr, | |
| 179 | + self->srv_addrlen); | |
| 180 | + ASSERT_EQ(0, err); | |
| 181 | + | |
| 182 | + err = listen(self->sk, 8); | |
| 183 | + EXPECT_EQ(-1, err); | |
| 184 | + EXPECT_EQ(EINVAL, errno); | |
| 185 | +} | |
| 186 | + | |
| 187 | +TEST_HARNESS_MAIN | |
| @@ -617,7 +617,11 @@ ip -6 -net "$nsr2" route add default via fee1:3::1 · 616 unmodified lines | ||
| 617 | 617 | ip -net "$ns2" route add default via 10.0.2.1 |
| 618 | 618 | ip -6 -net "$ns2" route add default via dead:2::1 |
| 619 | 619 | |
| 620 | +ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | |
| 621 | + 'meta oif tun0 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | |
| 620 | 622 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun0 accept' |
| 623 | +ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | |
| 624 | + 'meta oif tun6 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | |
| 621 | 625 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun6 accept' |
| 622 | 626 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ |
| 623 | 627 | 'meta oif "veth0" tcp sport 12345 ct mark set 1 flow add @f1 counter name routed_repl accept' |
| @@ -629,7 +633,7 @@ if ! test_tcp_forwarding_nat "$ns1" "$ns2" 1 "IPIP tunnel"; then · 5 unmodified lines | ||
| 629 | 633 | fi |
| 630 | 634 | |
| 631 | 635 | if test_tcp_forwarding "$ns1" "$ns2" 1 6 "[dead:2::99]" 12345; then |
| 632 | − echo "PASS: flow offload for ns1/ns2 IP6IP6 tunnel" | |
| 636 | + check_counters "flow offload for ns1/ns2 IP6IP6 tunnel" | |
| 633 | 637 | else |
| 634 | 638 | echo "FAIL: flow offload for ns1/ns2 with IP6IP6 tunnel" 1>&2 |
| 635 | 639 | ip netns exec "$nsr1" nft list ruleset |
| @@ -642,6 +646,8 @@ ip -net "$nsr1" link set veth1.10 up · 6 unmodified lines | ||
| 642 | 646 | ip -net "$nsr1" addr add 192.168.20.1/24 dev veth1.10 |
| 643 | 647 | ip -net "$nsr1" addr add fee1:4::1/64 dev veth1.10 nodad |
| 644 | 648 | ip netns exec "$nsr1" sysctl net.ipv4.conf.veth1/10.forwarding=1 > /dev/null |
| 649 | +ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | |
| 650 | + 'meta oif veth1.10 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | |
| 645 | 651 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif veth1.10 accept' |
| 646 | 652 | |
| 647 | 653 | ip -net "$nsr1" link add name tun0.10 type ipip local 192.168.20.1 remote 192.168.20.2 |
| @@ -649,6 +655,8 @@ ip -net "$nsr1" link set tun0.10 up · 1 unmodified line | ||
| 649 | 655 | ip -net "$nsr1" addr add 192.168.200.1/24 dev tun0.10 |
| 650 | 656 | ip -net "$nsr1" route change default via 192.168.200.2 |
| 651 | 657 | ip netns exec "$nsr1" sysctl net.ipv4.conf.tun0/10.forwarding=1 > /dev/null |
| 658 | +ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | |
| 659 | + 'meta oif tun0.10 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | |
| 652 | 660 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun0.10 accept' |
| 653 | 661 | |
| 654 | 662 | ip -net "$nsr1" link add name tun6.10 type ip6tnl local fee1:4::1 remote fee1:4::2 encaplimit none |
| @@ -656,6 +664,8 @@ ip -net "$nsr1" link set tun6.10 up · 1 unmodified line | ||
| 656 | 664 | ip -net "$nsr1" addr add fee1:5::1/64 dev tun6.10 nodad |
| 657 | 665 | ip -6 -net "$nsr1" route delete default |
| 658 | 666 | ip -6 -net "$nsr1" route add default via fee1:5::2 |
| 667 | +ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | |
| 668 | + 'meta oif tun6.10 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | |
| 659 | 669 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun6.10 accept' |
| 660 | 670 | |
| 661 | 671 | ip -net "$nsr2" link add link veth0 name veth0.10 type vlan id 10 |
| @@ -683,7 +693,7 @@ if ! test_tcp_forwarding_nat "$ns1" "$ns2" 1 "IPIP tunnel over vlan"; then · 21 unmodified lines | ||
| 683 | 693 | fi |
| 684 | 694 | |
| 685 | 695 | if test_tcp_forwarding "$ns1" "$ns2" 1 6 "[dead:2::99]" 12345; then |
| 686 | − echo "PASS: flow offload for ns1/ns2 IP6IP6 tunnel over vlan" | |
| 696 | + check_counters "flow offload for ns1/ns2 IP6IP6 tunnel over vlan" | |
| 687 | 697 | else |
| 688 | 698 | echo "FAIL: flow offload for ns1/ns2 with IP6IP6 tunnel over vlan" 1>&2 |
| 689 | 699 | ip netns exec "$nsr1" nft list ruleset |