Pull networking fixes from Paolo Abeni: "This is again larger than usual: the backlog accumulated in the past weeks is not done yet. I'm not aware of any known pending regression. Including fixes from netfilter, Bluetooth, WiFi and CAN. Current release - regressions: - bluetooth: remove unnecessary hci_conn_get in create_conn_sync - can: isotp: fix timer drain order, wakeup handling and tx_gen ordering - eth: - tun/vhost: revert avoid ptr_ring tail-drop when a qdisc is present Previous releases - regressions: - core: do not send ICMP/NDISC Redirects when peer allocation fails - ipv6: take nexthop lock for f6i_list walks in replace check and notify - wifi: fix an ath12k MLO regression impacting WCN7850/QCC2072. - netfilter: nf_tables: make nft_object rhltable per table - af_unix: fix listen() succeeding on sockets in the wrong state - openvswitch: fix potential UAF on meter attach failure - bluetooth: - fix advertising data UAFs - avoid deadlocks in iso_sock_timeout - smc: fix socket use-after-free during link group termination - dpll: use pin owner's dpll ref for pin-level attribute reporting - eth: - veth: convert frag_list skbs before running XDP - ice: wait for reset completion in ice_resume() - igc: remove napi_synchronize() in igc_down() - vxlan: use pskb_network_may_pull() for transmit path header pulls Previous releases - always broken: - xsk: fix AF_XDP multi-buffer Tx descriptor reclaim - psp: fix NULL genl_sock deref race with concurrent netns teardown - netfilter: widen NAT rewrite delta to s32 in sip_help_tcp() - can: peak_usb: fix double free of transfer buffer on URB submit error - dibs: fix use-after-free of dmb_node in loopback attach/detach/unregister - sctp: prevent peer transport count overflow - dsa: mt7530: error out on failed reads in MT7531 PHY polling - eth: - idpf: bound interrupt-vector register fill to the allocated array" * tag 'net-7.2-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (156 commits) qede: sync udp_tunnel ports outside qede_lock in the recovery path net: openvswitch: fix potential UAF on meter attach failure octeontx2-pf: Set correct sequence for carrier off and tx queue stop net: libwx: fix FDIR ATR queue mismatch for software VLAN packets net: dsa: realtek: use devm_mutex_init for l2_lock net: dsa: realtek: use devm_mutex_init for vlan_lock net: dsa: realtek: use devm_mutex_init for regmap lock net: dsa: realtek: rtl8365mb: use devm_mutex_init for mib_lock ptp: netc: fix potential interrupt storm caused by incorrect unbind order net: mana: Return error code from mana_create_rxq() net: openvswitch: fix skb leak on flow key update failure during ct net: openvswitch: fix skb leak on flow key update failure during recirculation net: stmmac: Fix E2E delay mechanism net: dsa: mt7530: error out on failed reads in MT7531 PHY polling net: dsa: mt7530: error out on failed reads in ATC/VTCR command polling net: dsa: mt7530: check bus->read() errors in the MDIO regmap backend Revert "tun/tap: add ptr_ring consume helper with netdev queue wakeup" Revert "vhost-net: wake queue of tun/tap after ptr_ring consume" Revert "ptr_ring: move free-space check into separate helper" Revert "tun/tap & vhost-net: avoid ptr_ring tail-drop when a qdisc is present" ...
Unified · Split
| @@ -72,6 +72,7 @@ Alice Mikityanska <alice.kernel@fastmail.im> <maximmi@nvidia.com> · 71 unmodified lines | |||
| 72 | Alice Mikityanska <alice.kernel@fastmail.im> <maxim@isovalent.com> | 72 | Alice Mikityanska <alice.kernel@fastmail.im> <maxim@isovalent.com> |
| 73 | Alice Mikityanska <alice.kernel@fastmail.im> <alice@isovalent.com> | 73 | Alice Mikityanska <alice.kernel@fastmail.im> <alice@isovalent.com> |
| 74 | Aloka Dixit <quic_alokad@quicinc.com> <alokad@codeaurora.org> | 74 | Aloka Dixit <quic_alokad@quicinc.com> <alokad@codeaurora.org> |
| 75 | Alvin Šipraga <alvin.sipraga@analog.com> <alsi@bang-olufsen.dk> | ||
| 75 | Al Viro <viro@ftp.linux.org.uk> | 76 | Al Viro <viro@ftp.linux.org.uk> |
| 76 | Al Viro <viro@zenIV.linux.org.uk> | 77 | Al Viro <viro@zenIV.linux.org.uk> |
| 77 | Amit Blay <quic_ablay@quicinc.com> <ablay@codeaurora.org> | 78 | Amit Blay <quic_ablay@quicinc.com> <ablay@codeaurora.org> |
| @@ -43,12 +43,13 @@ UMEM also has two rings: the FILL ring and the COMPLETION ring. The · 42 unmodified lines | |||
| 43 | FILL ring is used by the application to send down addr for the kernel | 43 | FILL ring is used by the application to send down addr for the kernel |
| 44 | to fill in with RX packet data. References to these frames will then | 44 | to fill in with RX packet data. References to these frames will then |
| 45 | appear in the RX ring once each packet has been received. The | 45 | appear in the RX ring once each packet has been received. The |
| 46 | COMPLETION ring, on the other hand, contains frame addr that the | 46 | COMPLETION ring, on the other hand, contains frame addresses from Tx |
| 47 | kernel has transmitted completely and can now be used again by user | 47 | descriptors that the kernel has finished processing and that can now be |
| 48 | space, for either TX or RX. Thus, the frame addrs appearing in the | 48 | used again by user space, for either Tx or Rx. This includes frames whose |
| 49 | COMPLETION ring are addrs that were previously transmitted using the | 49 | transmission has completed as well as frames referenced by invalid Tx |
| 50 | TX ring. In summary, the RX and FILL rings are used for the RX path | 50 | descriptors rejected by the kernel. A completion therefore returns |
| 51 | and the TX and COMPLETION rings are used for the TX path. | 51 | ownership of a frame to user space, but does not by itself guarantee that |
| 52 | the packet was successfully transmitted. | ||
| 52 | 53 | ||
| 53 | The socket is then finally bound with a bind() call to a device and a | 54 | The socket is then finally bound with a bind() call to a device and a |
| 54 | specific queue id on that device, and it is not until bind is | 55 | specific queue id on that device, and it is not until bind is |
| @@ -169,14 +170,15 @@ chunks mode, then the incoming addr will be left untouched. · 114 unmodified lines | |||
| 169 | UMEM Completion Ring | 170 | UMEM Completion Ring |
| 170 | ~~~~~~~~~~~~~~~~~~~~ | 171 | ~~~~~~~~~~~~~~~~~~~~ |
| 171 | 172 | ||
| 172 | The COMPLETION Ring is used transfer ownership of UMEM frames from | 173 | The COMPLETION Ring is used to transfer ownership of UMEM frames from |
| 173 | kernel-space to user-space. Just like the FILL ring, UMEM indices are | 174 | kernel-space to user-space. Just like the FILL ring, UMEM indices are |
| 174 | used. | 175 | used. Frames passed from the kernel to user-space are frames referenced |
| 175 | 176 | by Tx descriptors that the kernel has finished processing and can be | |
| 176 | Frames passed from the kernel to user-space are frames that has been | 177 | used by user-space again. This includes both frames whose transmission |
| 177 | sent (TX ring) and can be used by user-space again. | 178 | has completed and frames referenced by invalid Tx descriptors that were |
| 178 | 179 | rejected and reclaimed by the kernel. A completion entry does not | |
| 179 | The user application consumes UMEM addrs from this ring. | 180 | guarantee successful packet transmission. The user application consumes |
| 181 | UMEM addrs from this ring. | ||
| 180 | 182 | ||
| 181 | 183 | ||
| 182 | RX Ring | 184 | RX Ring |
| @@ -504,21 +506,25 @@ will be treated as an invalid descriptor. · 321 unmodified lines | |||
| 504 | These are the semantics for producing packets onto AF_XDP Tx ring | 506 | These are the semantics for producing packets onto AF_XDP Tx ring |
| 505 | consisting of multiple frames: | 507 | consisting of multiple frames: |
| 506 | 508 | ||
| 507 | * When an invalid descriptor is found, all the other | 509 | * When an invalid descriptor is found, the complete packet is treated as |
| 508 | descriptors/frames of this packet are marked as invalid and not | 510 | invalid. The kernel consumes descriptors through the descriptor marking |
| 509 | completed. The next descriptor is treated as the start of a new | 511 | the end of the packet and returns all their frame addresses through the |
| 510 | packet, even if this was not the intent (because we cannot guess | 512 | COMPLETION ring. A standalone invalid descriptor is treated as a |
| 511 | the intent). As before, if your program is producing invalid | 513 | one-descriptor invalid packet. The descriptor following the end of the |
| 512 | descriptors you have a bug that must be fixed. | 514 | invalid packet is treated as the start of a new packet. As before, if |
| 515 | your program is producing invalid descriptors you have a bug that must | ||
| 516 | be fixed. Rejected descriptors are reported in the ``tx_invalid_descs`` | ||
| 517 | statistic. | ||
| 513 | 518 | ||
| 514 | * Zero length descriptors are treated as invalid descriptors. | 519 | * Zero length descriptors are treated as invalid descriptors. |
| 515 | 520 | ||
| 516 | * For copy mode, the maximum supported number of frames in a packet is | 521 | * For copy mode, the maximum supported number of frames in a packet is |
| 517 | equal to CONFIG_MAX_SKB_FRAGS + 1. If it is exceeded, all | 522 | equal to CONFIG_MAX_SKB_FRAGS + 1. If it is exceeded, all descriptors |
| 518 | descriptors accumulated so far are dropped and treated as | 523 | through the end of the oversized packet are consumed, treated as invalid, |
| 519 | invalid. To produce an application that will work on any system | 524 | and their frame addresses are returned through the COMPLETION ring. To |
| 520 | regardless of this config setting, limit the number of frags to 18, | 525 | produce an application that will work on any system regardless of this |
| 521 | as the minimum value of the config is 17. | 526 | config setting, limit the number of frags to 18, as the minimum value of |
| 527 | the config is 17. | ||
| 522 | 528 | ||
| 523 | * For zero-copy mode, the limit is up to what the NIC HW | 529 | * For zero-copy mode, the limit is up to what the NIC HW |
| 524 | supports. Usually at least five on the NICs we have checked. We | 530 | supports. Usually at least five on the NICs we have checked. We |
| @@ -22736,7 +22736,8 @@ F: drivers/watchdog/realtek_otto_wdt.c · 22735 unmodified lines | |||
| 22736 | 22736 | ||
| 22737 | REALTEK RTL83xx SMI DSA ROUTER CHIPS | 22737 | REALTEK RTL83xx SMI DSA ROUTER CHIPS |
| 22738 | M: Linus Walleij <linusw@kernel.org> | 22738 | M: Linus Walleij <linusw@kernel.org> |
| 22739 | M: Alvin Šipraga <alsi@bang-olufsen.dk> | 22739 | M: Luiz Angelo Daros de Luca <luizluca@gmail.com> |
| 22740 | R: Alvin Šipraga <alvin.sipraga@analog.com> | ||
| 22740 | S: Maintained | 22741 | S: Maintained |
| 22741 | F: Documentation/devicetree/bindings/net/dsa/realtek.yaml | 22742 | F: Documentation/devicetree/bindings/net/dsa/realtek.yaml |
| 22742 | F: drivers/net/dsa/realtek/* | 22743 | F: drivers/net/dsa/realtek/* |
| @@ -24633,8 +24634,7 @@ SHARED MEMORY COMMUNICATIONS (SMC) SOCKETS · 1890 unmodified lines | |||
| 24633 | M: D. Wythe <alibuda@linux.alibaba.com> | 24634 | M: D. Wythe <alibuda@linux.alibaba.com> |
| 24634 | M: Dust Li <dust.li@linux.alibaba.com> | 24635 | M: Dust Li <dust.li@linux.alibaba.com> |
| 24635 | M: Sidraya Jayagond <sidraya@linux.ibm.com> | 24636 | M: Sidraya Jayagond <sidraya@linux.ibm.com> |
| 24636 | M: Wenjia Zhang <wenjia@linux.ibm.com> | 24637 | M: Mahanta Jambigi <mjambigi@linux.ibm.com> |
| 24637 | R: Mahanta Jambigi <mjambigi@linux.ibm.com> | ||
| 24638 | R: Tony Lu <tonylu@linux.alibaba.com> | 24638 | R: Tony Lu <tonylu@linux.alibaba.com> |
| 24639 | R: Wen Gu <guwen@linux.alibaba.com> | 24639 | R: Wen Gu <guwen@linux.alibaba.com> |
| 24640 | L: linux-rdma@vger.kernel.org | 24640 | L: linux-rdma@vger.kernel.org |
| @@ -3771,6 +3771,9 @@ static int btintel_diagnostics(struct hci_dev *hdev, struct sk_buff *skb) · 3770 unmodified lines | |||
| 3771 | { | 3771 | { |
| 3772 | struct intel_tlv *tlv = (void *)&skb->data[5]; | 3772 | struct intel_tlv *tlv = (void *)&skb->data[5]; |
| 3773 | 3773 | ||
| 3774 | if (skb->len < 5 + sizeof(*tlv) + sizeof(tlv->val[0])) | ||
| 3775 | goto recv_frame; | ||
| 3776 | |||
| 3774 | /* The first event is always an event type TLV */ | 3777 | /* The first event is always an event type TLV */ |
| 3775 | if (tlv->type != INTEL_TLV_TYPE_ID) | 3778 | if (tlv->type != INTEL_TLV_TYPE_ID) |
| 3776 | goto recv_frame; | 3779 | goto recv_frame; |
| @@ -804,56 +804,44 @@ static int btmtk_usb_uhw_reg_write(struct hci_dev *hdev, u32 reg, u32 val) · 803 unmodified lines | |||
| 804 | static int btmtk_usb_uhw_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) | 804 | static int btmtk_usb_uhw_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) |
| 805 | { | 805 | { |
| 806 | struct btmtk_data *data = hci_get_priv(hdev); | 806 | struct btmtk_data *data = hci_get_priv(hdev); |
| 807 | int pipe, err; | 807 | u8 buf[sizeof(u32)]; |
| 808 | void *buf; | 808 | int err; |
| 809 | |||
| 810 | buf = kzalloc(4, GFP_KERNEL); | ||
| 811 | if (!buf) | ||
| 812 | return -ENOMEM; | ||
| 813 | 809 | ||
| 814 | pipe = usb_rcvctrlpipe(data->udev, 0); | 810 | *val = 0; |
| 815 | err = usb_control_msg(data->udev, pipe, 0x01, | 811 | err = usb_control_msg_recv(data->udev, 0, 0x01, |
| 816 | 0xDE, | 812 | 0xDE, |
| 817 | reg >> 16, reg & 0xffff, | 813 | reg >> 16, reg & 0xffff, |
| 818 | buf, 4, USB_CTRL_GET_TIMEOUT); | 814 | buf, sizeof(buf), USB_CTRL_GET_TIMEOUT, |
| 819 | if (err < 0) { | 815 | GFP_KERNEL); |
| 816 | if (err) { | ||
| 820 | bt_dev_err(hdev, "Failed to read uhw reg(%d)", err); | 817 | bt_dev_err(hdev, "Failed to read uhw reg(%d)", err); |
| 821 | goto err_free_buf; | 818 | return err; |
| 822 | } | 819 | } |
| 823 | 820 | ||
| 824 | *val = get_unaligned_le32(buf); | 821 | *val = get_unaligned_le32(buf); |
| 825 | bt_dev_dbg(hdev, "reg=%x, value=0x%08x", reg, *val); | 822 | bt_dev_dbg(hdev, "reg=%x, value=0x%08x", reg, *val); |
| 826 | 823 | ||
| 827 | err_free_buf: | 824 | return 0; |
| 828 | kfree(buf); | ||
| 829 | |||
| 830 | return err; | ||
| 831 | } | 825 | } |
| 832 | 826 | ||
| 833 | static int btmtk_usb_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) | 827 | static int btmtk_usb_reg_read(struct hci_dev *hdev, u32 reg, u32 *val) |
| 834 | { | 828 | { |
| 835 | struct btmtk_data *data = hci_get_priv(hdev); | 829 | struct btmtk_data *data = hci_get_priv(hdev); |
| 836 | int pipe, err, size = sizeof(u32); | 830 | u8 buf[sizeof(u32)]; |
| 837 | void *buf; | 831 | int err; |
| 838 | |||
| 839 | buf = kzalloc(size, GFP_KERNEL); | ||
| 840 | if (!buf) | ||
| 841 | return -ENOMEM; | ||
| 842 | 832 | ||
| 843 | pipe = usb_rcvctrlpipe(data->udev, 0); | 833 | *val = 0; |
| 844 | err = usb_control_msg(data->udev, pipe, 0x63, | 834 | err = usb_control_msg_recv(data->udev, 0, 0x63, |
| 845 | USB_TYPE_VENDOR | USB_DIR_IN, | 835 | USB_TYPE_VENDOR | USB_DIR_IN, |
| 846 | reg >> 16, reg & 0xffff, | 836 | reg >> 16, reg & 0xffff, |
| 847 | buf, size, USB_CTRL_GET_TIMEOUT); | 837 | buf, sizeof(buf), USB_CTRL_GET_TIMEOUT, |
| 838 | GFP_KERNEL); | ||
| 848 | if (err < 0) | 839 | if (err < 0) |
| 849 | goto err_free_buf; | 840 | return err; |
| 850 | 841 | ||
| 851 | *val = get_unaligned_le32(buf); | 842 | *val = get_unaligned_le32(buf); |
| 852 | 843 | ||
| 853 | err_free_buf: | 844 | return 0; |
| 854 | kfree(buf); | ||
| 855 | |||
| 856 | return err; | ||
| 857 | } | 845 | } |
| 858 | 846 | ||
| 859 | static int btmtk_usb_id_get(struct hci_dev *hdev, u32 reg, u32 *id) | 847 | static int btmtk_usb_id_get(struct hci_dev *hdev, u32 reg, u32 *id) |
| @@ -877,7 +865,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 17 unmodified lines | |||
| 877 | 865 | ||
| 878 | if (dev_id == 0x7922) { | 866 | if (dev_id == 0x7922) { |
| 879 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); | 867 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 880 | if (err < 0) | 868 | if (err) |
| 881 | return err; | 869 | return err; |
| 882 | val |= 0x00002020; | 870 | val |= 0x00002020; |
| 883 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); | 871 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); |
| @@ -887,7 +875,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | |||
| 887 | if (err < 0) | 875 | if (err < 0) |
| 888 | return err; | 876 | return err; |
| 889 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); | 877 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 890 | if (err < 0) | 878 | if (err) |
| 891 | return err; | 879 | return err; |
| 892 | val |= BIT(0); | 880 | val |= BIT(0); |
| 893 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); | 881 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, val); |
| @@ -896,14 +884,14 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 2 unmodified lines | |||
| 896 | msleep(100); | 884 | msleep(100); |
| 897 | } else if (dev_id == 0x7925 || dev_id == 0x6639) { | 885 | } else if (dev_id == 0x7925 || dev_id == 0x6639) { |
| 898 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); | 886 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 899 | if (err < 0) | 887 | if (err) |
| 900 | return err; | 888 | return err; |
| 901 | val |= (1 << 5); | 889 | val |= (1 << 5); |
| 902 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); | 890 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); |
| 903 | if (err < 0) | 891 | if (err < 0) |
| 904 | return err; | 892 | return err; |
| 905 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); | 893 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 906 | if (err < 0) | 894 | if (err) |
| 907 | return err; | 895 | return err; |
| 908 | val &= 0xFFFF00FF; | 896 | val &= 0xFFFF00FF; |
| 909 | val |= (1 << 13); | 897 | val |= (1 << 13); |
| @@ -914,7 +902,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 4 unmodified lines | |||
| 914 | if (err < 0) | 902 | if (err < 0) |
| 915 | return err; | 903 | return err; |
| 916 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); | 904 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_RESET_REG_CONNV3, &val); |
| 917 | if (err < 0) | 905 | if (err) |
| 918 | return err; | 906 | return err; |
| 919 | val |= (1 << 0); | 907 | val |= (1 << 0); |
| 920 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); | 908 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_RESET_REG_CONNV3, val); |
| @@ -924,13 +912,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | |||
| 924 | if (err < 0) | 912 | if (err < 0) |
| 925 | return err; | 913 | return err; |
| 926 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); | 914 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); |
| 927 | if (err < 0) | 915 | if (err) |
| 928 | return err; | 916 | return err; |
| 929 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); | 917 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); |
| 930 | if (err < 0) | 918 | if (err < 0) |
| 931 | return err; | 919 | return err; |
| 932 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); | 920 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); |
| 933 | if (err < 0) | 921 | if (err) |
| 934 | return err; | 922 | return err; |
| 935 | msleep(100); | 923 | msleep(100); |
| 936 | } else { | 924 | } else { |
| @@ -940,7 +928,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | |||
| 940 | if (err < 0) | 928 | if (err < 0) |
| 941 | return err; | 929 | return err; |
| 942 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_WDT_STATUS, &val); | 930 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_WDT_STATUS, &val); |
| 943 | if (err < 0) | 931 | if (err) |
| 944 | return err; | 932 | return err; |
| 945 | /* Reset the bluetooth chip via USB interface. */ | 933 | /* Reset the bluetooth chip via USB interface. */ |
| 946 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, 1); | 934 | err = btmtk_usb_uhw_reg_write(hdev, MTK_BT_SUBSYS_RST, 1); |
| @@ -950,13 +938,13 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 3 unmodified lines | |||
| 950 | if (err < 0) | 938 | if (err < 0) |
| 951 | return err; | 939 | return err; |
| 952 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); | 940 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT, &val); |
| 953 | if (err < 0) | 941 | if (err) |
| 954 | return err; | 942 | return err; |
| 955 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); | 943 | err = btmtk_usb_uhw_reg_write(hdev, MTK_UDMA_INT_STA_BT1, 0x000000FF); |
| 956 | if (err < 0) | 944 | if (err < 0) |
| 957 | return err; | 945 | return err; |
| 958 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); | 946 | err = btmtk_usb_uhw_reg_read(hdev, MTK_UDMA_INT_STA_BT1, &val); |
| 959 | if (err < 0) | 947 | if (err) |
| 960 | return err; | 948 | return err; |
| 961 | /* MT7921 need to delay 20ms between toggle reset bit */ | 949 | /* MT7921 need to delay 20ms between toggle reset bit */ |
| 962 | msleep(20); | 950 | msleep(20); |
| @@ -964,7 +952,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 1 unmodified line | |||
| 964 | if (err < 0) | 952 | if (err < 0) |
| 965 | return err; | 953 | return err; |
| 966 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); | 954 | err = btmtk_usb_uhw_reg_read(hdev, MTK_BT_SUBSYS_RST, &val); |
| 967 | if (err < 0) | 955 | if (err) |
| 968 | return err; | 956 | return err; |
| 969 | } | 957 | } |
| 970 | 958 | ||
| @@ -980,7 +968,7 @@ int btmtk_usb_subsys_reset(struct hci_dev *hdev, u32 dev_id) · 9 unmodified lines | |||
| 980 | } | 968 | } |
| 981 | 969 | ||
| 982 | err = btmtk_usb_id_get(hdev, 0x70010200, &val); | 970 | err = btmtk_usb_id_get(hdev, 0x70010200, &val); |
| 983 | if (err < 0 || (!val && dev_id != 0x6639)) | 971 | if (err || (!val && dev_id != 0x6639)) |
| 984 | bt_dev_err(hdev, "Can't get device id, subsys reset fail."); | 972 | bt_dev_err(hdev, "Can't get device id, subsys reset fail."); |
| 985 | 973 | ||
| 986 | return err; | 974 | return err; |
| @@ -1324,24 +1312,24 @@ int btmtk_usb_setup(struct hci_dev *hdev) · 337 unmodified lines | |||
| 1324 | calltime = ktime_get(); | 1312 | calltime = ktime_get(); |
| 1325 | 1313 | ||
| 1326 | err = btmtk_usb_id_get(hdev, 0x80000008, &dev_id); | 1314 | err = btmtk_usb_id_get(hdev, 0x80000008, &dev_id); |
| 1327 | if (err < 0) { | 1315 | if (err) { |
| 1328 | bt_dev_err(hdev, "Failed to get device id (%d)", err); | 1316 | bt_dev_err(hdev, "Failed to get device id (%d)", err); |
| 1329 | return err; | 1317 | return err; |
| 1330 | } | 1318 | } |
| 1331 | 1319 | ||
| 1332 | if (!dev_id || dev_id != 0x7663) { | 1320 | if (!dev_id || dev_id != 0x7663) { |
| 1333 | err = btmtk_usb_id_get(hdev, 0x70010200, &dev_id); | 1321 | err = btmtk_usb_id_get(hdev, 0x70010200, &dev_id); |
| 1334 | if (err < 0) { | 1322 | if (err) { |
| 1335 | bt_dev_err(hdev, "Failed to get device id (%d)", err); | 1323 | bt_dev_err(hdev, "Failed to get device id (%d)", err); |
| 1336 | return err; | 1324 | return err; |
| 1337 | } | 1325 | } |
| 1338 | err = btmtk_usb_id_get(hdev, 0x80021004, &fw_version); | 1326 | err = btmtk_usb_id_get(hdev, 0x80021004, &fw_version); |
| 1339 | if (err < 0) { | 1327 | if (err) { |
| 1340 | bt_dev_err(hdev, "Failed to get fw version (%d)", err); | 1328 | bt_dev_err(hdev, "Failed to get fw version (%d)", err); |
| 1341 | return err; | 1329 | return err; |
| 1342 | } | 1330 | } |
| 1343 | err = btmtk_usb_id_get(hdev, 0x70010020, &fw_flavor); | 1331 | err = btmtk_usb_id_get(hdev, 0x70010020, &fw_flavor); |
| 1344 | if (err < 0) { | 1332 | if (err) { |
| 1345 | bt_dev_err(hdev, "Failed to get fw flavor (%d)", err); | 1333 | bt_dev_err(hdev, "Failed to get fw flavor (%d)", err); |
| 1346 | return err; | 1334 | return err; |
| 1347 | } | 1335 | } |
| @@ -3424,28 +3424,16 @@ static const char *qca_get_fw_subdirectory(const struct qca_version *ver) · 3423 unmodified lines | |||
| 3424 | static int btusb_qca_send_vendor_req(struct usb_device *udev, u8 request, | 3424 | static int btusb_qca_send_vendor_req(struct usb_device *udev, u8 request, |
| 3425 | void *data, u16 size) | 3425 | void *data, u16 size) |
| 3426 | { | 3426 | { |
| 3427 | int pipe, err; | 3427 | int err; |
| 3428 | u8 *buf; | ||
| 3429 | |||
| 3430 | buf = kmalloc(size, GFP_KERNEL); | ||
| 3431 | if (!buf) | ||
| 3432 | return -ENOMEM; | ||
| 3433 | 3428 | ||
| 3434 | /* Found some of USB hosts have IOT issues with ours so that we should | 3429 | /* Found some of USB hosts have IOT issues with ours so that we should |
| 3435 | * not wait until HCI layer is ready. | 3430 | * not wait until HCI layer is ready. |
| 3436 | */ | 3431 | */ |
| 3437 | pipe = usb_rcvctrlpipe(udev, 0); | 3432 | err = usb_control_msg_recv(udev, 0, request, USB_TYPE_VENDOR | USB_DIR_IN, |
| 3438 | err = usb_control_msg(udev, pipe, request, USB_TYPE_VENDOR | USB_DIR_IN, | 3433 | 0, 0, data, size, USB_CTRL_GET_TIMEOUT, |
| 3439 | 0, 0, buf, size, USB_CTRL_GET_TIMEOUT); | 3434 | GFP_KERNEL); |
| 3440 | if (err < 0) { | 3435 | if (err) |
| 3441 | dev_err(&udev->dev, "Failed to access otp area (%d)", err); | 3436 | dev_err(&udev->dev, "Failed to access otp area (%d)", err); |
| 3442 | goto done; | ||
| 3443 | } | ||
| 3444 | |||
| 3445 | memcpy(data, buf, size); | ||
| 3446 | |||
| 3447 | done: | ||
| 3448 | kfree(buf); | ||
| 3449 | 3437 | ||
| 3450 | return err; | 3438 | return err; |
| 3451 | } | 3439 | } |
| @@ -3652,7 +3640,7 @@ static bool btusb_qca_need_patch(struct usb_device *udev) · 200 unmodified lines | |||
| 3652 | struct qca_version ver; | 3640 | struct qca_version ver; |
| 3653 | 3641 | ||
| 3654 | if (btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, | 3642 | if (btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3655 | sizeof(ver)) < 0) | 3643 | sizeof(ver))) |
| 3656 | return false; | 3644 | return false; |
| 3657 | /* only low ROM versions need patches */ | 3645 | /* only low ROM versions need patches */ |
| 3658 | return !(le32_to_cpu(ver.rom_version) & ~0xffffU); | 3646 | return !(le32_to_cpu(ver.rom_version) & ~0xffffU); |
| @@ -3670,7 +3658,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 11 unmodified lines | |||
| 3670 | 3658 | ||
| 3671 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, | 3659 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3672 | sizeof(ver)); | 3660 | sizeof(ver)); |
| 3673 | if (err < 0) | 3661 | if (err) |
| 3674 | return err; | 3662 | return err; |
| 3675 | 3663 | ||
| 3676 | ver_rom = le32_to_cpu(ver.rom_version); | 3664 | ver_rom = le32_to_cpu(ver.rom_version); |
| @@ -3693,7 +3681,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 16 unmodified lines | |||
| 3693 | 3681 | ||
| 3694 | err = btusb_qca_send_vendor_req(udev, QCA_CHECK_STATUS, &status, | 3682 | err = btusb_qca_send_vendor_req(udev, QCA_CHECK_STATUS, &status, |
| 3695 | sizeof(status)); | 3683 | sizeof(status)); |
| 3696 | if (err < 0) | 3684 | if (err) |
| 3697 | return err; | 3685 | return err; |
| 3698 | 3686 | ||
| 3699 | if (!(status & QCA_PATCH_UPDATED)) { | 3687 | if (!(status & QCA_PATCH_UPDATED)) { |
| @@ -3704,7 +3692,7 @@ static int btusb_setup_qca(struct hci_dev *hdev) · 4 unmodified lines | |||
| 3704 | 3692 | ||
| 3705 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, | 3693 | err = btusb_qca_send_vendor_req(udev, QCA_GET_TARGET_VERSION, &ver, |
| 3706 | sizeof(ver)); | 3694 | sizeof(ver)); |
| 3707 | if (err < 0) | 3695 | if (err) |
| 3708 | return err; | 3696 | return err; |
| 3709 | 3697 | ||
| 3710 | btdata->qca_dump.fw_version = le32_to_cpu(ver.patch_version); | 3698 | btdata->qca_dump.fw_version = le32_to_cpu(ver.patch_version); |
| @@ -118,14 +118,9 @@ err_bit: · 117 unmodified lines | |||
| 118 | return rc; | 118 | return rc; |
| 119 | } | 119 | } |
| 120 | 120 | ||
| 121 | static void __dibs_lo_unregister_dmb(struct dibs_lo_dev *ldev, | 121 | static void dibs_lo_free_dmb(struct dibs_lo_dev *ldev, |
| 122 | struct dibs_lo_dmb_node *dmb_node) | 122 | struct dibs_lo_dmb_node *dmb_node) |
| 123 | { | 123 | { |
| 124 | /* remove dmb from hash table */ | ||
| 125 | write_lock_bh(&ldev->dmb_ht_lock); | ||
| 126 | hash_del(&dmb_node->list); | ||
| 127 | write_unlock_bh(&ldev->dmb_ht_lock); | ||
| 128 | |||
| 129 | clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); | 124 | clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); |
| 130 | folio_put(virt_to_folio(dmb_node->cpu_addr)); | 125 | folio_put(virt_to_folio(dmb_node->cpu_addr)); |
| 131 | kfree(dmb_node); | 126 | kfree(dmb_node); |
| @@ -139,27 +134,33 @@ static int dibs_lo_unregister_dmb(struct dibs_dev *dibs, struct dibs_dmb *dmb) · 7 unmodified lines | |||
| 139 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; | 134 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; |
| 140 | struct dibs_lo_dev *ldev; | 135 | struct dibs_lo_dev *ldev; |
| 141 | unsigned long flags; | 136 | unsigned long flags; |
| 137 | bool last; | ||
| 142 | 138 | ||
| 143 | ldev = dibs->drv_priv; | 139 | ldev = dibs->drv_priv; |
| 144 | 140 | ||
| 145 | /* find dmb from hash table */ | 141 | /* find dmb from hash table */ |
| 146 | read_lock_bh(&ldev->dmb_ht_lock); | 142 | write_lock_bh(&ldev->dmb_ht_lock); |
| 147 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { | 143 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { |
| 148 | if (tmp_node->token == dmb->dmb_tok) { | 144 | if (tmp_node->token == dmb->dmb_tok) { |
| 149 | dmb_node = tmp_node; | 145 | dmb_node = tmp_node; |
| 150 | break; | 146 | break; |
| 151 | } | 147 | } |
| 152 | } | 148 | } |
| 153 | read_unlock_bh(&ldev->dmb_ht_lock); | 149 | if (!dmb_node) { |
| 154 | if (!dmb_node) | 150 | write_unlock_bh(&ldev->dmb_ht_lock); |
| 155 | return -EINVAL; | 151 | return -EINVAL; |
| 152 | } | ||
| 153 | last = refcount_dec_and_test(&dmb_node->refcnt); | ||
| 154 | if (last) | ||
| 155 | hash_del(&dmb_node->list); | ||
| 156 | write_unlock_bh(&ldev->dmb_ht_lock); | ||
| 156 | 157 | ||
| 157 | if (refcount_dec_and_test(&dmb_node->refcnt)) { | 158 | if (last) { |
| 158 | spin_lock_irqsave(&dibs->lock, flags); | 159 | spin_lock_irqsave(&dibs->lock, flags); |
| 159 | dibs->dmb_clientid_arr[dmb_node->sba_idx] = NO_DIBS_CLIENT; | 160 | dibs->dmb_clientid_arr[dmb_node->sba_idx] = NO_DIBS_CLIENT; |
| 160 | spin_unlock_irqrestore(&dibs->lock, flags); | 161 | spin_unlock_irqrestore(&dibs->lock, flags); |
| 161 | 162 | ||
| 162 | __dibs_lo_unregister_dmb(ldev, dmb_node); | 163 | dibs_lo_free_dmb(ldev, dmb_node); |
| 163 | } | 164 | } |
| 164 | return 0; | 165 | return 0; |
| 165 | } | 166 | } |
| @@ -188,14 +189,9 @@ static int dibs_lo_attach_dmb(struct dibs_dev *dibs, struct dibs_dmb *dmb) · 22 unmodified lines | |||
| 188 | read_unlock_bh(&ldev->dmb_ht_lock); | 189 | read_unlock_bh(&ldev->dmb_ht_lock); |
| 189 | return -EINVAL; | 190 | return -EINVAL; |
| 190 | } | 191 | } |
| 192 | refcount_inc(&dmb_node->refcnt); | ||
| 191 | read_unlock_bh(&ldev->dmb_ht_lock); | 193 | read_unlock_bh(&ldev->dmb_ht_lock); |
| 192 | 194 | ||
| 193 | if (!refcount_inc_not_zero(&dmb_node->refcnt)) | ||
| 194 | /* the dmb is being unregistered, but has | ||
| 195 | * not been removed from the hash table. | ||
| 196 | */ | ||
| 197 | return -EINVAL; | ||
| 198 | |||
| 199 | /* provide dmb information */ | 195 | /* provide dmb information */ |
| 200 | dmb->idx = dmb_node->sba_idx; | 196 | dmb->idx = dmb_node->sba_idx; |
| 201 | dmb->dmb_tok = dmb_node->token; | 197 | dmb->dmb_tok = dmb_node->token; |
| @@ -209,11 +205,12 @@ static int dibs_lo_detach_dmb(struct dibs_dev *dibs, u64 token) · 7 unmodified lines | |||
| 209 | { | 205 | { |
| 210 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; | 206 | struct dibs_lo_dmb_node *dmb_node = NULL, *tmp_node; |
| 211 | struct dibs_lo_dev *ldev; | 207 | struct dibs_lo_dev *ldev; |
| 208 | bool last; | ||
| 212 | 209 | ||
| 213 | ldev = dibs->drv_priv; | 210 | ldev = dibs->drv_priv; |
| 214 | 211 | ||
| 215 | /* find dmb_node according to dmb->dmb_tok */ | 212 | /* find dmb_node according to dmb->dmb_tok */ |
| 216 | read_lock_bh(&ldev->dmb_ht_lock); | 213 | write_lock_bh(&ldev->dmb_ht_lock); |
| 217 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { | 214 | hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { |
| 218 | if (tmp_node->token == token) { | 215 | if (tmp_node->token == token) { |
| 219 | dmb_node = tmp_node; | 216 | dmb_node = tmp_node; |
| @@ -221,13 +218,17 @@ static int dibs_lo_detach_dmb(struct dibs_dev *dibs, u64 token) · 1 unmodified line | |||
| 221 | } | 218 | } |
| 222 | } | 219 | } |
| 223 | if (!dmb_node) { | 220 | if (!dmb_node) { |
| 224 | read_unlock_bh(&ldev->dmb_ht_lock); | 221 | write_unlock_bh(&ldev->dmb_ht_lock); |
| 225 | return -EINVAL; | 222 | return -EINVAL; |
| 226 | } | 223 | } |
| 227 | read_unlock_bh(&ldev->dmb_ht_lock); | 224 | last = refcount_dec_and_test(&dmb_node->refcnt); |
| 225 | if (last) | ||
| 226 | hash_del(&dmb_node->list); | ||
| 227 | write_unlock_bh(&ldev->dmb_ht_lock); | ||
| 228 | |||
| 229 | if (last) | ||
| 230 | dibs_lo_free_dmb(ldev, dmb_node); | ||
| 228 | 231 | ||
| 229 | if (refcount_dec_and_test(&dmb_node->refcnt)) | ||
| 230 | __dibs_lo_unregister_dmb(ldev, dmb_node); | ||
| 231 | return 0; | 232 | return 0; |
| 232 | } | 233 | } |
| 233 | 234 | ||
| @@ -1142,6 +1142,33 @@ void *dpll_pin_on_pin_priv(struct dpll_pin *parent, · 1141 unmodified lines | |||
| 1142 | return reg->priv; | 1142 | return reg->priv; |
| 1143 | } | 1143 | } |
| 1144 | 1144 | ||
| 1145 | /** | ||
| 1146 | * dpll_pin_own_dpll_ref_first - find the first owner dpll ref of a pin | ||
| 1147 | * @pin: pointer to a dpll pin | ||
| 1148 | * | ||
| 1149 | * Search pin's dpll_refs for a ref whose dpll matches the pin's | ||
| 1150 | * (module, clock_id) tuple, i.e. the dpll registered by the driver | ||
| 1151 | * that created the pin. This ensures pin-level attributes are | ||
| 1152 | * reported and modified using the owner's ops even when the pin is | ||
| 1153 | * also registered with dplls from other drivers. | ||
| 1154 | * | ||
| 1155 | * Return: pointer to the owner's dpll_pin_ref, or NULL if no | ||
| 1156 | * owner ref is found. | ||
| 1157 | */ | ||
| 1158 | struct dpll_pin_ref *dpll_pin_own_dpll_ref_first(struct dpll_pin *pin) | ||
| 1159 | { | ||
| 1160 | struct dpll_pin_ref *ref; | ||
| 1161 | unsigned long i; | ||
| 1162 | |||
| 1163 | xa_for_each(&pin->dpll_refs, i, ref) { | ||
| 1164 | if (ref->dpll->module == pin->module && | ||
| 1165 | ref->dpll->clock_id == pin->clock_id) | ||
| 1166 | return ref; | ||
| 1167 | } | ||
| 1168 | |||
| 1169 | return NULL; | ||
| 1170 | } | ||
| 1171 | |||
| 1145 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref) | 1172 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref) |
| 1146 | { | 1173 | { |
| 1147 | struct dpll_pin_registration *reg; | 1174 | struct dpll_pin_registration *reg; |
| @@ -93,6 +93,7 @@ void *dpll_pin_on_pin_priv(struct dpll_pin *parent, struct dpll_pin *pin); · 92 unmodified lines | |||
| 93 | 93 | ||
| 94 | const struct dpll_device_ops *dpll_device_ops(struct dpll_device *dpll); | 94 | const struct dpll_device_ops *dpll_device_ops(struct dpll_device *dpll); |
| 95 | struct dpll_device *dpll_device_get_by_id(int id); | 95 | struct dpll_device *dpll_device_get_by_id(int id); |
| 96 | struct dpll_pin_ref *dpll_pin_own_dpll_ref_first(struct dpll_pin *pin); | ||
| 96 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref); | 97 | const struct dpll_pin_ops *dpll_pin_ops(struct dpll_pin_ref *ref); |
| 97 | struct dpll_pin_ref *dpll_xa_ref_dpll_first(struct xarray *xa_refs); | 98 | struct dpll_pin_ref *dpll_xa_ref_dpll_first(struct xarray *xa_refs); |
| 98 | extern struct xarray dpll_device_xa; | 99 | extern struct xarray dpll_device_xa; |
| @@ -699,7 +699,9 @@ dpll_cmd_pin_get_one(struct sk_buff *msg, struct dpll_pin *pin, · 698 unmodified lines | |||
| 699 | struct dpll_pin_ref *ref; | 699 | struct dpll_pin_ref *ref; |
| 700 | int ret; | 700 | int ret; |
| 701 | 701 | ||
| 702 | ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | 702 | ref = dpll_pin_own_dpll_ref_first(pin); |
| 703 | if (!ref) | ||
| 704 | ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | ||
| 703 | ASSERT_NOT_NULL(ref); | 705 | ASSERT_NOT_NULL(ref); |
| 704 | 706 | ||
| 705 | ret = dpll_msg_add_pin_handle(msg, pin); | 707 | ret = dpll_msg_add_pin_handle(msg, pin); |
| @@ -1090,12 +1092,19 @@ dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a, · 384 unmodified lines | |||
| 1090 | 1092 | ||
| 1091 | xa_for_each(&pin->dpll_refs, i, ref) { | 1093 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1092 | ops = dpll_pin_ops(ref); | 1094 | ops = dpll_pin_ops(ref); |
| 1093 | if (!ops->frequency_set || !ops->frequency_get) { | 1095 | if ((!ops->frequency_set || !ops->frequency_get) && |
| 1094 | NL_SET_ERR_MSG(extack, "frequency set not supported by the device"); | 1096 | ref->dpll->module == pin->module && |
| 1097 | ref->dpll->clock_id == pin->clock_id) { | ||
| 1098 | NL_SET_ERR_MSG(extack, | ||
| 1099 | "frequency set not supported by the device"); | ||
| 1095 | return -EOPNOTSUPP; | 1100 | return -EOPNOTSUPP; |
| 1096 | } | 1101 | } |
| 1097 | } | 1102 | } |
| 1098 | ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | 1103 | ref = dpll_pin_own_dpll_ref_first(pin); |
| 1104 | if (!ref) { | ||
| 1105 | NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | ||
| 1106 | return -ENODEV; | ||
| 1107 | } | ||
| 1099 | ops = dpll_pin_ops(ref); | 1108 | ops = dpll_pin_ops(ref); |
| 1100 | dpll = ref->dpll; | 1109 | dpll = ref->dpll; |
| 1101 | ret = ops->frequency_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, | 1110 | ret = ops->frequency_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, |
| @@ -1109,6 +1118,8 @@ dpll_pin_freq_set(struct dpll_pin *pin, struct nlattr *a, · 7 unmodified lines | |||
| 1109 | 1118 | ||
| 1110 | xa_for_each(&pin->dpll_refs, i, ref) { | 1119 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1111 | ops = dpll_pin_ops(ref); | 1120 | ops = dpll_pin_ops(ref); |
| 1121 | if (!ops->frequency_set) | ||
| 1122 | continue; | ||
| 1112 | dpll = ref->dpll; | 1123 | dpll = ref->dpll; |
| 1113 | ret = ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), | 1124 | ret = ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1114 | dpll, dpll_priv(dpll), freq, extack); | 1125 | dpll, dpll_priv(dpll), freq, extack); |
| @@ -1128,6 +1139,8 @@ rollback: · 13 unmodified lines | |||
| 1128 | if (ref == failed) | 1139 | if (ref == failed) |
| 1129 | break; | 1140 | break; |
| 1130 | ops = dpll_pin_ops(ref); | 1141 | ops = dpll_pin_ops(ref); |
| 1142 | if (!ops->frequency_set) | ||
| 1143 | continue; | ||
| 1131 | dpll = ref->dpll; | 1144 | dpll = ref->dpll; |
| 1132 | if (ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), | 1145 | if (ops->frequency_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1133 | dpll, dpll_priv(dpll), old_freq, extack)) | 1146 | dpll, dpll_priv(dpll), old_freq, extack)) |
| @@ -1151,13 +1164,19 @@ dpll_pin_esync_set(struct dpll_pin *pin, struct nlattr *a, · 17 unmodified lines | |||
| 1151 | 1164 | ||
| 1152 | xa_for_each(&pin->dpll_refs, i, ref) { | 1165 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1153 | ops = dpll_pin_ops(ref); | 1166 | ops = dpll_pin_ops(ref); |
| 1154 | if (!ops->esync_set || !ops->esync_get) { | 1167 | if ((!ops->esync_set || !ops->esync_get) && |
| 1168 | ref->dpll->module == pin->module && | ||
| 1169 | ref->dpll->clock_id == pin->clock_id) { | ||
| 1155 | NL_SET_ERR_MSG(extack, | 1170 | NL_SET_ERR_MSG(extack, |
| 1156 | "embedded sync feature is not supported by this device"); | 1171 | "embedded sync feature is not supported by this device"); |
| 1157 | return -EOPNOTSUPP; | 1172 | return -EOPNOTSUPP; |
| 1158 | } | 1173 | } |
| 1159 | } | 1174 | } |
| 1160 | ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | 1175 | ref = dpll_pin_own_dpll_ref_first(pin); |
| 1176 | if (!ref) { | ||
| 1177 | NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | ||
| 1178 | return -ENODEV; | ||
| 1179 | } | ||
| 1161 | ops = dpll_pin_ops(ref); | 1180 | ops = dpll_pin_ops(ref); |
| 1162 | dpll = ref->dpll; | 1181 | dpll = ref->dpll; |
| 1163 | ret = ops->esync_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, | 1182 | ret = ops->esync_get(pin, dpll_pin_on_dpll_priv(dpll, pin), dpll, |
| @@ -1181,6 +1200,8 @@ dpll_pin_esync_set(struct dpll_pin *pin, struct nlattr *a, · 17 unmodified lines | |||
| 1181 | void *pin_dpll_priv; | 1200 | void *pin_dpll_priv; |
| 1182 | 1201 | ||
| 1183 | ops = dpll_pin_ops(ref); | 1202 | ops = dpll_pin_ops(ref); |
| 1203 | if (!ops->esync_set) | ||
| 1204 | continue; | ||
| 1184 | dpll = ref->dpll; | 1205 | dpll = ref->dpll; |
| 1185 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); | 1206 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); |
| 1186 | ret = ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), | 1207 | ret = ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), |
| @@ -1204,6 +1225,8 @@ rollback: · 17 unmodified lines | |||
| 1204 | if (ref == failed) | 1225 | if (ref == failed) |
| 1205 | break; | 1226 | break; |
| 1206 | ops = dpll_pin_ops(ref); | 1227 | ops = dpll_pin_ops(ref); |
| 1228 | if (!ops->esync_set) | ||
| 1229 | continue; | ||
| 1207 | dpll = ref->dpll; | 1230 | dpll = ref->dpll; |
| 1208 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); | 1231 | pin_dpll_priv = dpll_pin_on_dpll_priv(dpll, pin); |
| 1209 | if (ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), | 1232 | if (ops->esync_set(pin, pin_dpll_priv, dpll, dpll_priv(dpll), |
| @@ -1238,8 +1261,11 @@ dpll_pin_ref_sync_state_set(struct dpll_pin *pin, · 28 unmodified lines | |||
| 1238 | NL_SET_ERR_MSG(extack, "reference sync pin not available"); | 1261 | NL_SET_ERR_MSG(extack, "reference sync pin not available"); |
| 1239 | return -EINVAL; | 1262 | return -EINVAL; |
| 1240 | } | 1263 | } |
| 1241 | ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | 1264 | ref = dpll_pin_own_dpll_ref_first(pin); |
| 1242 | ASSERT_NOT_NULL(ref); | 1265 | if (!ref) { |
| 1266 | NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | ||
| 1267 | return -ENODEV; | ||
| 1268 | } | ||
| 1243 | ops = dpll_pin_ops(ref); | 1269 | ops = dpll_pin_ops(ref); |
| 1244 | if (!ops->ref_sync_set || !ops->ref_sync_get) { | 1270 | if (!ops->ref_sync_set || !ops->ref_sync_get) { |
| 1245 | NL_SET_ERR_MSG(extack, "reference sync not supported by this pin"); | 1271 | NL_SET_ERR_MSG(extack, "reference sync not supported by this pin"); |
| @@ -1258,6 +1284,8 @@ dpll_pin_ref_sync_state_set(struct dpll_pin *pin, · 12 unmodified lines | |||
| 1258 | return 0; | 1284 | return 0; |
| 1259 | xa_for_each(&pin->dpll_refs, i, ref) { | 1285 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1260 | ops = dpll_pin_ops(ref); | 1286 | ops = dpll_pin_ops(ref); |
| 1287 | if (!ops->ref_sync_set) | ||
| 1288 | continue; | ||
| 1261 | dpll = ref->dpll; | 1289 | dpll = ref->dpll; |
| 1262 | ret = ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), | 1290 | ret = ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1263 | ref_sync_pin, | 1291 | ref_sync_pin, |
| @@ -1280,6 +1308,8 @@ rollback: · 16 unmodified lines | |||
| 1280 | if (ref == failed) | 1308 | if (ref == failed) |
| 1281 | break; | 1309 | break; |
| 1282 | ops = dpll_pin_ops(ref); | 1310 | ops = dpll_pin_ops(ref); |
| 1311 | if (!ops->ref_sync_set) | ||
| 1312 | continue; | ||
| 1283 | dpll = ref->dpll; | 1313 | dpll = ref->dpll; |
| 1284 | if (ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), | 1314 | if (ops->ref_sync_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1285 | ref_sync_pin, | 1315 | ref_sync_pin, |
| @@ -1471,12 +1501,18 @@ dpll_pin_phase_adj_set(struct dpll_pin *pin, struct nlattr *phase_adj_attr, · 185 unmodified lines | |||
| 1471 | 1501 | ||
| 1472 | xa_for_each(&pin->dpll_refs, i, ref) { | 1502 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1473 | ops = dpll_pin_ops(ref); | 1503 | ops = dpll_pin_ops(ref); |
| 1474 | if (!ops->phase_adjust_set || !ops->phase_adjust_get) { | 1504 | if ((!ops->phase_adjust_set || !ops->phase_adjust_get) && |
| 1505 | ref->dpll->module == pin->module && | ||
| 1506 | ref->dpll->clock_id == pin->clock_id) { | ||
| 1475 | NL_SET_ERR_MSG(extack, "phase adjust not supported"); | 1507 | NL_SET_ERR_MSG(extack, "phase adjust not supported"); |
| 1476 | return -EOPNOTSUPP; | 1508 | return -EOPNOTSUPP; |
| 1477 | } | 1509 | } |
| 1478 | } | 1510 | } |
| 1479 | ref = dpll_xa_ref_dpll_first(&pin->dpll_refs); | 1511 | ref = dpll_pin_own_dpll_ref_first(pin); |
| 1512 | if (!ref) { | ||
| 1513 | NL_SET_ERR_MSG(extack, "pin owner dpll not found"); | ||
| 1514 | return -ENODEV; | ||
| 1515 | } | ||
| 1480 | ops = dpll_pin_ops(ref); | 1516 | ops = dpll_pin_ops(ref); |
| 1481 | dpll = ref->dpll; | 1517 | dpll = ref->dpll; |
| 1482 | ret = ops->phase_adjust_get(pin, dpll_pin_on_dpll_priv(dpll, pin), | 1518 | ret = ops->phase_adjust_get(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| @@ -1491,6 +1527,8 @@ dpll_pin_phase_adj_set(struct dpll_pin *pin, struct nlattr *phase_adj_attr, · 8 unmodified lines | |||
| 1491 | 1527 | ||
| 1492 | xa_for_each(&pin->dpll_refs, i, ref) { | 1528 | xa_for_each(&pin->dpll_refs, i, ref) { |
| 1493 | ops = dpll_pin_ops(ref); | 1529 | ops = dpll_pin_ops(ref); |
| 1530 | if (!ops->phase_adjust_set) | ||
| 1531 | continue; | ||
| 1494 | dpll = ref->dpll; | 1532 | dpll = ref->dpll; |
| 1495 | ret = ops->phase_adjust_set(pin, | 1533 | ret = ops->phase_adjust_set(pin, |
| 1496 | dpll_pin_on_dpll_priv(dpll, pin), | 1534 | dpll_pin_on_dpll_priv(dpll, pin), |
| @@ -1513,6 +1551,8 @@ rollback: · 16 unmodified lines | |||
| 1513 | if (ref == failed) | 1551 | if (ref == failed) |
| 1514 | break; | 1552 | break; |
| 1515 | ops = dpll_pin_ops(ref); | 1553 | ops = dpll_pin_ops(ref); |
| 1554 | if (!ops->phase_adjust_set) | ||
| 1555 | continue; | ||
| 1516 | dpll = ref->dpll; | 1556 | dpll = ref->dpll; |
| 1517 | if (ops->phase_adjust_set(pin, dpll_pin_on_dpll_priv(dpll, pin), | 1557 | if (ops->phase_adjust_set(pin, dpll_pin_on_dpll_priv(dpll, pin), |
| 1518 | dpll, dpll_priv(dpll), old_phase_adj, | 1558 | dpll, dpll_priv(dpll), old_phase_adj, |
| @@ -597,20 +597,20 @@ static int c_can_chip_config(struct net_device *dev) · 596 unmodified lines | |||
| 597 | return err; | 597 | return err; |
| 598 | 598 | ||
| 599 | /* enable automatic retransmission */ | 599 | /* enable automatic retransmission */ |
| 600 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR); | 600 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_ENABLE_AR | CONTROL_INIT); |
| 601 | 601 | ||
| 602 | if ((priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) && | 602 | if ((priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) && |
| 603 | (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)) { | 603 | (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK)) { |
| 604 | /* loopback + silent mode : useful for hot self-test */ | 604 | /* loopback + silent mode : useful for hot self-test */ |
| 605 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | 605 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); |
| 606 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK | TEST_SILENT); | 606 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK | TEST_SILENT); |
| 607 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK) { | 607 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LOOPBACK) { |
| 608 | /* loopback mode : useful for self-test function */ | 608 | /* loopback mode : useful for self-test function */ |
| 609 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | 609 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); |
| 610 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK); | 610 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_LBACK); |
| 611 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) { | 611 | } else if (priv->can.ctrlmode & CAN_CTRLMODE_LISTENONLY) { |
| 612 | /* silent mode : bus-monitoring mode */ | 612 | /* silent mode : bus-monitoring mode */ |
| 613 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST); | 613 | priv->write_reg(priv, C_CAN_CTRL_REG, CONTROL_TEST | CONTROL_INIT); |
| 614 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_SILENT); | 614 | priv->write_reg(priv, C_CAN_TEST_REG, TEST_SILENT); |
| 615 | } | 615 | } |
| 616 | 616 | ||
| @@ -340,8 +340,8 @@ static void ctucan_set_mode(struct ctucan_priv *priv, const struct can_ctrlmode · 339 unmodified lines | |||
| 340 | (mode_reg & ~REG_MODE_FDE); | 340 | (mode_reg & ~REG_MODE_FDE); |
| 341 | 341 | ||
| 342 | mode_reg = (mode->flags & CAN_CTRLMODE_PRESUME_ACK) ? | 342 | mode_reg = (mode->flags & CAN_CTRLMODE_PRESUME_ACK) ? |
| 343 | (mode_reg | REG_MODE_ACF) : | 343 | (mode_reg | REG_MODE_STM) : |
| 344 | (mode_reg & ~REG_MODE_ACF); | 344 | (mode_reg & ~REG_MODE_STM); |
| 345 | 345 | ||
| 346 | mode_reg = (mode->flags & CAN_CTRLMODE_FD_NON_ISO) ? | 346 | mode_reg = (mode->flags & CAN_CTRLMODE_FD_NON_ISO) ? |
| 347 | (mode_reg | REG_MODE_NISOFD) : | 347 | (mode_reg | REG_MODE_NISOFD) : |
| @@ -869,7 +869,7 @@ static void ctucan_err_interrupt(struct net_device *ndev, u32 isr) · 521 unmodified lines | |||
| 869 | break; | 869 | break; |
| 870 | case CAN_STATE_ERROR_ACTIVE: | 870 | case CAN_STATE_ERROR_ACTIVE: |
| 871 | if (skb) { | 871 | if (skb) { |
| 872 | cf->can_id |= CAN_ERR_CNT; | 872 | cf->can_id |= CAN_ERR_CRTL | CAN_ERR_CNT; |
| 873 | cf->data[1] = CAN_ERR_CRTL_ACTIVE; | 873 | cf->data[1] = CAN_ERR_CRTL_ACTIVE; |
| 874 | cf->data[6] = bec.txerr; | 874 | cf->data[6] = bec.txerr; |
| 875 | cf->data[7] = bec.rxerr; | 875 | cf->data[7] = bec.rxerr; |
| @@ -1136,8 +1136,12 @@ static irqreturn_t ctucan_interrupt(int irq, void *dev_id) · 260 unmodified lines | |||
| 1136 | /* Error interrupts */ | 1136 | /* Error interrupts */ |
| 1137 | if (FIELD_GET(REG_INT_STAT_EWLI, isr) || | 1137 | if (FIELD_GET(REG_INT_STAT_EWLI, isr) || |
| 1138 | FIELD_GET(REG_INT_STAT_FCSI, isr) || | 1138 | FIELD_GET(REG_INT_STAT_FCSI, isr) || |
| 1139 | FIELD_GET(REG_INT_STAT_ALI, isr)) { | 1139 | FIELD_GET(REG_INT_STAT_ALI, isr) || |
| 1140 | icr = isr & (REG_INT_STAT_EWLI | REG_INT_STAT_FCSI | REG_INT_STAT_ALI); | 1140 | FIELD_GET(REG_INT_STAT_BEI, isr)) { |
| 1141 | icr = isr & (REG_INT_STAT_EWLI | | ||
| 1142 | REG_INT_STAT_FCSI | | ||
| 1143 | REG_INT_STAT_ALI | | ||
| 1144 | REG_INT_STAT_BEI); | ||
| 1141 | 1145 | ||
| 1142 | ctucan_netdev_dbg(ndev, "some ERR interrupt: clearing 0x%08x\n", icr); | 1146 | ctucan_netdev_dbg(ndev, "some ERR interrupt: clearing 0x%08x\n", icr); |
| 1143 | ctucan_write32(priv, CTUCANFD_INT_STAT, icr); | 1147 | ctucan_write32(priv, CTUCANFD_INT_STAT, icr); |
| @@ -194,7 +194,7 @@ err_free_board: · 193 unmodified lines | |||
| 194 | pci_set_drvdata(pdev, NULL); | 194 | pci_set_drvdata(pdev, NULL); |
| 195 | kfree(bdata); | 195 | kfree(bdata); |
| 196 | err_pci_iounmap_bar0: | 196 | err_pci_iounmap_bar0: |
| 197 | pci_iounmap(pdev, cra_addr); | 197 | pci_iounmap(pdev, bar0_base); |
| 198 | err_pci_iounmap_bar1: | 198 | err_pci_iounmap_bar1: |
| 199 | pci_iounmap(pdev, addr); | 199 | pci_iounmap(pdev, addr); |
| 200 | err_release_regions: | 200 | err_release_regions: |
| @@ -266,6 +266,7 @@ static const struct pci_device_id ctucan_pci_tbl[] = { · 65 unmodified lines | |||
| 266 | CTUCAN_WITH_CTUCAN_ID)}, | 266 | CTUCAN_WITH_CTUCAN_ID)}, |
| 267 | {}, | 267 | {}, |
| 268 | }; | 268 | }; |
| 269 | MODULE_DEVICE_TABLE(pci, ctucan_pci_tbl); | ||
| 269 | 270 | ||
| 270 | static struct pci_driver ctucan_pci_driver = { | 271 | static struct pci_driver ctucan_pci_driver = { |
| 271 | .name = KBUILD_MODNAME, | 272 | .name = KBUILD_MODNAME, |
| @@ -2003,20 +2003,12 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 2002 unmodified lines | |||
| 2003 | u32 ch, sts; | 2003 | u32 ch, sts; |
| 2004 | int err; | 2004 | int err; |
| 2005 | 2005 | ||
| 2006 | err = reset_control_reset(gpriv->rstc1); | ||
| 2007 | if (err) | ||
| 2008 | return err; | ||
| 2009 | |||
| 2010 | err = reset_control_reset(gpriv->rstc2); | ||
| 2011 | if (err) | ||
| 2012 | goto fail_reset1; | ||
| 2013 | |||
| 2014 | /* Enable peripheral clock for register access */ | 2006 | /* Enable peripheral clock for register access */ |
| 2015 | err = clk_prepare_enable(gpriv->clkp); | 2007 | err = clk_prepare_enable(gpriv->clkp); |
| 2016 | if (err) { | 2008 | if (err) { |
| 2017 | dev_err(dev, "failed to enable peripheral clock: %pe\n", | 2009 | dev_err(dev, "failed to enable peripheral clock: %pe\n", |
| 2018 | ERR_PTR(err)); | 2010 | ERR_PTR(err)); |
| 2019 | goto fail_reset2; | 2011 | return err; |
| 2020 | } | 2012 | } |
| 2021 | 2013 | ||
| 2022 | /* Enable RAM clock */ | 2014 | /* Enable RAM clock */ |
| @@ -2027,10 +2019,18 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 4 unmodified lines | |||
| 2027 | goto fail_clk; | 2019 | goto fail_clk; |
| 2028 | } | 2020 | } |
| 2029 | 2021 | ||
| 2022 | err = reset_control_reset(gpriv->rstc1); | ||
| 2023 | if (err) | ||
| 2024 | goto fail_ram_clk; | ||
| 2025 | |||
| 2026 | err = reset_control_reset(gpriv->rstc2); | ||
| 2027 | if (err) | ||
| 2028 | goto fail_reset1; | ||
| 2029 | |||
| 2030 | err = rcar_canfd_reset_controller(gpriv); | 2030 | err = rcar_canfd_reset_controller(gpriv); |
| 2031 | if (err) { | 2031 | if (err) { |
| 2032 | dev_err(dev, "reset controller failed: %pe\n", ERR_PTR(err)); | 2032 | dev_err(dev, "reset controller failed: %pe\n", ERR_PTR(err)); |
| 2033 | goto fail_ram_clk; | 2033 | goto fail_reset2; |
| 2034 | } | 2034 | } |
| 2035 | 2035 | ||
| 2036 | /* Controller in Global reset & Channel reset mode */ | 2036 | /* Controller in Global reset & Channel reset mode */ |
| @@ -2068,14 +2068,14 @@ static int rcar_canfd_global_init(struct rcar_canfd_global *gpriv) · 31 unmodified lines | |||
| 2068 | 2068 | ||
| 2069 | fail_mode: | 2069 | fail_mode: |
| 2070 | rcar_canfd_disable_global_interrupts(gpriv); | 2070 | rcar_canfd_disable_global_interrupts(gpriv); |
| 2071 | fail_ram_clk: | ||
| 2072 | clk_disable_unprepare(gpriv->clk_ram); | ||
| 2073 | fail_clk: | ||
| 2074 | clk_disable_unprepare(gpriv->clkp); | ||
| 2075 | fail_reset2: | 2071 | fail_reset2: |
| 2076 | reset_control_assert(gpriv->rstc2); | 2072 | reset_control_assert(gpriv->rstc2); |
| 2077 | fail_reset1: | 2073 | fail_reset1: |
| 2078 | reset_control_assert(gpriv->rstc1); | 2074 | reset_control_assert(gpriv->rstc1); |
| 2075 | fail_ram_clk: | ||
| 2076 | clk_disable_unprepare(gpriv->clk_ram); | ||
| 2077 | fail_clk: | ||
| 2078 | clk_disable_unprepare(gpriv->clkp); | ||
| 2079 | return err; | 2079 | return err; |
| 2080 | } | 2080 | } |
| 2081 | 2081 | ||
| @@ -2090,10 +2090,10 @@ static void rcar_canfd_global_deinit(struct rcar_canfd_global *gpriv, bool full) · 8 unmodified lines | |||
| 2090 | rcar_canfd_set_bit(gpriv->base, RCANFD_GCTR, RCANFD_GCTR_GSLPR); | 2090 | rcar_canfd_set_bit(gpriv->base, RCANFD_GCTR, RCANFD_GCTR_GSLPR); |
| 2091 | } | 2091 | } |
| 2092 | 2092 | ||
| 2093 | clk_disable_unprepare(gpriv->clk_ram); | ||
| 2094 | clk_disable_unprepare(gpriv->clkp); | ||
| 2095 | reset_control_assert(gpriv->rstc2); | 2093 | reset_control_assert(gpriv->rstc2); |
| 2096 | reset_control_assert(gpriv->rstc1); | 2094 | reset_control_assert(gpriv->rstc1); |
| 2095 | clk_disable_unprepare(gpriv->clk_ram); | ||
| 2096 | clk_disable_unprepare(gpriv->clkp); | ||
| 2097 | } | 2097 | } |
| 2098 | 2098 | ||
| 2099 | static int rcar_canfd_probe(struct platform_device *pdev) | 2099 | static int rcar_canfd_probe(struct platform_device *pdev) |
| @@ -91,12 +91,12 @@ int softing_bootloader_command(struct softing *card, int16_t cmd, · 90 unmodified lines | |||
| 91 | return ret; | 91 | return ret; |
| 92 | } | 92 | } |
| 93 | 93 | ||
| 94 | static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, | 94 | static int fw_parse(const u8 **pmem, const u8 *limit, u16 *ptype, |
| 95 | uint16_t *plen, const uint8_t **pdat) | 95 | u32 *paddr, u16 *plen, const u8 **pdat) |
| 96 | { | 96 | { |
| 97 | uint16_t checksum[2]; | 97 | uint16_t checksum[2]; |
| 98 | const uint8_t *mem; | 98 | const u8 *mem; |
| 99 | const uint8_t *end; | 99 | const u8 *record_end; |
| 100 | 100 | ||
| 101 | /* | 101 | /* |
| 102 | * firmware records are a binary, unaligned stream composed of: | 102 | * firmware records are a binary, unaligned stream composed of: |
| @@ -114,14 +114,21 @@ static int fw_parse(const uint8_t **pmem, uint16_t *ptype, uint32_t *paddr, · 11 unmodified lines | |||
| 114 | * endianness & alignment. | 114 | * endianness & alignment. |
| 115 | */ | 115 | */ |
| 116 | mem = *pmem; | 116 | mem = *pmem; |
| 117 | /* A record needs an 8-byte prefix and a 2-byte checksum. */ | ||
| 118 | if (mem > limit || limit - mem < 10) | ||
| 119 | return -EINVAL; | ||
| 120 | |||
| 117 | *ptype = le16_to_cpup((void *)&mem[0]); | 121 | *ptype = le16_to_cpup((void *)&mem[0]); |
| 118 | *paddr = le32_to_cpup((void *)&mem[2]); | 122 | *paddr = le32_to_cpup((void *)&mem[2]); |
| 119 | *plen = le16_to_cpup((void *)&mem[6]); | 123 | *plen = le16_to_cpup((void *)&mem[6]); |
| 124 | if (*plen > limit - mem - 10) | ||
| 125 | return -EINVAL; | ||
| 126 | |||
| 120 | *pdat = &mem[8]; | 127 | *pdat = &mem[8]; |
| 121 | /* verify checksum */ | 128 | /* verify checksum */ |
| 122 | end = &mem[8 + *plen]; | 129 | record_end = &mem[8 + *plen]; |
| 123 | checksum[0] = le16_to_cpup((void *)end); | 130 | checksum[0] = le16_to_cpup((void *)record_end); |
| 124 | for (checksum[1] = 0; mem < end; ++mem) | 131 | for (checksum[1] = 0; mem < record_end; ++mem) |
| 125 | checksum[1] += *mem; | 132 | checksum[1] += *mem; |
| 126 | if (checksum[0] != checksum[1]) | 133 | if (checksum[0] != checksum[1]) |
| 127 | return -EINVAL; | 134 | return -EINVAL; |
| @@ -139,6 +146,7 @@ int softing_load_fw(const char *file, struct softing *card, · 11 unmodified lines | |||
| 139 | uint16_t type, len; | 146 | uint16_t type, len; |
| 140 | uint32_t addr; | 147 | uint32_t addr; |
| 141 | uint8_t *buf = NULL, *new_buf; | 148 | uint8_t *buf = NULL, *new_buf; |
| 149 | s64 dpram_offset; | ||
| 142 | int buflen = 0; | 150 | int buflen = 0; |
| 143 | int8_t type_end = 0; | 151 | int8_t type_end = 0; |
| 144 | 152 | ||
| @@ -153,7 +161,7 @@ int softing_load_fw(const char *file, struct softing *card, · 8 unmodified lines | |||
| 153 | mem = fw->data; | 161 | mem = fw->data; |
| 154 | end = &mem[fw->size]; | 162 | end = &mem[fw->size]; |
| 155 | /* look for header record */ | 163 | /* look for header record */ |
| 156 | ret = fw_parse(&mem, &type, &addr, &len, &dat); | 164 | ret = fw_parse(&mem, end, &type, &addr, &len, &dat); |
| 157 | if (ret < 0) | 165 | if (ret < 0) |
| 158 | goto failed; | 166 | goto failed; |
| 159 | if (type != 0xffff) | 167 | if (type != 0xffff) |
| @@ -164,7 +172,7 @@ int softing_load_fw(const char *file, struct softing *card, · 4 unmodified lines | |||
| 164 | } | 172 | } |
| 165 | /* ok, we had a header */ | 173 | /* ok, we had a header */ |
| 166 | while (mem < end) { | 174 | while (mem < end) { |
| 167 | ret = fw_parse(&mem, &type, &addr, &len, &dat); | 175 | ret = fw_parse(&mem, end, &type, &addr, &len, &dat); |
| 168 | if (ret < 0) | 176 | if (ret < 0) |
| 169 | goto failed; | 177 | goto failed; |
| 170 | if (type == 3) { | 178 | if (type == 3) { |
| @@ -179,9 +187,13 @@ int softing_load_fw(const char *file, struct softing *card, · 8 unmodified lines | |||
| 179 | goto failed; | 187 | goto failed; |
| 180 | } | 188 | } |
| 181 | 189 | ||
| 182 | if ((addr + len + offset) > size) | 190 | dpram_offset = (s64)addr + offset; |
| 191 | if (dpram_offset < 0 || dpram_offset > size || | ||
| 192 | len > size - dpram_offset) { | ||
| 193 | ret = -EINVAL; | ||
| 183 | goto failed; | 194 | goto failed; |
| 184 | memcpy_toio(&dpram[addr + offset], dat, len); | 195 | } |
| 196 | memcpy_toio(&dpram[dpram_offset], dat, len); | ||
| 185 | /* be sure to flush caches from IO space */ | 197 | /* be sure to flush caches from IO space */ |
| 186 | mb(); | 198 | mb(); |
| 187 | if (len > buflen) { | 199 | if (len > buflen) { |
| @@ -195,7 +207,7 @@ int softing_load_fw(const char *file, struct softing *card, · 7 unmodified lines | |||
| 195 | buf = new_buf; | 207 | buf = new_buf; |
| 196 | } | 208 | } |
| 197 | /* verify record data */ | 209 | /* verify record data */ |
| 198 | memcpy_fromio(buf, &dpram[addr + offset], len); | 210 | memcpy_fromio(buf, &dpram[dpram_offset], len); |
| 199 | if (memcmp(buf, dat, len)) { | 211 | if (memcmp(buf, dat, len)) { |
| 200 | /* is not ok */ | 212 | /* is not ok */ |
| 201 | dev_alert(&card->pdev->dev, "DPRAM readback failed\n"); | 213 | dev_alert(&card->pdev->dev, "DPRAM readback failed\n"); |
| @@ -237,7 +249,7 @@ int softing_load_app_fw(const char *file, struct softing *card) · 35 unmodified lines | |||
| 237 | mem = fw->data; | 249 | mem = fw->data; |
| 238 | end = &mem[fw->size]; | 250 | end = &mem[fw->size]; |
| 239 | /* look for header record */ | 251 | /* look for header record */ |
| 240 | ret = fw_parse(&mem, &type, &addr, &len, &dat); | 252 | ret = fw_parse(&mem, end, &type, &addr, &len, &dat); |
| 241 | if (ret) | 253 | if (ret) |
| 242 | goto failed; | 254 | goto failed; |
| 243 | ret = -EINVAL; | 255 | ret = -EINVAL; |
| @@ -253,7 +265,7 @@ int softing_load_app_fw(const char *file, struct softing *card) · 9 unmodified lines | |||
| 253 | } | 265 | } |
| 254 | /* ok, we had a header */ | 266 | /* ok, we had a header */ |
| 255 | while (mem < end) { | 267 | while (mem < end) { |
| 256 | ret = fw_parse(&mem, &type, &addr, &len, &dat); | 268 | ret = fw_parse(&mem, end, &type, &addr, &len, &dat); |
| 257 | if (ret) | 269 | if (ret) |
| 258 | goto failed; | 270 | goto failed; |
| 259 | 271 | ||
| @@ -279,6 +291,12 @@ int softing_load_app_fw(const char *file, struct softing *card) · 19 unmodified lines | |||
| 279 | /* work in 16bit (target) */ | 291 | /* work in 16bit (target) */ |
| 280 | sum &= 0xffff; | 292 | sum &= 0xffff; |
| 281 | 293 | ||
| 294 | if (card->pdat->app.offs > card->dpram_size || | ||
| 295 | len > card->dpram_size - card->pdat->app.offs) { | ||
| 296 | ret = -EINVAL; | ||
| 297 | goto failed; | ||
| 298 | } | ||
| 299 | |||
| 282 | memcpy_toio(&card->dpram[card->pdat->app.offs], dat, len); | 300 | memcpy_toio(&card->dpram[card->pdat->app.offs], dat, len); |
| 283 | iowrite32(card->pdat->app.offs + card->pdat->app.addr, | 301 | iowrite32(card->pdat->app.offs + card->pdat->app.addr, |
| 284 | &card->dpram[DPRAM_COMMAND + 2]); | 302 | &card->dpram[DPRAM_COMMAND + 2]); |
| @@ -409,6 +409,40 @@ static void ems_usb_rx_err(struct ems_usb *dev, struct ems_cpc_msg *msg) · 408 unmodified lines | |||
| 409 | netif_rx(skb); | 409 | netif_rx(skb); |
| 410 | } | 410 | } |
| 411 | 411 | ||
| 412 | static bool ems_usb_rx_msg_len_valid(struct ems_cpc_msg *msg) | ||
| 413 | { | ||
| 414 | size_t len = msg->length; | ||
| 415 | size_t can_len; | ||
| 416 | |||
| 417 | switch (msg->type) { | ||
| 418 | case CPC_MSG_TYPE_CAN_STATE: | ||
| 419 | return len >= sizeof(msg->msg.can_state); | ||
| 420 | |||
| 421 | case CPC_MSG_TYPE_CAN_FRAME: | ||
| 422 | case CPC_MSG_TYPE_EXT_CAN_FRAME: | ||
| 423 | case CPC_MSG_TYPE_RTR_FRAME: | ||
| 424 | case CPC_MSG_TYPE_EXT_RTR_FRAME: | ||
| 425 | if (len < CPC_CAN_MSG_MIN_SIZE) | ||
| 426 | return false; | ||
| 427 | |||
| 428 | if (msg->type == CPC_MSG_TYPE_RTR_FRAME || | ||
| 429 | msg->type == CPC_MSG_TYPE_EXT_RTR_FRAME) | ||
| 430 | return true; | ||
| 431 | |||
| 432 | can_len = can_cc_dlc2len(msg->msg.can_msg.length & 0xf); | ||
| 433 | return len >= CPC_CAN_MSG_MIN_SIZE + can_len; | ||
| 434 | |||
| 435 | case CPC_MSG_TYPE_CAN_FRAME_ERROR: | ||
| 436 | return len >= sizeof(msg->msg.error); | ||
| 437 | |||
| 438 | case CPC_MSG_TYPE_OVERRUN: | ||
| 439 | return len >= sizeof(msg->msg.overrun); | ||
| 440 | |||
| 441 | default: | ||
| 442 | return true; | ||
| 443 | } | ||
| 444 | } | ||
| 445 | |||
| 412 | /* | 446 | /* |
| 413 | * callback for bulk IN urb | 447 | * callback for bulk IN urb |
| 414 | */ | 448 | */ |
| @@ -451,6 +485,15 @@ static void ems_usb_read_bulk_callback(struct urb *urb) · 36 unmodified lines | |||
| 451 | } | 485 | } |
| 452 | 486 | ||
| 453 | msg = (struct ems_cpc_msg *)&ibuf[start]; | 487 | msg = (struct ems_cpc_msg *)&ibuf[start]; |
| 488 | if (msg->length > | ||
| 489 | urb->actual_length - start - CPC_MSG_HEADER_LEN) { | ||
| 490 | netdev_err(netdev, "format error\n"); | ||
| 491 | break; | ||
| 492 | } | ||
| 493 | if (!ems_usb_rx_msg_len_valid(msg)) { | ||
| 494 | netdev_err(netdev, "format error\n"); | ||
| 495 | break; | ||
| 496 | } | ||
| 454 | 497 | ||
| 455 | switch (msg->type) { | 498 | switch (msg->type) { |
| 456 | case CPC_MSG_TYPE_CAN_STATE: | 499 | case CPC_MSG_TYPE_CAN_STATE: |
| @@ -1476,7 +1476,6 @@ static void es58x_read_bulk_callback(struct urb *urb) · 1475 unmodified lines | |||
| 1476 | dev_err_ratelimited(dev, | 1476 | dev_err_ratelimited(dev, |
| 1477 | "Failed resubmitting read bulk urb: %pe\n", | 1477 | "Failed resubmitting read bulk urb: %pe\n", |
| 1478 | ERR_PTR(ret)); | 1478 | ERR_PTR(ret)); |
| 1479 | return; | ||
| 1480 | 1479 | ||
| 1481 | free_urb: | 1480 | free_urb: |
| 1482 | usb_free_coherent(urb->dev, urb->transfer_buffer_length, | 1481 | usb_free_coherent(urb->dev, urb->transfer_buffer_length, |
| @@ -674,7 +674,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) · 673 unmodified lines | |||
| 674 | if (hf->flags & GS_CAN_FLAG_FD) { | 674 | if (hf->flags & GS_CAN_FLAG_FD) { |
| 675 | skb = alloc_canfd_skb(netdev, &cfd); | 675 | skb = alloc_canfd_skb(netdev, &cfd); |
| 676 | if (!skb) | 676 | if (!skb) |
| 677 | return; | 677 | goto resubmit_urb; |
| 678 | 678 | ||
| 679 | cfd->can_id = le32_to_cpu(hf->can_id); | 679 | cfd->can_id = le32_to_cpu(hf->can_id); |
| 680 | cfd->len = data_length; | 680 | cfd->len = data_length; |
| @@ -687,7 +687,7 @@ static void gs_usb_receive_bulk_callback(struct urb *urb) · 6 unmodified lines | |||
| 687 | } else { | 687 | } else { |
| 688 | skb = alloc_can_skb(netdev, &cf); | 688 | skb = alloc_can_skb(netdev, &cf); |
| 689 | if (!skb) | 689 | if (!skb) |
| 690 | return; | 690 | goto resubmit_urb; |
| 691 | 691 | ||
| 692 | cf->can_id = le32_to_cpu(hf->can_id); | 692 | cf->can_id = le32_to_cpu(hf->can_id); |
| 693 | can_frame_set_cc_len(cf, hf->can_dlc, dev->can.ctrlmode); | 693 | can_frame_set_cc_len(cf, hf->can_dlc, dev->can.ctrlmode); |
| @@ -1626,6 +1626,7 @@ static int kvaser_usb_hydra_get_busparams(struct kvaser_usb_net_priv *priv, · 1625 unmodified lines | |||
| 1626 | reinit_completion(&priv->get_busparams_comp); | 1626 | reinit_completion(&priv->get_busparams_comp); |
| 1627 | 1627 | ||
| 1628 | err = kvaser_usb_send_cmd(dev, cmd, cmd_len); | 1628 | err = kvaser_usb_send_cmd(dev, cmd, cmd_len); |
| 1629 | kfree(cmd); | ||
| 1629 | if (err) | 1630 | if (err) |
| 1630 | return err; | 1631 | return err; |
| 1631 | 1632 | ||
| @@ -691,13 +691,22 @@ static int kvaser_usb_leaf_wait_cmd(const struct kvaser_usb *dev, u8 id, · 690 unmodified lines | |||
| 691 | continue; | 691 | continue; |
| 692 | } | 692 | } |
| 693 | 693 | ||
| 694 | if (pos + tmp->len > actual_len) { | 694 | if (tmp->len < CMD_HEADER_LEN || |
| 695 | tmp->len > actual_len - pos) { | ||
| 695 | dev_err_ratelimited(&dev->intf->dev, | 696 | dev_err_ratelimited(&dev->intf->dev, |
| 696 | "Format error\n"); | 697 | "Format error\n"); |
| 697 | break; | 698 | break; |
| 698 | } | 699 | } |
| 699 | 700 | ||
| 700 | if (tmp->id == id) { | 701 | if (tmp->id == id) { |
| 702 | if (tmp->len > sizeof(*cmd)) { | ||
| 703 | dev_err_ratelimited(&dev->intf->dev, | ||
| 704 | "Received command %u too large (%u)\n", | ||
| 705 | tmp->id, tmp->len); | ||
| 706 | err = -EIO; | ||
| 707 | goto end; | ||
| 708 | } | ||
| 709 | |||
| 701 | memcpy(cmd, tmp, tmp->len); | 710 | memcpy(cmd, tmp, tmp->len); |
| 702 | goto end; | 711 | goto end; |
| 703 | } | 712 | } |
| @@ -1737,7 +1746,7 @@ static void kvaser_usb_leaf_read_bulk_callback(struct kvaser_usb *dev, · 1033 unmodified lines | |||
| 1737 | continue; | 1746 | continue; |
| 1738 | } | 1747 | } |
| 1739 | 1748 | ||
| 1740 | if (pos + cmd->len > len) { | 1749 | if (cmd->len < CMD_HEADER_LEN || cmd->len > len - pos) { |
| 1741 | dev_err_ratelimited(&dev->intf->dev, "Format error\n"); | 1750 | dev_err_ratelimited(&dev->intf->dev, "Format error\n"); |
| 1742 | break; | 1751 | break; |
| 1743 | } | 1752 | } |
| @@ -470,7 +470,6 @@ static int peak_usb_start(struct peak_usb_device *dev) · 469 unmodified lines | |||
| 470 | netif_device_detach(dev->netdev); | 470 | netif_device_detach(dev->netdev); |
| 471 | 471 | ||
| 472 | usb_unanchor_urb(urb); | 472 | usb_unanchor_urb(urb); |
| 473 | kfree(buf); | ||
| 474 | usb_free_urb(urb); | 473 | usb_free_urb(urb); |
| 475 | break; | 474 | break; |
| 476 | } | 475 | } |
| @@ -566,6 +566,13 @@ static int pcan_usb_fd_decode_canmsg(struct pcan_usb_fd_if *usb_if, · 565 unmodified lines | |||
| 566 | dev->can.ctrlmode); | 566 | dev->can.ctrlmode); |
| 567 | } | 567 | } |
| 568 | 568 | ||
| 569 | if (!(rx_msg_flags & PUCAN_MSG_RTR) && | ||
| 570 | le16_to_cpu(rx_msg->size) - offsetof(struct pucan_rx_msg, d) < | ||
| 571 | cfd->len) { | ||
| 572 | kfree_skb(skb); | ||
| 573 | return -EBADMSG; | ||
| 574 | } | ||
| 575 | |||
| 569 | cfd->can_id = le32_to_cpu(rm->can_id); | 576 | cfd->can_id = le32_to_cpu(rm->can_id); |
| 570 | 577 | ||
| 571 | if (rx_msg_flags & PUCAN_MSG_EXT_ID) | 578 | if (rx_msg_flags & PUCAN_MSG_EXT_ID) |
| @@ -714,6 +721,24 @@ static void pcan_usb_fd_decode_ts(struct pcan_usb_fd_if *usb_if, · 142 unmodified lines | |||
| 714 | peak_usb_set_ts_now(&usb_if->time_ref, le32_to_cpu(ts->ts_low)); | 721 | peak_usb_set_ts_now(&usb_if->time_ref, le32_to_cpu(ts->ts_low)); |
| 715 | } | 722 | } |
| 716 | 723 | ||
| 724 | static size_t pcan_usb_fd_rx_msg_min_size(u16 rx_msg_type) | ||
| 725 | { | ||
| 726 | switch (rx_msg_type) { | ||
| 727 | case PUCAN_MSG_CAN_RX: | ||
| 728 | return offsetof(struct pucan_rx_msg, d); | ||
| 729 | case PCAN_UFD_MSG_CALIBRATION: | ||
| 730 | return sizeof(struct pcan_ufd_ts_msg); | ||
| 731 | case PUCAN_MSG_ERROR: | ||
| 732 | return sizeof(struct pucan_error_msg); | ||
| 733 | case PUCAN_MSG_STATUS: | ||
| 734 | return sizeof(struct pucan_status_msg); | ||
| 735 | case PCAN_UFD_MSG_OVERRUN: | ||
| 736 | return sizeof(struct pcan_ufd_ovr_msg); | ||
| 737 | default: | ||
| 738 | return sizeof(struct pucan_msg); | ||
| 739 | } | ||
| 740 | } | ||
| 741 | |||
| 717 | /* callback for bulk IN urb */ | 742 | /* callback for bulk IN urb */ |
| 718 | static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) | 743 | static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) |
| 719 | { | 744 | { |
| @@ -728,6 +753,12 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) · 8 unmodified lines | |||
| 728 | msg_end = urb->transfer_buffer + urb->actual_length; | 753 | msg_end = urb->transfer_buffer + urb->actual_length; |
| 729 | for (; msg_ptr < msg_end;) { | 754 | for (; msg_ptr < msg_end;) { |
| 730 | u16 rx_msg_type, rx_msg_size; | 755 | u16 rx_msg_type, rx_msg_size; |
| 756 | size_t rx_msg_min_size; | ||
| 757 | |||
| 758 | if (msg_end - msg_ptr < sizeof(*rx_msg)) { | ||
| 759 | err = -EBADMSG; | ||
| 760 | break; | ||
| 761 | } | ||
| 731 | 762 | ||
| 732 | rx_msg = (struct pucan_msg *)msg_ptr; | 763 | rx_msg = (struct pucan_msg *)msg_ptr; |
| 733 | if (!rx_msg->size) { | 764 | if (!rx_msg->size) { |
| @@ -739,13 +770,20 @@ static int pcan_usb_fd_decode_buf(struct peak_usb_device *dev, struct urb *urb) · 5 unmodified lines | |||
| 739 | rx_msg_type = le16_to_cpu(rx_msg->type); | 770 | rx_msg_type = le16_to_cpu(rx_msg->type); |
| 740 | 771 | ||
| 741 | /* check if the record goes out of current packet */ | 772 | /* check if the record goes out of current packet */ |
| 742 | if (msg_ptr + rx_msg_size > msg_end) { | 773 | if (rx_msg_size > msg_end - msg_ptr) { |
| 743 | netdev_err(netdev, | 774 | netdev_err(netdev, |
| 744 | "got frag rec: should inc usb rx buf sze\n"); | 775 | "got frag rec: should inc usb rx buf sze\n"); |
| 745 | err = -EBADMSG; | 776 | err = -EBADMSG; |
| 746 | break; | 777 | break; |
| 747 | } | 778 | } |
| 748 | 779 | ||
| 780 | rx_msg_min_size = pcan_usb_fd_rx_msg_min_size(rx_msg_type); | ||
| 781 | if (rx_msg_size < rx_msg_min_size) { | ||
| 782 | netdev_err(netdev, "got short rec\n"); | ||
| 783 | err = -EBADMSG; | ||
| 784 | break; | ||
| 785 | } | ||
| 786 | |||
| 749 | switch (rx_msg_type) { | 787 | switch (rx_msg_type) { |
| 750 | case PUCAN_MSG_CAN_RX: | 788 | case PUCAN_MSG_CAN_RX: |
| 751 | err = pcan_usb_fd_decode_canmsg(usb_if, rx_msg); | 789 | err = pcan_usb_fd_decode_canmsg(usb_if, rx_msg); |
| @@ -534,12 +534,18 @@ static int pcan_usb_pro_handle_canmsg(struct pcan_usb_pro_interface *usb_if, · 533 unmodified lines | |||
| 534 | struct pcan_usb_pro_rxmsg *rx) | 534 | struct pcan_usb_pro_rxmsg *rx) |
| 535 | { | 535 | { |
| 536 | const unsigned int ctrl_idx = (rx->len >> 4) & 0x0f; | 536 | const unsigned int ctrl_idx = (rx->len >> 4) & 0x0f; |
| 537 | struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; | 537 | struct peak_usb_device *dev; |
| 538 | struct net_device *netdev = dev->netdev; | 538 | struct net_device *netdev; |
| 539 | struct can_frame *can_frame; | 539 | struct can_frame *can_frame; |
| 540 | struct sk_buff *skb; | 540 | struct sk_buff *skb; |
| 541 | struct skb_shared_hwtstamps *hwts; | 541 | struct skb_shared_hwtstamps *hwts; |
| 542 | 542 | ||
| 543 | if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) | ||
| 544 | return -EINVAL; | ||
| 545 | |||
| 546 | dev = usb_if->dev[ctrl_idx]; | ||
| 547 | netdev = dev->netdev; | ||
| 548 | |||
| 543 | skb = alloc_can_skb(netdev, &can_frame); | 549 | skb = alloc_can_skb(netdev, &can_frame); |
| 544 | if (!skb) | 550 | if (!skb) |
| 545 | return -ENOMEM; | 551 | return -ENOMEM; |
| @@ -573,14 +579,20 @@ static int pcan_usb_pro_handle_error(struct pcan_usb_pro_interface *usb_if, · 27 unmodified lines | |||
| 573 | { | 579 | { |
| 574 | const u16 raw_status = le16_to_cpu(er->status); | 580 | const u16 raw_status = le16_to_cpu(er->status); |
| 575 | const unsigned int ctrl_idx = (er->channel >> 4) & 0x0f; | 581 | const unsigned int ctrl_idx = (er->channel >> 4) & 0x0f; |
| 576 | struct peak_usb_device *dev = usb_if->dev[ctrl_idx]; | 582 | struct peak_usb_device *dev; |
| 577 | struct net_device *netdev = dev->netdev; | 583 | struct net_device *netdev; |
| 578 | struct can_frame *can_frame; | 584 | struct can_frame *can_frame; |
| 579 | enum can_state new_state = CAN_STATE_ERROR_ACTIVE; | 585 | enum can_state new_state = CAN_STATE_ERROR_ACTIVE; |
| 580 | u8 err_mask = 0; | 586 | u8 err_mask = 0; |
| 581 | struct sk_buff *skb; | 587 | struct sk_buff *skb; |
| 582 | struct skb_shared_hwtstamps *hwts; | 588 | struct skb_shared_hwtstamps *hwts; |
| 583 | 589 | ||
| 590 | if (ctrl_idx >= ARRAY_SIZE(usb_if->dev)) | ||
| 591 | return -EINVAL; | ||
| 592 | |||
| 593 | dev = usb_if->dev[ctrl_idx]; | ||
| 594 | netdev = dev->netdev; | ||
| 595 | |||
| 584 | /* nothing should be sent while in BUS_OFF state */ | 596 | /* nothing should be sent while in BUS_OFF state */ |
| 585 | if (dev->can.state == CAN_STATE_BUS_OFF) | 597 | if (dev->can.state == CAN_STATE_BUS_OFF) |
| 586 | return 0; | 598 | return 0; |
| @@ -55,8 +55,15 @@ mt7530_regmap_read(void *context, unsigned int reg, unsigned int *val) · 54 unmodified lines | |||
| 55 | if (ret < 0) | 55 | if (ret < 0) |
| 56 | return ret; | 56 | return ret; |
| 57 | 57 | ||
| 58 | lo = bus->read(bus, priv->mdiodev->addr, r); | 58 | ret = bus->read(bus, priv->mdiodev->addr, r); |
| 59 | hi = bus->read(bus, priv->mdiodev->addr, 0x10); | 59 | if (ret < 0) |
| 60 | return ret; | ||
| 61 | lo = ret; | ||
| 62 | |||
| 63 | ret = bus->read(bus, priv->mdiodev->addr, 0x10); | ||
| 64 | if (ret < 0) | ||
| 65 | return ret; | ||
| 66 | hi = ret; | ||
| 60 | 67 | ||
| 61 | *val = (hi << 16) | (lo & 0xffff); | 68 | *val = (hi << 16) | (lo & 0xffff); |
| 62 | 69 | ||
| @@ -191,12 +191,6 @@ mt7530_write(struct mt7530_priv *priv, u32 reg, u32 val) · 190 unmodified lines | |||
| 191 | mt7530_mutex_unlock(priv); | 191 | mt7530_mutex_unlock(priv); |
| 192 | } | 192 | } |
| 193 | 193 | ||
| 194 | static u32 | ||
| 195 | _mt7530_unlocked_read(struct mt7530_dummy_poll *p) | ||
| 196 | { | ||
| 197 | return mt7530_mii_read(p->priv, p->reg); | ||
| 198 | } | ||
| 199 | |||
| 200 | static u32 | 194 | static u32 |
| 201 | _mt7530_read(struct mt7530_dummy_poll *p) | 195 | _mt7530_read(struct mt7530_dummy_poll *p) |
| 202 | { | 196 | { |
| @@ -248,15 +242,20 @@ mt7530_fdb_cmd(struct mt7530_priv *priv, enum mt7530_fdb_cmd cmd, u32 *rsp) · 45 unmodified lines | |||
| 248 | { | 242 | { |
| 249 | u32 val; | 243 | u32 val; |
| 250 | int ret; | 244 | int ret; |
| 251 | struct mt7530_dummy_poll p; | ||
| 252 | 245 | ||
| 253 | /* Set the command operating upon the MAC address entries */ | 246 | /* Set the command operating upon the MAC address entries */ |
| 254 | val = ATC_BUSY | ATC_MAT(0) | cmd; | 247 | val = ATC_BUSY | ATC_MAT(0) | cmd; |
| 255 | mt7530_write(priv, MT7530_ATC, val); | 248 | mt7530_write(priv, MT7530_ATC, val); |
| 256 | 249 | ||
| 257 | INIT_MT7530_DUMMY_POLL(&p, priv, MT7530_ATC); | 250 | mt7530_mutex_lock(priv); |
| 258 | ret = readx_poll_timeout(_mt7530_read, &p, val, | 251 | |
| 259 | !(val & ATC_BUSY), 20, 20000); | 252 | ret = regmap_read_poll_timeout(priv->regmap, MT7530_ATC, val, |
| 253 | !(val & ATC_BUSY), 20, 20000); | ||
| 254 | if (!ret) | ||
| 255 | ret = regmap_read(priv->regmap, MT7530_ATC, &val); | ||
| 256 | |||
| 257 | mt7530_mutex_unlock(priv); | ||
| 258 | |||
| 260 | if (ret < 0) { | 259 | if (ret < 0) { |
| 261 | dev_err(priv->dev, "reset timeout\n"); | 260 | dev_err(priv->dev, "reset timeout\n"); |
| 262 | return ret; | 261 | return ret; |
| @@ -265,7 +264,6 @@ mt7530_fdb_cmd(struct mt7530_priv *priv, enum mt7530_fdb_cmd cmd, u32 *rsp) · 2 unmodified lines | |||
| 265 | /* Additional sanity for read command if the specified | 264 | /* Additional sanity for read command if the specified |
| 266 | * entry is invalid | 265 | * entry is invalid |
| 267 | */ | 266 | */ |
| 268 | val = mt7530_read(priv, MT7530_ATC); | ||
| 269 | if ((cmd == MT7530_FDB_READ) && (val & ATC_INVALID)) | 267 | if ((cmd == MT7530_FDB_READ) && (val & ATC_INVALID)) |
| 270 | return -EINVAL; | 268 | return -EINVAL; |
| 271 | 269 | ||
| @@ -549,16 +547,13 @@ static int · 277 unmodified lines | |||
| 549 | mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, | 547 | mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, |
| 550 | int regnum) | 548 | int regnum) |
| 551 | { | 549 | { |
| 552 | struct mt7530_dummy_poll p; | ||
| 553 | u32 reg, val; | 550 | u32 reg, val; |
| 554 | int ret; | 551 | int ret; |
| 555 | 552 | ||
| 556 | INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | ||
| 557 | |||
| 558 | mt7530_mutex_lock(priv); | 553 | mt7530_mutex_lock(priv); |
| 559 | 554 | ||
| 560 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 555 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 561 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 556 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 562 | if (ret < 0) { | 557 | if (ret < 0) { |
| 563 | dev_err(priv->dev, "poll timeout\n"); | 558 | dev_err(priv->dev, "poll timeout\n"); |
| 564 | goto out; | 559 | goto out; |
| @@ -568,8 +563,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | |||
| 568 | MT7531_MDIO_DEV_ADDR(devad) | regnum; | 563 | MT7531_MDIO_DEV_ADDR(devad) | regnum; |
| 569 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); | 564 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 570 | 565 | ||
| 571 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 566 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 572 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 567 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 573 | if (ret < 0) { | 568 | if (ret < 0) { |
| 574 | dev_err(priv->dev, "poll timeout\n"); | 569 | dev_err(priv->dev, "poll timeout\n"); |
| 575 | goto out; | 570 | goto out; |
| @@ -579,8 +574,8 @@ mt7531_ind_c45_phy_read(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | |||
| 579 | MT7531_MDIO_DEV_ADDR(devad); | 574 | MT7531_MDIO_DEV_ADDR(devad); |
| 580 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); | 575 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 581 | 576 | ||
| 582 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 577 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 583 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 578 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 584 | if (ret < 0) { | 579 | if (ret < 0) { |
| 585 | dev_err(priv->dev, "poll timeout\n"); | 580 | dev_err(priv->dev, "poll timeout\n"); |
| 586 | goto out; | 581 | goto out; |
| @@ -597,16 +592,13 @@ static int · 10 unmodified lines | |||
| 597 | mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, | 592 | mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, |
| 598 | int regnum, u16 data) | 593 | int regnum, u16 data) |
| 599 | { | 594 | { |
| 600 | struct mt7530_dummy_poll p; | ||
| 601 | u32 val, reg; | 595 | u32 val, reg; |
| 602 | int ret; | 596 | int ret; |
| 603 | 597 | ||
| 604 | INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | ||
| 605 | |||
| 606 | mt7530_mutex_lock(priv); | 598 | mt7530_mutex_lock(priv); |
| 607 | 599 | ||
| 608 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 600 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 609 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 601 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 610 | if (ret < 0) { | 602 | if (ret < 0) { |
| 611 | dev_err(priv->dev, "poll timeout\n"); | 603 | dev_err(priv->dev, "poll timeout\n"); |
| 612 | goto out; | 604 | goto out; |
| @@ -616,8 +608,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | |||
| 616 | MT7531_MDIO_DEV_ADDR(devad) | regnum; | 608 | MT7531_MDIO_DEV_ADDR(devad) | regnum; |
| 617 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); | 609 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 618 | 610 | ||
| 619 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 611 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 620 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 612 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 621 | if (ret < 0) { | 613 | if (ret < 0) { |
| 622 | dev_err(priv->dev, "poll timeout\n"); | 614 | dev_err(priv->dev, "poll timeout\n"); |
| 623 | goto out; | 615 | goto out; |
| @@ -627,8 +619,8 @@ mt7531_ind_c45_phy_write(struct mt7530_priv *priv, int port, int devad, · 3 unmodified lines | |||
| 627 | MT7531_MDIO_DEV_ADDR(devad) | data; | 619 | MT7531_MDIO_DEV_ADDR(devad) | data; |
| 628 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); | 620 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 629 | 621 | ||
| 630 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 622 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 631 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 623 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 632 | if (ret < 0) { | 624 | if (ret < 0) { |
| 633 | dev_err(priv->dev, "poll timeout\n"); | 625 | dev_err(priv->dev, "poll timeout\n"); |
| 634 | goto out; | 626 | goto out; |
| @@ -643,16 +635,13 @@ out: · 8 unmodified lines | |||
| 643 | static int | 635 | static int |
| 644 | mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) | 636 | mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) |
| 645 | { | 637 | { |
| 646 | struct mt7530_dummy_poll p; | ||
| 647 | int ret; | 638 | int ret; |
| 648 | u32 val; | 639 | u32 val; |
| 649 | 640 | ||
| 650 | INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | ||
| 651 | |||
| 652 | mt7530_mutex_lock(priv); | 641 | mt7530_mutex_lock(priv); |
| 653 | 642 | ||
| 654 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 643 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 655 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 644 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 656 | if (ret < 0) { | 645 | if (ret < 0) { |
| 657 | dev_err(priv->dev, "poll timeout\n"); | 646 | dev_err(priv->dev, "poll timeout\n"); |
| 658 | goto out; | 647 | goto out; |
| @@ -663,8 +652,8 @@ mt7531_ind_c22_phy_read(struct mt7530_priv *priv, int port, int regnum) · 4 unmodified lines | |||
| 663 | 652 | ||
| 664 | mt7530_mii_write(priv, MT7531_PHY_IAC, val | MT7531_PHY_ACS_ST); | 653 | mt7530_mii_write(priv, MT7531_PHY_IAC, val | MT7531_PHY_ACS_ST); |
| 665 | 654 | ||
| 666 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, val, | 655 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, val, |
| 667 | !(val & MT7531_PHY_ACS_ST), 20, 100000); | 656 | !(val & MT7531_PHY_ACS_ST), 20, 100000); |
| 668 | if (ret < 0) { | 657 | if (ret < 0) { |
| 669 | dev_err(priv->dev, "poll timeout\n"); | 658 | dev_err(priv->dev, "poll timeout\n"); |
| 670 | goto out; | 659 | goto out; |
| @@ -681,16 +670,13 @@ static int · 10 unmodified lines | |||
| 681 | mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, | 670 | mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, |
| 682 | u16 data) | 671 | u16 data) |
| 683 | { | 672 | { |
| 684 | struct mt7530_dummy_poll p; | ||
| 685 | int ret; | 673 | int ret; |
| 686 | u32 reg; | 674 | u32 reg; |
| 687 | 675 | ||
| 688 | INIT_MT7530_DUMMY_POLL(&p, priv, MT7531_PHY_IAC); | ||
| 689 | |||
| 690 | mt7530_mutex_lock(priv); | 676 | mt7530_mutex_lock(priv); |
| 691 | 677 | ||
| 692 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, | 678 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, |
| 693 | !(reg & MT7531_PHY_ACS_ST), 20, 100000); | 679 | !(reg & MT7531_PHY_ACS_ST), 20, 100000); |
| 694 | if (ret < 0) { | 680 | if (ret < 0) { |
| 695 | dev_err(priv->dev, "poll timeout\n"); | 681 | dev_err(priv->dev, "poll timeout\n"); |
| 696 | goto out; | 682 | goto out; |
| @@ -701,8 +687,8 @@ mt7531_ind_c22_phy_write(struct mt7530_priv *priv, int port, int regnum, · 4 unmodified lines | |||
| 701 | 687 | ||
| 702 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); | 688 | mt7530_mii_write(priv, MT7531_PHY_IAC, reg | MT7531_PHY_ACS_ST); |
| 703 | 689 | ||
| 704 | ret = readx_poll_timeout(_mt7530_unlocked_read, &p, reg, | 690 | ret = regmap_read_poll_timeout(priv->regmap, MT7531_PHY_IAC, reg, |
| 705 | !(reg & MT7531_PHY_ACS_ST), 20, 100000); | 691 | !(reg & MT7531_PHY_ACS_ST), 20, 100000); |
| 706 | if (ret < 0) { | 692 | if (ret < 0) { |
| 707 | dev_err(priv->dev, "poll timeout\n"); | 693 | dev_err(priv->dev, "poll timeout\n"); |
| 708 | goto out; | 694 | goto out; |
| @@ -1626,22 +1612,26 @@ mt7530_port_bridge_join(struct dsa_switch *ds, int port, · 917 unmodified lines | |||
| 1626 | static int | 1612 | static int |
| 1627 | mt7530_vlan_cmd(struct mt7530_priv *priv, enum mt7530_vlan_cmd cmd, u16 vid) | 1613 | mt7530_vlan_cmd(struct mt7530_priv *priv, enum mt7530_vlan_cmd cmd, u16 vid) |
| 1628 | { | 1614 | { |
| 1629 | struct mt7530_dummy_poll p; | ||
| 1630 | u32 val; | 1615 | u32 val; |
| 1631 | int ret; | 1616 | int ret; |
| 1632 | 1617 | ||
| 1633 | val = VTCR_BUSY | VTCR_FUNC(cmd) | vid; | 1618 | val = VTCR_BUSY | VTCR_FUNC(cmd) | vid; |
| 1634 | mt7530_write(priv, MT7530_VTCR, val); | 1619 | mt7530_write(priv, MT7530_VTCR, val); |
| 1635 | 1620 | ||
| 1636 | INIT_MT7530_DUMMY_POLL(&p, priv, MT7530_VTCR); | 1621 | mt7530_mutex_lock(priv); |
| 1637 | ret = readx_poll_timeout(_mt7530_read, &p, val, | 1622 | |
| 1638 | !(val & VTCR_BUSY), 20, 20000); | 1623 | ret = regmap_read_poll_timeout(priv->regmap, MT7530_VTCR, val, |
| 1624 | !(val & VTCR_BUSY), 20, 20000); | ||
| 1625 | if (!ret) | ||
| 1626 | ret = regmap_read(priv->regmap, MT7530_VTCR, &val); | ||
| 1627 | |||
| 1628 | mt7530_mutex_unlock(priv); | ||
| 1629 | |||
| 1639 | if (ret < 0) { | 1630 | if (ret < 0) { |
| 1640 | dev_err(priv->dev, "poll timeout\n"); | 1631 | dev_err(priv->dev, "poll timeout\n"); |
| 1641 | return ret; | 1632 | return ret; |
| 1642 | } | 1633 | } |
| 1643 | 1634 | ||
| 1644 | val = mt7530_read(priv, MT7530_VTCR); | ||
| 1645 | if (val & VTCR_INVALID) { | 1635 | if (val & VTCR_INVALID) { |
| 1646 | dev_err(priv->dev, "read VTCR invalid\n"); | 1636 | dev_err(priv->dev, "read VTCR invalid\n"); |
| 1647 | return -EINVAL; | 1637 | return -EINVAL; |
| @@ -1988,16 +1988,19 @@ static void rtl8365mb_get_stats64(struct dsa_switch *ds, int port, · 1987 unmodified lines | |||
| 1988 | spin_unlock(&p->stats_lock); | 1988 | spin_unlock(&p->stats_lock); |
| 1989 | } | 1989 | } |
| 1990 | 1990 | ||
| 1991 | static void rtl8365mb_stats_setup(struct realtek_priv *priv) | 1991 | static int rtl8365mb_stats_setup(struct realtek_priv *priv) |
| 1992 | { | 1992 | { |
| 1993 | struct rtl8365mb *mb = priv->chip_data; | 1993 | struct rtl8365mb *mb = priv->chip_data; |
| 1994 | struct dsa_switch *ds = &priv->ds; | 1994 | struct dsa_switch *ds = &priv->ds; |
| 1995 | struct dsa_port *dp; | 1995 | struct dsa_port *dp; |
| 1996 | int ret; | ||
| 1996 | 1997 | ||
| 1997 | /* Per-chip global mutex to protect MIB counter access, since doing | 1998 | /* Per-chip global mutex to protect MIB counter access, since doing |
| 1998 | * so requires accessing a series of registers in a particular order. | 1999 | * so requires accessing a series of registers in a particular order. |
| 1999 | */ | 2000 | */ |
| 2000 | mutex_init(&mb->mib_lock); | 2001 | ret = devm_mutex_init(priv->dev, &mb->mib_lock); |
| 2002 | if (ret) | ||
| 2003 | return ret; | ||
| 2001 | 2004 | ||
| 2002 | dsa_switch_for_each_available_port(dp, ds) { | 2005 | dsa_switch_for_each_available_port(dp, ds) { |
| 2003 | struct rtl8365mb_port *p = &mb->ports[dp->index]; | 2006 | struct rtl8365mb_port *p = &mb->ports[dp->index]; |
| @@ -2010,6 +2013,8 @@ static void rtl8365mb_stats_setup(struct realtek_priv *priv) · 6 unmodified lines | |||
| 2010 | */ | 2013 | */ |
| 2011 | INIT_DELAYED_WORK(&p->mib_work, rtl8365mb_stats_poll); | 2014 | INIT_DELAYED_WORK(&p->mib_work, rtl8365mb_stats_poll); |
| 2012 | } | 2015 | } |
| 2016 | |||
| 2017 | return 0; | ||
| 2013 | } | 2018 | } |
| 2014 | 2019 | ||
| 2015 | static void rtl8365mb_stats_teardown(struct realtek_priv *priv) | 2020 | static void rtl8365mb_stats_teardown(struct realtek_priv *priv) |
| @@ -2567,7 +2572,12 @@ static int rtl8365mb_setup(struct dsa_switch *ds) · 551 unmodified lines | |||
| 2567 | } | 2572 | } |
| 2568 | 2573 | ||
| 2569 | /* Start statistics counter polling */ | 2574 | /* Start statistics counter polling */ |
| 2570 | rtl8365mb_stats_setup(priv); | 2575 | ret = rtl8365mb_stats_setup(priv); |
| 2576 | if (ret) { | ||
| 2577 | dev_err(priv->dev, "failed to setup stats: %pe\n", | ||
| 2578 | ERR_PTR(ret)); | ||
| 2579 | goto out_teardown_irq; | ||
| 2580 | } | ||
| 2571 | 2581 | ||
| 2572 | return 0; | 2582 | return 0; |
| 2573 | 2583 | ||
| @@ -156,9 +156,17 @@ rtl83xx_probe(struct device *dev, · 155 unmodified lines | |||
| 156 | if (!priv) | 156 | if (!priv) |
| 157 | return ERR_PTR(-ENOMEM); | 157 | return ERR_PTR(-ENOMEM); |
| 158 | 158 | ||
| 159 | mutex_init(&priv->map_lock); | 159 | ret = devm_mutex_init(dev, &priv->map_lock); |
| 160 | mutex_init(&priv->vlan_lock); | 160 | if (ret) |
| 161 | mutex_init(&priv->l2_lock); | 161 | return ERR_PTR(ret); |
| 162 | |||
| 163 | ret = devm_mutex_init(dev, &priv->vlan_lock); | ||
| 164 | if (ret) | ||
| 165 | return ERR_PTR(ret); | ||
| 166 | |||
| 167 | ret = devm_mutex_init(dev, &priv->l2_lock); | ||
| 168 | if (ret) | ||
| 169 | return ERR_PTR(ret); | ||
| 162 | 170 | ||
| 163 | rc.lock_arg = priv; | 171 | rc.lock_arg = priv; |
| 164 | priv->map = devm_regmap_init(dev, NULL, priv, &rc); | 172 | priv->map = devm_regmap_init(dev, NULL, priv, &rc); |
| @@ -1222,11 +1222,11 @@ err_eeprom: · 1221 unmodified lines | |||
| 1222 | 1222 | ||
| 1223 | if (hw->flash_address) | 1223 | if (hw->flash_address) |
| 1224 | iounmap(hw->flash_address); | 1224 | iounmap(hw->flash_address); |
| 1225 | err_mdio_ioremap: | ||
| 1225 | kfree(adapter->tx_ring); | 1226 | kfree(adapter->tx_ring); |
| 1226 | kfree(adapter->rx_ring); | 1227 | kfree(adapter->rx_ring); |
| 1227 | err_dma: | 1228 | err_dma: |
| 1228 | err_sw_init: | 1229 | err_sw_init: |
| 1229 | err_mdio_ioremap: | ||
| 1230 | iounmap(hw->ce4100_gbe_mdio_base_virt); | 1230 | iounmap(hw->ce4100_gbe_mdio_base_virt); |
| 1231 | iounmap(hw->hw_addr); | 1231 | iounmap(hw->hw_addr); |
| 1232 | err_ioremap: | 1232 | err_ioremap: |
| @@ -793,7 +793,7 @@ err: · 792 unmodified lines | |||
| 793 | ret, | 793 | ret, |
| 794 | libie_aq_str(pf->hw.adminq.sq_last_status), | 794 | libie_aq_str(pf->hw.adminq.sq_last_status), |
| 795 | pin_type_name[pin_type], pin->idx); | 795 | pin_type_name[pin_type], pin->idx); |
| 796 | else | 796 | else if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) |
| 797 | dev_err_ratelimited(ice_pf_to_dev(pf), | 797 | dev_err_ratelimited(ice_pf_to_dev(pf), |
| 798 | "err:%d %s failed to update %s pin:%u\n", | 798 | "err:%d %s failed to update %s pin:%u\n", |
| 799 | ret, | 799 | ret, |
| @@ -3024,7 +3024,8 @@ static int ice_dpll_pps_update_phase_offsets(struct ice_pf *pf, · 2224 unmodified lines | |||
| 3024 | *phase_offset_pins_updated = 0; | 3024 | *phase_offset_pins_updated = 0; |
| 3025 | ret = ice_aq_get_cgu_input_pin_measure(&pf->hw, DPLL_TYPE_PPS, meas, | 3025 | ret = ice_aq_get_cgu_input_pin_measure(&pf->hw, DPLL_TYPE_PPS, meas, |
| 3026 | ARRAY_SIZE(meas)); | 3026 | ARRAY_SIZE(meas)); |
| 3027 | if (ret && pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EAGAIN) { | 3027 | if (ret && (pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EAGAIN || |
| 3028 | pf->hw.adminq.sq_last_status == LIBIE_AQ_RC_EBUSY)) { | ||
| 3028 | return 0; | 3029 | return 0; |
| 3029 | } else if (ret) { | 3030 | } else if (ret) { |
| 3030 | dev_err(ice_pf_to_dev(pf), | 3031 | dev_err(ice_pf_to_dev(pf), |
| @@ -3086,10 +3087,12 @@ ice_dpll_update_state(struct ice_pf *pf, struct ice_dpll *d, bool init) · 55 unmodified lines | |||
| 3086 | d->dpll_idx, d->prev_input_idx, d->input_idx, | 3087 | d->dpll_idx, d->prev_input_idx, d->input_idx, |
| 3087 | d->dpll_state, d->prev_dpll_state, d->mode); | 3088 | d->dpll_state, d->prev_dpll_state, d->mode); |
| 3088 | if (ret) { | 3089 | if (ret) { |
| 3089 | dev_err(ice_pf_to_dev(pf), | 3090 | /* EBUSY is expected during reset recovery, don't log error */ |
| 3090 | "update dpll=%d state failed, ret=%d %s\n", | 3091 | if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) |
| 3091 | d->dpll_idx, ret, | 3092 | dev_err(ice_pf_to_dev(pf), |
| 3092 | libie_aq_str(pf->hw.adminq.sq_last_status)); | 3093 | "update dpll=%d state failed, ret=%d %s\n", |
| 3094 | d->dpll_idx, ret, | ||
| 3095 | libie_aq_str(pf->hw.adminq.sq_last_status)); | ||
| 3093 | return ret; | 3096 | return ret; |
| 3094 | } | 3097 | } |
| 3095 | if (init) { | 3098 | if (init) { |
| @@ -3158,7 +3161,9 @@ static void ice_dpll_periodic_work(struct kthread_work *work) · 62 unmodified lines | |||
| 3158 | d->periodic_counter % dp->phase_offset_monitor_period == 0) | 3161 | d->periodic_counter % dp->phase_offset_monitor_period == 0) |
| 3159 | ret = ice_dpll_pps_update_phase_offsets(pf, &phase_offset_ntf); | 3162 | ret = ice_dpll_pps_update_phase_offsets(pf, &phase_offset_ntf); |
| 3160 | if (ret) { | 3163 | if (ret) { |
| 3161 | d->cgu_state_acq_err_num++; | 3164 | /* EBUSY is expected during reset recovery */ |
| 3165 | if (pf->hw.adminq.sq_last_status != LIBIE_AQ_RC_EBUSY) | ||
| 3166 | d->cgu_state_acq_err_num++; | ||
| 3162 | /* stop rescheduling this worker */ | 3167 | /* stop rescheduling this worker */ |
| 3163 | if (d->cgu_state_acq_err_num > | 3168 | if (d->cgu_state_acq_err_num > |
| 3164 | ICE_CGU_STATE_ACQ_ERR_THRESHOLD) { | 3169 | ICE_CGU_STATE_ACQ_ERR_THRESHOLD) { |
| @@ -1069,18 +1069,18 @@ static int ice_lbtest_prepare_rings(struct ice_vsi *vsi) · 1068 unmodified lines | |||
| 1069 | 1069 | ||
| 1070 | status = ice_vsi_cfg_lan(vsi); | 1070 | status = ice_vsi_cfg_lan(vsi); |
| 1071 | if (status) | 1071 | if (status) |
| 1072 | goto err_setup_rx_ring; | 1072 | goto err_cfg_lan; |
| 1073 | 1073 | ||
| 1074 | status = ice_vsi_start_all_rx_rings(vsi); | 1074 | status = ice_vsi_start_all_rx_rings(vsi); |
| 1075 | if (status) | 1075 | if (status) |
| 1076 | goto err_start_rx_ring; | 1076 | goto err_cfg_lan; |
| 1077 | 1077 | ||
| 1078 | return 0; | 1078 | return 0; |
| 1079 | 1079 | ||
| 1080 | err_start_rx_ring: | 1080 | err_cfg_lan: |
| 1081 | ice_vsi_free_rx_rings(vsi); | ||
| 1082 | err_setup_rx_ring: | ||
| 1083 | ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, 0); | 1081 | ice_vsi_stop_lan_tx_rings(vsi, ICE_NO_RESET, 0); |
| 1082 | err_setup_rx_ring: | ||
| 1083 | ice_vsi_free_rx_rings(vsi); | ||
| 1084 | err_setup_tx_ring: | 1084 | err_setup_tx_ring: |
| 1085 | ice_vsi_free_tx_rings(vsi); | 1085 | ice_vsi_free_tx_rings(vsi); |
| 1086 | 1086 | ||
| @@ -5637,6 +5637,16 @@ static int ice_resume(struct device *dev) · 5636 unmodified lines | |||
| 5637 | /* Restart the service task */ | 5637 | /* Restart the service task */ |
| 5638 | mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period)); | 5638 | mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period)); |
| 5639 | 5639 | ||
| 5640 | /* Best-effort wait for the scheduled reset to finish so that the | ||
| 5641 | * device is operational before returning. Without this, userspace | ||
| 5642 | * (e.g. NetworkManager) may try to open the net device while the | ||
| 5643 | * asynchronous reset is still in progress, hitting -EBUSY. | ||
| 5644 | */ | ||
| 5645 | ret = ice_wait_for_reset(pf, secs_to_jiffies(10)); | ||
| 5646 | if (ret) | ||
| 5647 | dev_err(dev, "Wait for reset timed out (10s) during resume: %d\n", | ||
| 5648 | ret); | ||
| 5649 | |||
| 5640 | return 0; | 5650 | return 0; |
| 5641 | } | 5651 | } |
| 5642 | 5652 | ||
| @@ -848,6 +848,30 @@ static void ice_notify_vf_reset(struct ice_vf *vf) · 847 unmodified lines | |||
| 848 | NULL); | 848 | NULL); |
| 849 | } | 849 | } |
| 850 | 850 | ||
| 851 | /** | ||
| 852 | * ice_reset_interrupts - clear all queue interrupt configuration for a VSI | ||
| 853 | * @vsi: the VSI whose interrupt registers should be cleared | ||
| 854 | * | ||
| 855 | * Zero the QINT_RQCTL and QINT_TQCTL registers for all allocated queues | ||
| 856 | * in the VSI. This clears the entire register including MSIX_INDX, ITR_INDX, | ||
| 857 | * CAUSE_ENA and NEXTQ fields, unlike ice_vf_dis_rxq_interrupt() which only | ||
| 858 | * clears the CAUSE_ENA bit. | ||
| 859 | */ | ||
| 860 | void ice_reset_interrupts(struct ice_vsi *vsi) | ||
| 861 | { | ||
| 862 | struct ice_pf *pf = vsi->back; | ||
| 863 | struct ice_hw *hw = &pf->hw; | ||
| 864 | int i; | ||
| 865 | |||
| 866 | ice_for_each_alloc_rxq(vsi, i) | ||
| 867 | wr32(hw, QINT_RQCTL(vsi->rxq_map[i]), 0); | ||
| 868 | |||
| 869 | ice_for_each_alloc_txq(vsi, i) | ||
| 870 | wr32(hw, QINT_TQCTL(vsi->txq_map[i]), 0); | ||
| 871 | |||
| 872 | ice_flush(hw); | ||
| 873 | } | ||
| 874 | |||
| 851 | /** | 875 | /** |
| 852 | * ice_reset_vf - Reset a particular VF | 876 | * ice_reset_vf - Reset a particular VF |
| 853 | * @vf: pointer to the VF structure | 877 | * @vf: pointer to the VF structure |
| @@ -919,6 +943,9 @@ int ice_reset_vf(struct ice_vf *vf, u32 flags) · 65 unmodified lines | |||
| 919 | 943 | ||
| 920 | ice_dis_vf_qs(vf); | 944 | ice_dis_vf_qs(vf); |
| 921 | 945 | ||
| 946 | /* cleanup interrupt registers */ | ||
| 947 | ice_reset_interrupts(vsi); | ||
| 948 | |||
| 922 | /* Call Disable LAN Tx queue AQ whether or not queues are | 949 | /* Call Disable LAN Tx queue AQ whether or not queues are |
| 923 | * enabled. This is needed for successful completion of VFR. | 950 | * enabled. This is needed for successful completion of VFR. |
| 924 | */ | 951 | */ |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | |||
| 26 | void ice_initialize_vf_entry(struct ice_vf *vf); | 26 | void ice_initialize_vf_entry(struct ice_vf *vf); |
| 27 | void ice_deinitialize_vf_entry(struct ice_vf *vf); | 27 | void ice_deinitialize_vf_entry(struct ice_vf *vf); |
| 28 | void ice_dis_vf_qs(struct ice_vf *vf); | 28 | void ice_dis_vf_qs(struct ice_vf *vf); |
| 29 | void ice_reset_interrupts(struct ice_vsi *vsi); | ||
| 29 | int ice_check_vf_init(struct ice_vf *vf); | 30 | int ice_check_vf_init(struct ice_vf *vf); |
| 30 | enum virtchnl_status_code ice_err_to_virt_err(int err); | 31 | enum virtchnl_status_code ice_err_to_virt_err(int err); |
| 31 | struct ice_port_info *ice_vf_get_port_info(struct ice_vf *vf); | 32 | struct ice_port_info *ice_vf_get_port_info(struct ice_vf *vf); |
| @@ -224,6 +224,24 @@ void ice_vf_ena_rxq_interrupt(struct ice_vsi *vsi, u32 q_idx) · 223 unmodified lines | |||
| 224 | wr32(hw, QINT_RQCTL(pfq), reg | QINT_RQCTL_CAUSE_ENA_M); | 224 | wr32(hw, QINT_RQCTL(pfq), reg | QINT_RQCTL_CAUSE_ENA_M); |
| 225 | } | 225 | } |
| 226 | 226 | ||
| 227 | /** | ||
| 228 | * ice_vf_dis_rxq_interrupt - disable Rx queue interrupt via QINT_RQCTL | ||
| 229 | * @vsi: VSI of the VF to configure | ||
| 230 | * @q_idx: VF queue index used to determine the queue in the PF's space | ||
| 231 | */ | ||
| 232 | static void ice_vf_dis_rxq_interrupt(struct ice_vsi *vsi, u32 q_idx) | ||
| 233 | { | ||
| 234 | struct ice_hw *hw = &vsi->back->hw; | ||
| 235 | u32 pfq = vsi->rxq_map[q_idx]; | ||
| 236 | u32 reg; | ||
| 237 | |||
| 238 | reg = rd32(hw, QINT_RQCTL(pfq)); | ||
| 239 | reg &= ~QINT_RQCTL_CAUSE_ENA_M; | ||
| 240 | wr32(hw, QINT_RQCTL(pfq), reg); | ||
| 241 | |||
| 242 | ice_flush(hw); | ||
| 243 | } | ||
| 244 | |||
| 227 | /** | 245 | /** |
| 228 | * ice_vc_ena_qs_msg | 246 | * ice_vc_ena_qs_msg |
| 229 | * @vf: pointer to the VF info | 247 | * @vf: pointer to the VF info |
| @@ -416,6 +434,8 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg) · 186 unmodified lines | |||
| 416 | goto error_param; | 434 | goto error_param; |
| 417 | } | 435 | } |
| 418 | 436 | ||
| 437 | for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) | ||
| 438 | ice_vf_dis_rxq_interrupt(vsi, vf_q_id); | ||
| 419 | bitmap_zero(vf->rxq_ena, ICE_MAX_RSS_QS_PER_VF); | 439 | bitmap_zero(vf->rxq_ena, ICE_MAX_RSS_QS_PER_VF); |
| 420 | } else if (q_map) { | 440 | } else if (q_map) { |
| 421 | for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) { | 441 | for_each_set_bit(vf_q_id, &q_map, ICE_MAX_RSS_QS_PER_VF) { |
| @@ -436,6 +456,7 @@ int ice_vc_dis_qs_msg(struct ice_vf *vf, u8 *msg) · 14 unmodified lines | |||
| 436 | goto error_param; | 456 | goto error_param; |
| 437 | } | 457 | } |
| 438 | 458 | ||
| 459 | ice_vf_dis_rxq_interrupt(vsi, vf_q_id); | ||
| 439 | /* Clear enabled queues flag */ | 460 | /* Clear enabled queues flag */ |
| 440 | clear_bit(vf_q_id, vf->rxq_ena); | 461 | clear_bit(vf_q_id, vf->rxq_ena); |
| 441 | } | 462 | } |
| @@ -87,7 +87,7 @@ static int idpf_intr_reg_init(struct idpf_vport *vport, · 86 unmodified lines | |||
| 87 | if (!reg_vals) | 87 | if (!reg_vals) |
| 88 | return -ENOMEM; | 88 | return -ENOMEM; |
| 89 | 89 | ||
| 90 | num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals); | 90 | num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals, total_vecs); |
| 91 | if (num_regs < num_vecs) { | 91 | if (num_regs < num_vecs) { |
| 92 | err = -EINVAL; | 92 | err = -EINVAL; |
| 93 | goto free_reg_vals; | 93 | goto free_reg_vals; |
| @@ -139,7 +139,7 @@ static int idpf_mb_intr_req_irq(struct idpf_adapter *adapter) · 138 unmodified lines | |||
| 139 | if (err) { | 139 | if (err) { |
| 140 | dev_err(&adapter->pdev->dev, | 140 | dev_err(&adapter->pdev->dev, |
| 141 | "IRQ request for mailbox failed, error: %d\n", err); | 141 | "IRQ request for mailbox failed, error: %d\n", err); |
| 142 | 142 | kfree(name); | |
| 143 | return err; | 143 | return err; |
| 144 | } | 144 | } |
| 145 | 145 | ||
| @@ -3097,10 +3097,7 @@ static netdev_tx_t idpf_tx_splitq_frame(struct sk_buff *skb, · 3096 unmodified lines | |||
| 3097 | 3097 | ||
| 3098 | tx_params.dtype = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE; | 3098 | tx_params.dtype = IDPF_TX_DESC_DTYPE_FLEX_FLOW_SCHE; |
| 3099 | tx_params.eop_cmd = IDPF_TXD_FLEX_FLOW_CMD_EOP; | 3099 | tx_params.eop_cmd = IDPF_TXD_FLEX_FLOW_CMD_EOP; |
| 3100 | /* Set the RE bit to periodically "clean" the descriptor ring. | 3100 | /* Set the RE bit periodically to "clean" the descriptor ring */ |
| 3101 | * MIN_GAP is set to MIN_RING size to ensure it will be set at | ||
| 3102 | * least once each time around the ring. | ||
| 3103 | */ | ||
| 3104 | if (idpf_tx_splitq_need_re(tx_q)) { | 3101 | if (idpf_tx_splitq_need_re(tx_q)) { |
| 3105 | tx_params.eop_cmd |= IDPF_TXD_FLEX_FLOW_CMD_RE; | 3102 | tx_params.eop_cmd |= IDPF_TXD_FLEX_FLOW_CMD_RE; |
| 3106 | tx_q->txq_grp->num_completions_pending++; | 3103 | tx_q->txq_grp->num_completions_pending++; |
| @@ -21,7 +21,7 @@ · 20 unmodified lines | |||
| 21 | /* Mailbox Queue */ | 21 | /* Mailbox Queue */ |
| 22 | #define IDPF_MAX_MBXQ 1 | 22 | #define IDPF_MAX_MBXQ 1 |
| 23 | 23 | ||
| 24 | #define IDPF_MIN_TXQ_DESC 64 | 24 | #define IDPF_MIN_TXQ_DESC 128 |
| 25 | #define IDPF_MIN_RXQ_DESC 64 | 25 | #define IDPF_MIN_RXQ_DESC 64 |
| 26 | #define IDPF_MIN_TXQ_COMPLQ_DESC 256 | 26 | #define IDPF_MIN_TXQ_COMPLQ_DESC 256 |
| 27 | #define IDPF_MAX_QIDS 256 | 27 | #define IDPF_MAX_QIDS 256 |
| @@ -86,7 +86,7 @@ static int idpf_vf_intr_reg_init(struct idpf_vport *vport, · 85 unmodified lines | |||
| 86 | if (!reg_vals) | 86 | if (!reg_vals) |
| 87 | return -ENOMEM; | 87 | return -ENOMEM; |
| 88 | 88 | ||
| 89 | num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals); | 89 | num_regs = idpf_get_reg_intr_vecs(adapter, reg_vals, total_vecs); |
| 90 | if (num_regs < num_vecs) { | 90 | if (num_regs < num_vecs) { |
| 91 | err = -EINVAL; | 91 | err = -EINVAL; |
| 92 | goto free_reg_vals; | 92 | goto free_reg_vals; |
| @@ -1318,11 +1318,12 @@ idpf_vport_init_queue_reg_chunks(struct idpf_vport_config *vport_config, · 1317 unmodified lines | |||
| 1318 | * idpf_get_reg_intr_vecs - Get vector queue register offset | 1318 | * idpf_get_reg_intr_vecs - Get vector queue register offset |
| 1319 | * @adapter: adapter structure to get the vector chunks | 1319 | * @adapter: adapter structure to get the vector chunks |
| 1320 | * @reg_vals: Register offsets to store in | 1320 | * @reg_vals: Register offsets to store in |
| 1321 | * @num_vecs: number of entries the @reg_vals array can hold | ||
| 1321 | * | 1322 | * |
| 1322 | * Return: number of registers that got populated | 1323 | * Return: number of registers that got populated |
| 1323 | */ | 1324 | */ |
| 1324 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, | 1325 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, |
| 1325 | struct idpf_vec_regs *reg_vals) | 1326 | struct idpf_vec_regs *reg_vals, int num_vecs) |
| 1326 | { | 1327 | { |
| 1327 | struct virtchnl2_vector_chunks *chunks; | 1328 | struct virtchnl2_vector_chunks *chunks; |
| 1328 | struct idpf_vec_regs reg_val; | 1329 | struct idpf_vec_regs reg_val; |
| @@ -1346,7 +1347,7 @@ int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, · 17 unmodified lines | |||
| 1346 | dynctl_reg_spacing = le32_to_cpu(chunk->dynctl_reg_spacing); | 1347 | dynctl_reg_spacing = le32_to_cpu(chunk->dynctl_reg_spacing); |
| 1347 | itrn_reg_spacing = le32_to_cpu(chunk->itrn_reg_spacing); | 1348 | itrn_reg_spacing = le32_to_cpu(chunk->itrn_reg_spacing); |
| 1348 | 1349 | ||
| 1349 | for (i = 0; i < num_vec; i++) { | 1350 | for (i = 0; i < num_vec && num_regs < num_vecs; i++) { |
| 1350 | reg_vals[num_regs].dyn_ctl_reg = reg_val.dyn_ctl_reg; | 1351 | reg_vals[num_regs].dyn_ctl_reg = reg_val.dyn_ctl_reg; |
| 1351 | reg_vals[num_regs].itrn_reg = reg_val.itrn_reg; | 1352 | reg_vals[num_regs].itrn_reg = reg_val.itrn_reg; |
| 1352 | reg_vals[num_regs].itrn_index_spacing = | 1353 | reg_vals[num_regs].itrn_index_spacing = |
| @@ -104,7 +104,7 @@ int idpf_vc_core_init(struct idpf_adapter *adapter); · 103 unmodified lines | |||
| 104 | void idpf_vc_core_deinit(struct idpf_adapter *adapter); | 104 | void idpf_vc_core_deinit(struct idpf_adapter *adapter); |
| 105 | 105 | ||
| 106 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, | 106 | int idpf_get_reg_intr_vecs(struct idpf_adapter *adapter, |
| 107 | struct idpf_vec_regs *reg_vals); | 107 | struct idpf_vec_regs *reg_vals, int num_vecs); |
| 108 | int idpf_queue_reg_init(struct idpf_vport *vport, | 108 | int idpf_queue_reg_init(struct idpf_vport *vport, |
| 109 | struct idpf_q_vec_rsrc *rsrc, | 109 | struct idpf_q_vec_rsrc *rsrc, |
| 110 | struct idpf_queue_id_reg_info *chunks); | 110 | struct idpf_queue_id_reg_info *chunks); |
| @@ -2190,8 +2190,6 @@ dma_error: · 2189 unmodified lines | |||
| 2190 | buffer_info->time_stamp = 0; | 2190 | buffer_info->time_stamp = 0; |
| 2191 | buffer_info->length = 0; | 2191 | buffer_info->length = 0; |
| 2192 | buffer_info->mapped_as_page = false; | 2192 | buffer_info->mapped_as_page = false; |
| 2193 | if (count) | ||
| 2194 | count--; | ||
| 2195 | 2193 | ||
| 2196 | /* clear timestamp and dma mappings for remaining portion of packet */ | 2194 | /* clear timestamp and dma mappings for remaining portion of packet */ |
| 2197 | while (count--) { | 2195 | while (count--) { |
| @@ -5352,9 +5352,8 @@ void igc_down(struct igc_adapter *adapter) · 5351 unmodified lines | |||
| 5352 | 5352 | ||
| 5353 | for (i = 0; i < adapter->num_q_vectors; i++) { | 5353 | for (i = 0; i < adapter->num_q_vectors; i++) { |
| 5354 | if (adapter->q_vector[i]) { | 5354 | if (adapter->q_vector[i]) { |
| 5355 | napi_synchronize(&adapter->q_vector[i]->napi); | ||
| 5356 | igc_set_queue_napi(adapter, i, NULL); | ||
| 5357 | napi_disable(&adapter->q_vector[i]->napi); | 5355 | napi_disable(&adapter->q_vector[i]->napi); |
| 5356 | igc_set_queue_napi(adapter, i, NULL); | ||
| 5358 | } | 5357 | } |
| 5359 | } | 5358 | } |
| 5360 | 5359 | ||
| @@ -518,6 +518,18 @@ int cgx_set_pkind(void *cgxd, u8 lmac_id, int pkind) · 517 unmodified lines | |||
| 518 | return 0; | 518 | return 0; |
| 519 | } | 519 | } |
| 520 | 520 | ||
| 521 | int cgx_get_pkind(void *cgxd, u8 lmac_id, int *pkind) | ||
| 522 | { | ||
| 523 | struct cgx *cgx = cgxd; | ||
| 524 | |||
| 525 | if (!is_lmac_valid(cgx, lmac_id)) | ||
| 526 | return -ENODEV; | ||
| 527 | |||
| 528 | *pkind = cgx_read(cgx, lmac_id, cgx->mac_ops->rxid_map_offset); | ||
| 529 | *pkind = *pkind & 0x3F; | ||
| 530 | return 0; | ||
| 531 | } | ||
| 532 | |||
| 521 | static u8 cgx_get_lmac_type(void *cgxd, int lmac_id) | 533 | static u8 cgx_get_lmac_type(void *cgxd, int lmac_id) |
| 522 | { | 534 | { |
| 523 | struct cgx *cgx = cgxd; | 535 | struct cgx *cgx = cgxd; |
| @@ -141,6 +141,7 @@ int cgx_get_cgxid(void *cgxd); · 140 unmodified lines | |||
| 141 | int cgx_get_lmac_cnt(void *cgxd); | 141 | int cgx_get_lmac_cnt(void *cgxd); |
| 142 | void *cgx_get_pdata(int cgx_id); | 142 | void *cgx_get_pdata(int cgx_id); |
| 143 | int cgx_set_pkind(void *cgxd, u8 lmac_id, int pkind); | 143 | int cgx_set_pkind(void *cgxd, u8 lmac_id, int pkind); |
| 144 | int cgx_get_pkind(void *cgxd, u8 lmac_id, int *pkind); | ||
| 144 | int cgx_lmac_evh_register(struct cgx_event_cb *cb, void *cgxd, int lmac_id); | 145 | int cgx_lmac_evh_register(struct cgx_event_cb *cb, void *cgxd, int lmac_id); |
| 145 | int cgx_lmac_evh_unregister(void *cgxd, int lmac_id); | 146 | int cgx_lmac_evh_unregister(void *cgxd, int lmac_id); |
| 146 | int cgx_get_tx_stats(void *cgxd, int lmac_id, int idx, u64 *tx_stat); | 147 | int cgx_get_tx_stats(void *cgxd, int lmac_id, int idx, u64 *tx_stat); |
| @@ -1115,6 +1115,8 @@ void npc_read_mcam_entry(struct rvu *rvu, struct npc_mcam *mcam, · 1114 unmodified lines | |||
| 1115 | u8 *intf, u8 *ena); | 1115 | u8 *intf, u8 *ena); |
| 1116 | int npc_config_cntr_default_entries(struct rvu *rvu, bool enable); | 1116 | int npc_config_cntr_default_entries(struct rvu *rvu, bool enable); |
| 1117 | bool is_cgx_config_permitted(struct rvu *rvu, u16 pcifunc); | 1117 | bool is_cgx_config_permitted(struct rvu *rvu, u16 pcifunc); |
| 1118 | bool rvu_cgx_check_permission_and_set_pkind(struct rvu *rvu, u16 pcifunc, int pkind); | ||
| 1119 | bool rvu_cgx_is_pkind_config_permitted(struct rvu *rvu, u16 pcifunc); | ||
| 1118 | bool is_mac_feature_supported(struct rvu *rvu, int pf, int feature); | 1120 | bool is_mac_feature_supported(struct rvu *rvu, int pf, int feature); |
| 1119 | u32 rvu_cgx_get_fifolen(struct rvu *rvu); | 1121 | u32 rvu_cgx_get_fifolen(struct rvu *rvu); |
| 1120 | void *rvu_first_cgx_pdata(struct rvu *rvu); | 1122 | void *rvu_first_cgx_pdata(struct rvu *rvu); |
| @@ -1355,3 +1355,82 @@ void rvu_mac_reset(struct rvu *rvu, u16 pcifunc) · 1354 unmodified lines | |||
| 1355 | if (mac_ops->mac_reset(cgxd, lmac, !is_vf(pcifunc))) | 1355 | if (mac_ops->mac_reset(cgxd, lmac, !is_vf(pcifunc))) |
| 1356 | dev_err(rvu->dev, "Failed to reset MAC\n"); | 1356 | dev_err(rvu->dev, "Failed to reset MAC\n"); |
| 1357 | } | 1357 | } |
| 1358 | |||
| 1359 | /* Do not allow CGX-mapped VFs to overwrite PKIND when special parse kinds | ||
| 1360 | * (HiGig, EDSA, etc.) are in use on the shared LMAC. VFs must not program | ||
| 1361 | * NPC_TX_DEF_PKIND on NIX_AF_LFX_TX_PARSE_CFG in that case: the PF owns | ||
| 1362 | * parse mode and no separate NPC_TX_HIGIG_PKIND is installed on the VF LF. | ||
| 1363 | * TX-parse callers skip the write when denied; rvu_lf_reset() clears each LF | ||
| 1364 | * before alloc so the next permitted owner programs NPC_TX_DEF_PKIND. | ||
| 1365 | */ | ||
| 1366 | bool rvu_cgx_is_pkind_config_permitted(struct rvu *rvu, u16 pcifunc) | ||
| 1367 | { | ||
| 1368 | int pf, err, rxpkind; | ||
| 1369 | u8 cgx_id, lmac_id; | ||
| 1370 | void *cgxd; | ||
| 1371 | |||
| 1372 | pf = rvu_get_pf(rvu->pdev, pcifunc); | ||
| 1373 | |||
| 1374 | if (!(pcifunc & RVU_PFVF_FUNC_MASK)) | ||
| 1375 | return true; | ||
| 1376 | |||
| 1377 | if (!is_pf_cgxmapped(rvu, pf)) | ||
| 1378 | return true; | ||
| 1379 | |||
| 1380 | rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | ||
| 1381 | cgxd = rvu_cgx_pdata(cgx_id, rvu); | ||
| 1382 | err = cgx_get_pkind(cgxd, lmac_id, &rxpkind); | ||
| 1383 | if (err) | ||
| 1384 | return false; | ||
| 1385 | |||
| 1386 | switch (rxpkind) { | ||
| 1387 | case NPC_RX_HIGIG_PKIND: | ||
| 1388 | case NPC_RX_EDSA_PKIND: | ||
| 1389 | return false; | ||
| 1390 | default: | ||
| 1391 | return true; | ||
| 1392 | } | ||
| 1393 | } | ||
| 1394 | |||
| 1395 | /* Do not allow CGX-mapped VFs to overwrite PKIND when special parse kinds | ||
| 1396 | * (HiGig, EDSA, etc.) are in use on the shared LMAC. | ||
| 1397 | */ | ||
| 1398 | bool rvu_cgx_check_permission_and_set_pkind(struct rvu *rvu, u16 pcifunc, int pkind) | ||
| 1399 | { | ||
| 1400 | int pf, err, rxpkind; | ||
| 1401 | u8 cgx_id, lmac_id; | ||
| 1402 | struct cgx *cgxd; | ||
| 1403 | |||
| 1404 | pf = rvu_get_pf(rvu->pdev, pcifunc); | ||
| 1405 | |||
| 1406 | if (!is_pf_cgxmapped(rvu, pf)) | ||
| 1407 | return false; | ||
| 1408 | |||
| 1409 | rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | ||
| 1410 | cgxd = rvu_cgx_pdata(cgx_id, rvu); | ||
| 1411 | |||
| 1412 | mutex_lock(&cgxd->lock); | ||
| 1413 | if (!is_vf(pcifunc)) | ||
| 1414 | goto set; | ||
| 1415 | |||
| 1416 | err = cgx_get_pkind(cgxd, lmac_id, &rxpkind); | ||
| 1417 | if (err) | ||
| 1418 | goto err; | ||
| 1419 | |||
| 1420 | switch (rxpkind) { | ||
| 1421 | case NPC_RX_HIGIG_PKIND: | ||
| 1422 | case NPC_RX_EDSA_PKIND: | ||
| 1423 | goto err; | ||
| 1424 | default: | ||
| 1425 | break; | ||
| 1426 | } | ||
| 1427 | |||
| 1428 | set: | ||
| 1429 | cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, pkind); | ||
| 1430 | mutex_unlock(&cgxd->lock); | ||
| 1431 | return true; | ||
| 1432 | |||
| 1433 | err: | ||
| 1434 | mutex_unlock(&cgxd->lock); | ||
| 1435 | return false; | ||
| 1436 | } | ||
| @@ -363,8 +363,8 @@ static int nix_interface_init(struct rvu *rvu, u16 pcifunc, int type, int nixlf, · 362 unmodified lines | |||
| 363 | pfvf->tx_chan_cnt = 1; | 363 | pfvf->tx_chan_cnt = 1; |
| 364 | rsp->tx_link = cgx_id * hw->lmac_per_cgx + lmac_id; | 364 | rsp->tx_link = cgx_id * hw->lmac_per_cgx + lmac_id; |
| 365 | 365 | ||
| 366 | cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, pkind); | 366 | if (rvu_cgx_check_permission_and_set_pkind(rvu, pcifunc, pkind)) |
| 367 | rvu_npc_set_pkind(rvu, pkind, pfvf); | 367 | rvu_npc_set_pkind(rvu, pkind, pfvf); |
| 368 | break; | 368 | break; |
| 369 | case NIX_INTF_TYPE_LBK: | 369 | case NIX_INTF_TYPE_LBK: |
| 370 | vf = (pcifunc & RVU_PFVF_FUNC_MASK) - 1; | 370 | vf = (pcifunc & RVU_PFVF_FUNC_MASK) - 1; |
| @@ -1505,13 +1505,15 @@ int rvu_mbox_handler_nix_lf_alloc(struct rvu *rvu, · 1134 unmodified lines | |||
| 1505 | struct nix_lf_alloc_req *req, | 1505 | struct nix_lf_alloc_req *req, |
| 1506 | struct nix_lf_alloc_rsp *rsp) | 1506 | struct nix_lf_alloc_rsp *rsp) |
| 1507 | { | 1507 | { |
| 1508 | int nixlf, qints, hwctx_size, intf, rc = 0; | 1508 | int nixlf, qints, hwctx_size, intf, rc = 0, pf; |
| 1509 | u16 bcast, mcast, promisc, ucast; | 1509 | u16 bcast, mcast, promisc, ucast; |
| 1510 | struct rvu_hwinfo *hw = rvu->hw; | 1510 | struct rvu_hwinfo *hw = rvu->hw; |
| 1511 | u16 pcifunc = req->hdr.pcifunc; | 1511 | u16 pcifunc = req->hdr.pcifunc; |
| 1512 | u8 cgx_id = 0, lmac_id = 0; | ||
| 1512 | bool rules_created = false; | 1513 | bool rules_created = false; |
| 1513 | struct rvu_block *block; | 1514 | struct rvu_block *block; |
| 1514 | struct rvu_pfvf *pfvf; | 1515 | struct rvu_pfvf *pfvf; |
| 1516 | struct cgx *cgxd; | ||
| 1515 | u64 cfg, ctx_cfg; | 1517 | u64 cfg, ctx_cfg; |
| 1516 | int blkaddr; | 1518 | int blkaddr; |
| 1517 | 1519 | ||
| @@ -1685,8 +1687,20 @@ int rvu_mbox_handler_nix_lf_alloc(struct rvu *rvu, · 167 unmodified lines | |||
| 1685 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_RX_CFG(nixlf), req->rx_cfg); | 1687 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_RX_CFG(nixlf), req->rx_cfg); |
| 1686 | 1688 | ||
| 1687 | /* Configure pkind for TX parse config */ | 1689 | /* Configure pkind for TX parse config */ |
| 1690 | |||
| 1691 | pf = rvu_get_pf(rvu->pdev, pcifunc); | ||
| 1688 | cfg = NPC_TX_DEF_PKIND; | 1692 | cfg = NPC_TX_DEF_PKIND; |
| 1689 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | 1693 | |
| 1694 | if (is_pf_cgxmapped(rvu, pf) && is_vf(pcifunc)) { | ||
| 1695 | rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | ||
| 1696 | cgxd = rvu_cgx_pdata(cgx_id, rvu); | ||
| 1697 | mutex_lock(&cgxd->lock); | ||
| 1698 | if (rvu_cgx_is_pkind_config_permitted(rvu, pcifunc)) | ||
| 1699 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | ||
| 1700 | mutex_unlock(&cgxd->lock); | ||
| 1701 | } else { | ||
| 1702 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), cfg); | ||
| 1703 | } | ||
| 1690 | 1704 | ||
| 1691 | if (is_rep_dev(rvu, pcifunc)) { | 1705 | if (is_rep_dev(rvu, pcifunc)) { |
| 1692 | pfvf->tx_chan_base = RVU_SWITCH_LBK_CHAN; | 1706 | pfvf->tx_chan_base = RVU_SWITCH_LBK_CHAN; |
| @@ -19,6 +19,7 @@ · 18 unmodified lines | |||
| 19 | #include "cn20k/npc.h" | 19 | #include "cn20k/npc.h" |
| 20 | #include "rvu_npc.h" | 20 | #include "rvu_npc.h" |
| 21 | #include "cn20k/reg.h" | 21 | #include "cn20k/reg.h" |
| 22 | #include "lmac_common.h" | ||
| 22 | 23 | ||
| 23 | #define RSVD_MCAM_ENTRIES_PER_PF 3 /* Broadcast, Promisc and AllMulticast */ | 24 | #define RSVD_MCAM_ENTRIES_PER_PF 3 /* Broadcast, Promisc and AllMulticast */ |
| 24 | #define RSVD_MCAM_ENTRIES_PER_NIXLF 1 /* Ucast for LFs */ | 25 | #define RSVD_MCAM_ENTRIES_PER_NIXLF 1 /* Ucast for LFs */ |
| @@ -4200,10 +4201,11 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 4175 unmodified lines | |||
| 4200 | 4201 | ||
| 4201 | { | 4202 | { |
| 4202 | struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc); | 4203 | struct rvu_pfvf *pfvf = rvu_get_pfvf(rvu, pcifunc); |
| 4203 | int blkaddr, nixlf, rc, intf_mode; | ||
| 4204 | int pf = rvu_get_pf(rvu->pdev, pcifunc); | 4204 | int pf = rvu_get_pf(rvu->pdev, pcifunc); |
| 4205 | int blkaddr, nixlf, rc, intf_mode; | ||
| 4206 | u8 cgx_id = 0, lmac_id = 0; | ||
| 4205 | u64 rxpkind, txpkind; | 4207 | u64 rxpkind, txpkind; |
| 4206 | u8 cgx_id, lmac_id; | 4208 | struct cgx *cgxd; |
| 4207 | 4209 | ||
| 4208 | /* use default pkind to disable edsa/higig */ | 4210 | /* use default pkind to disable edsa/higig */ |
| 4209 | rxpkind = rvu_npc_get_pkind(rvu, pf); | 4211 | rxpkind = rvu_npc_get_pkind(rvu, pf); |
| @@ -4227,12 +4229,8 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 17 unmodified lines | |||
| 4227 | /* rx pkind set req valid only for cgx mapped PFs */ | 4229 | /* rx pkind set req valid only for cgx mapped PFs */ |
| 4228 | if (!is_cgx_config_permitted(rvu, pcifunc)) | 4230 | if (!is_cgx_config_permitted(rvu, pcifunc)) |
| 4229 | return 0; | 4231 | return 0; |
| 4230 | rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, &lmac_id); | 4232 | if (!rvu_cgx_check_permission_and_set_pkind(rvu, pcifunc, rxpkind)) |
| 4231 | 4233 | return -EINVAL; | |
| 4232 | rc = cgx_set_pkind(rvu_cgx_pdata(cgx_id, rvu), lmac_id, | ||
| 4233 | rxpkind); | ||
| 4234 | if (rc) | ||
| 4235 | return rc; | ||
| 4236 | } | 4234 | } |
| 4237 | 4235 | ||
| 4238 | if (dir & PKIND_TX) { | 4236 | if (dir & PKIND_TX) { |
| @@ -4241,8 +4239,19 @@ int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir, · 2 unmodified lines | |||
| 4241 | if (rc) | 4239 | if (rc) |
| 4242 | return rc; | 4240 | return rc; |
| 4243 | 4241 | ||
| 4244 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | 4242 | if (is_pf_cgxmapped(rvu, pf) && is_vf(pcifunc)) { |
| 4245 | txpkind); | 4243 | rvu_get_cgx_lmac_id(rvu->pf2cgxlmac_map[pf], &cgx_id, |
| 4244 | &lmac_id); | ||
| 4245 | cgxd = rvu_cgx_pdata(cgx_id, rvu); | ||
| 4246 | mutex_lock(&cgxd->lock); | ||
| 4247 | if (rvu_cgx_is_pkind_config_permitted(rvu, pcifunc)) | ||
| 4248 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | ||
| 4249 | txpkind); | ||
| 4250 | mutex_unlock(&cgxd->lock); | ||
| 4251 | } else { | ||
| 4252 | rvu_write64(rvu, blkaddr, NIX_AF_LFX_TX_PARSE_CFG(nixlf), | ||
| 4253 | txpkind); | ||
| 4254 | } | ||
| 4246 | } | 4255 | } |
| 4247 | 4256 | ||
| 4248 | pfvf->intf_mode = intf_mode; | 4257 | pfvf->intf_mode = intf_mode; |
| @@ -889,8 +889,8 @@ static void otx2_handle_link_event(struct otx2_nic *pf) · 888 unmodified lines | |||
| 889 | netif_carrier_on(netdev); | 889 | netif_carrier_on(netdev); |
| 890 | netif_tx_start_all_queues(netdev); | 890 | netif_tx_start_all_queues(netdev); |
| 891 | } else { | 891 | } else { |
| 892 | netif_tx_stop_all_queues(netdev); | ||
| 893 | netif_carrier_off(netdev); | 892 | netif_carrier_off(netdev); |
| 893 | netif_tx_stop_all_queues(netdev); | ||
| 894 | } | 894 | } |
| 895 | } | 895 | } |
| 896 | 896 | ||
| @@ -3467,7 +3467,7 @@ static void mtk_poll_controller(struct net_device *dev) · 3466 unmodified lines | |||
| 3467 | 3467 | ||
| 3468 | mtk_tx_irq_disable(eth, MTK_TX_DONE_INT); | 3468 | mtk_tx_irq_disable(eth, MTK_TX_DONE_INT); |
| 3469 | mtk_rx_irq_disable(eth, eth->soc->rx.irq_done_mask); | 3469 | mtk_rx_irq_disable(eth, eth->soc->rx.irq_done_mask); |
| 3470 | mtk_handle_irq_rx(eth->irq[MTK_FE_IRQ_RX], dev); | 3470 | mtk_handle_irq_rx(eth->irq[MTK_FE_IRQ_RX], eth); |
| 3471 | mtk_tx_irq_enable(eth, MTK_TX_DONE_INT); | 3471 | mtk_tx_irq_enable(eth, MTK_TX_DONE_INT); |
| 3472 | mtk_rx_irq_enable(eth, eth->soc->rx.irq_done_mask); | 3472 | mtk_rx_irq_enable(eth, eth->soc->rx.irq_done_mask); |
| 3473 | } | 3473 | } |
| @@ -984,7 +984,6 @@ struct mlx5e_priv { · 983 unmodified lines | |||
| 984 | struct mlx5e_mqprio_rl *mqprio_rl; | 984 | struct mlx5e_mqprio_rl *mqprio_rl; |
| 985 | struct dentry *dfs_root; | 985 | struct dentry *dfs_root; |
| 986 | struct mlx5_devcom_comp_dev *devcom; | 986 | struct mlx5_devcom_comp_dev *devcom; |
| 987 | struct ethtool_fec_hist_range *fec_ranges; | ||
| 988 | }; | 987 | }; |
| 989 | 988 | ||
| 990 | static inline u16 mlx5e_stats_nch_read(const struct mlx5e_priv *priv) | 989 | static inline u16 mlx5e_stats_nch_read(const struct mlx5e_priv *priv) |
| @@ -6415,14 +6415,8 @@ int mlx5e_priv_init(struct mlx5e_priv *priv, · 6414 unmodified lines | |||
| 6415 | if (!priv->channel_stats) | 6415 | if (!priv->channel_stats) |
| 6416 | goto err_free_tx_rates; | 6416 | goto err_free_tx_rates; |
| 6417 | 6417 | ||
| 6418 | priv->fec_ranges = kzalloc_objs(*priv->fec_ranges, ETHTOOL_FEC_HIST_MAX); | ||
| 6419 | if (!priv->fec_ranges) | ||
| 6420 | goto err_free_channel_stats; | ||
| 6421 | |||
| 6422 | return 0; | 6418 | return 0; |
| 6423 | 6419 | ||
| 6424 | err_free_channel_stats: | ||
| 6425 | kfree(priv->channel_stats); | ||
| 6426 | err_free_tx_rates: | 6420 | err_free_tx_rates: |
| 6427 | kfree(priv->tx_rates); | 6421 | kfree(priv->tx_rates); |
| 6428 | err_free_txq2sq_stats: | 6422 | err_free_txq2sq_stats: |
| @@ -6447,7 +6441,6 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv) · 18 unmodified lines | |||
| 6447 | if (!priv->mdev) | 6441 | if (!priv->mdev) |
| 6448 | return; | 6442 | return; |
| 6449 | 6443 | ||
| 6450 | kfree(priv->fec_ranges); | ||
| 6451 | for (i = 0; i < priv->stats_nch; i++) | 6444 | for (i = 0; i < priv->stats_nch; i++) |
| 6452 | kvfree(priv->channel_stats[i]); | 6445 | kvfree(priv->channel_stats[i]); |
| 6453 | kfree(priv->channel_stats); | 6446 | kfree(priv->channel_stats); |
| @@ -1550,7 +1550,7 @@ static bool fec_rs_validate_hist_type(int mode, int hist_type) · 1549 unmodified lines | |||
| 1550 | 1550 | ||
| 1551 | static u8 | 1551 | static u8 |
| 1552 | fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, | 1552 | fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, |
| 1553 | const struct ethtool_fec_hist_range **ranges) | 1553 | struct ethtool_fec_hist_range *ranges) |
| 1554 | { | 1554 | { |
| 1555 | struct mlx5_core_dev *mdev = priv->mdev; | 1555 | struct mlx5_core_dev *mdev = priv->mdev; |
| 1556 | u32 out[MLX5_ST_SZ_DW(pphcr_reg)] = {0}; | 1556 | u32 out[MLX5_ST_SZ_DW(pphcr_reg)] = {0}; |
| @@ -1558,8 +1558,6 @@ fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, · 1 unmodified line | |||
| 1558 | int sz = MLX5_ST_SZ_BYTES(pphcr_reg); | 1558 | int sz = MLX5_ST_SZ_BYTES(pphcr_reg); |
| 1559 | u8 hist_type, num_of_bins; | 1559 | u8 hist_type, num_of_bins; |
| 1560 | 1560 | ||
| 1561 | memset(priv->fec_ranges, 0, | ||
| 1562 | ETHTOOL_FEC_HIST_MAX * sizeof(*priv->fec_ranges)); | ||
| 1563 | MLX5_SET(pphcr_reg, in, local_port, 1); | 1561 | MLX5_SET(pphcr_reg, in, local_port, 1); |
| 1564 | if (mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPHCR, 0, 0)) | 1562 | if (mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_PPHCR, 0, 0)) |
| 1565 | return 0; | 1563 | return 0; |
| @@ -1575,12 +1573,11 @@ fec_rs_histogram_fill_ranges(struct mlx5e_priv *priv, int mode, · 9 unmodified lines | |||
| 1575 | for (int i = 0; i < num_of_bins; i++) { | 1573 | for (int i = 0; i < num_of_bins; i++) { |
| 1576 | void *bin_range = MLX5_ADDR_OF(pphcr_reg, out, bin_range[i]); | 1574 | void *bin_range = MLX5_ADDR_OF(pphcr_reg, out, bin_range[i]); |
| 1577 | 1575 | ||
| 1578 | priv->fec_ranges[i].high = MLX5_GET(bin_range_layout, bin_range, | 1576 | ranges[i].high = MLX5_GET(bin_range_layout, bin_range, |
| 1579 | high_val); | 1577 | high_val); |
| 1580 | priv->fec_ranges[i].low = MLX5_GET(bin_range_layout, bin_range, | 1578 | ranges[i].low = MLX5_GET(bin_range_layout, bin_range, |
| 1581 | low_val); | 1579 | low_val); |
| 1582 | } | 1580 | } |
| 1583 | *ranges = priv->fec_ranges; | ||
| 1584 | 1581 | ||
| 1585 | return num_of_bins; | 1582 | return num_of_bins; |
| 1586 | } | 1583 | } |
| @@ -1622,10 +1619,12 @@ static void fec_set_histograms_stats(struct mlx5e_priv *priv, int mode, · 35 unmodified lines | |||
| 1622 | case MLX5E_FEC_LLRS_272_257_1: | 1619 | case MLX5E_FEC_LLRS_272_257_1: |
| 1623 | case MLX5E_FEC_RS_544_514_INTERLEAVED_QUAD: | 1620 | case MLX5E_FEC_RS_544_514_INTERLEAVED_QUAD: |
| 1624 | num_of_bins = | 1621 | num_of_bins = |
| 1625 | fec_rs_histogram_fill_ranges(priv, mode, &hist->ranges); | 1622 | fec_rs_histogram_fill_ranges(priv, mode, hist->ranges_buf); |
| 1626 | if (num_of_bins) | 1623 | if (num_of_bins) { |
| 1624 | hist->ranges = hist->ranges_buf; | ||
| 1627 | return fec_rs_histogram_fill_stats(priv, num_of_bins, | 1625 | return fec_rs_histogram_fill_stats(priv, num_of_bins, |
| 1628 | hist); | 1626 | hist); |
| 1627 | } | ||
| 1629 | break; | 1628 | break; |
| 1630 | default: | 1629 | default: |
| 1631 | return; | 1630 | return; |
| @@ -2829,7 +2829,7 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc, · 2828 unmodified lines | |||
| 2829 | 2829 | ||
| 2830 | rxq = kvzalloc_flex(*rxq, rx_oobs, apc->rx_queue_size); | 2830 | rxq = kvzalloc_flex(*rxq, rx_oobs, apc->rx_queue_size); |
| 2831 | if (!rxq) | 2831 | if (!rxq) |
| 2832 | return NULL; | 2832 | return ERR_PTR(-ENOMEM); |
| 2833 | 2833 | ||
| 2834 | rxq->ndev = ndev; | 2834 | rxq->ndev = ndev; |
| 2835 | rxq->num_rx_buf = apc->rx_queue_size; | 2835 | rxq->num_rx_buf = apc->rx_queue_size; |
| @@ -2930,7 +2930,7 @@ out: · 94 unmodified lines | |||
| 2930 | 2930 | ||
| 2931 | mana_destroy_rxq(apc, rxq, false); | 2931 | mana_destroy_rxq(apc, rxq, false); |
| 2932 | 2932 | ||
| 2933 | return NULL; | 2933 | return ERR_PTR(err); |
| 2934 | } | 2934 | } |
| 2935 | 2935 | ||
| 2936 | static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx) | 2936 | static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx) |
| @@ -2964,8 +2964,8 @@ static int mana_add_rx_queues(struct mana_port_context *apc, · 27 unmodified lines | |||
| 2964 | 2964 | ||
| 2965 | for (i = 0; i < apc->num_queues; i++) { | 2965 | for (i = 0; i < apc->num_queues; i++) { |
| 2966 | rxq = mana_create_rxq(apc, i, &apc->eqs[i], ndev); | 2966 | rxq = mana_create_rxq(apc, i, &apc->eqs[i], ndev); |
| 2967 | if (!rxq) { | 2967 | if (IS_ERR(rxq)) { |
| 2968 | err = -ENOMEM; | 2968 | err = PTR_ERR(rxq); |
| 2969 | netdev_err(ndev, "Failed to create rxq %d : %d\n", i, err); | 2969 | netdev_err(ndev, "Failed to create rxq %d : %d\n", i, err); |
| 2970 | goto out; | 2970 | goto out; |
| 2971 | } | 2971 | } |
| @@ -6187,10 +6187,10 @@ static void nv_remove(struct pci_dev *pci_dev) · 6186 unmodified lines | |||
| 6187 | struct net_device *dev = pci_get_drvdata(pci_dev); | 6187 | struct net_device *dev = pci_get_drvdata(pci_dev); |
| 6188 | struct fe_priv *np = netdev_priv(dev); | 6188 | struct fe_priv *np = netdev_priv(dev); |
| 6189 | 6189 | ||
| 6190 | free_percpu(np->txrx_stats); | ||
| 6191 | |||
| 6192 | unregister_netdev(dev); | 6190 | unregister_netdev(dev); |
| 6193 | 6191 | ||
| 6192 | free_percpu(np->txrx_stats); | ||
| 6193 | |||
| 6194 | nv_restore_mac_addr(pci_dev); | 6194 | nv_restore_mac_addr(pci_dev); |
| 6195 | 6195 | ||
| 6196 | /* restore any phy related changes */ | 6196 | /* restore any phy related changes */ |
| @@ -107,7 +107,7 @@ static void qede_remove(struct pci_dev *pdev); · 106 unmodified lines | |||
| 107 | static void qede_shutdown(struct pci_dev *pdev); | 107 | static void qede_shutdown(struct pci_dev *pdev); |
| 108 | static void qede_link_update(void *dev, struct qed_link_output *link); | 108 | static void qede_link_update(void *dev, struct qed_link_output *link); |
| 109 | static void qede_schedule_recovery_handler(void *dev); | 109 | static void qede_schedule_recovery_handler(void *dev); |
| 110 | static void qede_recovery_handler(struct qede_dev *edev); | 110 | static bool qede_recovery_handler(struct qede_dev *edev); |
| 111 | static void qede_schedule_hw_err_handler(void *dev, | 111 | static void qede_schedule_hw_err_handler(void *dev, |
| 112 | enum qed_hw_err_type err_type); | 112 | enum qed_hw_err_type err_type); |
| 113 | static void qede_get_eth_tlv_data(void *edev, void *data); | 113 | static void qede_get_eth_tlv_data(void *edev, void *data); |
| @@ -1043,21 +1043,6 @@ void __qede_unlock(struct qede_dev *edev) · 929 unmodified lines | |||
| 1043 | mutex_unlock(&edev->qede_lock); | 1043 | mutex_unlock(&edev->qede_lock); |
| 1044 | } | 1044 | } |
| 1045 | 1045 | ||
| 1046 | /* This version of the lock should be used when acquiring the RTNL lock is also | ||
| 1047 | * needed in addition to the internal qede lock. | ||
| 1048 | */ | ||
| 1049 | static void qede_lock(struct qede_dev *edev) | ||
| 1050 | { | ||
| 1051 | rtnl_lock(); | ||
| 1052 | __qede_lock(edev); | ||
| 1053 | } | ||
| 1054 | |||
| 1055 | static void qede_unlock(struct qede_dev *edev) | ||
| 1056 | { | ||
| 1057 | __qede_unlock(edev); | ||
| 1058 | rtnl_unlock(); | ||
| 1059 | } | ||
| 1060 | |||
| 1061 | static void qede_periodic_task(struct work_struct *work) | 1046 | static void qede_periodic_task(struct work_struct *work) |
| 1062 | { | 1047 | { |
| 1063 | struct qede_dev *edev = container_of(work, struct qede_dev, | 1048 | struct qede_dev *edev = container_of(work, struct qede_dev, |
| @@ -1094,6 +1079,8 @@ static void qede_sp_task(struct work_struct *work) · 30 unmodified lines | |||
| 1094 | */ | 1079 | */ |
| 1095 | 1080 | ||
| 1096 | if (test_and_clear_bit(QEDE_SP_RECOVERY, &edev->sp_flags)) { | 1081 | if (test_and_clear_bit(QEDE_SP_RECOVERY, &edev->sp_flags)) { |
| 1082 | bool reloaded; | ||
| 1083 | |||
| 1097 | cancel_delayed_work_sync(&edev->periodic_task); | 1084 | cancel_delayed_work_sync(&edev->periodic_task); |
| 1098 | #ifdef CONFIG_QED_SRIOV | 1085 | #ifdef CONFIG_QED_SRIOV |
| 1099 | /* SRIOV must be disabled outside the lock to avoid a deadlock. | 1086 | /* SRIOV must be disabled outside the lock to avoid a deadlock. |
| @@ -1102,9 +1089,17 @@ static void qede_sp_task(struct work_struct *work) · 2 unmodified lines | |||
| 1102 | if (pci_num_vf(edev->pdev)) | 1089 | if (pci_num_vf(edev->pdev)) |
| 1103 | qede_sriov_configure(edev->pdev, 0); | 1090 | qede_sriov_configure(edev->pdev, 0); |
| 1104 | #endif | 1091 | #endif |
| 1105 | qede_lock(edev); | 1092 | rtnl_lock(); |
| 1106 | qede_recovery_handler(edev); | 1093 | __qede_lock(edev); |
| 1107 | qede_unlock(edev); | 1094 | reloaded = qede_recovery_handler(edev); |
| 1095 | __qede_unlock(edev); | ||
| 1096 | |||
| 1097 | /* The udp_tunnel core synchronously calls back into | ||
| 1098 | * qede_udp_tunnel_sync(), which takes the qede lock. | ||
| 1099 | */ | ||
| 1100 | if (reloaded) | ||
| 1101 | udp_tunnel_nic_reset_ntf(edev->ndev); | ||
| 1102 | rtnl_unlock(); | ||
| 1108 | } | 1103 | } |
| 1109 | 1104 | ||
| 1110 | __qede_lock(edev); | 1105 | __qede_lock(edev); |
| @@ -2645,9 +2640,13 @@ static void qede_recovery_failed(struct qede_dev *edev) · 1534 unmodified lines | |||
| 2645 | edev->ops->common->set_power_state(edev->cdev, PCI_D3hot); | 2640 | edev->ops->common->set_power_state(edev->cdev, PCI_D3hot); |
| 2646 | } | 2641 | } |
| 2647 | 2642 | ||
| 2648 | static void qede_recovery_handler(struct qede_dev *edev) | 2643 | /* Returns true if an open device was successfully reloaded and its |
| 2644 | * udp_tunnel ports need to be re-synced by the caller. | ||
| 2645 | */ | ||
| 2646 | static bool qede_recovery_handler(struct qede_dev *edev) | ||
| 2649 | { | 2647 | { |
| 2650 | u32 curr_state = edev->state; | 2648 | u32 curr_state = edev->state; |
| 2649 | bool reloaded = false; | ||
| 2651 | int rc; | 2650 | int rc; |
| 2652 | 2651 | ||
| 2653 | DP_NOTICE(edev, "Starting a recovery process\n"); | 2652 | DP_NOTICE(edev, "Starting a recovery process\n"); |
| @@ -2677,17 +2676,18 @@ static void qede_recovery_handler(struct qede_dev *edev) · 23 unmodified lines | |||
| 2677 | goto err; | 2676 | goto err; |
| 2678 | 2677 | ||
| 2679 | qede_config_rx_mode(edev->ndev); | 2678 | qede_config_rx_mode(edev->ndev); |
| 2680 | udp_tunnel_nic_reset_ntf(edev->ndev); | 2679 | reloaded = true; |
| 2681 | } | 2680 | } |
| 2682 | 2681 | ||
| 2683 | edev->state = curr_state; | 2682 | edev->state = curr_state; |
| 2684 | 2683 | ||
| 2685 | DP_NOTICE(edev, "Recovery handling is done\n"); | 2684 | DP_NOTICE(edev, "Recovery handling is done\n"); |
| 2686 | 2685 | ||
| 2687 | return; | 2686 | return reloaded; |
| 2688 | 2687 | ||
| 2689 | err: | 2688 | err: |
| 2690 | qede_recovery_failed(edev); | 2689 | qede_recovery_failed(edev); |
| 2690 | return false; | ||
| 2691 | } | 2691 | } |
| 2692 | 2692 | ||
| 2693 | static void qede_atomic_hw_err_handler(struct qede_dev *edev) | 2693 | static void qede_atomic_hw_err_handler(struct qede_dev *edev) |
| @@ -1623,6 +1623,9 @@ static netdev_tx_t rtase_start_xmit(struct sk_buff *skb, · 1622 unmodified lines | |||
| 1623 | err_dma_1: | 1623 | err_dma_1: |
| 1624 | ring->skbuff[entry] = NULL; | 1624 | ring->skbuff[entry] = NULL; |
| 1625 | rtase_tx_clear_range(ring, ring->cur_idx + 1, frags); | 1625 | rtase_tx_clear_range(ring, ring->cur_idx + 1, frags); |
| 1626 | if (frags) | ||
| 1627 | /* the frags were cleared above, along with the skb */ | ||
| 1628 | return NETDEV_TX_OK; | ||
| 1626 | 1629 | ||
| 1627 | err_dma_0: | 1630 | err_dma_0: |
| 1628 | tp->stats.tx_dropped++; | 1631 | tp->stats.tx_dropped++; |
| @@ -597,14 +597,13 @@ static int init_dma_desc_rings(struct net_device *netd) · 596 unmodified lines | |||
| 597 | 597 | ||
| 598 | return 0; | 598 | return 0; |
| 599 | 599 | ||
| 600 | txalloc_err: | ||
| 601 | while (queue_num--) | ||
| 602 | free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); | ||
| 603 | return ret; | ||
| 604 | |||
| 605 | rxalloc_err: | 600 | rxalloc_err: |
| 606 | while (queue_num--) | 601 | while (queue_num--) |
| 607 | free_rx_ring(priv->device, priv->rxq[queue_num], rx_rsize); | 602 | free_rx_ring(priv->device, priv->rxq[queue_num], rx_rsize); |
| 603 | queue_num = SXGBE_TX_QUEUES; | ||
| 604 | txalloc_err: | ||
| 605 | while (queue_num--) | ||
| 606 | free_tx_ring(priv->device, priv->txq[queue_num], tx_rsize); | ||
| 608 | return ret; | 607 | return ret; |
| 609 | } | 608 | } |
| 610 | 609 | ||
| @@ -1079,7 +1078,9 @@ static int sxgbe_open(struct net_device *dev) · 468 unmodified lines | |||
| 1079 | priv->dma_buf_sz = SXGBE_ALIGN(DMA_BUFFER_SIZE); | 1078 | priv->dma_buf_sz = SXGBE_ALIGN(DMA_BUFFER_SIZE); |
| 1080 | priv->tx_tc = TC_DEFAULT; | 1079 | priv->tx_tc = TC_DEFAULT; |
| 1081 | priv->rx_tc = TC_DEFAULT; | 1080 | priv->rx_tc = TC_DEFAULT; |
| 1082 | init_dma_desc_rings(dev); | 1081 | ret = init_dma_desc_rings(dev); |
| 1082 | if (ret) | ||
| 1083 | goto init_phy_error; | ||
| 1083 | 1084 | ||
| 1084 | /* DMA initialization and SW reset */ | 1085 | /* DMA initialization and SW reset */ |
| 1085 | ret = sxgbe_init_dma_engine(priv); | 1086 | ret = sxgbe_init_dma_engine(priv); |
| @@ -1188,6 +1189,7 @@ static int sxgbe_open(struct net_device *dev) · 102 unmodified lines | |||
| 1188 | 1189 | ||
| 1189 | init_error: | 1190 | init_error: |
| 1190 | free_dma_desc_resources(priv); | 1191 | free_dma_desc_resources(priv); |
| 1192 | init_phy_error: | ||
| 1191 | if (dev->phydev) | 1193 | if (dev->phydev) |
| 1192 | phy_disconnect(dev->phydev); | 1194 | phy_disconnect(dev->phydev); |
| 1193 | phy_error: | 1195 | phy_error: |
| @@ -755,7 +755,8 @@ static int stmmac_hwtstamp_set(struct net_device *dev, · 754 unmodified lines | |||
| 755 | config->rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; | 755 | config->rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; |
| 756 | ptp_v2 = PTP_TCR_TSVER2ENA; | 756 | ptp_v2 = PTP_TCR_TSVER2ENA; |
| 757 | snap_type_sel = PTP_TCR_SNAPTYPSEL_1; | 757 | snap_type_sel = PTP_TCR_SNAPTYPSEL_1; |
| 758 | if (priv->synopsys_id < DWMAC_CORE_4_10) | 758 | if (priv->synopsys_id < DWMAC_CORE_3_70 && |
| 759 | priv->plat->core_type != DWMAC_CORE_XGMAC) | ||
| 759 | ts_event_en = PTP_TCR_TSEVNTENA; | 760 | ts_event_en = PTP_TCR_TSEVNTENA; |
| 760 | ptp_over_ipv4_udp = PTP_TCR_TSIPV4ENA; | 761 | ptp_over_ipv4_udp = PTP_TCR_TSIPV4ENA; |
| 761 | ptp_over_ipv6_udp = PTP_TCR_TSIPV6ENA; | 762 | ptp_over_ipv6_udp = PTP_TCR_TSIPV6ENA; |
| @@ -1606,6 +1606,8 @@ static netdev_tx_t wx_xmit_frame_ring(struct sk_buff *skb, · 1605 unmodified lines | |||
| 1606 | if (skb_vlan_tag_present(skb)) { | 1606 | if (skb_vlan_tag_present(skb)) { |
| 1607 | tx_flags |= skb_vlan_tag_get(skb) << WX_TX_FLAGS_VLAN_SHIFT; | 1607 | tx_flags |= skb_vlan_tag_get(skb) << WX_TX_FLAGS_VLAN_SHIFT; |
| 1608 | tx_flags |= WX_TX_FLAGS_HW_VLAN; | 1608 | tx_flags |= WX_TX_FLAGS_HW_VLAN; |
| 1609 | } else if (eth_type_vlan(skb->protocol)) { | ||
| 1610 | tx_flags |= WX_TX_FLAGS_SW_VLAN; | ||
| 1609 | } | 1611 | } |
| 1610 | 1612 | ||
| 1611 | if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) && | 1613 | if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) && |
| @@ -1875,8 +1875,8 @@ struct phylink *phylink_create(struct phylink_config *config, · 1874 unmodified lines | |||
| 1875 | } else if (config->type == PHYLINK_DEV) { | 1875 | } else if (config->type == PHYLINK_DEV) { |
| 1876 | pl->dev = config->dev; | 1876 | pl->dev = config->dev; |
| 1877 | } else { | 1877 | } else { |
| 1878 | kfree(pl); | 1878 | ret = -EINVAL; |
| 1879 | return ERR_PTR(-EINVAL); | 1879 | goto free_pl; |
| 1880 | } | 1880 | } |
| 1881 | 1881 | ||
| 1882 | pl->mac_supports_eee_ops = phylink_mac_implements_lpi(mac_ops); | 1882 | pl->mac_supports_eee_ops = phylink_mac_implements_lpi(mac_ops); |
| @@ -1909,28 +1909,29 @@ struct phylink *phylink_create(struct phylink_config *config, · 26 unmodified lines | |||
| 1909 | phylink_validate(pl, pl->supported, &pl->link_config); | 1909 | phylink_validate(pl, pl->supported, &pl->link_config); |
| 1910 | 1910 | ||
| 1911 | ret = phylink_parse_mode(pl, fwnode); | 1911 | ret = phylink_parse_mode(pl, fwnode); |
| 1912 | if (ret < 0) { | 1912 | if (ret < 0) |
| 1913 | kfree(pl); | 1913 | goto free_pl; |
| 1914 | return ERR_PTR(ret); | ||
| 1915 | } | ||
| 1916 | 1914 | ||
| 1917 | if (pl->cfg_link_an_mode == MLO_AN_FIXED) { | 1915 | if (pl->cfg_link_an_mode == MLO_AN_FIXED) { |
| 1918 | ret = phylink_parse_fixedlink(pl, fwnode); | 1916 | ret = phylink_parse_fixedlink(pl, fwnode); |
| 1919 | if (ret < 0) { | 1917 | if (ret < 0) |
| 1920 | kfree(pl); | 1918 | goto release_link_gpio; |
| 1921 | return ERR_PTR(ret); | ||
| 1922 | } | ||
| 1923 | } | 1919 | } |
| 1924 | 1920 | ||
| 1925 | pl->req_link_an_mode = pl->cfg_link_an_mode; | 1921 | pl->req_link_an_mode = pl->cfg_link_an_mode; |
| 1926 | 1922 | ||
| 1927 | ret = phylink_register_sfp(pl, fwnode); | 1923 | ret = phylink_register_sfp(pl, fwnode); |
| 1928 | if (ret < 0) { | 1924 | if (ret < 0) |
| 1929 | kfree(pl); | 1925 | goto release_link_gpio; |
| 1930 | return ERR_PTR(ret); | ||
| 1931 | } | ||
| 1932 | 1926 | ||
| 1933 | return pl; | 1927 | return pl; |
| 1928 | |||
| 1929 | release_link_gpio: | ||
| 1930 | if (pl->link_gpio) | ||
| 1931 | gpiod_put(pl->link_gpio); | ||
| 1932 | free_pl: | ||
| 1933 | kfree(pl); | ||
| 1934 | return ERR_PTR(ret); | ||
| 1934 | } | 1935 | } |
| 1935 | EXPORT_SYMBOL_GPL(phylink_create); | 1936 | EXPORT_SYMBOL_GPL(phylink_create); |
| 1936 | 1937 | ||
| @@ -145,8 +145,6 @@ struct tun_file { · 144 unmodified lines | |||
| 145 | struct list_head next; | 145 | struct list_head next; |
| 146 | struct tun_struct *detached; | 146 | struct tun_struct *detached; |
| 147 | struct ptr_ring tx_ring; | 147 | struct ptr_ring tx_ring; |
| 148 | /* Protected by tx_ring.consumer_lock */ | ||
| 149 | int cons_cnt; | ||
| 150 | struct xdp_rxq_info xdp_rxq; | 148 | struct xdp_rxq_info xdp_rxq; |
| 151 | }; | 149 | }; |
| 152 | 150 | ||
| @@ -590,13 +588,8 @@ static void __tun_detach(struct tun_file *tfile, bool clean) · 437 unmodified lines | |||
| 590 | rcu_assign_pointer(tun->tfiles[index], | 588 | rcu_assign_pointer(tun->tfiles[index], |
| 591 | tun->tfiles[tun->numqueues - 1]); | 589 | tun->tfiles[tun->numqueues - 1]); |
| 592 | ntfile = rtnl_dereference(tun->tfiles[index]); | 590 | ntfile = rtnl_dereference(tun->tfiles[index]); |
| 593 | spin_lock(&ntfile->tx_ring.consumer_lock); | ||
| 594 | ntfile->queue_index = index; | 591 | ntfile->queue_index = index; |
| 595 | ntfile->xdp_rxq.queue_index = index; | 592 | ntfile->xdp_rxq.queue_index = index; |
| 596 | ntfile->cons_cnt = 0; | ||
| 597 | if (__ptr_ring_empty(&ntfile->tx_ring)) | ||
| 598 | netif_wake_subqueue(tun->dev, index); | ||
| 599 | spin_unlock(&ntfile->tx_ring.consumer_lock); | ||
| 600 | rcu_assign_pointer(tun->tfiles[tun->numqueues - 1], | 593 | rcu_assign_pointer(tun->tfiles[tun->numqueues - 1], |
| 601 | NULL); | 594 | NULL); |
| 602 | 595 | ||
| @@ -737,9 +730,6 @@ static int tun_attach(struct tun_struct *tun, struct file *file, · 134 unmodified lines | |||
| 737 | goto out; | 730 | goto out; |
| 738 | } | 731 | } |
| 739 | 732 | ||
| 740 | spin_lock(&tfile->tx_ring.consumer_lock); | ||
| 741 | tfile->cons_cnt = 0; | ||
| 742 | spin_unlock(&tfile->tx_ring.consumer_lock); | ||
| 743 | tfile->queue_index = tun->numqueues; | 733 | tfile->queue_index = tun->numqueues; |
| 744 | tfile->socket.sk->sk_shutdown &= ~RCV_SHUTDOWN; | 734 | tfile->socket.sk->sk_shutdown &= ~RCV_SHUTDOWN; |
| 745 | 735 | ||
| @@ -1018,7 +1008,6 @@ static netdev_tx_t tun_net_xmit(struct sk_buff *skb, struct net_device *dev) · 272 unmodified lines | |||
| 1018 | struct netdev_queue *queue; | 1008 | struct netdev_queue *queue; |
| 1019 | struct tun_file *tfile; | 1009 | struct tun_file *tfile; |
| 1020 | int len = skb->len; | 1010 | int len = skb->len; |
| 1021 | int ret; | ||
| 1022 | 1011 | ||
| 1023 | rcu_read_lock(); | 1012 | rcu_read_lock(); |
| 1024 | tfile = rcu_dereference(tun->tfiles[txq]); | 1013 | tfile = rcu_dereference(tun->tfiles[txq]); |
| @@ -1073,33 +1062,13 @@ static netdev_tx_t tun_net_xmit(struct sk_buff *skb, struct net_device *dev) · 48 unmodified lines | |||
| 1073 | 1062 | ||
| 1074 | nf_reset_ct(skb); | 1063 | nf_reset_ct(skb); |
| 1075 | 1064 | ||
| 1076 | queue = netdev_get_tx_queue(dev, txq); | 1065 | if (ptr_ring_produce(&tfile->tx_ring, skb)) { |
| 1077 | |||
| 1078 | spin_lock(&tfile->tx_ring.producer_lock); | ||
| 1079 | ret = __ptr_ring_produce(&tfile->tx_ring, skb); | ||
| 1080 | if (!qdisc_txq_has_no_queue(queue) && | ||
| 1081 | __ptr_ring_check_produce(&tfile->tx_ring) == -ENOSPC) { | ||
| 1082 | netif_tx_stop_queue(queue); | ||
| 1083 | /* Paired with smp_mb() in __tun_wake_queue() */ | ||
| 1084 | smp_mb__after_atomic(); | ||
| 1085 | if (!__ptr_ring_check_produce(&tfile->tx_ring)) | ||
| 1086 | netif_tx_wake_queue(queue); | ||
| 1087 | } | ||
| 1088 | spin_unlock(&tfile->tx_ring.producer_lock); | ||
| 1089 | |||
| 1090 | if (ret) { | ||
| 1091 | /* This should be a rare case if a qdisc is present, but | ||
| 1092 | * can happen due to lltx. | ||
| 1093 | * Since skb_tx_timestamp(), skb_orphan(), | ||
| 1094 | * run_ebpf_filter() and pskb_trim() could have tinkered | ||
| 1095 | * with the SKB, returning NETDEV_TX_BUSY is unsafe and | ||
| 1096 | * we must drop instead. | ||
| 1097 | */ | ||
| 1098 | drop_reason = SKB_DROP_REASON_FULL_RING; | 1066 | drop_reason = SKB_DROP_REASON_FULL_RING; |
| 1099 | goto drop; | 1067 | goto drop; |
| 1100 | } | 1068 | } |
| 1101 | 1069 | ||
| 1102 | /* dev->lltx requires to do our own update of trans_start */ | 1070 | /* dev->lltx requires to do our own update of trans_start */ |
| 1071 | queue = netdev_get_tx_queue(dev, txq); | ||
| 1103 | txq_trans_cond_update(queue); | 1072 | txq_trans_cond_update(queue); |
| 1104 | 1073 | ||
| 1105 | /* Notify and wake up reader process */ | 1074 | /* Notify and wake up reader process */ |
| @@ -2147,46 +2116,13 @@ done: · 1041 unmodified lines | |||
| 2147 | return total; | 2116 | return total; |
| 2148 | } | 2117 | } |
| 2149 | 2118 | ||
| 2150 | /* Callers must hold ring.consumer_lock */ | 2119 | static void *tun_ring_recv(struct tun_file *tfile, int noblock, int *err) |
| 2151 | static void __tun_wake_queue(struct tun_struct *tun, | ||
| 2152 | struct tun_file *tfile, int consumed) | ||
| 2153 | { | ||
| 2154 | struct netdev_queue *txq = netdev_get_tx_queue(tun->dev, | ||
| 2155 | tfile->queue_index); | ||
| 2156 | |||
| 2157 | /* Paired with smp_mb__after_atomic() in tun_net_xmit() */ | ||
| 2158 | smp_mb(); | ||
| 2159 | if (netif_tx_queue_stopped(txq)) { | ||
| 2160 | tfile->cons_cnt += consumed; | ||
| 2161 | if (tfile->cons_cnt >= tfile->tx_ring.size / 2 || | ||
| 2162 | __ptr_ring_empty(&tfile->tx_ring)) { | ||
| 2163 | netif_tx_wake_queue(txq); | ||
| 2164 | tfile->cons_cnt = 0; | ||
| 2165 | } | ||
| 2166 | } | ||
| 2167 | } | ||
| 2168 | |||
| 2169 | static void *tun_ring_consume(struct tun_struct *tun, struct tun_file *tfile) | ||
| 2170 | { | ||
| 2171 | void *ptr; | ||
| 2172 | |||
| 2173 | spin_lock(&tfile->tx_ring.consumer_lock); | ||
| 2174 | ptr = __ptr_ring_consume(&tfile->tx_ring); | ||
| 2175 | if (ptr) | ||
| 2176 | __tun_wake_queue(tun, tfile, 1); | ||
| 2177 | |||
| 2178 | spin_unlock(&tfile->tx_ring.consumer_lock); | ||
| 2179 | return ptr; | ||
| 2180 | } | ||
| 2181 | |||
| 2182 | static void *tun_ring_recv(struct tun_struct *tun, struct tun_file *tfile, | ||
| 2183 | int noblock, int *err) | ||
| 2184 | { | 2120 | { |
| 2185 | DECLARE_WAITQUEUE(wait, current); | 2121 | DECLARE_WAITQUEUE(wait, current); |
| 2186 | void *ptr = NULL; | 2122 | void *ptr = NULL; |
| 2187 | int error = 0; | 2123 | int error = 0; |
| 2188 | 2124 | ||
| 2189 | ptr = tun_ring_consume(tun, tfile); | 2125 | ptr = ptr_ring_consume(&tfile->tx_ring); |
| 2190 | if (ptr) | 2126 | if (ptr) |
| 2191 | goto out; | 2127 | goto out; |
| 2192 | if (noblock) { | 2128 | if (noblock) { |
| @@ -2198,7 +2134,7 @@ static void *tun_ring_recv(struct tun_struct *tun, struct tun_file *tfile, · 5 unmodified lines | |||
| 2198 | 2134 | ||
| 2199 | while (1) { | 2135 | while (1) { |
| 2200 | set_current_state(TASK_INTERRUPTIBLE); | 2136 | set_current_state(TASK_INTERRUPTIBLE); |
| 2201 | ptr = tun_ring_consume(tun, tfile); | 2137 | ptr = ptr_ring_consume(&tfile->tx_ring); |
| 2202 | if (ptr) | 2138 | if (ptr) |
| 2203 | break; | 2139 | break; |
| 2204 | if (signal_pending(current)) { | 2140 | if (signal_pending(current)) { |
| @@ -2235,7 +2171,7 @@ static ssize_t tun_do_read(struct tun_struct *tun, struct tun_file *tfile, · 30 unmodified lines | |||
| 2235 | 2171 | ||
| 2236 | if (!ptr) { | 2172 | if (!ptr) { |
| 2237 | /* Read frames from ring */ | 2173 | /* Read frames from ring */ |
| 2238 | ptr = tun_ring_recv(tun, tfile, noblock, &err); | 2174 | ptr = tun_ring_recv(tfile, noblock, &err); |
| 2239 | if (!ptr) | 2175 | if (!ptr) |
| 2240 | return err; | 2176 | return err; |
| 2241 | } | 2177 | } |
| @@ -3690,16 +3626,6 @@ static int tun_queue_resize(struct tun_struct *tun) · 1448 unmodified lines | |||
| 3690 | dev->tx_queue_len, GFP_KERNEL, | 3626 | dev->tx_queue_len, GFP_KERNEL, |
| 3691 | tun_ptr_free); | 3627 | tun_ptr_free); |
| 3692 | 3628 | ||
| 3693 | if (!ret) { | ||
| 3694 | for (i = 0; i < tun->numqueues; i++) { | ||
| 3695 | tfile = rtnl_dereference(tun->tfiles[i]); | ||
| 3696 | spin_lock(&tfile->tx_ring.consumer_lock); | ||
| 3697 | netif_wake_subqueue(tun->dev, tfile->queue_index); | ||
| 3698 | tfile->cons_cnt = 0; | ||
| 3699 | spin_unlock(&tfile->tx_ring.consumer_lock); | ||
| 3700 | } | ||
| 3701 | } | ||
| 3702 | |||
| 3703 | kfree(rings); | 3629 | kfree(rings); |
| 3704 | return ret; | 3630 | return ret; |
| 3705 | } | 3631 | } |
| @@ -3808,29 +3734,6 @@ struct ptr_ring *tun_get_tx_ring(struct file *file) · 102 unmodified lines | |||
| 3808 | } | 3734 | } |
| 3809 | EXPORT_SYMBOL_GPL(tun_get_tx_ring); | 3735 | EXPORT_SYMBOL_GPL(tun_get_tx_ring); |
| 3810 | 3736 | ||
| 3811 | /* Callers must hold ring.consumer_lock */ | ||
| 3812 | void tun_wake_queue(struct file *file, int consumed) | ||
| 3813 | { | ||
| 3814 | struct tun_file *tfile; | ||
| 3815 | struct tun_struct *tun; | ||
| 3816 | |||
| 3817 | if (file->f_op != &tun_fops) | ||
| 3818 | return; | ||
| 3819 | |||
| 3820 | tfile = file->private_data; | ||
| 3821 | if (!tfile) | ||
| 3822 | return; | ||
| 3823 | |||
| 3824 | rcu_read_lock(); | ||
| 3825 | |||
| 3826 | tun = rcu_dereference(tfile->tun); | ||
| 3827 | if (tun) | ||
| 3828 | __tun_wake_queue(tun, tfile, consumed); | ||
| 3829 | |||
| 3830 | rcu_read_unlock(); | ||
| 3831 | } | ||
| 3832 | EXPORT_SYMBOL_GPL(tun_wake_queue); | ||
| 3833 | |||
| 3834 | module_init(tun_init); | 3737 | module_init(tun_init); |
| 3835 | module_exit(tun_cleanup); | 3738 | module_exit(tun_cleanup); |
| 3836 | MODULE_DESCRIPTION(DRV_DESCRIPTION); | 3739 | MODULE_DESCRIPTION(DRV_DESCRIPTION); |
| @@ -756,7 +756,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, · 755 unmodified lines | |||
| 756 | u32 frame_sz; | 756 | u32 frame_sz; |
| 757 | 757 | ||
| 758 | if (skb_shared(skb) || skb_head_is_locked(skb) || | 758 | if (skb_shared(skb) || skb_head_is_locked(skb) || |
| 759 | skb_shinfo(skb)->nr_frags || | 759 | skb_is_nonlinear(skb) || |
| 760 | skb_headroom(skb) < XDP_PACKET_HEADROOM) { | 760 | skb_headroom(skb) < XDP_PACKET_HEADROOM) { |
| 761 | if (skb_pp_cow_data(rq->page_pool, pskb, XDP_PACKET_HEADROOM)) | 761 | if (skb_pp_cow_data(rq->page_pool, pskb, XDP_PACKET_HEADROOM)) |
| 762 | goto drop; | 762 | goto drop; |
| @@ -771,7 +771,7 @@ static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, · 8 unmodified lines | |||
| 771 | xdp_prepare_buff(xdp, skb->head, skb_headroom(skb), | 771 | xdp_prepare_buff(xdp, skb->head, skb_headroom(skb), |
| 772 | skb_headlen(skb), true); | 772 | skb_headlen(skb), true); |
| 773 | 773 | ||
| 774 | if (skb_is_nonlinear(skb)) { | 774 | if (skb_shinfo(skb)->nr_frags) { |
| 775 | skb_shinfo(skb)->xdp_frags_size = skb->data_len; | 775 | skb_shinfo(skb)->xdp_frags_size = skb->data_len; |
| 776 | xdp_buff_set_frags_flag(xdp); | 776 | xdp_buff_set_frags_flag(xdp); |
| 777 | } else { | 777 | } else { |
| @@ -1850,7 +1850,7 @@ static int arp_reduce(struct net_device *dev, struct sk_buff *skb, __be32 vni) · 1849 unmodified lines | |||
| 1850 | if (dev->flags & IFF_NOARP) | 1850 | if (dev->flags & IFF_NOARP) |
| 1851 | goto out; | 1851 | goto out; |
| 1852 | 1852 | ||
| 1853 | if (!pskb_may_pull(skb, arp_hdr_len(dev))) { | 1853 | if (!pskb_network_may_pull(skb, arp_hdr_len(dev))) { |
| 1854 | dev_dstats_tx_dropped(dev); | 1854 | dev_dstats_tx_dropped(dev); |
| 1855 | vxlan_vnifilter_count(vxlan, vni, NULL, | 1855 | vxlan_vnifilter_count(vxlan, vni, NULL, |
| 1856 | VXLAN_VNI_STATS_TX_DROPS, 0); | 1856 | VXLAN_VNI_STATS_TX_DROPS, 0); |
| @@ -2111,7 +2111,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 254 unmodified lines | |||
| 2111 | { | 2111 | { |
| 2112 | struct iphdr *pip; | 2112 | struct iphdr *pip; |
| 2113 | 2113 | ||
| 2114 | if (!pskb_may_pull(skb, sizeof(struct iphdr))) | 2114 | if (!pskb_network_may_pull(skb, sizeof(struct iphdr))) |
| 2115 | return false; | 2115 | return false; |
| 2116 | pip = ip_hdr(skb); | 2116 | pip = ip_hdr(skb); |
| 2117 | n = neigh_lookup(&arp_tbl, &pip->daddr, dev); | 2117 | n = neigh_lookup(&arp_tbl, &pip->daddr, dev); |
| @@ -2137,7 +2137,7 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 19 unmodified lines | |||
| 2137 | */ | 2137 | */ |
| 2138 | if (!ipv6_mod_enabled()) | 2138 | if (!ipv6_mod_enabled()) |
| 2139 | return false; | 2139 | return false; |
| 2140 | if (!pskb_may_pull(skb, sizeof(struct ipv6hdr))) | 2140 | if (!pskb_network_may_pull(skb, sizeof(struct ipv6hdr))) |
| 2141 | return false; | 2141 | return false; |
| 2142 | pip6 = ipv6_hdr(skb); | 2142 | pip6 = ipv6_hdr(skb); |
| 2143 | n = neigh_lookup(&nd_tbl, &pip6->daddr, dev); | 2143 | n = neigh_lookup(&nd_tbl, &pip6->daddr, dev); |
| @@ -2159,13 +2159,19 @@ static bool route_shortcircuit(struct net_device *dev, struct sk_buff *skb) · 15 unmodified lines | |||
| 2159 | } | 2159 | } |
| 2160 | 2160 | ||
| 2161 | if (n) { | 2161 | if (n) { |
| 2162 | u8 haddr[ETH_ALEN]; | ||
| 2162 | bool diff; | 2163 | bool diff; |
| 2163 | 2164 | ||
| 2164 | diff = !ether_addr_equal(eth_hdr(skb)->h_dest, n->ha); | 2165 | neigh_ha_snapshot(haddr, n, dev); |
| 2166 | diff = !ether_addr_equal_unaligned(eth_hdr(skb)->h_dest, haddr); | ||
| 2165 | if (diff) { | 2167 | if (diff) { |
| 2168 | if (skb_cow_head(skb, 0)) { | ||
| 2169 | neigh_release(n); | ||
| 2170 | return false; | ||
| 2171 | } | ||
| 2166 | memcpy(eth_hdr(skb)->h_source, eth_hdr(skb)->h_dest, | 2172 | memcpy(eth_hdr(skb)->h_source, eth_hdr(skb)->h_dest, |
| 2167 | dev->addr_len); | 2173 | dev->addr_len); |
| 2168 | memcpy(eth_hdr(skb)->h_dest, n->ha, dev->addr_len); | 2174 | memcpy(eth_hdr(skb)->h_dest, haddr, dev->addr_len); |
| 2169 | } | 2175 | } |
| 2170 | neigh_release(n); | 2176 | neigh_release(n); |
| 2171 | return diff; | 2177 | return diff; |
| @@ -2757,8 +2763,8 @@ static netdev_tx_t vxlan_xmit(struct sk_buff *skb, struct net_device *dev) · 585 unmodified lines | |||
| 2757 | return arp_reduce(dev, skb, vni); | 2763 | return arp_reduce(dev, skb, vni); |
| 2758 | #if IS_ENABLED(CONFIG_IPV6) | 2764 | #if IS_ENABLED(CONFIG_IPV6) |
| 2759 | else if (ntohs(eth->h_proto) == ETH_P_IPV6 && | 2765 | else if (ntohs(eth->h_proto) == ETH_P_IPV6 && |
| 2760 | pskb_may_pull(skb, sizeof(struct ipv6hdr) + | 2766 | pskb_network_may_pull(skb, sizeof(struct ipv6hdr) + |
| 2761 | sizeof(struct nd_msg)) && | 2767 | sizeof(struct nd_msg)) && |
| 2762 | ipv6_hdr(skb)->nexthdr == IPPROTO_ICMPV6) { | 2768 | ipv6_hdr(skb)->nexthdr == IPPROTO_ICMPV6) { |
| 2763 | struct nd_msg *m = (struct nd_msg *)(ipv6_hdr(skb) + 1); | 2769 | struct nd_msg *m = (struct nd_msg *)(ipv6_hdr(skb) + 1); |
| 2764 | 2770 | ||
| @@ -2796,6 +2802,7 @@ static netdev_tx_t vxlan_xmit(struct sk_buff *skb, struct net_device *dev) · 31 unmodified lines | |||
| 2796 | (ntohs(eth->h_proto) == ETH_P_IP || | 2802 | (ntohs(eth->h_proto) == ETH_P_IP || |
| 2797 | ntohs(eth->h_proto) == ETH_P_IPV6)) { | 2803 | ntohs(eth->h_proto) == ETH_P_IPV6)) { |
| 2798 | did_rsc = route_shortcircuit(dev, skb); | 2804 | did_rsc = route_shortcircuit(dev, skb); |
| 2805 | eth = eth_hdr(skb); | ||
| 2799 | if (did_rsc) | 2806 | if (did_rsc) |
| 2800 | f = vxlan_find_mac_tx(vxlan, eth->h_dest, vni); | 2807 | f = vxlan_find_mac_tx(vxlan, eth->h_dest, vni); |
| 2801 | } | 2808 | } |
| @@ -1631,7 +1631,7 @@ struct vxlan_mdb_entry *vxlan_mdb_entry_skb_get(struct vxlan_dev *vxlan, · 1630 unmodified lines | |||
| 1631 | 1631 | ||
| 1632 | switch (skb->protocol) { | 1632 | switch (skb->protocol) { |
| 1633 | case htons(ETH_P_IP): | 1633 | case htons(ETH_P_IP): |
| 1634 | if (!pskb_may_pull(skb, sizeof(struct iphdr))) | 1634 | if (!pskb_network_may_pull(skb, sizeof(struct iphdr))) |
| 1635 | return NULL; | 1635 | return NULL; |
| 1636 | group.dst.sa.sa_family = AF_INET; | 1636 | group.dst.sa.sa_family = AF_INET; |
| 1637 | group.dst.sin.sin_addr.s_addr = ip_hdr(skb)->daddr; | 1637 | group.dst.sin.sin_addr.s_addr = ip_hdr(skb)->daddr; |
| @@ -1640,7 +1640,7 @@ struct vxlan_mdb_entry *vxlan_mdb_entry_skb_get(struct vxlan_dev *vxlan, · 2 unmodified lines | |||
| 1640 | break; | 1640 | break; |
| 1641 | #if IS_ENABLED(CONFIG_IPV6) | 1641 | #if IS_ENABLED(CONFIG_IPV6) |
| 1642 | case htons(ETH_P_IPV6): | 1642 | case htons(ETH_P_IPV6): |
| 1643 | if (!pskb_may_pull(skb, sizeof(struct ipv6hdr))) | 1643 | if (!pskb_network_may_pull(skb, sizeof(struct ipv6hdr))) |
| 1644 | return NULL; | 1644 | return NULL; |
| 1645 | group.dst.sa.sa_family = AF_INET6; | 1645 | group.dst.sa.sa_family = AF_INET6; |
| 1646 | group.dst.sin6.sin6_addr = ipv6_hdr(skb)->daddr; | 1646 | group.dst.sin6.sin6_addr = ipv6_hdr(skb)->daddr; |
| @@ -1544,6 +1544,8 @@ static void ath12k_core_pre_reconfigure_recovery(struct ath12k_base *ab) · 1543 unmodified lines | |||
| 1544 | } | 1544 | } |
| 1545 | 1545 | ||
| 1546 | wiphy_unlock(ah->hw->wiphy); | 1546 | wiphy_unlock(ah->hw->wiphy); |
| 1547 | |||
| 1548 | complete(&ah->peer_ml_id_done); | ||
| 1547 | } | 1549 | } |
| 1548 | 1550 | ||
| 1549 | wake_up(&ab->wmi_ab.tx_credits_wq); | 1551 | wake_up(&ab->wmi_ab.tx_credits_wq); |
| @@ -72,6 +72,7 @@ · 71 unmodified lines | |||
| 72 | 72 | ||
| 73 | #define ATH12K_MAX_MLO_PEERS 256 | 73 | #define ATH12K_MAX_MLO_PEERS 256 |
| 74 | #define ATH12K_MLO_PEER_ID_INVALID 0xFFFF | 74 | #define ATH12K_MLO_PEER_ID_INVALID 0xFFFF |
| 75 | #define ATH12K_MLO_PEER_ID_PENDING 0xFFFE | ||
| 75 | 76 | ||
| 76 | #define ATH12K_INVALID_RSSI_FULL -1 | 77 | #define ATH12K_INVALID_RSSI_FULL -1 |
| 77 | #define ATH12K_INVALID_RSSI_EMPTY -128 | 78 | #define ATH12K_INVALID_RSSI_EMPTY -128 |
| @@ -793,6 +794,8 @@ struct ath12k_hw { · 715 unmodified lines | |||
| 793 | enum ath12k_hw_state state; | 794 | enum ath12k_hw_state state; |
| 794 | bool regd_updated; | 795 | bool regd_updated; |
| 795 | bool use_6ghz_regd; | 796 | bool use_6ghz_regd; |
| 797 | bool host_alloc_ml_id; | ||
| 798 | struct completion peer_ml_id_done; | ||
| 796 | 799 | ||
| 797 | u8 num_radio; | 800 | u8 num_radio; |
| 798 | 801 | ||
| @@ -6,6 +6,7 @@ · 5 unmodified lines | |||
| 6 | 6 | ||
| 7 | #include "core.h" | 7 | #include "core.h" |
| 8 | #include "peer.h" | 8 | #include "peer.h" |
| 9 | #include "dp_peer.h" | ||
| 9 | #include "htc.h" | 10 | #include "htc.h" |
| 10 | #include "dp_htt.h" | 11 | #include "dp_htt.h" |
| 11 | #include "debugfs_htt_stats.h" | 12 | #include "debugfs_htt_stats.h" |
| @@ -575,6 +576,51 @@ exit: · 563 unmodified lines | |||
| 575 | rcu_read_unlock(); | 576 | rcu_read_unlock(); |
| 576 | } | 577 | } |
| 577 | 578 | ||
| 579 | static void ath12k_dp_htt_mlo_peer_map_handler(struct ath12k_base *ab, | ||
| 580 | struct sk_buff *skb) | ||
| 581 | { | ||
| 582 | struct htt_resp_msg *resp = (struct htt_resp_msg *)skb->data; | ||
| 583 | struct htt_t2h_mlo_peer_map_event *ev = &resp->mlo_peer_map_ev; | ||
| 584 | u16 raw_peer_id, peer_id, addr_h16; | ||
| 585 | u8 peer_addr[ETH_ALEN]; | ||
| 586 | int ret; | ||
| 587 | |||
| 588 | if (skb->len < sizeof(*ev)) { | ||
| 589 | ath12k_warn(ab, "unexpected htt mlo peer map event len %u\n", | ||
| 590 | skb->len); | ||
| 591 | return; | ||
| 592 | } | ||
| 593 | |||
| 594 | raw_peer_id = le32_get_bits(ev->info0, | ||
| 595 | HTT_T2H_MLO_PEER_MAP_INFO0_MLO_PEER_ID); | ||
| 596 | peer_id = raw_peer_id | ATH12K_PEER_ML_ID_VALID; | ||
| 597 | |||
| 598 | addr_h16 = le32_get_bits(ev->info1, | ||
| 599 | HTT_T2H_MLO_PEER_MAP_INFO1_MAC_ADDR_H16); | ||
| 600 | ath12k_dp_get_mac_addr(le32_to_cpu(ev->mac_addr_l32), addr_h16, | ||
| 601 | peer_addr); | ||
| 602 | |||
| 603 | ath12k_dbg(ab, ATH12K_DBG_DP_HTT, "htt mlo peer map peer %pM id %u\n", | ||
| 604 | peer_addr, peer_id); | ||
| 605 | |||
| 606 | /* | ||
| 607 | * Fix up the dp_peer entry created with ATH12K_MLO_PEER_ID_PENDING | ||
| 608 | * earlier; on chips with host_alloc_ml_id == false this is the only | ||
| 609 | * point at which the host learns the firmware-assigned ID. Chips | ||
| 610 | * that allocate the ID on the host also receive this event but the | ||
| 611 | * firmware-reported ID matches the host-allocated one, so there is | ||
| 612 | * nothing to fix up. | ||
| 613 | */ | ||
| 614 | if (!ab->hw_params->host_alloc_ml_id) { | ||
| 615 | ret = ath12k_dp_peer_fixup_peer_id(ab, peer_addr, | ||
| 616 | peer_id); | ||
| 617 | if (ret) | ||
| 618 | ath12k_warn(ab, | ||
| 619 | "failed to fix up peer id %u for dp peer %pM: %d\n", | ||
| 620 | peer_id, peer_addr, ret); | ||
| 621 | } | ||
| 622 | } | ||
| 623 | |||
| 578 | void ath12k_dp_htt_htc_t2h_msg_handler(struct ath12k_base *ab, | 624 | void ath12k_dp_htt_htc_t2h_msg_handler(struct ath12k_base *ab, |
| 579 | struct sk_buff *skb) | 625 | struct sk_buff *skb) |
| 580 | { | 626 | { |
| @@ -659,6 +705,9 @@ void ath12k_dp_htt_htc_t2h_msg_handler(struct ath12k_base *ab, · 78 unmodified lines | |||
| 659 | case HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND: | 705 | case HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND: |
| 660 | ath12k_htt_mlo_offset_event_handler(ab, skb); | 706 | ath12k_htt_mlo_offset_event_handler(ab, skb); |
| 661 | break; | 707 | break; |
| 708 | case HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP: | ||
| 709 | ath12k_dp_htt_mlo_peer_map_handler(ab, skb); | ||
| 710 | break; | ||
| 662 | default: | 711 | default: |
| 663 | ath12k_dbg(ab, ATH12K_DBG_DP_HTT, "dp_htt event %d not handled\n", | 712 | ath12k_dbg(ab, ATH12K_DBG_DP_HTT, "dp_htt event %d not handled\n", |
| 664 | type); | 713 | type); |
| @@ -930,6 +930,7 @@ enum htt_t2h_msg_type { · 929 unmodified lines | |||
| 930 | HTT_T2H_MSG_TYPE_EXT_STATS_CONF = 0x1c, | 930 | HTT_T2H_MSG_TYPE_EXT_STATS_CONF = 0x1c, |
| 931 | HTT_T2H_MSG_TYPE_BKPRESSURE_EVENT_IND = 0x24, | 931 | HTT_T2H_MSG_TYPE_BKPRESSURE_EVENT_IND = 0x24, |
| 932 | HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND = 0x28, | 932 | HTT_T2H_MSG_TYPE_MLO_TIMESTAMP_OFFSET_IND = 0x28, |
| 933 | HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP = 0x29, | ||
| 933 | HTT_T2H_MSG_TYPE_PEER_MAP3 = 0x2b, | 934 | HTT_T2H_MSG_TYPE_PEER_MAP3 = 0x2b, |
| 934 | HTT_T2H_MSG_TYPE_VDEV_TXRX_STATS_PERIODIC_IND = 0x2c, | 935 | HTT_T2H_MSG_TYPE_VDEV_TXRX_STATS_PERIODIC_IND = 0x2c, |
| 935 | }; | 936 | }; |
| @@ -974,11 +975,22 @@ struct htt_t2h_peer_unmap_event { · 38 unmodified lines | |||
| 974 | __le32 info1; | 975 | __le32 info1; |
| 975 | } __packed; | 976 | } __packed; |
| 976 | 977 | ||
| 978 | #define HTT_T2H_MLO_PEER_MAP_INFO0_MLO_PEER_ID GENMASK(23, 8) | ||
| 979 | #define HTT_T2H_MLO_PEER_MAP_INFO1_MAC_ADDR_H16 GENMASK(15, 0) | ||
| 980 | |||
| 981 | struct htt_t2h_mlo_peer_map_event { | ||
| 982 | __le32 info0; | ||
| 983 | __le32 mac_addr_l32; | ||
| 984 | __le32 info1; | ||
| 985 | __le32 reserved[5]; | ||
| 986 | } __packed; | ||
| 987 | |||
| 977 | struct htt_resp_msg { | 988 | struct htt_resp_msg { |
| 978 | union { | 989 | union { |
| 979 | struct htt_t2h_version_conf_msg version_msg; | 990 | struct htt_t2h_version_conf_msg version_msg; |
| 980 | struct htt_t2h_peer_map_event peer_map_ev; | 991 | struct htt_t2h_peer_map_event peer_map_ev; |
| 981 | struct htt_t2h_peer_unmap_event peer_unmap_ev; | 992 | struct htt_t2h_peer_unmap_event peer_unmap_ev; |
| 993 | struct htt_t2h_mlo_peer_map_event mlo_peer_map_ev; | ||
| 982 | }; | 994 | }; |
| 983 | } __packed; | 995 | } __packed; |
| 984 | 996 | ||
| @@ -475,7 +475,9 @@ int ath12k_dp_peer_create(struct ath12k_dp_hw *dp_hw, u8 *addr, · 474 unmodified lines | |||
| 475 | dp_peer->is_mlo = params->is_mlo; | 475 | dp_peer->is_mlo = params->is_mlo; |
| 476 | 476 | ||
| 477 | /* | 477 | /* |
| 478 | * For MLO client, the host assigns the ML peer ID, so set peer_id in dp_peer | 478 | * For MLO client, the ML peer ID, either known or PENDING, needs to be |
| 479 | * initialized here since the following logic depends on it. | ||
| 480 | * | ||
| 479 | * For non-MLO client, host gets link peer ID from firmware and will be | 481 | * For non-MLO client, host gets link peer ID from firmware and will be |
| 480 | * assigned at the time of link peer creation | 482 | * assigned at the time of link peer creation |
| 481 | */ | 483 | */ |
| @@ -491,13 +493,17 @@ int ath12k_dp_peer_create(struct ath12k_dp_hw *dp_hw, u8 *addr, · 9 unmodified lines | |||
| 491 | list_add(&dp_peer->list, &dp_hw->dp_peers_list); | 493 | list_add(&dp_peer->list, &dp_hw->dp_peers_list); |
| 492 | 494 | ||
| 493 | /* | 495 | /* |
| 494 | * For MLO client, the peer_id for ath12k_dp_peer is allocated by host | 496 | * For an MLO client whose ML peer ID is allocated by the host, the |
| 495 | * and that peer_id is known at this point, and hence this ath12k_dp_peer | 497 | * peer_id is known here and the dp_peer can be added to the RCU |
| 496 | * can be added to the RCU table using the peer_id. | 498 | * table using it. For an MLO client on chips where the firmware |
| 497 | * For non-MLO client, this addition to RCU table shall be done at the | 499 | * allocates the ID, peer_id is ATH12K_MLO_PEER_ID_PENDING and the |
| 498 | * time of assignment of ath12k_dp_link_peer to ath12k_dp_peer. | 500 | * RCU table publish is deferred to the |
| 501 | * HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP handler. For a non-MLO client | ||
| 502 | * the publish happens later, at the time of assignment of | ||
| 503 | * ath12k_dp_link_peer to ath12k_dp_peer. | ||
| 499 | */ | 504 | */ |
| 500 | if (dp_peer->is_mlo) | 505 | if (dp_peer->is_mlo && |
| 506 | dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | ||
| 501 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], dp_peer); | 507 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], dp_peer); |
| 502 | 508 | ||
| 503 | spin_unlock_bh(&dp_hw->peer_lock); | 509 | spin_unlock_bh(&dp_hw->peer_lock); |
| @@ -518,7 +524,8 @@ void ath12k_dp_peer_delete(struct ath12k_dp_hw *dp_hw, u8 *addr, · 14 unmodified lines | |||
| 518 | return; | 524 | return; |
| 519 | } | 525 | } |
| 520 | 526 | ||
| 521 | if (dp_peer->is_mlo) | 527 | if (dp_peer->is_mlo && |
| 528 | dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) | ||
| 522 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], NULL); | 529 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], NULL); |
| 523 | 530 | ||
| 524 | list_del(&dp_peer->list); | 531 | list_del(&dp_peer->list); |
| @@ -695,3 +702,55 @@ void ath12k_dp_link_peer_reset_rx_stats(struct ath12k_dp *dp, const u8 *addr) · 170 unmodified lines | |||
| 695 | if (rx_stats) | 702 | if (rx_stats) |
| 696 | memset(rx_stats, 0, sizeof(*rx_stats)); | 703 | memset(rx_stats, 0, sizeof(*rx_stats)); |
| 697 | } | 704 | } |
| 705 | |||
| 706 | int ath12k_dp_peer_fixup_peer_id(struct ath12k_base *ab, | ||
| 707 | const u8 *peer_addr, u16 peer_id) | ||
| 708 | { | ||
| 709 | struct ath12k_dp_link_peer *link_peer; | ||
| 710 | struct ath12k_dp_peer *dp_peer = NULL; | ||
| 711 | struct ath12k_hw_group *ag = ab->ag; | ||
| 712 | struct ath12k_dp_hw *dp_hw = NULL; | ||
| 713 | struct ath12k_hw *ah; | ||
| 714 | int i; | ||
| 715 | |||
| 716 | if (peer_id >= (ATH12K_PEER_ML_ID_VALID | ATH12K_MAX_MLO_PEERS)) | ||
| 717 | return -EINVAL; | ||
| 718 | |||
| 719 | for (i = 0; i < ag->num_hw; i++) { | ||
| 720 | ah = ag->ah[i]; | ||
| 721 | if (!ah) | ||
| 722 | continue; | ||
| 723 | |||
| 724 | spin_lock_bh(&ah->dp_hw.peer_lock); | ||
| 725 | dp_peer = ath12k_dp_peer_find_by_addr(&ah->dp_hw, | ||
| 726 | (u8 *)peer_addr); | ||
| 727 | if (dp_peer) { | ||
| 728 | dp_hw = &ah->dp_hw; | ||
| 729 | break; | ||
| 730 | } | ||
| 731 | spin_unlock_bh(&ah->dp_hw.peer_lock); | ||
| 732 | } | ||
| 733 | |||
| 734 | if (!dp_peer) | ||
| 735 | return -ENOENT; | ||
| 736 | |||
| 737 | /* dp_hw->peer_lock is held */ | ||
| 738 | |||
| 739 | dp_peer->peer_id = peer_id; | ||
| 740 | rcu_assign_pointer(dp_hw->dp_peers[peer_id], dp_peer); | ||
| 741 | |||
| 742 | for (i = 0; i < ATH12K_NUM_MAX_LINKS; i++) { | ||
| 743 | link_peer = rcu_dereference_protected(dp_peer->link_peers[i], | ||
| 744 | lockdep_is_held(&dp_hw->peer_lock)); | ||
| 745 | if (link_peer) | ||
| 746 | link_peer->ml_id = peer_id; | ||
| 747 | } | ||
| 748 | |||
| 749 | ath12k_sta_to_ahsta(dp_peer->sta)->ml_peer_id = peer_id; | ||
| 750 | |||
| 751 | spin_unlock_bh(&dp_hw->peer_lock); | ||
| 752 | |||
| 753 | complete(&ah->peer_ml_id_done); | ||
| 754 | |||
| 755 | return 0; | ||
| 756 | } | ||
| @@ -181,4 +181,6 @@ struct ath12k_dp_peer *ath12k_dp_peer_find_by_peerid(struct ath12k_pdev_dp *dp_p · 180 unmodified lines | |||
| 181 | struct ath12k_dp_link_peer * | 181 | struct ath12k_dp_link_peer * |
| 182 | ath12k_dp_link_peer_find_by_peerid(struct ath12k_pdev_dp *dp_pdev, u16 peer_id); | 182 | ath12k_dp_link_peer_find_by_peerid(struct ath12k_pdev_dp *dp_pdev, u16 peer_id); |
| 183 | void ath12k_dp_link_peer_free(struct ath12k_dp_link_peer *peer); | 183 | void ath12k_dp_link_peer_free(struct ath12k_dp_link_peer *peer); |
| 184 | int ath12k_dp_peer_fixup_peer_id(struct ath12k_base *ab, const u8 *peer_addr, | ||
| 185 | u16 peer_id); | ||
| 184 | #endif | 186 | #endif |
| @@ -236,6 +236,8 @@ struct ath12k_hw_params { · 235 unmodified lines | |||
| 236 | u32 max_client_dbs; | 236 | u32 max_client_dbs; |
| 237 | u32 max_client_dbs_sbs; | 237 | u32 max_client_dbs_sbs; |
| 238 | } client; | 238 | } client; |
| 239 | |||
| 240 | bool host_alloc_ml_id; | ||
| 239 | }; | 241 | }; |
| 240 | 242 | ||
| 241 | struct ath12k_hw_ops { | 243 | struct ath12k_hw_ops { |
| @@ -1282,13 +1282,17 @@ void ath12k_mac_dp_peer_cleanup(struct ath12k_hw *ah) · 1281 unmodified lines | |||
| 1282 | struct ath12k_dp_peer *dp_peer, *tmp; | 1282 | struct ath12k_dp_peer *dp_peer, *tmp; |
| 1283 | struct ath12k_dp_hw *dp_hw = &ah->dp_hw; | 1283 | struct ath12k_dp_hw *dp_hw = &ah->dp_hw; |
| 1284 | 1284 | ||
| 1285 | lockdep_assert_wiphy(ah->hw->wiphy); | ||
| 1286 | |||
| 1285 | INIT_LIST_HEAD(&peers); | 1287 | INIT_LIST_HEAD(&peers); |
| 1286 | 1288 | ||
| 1287 | spin_lock_bh(&dp_hw->peer_lock); | 1289 | spin_lock_bh(&dp_hw->peer_lock); |
| 1288 | list_for_each_entry_safe(dp_peer, tmp, &dp_hw->dp_peers_list, list) { | 1290 | list_for_each_entry_safe(dp_peer, tmp, &dp_hw->dp_peers_list, list) { |
| 1289 | if (dp_peer->is_mlo) { | 1291 | if (dp_peer->is_mlo) { |
| 1290 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], NULL); | 1292 | if (dp_peer->peer_id != ATH12K_MLO_PEER_ID_PENDING) |
| 1291 | clear_bit(dp_peer->peer_id, ah->free_ml_peer_id_map); | 1293 | rcu_assign_pointer(dp_hw->dp_peers[dp_peer->peer_id], |
| 1294 | NULL); | ||
| 1295 | ath12k_peer_ml_free(ah, ath12k_sta_to_ahsta(dp_peer->sta)); | ||
| 1292 | } | 1296 | } |
| 1293 | 1297 | ||
| 1294 | list_move(&dp_peer->list, &peers); | 1298 | list_move(&dp_peer->list, &peers); |
| @@ -3531,11 +3535,16 @@ static void ath12k_peer_assoc_h_mlo(struct ath12k_link_sta *arsta, · 2236 unmodified lines | |||
| 3531 | struct ath12k_sta *ahsta = arsta->ahsta; | 3535 | struct ath12k_sta *ahsta = arsta->ahsta; |
| 3532 | struct ath12k_link_sta *arsta_p; | 3536 | struct ath12k_link_sta *arsta_p; |
| 3533 | struct ath12k_link_vif *arvif; | 3537 | struct ath12k_link_vif *arvif; |
| 3538 | struct ath12k_hw *ah = arsta->arvif->ar->ah; | ||
| 3534 | unsigned long links; | 3539 | unsigned long links; |
| 3535 | u8 link_id; | 3540 | u8 link_id; |
| 3536 | int i; | 3541 | int i; |
| 3537 | 3542 | ||
| 3538 | if (!sta->mlo || ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) | 3543 | if (!sta->mlo) |
| 3544 | return; | ||
| 3545 | |||
| 3546 | if (ah->host_alloc_ml_id && | ||
| 3547 | ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) | ||
| 3539 | return; | 3548 | return; |
| 3540 | 3549 | ||
| 3541 | ml->enabled = true; | 3550 | ml->enabled = true; |
| @@ -3543,12 +3552,25 @@ static void ath12k_peer_assoc_h_mlo(struct ath12k_link_sta *arsta, · 1 unmodified line | |||
| 3543 | 3552 | ||
| 3544 | /* For now considering the primary umac based on assoc link */ | 3553 | /* For now considering the primary umac based on assoc link */ |
| 3545 | ml->primary_umac = arsta->is_assoc_link; | 3554 | ml->primary_umac = arsta->is_assoc_link; |
| 3546 | ml->peer_id_valid = true; | 3555 | /* |
| 3556 | * Only chips that allocate the MLD peer ID on the host send a valid | ||
| 3557 | * ml_peer_id in WMI_PEER_ASSOC_CMDID. For chips where the firmware | ||
| 3558 | * picks the ID, leave peer_id_valid false to avoid unexpected issues. | ||
| 3559 | */ | ||
| 3560 | ml->peer_id_valid = ah->host_alloc_ml_id; | ||
| 3547 | ml->logical_link_idx_valid = true; | 3561 | ml->logical_link_idx_valid = true; |
| 3548 | 3562 | ||
| 3549 | ether_addr_copy(ml->mld_addr, sta->addr); | 3563 | ether_addr_copy(ml->mld_addr, sta->addr); |
| 3550 | ml->logical_link_idx = arsta->link_idx; | 3564 | ml->logical_link_idx = arsta->link_idx; |
| 3551 | ml->ml_peer_id = ahsta->ml_peer_id; | 3565 | /* |
| 3566 | * WMI_MLO_PEER_ASSOC_PARAMS expects the raw ML peer ID without | ||
| 3567 | * the host-side ATH12K_PEER_ML_ID_VALID bookkeeping bit. For chips | ||
| 3568 | * where the firmware allocates the ID, the field is unused (the | ||
| 3569 | * firmware always allocates regardless of the value here); send 0 | ||
| 3570 | * to make that intent explicit. | ||
| 3571 | */ | ||
| 3572 | ml->ml_peer_id = ah->host_alloc_ml_id ? | ||
| 3573 | (ahsta->ml_peer_id & ~ATH12K_PEER_ML_ID_VALID) : 0; | ||
| 3552 | ml->ieee_link_id = arsta->link_id; | 3574 | ml->ieee_link_id = arsta->link_id; |
| 3553 | ml->num_partner_links = 0; | 3575 | ml->num_partner_links = 0; |
| 3554 | ml->eml_cap = sta->eml_cap; | 3576 | ml->eml_cap = sta->eml_cap; |
| @@ -3595,8 +3617,6 @@ static void ath12k_peer_assoc_prepare(struct ath12k *ar, · 40 unmodified lines | |||
| 3595 | 3617 | ||
| 3596 | memset(arg, 0, sizeof(*arg)); | 3618 | memset(arg, 0, sizeof(*arg)); |
| 3597 | 3619 | ||
| 3598 | reinit_completion(&ar->peer_assoc_done); | ||
| 3599 | |||
| 3600 | arg->peer_new_assoc = !reassoc; | 3620 | arg->peer_new_assoc = !reassoc; |
| 3601 | ath12k_peer_assoc_h_basic(ar, arvif, arsta, arg); | 3621 | ath12k_peer_assoc_h_basic(ar, arvif, arsta, arg); |
| 3602 | ath12k_peer_assoc_h_crypto(ar, arvif, arsta, arg); | 3622 | ath12k_peer_assoc_h_crypto(ar, arvif, arsta, arg); |
| @@ -3836,6 +3856,52 @@ static u32 ath12k_mac_ieee80211_sta_bw_to_wmi(struct ath12k *ar, · 233 unmodified lines | |||
| 3836 | return bw; | 3856 | return bw; |
| 3837 | } | 3857 | } |
| 3838 | 3858 | ||
| 3859 | static int ath12k_mac_peer_assoc(struct ath12k *ar, | ||
| 3860 | struct ath12k_wmi_peer_assoc_arg *peer_arg) | ||
| 3861 | { | ||
| 3862 | struct ath12k_hw *ah = ath12k_ar_to_ah(ar); | ||
| 3863 | int ret; | ||
| 3864 | |||
| 3865 | reinit_completion(&ar->peer_assoc_done); | ||
| 3866 | reinit_completion(&ah->peer_ml_id_done); | ||
| 3867 | |||
| 3868 | ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | ||
| 3869 | if (ret) { | ||
| 3870 | ath12k_warn(ar->ab, "failed to run peer assoc for %pM vdev %i: %d\n", | ||
| 3871 | peer_arg->peer_mac, peer_arg->vdev_id, ret); | ||
| 3872 | return ret; | ||
| 3873 | } | ||
| 3874 | |||
| 3875 | if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | ||
| 3876 | ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | ||
| 3877 | peer_arg->peer_mac, peer_arg->vdev_id); | ||
| 3878 | return -ETIMEDOUT; | ||
| 3879 | } | ||
| 3880 | |||
| 3881 | /* | ||
| 3882 | * For devices where the firmware allocates the MLD peer ID, the host | ||
| 3883 | * learns the real ID only from the MLO_RX_PEER_MAP HTT event, which is | ||
| 3884 | * handled in a softirq (BH workqueue) context that cannot take the | ||
| 3885 | * wiphy lock. Block here, while still holding the wiphy lock, until | ||
| 3886 | * that event has fixed up the ID. This serialises the fixup against | ||
| 3887 | * all other wiphy-locked ml_peer_id accesses. | ||
| 3888 | * | ||
| 3889 | * The firmware sends the event only once, in response to the assoc-link | ||
| 3890 | * peer assoc, so block only for that link. | ||
| 3891 | */ | ||
| 3892 | if (!ah->host_alloc_ml_id && | ||
| 3893 | peer_arg->is_assoc && | ||
| 3894 | peer_arg->ml.enabled && | ||
| 3895 | peer_arg->ml.assoc_link && | ||
| 3896 | !wait_for_completion_timeout(&ah->peer_ml_id_done, 1 * HZ)) { | ||
| 3897 | ath12k_warn(ar->ab, "failed to get MLO peer map event for %pM vdev %i\n", | ||
| 3898 | peer_arg->peer_mac, peer_arg->vdev_id); | ||
| 3899 | return -ETIMEDOUT; | ||
| 3900 | } | ||
| 3901 | |||
| 3902 | return 0; | ||
| 3903 | } | ||
| 3904 | |||
| 3839 | static void ath12k_bss_assoc(struct ath12k *ar, | 3905 | static void ath12k_bss_assoc(struct ath12k *ar, |
| 3840 | struct ath12k_link_vif *arvif, | 3906 | struct ath12k_link_vif *arvif, |
| 3841 | struct ieee80211_bss_conf *bss_conf) | 3907 | struct ieee80211_bss_conf *bss_conf) |
| @@ -3916,18 +3982,10 @@ static void ath12k_bss_assoc(struct ath12k *ar, · 74 unmodified lines | |||
| 3916 | } | 3982 | } |
| 3917 | 3983 | ||
| 3918 | peer_arg->is_assoc = true; | 3984 | peer_arg->is_assoc = true; |
| 3919 | ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | ||
| 3920 | if (ret) { | ||
| 3921 | ath12k_warn(ar->ab, "failed to run peer assoc for %pM vdev %i: %d\n", | ||
| 3922 | bss_conf->bssid, arvif->vdev_id, ret); | ||
| 3923 | return; | ||
| 3924 | } | ||
| 3925 | 3985 | ||
| 3926 | if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | 3986 | ret = ath12k_mac_peer_assoc(ar, peer_arg); |
| 3927 | ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | 3987 | if (ret) |
| 3928 | bss_conf->bssid, arvif->vdev_id); | ||
| 3929 | return; | 3988 | return; |
| 3930 | } | ||
| 3931 | 3989 | ||
| 3932 | ret = ath12k_setup_peer_smps(ar, arvif, bss_conf->bssid, | 3990 | ret = ath12k_setup_peer_smps(ar, arvif, bss_conf->bssid, |
| 3933 | &link_sta->ht_cap, &link_sta->he_6ghz_capa); | 3991 | &link_sta->ht_cap, &link_sta->he_6ghz_capa); |
| @@ -6481,18 +6539,10 @@ static int ath12k_mac_station_assoc(struct ath12k *ar, · 2547 unmodified lines | |||
| 6481 | } | 6539 | } |
| 6482 | 6540 | ||
| 6483 | peer_arg->is_assoc = true; | 6541 | peer_arg->is_assoc = true; |
| 6484 | ret = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | ||
| 6485 | if (ret) { | ||
| 6486 | ath12k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n", | ||
| 6487 | arsta->addr, arvif->vdev_id, ret); | ||
| 6488 | return ret; | ||
| 6489 | } | ||
| 6490 | 6542 | ||
| 6491 | if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) { | 6543 | ret = ath12k_mac_peer_assoc(ar, peer_arg); |
| 6492 | ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | 6544 | if (ret) |
| 6493 | arsta->addr, arvif->vdev_id); | 6545 | return ret; |
| 6494 | return -ETIMEDOUT; | ||
| 6495 | } | ||
| 6496 | 6546 | ||
| 6497 | num_vht_rates = ath12k_mac_bitrate_mask_num_vht_rates(ar, band, mask); | 6547 | num_vht_rates = ath12k_mac_bitrate_mask_num_vht_rates(ar, band, mask); |
| 6498 | num_he_rates = ath12k_mac_bitrate_mask_num_he_rates(ar, band, mask); | 6548 | num_he_rates = ath12k_mac_bitrate_mask_num_he_rates(ar, band, mask); |
| @@ -6841,14 +6891,8 @@ static void ath12k_sta_rc_update_wk(struct wiphy *wiphy, struct wiphy_work *wk) · 342 unmodified lines | |||
| 6841 | peer_arg, true); | 6891 | peer_arg, true); |
| 6842 | 6892 | ||
| 6843 | peer_arg->is_assoc = false; | 6893 | peer_arg->is_assoc = false; |
| 6844 | err = ath12k_wmi_send_peer_assoc_cmd(ar, peer_arg); | ||
| 6845 | if (err) | ||
| 6846 | ath12k_warn(ar->ab, "failed to run peer assoc for STA %pM vdev %i: %d\n", | ||
| 6847 | arsta->addr, arvif->vdev_id, err); | ||
| 6848 | 6894 | ||
| 6849 | if (!wait_for_completion_timeout(&ar->peer_assoc_done, 1 * HZ)) | 6895 | ath12k_mac_peer_assoc(ar, peer_arg); |
| 6850 | ath12k_warn(ar->ab, "failed to get peer assoc conf event for %pM vdev %i\n", | ||
| 6851 | arsta->addr, arvif->vdev_id); | ||
| 6852 | } | 6896 | } |
| 6853 | } | 6897 | } |
| 6854 | } | 6898 | } |
| @@ -7266,10 +7310,8 @@ static void ath12k_mac_ml_station_remove(struct ath12k_vif *ahvif, · 411 unmodified lines | |||
| 7266 | ath12k_mac_free_unassign_link_sta(ah, ahsta, link_id); | 7310 | ath12k_mac_free_unassign_link_sta(ah, ahsta, link_id); |
| 7267 | } | 7311 | } |
| 7268 | 7312 | ||
| 7269 | if (sta->mlo) { | 7313 | if (sta->mlo) |
| 7270 | clear_bit(ahsta->ml_peer_id, ah->free_ml_peer_id_map); | 7314 | ath12k_peer_ml_free(ah, ahsta); |
| 7271 | ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | ||
| 7272 | } | ||
| 7273 | } | 7315 | } |
| 7274 | 7316 | ||
| 7275 | static int ath12k_mac_handle_link_sta_state(struct ieee80211_hw *hw, | 7317 | static int ath12k_mac_handle_link_sta_state(struct ieee80211_hw *hw, |
| @@ -7733,15 +7775,23 @@ int ath12k_mac_op_sta_state(struct ieee80211_hw *hw, · 457 unmodified lines | |||
| 7733 | /* ML sta */ | 7775 | /* ML sta */ |
| 7734 | if (sta->mlo && !ahsta->links_map && | 7776 | if (sta->mlo && !ahsta->links_map && |
| 7735 | (hweight16(sta->valid_links) == 1)) { | 7777 | (hweight16(sta->valid_links) == 1)) { |
| 7736 | ahsta->ml_peer_id = ath12k_peer_ml_alloc(ah); | 7778 | if (ah->host_alloc_ml_id) { |
| 7737 | if (ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) { | 7779 | ahsta->ml_peer_id = ath12k_peer_ml_alloc(ah); |
| 7738 | ath12k_hw_warn(ah, "unable to allocate ML peer id for sta %pM", | 7780 | if (ahsta->ml_peer_id == ATH12K_MLO_PEER_ID_INVALID) { |
| 7739 | sta->addr); | 7781 | ath12k_hw_warn(ah, "unable to allocate ML peer id for sta %pM", |
| 7740 | goto exit; | 7782 | sta->addr); |
| 7783 | goto exit; | ||
| 7784 | } | ||
| 7785 | } else { | ||
| 7786 | /* | ||
| 7787 | * firmware allocates the ML peer ID and notifies | ||
| 7788 | * the host via HTT_T2H_MSG_TYPE_MLO_RX_PEER_MAP | ||
| 7789 | */ | ||
| 7790 | ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_PENDING; | ||
| 7741 | } | 7791 | } |
| 7742 | 7792 | ||
| 7743 | dp_params.is_mlo = true; | 7793 | dp_params.is_mlo = true; |
| 7744 | dp_params.peer_id = ahsta->ml_peer_id | ATH12K_PEER_ML_ID_VALID; | 7794 | dp_params.peer_id = ahsta->ml_peer_id; |
| 7745 | } | 7795 | } |
| 7746 | 7796 | ||
| 7747 | dp_params.sta = sta; | 7797 | dp_params.sta = sta; |
| @@ -7878,10 +7928,8 @@ int ath12k_mac_op_sta_state(struct ieee80211_hw *hw, · 130 unmodified lines | |||
| 7878 | peer_delete: | 7928 | peer_delete: |
| 7879 | ath12k_dp_peer_delete(&ah->dp_hw, sta->addr, sta); | 7929 | ath12k_dp_peer_delete(&ah->dp_hw, sta->addr, sta); |
| 7880 | ml_peer_id_clear: | 7930 | ml_peer_id_clear: |
| 7881 | if (sta->mlo) { | 7931 | if (sta->mlo) |
| 7882 | clear_bit(ahsta->ml_peer_id, ah->free_ml_peer_id_map); | 7932 | ath12k_peer_ml_free(ah, ahsta); |
| 7883 | ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | ||
| 7884 | } | ||
| 7885 | exit: | 7933 | exit: |
| 7886 | /* update the state if everything went well */ | 7934 | /* update the state if everything went well */ |
| 7887 | if (!ret) | 7935 | if (!ret) |
| @@ -15310,6 +15358,7 @@ static struct ath12k_hw *ath12k_mac_hw_allocate(struct ath12k_hw_group *ag, · 7422 unmodified lines | |||
| 15310 | ah->num_radio = num_pdev_map; | 15358 | ah->num_radio = num_pdev_map; |
| 15311 | 15359 | ||
| 15312 | mutex_init(&ah->hw_mutex); | 15360 | mutex_init(&ah->hw_mutex); |
| 15361 | init_completion(&ah->peer_ml_id_done); | ||
| 15313 | 15362 | ||
| 15314 | spin_lock_init(&ah->dp_hw.peer_lock); | 15363 | spin_lock_init(&ah->dp_hw.peer_lock); |
| 15315 | INIT_LIST_HEAD(&ah->dp_hw.dp_peers_list); | 15364 | INIT_LIST_HEAD(&ah->dp_hw.dp_peers_list); |
| @@ -15384,8 +15433,9 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 68 unmodified lines | |||
| 15384 | int mac_id, device_id, total_radio, num_hw; | 15433 | int mac_id, device_id, total_radio, num_hw; |
| 15385 | struct ath12k_base *ab; | 15434 | struct ath12k_base *ab; |
| 15386 | struct ath12k_hw *ah; | 15435 | struct ath12k_hw *ah; |
| 15387 | int ret, i, j; | 15436 | bool conf = false; |
| 15388 | u8 radio_per_hw; | 15437 | u8 radio_per_hw; |
| 15438 | int ret, i, j; | ||
| 15389 | 15439 | ||
| 15390 | total_radio = 0; | 15440 | total_radio = 0; |
| 15391 | for (i = 0; i < ag->num_devices; i++) { | 15441 | for (i = 0; i < ag->num_devices; i++) { |
| @@ -15425,6 +15475,20 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 33 unmodified lines | |||
| 15425 | } | 15475 | } |
| 15426 | 15476 | ||
| 15427 | ab = ag->ab[device_id]; | 15477 | ab = ag->ab[device_id]; |
| 15478 | |||
| 15479 | /* | ||
| 15480 | * the assumption is all devices within an ah | ||
| 15481 | * share the same host_alloc_ml_id configuration | ||
| 15482 | */ | ||
| 15483 | if (j == 0) { | ||
| 15484 | conf = ab->hw_params->host_alloc_ml_id; | ||
| 15485 | } else if (conf != ab->hw_params->host_alloc_ml_id) { | ||
| 15486 | ath12k_warn(ab, "inconsistent ML ID config within ah, device 0 uses %s allocated ID, while device %u doesn't\n", | ||
| 15487 | conf ? "host" : "firmware", device_id); | ||
| 15488 | ret = -EINVAL; | ||
| 15489 | goto err; | ||
| 15490 | } | ||
| 15491 | |||
| 15428 | pdev_map[j].ab = ab; | 15492 | pdev_map[j].ab = ab; |
| 15429 | pdev_map[j].pdev_idx = mac_id; | 15493 | pdev_map[j].pdev_idx = mac_id; |
| 15430 | mac_id++; | 15494 | mac_id++; |
| @@ -15449,6 +15513,7 @@ int ath12k_mac_allocate(struct ath12k_hw_group *ag) · 18 unmodified lines | |||
| 15449 | } | 15513 | } |
| 15450 | 15514 | ||
| 15451 | ah->dev = ab->dev; | 15515 | ah->dev = ab->dev; |
| 15516 | ah->host_alloc_ml_id = conf; | ||
| 15452 | 15517 | ||
| 15453 | ag->ah[i] = ah; | 15518 | ag->ah[i] = ah; |
| 15454 | ag->num_hw++; | 15519 | ag->num_hw++; |
| @@ -230,7 +230,16 @@ int ath12k_peer_create(struct ath12k *ar, struct ath12k_link_vif *arvif, · 229 unmodified lines | |||
| 230 | /* Fill ML info into created peer */ | 230 | /* Fill ML info into created peer */ |
| 231 | if (sta->mlo) { | 231 | if (sta->mlo) { |
| 232 | ml_peer_id = ahsta->ml_peer_id; | 232 | ml_peer_id = ahsta->ml_peer_id; |
| 233 | peer->ml_id = ml_peer_id | ATH12K_PEER_ML_ID_VALID; | 233 | /* |
| 234 | * For chips where firmware allocates the ML peer ID, | ||
| 235 | * ml_peer_id is ATH12K_MLO_PEER_ID_PENDING here. The | ||
| 236 | * MLO_RX_PEER_MAP HTT event handler fixes up | ||
| 237 | * peer->ml_id once the ID is known. | ||
| 238 | */ | ||
| 239 | if (ml_peer_id == ATH12K_MLO_PEER_ID_PENDING) | ||
| 240 | peer->ml_id = ATH12K_MLO_PEER_ID_INVALID; | ||
| 241 | else | ||
| 242 | peer->ml_id = ml_peer_id; | ||
| 234 | ether_addr_copy(peer->ml_addr, sta->addr); | 243 | ether_addr_copy(peer->ml_addr, sta->addr); |
| 235 | 244 | ||
| 236 | /* the assoc link is considered primary for now */ | 245 | /* the assoc link is considered primary for now */ |
| @@ -276,9 +285,25 @@ u16 ath12k_peer_ml_alloc(struct ath12k_hw *ah) · 39 unmodified lines | |||
| 276 | } | 285 | } |
| 277 | 286 | ||
| 278 | if (ml_peer_id == ATH12K_MAX_MLO_PEERS) | 287 | if (ml_peer_id == ATH12K_MAX_MLO_PEERS) |
| 279 | ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | 288 | return ATH12K_MLO_PEER_ID_INVALID; |
| 280 | 289 | ||
| 281 | return ml_peer_id; | 290 | return ml_peer_id | ATH12K_PEER_ML_ID_VALID; |
| 291 | } | ||
| 292 | |||
| 293 | void ath12k_peer_ml_free(struct ath12k_hw *ah, struct ath12k_sta *ahsta) | ||
| 294 | { | ||
| 295 | lockdep_assert_wiphy(ah->hw->wiphy); | ||
| 296 | |||
| 297 | /* | ||
| 298 | * Only devices that allocate the ID on the host own a slot in | ||
| 299 | * free_ml_peer_id_map. | ||
| 300 | */ | ||
| 301 | if (ah->host_alloc_ml_id && | ||
| 302 | (ahsta->ml_peer_id < | ||
| 303 | (ATH12K_MAX_MLO_PEERS | ATH12K_PEER_ML_ID_VALID))) | ||
| 304 | clear_bit(ahsta->ml_peer_id & ~ATH12K_PEER_ML_ID_VALID, | ||
| 305 | ah->free_ml_peer_id_map); | ||
| 306 | ahsta->ml_peer_id = ATH12K_MLO_PEER_ID_INVALID; | ||
| 282 | } | 307 | } |
| 283 | 308 | ||
| 284 | int ath12k_peer_mlo_link_peers_delete(struct ath12k_vif *ahvif, struct ath12k_sta *ahsta) | 309 | int ath12k_peer_mlo_link_peers_delete(struct ath12k_vif *ahvif, struct ath12k_sta *ahsta) |
| @@ -26,4 +26,5 @@ int ath12k_link_sta_rhash_add(struct ath12k_base *ab, struct ath12k_link_sta *ar · 25 unmodified lines | |||
| 26 | struct ath12k_link_sta *ath12k_link_sta_find_by_addr(struct ath12k_base *ab, | 26 | struct ath12k_link_sta *ath12k_link_sta_find_by_addr(struct ath12k_base *ab, |
| 27 | const u8 *addr); | 27 | const u8 *addr); |
| 28 | u16 ath12k_peer_ml_alloc(struct ath12k_hw *ah); | 28 | u16 ath12k_peer_ml_alloc(struct ath12k_hw *ah); |
| 29 | void ath12k_peer_ml_free(struct ath12k_hw *ah, struct ath12k_sta *ahsta); | ||
| 29 | #endif /* _PEER_H_ */ | 30 | #endif /* _PEER_H_ */ |
| @@ -442,6 +442,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 441 unmodified lines | |||
| 442 | .max_client_dbs = 128, | 442 | .max_client_dbs = 128, |
| 443 | .max_client_dbs_sbs = 128, | 443 | .max_client_dbs_sbs = 128, |
| 444 | }, | 444 | }, |
| 445 | |||
| 446 | .host_alloc_ml_id = true, | ||
| 445 | }, | 447 | }, |
| 446 | { | 448 | { |
| 447 | .name = "wcn7850 hw2.0", | 449 | .name = "wcn7850 hw2.0", |
| @@ -533,6 +535,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 85 unmodified lines | |||
| 533 | .max_client_dbs = 128, | 535 | .max_client_dbs = 128, |
| 534 | .max_client_dbs_sbs = 128, | 536 | .max_client_dbs_sbs = 128, |
| 535 | }, | 537 | }, |
| 538 | |||
| 539 | .host_alloc_ml_id = false, | ||
| 536 | }, | 540 | }, |
| 537 | { | 541 | { |
| 538 | .name = "qcn9274 hw2.0", | 542 | .name = "qcn9274 hw2.0", |
| @@ -620,6 +624,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 81 unmodified lines | |||
| 620 | .max_client_dbs = 128, | 624 | .max_client_dbs = 128, |
| 621 | .max_client_dbs_sbs = 128, | 625 | .max_client_dbs_sbs = 128, |
| 622 | }, | 626 | }, |
| 627 | |||
| 628 | .host_alloc_ml_id = true, | ||
| 623 | }, | 629 | }, |
| 624 | { | 630 | { |
| 625 | .name = "ipq5332 hw1.0", | 631 | .name = "ipq5332 hw1.0", |
| @@ -700,6 +706,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 74 unmodified lines | |||
| 700 | .max_client_dbs = 128, | 706 | .max_client_dbs = 128, |
| 701 | .max_client_dbs_sbs = 128, | 707 | .max_client_dbs_sbs = 128, |
| 702 | }, | 708 | }, |
| 709 | |||
| 710 | .host_alloc_ml_id = true, | ||
| 703 | }, | 711 | }, |
| 704 | { | 712 | { |
| 705 | .name = "qcc2072 hw1.0", | 713 | .name = "qcc2072 hw1.0", |
| @@ -792,6 +800,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 86 unmodified lines | |||
| 792 | .max_client_dbs = 128, | 800 | .max_client_dbs = 128, |
| 793 | .max_client_dbs_sbs = 128, | 801 | .max_client_dbs_sbs = 128, |
| 794 | }, | 802 | }, |
| 803 | |||
| 804 | .host_alloc_ml_id = false, | ||
| 795 | }, | 805 | }, |
| 796 | { | 806 | { |
| 797 | .name = "ipq5424 hw1.0", | 807 | .name = "ipq5424 hw1.0", |
| @@ -876,6 +886,8 @@ static const struct ath12k_hw_params ath12k_wifi7_hw_params[] = { · 78 unmodified lines | |||
| 876 | .max_client_dbs = 128, | 886 | .max_client_dbs = 128, |
| 877 | .max_client_dbs_sbs = 128, | 887 | .max_client_dbs_sbs = 128, |
| 878 | }, | 888 | }, |
| 889 | |||
| 890 | .host_alloc_ml_id = true, | ||
| 879 | }, | 891 | }, |
| 880 | }; | 892 | }; |
| 881 | 893 | ||
| @@ -2179,8 +2179,8 @@ il_remove_station(struct il_priv *il, const u8 sta_id, const u8 * addr) · 2178 unmodified lines | |||
| 2179 | il->stations[sta_id].used &= ~IL_STA_DRIVER_ACTIVE; | 2179 | il->stations[sta_id].used &= ~IL_STA_DRIVER_ACTIVE; |
| 2180 | 2180 | ||
| 2181 | il->num_stations--; | 2181 | il->num_stations--; |
| 2182 | 2182 | if (WARN_ON(il->num_stations < 0)) | |
| 2183 | BUG_ON(il->num_stations < 0); | 2183 | il->num_stations = 0; |
| 2184 | 2184 | ||
| 2185 | spin_unlock_irqrestore(&il->sta_lock, flags); | 2185 | spin_unlock_irqrestore(&il->sta_lock, flags); |
| 2186 | 2186 | ||
| @@ -2328,7 +2328,8 @@ il_dealloc_bcast_stations(struct il_priv *il) · 141 unmodified lines | |||
| 2328 | 2328 | ||
| 2329 | il->stations[i].used &= ~IL_STA_UCODE_ACTIVE; | 2329 | il->stations[i].used &= ~IL_STA_UCODE_ACTIVE; |
| 2330 | il->num_stations--; | 2330 | il->num_stations--; |
| 2331 | BUG_ON(il->num_stations < 0); | 2331 | if (WARN_ON(il->num_stations < 0)) |
| 2332 | il->num_stations = 0; | ||
| 2332 | kfree(il->stations[i].lq); | 2333 | kfree(il->stations[i].lq); |
| 2333 | il->stations[i].lq = NULL; | 2334 | il->stations[i].lq = NULL; |
| 2334 | } | 2335 | } |
| @@ -396,7 +396,7 @@ il_dbgfs_stations_read(struct file *file, char __user *user_buf, size_t count, · 395 unmodified lines | |||
| 396 | int i, j, pos = 0; | 396 | int i, j, pos = 0; |
| 397 | ssize_t ret; | 397 | ssize_t ret; |
| 398 | /* Add 30 for initial string */ | 398 | /* Add 30 for initial string */ |
| 399 | const size_t bufsz = 30 + sizeof(char) * 500 * (il->num_stations); | 399 | const size_t bufsz = 30 + sizeof(char) * 500 * max_sta; |
| 400 | 400 | ||
| 401 | buf = kmalloc(bufsz, GFP_KERNEL); | 401 | buf = kmalloc(bufsz, GFP_KERNEL); |
| 402 | if (!buf) | 402 | if (!buf) |
| @@ -44,7 +44,7 @@ static int mwifiex_11n_dispatch_amsdu_pkt(struct mwifiex_private *priv, · 43 unmodified lines | |||
| 44 | ntohs(rx_hdr->eth803_hdr.h_proto) == ETH_P_TDLS) { | 44 | ntohs(rx_hdr->eth803_hdr.h_proto) == ETH_P_TDLS) { |
| 45 | mwifiex_process_tdls_action_frame(priv, | 45 | mwifiex_process_tdls_action_frame(priv, |
| 46 | (u8 *)rx_hdr, | 46 | (u8 *)rx_hdr, |
| 47 | skb->len); | 47 | rx_skb->len); |
| 48 | } | 48 | } |
| 49 | 49 | ||
| 50 | if (priv->bss_role == MWIFIEX_BSS_ROLE_UAP) | 50 | if (priv->bss_role == MWIFIEX_BSS_ROLE_UAP) |
| @@ -769,6 +769,7 @@ static void netc_timer_init(struct netc_timer *priv) · 768 unmodified lines | |||
| 769 | TMR_CTRL_TE | TMR_CTRL_FS; | 769 | TMR_CTRL_TE | TMR_CTRL_FS; |
| 770 | netc_timer_wr(priv, NETC_TMR_CTRL, tmr_ctrl); | 770 | netc_timer_wr(priv, NETC_TMR_CTRL, tmr_ctrl); |
| 771 | netc_timer_wr(priv, NETC_TMR_PRSC, priv->oclk_prsc); | 771 | netc_timer_wr(priv, NETC_TMR_PRSC, priv->oclk_prsc); |
| 772 | netc_timer_wr(priv, NETC_TMR_TEMASK, 0); | ||
| 772 | 773 | ||
| 773 | /* Disable FIPER by default */ | 774 | /* Disable FIPER by default */ |
| 774 | fiper_ctrl = netc_timer_rd(priv, NETC_TMR_FIPER_CTRL); | 775 | fiper_ctrl = netc_timer_rd(priv, NETC_TMR_FIPER_CTRL); |
| @@ -901,6 +902,11 @@ static irqreturn_t netc_timer_isr(int irq, void *data) · 126 unmodified lines | |||
| 901 | /* Clear interrupts status */ | 902 | /* Clear interrupts status */ |
| 902 | netc_timer_wr(priv, NETC_TMR_TEVENT, tmr_event); | 903 | netc_timer_wr(priv, NETC_TMR_TEVENT, tmr_event); |
| 903 | 904 | ||
| 905 | if (!tmr_event) { | ||
| 906 | spin_unlock(&priv->lock); | ||
| 907 | return IRQ_NONE; | ||
| 908 | } | ||
| 909 | |||
| 904 | if (tmr_event & TMR_TEVENT_ALMEN(0)) | 910 | if (tmr_event & TMR_TEVENT_ALMEN(0)) |
| 905 | netc_timer_alarm_write(priv, NETC_TMR_DEFAULT_ALARM, 0); | 911 | netc_timer_alarm_write(priv, NETC_TMR_DEFAULT_ALARM, 0); |
| 906 | 912 | ||
| @@ -936,7 +942,8 @@ static int netc_timer_init_msix_irq(struct netc_timer *priv) · 29 unmodified lines | |||
| 936 | } | 942 | } |
| 937 | 943 | ||
| 938 | priv->irq = pci_irq_vector(pdev, 0); | 944 | priv->irq = pci_irq_vector(pdev, 0); |
| 939 | err = request_irq(priv->irq, netc_timer_isr, 0, priv->irq_name, priv); | 945 | err = request_irq(priv->irq, netc_timer_isr, IRQF_NO_AUTOEN, |
| 946 | priv->irq_name, priv); | ||
| 940 | if (err) { | 947 | if (err) { |
| 941 | dev_err(&pdev->dev, "request_irq() failed\n"); | 948 | dev_err(&pdev->dev, "request_irq() failed\n"); |
| 942 | pci_free_irq_vectors(pdev); | 949 | pci_free_irq_vectors(pdev); |
| @@ -951,7 +958,6 @@ static void netc_timer_free_msix_irq(struct netc_timer *priv) · 8 unmodified lines | |||
| 951 | { | 958 | { |
| 952 | struct pci_dev *pdev = priv->pdev; | 959 | struct pci_dev *pdev = priv->pdev; |
| 953 | 960 | ||
| 954 | disable_irq(priv->irq); | ||
| 955 | free_irq(priv->irq, priv); | 961 | free_irq(priv->irq, priv); |
| 956 | pci_free_irq_vectors(pdev); | 962 | pci_free_irq_vectors(pdev); |
| 957 | } | 963 | } |
| @@ -1005,6 +1011,8 @@ static int netc_timer_probe(struct pci_dev *pdev, · 47 unmodified lines | |||
| 1005 | goto free_msix_irq; | 1011 | goto free_msix_irq; |
| 1006 | } | 1012 | } |
| 1007 | 1013 | ||
| 1014 | enable_irq(priv->irq); | ||
| 1015 | |||
| 1008 | return 0; | 1016 | return 0; |
| 1009 | 1017 | ||
| 1010 | free_msix_irq: | 1018 | free_msix_irq: |
| @@ -1019,9 +1027,10 @@ static void netc_timer_remove(struct pci_dev *pdev) · 8 unmodified lines | |||
| 1019 | { | 1027 | { |
| 1020 | struct netc_timer *priv = pci_get_drvdata(pdev); | 1028 | struct netc_timer *priv = pci_get_drvdata(pdev); |
| 1021 | 1029 | ||
| 1030 | disable_irq(priv->irq); | ||
| 1031 | ptp_clock_unregister(priv->clock); | ||
| 1022 | netc_timer_wr(priv, NETC_TMR_TEMASK, 0); | 1032 | netc_timer_wr(priv, NETC_TMR_TEMASK, 0); |
| 1023 | netc_timer_wr(priv, NETC_TMR_CTRL, 0); | 1033 | netc_timer_wr(priv, NETC_TMR_CTRL, 0); |
| 1024 | ptp_clock_unregister(priv->clock); | ||
| 1025 | netc_timer_free_msix_irq(priv); | 1034 | netc_timer_free_msix_irq(priv); |
| 1026 | netc_timer_pci_remove(pdev); | 1035 | netc_timer_pci_remove(pdev); |
| 1027 | } | 1036 | } |
| @@ -6525,6 +6525,9 @@ int qeth_siocdevprivate(struct net_device *dev, struct ifreq *rq, void __user *d · 6524 unmodified lines | |||
| 6525 | struct qeth_card *card = dev->ml_priv; | 6525 | struct qeth_card *card = dev->ml_priv; |
| 6526 | int rc = 0; | 6526 | int rc = 0; |
| 6527 | 6527 | ||
| 6528 | if (!capable(CAP_NET_ADMIN)) | ||
| 6529 | return -EPERM; | ||
| 6530 | |||
| 6528 | switch (cmd) { | 6531 | switch (cmd) { |
| 6529 | case SIOC_QETH_ADP_SET_SNMP_CONTROL: | 6532 | case SIOC_QETH_ADP_SET_SNMP_CONTROL: |
| 6530 | rc = qeth_snmp_command(card, data); | 6533 | rc = qeth_snmp_command(card, data); |
| @@ -176,21 +176,13 @@ static void *vhost_net_buf_consume(struct vhost_net_buf *rxq) · 175 unmodified lines | |||
| 176 | return ret; | 176 | return ret; |
| 177 | } | 177 | } |
| 178 | 178 | ||
| 179 | static int vhost_net_buf_produce(struct sock *sk, | 179 | static int vhost_net_buf_produce(struct vhost_net_virtqueue *nvq) |
| 180 | struct vhost_net_virtqueue *nvq) | ||
| 181 | { | 180 | { |
| 182 | struct file *file = sk->sk_socket->file; | ||
| 183 | struct vhost_net_buf *rxq = &nvq->rxq; | 181 | struct vhost_net_buf *rxq = &nvq->rxq; |
| 184 | 182 | ||
| 185 | rxq->head = 0; | 183 | rxq->head = 0; |
| 186 | spin_lock(&nvq->rx_ring->consumer_lock); | 184 | rxq->tail = ptr_ring_consume_batched(nvq->rx_ring, rxq->queue, |
| 187 | rxq->tail = __ptr_ring_consume_batched(nvq->rx_ring, rxq->queue, | 185 | VHOST_NET_BATCH); |
| 188 | VHOST_NET_BATCH); | ||
| 189 | |||
| 190 | if (rxq->tail) | ||
| 191 | tun_wake_queue(file, rxq->tail); | ||
| 192 | |||
| 193 | spin_unlock(&nvq->rx_ring->consumer_lock); | ||
| 194 | return rxq->tail; | 186 | return rxq->tail; |
| 195 | } | 187 | } |
| 196 | 188 | ||
| @@ -217,15 +209,14 @@ static int vhost_net_buf_peek_len(void *ptr) · 20 unmodified lines | |||
| 217 | return __skb_array_len_with_tag(ptr); | 209 | return __skb_array_len_with_tag(ptr); |
| 218 | } | 210 | } |
| 219 | 211 | ||
| 220 | static int vhost_net_buf_peek(struct sock *sk, | 212 | static int vhost_net_buf_peek(struct vhost_net_virtqueue *nvq) |
| 221 | struct vhost_net_virtqueue *nvq) | ||
| 222 | { | 213 | { |
| 223 | struct vhost_net_buf *rxq = &nvq->rxq; | 214 | struct vhost_net_buf *rxq = &nvq->rxq; |
| 224 | 215 | ||
| 225 | if (!vhost_net_buf_is_empty(rxq)) | 216 | if (!vhost_net_buf_is_empty(rxq)) |
| 226 | goto out; | 217 | goto out; |
| 227 | 218 | ||
| 228 | if (!vhost_net_buf_produce(sk, nvq)) | 219 | if (!vhost_net_buf_produce(nvq)) |
| 229 | return 0; | 220 | return 0; |
| 230 | 221 | ||
| 231 | out: | 222 | out: |
| @@ -1013,7 +1004,7 @@ static int peek_head_len(struct vhost_net_virtqueue *rvq, struct sock *sk) · 781 unmodified lines | |||
| 1013 | unsigned long flags; | 1004 | unsigned long flags; |
| 1014 | 1005 | ||
| 1015 | if (rvq->rx_ring) | 1006 | if (rvq->rx_ring) |
| 1016 | return vhost_net_buf_peek(sk, rvq); | 1007 | return vhost_net_buf_peek(rvq); |
| 1017 | 1008 | ||
| 1018 | spin_lock_irqsave(&sk->sk_receive_queue.lock, flags); | 1009 | spin_lock_irqsave(&sk->sk_receive_queue.lock, flags); |
| 1019 | head = skb_peek(&sk->sk_receive_queue); | 1010 | head = skb_peek(&sk->sk_receive_queue); |
| @@ -562,6 +562,7 @@ struct ethtool_fec_hist { · 561 unmodified lines | |||
| 562 | u64 per_lane[ETHTOOL_MAX_LANES]; | 562 | u64 per_lane[ETHTOOL_MAX_LANES]; |
| 563 | } values[ETHTOOL_FEC_HIST_MAX]; | 563 | } values[ETHTOOL_FEC_HIST_MAX]; |
| 564 | const struct ethtool_fec_hist_range *ranges; | 564 | const struct ethtool_fec_hist_range *ranges; |
| 565 | struct ethtool_fec_hist_range ranges_buf[ETHTOOL_FEC_HIST_MAX]; | ||
| 565 | }; | 566 | }; |
| 566 | /** | 567 | /** |
| 567 | * struct ethtool_fec_stats - statistics for IEEE 802.3 FEC | 568 | * struct ethtool_fec_stats - statistics for IEEE 802.3 FEC |
| @@ -22,7 +22,6 @@ struct tun_msg_ctl { · 21 unmodified lines | |||
| 22 | #if defined(CONFIG_TUN) || defined(CONFIG_TUN_MODULE) | 22 | #if defined(CONFIG_TUN) || defined(CONFIG_TUN_MODULE) |
| 23 | struct socket *tun_get_socket(struct file *); | 23 | struct socket *tun_get_socket(struct file *); |
| 24 | struct ptr_ring *tun_get_tx_ring(struct file *file); | 24 | struct ptr_ring *tun_get_tx_ring(struct file *file); |
| 25 | void tun_wake_queue(struct file *file, int consumed); | ||
| 26 | 25 | ||
| 27 | static inline bool tun_is_xdp_frame(void *ptr) | 26 | static inline bool tun_is_xdp_frame(void *ptr) |
| 28 | { | 27 | { |
| @@ -56,8 +55,6 @@ static inline struct ptr_ring *tun_get_tx_ring(struct file *f) · 27 unmodified lines | |||
| 56 | return ERR_PTR(-EINVAL); | 55 | return ERR_PTR(-EINVAL); |
| 57 | } | 56 | } |
| 58 | 57 | ||
| 59 | static inline void tun_wake_queue(struct file *f, int consumed) {} | ||
| 60 | |||
| 61 | static inline bool tun_is_xdp_frame(void *ptr) | 58 | static inline bool tun_is_xdp_frame(void *ptr) |
| 62 | { | 59 | { |
| 63 | return false; | 60 | return false; |
| @@ -115,7 +115,7 @@ struct nf_nat_sip_hooks { · 114 unmodified lines | |||
| 115 | unsigned int *datalen); | 115 | unsigned int *datalen); |
| 116 | 116 | ||
| 117 | void (*seq_adjust)(struct sk_buff *skb, | 117 | void (*seq_adjust)(struct sk_buff *skb, |
| 118 | unsigned int protoff, s16 off); | 118 | unsigned int protoff, s32 off); |
| 119 | 119 | ||
| 120 | unsigned int (*expect)(struct sk_buff *skb, | 120 | unsigned int (*expect)(struct sk_buff *skb, |
| 121 | unsigned int protoff, | 121 | unsigned int protoff, |
| @@ -96,20 +96,6 @@ static inline bool ptr_ring_full_bh(struct ptr_ring *r) · 95 unmodified lines | |||
| 96 | return ret; | 96 | return ret; |
| 97 | } | 97 | } |
| 98 | 98 | ||
| 99 | /* Note: callers invoking this in a loop must use a compiler barrier, | ||
| 100 | * for example cpu_relax(). Callers must hold producer_lock. | ||
| 101 | */ | ||
| 102 | static inline int __ptr_ring_check_produce(struct ptr_ring *r) | ||
| 103 | { | ||
| 104 | if (unlikely(!r->size)) | ||
| 105 | return -EINVAL; | ||
| 106 | |||
| 107 | if (data_race(r->queue[r->producer])) | ||
| 108 | return -ENOSPC; | ||
| 109 | |||
| 110 | return 0; | ||
| 111 | } | ||
| 112 | |||
| 113 | /* Note: callers invoking this in a loop must use a compiler barrier, | 99 | /* Note: callers invoking this in a loop must use a compiler barrier, |
| 114 | * for example cpu_relax(). Callers must hold producer_lock. | 100 | * for example cpu_relax(). Callers must hold producer_lock. |
| 115 | * Callers are responsible for making sure pointer that is being queued | 101 | * Callers are responsible for making sure pointer that is being queued |
| @@ -117,10 +103,8 @@ static inline int __ptr_ring_check_produce(struct ptr_ring *r) · 1 unmodified line | |||
| 117 | */ | 103 | */ |
| 118 | static inline int __ptr_ring_produce(struct ptr_ring *r, void *ptr) | 104 | static inline int __ptr_ring_produce(struct ptr_ring *r, void *ptr) |
| 119 | { | 105 | { |
| 120 | int p = __ptr_ring_check_produce(r); | 106 | if (unlikely(!r->size) || data_race(r->queue[r->producer])) |
| 121 | 107 | return -ENOSPC; | |
| 122 | if (p) | ||
| 123 | return p; | ||
| 124 | 108 | ||
| 125 | /* Make sure the pointer we are storing points to a valid data. */ | 109 | /* Make sure the pointer we are storing points to a valid data. */ |
| 126 | /* Pairs with the dependency ordering in __ptr_ring_consume. */ | 110 | /* Pairs with the dependency ordering in __ptr_ring_consume. */ |
| @@ -767,9 +767,11 @@ struct hci_conn { · 766 unmodified lines | |||
| 767 | struct dentry *debugfs; | 767 | struct dentry *debugfs; |
| 768 | 768 | ||
| 769 | struct hci_dev *hdev; | 769 | struct hci_dev *hdev; |
| 770 | |||
| 771 | spinlock_t proto_lock; /* lock guarding protocol data */ | ||
| 770 | void *l2cap_data; | 772 | void *l2cap_data; |
| 771 | void *sco_data; | 773 | void *sco_data; |
| 772 | void *iso_data; | 774 | void *iso_data __guarded_by(&proto_lock); |
| 773 | 775 | ||
| 774 | struct list_head link_list; | 776 | struct list_head link_list; |
| 775 | struct hci_conn *parent; | 777 | struct hci_conn *parent; |
| @@ -25,7 +25,9 @@ · 24 unmodified lines | |||
| 25 | #include <linux/netfilter.h> /* for union nf_inet_addr */ | 25 | #include <linux/netfilter.h> /* for union nf_inet_addr */ |
| 26 | #include <linux/ip.h> | 26 | #include <linux/ip.h> |
| 27 | #include <linux/ipv6.h> /* for struct ipv6hdr */ | 27 | #include <linux/ipv6.h> /* for struct ipv6hdr */ |
| 28 | #include <net/route.h> | ||
| 28 | #include <net/ipv6.h> | 29 | #include <net/ipv6.h> |
| 30 | #include <net/ip6_fib.h> | ||
| 29 | #if IS_ENABLED(CONFIG_NF_CONNTRACK) | 31 | #if IS_ENABLED(CONFIG_NF_CONNTRACK) |
| 30 | #include <net/netfilter/nf_conntrack.h> | 32 | #include <net/netfilter/nf_conntrack.h> |
| 31 | #endif | 33 | #endif |
| @@ -1972,8 +1974,8 @@ int ip_vs_tunnel_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 1940 unmodified lines | |||
| 1972 | int ip_vs_dr_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, | 1974 | int ip_vs_dr_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1973 | struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); | 1975 | struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); |
| 1974 | int ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, | 1976 | int ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1975 | struct ip_vs_protocol *pp, int offset, | 1977 | struct ip_vs_protocol *pp, unsigned int toff, |
| 1976 | unsigned int hooknum, struct ip_vs_iphdr *iph); | 1978 | unsigned int hooknum, struct ip_vs_iphdr *ciph); |
| 1977 | void ip_vs_dest_dst_rcu_free(struct rcu_head *head); | 1979 | void ip_vs_dest_dst_rcu_free(struct rcu_head *head); |
| 1978 | 1980 | ||
| 1979 | #ifdef CONFIG_IP_VS_IPV6 | 1981 | #ifdef CONFIG_IP_VS_IPV6 |
| @@ -1986,8 +1988,8 @@ int ip_vs_tunnel_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 6 unmodified lines | |||
| 1986 | int ip_vs_dr_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, | 1988 | int ip_vs_dr_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1987 | struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); | 1989 | struct ip_vs_protocol *pp, struct ip_vs_iphdr *iph); |
| 1988 | int ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, | 1990 | int ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1989 | struct ip_vs_protocol *pp, int offset, | 1991 | struct ip_vs_protocol *pp, unsigned int toff, |
| 1990 | unsigned int hooknum, struct ip_vs_iphdr *iph); | 1992 | unsigned int hooknum, struct ip_vs_iphdr *ciph); |
| 1991 | #endif | 1993 | #endif |
| 1992 | 1994 | ||
| 1993 | #ifdef CONFIG_SYSCTL | 1995 | #ifdef CONFIG_SYSCTL |
| @@ -2059,15 +2061,15 @@ static inline bool ip_vs_conn_use_hash2(struct ip_vs_conn *cp) · 65 unmodified lines | |||
| 2059 | } | 2061 | } |
| 2060 | 2062 | ||
| 2061 | void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, | 2063 | void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 2062 | struct ip_vs_conn *cp, int dir); | 2064 | struct ip_vs_conn *cp, int dir, unsigned int toff, |
| 2065 | bool has_ports); | ||
| 2063 | 2066 | ||
| 2064 | #ifdef CONFIG_IP_VS_IPV6 | 2067 | #ifdef CONFIG_IP_VS_IPV6 |
| 2065 | void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, | 2068 | void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 2066 | struct ip_vs_conn *cp, int dir); | 2069 | struct ip_vs_conn *cp, int dir, unsigned int toff, |
| 2070 | bool has_ports, struct ip_vs_iphdr *ciph); | ||
| 2067 | #endif | 2071 | #endif |
| 2068 | 2072 | ||
| 2069 | __sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset); | ||
| 2070 | |||
| 2071 | static inline __wsum ip_vs_check_diff4(__be32 old, __be32 new, __wsum oldsum) | 2073 | static inline __wsum ip_vs_check_diff4(__be32 old, __be32 new, __wsum oldsum) |
| 2072 | { | 2074 | { |
| 2073 | __be32 diff[2] = { ~old, new }; | 2075 | __be32 diff[2] = { ~old, new }; |
| @@ -2093,6 +2095,33 @@ static inline __wsum ip_vs_check_diff2(__be16 old, __be16 new, __wsum oldsum) · 19 unmodified lines | |||
| 2093 | return csum_partial(diff, sizeof(diff), oldsum); | 2095 | return csum_partial(diff, sizeof(diff), oldsum); |
| 2094 | } | 2096 | } |
| 2095 | 2097 | ||
| 2098 | static inline bool ip_vs_checksum_needed(struct sk_buff *skb, int af) | ||
| 2099 | { | ||
| 2100 | /* Checksum unnecessary or already validated? */ | ||
| 2101 | if (skb_csum_unnecessary(skb)) | ||
| 2102 | return false; | ||
| 2103 | /* LOCAL_OUT ? */ | ||
| 2104 | if (!skb->dev || skb->dev->flags & IFF_LOOPBACK) | ||
| 2105 | return false; | ||
| 2106 | /* !LOCAL_IN (FORWARD) ? */ | ||
| 2107 | if (af == AF_INET6) { | ||
| 2108 | if (!(dst_rt6_info(skb_dst(skb))->rt6i_flags & RTF_LOCAL)) | ||
| 2109 | return false; | ||
| 2110 | } else { | ||
| 2111 | if (!(skb_rtable(skb)->rt_flags & RTCF_LOCAL)) | ||
| 2112 | return false; | ||
| 2113 | } | ||
| 2114 | return true; | ||
| 2115 | } | ||
| 2116 | |||
| 2117 | static inline bool ip_vs_checksum_common_check(struct sk_buff *skb, | ||
| 2118 | int offset, int proto, int af) | ||
| 2119 | { | ||
| 2120 | if (!ip_vs_checksum_needed(skb, af)) | ||
| 2121 | return true; | ||
| 2122 | return !nf_checksum(skb, NF_INET_LOCAL_IN, offset, proto, af); | ||
| 2123 | } | ||
| 2124 | |||
| 2096 | /* Forget current conntrack (unconfirmed) and attach notrack entry */ | 2125 | /* Forget current conntrack (unconfirmed) and attach notrack entry */ |
| 2097 | static inline void ip_vs_notrack(struct sk_buff *skb) | 2126 | static inline void ip_vs_notrack(struct sk_buff *skb) |
| 2098 | { | 2127 | { |
| @@ -161,6 +161,9 @@ static inline int nf_ct_expect_related(struct nf_conntrack_expect *expect, · 160 unmodified lines | |||
| 161 | return nf_ct_expect_related_report(expect, 0, 0, flags); | 161 | return nf_ct_expect_related_report(expect, 0, 0, flags); |
| 162 | } | 162 | } |
| 163 | 163 | ||
| 164 | int nf_ct_expect_related_pair(struct nf_conntrack_expect *expect[], | ||
| 165 | unsigned int flag); | ||
| 166 | |||
| 164 | struct nf_conn_help; | 167 | struct nf_conn_help; |
| 165 | void nf_ct_expectation_gc(struct nf_conn_help *master_help); | 168 | void nf_ct_expectation_gc(struct nf_conn_help *master_help); |
| 166 | 169 | ||
| @@ -1294,6 +1294,7 @@ static inline void nft_use_inc_restore(u32 *use) · 1293 unmodified lines | |||
| 1294 | * @sets: sets in the table | 1294 | * @sets: sets in the table |
| 1295 | * @objects: stateful objects in the table | 1295 | * @objects: stateful objects in the table |
| 1296 | * @flowtables: flow tables in the table | 1296 | * @flowtables: flow tables in the table |
| 1297 | * @objname_ht: hashtable for objects lookup by name | ||
| 1297 | * @hgenerator: handle generator state | 1298 | * @hgenerator: handle generator state |
| 1298 | * @handle: table handle | 1299 | * @handle: table handle |
| 1299 | * @use: number of chain references to this table | 1300 | * @use: number of chain references to this table |
| @@ -1313,6 +1314,7 @@ struct nft_table { · 13 unmodified lines | |||
| 1313 | struct list_head sets; | 1314 | struct list_head sets; |
| 1314 | struct list_head objects; | 1315 | struct list_head objects; |
| 1315 | struct list_head flowtables; | 1316 | struct list_head flowtables; |
| 1317 | struct rhltable objname_ht; | ||
| 1316 | u64 hgenerator; | 1318 | u64 hgenerator; |
| 1317 | u64 handle; | 1319 | u64 handle; |
| 1318 | u32 use; | 1320 | u32 use; |
| @@ -1400,7 +1402,7 @@ static inline void *nft_obj_data(const struct nft_object *obj) · 81 unmodified lines | |||
| 1400 | #define nft_expr_obj(expr) *((struct nft_object **)nft_expr_priv(expr)) | 1402 | #define nft_expr_obj(expr) *((struct nft_object **)nft_expr_priv(expr)) |
| 1401 | 1403 | ||
| 1402 | struct nft_object *nft_obj_lookup(const struct net *net, | 1404 | struct nft_object *nft_obj_lookup(const struct net *net, |
| 1403 | const struct nft_table *table, | 1405 | struct nft_table *table, |
| 1404 | const struct nlattr *nla, u32 objtype, | 1406 | const struct nlattr *nla, u32 objtype, |
| 1405 | u8 genmask); | 1407 | u8 genmask); |
| 1406 | 1408 | ||
| @@ -80,6 +80,7 @@ struct xdp_sock { · 79 unmodified lines | |||
| 80 | * call of __xsk_generic_xmit(). | 80 | * call of __xsk_generic_xmit(). |
| 81 | */ | 81 | */ |
| 82 | struct sk_buff *skb; | 82 | struct sk_buff *skb; |
| 83 | bool drain_cont; | ||
| 83 | 84 | ||
| 84 | struct list_head map_list; | 85 | struct list_head map_list; |
| 85 | /* Protects map_list */ | 86 | /* Protects map_list */ |
| @@ -78,6 +78,9 @@ struct xsk_buff_pool { · 77 unmodified lines | |||
| 78 | u32 chunk_size; | 78 | u32 chunk_size; |
| 79 | u32 chunk_shift; | 79 | u32 chunk_shift; |
| 80 | u32 frame_len; | 80 | u32 frame_len; |
| 81 | u32 tx_descs_nentries; | ||
| 82 | u32 reclaim_descs; | ||
| 83 | u32 tx_zc_pending_descs; | ||
| 81 | u32 xdp_zc_max_segs; | 84 | u32 xdp_zc_max_segs; |
| 82 | u8 tx_metadata_len; /* inherited from umem */ | 85 | u8 tx_metadata_len; /* inherited from umem */ |
| 83 | u8 cached_need_wakeup; | 86 | u8 cached_need_wakeup; |
| @@ -102,12 +105,14 @@ struct xsk_buff_pool { · 18 unmodified lines | |||
| 102 | 105 | ||
| 103 | /* AF_XDP core. */ | 106 | /* AF_XDP core. */ |
| 104 | struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, | 107 | struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, |
| 105 | struct xdp_umem *umem); | 108 | struct xdp_umem *umem, |
| 109 | u32 max_segs); | ||
| 106 | int xp_assign_dev(struct xsk_buff_pool *pool, struct net_device *dev, | 110 | int xp_assign_dev(struct xsk_buff_pool *pool, struct net_device *dev, |
| 107 | u16 queue_id, u16 flags); | 111 | u16 queue_id, u16 flags); |
| 108 | int xp_assign_dev_shared(struct xsk_buff_pool *pool, struct xdp_sock *umem_xs, | 112 | int xp_assign_dev_shared(struct xsk_buff_pool *pool, struct xdp_sock *umem_xs, |
| 109 | struct net_device *dev, u16 queue_id); | 113 | struct net_device *dev, u16 queue_id); |
| 110 | int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs); | 114 | int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs, |
| 115 | u32 max_segs); | ||
| 111 | void xp_destroy(struct xsk_buff_pool *pool); | 116 | void xp_destroy(struct xsk_buff_pool *pool); |
| 112 | void xp_get_pool(struct xsk_buff_pool *pool); | 117 | void xp_get_pool(struct xsk_buff_pool *pool); |
| 113 | bool xp_put_pool(struct xsk_buff_pool *pool); | 118 | bool xp_put_pool(struct xsk_buff_pool *pool); |
| @@ -1123,6 +1123,8 @@ static struct hci_conn *__hci_conn_add(struct hci_dev *hdev, int type, · 1122 unmodified lines | |||
| 1123 | INIT_DELAYED_WORK(&conn->idle_work, hci_conn_idle); | 1123 | INIT_DELAYED_WORK(&conn->idle_work, hci_conn_idle); |
| 1124 | INIT_DELAYED_WORK(&conn->le_conn_timeout, le_conn_timeout); | 1124 | INIT_DELAYED_WORK(&conn->le_conn_timeout, le_conn_timeout); |
| 1125 | 1125 | ||
| 1126 | spin_lock_init(&conn->proto_lock); | ||
| 1127 | |||
| 1126 | atomic_set(&conn->refcnt, 0); | 1128 | atomic_set(&conn->refcnt, 0); |
| 1127 | 1129 | ||
| 1128 | hci_dev_hold(hdev); | 1130 | hci_dev_hold(hdev); |
| @@ -3163,6 +3165,13 @@ static int abort_conn_sync(struct hci_dev *hdev, void *data) · 2034 unmodified lines | |||
| 3163 | return hci_abort_conn_sync(hdev, conn, conn->abort_reason); | 3165 | return hci_abort_conn_sync(hdev, conn, conn->abort_reason); |
| 3164 | } | 3166 | } |
| 3165 | 3167 | ||
| 3168 | static void abort_conn_destroy(struct hci_dev *hdev, void *data, int err) | ||
| 3169 | { | ||
| 3170 | struct hci_conn *conn = data; | ||
| 3171 | |||
| 3172 | hci_conn_put(conn); | ||
| 3173 | } | ||
| 3174 | |||
| 3166 | int hci_abort_conn(struct hci_conn *conn, u8 reason) | 3175 | int hci_abort_conn(struct hci_conn *conn, u8 reason) |
| 3167 | { | 3176 | { |
| 3168 | struct hci_dev *hdev = conn->hdev; | 3177 | struct hci_dev *hdev = conn->hdev; |
| @@ -3188,7 +3197,10 @@ int hci_abort_conn(struct hci_conn *conn, u8 reason) · 19 unmodified lines | |||
| 3188 | * as a result to MGMT_OP_DISCONNECT/MGMT_OP_UNPAIR which does | 3197 | * as a result to MGMT_OP_DISCONNECT/MGMT_OP_UNPAIR which does |
| 3189 | * already queue its callback on cmd_sync_work. | 3198 | * already queue its callback on cmd_sync_work. |
| 3190 | */ | 3199 | */ |
| 3191 | err = hci_cmd_sync_run_once(hdev, abort_conn_sync, conn, NULL); | 3200 | err = hci_cmd_sync_run_once(hdev, abort_conn_sync, hci_conn_get(conn), |
| 3201 | abort_conn_destroy); | ||
| 3202 | if (err) | ||
| 3203 | hci_conn_put(conn); | ||
| 3192 | return (err == -EEXIST) ? 0 : err; | 3204 | return (err == -EEXIST) ? 0 : err; |
| 3193 | } | 3205 | } |
| 3194 | 3206 | ||
| @@ -1233,10 +1233,11 @@ static int hci_set_adv_set_random_addr_sync(struct hci_dev *hdev, u8 instance, · 1232 unmodified lines | |||
| 1233 | } | 1233 | } |
| 1234 | 1234 | ||
| 1235 | static int | 1235 | static int |
| 1236 | hci_set_ext_adv_params_sync(struct hci_dev *hdev, struct adv_info *adv, | 1236 | hci_set_ext_adv_params_sync(struct hci_dev *hdev, u8 instance, |
| 1237 | const struct hci_cp_le_set_ext_adv_params *cp, | 1237 | const struct hci_cp_le_set_ext_adv_params *cp, |
| 1238 | struct hci_rp_le_set_ext_adv_params *rp) | 1238 | struct hci_rp_le_set_ext_adv_params *rp) |
| 1239 | { | 1239 | { |
| 1240 | struct adv_info *adv; | ||
| 1240 | struct sk_buff *skb; | 1241 | struct sk_buff *skb; |
| 1241 | 1242 | ||
| 1242 | skb = __hci_cmd_sync(hdev, HCI_OP_LE_SET_EXT_ADV_PARAMS, sizeof(*cp), | 1243 | skb = __hci_cmd_sync(hdev, HCI_OP_LE_SET_EXT_ADV_PARAMS, sizeof(*cp), |
| @@ -1264,11 +1265,15 @@ hci_set_ext_adv_params_sync(struct hci_dev *hdev, struct adv_info *adv, · 21 unmodified lines | |||
| 1264 | 1265 | ||
| 1265 | if (!rp->status) { | 1266 | if (!rp->status) { |
| 1266 | hdev->adv_addr_type = cp->own_addr_type; | 1267 | hdev->adv_addr_type = cp->own_addr_type; |
| 1267 | if (!cp->handle) { | 1268 | if (!instance) { |
| 1268 | /* Store in hdev for instance 0 */ | 1269 | /* Store in hdev for instance 0 */ |
| 1269 | hdev->adv_tx_power = rp->tx_power; | 1270 | hdev->adv_tx_power = rp->tx_power; |
| 1270 | } else if (adv) { | 1271 | } else { |
| 1271 | adv->tx_power = rp->tx_power; | 1272 | hci_dev_lock(hdev); |
| 1273 | adv = hci_find_adv_instance(hdev, instance); | ||
| 1274 | if (adv) | ||
| 1275 | adv->tx_power = rp->tx_power; | ||
| 1276 | hci_dev_unlock(hdev); | ||
| 1272 | } | 1277 | } |
| 1273 | } | 1278 | } |
| 1274 | 1279 | ||
| @@ -1284,9 +1289,13 @@ static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance) · 9 unmodified lines | |||
| 1284 | int err; | 1289 | int err; |
| 1285 | 1290 | ||
| 1286 | if (instance) { | 1291 | if (instance) { |
| 1292 | hci_dev_lock(hdev); | ||
| 1293 | |||
| 1287 | adv = hci_find_adv_instance(hdev, instance); | 1294 | adv = hci_find_adv_instance(hdev, instance); |
| 1288 | if (!adv || !adv->adv_data_changed) | 1295 | if (!adv || !adv->adv_data_changed) { |
| 1296 | hci_dev_unlock(hdev); | ||
| 1289 | return 0; | 1297 | return 0; |
| 1298 | } | ||
| 1290 | } | 1299 | } |
| 1291 | 1300 | ||
| 1292 | len = eir_create_adv_data(hdev, instance, pdu->data, | 1301 | len = eir_create_adv_data(hdev, instance, pdu->data, |
| @@ -1297,16 +1306,27 @@ static int hci_set_ext_adv_data_sync(struct hci_dev *hdev, u8 instance) · 4 unmodified lines | |||
| 1297 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; | 1306 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; |
| 1298 | pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; | 1307 | pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; |
| 1299 | 1308 | ||
| 1309 | if (adv) { | ||
| 1310 | adv->adv_data_changed = false; | ||
| 1311 | hci_dev_unlock(hdev); | ||
| 1312 | } | ||
| 1313 | |||
| 1300 | err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_DATA, | 1314 | err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_ADV_DATA, |
| 1301 | struct_size(pdu, data, len), pdu, | 1315 | struct_size(pdu, data, len), pdu, |
| 1302 | HCI_CMD_TIMEOUT); | 1316 | HCI_CMD_TIMEOUT); |
| 1303 | if (err) | 1317 | if (err) { |
| 1318 | if (instance) { | ||
| 1319 | hci_dev_lock(hdev); | ||
| 1320 | adv = hci_find_adv_instance(hdev, instance); | ||
| 1321 | if (adv) | ||
| 1322 | adv->adv_data_changed = true; | ||
| 1323 | hci_dev_unlock(hdev); | ||
| 1324 | } | ||
| 1325 | |||
| 1304 | return err; | 1326 | return err; |
| 1327 | } | ||
| 1305 | 1328 | ||
| 1306 | /* Update data if the command succeed */ | 1329 | if (!instance) { |
| 1307 | if (adv) { | ||
| 1308 | adv->adv_data_changed = false; | ||
| 1309 | } else { | ||
| 1310 | memcpy(hdev->adv_data, pdu->data, len); | 1330 | memcpy(hdev->adv_data, pdu->data, len); |
| 1311 | hdev->adv_data_len = len; | 1331 | hdev->adv_data_len = len; |
| 1312 | } | 1332 | } |
| @@ -1360,22 +1380,22 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 47 unmodified lines | |||
| 1360 | struct adv_info *adv; | 1380 | struct adv_info *adv; |
| 1361 | bool secondary_adv; | 1381 | bool secondary_adv; |
| 1362 | 1382 | ||
| 1363 | if (instance > 0) { | ||
| 1364 | adv = hci_find_adv_instance(hdev, instance); | ||
| 1365 | if (!adv) | ||
| 1366 | return -EINVAL; | ||
| 1367 | } else { | ||
| 1368 | adv = NULL; | ||
| 1369 | } | ||
| 1370 | |||
| 1371 | /* Updating parameters of an active instance will return a | 1383 | /* Updating parameters of an active instance will return a |
| 1372 | * Command Disallowed error, so we must first disable the | 1384 | * Command Disallowed error, so disable it before taking a snapshot. |
| 1373 | * instance if it is active. | ||
| 1374 | */ | 1385 | */ |
| 1375 | if (adv) { | 1386 | if (instance > 0) { |
| 1376 | err = hci_disable_ext_adv_instance_sync(hdev, instance); | 1387 | err = hci_disable_ext_adv_instance_sync(hdev, instance); |
| 1377 | if (err) | 1388 | if (err) |
| 1378 | return err; | 1389 | return err; |
| 1390 | |||
| 1391 | hci_dev_lock(hdev); | ||
| 1392 | adv = hci_find_adv_instance(hdev, instance); | ||
| 1393 | if (!adv) { | ||
| 1394 | hci_dev_unlock(hdev); | ||
| 1395 | return -EINVAL; | ||
| 1396 | } | ||
| 1397 | } else { | ||
| 1398 | adv = NULL; | ||
| 1379 | } | 1399 | } |
| 1380 | 1400 | ||
| 1381 | flags = hci_adv_instance_flags(hdev, instance); | 1401 | flags = hci_adv_instance_flags(hdev, instance); |
| @@ -1386,8 +1406,11 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 4 unmodified lines | |||
| 1386 | connectable = (flags & MGMT_ADV_FLAG_CONNECTABLE) || | 1406 | connectable = (flags & MGMT_ADV_FLAG_CONNECTABLE) || |
| 1387 | mgmt_get_connectable(hdev); | 1407 | mgmt_get_connectable(hdev); |
| 1388 | 1408 | ||
| 1389 | if (!is_advertising_allowed(hdev, connectable)) | 1409 | if (!is_advertising_allowed(hdev, connectable)) { |
| 1410 | if (instance) | ||
| 1411 | hci_dev_unlock(hdev); | ||
| 1390 | return -EPERM; | 1412 | return -EPERM; |
| 1413 | } | ||
| 1391 | 1414 | ||
| 1392 | /* Set require_privacy to true only when non-connectable | 1415 | /* Set require_privacy to true only when non-connectable |
| 1393 | * advertising is used and it is not periodic. | 1416 | * advertising is used and it is not periodic. |
| @@ -1398,8 +1421,11 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 4 unmodified lines | |||
| 1398 | err = hci_get_random_address(hdev, require_privacy, | 1421 | err = hci_get_random_address(hdev, require_privacy, |
| 1399 | adv_use_rpa(hdev, flags), adv, | 1422 | adv_use_rpa(hdev, flags), adv, |
| 1400 | &own_addr_type, &random_addr); | 1423 | &own_addr_type, &random_addr); |
| 1401 | if (err < 0) | 1424 | if (err < 0) { |
| 1425 | if (instance) | ||
| 1426 | hci_dev_unlock(hdev); | ||
| 1402 | return err; | 1427 | return err; |
| 1428 | } | ||
| 1403 | 1429 | ||
| 1404 | memset(&cp, 0, sizeof(cp)); | 1430 | memset(&cp, 0, sizeof(cp)); |
| 1405 | 1431 | ||
| @@ -1450,6 +1476,9 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 44 unmodified lines | |||
| 1450 | cp.channel_map = hdev->le_adv_channel_map; | 1476 | cp.channel_map = hdev->le_adv_channel_map; |
| 1451 | cp.handle = adv ? adv->handle : instance; | 1477 | cp.handle = adv ? adv->handle : instance; |
| 1452 | 1478 | ||
| 1479 | if (instance) | ||
| 1480 | hci_dev_unlock(hdev); | ||
| 1481 | |||
| 1453 | if (flags & MGMT_ADV_FLAG_SEC_2M) { | 1482 | if (flags & MGMT_ADV_FLAG_SEC_2M) { |
| 1454 | cp.primary_phy = HCI_ADV_PHY_1M; | 1483 | cp.primary_phy = HCI_ADV_PHY_1M; |
| 1455 | cp.secondary_phy = HCI_ADV_PHY_2M; | 1484 | cp.secondary_phy = HCI_ADV_PHY_2M; |
| @@ -1462,12 +1491,12 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 6 unmodified lines | |||
| 1462 | cp.secondary_phy = HCI_ADV_PHY_1M; | 1491 | cp.secondary_phy = HCI_ADV_PHY_1M; |
| 1463 | } | 1492 | } |
| 1464 | 1493 | ||
| 1465 | err = hci_set_ext_adv_params_sync(hdev, adv, &cp, &rp); | 1494 | err = hci_set_ext_adv_params_sync(hdev, instance, &cp, &rp); |
| 1466 | if (err) | 1495 | if (err) |
| 1467 | return err; | 1496 | return err; |
| 1468 | 1497 | ||
| 1469 | /* Update adv data as tx power is known now */ | 1498 | /* Update adv data as tx power is known now */ |
| 1470 | err = hci_set_ext_adv_data_sync(hdev, cp.handle); | 1499 | err = hci_set_ext_adv_data_sync(hdev, instance); |
| 1471 | if (err) | 1500 | if (err) |
| 1472 | return err; | 1501 | return err; |
| 1473 | 1502 | ||
| @@ -1475,9 +1504,14 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 1 unmodified line | |||
| 1475 | own_addr_type == ADDR_LE_DEV_RANDOM_RESOLVED) && | 1504 | own_addr_type == ADDR_LE_DEV_RANDOM_RESOLVED) && |
| 1476 | bacmp(&random_addr, BDADDR_ANY)) { | 1505 | bacmp(&random_addr, BDADDR_ANY)) { |
| 1477 | /* Check if random address need to be updated */ | 1506 | /* Check if random address need to be updated */ |
| 1478 | if (adv) { | 1507 | if (instance) { |
| 1479 | if (!bacmp(&random_addr, &adv->random_addr)) | 1508 | hci_dev_lock(hdev); |
| 1509 | adv = hci_find_adv_instance(hdev, instance); | ||
| 1510 | if (!adv || !bacmp(&random_addr, &adv->random_addr)) { | ||
| 1511 | hci_dev_unlock(hdev); | ||
| 1480 | return 0; | 1512 | return 0; |
| 1513 | } | ||
| 1514 | hci_dev_unlock(hdev); | ||
| 1481 | } else { | 1515 | } else { |
| 1482 | if (!bacmp(&random_addr, &hdev->random_addr)) | 1516 | if (!bacmp(&random_addr, &hdev->random_addr)) |
| 1483 | return 0; | 1517 | return 0; |
| @@ -1499,9 +1533,13 @@ static int hci_set_ext_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) · 15 unmodified lines | |||
| 1499 | int err; | 1533 | int err; |
| 1500 | 1534 | ||
| 1501 | if (instance) { | 1535 | if (instance) { |
| 1536 | hci_dev_lock(hdev); | ||
| 1537 | |||
| 1502 | adv = hci_find_adv_instance(hdev, instance); | 1538 | adv = hci_find_adv_instance(hdev, instance); |
| 1503 | if (!adv || !adv->scan_rsp_changed) | 1539 | if (!adv || !adv->scan_rsp_changed) { |
| 1540 | hci_dev_unlock(hdev); | ||
| 1504 | return 0; | 1541 | return 0; |
| 1542 | } | ||
| 1505 | } | 1543 | } |
| 1506 | 1544 | ||
| 1507 | len = eir_create_scan_rsp(hdev, instance, pdu->data); | 1545 | len = eir_create_scan_rsp(hdev, instance, pdu->data); |
| @@ -1511,15 +1549,27 @@ static int hci_set_ext_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) · 3 unmodified lines | |||
| 1511 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; | 1549 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; |
| 1512 | pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; | 1550 | pdu->frag_pref = LE_SET_ADV_DATA_NO_FRAG; |
| 1513 | 1551 | ||
| 1552 | if (adv) { | ||
| 1553 | adv->scan_rsp_changed = false; | ||
| 1554 | hci_dev_unlock(hdev); | ||
| 1555 | } | ||
| 1556 | |||
| 1514 | err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_RSP_DATA, | 1557 | err = __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_EXT_SCAN_RSP_DATA, |
| 1515 | struct_size(pdu, data, len), pdu, | 1558 | struct_size(pdu, data, len), pdu, |
| 1516 | HCI_CMD_TIMEOUT); | 1559 | HCI_CMD_TIMEOUT); |
| 1517 | if (err) | 1560 | if (err) { |
| 1561 | if (instance) { | ||
| 1562 | hci_dev_lock(hdev); | ||
| 1563 | adv = hci_find_adv_instance(hdev, instance); | ||
| 1564 | if (adv) | ||
| 1565 | adv->scan_rsp_changed = true; | ||
| 1566 | hci_dev_unlock(hdev); | ||
| 1567 | } | ||
| 1568 | |||
| 1518 | return err; | 1569 | return err; |
| 1570 | } | ||
| 1519 | 1571 | ||
| 1520 | if (adv) { | 1572 | if (!instance) { |
| 1521 | adv->scan_rsp_changed = false; | ||
| 1522 | } else { | ||
| 1523 | memcpy(hdev->scan_rsp_data, pdu->data, len); | 1573 | memcpy(hdev->scan_rsp_data, pdu->data, len); |
| 1524 | hdev->scan_rsp_data_len = len; | 1574 | hdev->scan_rsp_data_len = len; |
| 1525 | } | 1575 | } |
| @@ -1534,8 +1584,14 @@ static int __hci_set_scan_rsp_data_sync(struct hci_dev *hdev, u8 instance) · 8 unmodified lines | |||
| 1534 | 1584 | ||
| 1535 | memset(&cp, 0, sizeof(cp)); | 1585 | memset(&cp, 0, sizeof(cp)); |
| 1536 | 1586 | ||
| 1587 | if (instance) | ||
| 1588 | hci_dev_lock(hdev); | ||
| 1589 | |||
| 1537 | len = eir_create_scan_rsp(hdev, instance, cp.data); | 1590 | len = eir_create_scan_rsp(hdev, instance, cp.data); |
| 1538 | 1591 | ||
| 1592 | if (instance) | ||
| 1593 | hci_dev_unlock(hdev); | ||
| 1594 | |||
| 1539 | if (hdev->scan_rsp_data_len == len && | 1595 | if (hdev->scan_rsp_data_len == len && |
| 1540 | !memcmp(cp.data, hdev->scan_rsp_data, len)) | 1596 | !memcmp(cp.data, hdev->scan_rsp_data, len)) |
| 1541 | return 0; | 1597 | return 0; |
| @@ -1670,9 +1726,13 @@ static int hci_set_per_adv_data_sync(struct hci_dev *hdev, u8 instance) · 128 unmodified lines | |||
| 1670 | struct adv_info *adv = NULL; | 1726 | struct adv_info *adv = NULL; |
| 1671 | 1727 | ||
| 1672 | if (instance) { | 1728 | if (instance) { |
| 1729 | hci_dev_lock(hdev); | ||
| 1730 | |||
| 1673 | adv = hci_find_adv_instance(hdev, instance); | 1731 | adv = hci_find_adv_instance(hdev, instance); |
| 1674 | if (!adv || !adv->periodic) | 1732 | if (!adv || !adv->periodic) { |
| 1733 | hci_dev_unlock(hdev); | ||
| 1675 | return 0; | 1734 | return 0; |
| 1735 | } | ||
| 1676 | } | 1736 | } |
| 1677 | 1737 | ||
| 1678 | len = eir_create_per_adv_data(hdev, instance, pdu->data); | 1738 | len = eir_create_per_adv_data(hdev, instance, pdu->data); |
| @@ -1681,6 +1741,9 @@ static int hci_set_per_adv_data_sync(struct hci_dev *hdev, u8 instance) · 2 unmodified lines | |||
| 1681 | pdu->handle = adv ? adv->handle : instance; | 1741 | pdu->handle = adv ? adv->handle : instance; |
| 1682 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; | 1742 | pdu->operation = LE_SET_ADV_DATA_OP_COMPLETE; |
| 1683 | 1743 | ||
| 1744 | if (adv) | ||
| 1745 | hci_dev_unlock(hdev); | ||
| 1746 | |||
| 1684 | return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_PER_ADV_DATA, | 1747 | return __hci_cmd_sync_status(hdev, HCI_OP_LE_SET_PER_ADV_DATA, |
| 1685 | struct_size(pdu, data, len), pdu, | 1748 | struct_size(pdu, data, len), pdu, |
| 1686 | HCI_CMD_TIMEOUT); | 1749 | HCI_CMD_TIMEOUT); |
| @@ -6523,7 +6586,7 @@ static int hci_le_ext_directed_advertising_sync(struct hci_dev *hdev, · 4836 unmodified lines | |||
| 6523 | if (err) | 6586 | if (err) |
| 6524 | return err; | 6587 | return err; |
| 6525 | 6588 | ||
| 6526 | err = hci_set_ext_adv_params_sync(hdev, NULL, &cp, &rp); | 6589 | err = hci_set_ext_adv_params_sync(hdev, 0, &cp, &rp); |
| 6527 | if (err) | 6590 | if (err) |
| 6528 | return err; | 6591 | return err; |
| 6529 | 6592 | ||
| @@ -6678,11 +6741,6 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data) · 148 unmodified lines | |||
| 6678 | 6741 | ||
| 6679 | bt_dev_dbg(hdev, "conn %p", conn); | 6742 | bt_dev_dbg(hdev, "conn %p", conn); |
| 6680 | 6743 | ||
| 6681 | /* Hold a reference so conn stays valid for the HCI_CONN_CREATE | ||
| 6682 | * clear_bit() at done. | ||
| 6683 | */ | ||
| 6684 | hci_conn_get(conn); | ||
| 6685 | |||
| 6686 | clear_bit(HCI_CONN_SCANNING, &conn->flags); | 6744 | clear_bit(HCI_CONN_SCANNING, &conn->flags); |
| 6687 | conn->state = BT_CONNECT; | 6745 | conn->state = BT_CONNECT; |
| 6688 | 6746 | ||
| @@ -6694,8 +6752,9 @@ static int hci_le_create_conn_sync(struct hci_dev *hdev, void *data) · 5 unmodified lines | |||
| 6694 | if (hci_dev_test_flag(hdev, HCI_LE_SCAN) && | 6752 | if (hci_dev_test_flag(hdev, HCI_LE_SCAN) && |
| 6695 | hdev->le_scan_type == LE_SCAN_ACTIVE && | 6753 | hdev->le_scan_type == LE_SCAN_ACTIVE && |
| 6696 | !hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES)) { | 6754 | !hci_dev_test_flag(hdev, HCI_LE_SIMULTANEOUS_ROLES)) { |
| 6697 | hci_conn_del(conn); | 6755 | conn->state = BT_OPEN; |
| 6698 | hci_conn_put(conn); | 6756 | hci_abort_conn_sync(hdev, conn, |
| 6757 | HCI_ERROR_REJ_LIMITED_RESOURCES); | ||
| 6699 | return -EBUSY; | 6758 | return -EBUSY; |
| 6700 | } | 6759 | } |
| 6701 | 6760 | ||
| @@ -6793,7 +6852,6 @@ done: · 91 unmodified lines | |||
| 6793 | 6852 | ||
| 6794 | /* Re-enable advertising after the connection attempt is finished. */ | 6853 | /* Re-enable advertising after the connection attempt is finished. */ |
| 6795 | hci_resume_advertising_sync(hdev); | 6854 | hci_resume_advertising_sync(hdev); |
| 6796 | hci_conn_put(conn); | ||
| 6797 | return err; | 6855 | return err; |
| 6798 | } | 6856 | } |
| 6799 | 6857 | ||
| @@ -7068,11 +7126,6 @@ static int hci_acl_create_conn_sync(struct hci_dev *hdev, void *data) · 268 unmodified lines | |||
| 7068 | else | 7126 | else |
| 7069 | cp.role_switch = 0x00; | 7127 | cp.role_switch = 0x00; |
| 7070 | 7128 | ||
| 7071 | /* Hold a reference so conn stays valid for the HCI_CONN_CREATE | ||
| 7072 | * clear_bit() below. | ||
| 7073 | */ | ||
| 7074 | hci_conn_get(conn); | ||
| 7075 | |||
| 7076 | /* Mark create connection in flight so hci_cancel_connect_sync() can | 7129 | /* Mark create connection in flight so hci_cancel_connect_sync() can |
| 7077 | * cancel it while blocking on the connection complete event. | 7130 | * cancel it while blocking on the connection complete event. |
| 7078 | */ | 7131 | */ |
| @@ -7084,17 +7137,27 @@ static int hci_acl_create_conn_sync(struct hci_dev *hdev, void *data) · 5 unmodified lines | |||
| 7084 | conn->conn_timeout, NULL); | 7137 | conn->conn_timeout, NULL); |
| 7085 | 7138 | ||
| 7086 | clear_bit(HCI_CONN_CREATE, &conn->flags); | 7139 | clear_bit(HCI_CONN_CREATE, &conn->flags); |
| 7087 | hci_conn_put(conn); | ||
| 7088 | 7140 | ||
| 7089 | return err; | 7141 | return err; |
| 7090 | } | 7142 | } |
| 7091 | 7143 | ||
| 7144 | static void hci_acl_create_conn_sync_complete(struct hci_dev *hdev, void *data, | ||
| 7145 | int err) | ||
| 7146 | { | ||
| 7147 | struct hci_conn *conn = data; | ||
| 7148 | |||
| 7149 | hci_conn_put(conn); | ||
| 7150 | } | ||
| 7151 | |||
| 7092 | int hci_connect_acl_sync(struct hci_dev *hdev, struct hci_conn *conn) | 7152 | int hci_connect_acl_sync(struct hci_dev *hdev, struct hci_conn *conn) |
| 7093 | { | 7153 | { |
| 7094 | int err; | 7154 | int err; |
| 7095 | 7155 | ||
| 7096 | err = hci_cmd_sync_queue_once(hdev, hci_acl_create_conn_sync, conn, | 7156 | err = hci_cmd_sync_queue_once(hdev, hci_acl_create_conn_sync, |
| 7097 | NULL); | 7157 | hci_conn_get(conn), |
| 7158 | hci_acl_create_conn_sync_complete); | ||
| 7159 | if (err) | ||
| 7160 | hci_conn_put(conn); | ||
| 7098 | return (err == -EEXIST) ? 0 : err; | 7161 | return (err == -EEXIST) ? 0 : err; |
| 7099 | } | 7162 | } |
| 7100 | 7163 | ||
| @@ -7105,36 +7168,41 @@ static void create_le_conn_complete(struct hci_dev *hdev, void *data, int err) · 4 unmodified lines | |||
| 7105 | bt_dev_dbg(hdev, "err %d", err); | 7168 | bt_dev_dbg(hdev, "err %d", err); |
| 7106 | 7169 | ||
| 7107 | if (err == -ECANCELED) | 7170 | if (err == -ECANCELED) |
| 7108 | return; | 7171 | goto done; |
| 7109 | 7172 | ||
| 7110 | hci_dev_lock(hdev); | 7173 | hci_dev_lock(hdev); |
| 7111 | 7174 | ||
| 7112 | if (!hci_conn_valid(hdev, conn)) | 7175 | if (!hci_conn_valid(hdev, conn)) |
| 7113 | goto done; | 7176 | goto unlock; |
| 7114 | 7177 | ||
| 7115 | if (!err) { | 7178 | if (!err) { |
| 7116 | hci_connect_le_scan_cleanup(conn, 0x00); | 7179 | hci_connect_le_scan_cleanup(conn, 0x00); |
| 7117 | goto done; | 7180 | goto unlock; |
| 7118 | } | 7181 | } |
| 7119 | 7182 | ||
| 7120 | /* Check if connection is still pending */ | 7183 | /* Check if connection is still pending */ |
| 7121 | if (conn != hci_lookup_le_connect(hdev)) | 7184 | if (conn != hci_lookup_le_connect(hdev)) |
| 7122 | goto done; | 7185 | goto unlock; |
| 7123 | 7186 | ||
| 7124 | /* Flush to make sure we send create conn cancel command if needed */ | 7187 | /* Flush to make sure we send create conn cancel command if needed */ |
| 7125 | flush_delayed_work(&conn->le_conn_timeout); | 7188 | flush_delayed_work(&conn->le_conn_timeout); |
| 7126 | hci_conn_failed(conn, bt_status(err)); | 7189 | hci_conn_failed(conn, bt_status(err)); |
| 7127 | 7190 | ||
| 7128 | done: | 7191 | unlock: |
| 7129 | hci_dev_unlock(hdev); | 7192 | hci_dev_unlock(hdev); |
| 7193 | done: | ||
| 7194 | hci_conn_put(conn); | ||
| 7130 | } | 7195 | } |
| 7131 | 7196 | ||
| 7132 | int hci_connect_le_sync(struct hci_dev *hdev, struct hci_conn *conn) | 7197 | int hci_connect_le_sync(struct hci_dev *hdev, struct hci_conn *conn) |
| 7133 | { | 7198 | { |
| 7134 | int err; | 7199 | int err; |
| 7135 | 7200 | ||
| 7136 | err = hci_cmd_sync_queue_once(hdev, hci_le_create_conn_sync, conn, | 7201 | err = hci_cmd_sync_queue_once(hdev, hci_le_create_conn_sync, |
| 7202 | hci_conn_get(conn), | ||
| 7137 | create_le_conn_complete); | 7203 | create_le_conn_complete); |
| 7204 | if (err) | ||
| 7205 | hci_conn_put(conn); | ||
| 7138 | return (err == -EEXIST) ? 0 : err; | 7206 | return (err == -EEXIST) ? 0 : err; |
| 7139 | } | 7207 | } |
| 7140 | 7208 | ||
| @@ -7257,7 +7325,7 @@ static void create_pa_complete(struct hci_dev *hdev, void *data, int err) · 116 unmodified lines | |||
| 7257 | bt_dev_dbg(hdev, "err %d", err); | 7325 | bt_dev_dbg(hdev, "err %d", err); |
| 7258 | 7326 | ||
| 7259 | if (err == -ECANCELED) | 7327 | if (err == -ECANCELED) |
| 7260 | return; | 7328 | goto done; |
| 7261 | 7329 | ||
| 7262 | hci_dev_lock(hdev); | 7330 | hci_dev_lock(hdev); |
| 7263 | 7331 | ||
| @@ -7281,6 +7349,8 @@ static void create_pa_complete(struct hci_dev *hdev, void *data, int err) · 17 unmodified lines | |||
| 7281 | 7349 | ||
| 7282 | unlock: | 7350 | unlock: |
| 7283 | hci_dev_unlock(hdev); | 7351 | hci_dev_unlock(hdev); |
| 7352 | done: | ||
| 7353 | hci_conn_put(conn); | ||
| 7284 | } | 7354 | } |
| 7285 | 7355 | ||
| 7286 | static int hci_le_past_params_sync(struct hci_dev *hdev, struct hci_conn *conn, | 7356 | static int hci_le_past_params_sync(struct hci_dev *hdev, struct hci_conn *conn, |
| @@ -7431,8 +7501,11 @@ int hci_connect_pa_sync(struct hci_dev *hdev, struct hci_conn *conn) · 144 unmodified lines | |||
| 7431 | { | 7501 | { |
| 7432 | int err; | 7502 | int err; |
| 7433 | 7503 | ||
| 7434 | err = hci_cmd_sync_queue_once(hdev, hci_le_pa_create_sync, conn, | 7504 | err = hci_cmd_sync_queue_once(hdev, hci_le_pa_create_sync, |
| 7505 | hci_conn_get(conn), | ||
| 7435 | create_pa_complete); | 7506 | create_pa_complete); |
| 7507 | if (err) | ||
| 7508 | hci_conn_put(conn); | ||
| 7436 | return (err == -EEXIST) ? 0 : err; | 7509 | return (err == -EEXIST) ? 0 : err; |
| 7437 | } | 7510 | } |
| 7438 | 7511 | ||
| @@ -7443,10 +7516,12 @@ static void create_big_complete(struct hci_dev *hdev, void *data, int err) · 4 unmodified lines | |||
| 7443 | bt_dev_dbg(hdev, "err %d", err); | 7516 | bt_dev_dbg(hdev, "err %d", err); |
| 7444 | 7517 | ||
| 7445 | if (err == -ECANCELED) | 7518 | if (err == -ECANCELED) |
| 7446 | return; | 7519 | goto done; |
| 7447 | 7520 | ||
| 7448 | if (hci_conn_valid(hdev, conn)) | 7521 | clear_bit(HCI_CONN_CREATE_BIG_SYNC, &conn->flags); |
| 7449 | clear_bit(HCI_CONN_CREATE_BIG_SYNC, &conn->flags); | 7522 | |
| 7523 | done: | ||
| 7524 | hci_conn_put(conn); | ||
| 7450 | } | 7525 | } |
| 7451 | 7526 | ||
| 7452 | static int hci_le_big_create_sync(struct hci_dev *hdev, void *data) | 7527 | static int hci_le_big_create_sync(struct hci_dev *hdev, void *data) |
| @@ -7498,8 +7573,14 @@ int hci_connect_big_sync(struct hci_dev *hdev, struct hci_conn *conn) · 45 unmodified lines | |||
| 7498 | { | 7573 | { |
| 7499 | int err; | 7574 | int err; |
| 7500 | 7575 | ||
| 7501 | err = hci_cmd_sync_queue_once(hdev, hci_le_big_create_sync, conn, | 7576 | if (!conn) |
| 7577 | return 0; | ||
| 7578 | |||
| 7579 | err = hci_cmd_sync_queue_once(hdev, hci_le_big_create_sync, | ||
| 7580 | hci_conn_get(conn), | ||
| 7502 | create_big_complete); | 7581 | create_big_complete); |
| 7582 | if (err) | ||
| 7583 | hci_conn_put(conn); | ||
| 7503 | return (err == -EEXIST) ? 0 : err; | 7584 | return (err == -EEXIST) ? 0 : err; |
| 7504 | } | 7585 | } |
| 7505 | 7586 | ||
| @@ -7514,6 +7595,8 @@ static void past_complete(struct hci_dev *hdev, void *data, int err) · 8 unmodified lines | |||
| 7514 | 7595 | ||
| 7515 | bt_dev_dbg(hdev, "err %d", err); | 7596 | bt_dev_dbg(hdev, "err %d", err); |
| 7516 | 7597 | ||
| 7598 | hci_conn_put(past->conn); | ||
| 7599 | hci_conn_put(past->le); | ||
| 7517 | kfree(past); | 7600 | kfree(past); |
| 7518 | } | 7601 | } |
| 7519 | 7602 | ||
| @@ -7578,8 +7661,8 @@ int hci_past_sync(struct hci_conn *conn, struct hci_conn *le) · 58 unmodified lines | |||
| 7578 | if (!data) | 7661 | if (!data) |
| 7579 | return -ENOMEM; | 7662 | return -ENOMEM; |
| 7580 | 7663 | ||
| 7581 | data->conn = conn; | 7664 | data->conn = hci_conn_get(conn); |
| 7582 | data->le = le; | 7665 | data->le = hci_conn_get(le); |
| 7583 | 7666 | ||
| 7584 | if (conn->role == HCI_ROLE_MASTER) | 7667 | if (conn->role == HCI_ROLE_MASTER) |
| 7585 | err = hci_cmd_sync_queue_once(conn->hdev, | 7668 | err = hci_cmd_sync_queue_once(conn->hdev, |
| @@ -7589,8 +7672,11 @@ int hci_past_sync(struct hci_conn *conn, struct hci_conn *le) · 3 unmodified lines | |||
| 7589 | err = hci_cmd_sync_queue_once(conn->hdev, hci_le_past_sync, | 7672 | err = hci_cmd_sync_queue_once(conn->hdev, hci_le_past_sync, |
| 7590 | data, past_complete); | 7673 | data, past_complete); |
| 7591 | 7674 | ||
| 7592 | if (err) | 7675 | if (err) { |
| 7676 | hci_conn_put(data->conn); | ||
| 7677 | hci_conn_put(data->le); | ||
| 7593 | kfree(data); | 7678 | kfree(data); |
| 7679 | } | ||
| 7594 | 7680 | ||
| 7595 | return (err == -EEXIST) ? 0 : err; | 7681 | return (err == -EEXIST) ? 0 : err; |
| 7596 | } | 7682 | } |
| @@ -543,9 +543,10 @@ static int hidp_process_data(struct hidp_session *session, struct sk_buff *skb, · 542 unmodified lines | |||
| 543 | } | 543 | } |
| 544 | 544 | ||
| 545 | if (test_bit(HIDP_WAITING_FOR_RETURN, &session->flags) && | 545 | if (test_bit(HIDP_WAITING_FOR_RETURN, &session->flags) && |
| 546 | param == session->waiting_report_type) { | 546 | param == session->waiting_report_type) { |
| 547 | if (session->waiting_report_number < 0 || | 547 | if (session->waiting_report_number < 0 || |
| 548 | session->waiting_report_number == skb->data[0]) { | 548 | (skb->len && |
| 549 | session->waiting_report_number == skb->data[0])) { | ||
| 549 | /* hidp_get_raw_report() is waiting on this report. */ | 550 | /* hidp_get_raw_report() is waiting on this report. */ |
| 550 | session->report_return = skb; | 551 | session->report_return = skb; |
| 551 | done_with_skb = 0; | 552 | done_with_skb = 0; |
| @@ -560,16 +561,18 @@ static int hidp_process_data(struct hidp_session *session, struct sk_buff *skb, · 8 unmodified lines | |||
| 560 | static void hidp_recv_ctrl_frame(struct hidp_session *session, | 561 | static void hidp_recv_ctrl_frame(struct hidp_session *session, |
| 561 | struct sk_buff *skb) | 562 | struct sk_buff *skb) |
| 562 | { | 563 | { |
| 563 | unsigned char hdr, type, param; | 564 | unsigned char type, param; |
| 565 | u8 *hdr; | ||
| 564 | int free_skb = 1; | 566 | int free_skb = 1; |
| 565 | 567 | ||
| 566 | BT_DBG("session %p skb %p len %u", session, skb, skb->len); | 568 | BT_DBG("session %p skb %p len %u", session, skb, skb->len); |
| 567 | 569 | ||
| 568 | hdr = skb->data[0]; | 570 | hdr = skb_pull_data(skb, 1); |
| 569 | skb_pull(skb, 1); | 571 | if (!hdr) |
| 572 | goto free; | ||
| 570 | 573 | ||
| 571 | type = hdr & HIDP_HEADER_TRANS_MASK; | 574 | type = *hdr & HIDP_HEADER_TRANS_MASK; |
| 572 | param = hdr & HIDP_HEADER_PARAM_MASK; | 575 | param = *hdr & HIDP_HEADER_PARAM_MASK; |
| 573 | 576 | ||
| 574 | switch (type) { | 577 | switch (type) { |
| 575 | case HIDP_TRANS_HANDSHAKE: | 578 | case HIDP_TRANS_HANDSHAKE: |
| @@ -590,6 +593,7 @@ static void hidp_recv_ctrl_frame(struct hidp_session *session, · 14 unmodified lines | |||
| 590 | break; | 593 | break; |
| 591 | } | 594 | } |
| 592 | 595 | ||
| 596 | free: | ||
| 593 | if (free_skb) | 597 | if (free_skb) |
| 594 | kfree_skb(skb); | 598 | kfree_skb(skb); |
| 595 | } | 599 | } |
| @@ -597,14 +601,15 @@ static void hidp_recv_ctrl_frame(struct hidp_session *session, · 1 unmodified line | |||
| 597 | static void hidp_recv_intr_frame(struct hidp_session *session, | 601 | static void hidp_recv_intr_frame(struct hidp_session *session, |
| 598 | struct sk_buff *skb) | 602 | struct sk_buff *skb) |
| 599 | { | 603 | { |
| 600 | unsigned char hdr; | 604 | u8 *hdr; |
| 601 | 605 | ||
| 602 | BT_DBG("session %p skb %p len %u", session, skb, skb->len); | 606 | BT_DBG("session %p skb %p len %u", session, skb, skb->len); |
| 603 | 607 | ||
| 604 | hdr = skb->data[0]; | 608 | hdr = skb_pull_data(skb, 1); |
| 605 | skb_pull(skb, 1); | 609 | if (!hdr) |
| 610 | goto free; | ||
| 606 | 611 | ||
| 607 | if (hdr == (HIDP_TRANS_DATA | HIDP_DATA_RTYPE_INPUT)) { | 612 | if (*hdr == (HIDP_TRANS_DATA | HIDP_DATA_RTYPE_INPUT)) { |
| 608 | hidp_set_timer(session); | 613 | hidp_set_timer(session); |
| 609 | 614 | ||
| 610 | if (session->input) | 615 | if (session->input) |
| @@ -616,9 +621,10 @@ static void hidp_recv_intr_frame(struct hidp_session *session, · 5 unmodified lines | |||
| 616 | BT_DBG("report len %d", skb->len); | 621 | BT_DBG("report len %d", skb->len); |
| 617 | } | 622 | } |
| 618 | } else { | 623 | } else { |
| 619 | BT_DBG("Unsupported protocol header 0x%02x", hdr); | 624 | BT_DBG("Unsupported protocol header 0x%02x", *hdr); |
| 620 | } | 625 | } |
| 621 | 626 | ||
| 627 | free: | ||
| 622 | kfree_skb(skb); | 628 | kfree_skb(skb); |
| 623 | } | 629 | } |
| 624 | 630 | ||
| @@ -24,15 +24,19 @@ static struct bt_sock_list iso_sk_list = { · 23 unmodified lines | |||
| 24 | }; | 24 | }; |
| 25 | 25 | ||
| 26 | /* ---- ISO connections ---- */ | 26 | /* ---- ISO connections ---- */ |
| 27 | enum { | ||
| 28 | ISO_CONN_DROPPED, | ||
| 29 | __ISO_CONN_NUM_FLAGS | ||
| 30 | }; | ||
| 31 | |||
| 27 | struct iso_conn { | 32 | struct iso_conn { |
| 28 | struct hci_conn *hcon; | 33 | struct hci_conn *hcon; |
| 34 | DECLARE_BITMAP(flags, __ISO_CONN_NUM_FLAGS); | ||
| 29 | 35 | ||
| 30 | /* @lock: spinlock protecting changes to iso_conn fields */ | 36 | /* @lock: spinlock protecting changes to iso_conn fields */ |
| 31 | spinlock_t lock; | 37 | spinlock_t lock; |
| 32 | struct sock *sk; | 38 | struct sock *sk; |
| 33 | 39 | ||
| 34 | struct delayed_work timeout_work; | ||
| 35 | |||
| 36 | struct sk_buff *rx_skb; | 40 | struct sk_buff *rx_skb; |
| 37 | __u32 rx_len; | 41 | __u32 rx_len; |
| 38 | __u16 tx_sn; | 42 | __u16 tx_sn; |
| @@ -56,6 +60,7 @@ static void iso_sock_kill(struct sock *sk); · 17 unmodified lines | |||
| 56 | enum { | 60 | enum { |
| 57 | BT_SK_BIG_SYNC, | 61 | BT_SK_BIG_SYNC, |
| 58 | BT_SK_PA_SYNC, | 62 | BT_SK_PA_SYNC, |
| 63 | BT_SK_KILLED, | ||
| 59 | }; | 64 | }; |
| 60 | 65 | ||
| 61 | struct iso_pinfo { | 66 | struct iso_pinfo { |
| @@ -74,6 +79,7 @@ struct iso_pinfo { · 12 unmodified lines | |||
| 74 | __u8 base_len; | 79 | __u8 base_len; |
| 75 | __u8 base[BASE_MAX_LENGTH]; | 80 | __u8 base[BASE_MAX_LENGTH]; |
| 76 | struct iso_conn *conn; | 81 | struct iso_conn *conn; |
| 82 | struct delayed_work timeout_work; | ||
| 77 | }; | 83 | }; |
| 78 | 84 | ||
| 79 | static struct bt_iso_qos default_qos; | 85 | static struct bt_iso_qos default_qos; |
| @@ -102,16 +108,18 @@ static void iso_conn_free(struct kref *ref) · 22 unmodified lines | |||
| 102 | 108 | ||
| 103 | BT_DBG("conn %p", conn); | 109 | BT_DBG("conn %p", conn); |
| 104 | 110 | ||
| 105 | if (conn->sk) | ||
| 106 | iso_pi(conn->sk)->conn = NULL; | ||
| 107 | |||
| 108 | if (conn->hcon) { | 111 | if (conn->hcon) { |
| 109 | conn->hcon->iso_data = NULL; | 112 | spin_lock(&conn->hcon->proto_lock); |
| 110 | hci_conn_drop(conn->hcon); | ||
| 111 | } | ||
| 112 | 113 | ||
| 113 | /* Ensure no more work items will run since hci_conn has been dropped */ | 114 | /* Check we are not racing with iso_conn_add */ |
| 114 | disable_delayed_work_sync(&conn->timeout_work); | 115 | if (conn->hcon->iso_data == conn) { |
| 116 | conn->hcon->iso_data = NULL; | ||
| 117 | if (!test_and_set_bit(ISO_CONN_DROPPED, conn->flags)) | ||
| 118 | hci_conn_drop(conn->hcon); | ||
| 119 | } | ||
| 120 | |||
| 121 | spin_unlock(&conn->hcon->proto_lock); | ||
| 122 | } | ||
| 115 | 123 | ||
| 116 | kfree_skb(conn->rx_skb); | 124 | kfree_skb(conn->rx_skb); |
| 117 | 125 | ||
| @@ -125,7 +133,21 @@ static void iso_conn_put(struct iso_conn *conn) · 7 unmodified lines | |||
| 125 | 133 | ||
| 126 | BT_DBG("conn %p refcnt %d", conn, kref_read(&conn->ref)); | 134 | BT_DBG("conn %p refcnt %d", conn, kref_read(&conn->ref)); |
| 127 | 135 | ||
| 136 | /* The following race vs. iso_conn_del() is possible: | ||
| 137 | * | ||
| 138 | * 1. conn->hcon != NULL here | ||
| 139 | * 2. kref_put puts the last reference | ||
| 140 | * 3. concurrent iso_conn_del() gets iso_conn_hold_unless_zero() -> NULL | ||
| 141 | * and returns immediately, so conn->hcon is not cleared | ||
| 142 | * 4. iso_conn_free() dereferences conn->hcon | ||
| 143 | * | ||
| 144 | * To avoid UAF in step 4, take RCU before decrementing the refcount. | ||
| 145 | */ | ||
| 146 | rcu_read_lock(); | ||
| 147 | |||
| 128 | kref_put(&conn->ref, iso_conn_free); | 148 | kref_put(&conn->ref, iso_conn_free); |
| 149 | |||
| 150 | rcu_read_unlock(); | ||
| 129 | } | 151 | } |
| 130 | 152 | ||
| 131 | static struct iso_conn *iso_conn_hold_unless_zero(struct iso_conn *conn) | 153 | static struct iso_conn *iso_conn_hold_unless_zero(struct iso_conn *conn) |
| @@ -141,6 +163,14 @@ static struct iso_conn *iso_conn_hold_unless_zero(struct iso_conn *conn) · 9 unmodified lines | |||
| 141 | return conn; | 163 | return conn; |
| 142 | } | 164 | } |
| 143 | 165 | ||
| 166 | static struct iso_conn *iso_conn_hold(struct iso_conn *conn) | ||
| 167 | { | ||
| 168 | BT_DBG("conn %p refcnt %u", conn, kref_read(&conn->ref)); | ||
| 169 | |||
| 170 | kref_get(&conn->ref); | ||
| 171 | return conn; | ||
| 172 | } | ||
| 173 | |||
| 144 | static struct sock *iso_sock_hold(struct iso_conn *conn) | 174 | static struct sock *iso_sock_hold(struct iso_conn *conn) |
| 145 | { | 175 | { |
| 146 | if (!conn || !bt_sock_linked(&iso_sk_list, conn->sk)) | 176 | if (!conn || !bt_sock_linked(&iso_sk_list, conn->sk)) |
| @@ -153,78 +183,81 @@ static struct sock *iso_sock_hold(struct iso_conn *conn) · 6 unmodified lines | |||
| 153 | 183 | ||
| 154 | static void iso_sock_timeout(struct work_struct *work) | 184 | static void iso_sock_timeout(struct work_struct *work) |
| 155 | { | 185 | { |
| 156 | struct iso_conn *conn = container_of(work, struct iso_conn, | 186 | struct iso_pinfo *pi = container_of(work, struct iso_pinfo, |
| 157 | timeout_work.work); | 187 | timeout_work.work); |
| 158 | struct sock *sk; | 188 | struct sock *sk = &pi->bt.sk; |
| 159 | |||
| 160 | conn = iso_conn_hold_unless_zero(conn); | ||
| 161 | if (!conn) | ||
| 162 | return; | ||
| 163 | |||
| 164 | iso_conn_lock(conn); | ||
| 165 | sk = iso_sock_hold(conn); | ||
| 166 | iso_conn_unlock(conn); | ||
| 167 | iso_conn_put(conn); | ||
| 168 | |||
| 169 | if (!sk) | ||
| 170 | return; | ||
| 171 | 189 | ||
| 172 | BT_DBG("sock %p state %d", sk, sk->sk_state); | 190 | BT_DBG("sock %p state %d", sk, sk->sk_state); |
| 173 | 191 | ||
| 174 | lock_sock(sk); | 192 | lock_sock(sk); |
| 175 | sk->sk_err = ETIMEDOUT; | 193 | if (!sock_flag(sk, SOCK_ZAPPED)) { |
| 176 | sk->sk_state_change(sk); | 194 | sk->sk_err = ETIMEDOUT; |
| 195 | sk->sk_state_change(sk); | ||
| 196 | } | ||
| 177 | release_sock(sk); | 197 | release_sock(sk); |
| 178 | sock_put(sk); | ||
| 179 | } | 198 | } |
| 180 | 199 | ||
| 181 | static void iso_sock_set_timer(struct sock *sk, long timeout) | 200 | static void iso_sock_set_timer(struct sock *sk, long timeout) |
| 182 | { | 201 | { |
| 202 | lockdep_assert(lockdep_sock_is_held(sk)); | ||
| 203 | |||
| 204 | cancel_delayed_work(&iso_pi(sk)->timeout_work); | ||
| 205 | |||
| 183 | if (!iso_pi(sk)->conn) | 206 | if (!iso_pi(sk)->conn) |
| 184 | return; | 207 | return; |
| 185 | 208 | ||
| 186 | BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout); | 209 | BT_DBG("sock %p state %d timeout %ld", sk, sk->sk_state, timeout); |
| 187 | cancel_delayed_work(&iso_pi(sk)->conn->timeout_work); | 210 | schedule_delayed_work(&iso_pi(sk)->timeout_work, timeout); |
| 188 | schedule_delayed_work(&iso_pi(sk)->conn->timeout_work, timeout); | ||
| 189 | } | 211 | } |
| 190 | 212 | ||
| 191 | static void iso_sock_clear_timer(struct sock *sk) | 213 | static void iso_sock_clear_timer(struct sock *sk) |
| 192 | { | 214 | { |
| 193 | if (!iso_pi(sk)->conn) | 215 | BT_DBG("sock %p state %d", sk, sk->sk_state); |
| 194 | return; | 216 | cancel_delayed_work(&iso_pi(sk)->timeout_work); |
| 217 | } | ||
| 218 | |||
| 219 | static void iso_sock_disable_timer(struct sock *sk) | ||
| 220 | { | ||
| 221 | lockdep_assert(!lockdep_sock_is_held(sk)); | ||
| 195 | 222 | ||
| 196 | BT_DBG("sock %p state %d", sk, sk->sk_state); | 223 | BT_DBG("sock %p state %d", sk, sk->sk_state); |
| 197 | cancel_delayed_work(&iso_pi(sk)->conn->timeout_work); | 224 | disable_delayed_work_sync(&iso_pi(sk)->timeout_work); |
| 198 | } | 225 | } |
| 199 | 226 | ||
| 200 | /* ---- ISO connections ---- */ | 227 | /* ---- ISO connections ---- */ |
| 201 | static struct iso_conn *iso_conn_add(struct hci_conn *hcon) | 228 | static struct iso_conn *iso_conn_add(struct hci_conn *hcon) |
| 229 | __must_hold(&hcon->hdev->lock) | ||
| 202 | { | 230 | { |
| 203 | struct iso_conn *conn = hcon->iso_data; | 231 | struct iso_conn *conn; |
| 204 | 232 | ||
| 205 | conn = iso_conn_hold_unless_zero(conn); | 233 | spin_lock(&hcon->proto_lock); |
| 234 | |||
| 235 | conn = iso_conn_hold_unless_zero(hcon->iso_data); | ||
| 206 | if (conn) { | 236 | if (conn) { |
| 207 | if (!conn->hcon) { | 237 | if (!conn->hcon) { |
| 208 | iso_conn_lock(conn); | 238 | iso_conn_lock(conn); |
| 209 | conn->hcon = hcon; | 239 | conn->hcon = hcon; |
| 210 | iso_conn_unlock(conn); | 240 | iso_conn_unlock(conn); |
| 211 | } | 241 | } |
| 212 | iso_conn_put(conn); | 242 | spin_unlock(&hcon->proto_lock); |
| 213 | return conn; | 243 | return conn; |
| 214 | } | 244 | } |
| 215 | 245 | ||
| 216 | conn = kzalloc_obj(*conn); | 246 | conn = kzalloc_obj(*conn, GFP_ATOMIC); |
| 217 | if (!conn) | 247 | if (!conn) { |
| 248 | spin_unlock(&hcon->proto_lock); | ||
| 218 | return NULL; | 249 | return NULL; |
| 250 | } | ||
| 219 | 251 | ||
| 220 | kref_init(&conn->ref); | 252 | kref_init(&conn->ref); |
| 221 | spin_lock_init(&conn->lock); | 253 | spin_lock_init(&conn->lock); |
| 222 | INIT_DELAYED_WORK(&conn->timeout_work, iso_sock_timeout); | ||
| 223 | 254 | ||
| 224 | hcon->iso_data = conn; | 255 | hcon->iso_data = conn; |
| 225 | conn->hcon = hcon; | 256 | conn->hcon = hcon; |
| 226 | conn->tx_sn = 0; | 257 | conn->tx_sn = 0; |
| 227 | 258 | ||
| 259 | spin_unlock(&hcon->proto_lock); | ||
| 260 | |||
| 228 | BT_DBG("hcon %p conn %p", hcon, conn); | 261 | BT_DBG("hcon %p conn %p", hcon, conn); |
| 229 | 262 | ||
| 230 | return conn; | 263 | return conn; |
| @@ -263,11 +296,14 @@ static void iso_chan_del(struct sock *sk, int err) · 32 unmodified lines | |||
| 263 | } | 296 | } |
| 264 | 297 | ||
| 265 | static void iso_conn_del(struct hci_conn *hcon, int err) | 298 | static void iso_conn_del(struct hci_conn *hcon, int err) |
| 299 | __must_hold(&hcon->hdev->lock) | ||
| 266 | { | 300 | { |
| 267 | struct iso_conn *conn = hcon->iso_data; | 301 | struct iso_conn *conn; |
| 268 | struct sock *sk; | 302 | struct sock *sk; |
| 269 | 303 | ||
| 270 | conn = iso_conn_hold_unless_zero(conn); | 304 | spin_lock(&hcon->proto_lock); |
| 305 | conn = iso_conn_hold_unless_zero(hcon->iso_data); | ||
| 306 | spin_unlock(&hcon->proto_lock); | ||
| 271 | if (!conn) | 307 | if (!conn) |
| 272 | return; | 308 | return; |
| 273 | 309 | ||
| @@ -277,18 +313,28 @@ static void iso_conn_del(struct hci_conn *hcon, int err) · 3 unmodified lines | |||
| 277 | iso_conn_lock(conn); | 313 | iso_conn_lock(conn); |
| 278 | sk = iso_sock_hold(conn); | 314 | sk = iso_sock_hold(conn); |
| 279 | iso_conn_unlock(conn); | 315 | iso_conn_unlock(conn); |
| 280 | iso_conn_put(conn); | ||
| 281 | 316 | ||
| 282 | if (!sk) { | 317 | if (!sk) |
| 283 | iso_conn_put(conn); | 318 | goto done; |
| 284 | return; | 319 | |
| 285 | } | 320 | iso_sock_disable_timer(sk); |
| 286 | 321 | ||
| 287 | lock_sock(sk); | 322 | lock_sock(sk); |
| 288 | iso_sock_clear_timer(sk); | ||
| 289 | iso_chan_del(sk, err); | 323 | iso_chan_del(sk, err); |
| 290 | release_sock(sk); | 324 | release_sock(sk); |
| 325 | iso_sock_kill(sk); | ||
| 291 | sock_put(sk); | 326 | sock_put(sk); |
| 327 | |||
| 328 | done: | ||
| 329 | /* No sk access to conn->hcon any more (lock_sock + hdev->lock) */ | ||
| 330 | spin_lock(&hcon->proto_lock); | ||
| 331 | iso_conn_lock(conn); | ||
| 332 | conn->hcon = NULL; | ||
| 333 | hcon->iso_data = NULL; | ||
| 334 | iso_conn_unlock(conn); | ||
| 335 | spin_unlock(&hcon->proto_lock); | ||
| 336 | |||
| 337 | iso_conn_put(conn); | ||
| 292 | } | 338 | } |
| 293 | 339 | ||
| 294 | static int __iso_chan_add(struct iso_conn *conn, struct sock *sk, | 340 | static int __iso_chan_add(struct iso_conn *conn, struct sock *sk, |
| @@ -304,8 +350,14 @@ static int __iso_chan_add(struct iso_conn *conn, struct sock *sk, · 9 unmodified lines | |||
| 304 | return -EBUSY; | 350 | return -EBUSY; |
| 305 | } | 351 | } |
| 306 | 352 | ||
| 307 | iso_pi(sk)->conn = conn; | 353 | if (!conn->hcon) { |
| 354 | BT_ERR("conn->hcon missing"); | ||
| 355 | return -EIO; | ||
| 356 | } | ||
| 357 | |||
| 358 | iso_pi(sk)->conn = iso_conn_hold(conn); | ||
| 308 | conn->sk = sk; | 359 | conn->sk = sk; |
| 360 | clear_bit(ISO_CONN_DROPPED, conn->flags); | ||
| 309 | 361 | ||
| 310 | if (parent) | 362 | if (parent) |
| 311 | bt_accept_enqueue(parent, sk, true); | 363 | bt_accept_enqueue(parent, sk, true); |
| @@ -402,6 +454,8 @@ static int iso_connect_bis(struct sock *sk) · 90 unmodified lines | |||
| 402 | iso_pi(sk)->bc_sid = hcon->sid; | 454 | iso_pi(sk)->bc_sid = hcon->sid; |
| 403 | } | 455 | } |
| 404 | 456 | ||
| 457 | lockdep_assert_held(&hcon->hdev->lock); | ||
| 458 | |||
| 405 | conn = iso_conn_add(hcon); | 459 | conn = iso_conn_add(hcon); |
| 406 | if (!conn) { | 460 | if (!conn) { |
| 407 | hci_conn_drop(hcon); | 461 | hci_conn_drop(hcon); |
| @@ -410,6 +464,7 @@ static int iso_connect_bis(struct sock *sk) · 2 unmodified lines | |||
| 410 | } | 464 | } |
| 411 | 465 | ||
| 412 | err = iso_chan_add(conn, sk, NULL); | 466 | err = iso_chan_add(conn, sk, NULL); |
| 467 | iso_conn_put(conn); | ||
| 413 | if (err) | 468 | if (err) |
| 414 | goto unlock; | 469 | goto unlock; |
| 415 | 470 | ||
| @@ -504,6 +559,8 @@ static int iso_connect_cis(struct sock *sk) · 88 unmodified lines | |||
| 504 | } | 559 | } |
| 505 | } | 560 | } |
| 506 | 561 | ||
| 562 | lockdep_assert_held(&hcon->hdev->lock); | ||
| 563 | |||
| 507 | conn = iso_conn_add(hcon); | 564 | conn = iso_conn_add(hcon); |
| 508 | if (!conn) { | 565 | if (!conn) { |
| 509 | hci_conn_drop(hcon); | 566 | hci_conn_drop(hcon); |
| @@ -512,6 +569,7 @@ static int iso_connect_cis(struct sock *sk) · 2 unmodified lines | |||
| 512 | } | 569 | } |
| 513 | 570 | ||
| 514 | err = iso_chan_add(conn, sk, NULL); | 571 | err = iso_chan_add(conn, sk, NULL); |
| 572 | iso_conn_put(conn); | ||
| 515 | if (err) | 573 | if (err) |
| 516 | goto unlock; | 574 | goto unlock; |
| 517 | 575 | ||
| @@ -790,9 +848,15 @@ static void iso_sock_cleanup_listen(struct sock *parent) · 272 unmodified lines | |||
| 790 | */ | 848 | */ |
| 791 | static void iso_sock_kill(struct sock *sk) | 849 | static void iso_sock_kill(struct sock *sk) |
| 792 | { | 850 | { |
| 851 | iso_sock_disable_timer(sk); | ||
| 852 | |||
| 853 | lock_sock(sk); | ||
| 854 | |||
| 793 | if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket || | 855 | if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket || |
| 794 | sock_flag(sk, SOCK_DEAD)) | 856 | test_bit(BT_SK_KILLED, &iso_pi(sk)->flags)) { |
| 857 | release_sock(sk); | ||
| 795 | return; | 858 | return; |
| 859 | } | ||
| 796 | 860 | ||
| 797 | BT_DBG("sk %p state %d", sk, sk->sk_state); | 861 | BT_DBG("sk %p state %d", sk, sk->sk_state); |
| 798 | 862 | ||
| @@ -806,6 +870,9 @@ static void iso_sock_kill(struct sock *sk) · 7 unmodified lines | |||
| 806 | /* Kill poor orphan */ | 870 | /* Kill poor orphan */ |
| 807 | bt_sock_unlink(&iso_sk_list, sk); | 871 | bt_sock_unlink(&iso_sk_list, sk); |
| 808 | sock_set_flag(sk, SOCK_DEAD); | 872 | sock_set_flag(sk, SOCK_DEAD); |
| 873 | set_bit(BT_SK_KILLED, &iso_pi(sk)->flags); | ||
| 874 | |||
| 875 | release_sock(sk); | ||
| 809 | sock_put(sk); | 876 | sock_put(sk); |
| 810 | } | 877 | } |
| 811 | 878 | ||
| @@ -825,8 +892,8 @@ static void iso_sock_disconn(struct sock *sk) · 13 unmodified lines | |||
| 825 | */ | 892 | */ |
| 826 | if (bis_sk) { | 893 | if (bis_sk) { |
| 827 | hcon->state = BT_OPEN; | 894 | hcon->state = BT_OPEN; |
| 828 | hcon->iso_data = NULL; | 895 | set_bit(ISO_CONN_DROPPED, iso_pi(sk)->conn->flags); |
| 829 | iso_pi(sk)->conn->hcon = NULL; | 896 | |
| 830 | iso_sock_clear_timer(sk); | 897 | iso_sock_clear_timer(sk); |
| 831 | iso_chan_del(sk, bt_to_errno(hcon->abort_reason)); | 898 | iso_chan_del(sk, bt_to_errno(hcon->abort_reason)); |
| 832 | sock_put(bis_sk); | 899 | sock_put(bis_sk); |
| @@ -835,10 +902,8 @@ static void iso_sock_disconn(struct sock *sk) · 2 unmodified lines | |||
| 835 | } | 902 | } |
| 836 | 903 | ||
| 837 | sk->sk_state = BT_DISCONN; | 904 | sk->sk_state = BT_DISCONN; |
| 838 | iso_conn_lock(iso_pi(sk)->conn); | 905 | if (!test_and_set_bit(ISO_CONN_DROPPED, iso_pi(sk)->conn->flags)) |
| 839 | hci_conn_drop(iso_pi(sk)->conn->hcon); | 906 | hci_conn_drop(iso_pi(sk)->conn->hcon); |
| 840 | iso_pi(sk)->conn->hcon = NULL; | ||
| 841 | iso_conn_unlock(iso_pi(sk)->conn); | ||
| 842 | } | 907 | } |
| 843 | 908 | ||
| 844 | static void __iso_sock_close(struct sock *sk) | 909 | static void __iso_sock_close(struct sock *sk) |
| @@ -880,11 +945,11 @@ static void __iso_sock_close(struct sock *sk) · 35 unmodified lines | |||
| 880 | /* Must be called on unlocked socket. */ | 945 | /* Must be called on unlocked socket. */ |
| 881 | static void iso_sock_close(struct sock *sk) | 946 | static void iso_sock_close(struct sock *sk) |
| 882 | { | 947 | { |
| 948 | iso_sock_disable_timer(sk); | ||
| 949 | |||
| 883 | lock_sock(sk); | 950 | lock_sock(sk); |
| 884 | iso_sock_clear_timer(sk); | ||
| 885 | __iso_sock_close(sk); | 951 | __iso_sock_close(sk); |
| 886 | release_sock(sk); | 952 | release_sock(sk); |
| 887 | iso_sock_kill(sk); | ||
| 888 | } | 953 | } |
| 889 | 954 | ||
| 890 | static void iso_sock_init(struct sock *sk, struct sock *parent) | 955 | static void iso_sock_init(struct sock *sk, struct sock *parent) |
| @@ -951,6 +1016,8 @@ static struct sock *iso_sock_alloc(struct net *net, struct socket *sock, · 60 unmodified lines | |||
| 951 | iso_pi(sk)->qos = default_qos; | 1016 | iso_pi(sk)->qos = default_qos; |
| 952 | iso_pi(sk)->sync_handle = -1; | 1017 | iso_pi(sk)->sync_handle = -1; |
| 953 | 1018 | ||
| 1019 | INIT_DELAYED_WORK(&iso_pi(sk)->timeout_work, iso_sock_timeout); | ||
| 1020 | |||
| 954 | bt_sock_link(&iso_sk_list, sk); | 1021 | bt_sock_link(&iso_sk_list, sk); |
| 955 | return sk; | 1022 | return sk; |
| 956 | } | 1023 | } |
| @@ -1033,15 +1100,15 @@ static int iso_sock_rebind_bis(struct sock *sk, struct sockaddr_iso *sa, · 76 unmodified lines | |||
| 1033 | goto done; | 1100 | goto done; |
| 1034 | } | 1101 | } |
| 1035 | 1102 | ||
| 1036 | iso_pi(sk)->bc_num_bis = sa->iso_bc->bc_num_bis; | 1103 | for (int i = 0; i < sa->iso_bc->bc_num_bis; i++) |
| 1037 | |||
| 1038 | for (int i = 0; i < iso_pi(sk)->bc_num_bis; i++) | ||
| 1039 | if (sa->iso_bc->bc_bis[i] < 0x01 || | 1104 | if (sa->iso_bc->bc_bis[i] < 0x01 || |
| 1040 | sa->iso_bc->bc_bis[i] > 0x1f) { | 1105 | sa->iso_bc->bc_bis[i] > 0x1f) { |
| 1041 | err = -EINVAL; | 1106 | err = -EINVAL; |
| 1042 | goto done; | 1107 | goto done; |
| 1043 | } | 1108 | } |
| 1044 | 1109 | ||
| 1110 | iso_pi(sk)->bc_num_bis = sa->iso_bc->bc_num_bis; | ||
| 1111 | |||
| 1045 | memcpy(iso_pi(sk)->bc_bis, sa->iso_bc->bc_bis, | 1112 | memcpy(iso_pi(sk)->bc_bis, sa->iso_bc->bc_bis, |
| 1046 | iso_pi(sk)->bc_num_bis); | 1113 | iso_pi(sk)->bc_num_bis); |
| 1047 | 1114 | ||
| @@ -1276,6 +1343,8 @@ static int iso_listen_bis(struct sock *sk) · 228 unmodified lines | |||
| 1276 | goto unlock; | 1343 | goto unlock; |
| 1277 | } | 1344 | } |
| 1278 | 1345 | ||
| 1346 | lockdep_assert_held(&hcon->hdev->lock); | ||
| 1347 | |||
| 1279 | conn = iso_conn_add(hcon); | 1348 | conn = iso_conn_add(hcon); |
| 1280 | if (!conn) { | 1349 | if (!conn) { |
| 1281 | hci_conn_drop(hcon); | 1350 | hci_conn_drop(hcon); |
| @@ -1284,10 +1353,9 @@ static int iso_listen_bis(struct sock *sk) · 2 unmodified lines | |||
| 1284 | } | 1353 | } |
| 1285 | 1354 | ||
| 1286 | err = iso_chan_add(conn, sk, NULL); | 1355 | err = iso_chan_add(conn, sk, NULL); |
| 1287 | if (err) { | 1356 | iso_conn_put(conn); |
| 1288 | hci_conn_drop(hcon); | 1357 | if (err) |
| 1289 | goto unlock; | 1358 | goto unlock; |
| 1290 | } | ||
| 1291 | 1359 | ||
| 1292 | unlock: | 1360 | unlock: |
| 1293 | release_sock(sk); | 1361 | release_sock(sk); |
| @@ -1466,6 +1534,8 @@ static int iso_sock_getname(struct socket *sock, struct sockaddr *addr, · 172 unmodified lines | |||
| 1466 | 1534 | ||
| 1467 | BT_DBG("sock %p, sk %p", sock, sk); | 1535 | BT_DBG("sock %p, sk %p", sock, sk); |
| 1468 | 1536 | ||
| 1537 | lock_sock(sk); | ||
| 1538 | |||
| 1469 | addr->sa_family = AF_BLUETOOTH; | 1539 | addr->sa_family = AF_BLUETOOTH; |
| 1470 | 1540 | ||
| 1471 | if (peer) { | 1541 | if (peer) { |
| @@ -1487,6 +1557,8 @@ static int iso_sock_getname(struct socket *sock, struct sockaddr *addr, · 15 unmodified lines | |||
| 1487 | sa->iso_bdaddr_type = iso_pi(sk)->src_type; | 1557 | sa->iso_bdaddr_type = iso_pi(sk)->src_type; |
| 1488 | } | 1558 | } |
| 1489 | 1559 | ||
| 1560 | release_sock(sk); | ||
| 1561 | |||
| 1490 | return len; | 1562 | return len; |
| 1491 | } | 1563 | } |
| 1492 | 1564 | ||
| @@ -1786,7 +1858,7 @@ static bool check_bcast_qos(struct bt_iso_qos *qos) · 293 unmodified lines | |||
| 1786 | return false; | 1858 | return false; |
| 1787 | 1859 | ||
| 1788 | if (!qos->bcast.timeout) | 1860 | if (!qos->bcast.timeout) |
| 1789 | qos->bcast.sync_timeout = BT_ISO_SYNC_TIMEOUT; | 1861 | qos->bcast.timeout = BT_ISO_SYNC_TIMEOUT; |
| 1790 | 1862 | ||
| 1791 | if (qos->bcast.timeout < 0x000a || qos->bcast.timeout > 0x4000) | 1863 | if (qos->bcast.timeout < 0x000a || qos->bcast.timeout > 0x4000) |
| 1792 | return false; | 1864 | return false; |
| @@ -2028,8 +2100,16 @@ static int iso_sock_release(struct socket *sock) · 235 unmodified lines | |||
| 2028 | release_sock(sk); | 2100 | release_sock(sk); |
| 2029 | } | 2101 | } |
| 2030 | 2102 | ||
| 2103 | /* Make sure sk is valid even if iso_conn_del() is concurrent */ | ||
| 2104 | sock_hold(sk); | ||
| 2105 | |||
| 2106 | lock_sock(sk); | ||
| 2031 | sock_orphan(sk); | 2107 | sock_orphan(sk); |
| 2108 | release_sock(sk); | ||
| 2109 | |||
| 2032 | iso_sock_kill(sk); | 2110 | iso_sock_kill(sk); |
| 2111 | |||
| 2112 | sock_put(sk); | ||
| 2033 | return err; | 2113 | return err; |
| 2034 | } | 2114 | } |
| 2035 | 2115 | ||
| @@ -2037,14 +2117,17 @@ static void iso_sock_ready(struct sock *sk) · 1 unmodified line | |||
| 2037 | { | 2117 | { |
| 2038 | BT_DBG("sk %p", sk); | 2118 | BT_DBG("sk %p", sk); |
| 2039 | 2119 | ||
| 2040 | if (!sk) | 2120 | lockdep_assert(lockdep_sock_is_held(sk)); |
| 2121 | |||
| 2122 | switch (sk->sk_state) { | ||
| 2123 | case BT_DISCONN: | ||
| 2124 | case BT_CLOSED: | ||
| 2041 | return; | 2125 | return; |
| 2126 | } | ||
| 2042 | 2127 | ||
| 2043 | lock_sock(sk); | ||
| 2044 | iso_sock_clear_timer(sk); | 2128 | iso_sock_clear_timer(sk); |
| 2045 | sk->sk_state = BT_CONNECTED; | 2129 | sk->sk_state = BT_CONNECTED; |
| 2046 | sk->sk_state_change(sk); | 2130 | sk->sk_state_change(sk); |
| 2047 | release_sock(sk); | ||
| 2048 | } | 2131 | } |
| 2049 | 2132 | ||
| 2050 | static bool iso_match_big(struct sock *sk, void *data) | 2133 | static bool iso_match_big(struct sock *sk, void *data) |
| @@ -2074,7 +2157,7 @@ static bool iso_match_dst(struct sock *sk, void *data) · 23 unmodified lines | |||
| 2074 | static void iso_conn_ready(struct iso_conn *conn) | 2157 | static void iso_conn_ready(struct iso_conn *conn) |
| 2075 | { | 2158 | { |
| 2076 | struct sock *parent = NULL; | 2159 | struct sock *parent = NULL; |
| 2077 | struct sock *sk = conn->sk; | 2160 | struct sock *sk; |
| 2078 | struct hci_ev_le_big_sync_established *ev = NULL; | 2161 | struct hci_ev_le_big_sync_established *ev = NULL; |
| 2079 | struct hci_ev_le_pa_sync_established *ev2 = NULL; | 2162 | struct hci_ev_le_pa_sync_established *ev2 = NULL; |
| 2080 | struct hci_ev_le_per_adv_report *ev3 = NULL; | 2163 | struct hci_ev_le_per_adv_report *ev3 = NULL; |
| @@ -2083,7 +2166,22 @@ static void iso_conn_ready(struct iso_conn *conn) · 2 unmodified lines | |||
| 2083 | 2166 | ||
| 2084 | BT_DBG("conn %p", conn); | 2167 | BT_DBG("conn %p", conn); |
| 2085 | 2168 | ||
| 2169 | iso_conn_lock(conn); | ||
| 2170 | sk = iso_sock_hold(conn); | ||
| 2171 | iso_conn_unlock(conn); | ||
| 2172 | |||
| 2086 | if (sk) { | 2173 | if (sk) { |
| 2174 | lock_sock(sk); | ||
| 2175 | |||
| 2176 | /* conn->sk may have become NULL if racing with sk close, but | ||
| 2177 | * due to held hdev->lock, it can't become different sk. | ||
| 2178 | */ | ||
| 2179 | if (!conn->sk) { | ||
| 2180 | release_sock(sk); | ||
| 2181 | sock_put(sk); | ||
| 2182 | return; | ||
| 2183 | } | ||
| 2184 | |||
| 2087 | /* Attempt to update source address in case of BIS Sender if | 2185 | /* Attempt to update source address in case of BIS Sender if |
| 2088 | * the advertisement is using a random address. | 2186 | * the advertisement is using a random address. |
| 2089 | */ | 2187 | */ |
| @@ -2096,14 +2194,15 @@ static void iso_conn_ready(struct iso_conn *conn) · 6 unmodified lines | |||
| 2096 | adv = hci_find_adv_instance(bis->hdev, | 2194 | adv = hci_find_adv_instance(bis->hdev, |
| 2097 | bis->iso_qos.bcast.bis); | 2195 | bis->iso_qos.bcast.bis); |
| 2098 | if (adv && bacmp(&adv->random_addr, BDADDR_ANY)) { | 2196 | if (adv && bacmp(&adv->random_addr, BDADDR_ANY)) { |
| 2099 | lock_sock(sk); | ||
| 2100 | iso_pi(sk)->src_type = BDADDR_LE_RANDOM; | 2197 | iso_pi(sk)->src_type = BDADDR_LE_RANDOM; |
| 2101 | bacpy(&iso_pi(sk)->src, &adv->random_addr); | 2198 | bacpy(&iso_pi(sk)->src, &adv->random_addr); |
| 2102 | release_sock(sk); | ||
| 2103 | } | 2199 | } |
| 2104 | } | 2200 | } |
| 2105 | 2201 | ||
| 2106 | iso_sock_ready(conn->sk); | 2202 | iso_sock_ready(sk); |
| 2203 | |||
| 2204 | release_sock(sk); | ||
| 2205 | sock_put(sk); | ||
| 2107 | } else { | 2206 | } else { |
| 2108 | hcon = conn->hcon; | 2207 | hcon = conn->hcon; |
| 2109 | if (!hcon) | 2208 | if (!hcon) |
| @@ -2350,7 +2449,7 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 240 unmodified lines | |||
| 2350 | 2449 | ||
| 2351 | lock_sock(sk); | 2450 | lock_sock(sk); |
| 2352 | 2451 | ||
| 2353 | hcon = iso_pi(sk)->conn->hcon; | 2452 | hcon = iso_pi(sk)->conn ? iso_pi(sk)->conn->hcon : NULL; |
| 2354 | iso_pi(sk)->qos.bcast.encryption = ev2->encryption; | 2453 | iso_pi(sk)->qos.bcast.encryption = ev2->encryption; |
| 2355 | 2454 | ||
| 2356 | if (ev2->num_bis < iso_pi(sk)->bc_num_bis) | 2455 | if (ev2->num_bis < iso_pi(sk)->bc_num_bis) |
| @@ -2390,9 +2489,11 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 33 unmodified lines | |||
| 2390 | if (!sk) | 2489 | if (!sk) |
| 2391 | goto done; | 2490 | goto done; |
| 2392 | 2491 | ||
| 2393 | hcon = iso_pi(sk)->conn->hcon; | 2492 | lock_sock(sk); |
| 2493 | |||
| 2494 | hcon = iso_pi(sk)->conn ? iso_pi(sk)->conn->hcon : NULL; | ||
| 2394 | if (!hcon) | 2495 | if (!hcon) |
| 2395 | goto done; | 2496 | goto release3; |
| 2396 | 2497 | ||
| 2397 | if (ev3->data_status == LE_PA_DATA_TRUNCATED) { | 2498 | if (ev3->data_status == LE_PA_DATA_TRUNCATED) { |
| 2398 | /* The controller was unable to retrieve PA data. */ | 2499 | /* The controller was unable to retrieve PA data. */ |
| @@ -2400,12 +2501,12 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 1 unmodified line | |||
| 2400 | HCI_MAX_PER_AD_TOT_LEN); | 2501 | HCI_MAX_PER_AD_TOT_LEN); |
| 2401 | hcon->le_per_adv_data_len = 0; | 2502 | hcon->le_per_adv_data_len = 0; |
| 2402 | hcon->le_per_adv_data_offset = 0; | 2503 | hcon->le_per_adv_data_offset = 0; |
| 2403 | goto done; | 2504 | goto release3; |
| 2404 | } | 2505 | } |
| 2405 | 2506 | ||
| 2406 | if (hcon->le_per_adv_data_offset + ev3->length > | 2507 | if (hcon->le_per_adv_data_offset + ev3->length > |
| 2407 | HCI_MAX_PER_AD_TOT_LEN) | 2508 | HCI_MAX_PER_AD_TOT_LEN) |
| 2408 | goto done; | 2509 | goto release3; |
| 2409 | 2510 | ||
| 2410 | memcpy(hcon->le_per_adv_data + hcon->le_per_adv_data_offset, | 2511 | memcpy(hcon->le_per_adv_data + hcon->le_per_adv_data_offset, |
| 2411 | ev3->data, ev3->length); | 2512 | ev3->data, ev3->length); |
| @@ -2424,18 +2525,19 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags) · 12 unmodified lines | |||
| 2424 | &base_len); | 2525 | &base_len); |
| 2425 | 2526 | ||
| 2426 | if (!base || base_len > BASE_MAX_LENGTH) | 2527 | if (!base || base_len > BASE_MAX_LENGTH) |
| 2427 | goto done; | 2528 | goto release3; |
| 2428 | 2529 | ||
| 2429 | lock_sock(sk); | ||
| 2430 | memcpy(iso_pi(sk)->base, base, base_len); | 2530 | memcpy(iso_pi(sk)->base, base, base_len); |
| 2431 | iso_pi(sk)->base_len = base_len; | 2531 | iso_pi(sk)->base_len = base_len; |
| 2432 | release_sock(sk); | ||
| 2433 | } else { | 2532 | } else { |
| 2434 | /* This is a PA data fragment. Keep pa_data_len set to 0 | 2533 | /* This is a PA data fragment. Keep pa_data_len set to 0 |
| 2435 | * until all data has been reassembled. | 2534 | * until all data has been reassembled. |
| 2436 | */ | 2535 | */ |
| 2437 | hcon->le_per_adv_data_len = 0; | 2536 | hcon->le_per_adv_data_len = 0; |
| 2438 | } | 2537 | } |
| 2538 | |||
| 2539 | release3: | ||
| 2540 | release_sock(sk); | ||
| 2439 | } else { | 2541 | } else { |
| 2440 | sk = iso_get_sock(hdev, &hdev->bdaddr, BDADDR_ANY, | 2542 | sk = iso_get_sock(hdev, &hdev->bdaddr, BDADDR_ANY, |
| 2441 | BT_LISTEN, iso_match_dst, BDADDR_ANY); | 2543 | BT_LISTEN, iso_match_dst, BDADDR_ANY); |
| @@ -2454,6 +2556,7 @@ done: · 12 unmodified lines | |||
| 2454 | } | 2556 | } |
| 2455 | 2557 | ||
| 2456 | static void iso_connect_cfm(struct hci_conn *hcon, __u8 status) | 2558 | static void iso_connect_cfm(struct hci_conn *hcon, __u8 status) |
| 2559 | __must_hold(&hcon->hdev->lock) | ||
| 2457 | { | 2560 | { |
| 2458 | if (hcon->type != CIS_LINK && hcon->type != BIS_LINK && | 2561 | if (hcon->type != CIS_LINK && hcon->type != BIS_LINK && |
| 2459 | hcon->type != PA_LINK) { | 2562 | hcon->type != PA_LINK) { |
| @@ -2465,8 +2568,10 @@ static void iso_connect_cfm(struct hci_conn *hcon, __u8 status) · 5 unmodified lines | |||
| 2465 | struct hci_link *link, *t; | 2568 | struct hci_link *link, *t; |
| 2466 | 2569 | ||
| 2467 | list_for_each_entry_safe(link, t, &hcon->link_list, | 2570 | list_for_each_entry_safe(link, t, &hcon->link_list, |
| 2468 | list) | 2571 | list) { |
| 2572 | lockdep_assert_held(&link->conn->hdev->lock); | ||
| 2469 | iso_conn_del(link->conn, bt_to_errno(status)); | 2573 | iso_conn_del(link->conn, bt_to_errno(status)); |
| 2574 | } | ||
| 2470 | 2575 | ||
| 2471 | return; | 2576 | return; |
| 2472 | } | 2577 | } |
| @@ -2488,14 +2593,17 @@ static void iso_connect_cfm(struct hci_conn *hcon, __u8 status) · 15 unmodified lines | |||
| 2488 | struct iso_conn *conn; | 2593 | struct iso_conn *conn; |
| 2489 | 2594 | ||
| 2490 | conn = iso_conn_add(hcon); | 2595 | conn = iso_conn_add(hcon); |
| 2491 | if (conn) | 2596 | if (conn) { |
| 2492 | iso_conn_ready(conn); | 2597 | iso_conn_ready(conn); |
| 2598 | iso_conn_put(conn); | ||
| 2599 | } | ||
| 2493 | } else { | 2600 | } else { |
| 2494 | iso_conn_del(hcon, bt_to_errno(status)); | 2601 | iso_conn_del(hcon, bt_to_errno(status)); |
| 2495 | } | 2602 | } |
| 2496 | } | 2603 | } |
| 2497 | 2604 | ||
| 2498 | static void iso_disconn_cfm(struct hci_conn *hcon, __u8 reason) | 2605 | static void iso_disconn_cfm(struct hci_conn *hcon, __u8 reason) |
| 2606 | __must_hold(&hcon->hdev->lock) | ||
| 2499 | { | 2607 | { |
| 2500 | if (hcon->type != CIS_LINK && hcon->type != BIS_LINK && | 2608 | if (hcon->type != CIS_LINK && hcon->type != BIS_LINK && |
| 2501 | hcon->type != PA_LINK) | 2609 | hcon->type != PA_LINK) |
| @@ -2522,7 +2630,10 @@ int iso_recv(struct hci_dev *hdev, u16 handle, struct sk_buff *skb, u16 flags) · 20 unmodified lines | |||
| 2522 | return -ENOENT; | 2630 | return -ENOENT; |
| 2523 | } | 2631 | } |
| 2524 | 2632 | ||
| 2633 | spin_lock(&hcon->proto_lock); | ||
| 2525 | conn = iso_conn_hold_unless_zero(hcon->iso_data); | 2634 | conn = iso_conn_hold_unless_zero(hcon->iso_data); |
| 2635 | spin_unlock(&hcon->proto_lock); | ||
| 2636 | |||
| 2526 | hcon = NULL; | 2637 | hcon = NULL; |
| 2527 | 2638 | ||
| 2528 | hci_dev_unlock(hdev); | 2639 | hci_dev_unlock(hdev); |
| @@ -4820,6 +4820,10 @@ static int l2cap_le_connect_rsp(struct l2cap_conn *conn, · 4819 unmodified lines | |||
| 4820 | if (!chan) | 4820 | if (!chan) |
| 4821 | return -EBADSLT; | 4821 | return -EBADSLT; |
| 4822 | 4822 | ||
| 4823 | chan = l2cap_chan_hold_unless_zero(chan); | ||
| 4824 | if (!chan) | ||
| 4825 | return -EBADSLT; | ||
| 4826 | |||
| 4823 | err = 0; | 4827 | err = 0; |
| 4824 | 4828 | ||
| 4825 | l2cap_chan_lock(chan); | 4829 | l2cap_chan_lock(chan); |
| @@ -4865,6 +4869,7 @@ static int l2cap_le_connect_rsp(struct l2cap_conn *conn, · 39 unmodified lines | |||
| 4865 | } | 4869 | } |
| 4866 | 4870 | ||
| 4867 | l2cap_chan_unlock(chan); | 4871 | l2cap_chan_unlock(chan); |
| 4872 | l2cap_chan_put(chan); | ||
| 4868 | 4873 | ||
| 4869 | return err; | 4874 | return err; |
| 4870 | } | 4875 | } |
| @@ -2696,18 +2696,28 @@ static int mgmt_hci_cmd_sync(struct sock *sk, struct hci_dev *hdev, · 2695 unmodified lines | |||
| 2696 | static bool pending_eir_or_class(struct hci_dev *hdev) | 2696 | static bool pending_eir_or_class(struct hci_dev *hdev) |
| 2697 | { | 2697 | { |
| 2698 | struct mgmt_pending_cmd *cmd; | 2698 | struct mgmt_pending_cmd *cmd; |
| 2699 | bool pending = false; | ||
| 2700 | |||
| 2701 | mutex_lock(&hdev->mgmt_pending_lock); | ||
| 2699 | 2702 | ||
| 2700 | list_for_each_entry(cmd, &hdev->mgmt_pending, list) { | 2703 | list_for_each_entry(cmd, &hdev->mgmt_pending, list) { |
| 2701 | switch (cmd->opcode) { | 2704 | switch (cmd->opcode) { |
| 2702 | case MGMT_OP_ADD_UUID: | 2705 | case MGMT_OP_ADD_UUID: |
| 2703 | case MGMT_OP_REMOVE_UUID: | 2706 | case MGMT_OP_REMOVE_UUID: |
| 2704 | case MGMT_OP_SET_DEV_CLASS: | 2707 | case MGMT_OP_SET_DEV_CLASS: |
| 2708 | case MGMT_OP_SET_LOCAL_NAME: | ||
| 2705 | case MGMT_OP_SET_POWERED: | 2709 | case MGMT_OP_SET_POWERED: |
| 2706 | return true; | 2710 | pending = true; |
| 2711 | break; | ||
| 2707 | } | 2712 | } |
| 2713 | |||
| 2714 | if (pending) | ||
| 2715 | break; | ||
| 2708 | } | 2716 | } |
| 2709 | 2717 | ||
| 2710 | return false; | 2718 | mutex_unlock(&hdev->mgmt_pending_lock); |
| 2719 | |||
| 2720 | return pending; | ||
| 2711 | } | 2721 | } |
| 2712 | 2722 | ||
| 2713 | static const u8 bluetooth_base_uuid[] = { | 2723 | static const u8 bluetooth_base_uuid[] = { |
| @@ -3514,11 +3524,13 @@ static int set_io_capability(struct sock *sk, struct hci_dev *hdev, void *data, · 800 unmodified lines | |||
| 3514 | NULL, 0); | 3524 | NULL, 0); |
| 3515 | } | 3525 | } |
| 3516 | 3526 | ||
| 3517 | static struct mgmt_pending_cmd *find_pairing(struct hci_conn *conn) | 3527 | static struct mgmt_pending_cmd *remove_pairing(struct hci_conn *conn) |
| 3518 | { | 3528 | { |
| 3519 | struct hci_dev *hdev = conn->hdev; | 3529 | struct hci_dev *hdev = conn->hdev; |
| 3520 | struct mgmt_pending_cmd *cmd; | 3530 | struct mgmt_pending_cmd *cmd; |
| 3521 | 3531 | ||
| 3532 | mutex_lock(&hdev->mgmt_pending_lock); | ||
| 3533 | |||
| 3522 | list_for_each_entry(cmd, &hdev->mgmt_pending, list) { | 3534 | list_for_each_entry(cmd, &hdev->mgmt_pending, list) { |
| 3523 | if (cmd->opcode != MGMT_OP_PAIR_DEVICE) | 3535 | if (cmd->opcode != MGMT_OP_PAIR_DEVICE) |
| 3524 | continue; | 3536 | continue; |
| @@ -3526,9 +3538,39 @@ static struct mgmt_pending_cmd *find_pairing(struct hci_conn *conn) · 1 unmodified line | |||
| 3526 | if (cmd->user_data != conn) | 3538 | if (cmd->user_data != conn) |
| 3527 | continue; | 3539 | continue; |
| 3528 | 3540 | ||
| 3541 | list_del(&cmd->list); | ||
| 3542 | mutex_unlock(&hdev->mgmt_pending_lock); | ||
| 3529 | return cmd; | 3543 | return cmd; |
| 3530 | } | 3544 | } |
| 3531 | 3545 | ||
| 3546 | mutex_unlock(&hdev->mgmt_pending_lock); | ||
| 3547 | |||
| 3548 | return NULL; | ||
| 3549 | } | ||
| 3550 | |||
| 3551 | static struct mgmt_pending_cmd *remove_pairing_by_addr(struct hci_dev *hdev, | ||
| 3552 | bdaddr_t *bdaddr) | ||
| 3553 | { | ||
| 3554 | struct mgmt_pending_cmd *cmd; | ||
| 3555 | struct hci_conn *conn; | ||
| 3556 | |||
| 3557 | mutex_lock(&hdev->mgmt_pending_lock); | ||
| 3558 | |||
| 3559 | list_for_each_entry(cmd, &hdev->mgmt_pending, list) { | ||
| 3560 | if (cmd->opcode != MGMT_OP_PAIR_DEVICE) | ||
| 3561 | continue; | ||
| 3562 | |||
| 3563 | conn = cmd->user_data; | ||
| 3564 | if (bacmp(bdaddr, &conn->dst) != 0) | ||
| 3565 | continue; | ||
| 3566 | |||
| 3567 | list_del(&cmd->list); | ||
| 3568 | mutex_unlock(&hdev->mgmt_pending_lock); | ||
| 3569 | return cmd; | ||
| 3570 | } | ||
| 3571 | |||
| 3572 | mutex_unlock(&hdev->mgmt_pending_lock); | ||
| 3573 | |||
| 3532 | return NULL; | 3574 | return NULL; |
| 3533 | } | 3575 | } |
| 3534 | 3576 | ||
| @@ -3566,10 +3608,10 @@ void mgmt_smp_complete(struct hci_conn *conn, bool complete) · 31 unmodified lines | |||
| 3566 | u8 status = complete ? MGMT_STATUS_SUCCESS : MGMT_STATUS_FAILED; | 3608 | u8 status = complete ? MGMT_STATUS_SUCCESS : MGMT_STATUS_FAILED; |
| 3567 | struct mgmt_pending_cmd *cmd; | 3609 | struct mgmt_pending_cmd *cmd; |
| 3568 | 3610 | ||
| 3569 | cmd = find_pairing(conn); | 3611 | cmd = remove_pairing(conn); |
| 3570 | if (cmd) { | 3612 | if (cmd) { |
| 3571 | cmd->cmd_complete(cmd, status); | 3613 | cmd->cmd_complete(cmd, status); |
| 3572 | mgmt_pending_remove(cmd); | 3614 | mgmt_pending_free(cmd); |
| 3573 | } | 3615 | } |
| 3574 | } | 3616 | } |
| 3575 | 3617 | ||
| @@ -3579,14 +3621,14 @@ static void pairing_complete_cb(struct hci_conn *conn, u8 status) · 3 unmodified lines | |||
| 3579 | 3621 | ||
| 3580 | BT_DBG("status %u", status); | 3622 | BT_DBG("status %u", status); |
| 3581 | 3623 | ||
| 3582 | cmd = find_pairing(conn); | 3624 | cmd = remove_pairing(conn); |
| 3583 | if (!cmd) { | 3625 | if (!cmd) { |
| 3584 | BT_DBG("Unable to find a pending command"); | 3626 | BT_DBG("Unable to find a pending command"); |
| 3585 | return; | 3627 | return; |
| 3586 | } | 3628 | } |
| 3587 | 3629 | ||
| 3588 | cmd->cmd_complete(cmd, mgmt_status(status)); | 3630 | cmd->cmd_complete(cmd, mgmt_status(status)); |
| 3589 | mgmt_pending_remove(cmd); | 3631 | mgmt_pending_free(cmd); |
| 3590 | } | 3632 | } |
| 3591 | 3633 | ||
| 3592 | static void le_pairing_complete_cb(struct hci_conn *conn, u8 status) | 3634 | static void le_pairing_complete_cb(struct hci_conn *conn, u8 status) |
| @@ -3598,14 +3640,14 @@ static void le_pairing_complete_cb(struct hci_conn *conn, u8 status) · 5 unmodified lines | |||
| 3598 | if (!status) | 3640 | if (!status) |
| 3599 | return; | 3641 | return; |
| 3600 | 3642 | ||
| 3601 | cmd = find_pairing(conn); | 3643 | cmd = remove_pairing(conn); |
| 3602 | if (!cmd) { | 3644 | if (!cmd) { |
| 3603 | BT_DBG("Unable to find a pending command"); | 3645 | BT_DBG("Unable to find a pending command"); |
| 3604 | return; | 3646 | return; |
| 3605 | } | 3647 | } |
| 3606 | 3648 | ||
| 3607 | cmd->cmd_complete(cmd, mgmt_status(status)); | 3649 | cmd->cmd_complete(cmd, mgmt_status(status)); |
| 3608 | mgmt_pending_remove(cmd); | 3650 | mgmt_pending_free(cmd); |
| 3609 | } | 3651 | } |
| 3610 | 3652 | ||
| 3611 | static int pair_device(struct sock *sk, struct hci_dev *hdev, void *data, | 3653 | static int pair_device(struct sock *sk, struct hci_dev *hdev, void *data, |
| @@ -3762,23 +3804,17 @@ static int cancel_pair_device(struct sock *sk, struct hci_dev *hdev, void *data, · 150 unmodified lines | |||
| 3762 | goto unlock; | 3804 | goto unlock; |
| 3763 | } | 3805 | } |
| 3764 | 3806 | ||
| 3765 | cmd = pending_find(MGMT_OP_PAIR_DEVICE, hdev); | 3807 | cmd = remove_pairing_by_addr(hdev, &addr->bdaddr); |
| 3766 | if (!cmd) { | 3808 | if (!cmd) { |
| 3767 | err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, | 3809 | err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, |
| 3768 | MGMT_STATUS_INVALID_PARAMS); | 3810 | MGMT_STATUS_INVALID_PARAMS); |
| 3769 | goto unlock; | 3811 | goto unlock; |
| 3770 | } | 3812 | } |
| 3771 | 3813 | ||
| 3772 | conn = cmd->user_data; | 3814 | conn = hci_conn_get(cmd->user_data); |
| 3773 | |||
| 3774 | if (bacmp(&addr->bdaddr, &conn->dst) != 0) { | ||
| 3775 | err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, | ||
| 3776 | MGMT_STATUS_INVALID_PARAMS); | ||
| 3777 | goto unlock; | ||
| 3778 | } | ||
| 3779 | 3815 | ||
| 3780 | cmd->cmd_complete(cmd, MGMT_STATUS_CANCELLED); | 3816 | cmd->cmd_complete(cmd, MGMT_STATUS_CANCELLED); |
| 3781 | mgmt_pending_remove(cmd); | 3817 | mgmt_pending_free(cmd); |
| 3782 | 3818 | ||
| 3783 | err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, 0, | 3819 | err = mgmt_cmd_complete(sk, hdev->id, MGMT_OP_CANCEL_PAIR_DEVICE, 0, |
| 3784 | addr, sizeof(*addr)); | 3820 | addr, sizeof(*addr)); |
| @@ -3796,6 +3832,8 @@ static int cancel_pair_device(struct sock *sk, struct hci_dev *hdev, void *data, · 11 unmodified lines | |||
| 3796 | if (conn->conn_reason == CONN_REASON_PAIR_DEVICE) | 3832 | if (conn->conn_reason == CONN_REASON_PAIR_DEVICE) |
| 3797 | hci_abort_conn(conn, HCI_ERROR_REMOTE_USER_TERM); | 3833 | hci_abort_conn(conn, HCI_ERROR_REMOTE_USER_TERM); |
| 3798 | 3834 | ||
| 3835 | hci_conn_put(conn); | ||
| 3836 | |||
| 3799 | unlock: | 3837 | unlock: |
| 3800 | hci_dev_unlock(hdev); | 3838 | hci_dev_unlock(hdev); |
| 3801 | return err; | 3839 | return err; |
| @@ -4043,6 +4081,12 @@ static int set_local_name(struct sock *sk, struct hci_dev *hdev, void *data, · 241 unmodified lines | |||
| 4043 | goto failed; | 4081 | goto failed; |
| 4044 | } | 4082 | } |
| 4045 | 4083 | ||
| 4084 | if (hdev_is_powered(hdev) && pending_eir_or_class(hdev)) { | ||
| 4085 | err = mgmt_cmd_status(sk, hdev->id, MGMT_OP_SET_LOCAL_NAME, | ||
| 4086 | MGMT_STATUS_BUSY); | ||
| 4087 | goto failed; | ||
| 4088 | } | ||
| 4089 | |||
| 4046 | memcpy(hdev->short_name, cp->short_name, sizeof(hdev->short_name)); | 4090 | memcpy(hdev->short_name, cp->short_name, sizeof(hdev->short_name)); |
| 4047 | 4091 | ||
| 4048 | if (!hdev_is_powered(hdev)) { | 4092 | if (!hdev_is_powered(hdev)) { |
| @@ -10137,14 +10181,14 @@ void mgmt_auth_failed(struct hci_conn *conn, u8 hci_status) · 6088 unmodified lines | |||
| 10137 | ev.addr.type = link_to_bdaddr(conn->type, conn->dst_type); | 10181 | ev.addr.type = link_to_bdaddr(conn->type, conn->dst_type); |
| 10138 | ev.status = status; | 10182 | ev.status = status; |
| 10139 | 10183 | ||
| 10140 | cmd = find_pairing(conn); | 10184 | cmd = remove_pairing(conn); |
| 10141 | 10185 | ||
| 10142 | mgmt_event(MGMT_EV_AUTH_FAILED, conn->hdev, &ev, sizeof(ev), | 10186 | mgmt_event(MGMT_EV_AUTH_FAILED, conn->hdev, &ev, sizeof(ev), |
| 10143 | cmd ? cmd->sk : NULL); | 10187 | cmd ? cmd->sk : NULL); |
| 10144 | 10188 | ||
| 10145 | if (cmd) { | 10189 | if (cmd) { |
| 10146 | cmd->cmd_complete(cmd, status); | 10190 | cmd->cmd_complete(cmd, status); |
| 10147 | mgmt_pending_remove(cmd); | 10191 | mgmt_pending_free(cmd); |
| 10148 | } | 10192 | } |
| 10149 | } | 10193 | } |
| 10150 | 10194 | ||
| @@ -1795,6 +1795,11 @@ static struct rfcomm_session *rfcomm_recv_frame(struct rfcomm_session *s, · 1794 unmodified lines | |||
| 1795 | return s; | 1795 | return s; |
| 1796 | } | 1796 | } |
| 1797 | 1797 | ||
| 1798 | if (skb->len < sizeof(*hdr) + 1) { | ||
| 1799 | kfree_skb(skb); | ||
| 1800 | return s; | ||
| 1801 | } | ||
| 1802 | |||
| 1798 | dlci = __get_dlci(hdr->addr); | 1803 | dlci = __get_dlci(hdr->addr); |
| 1799 | type = __get_type(hdr->ctrl); | 1804 | type = __get_type(hdr->ctrl); |
| 1800 | 1805 | ||
| @@ -188,6 +188,9 @@ static void sco_sock_clear_timer(struct sock *sk) · 187 unmodified lines | |||
| 188 | } | 188 | } |
| 189 | 189 | ||
| 190 | /* ---- SCO connections ---- */ | 190 | /* ---- SCO connections ---- */ |
| 191 | /* Consumes a reference on @hcon, which the returned sco_conn owns until it is | ||
| 192 | * freed. On failure (NULL return) the reference is left for the caller to drop. | ||
| 193 | */ | ||
| 191 | static struct sco_conn *sco_conn_add(struct hci_conn *hcon) | 194 | static struct sco_conn *sco_conn_add(struct hci_conn *hcon) |
| 192 | { | 195 | { |
| 193 | struct sco_conn *conn = hcon->sco_data; | 196 | struct sco_conn *conn = hcon->sco_data; |
| @@ -198,6 +201,9 @@ static struct sco_conn *sco_conn_add(struct hci_conn *hcon) · 4 unmodified lines | |||
| 198 | sco_conn_lock(conn); | 201 | sco_conn_lock(conn); |
| 199 | conn->hcon = hcon; | 202 | conn->hcon = hcon; |
| 200 | sco_conn_unlock(conn); | 203 | sco_conn_unlock(conn); |
| 204 | } else { | ||
| 205 | /* conn already owns a reference on hcon */ | ||
| 206 | hci_conn_drop(hcon); | ||
| 201 | } | 207 | } |
| 202 | return conn; | 208 | return conn; |
| 203 | } | 209 | } |
| @@ -265,10 +271,8 @@ static void sco_conn_del(struct hci_conn *hcon, int err) · 61 unmodified lines | |||
| 265 | sco_conn_unlock(conn); | 271 | sco_conn_unlock(conn); |
| 266 | sco_conn_put(conn); | 272 | sco_conn_put(conn); |
| 267 | 273 | ||
| 268 | if (!sk) { | 274 | if (!sk) |
| 269 | sco_conn_put(conn); | ||
| 270 | return; | 275 | return; |
| 271 | } | ||
| 272 | 276 | ||
| 273 | /* Kill socket */ | 277 | /* Kill socket */ |
| 274 | lock_sock(sk); | 278 | lock_sock(sk); |
| @@ -283,7 +287,7 @@ static void __sco_chan_add(struct sco_conn *conn, struct sock *sk, · 8 unmodified lines | |||
| 283 | { | 287 | { |
| 284 | BT_DBG("conn %p", conn); | 288 | BT_DBG("conn %p", conn); |
| 285 | 289 | ||
| 286 | sco_pi(sk)->conn = conn; | 290 | sco_pi(sk)->conn = sco_conn_hold(conn); |
| 287 | conn->sk = sk; | 291 | conn->sk = sk; |
| 288 | 292 | ||
| 289 | if (parent) | 293 | if (parent) |
| @@ -366,15 +370,15 @@ static int sco_connect(struct sock *sk) · 76 unmodified lines | |||
| 366 | */ | 370 | */ |
| 367 | if (sk->sk_state != BT_OPEN && sk->sk_state != BT_BOUND) { | 371 | if (sk->sk_state != BT_OPEN && sk->sk_state != BT_BOUND) { |
| 368 | release_sock(sk); | 372 | release_sock(sk); |
| 369 | hci_conn_drop(hcon); | 373 | sco_conn_put(conn); |
| 370 | err = -EBADFD; | 374 | err = -EBADFD; |
| 371 | goto unlock; | 375 | goto unlock; |
| 372 | } | 376 | } |
| 373 | 377 | ||
| 374 | err = sco_chan_add(conn, sk, NULL); | 378 | err = sco_chan_add(conn, sk, NULL); |
| 379 | sco_conn_put(conn); | ||
| 375 | if (err) { | 380 | if (err) { |
| 376 | release_sock(sk); | 381 | release_sock(sk); |
| 377 | hci_conn_drop(hcon); | ||
| 378 | goto unlock; | 382 | goto unlock; |
| 379 | } | 383 | } |
| 380 | 384 | ||
| @@ -1452,8 +1456,6 @@ static void sco_conn_ready(struct sco_conn *conn) · 1071 unmodified lines | |||
| 1452 | bacpy(&sco_pi(sk)->src, &conn->hcon->src); | 1456 | bacpy(&sco_pi(sk)->src, &conn->hcon->src); |
| 1453 | bacpy(&sco_pi(sk)->dst, &conn->hcon->dst); | 1457 | bacpy(&sco_pi(sk)->dst, &conn->hcon->dst); |
| 1454 | 1458 | ||
| 1455 | sco_conn_hold(conn); | ||
| 1456 | hci_conn_hold(conn->hcon); | ||
| 1457 | __sco_chan_add(conn, sk, parent); | 1459 | __sco_chan_add(conn, sk, parent); |
| 1458 | 1460 | ||
| 1459 | if (test_bit(BT_SK_DEFER_SETUP, &bt_sk(parent)->flags)) | 1461 | if (test_bit(BT_SK_DEFER_SETUP, &bt_sk(parent)->flags)) |
| @@ -1509,10 +1511,12 @@ static void sco_connect_cfm(struct hci_conn *hcon, __u8 status) · 49 unmodified lines | |||
| 1509 | if (!status) { | 1511 | if (!status) { |
| 1510 | struct sco_conn *conn; | 1512 | struct sco_conn *conn; |
| 1511 | 1513 | ||
| 1512 | conn = sco_conn_add(hcon); | 1514 | conn = sco_conn_add(hci_conn_hold(hcon)); |
| 1513 | if (conn) { | 1515 | if (conn) { |
| 1514 | sco_conn_ready(conn); | 1516 | sco_conn_ready(conn); |
| 1515 | sco_conn_put(conn); | 1517 | sco_conn_put(conn); |
| 1518 | } else { | ||
| 1519 | hci_conn_drop(hcon); | ||
| 1516 | } | 1520 | } |
| 1517 | } else | 1521 | } else |
| 1518 | sco_conn_del(hcon, bt_to_errno(status)); | 1522 | sco_conn_del(hcon, bt_to_errno(status)); |
| @@ -215,7 +215,7 @@ static struct sk_buff *br_mrp_alloc_test_skb(struct br_mrp *mrp, · 214 unmodified lines | |||
| 215 | struct br_mrp_oui_hdr *oui = NULL; | 215 | struct br_mrp_oui_hdr *oui = NULL; |
| 216 | u8 length; | 216 | u8 length; |
| 217 | 217 | ||
| 218 | length = sizeof(*sub_opt) + sizeof(*sub_tlv) + sizeof(oui) + | 218 | length = sizeof(*sub_opt) + sizeof(*sub_tlv) + sizeof(*oui) + |
| 219 | MRP_OPT_PADDING; | 219 | MRP_OPT_PADDING; |
| 220 | br_mrp_skb_tlv(skb, BR_MRP_TLV_HEADER_OPTION, length); | 220 | br_mrp_skb_tlv(skb, BR_MRP_TLV_HEADER_OPTION, length); |
| 221 | 221 | ||
| @@ -3687,6 +3687,7 @@ br_multicast_leave_group(struct net_bridge_mcast *brmctx, · 3686 unmodified lines | |||
| 3687 | 3687 | ||
| 3688 | p->flags |= MDB_PG_FLAGS_FAST_LEAVE; | 3688 | p->flags |= MDB_PG_FLAGS_FAST_LEAVE; |
| 3689 | br_multicast_del_pg(mp, p, pp); | 3689 | br_multicast_del_pg(mp, p, pp); |
| 3690 | break; | ||
| 3690 | } | 3691 | } |
| 3691 | goto out; | 3692 | goto out; |
| 3692 | } | 3693 | } |
| @@ -127,6 +127,15 @@ MODULE_PARM_DESC(max_pdu_size, "maximum isotp pdu size (default " · 126 unmodified lines | |||
| 127 | #define ISOTP_FC_TIMEOUT 1 /* 1 sec */ | 127 | #define ISOTP_FC_TIMEOUT 1 /* 1 sec */ |
| 128 | #define ISOTP_ECHO_TIMEOUT 2 /* 2 secs */ | 128 | #define ISOTP_ECHO_TIMEOUT 2 /* 2 secs */ |
| 129 | 129 | ||
| 130 | /* so->tx_result[so->tx_gen % ISOTP_TX_RESULT_SLOTS] holds the packed value | ||
| 131 | * (err << ISOTP_TX_RESULT_GEN_BITS | gen) for each tx generation slot, so it | ||
| 132 | * can be handled with a single READ_ONCE()/WRITE_ONCE() access. | ||
| 133 | */ | ||
| 134 | #define ISOTP_TX_RESULT_SLOTS 4 | ||
| 135 | #define ISOTP_TX_RESULT_GEN_BITS 24 | ||
| 136 | #define ISOTP_TX_RESULT_GEN_MASK ((1U << ISOTP_TX_RESULT_GEN_BITS) - 1) | ||
| 137 | #define ISOTP_TX_RESULT_ERR_MASK 0xFF | ||
| 138 | |||
| 130 | enum { | 139 | enum { |
| 131 | ISOTP_IDLE = 0, | 140 | ISOTP_IDLE = 0, |
| 132 | ISOTP_WAIT_FIRST_FC, | 141 | ISOTP_WAIT_FIRST_FC, |
| @@ -166,7 +175,8 @@ struct isotp_sock { · 33 unmodified lines | |||
| 166 | u32 force_tx_stmin; | 175 | u32 force_tx_stmin; |
| 167 | u32 force_rx_stmin; | 176 | u32 force_rx_stmin; |
| 168 | u32 cfecho; /* consecutive frame echo tag */ | 177 | u32 cfecho; /* consecutive frame echo tag */ |
| 169 | u32 tx_gen; /* generation, bumped per new tx transfer */ | 178 | u32 tx_gen; /* transfer generation, increased per new tx transfer */ |
| 179 | u32 tx_result[ISOTP_TX_RESULT_SLOTS]; /* per-generation result slots */ | ||
| 170 | struct tpcon rx, tx; | 180 | struct tpcon rx, tx; |
| 171 | struct list_head notifier; | 181 | struct list_head notifier; |
| 172 | wait_queue_head_t wait; | 182 | wait_queue_head_t wait; |
| @@ -177,6 +187,65 @@ static LIST_HEAD(isotp_notifier_list); · 4 unmodified lines | |||
| 177 | static DEFINE_SPINLOCK(isotp_notifier_lock); | 187 | static DEFINE_SPINLOCK(isotp_notifier_lock); |
| 178 | static struct isotp_sock *isotp_busy_notifier; | 188 | static struct isotp_sock *isotp_busy_notifier; |
| 179 | 189 | ||
| 190 | /* increase (24 bit) tx generation value */ | ||
| 191 | static u32 isotp_inc_tx_gen(u32 gen) | ||
| 192 | { | ||
| 193 | return (gen + 1) & ISOTP_TX_RESULT_GEN_MASK; | ||
| 194 | } | ||
| 195 | |||
| 196 | /* store 8 bit error and 24 bit tx generation values in packed u32 element */ | ||
| 197 | static u32 isotp_pack_tx_result(u32 gen, int err) | ||
| 198 | { | ||
| 199 | return gen | ((u32)err << ISOTP_TX_RESULT_GEN_BITS); | ||
| 200 | } | ||
| 201 | |||
| 202 | /* get the 24 bit tx generation value from the tx result */ | ||
| 203 | static u32 isotp_get_tx_gen(u32 gen_err) | ||
| 204 | { | ||
| 205 | return gen_err & ISOTP_TX_RESULT_GEN_MASK; | ||
| 206 | } | ||
| 207 | |||
| 208 | /* get the 8 bit error value from the tx result */ | ||
| 209 | static u32 isotp_get_tx_err(u32 gen_err) | ||
| 210 | { | ||
| 211 | return (gen_err >> ISOTP_TX_RESULT_GEN_BITS) & ISOTP_TX_RESULT_ERR_MASK; | ||
| 212 | } | ||
| 213 | |||
| 214 | /* store transfer result in per-generation%4 so->tx_result[] slot */ | ||
| 215 | static void isotp_set_tx_result(struct isotp_sock *so, u32 gen, int err) | ||
| 216 | { | ||
| 217 | WRITE_ONCE(so->tx_result[gen % ISOTP_TX_RESULT_SLOTS], | ||
| 218 | isotp_pack_tx_result(gen, err)); | ||
| 219 | } | ||
| 220 | |||
| 221 | /* fetch the result recorded for 'gen', as a (negative) errno (0 for success) */ | ||
| 222 | static int isotp_get_tx_result(struct isotp_sock *so, u32 gen) | ||
| 223 | { | ||
| 224 | u32 result = READ_ONCE(so->tx_result[gen % ISOTP_TX_RESULT_SLOTS]); | ||
| 225 | |||
| 226 | if (isotp_get_tx_gen(result) != gen) { | ||
| 227 | pr_notice_once("can-isotp: tx_result[] slot reused before read\n"); | ||
| 228 | |||
| 229 | /* report failure rather than risk a false success */ | ||
| 230 | return -ECOMM; | ||
| 231 | } | ||
| 232 | |||
| 233 | return -(isotp_get_tx_err(result)); | ||
| 234 | } | ||
| 235 | |||
| 236 | /* true if done, shut down or superseded ('gen' is no longer the active | ||
| 237 | * transfer). Reads tx.state first (acquire) so tx_gen/tx_result reads | ||
| 238 | * below see at least what that state write published (common sequence). | ||
| 239 | */ | ||
| 240 | static bool isotp_tx_gen_done(struct isotp_sock *so, u32 gen) | ||
| 241 | { | ||
| 242 | /* read tx.state first for the common sequence */ | ||
| 243 | u32 state = smp_load_acquire(&so->tx.state); | ||
| 244 | |||
| 245 | return state == ISOTP_IDLE || state == ISOTP_SHUTDOWN || | ||
| 246 | READ_ONCE(so->tx_gen) != gen; | ||
| 247 | } | ||
| 248 | |||
| 180 | static inline struct isotp_sock *isotp_sk(const struct sock *sk) | 249 | static inline struct isotp_sock *isotp_sk(const struct sock *sk) |
| 181 | { | 250 | { |
| 182 | return (struct isotp_sock *)sk; | 251 | return (struct isotp_sock *)sk; |
| @@ -199,7 +268,7 @@ static enum hrtimer_restart isotp_rx_timer_handler(struct hrtimer *hrtimer) · 16 unmodified lines | |||
| 199 | rxtimer); | 268 | rxtimer); |
| 200 | struct sock *sk = &so->sk; | 269 | struct sock *sk = &so->sk; |
| 201 | 270 | ||
| 202 | if (so->rx.state == ISOTP_WAIT_DATA) { | 271 | if (READ_ONCE(so->rx.state) == ISOTP_WAIT_DATA) { |
| 203 | /* we did not get new data frames in time */ | 272 | /* we did not get new data frames in time */ |
| 204 | 273 | ||
| 205 | /* report 'connection timed out' */ | 274 | /* report 'connection timed out' */ |
| @@ -208,7 +277,7 @@ static enum hrtimer_restart isotp_rx_timer_handler(struct hrtimer *hrtimer) · 2 unmodified lines | |||
| 208 | sk_error_report(sk); | 277 | sk_error_report(sk); |
| 209 | 278 | ||
| 210 | /* reset rx state */ | 279 | /* reset rx state */ |
| 211 | so->rx.state = ISOTP_IDLE; | 280 | WRITE_ONCE(so->rx.state, ISOTP_IDLE); |
| 212 | } | 281 | } |
| 213 | 282 | ||
| 214 | return HRTIMER_NORESTART; | 283 | return HRTIMER_NORESTART; |
| @@ -372,20 +441,19 @@ static void isotp_send_cframe(struct isotp_sock *so); · 157 unmodified lines | |||
| 372 | static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) | 441 | static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) |
| 373 | { | 442 | { |
| 374 | struct sock *sk = &so->sk; | 443 | struct sock *sk = &so->sk; |
| 444 | int tx_err = EBADMSG; /* default for unknown FC status */ | ||
| 375 | 445 | ||
| 376 | if (so->tx.state != ISOTP_WAIT_FC && | 446 | if (READ_ONCE(so->tx.state) != ISOTP_WAIT_FC && |
| 377 | so->tx.state != ISOTP_WAIT_FIRST_FC) | 447 | READ_ONCE(so->tx.state) != ISOTP_WAIT_FIRST_FC) |
| 378 | return 0; | 448 | return 0; |
| 379 | 449 | ||
| 380 | hrtimer_cancel(&so->txtimer); | 450 | hrtimer_cancel(&so->txtimer); |
| 381 | 451 | ||
| 382 | /* isotp_tx_timeout() may have given up on this job while | 452 | /* isotp_tx_timeout() may have given up on this job while |
| 383 | * hrtimer_cancel() above waited for it to finish; so->rx_lock | 453 | * hrtimer_cancel() above waited for it to finish => recheck |
| 384 | * (held by our caller isotp_rcv()) rules out a concurrent claim, | ||
| 385 | * so a plain recheck is enough here. | ||
| 386 | */ | 454 | */ |
| 387 | if (so->tx.state != ISOTP_WAIT_FC && | 455 | if (READ_ONCE(so->tx.state) != ISOTP_WAIT_FC && |
| 388 | so->tx.state != ISOTP_WAIT_FIRST_FC) | 456 | READ_ONCE(so->tx.state) != ISOTP_WAIT_FIRST_FC) |
| 389 | return 1; | 457 | return 1; |
| 390 | 458 | ||
| 391 | if ((cf->len < ae + FC_CONTENT_SZ) || | 459 | if ((cf->len < ae + FC_CONTENT_SZ) || |
| @@ -396,13 +464,15 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) · 4 unmodified lines | |||
| 396 | if (!sock_flag(sk, SOCK_DEAD)) | 464 | if (!sock_flag(sk, SOCK_DEAD)) |
| 397 | sk_error_report(sk); | 465 | sk_error_report(sk); |
| 398 | 466 | ||
| 399 | so->tx.state = ISOTP_IDLE; | 467 | isotp_set_tx_result(so, so->tx_gen, EBADMSG); |
| 468 | /* set to IDLE after publishing tx_result */ | ||
| 469 | smp_store_release(&so->tx.state, ISOTP_IDLE); | ||
| 400 | wake_up_interruptible(&so->wait); | 470 | wake_up_interruptible(&so->wait); |
| 401 | return 1; | 471 | return 1; |
| 402 | } | 472 | } |
| 403 | 473 | ||
| 404 | /* get static/dynamic communication params from first/every FC frame */ | 474 | /* get static/dynamic communication params from first/every FC frame */ |
| 405 | if (so->tx.state == ISOTP_WAIT_FIRST_FC || | 475 | if (READ_ONCE(so->tx.state) == ISOTP_WAIT_FIRST_FC || |
| 406 | so->opt.flags & CAN_ISOTP_DYN_FC_PARMS) { | 476 | so->opt.flags & CAN_ISOTP_DYN_FC_PARMS) { |
| 407 | so->txfc.bs = cf->data[ae + 1]; | 477 | so->txfc.bs = cf->data[ae + 1]; |
| 408 | so->txfc.stmin = cf->data[ae + 2]; | 478 | so->txfc.stmin = cf->data[ae + 2]; |
| @@ -426,13 +496,13 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) · 17 unmodified lines | |||
| 426 | so->tx_gap = ktime_add_ns(so->tx_gap, | 496 | so->tx_gap = ktime_add_ns(so->tx_gap, |
| 427 | (so->txfc.stmin - 0xF0) | 497 | (so->txfc.stmin - 0xF0) |
| 428 | * 100000); | 498 | * 100000); |
| 429 | so->tx.state = ISOTP_WAIT_FC; | 499 | WRITE_ONCE(so->tx.state, ISOTP_WAIT_FC); |
| 430 | } | 500 | } |
| 431 | 501 | ||
| 432 | switch (cf->data[ae] & 0x0F) { | 502 | switch (cf->data[ae] & 0x0F) { |
| 433 | case ISOTP_FC_CTS: | 503 | case ISOTP_FC_CTS: |
| 434 | so->tx.bs = 0; | 504 | so->tx.bs = 0; |
| 435 | so->tx.state = ISOTP_SENDING; | 505 | WRITE_ONCE(so->tx.state, ISOTP_SENDING); |
| 436 | /* send CF frame and enable echo timeout handling */ | 506 | /* send CF frame and enable echo timeout handling */ |
| 437 | hrtimer_start(&so->echotimer, ktime_set(ISOTP_ECHO_TIMEOUT, 0), | 507 | hrtimer_start(&so->echotimer, ktime_set(ISOTP_ECHO_TIMEOUT, 0), |
| 438 | HRTIMER_MODE_REL_SOFT); | 508 | HRTIMER_MODE_REL_SOFT); |
| @@ -447,14 +517,19 @@ static int isotp_rcv_fc(struct isotp_sock *so, struct canfd_frame *cf, int ae) · 8 unmodified lines | |||
| 447 | 517 | ||
| 448 | case ISOTP_FC_OVFLW: | 518 | case ISOTP_FC_OVFLW: |
| 449 | /* overflow on receiver side - report 'message too long' */ | 519 | /* overflow on receiver side - report 'message too long' */ |
| 450 | sk->sk_err = EMSGSIZE; | 520 | tx_err = EMSGSIZE; |
| 451 | if (!sock_flag(sk, SOCK_DEAD)) | ||
| 452 | sk_error_report(sk); | ||
| 453 | fallthrough; | 521 | fallthrough; |
| 454 | 522 | ||
| 455 | default: | 523 | default: |
| 456 | /* stop this tx job */ | 524 | /* reserved/unknown flow status (tx_err defaults to EBADMSG) */ |
| 457 | so->tx.state = ISOTP_IDLE; | 525 | |
| 526 | sk->sk_err = tx_err; | ||
| 527 | if (!sock_flag(sk, SOCK_DEAD)) | ||
| 528 | sk_error_report(sk); | ||
| 529 | |||
| 530 | isotp_set_tx_result(so, so->tx_gen, tx_err); | ||
| 531 | /* set to IDLE after publishing tx_result */ | ||
| 532 | smp_store_release(&so->tx.state, ISOTP_IDLE); | ||
| 458 | wake_up_interruptible(&so->wait); | 533 | wake_up_interruptible(&so->wait); |
| 459 | } | 534 | } |
| 460 | return 0; | 535 | return 0; |
| @@ -467,7 +542,7 @@ static int isotp_rcv_sf(struct sock *sk, struct canfd_frame *cf, int pcilen, · 6 unmodified lines | |||
| 467 | struct sk_buff *nskb; | 542 | struct sk_buff *nskb; |
| 468 | 543 | ||
| 469 | hrtimer_cancel(&so->rxtimer); | 544 | hrtimer_cancel(&so->rxtimer); |
| 470 | so->rx.state = ISOTP_IDLE; | 545 | WRITE_ONCE(so->rx.state, ISOTP_IDLE); |
| 471 | 546 | ||
| 472 | if (!len || len > cf->len - pcilen) | 547 | if (!len || len > cf->len - pcilen) |
| 473 | return 1; | 548 | return 1; |
| @@ -501,7 +576,7 @@ static int isotp_rcv_ff(struct sock *sk, struct canfd_frame *cf, int ae) · 27 unmodified lines | |||
| 501 | int ff_pci_sz; | 576 | int ff_pci_sz; |
| 502 | 577 | ||
| 503 | hrtimer_cancel(&so->rxtimer); | 578 | hrtimer_cancel(&so->rxtimer); |
| 504 | so->rx.state = ISOTP_IDLE; | 579 | WRITE_ONCE(so->rx.state, ISOTP_IDLE); |
| 505 | 580 | ||
| 506 | /* get the used sender LL_DL from the (first) CAN frame data length */ | 581 | /* get the used sender LL_DL from the (first) CAN frame data length */ |
| 507 | so->rx.ll_dl = padlen(cf->len); | 582 | so->rx.ll_dl = padlen(cf->len); |
| @@ -555,7 +630,7 @@ static int isotp_rcv_ff(struct sock *sk, struct canfd_frame *cf, int ae) · 47 unmodified lines | |||
| 555 | 630 | ||
| 556 | /* initial setup for this pdu reception */ | 631 | /* initial setup for this pdu reception */ |
| 557 | so->rx.sn = 1; | 632 | so->rx.sn = 1; |
| 558 | so->rx.state = ISOTP_WAIT_DATA; | 633 | WRITE_ONCE(so->rx.state, ISOTP_WAIT_DATA); |
| 559 | 634 | ||
| 560 | /* no creation of flow control frames */ | 635 | /* no creation of flow control frames */ |
| 561 | if (so->opt.flags & CAN_ISOTP_LISTEN_MODE) | 636 | if (so->opt.flags & CAN_ISOTP_LISTEN_MODE) |
| @@ -573,7 +648,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 11 unmodified lines | |||
| 573 | struct sk_buff *nskb; | 648 | struct sk_buff *nskb; |
| 574 | int i; | 649 | int i; |
| 575 | 650 | ||
| 576 | if (so->rx.state != ISOTP_WAIT_DATA) | 651 | if (READ_ONCE(so->rx.state) != ISOTP_WAIT_DATA) |
| 577 | return 0; | 652 | return 0; |
| 578 | 653 | ||
| 579 | /* drop if timestamp gap is less than force_rx_stmin nano secs */ | 654 | /* drop if timestamp gap is less than force_rx_stmin nano secs */ |
| @@ -588,11 +663,9 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 8 unmodified lines | |||
| 588 | hrtimer_cancel(&so->rxtimer); | 663 | hrtimer_cancel(&so->rxtimer); |
| 589 | 664 | ||
| 590 | /* isotp_rx_timer_handler() may have raced us for so->rx.state | 665 | /* isotp_rx_timer_handler() may have raced us for so->rx.state |
| 591 | * while hrtimer_cancel() above waited for it to finish, already | 666 | * while hrtimer_cancel() above waited for it to finish => recheck |
| 592 | * reporting ETIMEDOUT and resetting the reception; don't process | ||
| 593 | * this CF into a reassembly that has already been given up on. | ||
| 594 | */ | 667 | */ |
| 595 | if (so->rx.state != ISOTP_WAIT_DATA) | 668 | if (READ_ONCE(so->rx.state) != ISOTP_WAIT_DATA) |
| 596 | return 1; | 669 | return 1; |
| 597 | 670 | ||
| 598 | /* CFs are never longer than the FF */ | 671 | /* CFs are never longer than the FF */ |
| @@ -613,7 +686,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 14 unmodified lines | |||
| 613 | sk_error_report(sk); | 686 | sk_error_report(sk); |
| 614 | 687 | ||
| 615 | /* reset rx state */ | 688 | /* reset rx state */ |
| 616 | so->rx.state = ISOTP_IDLE; | 689 | WRITE_ONCE(so->rx.state, ISOTP_IDLE); |
| 617 | return 1; | 690 | return 1; |
| 618 | } | 691 | } |
| 619 | so->rx.sn++; | 692 | so->rx.sn++; |
| @@ -627,7 +700,7 @@ static int isotp_rcv_cf(struct sock *sk, struct canfd_frame *cf, int ae, · 7 unmodified lines | |||
| 627 | 700 | ||
| 628 | if (so->rx.idx >= so->rx.len) { | 701 | if (so->rx.idx >= so->rx.len) { |
| 629 | /* we are done */ | 702 | /* we are done */ |
| 630 | so->rx.state = ISOTP_IDLE; | 703 | WRITE_ONCE(so->rx.state, ISOTP_IDLE); |
| 631 | 704 | ||
| 632 | if ((so->opt.flags & ISOTP_CHECK_PADDING) && | 705 | if ((so->opt.flags & ISOTP_CHECK_PADDING) && |
| 633 | check_pad(so, cf, i + 1, so->opt.rxpad_content)) { | 706 | check_pad(so, cf, i + 1, so->opt.rxpad_content)) { |
| @@ -698,8 +771,10 @@ static void isotp_rcv(struct sk_buff *skb, void *data) · 64 unmodified lines | |||
| 698 | 771 | ||
| 699 | if (so->opt.flags & CAN_ISOTP_HALF_DUPLEX) { | 772 | if (so->opt.flags & CAN_ISOTP_HALF_DUPLEX) { |
| 700 | /* check rx/tx path half duplex expectations */ | 773 | /* check rx/tx path half duplex expectations */ |
| 701 | if ((so->tx.state != ISOTP_IDLE && n_pci_type != N_PCI_FC) || | 774 | if ((READ_ONCE(so->tx.state) != ISOTP_IDLE && |
| 702 | (so->rx.state != ISOTP_IDLE && n_pci_type == N_PCI_FC)) | 775 | n_pci_type != N_PCI_FC) || |
| 776 | (READ_ONCE(so->rx.state) != ISOTP_IDLE && | ||
| 777 | n_pci_type == N_PCI_FC)) | ||
| 703 | goto out_unlock; | 778 | goto out_unlock; |
| 704 | } | 779 | } |
| 705 | 780 | ||
| @@ -794,6 +869,7 @@ static void isotp_send_cframe(struct isotp_sock *so) · 88 unmodified lines | |||
| 794 | struct canfd_frame *cf; | 869 | struct canfd_frame *cf; |
| 795 | int can_send_ret; | 870 | int can_send_ret; |
| 796 | int ae = (so->opt.flags & CAN_ISOTP_EXTEND_ADDR) ? 1 : 0; | 871 | int ae = (so->opt.flags & CAN_ISOTP_EXTEND_ADDR) ? 1 : 0; |
| 872 | u32 old_cfecho; | ||
| 797 | 873 | ||
| 798 | dev = dev_get_by_index(sock_net(sk), so->ifindex); | 874 | dev = dev_get_by_index(sock_net(sk), so->ifindex); |
| 799 | if (!dev) | 875 | if (!dev) |
| @@ -814,6 +890,9 @@ static void isotp_send_cframe(struct isotp_sock *so) · 14 unmodified lines | |||
| 814 | 890 | ||
| 815 | csx->can_iif = dev->ifindex; | 891 | csx->can_iif = dev->ifindex; |
| 816 | 892 | ||
| 893 | /* set uid in tx skb to identify CF echo frames */ | ||
| 894 | can_set_skb_uid(skb); | ||
| 895 | |||
| 817 | cf = (struct canfd_frame *)skb->data; | 896 | cf = (struct canfd_frame *)skb->data; |
| 818 | skb_put_zero(skb, so->ll.mtu); | 897 | skb_put_zero(skb, so->ll.mtu); |
| 819 | 898 | ||
| @@ -830,12 +909,15 @@ static void isotp_send_cframe(struct isotp_sock *so) · 10 unmodified lines | |||
| 830 | skb->dev = dev; | 909 | skb->dev = dev; |
| 831 | can_skb_set_owner(skb, sk); | 910 | can_skb_set_owner(skb, sk); |
| 832 | 911 | ||
| 833 | /* cfecho should have been zero'ed by init/isotp_rcv_echo() */ | 912 | /* zero'ed by init/isotp_rcv_echo(); reached lock-free via |
| 834 | if (so->cfecho) | 913 | * isotp_txfr_timer_handler() too, so use READ_ONCE()/WRITE_ONCE() |
| 835 | pr_notice_once("can-isotp: cfecho is %08X != 0\n", so->cfecho); | 914 | */ |
| 915 | old_cfecho = READ_ONCE(so->cfecho); | ||
| 916 | if (old_cfecho) | ||
| 917 | pr_notice_once("can-isotp: cfecho is %08X != 0\n", old_cfecho); | ||
| 836 | 918 | ||
| 837 | /* set consecutive frame echo tag */ | 919 | /* set consecutive frame echo tag */ |
| 838 | so->cfecho = *(u32 *)cf->data; | 920 | WRITE_ONCE(so->cfecho, skb->hash); |
| 839 | 921 | ||
| 840 | /* send frame with local echo enabled */ | 922 | /* send frame with local echo enabled */ |
| 841 | can_send_ret = can_send(skb, 1); | 923 | can_send_ret = can_send(skb, 1); |
| @@ -887,7 +969,6 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) · 45 unmodified lines | |||
| 887 | { | 969 | { |
| 888 | struct sock *sk = (struct sock *)data; | 970 | struct sock *sk = (struct sock *)data; |
| 889 | struct isotp_sock *so = isotp_sk(sk); | 971 | struct isotp_sock *so = isotp_sk(sk); |
| 890 | struct canfd_frame *cf = (struct canfd_frame *)skb->data; | ||
| 891 | 972 | ||
| 892 | /* only handle my own local echo CF/SF skb's (no FF!) */ | 973 | /* only handle my own local echo CF/SF skb's (no FF!) */ |
| 893 | if (skb->sk != sk) | 974 | if (skb->sk != sk) |
| @@ -899,32 +980,35 @@ static void isotp_rcv_echo(struct sk_buff *skb, void *data) · 5 unmodified lines | |||
| 899 | spin_lock(&so->rx_lock); | 980 | spin_lock(&so->rx_lock); |
| 900 | 981 | ||
| 901 | /* so->cfecho may since belong to a new transfer; recheck under lock */ | 982 | /* so->cfecho may since belong to a new transfer; recheck under lock */ |
| 902 | if (so->cfecho != *(u32 *)cf->data) | 983 | if (READ_ONCE(so->cfecho) != skb->hash) |
| 903 | goto out_unlock; | 984 | goto out_unlock; |
| 904 | 985 | ||
| 905 | /* cancel local echo timeout */ | 986 | /* cancel local echo timeout */ |
| 906 | hrtimer_cancel(&so->echotimer); | 987 | hrtimer_cancel(&so->echotimer); |
| 907 | 988 | ||
| 908 | /* local echo skb with consecutive frame has been consumed */ | 989 | /* local echo skb with consecutive frame has been consumed */ |
| 909 | so->cfecho = 0; | 990 | WRITE_ONCE(so->cfecho, 0); |
| 910 | 991 | ||
| 911 | /* claiming a transfer also takes so->rx_lock, so a plain recheck | 992 | /* claiming a transfer also takes so->rx_lock, so a plain recheck |
| 912 | * is enough: so->tx.state can't have flipped to ISOTP_SENDING for | 993 | * is enough: so->tx.state can't have flipped to ISOTP_SENDING for |
| 913 | * a new claim while we're still in here | 994 | * a new claim while we're still in here |
| 914 | */ | 995 | */ |
| 915 | if (so->tx.state != ISOTP_SENDING) | 996 | if (READ_ONCE(so->tx.state) != ISOTP_SENDING) |
| 916 | goto out_unlock; | 997 | goto out_unlock; |
| 917 | 998 | ||
| 918 | if (so->tx.idx >= so->tx.len) { | 999 | if (so->tx.idx >= so->tx.len) { |
| 919 | /* we are done */ | 1000 | /* we are done */ |
| 920 | so->tx.state = ISOTP_IDLE; | 1001 | |
| 1002 | isotp_set_tx_result(so, so->tx_gen, 0); | ||
| 1003 | /* set to IDLE after publishing tx_result */ | ||
| 1004 | smp_store_release(&so->tx.state, ISOTP_IDLE); | ||
| 921 | wake_up_interruptible(&so->wait); | 1005 | wake_up_interruptible(&so->wait); |
| 922 | goto out_unlock; | 1006 | goto out_unlock; |
| 923 | } | 1007 | } |
| 924 | 1008 | ||
| 925 | if (so->txfc.bs && so->tx.bs >= so->txfc.bs) { | 1009 | if (so->txfc.bs && so->tx.bs >= so->txfc.bs) { |
| 926 | /* stop and wait for FC with timeout */ | 1010 | /* stop and wait for FC with timeout */ |
| 927 | so->tx.state = ISOTP_WAIT_FC; | 1011 | WRITE_ONCE(so->tx.state, ISOTP_WAIT_FC); |
| 928 | hrtimer_start(&so->txtimer, ktime_set(ISOTP_FC_TIMEOUT, 0), | 1012 | hrtimer_start(&so->txtimer, ktime_set(ISOTP_FC_TIMEOUT, 0), |
| 929 | HRTIMER_MODE_REL_SOFT); | 1013 | HRTIMER_MODE_REL_SOFT); |
| 930 | goto out_unlock; | 1014 | goto out_unlock; |
| @@ -946,16 +1030,20 @@ out_unlock: · 15 unmodified lines | |||
| 946 | spin_unlock(&so->rx_lock); | 1030 | spin_unlock(&so->rx_lock); |
| 947 | } | 1031 | } |
| 948 | 1032 | ||
| 949 | /* shared by so->txtimer's and so->echotimer's callbacks. Both timers get | 1033 | /* isotp_tx_timeout: we did not get any flow control or echo frame in time |
| 950 | * cancelled under so->rx_lock elsewhere, so this must stay lock-free to | 1034 | * |
| 951 | * avoid deadlocking with that; uses so->tx_gen instead to avoid tainting | 1035 | * Shared by so->txtimer's and so->echotimer's callbacks. Both timers get |
| 952 | * a new transfer with an error from the one that just timed out. | 1036 | * cancelled under so->rx_lock elsewhere, so this must stay lock-free. |
| 1037 | * | ||
| 1038 | * tx.state is acquired before tx_gen. Common sequence in isotp_tx_gen_done(). | ||
| 1039 | * cmpxchg() only orders itself, not the two preceding loads. | ||
| 953 | */ | 1040 | */ |
| 954 | static enum hrtimer_restart isotp_tx_timeout(struct isotp_sock *so) | 1041 | static enum hrtimer_restart isotp_tx_timeout(struct isotp_sock *so) |
| 955 | { | 1042 | { |
| 956 | struct sock *sk = &so->sk; | 1043 | struct sock *sk = &so->sk; |
| 1044 | /* read tx.state first for the common sequence */ | ||
| 1045 | u32 old_state = smp_load_acquire(&so->tx.state); | ||
| 957 | u32 gen = READ_ONCE(so->tx_gen); | 1046 | u32 gen = READ_ONCE(so->tx_gen); |
| 958 | u32 old_state = READ_ONCE(so->tx.state); | ||
| 959 | 1047 | ||
| 960 | /* don't handle timeouts in IDLE or SHUTDOWN state */ | 1048 | /* don't handle timeouts in IDLE or SHUTDOWN state */ |
| 961 | if (old_state == ISOTP_IDLE || old_state == ISOTP_SHUTDOWN) | 1049 | if (old_state == ISOTP_IDLE || old_state == ISOTP_SHUTDOWN) |
| @@ -965,14 +1053,14 @@ static enum hrtimer_restart isotp_tx_timeout(struct isotp_sock *so) · 3 unmodified lines | |||
| 965 | if (cmpxchg(&so->tx.state, old_state, ISOTP_IDLE) != old_state) | 1053 | if (cmpxchg(&so->tx.state, old_state, ISOTP_IDLE) != old_state) |
| 966 | return HRTIMER_NORESTART; | 1054 | return HRTIMER_NORESTART; |
| 967 | 1055 | ||
| 968 | /* we did not get any flow control or echo frame in time */ | 1056 | /* detected timeout: report 'communication error on send' */ |
| 969 | 1057 | ||
| 970 | if (READ_ONCE(so->tx_gen) == gen) { | 1058 | /* a stale read of this slot by a waiter still falls back to ECOMM */ |
| 971 | /* report 'communication error on send' */ | 1059 | isotp_set_tx_result(so, gen, ECOMM); |
| 972 | sk->sk_err = ECOMM; | 1060 | |
| 973 | if (!sock_flag(sk, SOCK_DEAD)) | 1061 | sk->sk_err = ECOMM; |
| 974 | sk_error_report(sk); | 1062 | if (!sock_flag(sk, SOCK_DEAD)) |
| 975 | } | 1063 | sk_error_report(sk); |
| 976 | 1064 | ||
| 977 | wake_up_interruptible(&so->wait); | 1065 | wake_up_interruptible(&so->wait); |
| 978 | 1066 | ||
| @@ -1007,7 +1095,7 @@ static enum hrtimer_restart isotp_txfr_timer_handler(struct hrtimer *hrtimer) · 28 unmodified lines | |||
| 1007 | HRTIMER_MODE_REL_SOFT); | 1095 | HRTIMER_MODE_REL_SOFT); |
| 1008 | 1096 | ||
| 1009 | /* cfecho should be consumed by isotp_rcv_echo() here */ | 1097 | /* cfecho should be consumed by isotp_rcv_echo() here */ |
| 1010 | if (so->tx.state == ISOTP_SENDING && !so->cfecho) | 1098 | if (READ_ONCE(so->tx.state) == ISOTP_SENDING && !READ_ONCE(so->cfecho)) |
| 1011 | isotp_send_cframe(so); | 1099 | isotp_send_cframe(so); |
| 1012 | 1100 | ||
| 1013 | return HRTIMER_NORESTART; | 1101 | return HRTIMER_NORESTART; |
| @@ -1026,10 +1114,12 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 12 unmodified lines | |||
| 1026 | s64 hrtimer_sec = ISOTP_ECHO_TIMEOUT; | 1114 | s64 hrtimer_sec = ISOTP_ECHO_TIMEOUT; |
| 1027 | struct hrtimer *tx_hrt = &so->echotimer; | 1115 | struct hrtimer *tx_hrt = &so->echotimer; |
| 1028 | u32 new_state = ISOTP_SENDING; | 1116 | u32 new_state = ISOTP_SENDING; |
| 1117 | u32 my_gen; | ||
| 1118 | u32 old_cfecho; | ||
| 1029 | int off; | 1119 | int off; |
| 1030 | int err; | 1120 | int err; |
| 1031 | 1121 | ||
| 1032 | if (!so->bound || so->tx.state == ISOTP_SHUTDOWN) | 1122 | if (!so->bound || READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) |
| 1033 | return -EADDRNOTAVAIL; | 1123 | return -EADDRNOTAVAIL; |
| 1034 | 1124 | ||
| 1035 | /* claim the socket under so->rx_lock: this serializes the claim | 1125 | /* claim the socket under so->rx_lock: this serializes the claim |
| @@ -1046,29 +1136,33 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 10 unmodified lines | |||
| 1046 | if (msg->msg_flags & MSG_DONTWAIT) | 1136 | if (msg->msg_flags & MSG_DONTWAIT) |
| 1047 | return -EAGAIN; | 1137 | return -EAGAIN; |
| 1048 | 1138 | ||
| 1049 | if (so->tx.state == ISOTP_SHUTDOWN) | 1139 | if (READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) |
| 1050 | return -EADDRNOTAVAIL; | 1140 | return -EADDRNOTAVAIL; |
| 1051 | 1141 | ||
| 1052 | /* wait for complete transmission of current pdu */ | 1142 | /* wait for complete transmission of current pdu */ |
| 1053 | err = wait_event_interruptible(so->wait, | 1143 | err = wait_event_interruptible(so->wait, |
| 1054 | so->tx.state == ISOTP_IDLE); | 1144 | READ_ONCE(so->tx.state) == ISOTP_IDLE || |
| 1145 | READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN); | ||
| 1055 | if (err) | 1146 | if (err) |
| 1056 | return err; | 1147 | return err; |
| 1057 | } | 1148 | } |
| 1058 | 1149 | ||
| 1059 | /* new transfer: bump so->tx_gen and drain the old one's timers, | 1150 | /* txfrtimer's callback re-arms echotimer lock-free: drain it first */ |
| 1060 | * still under the so->rx_lock we just claimed the socket with | 1151 | hrtimer_cancel(&so->txfrtimer); |
| 1061 | */ | ||
| 1062 | WRITE_ONCE(so->tx.state, ISOTP_SENDING); | ||
| 1063 | WRITE_ONCE(so->tx_gen, READ_ONCE(so->tx_gen) + 1); | ||
| 1064 | hrtimer_cancel(&so->txtimer); | 1152 | hrtimer_cancel(&so->txtimer); |
| 1065 | hrtimer_cancel(&so->echotimer); | 1153 | hrtimer_cancel(&so->echotimer); |
| 1066 | hrtimer_cancel(&so->txfrtimer); | 1154 | |
| 1067 | so->cfecho = 0; | 1155 | /* new transfer: increment so->tx_gen and set tx.state after barrier */ |
| 1156 | my_gen = isotp_inc_tx_gen(READ_ONCE(so->tx_gen)); | ||
| 1157 | isotp_set_tx_result(so, my_gen, ECOMM); /* prevent stale slot matching */ | ||
| 1158 | WRITE_ONCE(so->tx_gen, my_gen); | ||
| 1159 | smp_wmb(); /* see smp_load_acquire() in isotp_tx_[timeout|gen_done] */ | ||
| 1160 | WRITE_ONCE(so->tx.state, ISOTP_SENDING); | ||
| 1161 | WRITE_ONCE(so->cfecho, 0); | ||
| 1068 | spin_unlock_bh(&so->rx_lock); | 1162 | spin_unlock_bh(&so->rx_lock); |
| 1069 | 1163 | ||
| 1070 | /* so->bound is only checked once above - a wakeup may have | 1164 | /* so->bound is only checked once above - a wakeup may have |
| 1071 | * unbound/rebound the socket meanwhile, so re-validate it | 1165 | * unbound/rebound the socket meanwhile => recheck |
| 1072 | */ | 1166 | */ |
| 1073 | if (!so->bound) { | 1167 | if (!so->bound) { |
| 1074 | err = -EADDRNOTAVAIL; | 1168 | err = -EADDRNOTAVAIL; |
| @@ -1127,6 +1221,9 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 52 unmodified lines | |||
| 1127 | 1221 | ||
| 1128 | csx->can_iif = dev->ifindex; | 1222 | csx->can_iif = dev->ifindex; |
| 1129 | 1223 | ||
| 1224 | /* set uid in tx skb to identify CF echo frames */ | ||
| 1225 | can_set_skb_uid(skb); | ||
| 1226 | |||
| 1130 | so->tx.len = size; | 1227 | so->tx.len = size; |
| 1131 | so->tx.idx = 0; | 1228 | so->tx.idx = 0; |
| 1132 | 1229 | ||
| @@ -1134,8 +1231,9 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 1 unmodified line | |||
| 1134 | skb_put_zero(skb, so->ll.mtu); | 1231 | skb_put_zero(skb, so->ll.mtu); |
| 1135 | 1232 | ||
| 1136 | /* cfecho should have been zero'ed by init / former isotp_rcv_echo() */ | 1233 | /* cfecho should have been zero'ed by init / former isotp_rcv_echo() */ |
| 1137 | if (so->cfecho) | 1234 | old_cfecho = READ_ONCE(so->cfecho); |
| 1138 | pr_notice_once("can-isotp: uninit cfecho %08X\n", so->cfecho); | 1235 | if (old_cfecho) |
| 1236 | pr_notice_once("can-isotp: uninit cfecho %08X\n", old_cfecho); | ||
| 1139 | 1237 | ||
| 1140 | /* check for single frame transmission depending on TX_DL */ | 1238 | /* check for single frame transmission depending on TX_DL */ |
| 1141 | if (size <= so->tx.ll_dl - SF_PCI_SZ4 - ae - off) { | 1239 | if (size <= so->tx.ll_dl - SF_PCI_SZ4 - ae - off) { |
| @@ -1163,7 +1261,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 21 unmodified lines | |||
| 1163 | cf->data[ae] |= size; | 1261 | cf->data[ae] |= size; |
| 1164 | 1262 | ||
| 1165 | /* set CF echo tag for isotp_rcv_echo() (SF-mode) */ | 1263 | /* set CF echo tag for isotp_rcv_echo() (SF-mode) */ |
| 1166 | so->cfecho = *(u32 *)cf->data; | 1264 | WRITE_ONCE(so->cfecho, skb->hash); |
| 1167 | } else { | 1265 | } else { |
| 1168 | /* send first frame */ | 1266 | /* send first frame */ |
| 1169 | 1267 | ||
| @@ -1180,7 +1278,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 10 unmodified lines | |||
| 1180 | so->txfc.bs = 0; | 1278 | so->txfc.bs = 0; |
| 1181 | 1279 | ||
| 1182 | /* set CF echo tag for isotp_rcv_echo() (CF-mode) */ | 1280 | /* set CF echo tag for isotp_rcv_echo() (CF-mode) */ |
| 1183 | so->cfecho = *(u32 *)cf->data; | 1281 | WRITE_ONCE(so->cfecho, skb->hash); |
| 1184 | } else { | 1282 | } else { |
| 1185 | /* standard flow control check */ | 1283 | /* standard flow control check */ |
| 1186 | new_state = ISOTP_WAIT_FIRST_FC; | 1284 | new_state = ISOTP_WAIT_FIRST_FC; |
| @@ -1190,12 +1288,12 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 3 unmodified lines | |||
| 1190 | tx_hrt = &so->txtimer; | 1288 | tx_hrt = &so->txtimer; |
| 1191 | 1289 | ||
| 1192 | /* no CF echo tag for isotp_rcv_echo() (FF-mode) */ | 1290 | /* no CF echo tag for isotp_rcv_echo() (FF-mode) */ |
| 1193 | so->cfecho = 0; | 1291 | WRITE_ONCE(so->cfecho, 0); |
| 1194 | } | 1292 | } |
| 1195 | } | 1293 | } |
| 1196 | 1294 | ||
| 1197 | spin_lock_bh(&so->rx_lock); | 1295 | spin_lock_bh(&so->rx_lock); |
| 1198 | if (so->tx.state == ISOTP_SHUTDOWN) { | 1296 | if (READ_ONCE(so->tx.state) == ISOTP_SHUTDOWN) { |
| 1199 | /* isotp_release() has since taken over and already drained | 1297 | /* isotp_release() has since taken over and already drained |
| 1200 | * our timers - don't send into a socket that's going away | 1298 | * our timers - don't send into a socket that's going away |
| 1201 | */ | 1299 | */ |
| @@ -1206,7 +1304,7 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 4 unmodified lines | |||
| 1206 | return -EADDRNOTAVAIL; | 1304 | return -EADDRNOTAVAIL; |
| 1207 | } | 1305 | } |
| 1208 | /* WAIT_FIRST_FC for standard FF, else stays ISOTP_SENDING */ | 1306 | /* WAIT_FIRST_FC for standard FF, else stays ISOTP_SENDING */ |
| 1209 | so->tx.state = new_state; | 1307 | WRITE_ONCE(so->tx.state, new_state); |
| 1210 | hrtimer_start(tx_hrt, ktime_set(hrtimer_sec, 0), | 1308 | hrtimer_start(tx_hrt, ktime_set(hrtimer_sec, 0), |
| 1211 | HRTIMER_MODE_REL_SOFT); | 1309 | HRTIMER_MODE_REL_SOFT); |
| 1212 | spin_unlock_bh(&so->rx_lock); | 1310 | spin_unlock_bh(&so->rx_lock); |
| @@ -1223,20 +1321,49 @@ static int isotp_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) · 10 unmodified lines | |||
| 1223 | __func__, ERR_PTR(err)); | 1321 | __func__, ERR_PTR(err)); |
| 1224 | 1322 | ||
| 1225 | spin_lock_bh(&so->rx_lock); | 1323 | spin_lock_bh(&so->rx_lock); |
| 1324 | |||
| 1325 | /* new transfer already claimed by a concurrent completion, | ||
| 1326 | * timeout or sendmsg() while we were stuck in can_send()? | ||
| 1327 | */ | ||
| 1328 | if (READ_ONCE(so->tx_gen) != my_gen) { | ||
| 1329 | /* don't touch timers and state of the new transfer */ | ||
| 1330 | spin_unlock_bh(&so->rx_lock); | ||
| 1331 | return err; | ||
| 1332 | } | ||
| 1333 | |||
| 1226 | /* no transmission -> no timeout monitoring */ | 1334 | /* no transmission -> no timeout monitoring */ |
| 1227 | hrtimer_cancel(tx_hrt); | 1335 | hrtimer_cancel(tx_hrt); |
| 1228 | goto err_out_drop_locked; | 1336 | goto err_out_drop_locked; |
| 1229 | } | 1337 | } |
| 1230 | 1338 | ||
| 1231 | if (wait_tx_done) { | 1339 | if (wait_tx_done) { |
| 1232 | /* wait for complete transmission of current pdu */ | 1340 | /* wake up for: |
| 1233 | err = wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE); | 1341 | * - concurrent sendmsg() claiming a new transfer |
| 1342 | * - complete transmission of current PDU | ||
| 1343 | * - shutdown state change in isotp_release() | ||
| 1344 | * isotp_tx_gen_done() uses common tx.state/tx_gen read sequence | ||
| 1345 | */ | ||
| 1346 | err = wait_event_interruptible(so->wait, | ||
| 1347 | isotp_tx_gen_done(so, my_gen)); | ||
| 1234 | if (err) | 1348 | if (err) |
| 1235 | goto err_event_drop; | 1349 | goto err_event_drop; |
| 1236 | 1350 | ||
| 1237 | err = sock_error(sk); | 1351 | /* still our claim, but isotp_release() force-shut it down */ |
| 1238 | if (err) | 1352 | if (smp_load_acquire(&so->tx.state) == ISOTP_SHUTDOWN && |
| 1239 | return err; | 1353 | READ_ONCE(so->tx_gen) == my_gen) { |
| 1354 | err = -EADDRNOTAVAIL; | ||
| 1355 | goto err_event_drop; | ||
| 1356 | } | ||
| 1357 | |||
| 1358 | /* own completion, or tx_gen moved on - either way this is | ||
| 1359 | * what isotp_get_tx_result() recorded for my_gen | ||
| 1360 | */ | ||
| 1361 | err = isotp_get_tx_result(so, my_gen); | ||
| 1362 | |||
| 1363 | /* drain to avoid stale error for a later poll()/SO_ERROR */ | ||
| 1364 | sock_error(sk); | ||
| 1365 | |||
| 1366 | return err ? err : size; | ||
| 1240 | } | 1367 | } |
| 1241 | 1368 | ||
| 1242 | return size; | 1369 | return size; |
| @@ -1246,15 +1373,26 @@ err_out_drop: · 3 unmodified lines | |||
| 1246 | spin_lock_bh(&so->rx_lock); | 1373 | spin_lock_bh(&so->rx_lock); |
| 1247 | goto err_out_drop_locked; | 1374 | goto err_out_drop_locked; |
| 1248 | err_event_drop: | 1375 | err_event_drop: |
| 1249 | /* interrupted waiting on our own transfer - drain its timers */ | 1376 | /* interrupted or shut down while waiting on our own transfer */ |
| 1250 | spin_lock_bh(&so->rx_lock); | 1377 | spin_lock_bh(&so->rx_lock); |
| 1378 | |||
| 1379 | /* new transfer already started by concurrent sendmsg()? */ | ||
| 1380 | if (READ_ONCE(so->tx_gen) != my_gen) { | ||
| 1381 | /* don't touch timers and states of the new transfer */ | ||
| 1382 | spin_unlock_bh(&so->rx_lock); | ||
| 1383 | return err; | ||
| 1384 | } | ||
| 1385 | |||
| 1251 | hrtimer_cancel(&so->txfrtimer); | 1386 | hrtimer_cancel(&so->txfrtimer); |
| 1252 | hrtimer_cancel(&so->txtimer); | 1387 | hrtimer_cancel(&so->txtimer); |
| 1253 | hrtimer_cancel(&so->echotimer); | 1388 | hrtimer_cancel(&so->echotimer); |
| 1254 | err_out_drop_locked: | 1389 | err_out_drop_locked: |
| 1255 | /* release the claim; so->rx_lock still held from above */ | 1390 | /* release the claim; so->rx_lock still held from above */ |
| 1256 | so->cfecho = 0; | 1391 | WRITE_ONCE(so->cfecho, 0); |
| 1257 | so->tx.state = ISOTP_IDLE; | 1392 | |
| 1393 | /* only claim to IDLE if isotp_release() has not taken over */ | ||
| 1394 | if (READ_ONCE(so->tx.state) != ISOTP_SHUTDOWN) | ||
| 1395 | WRITE_ONCE(so->tx.state, ISOTP_IDLE); | ||
| 1258 | spin_unlock_bh(&so->rx_lock); | 1396 | spin_unlock_bh(&so->rx_lock); |
| 1259 | wake_up_interruptible(&so->wait); | 1397 | wake_up_interruptible(&so->wait); |
| 1260 | 1398 | ||
| @@ -1320,8 +1458,9 @@ static int isotp_release(struct socket *sock) · 59 unmodified lines | |||
| 1320 | /* best-effort: wait for a running pdu to finish, but don't block on | 1458 | /* best-effort: wait for a running pdu to finish, but don't block on |
| 1321 | * it forever - give up after the first signal | 1459 | * it forever - give up after the first signal |
| 1322 | */ | 1460 | */ |
| 1323 | while (so->tx.state != ISOTP_IDLE && | 1461 | while (READ_ONCE(so->tx.state) != ISOTP_IDLE && |
| 1324 | wait_event_interruptible(so->wait, so->tx.state == ISOTP_IDLE) == 0) | 1462 | wait_event_interruptible(so->wait, |
| 1463 | READ_ONCE(so->tx.state) == ISOTP_IDLE) == 0) | ||
| 1325 | ; | 1464 | ; |
| 1326 | 1465 | ||
| 1327 | /* claim the socket under so->rx_lock like sendmsg() does, so its | 1466 | /* claim the socket under so->rx_lock like sendmsg() does, so its |
| @@ -1329,9 +1468,12 @@ static int isotp_release(struct socket *sock) · 1 unmodified line | |||
| 1329 | * unconditionally, even when a signal cut the wait above short | 1468 | * unconditionally, even when a signal cut the wait above short |
| 1330 | */ | 1469 | */ |
| 1331 | spin_lock_bh(&so->rx_lock); | 1470 | spin_lock_bh(&so->rx_lock); |
| 1332 | so->tx.state = ISOTP_SHUTDOWN; | 1471 | WRITE_ONCE(so->tx.state, ISOTP_SHUTDOWN); |
| 1333 | spin_unlock_bh(&so->rx_lock); | 1472 | spin_unlock_bh(&so->rx_lock); |
| 1334 | so->rx.state = ISOTP_IDLE; | 1473 | WRITE_ONCE(so->rx.state, ISOTP_IDLE); |
| 1474 | |||
| 1475 | /* forced SHUTDOWN may have skipped IDLE (gave up on a signal) */ | ||
| 1476 | wake_up_interruptible(&so->wait); | ||
| 1335 | 1477 | ||
| 1336 | spin_lock(&isotp_notifier_lock); | 1478 | spin_lock(&isotp_notifier_lock); |
| 1337 | while (isotp_busy_notifier == so) { | 1479 | while (isotp_busy_notifier == so) { |
| @@ -1447,7 +1589,8 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l · 109 unmodified lines | |||
| 1447 | * with so->bound in the same lock_sock() section above, so there is | 1589 | * with so->bound in the same lock_sock() section above, so there is |
| 1448 | * no window in which a concurrent isotp_notify() could be missed. | 1590 | * no window in which a concurrent isotp_notify() could be missed. |
| 1449 | */ | 1591 | */ |
| 1450 | if (so->tx.state != ISOTP_IDLE || so->rx.state != ISOTP_IDLE) { | 1592 | if (READ_ONCE(so->tx.state) != ISOTP_IDLE || |
| 1593 | READ_ONCE(so->rx.state) != ISOTP_IDLE) { | ||
| 1451 | err = -EAGAIN; | 1594 | err = -EAGAIN; |
| 1452 | goto out; | 1595 | goto out; |
| 1453 | } | 1596 | } |
| @@ -1481,7 +1624,7 @@ static int isotp_bind(struct socket *sock, struct sockaddr_unsized *uaddr, int l · 27 unmodified lines | |||
| 1481 | isotp_rcv, sk, "isotp", sk); | 1624 | isotp_rcv, sk, "isotp", sk); |
| 1482 | 1625 | ||
| 1483 | /* no consecutive frame echo skb in flight */ | 1626 | /* no consecutive frame echo skb in flight */ |
| 1484 | so->cfecho = 0; | 1627 | WRITE_ONCE(so->cfecho, 0); |
| 1485 | 1628 | ||
| 1486 | /* register for echo skb's */ | 1629 | /* register for echo skb's */ |
| 1487 | can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), | 1630 | can_rx_register(net, dev, tx_id, SINGLE_MASK(tx_id), |
| @@ -1847,7 +1990,7 @@ static __poll_t isotp_poll(struct file *file, struct socket *sock, poll_table *w · 359 unmodified lines | |||
| 1847 | poll_wait(file, &so->wait, wait); | 1990 | poll_wait(file, &so->wait, wait); |
| 1848 | 1991 | ||
| 1849 | /* Check for false positives due to TX state */ | 1992 | /* Check for false positives due to TX state */ |
| 1850 | if ((mask & EPOLLWRNORM) && (so->tx.state != ISOTP_IDLE)) | 1993 | if ((mask & EPOLLWRNORM) && (READ_ONCE(so->tx.state) != ISOTP_IDLE)) |
| 1851 | mask &= ~(EPOLLOUT | EPOLLWRNORM); | 1994 | mask &= ~(EPOLLOUT | EPOLLWRNORM); |
| 1852 | 1995 | ||
| 1853 | return mask; | 1996 | return mask; |
| @@ -1907,13 +2050,18 @@ static __init int isotp_module_init(void) · 53 unmodified lines | |||
| 1907 | 2050 | ||
| 1908 | pr_info("can: isotp protocol (max_pdu_size %d)\n", max_pdu_size); | 2051 | pr_info("can: isotp protocol (max_pdu_size %d)\n", max_pdu_size); |
| 1909 | 2052 | ||
| 2053 | err = register_netdevice_notifier(&canisotp_notifier); | ||
| 2054 | if (err) | ||
| 2055 | return err; | ||
| 2056 | |||
| 1910 | err = can_proto_register(&isotp_can_proto); | 2057 | err = can_proto_register(&isotp_can_proto); |
| 1911 | if (err < 0) | 2058 | if (err < 0) { |
| 1912 | pr_err("can: registration of isotp protocol failed %pe\n", ERR_PTR(err)); | 2059 | pr_err("can: registration of isotp protocol failed %pe\n", ERR_PTR(err)); |
| 1913 | else | 2060 | unregister_netdevice_notifier(&canisotp_notifier); |
| 1914 | register_netdevice_notifier(&canisotp_notifier); | 2061 | return err; |
| 2062 | } | ||
| 1915 | 2063 | ||
| 1916 | return err; | 2064 | return 0; |
| 1917 | } | 2065 | } |
| 1918 | 2066 | ||
| 1919 | static __exit void isotp_module_exit(void) | 2067 | static __exit void isotp_module_exit(void) |
| @@ -20,6 +20,7 @@ static void __j1939_ecu_release(struct kref *kref) · 19 unmodified lines | |||
| 20 | struct j1939_priv *priv = ecu->priv; | 20 | struct j1939_priv *priv = ecu->priv; |
| 21 | 21 | ||
| 22 | list_del(&ecu->list); | 22 | list_del(&ecu->list); |
| 23 | netdev_put(priv->ndev, &ecu->priv_dev_tracker); | ||
| 23 | kfree(ecu); | 24 | kfree(ecu); |
| 24 | j1939_priv_put(priv); | 25 | j1939_priv_put(priv); |
| 25 | } | 26 | } |
| @@ -155,6 +156,7 @@ struct j1939_ecu *j1939_ecu_create_locked(struct j1939_priv *priv, name_t name) · 129 unmodified lines | |||
| 155 | if (!ecu) | 156 | if (!ecu) |
| 156 | return ERR_PTR(-ENOMEM); | 157 | return ERR_PTR(-ENOMEM); |
| 157 | kref_init(&ecu->kref); | 158 | kref_init(&ecu->kref); |
| 159 | netdev_hold(priv->ndev, &ecu->priv_dev_tracker, gfp_any()); | ||
| 158 | ecu->addr = J1939_IDLE_ADDR; | 160 | ecu->addr = J1939_IDLE_ADDR; |
| 159 | ecu->name = name; | 161 | ecu->name = name; |
| 160 | 162 | ||
| @@ -38,6 +38,7 @@ struct j1939_ecu { · 37 unmodified lines | |||
| 38 | struct hrtimer ac_timer; | 38 | struct hrtimer ac_timer; |
| 39 | struct kref kref; | 39 | struct kref kref; |
| 40 | struct j1939_priv *priv; | 40 | struct j1939_priv *priv; |
| 41 | netdevice_tracker priv_dev_tracker; | ||
| 41 | 42 | ||
| 42 | /* count users, to help transport protocol decide for interaction */ | 43 | /* count users, to help transport protocol decide for interaction */ |
| 43 | int nusers; | 44 | int nusers; |
| @@ -60,6 +61,7 @@ struct j1939_priv { · 16 unmodified lines | |||
| 60 | rwlock_t lock; | 61 | rwlock_t lock; |
| 61 | 62 | ||
| 62 | struct net_device *ndev; | 63 | struct net_device *ndev; |
| 64 | netdevice_tracker dev_tracker; | ||
| 63 | 65 | ||
| 64 | /* list of 256 ecu ptrs, that cache the claimed addresses. | 66 | /* list of 256 ecu ptrs, that cache the claimed addresses. |
| 65 | * also protected by the above lock | 67 | * also protected by the above lock |
| @@ -230,6 +232,7 @@ enum j1939_session_state { · 164 unmodified lines | |||
| 230 | 232 | ||
| 231 | struct j1939_session { | 233 | struct j1939_session { |
| 232 | struct j1939_priv *priv; | 234 | struct j1939_priv *priv; |
| 235 | netdevice_tracker priv_dev_tracker; | ||
| 233 | struct list_head active_session_list_entry; | 236 | struct list_head active_session_list_entry; |
| 234 | struct list_head sk_session_queue_entry; | 237 | struct list_head sk_session_queue_entry; |
| 235 | struct kref kref; | 238 | struct kref kref; |
| @@ -137,7 +137,7 @@ static struct j1939_priv *j1939_priv_create(struct net_device *ndev) · 136 unmodified lines | |||
| 137 | priv->ndev = ndev; | 137 | priv->ndev = ndev; |
| 138 | kref_init(&priv->kref); | 138 | kref_init(&priv->kref); |
| 139 | kref_init(&priv->rx_kref); | 139 | kref_init(&priv->rx_kref); |
| 140 | dev_hold(ndev); | 140 | netdev_hold(ndev, &priv->dev_tracker, GFP_KERNEL); |
| 141 | 141 | ||
| 142 | netdev_dbg(priv->ndev, "%s : 0x%p\n", __func__, priv); | 142 | netdev_dbg(priv->ndev, "%s : 0x%p\n", __func__, priv); |
| 143 | 143 | ||
| @@ -163,7 +163,7 @@ static void __j1939_priv_release(struct kref *kref) · 19 unmodified lines | |||
| 163 | WARN_ON_ONCE(!list_empty(&priv->ecus)); | 163 | WARN_ON_ONCE(!list_empty(&priv->ecus)); |
| 164 | WARN_ON_ONCE(!list_empty(&priv->j1939_socks)); | 164 | WARN_ON_ONCE(!list_empty(&priv->j1939_socks)); |
| 165 | 165 | ||
| 166 | dev_put(ndev); | 166 | netdev_put(ndev, &priv->dev_tracker); |
| 167 | kfree(priv); | 167 | kfree(priv); |
| 168 | } | 168 | } |
| 169 | 169 | ||
| @@ -281,7 +281,7 @@ struct j1939_priv *j1939_netdev_start(struct net_device *ndev) · 111 unmodified lines | |||
| 281 | */ | 281 | */ |
| 282 | kref_get(&priv_new->rx_kref); | 282 | kref_get(&priv_new->rx_kref); |
| 283 | mutex_unlock(&j1939_netdev_lock); | 283 | mutex_unlock(&j1939_netdev_lock); |
| 284 | dev_put(ndev); | 284 | netdev_put(ndev, &priv->dev_tracker); |
| 285 | kfree(priv); | 285 | kfree(priv); |
| 286 | return priv_new; | 286 | return priv_new; |
| 287 | } | 287 | } |
| @@ -298,7 +298,7 @@ struct j1939_priv *j1939_netdev_start(struct net_device *ndev) · 10 unmodified lines | |||
| 298 | j1939_priv_set(ndev, NULL); | 298 | j1939_priv_set(ndev, NULL); |
| 299 | mutex_unlock(&j1939_netdev_lock); | 299 | mutex_unlock(&j1939_netdev_lock); |
| 300 | 300 | ||
| 301 | dev_put(ndev); | 301 | netdev_put(ndev, &priv->dev_tracker); |
| 302 | kfree(priv); | 302 | kfree(priv); |
| 303 | 303 | ||
| 304 | return ERR_PTR(ret); | 304 | return ERR_PTR(ret); |
| @@ -283,6 +283,7 @@ static void j1939_session_destroy(struct j1939_session *session) · 282 unmodified lines | |||
| 283 | kfree_skb(skb); | 283 | kfree_skb(skb); |
| 284 | } | 284 | } |
| 285 | __j1939_session_drop(session); | 285 | __j1939_session_drop(session); |
| 286 | netdev_put(session->priv->ndev, &session->priv_dev_tracker); | ||
| 286 | j1939_priv_put(session->priv); | 287 | j1939_priv_put(session->priv); |
| 287 | kfree(session); | 288 | kfree(session); |
| 288 | } | 289 | } |
| @@ -1526,6 +1527,7 @@ static struct j1939_session *j1939_session_new(struct j1939_priv *priv, · 1237 unmodified lines | |||
| 1526 | INIT_LIST_HEAD(&session->active_session_list_entry); | 1527 | INIT_LIST_HEAD(&session->active_session_list_entry); |
| 1527 | INIT_LIST_HEAD(&session->sk_session_queue_entry); | 1528 | INIT_LIST_HEAD(&session->sk_session_queue_entry); |
| 1528 | kref_init(&session->kref); | 1529 | kref_init(&session->kref); |
| 1530 | netdev_hold(priv->ndev, &session->priv_dev_tracker, gfp_any()); | ||
| 1529 | 1531 | ||
| 1530 | j1939_priv_get(priv); | 1532 | j1939_priv_get(priv); |
| 1531 | session->priv = priv; | 1533 | session->priv = priv; |
| @@ -1579,7 +1581,7 @@ j1939_session *j1939_session_fresh_new(struct j1939_priv *priv, · 47 unmodified lines | |||
| 1579 | } | 1581 | } |
| 1580 | 1582 | ||
| 1581 | /* alloc data area */ | 1583 | /* alloc data area */ |
| 1582 | skb_put(skb, size); | 1584 | skb_put_zero(skb, size); |
| 1583 | /* skb is recounted in j1939_session_new() */ | 1585 | /* skb is recounted in j1939_session_new() */ |
| 1584 | return session; | 1586 | return session; |
| 1585 | } | 1587 | } |
| @@ -255,8 +255,10 @@ static int bpf_lwt_xmit_reroute(struct sk_buff *skb) · 254 unmodified lines | |||
| 255 | * if there is enough header space in skb. | 255 | * if there is enough header space in skb. |
| 256 | */ | 256 | */ |
| 257 | err = skb_cow_head(skb, LL_RESERVED_SPACE(dst->dev)); | 257 | err = skb_cow_head(skb, LL_RESERVED_SPACE(dst->dev)); |
| 258 | if (unlikely(err)) | 258 | if (unlikely(err)) { |
| 259 | dst_release(dst); | ||
| 259 | goto err; | 260 | goto err; |
| 261 | } | ||
| 260 | 262 | ||
| 261 | skb_dst_drop(skb); | 263 | skb_dst_drop(skb); |
| 262 | skb_dst_set(skb, dst); | 264 | skb_dst_set(skb, dst); |
| @@ -3972,6 +3972,7 @@ static void _rem_dev_from_if_list(struct pktgen_thread *t, · 3971 unmodified lines | |||
| 3972 | struct pktgen_dev *p; | 3972 | struct pktgen_dev *p; |
| 3973 | 3973 | ||
| 3974 | if_lock(t); | 3974 | if_lock(t); |
| 3975 | proc_remove(pkt_dev->entry); | ||
| 3975 | list_for_each_safe(q, n, &t->if_list) { | 3976 | list_for_each_safe(q, n, &t->if_list) { |
| 3976 | p = list_entry(q, struct pktgen_dev, list); | 3977 | p = list_entry(q, struct pktgen_dev, list); |
| 3977 | if (p == pkt_dev) | 3978 | if (p == pkt_dev) |
| @@ -4001,9 +4002,6 @@ static int pktgen_remove_device(struct pktgen_thread *t, · 23 unmodified lines | |||
| 4001 | * list to determine if interface already exist, avoid race | 4002 | * list to determine if interface already exist, avoid race |
| 4002 | * with proc_create_data() | 4003 | * with proc_create_data() |
| 4003 | */ | 4004 | */ |
| 4004 | proc_remove(pkt_dev->entry); | ||
| 4005 | |||
| 4006 | /* And update the thread if_list */ | ||
| 4007 | _rem_dev_from_if_list(t, pkt_dev); | 4005 | _rem_dev_from_if_list(t, pkt_dev); |
| 4008 | 4006 | ||
| 4009 | #ifdef CONFIG_XFRM | 4007 | #ifdef CONFIG_XFRM |
| @@ -927,6 +927,18 @@ static void skb_clone_fraglist(struct sk_buff *skb) · 926 unmodified lines | |||
| 927 | skb_get(list); | 927 | skb_get(list); |
| 928 | } | 928 | } |
| 929 | 929 | ||
| 930 | /** | ||
| 931 | * skb_pp_cow_data() - copy skb data into page-pool backed storage | ||
| 932 | * @pool: page pool to allocate from | ||
| 933 | * @pskb: pointer to skb pointer, replaced with the copied skb on success | ||
| 934 | * @headroom: headroom to reserve in the copied skb | ||
| 935 | * | ||
| 936 | * skb_copy_bits() handles both frags[] and frag_list input. If the copied | ||
| 937 | * skb remains non-linear, it uses frags[], which is the representation used | ||
| 938 | * by XDP multi-buffer. | ||
| 939 | * | ||
| 940 | * Return: 0 on success or a negative errno on failure. | ||
| 941 | */ | ||
| 930 | int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, | 942 | int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, |
| 931 | unsigned int headroom) | 943 | unsigned int headroom) |
| 932 | { | 944 | { |
| @@ -936,12 +948,6 @@ int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, · 3 unmodified lines | |||
| 936 | int err, i, head_off; | 948 | int err, i, head_off; |
| 937 | void *data; | 949 | void *data; |
| 938 | 950 | ||
| 939 | /* XDP does not support fraglist so we need to linearize | ||
| 940 | * the skb. | ||
| 941 | */ | ||
| 942 | if (skb_has_frag_list(skb)) | ||
| 943 | return -EOPNOTSUPP; | ||
| 944 | |||
| 945 | max_head_size = SKB_WITH_OVERHEAD(PAGE_SIZE - headroom); | 951 | max_head_size = SKB_WITH_OVERHEAD(PAGE_SIZE - headroom); |
| 946 | if (skb->len > max_head_size + MAX_SKB_FRAGS * PAGE_SIZE) | 952 | if (skb->len > max_head_size + MAX_SKB_FRAGS * PAGE_SIZE) |
| 947 | return -ENOMEM; | 953 | return -ENOMEM; |
| @@ -629,9 +629,9 @@ static int fou_create(struct net *net, struct fou_cfg *cfg, · 628 unmodified lines | |||
| 629 | return 0; | 629 | return 0; |
| 630 | 630 | ||
| 631 | error: | 631 | error: |
| 632 | kfree(fou); | ||
| 633 | if (sock) | 632 | if (sock) |
| 634 | udp_tunnel_sock_release(sock->sk); | 633 | udp_tunnel_sock_release(sock->sk); |
| 634 | kfree_rcu(fou, rcu); | ||
| 635 | 635 | ||
| 636 | return err; | 636 | return err; |
| 637 | } | 637 | } |
| @@ -182,6 +182,7 @@ static int nat_rtp_rtcp(struct sk_buff *skb, struct nf_conn *ct, · 181 unmodified lines | |||
| 182 | struct nf_conntrack_expect *rtp_exp, | 182 | struct nf_conntrack_expect *rtp_exp, |
| 183 | struct nf_conntrack_expect *rtcp_exp) | 183 | struct nf_conntrack_expect *rtcp_exp) |
| 184 | { | 184 | { |
| 185 | struct nf_conntrack_expect *rtp_pair[2] = { rtp_exp, rtcp_exp }; | ||
| 185 | struct nf_ct_h323_master *info = nfct_help_data(ct); | 186 | struct nf_ct_h323_master *info = nfct_help_data(ct); |
| 186 | int dir = CTINFO2DIR(ctinfo); | 187 | int dir = CTINFO2DIR(ctinfo); |
| 187 | int i; | 188 | int i; |
| @@ -227,22 +228,13 @@ static int nat_rtp_rtcp(struct sk_buff *skb, struct nf_conn *ct, · 39 unmodified lines | |||
| 227 | int ret; | 228 | int ret; |
| 228 | 229 | ||
| 229 | rtp_exp->tuple.dst.u.udp.port = htons(nated_port); | 230 | rtp_exp->tuple.dst.u.udp.port = htons(nated_port); |
| 230 | ret = nf_ct_expect_related(rtp_exp, 0); | 231 | rtcp_exp->tuple.dst.u.udp.port = htons(nated_port + 1); |
| 232 | ret = nf_ct_expect_related_pair(rtp_pair, 0); | ||
| 231 | if (ret == 0) { | 233 | if (ret == 0) { |
| 232 | rtcp_exp->tuple.dst.u.udp.port = | 234 | break; |
| 233 | htons(nated_port + 1); | 235 | } else if (ret == -EBUSY) { |
| 234 | ret = nf_ct_expect_related(rtcp_exp, 0); | 236 | continue; |
| 235 | if (ret == 0) | 237 | } else if (ret < 0) { |
| 236 | break; | ||
| 237 | else if (ret == -EBUSY) { | ||
| 238 | nf_ct_unexpect_related(rtp_exp); | ||
| 239 | continue; | ||
| 240 | } else if (ret < 0) { | ||
| 241 | nf_ct_unexpect_related(rtp_exp); | ||
| 242 | nated_port = 0; | ||
| 243 | break; | ||
| 244 | } | ||
| 245 | } else if (ret != -EBUSY) { | ||
| 246 | nated_port = 0; | 238 | nated_port = 0; |
| 247 | break; | 239 | break; |
| 248 | } | 240 | } |
| @@ -1597,14 +1597,21 @@ static int fib6_check_nh_list(struct nexthop *old, struct nexthop *new, · 1596 unmodified lines | |||
| 1597 | struct netlink_ext_ack *extack) | 1597 | struct netlink_ext_ack *extack) |
| 1598 | { | 1598 | { |
| 1599 | struct fib6_info *f6i; | 1599 | struct fib6_info *f6i; |
| 1600 | int err = 0; | ||
| 1600 | 1601 | ||
| 1601 | if (list_empty(&old->f6i_list)) | 1602 | if (list_empty(&old->f6i_list)) |
| 1602 | return 0; | 1603 | return 0; |
| 1603 | 1604 | ||
| 1605 | spin_lock_bh(&old->lock); | ||
| 1604 | list_for_each_entry(f6i, &old->f6i_list, nh_list) { | 1606 | list_for_each_entry(f6i, &old->f6i_list, nh_list) { |
| 1605 | if (check_src_addr(&f6i->fib6_src.addr, extack) < 0) | 1607 | err = check_src_addr(&f6i->fib6_src.addr, extack); |
| 1606 | return -EINVAL; | 1608 | if (err) |
| 1609 | break; | ||
| 1607 | } | 1610 | } |
| 1611 | spin_unlock_bh(&old->lock); | ||
| 1612 | |||
| 1613 | if (err) | ||
| 1614 | return err; | ||
| 1608 | 1615 | ||
| 1609 | return fib6_check_nexthop(new, NULL, extack); | 1616 | return fib6_check_nexthop(new, NULL, extack); |
| 1610 | } | 1617 | } |
| @@ -2233,18 +2240,18 @@ static void remove_one_nexthop(struct net *net, struct nexthop *nh, · 622 unmodified lines | |||
| 2233 | static void nh_rt_cache_flush(struct net *net, struct nexthop *nh, | 2240 | static void nh_rt_cache_flush(struct net *net, struct nexthop *nh, |
| 2234 | struct nexthop *replaced_nh) | 2241 | struct nexthop *replaced_nh) |
| 2235 | { | 2242 | { |
| 2236 | struct fib6_info *f6i; | ||
| 2237 | struct nh_group *nhg; | 2243 | struct nh_group *nhg; |
| 2244 | bool have_f6i; | ||
| 2238 | int i; | 2245 | int i; |
| 2239 | 2246 | ||
| 2240 | if (!list_empty(&nh->fi_list)) | 2247 | if (!list_empty(&nh->fi_list)) |
| 2241 | rt_cache_flush(net); | 2248 | rt_cache_flush(net); |
| 2242 | 2249 | ||
| 2243 | list_for_each_entry(f6i, &nh->f6i_list, nh_list) { | 2250 | spin_lock_bh(&nh->lock); |
| 2244 | spin_lock_bh(&f6i->fib6_table->tb6_lock); | 2251 | have_f6i = !list_empty(&nh->f6i_list); |
| 2245 | fib6_update_sernum_upto_root(net, f6i); | 2252 | spin_unlock_bh(&nh->lock); |
| 2246 | spin_unlock_bh(&f6i->fib6_table->tb6_lock); | 2253 | if (have_f6i) |
| 2247 | } | 2254 | rt_genid_bump_ipv6(net); |
| 2248 | 2255 | ||
| 2249 | /* if an IPv6 group was replaced, we have to release all old | 2256 | /* if an IPv6 group was replaced, we have to release all old |
| 2250 | * dsts to make sure all refcounts are released | 2257 | * dsts to make sure all refcounts are released |
| @@ -2538,8 +2545,10 @@ static void __nexthop_replace_notify(struct net *net, struct nexthop *nh, · 287 unmodified lines | |||
| 2538 | fi->nh_updated = false; | 2545 | fi->nh_updated = false; |
| 2539 | } | 2546 | } |
| 2540 | 2547 | ||
| 2548 | spin_lock_bh(&nh->lock); | ||
| 2541 | list_for_each_entry(f6i, &nh->f6i_list, nh_list) | 2549 | list_for_each_entry(f6i, &nh->f6i_list, nh_list) |
| 2542 | fib6_rt_update(net, f6i, info); | 2550 | fib6_rt_update(net, f6i, info); |
| 2551 | spin_unlock_bh(&nh->lock); | ||
| 2543 | } | 2552 | } |
| 2544 | 2553 | ||
| 2545 | /* send RTM_NEWROUTE with REPLACE flag set for all FIB entries | 2554 | /* send RTM_NEWROUTE with REPLACE flag set for all FIB entries |
| @@ -892,8 +892,6 @@ void ip_rt_send_redirect(struct sk_buff *skb) · 891 unmodified lines | |||
| 892 | peer = inet_getpeer_v4(net->ipv4.peers, ip_hdr(skb)->saddr, vif); | 892 | peer = inet_getpeer_v4(net->ipv4.peers, ip_hdr(skb)->saddr, vif); |
| 893 | if (!peer) { | 893 | if (!peer) { |
| 894 | rcu_read_unlock(); | 894 | rcu_read_unlock(); |
| 895 | icmp_send(skb, ICMP_REDIRECT, ICMP_REDIR_HOST, | ||
| 896 | rt_nexthop(rt, ip_hdr(skb)->daddr)); | ||
| 897 | return; | 895 | return; |
| 898 | } | 896 | } |
| 899 | 897 | ||
| @@ -32,13 +32,12 @@ struct udp_tunnel_nic_table_entry { · 31 unmodified lines | |||
| 32 | * @lock: protects all fields | 32 | * @lock: protects all fields |
| 33 | * @need_sync: at least one port start changed | 33 | * @need_sync: at least one port start changed |
| 34 | * @need_replay: space was freed, we need a replay of all ports | 34 | * @need_replay: space was freed, we need a replay of all ports |
| 35 | * @work_pending: @work is currently scheduled | ||
| 36 | * @n_tables: number of tables under @entries | 35 | * @n_tables: number of tables under @entries |
| 37 | * @missed: bitmap of tables which overflown | 36 | * @missed: bitmap of tables which overflown |
| 38 | * @entries: table of tables of ports currently offloaded | 37 | * @entries: table of tables of ports currently offloaded |
| 39 | */ | 38 | */ |
| 40 | struct udp_tunnel_nic { | 39 | struct udp_tunnel_nic { |
| 41 | struct work_struct work; | 40 | struct delayed_work work; |
| 42 | 41 | ||
| 43 | struct net_device *dev; | 42 | struct net_device *dev; |
| 44 | 43 | ||
| @@ -46,7 +45,6 @@ struct udp_tunnel_nic { · 1 unmodified line | |||
| 46 | 45 | ||
| 47 | u8 need_sync:1; | 46 | u8 need_sync:1; |
| 48 | u8 need_replay:1; | 47 | u8 need_replay:1; |
| 49 | u8 work_pending:1; | ||
| 50 | 48 | ||
| 51 | unsigned int n_tables; | 49 | unsigned int n_tables; |
| 52 | unsigned long missed; | 50 | unsigned long missed; |
| @@ -301,11 +299,10 @@ __udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn) · 248 unmodified lines | |||
| 301 | static void | 299 | static void |
| 302 | udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn) | 300 | udp_tunnel_nic_device_sync(struct net_device *dev, struct udp_tunnel_nic *utn) |
| 303 | { | 301 | { |
| 304 | if (!utn->need_sync || utn->work_pending) | 302 | if (!utn->need_sync) |
| 305 | return; | 303 | return; |
| 306 | 304 | ||
| 307 | queue_work(udp_tunnel_nic_workqueue, &utn->work); | 305 | queue_delayed_work(udp_tunnel_nic_workqueue, &utn->work, 0); |
| 308 | utn->work_pending = 1; | ||
| 309 | } | 306 | } |
| 310 | 307 | ||
| 311 | static bool | 308 | static bool |
| @@ -731,12 +728,17 @@ udp_tunnel_nic_replay(struct net_device *dev, struct udp_tunnel_nic *utn) · 419 unmodified lines | |||
| 731 | static void udp_tunnel_nic_device_sync_work(struct work_struct *work) | 728 | static void udp_tunnel_nic_device_sync_work(struct work_struct *work) |
| 732 | { | 729 | { |
| 733 | struct udp_tunnel_nic *utn = | 730 | struct udp_tunnel_nic *utn = |
| 734 | container_of(work, struct udp_tunnel_nic, work); | 731 | container_of(work, struct udp_tunnel_nic, work.work); |
| 735 | 732 | ||
| 736 | rtnl_lock(); | 733 | /* We cannot block on RTNL here, otherwise we would deadlock with |
| 734 | * udp_tunnel_nic_unregister() calling cancel_delayed_work_sync() | ||
| 735 | * while holding RTNL. Requeue with 1 jiffy delay if RTNL is contended. | ||
| 736 | */ | ||
| 737 | if (!rtnl_trylock()) { | ||
| 738 | queue_delayed_work(udp_tunnel_nic_workqueue, &utn->work, 1); | ||
| 739 | return; | ||
| 740 | } | ||
| 737 | mutex_lock(&utn->lock); | 741 | mutex_lock(&utn->lock); |
| 738 | |||
| 739 | utn->work_pending = 0; | ||
| 740 | __udp_tunnel_nic_device_sync(utn->dev, utn); | 742 | __udp_tunnel_nic_device_sync(utn->dev, utn); |
| 741 | 743 | ||
| 742 | if (utn->need_replay) | 744 | if (utn->need_replay) |
| @@ -757,7 +759,7 @@ udp_tunnel_nic_alloc(const struct udp_tunnel_nic_info *info, · 14 unmodified lines | |||
| 757 | if (!utn) | 759 | if (!utn) |
| 758 | return NULL; | 760 | return NULL; |
| 759 | utn->n_tables = n_tables; | 761 | utn->n_tables = n_tables; |
| 760 | INIT_WORK(&utn->work, udp_tunnel_nic_device_sync_work); | 762 | INIT_DELAYED_WORK(&utn->work, udp_tunnel_nic_device_sync_work); |
| 761 | mutex_init(&utn->lock); | 763 | mutex_init(&utn->lock); |
| 762 | 764 | ||
| 763 | for (i = 0; i < n_tables; i++) { | 765 | for (i = 0; i < n_tables; i++) { |
| @@ -901,11 +903,11 @@ udp_tunnel_nic_unregister(struct net_device *dev, struct udp_tunnel_nic *utn) · 137 unmodified lines | |||
| 901 | udp_tunnel_nic_flush(dev, utn); | 903 | udp_tunnel_nic_flush(dev, utn); |
| 902 | udp_tunnel_nic_unlock(dev); | 904 | udp_tunnel_nic_unlock(dev); |
| 903 | 905 | ||
| 904 | /* Wait for the work to be done using the state, netdev core will | 906 | /* Make sure no work is running or queued before freeing @utn. |
| 905 | * retry unregister until we give up our reference on this device. | 907 | * The work handler uses rtnl_trylock(), so it will not deadlock |
| 908 | * against the RTNL we are holding here. | ||
| 906 | */ | 909 | */ |
| 907 | if (utn->work_pending) | 910 | cancel_delayed_work_sync(&utn->work); |
| 908 | return; | ||
| 909 | 911 | ||
| 910 | udp_tunnel_nic_free(utn); | 912 | udp_tunnel_nic_free(utn); |
| 911 | release_dev: | 913 | release_dev: |
| @@ -308,6 +308,7 @@ INDIRECT_CALLABLE_SCOPE bool fib6_rule_suppress(struct fib_rule *rule, · 307 unmodified lines | |||
| 308 | 308 | ||
| 309 | suppress_route: | 309 | suppress_route: |
| 310 | ip6_rt_put_flags(rt, flags); | 310 | ip6_rt_put_flags(rt, flags); |
| 311 | res->rt6 = NULL; | ||
| 311 | return true; | 312 | return true; |
| 312 | } | 313 | } |
| 313 | 314 | ||
| @@ -1494,6 +1494,7 @@ int fib6_add(struct fib6_node *root, struct fib6_info *rt, · 1493 unmodified lines | |||
| 1494 | root, and then (in failure) stale node | 1494 | root, and then (in failure) stale node |
| 1495 | in main tree. | 1495 | in main tree. |
| 1496 | */ | 1496 | */ |
| 1497 | fib6_info_release(info->nl_net->ipv6.fib6_null_entry); | ||
| 1497 | node_free_immediate(info->nl_net, sfn); | 1498 | node_free_immediate(info->nl_net, sfn); |
| 1498 | err = PTR_ERR(sn); | 1499 | err = PTR_ERR(sn); |
| 1499 | goto failure; | 1500 | goto failure; |
| @@ -641,7 +641,7 @@ int ip6_forward(struct sk_buff *skb) · 640 unmodified lines | |||
| 641 | /* Limit redirects both by destination (here) | 641 | /* Limit redirects both by destination (here) |
| 642 | and by source (inside ndisc_send_redirect) | 642 | and by source (inside ndisc_send_redirect) |
| 643 | */ | 643 | */ |
| 644 | if (inet_peer_xrlim_allow(peer, 1*HZ)) | 644 | if (peer && inet_peer_xrlim_allow(peer, 1*HZ)) |
| 645 | ndisc_send_redirect(skb, target); | 645 | ndisc_send_redirect(skb, target); |
| 646 | rcu_read_unlock(); | 646 | rcu_read_unlock(); |
| 647 | } else { | 647 | } else { |
| @@ -1707,6 +1707,8 @@ void ndisc_send_redirect(struct sk_buff *skb, const struct in6_addr *target) · 1706 unmodified lines | |||
| 1707 | } | 1707 | } |
| 1708 | 1708 | ||
| 1709 | peer = inet_getpeer_v6(net->ipv6.peers, &ipv6_hdr(skb)->saddr); | 1709 | peer = inet_getpeer_v6(net->ipv6.peers, &ipv6_hdr(skb)->saddr); |
| 1710 | if (!peer) | ||
| 1711 | goto release; | ||
| 1710 | ret = inet_peer_xrlim_allow(peer, 1*HZ); | 1712 | ret = inet_peer_xrlim_allow(peer, 1*HZ); |
| 1711 | 1713 | ||
| 1712 | if (!ret) | 1714 | if (!ret) |
| @@ -915,6 +915,7 @@ void ieee80211_stop_tx_ba_cb(struct sta_info *sta, int tid, · 914 unmodified lines | |||
| 915 | struct tid_ampdu_tx *tid_tx) | 915 | struct tid_ampdu_tx *tid_tx) |
| 916 | { | 916 | { |
| 917 | struct ieee80211_sub_if_data *sdata = sta->sdata; | 917 | struct ieee80211_sub_if_data *sdata = sta->sdata; |
| 918 | bool ndp = ndp = tid_tx->ndp; | ||
| 918 | bool send_delba = false; | 919 | bool send_delba = false; |
| 919 | bool start_txq = false; | 920 | bool start_txq = false; |
| 920 | 921 | ||
| @@ -934,6 +935,7 @@ void ieee80211_stop_tx_ba_cb(struct sta_info *sta, int tid, · 13 unmodified lines | |||
| 934 | send_delba = true; | 935 | send_delba = true; |
| 935 | 936 | ||
| 936 | ieee80211_remove_tid_tx(sta, tid); | 937 | ieee80211_remove_tid_tx(sta, tid); |
| 938 | /* tid_tx is now invalid since ieee80211_remove_tid_tx() frees it */ | ||
| 937 | start_txq = true; | 939 | start_txq = true; |
| 938 | 940 | ||
| 939 | unlock_sta: | 941 | unlock_sta: |
| @@ -946,7 +948,7 @@ void ieee80211_stop_tx_ba_cb(struct sta_info *sta, int tid, · 6 unmodified lines | |||
| 946 | ieee80211_send_delba(sdata, sta->sta.addr, tid, | 948 | ieee80211_send_delba(sdata, sta->sta.addr, tid, |
| 947 | WLAN_BACK_INITIATOR, | 949 | WLAN_BACK_INITIATOR, |
| 948 | WLAN_REASON_QSTA_NOT_USE, | 950 | WLAN_REASON_QSTA_NOT_USE, |
| 949 | tid_tx->ndp); | 951 | ndp); |
| 950 | } | 952 | } |
| 951 | 953 | ||
| 952 | void ieee80211_stop_tx_ba_cb_irqsafe(struct ieee80211_vif *vif, | 954 | void ieee80211_stop_tx_ba_cb_irqsafe(struct ieee80211_vif *vif, |
| @@ -101,6 +101,10 @@ ieee80211_s1g_rx_twt_setup(struct ieee80211_sub_if_data *sdata, · 100 unmodified lines | |||
| 101 | struct ieee80211_twt_setup *twt = (void *)mgmt->u.action.s1g.variable; | 101 | struct ieee80211_twt_setup *twt = (void *)mgmt->u.action.s1g.variable; |
| 102 | struct ieee80211_twt_params *twt_agrt = (void *)twt->params; | 102 | struct ieee80211_twt_params *twt_agrt = (void *)twt->params; |
| 103 | 103 | ||
| 104 | if (!(twt->control & IEEE80211_TWT_CONTROL_NEG_TYPE_BROADCAST) && | ||
| 105 | twt->length < sizeof(twt->control) + sizeof(*twt_agrt)) | ||
| 106 | return; | ||
| 107 | |||
| 104 | twt_agrt->req_type &= cpu_to_le16(~IEEE80211_TWT_REQTYPE_REQUEST); | 108 | twt_agrt->req_type &= cpu_to_le16(~IEEE80211_TWT_REQTYPE_REQUEST); |
| 105 | 109 | ||
| 106 | /* broadcast TWT not supported yet */ | 110 | /* broadcast TWT not supported yet */ |
| @@ -2539,6 +2539,7 @@ static int mpls_getroute(struct sk_buff *in_skb, struct nlmsghdr *in_nlh, · 2538 unmodified lines | |||
| 2539 | r->rtm_family = AF_MPLS; | 2539 | r->rtm_family = AF_MPLS; |
| 2540 | r->rtm_dst_len = 20; | 2540 | r->rtm_dst_len = 20; |
| 2541 | r->rtm_src_len = 0; | 2541 | r->rtm_src_len = 0; |
| 2542 | r->rtm_tos = 0; | ||
| 2542 | r->rtm_table = RT_TABLE_MAIN; | 2543 | r->rtm_table = RT_TABLE_MAIN; |
| 2543 | r->rtm_type = RTN_UNICAST; | 2544 | r->rtm_type = RTN_UNICAST; |
| 2544 | r->rtm_scope = RT_SCOPE_UNIVERSE; | 2545 | r->rtm_scope = RT_SCOPE_UNIVERSE; |
| @@ -1005,7 +1005,7 @@ overwrite_extensions: · 1004 unmodified lines | |||
| 1005 | #endif | 1005 | #endif |
| 1006 | if (SET_WITH_COUNTER(set)) | 1006 | if (SET_WITH_COUNTER(set)) |
| 1007 | ip_set_init_counter(ext_counter(data, set), ext); | 1007 | ip_set_init_counter(ext_counter(data, set), ext); |
| 1008 | if (SET_WITH_COMMENT(set)) | 1008 | if (SET_WITH_COMMENT(set) && !ext->target) |
| 1009 | ip_set_init_comment(set, ext_comment(data, set), ext); | 1009 | ip_set_init_comment(set, ext_comment(data, set), ext); |
| 1010 | if (SET_WITH_SKBINFO(set)) | 1010 | if (SET_WITH_SKBINFO(set)) |
| 1011 | ip_set_init_skbinfo(ext_skbinfo(data, set), ext); | 1011 | ip_set_init_skbinfo(ext_skbinfo(data, set), ext); |
| @@ -367,7 +367,7 @@ static inline int app_tcp_pkt_out(struct ip_vs_conn *cp, struct sk_buff *skb, · 366 unmodified lines | |||
| 367 | if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) | 367 | if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) |
| 368 | return 0; | 368 | return 0; |
| 369 | 369 | ||
| 370 | th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len); | 370 | th = (struct tcphdr *)(skb->data + ipvsh->len); |
| 371 | 371 | ||
| 372 | /* | 372 | /* |
| 373 | * Remember seq number in case this pkt gets resized | 373 | * Remember seq number in case this pkt gets resized |
| @@ -443,7 +443,7 @@ static inline int app_tcp_pkt_in(struct ip_vs_conn *cp, struct sk_buff *skb, · 69 unmodified lines | |||
| 443 | if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) | 443 | if (skb_ensure_writable(skb, ipvsh->len + sizeof(*th))) |
| 444 | return 0; | 444 | return 0; |
| 445 | 445 | ||
| 446 | th = (struct tcphdr *)(skb_network_header(skb) + ipvsh->len); | 446 | th = (struct tcphdr *)(skb->data + ipvsh->len); |
| 447 | 447 | ||
| 448 | /* | 448 | /* |
| 449 | * Remember seq number in case this pkt gets resized | 449 | * Remember seq number in case this pkt gets resized |
| @@ -70,25 +70,45 @@ static struct kmem_cache *ip_vs_conn_cachep __read_mostly; · 69 unmodified lines | |||
| 70 | * bucket or hash table | 70 | * bucket or hash table |
| 71 | * - hash table resize works like rehash but always rehashes into new table | 71 | * - hash table resize works like rehash but always rehashes into new table |
| 72 | * - bit lock on bucket serializes all operations that modify the chain | 72 | * - bit lock on bucket serializes all operations that modify the chain |
| 73 | * - on resize, bucket from the old table is locked before bucket from the | ||
| 74 | * new table | ||
| 73 | * - cp->lock protects conn fields like cp->flags, cp->dest | 75 | * - cp->lock protects conn fields like cp->flags, cp->dest |
| 74 | */ | 76 | */ |
| 75 | 77 | ||
| 76 | /* Lock conn_tab bucket for conn hash/unhash, not for rehash */ | 78 | /** |
| 79 | * conn_tab_lock - Lock conn_tab buckets for conn hash/unhash, not for rehash | ||
| 80 | * @t: hash table for hn0, new_tbl when new_hash=true | ||
| 81 | * @t2: hash table for hn1, new_tbl when new_hash2=true | ||
| 82 | * @cp: connection | ||
| 83 | * @hash_key: hash key for hn0 | ||
| 84 | * @hash_key2: hash key for hn1 | ||
| 85 | * @use2: using hn1 (double hashing) based on the forwarding method | ||
| 86 | * @new_hash: mode for hn0, hash node (true) or seek node (false) | ||
| 87 | * @new_hash2: mode for hn1, hash node (true) or seek node (false) | ||
| 88 | * @head_ret: returned head for hn0 | ||
| 89 | * @head2_ret: returned head for hn1 | ||
| 90 | * | ||
| 91 | * We support 3 modes: | ||
| 92 | * - seek mode for both nodes, used for unhashing | ||
| 93 | * - hash mode for both nodes, used for hashing | ||
| 94 | * - seek hn0 and hash hn1, used when forwarding method is changed | ||
| 95 | */ | ||
| 77 | static __always_inline void | 96 | static __always_inline void |
| 78 | conn_tab_lock(struct ip_vs_rht *t, struct ip_vs_conn *cp, u32 hash_key, | 97 | conn_tab_lock(struct ip_vs_rht *t, struct ip_vs_rht *t2, struct ip_vs_conn *cp, |
| 79 | u32 hash_key2, bool use2, bool new_hash, | 98 | u32 hash_key, u32 hash_key2, bool use2, bool new_hash, |
| 80 | struct hlist_bl_head **head_ret, struct hlist_bl_head **head2_ret) | 99 | bool new_hash2, struct hlist_bl_head **head_ret, |
| 100 | struct hlist_bl_head **head2_ret) | ||
| 81 | { | 101 | { |
| 82 | struct hlist_bl_head *head, *head2; | 102 | struct hlist_bl_head *head, *head2; |
| 83 | u32 hash_key_new, hash_key_new2; | 103 | u32 hash_key_new, hash_key_new2; |
| 84 | struct ip_vs_rht *t2 = t; | 104 | int idx = 0, idx2 = 0; |
| 85 | u32 idx, idx2; | 105 | |
| 106 | /* Advance idx2 when new_hash is not set but hash_key2 | ||
| 107 | * is for new table | ||
| 108 | */ | ||
| 109 | if (new_hash2 && use2 && t != t2) | ||
| 110 | idx2++; | ||
| 86 | 111 | ||
| 87 | idx = hash_key & t->mask; | ||
| 88 | if (use2) | ||
| 89 | idx2 = hash_key2 & t->mask; | ||
| 90 | else | ||
| 91 | idx2 = idx; | ||
| 92 | if (!new_hash) { | 112 | if (!new_hash) { |
| 93 | /* We need to lock the bucket in the right table */ | 113 | /* We need to lock the bucket in the right table */ |
| 94 | 114 | ||
| @@ -100,46 +120,45 @@ retry: · 5 unmodified lines | |||
| 100 | * both nodes in different tables, use idx/idx2 | 120 | * both nodes in different tables, use idx/idx2 |
| 101 | * for proper lock ordering for heads. | 121 | * for proper lock ordering for heads. |
| 102 | */ | 122 | */ |
| 103 | idx = hash_key & t->mask; | 123 | idx++; |
| 104 | idx |= IP_VS_RHT_TABLE_ID_MASK; | ||
| 105 | } | ||
| 106 | if (use2) { | ||
| 107 | if (!ip_vs_rht_same_table(t2, hash_key2)) { | ||
| 108 | /* It is already moved to new table */ | ||
| 109 | t2 = rcu_dereference(t2->new_tbl); | ||
| 110 | idx2 = hash_key2 & t2->mask; | ||
| 111 | idx2 |= IP_VS_RHT_TABLE_ID_MASK; | ||
| 112 | } | ||
| 113 | } else { | ||
| 114 | idx2 = idx; | ||
| 115 | } | 124 | } |
| 116 | } | 125 | } |
| 126 | if (use2 && !new_hash2 && !ip_vs_rht_same_table(t2, hash_key2)) { | ||
| 127 | /* It is already moved to new table */ | ||
| 128 | t2 = rcu_dereference(t2->new_tbl); | ||
| 129 | idx2++; | ||
| 130 | } | ||
| 117 | 131 | ||
| 132 | if (!use2) | ||
| 133 | idx2 = idx; | ||
| 118 | head = t->buckets + (hash_key & t->mask); | 134 | head = t->buckets + (hash_key & t->mask); |
| 119 | head2 = use2 ? t2->buckets + (hash_key2 & t2->mask) : head; | 135 | head2 = use2 ? t2->buckets + (hash_key2 & t2->mask) : head; |
| 120 | 136 | ||
| 121 | local_bh_disable(); | 137 | if (idx > idx2 || (head > head2 && idx == idx2)) { |
| 122 | /* Do not touch seqcount, this is a safe operation */ | 138 | hlist_bl_lock(head2); |
| 123 | |||
| 124 | if (idx <= idx2) { | ||
| 125 | hlist_bl_lock(head); | 139 | hlist_bl_lock(head); |
| 126 | if (head != head2) | ||
| 127 | hlist_bl_lock(head2); | ||
| 128 | } else { | 140 | } else { |
| 129 | hlist_bl_lock(head2); | ||
| 130 | hlist_bl_lock(head); | 141 | hlist_bl_lock(head); |
| 142 | if (head != head2) | ||
| 143 | hlist_bl_lock(head2); | ||
| 131 | } | 144 | } |
| 132 | if (!new_hash) { | 145 | if (!new_hash) { |
| 146 | bool changed; | ||
| 147 | |||
| 133 | /* Ensure hash_key is read under lock */ | 148 | /* Ensure hash_key is read under lock */ |
| 134 | hash_key_new = READ_ONCE(cp->hn0.hash_key); | 149 | hash_key_new = READ_ONCE(cp->hn0.hash_key); |
| 135 | hash_key_new2 = READ_ONCE(cp->hn1.hash_key); | 150 | changed = hash_key != hash_key_new; |
| 151 | if (use2 && !new_hash2) { | ||
| 152 | hash_key_new2 = READ_ONCE(cp->hn1.hash_key); | ||
| 153 | changed |= hash_key2 != hash_key_new2; | ||
| 154 | } else { | ||
| 155 | hash_key_new2 = hash_key2; | ||
| 156 | } | ||
| 136 | /* Hash changed ? */ | 157 | /* Hash changed ? */ |
| 137 | if (hash_key != hash_key_new || | 158 | if (changed) { |
| 138 | (hash_key2 != hash_key_new2 && use2)) { | ||
| 139 | if (head != head2) | 159 | if (head != head2) |
| 140 | hlist_bl_unlock(head2); | 160 | hlist_bl_unlock(head2); |
| 141 | hlist_bl_unlock(head); | 161 | hlist_bl_unlock(head); |
| 142 | local_bh_enable(); | ||
| 143 | hash_key = hash_key_new; | 162 | hash_key = hash_key_new; |
| 144 | hash_key2 = hash_key_new2; | 163 | hash_key2 = hash_key_new2; |
| 145 | goto retry; | 164 | goto retry; |
| @@ -155,7 +174,6 @@ static inline void conn_tab_unlock(struct hlist_bl_head *head, · 9 unmodified lines | |||
| 155 | if (head != head2) | 174 | if (head != head2) |
| 156 | hlist_bl_unlock(head2); | 175 | hlist_bl_unlock(head2); |
| 157 | hlist_bl_unlock(head); | 176 | hlist_bl_unlock(head); |
| 158 | local_bh_enable(); | ||
| 159 | } | 177 | } |
| 160 | 178 | ||
| 161 | static void ip_vs_conn_expire(struct timer_list *t); | 179 | static void ip_vs_conn_expire(struct timer_list *t); |
| @@ -268,8 +286,9 @@ static inline int ip_vs_conn_hash(struct ip_vs_conn *cp) · 106 unmodified lines | |||
| 268 | use2 = false; | 286 | use2 = false; |
| 269 | } | 287 | } |
| 270 | 288 | ||
| 271 | conn_tab_lock(t, cp, hash_key, hash_key2, use2, true /* new_hash */, | 289 | local_bh_disable(); |
| 272 | &head, &head2); | 290 | conn_tab_lock(t, t, cp, hash_key, hash_key2, use2, true /* new_hash */, |
| 291 | true /* new_hash2 */, &head, &head2); | ||
| 273 | 292 | ||
| 274 | cp->flags |= IP_VS_CONN_F_HASHED; | 293 | cp->flags |= IP_VS_CONN_F_HASHED; |
| 275 | WRITE_ONCE(cp->hn0.hash_key, hash_key); | 294 | WRITE_ONCE(cp->hn0.hash_key, hash_key); |
| @@ -280,6 +299,7 @@ static inline int ip_vs_conn_hash(struct ip_vs_conn *cp) · 4 unmodified lines | |||
| 280 | hlist_bl_add_head_rcu(&cp->hn1.node, head2); | 299 | hlist_bl_add_head_rcu(&cp->hn1.node, head2); |
| 281 | 300 | ||
| 282 | conn_tab_unlock(head, head2); | 301 | conn_tab_unlock(head, head2); |
| 302 | local_bh_enable(); | ||
| 283 | ret = 1; | 303 | ret = 1; |
| 284 | 304 | ||
| 285 | /* Schedule resizing if load increases */ | 305 | /* Schedule resizing if load increases */ |
| @@ -306,18 +326,20 @@ static inline bool ip_vs_conn_unlink(struct ip_vs_conn *cp) · 20 unmodified lines | |||
| 306 | return refcount_dec_if_one(&cp->refcnt); | 326 | return refcount_dec_if_one(&cp->refcnt); |
| 307 | 327 | ||
| 308 | rcu_read_lock(); | 328 | rcu_read_lock(); |
| 329 | local_bh_disable(); | ||
| 309 | 330 | ||
| 310 | t = rcu_dereference(ipvs->conn_tab); | 331 | t = rcu_dereference(ipvs->conn_tab); |
| 311 | hash_key = READ_ONCE(cp->hn0.hash_key); | 332 | hash_key = READ_ONCE(cp->hn0.hash_key); |
| 312 | hash_key2 = READ_ONCE(cp->hn1.hash_key); | 333 | hash_key2 = READ_ONCE(cp->hn1.hash_key); |
| 313 | use2 = ip_vs_conn_use_hash2(cp); | 334 | use2 = ip_vs_conn_use_hash2(cp); |
| 314 | 335 | ||
| 315 | conn_tab_lock(t, cp, hash_key, hash_key2, use2, false /* new_hash */, | 336 | conn_tab_lock(t, t, cp, hash_key, hash_key2, use2, false /* new_hash */, |
| 316 | &head, &head2); | 337 | false /* new_hash2 */, &head, &head2); |
| 317 | 338 | ||
| 318 | if (cp->flags & IP_VS_CONN_F_HASHED) { | 339 | if (cp->flags & IP_VS_CONN_F_HASHED) { |
| 319 | /* Decrease refcnt and unlink conn only if we are last user */ | 340 | /* Decrease refcnt and unlink conn only if we are last user */ |
| 320 | if (refcount_dec_if_one(&cp->refcnt)) { | 341 | if (use2 == ip_vs_conn_use_hash2(cp) && |
| 342 | refcount_dec_if_one(&cp->refcnt)) { | ||
| 321 | hlist_bl_del_rcu(&cp->hn0.node); | 343 | hlist_bl_del_rcu(&cp->hn0.node); |
| 322 | if (use2) | 344 | if (use2) |
| 323 | hlist_bl_del_rcu(&cp->hn1.node); | 345 | hlist_bl_del_rcu(&cp->hn1.node); |
| @@ -328,6 +350,7 @@ static inline bool ip_vs_conn_unlink(struct ip_vs_conn *cp) · 4 unmodified lines | |||
| 328 | 350 | ||
| 329 | conn_tab_unlock(head, head2); | 351 | conn_tab_unlock(head, head2); |
| 330 | 352 | ||
| 353 | local_bh_enable(); | ||
| 331 | rcu_read_unlock(); | 354 | rcu_read_unlock(); |
| 332 | 355 | ||
| 333 | return ret; | 356 | return ret; |
| @@ -632,6 +655,7 @@ void ip_vs_conn_fill_cport(struct ip_vs_conn *cp, __be16 cport) · 298 unmodified lines | |||
| 632 | int ntbl; | 655 | int ntbl; |
| 633 | int dir; | 656 | int dir; |
| 634 | 657 | ||
| 658 | restart: | ||
| 635 | /* No packets from inside, so we can do it in 2 steps. */ | 659 | /* No packets from inside, so we can do it in 2 steps. */ |
| 636 | dir = use2 ? 1 : 0; | 660 | dir = use2 ? 1 : 0; |
| 637 | 661 | ||
| @@ -686,6 +710,23 @@ retry: · 48 unmodified lines | |||
| 686 | /* Protect the cp->flags modification */ | 710 | /* Protect the cp->flags modification */ |
| 687 | spin_lock_bh(&cp->lock); | 711 | spin_lock_bh(&cp->lock); |
| 688 | 712 | ||
| 713 | /* Recheck the forwarding method under lock */ | ||
| 714 | if (use2 != ip_vs_conn_use_hash2(cp)) { | ||
| 715 | use2 = !use2; | ||
| 716 | if (use2) { | ||
| 717 | spin_unlock_bh(&cp->lock); | ||
| 718 | /* Restart with new use2 value */ | ||
| 719 | goto restart; | ||
| 720 | } | ||
| 721 | if (dir) { | ||
| 722 | /* Not started yet, so just skip dir 1 */ | ||
| 723 | spin_unlock_bh(&cp->lock); | ||
| 724 | dir--; | ||
| 725 | goto next_dir; | ||
| 726 | } | ||
| 727 | /* Just finish dir 0 */ | ||
| 728 | } | ||
| 729 | |||
| 689 | /* Lock seqcount only for the old bucket, even if we are on new table | 730 | /* Lock seqcount only for the old bucket, even if we are on new table |
| 690 | * because it affects the del operation, not the adding. | 731 | * because it affects the del operation, not the adding. |
| 691 | */ | 732 | */ |
| @@ -752,6 +793,61 @@ retry: · 60 unmodified lines | |||
| 752 | goto next_dir; | 793 | goto next_dir; |
| 753 | } | 794 | } |
| 754 | 795 | ||
| 796 | /* Change forwarding method for hashed conn */ | ||
| 797 | static void ip_vs_conn_change_fwd_mask(struct ip_vs_conn *cp, u32 new_flags) | ||
| 798 | { | ||
| 799 | struct netns_ipvs *ipvs = cp->ipvs; | ||
| 800 | struct hlist_bl_head *head, *head2; | ||
| 801 | u32 hash2, hash_key, hash_key2; | ||
| 802 | struct ip_vs_rht *t, *t2; | ||
| 803 | |||
| 804 | /* See ip_vs_conn_use_hash2() for reference */ | ||
| 805 | if ((cp->flags & IP_VS_CONN_F_TEMPLATE) || | ||
| 806 | /* No change in double hashing ? */ | ||
| 807 | (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ) == | ||
| 808 | ((new_flags & IP_VS_CONN_F_FWD_MASK) == IP_VS_CONN_F_MASQ)) { | ||
| 809 | cp->flags = new_flags; | ||
| 810 | return; | ||
| 811 | } | ||
| 812 | t = rcu_dereference(ipvs->conn_tab); | ||
| 813 | if (ip_vs_conn_use_hash2(cp)) { | ||
| 814 | /* Stop double hashing */ | ||
| 815 | hash_key = READ_ONCE(cp->hn0.hash_key); | ||
| 816 | hash_key2 = READ_ONCE(cp->hn1.hash_key); | ||
| 817 | |||
| 818 | conn_tab_lock(t, t, cp, hash_key, hash_key2, true /* use2 */, | ||
| 819 | false /* new_hash */, false /* new_hash2 */, | ||
| 820 | &head, &head2); | ||
| 821 | |||
| 822 | /* Keep both hash keys in same table */ | ||
| 823 | hash_key = READ_ONCE(cp->hn0.hash_key); | ||
| 824 | WRITE_ONCE(cp->hn1.hash_key, hash_key); | ||
| 825 | hlist_bl_del_rcu(&cp->hn1.node); | ||
| 826 | cp->flags = new_flags; | ||
| 827 | |||
| 828 | conn_tab_unlock(head, head2); | ||
| 829 | } else { | ||
| 830 | /* Start double hashing */ | ||
| 831 | |||
| 832 | hash_key = READ_ONCE(cp->hn0.hash_key); | ||
| 833 | |||
| 834 | t2 = rcu_dereference(t->new_tbl); | ||
| 835 | hash2 = ip_vs_conn_hashkey_conn(t2, cp, true); | ||
| 836 | hash_key2 = ip_vs_rht_build_hash_key(t2, hash2); | ||
| 837 | |||
| 838 | /* Change the forwarding method under locked hn0 */ | ||
| 839 | conn_tab_lock(t, t2, cp, hash_key, hash_key2, true /* use2 */, | ||
| 840 | false /* new_hash */, true /* new_hash2 */, | ||
| 841 | &head, &head2); | ||
| 842 | |||
| 843 | WRITE_ONCE(cp->hn1.hash_key, hash_key2); | ||
| 844 | cp->flags = new_flags; | ||
| 845 | hlist_bl_add_head_rcu(&cp->hn1.node, head2); | ||
| 846 | |||
| 847 | conn_tab_unlock(head, head2); | ||
| 848 | } | ||
| 849 | } | ||
| 850 | |||
| 755 | /* Get default load factor to map conn_count/u_thresh to t->size */ | 851 | /* Get default load factor to map conn_count/u_thresh to t->size */ |
| 756 | static int ip_vs_conn_default_load_factor(struct netns_ipvs *ipvs) | 852 | static int ip_vs_conn_default_load_factor(struct netns_ipvs *ipvs) |
| 757 | { | 853 | { |
| @@ -1014,6 +1110,9 @@ ip_vs_bind_dest(struct ip_vs_conn *cp, struct ip_vs_dest *dest) · 256 unmodified lines | |||
| 1014 | flags = cp->flags; | 1110 | flags = cp->flags; |
| 1015 | /* Bind with the destination and its corresponding transmitter */ | 1111 | /* Bind with the destination and its corresponding transmitter */ |
| 1016 | if (flags & IP_VS_CONN_F_SYNC) { | 1112 | if (flags & IP_VS_CONN_F_SYNC) { |
| 1113 | /* Synced conns are hashed, so they can not get this flag */ | ||
| 1114 | conn_flags &= ~IP_VS_CONN_F_ONE_PACKET; | ||
| 1115 | |||
| 1017 | /* if the connection is not template and is created | 1116 | /* if the connection is not template and is created |
| 1018 | * by sync, preserve the activity flag. | 1117 | * by sync, preserve the activity flag. |
| 1019 | */ | 1118 | */ |
| @@ -1021,9 +1120,18 @@ ip_vs_bind_dest(struct ip_vs_conn *cp, struct ip_vs_dest *dest) · 1 unmodified line | |||
| 1021 | conn_flags &= ~IP_VS_CONN_F_INACTIVE; | 1120 | conn_flags &= ~IP_VS_CONN_F_INACTIVE; |
| 1022 | /* connections inherit forwarding method from dest */ | 1121 | /* connections inherit forwarding method from dest */ |
| 1023 | flags &= ~(IP_VS_CONN_F_FWD_MASK | IP_VS_CONN_F_NOOUTPUT); | 1122 | flags &= ~(IP_VS_CONN_F_FWD_MASK | IP_VS_CONN_F_NOOUTPUT); |
| 1123 | flags |= conn_flags; | ||
| 1124 | /* Changing forwarding method for hashed conn can | ||
| 1125 | * happen only under locks | ||
| 1126 | */ | ||
| 1127 | if (cp->flags & IP_VS_CONN_F_HASHED) | ||
| 1128 | ip_vs_conn_change_fwd_mask(cp, flags); | ||
| 1129 | else | ||
| 1130 | cp->flags = flags; | ||
| 1131 | } else { | ||
| 1132 | flags |= conn_flags; | ||
| 1133 | cp->flags = flags; | ||
| 1024 | } | 1134 | } |
| 1025 | flags |= conn_flags; | ||
| 1026 | cp->flags = flags; | ||
| 1027 | cp->dest = dest; | 1135 | cp->dest = dest; |
| 1028 | 1136 | ||
| 1029 | IP_VS_DBG_BUF(7, "Bind-dest %s c:%s:%d v:%s:%d " | 1137 | IP_VS_DBG_BUF(7, "Bind-dest %s c:%s:%d v:%s:%d " |
| @@ -867,7 +867,7 @@ static int sysctl_nat_icmp_send(struct netns_ipvs *ipvs) { return 0; } · 866 unmodified lines | |||
| 867 | 867 | ||
| 868 | #endif | 868 | #endif |
| 869 | 869 | ||
| 870 | __sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset) | 870 | static __sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset) |
| 871 | { | 871 | { |
| 872 | return csum_fold(skb_checksum(skb, offset, skb->len - offset, 0)); | 872 | return csum_fold(skb_checksum(skb, offset, skb->len - offset, 0)); |
| 873 | } | 873 | } |
| @@ -924,13 +924,13 @@ static int ip_vs_route_me_harder(struct netns_ipvs *ipvs, int af, · 50 unmodified lines | |||
| 924 | * - inout: 1=in->out, 0=out->in | 924 | * - inout: 1=in->out, 0=out->in |
| 925 | */ | 925 | */ |
| 926 | void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, | 926 | void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 927 | struct ip_vs_conn *cp, int inout) | 927 | struct ip_vs_conn *cp, int inout, unsigned int toff, |
| 928 | bool has_ports) | ||
| 928 | { | 929 | { |
| 929 | struct iphdr *iph = ip_hdr(skb); | 930 | struct iphdr *iph = ip_hdr(skb); |
| 930 | unsigned int icmp_offset = iph->ihl*4; | 931 | struct icmphdr *icmph = (struct icmphdr *)(skb->data + toff); |
| 931 | struct icmphdr *icmph = (struct icmphdr *)(skb_network_header(skb) + | ||
| 932 | icmp_offset); | ||
| 933 | struct iphdr *ciph = (struct iphdr *)(icmph + 1); | 932 | struct iphdr *ciph = (struct iphdr *)(icmph + 1); |
| 933 | unsigned int coff __maybe_unused = toff + sizeof(struct icmphdr); | ||
| 934 | 934 | ||
| 935 | if (inout) { | 935 | if (inout) { |
| 936 | iph->saddr = cp->vaddr.ip; | 936 | iph->saddr = cp->vaddr.ip; |
| @@ -945,8 +945,7 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, · 8 unmodified lines | |||
| 945 | } | 945 | } |
| 946 | 946 | ||
| 947 | /* the TCP/UDP/SCTP port */ | 947 | /* the TCP/UDP/SCTP port */ |
| 948 | if (IPPROTO_TCP == ciph->protocol || IPPROTO_UDP == ciph->protocol || | 948 | if (has_ports) { |
| 949 | IPPROTO_SCTP == ciph->protocol) { | ||
| 950 | __be16 *ports = (void *)ciph + ciph->ihl*4; | 949 | __be16 *ports = (void *)ciph + ciph->ihl*4; |
| 951 | 950 | ||
| 952 | if (inout) | 951 | if (inout) |
| @@ -957,48 +956,40 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, · 4 unmodified lines | |||
| 957 | 956 | ||
| 958 | /* And finally the ICMP checksum */ | 957 | /* And finally the ICMP checksum */ |
| 959 | icmph->checksum = 0; | 958 | icmph->checksum = 0; |
| 960 | icmph->checksum = ip_vs_checksum_complete(skb, icmp_offset); | 959 | icmph->checksum = ip_vs_checksum_complete(skb, toff); |
| 961 | skb->ip_summed = CHECKSUM_UNNECESSARY; | 960 | skb->ip_summed = CHECKSUM_UNNECESSARY; |
| 962 | 961 | ||
| 963 | if (inout) | 962 | if (inout) |
| 964 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, (void *)ciph - (void *)iph, | 963 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, |
| 965 | "Forwarding altered outgoing ICMP"); | 964 | "Forwarding altered outgoing ICMP"); |
| 966 | else | 965 | else |
| 967 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, (void *)ciph - (void *)iph, | 966 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, |
| 968 | "Forwarding altered incoming ICMP"); | 967 | "Forwarding altered incoming ICMP"); |
| 969 | } | 968 | } |
| 970 | 969 | ||
| 971 | #ifdef CONFIG_IP_VS_IPV6 | 970 | #ifdef CONFIG_IP_VS_IPV6 |
| 972 | void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, | 971 | void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 973 | struct ip_vs_conn *cp, int inout) | 972 | struct ip_vs_conn *cp, int inout, unsigned int toff, |
| 973 | bool has_ports, struct ip_vs_iphdr *ciph) | ||
| 974 | { | 974 | { |
| 975 | struct ipv6hdr *iph = ipv6_hdr(skb); | 975 | struct ipv6hdr *iph = ipv6_hdr(skb); |
| 976 | unsigned int icmp_offset = 0; | ||
| 977 | unsigned int offs = 0; /* header offset*/ | ||
| 978 | int protocol; | ||
| 979 | struct icmp6hdr *icmph; | 976 | struct icmp6hdr *icmph; |
| 980 | struct ipv6hdr *ciph; | 977 | struct ipv6hdr *cih; |
| 981 | unsigned short fragoffs; | ||
| 982 | 978 | ||
| 983 | ipv6_find_hdr(skb, &icmp_offset, IPPROTO_ICMPV6, &fragoffs, NULL); | 979 | icmph = (struct icmp6hdr *)(skb->data + toff); |
| 984 | icmph = (struct icmp6hdr *)(skb_network_header(skb) + icmp_offset); | 980 | cih = (struct ipv6hdr *)(skb->data + ciph->off); |
| 985 | offs = icmp_offset + sizeof(struct icmp6hdr); | ||
| 986 | ciph = (struct ipv6hdr *)(skb_network_header(skb) + offs); | ||
| 987 | |||
| 988 | protocol = ipv6_find_hdr(skb, &offs, -1, &fragoffs, NULL); | ||
| 989 | 981 | ||
| 990 | if (inout) { | 982 | if (inout) { |
| 991 | iph->saddr = cp->vaddr.in6; | 983 | iph->saddr = cp->vaddr.in6; |
| 992 | ciph->daddr = cp->vaddr.in6; | 984 | cih->daddr = cp->vaddr.in6; |
| 993 | } else { | 985 | } else { |
| 994 | iph->daddr = cp->daddr.in6; | 986 | iph->daddr = cp->daddr.in6; |
| 995 | ciph->saddr = cp->daddr.in6; | 987 | cih->saddr = cp->daddr.in6; |
| 996 | } | 988 | } |
| 997 | 989 | ||
| 998 | /* the TCP/UDP/SCTP port */ | 990 | /* the TCP/UDP/SCTP port */ |
| 999 | if (!fragoffs && (IPPROTO_TCP == protocol || IPPROTO_UDP == protocol || | 991 | if (has_ports) { |
| 1000 | IPPROTO_SCTP == protocol)) { | 992 | __be16 *ports = (void *)(skb->data + ciph->len); |
| 1001 | __be16 *ports = (void *)(skb_network_header(skb) + offs); | ||
| 1002 | 993 | ||
| 1003 | IP_VS_DBG(11, "%s() changed port %d to %d\n", __func__, | 994 | IP_VS_DBG(11, "%s() changed port %d to %d\n", __func__, |
| 1004 | ntohs(inout ? ports[1] : ports[0]), | 995 | ntohs(inout ? ports[1] : ports[0]), |
| @@ -1011,19 +1002,17 @@ void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, · 6 unmodified lines | |||
| 1011 | 1002 | ||
| 1012 | /* And finally the ICMP checksum */ | 1003 | /* And finally the ICMP checksum */ |
| 1013 | icmph->icmp6_cksum = ~csum_ipv6_magic(&iph->saddr, &iph->daddr, | 1004 | icmph->icmp6_cksum = ~csum_ipv6_magic(&iph->saddr, &iph->daddr, |
| 1014 | skb->len - icmp_offset, | 1005 | skb->len - toff, |
| 1015 | IPPROTO_ICMPV6, 0); | 1006 | IPPROTO_ICMPV6, 0); |
| 1016 | skb->csum_start = skb_network_header(skb) - skb->head + icmp_offset; | 1007 | skb->csum_start = skb_headroom(skb) + toff; |
| 1017 | skb->csum_offset = offsetof(struct icmp6hdr, icmp6_cksum); | 1008 | skb->csum_offset = offsetof(struct icmp6hdr, icmp6_cksum); |
| 1018 | skb->ip_summed = CHECKSUM_PARTIAL; | 1009 | skb->ip_summed = CHECKSUM_PARTIAL; |
| 1019 | 1010 | ||
| 1020 | if (inout) | 1011 | if (inout) |
| 1021 | IP_VS_DBG_PKT(11, AF_INET6, pp, skb, | 1012 | IP_VS_DBG_PKT(11, AF_INET6, pp, skb, ciph->off, |
| 1022 | (void *)ciph - (void *)iph, | ||
| 1023 | "Forwarding altered outgoing ICMPv6"); | 1013 | "Forwarding altered outgoing ICMPv6"); |
| 1024 | else | 1014 | else |
| 1025 | IP_VS_DBG_PKT(11, AF_INET6, pp, skb, | 1015 | IP_VS_DBG_PKT(11, AF_INET6, pp, skb, ciph->off, |
| 1026 | (void *)ciph - (void *)iph, | ||
| 1027 | "Forwarding altered incoming ICMPv6"); | 1016 | "Forwarding altered incoming ICMPv6"); |
| 1028 | } | 1017 | } |
| 1029 | #endif | 1018 | #endif |
| @@ -1033,36 +1022,41 @@ void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, · 3 unmodified lines | |||
| 1033 | */ | 1022 | */ |
| 1034 | static int handle_response_icmp(int af, struct sk_buff *skb, | 1023 | static int handle_response_icmp(int af, struct sk_buff *skb, |
| 1035 | union nf_inet_addr *snet, | 1024 | union nf_inet_addr *snet, |
| 1036 | __u8 protocol, struct ip_vs_conn *cp, | 1025 | struct ip_vs_conn *cp, |
| 1037 | struct ip_vs_protocol *pp, | 1026 | struct ip_vs_protocol *pp, |
| 1038 | unsigned int offset, unsigned int ihl, | 1027 | struct ip_vs_iphdr *ciph, |
| 1039 | unsigned int hooknum) | 1028 | unsigned int toff, unsigned int hooknum) |
| 1040 | { | 1029 | { |
| 1030 | int iproto = af == AF_INET6 ? IPPROTO_ICMPV6 : IPPROTO_ICMP; | ||
| 1041 | unsigned int verdict = NF_DROP; | 1031 | unsigned int verdict = NF_DROP; |
| 1032 | unsigned int ctoff = ciph->len; | ||
| 1033 | bool has_ports = false; | ||
| 1042 | 1034 | ||
| 1043 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) | 1035 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) |
| 1044 | goto after_nat; | 1036 | goto after_nat; |
| 1045 | 1037 | ||
| 1046 | /* Ensure the checksum is correct */ | 1038 | /* Ensure the checksum is correct */ |
| 1047 | if (!skb_csum_unnecessary(skb) && ip_vs_checksum_complete(skb, ihl)) { | 1039 | if (!ip_vs_checksum_common_check(skb, toff, iproto, af)) { |
| 1048 | /* Failed checksum! */ | 1040 | /* Failed checksum! */ |
| 1049 | IP_VS_DBG_BUF(1, "Forward ICMP: failed checksum from %s!\n", | 1041 | IP_VS_DBG_BUF(1, "Forward ICMP: failed checksum from %s!\n", |
| 1050 | IP_VS_DBG_ADDR(af, snet)); | 1042 | IP_VS_DBG_ADDR(af, snet)); |
| 1051 | goto out; | 1043 | goto out; |
| 1052 | } | 1044 | } |
| 1053 | 1045 | ||
| 1054 | if (IPPROTO_TCP == protocol || IPPROTO_UDP == protocol || | 1046 | if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || |
| 1055 | IPPROTO_SCTP == protocol) | 1047 | ciph->protocol == IPPROTO_SCTP) { |
| 1056 | offset += 2 * sizeof(__u16); | 1048 | ctoff += 2 * sizeof(__u16); |
| 1057 | if (skb_ensure_writable(skb, offset)) | 1049 | has_ports = true; |
| 1050 | } | ||
| 1051 | if (skb_ensure_writable(skb, ctoff)) | ||
| 1058 | goto out; | 1052 | goto out; |
| 1059 | 1053 | ||
| 1060 | #ifdef CONFIG_IP_VS_IPV6 | 1054 | #ifdef CONFIG_IP_VS_IPV6 |
| 1061 | if (af == AF_INET6) | 1055 | if (af == AF_INET6) |
| 1062 | ip_vs_nat_icmp_v6(skb, pp, cp, 1); | 1056 | ip_vs_nat_icmp_v6(skb, pp, cp, 1, toff, has_ports, ciph); |
| 1063 | else | 1057 | else |
| 1064 | #endif | 1058 | #endif |
| 1065 | ip_vs_nat_icmp(skb, pp, cp, 1); | 1059 | ip_vs_nat_icmp(skb, pp, cp, 1, toff, has_ports); |
| 1066 | 1060 | ||
| 1067 | if (ip_vs_route_me_harder(cp->ipvs, af, skb, hooknum)) | 1061 | if (ip_vs_route_me_harder(cp->ipvs, af, skb, hooknum)) |
| 1068 | goto out; | 1062 | goto out; |
| @@ -1090,9 +1084,9 @@ out: · 21 unmodified lines | |||
| 1090 | * Currently handles error types - unreachable, quench, ttl exceeded. | 1084 | * Currently handles error types - unreachable, quench, ttl exceeded. |
| 1091 | */ | 1085 | */ |
| 1092 | static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, | 1086 | static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, |
| 1093 | int *related, unsigned int hooknum) | 1087 | int *related, unsigned int hooknum, |
| 1088 | struct ip_vs_iphdr *ipvsh) | ||
| 1094 | { | 1089 | { |
| 1095 | struct iphdr *iph; | ||
| 1096 | struct icmphdr _icmph, *ic; | 1090 | struct icmphdr _icmph, *ic; |
| 1097 | struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ | 1091 | struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ |
| 1098 | struct ip_vs_iphdr ciph; | 1092 | struct ip_vs_iphdr ciph; |
| @@ -1107,17 +1101,19 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 8 unmodified lines | |||
| 1107 | if (ip_is_fragment(ip_hdr(skb))) { | 1101 | if (ip_is_fragment(ip_hdr(skb))) { |
| 1108 | if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) | 1102 | if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) |
| 1109 | return NF_STOLEN; | 1103 | return NF_STOLEN; |
| 1104 | if (!ip_vs_fill_iph_skb(AF_INET, skb, false, ipvsh)) | ||
| 1105 | return NF_ACCEPT; | ||
| 1110 | } | 1106 | } |
| 1111 | 1107 | ||
| 1112 | iph = ip_hdr(skb); | 1108 | ihl = ipvsh->len; |
| 1113 | offset = ihl = iph->ihl * 4; | 1109 | offset = ipvsh->len; |
| 1114 | ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); | 1110 | ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); |
| 1115 | if (ic == NULL) | 1111 | if (ic == NULL) |
| 1116 | return NF_DROP; | 1112 | return NF_DROP; |
| 1117 | 1113 | ||
| 1118 | IP_VS_DBG(12, "Outgoing ICMP (%d,%d) %pI4->%pI4\n", | 1114 | IP_VS_DBG(12, "Outgoing ICMP (%d,%d) %pI4->%pI4\n", |
| 1119 | ic->type, ntohs(icmp_id(ic)), | 1115 | ic->type, ntohs(icmp_id(ic)), |
| 1120 | &iph->saddr, &iph->daddr); | 1116 | &ipvsh->saddr.ip, &ipvsh->daddr.ip); |
| 1121 | 1117 | ||
| 1122 | /* | 1118 | /* |
| 1123 | * Work through seeing if this is for us. | 1119 | * Work through seeing if this is for us. |
| @@ -1136,7 +1132,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 12 unmodified lines | |||
| 1136 | /* Now find the contained IP header */ | 1132 | /* Now find the contained IP header */ |
| 1137 | offset += sizeof(_icmph); | 1133 | offset += sizeof(_icmph); |
| 1138 | cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); | 1134 | cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); |
| 1139 | if (cih == NULL) | 1135 | if (!(cih && cih->version == 4 && cih->ihl >= 5)) |
| 1140 | return NF_ACCEPT; /* The packet looks wrong, ignore */ | 1136 | return NF_ACCEPT; /* The packet looks wrong, ignore */ |
| 1141 | 1137 | ||
| 1142 | pp = ip_vs_proto_get(cih->protocol); | 1138 | pp = ip_vs_proto_get(cih->protocol); |
| @@ -1144,8 +1140,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 1 unmodified line | |||
| 1144 | return NF_ACCEPT; | 1140 | return NF_ACCEPT; |
| 1145 | 1141 | ||
| 1146 | /* Is the embedded protocol header present? */ | 1142 | /* Is the embedded protocol header present? */ |
| 1147 | if (unlikely(cih->frag_off & htons(IP_OFFSET) && | 1143 | if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) |
| 1148 | pp->dont_defrag)) | ||
| 1149 | return NF_ACCEPT; | 1144 | return NF_ACCEPT; |
| 1150 | 1145 | ||
| 1151 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, | 1146 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, |
| @@ -1159,9 +1154,9 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, · 7 unmodified lines | |||
| 1159 | if (!cp) | 1154 | if (!cp) |
| 1160 | return NF_ACCEPT; | 1155 | return NF_ACCEPT; |
| 1161 | 1156 | ||
| 1162 | snet.ip = iph->saddr; | 1157 | snet.ip = ipvsh->saddr.ip; |
| 1163 | return handle_response_icmp(AF_INET, skb, &snet, cih->protocol, cp, | 1158 | return handle_response_icmp(AF_INET, skb, &snet, cp, pp, &ciph, ihl, |
| 1164 | pp, ciph.len, ihl, hooknum); | 1159 | hooknum); |
| 1165 | } | 1160 | } |
| 1166 | 1161 | ||
| 1167 | #ifdef CONFIG_IP_VS_IPV6 | 1162 | #ifdef CONFIG_IP_VS_IPV6 |
| @@ -1174,7 +1169,6 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 6 unmodified lines | |||
| 1174 | struct ip_vs_conn *cp; | 1169 | struct ip_vs_conn *cp; |
| 1175 | struct ip_vs_protocol *pp; | 1170 | struct ip_vs_protocol *pp; |
| 1176 | union nf_inet_addr snet; | 1171 | union nf_inet_addr snet; |
| 1177 | unsigned int offset; | ||
| 1178 | 1172 | ||
| 1179 | *related = 1; | 1173 | *related = 1; |
| 1180 | ic = frag_safe_skb_hp(skb, ipvsh->len, sizeof(_icmph), &_icmph); | 1174 | ic = frag_safe_skb_hp(skb, ipvsh->len, sizeof(_icmph), &_icmph); |
| @@ -1210,6 +1204,10 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 29 unmodified lines | |||
| 1210 | if (!pp) | 1204 | if (!pp) |
| 1211 | return NF_ACCEPT; | 1205 | return NF_ACCEPT; |
| 1212 | 1206 | ||
| 1207 | /* Is the embedded protocol header present? */ | ||
| 1208 | if (unlikely(ciph.fragoffs && !pp->dont_defrag)) | ||
| 1209 | return NF_ACCEPT; | ||
| 1210 | |||
| 1213 | /* The embedded headers contain source and dest in reverse order */ | 1211 | /* The embedded headers contain source and dest in reverse order */ |
| 1214 | cp = INDIRECT_CALL_1(pp->conn_out_get, ip_vs_conn_out_get_proto, | 1212 | cp = INDIRECT_CALL_1(pp->conn_out_get, ip_vs_conn_out_get_proto, |
| 1215 | ipvs, AF_INET6, skb, &ciph); | 1213 | ipvs, AF_INET6, skb, &ciph); |
| @@ -1217,9 +1215,8 @@ static int ip_vs_out_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 1 unmodified line | |||
| 1217 | return NF_ACCEPT; | 1215 | return NF_ACCEPT; |
| 1218 | 1216 | ||
| 1219 | snet.in6 = ciph.saddr.in6; | 1217 | snet.in6 = ciph.saddr.in6; |
| 1220 | offset = ciph.len; | 1218 | return handle_response_icmp(AF_INET6, skb, &snet, cp, pp, &ciph, |
| 1221 | return handle_response_icmp(AF_INET6, skb, &snet, ciph.protocol, cp, | 1219 | ipvsh->len, hooknum); |
| 1222 | pp, offset, ipvsh->len, hooknum); | ||
| 1223 | } | 1220 | } |
| 1224 | #endif | 1221 | #endif |
| 1225 | 1222 | ||
| @@ -1545,7 +1542,8 @@ ip_vs_out_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *stat · 319 unmodified lines | |||
| 1545 | #endif | 1542 | #endif |
| 1546 | if (unlikely(iph.protocol == IPPROTO_ICMP)) { | 1543 | if (unlikely(iph.protocol == IPPROTO_ICMP)) { |
| 1547 | int related; | 1544 | int related; |
| 1548 | int verdict = ip_vs_out_icmp(ipvs, skb, &related, hooknum); | 1545 | int verdict = ip_vs_out_icmp(ipvs, skb, &related, |
| 1546 | hooknum, &iph); | ||
| 1549 | 1547 | ||
| 1550 | if (related) | 1548 | if (related) |
| 1551 | return verdict; | 1549 | return verdict; |
| @@ -1753,9 +1751,8 @@ unk: · 201 unmodified lines | |||
| 1753 | */ | 1751 | */ |
| 1754 | static int | 1752 | static int |
| 1755 | ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, | 1753 | ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, |
| 1756 | unsigned int hooknum) | 1754 | unsigned int hooknum, struct ip_vs_iphdr *iph) |
| 1757 | { | 1755 | { |
| 1758 | struct iphdr *iph; | ||
| 1759 | struct icmphdr _icmph, *ic; | 1756 | struct icmphdr _icmph, *ic; |
| 1760 | struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ | 1757 | struct iphdr _ciph, *cih; /* The ip header contained within the ICMP */ |
| 1761 | struct ip_vs_iphdr ciph; | 1758 | struct ip_vs_iphdr ciph; |
| @@ -1765,7 +1762,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 3 unmodified lines | |||
| 1765 | unsigned int offset, offset2, ihl, verdict; | 1762 | unsigned int offset, offset2, ihl, verdict; |
| 1766 | bool tunnel, new_cp = false; | 1763 | bool tunnel, new_cp = false; |
| 1767 | union nf_inet_addr *raddr; | 1764 | union nf_inet_addr *raddr; |
| 1768 | char *outer_proto = "IPIP"; | 1765 | char *outer_proto __maybe_unused = "IPIP"; |
| 1769 | unsigned int hlen_ipip; | 1766 | unsigned int hlen_ipip; |
| 1770 | int ulen = 0; | 1767 | int ulen = 0; |
| 1771 | 1768 | ||
| @@ -1775,17 +1772,19 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 3 unmodified lines | |||
| 1775 | if (ip_is_fragment(ip_hdr(skb))) { | 1772 | if (ip_is_fragment(ip_hdr(skb))) { |
| 1776 | if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) | 1773 | if (ip_vs_gather_frags(ipvs, skb, ip_vs_defrag_user(hooknum))) |
| 1777 | return NF_STOLEN; | 1774 | return NF_STOLEN; |
| 1775 | if (!ip_vs_fill_iph_skb(AF_INET, skb, false, iph)) | ||
| 1776 | return NF_ACCEPT; | ||
| 1778 | } | 1777 | } |
| 1779 | 1778 | ||
| 1780 | iph = ip_hdr(skb); | 1779 | ihl = iph->len; |
| 1781 | offset = ihl = iph->ihl * 4; | 1780 | offset = iph->len; |
| 1782 | ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); | 1781 | ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); |
| 1783 | if (ic == NULL) | 1782 | if (ic == NULL) |
| 1784 | return NF_DROP; | 1783 | return NF_DROP; |
| 1785 | 1784 | ||
| 1786 | IP_VS_DBG(12, "Incoming ICMP (%d,%d) %pI4->%pI4\n", | 1785 | IP_VS_DBG(12, "Incoming ICMP (%d,%d) %pI4->%pI4\n", |
| 1787 | ic->type, ntohs(icmp_id(ic)), | 1786 | ic->type, ntohs(icmp_id(ic)), |
| 1788 | &iph->saddr, &iph->daddr); | 1787 | &iph->saddr.ip, &iph->daddr.ip); |
| 1789 | 1788 | ||
| 1790 | /* | 1789 | /* |
| 1791 | * Work through seeing if this is for us. | 1790 | * Work through seeing if this is for us. |
| @@ -1867,8 +1866,7 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 75 unmodified lines | |||
| 1867 | pp = pd->pp; | 1866 | pp = pd->pp; |
| 1868 | 1867 | ||
| 1869 | /* Is the embedded protocol header present? */ | 1868 | /* Is the embedded protocol header present? */ |
| 1870 | if (unlikely(cih->frag_off & htons(IP_OFFSET) && | 1869 | if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) |
| 1871 | pp->dont_defrag)) | ||
| 1872 | return NF_ACCEPT; | 1870 | return NF_ACCEPT; |
| 1873 | 1871 | ||
| 1874 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, | 1872 | IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, |
| @@ -1876,7 +1874,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 1 unmodified line | |||
| 1876 | 1874 | ||
| 1877 | offset2 = offset; | 1875 | offset2 = offset; |
| 1878 | ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph); | 1876 | ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph); |
| 1879 | offset = ciph.len; | ||
| 1880 | 1877 | ||
| 1881 | /* The embedded headers contain source and dest in reverse order. | 1878 | /* The embedded headers contain source and dest in reverse order. |
| 1882 | * For IPIP/UDP/GRE tunnel this is error for request, not for reply. | 1879 | * For IPIP/UDP/GRE tunnel this is error for request, not for reply. |
| @@ -1898,10 +1895,11 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, · 15 unmodified lines | |||
| 1898 | verdict = NF_DROP; | 1895 | verdict = NF_DROP; |
| 1899 | 1896 | ||
| 1900 | /* Ensure the checksum is correct */ | 1897 | /* Ensure the checksum is correct */ |
| 1901 | if (!skb_csum_unnecessary(skb) && ip_vs_checksum_complete(skb, ihl)) { | 1898 | if ((IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ || tunnel) && |
| 1899 | !ip_vs_checksum_common_check(skb, ihl, IPPROTO_ICMP, AF_INET)) { | ||
| 1902 | /* Failed checksum! */ | 1900 | /* Failed checksum! */ |
| 1903 | IP_VS_DBG(1, "Incoming ICMP: failed checksum from %pI4!\n", | 1901 | IP_VS_DBG(1, "Incoming ICMP: failed checksum from %pI4!\n", |
| 1904 | &iph->saddr); | 1902 | &iph->saddr.ip); |
| 1905 | goto out; | 1903 | goto out; |
| 1906 | } | 1904 | } |
| 1907 | 1905 | ||
| @@ -1969,10 +1967,7 @@ ignore_tunnel: · 61 unmodified lines | |||
| 1969 | 1967 | ||
| 1970 | /* do the statistics and put it back */ | 1968 | /* do the statistics and put it back */ |
| 1971 | ip_vs_in_stats(cp, skb); | 1969 | ip_vs_in_stats(cp, skb); |
| 1972 | if (IPPROTO_TCP == cih->protocol || IPPROTO_UDP == cih->protocol || | 1970 | verdict = ip_vs_icmp_xmit(skb, cp, pp, iph->len, hooknum, &ciph); |
| 1973 | IPPROTO_SCTP == cih->protocol) | ||
| 1974 | offset += 2 * sizeof(__u16); | ||
| 1975 | verdict = ip_vs_icmp_xmit(skb, cp, pp, offset, hooknum, &ciph); | ||
| 1976 | 1971 | ||
| 1977 | out: | 1972 | out: |
| 1978 | if (likely(!new_cp)) | 1973 | if (likely(!new_cp)) |
| @@ -2032,8 +2027,8 @@ static int ip_vs_in_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 53 unmodified lines | |||
| 2032 | return NF_ACCEPT; | 2027 | return NF_ACCEPT; |
| 2033 | pp = pd->pp; | 2028 | pp = pd->pp; |
| 2034 | 2029 | ||
| 2035 | /* Cannot handle fragmented embedded protocol */ | 2030 | /* Is the embedded protocol header present? */ |
| 2036 | if (ciph.fragoffs) | 2031 | if (ciph.fragoffs && !pp->dont_defrag) |
| 2037 | return NF_ACCEPT; | 2032 | return NF_ACCEPT; |
| 2038 | 2033 | ||
| 2039 | IP_VS_DBG_PKT(11, AF_INET6, pp, skb, offset, | 2034 | IP_VS_DBG_PKT(11, AF_INET6, pp, skb, offset, |
| @@ -2057,23 +2052,22 @@ static int ip_vs_in_icmp_v6(struct netns_ipvs *ipvs, struct sk_buff *skb, · 17 unmodified lines | |||
| 2057 | new_cp = true; | 2052 | new_cp = true; |
| 2058 | } | 2053 | } |
| 2059 | 2054 | ||
| 2060 | /* VS/TUN, VS/DR and LOCALNODE just let it go */ | 2055 | verdict = NF_DROP; |
| 2061 | if ((hooknum == NF_INET_LOCAL_OUT) && | 2056 | |
| 2062 | (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ)) { | 2057 | /* Ensure the checksum is correct */ |
| 2063 | verdict = NF_ACCEPT; | 2058 | if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ && |
| 2059 | !ip_vs_checksum_common_check(skb, iph->len, IPPROTO_ICMPV6, | ||
| 2060 | AF_INET6)) { | ||
| 2061 | /* Failed checksum! */ | ||
| 2062 | IP_VS_DBG(1, "Incoming ICMPv6: failed checksum from %pI6c!\n", | ||
| 2063 | &iph->saddr); | ||
| 2064 | goto out; | 2064 | goto out; |
| 2065 | } | 2065 | } |
| 2066 | 2066 | ||
| 2067 | /* do the statistics and put it back */ | 2067 | /* do the statistics and put it back */ |
| 2068 | ip_vs_in_stats(cp, skb); | 2068 | ip_vs_in_stats(cp, skb); |
| 2069 | 2069 | ||
| 2070 | /* Need to mangle contained IPv6 header in ICMPv6 packet */ | 2070 | verdict = ip_vs_icmp_xmit_v6(skb, cp, pp, iph->len, hooknum, &ciph); |
| 2071 | offset = ciph.len; | ||
| 2072 | if (IPPROTO_TCP == ciph.protocol || IPPROTO_UDP == ciph.protocol || | ||
| 2073 | IPPROTO_SCTP == ciph.protocol) | ||
| 2074 | offset += 2 * sizeof(__u16); /* Also mangle ports */ | ||
| 2075 | |||
| 2076 | verdict = ip_vs_icmp_xmit_v6(skb, cp, pp, offset, hooknum, &ciph); | ||
| 2077 | 2071 | ||
| 2078 | out: | 2072 | out: |
| 2079 | if (likely(!new_cp)) | 2073 | if (likely(!new_cp)) |
| @@ -2152,7 +2146,7 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state · 72 unmodified lines | |||
| 2152 | if (unlikely(iph.protocol == IPPROTO_ICMP)) { | 2146 | if (unlikely(iph.protocol == IPPROTO_ICMP)) { |
| 2153 | int related; | 2147 | int related; |
| 2154 | int verdict = ip_vs_in_icmp(ipvs, skb, &related, | 2148 | int verdict = ip_vs_in_icmp(ipvs, skb, &related, |
| 2155 | hooknum); | 2149 | hooknum, &iph); |
| 2156 | 2150 | ||
| 2157 | if (related) | 2151 | if (related) |
| 2158 | return verdict; | 2152 | return verdict; |
| @@ -2200,8 +2194,11 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state · 41 unmodified lines | |||
| 2200 | } | 2194 | } |
| 2201 | 2195 | ||
| 2202 | if (resched) { | 2196 | if (resched) { |
| 2203 | if (!old_ct) | 2197 | if (!old_ct) { |
| 2198 | spin_lock_bh(&cp->lock); | ||
| 2204 | cp->flags &= ~IP_VS_CONN_F_NFCT; | 2199 | cp->flags &= ~IP_VS_CONN_F_NFCT; |
| 2200 | spin_unlock_bh(&cp->lock); | ||
| 2201 | } | ||
| 2205 | if (!atomic_read(&cp->n_control)) | 2202 | if (!atomic_read(&cp->n_control)) |
| 2206 | ip_vs_conn_expire_now(cp); | 2203 | ip_vs_conn_expire_now(cp); |
| 2207 | __ip_vs_conn_put(cp); | 2204 | __ip_vs_conn_put(cp); |
| @@ -2217,8 +2214,11 @@ ip_vs_in_hook(void *priv, struct sk_buff *skb, const struct nf_hook_state *state · 9 unmodified lines | |||
| 2217 | if (sysctl_expire_nodest_conn(ipvs)) { | 2214 | if (sysctl_expire_nodest_conn(ipvs)) { |
| 2218 | bool old_ct = ip_vs_conn_uses_old_conntrack(cp, skb); | 2215 | bool old_ct = ip_vs_conn_uses_old_conntrack(cp, skb); |
| 2219 | 2216 | ||
| 2220 | if (!old_ct) | 2217 | if (!old_ct) { |
| 2218 | spin_lock_bh(&cp->lock); | ||
| 2221 | cp->flags &= ~IP_VS_CONN_F_NFCT; | 2219 | cp->flags &= ~IP_VS_CONN_F_NFCT; |
| 2220 | spin_unlock_bh(&cp->lock); | ||
| 2221 | } | ||
| 2222 | 2222 | ||
| 2223 | ip_vs_conn_expire_now(cp); | 2223 | ip_vs_conn_expire_now(cp); |
| 2224 | __ip_vs_conn_put(cp); | 2224 | __ip_vs_conn_put(cp); |
| @@ -2288,6 +2288,7 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, · 63 unmodified lines | |||
| 2288 | const struct nf_hook_state *state) | 2288 | const struct nf_hook_state *state) |
| 2289 | { | 2289 | { |
| 2290 | struct netns_ipvs *ipvs = net_ipvs(state->net); | 2290 | struct netns_ipvs *ipvs = net_ipvs(state->net); |
| 2291 | struct ip_vs_iphdr iphdr; | ||
| 2291 | int r; | 2292 | int r; |
| 2292 | 2293 | ||
| 2293 | /* ipvs enabled in this netns ? */ | 2294 | /* ipvs enabled in this netns ? */ |
| @@ -2297,10 +2298,9 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, · 3 unmodified lines | |||
| 2297 | if (state->pf == NFPROTO_IPV4) { | 2298 | if (state->pf == NFPROTO_IPV4) { |
| 2298 | if (ip_hdr(skb)->protocol != IPPROTO_ICMP) | 2299 | if (ip_hdr(skb)->protocol != IPPROTO_ICMP) |
| 2299 | return NF_ACCEPT; | 2300 | return NF_ACCEPT; |
| 2301 | ip_vs_fill_iph_skb(AF_INET, skb, false, &iphdr); | ||
| 2300 | #ifdef CONFIG_IP_VS_IPV6 | 2302 | #ifdef CONFIG_IP_VS_IPV6 |
| 2301 | } else { | 2303 | } else { |
| 2302 | struct ip_vs_iphdr iphdr; | ||
| 2303 | |||
| 2304 | ip_vs_fill_iph_skb(AF_INET6, skb, false, &iphdr); | 2304 | ip_vs_fill_iph_skb(AF_INET6, skb, false, &iphdr); |
| 2305 | 2305 | ||
| 2306 | if (iphdr.protocol != IPPROTO_ICMPV6) | 2306 | if (iphdr.protocol != IPPROTO_ICMPV6) |
| @@ -2310,7 +2310,7 @@ ip_vs_forward_icmp(void *priv, struct sk_buff *skb, · 3 unmodified lines | |||
| 2310 | #endif | 2310 | #endif |
| 2311 | } | 2311 | } |
| 2312 | 2312 | ||
| 2313 | return ip_vs_in_icmp(ipvs, skb, &r, state->hook); | 2313 | return ip_vs_in_icmp(ipvs, skb, &r, state->hook, &iphdr); |
| 2314 | } | 2314 | } |
| 2315 | 2315 | ||
| 2316 | static const struct nf_hook_ops ip_vs_ops4[] = { | 2316 | static const struct nf_hook_ops ip_vs_ops4[] = { |
| @@ -11,7 +11,7 @@ · 10 unmodified lines | |||
| 11 | 11 | ||
| 12 | static int | 12 | static int |
| 13 | sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, | 13 | sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 14 | unsigned int sctphoff); | 14 | struct ip_vs_iphdr *iph); |
| 15 | 15 | ||
| 16 | static int | 16 | static int |
| 17 | sctp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, | 17 | sctp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, |
| @@ -109,7 +109,7 @@ sctp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 91 unmodified lines | |||
| 109 | int ret; | 109 | int ret; |
| 110 | 110 | ||
| 111 | /* Some checks before mangling */ | 111 | /* Some checks before mangling */ |
| 112 | if (!sctp_csum_check(cp->af, skb, pp, sctphoff)) | 112 | if (!sctp_csum_check(cp->af, skb, pp, iph)) |
| 113 | return 0; | 113 | return 0; |
| 114 | 114 | ||
| 115 | /* Call application helper if needed */ | 115 | /* Call application helper if needed */ |
| @@ -121,7 +121,7 @@ sctp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 5 unmodified lines | |||
| 121 | payload_csum = true; | 121 | payload_csum = true; |
| 122 | } | 122 | } |
| 123 | 123 | ||
| 124 | sctph = (void *) skb_network_header(skb) + sctphoff; | 124 | sctph = (void *)skb->data + sctphoff; |
| 125 | 125 | ||
| 126 | /* Only update csum if we really have to */ | 126 | /* Only update csum if we really have to */ |
| 127 | if (sctph->source != cp->vport || payload_csum || | 127 | if (sctph->source != cp->vport || payload_csum || |
| @@ -157,7 +157,7 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 29 unmodified lines | |||
| 157 | int ret; | 157 | int ret; |
| 158 | 158 | ||
| 159 | /* Some checks before mangling */ | 159 | /* Some checks before mangling */ |
| 160 | if (!sctp_csum_check(cp->af, skb, pp, sctphoff)) | 160 | if (!sctp_csum_check(cp->af, skb, pp, iph)) |
| 161 | return 0; | 161 | return 0; |
| 162 | 162 | ||
| 163 | /* Call application helper if needed */ | 163 | /* Call application helper if needed */ |
| @@ -169,7 +169,7 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 5 unmodified lines | |||
| 169 | payload_csum = true; | 169 | payload_csum = true; |
| 170 | } | 170 | } |
| 171 | 171 | ||
| 172 | sctph = (void *) skb_network_header(skb) + sctphoff; | 172 | sctph = (void *)skb->data + sctphoff; |
| 173 | 173 | ||
| 174 | /* Only update csum if we really have to */ | 174 | /* Only update csum if we really have to */ |
| 175 | if (sctph->dest != cp->dport || payload_csum || | 175 | if (sctph->dest != cp->dport || payload_csum || |
| @@ -187,19 +187,22 @@ sctp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 11 unmodified lines | |||
| 187 | 187 | ||
| 188 | static int | 188 | static int |
| 189 | sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, | 189 | sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 190 | unsigned int sctphoff) | 190 | struct ip_vs_iphdr *iph) |
| 191 | { | 191 | { |
| 192 | unsigned int sctphoff = iph->len; | ||
| 192 | struct sctphdr *sh; | 193 | struct sctphdr *sh; |
| 193 | __le32 cmp, val; | 194 | __le32 cmp, val; |
| 194 | 195 | ||
| 196 | if (!ip_vs_checksum_needed(skb, af)) | ||
| 197 | return 1; | ||
| 195 | sh = (struct sctphdr *)(skb->data + sctphoff); | 198 | sh = (struct sctphdr *)(skb->data + sctphoff); |
| 196 | cmp = sh->checksum; | 199 | cmp = sh->checksum; |
| 197 | val = sctp_compute_cksum(skb, sctphoff); | 200 | val = sctp_compute_cksum(skb, sctphoff); |
| 198 | 201 | ||
| 199 | if (val != cmp) { | 202 | if (val != cmp) { |
| 200 | /* CRC failure, dump it. */ | 203 | /* CRC failure, dump it. */ |
| 201 | IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | 204 | IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, |
| 202 | "Failed checksum for"); | 205 | "Failed checksum for"); |
| 203 | return 0; | 206 | return 0; |
| 204 | } | 207 | } |
| 205 | return 1; | 208 | return 1; |
| @@ -29,7 +29,7 @@ · 28 unmodified lines | |||
| 29 | 29 | ||
| 30 | static int | 30 | static int |
| 31 | tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, | 31 | tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 32 | unsigned int tcphoff); | 32 | struct ip_vs_iphdr *iph); |
| 33 | 33 | ||
| 34 | static int | 34 | static int |
| 35 | tcp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, | 35 | tcp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, |
| @@ -166,7 +166,7 @@ tcp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 130 unmodified lines | |||
| 166 | int ret; | 166 | int ret; |
| 167 | 167 | ||
| 168 | /* Some checks before mangling */ | 168 | /* Some checks before mangling */ |
| 169 | if (!tcp_csum_check(cp->af, skb, pp, tcphoff)) | 169 | if (!tcp_csum_check(cp->af, skb, pp, iph)) |
| 170 | return 0; | 170 | return 0; |
| 171 | 171 | ||
| 172 | /* Call application helper if needed */ | 172 | /* Call application helper if needed */ |
| @@ -179,7 +179,7 @@ tcp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 6 unmodified lines | |||
| 179 | payload_csum = true; | 179 | payload_csum = true; |
| 180 | } | 180 | } |
| 181 | 181 | ||
| 182 | tcph = (void *)skb_network_header(skb) + tcphoff; | 182 | tcph = (void *)skb->data + tcphoff; |
| 183 | tcph->source = cp->vport; | 183 | tcph->source = cp->vport; |
| 184 | 184 | ||
| 185 | /* Adjust TCP checksums */ | 185 | /* Adjust TCP checksums */ |
| @@ -244,7 +244,7 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 58 unmodified lines | |||
| 244 | int ret; | 244 | int ret; |
| 245 | 245 | ||
| 246 | /* Some checks before mangling */ | 246 | /* Some checks before mangling */ |
| 247 | if (!tcp_csum_check(cp->af, skb, pp, tcphoff)) | 247 | if (!tcp_csum_check(cp->af, skb, pp, iph)) |
| 248 | return 0; | 248 | return 0; |
| 249 | 249 | ||
| 250 | /* | 250 | /* |
| @@ -260,7 +260,7 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 9 unmodified lines | |||
| 260 | payload_csum = true; | 260 | payload_csum = true; |
| 261 | } | 261 | } |
| 262 | 262 | ||
| 263 | tcph = (void *)skb_network_header(skb) + tcphoff; | 263 | tcph = (void *)skb->data + tcphoff; |
| 264 | tcph->dest = cp->dport; | 264 | tcph->dest = cp->dport; |
| 265 | 265 | ||
| 266 | /* | 266 | /* |
| @@ -302,41 +302,13 @@ tcp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 35 unmodified lines | |||
| 302 | 302 | ||
| 303 | static int | 303 | static int |
| 304 | tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, | 304 | tcp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 305 | unsigned int tcphoff) | 305 | struct ip_vs_iphdr *iph) |
| 306 | { | 306 | { |
| 307 | switch (skb->ip_summed) { | 307 | if (!ip_vs_checksum_common_check(skb, iph->len, IPPROTO_TCP, af)) { |
| 308 | case CHECKSUM_NONE: | 308 | IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, |
| 309 | skb->csum = skb_checksum(skb, tcphoff, skb->len - tcphoff, 0); | 309 | "Failed checksum for"); |
| 310 | fallthrough; | 310 | return 0; |
| 311 | case CHECKSUM_COMPLETE: | ||
| 312 | #ifdef CONFIG_IP_VS_IPV6 | ||
| 313 | if (af == AF_INET6) { | ||
| 314 | if (csum_ipv6_magic(&ipv6_hdr(skb)->saddr, | ||
| 315 | &ipv6_hdr(skb)->daddr, | ||
| 316 | skb->len - tcphoff, | ||
| 317 | IPPROTO_TCP, | ||
| 318 | skb->csum)) { | ||
| 319 | IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | ||
| 320 | "Failed checksum for"); | ||
| 321 | return 0; | ||
| 322 | } | ||
| 323 | } else | ||
| 324 | #endif | ||
| 325 | if (csum_tcpudp_magic(ip_hdr(skb)->saddr, | ||
| 326 | ip_hdr(skb)->daddr, | ||
| 327 | skb->len - tcphoff, | ||
| 328 | ip_hdr(skb)->protocol, | ||
| 329 | skb->csum)) { | ||
| 330 | IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | ||
| 331 | "Failed checksum for"); | ||
| 332 | return 0; | ||
| 333 | } | ||
| 334 | break; | ||
| 335 | default: | ||
| 336 | /* No need to checksum. */ | ||
| 337 | break; | ||
| 338 | } | 311 | } |
| 339 | |||
| 340 | return 1; | 312 | return 1; |
| 341 | } | 313 | } |
| 342 | 314 | ||
| @@ -25,7 +25,7 @@ · 24 unmodified lines | |||
| 25 | 25 | ||
| 26 | static int | 26 | static int |
| 27 | udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, | 27 | udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 28 | unsigned int udphoff); | 28 | struct ip_vs_iphdr *iph); |
| 29 | 29 | ||
| 30 | static int | 30 | static int |
| 31 | udp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, | 31 | udp_conn_schedule(struct netns_ipvs *ipvs, int af, struct sk_buff *skb, |
| @@ -155,7 +155,7 @@ udp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 123 unmodified lines | |||
| 155 | int ret; | 155 | int ret; |
| 156 | 156 | ||
| 157 | /* Some checks before mangling */ | 157 | /* Some checks before mangling */ |
| 158 | if (!udp_csum_check(cp->af, skb, pp, udphoff)) | 158 | if (!udp_csum_check(cp->af, skb, pp, iph)) |
| 159 | return 0; | 159 | return 0; |
| 160 | 160 | ||
| 161 | /* | 161 | /* |
| @@ -170,7 +170,7 @@ udp_snat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 8 unmodified lines | |||
| 170 | payload_csum = true; | 170 | payload_csum = true; |
| 171 | } | 171 | } |
| 172 | 172 | ||
| 173 | udph = (void *)skb_network_header(skb) + udphoff; | 173 | udph = (void *)skb->data + udphoff; |
| 174 | udph->source = cp->vport; | 174 | udph->source = cp->vport; |
| 175 | 175 | ||
| 176 | /* | 176 | /* |
| @@ -238,7 +238,7 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 61 unmodified lines | |||
| 238 | int ret; | 238 | int ret; |
| 239 | 239 | ||
| 240 | /* Some checks before mangling */ | 240 | /* Some checks before mangling */ |
| 241 | if (!udp_csum_check(cp->af, skb, pp, udphoff)) | 241 | if (!udp_csum_check(cp->af, skb, pp, iph)) |
| 242 | return 0; | 242 | return 0; |
| 243 | 243 | ||
| 244 | /* | 244 | /* |
| @@ -254,7 +254,7 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 9 unmodified lines | |||
| 254 | payload_csum = true; | 254 | payload_csum = true; |
| 255 | } | 255 | } |
| 256 | 256 | ||
| 257 | udph = (void *)skb_network_header(skb) + udphoff; | 257 | udph = (void *)skb->data + udphoff; |
| 258 | udph->dest = cp->dport; | 258 | udph->dest = cp->dport; |
| 259 | 259 | ||
| 260 | /* | 260 | /* |
| @@ -298,48 +298,20 @@ udp_dnat_handler(struct sk_buff *skb, struct ip_vs_protocol *pp, · 37 unmodified lines | |||
| 298 | 298 | ||
| 299 | static int | 299 | static int |
| 300 | udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, | 300 | udp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, |
| 301 | unsigned int udphoff) | 301 | struct ip_vs_iphdr *iph) |
| 302 | { | 302 | { |
| 303 | struct udphdr _udph, *uh; | 303 | struct udphdr _udph, *uh; |
| 304 | 304 | ||
| 305 | uh = skb_header_pointer(skb, udphoff, sizeof(_udph), &_udph); | 305 | uh = skb_header_pointer(skb, iph->len, sizeof(_udph), &_udph); |
| 306 | if (uh == NULL) | 306 | if (uh == NULL) |
| 307 | return 0; | 307 | return 0; |
| 308 | 308 | ||
| 309 | if (uh->check != 0) { | 309 | if (!uh->check) |
| 310 | switch (skb->ip_summed) { | 310 | return 1; |
| 311 | case CHECKSUM_NONE: | 311 | if (!ip_vs_checksum_common_check(skb, iph->len, IPPROTO_UDP, af)) { |
| 312 | skb->csum = skb_checksum(skb, udphoff, | 312 | IP_VS_DBG_RL_PKT(0, af, pp, skb, iph->off, |
| 313 | skb->len - udphoff, 0); | 313 | "Failed checksum for"); |
| 314 | fallthrough; | 314 | return 0; |
| 315 | case CHECKSUM_COMPLETE: | ||
| 316 | #ifdef CONFIG_IP_VS_IPV6 | ||
| 317 | if (af == AF_INET6) { | ||
| 318 | if (csum_ipv6_magic(&ipv6_hdr(skb)->saddr, | ||
| 319 | &ipv6_hdr(skb)->daddr, | ||
| 320 | skb->len - udphoff, | ||
| 321 | IPPROTO_UDP, | ||
| 322 | skb->csum)) { | ||
| 323 | IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | ||
| 324 | "Failed checksum for"); | ||
| 325 | return 0; | ||
| 326 | } | ||
| 327 | } else | ||
| 328 | #endif | ||
| 329 | if (csum_tcpudp_magic(ip_hdr(skb)->saddr, | ||
| 330 | ip_hdr(skb)->daddr, | ||
| 331 | skb->len - udphoff, | ||
| 332 | ip_hdr(skb)->protocol, | ||
| 333 | skb->csum)) { | ||
| 334 | IP_VS_DBG_RL_PKT(0, af, pp, skb, 0, | ||
| 335 | "Failed checksum for"); | ||
| 336 | return 0; | ||
| 337 | } | ||
| 338 | break; | ||
| 339 | default: | ||
| 340 | /* No need to checksum. */ | ||
| 341 | break; | ||
| 342 | } | ||
| 343 | } | 315 | } |
| 344 | return 1; | 316 | return 1; |
| 345 | } | 317 | } |
| @@ -1502,20 +1502,22 @@ tx_error: · 1501 unmodified lines | |||
| 1502 | */ | 1502 | */ |
| 1503 | int | 1503 | int |
| 1504 | ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, | 1504 | ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1505 | struct ip_vs_protocol *pp, int offset, unsigned int hooknum, | 1505 | struct ip_vs_protocol *pp, unsigned int toff, |
| 1506 | struct ip_vs_iphdr *iph) | 1506 | unsigned int hooknum, struct ip_vs_iphdr *ciph) |
| 1507 | { | 1507 | { |
| 1508 | struct rtable *rt; /* Route to the other host */ | 1508 | struct rtable *rt; /* Route to the other host */ |
| 1509 | int rc; | 1509 | int rc; |
| 1510 | int local; | 1510 | int local; |
| 1511 | int rt_mode, was_input; | 1511 | int rt_mode, was_input; |
| 1512 | bool has_ports = false; | ||
| 1513 | unsigned int wlen; | ||
| 1512 | 1514 | ||
| 1513 | /* The ICMP packet for VS/TUN, VS/DR and LOCALNODE will be | 1515 | /* The ICMP packet for VS/TUN, VS/DR and LOCALNODE will be |
| 1514 | forwarded directly here, because there is no need to | 1516 | forwarded directly here, because there is no need to |
| 1515 | translate address/port back */ | 1517 | translate address/port back */ |
| 1516 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { | 1518 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { |
| 1517 | if (cp->packet_xmit) | 1519 | if (cp->packet_xmit) |
| 1518 | rc = cp->packet_xmit(skb, cp, pp, iph); | 1520 | rc = cp->packet_xmit(skb, cp, pp, ciph); |
| 1519 | else | 1521 | else |
| 1520 | rc = NF_ACCEPT; | 1522 | rc = NF_ACCEPT; |
| 1521 | /* do not touch skb anymore */ | 1523 | /* do not touch skb anymore */ |
| @@ -1533,7 +1535,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 11 unmodified lines | |||
| 1533 | IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | | 1535 | IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | |
| 1534 | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; | 1536 | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; |
| 1535 | local = __ip_vs_get_out_rt(cp->ipvs, cp->af, skb, cp->dest, cp->daddr.ip, rt_mode, | 1537 | local = __ip_vs_get_out_rt(cp->ipvs, cp->af, skb, cp->dest, cp->daddr.ip, rt_mode, |
| 1536 | NULL, iph); | 1538 | NULL, ciph); |
| 1537 | if (local < 0) | 1539 | if (local < 0) |
| 1538 | goto tx_error; | 1540 | goto tx_error; |
| 1539 | rt = skb_rtable(skb); | 1541 | rt = skb_rtable(skb); |
| @@ -1564,14 +1566,21 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 24 unmodified lines | |||
| 1564 | goto tx_error; | 1566 | goto tx_error; |
| 1565 | } | 1567 | } |
| 1566 | 1568 | ||
| 1569 | wlen = ciph->len; | ||
| 1570 | if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || | ||
| 1571 | ciph->protocol == IPPROTO_SCTP) { | ||
| 1572 | wlen += 2 * sizeof(__u16); /* Also mangle ports */ | ||
| 1573 | has_ports = true; | ||
| 1574 | } | ||
| 1575 | |||
| 1567 | /* copy-on-write the packet before mangling it */ | 1576 | /* copy-on-write the packet before mangling it */ |
| 1568 | if (skb_ensure_writable(skb, offset)) | 1577 | if (skb_ensure_writable(skb, wlen)) |
| 1569 | goto tx_error; | 1578 | goto tx_error; |
| 1570 | 1579 | ||
| 1571 | if (skb_cow(skb, rt->dst.dev->hard_header_len)) | 1580 | if (skb_cow(skb, rt->dst.dev->hard_header_len)) |
| 1572 | goto tx_error; | 1581 | goto tx_error; |
| 1573 | 1582 | ||
| 1574 | ip_vs_nat_icmp(skb, pp, cp, 0); | 1583 | ip_vs_nat_icmp(skb, pp, cp, 0, toff, has_ports); |
| 1575 | 1584 | ||
| 1576 | /* Another hack: avoid icmp_send in ip_fragment */ | 1585 | /* Another hack: avoid icmp_send in ip_fragment */ |
| 1577 | skb->ignore_df = 1; | 1586 | skb->ignore_df = 1; |
| @@ -1587,10 +1596,12 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, · 9 unmodified lines | |||
| 1587 | #ifdef CONFIG_IP_VS_IPV6 | 1596 | #ifdef CONFIG_IP_VS_IPV6 |
| 1588 | int | 1597 | int |
| 1589 | ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, | 1598 | ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, |
| 1590 | struct ip_vs_protocol *pp, int offset, unsigned int hooknum, | 1599 | struct ip_vs_protocol *pp, unsigned int toff, |
| 1591 | struct ip_vs_iphdr *ipvsh) | 1600 | unsigned int hooknum, struct ip_vs_iphdr *ciph) |
| 1592 | { | 1601 | { |
| 1602 | bool has_ports = false; | ||
| 1593 | struct rt6_info *rt; /* Route to the other host */ | 1603 | struct rt6_info *rt; /* Route to the other host */ |
| 1604 | unsigned int wlen; | ||
| 1594 | int rc; | 1605 | int rc; |
| 1595 | int local; | 1606 | int local; |
| 1596 | int rt_mode; | 1607 | int rt_mode; |
| @@ -1600,7 +1611,7 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 3 unmodified lines | |||
| 1600 | translate address/port back */ | 1611 | translate address/port back */ |
| 1601 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { | 1612 | if (IP_VS_FWD_METHOD(cp) != IP_VS_CONN_F_MASQ) { |
| 1602 | if (cp->packet_xmit) | 1613 | if (cp->packet_xmit) |
| 1603 | rc = cp->packet_xmit(skb, cp, pp, ipvsh); | 1614 | rc = cp->packet_xmit(skb, cp, pp, ciph); |
| 1604 | else | 1615 | else |
| 1605 | rc = NF_ACCEPT; | 1616 | rc = NF_ACCEPT; |
| 1606 | /* do not touch skb anymore */ | 1617 | /* do not touch skb anymore */ |
| @@ -1617,7 +1628,7 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 10 unmodified lines | |||
| 1617 | IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | | 1628 | IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | |
| 1618 | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; | 1629 | IP_VS_RT_MODE_RDR : IP_VS_RT_MODE_NON_LOCAL; |
| 1619 | local = __ip_vs_get_out_rt_v6(cp->ipvs, cp->af, skb, cp->dest, | 1630 | local = __ip_vs_get_out_rt_v6(cp->ipvs, cp->af, skb, cp->dest, |
| 1620 | &cp->daddr.in6, NULL, ipvsh, 0, rt_mode); | 1631 | &cp->daddr.in6, NULL, ciph, 0, rt_mode); |
| 1621 | if (local < 0) | 1632 | if (local < 0) |
| 1622 | goto tx_error; | 1633 | goto tx_error; |
| 1623 | rt = dst_rt6_info(skb_dst(skb)); | 1634 | rt = dst_rt6_info(skb_dst(skb)); |
| @@ -1648,14 +1659,21 @@ ip_vs_icmp_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp, · 24 unmodified lines | |||
| 1648 | goto tx_error; | 1659 | goto tx_error; |
| 1649 | } | 1660 | } |
| 1650 | 1661 | ||
| 1662 | wlen = ciph->len; | ||
| 1663 | if (ciph->protocol == IPPROTO_TCP || ciph->protocol == IPPROTO_UDP || | ||
| 1664 | ciph->protocol == IPPROTO_SCTP) { | ||
| 1665 | wlen += 2 * sizeof(__u16); /* Also mangle ports */ | ||
| 1666 | has_ports = true; | ||
| 1667 | } | ||
| 1668 | |||
| 1651 | /* copy-on-write the packet before mangling it */ | 1669 | /* copy-on-write the packet before mangling it */ |
| 1652 | if (skb_ensure_writable(skb, offset)) | 1670 | if (skb_ensure_writable(skb, wlen)) |
| 1653 | goto tx_error; | 1671 | goto tx_error; |
| 1654 | 1672 | ||
| 1655 | if (skb_cow(skb, rt->dst.dev->hard_header_len)) | 1673 | if (skb_cow(skb, rt->dst.dev->hard_header_len)) |
| 1656 | goto tx_error; | 1674 | goto tx_error; |
| 1657 | 1675 | ||
| 1658 | ip_vs_nat_icmp_v6(skb, pp, cp, 0); | 1676 | ip_vs_nat_icmp_v6(skb, pp, cp, 0, toff, has_ports, ciph); |
| 1659 | 1677 | ||
| 1660 | /* Another hack: avoid icmp_send in ip_fragment */ | 1678 | /* Another hack: avoid icmp_send in ip_fragment */ |
| 1661 | skb->ignore_df = 1; | 1679 | skb->ignore_df = 1; |
| @@ -427,7 +427,6 @@ static void nf_ct_expect_insert(struct nf_conntrack_expect *exp, · 426 unmodified lines | |||
| 427 | exp->timeout += helper->expect_policy[exp->class].timeout * HZ; | 427 | exp->timeout += helper->expect_policy[exp->class].timeout * HZ; |
| 428 | 428 | ||
| 429 | hlist_add_head_rcu(&exp->lnode, &master_help->expectations); | 429 | hlist_add_head_rcu(&exp->lnode, &master_help->expectations); |
| 430 | master_help->expecting[exp->class]++; | ||
| 431 | 430 | ||
| 432 | hlist_add_head_rcu(&exp->hnode, &nf_ct_expect_hash[h]); | 431 | hlist_add_head_rcu(&exp->hnode, &nf_ct_expect_hash[h]); |
| 433 | cnet = nf_ct_pernet(net); | 432 | cnet = nf_ct_pernet(net); |
| @@ -534,6 +533,7 @@ int nf_ct_expect_related_report(struct nf_conntrack_expect *expect, · 100 unmodified lines | |||
| 534 | if (ret < 0) | 533 | if (ret < 0) |
| 535 | goto out; | 534 | goto out; |
| 536 | 535 | ||
| 536 | master_help->expecting[expect->class]++; | ||
| 537 | nf_ct_expect_insert(expect, master_help); | 537 | nf_ct_expect_insert(expect, master_help); |
| 538 | 538 | ||
| 539 | nf_ct_expect_event_report(IPEXP_NEW, expect, portid, report); | 539 | nf_ct_expect_event_report(IPEXP_NEW, expect, portid, report); |
| @@ -546,6 +546,39 @@ out: · 6 unmodified lines | |||
| 546 | } | 546 | } |
| 547 | EXPORT_SYMBOL_GPL(nf_ct_expect_related_report); | 547 | EXPORT_SYMBOL_GPL(nf_ct_expect_related_report); |
| 548 | 548 | ||
| 549 | int nf_ct_expect_related_pair(struct nf_conntrack_expect *expect[], | ||
| 550 | unsigned int flags) | ||
| 551 | { | ||
| 552 | struct nf_conn_help *master_help; | ||
| 553 | int i, ret; | ||
| 554 | |||
| 555 | spin_lock_bh(&nf_conntrack_expect_lock); | ||
| 556 | master_help = nfct_help(expect[0]->master); | ||
| 557 | if (!master_help || master_help != nfct_help(expect[1]->master)) { | ||
| 558 | ret = -EINVAL; | ||
| 559 | goto out; | ||
| 560 | } | ||
| 561 | |||
| 562 | for (i = 0; i < 2; i++) { | ||
| 563 | ret = __nf_ct_expect_check(expect[i], master_help, flags); | ||
| 564 | if (ret < 0) { | ||
| 565 | if (i == 1) | ||
| 566 | master_help->expecting[expect[0]->class]--; | ||
| 567 | goto out; | ||
| 568 | } | ||
| 569 | master_help->expecting[expect[i]->class]++; | ||
| 570 | } | ||
| 571 | |||
| 572 | for (i = 0; i < 2; i++) { | ||
| 573 | nf_ct_expect_insert(expect[i], master_help); | ||
| 574 | nf_ct_expect_event_report(IPEXP_NEW, expect[i], 0, 0); | ||
| 575 | } | ||
| 576 | out: | ||
| 577 | spin_unlock_bh(&nf_conntrack_expect_lock); | ||
| 578 | return ret; | ||
| 579 | } | ||
| 580 | EXPORT_SYMBOL_GPL(nf_ct_expect_related_pair); | ||
| 581 | |||
| 549 | void nf_ct_expect_iterate_destroy(bool (*iter)(struct nf_conntrack_expect *e, void *data), | 582 | void nf_ct_expect_iterate_destroy(bool (*iter)(struct nf_conntrack_expect *e, void *data), |
| 550 | void *data) | 583 | void *data) |
| 551 | { | 584 | { |
| @@ -1663,7 +1663,7 @@ static int sip_help_tcp(struct sk_buff *skb, unsigned int protoff, · 1662 unmodified lines | |||
| 1663 | unsigned int matchoff, matchlen; | 1663 | unsigned int matchoff, matchlen; |
| 1664 | unsigned int msglen, origlen; | 1664 | unsigned int msglen, origlen; |
| 1665 | const char *dptr, *end; | 1665 | const char *dptr, *end; |
| 1666 | s16 diff, tdiff = 0; | 1666 | s32 diff, tdiff = 0; |
| 1667 | int ret = NF_ACCEPT; | 1667 | int ret = NF_ACCEPT; |
| 1668 | unsigned long clen; | 1668 | unsigned long clen; |
| 1669 | bool term; | 1669 | bool term; |
| @@ -321,7 +321,7 @@ next: · 320 unmodified lines | |||
| 321 | } | 321 | } |
| 322 | 322 | ||
| 323 | static void nf_nat_sip_seq_adjust(struct sk_buff *skb, unsigned int protoff, | 323 | static void nf_nat_sip_seq_adjust(struct sk_buff *skb, unsigned int protoff, |
| 324 | s16 off) | 324 | s32 off) |
| 325 | { | 325 | { |
| 326 | enum ip_conntrack_info ctinfo; | 326 | enum ip_conntrack_info ctinfo; |
| 327 | struct nf_conn *ct = nf_ct_get(skb, &ctinfo); | 327 | struct nf_conn *ct = nf_ct_get(skb, &ctinfo); |
| @@ -592,6 +592,7 @@ static unsigned int nf_nat_sdp_media(struct sk_buff *skb, unsigned int protoff, · 264 unmodified lines | |||
| 592 | unsigned int medialen, | 592 | unsigned int medialen, |
| 593 | union nf_inet_addr *rtp_addr) | 593 | union nf_inet_addr *rtp_addr) |
| 594 | { | 594 | { |
| 595 | struct nf_conntrack_expect *rtp_pair[2] = { rtp_exp, rtcp_exp }; | ||
| 595 | enum ip_conntrack_info ctinfo; | 596 | enum ip_conntrack_info ctinfo; |
| 596 | struct nf_conn *ct = nf_ct_get(skb, &ctinfo); | 597 | struct nf_conn *ct = nf_ct_get(skb, &ctinfo); |
| 597 | enum ip_conntrack_dir dir = CTINFO2DIR(ctinfo); | 598 | enum ip_conntrack_dir dir = CTINFO2DIR(ctinfo); |
| @@ -622,24 +623,15 @@ static unsigned int nf_nat_sdp_media(struct sk_buff *skb, unsigned int protoff, · 24 unmodified lines | |||
| 622 | int ret; | 623 | int ret; |
| 623 | 624 | ||
| 624 | rtp_exp->tuple.dst.u.udp.port = htons(port); | 625 | rtp_exp->tuple.dst.u.udp.port = htons(port); |
| 625 | ret = nf_ct_expect_related(rtp_exp, | ||
| 626 | NF_CT_EXP_F_SKIP_MASTER); | ||
| 627 | if (ret == -EBUSY) | ||
| 628 | continue; | ||
| 629 | else if (ret < 0) { | ||
| 630 | port = 0; | ||
| 631 | break; | ||
| 632 | } | ||
| 633 | rtcp_exp->tuple.dst.u.udp.port = htons(port + 1); | 626 | rtcp_exp->tuple.dst.u.udp.port = htons(port + 1); |
| 634 | ret = nf_ct_expect_related(rtcp_exp, | 627 | |
| 635 | NF_CT_EXP_F_SKIP_MASTER); | 628 | ret = nf_ct_expect_related_pair(rtp_pair, |
| 629 | NF_CT_EXP_F_SKIP_MASTER); | ||
| 636 | if (ret == 0) | 630 | if (ret == 0) |
| 637 | break; | 631 | break; |
| 638 | else if (ret == -EBUSY) { | 632 | else if (ret == -EBUSY) |
| 639 | nf_ct_unexpect_related(rtp_exp); | ||
| 640 | continue; | 633 | continue; |
| 641 | } else if (ret < 0) { | 634 | else if (ret < 0) { |
| 642 | nf_ct_unexpect_related(rtp_exp); | ||
| 643 | port = 0; | 635 | port = 0; |
| 644 | break; | 636 | break; |
| 645 | } | 637 | } |
| @@ -45,8 +45,6 @@ enum { · 44 unmodified lines | |||
| 45 | NFT_VALIDATE_DO, | 45 | NFT_VALIDATE_DO, |
| 46 | }; | 46 | }; |
| 47 | 47 | ||
| 48 | static struct rhltable nft_objname_ht; | ||
| 49 | |||
| 50 | static u32 nft_chain_hash(const void *data, u32 len, u32 seed); | 48 | static u32 nft_chain_hash(const void *data, u32 len, u32 seed); |
| 51 | static u32 nft_chain_hash_obj(const void *data, u32 len, u32 seed); | 49 | static u32 nft_chain_hash_obj(const void *data, u32 len, u32 seed); |
| 52 | static int nft_chain_hash_cmp(struct rhashtable_compare_arg *, const void *); | 50 | static int nft_chain_hash_cmp(struct rhashtable_compare_arg *, const void *); |
| @@ -1635,6 +1633,10 @@ static int nf_tables_newtable(struct sk_buff *skb, const struct nfnl_info *info, · 1582 unmodified lines | |||
| 1635 | if (err) | 1633 | if (err) |
| 1636 | goto err_chain_ht; | 1634 | goto err_chain_ht; |
| 1637 | 1635 | ||
| 1636 | err = rhltable_init(&table->objname_ht, &nft_objname_ht_params); | ||
| 1637 | if (err < 0) | ||
| 1638 | goto err_obj_ht; | ||
| 1639 | |||
| 1638 | INIT_LIST_HEAD(&table->chains); | 1640 | INIT_LIST_HEAD(&table->chains); |
| 1639 | INIT_LIST_HEAD(&table->sets); | 1641 | INIT_LIST_HEAD(&table->sets); |
| 1640 | INIT_LIST_HEAD(&table->objects); | 1642 | INIT_LIST_HEAD(&table->objects); |
| @@ -1653,6 +1655,8 @@ static int nf_tables_newtable(struct sk_buff *skb, const struct nfnl_info *info, · 12 unmodified lines | |||
| 1653 | list_add_tail_rcu(&table->list, &nft_net->tables); | 1655 | list_add_tail_rcu(&table->list, &nft_net->tables); |
| 1654 | return 0; | 1656 | return 0; |
| 1655 | err_trans: | 1657 | err_trans: |
| 1658 | rhltable_destroy(&table->objname_ht); | ||
| 1659 | err_obj_ht: | ||
| 1656 | rhltable_destroy(&table->chains_ht); | 1660 | rhltable_destroy(&table->chains_ht); |
| 1657 | err_chain_ht: | 1661 | err_chain_ht: |
| 1658 | kfree(table->udata); | 1662 | kfree(table->udata); |
| @@ -1819,6 +1823,7 @@ static void nf_tables_table_destroy(struct nft_table *table) · 160 unmodified lines | |||
| 1819 | return; | 1823 | return; |
| 1820 | 1824 | ||
| 1821 | rhltable_destroy(&table->chains_ht); | 1825 | rhltable_destroy(&table->chains_ht); |
| 1826 | rhltable_destroy(&table->objname_ht); | ||
| 1822 | kfree(table->name); | 1827 | kfree(table->name); |
| 1823 | kfree(table->udata); | 1828 | kfree(table->udata); |
| 1824 | kfree(table); | 1829 | kfree(table); |
| @@ -8086,7 +8091,7 @@ void nft_unregister_obj(struct nft_object_type *obj_type) · 6261 unmodified lines | |||
| 8086 | EXPORT_SYMBOL_GPL(nft_unregister_obj); | 8091 | EXPORT_SYMBOL_GPL(nft_unregister_obj); |
| 8087 | 8092 | ||
| 8088 | struct nft_object *nft_obj_lookup(const struct net *net, | 8093 | struct nft_object *nft_obj_lookup(const struct net *net, |
| 8089 | const struct nft_table *table, | 8094 | struct nft_table *table, |
| 8090 | const struct nlattr *nla, u32 objtype, | 8095 | const struct nlattr *nla, u32 objtype, |
| 8091 | u8 genmask) | 8096 | u8 genmask) |
| 8092 | { | 8097 | { |
| @@ -8102,7 +8107,7 @@ struct nft_object *nft_obj_lookup(const struct net *net, · 9 unmodified lines | |||
| 8102 | !lockdep_commit_lock_is_held(net)); | 8107 | !lockdep_commit_lock_is_held(net)); |
| 8103 | 8108 | ||
| 8104 | rcu_read_lock(); | 8109 | rcu_read_lock(); |
| 8105 | list = rhltable_lookup(&nft_objname_ht, &k, nft_objname_ht_params); | 8110 | list = rhltable_lookup(&table->objname_ht, &k, nft_objname_ht_params); |
| 8106 | if (!list) | 8111 | if (!list) |
| 8107 | goto out; | 8112 | goto out; |
| 8108 | 8113 | ||
| @@ -8382,7 +8387,7 @@ static int nf_tables_newobj(struct sk_buff *skb, const struct nfnl_info *info, · 273 unmodified lines | |||
| 8382 | if (err < 0) | 8387 | if (err < 0) |
| 8383 | goto err_trans; | 8388 | goto err_trans; |
| 8384 | 8389 | ||
| 8385 | err = rhltable_insert(&nft_objname_ht, &obj->rhlhead, | 8390 | err = rhltable_insert(&table->objname_ht, &obj->rhlhead, |
| 8386 | nft_objname_ht_params); | 8391 | nft_objname_ht_params); |
| 8387 | if (err < 0) | 8392 | if (err < 0) |
| 8388 | goto err_obj_ht; | 8393 | goto err_obj_ht; |
| @@ -8567,8 +8572,8 @@ nf_tables_getobj_single(u32 portid, const struct nfnl_info *info, · 178 unmodified lines | |||
| 8567 | struct netlink_ext_ack *extack = info->extack; | 8572 | struct netlink_ext_ack *extack = info->extack; |
| 8568 | u8 genmask = nft_genmask_cur(info->net); | 8573 | u8 genmask = nft_genmask_cur(info->net); |
| 8569 | u8 family = info->nfmsg->nfgen_family; | 8574 | u8 family = info->nfmsg->nfgen_family; |
| 8570 | const struct nft_table *table; | ||
| 8571 | struct net *net = info->net; | 8575 | struct net *net = info->net; |
| 8576 | struct nft_table *table; | ||
| 8572 | struct nft_object *obj; | 8577 | struct nft_object *obj; |
| 8573 | struct sk_buff *skb2; | 8578 | struct sk_buff *skb2; |
| 8574 | u32 objtype; | 8579 | u32 objtype; |
| @@ -10437,9 +10442,9 @@ static void nf_tables_commit_chain(struct net *net, struct nft_chain *chain) · 1862 unmodified lines | |||
| 10437 | nf_tables_commit_chain_free_rules_old(g0); | 10442 | nf_tables_commit_chain_free_rules_old(g0); |
| 10438 | } | 10443 | } |
| 10439 | 10444 | ||
| 10440 | static void nft_obj_del(struct nft_object *obj) | 10445 | static void nft_obj_del(struct nft_table *table, struct nft_object *obj) |
| 10441 | { | 10446 | { |
| 10442 | rhltable_remove(&nft_objname_ht, &obj->rhlhead, nft_objname_ht_params); | 10447 | rhltable_remove(&table->objname_ht, &obj->rhlhead, nft_objname_ht_params); |
| 10443 | list_del_rcu(&obj->list); | 10448 | list_del_rcu(&obj->list); |
| 10444 | } | 10449 | } |
| 10445 | 10450 | ||
| @@ -11124,7 +11129,7 @@ static int nf_tables_commit(struct net *net, struct sk_buff *skb) · 678 unmodified lines | |||
| 11124 | break; | 11129 | break; |
| 11125 | case NFT_MSG_DELOBJ: | 11130 | case NFT_MSG_DELOBJ: |
| 11126 | case NFT_MSG_DESTROYOBJ: | 11131 | case NFT_MSG_DESTROYOBJ: |
| 11127 | nft_obj_del(nft_trans_obj(trans)); | 11132 | nft_obj_del(table, nft_trans_obj(trans)); |
| 11128 | nf_tables_obj_notify(&ctx, nft_trans_obj(trans), | 11133 | nf_tables_obj_notify(&ctx, nft_trans_obj(trans), |
| 11129 | trans->msg_type); | 11134 | trans->msg_type); |
| 11130 | break; | 11135 | break; |
| @@ -11416,7 +11421,7 @@ static int __nf_tables_abort(struct net *net, enum nfnl_abort_action action) · 285 unmodified lines | |||
| 11416 | nft_trans_destroy(trans); | 11421 | nft_trans_destroy(trans); |
| 11417 | } else { | 11422 | } else { |
| 11418 | nft_use_dec_restore(&table->use); | 11423 | nft_use_dec_restore(&table->use); |
| 11419 | nft_obj_del(nft_trans_obj(trans)); | 11424 | nft_obj_del(table, nft_trans_obj(trans)); |
| 11420 | } | 11425 | } |
| 11421 | break; | 11426 | break; |
| 11422 | case NFT_MSG_DELOBJ: | 11427 | case NFT_MSG_DELOBJ: |
| @@ -12043,7 +12048,7 @@ static void __nft_release_table(struct net *net, struct nft_table *table) · 620 unmodified lines | |||
| 12043 | nft_set_destroy(&ctx, set); | 12048 | nft_set_destroy(&ctx, set); |
| 12044 | } | 12049 | } |
| 12045 | list_for_each_entry_safe(obj, ne, &table->objects, list) { | 12050 | list_for_each_entry_safe(obj, ne, &table->objects, list) { |
| 12046 | nft_obj_del(obj); | 12051 | nft_obj_del(table, obj); |
| 12047 | nft_use_dec(&table->use); | 12052 | nft_use_dec(&table->use); |
| 12048 | nft_obj_destroy(&ctx, obj); | 12053 | nft_obj_destroy(&ctx, obj); |
| 12049 | } | 12054 | } |
| @@ -12225,10 +12230,6 @@ static int __init nf_tables_module_init(void) · 175 unmodified lines | |||
| 12225 | if (err < 0) | 12230 | if (err < 0) |
| 12226 | goto err_netdev_notifier; | 12231 | goto err_netdev_notifier; |
| 12227 | 12232 | ||
| 12228 | err = rhltable_init(&nft_objname_ht, &nft_objname_ht_params); | ||
| 12229 | if (err < 0) | ||
| 12230 | goto err_rht_objname; | ||
| 12231 | |||
| 12232 | err = nft_offload_init(); | 12233 | err = nft_offload_init(); |
| 12233 | if (err < 0) | 12234 | if (err < 0) |
| 12234 | goto err_offload; | 12235 | goto err_offload; |
| @@ -12251,8 +12252,6 @@ err_nfnl_subsys: · 16 unmodified lines | |||
| 12251 | err_netlink_notifier: | 12252 | err_netlink_notifier: |
| 12252 | nft_offload_exit(); | 12253 | nft_offload_exit(); |
| 12253 | err_offload: | 12254 | err_offload: |
| 12254 | rhltable_destroy(&nft_objname_ht); | ||
| 12255 | err_rht_objname: | ||
| 12256 | unregister_netdevice_notifier(&nf_tables_flowtable_notifier); | 12255 | unregister_netdevice_notifier(&nf_tables_flowtable_notifier); |
| 12257 | err_netdev_notifier: | 12256 | err_netdev_notifier: |
| 12258 | nf_tables_core_module_exit(); | 12257 | nf_tables_core_module_exit(); |
| @@ -12274,7 +12273,6 @@ static void __exit nf_tables_module_exit(void) · 15 unmodified lines | |||
| 12274 | unregister_pernet_subsys(&nf_tables_net_ops); | 12273 | unregister_pernet_subsys(&nf_tables_net_ops); |
| 12275 | cancel_work_sync(&trans_gc_work); | 12274 | cancel_work_sync(&trans_gc_work); |
| 12276 | rcu_barrier(); | 12275 | rcu_barrier(); |
| 12277 | rhltable_destroy(&nft_objname_ht); | ||
| 12278 | nf_tables_core_module_exit(); | 12276 | nf_tables_core_module_exit(); |
| 12279 | } | 12277 | } |
| 12280 | 12278 | ||
| @@ -259,9 +259,7 @@ nla_put_failure: · 258 unmodified lines | |||
| 259 | static bool nft_payload_offload_mask(struct nft_offload_reg *reg, | 259 | static bool nft_payload_offload_mask(struct nft_offload_reg *reg, |
| 260 | u32 priv_len, u32 field_len) | 260 | u32 priv_len, u32 field_len) |
| 261 | { | 261 | { |
| 262 | unsigned int remainder, delta, k; | ||
| 263 | struct nft_data mask = {}; | 262 | struct nft_data mask = {}; |
| 264 | __be32 remainder_mask; | ||
| 265 | 263 | ||
| 266 | if (priv_len == field_len) { | 264 | if (priv_len == field_len) { |
| 267 | memset(®->mask, 0xff, priv_len); | 265 | memset(®->mask, 0xff, priv_len); |
| @@ -270,15 +268,7 @@ static bool nft_payload_offload_mask(struct nft_offload_reg *reg, · 2 unmodified lines | |||
| 270 | return false; | 268 | return false; |
| 271 | } | 269 | } |
| 272 | 270 | ||
| 273 | memset(&mask, 0xff, field_len); | 271 | memset(&mask, 0xff, priv_len); |
| 274 | remainder = priv_len % sizeof(u32); | ||
| 275 | if (remainder) { | ||
| 276 | k = priv_len / sizeof(u32); | ||
| 277 | delta = field_len - priv_len; | ||
| 278 | remainder_mask = htonl(~((1 << (delta * BITS_PER_BYTE)) - 1)); | ||
| 279 | mask.data[k] = (__force u32)remainder_mask; | ||
| 280 | } | ||
| 281 | |||
| 282 | memcpy(®->mask, &mask, field_len); | 272 | memcpy(®->mask, &mask, field_len); |
| 283 | 273 | ||
| 284 | return true; | 274 | return true; |
| @@ -117,6 +117,7 @@ struct xt_hashlimit_htable { · 116 unmodified lines | |||
| 117 | refcount_t use; | 117 | refcount_t use; |
| 118 | u_int8_t family; | 118 | u_int8_t family; |
| 119 | bool rnd_initialized; | 119 | bool rnd_initialized; |
| 120 | bool ratematch; | ||
| 120 | 121 | ||
| 121 | struct hashlimit_cfg3 cfg; /* config */ | 122 | struct hashlimit_cfg3 cfg; /* config */ |
| 122 | 123 | ||
| @@ -323,6 +324,7 @@ static int htable_create(struct net *net, struct hashlimit_cfg3 *cfg, · 200 unmodified lines | |||
| 323 | kvfree(hinfo); | 324 | kvfree(hinfo); |
| 324 | return -ENOMEM; | 325 | return -ENOMEM; |
| 325 | } | 326 | } |
| 327 | hinfo->ratematch = !!(cfg->mode & XT_HASHLIMIT_RATE_MATCH); | ||
| 326 | spin_lock_init(&hinfo->lock); | 328 | spin_lock_init(&hinfo->lock); |
| 327 | 329 | ||
| 328 | switch (revision) { | 330 | switch (revision) { |
| @@ -872,7 +874,10 @@ static int hashlimit_mt_check_common(const struct xt_mtchk_param *par, · 543 unmodified lines | |||
| 872 | } | 874 | } |
| 873 | 875 | ||
| 874 | /* Check for overflow. */ | 876 | /* Check for overflow. */ |
| 875 | if (revision >= 3 && cfg->mode & XT_HASHLIMIT_RATE_MATCH) { | 877 | if (cfg->mode & XT_HASHLIMIT_RATE_MATCH) { |
| 878 | if (revision < 3) | ||
| 879 | return -EINVAL; | ||
| 880 | |||
| 876 | if (cfg->avg == 0 || cfg->avg > U32_MAX) { | 881 | if (cfg->avg == 0 || cfg->avg > U32_MAX) { |
| 877 | pr_info_ratelimited("invalid rate\n"); | 882 | pr_info_ratelimited("invalid rate\n"); |
| 878 | return -ERANGE; | 883 | return -ERANGE; |
| @@ -905,6 +910,15 @@ static int hashlimit_mt_check_common(const struct xt_mtchk_param *par, · 26 unmodified lines | |||
| 905 | mutex_unlock(&hashlimit_mutex); | 910 | mutex_unlock(&hashlimit_mutex); |
| 906 | return ret; | 911 | return ret; |
| 907 | } | 912 | } |
| 913 | } else { | ||
| 914 | if ((cfg->mode & XT_HASHLIMIT_RATE_MATCH && | ||
| 915 | !(*hinfo)->ratematch) || | ||
| 916 | (!(cfg->mode & XT_HASHLIMIT_RATE_MATCH) && | ||
| 917 | (*hinfo)->ratematch)) { | ||
| 918 | mutex_unlock(&hashlimit_mutex); | ||
| 919 | htable_put(*hinfo); | ||
| 920 | return -EINVAL; | ||
| 921 | } | ||
| 908 | } | 922 | } |
| 909 | mutex_unlock(&hashlimit_mutex); | 923 | mutex_unlock(&hashlimit_mutex); |
| 910 | 924 | ||
| @@ -1108,6 +1108,10 @@ static int execute_masked_set_action(struct sk_buff *skb, · 1107 unmodified lines | |||
| 1108 | return err; | 1108 | return err; |
| 1109 | } | 1109 | } |
| 1110 | 1110 | ||
| 1111 | /* When 'last' is true, recirc() should always consume the 'skb'. | ||
| 1112 | * Otherwise, recirc() should keep 'skb' intact regardless what | ||
| 1113 | * actions are executed on recirculation. | ||
| 1114 | */ | ||
| 1111 | static int execute_recirc(struct datapath *dp, struct sk_buff *skb, | 1115 | static int execute_recirc(struct datapath *dp, struct sk_buff *skb, |
| 1112 | struct sw_flow_key *key, | 1116 | struct sw_flow_key *key, |
| 1113 | const struct nlattr *a, bool last) | 1117 | const struct nlattr *a, bool last) |
| @@ -1118,8 +1122,12 @@ static int execute_recirc(struct datapath *dp, struct sk_buff *skb, · 4 unmodified lines | |||
| 1118 | int err; | 1122 | int err; |
| 1119 | 1123 | ||
| 1120 | err = ovs_flow_key_update(skb, key); | 1124 | err = ovs_flow_key_update(skb, key); |
| 1121 | if (err) | 1125 | if (err) { |
| 1126 | if (last) | ||
| 1127 | ovs_kfree_skb_reason(skb, | ||
| 1128 | OVS_DROP_ACTION_ERROR); | ||
| 1122 | return err; | 1129 | return err; |
| 1130 | } | ||
| 1123 | } | 1131 | } |
| 1124 | BUG_ON(!is_flow_key_valid(key)); | 1132 | BUG_ON(!is_flow_key_valid(key)); |
| 1125 | 1133 | ||
| @@ -1372,7 +1380,7 @@ static int do_execute_actions(struct datapath *dp, struct sk_buff *skb, · 246 unmodified lines | |||
| 1372 | if (!is_flow_key_valid(key)) { | 1380 | if (!is_flow_key_valid(key)) { |
| 1373 | err = ovs_flow_key_update(skb, key); | 1381 | err = ovs_flow_key_update(skb, key); |
| 1374 | if (err) | 1382 | if (err) |
| 1375 | return err; | 1383 | break; |
| 1376 | } | 1384 | } |
| 1377 | 1385 | ||
| 1378 | err = ovs_ct_execute(ovs_dp_get_net(dp), skb, key, | 1386 | err = ovs_ct_execute(ovs_dp_get_net(dp), skb, key, |
| @@ -133,18 +133,10 @@ static void dp_meter_instance_remove(struct dp_meter_instance *ti, · 132 unmodified lines | |||
| 133 | 133 | ||
| 134 | static int attach_meter(struct dp_meter_table *tbl, struct dp_meter *meter) | 134 | static int attach_meter(struct dp_meter_table *tbl, struct dp_meter *meter) |
| 135 | { | 135 | { |
| 136 | struct dp_meter_instance *ti = rcu_dereference_ovsl(tbl->ti); | 136 | struct dp_meter_instance *ti; |
| 137 | u32 hash = meter_hash(ti, meter->id); | 137 | u32 hash; |
| 138 | int err; | 138 | int err; |
| 139 | 139 | ||
| 140 | /* In generally, slots selected should be empty, because | ||
| 141 | * OvS uses id-pool to fetch a available id. | ||
| 142 | */ | ||
| 143 | if (unlikely(rcu_dereference_ovsl(ti->dp_meters[hash]))) | ||
| 144 | return -EBUSY; | ||
| 145 | |||
| 146 | dp_meter_instance_insert(ti, meter); | ||
| 147 | |||
| 148 | /* That function is thread-safe. */ | 140 | /* That function is thread-safe. */ |
| 149 | tbl->count++; | 141 | tbl->count++; |
| 150 | if (tbl->count >= tbl->max_meters_allowed) { | 142 | if (tbl->count >= tbl->max_meters_allowed) { |
| @@ -152,16 +144,29 @@ static int attach_meter(struct dp_meter_table *tbl, struct dp_meter *meter) · 1 unmodified line | |||
| 152 | goto attach_err; | 144 | goto attach_err; |
| 153 | } | 145 | } |
| 154 | 146 | ||
| 155 | if (tbl->count >= ti->n_meters && | 147 | ti = rcu_dereference_ovsl(tbl->ti); |
| 156 | dp_meter_instance_realloc(tbl, ti->n_meters * 2)) { | 148 | if (tbl->count >= ti->n_meters) { |
| 157 | err = -ENOMEM; | 149 | err = dp_meter_instance_realloc(tbl, ti->n_meters * 2); |
| 150 | if (err) | ||
| 151 | goto attach_err; | ||
| 152 | |||
| 153 | ti = rcu_dereference_ovsl(tbl->ti); | ||
| 154 | } | ||
| 155 | |||
| 156 | hash = meter_hash(ti, meter->id); | ||
| 157 | |||
| 158 | /* In general, selected slots should be empty, because | ||
| 159 | * OvS uses id-pool to fetch available ids. | ||
| 160 | */ | ||
| 161 | if (unlikely(rcu_dereference_ovsl(ti->dp_meters[hash]))) { | ||
| 162 | err = -EBUSY; | ||
| 158 | goto attach_err; | 163 | goto attach_err; |
| 159 | } | 164 | } |
| 160 | 165 | ||
| 166 | dp_meter_instance_insert(ti, meter); | ||
| 161 | return 0; | 167 | return 0; |
| 162 | 168 | ||
| 163 | attach_err: | 169 | attach_err: |
| 164 | dp_meter_instance_remove(ti, meter); | ||
| 165 | tbl->count--; | 170 | tbl->count--; |
| 166 | return err; | 171 | return err; |
| 167 | } | 172 | } |
| @@ -62,7 +62,14 @@ psp_nl_multicast_per_ns(struct psp_dev *psd, unsigned int group, · 61 unmodified lines | |||
| 62 | struct net *main_net; | 62 | struct net *main_net; |
| 63 | struct sk_buff *ntf; | 63 | struct sk_buff *ntf; |
| 64 | 64 | ||
| 65 | main_net = dev_net(psd->main_netdev); | 65 | /* device may be changing netns in parallel */ |
| 66 | rcu_read_lock(); | ||
| 67 | main_net = maybe_get_net(dev_net_rcu(psd->main_netdev)); | ||
| 68 | rcu_read_unlock(); | ||
| 69 | |||
| 70 | if (!main_net) | ||
| 71 | return; | ||
| 72 | |||
| 66 | xa_init(&sent_nets); | 73 | xa_init(&sent_nets); |
| 67 | 74 | ||
| 68 | list_for_each_entry(entry, &psd->assoc_dev_list, dev_list) { | 75 | list_for_each_entry(entry, &psd->assoc_dev_list, dev_list) { |
| @@ -88,10 +95,10 @@ psp_nl_multicast_per_ns(struct psp_dev *psd, unsigned int group, · 19 unmodified lines | |||
| 88 | 95 | ||
| 89 | /* Send to main device netns */ | 96 | /* Send to main device netns */ |
| 90 | ntf = build_ntf(psd, main_net, ctx); | 97 | ntf = build_ntf(psd, main_net, ctx); |
| 91 | if (!ntf) | 98 | if (ntf) |
| 92 | return; | 99 | genlmsg_multicast_netns(&psp_nl_family, main_net, ntf, 0, group, |
| 93 | genlmsg_multicast_netns(&psp_nl_family, main_net, ntf, 0, group, | 100 | GFP_KERNEL); |
| 94 | GFP_KERNEL); | 101 | put_net(main_net); |
| 95 | } | 102 | } |
| 96 | 103 | ||
| 97 | static struct sk_buff *psp_nl_clone_ntf(struct psp_dev *psd, struct net *net, | 104 | static struct sk_buff *psp_nl_clone_ntf(struct psp_dev *psd, struct net *net, |
| @@ -355,23 +355,25 @@ int rds_tcp_laddr_check(struct net *net, const struct in6_addr *addr, · 354 unmodified lines | |||
| 355 | /* If the scope_id is specified, check only those addresses | 355 | /* If the scope_id is specified, check only those addresses |
| 356 | * hosted on the specified interface. | 356 | * hosted on the specified interface. |
| 357 | */ | 357 | */ |
| 358 | rcu_read_lock(); | ||
| 358 | if (scope_id != 0) { | 359 | if (scope_id != 0) { |
| 359 | rcu_read_lock(); | ||
| 360 | dev = dev_get_by_index_rcu(net, scope_id); | 360 | dev = dev_get_by_index_rcu(net, scope_id); |
| 361 | /* scope_id is not valid... */ | 361 | /* scope_id is not valid... */ |
| 362 | if (!dev) { | 362 | if (!dev) { |
| 363 | rcu_read_unlock(); | 363 | rcu_read_unlock(); |
| 364 | return -EADDRNOTAVAIL; | 364 | return -EADDRNOTAVAIL; |
| 365 | } | 365 | } |
| 366 | rcu_read_unlock(); | ||
| 367 | } | 366 | } |
| 368 | #if IS_ENABLED(CONFIG_IPV6) | 367 | #if IS_ENABLED(CONFIG_IPV6) |
| 369 | if (ipv6_mod_enabled()) { | 368 | if (ipv6_mod_enabled()) { |
| 370 | ret = ipv6_chk_addr(net, addr, dev, 0); | 369 | ret = ipv6_chk_addr(net, addr, dev, 0); |
| 371 | if (ret) | 370 | if (ret) { |
| 371 | rcu_read_unlock(); | ||
| 372 | return 0; | 372 | return 0; |
| 373 | } | ||
| 373 | } | 374 | } |
| 374 | #endif | 375 | #endif |
| 376 | rcu_read_unlock(); | ||
| 375 | return -EADDRNOTAVAIL; | 377 | return -EADDRNOTAVAIL; |
| 376 | } | 378 | } |
| 377 | 379 | ||
| @@ -1107,6 +1107,13 @@ static int u32_change(struct net *net, struct sk_buff *in_skb, · 1106 unmodified lines | |||
| 1107 | goto erridr; | 1107 | goto erridr; |
| 1108 | } | 1108 | } |
| 1109 | 1109 | ||
| 1110 | if (s->offshift >= 16) { | ||
| 1111 | NL_SET_ERR_MSG_MOD(extack, | ||
| 1112 | "offshift must be less than 16"); | ||
| 1113 | err = -EINVAL; | ||
| 1114 | goto erridr; | ||
| 1115 | } | ||
| 1116 | |||
| 1110 | n = kzalloc_flex(*n, sel.keys, s->nkeys); | 1117 | n = kzalloc_flex(*n, sel.keys, s->nkeys); |
| 1111 | if (n == NULL) { | 1118 | if (n == NULL) { |
| 1112 | err = -ENOBUFS; | 1119 | err = -ENOBUFS; |
| @@ -2609,9 +2609,11 @@ static void cake_configure_rates(struct Qdisc *sch, u64 rate, bool rate_adjust) · 2608 unmodified lines | |||
| 2609 | break; | 2609 | break; |
| 2610 | } | 2610 | } |
| 2611 | 2611 | ||
| 2612 | for (c = qd->tin_cnt; c < CAKE_MAX_TINS; c++) { | 2612 | if (!rate_adjust) { |
| 2613 | cake_clear_tin(sch, c); | 2613 | for (c = qd->tin_cnt; c < CAKE_MAX_TINS; c++) { |
| 2614 | qd->tins[c].cparams.mtu_time = qd->tins[ft].cparams.mtu_time; | 2614 | cake_clear_tin(sch, c); |
| 2615 | qd->tins[c].cparams.mtu_time = qd->tins[ft].cparams.mtu_time; | ||
| 2616 | } | ||
| 2615 | } | 2617 | } |
| 2616 | 2618 | ||
| 2617 | qd->rate_ns = qd->tins[ft].tin_rate_ns; | 2619 | qd->rate_ns = qd->tins[ft].tin_rate_ns; |
| @@ -614,6 +614,9 @@ struct sctp_transport *sctp_assoc_add_peer(struct sctp_association *asoc, · 613 unmodified lines | |||
| 614 | return peer; | 614 | return peer; |
| 615 | } | 615 | } |
| 616 | 616 | ||
| 617 | if (asoc->peer.transport_count == U16_MAX) | ||
| 618 | return NULL; | ||
| 619 | |||
| 617 | peer = sctp_transport_new(asoc->base.net, addr, gfp); | 620 | peer = sctp_transport_new(asoc->base.net, addr, gfp); |
| 618 | if (!peer) | 621 | if (!peer) |
| 619 | return NULL; | 622 | return NULL; |
| @@ -1802,9 +1802,9 @@ struct sctp_association *sctp_unpack_cookie( · 1801 unmodified lines | |||
| 1802 | goto fail; | 1802 | goto fail; |
| 1803 | } | 1803 | } |
| 1804 | 1804 | ||
| 1805 | /* Check to see if the cookie is stale. If there is already | 1805 | /* Check to see if the cookie is stale. RFC 9260 Section 5.2.4 |
| 1806 | * an association, there is no need to check cookie's expiration | 1806 | * exempts an expired cookie only when both Verification Tags match |
| 1807 | * for init collision case of lost COOKIE ACK. | 1807 | * the current association. |
| 1808 | * If skb has been timestamped, then use the stamp, otherwise | 1808 | * If skb has been timestamped, then use the stamp, otherwise |
| 1809 | * use current time. This introduces a small possibility that | 1809 | * use current time. This introduces a small possibility that |
| 1810 | * a cookie may be considered expired, but this would only slow | 1810 | * a cookie may be considered expired, but this would only slow |
| @@ -1815,7 +1815,10 @@ struct sctp_association *sctp_unpack_cookie( · 4 unmodified lines | |||
| 1815 | else | 1815 | else |
| 1816 | kt = ktime_get_real(); | 1816 | kt = ktime_get_real(); |
| 1817 | 1817 | ||
| 1818 | if (!asoc && ktime_before(bear_cookie->expiration, kt)) { | 1818 | if ((!asoc || |
| 1819 | asoc->c.my_vtag != bear_cookie->my_vtag || | ||
| 1820 | asoc->c.peer_vtag != bear_cookie->peer_vtag) && | ||
| 1821 | ktime_before(bear_cookie->expiration, kt)) { | ||
| 1819 | suseconds_t usecs = ktime_to_us(ktime_sub(kt, bear_cookie->expiration)); | 1822 | suseconds_t usecs = ktime_to_us(ktime_sub(kt, bear_cookie->expiration)); |
| 1820 | __be32 n = htonl(usecs); | 1823 | __be32 n = htonl(usecs); |
| 1821 | 1824 | ||
| @@ -2168,7 +2171,13 @@ static enum sctp_ierror sctp_verify_param(struct net *net, · 346 unmodified lines | |||
| 2168 | case SCTP_PARAM_HEARTBEAT_INFO: | 2171 | case SCTP_PARAM_HEARTBEAT_INFO: |
| 2169 | case SCTP_PARAM_UNRECOGNIZED_PARAMETERS: | 2172 | case SCTP_PARAM_UNRECOGNIZED_PARAMETERS: |
| 2170 | case SCTP_PARAM_ECN_CAPABLE: | 2173 | case SCTP_PARAM_ECN_CAPABLE: |
| 2174 | break; | ||
| 2171 | case SCTP_PARAM_ADAPTATION_LAYER_IND: | 2175 | case SCTP_PARAM_ADAPTATION_LAYER_IND: |
| 2176 | if (ntohs(param.p->length) != sizeof(*param.aind)) { | ||
| 2177 | sctp_process_inv_paramlength(asoc, param.p, | ||
| 2178 | chunk, err_chunk); | ||
| 2179 | retval = SCTP_IERROR_ABORT; | ||
| 2180 | } | ||
| 2172 | break; | 2181 | break; |
| 2173 | 2182 | ||
| 2174 | case SCTP_PARAM_SUPPORTED_EXT: | 2183 | case SCTP_PARAM_SUPPORTED_EXT: |
| @@ -1572,10 +1572,10 @@ static void __smc_lgr_terminate(struct smc_link_group *lgr, bool soft) · 1571 unmodified lines | |||
| 1572 | read_lock_bh(&lgr->conns_lock); | 1572 | read_lock_bh(&lgr->conns_lock); |
| 1573 | node = rb_first(&lgr->conns_all); | 1573 | node = rb_first(&lgr->conns_all); |
| 1574 | while (node) { | 1574 | while (node) { |
| 1575 | read_unlock_bh(&lgr->conns_lock); | ||
| 1576 | conn = rb_entry(node, struct smc_connection, alert_node); | 1575 | conn = rb_entry(node, struct smc_connection, alert_node); |
| 1577 | smc = container_of(conn, struct smc_sock, conn); | 1576 | smc = container_of(conn, struct smc_sock, conn); |
| 1578 | sock_hold(&smc->sk); /* sock_put below */ | 1577 | sock_hold(&smc->sk); /* sock_put below */ |
| 1578 | read_unlock_bh(&lgr->conns_lock); | ||
| 1579 | lock_sock(&smc->sk); | 1579 | lock_sock(&smc->sk); |
| 1580 | smc_conn_kill(conn, soft); | 1580 | smc_conn_kill(conn, soft); |
| 1581 | release_sock(&smc->sk); | 1581 | release_sock(&smc->sk); |
| @@ -796,7 +796,7 @@ static __poll_t tipc_poll(struct file *file, struct socket *sock, · 795 unmodified lines | |||
| 796 | __poll_t revents = 0; | 796 | __poll_t revents = 0; |
| 797 | 797 | ||
| 798 | sock_poll_wait(file, sock, wait); | 798 | sock_poll_wait(file, sock, wait); |
| 799 | trace_tipc_sk_poll(sk, NULL, TIPC_DUMP_ALL, " "); | 799 | trace_tipc_sk_poll(sk, NULL, TIPC_DUMP_NONE, " "); |
| 800 | 800 | ||
| 801 | if (sk->sk_shutdown & RCV_SHUTDOWN) | 801 | if (sk->sk_shutdown & RCV_SHUTDOWN) |
| 802 | revents |= EPOLLRDHUP | EPOLLIN | EPOLLRDNORM; | 802 | revents |= EPOLLRDHUP | EPOLLIN | EPOLLRDNORM; |
| @@ -823,6 +823,7 @@ static int unix_listen(struct socket *sock, int backlog) · 822 unmodified lines | |||
| 823 | if (err) | 823 | if (err) |
| 824 | goto out; | 824 | goto out; |
| 825 | unix_state_lock(sk); | 825 | unix_state_lock(sk); |
| 826 | err = -EINVAL; | ||
| 826 | if (sk->sk_state != TCP_CLOSE && sk->sk_state != TCP_LISTEN) | 827 | if (sk->sk_state != TCP_CLOSE && sk->sk_state != TCP_LISTEN) |
| 827 | goto out_unlock; | 828 | goto out_unlock; |
| 828 | if (backlog > sk->sk_max_ack_backlog) | 829 | if (backlog > sk->sk_max_ack_backlog) |
| @@ -420,6 +420,7 @@ int nl80211_pmsr_start(struct sk_buff *skb, struct genl_info *info) · 419 unmodified lines | |||
| 420 | const struct cfg80211_pmsr_capabilities *capa; | 420 | const struct cfg80211_pmsr_capabilities *capa; |
| 421 | struct cfg80211_pmsr_request *req; | 421 | struct cfg80211_pmsr_request *req; |
| 422 | struct nlattr *peers, *peer; | 422 | struct nlattr *peers, *peer; |
| 423 | u64 cookie; | ||
| 423 | 424 | ||
| 424 | capa = rdev->wiphy.pmsr_capa; | 425 | capa = rdev->wiphy.pmsr_capa; |
| 425 | 426 | ||
| @@ -521,14 +522,27 @@ int nl80211_pmsr_start(struct sk_buff *skb, struct genl_info *info) · 95 unmodified lines | |||
| 521 | } | 522 | } |
| 522 | req->cookie = cfg80211_assign_cookie(rdev); | 523 | req->cookie = cfg80211_assign_cookie(rdev); |
| 523 | req->nl_portid = info->snd_portid; | 524 | req->nl_portid = info->snd_portid; |
| 525 | cookie = req->cookie; | ||
| 526 | |||
| 527 | /* | ||
| 528 | * Add to the list before the driver call; under races or broken | ||
| 529 | * drivers, completion may free the request before rdev_start_pmsr() | ||
| 530 | * returns. Use the saved cookie below. | ||
| 531 | */ | ||
| 532 | spin_lock_bh(&wdev->pmsr_lock); | ||
| 533 | list_add_tail(&req->list, &wdev->pmsr_list); | ||
| 534 | spin_unlock_bh(&wdev->pmsr_lock); | ||
| 524 | 535 | ||
| 525 | err = rdev_start_pmsr(rdev, wdev, req); | 536 | err = rdev_start_pmsr(rdev, wdev, req); |
| 526 | if (err) | 537 | if (err) { |
| 538 | /* An error return leaves the request owned by this path. */ | ||
| 539 | spin_lock_bh(&wdev->pmsr_lock); | ||
| 540 | list_del(&req->list); | ||
| 541 | spin_unlock_bh(&wdev->pmsr_lock); | ||
| 527 | goto out_err; | 542 | goto out_err; |
| 543 | } | ||
| 528 | 544 | ||
| 529 | list_add_tail(&req->list, &wdev->pmsr_list); | 545 | nl_set_extack_cookie_u64(info->extack, cookie); |
| 530 | |||
| 531 | nl_set_extack_cookie_u64(info->extack, req->cookie); | ||
| 532 | return 0; | 546 | return 0; |
| 533 | out_err: | 547 | out_err: |
| 534 | kfree(req); | 548 | kfree(req); |
| @@ -319,6 +319,15 @@ int cfg80211_wext_siwgenie(struct net_device *dev, · 318 unmodified lines | |||
| 319 | return 0; | 319 | return 0; |
| 320 | 320 | ||
| 321 | if (ie_len) { | 321 | if (ie_len) { |
| 322 | const struct element *elem; | ||
| 323 | |||
| 324 | for_each_element(elem, extra, ie_len) { | ||
| 325 | /* nothing */ | ||
| 326 | } | ||
| 327 | |||
| 328 | if (!for_each_element_completed(elem, extra, ie_len)) | ||
| 329 | return -EINVAL; | ||
| 330 | |||
| 322 | ie = kmemdup(extra, ie_len, GFP_KERNEL); | 331 | ie = kmemdup(extra, ie_len, GFP_KERNEL); |
| 323 | if (!ie) | 332 | if (!ie) |
| 324 | return -ENOMEM; | 333 | return -ENOMEM; |
| @@ -499,6 +499,23 @@ void __xsk_map_flush(struct list_head *flush_list) · 498 unmodified lines | |||
| 499 | 499 | ||
| 500 | void xsk_tx_completed(struct xsk_buff_pool *pool, u32 nb_entries) | 500 | void xsk_tx_completed(struct xsk_buff_pool *pool, u32 nb_entries) |
| 501 | { | 501 | { |
| 502 | u32 reclaim_descs = READ_ONCE(pool->reclaim_descs); | ||
| 503 | |||
| 504 | if (unlikely(reclaim_descs)) { | ||
| 505 | u32 pending_descs = READ_ONCE(pool->tx_zc_pending_descs); | ||
| 506 | |||
| 507 | if (nb_entries < pending_descs) { | ||
| 508 | WRITE_ONCE(pool->tx_zc_pending_descs, | ||
| 509 | pending_descs - nb_entries); | ||
| 510 | xskq_prod_submit_n(pool->cq, nb_entries); | ||
| 511 | return; | ||
| 512 | } | ||
| 513 | |||
| 514 | WRITE_ONCE(pool->tx_zc_pending_descs, 0); | ||
| 515 | nb_entries += reclaim_descs; | ||
| 516 | WRITE_ONCE(pool->reclaim_descs, 0); | ||
| 517 | } | ||
| 518 | |||
| 502 | xskq_prod_submit_n(pool->cq, nb_entries); | 519 | xskq_prod_submit_n(pool->cq, nb_entries); |
| 503 | } | 520 | } |
| 504 | EXPORT_SYMBOL(xsk_tx_completed); | 521 | EXPORT_SYMBOL(xsk_tx_completed); |
| @@ -574,24 +591,157 @@ static u32 xsk_tx_peek_release_fallback(struct xsk_buff_pool *pool, u32 max_entr · 69 unmodified lines | |||
| 574 | return nb_pkts; | 591 | return nb_pkts; |
| 575 | } | 592 | } |
| 576 | 593 | ||
| 594 | static void xsk_tx_commit_batch(struct xsk_buff_pool *pool, | ||
| 595 | struct xsk_tx_batch *batch) | ||
| 596 | { | ||
| 597 | u32 nb_descs = xsk_tx_batch_cq_descs(batch); | ||
| 598 | u32 cq_cached_prod; | ||
| 599 | |||
| 600 | if (!nb_descs) | ||
| 601 | return; | ||
| 602 | |||
| 603 | cq_cached_prod = pool->cq->cached_prod; | ||
| 604 | xskq_prod_write_addr_batch(pool->cq, pool->tx_descs, nb_descs); | ||
| 605 | |||
| 606 | if (unlikely(batch->reclaim_descs)) { | ||
| 607 | u32 cq_pending_descs; | ||
| 608 | |||
| 609 | /* CQ is positional. Descriptors already written but not | ||
| 610 | * submitted must complete before any reclaim-only descriptors | ||
| 611 | * appended below. | ||
| 612 | */ | ||
| 613 | cq_pending_descs = cq_cached_prod - xskq_get_prod(pool->cq); | ||
| 614 | |||
| 615 | WRITE_ONCE(pool->tx_zc_pending_descs, | ||
| 616 | batch->tx_descs + cq_pending_descs); | ||
| 617 | WRITE_ONCE(pool->reclaim_descs, batch->reclaim_descs); | ||
| 618 | if (unlikely(!pool->tx_zc_pending_descs)) | ||
| 619 | xsk_tx_completed(pool, 0); | ||
| 620 | } | ||
| 621 | } | ||
| 622 | |||
| 623 | static struct xsk_tx_batch | ||
| 624 | __xsk_tx_peek_release_desc_batch(struct xsk_buff_pool *pool, struct xdp_sock *xs, | ||
| 625 | struct xdp_desc *descs, u32 max_descs) | ||
| 626 | { | ||
| 627 | struct xsk_tx_batch batch = {}; | ||
| 628 | u32 entries; | ||
| 629 | |||
| 630 | entries = xskq_cons_nb_entries(xs->tx, max_descs); | ||
| 631 | if (!entries) | ||
| 632 | return batch; | ||
| 633 | |||
| 634 | batch = xskq_cons_read_desc_batch(xs, pool, descs, max_descs); | ||
| 635 | if (!xsk_tx_batch_cq_descs(&batch)) { | ||
| 636 | xs->tx->queue_empty_descs++; | ||
| 637 | } else { | ||
| 638 | __xskq_cons_release(xs->tx); | ||
| 639 | xs->sk.sk_write_space(&xs->sk); | ||
| 640 | } | ||
| 641 | return batch; | ||
| 642 | } | ||
| 643 | |||
| 644 | static struct xsk_tx_batch | ||
| 645 | xsk_tx_peek_release_shared_desc_batch(struct xsk_buff_pool *pool, u32 max_descs) | ||
| 646 | { | ||
| 647 | u32 cq_descs_before, cq_descs_after; | ||
| 648 | struct xsk_tx_batch sum_batch = {}; | ||
| 649 | bool budget_exhausted; | ||
| 650 | u32 per_socket_budget; | ||
| 651 | struct xdp_sock *xs; | ||
| 652 | |||
| 653 | /* The fairness quota must allow one maximum-sized valid packet. */ | ||
| 654 | per_socket_budget = max_t(u32, MAX_PER_SOCKET_BUDGET, | ||
| 655 | pool->xdp_zc_max_segs); | ||
| 656 | |||
| 657 | again: | ||
| 658 | budget_exhausted = false; | ||
| 659 | cq_descs_before = xsk_tx_batch_cq_descs(&sum_batch); | ||
| 660 | list_for_each_entry_rcu(xs, &pool->xsk_tx_list, tx_list) { | ||
| 661 | u32 budget, budget_left, offset, remaining, used; | ||
| 662 | struct xsk_tx_batch curr_batch; | ||
| 663 | |||
| 664 | /* Once reclaim-only descriptors have been appended to the CQ | ||
| 665 | * address area, do not append driver-visible Tx descriptors | ||
| 666 | * from another socket after them. xsk_tx_completed() relies on | ||
| 667 | * all driver-visible descriptors preceding all reclaim-only | ||
| 668 | * descriptors in CQ order. | ||
| 669 | */ | ||
| 670 | if (sum_batch.reclaim_descs) | ||
| 671 | break; | ||
| 672 | |||
| 673 | /* be gentle when playing with pool->tx_descs */ | ||
| 674 | offset = xsk_tx_batch_cq_descs(&sum_batch); | ||
| 675 | if (offset >= max_descs) | ||
| 676 | break; | ||
| 677 | |||
| 678 | if (xs->tx_budget_spent >= per_socket_budget) { | ||
| 679 | if (xskq_cons_nb_entries(xs->tx, 1)) | ||
| 680 | budget_exhausted = true; | ||
| 681 | continue; | ||
| 682 | } | ||
| 683 | |||
| 684 | budget_left = per_socket_budget - xs->tx_budget_spent; | ||
| 685 | remaining = max_descs - offset; | ||
| 686 | budget = min(remaining, budget_left); | ||
| 687 | |||
| 688 | curr_batch = __xsk_tx_peek_release_desc_batch(pool, xs, | ||
| 689 | pool->tx_descs + offset, | ||
| 690 | budget); | ||
| 691 | used = xsk_tx_batch_cq_descs(&curr_batch); | ||
| 692 | if (!used) { | ||
| 693 | if (curr_batch.budget_limited && budget_left < remaining) | ||
| 694 | budget_exhausted = true; | ||
| 695 | continue; | ||
| 696 | } | ||
| 697 | |||
| 698 | xs->tx_budget_spent += used; | ||
| 699 | sum_batch.tx_descs += curr_batch.tx_descs; | ||
| 700 | sum_batch.reclaim_descs = curr_batch.reclaim_descs; | ||
| 701 | } | ||
| 702 | |||
| 703 | cq_descs_after = xsk_tx_batch_cq_descs(&sum_batch); | ||
| 704 | |||
| 705 | if (sum_batch.reclaim_descs || cq_descs_after >= max_descs) | ||
| 706 | return sum_batch; | ||
| 707 | |||
| 708 | /* Continue filling the batch while this pass made progress */ | ||
| 709 | if (cq_descs_before != cq_descs_after) | ||
| 710 | goto again; | ||
| 711 | |||
| 712 | if (!budget_exhausted) | ||
| 713 | return sum_batch; | ||
| 714 | |||
| 715 | list_for_each_entry_rcu(xs, &pool->xsk_tx_list, tx_list) | ||
| 716 | xs->tx_budget_spent = 0; | ||
| 717 | goto again; | ||
| 718 | } | ||
| 719 | |||
| 577 | u32 xsk_tx_peek_release_desc_batch(struct xsk_buff_pool *pool, u32 nb_pkts) | 720 | u32 xsk_tx_peek_release_desc_batch(struct xsk_buff_pool *pool, u32 nb_pkts) |
| 578 | { | 721 | { |
| 722 | struct xsk_tx_batch batch = {}; | ||
| 579 | struct xdp_sock *xs; | 723 | struct xdp_sock *xs; |
| 724 | bool umem_shared; | ||
| 580 | 725 | ||
| 581 | rcu_read_lock(); | 726 | rcu_read_lock(); |
| 582 | if (!list_is_singular(&pool->xsk_tx_list)) { | 727 | if (unlikely(READ_ONCE(pool->reclaim_descs))) |
| 583 | /* Fallback to the non-batched version */ | 728 | goto out; |
| 584 | rcu_read_unlock(); | ||
| 585 | return xsk_tx_peek_release_fallback(pool, nb_pkts); | ||
| 586 | } | ||
| 587 | 729 | ||
| 588 | xs = list_first_or_null_rcu(&pool->xsk_tx_list, struct xdp_sock, tx_list); | 730 | xs = list_first_or_null_rcu(&pool->xsk_tx_list, struct xdp_sock, |
| 589 | if (!xs) { | 731 | tx_list); |
| 590 | nb_pkts = 0; | 732 | if (!xs) |
| 591 | goto out; | 733 | goto out; |
| 592 | } | ||
| 593 | 734 | ||
| 594 | nb_pkts = xskq_cons_nb_entries(xs->tx, nb_pkts); | 735 | nb_pkts = min(nb_pkts, pool->tx_descs_nentries); |
| 736 | if (!nb_pkts) | ||
| 737 | goto out; | ||
| 738 | |||
| 739 | umem_shared = !list_is_singular(&pool->xsk_tx_list); | ||
| 740 | |||
| 741 | if (umem_shared && !(pool->umem->flags & XDP_UMEM_SG_FLAG)) { | ||
| 742 | rcu_read_unlock(); | ||
| 743 | return xsk_tx_peek_release_fallback(pool, nb_pkts); | ||
| 744 | } | ||
| 595 | 745 | ||
| 596 | /* This is the backpressure mechanism for the Tx path. Try to | 746 | /* This is the backpressure mechanism for the Tx path. Try to |
| 597 | * reserve space in the completion queue for all packets, but | 747 | * reserve space in the completion queue for all packets, but |
| @@ -603,19 +753,16 @@ u32 xsk_tx_peek_release_desc_batch(struct xsk_buff_pool *pool, u32 nb_pkts) · 5 unmodified lines | |||
| 603 | if (!nb_pkts) | 753 | if (!nb_pkts) |
| 604 | goto out; | 754 | goto out; |
| 605 | 755 | ||
| 606 | nb_pkts = xskq_cons_read_desc_batch(xs->tx, pool, nb_pkts); | 756 | batch = umem_shared ? |
| 607 | if (!nb_pkts) { | 757 | xsk_tx_peek_release_shared_desc_batch(pool, nb_pkts) : |
| 608 | xs->tx->queue_empty_descs++; | 758 | __xsk_tx_peek_release_desc_batch(pool, xs, |
| 609 | goto out; | 759 | pool->tx_descs, |
| 610 | } | 760 | nb_pkts); |
| 611 | 761 | xsk_tx_commit_batch(pool, &batch); | |
| 612 | __xskq_cons_release(xs->tx); | ||
| 613 | xskq_prod_write_addr_batch(pool->cq, pool->tx_descs, nb_pkts); | ||
| 614 | xs->sk.sk_write_space(&xs->sk); | ||
| 615 | 762 | ||
| 616 | out: | 763 | out: |
| 617 | rcu_read_unlock(); | 764 | rcu_read_unlock(); |
| 618 | return nb_pkts; | 765 | return batch.tx_descs; |
| 619 | } | 766 | } |
| 620 | EXPORT_SYMBOL(xsk_tx_peek_release_desc_batch); | 767 | EXPORT_SYMBOL(xsk_tx_peek_release_desc_batch); |
| 621 | 768 | ||
| @@ -737,6 +884,19 @@ static void xsk_cq_submit_addr_locked(struct xsk_buff_pool *pool, · 115 unmodified lines | |||
| 737 | spin_unlock_irqrestore(&pool->cq_prod_lock, flags); | 884 | spin_unlock_irqrestore(&pool->cq_prod_lock, flags); |
| 738 | } | 885 | } |
| 739 | 886 | ||
| 887 | static void xsk_cq_submit_addr_single_locked(struct xsk_buff_pool *pool, | ||
| 888 | struct xdp_desc *desc) | ||
| 889 | { | ||
| 890 | unsigned long flags; | ||
| 891 | u32 idx; | ||
| 892 | |||
| 893 | spin_lock_irqsave(&pool->cq_prod_lock, flags); | ||
| 894 | idx = xskq_get_prod(pool->cq); | ||
| 895 | xskq_prod_write_addr(pool->cq, idx, desc->addr); | ||
| 896 | xskq_prod_submit_n(pool->cq, 1); | ||
| 897 | spin_unlock_irqrestore(&pool->cq_prod_lock, flags); | ||
| 898 | } | ||
| 899 | |||
| 740 | static void xsk_cq_cancel_locked(struct xsk_buff_pool *pool, u32 n) | 900 | static void xsk_cq_cancel_locked(struct xsk_buff_pool *pool, u32 n) |
| 741 | { | 901 | { |
| 742 | spin_lock(&pool->cq->cq_cached_prod_lock); | 902 | spin_lock(&pool->cq->cq_cached_prod_lock); |
| @@ -794,8 +954,11 @@ static void xsk_consume_skb(struct sk_buff *skb) · 51 unmodified lines | |||
| 794 | 954 | ||
| 795 | static void xsk_drop_skb(struct sk_buff *skb) | 955 | static void xsk_drop_skb(struct sk_buff *skb) |
| 796 | { | 956 | { |
| 797 | xdp_sk(skb->sk)->tx->invalid_descs += xsk_get_num_desc(skb); | 957 | struct xdp_sock *xs = xdp_sk(skb->sk); |
| 798 | xsk_consume_skb(skb); | 958 | |
| 959 | xs->tx->invalid_descs += xsk_get_num_desc(skb); | ||
| 960 | consume_skb(skb); | ||
| 961 | xs->skb = NULL; | ||
| 799 | } | 962 | } |
| 800 | 963 | ||
| 801 | static int xsk_skb_metadata(struct sk_buff *skb, void *buffer, | 964 | static int xsk_skb_metadata(struct sk_buff *skb, void *buffer, |
| @@ -877,7 +1040,7 @@ static struct sk_buff *xsk_build_skb_zerocopy(struct xdp_sock *xs, · 75 unmodified lines | |||
| 877 | return ERR_PTR(-ENOMEM); | 1040 | return ERR_PTR(-ENOMEM); |
| 878 | 1041 | ||
| 879 | /* in case of -EOVERFLOW that could happen below, | 1042 | /* in case of -EOVERFLOW that could happen below, |
| 880 | * xsk_consume_skb() will release this node as whole skb | 1043 | * xsk_drop_skb() will release this node as whole skb |
| 881 | * would be dropped, which implies freeing all list elements | 1044 | * would be dropped, which implies freeing all list elements |
| 882 | */ | 1045 | */ |
| 883 | xsk_addr->addrs[xsk_addr->num_descs] = desc->addr; | 1046 | xsk_addr->addrs[xsk_addr->num_descs] = desc->addr; |
| @@ -969,6 +1132,8 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs, · 85 unmodified lines | |||
| 969 | goto free_err; | 1132 | goto free_err; |
| 970 | } | 1133 | } |
| 971 | 1134 | ||
| 1135 | xsk_addr->addrs[xsk_addr->num_descs] = desc->addr; | ||
| 1136 | |||
| 972 | if (unlikely(nr_frags == (MAX_SKB_FRAGS - 1) && xp_mb_desc(desc))) { | 1137 | if (unlikely(nr_frags == (MAX_SKB_FRAGS - 1) && xp_mb_desc(desc))) { |
| 973 | err = -EOVERFLOW; | 1138 | err = -EOVERFLOW; |
| 974 | goto free_err; | 1139 | goto free_err; |
| @@ -986,8 +1151,6 @@ static struct sk_buff *xsk_build_skb(struct xdp_sock *xs, · 11 unmodified lines | |||
| 986 | 1151 | ||
| 987 | skb_add_rx_frag(skb, nr_frags, page, 0, len, PAGE_SIZE); | 1152 | skb_add_rx_frag(skb, nr_frags, page, 0, len, PAGE_SIZE); |
| 988 | refcount_add(PAGE_SIZE, &xs->sk.sk_wmem_alloc); | 1153 | refcount_add(PAGE_SIZE, &xs->sk.sk_wmem_alloc); |
| 989 | |||
| 990 | xsk_addr->addrs[xsk_addr->num_descs] = desc->addr; | ||
| 991 | } | 1154 | } |
| 992 | } | 1155 | } |
| 993 | 1156 | ||
| @@ -1025,13 +1188,14 @@ free_err: · 31 unmodified lines | |||
| 1025 | static int __xsk_generic_xmit(struct sock *sk) | 1188 | static int __xsk_generic_xmit(struct sock *sk) |
| 1026 | { | 1189 | { |
| 1027 | struct xdp_sock *xs = xdp_sk(sk); | 1190 | struct xdp_sock *xs = xdp_sk(sk); |
| 1028 | bool sent_frame = false; | ||
| 1029 | struct xdp_desc desc; | 1191 | struct xdp_desc desc; |
| 1030 | struct sk_buff *skb; | 1192 | struct sk_buff *skb; |
| 1193 | u32 cached_cons; | ||
| 1031 | u32 max_batch; | 1194 | u32 max_batch; |
| 1032 | int err = 0; | 1195 | int err = 0; |
| 1033 | 1196 | ||
| 1034 | mutex_lock(&xs->mutex); | 1197 | mutex_lock(&xs->mutex); |
| 1198 | cached_cons = xs->tx->cached_cons; | ||
| 1035 | 1199 | ||
| 1036 | /* Since we dropped the RCU read lock, the socket state might have changed. */ | 1200 | /* Since we dropped the RCU read lock, the socket state might have changed. */ |
| 1037 | if (unlikely(!xsk_is_bound(xs))) { | 1201 | if (unlikely(!xsk_is_bound(xs))) { |
| @@ -1060,11 +1224,21 @@ static int __xsk_generic_xmit(struct sock *sk) · 22 unmodified lines | |||
| 1060 | goto out; | 1224 | goto out; |
| 1061 | } | 1225 | } |
| 1062 | 1226 | ||
| 1227 | if (unlikely(xs->drain_cont)) { | ||
| 1228 | xsk_cq_submit_addr_single_locked(xs->pool, &desc); | ||
| 1229 | xs->tx->invalid_descs++; | ||
| 1230 | xskq_cons_release(xs->tx); | ||
| 1231 | xs->drain_cont = xp_mb_desc(&desc); | ||
| 1232 | continue; | ||
| 1233 | } | ||
| 1234 | |||
| 1063 | skb = xsk_build_skb(xs, &desc); | 1235 | skb = xsk_build_skb(xs, &desc); |
| 1064 | if (IS_ERR(skb)) { | 1236 | if (IS_ERR(skb)) { |
| 1065 | err = PTR_ERR(skb); | 1237 | err = PTR_ERR(skb); |
| 1066 | if (err != -EOVERFLOW) | 1238 | if (err != -EOVERFLOW) |
| 1067 | goto out; | 1239 | goto out; |
| 1240 | if (xp_mb_desc(&desc)) | ||
| 1241 | xs->drain_cont = true; | ||
| 1068 | err = 0; | 1242 | err = 0; |
| 1069 | continue; | 1243 | continue; |
| 1070 | } | 1244 | } |
| @@ -1093,18 +1267,33 @@ static int __xsk_generic_xmit(struct sock *sk) · 22 unmodified lines | |||
| 1093 | goto out; | 1267 | goto out; |
| 1094 | } | 1268 | } |
| 1095 | 1269 | ||
| 1096 | sent_frame = true; | ||
| 1097 | xs->skb = NULL; | 1270 | xs->skb = NULL; |
| 1098 | } | 1271 | } |
| 1099 | 1272 | ||
| 1100 | if (xskq_has_descs(xs->tx)) { | 1273 | if (xskq_has_descs(xs->tx)) { |
| 1274 | bool drain = xs->skb || xs->drain_cont || xp_mb_desc(&desc); | ||
| 1275 | |||
| 1276 | err = xsk_cq_reserve_locked(xs->pool); | ||
| 1277 | if (err) { | ||
| 1278 | xs->tx->invalid_descs--; | ||
| 1279 | if (xs->skb) | ||
| 1280 | xsk_drop_skb(xs->skb); | ||
| 1281 | xs->drain_cont = drain; | ||
| 1282 | err = -EAGAIN; | ||
| 1283 | goto out; | ||
| 1284 | } | ||
| 1285 | |||
| 1101 | if (xs->skb) | 1286 | if (xs->skb) |
| 1102 | xsk_drop_skb(xs->skb); | 1287 | xsk_drop_skb(xs->skb); |
| 1288 | |||
| 1289 | xsk_cq_submit_addr_single_locked(xs->pool, &desc); | ||
| 1290 | |||
| 1103 | xskq_cons_release(xs->tx); | 1291 | xskq_cons_release(xs->tx); |
| 1292 | xs->drain_cont = xp_mb_desc(&desc); | ||
| 1104 | } | 1293 | } |
| 1105 | 1294 | ||
| 1106 | out: | 1295 | out: |
| 1107 | if (sent_frame) | 1296 | if (xs->tx->cached_cons != cached_cons) |
| 1108 | __xsk_tx_release(xs); | 1297 | __xsk_tx_release(xs); |
| 1109 | 1298 | ||
| 1110 | mutex_unlock(&xs->mutex); | 1299 | mutex_unlock(&xs->mutex); |
| @@ -1483,7 +1672,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr_unsized *addr, int addr · 372 unmodified lines | |||
| 1483 | * and/or device. | 1672 | * and/or device. |
| 1484 | */ | 1673 | */ |
| 1485 | xs->pool = xp_create_and_assign_umem(xs, | 1674 | xs->pool = xp_create_and_assign_umem(xs, |
| 1486 | umem_xs->umem); | 1675 | umem_xs->umem, |
| 1676 | dev->xdp_zc_max_segs); | ||
| 1487 | if (!xs->pool) { | 1677 | if (!xs->pool) { |
| 1488 | err = -ENOMEM; | 1678 | err = -ENOMEM; |
| 1489 | sockfd_put(sock); | 1679 | sockfd_put(sock); |
| @@ -1515,7 +1705,8 @@ static int xsk_bind(struct socket *sock, struct sockaddr_unsized *addr, int addr · 25 unmodified lines | |||
| 1515 | * utilizes | 1705 | * utilizes |
| 1516 | */ | 1706 | */ |
| 1517 | if (xs->tx && !xs->pool->tx_descs) { | 1707 | if (xs->tx && !xs->pool->tx_descs) { |
| 1518 | err = xp_alloc_tx_descs(xs->pool, xs); | 1708 | err = xp_alloc_tx_descs(xs->pool, xs, |
| 1709 | dev->xdp_zc_max_segs); | ||
| 1519 | if (err) { | 1710 | if (err) { |
| 1520 | xp_put_pool(xs->pool); | 1711 | xp_put_pool(xs->pool); |
| 1521 | xs->pool = NULL; | 1712 | xs->pool = NULL; |
| @@ -1533,7 +1724,9 @@ static int xsk_bind(struct socket *sock, struct sockaddr_unsized *addr, int addr · 11 unmodified lines | |||
| 1533 | goto out_unlock; | 1724 | goto out_unlock; |
| 1534 | } else { | 1725 | } else { |
| 1535 | /* This xsk has its own umem. */ | 1726 | /* This xsk has its own umem. */ |
| 1536 | xs->pool = xp_create_and_assign_umem(xs, xs->umem); | 1727 | xs->pool = xp_create_and_assign_umem(xs, xs->umem, |
| 1728 | dev->xdp_zc_max_segs); | ||
| 1729 | |||
| 1537 | if (!xs->pool) { | 1730 | if (!xs->pool) { |
| 1538 | err = -ENOMEM; | 1731 | err = -ENOMEM; |
| 1539 | goto out_unlock; | 1732 | goto out_unlock; |
| @@ -42,17 +42,22 @@ void xp_destroy(struct xsk_buff_pool *pool) · 41 unmodified lines | |||
| 42 | kvfree(pool); | 42 | kvfree(pool); |
| 43 | } | 43 | } |
| 44 | 44 | ||
| 45 | int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs) | 45 | int xp_alloc_tx_descs(struct xsk_buff_pool *pool, struct xdp_sock *xs, |
| 46 | u32 max_segs) | ||
| 46 | { | 47 | { |
| 47 | pool->tx_descs = kvzalloc_objs(*pool->tx_descs, xs->tx->nentries); | 48 | u32 nentries = max(xs->tx->nentries, max_segs); |
| 49 | |||
| 50 | pool->tx_descs = kvzalloc_objs(*pool->tx_descs, nentries); | ||
| 48 | if (!pool->tx_descs) | 51 | if (!pool->tx_descs) |
| 49 | return -ENOMEM; | 52 | return -ENOMEM; |
| 50 | 53 | ||
| 54 | pool->tx_descs_nentries = nentries; | ||
| 51 | return 0; | 55 | return 0; |
| 52 | } | 56 | } |
| 53 | 57 | ||
| 54 | struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, | 58 | struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, |
| 55 | struct xdp_umem *umem) | 59 | struct xdp_umem *umem, |
| 60 | u32 max_segs) | ||
| 56 | { | 61 | { |
| 57 | bool unaligned = umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG; | 62 | bool unaligned = umem->flags & XDP_UMEM_UNALIGNED_CHUNK_FLAG; |
| 58 | struct xsk_buff_pool *pool; | 63 | struct xsk_buff_pool *pool; |
| @@ -69,7 +74,7 @@ struct xsk_buff_pool *xp_create_and_assign_umem(struct xdp_sock *xs, · 10 unmodified lines | |||
| 69 | goto out; | 74 | goto out; |
| 70 | 75 | ||
| 71 | if (xs->tx) | 76 | if (xs->tx) |
| 72 | if (xp_alloc_tx_descs(pool, xs)) | 77 | if (xp_alloc_tx_descs(pool, xs, max_segs)) |
| 73 | goto out; | 78 | goto out; |
| 74 | 79 | ||
| 75 | pool->chunk_mask = ~((u64)umem->chunk_size - 1); | 80 | pool->chunk_mask = ~((u64)umem->chunk_size - 1); |
| @@ -58,6 +58,17 @@ struct parsed_desc { · 57 unmodified lines | |||
| 58 | u32 valid; | 58 | u32 valid; |
| 59 | }; | 59 | }; |
| 60 | 60 | ||
| 61 | struct xsk_tx_batch { | ||
| 62 | u32 tx_descs; | ||
| 63 | u32 reclaim_descs; | ||
| 64 | bool budget_limited; | ||
| 65 | }; | ||
| 66 | |||
| 67 | static inline u32 xsk_tx_batch_cq_descs(const struct xsk_tx_batch *batch) | ||
| 68 | { | ||
| 69 | return batch->tx_descs + batch->reclaim_descs; | ||
| 70 | } | ||
| 71 | |||
| 61 | /* The structure of the shared state of the rings are a simple | 72 | /* The structure of the shared state of the rings are a simple |
| 62 | * circular buffer, as outlined in | 73 | * circular buffer, as outlined in |
| 63 | * Documentation/core-api/circular-buffers.rst. For the Rx and | 74 | * Documentation/core-api/circular-buffers.rst. For the Rx and |
| @@ -263,17 +274,18 @@ static inline void parse_desc(struct xsk_queue *q, struct xsk_buff_pool *pool, · 199 unmodified lines | |||
| 263 | parsed->mb = xp_mb_desc(desc); | 274 | parsed->mb = xp_mb_desc(desc); |
| 264 | } | 275 | } |
| 265 | 276 | ||
| 266 | static inline | 277 | static inline struct xsk_tx_batch |
| 267 | u32 xskq_cons_read_desc_batch(struct xsk_queue *q, struct xsk_buff_pool *pool, | 278 | xskq_cons_read_desc_batch(struct xdp_sock *xs, struct xsk_buff_pool *pool, |
| 268 | u32 max) | 279 | struct xdp_desc *descs, u32 max) |
| 269 | { | 280 | { |
| 270 | u32 cached_cons = q->cached_cons, nb_entries = 0; | 281 | bool drain = READ_ONCE(xs->drain_cont); |
| 271 | struct xdp_desc *descs = pool->tx_descs; | 282 | u32 cached_cons, nb_entries = 0; |
| 272 | u32 total_descs = 0, nr_frags = 0; | 283 | struct xsk_tx_batch batch = {}; |
| 284 | struct xsk_queue *q = xs->tx; | ||
| 285 | u32 nr_frags = 0; | ||
| 286 | |||
| 287 | cached_cons = q->cached_cons; | ||
| 273 | 288 | ||
| 274 | /* track first entry, if stumble upon *any* invalid descriptor, rewind | ||
| 275 | * current packet that consists of frags and stop the processing | ||
| 276 | */ | ||
| 277 | while (cached_cons != q->cached_prod && nb_entries < max) { | 289 | while (cached_cons != q->cached_prod && nb_entries < max) { |
| 278 | struct xdp_rxtx_ring *ring = (struct xdp_rxtx_ring *)q->ring; | 290 | struct xdp_rxtx_ring *ring = (struct xdp_rxtx_ring *)q->ring; |
| 279 | u32 idx = cached_cons & q->ring_mask; | 291 | u32 idx = cached_cons & q->ring_mask; |
| @@ -283,25 +295,42 @@ u32 xskq_cons_read_desc_batch(struct xsk_queue *q, struct xsk_buff_pool *pool, · 3 unmodified lines | |||
| 283 | cached_cons++; | 295 | cached_cons++; |
| 284 | parse_desc(q, pool, &descs[nb_entries], &parsed); | 296 | parse_desc(q, pool, &descs[nb_entries], &parsed); |
| 285 | if (unlikely(!parsed.valid)) | 297 | if (unlikely(!parsed.valid)) |
| 286 | break; | 298 | drain = true; |
| 299 | |||
| 300 | nr_frags++; | ||
| 301 | nb_entries++; | ||
| 287 | 302 | ||
| 288 | if (likely(!parsed.mb)) { | 303 | if (likely(!parsed.mb)) { |
| 289 | total_descs += (nr_frags + 1); | 304 | if (unlikely(drain)) { |
| 290 | nr_frags = 0; | 305 | batch.reclaim_descs = nr_frags; |
| 291 | } else { | 306 | WRITE_ONCE(xs->drain_cont, false); |
| 292 | nr_frags++; | ||
| 293 | if (nr_frags == pool->xdp_zc_max_segs) { | ||
| 294 | nr_frags = 0; | 307 | nr_frags = 0; |
| 295 | break; | 308 | break; |
| 296 | } | 309 | } |
| 310 | |||
| 311 | batch.tx_descs += nr_frags; | ||
| 312 | nr_frags = 0; | ||
| 313 | continue; | ||
| 314 | } | ||
| 315 | |||
| 316 | if (nr_frags == pool->xdp_zc_max_segs) | ||
| 317 | drain = true; | ||
| 318 | } | ||
| 319 | |||
| 320 | if (nr_frags) { | ||
| 321 | if (drain) { | ||
| 322 | batch.reclaim_descs = nr_frags; | ||
| 323 | WRITE_ONCE(xs->drain_cont, true); | ||
| 324 | } else { | ||
| 325 | if (nb_entries == max) | ||
| 326 | batch.budget_limited = true; | ||
| 327 | cached_cons -= nr_frags; | ||
| 297 | } | 328 | } |
| 298 | nb_entries++; | ||
| 299 | } | 329 | } |
| 300 | 330 | ||
| 301 | cached_cons -= nr_frags; | ||
| 302 | /* Release valid plus any invalid entries */ | 331 | /* Release valid plus any invalid entries */ |
| 303 | xskq_cons_release_n(q, cached_cons - q->cached_cons); | 332 | xskq_cons_release_n(q, cached_cons - q->cached_cons); |
| 304 | return total_descs; | 333 | return batch; |
| 305 | } | 334 | } |
| 306 | 335 | ||
| 307 | /* Functions for consumers */ | 336 | /* Functions for consumers */ |
| @@ -427,14 +427,14 @@ static u32 pkt_nb_frags(u32 frame_size, struct pkt_stream *pkt_stream, struct pk · 426 unmodified lines | |||
| 427 | } | 427 | } |
| 428 | 428 | ||
| 429 | /* Search for the end of the packet in verbatim mode */ | 429 | /* Search for the end of the packet in verbatim mode */ |
| 430 | if (!pkt_continues(pkt->options) || !pkt->valid) | 430 | if (!pkt_continues(pkt->options)) |
| 431 | return nb_frags; | 431 | return nb_frags; |
| 432 | 432 | ||
| 433 | next_frag = pkt_stream->current_pkt_nb; | 433 | next_frag = pkt_stream->current_pkt_nb; |
| 434 | pkt++; | 434 | pkt++; |
| 435 | while (next_frag++ < pkt_stream->nb_pkts) { | 435 | while (next_frag++ < pkt_stream->nb_pkts) { |
| 436 | nb_frags++; | 436 | nb_frags++; |
| 437 | if (!pkt_continues(pkt->options) || !pkt->valid) | 437 | if (!pkt_continues(pkt->options)) |
| 438 | break; | 438 | break; |
| 439 | pkt++; | 439 | pkt++; |
| 440 | } | 440 | } |
| @@ -665,11 +665,11 @@ static struct pkt_stream *__pkt_stream_generate_custom(struct ifobject *ifobj, s · 224 unmodified lines | |||
| 665 | if (!frame->valid || !pkt_continues(frame->options)) | 665 | if (!frame->valid || !pkt_continues(frame->options)) |
| 666 | payload++; | 666 | payload++; |
| 667 | } else { | 667 | } else { |
| 668 | if (frame->valid) | 668 | if (frame->valid) { |
| 669 | len += frame->len; | 669 | len += frame->len; |
| 670 | if (frame->valid && pkt_continues(frame->options)) | 670 | if (pkt_continues(frame->options)) |
| 671 | continue; | 671 | continue; |
| 672 | 672 | } | |
| 673 | pkt->pkt_nb = pkt_nb; | 673 | pkt->pkt_nb = pkt_nb; |
| 674 | pkt->len = len; | 674 | pkt->len = len; |
| 675 | pkt->valid = frame->valid; | 675 | pkt->valid = frame->valid; |
| @@ -1250,10 +1250,9 @@ static int __send_pkts(struct ifobject *ifobject, struct xsk_socket_info *xsk, · 574 unmodified lines | |||
| 1250 | } | 1250 | } |
| 1251 | } | 1251 | } |
| 1252 | 1252 | ||
| 1253 | if (pkt && pkt->valid) { | 1253 | if (pkt && pkt->valid) |
| 1254 | valid_pkts++; | 1254 | valid_pkts++; |
| 1255 | valid_frags += nb_frags; | 1255 | valid_frags += nb_frags; |
| 1256 | } | ||
| 1257 | } | 1256 | } |
| 1258 | 1257 | ||
| 1259 | pthread_mutex_lock(&pacing_mutex); | 1258 | pthread_mutex_lock(&pacing_mutex); |
| @@ -2099,13 +2098,16 @@ int testapp_invalid_desc_mb(struct test_spec *test) · 839 unmodified lines | |||
| 2099 | {0, 0, 0, false, 0}, | 2098 | {0, 0, 0, false, 0}, |
| 2100 | /* Invalid address in the second frame */ | 2099 | /* Invalid address in the second frame */ |
| 2101 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, | 2100 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2102 | {umem_sz, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, | 2101 | {umem_sz * 2, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2102 | {0, MIN_PKT_SIZE, 0, false, 0}, | ||
| 2103 | /* Invalid len in the middle */ | 2103 | /* Invalid len in the middle */ |
| 2104 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, | 2104 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2105 | {0, XSK_UMEM__INVALID_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, | 2105 | {0, XSK_UMEM__INVALID_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2106 | {0, MIN_PKT_SIZE, 0, false, 0}, | ||
| 2106 | /* Invalid options in the middle */ | 2107 | /* Invalid options in the middle */ |
| 2107 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, | 2108 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XDP_PKT_CONTD}, |
| 2108 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XSK_DESC__INVALID_OPTION}, | 2109 | {0, XSK_UMEM__LARGE_FRAME_SIZE, 0, false, XSK_DESC__INVALID_OPTION}, |
| 2110 | {0, MIN_PKT_SIZE, 0, false, 0}, | ||
| 2109 | /* Transmit 2 frags, receive 3 */ | 2111 | /* Transmit 2 frags, receive 3 */ |
| 2110 | {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, XDP_PKT_CONTD}, | 2112 | {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, XDP_PKT_CONTD}, |
| 2111 | {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, 0}, | 2113 | {0, XSK_UMEM__MAX_FRAME_SIZE, 0, true, 0}, |
| @@ -2117,8 +2119,8 @@ int testapp_invalid_desc_mb(struct test_spec *test) · 5 unmodified lines | |||
| 2117 | 2119 | ||
| 2118 | if (umem->unaligned_mode) { | 2120 | if (umem->unaligned_mode) { |
| 2119 | /* Crossing a chunk boundary allowed */ | 2121 | /* Crossing a chunk boundary allowed */ |
| 2120 | pkts[12].valid = true; | 2122 | pkts[15].valid = true; |
| 2121 | pkts[13].valid = true; | 2123 | pkts[16].valid = true; |
| 2122 | } | 2124 | } |
| 2123 | 2125 | ||
| 2124 | test->mtu = MAX_ETH_JUMBO_SIZE; | 2126 | test->mtu = MAX_ETH_JUMBO_SIZE; |
| @@ -2270,7 +2272,7 @@ int testapp_too_many_frags(struct test_spec *test) · 145 unmodified lines | |||
| 2270 | max_frags += 1; | 2272 | max_frags += 1; |
| 2271 | } | 2273 | } |
| 2272 | 2274 | ||
| 2273 | pkts = calloc(2 * max_frags + 2, sizeof(struct pkt)); | 2275 | pkts = calloc(2 * max_frags + 3, sizeof(struct pkt)); |
| 2274 | if (!pkts) | 2276 | if (!pkts) |
| 2275 | return TEST_FAILURE; | 2277 | return TEST_FAILURE; |
| 2276 | 2278 | ||
| @@ -2288,24 +2290,30 @@ int testapp_too_many_frags(struct test_spec *test) · 11 unmodified lines | |||
| 2288 | } | 2290 | } |
| 2289 | pkts[max_frags].options = 0; | 2291 | pkts[max_frags].options = 0; |
| 2290 | 2292 | ||
| 2291 | /* An invalid packet with the max amount of frags but signals packet | 2293 | /* An invalid packet with the max + 1 amount of frags */ |
| 2292 | * continues on the last frag | 2294 | for (i = max_frags + 1; i < 2 * max_frags + 2; i++) { |
| 2293 | */ | ||
| 2294 | for (i = max_frags + 1; i < 2 * max_frags + 1; i++) { | ||
| 2295 | pkts[i].len = MIN_PKT_SIZE; | 2295 | pkts[i].len = MIN_PKT_SIZE; |
| 2296 | pkts[i].options = XDP_PKT_CONTD; | 2296 | pkts[i].options = XDP_PKT_CONTD; |
| 2297 | pkts[i].valid = false; | 2297 | pkts[i].valid = true; |
| 2298 | } | 2298 | } |
| 2299 | pkts[2 * max_frags + 1].options = 0; | ||
| 2299 | 2300 | ||
| 2300 | /* Valid packet for synch */ | 2301 | /* Valid packet for synch */ |
| 2301 | pkts[2 * max_frags + 1].len = MIN_PKT_SIZE; | 2302 | pkts[2 * max_frags + 2].len = MIN_PKT_SIZE; |
| 2302 | pkts[2 * max_frags + 1].valid = true; | 2303 | pkts[2 * max_frags + 2].valid = true; |
| 2303 | 2304 | ||
| 2304 | if (pkt_stream_generate_custom(test, pkts, 2 * max_frags + 2)) { | 2305 | if (pkt_stream_generate_custom(test, pkts, 2 * max_frags + 3)) { |
| 2305 | free(pkts); | 2306 | free(pkts); |
| 2306 | return TEST_FAILURE; | 2307 | return TEST_FAILURE; |
| 2307 | } | 2308 | } |
| 2308 | 2309 | ||
| 2310 | /* The generated Tx stream must keep the too-big packet valid so that | ||
| 2311 | * __send_pkts() accounts its descriptors in outstanding_tx. The Rx | ||
| 2312 | * stream, however, must not expect this packet on the wire. | ||
| 2313 | */ | ||
| 2314 | test->ifobj_rx->xsk->pkt_stream->pkts[2].valid = false; | ||
| 2315 | test->ifobj_rx->xsk->pkt_stream->nb_valid_entries--; | ||
| 2316 | |||
| 2309 | ret = testapp_validate_traffic(test); | 2317 | ret = testapp_validate_traffic(test); |
| 2310 | free(pkts); | 2318 | free(pkts); |
| 2311 | return ret; | 2319 | return ret; |
| @@ -6,3 +6,4 @@ scm_rights · 5 unmodified lines | |||
| 6 | so_peek_off | 6 | so_peek_off |
| 7 | unix_connect | 7 | unix_connect |
| 8 | unix_connreset | 8 | unix_connreset |
| 9 | unix_listen | ||
| @@ -14,6 +14,7 @@ TEST_GEN_PROGS := \ · 13 unmodified lines | |||
| 14 | so_peek_off \ | 14 | so_peek_off \ |
| 15 | unix_connect \ | 15 | unix_connect \ |
| 16 | unix_connreset \ | 16 | unix_connreset \ |
| 17 | unix_listen \ | ||
| 17 | # end of TEST_GEN_PROGS | 18 | # end of TEST_GEN_PROGS |
| 18 | 19 | ||
| 19 | include ../../lib.mk | 20 | include ../../lib.mk |
| @@ -0,0 +1,187 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0 | ||
| 2 | /* | ||
| 3 | * Tests for the state checks in AF_UNIX listen(). | ||
| 4 | * | ||
| 5 | * The central case is a regression test: listen() on a bound socket that | ||
| 6 | * is already connected (i.e. not in TCP_CLOSE or TCP_LISTEN state) must | ||
| 7 | * fail with EINVAL. A prior change accidentally let it return success | ||
| 8 | * without doing anything, because a helper called in between reset the | ||
| 9 | * error code to 0. The neighbouring checks (unbound, already listening) | ||
| 10 | * are tested too so they cannot silently regress the same way. | ||
| 11 | * | ||
| 12 | * Every case runs for both listenable socket types (SOCK_STREAM and | ||
| 13 | * SOCK_SEQPACKET) and both pathname and abstract addresses. | ||
| 14 | */ | ||
| 15 | #define _GNU_SOURCE | ||
| 16 | |||
| 17 | #include <errno.h> | ||
| 18 | #include <stddef.h> | ||
| 19 | #include <stdio.h> | ||
| 20 | #include <string.h> | ||
| 21 | #include <unistd.h> | ||
| 22 | |||
| 23 | #include <sys/socket.h> | ||
| 24 | #include <sys/un.h> | ||
| 25 | |||
| 26 | #include "kselftest_harness.h" | ||
| 27 | |||
| 28 | #define SK_NAME "unix_listen_sk" | ||
| 29 | #define SRV_NAME "unix_listen_srv" | ||
| 30 | |||
| 31 | FIXTURE(unix_listen) | ||
| 32 | { | ||
| 33 | int sk; /* socket under test */ | ||
| 34 | int server; /* a listening peer, when a test needs one */ | ||
| 35 | struct sockaddr_un addr, srv_addr; | ||
| 36 | socklen_t addrlen, srv_addrlen; | ||
| 37 | }; | ||
| 38 | |||
| 39 | FIXTURE_VARIANT(unix_listen) | ||
| 40 | { | ||
| 41 | int type; | ||
| 42 | int abstract; | ||
| 43 | }; | ||
| 44 | |||
| 45 | FIXTURE_VARIANT_ADD(unix_listen, stream_pathname) | ||
| 46 | { | ||
| 47 | .type = SOCK_STREAM, | ||
| 48 | .abstract = 0, | ||
| 49 | }; | ||
| 50 | |||
| 51 | FIXTURE_VARIANT_ADD(unix_listen, stream_abstract) | ||
| 52 | { | ||
| 53 | .type = SOCK_STREAM, | ||
| 54 | .abstract = 1, | ||
| 55 | }; | ||
| 56 | |||
| 57 | FIXTURE_VARIANT_ADD(unix_listen, seqpacket_pathname) | ||
| 58 | { | ||
| 59 | .type = SOCK_SEQPACKET, | ||
| 60 | .abstract = 0, | ||
| 61 | }; | ||
| 62 | |||
| 63 | FIXTURE_VARIANT_ADD(unix_listen, seqpacket_abstract) | ||
| 64 | { | ||
| 65 | .type = SOCK_SEQPACKET, | ||
| 66 | .abstract = 1, | ||
| 67 | }; | ||
| 68 | |||
| 69 | /* Fill @addr with a pathname or abstract address named @name. */ | ||
| 70 | static socklen_t unix_set_addr(struct sockaddr_un *addr, const char *name, | ||
| 71 | int abstract) | ||
| 72 | { | ||
| 73 | size_t len = strlen(name); | ||
| 74 | |||
| 75 | memset(addr, 0, sizeof(*addr)); | ||
| 76 | addr->sun_family = AF_UNIX; | ||
| 77 | /* An abstract address leads with a NUL and has no filesystem entry. */ | ||
| 78 | memcpy(addr->sun_path + (abstract ? 1 : 0), name, len); | ||
| 79 | |||
| 80 | return offsetof(struct sockaddr_un, sun_path) + len + 1; | ||
| 81 | } | ||
| 82 | |||
| 83 | FIXTURE_SETUP(unix_listen) | ||
| 84 | { | ||
| 85 | self->sk = -1; | ||
| 86 | self->server = -1; | ||
| 87 | self->addrlen = unix_set_addr(&self->addr, SK_NAME, variant->abstract); | ||
| 88 | self->srv_addrlen = unix_set_addr(&self->srv_addr, SRV_NAME, | ||
| 89 | variant->abstract); | ||
| 90 | } | ||
| 91 | |||
| 92 | FIXTURE_TEARDOWN(unix_listen) | ||
| 93 | { | ||
| 94 | if (self->sk >= 0) | ||
| 95 | close(self->sk); | ||
| 96 | if (self->server >= 0) | ||
| 97 | close(self->server); | ||
| 98 | |||
| 99 | /* Pathname sockets leave a filesystem entry behind; abstract ones do not. */ | ||
| 100 | if (!variant->abstract) { | ||
| 101 | remove(SK_NAME); | ||
| 102 | remove(SRV_NAME); | ||
| 103 | } | ||
| 104 | } | ||
| 105 | |||
| 106 | /* A bound socket in TCP_CLOSE is the normal, allowed case. */ | ||
| 107 | TEST_F(unix_listen, bound_is_ok) | ||
| 108 | { | ||
| 109 | int err; | ||
| 110 | |||
| 111 | self->sk = socket(AF_UNIX, variant->type, 0); | ||
| 112 | ASSERT_LE(0, self->sk); | ||
| 113 | |||
| 114 | err = bind(self->sk, (struct sockaddr *)&self->addr, self->addrlen); | ||
| 115 | ASSERT_EQ(0, err); | ||
| 116 | |||
| 117 | err = listen(self->sk, 8); | ||
| 118 | EXPECT_EQ(0, err); | ||
| 119 | } | ||
| 120 | |||
| 121 | /* Listening again on an already-listening socket (TCP_LISTEN) is allowed. */ | ||
| 122 | TEST_F(unix_listen, relisten_is_ok) | ||
| 123 | { | ||
| 124 | int err; | ||
| 125 | |||
| 126 | self->sk = socket(AF_UNIX, variant->type, 0); | ||
| 127 | ASSERT_LE(0, self->sk); | ||
| 128 | |||
| 129 | err = bind(self->sk, (struct sockaddr *)&self->addr, self->addrlen); | ||
| 130 | ASSERT_EQ(0, err); | ||
| 131 | |||
| 132 | err = listen(self->sk, 8); | ||
| 133 | ASSERT_EQ(0, err); | ||
| 134 | |||
| 135 | err = listen(self->sk, 16); | ||
| 136 | EXPECT_EQ(0, err); | ||
| 137 | } | ||
| 138 | |||
| 139 | /* listen() on an unbound socket fails: there is nothing to listen on. */ | ||
| 140 | TEST_F(unix_listen, unbound_is_einval) | ||
| 141 | { | ||
| 142 | int err; | ||
| 143 | |||
| 144 | self->sk = socket(AF_UNIX, variant->type, 0); | ||
| 145 | ASSERT_LE(0, self->sk); | ||
| 146 | |||
| 147 | err = listen(self->sk, 8); | ||
| 148 | EXPECT_EQ(-1, err); | ||
| 149 | EXPECT_EQ(EINVAL, errno); | ||
| 150 | } | ||
| 151 | |||
| 152 | /* | ||
| 153 | * The regression: a bound socket that has already been connected is not in | ||
| 154 | * TCP_CLOSE or TCP_LISTEN, so listen() must reject it with EINVAL rather | ||
| 155 | * than quietly succeeding. | ||
| 156 | */ | ||
| 157 | TEST_F(unix_listen, connected_is_einval) | ||
| 158 | { | ||
| 159 | int err; | ||
| 160 | |||
| 161 | self->server = socket(AF_UNIX, variant->type, 0); | ||
| 162 | ASSERT_LE(0, self->server); | ||
| 163 | |||
| 164 | err = bind(self->server, (struct sockaddr *)&self->srv_addr, | ||
| 165 | self->srv_addrlen); | ||
| 166 | ASSERT_EQ(0, err); | ||
| 167 | |||
| 168 | err = listen(self->server, 8); | ||
| 169 | ASSERT_EQ(0, err); | ||
| 170 | |||
| 171 | self->sk = socket(AF_UNIX, variant->type, 0); | ||
| 172 | ASSERT_LE(0, self->sk); | ||
| 173 | |||
| 174 | /* Bind first so the unbound check does not mask the state check. */ | ||
| 175 | err = bind(self->sk, (struct sockaddr *)&self->addr, self->addrlen); | ||
| 176 | ASSERT_EQ(0, err); | ||
| 177 | |||
| 178 | err = connect(self->sk, (struct sockaddr *)&self->srv_addr, | ||
| 179 | self->srv_addrlen); | ||
| 180 | ASSERT_EQ(0, err); | ||
| 181 | |||
| 182 | err = listen(self->sk, 8); | ||
| 183 | EXPECT_EQ(-1, err); | ||
| 184 | EXPECT_EQ(EINVAL, errno); | ||
| 185 | } | ||
| 186 | |||
| 187 | TEST_HARNESS_MAIN | ||
| @@ -617,7 +617,11 @@ ip -6 -net "$nsr2" route add default via fee1:3::1 · 616 unmodified lines | |||
| 617 | ip -net "$ns2" route add default via 10.0.2.1 | 617 | ip -net "$ns2" route add default via 10.0.2.1 |
| 618 | ip -6 -net "$ns2" route add default via dead:2::1 | 618 | ip -6 -net "$ns2" route add default via dead:2::1 |
| 619 | 619 | ||
| 620 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | ||
| 621 | 'meta oif tun0 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | ||
| 620 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun0 accept' | 622 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun0 accept' |
| 623 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | ||
| 624 | 'meta oif tun6 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | ||
| 621 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun6 accept' | 625 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun6 accept' |
| 622 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | 626 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ |
| 623 | 'meta oif "veth0" tcp sport 12345 ct mark set 1 flow add @f1 counter name routed_repl accept' | 627 | 'meta oif "veth0" tcp sport 12345 ct mark set 1 flow add @f1 counter name routed_repl accept' |
| @@ -629,7 +633,7 @@ if ! test_tcp_forwarding_nat "$ns1" "$ns2" 1 "IPIP tunnel"; then · 5 unmodified lines | |||
| 629 | fi | 633 | fi |
| 630 | 634 | ||
| 631 | if test_tcp_forwarding "$ns1" "$ns2" 1 6 "[dead:2::99]" 12345; then | 635 | if test_tcp_forwarding "$ns1" "$ns2" 1 6 "[dead:2::99]" 12345; then |
| 632 | echo "PASS: flow offload for ns1/ns2 IP6IP6 tunnel" | 636 | check_counters "flow offload for ns1/ns2 IP6IP6 tunnel" |
| 633 | else | 637 | else |
| 634 | echo "FAIL: flow offload for ns1/ns2 with IP6IP6 tunnel" 1>&2 | 638 | echo "FAIL: flow offload for ns1/ns2 with IP6IP6 tunnel" 1>&2 |
| 635 | ip netns exec "$nsr1" nft list ruleset | 639 | ip netns exec "$nsr1" nft list ruleset |
| @@ -642,6 +646,8 @@ ip -net "$nsr1" link set veth1.10 up · 6 unmodified lines | |||
| 642 | ip -net "$nsr1" addr add 192.168.20.1/24 dev veth1.10 | 646 | ip -net "$nsr1" addr add 192.168.20.1/24 dev veth1.10 |
| 643 | ip -net "$nsr1" addr add fee1:4::1/64 dev veth1.10 nodad | 647 | ip -net "$nsr1" addr add fee1:4::1/64 dev veth1.10 nodad |
| 644 | ip netns exec "$nsr1" sysctl net.ipv4.conf.veth1/10.forwarding=1 > /dev/null | 648 | ip netns exec "$nsr1" sysctl net.ipv4.conf.veth1/10.forwarding=1 > /dev/null |
| 649 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | ||
| 650 | 'meta oif veth1.10 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | ||
| 645 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif veth1.10 accept' | 651 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif veth1.10 accept' |
| 646 | 652 | ||
| 647 | ip -net "$nsr1" link add name tun0.10 type ipip local 192.168.20.1 remote 192.168.20.2 | 653 | ip -net "$nsr1" link add name tun0.10 type ipip local 192.168.20.1 remote 192.168.20.2 |
| @@ -649,6 +655,8 @@ ip -net "$nsr1" link set tun0.10 up · 1 unmodified line | |||
| 649 | ip -net "$nsr1" addr add 192.168.200.1/24 dev tun0.10 | 655 | ip -net "$nsr1" addr add 192.168.200.1/24 dev tun0.10 |
| 650 | ip -net "$nsr1" route change default via 192.168.200.2 | 656 | ip -net "$nsr1" route change default via 192.168.200.2 |
| 651 | ip netns exec "$nsr1" sysctl net.ipv4.conf.tun0/10.forwarding=1 > /dev/null | 657 | ip netns exec "$nsr1" sysctl net.ipv4.conf.tun0/10.forwarding=1 > /dev/null |
| 658 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | ||
| 659 | 'meta oif tun0.10 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | ||
| 652 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun0.10 accept' | 660 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun0.10 accept' |
| 653 | 661 | ||
| 654 | ip -net "$nsr1" link add name tun6.10 type ip6tnl local fee1:4::1 remote fee1:4::2 encaplimit none | 662 | ip -net "$nsr1" link add name tun6.10 type ip6tnl local fee1:4::1 remote fee1:4::2 encaplimit none |
| @@ -656,6 +664,8 @@ ip -net "$nsr1" link set tun6.10 up · 1 unmodified line | |||
| 656 | ip -net "$nsr1" addr add fee1:5::1/64 dev tun6.10 nodad | 664 | ip -net "$nsr1" addr add fee1:5::1/64 dev tun6.10 nodad |
| 657 | ip -6 -net "$nsr1" route delete default | 665 | ip -6 -net "$nsr1" route delete default |
| 658 | ip -6 -net "$nsr1" route add default via fee1:5::2 | 666 | ip -6 -net "$nsr1" route add default via fee1:5::2 |
| 667 | ip netns exec "$nsr1" nft -a insert rule inet filter forward \ | ||
| 668 | 'meta oif tun6.10 tcp dport 12345 ct mark set 1 flow add @f1 counter name routed_orig accept' | ||
| 659 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun6.10 accept' | 669 | ip netns exec "$nsr1" nft -a insert rule inet filter forward 'meta oif tun6.10 accept' |
| 660 | 670 | ||
| 661 | ip -net "$nsr2" link add link veth0 name veth0.10 type vlan id 10 | 671 | ip -net "$nsr2" link add link veth0 name veth0.10 type vlan id 10 |
| @@ -683,7 +693,7 @@ if ! test_tcp_forwarding_nat "$ns1" "$ns2" 1 "IPIP tunnel over vlan"; then · 21 unmodified lines | |||
| 683 | fi | 693 | fi |
| 684 | 694 | ||
| 685 | if test_tcp_forwarding "$ns1" "$ns2" 1 6 "[dead:2::99]" 12345; then | 695 | if test_tcp_forwarding "$ns1" "$ns2" 1 6 "[dead:2::99]" 12345; then |
| 686 | echo "PASS: flow offload for ns1/ns2 IP6IP6 tunnel over vlan" | 696 | check_counters "flow offload for ns1/ns2 IP6IP6 tunnel over vlan" |
| 687 | else | 697 | else |
| 688 | echo "FAIL: flow offload for ns1/ns2 with IP6IP6 tunnel over vlan" 1>&2 | 698 | echo "FAIL: flow offload for ns1/ns2 with IP6IP6 tunnel over vlan" 1>&2 |
| 689 | ip netns exec "$nsr1" nft list ruleset | 699 | ip netns exec "$nsr1" nft list ruleset |