History

xsk: clear metadata pointer when no timestamp is requested

User space can change metadata flags after request processing. Rereading them during completion can therefore make the kernel write a timestamp that was not requested when the packet was submitted. Clear the metadata pointer during request processing unless timestamp completion is requested. Completion handling can then use the pointer itself instead of rereading the flags. On the mlx5 multi-packet WQE path metadata is evaluated per batch: xsk_tx_metadata_request() runs only for the descriptor that starts a session, just like the checksum offload that is applied once through the shared WQE. Only that descriptor's pointer is reset, so completion handling can record a timestamp for the other descriptors of the session regardless of their own XDP_TXMD_FLAGS_TIMESTAMP bit. The write stays inside the metadata area; the single-WQE, other zero-copy, and generic paths reset the pointer per descriptor and are unaffected.

Stanislav Fomichev committed 3 months ago

Commit 9f60a671 parent 19366db2 files · +12 −4Browse files

Unified · Split

2 files
  1. include/net/xdp_sock.h +10 −4
  2. net/xdp/xsk.c +2 −0
include/net/xdp_sock.h+10−4
@@ -141,13 +141,16 @@ INDIRECT_CALLABLE_DECLARE(void xsk_destruct_skb(struct sk_buff *)); · 140 unmodified lines
141static inline void xsk_tx_metadata_to_compl(struct xsk_tx_metadata *meta,141static inline void xsk_tx_metadata_to_compl(struct xsk_tx_metadata *meta,
142 struct xsk_tx_metadata_compl *compl)142 struct xsk_tx_metadata_compl *compl)
143{143{
144 compl->tx_timestamp = NULL;
145
144 if (!meta)146 if (!meta)
145 return;147 return;
146 148
147 if (meta->flags & XDP_TXMD_FLAGS_TIMESTAMP)149 /* we can only arrive here if the completion timestamp has been
148 compl->tx_timestamp = &meta->completion.tx_timestamp;150 * requested via XDP_TXMD_FLAGS_TIMESTAMP, see xsk_tx_metadata_request
149 else151 */
150 compl->tx_timestamp = NULL;152
153 compl->tx_timestamp = &meta->completion.tx_timestamp;
151}154}
152 155
153/**156/**
@@ -182,6 +185,9 @@ static inline void xsk_tx_metadata_request(struct xsk_tx_metadata **pmeta, · 28 unmodified lines
182 if (meta->flags & XDP_TXMD_FLAGS_CHECKSUM)185 if (meta->flags & XDP_TXMD_FLAGS_CHECKSUM)
183 ops->tmo_request_checksum(meta->request.csum_start,186 ops->tmo_request_checksum(meta->request.csum_start,
184 meta->request.csum_offset, priv);187 meta->request.csum_offset, priv);
188
189 if (!(meta->flags & XDP_TXMD_FLAGS_TIMESTAMP))
190 *pmeta = NULL;
185}191}
186 192
187/**193/**
net/xdp/xsk.c+2−0
@@ -998,6 +998,8 @@ static int xsk_skb_metadata(struct sk_buff *skb, void *buffer, · 997 unmodified lines
998 998
999 if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME)999 if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME)
1000 skb->skb_mstamp_ns = meta->request.launch_time;1000 skb->skb_mstamp_ns = meta->request.launch_time;
1001 if (!(meta->flags & XDP_TXMD_FLAGS_TIMESTAMP))
1002 meta = NULL;
1001 xsk_tx_metadata_to_compl(meta, &skb_shinfo(skb)->xsk_meta);1003 xsk_tx_metadata_to_compl(meta, &skb_shinfo(skb)->xsk_meta);
1002 1004
1003 return 0;1005 return 0;

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t