leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

net/udp: do not touch skb->peeked unless really needed

Browse Source 
In UDP recvmsg() path we currently access 3 cache lines from an skb
while holding receive queue lock, plus another one if packet is
dequeued, since we need to change skb->next->prev

1st cache line (contains ->next/prev pointers, offsets 0x00 and 0x08)
2nd cache line (skb->len & skb->peeked, offsets 0x80 and 0x8e)
3rd cache line (skb->truesize/users, offsets 0xe0 and 0xe4)

skb->peeked is only needed to make sure 0-length packets are properly
handled while MSG_PEEK is operated.

I had first the intent to remove skb->peeked but the "MSG_PEEK at
non-zero offset" support added by Sam Kumar makes this not possible.

This patch avoids one cache line miss during the locked section, when
skb->len and skb->peeked do not have to be read.

It also avoids the skb_set_peeked() cost for non empty UDP datagrams.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This commit is contained in:
Eric Dumazet 2016-12-05 09:57:19 -08:00 committed by David S. Miller
parent e466af66c7
commit a297569fe0
1 changed files with 10 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
net/core/datagram.c
View File

@ -214,6 +214,7 @@ struct sk_buff *__skb_try_recv_datagram(struct sock *sk, unsigned int flags,
if (error) if (error)
goto no_packet; goto no_packet;
*peeked = 0;
do { do {
/* Again only user level code calls this function, so nothing /* Again only user level code calls this function, so nothing
* interrupt level will suddenly eat the receive_queue. * interrupt level will suddenly eat the receive_queue.
@ -227,22 +228,22 @@ struct sk_buff *__skb_try_recv_datagram(struct sock *sk, unsigned int flags,
spin_lock_irqsave(&queue->lock, cpu_flags); spin_lock_irqsave(&queue->lock, cpu_flags);
skb_queue_walk(queue, skb) { skb_queue_walk(queue, skb) {
*last = skb; *last = skb;
*peeked = skb->peeked;
if (flags & MSG_PEEK) { if (flags & MSG_PEEK) {
if (_off >= skb->len && (skb->len || _off || if (_off >= skb->len && (skb->len || _off ||
skb->peeked)) { skb->peeked)) {
_off -= skb->len; _off -= skb->len;
continue; continue;
} }
if (!skb->len) {
skb = skb_set_peeked(skb); skb = skb_set_peeked(skb);
error = PTR_ERR(skb); if (IS_ERR(skb)) {
if (IS_ERR(skb)) { error = PTR_ERR(skb);
spin_unlock_irqrestore(&queue->lock, spin_unlock_irqrestore(&queue->lock,
cpu_flags); cpu_flags);
goto no_packet; goto no_packet;
}
} }
*peeked = 1;
atomic_inc(&skb->users); atomic_inc(&skb->users);
} else { } else {
__skb_unlink(skb, queue); __skb_unlink(skb, queue);