Commit 501e7ef5 authored by Eric Dumazet's avatar Eric Dumazet Committed by David S. Miller
Browse files

net-rfs: fix false sharing accessing sd->input_queue_head

sd->input_queue_head is incremented for each processed packet
in process_backlog(), and read from other cpus performing
Out Of Order avoidance in get_rps_cpu()

Moving this field in a separate cache line keeps it mostly
hot for the cpu in process_backlog(), as other cpus will
only read it.

In a stress test, process_backlog() was consuming 6.80 % of cpu cycles,
and the patch reduced the cost to 0.65 %
Signed-off-by: default avatarEric Dumazet <>
Acked-by: default avatarTom Herbert <>
Signed-off-by: default avatarDavid S. Miller <>
parent 35ef7d68
......@@ -2747,11 +2747,15 @@ struct softnet_data {
struct sk_buff *completion_queue;
/* Elements below can be accessed between CPUs for RPS */
/* input_queue_head should be written by cpu owning this struct,
* and only read by other cpus. Worth using a cache line.
unsigned int input_queue_head ____cacheline_aligned_in_smp;
/* Elements below can be accessed between CPUs for RPS/RFS */
struct call_single_data csd ____cacheline_aligned_in_smp;
struct softnet_data *rps_ipi_next;
unsigned int cpu;
unsigned int input_queue_head;
unsigned int input_queue_tail;
unsigned int dropped;
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment