Re: [RFC net-next PATCH V1 7/9] net: frag queue locking per hash bucket
From: Jesper Dangaard Brouer <hidden>
Date: 2012-11-27 15:02:31
On Fri, 2012-11-23 at 14:08 +0100, Jesper Dangaard Brouer wrote:
DO NOT apply - patch not finished, can cause on OOPS/PANIC during hash rebuild This patch implements per hash bucket locking for the frag queue hash. This removes two write locks, and the only remaining write lock is for protecting hash rebuild. This essentially reduce the readers-writer lock to a rebuild lock. NOT-Signed-off-by: Jesper Dangaard Brouer [off-list ref]
Last bug mentioned, were not the only one... fixing hopefully the last bug in this patch.
quoted hunk ↗ jump to hunk
diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c index 1620a21..447423f 100644 --- a/net/ipv4/inet_fragment.c +++ b/net/ipv4/inet_fragment.c@@ -35,20 +35,27 @@ static void inet_frag_secret_rebuild(unsigned long dummy) unsigned long now = jiffies; int i; + /* Per bucket lock NOT needed here, due to write lock protection */ write_lock(&f->lock); + get_random_bytes(&f->rnd, sizeof(u32)); for (i = 0; i < INETFRAGS_HASHSZ; i++) { + struct inet_frag_bucket *hb; struct inet_frag_queue *q; struct hlist_node *p, *n; - hlist_for_each_entry_safe(q, p, n, &f->hash[i], list) { + hb = &f->hash[i]; + hlist_for_each_entry_safe(q, p, n, &hb->chain, list) { unsigned int hval = f->hashfn(q); if (hval != i) { + struct inet_frag_bucket *hb_dest; + hlist_del(&q->list); /* Relink to new hash chain. */ - hlist_add_head(&q->list, &f->hash[hval]); + hb_dest = &f->hash[hval]; + hlist_add_head(&q->list, &hb->chain);
The above line were wrong, it should have been: hlist_add_head(&q->list, &hb_dest->chain);
} } }
The patch seem quite stable now. My test is to adjust to rebuild interval to 2 sec and then run 4x 10G with two fragments (packet size 1472*2) to create as many fragments as possible (approx 300 inet_frag_queue elements). 30 min test run: 3726+3896+3960+3608 = 15190 Mbit/s (For reproducers, note, that changing ipfrag_secret_interval (e.g. sysctl -w net/ipv4/ipfrag_secret_interval=2), first take effect after the first interval/timer expires, which default is 10 min) -- Best regards, Jesper Dangaard Brouer MSc.CS, Sr. Network Kernel Developer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer