On Mon, May 28, 2007 at 04:20:32PM +0200, Sebastien Estienne wrote:

On 5/28/07, Horms [off-list ref] wrote:

quoted

On Sat, May 26, 2007 at 11:22:40AM +0900, Horms wrote:

quoted

On Fri, May 25, 2007 at 09:30:52AM +0000, Sebastien Estienne wrote:

quoted

I didn't try 2.6.21 yet, but using ubuntu dapper kernel (2.6.15) i
can't reproduce the bug.
When i was using feisty kernel (2.6.20), i can reproduce in less than 5
minutes.

I'm using lvs to loadbalance some mysql servers, i wrote a deamon that
check the synchro of the mysql replication on each slave and adjust
the wieght on the lvs every 500ms

It does look a lot like there is some sort of locking problem in there.
Would it be possible to send your kernel config, as the locking
deatails to change a little with different configs.

About the kernel .config, i'm using the vanilla kernel "-server" from
ubuntu feisty

quoted

If you also have some details of you ipvs configuration,
that might help narrow down which code-paths to investigate.

i attached the output of ipvsadm-save

i'm adjusting the weight every 500ms by generating lines like this:
-e -t 10.33.1.231:3306 -r 10.33.1.1 -w 100

Thanks.

and piping all the needed changes in ipvadm -R

it can represent something like 20 to 40 updates in one time.

i also noticed that sometimes when i execute "ipvsadm" the display get
locked in the middle for a second and then finish.

That is probably related. Though the fact that it eventually
exits seems to indicate that you're not hitting II.

quoted

II.

ip_vs_set_ctl() does seem to leak svc->usecnt in one corner case,
but I doubt that is what you are seeing - if it was your ipvsadm
command(s) would hang. The problem is a bit wordy to describe,
but this fix should illustrate the problem.

--- linux-2.6.orig/net/ipv4/ipvs/ip_vs_ctl.c
+++ linux-2.6/net/ipv4/ipvs/ip_vs_ctl.c

@@ -2000,7 +2000,7 @@ do_ip_vs_set_ctl(struct sock *sk, int cm
       if (cmd != IP_VS_SO_SET_ADD
           && (svc == NULL || svc->protocol != usvc->protocol)) {
               ret = -ESRCH;
-               goto out_unlock;
+               goto out_svc;
       }

       switch (cmd) {

@@ -2034,9 +2034,9 @@ do_ip_vs_set_ctl(struct sock *sk, int cm
               ret = -EINVAL;
       }

+  out_svc:
       if (svc)
               ip_vs_service_put(svc);
-
  out_unlock:
       mutex_unlock(&__ip_vs_mutex);
  out_dec:

-- 
Horms
  H: http://www.vergenet.net/~horms/
  W: http://www.valinux.co.jp/en/