-----Original Message-----
From: Thomas Graf [mailto:tgraf@suug.ch] 
Sent: Friday, March 09, 2007 6:35 PM
To: Waskiewicz Jr, Peter P
Cc: Kok, Auke-jan H; David Miller; Garzik, Jeff; 
netdev@vger.kernel.org; linux-kernel@vger.kernel.org; 
Brandeburg, Jesse; Kok, Auke; Ronciak, John
Subject: Re: [PATCH 1/2] NET: Multiple queue network device support

* Waskiewicz Jr, Peter P [off-list ref] 
2007-03-09 15:27

quoted

That's the entire point of this extra locking.  enqueue() 

is going to 

quoted

put an skb into a band somewhere that maps to some queue, 

and there is 

quoted

no way to guarantee the skb I retrieve from dequeue() is headed for 
the same queue.  Therefore, I need to unlock the queue 

after I finish 

quoted

enqueuing, since having that lock makes little sense to dequeue().
dequeue() will then grab *a* lock on a queue; it may be the 

same one 

quoted

we had during enqueue(), but it may not be.  And the 

placement of the 

quoted

unlock of that queue is exactly where it happens in non-multiqueue, 
which is right before the hard_start_xmit().

The lock is already unlocked after dequeue, from your prio_dequeue():

       if (netif_is_multiqueue(sch->dev)) {
               queue = q->band2queue[prio];
               if 
(spin_trylock(&sch->dev->egress_subqueue[queue].queue_lock)) {
                       qdisc = q->queues[prio];
                       skb = qdisc->dequeue(qdisc);
                       if (skb) {
                               sch->q.qlen--;
                               skb->priority = prio;
                               
spin_unlock(&sch->dev->egress_subqueue[queue].queue_lock);
                               return skb;
                       }
                       
spin_unlock(&sch->dev->egress_subqueue[queue].queue_lock);
       }

Ok, now I see what's wrong.  Taking Dave M.'s recommendation to store
the queue mapping in the skb will let me unlock the queue when dequeue()
returns.  I'll fix this locking issue; thanks for the feedback and
persistent drilling into my thick head.

-PJ Waskiewicz
peter.p.waskiewicz.jr@intel.com