From: Eric Dumazet <hidden> Date: 2011-11-23 08:12:22
Le mercredi 23 novembre 2011 à 15:54 +0800, Herbert Xu a écrit :
David Miller [off-list ref] wrote:
quoted
I would like to see some discussion wrt. Jamal's feedback, which is that
a lot of the side-band functionality added by this code is either 1) already
doable with packet scheduler actions or 2) should be implemented there.
I mostly agree with Jamal. As far as the concept of a policy
lookup cache goes (which appears to be at the core of OVS), this
almost fits exactly onto a u32 hash table. All that would be needed
is to add the tail end of the policies, e.g., with new packet
actions.
However, this is purely based on my conceptual view of OVS, which
may or may not be accurate. I'll dig into the patches over the
next couple of days to see if they could be easily turned into
packet actions or whether this is difficult for reasons that we
have not yet discovered.
I had no time to look at OVS, but current tc model is not scalable,
everything is performed under a queue lock.
Maybe its time to redesign a new model, based on modern techniques.
By the way, we seriously lack good documentation on tc, not counting
many features. Code might be there, but without documenation, working
samples, who can use it ?
Take a look at last cls_flow extension, and try to use it on a real
setup, you'll find its almost not possible...
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
From: Herbert Xu <herbert@gondor.apana.org.au> Date: 2011-11-23 08:21:31
On Wed, Nov 23, 2011 at 09:12:22AM +0100, Eric Dumazet wrote:
I had no time to look at OVS, but current tc model is not scalable,
everything is performed under a queue lock.
Maybe its time to redesign a new model, based on modern techniques.
Indeed, I pointed this out numerous times over the past few years :)
However, this is something that we need to solve regardless of
whether OVS is added, since OVS isn't exactly going to replace the
packet scheduling layer.
By the way, we seriously lack good documentation on tc, not counting
many features. Code might be there, but without documenation, working
samples, who can use it ?
Take a look at last cls_flow extension, and try to use it on a real
setup, you'll find its almost not possible...
lartc.org is surprisingly good. But yes new features won't show
up there unless somebody contributes time to write it up.
Unfortunately while many love documentation, few are willing to
pay for it.
Cheers,
--
Email: Herbert Xu [off-list ref]
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
On Wed, 2011-11-23 at 09:12 +0100, Eric Dumazet wrote:
I had no time to look at OVS, but current tc model is not scalable,
everything is performed under a queue lock.
Maybe its time to redesign a new model, based on modern techniques.
Making the enqueur/dequeuer lockless would be a big win. What happened
to your idea of ring buffer?
What other hot areas do you see? It used to be ingress/egress share
the qdisc lock - but that is now gone.
By the way, we seriously lack good documentation on tc, not counting
many features. Code might be there, but without documenation, working
samples, who can use it ?
Take a look at last cls_flow extension, and try to use it on a real
setup, you'll find its almost not possible...
There's no tc-central.org unlike the nice effort the netfilter guys have
put over the years. Documentation is there - sometimes a little too much
with differing "opinions" (lartc that Herbert pointed to is a good
starting point); but googling also helps.
Unfortunately, sometimes the people who understand stuff have no
motivation to do docs.
cheers,
jamal
From: Eric Dumazet <hidden> Date: 2011-11-23 12:55:38
Le mercredi 23 novembre 2011 à 07:47 -0500, jamal a écrit :
On Wed, 2011-11-23 at 09:12 +0100, Eric Dumazet wrote:
quoted
I had no time to look at OVS, but current tc model is not scalable,
everything is performed under a queue lock.
Maybe its time to redesign a new model, based on modern techniques.
Making the enqueur/dequeuer lockless would be a big win. What happened
to your idea of ring buffer?
Currently thinking about it. I was also waiting Tom Herbert BQL patches.
Several people are interested, and John Fastabend told me he plans to :
(1) rcu'ify classifiers/actions as needed
(2) add flag to drop qdisc lock on simple or hw qdiscs
(3) mq and mqprio call root qdisc and run a pass over classifiers
actions possibly resetting queue_mapping.
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
From: David Täht <hidden> Date: 2011-11-23 13:13:39
On 11/23/2011 01:47 PM, jamal wrote:
On Wed, 2011-11-23 at 09:12 +0100, Eric Dumazet wrote:
quoted
I had no time to look at OVS, but current tc model is not scalable,
everything is performed under a queue lock.
Maybe its time to redesign a new model, based on modern techniques.
Making the enqueur/dequeuer lockless would be a big win. What happened
to your idea of ring buffer?
It's not so much 'modern tecniques', as modern environments.
High on my list would be a way to more easily expose QoS and AQM
features in the hardware all the way up the stack.
I'd like the hardware to be able to express 'I have FQ', or 'I have red',
much like we express many other features in ethtool, only abstractly
enough so that a qdisc setup can be made generic.
What other hot areas do you see? It used to be ingress/egress share
the qdisc lock - but that is now gone.
I find the mapping from hardware queues to any sort of complex software
queuing scheme hard to conceptualize. Also, as structured, tc cannot be
easily applied to wireless APs.
quoted
By the way, we seriously lack good documentation on tc, not counting
many features. Code might be there, but without documenation, working
samples, who can use it ?
I find tc's concepts incredibly difficult to use effectively. They start
with the presumption that what you are working with is a 1998 point to
point link and get harder from there. That said I think I've almost
managed to bend it to my will of late...
(this email written under the influence of Byte Queue Limits + QFQ +
RED, on ethernet)
quoted
Take a look at last cls_flow extension, and try to use it on a real
setup, you'll find its almost not possible...
There's no tc-central.org unlike the nice effort the netfilter guys have
put over the years. Documentation is there - sometimes a little too much
with differing "opinions" (lartc that Herbert pointed to is a good
starting point); but googling also helps.
Unfortunately, sometimes the people who understand stuff have no
motivation to do docs.
After burning the last several months getting good enough at the tc layer
to do stuff in it, I would certainly like to have a place to put
documentation,
and also easily update what already exists.
If it helps any I could offer a redmine instance on bufferbloat.net for
this.
redmine has bug tracking and a wiki...
It would be nice also if the iproute2 code contained more working examples,
and man pages.
It's a ton of doc work, but I'd be willing to do some of it.
cheers,
jamal
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Jamal Hadi Salim <jhs@mojatatu.com> Date: 2011-11-23 13:44:23
On Wed, 2011-11-23 at 13:55 +0100, Eric Dumazet wrote:
Currently thinking about it. I was also waiting Tom Herbert BQL patches.
Excellent. I can test when you have something.
Several people are interested, and John Fastabend told me he plans to :
(1) rcu'ify classifiers/actions as needed
Makes sense in most cases. If you have a lot of flow setup/teardown
it may harm.
Another one - but dont see how much you can do about this; useful
when you want to share state (eg multiple flows being policed
by a single rate meter);
An action could be shared across multiple policies i.e you can
have:
match1, action foo instance 1, action bar instance 3
match2, action bar instance3
match3, ....
This could would mean a lock contended across cpus when different
flows hitting match1/2 show up on different cpus.
(2) add flag to drop qdisc lock on simple or hw qdiscs
Where does config for the hardware happen from?
(3) mq and mqprio call root qdisc and run a pass over classifiers
actions possibly resetting queue_mapping.
It seems to make sense - but I will wait and see to have better
understanding.
cheers,
jamal
From: John Fastabend <hidden> Date: 2011-11-23 16:05:00
On 11/23/2011 5:44 AM, Jamal Hadi Salim wrote:
On Wed, 2011-11-23 at 13:55 +0100, Eric Dumazet wrote:
quoted
Currently thinking about it. I was also waiting Tom Herbert BQL patches.
Excellent. I can test when you have something.
quoted
Several people are interested, and John Fastabend told me he plans to :
(1) rcu'ify classifiers/actions as needed
Makes sense in most cases. If you have a lot of flow setup/teardown
it may harm.
We could have a CONFIG option to always do locking in some
cases if thats not too ugly.
Another one - but dont see how much you can do about this; useful
when you want to share state (eg multiple flows being policed
by a single rate meter);
An action could be shared across multiple policies i.e you can
have:
match1, action foo instance 1, action bar instance 3
match2, action bar instance3
match3, ....
This could would mean a lock contended across cpus when different
flows hitting match1/2 show up on different cpus.
quoted
(2) add flag to drop qdisc lock on simple or hw qdiscs
Where does config for the hardware happen from?
I assume you mean something like setup_tc() which we have
today to call into into the driver at qdisc create time. This
happens with the RTNL held. I don't see any reason not to also
call into the hardware on qdisc_change() I just haven't done
it yet.
Although I'm pretty sure we don't want to add a new ndo_ops
ever time we have some hardware feature we want to expose.
Assuming there are more than 1 or 2 hw features. So maybe
we could convert to something more generic. A setup_qos()
call that passes an skb with nl attributes.
Is that what you were asking?
quoted
(3) mq and mqprio call root qdisc and run a pass over classifiers
actions possibly resetting queue_mapping.
It seems to make sense - but I will wait and see to have better
understanding.
One of the problems this resolves is not being able to
call the classifier-actions until after the queue is
already selected. At this point you can't send it to
a higher/lower priority queue.
I'm traveling for a couple days, but I'll try to get
some actual patches out next week to illustrate this.
Thanks,
John
cheers,
jamal
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html