-----Original Message-----
From: jamal [mailto:hadi@cyberus.ca]
Sent: Saturday, August 09, 2003 1:01 AM
To: Hen, Shmulik
Cc: bonding-devel@lists.sourceforge.net; netdev@oss.sgi.com
Subject: Re: [SET 2][PATCH 2/8][bonding] Propagating master's settings
to slaves
Shmulik,
Some of this bonding stuff is pretty scary. Lotsa policies in the
kernel and communication seems to be centred around /proc.
Shouldnt policies on failover be really driven from user space?
Also shouldnt communication be using something like netlink?
cheers,
jamal
On Fri, 2003-08-08 at 10:44, Shmulik Hen wrote:
quoted
2 - Change monitoring function use the new functionality.
Not sure I fully understood the concerns above, but I'll try
to explain what the change was all about.
By monitoring, I meant the 3 timer function running in bonding
to monitor link changes and act once a link fail/recovery is
detected. The old code used to do all the activity related to
changing the current active slave separately in each timer
function and it seemed redundant since it was basically the
same thing repeated 3 times. Instead, we thought it would be
best if we put that into 3 new functions - reselect_active,
find_best_slave and change_active that does all the actual stuff
of swapping an old current with the new one.
The change we did in /proc was to reduce the amount of data
extarcted each time the proc entry is polled. Instead of dumping
all the data of all the bond devices that exist, each bond returns
just data that is relevant to itself.
In the lonf term, the drive is to move any *smart* code done in
the config application into the driver itself and be left with
the smallest, most compact application as possible. This is the
trend we've seen in the VLAN config app, and the bridge module.
All the "brain" is in the kernel module and very little should be
done in the application.
Shmulik.
Not sure I fully understood the concerns above, but I'll try
to explain what the change was all about.
I think it wasnt the one specific change rather a few posted that i
spent a minute or two staring at. And you confirm my suspicion below.
[..]
In the lonf term, the drive is to move any *smart* code done in
the config application into the driver itself and be left with
the smallest, most compact application as possible. This is the
trend we've seen in the VLAN config app, and the bridge module.
All the "brain" is in the kernel module and very little should be
done in the application.
I am not very familiar with the bonding code although i think you guys
have been doing very good work since you got involved.
In any case the approach you state above is wrong. Actually Stephen
Hemminger and I discussed this for bridging. Post 2.6 he is going to
remove a lot of the bridge policy (or "brain" as you call it) out of the
kernel. Netlink for kernel<->userspace not /proc. I think we should head
towards that direction so we can have more sophisticated management.
Thoughts?
cheers,
jamal
From: Shmulik Hen <hidden> Date: 2003-08-11 10:08:48
On Monday 11 August 2003 05:51 am, you wrote:
On Sat, 2003-08-09 at 06:29, Hen, Shmulik wrote:
quoted
Not sure I fully understood the concerns above, but I'll try
to explain what the change was all about.
I think it wasnt the one specific change rather a few posted that i
spent a minute or two staring at. And you confirm my suspicion
below.
I probably didn't make myself clear - by "understood" I wanted to say
I probably didn't get the *meaning* of the whole sentence , and not
"I don't under stand why you are concerned".
(English is not my native tongue :) ).
I am not very familiar with the bonding code although i think you
guys have been doing very good work since you got involved.
In any case the approach you state above is wrong. Actually Stephen
Hemminger and I discussed this for bridging. Post 2.6 he is going
to remove a lot of the bridge policy (or "brain" as you call it)
out of the kernel. Netlink for kernel<->userspace not /proc. I
think we should head towards that direction so we can have more
sophisticated management.
I, on the other hand, am not familiar with the bridging code and I
don't know what it actually does internally, I just noticed that
regarding config operations, most of the code is done at the kernel
level as response to ioctl commands.
I'll try to clarify how that relates to bonding. The ifenslave utility
has very little "brain" as it is, and all it knows how to do
currently is enslave/release slave devices and change the current
active slave. It also has some ability to extract status info from
the bond and present it nicely for a user.
The "brain" I was referring to in the bonding module itself has to do
with timer functions monitoring link status or Tx/Rx activity of the
slaves, and once a faulty slave is detected, switch to use another
one instead according to the teaming mode. There are no large scale
decision making nor major CPU consuming computations that are part of
the continuous operation of the module that is basically handle Rx/Tx
on slaves.
The bonding module doesn't need to access any special info that is
normally available to user space apps. What it does need is very
short response time and accessibility to kernel internal resources
like net devices info to make it a high availability intermediate
driver.
Trying to move that from the kernel module into the config application
seems to be a very hard task to implement since we'll have to find a
way to make the application constantly aware to the specifics like
current topology, slave-to-bond affiliation, updated status of each
slave, etc., etc. It would also mean that the driver will have to
wait for the application to tell it what to do each time it needs a
decision, and by that we'll surely suffer some performance hit and
probably get low availability or temporary loss of communications.
Going back to the first problem, discussions on the bonding
development list pointed that it might be better if we moved the
configuration-time decisions making to the driver, so the application
wouldn't have to deal with situations like:
1) get the master's MTU settings, master's teaming mode, communication
version, backwards compatibility issues, etc.
2) figure if need to set MTU to slave according to all that,
3) try to set that on the new slave being added,
4) if not successfull, decide if may enslave anyway or,
5) maybe undo all previous settings already done to the slave
(needs a way to retrieve old values)
6) decide if should go on or fail any further operations
7) repeat the above for all other settings
On the other hand, what we want to get to is something more like:
1) tell bonding to add slave X to bond Y,
2) watch for error returns,
3) print a nice message according to the type of the error.
While the driver, already aware of all possible relevant data, makes
all decisions, performs settings, handles compatibility issues,
checks for failures at each stage, handles any undo steps, and return
success/error values accordingly.
Thoughts?
Mostly explanations :)
Is there anywhere I can see what you refereed to as discussions with
Stephen Hemminger ? I would really like to know how and what could
also be applied to bonding.
Regards,
Shmulik.
Not sure I fully understood the concerns above, but I'll try
to explain what the change was all about.
I think it wasnt the one specific change rather a few posted that i
spent a minute or two staring at. And you confirm my suspicion
below.
I probably didn't make myself clear - by "understood" I wanted to say
I probably didn't get the *meaning* of the whole sentence , and not
"I don't under stand why you are concerned".
(English is not my native tongue :) ).
quoted
I am not very familiar with the bonding code although i think you
guys have been doing very good work since you got involved.
In any case the approach you state above is wrong. Actually Stephen
Hemminger and I discussed this for bridging. Post 2.6 he is going
to remove a lot of the bridge policy (or "brain" as you call it)
out of the kernel. Netlink for kernel<->userspace not /proc. I
think we should head towards that direction so we can have more
sophisticated management.
I, on the other hand, am not familiar with the bridging code and I
don't know what it actually does internally, I just noticed that
regarding config operations, most of the code is done at the kernel
level as response to ioctl commands.
Theres two main components to it: a control protocol and a forwarding
path. The control protocol known as STP tells the forwarding path how to
behave. Essentially, STP carries the policy implemented by the
forwarding path. This is the same breakdown to say routing protocols
like OSPF and regular forwarding path. At the moment STP sits in the
kernel. STP is really the "brains".
I'll try to clarify how that relates to bonding. The ifenslave utility
has very little "brain" as it is, and all it knows how to do
currently is enslave/release slave devices and change the current
active slave. It also has some ability to extract status info from
the bond and present it nicely for a user.
The "brain" I was referring to in the bonding module itself has to do
with timer functions monitoring link status or Tx/Rx activity of the
slaves, and once a faulty slave is detected, switch to use another
one instead according to the teaming mode.
There are no large scale
decision making nor major CPU consuming computations that are part of
the continuous operation of the module that is basically handle Rx/Tx
on slaves.
The bonding module doesn't need to access any special info that is
normally available to user space apps. What it does need is very
short response time and accessibility to kernel internal resources
like net devices info to make it a high availability intermediate
driver.
Trying to move that from the kernel module into the config application
seems to be a very hard task to implement since we'll have to find a
way to make the application constantly aware to the specifics like
current topology, slave-to-bond affiliation, updated status of each
slave, etc., etc. It would also mean that the driver will have to
wait for the application to tell it what to do each time it needs a
decision, and by that we'll surely suffer some performance hit and
probably get low availability or temporary loss of communications.
Not at all. If you let some app control this i am sure whoever writes
the app has vested interest in getting fast failovers etc.
Going back to the first problem, discussions on the bonding
development list pointed that it might be better if we moved the
configuration-time decisions making to the driver, so the application
wouldn't have to deal with situations like:
1) get the master's MTU settings, master's teaming mode, communication
version, backwards compatibility issues, etc.
2) figure if need to set MTU to slave according to all that,
3) try to set that on the new slave being added,
4) if not successfull, decide if may enslave anyway or,
5) maybe undo all previous settings already done to the slave
(needs a way to retrieve old values)
6) decide if should go on or fail any further operations
7) repeat the above for all other settings
On the other hand, what we want to get to is something more like:
1) tell bonding to add slave X to bond Y,
2) watch for error returns,
3) print a nice message according to the type of the error.
Dont you think that anything thats "rich" like you list above should
stay out of the kernel? In any case, if you have a controlling app, you
could do more interesting things; example add or delete routes, firewall
rules, qos policies etc which all have very strong correlation with
availability - these are examples btw, not an exhaustive list. If all
you are satisfied with is link management alone, then by all means
hardcoding behavior into the kernel is fine. I dont think it is
sufficient.
While the driver, already aware of all possible relevant data, makes
all decisions, performs settings, handles compatibility issues,
checks for failures at each stage, handles any undo steps, and return
success/error values accordingly.
Driver - actually bonding - should have minimal failover policy built in
for the lazy; example what i used to know about bonding - failover to
the next link, maybe send a grat arp etc. If I want more than basic,
then send me netlink events to user space and let me control how it
goes. Maybe i dont want to go to the second link but rather the 4th
link.
quoted
Thoughts?
Mostly explanations :)
Is there anywhere I can see what you refereed to as discussions with
Stephen Hemminger ? I would really like to know how and what could
also be applied to bonding.
Basically what i described at the top. Move any "richness" to user
space.
cheers,
jamal
Trying to move that from the kernel module into the config application
seems to be a very hard task to implement since we'll have to find a
way to make the application constantly aware to the specifics like
current topology, slave-to-bond affiliation, updated status of each
slave, etc., etc. It would also mean that the driver will have to
wait for the application to tell it what to do each time it needs a
decision, and by that we'll surely suffer some performance hit and
probably get low availability or temporary loss of communications.
Not at all. If you let some app control this i am sure whoever writes
the app has vested interest in getting fast failovers etc.
Basically what i described at the top. Move any "richness" to user
space.
HP/Compaq/Digital used to have the same approach with their Netrain
implementation, and from one release of Tru64 UNIX to another, they
could no longer support resolution ala milli-seconds but only seconds
due to the move of such "richness" to user space (among other things).
I am not saying that doing so on Linux will result to the same, but
a minimal failover policy shall remain in the kernel for performance
reason ... (or a user space facility could exist to *configure* such
policy but without direct interaction with user space when the kernel
has to decide).
Laurent
From: Shmulik Hen <hidden> Date: 2003-08-11 14:20:38
On Monday 11 August 2003 05:07 pm, Laurent DENIEL wrote:
HP/Compaq/Digital used to have the same approach with their Netrain
implementation, and from one release of Tru64 UNIX to another, they
could no longer support resolution ala milli-seconds but only
seconds due to the move of such "richness" to user space (among
other things). I am not saying that doing so on Linux will result
to the same, but a minimal failover policy shall remain in the
kernel for performance reason ... (or a user space facility could
exist to *configure* such policy but without direct interaction
with user space when the kernel has to decide).
Laurent
That was my point. Thank you for putting it into better words.
If high availbilty and fast failovers are what's needed, why move it
out of kernel space and put it in an application ? How fast could it
work compared to a kernel module ? Why need an extra piece of code
running in user space (daemon?) to monitor a module when the module
can do that itself ?
If smarter behavior is needed (e.g. falling to eth4 instead of eth1
when eth0 fails), we can add some priority mechanism to the driver to
do that when it decides to swap. Otherwise, we'll be devleoping
applications from now on, not the Linux kernel :)
Shmulik.
On Monday 11 August 2003 05:07 pm, Laurent DENIEL wrote:
quoted
HP/Compaq/Digital used to have the same approach with their Netrain
implementation, and from one release of Tru64 UNIX to another, they
could no longer support resolution ala milli-seconds but only
seconds due to the move of such "richness" to user space (among
other things). I am not saying that doing so on Linux will result
to the same, but a minimal failover policy shall remain in the
kernel for performance reason ... (or a user space facility could
exist to *configure* such policy but without direct interaction
with user space when the kernel has to decide).
Laurent
That was my point. Thank you for putting it into better words.
If high availbilty and fast failovers are what's needed, why move it
out of kernel space and put it in an application ? How fast could it
work compared to a kernel module ? Why need an extra piece of code
running in user space (daemon?) to monitor a module when the module
can do that itself ?
If smarter behavior is needed (e.g. falling to eth4 instead of eth1
when eth0 fails), we can add some priority mechanism to the driver to
do that when it decides to swap. Otherwise, we'll be devleoping
applications from now on, not the Linux kernel :)
So how many smart things are you going to add to the driver? ;->
Do you wanna add the qos policy changeover as well? What about route
changes, firewalling etc. What about sliceing bread and adding butter?
Where do you draw the line?
BTW, I dont understand why it would slow down failover; sure it will a
tiny bit because you have to cross user space to lookup the policy.
Maybe this is the part that i havent made clear, heres an example:
- User space gets notified link eth0 went down
- User space looks up a policy config on what to do when eth0 goes down
- user space executes commands which may include telling kernel to
move activity to eth1.
Note: I agree on a minimal failover policy staying in the kernel; very
basic stuff like what bonding used to do (may still do, dont know).
cheers,
jamal
From: Shmulik Hen <hidden> Date: 2003-08-11 16:25:38
On Monday 11 August 2003 05:34 pm, jamal wrote:
So how many smart things are you going to add to the driver? ;->
Do you wanna add the qos policy changeover as well? What about
route changes, firewalling etc. What about sliceing bread and
adding butter? Where do you draw the line?
BTW, I dont understand why it would slow down failover; sure it
will a tiny bit because you have to cross user space to lookup the
policy. Maybe this is the part that i havent made clear, heres an
example: - User space gets notified link eth0 went down
- User space looks up a policy config on what to do when eth0 goes
down - user space executes commands which may include telling
kernel to move activity to eth1.
Note: I agree on a minimal failover policy staying in the kernel;
very basic stuff like what bonding used to do (may still do, dont
know).
cheers,
jamal
Why have any kernel code other than device drivers in the first place
?
Why not move all the TCP/IP stack out of kernel space and put it in an
application ? Lets have the entire ARP mechanism in an appliaction
and let it handle everything from routing tables management to arp
negotiation while the kernel will only know how to create arp packets
that it gets from that app and send them away ? It doesn't need to
have the know how.
Say we do thing s your way and use the notification mechanism, how
long do you think it's going to take for the whole operation to
finish taking into consideration how the kernel runs user space
applications in comparison with kernel code? what happens when the
system is heavily loaded ? What happens if the application dies for
some reason ?
Why should the bonding driver even care about routes or firewalling ?
It's only meant to group several physical ethernet devices and group
them under one logical device to handle teaming solutions.
--
| Shmulik Hen Advanced Network Services |
| Israel Design Center, Jerusalem |
| LAN Access Division, Platform Networking |
| Intel Communications Group, Intel corp. |
From: Jeff Garzik <hidden> Date: 2003-08-11 16:43:47
The answer is, like life, it's a balance.
As a general rule, we do prefer to move all code possible out of the
Linux kernel. We have even created "initramfs", which for 2.7, will be
used as a vehicle to move code from the kernel to userspace, that
previously had to be in the kernel only because it was a task that "had
to be performed at boot time".
However, one must consider
(1) does moving code to userspace create any security holes?
(2) does moving code to userspace dramatically increase the number of
context switches?
(3) does moving code to userspace violate some atomicity that being
inside the kernel guarantees?
In practice, #3 is the showstopper that occurs most often.
This is why I push for a "bonding-utils" package from Jay.... because of
the general rule above: put it into userspace, where possible.
Jeff
The answer is, like life, it's a balance.
As a general rule, we do prefer to move all code possible out of the
Linux kernel. We have even created "initramfs", which for 2.7, will be
used as a vehicle to move code from the kernel to userspace, that
previously had to be in the kernel only because it was a task that "had
to be performed at boot time".
However, one must consider
(1) does moving code to userspace create any security holes?
(2) does moving code to userspace dramatically increase the number of
context switches?
(3) does moving code to userspace violate some atomicity that being
inside the kernel guarantees?
You forgot one important aspect :
(4) does moving code to userspace break compatibility (or behavior)
with user land applications (or systems)
What can one do if say, kernel 2.[4|5] switches the NIC in 10 mseconds
while kernel 2.7 with user land daemon switches in a few seconds ?
nothing but stay with the previous version or fork the driver development ;-(
But I agree that it is interesting to do some stuff at user land, and if
the bonding had an option to disable the automatic failover policy,
this could be implemented with trigger towards user land application that
could use an ioctl call to switch to the appropriate NIC according to
the user lan configuration ...
But the fast and simple failover policy shall remain in kernel code.
Laurent
From: Jeff Garzik <hidden> Date: 2003-08-11 17:43:27
Laurent DENIEL wrote:
Jeff Garzik a écrit :
quoted
The answer is, like life, it's a balance.
As a general rule, we do prefer to move all code possible out of the
Linux kernel. We have even created "initramfs", which for 2.7, will be
used as a vehicle to move code from the kernel to userspace, that
previously had to be in the kernel only because it was a task that "had
to be performed at boot time".
However, one must consider
(1) does moving code to userspace create any security holes?
(2) does moving code to userspace dramatically increase the number of
context switches?
(3) does moving code to userspace violate some atomicity that being
inside the kernel guarantees?
You forgot one important aspect :
(4) does moving code to userspace break compatibility (or behavior)
with user land applications (or systems)
I agree... assuming these userland interfaces are fairly standard and
widely deployed.
What can one do if say, kernel 2.[4|5] switches the NIC in 10 mseconds
while kernel 2.7 with user land daemon switches in a few seconds ?
nothing but stay with the previous version or fork the driver development ;-(
This is a silly example. If that happens in practice, then that is a
bug in the configuration of the userland daemon, or a bug in the
kernel<->userland ABI.
But I agree that it is interesting to do some stuff at user land, and if
the bonding had an option to disable the automatic failover policy,
this could be implemented with trigger towards user land application that
could use an ioctl call to switch to the appropriate NIC according to
the user lan configuration ...
Remember, ioctls are bad. :) Unix design mistake.
But the fast and simple failover policy shall remain in kernel code.
I would not make such absolute predictions, especially about policy :)
Jeff
From: Mark Huth <hidden> Date: 2003-08-11 21:27:04
Jeff Garzik wrote:
The answer is, like life, it's a balance.
As a general rule, we do prefer to move all code possible out of the
Linux kernel. We have even created "initramfs", which for 2.7, will
be used as a vehicle to move code from the kernel to userspace, that
previously had to be in the kernel only because it was a task that
"had to be performed at boot time".
However, one must consider
(1) does moving code to userspace create any security holes?
(2) does moving code to userspace dramatically increase the number of
context switches?
(3) does moving code to userspace violate some atomicity that being
inside the kernel guarantees?
In practice, #3 is the showstopper that occurs most often.
This is why I push for a "bonding-utils" package from Jay.... because
of the general rule above: put it into userspace, where possible.
Jeff
Yes, the answer is balance - the complicated, but non-time critical
things should go into applications. However, we need to retain a basic
ability to perform the failover according to pre-configured rules within
the kernel. Many of our customers use bonding to provide a redundant
network path through the wires and switches for what turn out to be
heavily network dependant applications. In many cases, the systems do
not have a local disk, and everything is obtained via say an NFS mount.
When the MAC breaks, you may not be able to run userland!
In HA systems at this level, guarding against the failure of a redundant
hardware component, we find that it is very helpful for the kernel to be
able to perform a variety of simple, pre-programmed operations without
resort to userland - this keeps the interacting fault domains smaller.
Sure, the decisons about how to configure the behaviors - that is the
policies - belong in applications. But the response to an event which
triggers the actions may well _need_ to be in the kernel.
While the issue may not be so much one of speed - the applications may
well respond in an adequate manner, depending on design and load - the
issue of the amount of the system that must work for recovery is quite
important when trying to push system availabilities into the mythical 5
9's plus region. For an application to run, the system has to be able
to fork and exec, access the file system, allocate memory, etc. Sure,
through careful configuration it is possible to reduce the transient
resources required (run a pre-loaded/locked daemon, make sure the files
are locally cached, etc) then the configuration and testing are
complicated. It worked fine in the lab, because a resource we didn't
realize was critical, never got pushed out of the dcache, for example.
Mark Huth
Why have any kernel code other than device drivers in the first place
?
Why not move all the TCP/IP stack out of kernel space and put it in an
application ? Lets have the entire ARP mechanism in an appliaction
Of course ARP should be moved out. Infact you bring out a very good
point;-> If ARP was out of the kernel a lot of the recent discussions on
ARP in relation to HA would have been nonexistent. The only reason they
are even being discussed its because so much of the policy exists in the
kernel.
[BTW, there are several ARP implementations on Linux which try to move
it out of the kernel and leave some minimal functionality in the
kernel].
and let it handle everything from routing tables management to arp
negotiation while the kernel will only know how to create arp packets
that it gets from that app and send them away ? It doesn't need to
have the know how.
Actually if you do this, then ARP packets need be generated in user
space.
Say we do thing s your way and use the notification mechanism, how
long do you think it's going to take for the whole operation to
finish taking into consideration how the kernel runs user space
applications in comparison with kernel code?
I dont have numbers. I would estimate less than a ms to go to user
space, execute some simple policy config and come back.
what happens when the
system is heavily loaded ?
What happens now ?
What happens if the application dies for
some reason ?
What happens when the kernel oopses? ;->
Why should the bonding driver even care about routes or firewalling ?
I gave those as examples of policies that could be executed. There are
many reasons why you would wanna do a route redirection should a
specific link fail, most important being for path availability which is
a layer above link availability. Again, this is just an example, i may
wanna slice bread everytime a link goes down and add butter every time
one comes up. Ridiculuos as that may sound, point is you cant predict
what i wanna do with such events.
It's only meant to group several physical ethernet devices and group
them under one logical device to handle teaming solutions.
yes, those are basic services. You need to allow for more interesting
things.
cheers,
jamal
On Mon, 2003-08-11 at 13:31, Laurent DENIEL wrote:
But I agree that it is interesting to do some stuff at user land, and if
the bonding had an option to disable the automatic failover policy,
this could be implemented with trigger towards user land application that
could use an ioctl call to switch to the appropriate NIC according to
You spoilt otherwise sane text by mentioning ioctl;->
But the fast and simple failover policy shall remain in kernel code.
nod from here. Simple failover policies should stay in the kernel.
cheers,
jamal
You forgot one important aspect :
(4) does moving code to userspace break compatibility (or behavior)
with user land applications (or systems)
I agree... assuming these userland interfaces are fairly standard and
widely deployed.
quoted
What can one do if say, kernel 2.[4|5] switches the NIC in 10 mseconds
while kernel 2.7 with user land daemon switches in a few seconds ?
nothing but stay with the previous version or fork the driver development ;-(
This is a silly example. If that happens in practice, then that is a
bug in the configuration of the userland daemon, or a bug in the
kernel<->userland ABI.
Not a silly example but a real case that happened to me with another
operating system and I'd hate if it happens also with Linux ...
quoted
But I agree that it is interesting to do some stuff at user land, and if
the bonding had an option to disable the automatic failover policy,
this could be implemented with trigger towards user land application that
could use an ioctl call to switch to the appropriate NIC according to
the user lan configuration ...
Remember, ioctls are bad. :) Unix design mistake.
ioctl (which already exist) or something else, this is not the point here.
quoted
what happens when the
system is heavily loaded ?
What happens now ?
quoted
What happens if the application dies for
some reason ?
What happens when the kernel oopses? ;->
Such silly responses make me think that it is no longer worth to argue ...
Laurent
From: David S. Miller <hidden> Date: 2003-08-12 13:08:45
On 12 Aug 2003 08:59:17 -0400
jamal [off-list ref] wrote:
You dont think asking "what if the application dies" is in the same
calibre as "what happens when the kernel oopses"?
Don't sweat it Jamal, some people just don't get it :-)
Look, people, when userlevel routing daemon dies your system
effectively stops to route.
There is zero difference between that example and the ones
we are discussing here.
Policy belongs strictly at user space.
One of the great things about what Jamal spends his time working
on is finally a strict seperation of the control layer from everything
else. And part of this is moving all of the control logic into userspace.
Once that is accomplished, I can have my toilet flush every time a TCP
packet is routed through my system and this won't crap up the kernel.
If you don't see the value in that, perhaps you shouldn't be partaking
in this discussion :-)
On 12 Aug 2003 08:59:17 -0400
jamal [off-list ref] wrote:
quoted
You dont think asking "what if the application dies" is in the same
calibre as "what happens when the kernel oopses"?
Don't sweat it Jamal, some people just don't get it :-)
Look, people, when userlevel routing daemon dies your system
effectively stops to route.
That's why in really *safe* systems, we do not use routing daemon
but only static routes ;-)
And there is a BIG difference :
When user level daemon dies, you have to be sure that some stuff
exists to monitor and recover from that situation (either by
restarting the faulty deamon (if it could recover in time which
I doubt with the bonding case), or by switching to a new machine
in a fault tolerant configuration). With kernel ooops, there is
NOTHING to do in such in such a fault tolerant systems, since the
machine is unusable (this is the same as a hardware failure).
But people does not understand the constraints of really safe
systems.
Policy belongs strictly at user space.
One of the great things about what Jamal spends his time working
on is finally a strict seperation of the control layer from everything
else. And part of this is moving all of the control logic into userspace.
Once that is accomplished, I can have my toilet flush every time a TCP
packet is routed through my system and this won't crap up the kernel.
If you don't see the value in that, perhaps you shouldn't be partaking
in this discussion :-)
This is OK as long as your kernel and user space stuff remains suitable
for highly fault tolerant systems and which does not require big montains
of user stuff to do the same as a few line of code in the kernel. Remember
the aim of bonding : NIC fault tolerance or load balancing. I am not against
a user space configurable policy for more complex job but the initial aim of
the bonding shall remain coded in the kernel (and is only usable in the above
mentioned systems in that way).
Laurent