From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:48:43
The following patch set adds hardware offload of the crypto operations for
IPv4 IPSEC processing. It gives a noticible speedup on my (admittedly older)
hardware, but given the recent numbers posted, can be a speedup even for
more recent hardware.
There are a few known issues with the current patchset, but I think it is
ready for wider review.
* Only the 3Com 3CR990 family of NICs are supported. I don't have hardware
or documentation for the Intel cards.
* Doesn't do IPv6. Need someone to implement map_direction(), and
AH/ESP handling, as well as come up with a card that supports it.
* The use of GFP_ATOMIC in xfrm_offload_alloc() is probably not a good idea.
* linux/skbuff.h cannot include net/xfrm.h currently, so there are
redundant defines (requires some header cleanup, which I'm not
very inclined to tackle at the moment.)
* TCP Segmentation offload seems broken by firmware 03.001.008. It could be
my changes to support the offload, but that seems unlikely. I will
have to investigate this.
* Latency suffers somewhat on smaller packets, it may be advisable to have
a minimum packet size to offload.
* No real feedback on which xfrm_states have been offloaded or not.
The patch set will be sent as follow-ups to this post, or is available via:
bk pull http://typhoon.bkbits.net/ipsec-2.6
It will update the following files:
Documentation/networking/netdevices.txt | 16
drivers/net/typhoon.c | 687 +++++++++++++++++++++++++++++++-
drivers/net/typhoon.h | 38 +
include/linux/ethtool.h | 8
include/linux/netdevice.h | 11
include/linux/skbuff.h | 55 ++
include/net/dst.h | 1
include/net/xfrm.h | 108 +++++
net/core/ethtool.c | 54 ++
net/core/skbuff.c | 31 +
net/ipv4/ah4.c | 99 ++--
net/ipv4/esp4.c | 102 +++-
net/ipv4/xfrm4_state.c | 10
net/ipv6/xfrm6_state.c | 9
net/xfrm/xfrm_export.c | 4
net/xfrm/xfrm_policy.c | 64 ++
net/xfrm/xfrm_state.c | 101 ++++
17 files changed, 1284 insertions(+), 114 deletions(-)
If you work from the mailed patches, you will want the netdev-2.6 updates
to the typhoon driver, as the 3CR990B series needs the newest firmware to
correctly offload IPSEC processing. That patch is available from
http://www.thedillows.org/typhoon-netdev-2.6.patch.bz2
The following results were generated using a dual processor PIII 1GHz/512MB
with a 3CR990SVR97 (ori) and an Athlon 550 MHz/256MB with a 3CR990B (tank).
Latency testing was performed with lmbench's lat_tcp, and bandwith testing
was performed with Andrew Morton's zcc/zcs/cyclesoak. I ran the tests
multiple times, and picked the median results to report. There was not much
deviation in the results (+/- 1.5 us +/- 50KBytes/s +/- 1.5% CPU usage).
TCP Latency tests (1 byte msg)
Config Latency
No IPSEC 196 us
AH/SHA1 (sw) 256 us
AH/SHA1 (hw) 317 us
ESP/3DES,SHA1 (sw) 333 us
ESP/3DES,SHA1 (hw) 347 us
ESP-AH/3DES,SHA1-SHA1 (sw) 387 us
ESP-AH/3DES,SHA1-SHA1 (hw) 467 us
TCP Latency tests (1024 byte msg)
Config Latency
No IPSEC 625 us
AH/SHA1 (sw) 771 us
AH/SHA1 (hw) 858 us
ESP/3DES,SHA1 (sw) 1999 us
ESP/3DES,SHA1 (hw) 902 us
ESP-AH/3DES,SHA1-SHA1 (sw) 2140 us
ESP-AH/3DES,SHA1-SHA1 (hw) 1131 us
Bandwidth tests
Config (sender -> receiver) Bandwidth ori CPU tank CPU
No IPSEC (tank->ori) 11494 KB/s 11.9% 18.7%
No IPSEC (ori->tank) 11492 KB/s 9.5% 34.3%
AH/SHA1 (sw) (tank->ori) 11303 KB/s 29.2% 79.3%
AH/SHA1 (sw) (ori->tank) 11302 KB/s 28.6% 91.1%
ESP/3DES,SHA1 (sw) (tank->ori) 2130 KB/s 29.6% 100%
ESP/3DES,SHA1 (sw) (ori->tank) 2263 KB/s 29.3% 99.7%
ESP-AH/3DES,SHA1-SHA1 (sw) (tank->ori) 1906 KB/s 29.1% 100%
ESP-AH/3DES,SHA1-SHA1 (sw) (ori->tank) 2051 KB/s 29.3% 99.7%
AH/SHA1 (hw) (tank->ori) 11303 KB/s 14.0% 30.2%
AH/SHA1 (hw) (ori->tank) 11301 KB/s 14.1% 39.8%
ESP/3DES,SHA1 (hw) (tank->ori) 11221 KB/s 15.4% 44.9%
ESP/3DES,SHA1 (hw) (ori->tank) 11220 KB/s 21.5% 48.1%
ESP-AH/3DES,SHA1-SHA1 (hw) (tank->ori) 5920 KB/s 10.8% 35.9%
ESP-AH/3DES,SHA1-SHA1 (hw) (ori->tank) 7189 KB/s 14.3% 35.4%
The last line seems suspicious, and should probably be retested.
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:48:50
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:28:25-05:00 dave@thedillows.org
# Add the xfrm offload management calls to struct netdevice.
#
# xfrm_state_add() is called for inbound xfrm states
# xfrm_bundle_add() is called for outbound xfrm bundles
# xfrm_state_del() is called for all offloaded xfrms,
# inbound or outbound.
#
# If a driver adds NETIF_F_IPSEC to its features, it must
# provide all three callbacks.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/linux/netdevice.h
# 2004/12/30 00:28:07-05:00 dave@thedillows.org +11 -0
# Add the xfrm offload management calls to struct netdevice.
#
# xfrm_state_add() is called for inbound xfrm states
# xfrm_bundle_add() is called for outbound xfrm bundles
# xfrm_state_del() is called for all offloaded xfrms,
# inbound or outbound.
#
# If a driver adds NETIF_F_IPSEC to its features, it must
# provide all three callbacks.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/include/linux/netdevice.h b/include/linux/netdevice.h
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:50:52
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:27:15-05:00 dave@thedillows.org
# Add direction information to xfrm_state. This will be needed to
# offload xfrm processing to the NIC.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_state.c
# 2004/12/30 00:25:42-05:00 dave@thedillows.org +5 -0
# Add direction information to xfrm_state. This will be needed to
# offload xfrm processing to the NIC.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/ipv6/xfrm6_state.c
# 2004/12/30 00:25:42-05:00 dave@thedillows.org +9 -0
# Place holder for adding IPv6 direction mapping routine.
#
# net/ipv4/xfrm4_state.c
# 2004/12/30 00:25:42-05:00 dave@thedillows.org +10 -0
# Add direction information to xfrm_state. This will be needed to
# offload xfrm processing to the NIC.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/net/xfrm.h
# 2004/12/30 00:25:42-05:00 dave@thedillows.org +10 -0
# Add direction information to xfrm_state. This will be needed to
# offload xfrm processing to the NIC.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
@@ -146,6 +146,9 @@/* Private data of this transformer, format is opaque,*interpretedbyxfrm_typemethods.*/void*data;++/* Intended direction of this state, used for offloading */+intdir;};enum{
@@ -116,12 +116,21 @@returnx0;}+staticvoid+__xfrm6_map_direction(structxfrm_state*x)+{+/* XXX This needs to be implemented by someone who knows+*IPv6betterthenI.+*/+}+staticstructxfrm_state_afinfoxfrm6_state_afinfo={.family=AF_INET6,.lock=RW_LOCK_UNLOCKED,.init_tempsel=__xfrm6_init_tempsel,.state_lookup=__xfrm6_state_lookup,.find_acq=__xfrm6_find_acq,+.map_direction=__xfrm6_map_direction,};void__initxfrm6_state_init(void)
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:50:57
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:37:44-05:00 dave@thedillows.org
# Give device drivers a method to allow the use of crypto
# offload features for existing xfrm_states and bundles, as
# well as dynamically remove crypto offload capabilities.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_state.c
# 2004/12/30 00:37:26-05:00 dave@thedillows.org +39 -0
# When we've been informed of a new device that can offload
# xfrm crypto operations, go ahead and offload existing inbound
# xfrm_states to it.
#
# When we're removing crypto offload capabilities, remove every
# offload instance for that device.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_policy.c
# 2004/12/30 00:37:26-05:00 dave@thedillows.org +17 -0
# When adding/removing xfrm offload capable device, give the xfrm_state
# engine a chance to make the changes it needs, then flush any existing
# bundles that use the device so that future flows get a chance to use
# the offload features (for add), or resume using the software crypto
# routines (for remove).
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_export.c
# 2004/12/30 00:37:26-05:00 dave@thedillows.org +2 -0
# Export the driver-visible API.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/net/xfrm.h
# 2004/12/30 00:37:26-05:00 dave@thedillows.org +4 -0
# Prototypes for the new routines.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
@@ -1121,6 +1121,23 @@return0;}+staticintbundle_uses_dev(structdst_entry*dst,void*dev)+{+return(dst->dev==dev);+}++voidxfrm_accel_add(structnet_device*dev)+{+xfrm_state_accel_add(dev);+xfrm_prune_bundles(bundle_uses_dev,dev);+}++voidxfrm_accel_flush(structnet_device*dev)+{+xfrm_state_accel_flush(dev);+xfrm_prune_bundles(bundle_uses_dev,dev);+}+/* Well... that's _TASK_. We need to scan through transformation*listandfigureoutwhatmsstcpshouldgenerateinorderto*finaldatagramfittomtu.Mamamia...:-)
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:50:57
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:35:34-05:00 dave@thedillows.org
# Add a parameter to the decision function(s) used by
# xfrm_prune_bundles(). This will allow us to have more
# fine grained selection of bundles pruned (like, say,
# per device.)
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_policy.c
# 2004/12/30 00:35:16-05:00 dave@thedillows.org +10 -9
# Add a parameter to the decision function(s) used by
# xfrm_prune_bundles(). This will allow us to have more
# fine grained selection of bundles pruned (like, say,
# per device.)
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
@@ -730,7 +730,7 @@kfree(xbl);}-staticintstale_bundle(structdst_entry*dst);+staticintstale_bundle(structdst_entry*dst,void*unused);/* Main function: finds/creates a bundle for given flow.*
@@ -841,7 +841,7 @@}write_lock_bh(&policy->lock);-if(unlikely(policy->dead||stale_bundle(dst))){+if(unlikely(policy->dead||stale_bundle(dst,NULL))){/* Wow! While we worked on resolving, this*policyhasgone.Retry.Itisnotparanoia,*wejustcannotenlistnewbundletodeadobject.
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:50:58
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:33:11-05:00 dave@thedillows.org
# Plumb in offloading of inbound xfrm_states.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_state.c
# 2004/12/30 00:32:54-05:00 dave@thedillows.org +28 -1
# Try to offload an inbound xfrm_state when it is added or updated.
# Since it could potentially come in from any interface, try to
# offload it on all devices that support it.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/net/xfrm/xfrm_state.c b/net/xfrm/xfrm_state.c
@@ -398,6 +398,26 @@spin_unlock_bh(&xfrm_state_lock);}+staticvoidxfrm_state_inbound_accel(structxfrm_state*x)+{+/* Only called for an inbound xfrm_state. Since it could+*possiblyarriveonanyinterface,trytooffloadit+*onalldevicesthatarecapable.+*/+structnet_device*dev;++rtnl_lock();+read_lock(&dev_base_lock);+dev=dev_base;+while(dev){+if(netif_running(dev)&&(dev->features&NETIF_F_IPSEC))+dev->xfrm_state_add(dev,x);+dev=dev->next;+}+read_unlock(&dev_base_lock);+rtnl_unlock();+}+staticstructxfrm_state*__xfrm_find_acq_byseq(u32seq);intxfrm_state_add(structxfrm_state*x)
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:55:32
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:34:46-05:00 dave@thedillows.org
# Plumb in offloading new bundles for outgoing packets.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_policy.c
# 2004/12/30 00:34:28-05:00 dave@thedillows.org +28 -0
# When we create a new bundle for an outbound flow, try to
# offload as much as the destination driver will allow.
#
# Don't forget to clean up....
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/net/xfrm.h
# 2004/12/30 00:34:28-05:00 dave@thedillows.org +6 -0
# A convenience structure for offloading bundles.
#
# The dst->child field gives us a singly linked list
# from upper protocols to outer transforms. Drivers, however,
# will likely have a limited number of offloads they can
# perform on a particular packet, so they need to offload
# the bundle from the outside in. This list makes it easier
# for them.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/net/dst.h
# 2004/12/30 00:34:28-05:00 dave@thedillows.org +1 -0
# Add a field to store the offload information for this part
# of the outgoing bundle (non-NULL if this dst is offloaded.)
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/include/net/dst.h b/include/net/dst.h
@@ -705,6 +705,31 @@};}+staticvoidxfrm_accel_bundle(structdst_entry*dst)+{+structxfrm_bundle_listbundle,*xbl,*tmp;+structnet_device*dev=dst->dev;+INIT_LIST_HEAD(&bundle.node);++if(dev&&netif_running(dev)&&(dev->features&NETIF_F_IPSEC)){+while(dst){+xbl=kmalloc(sizeof(*xbl),GFP_ATOMIC);+if(!xbl)+gotoout;++xbl->dst=dst;+list_add_tail(&xbl->node,&bundle.node);+dst=dst->child;+}++dev->xfrm_bundle_add(dev,&bundle);+}++out:+list_for_each_entry_safe(xbl,tmp,&bundle.node,node)+kfree(xbl);+}+staticintstale_bundle(structdst_entry*dst);/* Main function: finds/creates a bundle for given flow.
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:55:35
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:44:50-05:00 dave@thedillows.org
# Add crypto processing for outbound AH and ESP xfrms (IPv4).
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/ipv4/esp4.c
# 2004/12/30 00:44:32-05:00 dave@thedillows.org +35 -21
# Add crypto offload for outbound ESP (IPv4) xfrms. Note that we always
# generate a random IV, as we are not guaranteed to have any state in
# the software crypto engine (we may have always been offloaded), and
# we cannot rely on secure IV generation by the NIC driver/hw.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/ipv4/ah4.c
# 2004/12/30 00:44:32-05:00 dave@thedillows.org +31 -21
# Add crypto offload for outbound AH (IPv4) xfrms. Note that the NIC
# driver/hw is responsible for zeroing the mutable IP header fields.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/net/ipv4/ah4.c b/net/ipv4/ah4.c
@@ -83,31 +83,41 @@ah->spi=x->id.spi;ah->seq_no=htonl(x->replay.oseq+1);-iph->tos=top_iph->tos;-iph->ttl=top_iph->ttl;-iph->frag_off=top_iph->frag_off;--if(top_iph->ihl!=5){-iph->daddr=top_iph->daddr;-memcpy(iph+1,top_iph+1,top_iph->ihl*4-sizeof(structiphdr));-err=ip_clear_mutable_options(top_iph,&top_iph->daddr);-if(err)+if(dst->xfrm_offload){+err=-ENOMEM;+xfrm_offload_hold(dst->xfrm_offload);+if(skb_push_xfrm_offload(skb,dst->xfrm_offload)){+xfrm_offload_release(dst->xfrm_offload);gotoerror;-}+}+}else{+/* Not offloaded, manually calculate the auth hash */+iph->tos=top_iph->tos;+iph->ttl=top_iph->ttl;+iph->frag_off=top_iph->frag_off;++if(top_iph->ihl!=5){+iph->daddr=top_iph->daddr;+memcpy(iph+1,top_iph+1,top_iph->ihl*4-sizeof(structiphdr));+err=ip_clear_mutable_options(top_iph,&top_iph->daddr);+if(err)+gotoerror;+}-top_iph->tos=0;-top_iph->frag_off=0;-top_iph->ttl=0;-top_iph->check=0;+top_iph->tos=0;+top_iph->frag_off=0;+top_iph->ttl=0;+top_iph->check=0;-ahp->icv(ahp,skb,ah->auth_data);+ahp->icv(ahp,skb,ah->auth_data);-top_iph->tos=iph->tos;-top_iph->ttl=iph->ttl;-top_iph->frag_off=iph->frag_off;-if(top_iph->ihl!=5){-top_iph->daddr=iph->daddr;-memcpy(top_iph+1,iph+1,top_iph->ihl*4-sizeof(structiphdr));+top_iph->tos=iph->tos;+top_iph->ttl=iph->ttl;+top_iph->frag_off=iph->frag_off;+if(top_iph->ihl!=5){+top_iph->daddr=iph->daddr;+memcpy(top_iph+1,iph+1,top_iph->ihl*4-sizeof(structiphdr));+}}/* Delay incrementing the replay sequence until we know we're going
@@ -98,33 +98,47 @@esph->spi=x->id.spi;esph->seq_no=htonl(++x->replay.oseq);-if(esp->conf.ivlen)-crypto_cipher_set_iv(tfm,esp->conf.ivec,crypto_tfm_alg_ivsize(tfm));+if(dst->xfrm_offload){+xfrm_offload_hold(dst->xfrm_offload);+if(skb_push_xfrm_offload(skb,dst->xfrm_offload)){+xfrm_offload_release(dst->xfrm_offload);+gotoerror;+}++if(esp->conf.ivlen)+get_random_bytes(esph->enc_data,esp->conf.ivlen);+}else{+if(esp->conf.ivlen)+crypto_cipher_set_iv(tfm,esp->conf.ivec,crypto_tfm_alg_ivsize(tfm));++do{+structscatterlist*sg=&esp->sgbuf[0];-do{-structscatterlist*sg=&esp->sgbuf[0];+if(unlikely(nfrags>ESP_NUM_FAST_SG)){+sg=kmalloc(sizeof(structscatterlist)*nfrags,GFP_ATOMIC);+if(!sg)+gotoerror;+}+skb_to_sgvec(skb,sg,esph->enc_data+esp->conf.ivlen-skb->data,clen);+crypto_cipher_encrypt(tfm,sg,sg,clen);+if(unlikely(sg!=&esp->sgbuf[0]))+kfree(sg);+}while(0);-if(unlikely(nfrags>ESP_NUM_FAST_SG)){-sg=kmalloc(sizeof(structscatterlist)*nfrags,GFP_ATOMIC);-if(!sg)-gotoerror;+if(esp->conf.ivlen){+memcpy(esph->enc_data,esp->conf.ivec,crypto_tfm_alg_ivsize(tfm));+crypto_cipher_get_iv(tfm,esp->conf.ivec,crypto_tfm_alg_ivsize(tfm));+}++if(esp->auth.icv_full_len){+esp->auth.icv(esp,skb,(u8*)esph-skb->data,+sizeof(structip_esp_hdr)+esp->conf.ivlen+clen,trailer->tail);}-skb_to_sgvec(skb,sg,esph->enc_data+esp->conf.ivlen-skb->data,clen);-crypto_cipher_encrypt(tfm,sg,sg,clen);-if(unlikely(sg!=&esp->sgbuf[0]))-kfree(sg);-}while(0);--if(esp->conf.ivlen){-memcpy(esph->enc_data,esp->conf.ivec,crypto_tfm_alg_ivsize(tfm));-crypto_cipher_get_iv(tfm,esp->conf.ivec,crypto_tfm_alg_ivsize(tfm));}-if(esp->auth.icv_full_len){-esp->auth.icv(esp,skb,(u8*)esph-skb->data,-sizeof(structip_esp_hdr)+esp->conf.ivlen+clen,trailer->tail);+/* Need to account for auth data, offloading or not... */+if(esp->auth.icv_full_len)pskb_put(skb,trailer,alen);-}ip_send_check(top_iph);
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:55:36
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:40:20-05:00 dave@thedillows.org
# Add fields to sk_buff to track offloaded xfrm_states for this
# packet. On Tx, these will be pointers to struct xfrm_offload.
# On Rx, these will be a 4 bit bitfield indicating what operations
# were performed, and the result of those operations.
#
# skb_push_xfrm_offload() records an offloaded xfrm on Tx. It will
# return an error code if it is unable to record the offload.
# skb_get_xfrm_offload() returns the xfrm_offload struct at the
# given position on the stack. It will return NULL if there
# are no more offloads available.
# skb_has_xfrm_offload() returns true if the sk_buff has offload
# information available.
# skb_put_xfrm_result() records an offload result on Rx at the given
# index. It will return an error code if it is unable to
# record the result.
# skb_pop_xfrm_result() pops the current offload result from the.
# stack. If there are no more results, it will return
# XFRM_OFFLOAD_NONE.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/core/skbuff.c
# 2004/12/30 00:40:02-05:00 dave@thedillows.org +31 -0
# When an sk_buff is cloned, we must gain a reference to each
# xfrm_offload that it references.
#
# When an sk_buff is freed, we must release our references
# to the xfrm_offloads attached to it.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/net/xfrm.h
# 2004/12/30 00:40:02-05:00 dave@thedillows.org +9 -0
# Add the values for the result bitfield.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/linux/skbuff.h
# 2004/12/30 00:40:02-05:00 dave@thedillows.org +55 -0
# Add the fields and functins to track offloads and results, as
# well as the current position in the stack.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/include/linux/skbuff.h b/include/linux/skbuff.h
@@ -146,6 +146,14 @@skb_frag_tfrags[MAX_SKB_FRAGS];};+/* XXX UGH. We cannot include <net/xfrm.h> in this file without some+*headerfilesurgery,sodefineourownmaxxfrmdepth.Thisshould+*bekept>=XFRM_MAX_DEPTHuntilwefixtheincludes,anditcan+*goaway.+*/+#define SKB_XFRM_MAX_DEPTH 4+structxfrm_offload;+/** *structsk_buff-socketbuffer*@next:Nextbufferinlist
@@ -272,6 +282,8 @@#endif+intxfrm_offload_idx;+structxfrm_offload*xfrm_offload[SKB_XFRM_MAX_DEPTH];/* These elements must be at the end, see alloc_skb() for details. */unsignedinttruesize;
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:55:37
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:31:03-05:00 dave@thedillows.org
# Add offload management to xfrm_state.
#
# xfrm_offload_alloc() creates a new xfrm_offload, with a private
# part to be used by the driver (ala net_device->priv)
# The returned offload may be kfree'd if it has not been
# added to a xfrm_state using xfrm_state_offload_add().
# xfrm_offload_priv() returns a pointer to the private area of
# the xfrm_offload. This will be 8-byte aligned.
# xfrm_offload_hold()/xfrm_offload_release() do the reference
# counting of the xfrm_offload
# xfrm_offload_get() looks up the xfrm_offload from a given device
# The caller should call xfrm_offload_release() when it is
# finished with this offload.
# xfrm_state_offload_add() adds a new offload to the xfrm_state,
# replacing any existing offload for the device that may
# exist for this xfrm_state.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_state.c
# 2004/12/30 00:30:46-05:00 dave@thedillows.org +28 -0
# Clean up any offloads on destruction of an xfrm_state, and
# allow the addition of new xfrm_offloads to a xfrm_state.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/xfrm/xfrm_export.c
# 2004/12/30 00:30:46-05:00 dave@thedillows.org +2 -0
# Export xfrm_state_offload_add() to drivers.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/net/xfrm.h
# 2004/12/30 00:30:46-05:00 dave@thedillows.org +79 -0
# Add offload management to xfrm_state.
#
# Add xfrm_offload_alloc(), xfrm_offload_priv(), xfrm_offload_hold()
# xfrm_offload_release(), and xfrm_offload_get().
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/include/net/xfrm.h b/include/net/xfrm.h
@@ -81,6 +81,8 @@metrics.Plus,itwillbemadeviask->sk_dst_cache.Solved.*/+structxfrm_offload;+/* Full description of state of transformer. */structxfrm_state{
@@ -149,6 +151,9 @@/* Intended direction of this state, used for offloading */intdir;++/* List of offload cookies, per device */+structlist_headoffloads;};enum{
@@ -1681,6 +1682,43 @@return0;}+staticinlinevoid+typhoon_ipsec_rx(structsk_buff*skb,u16results)+{+#define CHECK_OFFLOAD(good, bad) \+do{if(results&(good|bad)){\+unsignedinttmp=XFRM_OFFLOAD_CONF|XFRM_OFFLOAD_AUTH;\+tmp|=(results&good)?XFRM_OFFLOAD_AUTH_OK:\+XFRM_OFFLOAD_AUTH_FAIL;\+if(skb_put_xfrm_result(skb,tmp,i))\+return;\+i++;\+}}while(0)++/* We have no way to determine what the order of the SAs were on+*thewire,justthe1stAHseen,the1stESPseen,etc.+*+*Wejustwalkthestack,andpretendthatAHSAsgetdecypted+*sothatifwegettheorderwrong,theworstcasesceneriois+*thatweindicatethefailureonthewrongSA,sincewe'llneed+*tomatchallSAsagainstthepolicy.+*+*Wegeta"ESP good"indicationfornullauthhashonESP.+*/+/* XXX think more about security indications -- can I craft a+*packettodobadthings--maybeaNULLauthESPpacket,+*andafailedAHpacket?+*/+inti=0;++CHECK_OFFLOAD(TYPHOON_RX_AH1_GOOD,TYPHOON_RX_AH1_FAIL);+CHECK_OFFLOAD(TYPHOON_RX_ESP1_GOOD,TYPHOON_RX_ESP1_FAIL);+CHECK_OFFLOAD(TYPHOON_RX_AH2_GOOD,TYPHOON_RX_AH2_FAIL);+CHECK_OFFLOAD(TYPHOON_RX_ESP2_GOOD,TYPHOON_RX_ESP2_FAIL);++#undef CHECK_OFFLOAD+}+staticinttyphoon_rx(structtyphoon*tp,structbasic_ring*rxRing,volatileu32*ready,volatileu32*cleared,intbudget)
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:59:54
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:53:15-05:00 dave@thedillows.org
# Make the crypto structures better match actual usage.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.h
# 2004/12/30 00:52:57-05:00 dave@thedillows.org +13 -11
# Make the crypto structures better match actual usage.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/drivers/net/typhoon.h b/drivers/net/typhoon.h
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:59:55
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:51:19-05:00 dave@thedillows.org
# Add support for querying and changing the status of the
# IPSEC crypto offload feature of a NIC.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/core/ethtool.c
# 2004/12/30 00:51:00-05:00 dave@thedillows.org +54 -0
# Add support for querying and changing the status of the IPSEC
# crypto offload feature of a NIC.
#
# Turn on/off the feature flag before informing the xfrm engine
# of the change so that existing xfrms get the new settings.
#
# Signed-off-by: David Dillow [off-list ref]
#
# include/linux/ethtool.h
# 2004/12/30 00:51:00-05:00 dave@thedillows.org +8 -0
# Add support for querying and changing the status of the
# IPSEC crypto offload feature of a NIC.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/include/linux/ethtool.h b/include/linux/ethtool.h
@@ -72,6 +73,24 @@return0;}+u32ethtool_op_get_ipsec(structnet_device*dev)+{+return(dev->features&NETIF_F_IPSEC)!=0;+}++intethtool_op_set_ipsec(structnet_device*dev,u32data)+{+if(data){+dev->features|=NETIF_F_IPSEC;+xfrm_accel_add(dev);+}else{+dev->features&=~NETIF_F_IPSEC;+xfrm_accel_flush(dev);+}++return0;+}+/* Handlers for each ethtool command */staticintethtool_get_settings(structnet_device*dev,void__user*useraddr)
@@ -171,22 +172,32 @@if(elen<=0||(elen&(blksize-1)))gotoout;+offload=skb_pop_xfrm_result(skb);+/* If integrity check is required, do this. */if(esp->auth.icv_full_len){-u8sum[esp->auth.icv_full_len];-u8sum1[alen];+if(unlikely(offload&XFRM_OFFLOAD_AUTH_FAIL)){+x->stats.integrity_failed++;+gotoout;+}++if(!(offload&XFRM_OFFLOAD_AUTH)){+u8sum[esp->auth.icv_full_len];+u8sum1[alen];-esp->auth.icv(esp,skb,0,skb->len-alen,sum);+esp->auth.icv(esp,skb,0,skb->len-alen,sum);-if(skb_copy_bits(skb,skb->len-alen,sum1,alen))-BUG();+if(skb_copy_bits(skb,skb->len-alen,sum1,alen))+BUG();-if(unlikely(memcmp(sum,sum1,alen))){-x->stats.integrity_failed++;-gotoout;+if(unlikely(memcmp(sum,sum1,alen))){+x->stats.integrity_failed++;+gotoout;+}}}+/* XXX I think this can be moved to the !offload case */if((nfrags=skb_cow_data(skb,0,&trailer))<0)gotoout;
@@ -195,15 +206,12 @@esph=(structip_esp_hdr*)skb->data;iph=skb->nh.iph;-/* Get ivec. This can be wrong, check against another impls. */-if(esp->conf.ivlen)-crypto_cipher_set_iv(esp->conf.tfm,esph->enc_data,crypto_tfm_alg_ivsize(esp->conf.tfm));--{-u8nexthdr[2];+if(!(offload&XFRM_OFFLOAD_CONF)){structscatterlist*sg=&esp->sgbuf[0];-u8workbuf[60];-intpadlen;++/* Get ivec. This can be wrong, check against another impls. */+if(esp->conf.ivlen)+crypto_cipher_set_iv(esp->conf.tfm,esph->enc_data,crypto_tfm_alg_ivsize(esp->conf.tfm));if(unlikely(nfrags>ESP_NUM_FAST_SG)){sg=kmalloc(sizeof(structscatterlist)*nfrags,GFP_ATOMIC);
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 08:59:57
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:42:33-05:00 dave@thedillows.org
# Seperate AH header initialization from the zeroing of mutable
# IP header fields in preparation for offloading the crypto
# processing of the packet.
#
# Signed-off-by: David Dillow [off-list ref]
#
# net/ipv4/ah4.c
# 2004/12/30 00:42:15-05:00 dave@thedillows.org +18 -12
# Seperate AH header initialization from the zeroing of mutable
# IP header fields in preparation for offloading the crypto
# processing of the packet.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/net/ipv4/ah4.c b/net/ipv4/ah4.c
@@ -108,6 +110,10 @@memcpy(top_iph+1,iph+1,top_iph->ihl*4-sizeof(structiphdr));}+/* Delay incrementing the replay sequence until we know we're going+*tosendthispackettopreventgaps.+*/+x->replay.oseq++;ip_send_check(top_iph);err=0;
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 09:04:34
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 01:02:32-05:00 dave@thedillows.org
# Teach the Typhoon driver how to add and remove xfrm_states to
# the 3XP for later packet processing.
#
# When the first xfrm_state is added, we turn on IPSEC offloads
# for the 3XP, and we turn it off when the last one is removed.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.c
# 2004/12/30 01:02:14-05:00 dave@thedillows.org +167 -0
# Teach the Typhoon driver how to add and remove xfrm_states to
# the 3XP for later packet processing.
#
# When the first xfrm_state is added, we turn on IPSEC offloads
# for the 3XP, and we turn it off when the last one is removed.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
@@ -2420,6 +2420,173 @@#undef REQUIRED#undef UNSUPPORTED+staticstructxfrm_offload*+typhoon_offload_ipsec(structtyphoon*tp,structxfrm_state*x)+{+structcmd_descxp_cmd[5];+structresp_descxp_resp;+structsa_descriptor*sa=(structsa_descriptor*)xp_cmd;+structxfrm_offload*xol;+structtyphoon_xfrm_offload*txo;+u16*dir_sa_avail=&tp->rx_sa_avail;+u16cookie;+intkeylen,err;++if(!typhoon_validate_xfrm(tp,x))+gotoerror;++memset(xp_cmd,0,5*sizeof(xp_cmd[0]));+INIT_COMMAND_WITH_RESPONSE(xp_cmd,TYPHOON_CMD_CREATE_SA);+sa->numDesc=4;++sa->mode=TYPHOON_SA_MODE_AH;+if(x->type->proto==IPPROTO_ESP)+sa->mode=TYPHOON_SA_MODE_ESP;++if(x->dir==XFRM_STATE_DIR_OUT){+sa->direction=TYPHOON_SA_DIR_TX;+dir_sa_avail=&tp->tx_sa_avail;+}++spin_lock_bh(&tp->offload_lock);+if(!*dir_sa_avail){+spin_unlock_bh(&tp->offload_lock);+gotoerror;+}+*dir_sa_avail--;+if(!tp->sa_count++){+tp->offload|=TYPHOON_OFFLOAD_IPSEC;+err=typhoon_set_offload(tp);+if(err<0){+spin_unlock_bh(&tp->offload_lock);+printk(KERN_ERR"%s: unable to enable IPSEC "+"offload (%d)\n",tp->name,-err);+gotoerror_counted;+}+}+spin_unlock_bh(&tp->offload_lock);++if(x->props.aalgo!=SADB_X_AALG_NULL&&x->aalg){+keylen=(x->aalg->alg_key_len+7)/8;++sa->hashFlags=TYPHOON_SA_HASH_SHA1;+if(x->props.aalgo==SADB_AALG_MD5HMAC)+sa->hashFlags=TYPHOON_SA_HASH_MD5;+sa->hashFlags|=TYPHOON_SA_HASH_ENABLE;++memcpy(sa->integKey,x->aalg->alg_key,keylen);+}++if(x->props.ealgo!=SADB_EALG_NULL&&x->ealg){+keylen=(x->ealg->alg_key_len+7)/8;++sa->encryptionFlags=TYPHOON_SA_ENCRYPT_ENABLE|+TYPHOON_SA_ENCRYPT_CBC;+if(x->props.ealgo==SADB_EALG_DESCBC)+sa->encryptionFlags|=TYPHOON_SA_ENCRYPT_DES;+elseif(x->ealg->alg_key_len==192)+sa->encryptionFlags|=TYPHOON_SA_ENCRYPT_3DES_3KEY;+else{+sa->encryptionFlags|=TYPHOON_SA_ENCRYPT_3DES_2KEY;+memcpy(&sa->confKey[16],x->ealg->alg_key,8);+}++memcpy(sa->confKey,x->ealg->alg_key,keylen);+}++/* The 3XP expects the SPI to be in host order, litte endian.+*Itexpectstheaddresstobeinnetworkorder.+*/+sa->SPI=cpu_to_le32(ntohl(x->id.spi));+sa->destAddr=x->id.daddr.a4;+sa->destMask=(u32)~0UL;++err=typhoon_issue_command(tp,5,xp_cmd,1,&xp_resp);+cookie=le16_to_cpu(xp_resp.parm1);+if(err<0||!cookie||cookie==0xffff)+gotoerror_counted;++xol=xfrm_offload_alloc(sizeof(*txo),tp->dev);+if(!xol)+gotoerror_cookie;++txo=xfrm_offload_priv(xol);+txo->sa_cookie=cookie;+txo->tunnel=!!x->props.mode;+txo->ah=(x->id.proto==IPPROTO_AH);+txo->inbound=(x->dir==XFRM_STATE_DIR_IN);++xfrm_state_offload_add(x,xol);++returnxol;++error_cookie:+INIT_COMMAND_NO_RESPONSE(xp_cmd,TYPHOON_CMD_DELETE_SA);+xp_cmd[0].parm1=xp_resp.parm1;+typhoon_issue_command(tp,1,xp_cmd,0,NULL);++error_counted:+spin_lock_bh(&tp->offload_lock);+*dir_sa_avail++;+tp->sa_count--;+if(!tp->sa_count){+tp->offload&=~TYPHOON_OFFLOAD_IPSEC;+err=typhoon_set_offload(tp);+if(err<0)+printk(KERN_ERR"%s: unable to disable IPSEC "+"offload (%d)\n",tp->name,-err);+}+spin_unlock_bh(&tp->offload_lock);++error:+returnNULL;+}++staticvoid+typhoon_xfrm_state_add(structnet_device*dev,structxfrm_state*x)+{+structtyphoon*tp=netdev_priv(dev);++smp_rmb();+if(tp->card_state==Running)+typhoon_offload_ipsec(tp,x);+}++staticvoid+typhoon_xfrm_state_del(structnet_device*dev,structxfrm_offload*xol)+{+structtyphoon*tp=netdev_priv(dev);+structtyphoon_xfrm_offload*txo=xfrm_offload_priv(xol);+structcmd_descxp_cmd;+interr;++smp_rmb();+if(tp->card_state!=Running)+return;++INIT_COMMAND_NO_RESPONSE(&xp_cmd,TYPHOON_CMD_DELETE_SA);+xp_cmd.parm1=cpu_to_le16(txo->sa_cookie);+if(typhoon_issue_command(tp,1,&xp_cmd,0,NULL)<0){+printk(KERN_ERR"%s: unable to remove offloaded SA 0x%04x\n",+tp->name,txo->sa_cookie);+}++spin_lock_bh(&tp->offload_lock);+if(txo->inbound)+tp->rx_sa_avail++;+else+tp->tx_sa_avail++;+tp->sa_count--;+if(!tp->sa_count){+tp->offload&=~TYPHOON_OFFLOAD_IPSEC;+err=typhoon_set_offload(tp);+if(err<0)+printk(KERN_ERR"%s: unable to disable IPSEC "+"offload (%d)\n",tp->name,-err);+}+spin_unlock_bh(&tp->offload_lock);+}+staticvoidtyphoon_tx_timeout(structnet_device*dev){
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 09:04:35
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:57:35-05:00 dave@thedillows.org
# Move the setting of the currently offloaded tasks to its own
# function, as we'll be making use of it to change the crypto
# offload status when adding/removing xfrms.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.c
# 2004/12/30 00:57:17-05:00 dave@thedillows.org +26 -15
# Move the setting of the currently offloaded tasks to its own
# function, as we'll be making use of it to change the crypto
# offload status when adding/removing xfrms.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
@@ -725,11 +726,28 @@returnerr;}+staticint+typhoon_set_offload(structtyphoon*tp)+{+/* Caller should hold tp->offload_lock, or otherwise guarantee+*exclusitivitytothisroutine.+*/+structcmd_descxp_cmd;++smp_rmb();+if(tp->card_state!=Running)+return0;++INIT_COMMAND_WITH_RESPONSE(&xp_cmd,TYPHOON_CMD_SET_OFFLOAD_TASKS);+xp_cmd.parm2=tp->offload;+xp_cmd.parm3=tp->offload;+returntyphoon_issue_command(tp,1,&xp_cmd,0,NULL);+}+staticvoidtyphoon_vlan_rx_register(structnet_device*dev,structvlan_group*grp){structtyphoon*tp=netdev_priv(dev);-structcmd_descxp_cmd;interr;spin_lock_bh(&tp->state_lock);
@@ -737,25 +755,16 @@/* We've either been turned on for the first time, or we've*beenturnedoff.Updatethe3XP.*/+spin_lock_bh(&tp->offload_lock);if(grp)tp->offload|=TYPHOON_OFFLOAD_VLAN;elsetp->offload&=~TYPHOON_OFFLOAD_VLAN;+err=typhoon_set_offload(tp);+spin_unlock_bh(&tp->offload_lock);-/* If the interface is up, the runtime is running -- and we-*mustbeupforthevlancoretocallus.-*-*Dothecommandoutsideofthespinlock,asitisslow.-*/-INIT_COMMAND_WITH_RESPONSE(&xp_cmd,-TYPHOON_CMD_SET_OFFLOAD_TASKS);-xp_cmd.parm2=tp->offload;-xp_cmd.parm3=tp->offload;-spin_unlock_bh(&tp->state_lock);-err=typhoon_issue_command(tp,1,&xp_cmd,0,NULL);if(err<0)printk("%s: vlan offload error %d\n",tp->name,-err);-spin_lock_bh(&tp->state_lock);}/* now make the change visible */
@@ -2218,12 +2228,13 @@if(err<0)gotoerror_out;+/* tp->card_state != Running, so nothing will change this out+*fromunderus.+*/INIT_COMMAND_NO_RESPONSE(&xp_cmd,TYPHOON_CMD_SET_OFFLOAD_TASKS);-spin_lock_bh(&tp->state_lock);xp_cmd.parm2=tp->offload;xp_cmd.parm3=tp->offload;err=typhoon_issue_command(tp,1,&xp_cmd,0,NULL);-spin_unlock_bh(&tp->state_lock);if(err<0)gotoerror_out;
@@ -73,3 +73,19 @@ dev_close code and comments in net/core/dev.c for more info. Context: softirq+dev->xfrm_state_add:+ Synchronization: None, but can be called inside dev_base_lock rwlock+ Context: nominally process, but don't sleep inside an rwlock+ Notes: Only called for inbound xfrm_state(s). Can be invoked during+ xfrm_accel_add() call.++dev->xfrm_state_del:+ Synchronization: None, but can be called inside dev->xmit_lock spinlock.+ Context: BHs disabled/softirq+ Notes: Called for all offloaded xfrm_state(s). Can be invoked during+ xfrm_accel_flush() call.++dev->xfrm_bundle_add:+ Synchronization: None+ Context: softirq/process+ Notes: Called for newly created outbound xfrm bundles.
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 09:09:54
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 01:05:45-05:00 dave@thedillows.org
# Export the xfrm offload callbacks, and let the world know we
# support IPSEC offload.
#
# While we're at it, allow this to controlled by ethtool.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.c
# 2004/12/30 01:05:27-05:00 dave@thedillows.org +23 -4
# Export the xfrm offload callbacks, and let the world know we
# support IPSEC offload.
#
# While we're at it, allow this to controlled by ethtool.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
@@ -2327,6 +2335,14 @@typhoon_clean_tx(tp,&tp->txLoRing,&indexes->txLoCleared);}+if(tp->dev->features&NETIF_F_IPSEC)+xfrm_accel_flush(tp->dev);++/* tp->card_state != Running, so nothing will change this out+*fromunderus.+*/+tp->offload&=~TYPHOON_OFFLOAD_IPSEC;+return0;}
@@ -3183,6 +3199,9 @@dev->set_mac_address=typhoon_set_mac_address;dev->vlan_rx_register=typhoon_vlan_rx_register;dev->vlan_rx_kill_vid=typhoon_vlan_rx_kill_vid;+dev->xfrm_state_add=typhoon_xfrm_state_add;+dev->xfrm_state_del=typhoon_xfrm_state_del;+dev->xfrm_bundle_add=typhoon_xfrm_bundle_add;SET_ETHTOOL_OPS(dev,&typhoon_ethtool_ops);/* We can handle scatter gather, up to 16 entries, and
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 09:19:36
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 01:00:43-05:00 dave@thedillows.org
# Add routines to validate that the xfrm_state passed to them is
# one that we can offload to the 3XP.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.c
# 2004/12/30 01:00:25-05:00 dave@thedillows.org +90 -0
# Add routines to validate that the xfrm_state passed to them is
# one that we can offload to the 3XP.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 09:19:37
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 01:04:39-05:00 dave@thedillows.org
# Add the offloading of outbound bundles.
#
# This is a tricky business, because there are restrictions on
# the types and order of the xfrms we can offload. Some combinations
# also yield incorrect results, so we have to reduce the amount of
# offloading we do in those cases.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.c
# 2004/12/30 01:04:20-05:00 dave@thedillows.org +134 -0
# Add the offloading of outbound bundles.
#
# This is a tricky business, because there are restrictions on
# the types and order of the xfrms we can offload. Some combinations
# also yield incorrect results, so we have to reduce the amount of
# offloading we do in those cases.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
@@ -2587,6 +2587,140 @@spin_unlock_bh(&tp->offload_lock);}+staticinlineint+typhoon_max_offload(structxfrm_bundle_list*xbl)+{+/* Pre-scan the bundle to avoid offloading problematic sequences.+*Onlyreducestheoffloadleveltokeepasmuchadvantageas+*possible.+*+*For03.001.002--stillproblematicfor03.001.008,butneed+*re-verifysymptoms.+*+*innerAHtunnel,outerAHtransport+*-->3XPseemstoputtheinnerhashatthewronglocation+*innerAHtunnel,outerESPtunnel+*-->3XPcorruptsouterhash,maybewrongplace?+*innerAHtransport,outerESPtunnel+*-->3XPseemstoencryptthewrongportionofthepacket+*innerESPtransport,outerAHtunnel+*-->3XPlockup,requiresreset+*/+structxfrm_bundle_list*bundle;+structdst_entry*dst;+structxfrm_state*x;+intlast_was_ah=0,last_was_tunnel=0;+intmax_level=2;+intproto;++list_for_each_entry_reverse(bundle,&xbl->node,node){+dst=bundle->dst;+x=dst->xfrm;++proto=x?x->type->proto:IPPROTO_IP;++if(proto==IPPROTO_AH&&x->props.mode&&+(last_was_ah^last_was_tunnel))+gotoproblem_offload;++if(proto==IPPROTO_AH&&!x->props.mode&&+(!last_was_ah&&last_was_tunnel))+gotoproblem_offload;++if(proto==IPPROTO_ESP&&last_was_ah&&last_was_tunnel)+gotoproblem_offload;++last_was_ah=(proto==IPPROTO_AH)?1:0;+last_was_tunnel=x?x->props.mode:0;+continue;++problem_offload:+max_level--;+break;+}++returnmax_level;+}++staticvoid+typhoon_xfrm_bundle_add(structnet_device*dev,structxfrm_bundle_list*xbl)+{+/* Walk from the outermost dst back up the chain, offloading+*untilwehitsomethingwecannotdealwith.+*/+structtyphoon*tp=netdev_priv(dev);+structxfrm_bundle_list*bundle;+structdst_entry*dst;+structxfrm_state*x;+structxfrm_offload*xol;+structtyphoon_xfrm_offload*txo;+intproto;+intlevel=0,max_level;+intlast=-1;++smp_rmb();+if(tp->card_state!=Running)+return;++max_level=typhoon_max_offload(xbl);++list_for_each_entry_reverse(bundle,&xbl->node,node){+dst=bundle->dst;+x=dst->xfrm;++/* Only support IPv4 */+if(dst->ops->family!=AF_INET)+gotocannot_offload;++proto=x?x->type->proto:IPPROTO_IP;++switch(proto){+caseIPPROTO_IP:+caseIPPROTO_IPIP:+if(last==IPPROTO_IP||last==IPPROTO_IPIP)+gotocannot_offload;+if(level)+level++;+last=proto;+continue;+caseIPPROTO_ESP:+if(last!=IPPROTO_AH)+level++;+break;+caseIPPROTO_AH:+level++;+break;+default:+gotocannot_offload;+}++last=proto;+if(level>max_level)+gotocannot_offload;++if(dst->xfrm_offload)+continue;++xol=xfrm_offload_get(x,dev);+if(!xol){+xol=typhoon_offload_ipsec(tp,x);+if(xol)+xfrm_offload_hold(xol);+}++if(!xol)+gotocannot_offload;++dst->xfrm_offload=xol;+txo=xfrm_offload_priv(xol);+if(txo->tunnel)+last=IPPROTO_IPIP;+}++cannot_offload:+return;+}+staticvoidtyphoon_tx_timeout(structnet_device*dev){
From: David Dillow <dave@thedillows.org> Date: 2004-12-30 09:23:18
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:56:40-05:00 dave@thedillows.org
# Collect some information about the Typhoon's offload capabilities,
# and store it for future use.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.h
# 2004/12/30 00:56:22-05:00 dave@thedillows.org +14 -0
# Add the reply message format for the crypto capability query command.
#
# Signed-off-by: David Dillow [off-list ref]
#
# drivers/net/typhoon.c
# 2004/12/30 00:56:22-05:00 dave@thedillows.org +56 -0
# Collect some information about the Typhoon's offload capabilities,
# and store it for future use.
#
# Signed-off-by: David Dillow [off-list ref]
#
diff -Nru a/drivers/net/typhoon.c b/drivers/net/typhoon.c
@@ -2105,6 +2111,53 @@return0;}+staticinlineint+typhoon_ipsec_init(structtyphoon*tp)+{+structcmd_descxp_cmd;+structresp_descxp_resp;+structipsec_info_desc*info=(structipsec_info_desc*)&xp_resp;+u16last_tx,last_rx,last_cap;+interr;++last_tx=tp->tx_sa_max;+last_rx=tp->rx_sa_max;+last_cap=tp->capabilities;++INIT_COMMAND_WITH_RESPONSE(&xp_cmd,TYPHOON_CMD_READ_IPSEC_INFO);+err=typhoon_issue_command(tp,1,&xp_cmd,1,&xp_resp);+if(err<0)+gotoout;++/* We're not up yet, so no need to lock this -- we cannot modify+*thesefieldsyet.+*/+tp->tx_sa_avail=tp->tx_sa_max=le16_to_cpu(info->tx_sa_max);+tp->rx_sa_avail=tp->rx_sa_max=le16_to_cpu(info->rx_sa_max);+tp->sa_count=0;++/* Typhoon2 was originally going to have variable crypto capabilities,+*subjecttoregistrationwith3Com.Itappearstheyhavedecided+*tojustenable3DESaswell.+*/+if(tp->capabilities&TYPHOON_CRYPTO_VARIABLE){+tp->capabilities&=~TYPHOON_CRYPTO_VARIABLE;+tp->capabilities|=TYPHOON_CRYPTO_DES|TYPHOON_CRYPTO_3DES;+}++if(last_tx!=tp->tx_sa_max||last_rx!=tp->rx_sa_max||+last_cap!=tp->capabilities){+printk(KERN_INFO"%s: IPSEC offload %s%s %d Tx %d Rx\n",+tp->name,+tp->capabilities&TYPHOON_CRYPTO_DES?"DES ":"",+tp->capabilities&TYPHOON_CRYPTO_3DES?"3DES":"",+tp->tx_sa_max,tp->rx_sa_max);+}++out:+returnerr;+}+staticinttyphoon_start_runtime(structtyphoon*tp){
@@ -779,12 +788,115 @@tcpd->status=0;}+staticinlineint+typhoon_ipsec_fill(structtyphoon*tp,structsk_buff*skb,+structtransmit_ring*txRing)+{+structxfrm_offload*xol;+structtyphoon_xfrm_offload*txo;+structipsec_desc*ipsec;+intlast_was_esp=0;+inti,entry;+u32sa[3];++ipsec=(structipsec_desc*)(txRing->ringBase+txRing->lastWrite);+typhoon_inc_tx_index(&txRing->lastWrite,1);++ipsec->flags=TYPHOON_OPT_DESC|TYPHOON_OPT_IPSEC;+ipsec->numDesc=1;+ipsec->ipsecFlags=TYPHOON_IPSEC_USE_IV;+ipsec->reserved=0;+sa[0]=sa[1]=sa[2]=0;++/* Fill the offload descriptor with the cookies to indicate+*whichkeysettousewhen.Whilewe'reloopingthroughthe+*offloadedxfrms,ifthelastxfrmwasESP,andwe'redoing+*AHnow,*thenwecanmovetheESPparttothetopofthe+*descriptor.Otherwise,we'llneedtomovetothenextone.+*Weoverrunintosa[2]topreventneedingtochecktheentry+*limitinthemiddileofthings.+*/+entry=i=0;+xol=skb_get_xfrm_offload(skb,i++);+while(xol&&entry<2){+xfrm_offload_hold(xol);+txo=xfrm_offload_priv(xol);+if(sa[entry]&&txo->tunnel)+entry++;+if(sa[entry]&0xffff){+if(last_was_esp&&txo->ah)+sa[entry]<<=16;+else+entry++;+}++sa[entry]|=txo->sa_cookie;+last_was_esp=!txo->ah;++xol=skb_get_xfrm_offload(skb,i++);+}++/* Make sure we used all of the xfrms that were offloaded.+*/+if(unlikely(entry==2&&xol)){+if(net_ratelimit())+printk(KERN_ERR"%s: failing to offload IPSEC packet "+"with too many xfrms!\n",tp->name);+gotobad_packet;+}++ipsec->sa[0]=cpu_to_le16(sa[0]&0xffff);+ipsec->sa[1]=cpu_to_le16(sa[0]>>16);+ipsec->sa[2]=cpu_to_le16(sa[1]&0xffff);+ipsec->sa[3]=cpu_to_le16(sa[1]>>16);++/* The current 3XP firmware seems to hang if we try to feed it+*thesame(non-zero)SAtwiceonthesamepacket.So,detect+*anddropthosepacketsasitislikelyastackbug,or+*misconfigurationofpolicy.+*+*I.e.,weshouldneverhitthis.+*/+if(unlikely(ipsec->sa[2])){+if(unlikely(ipsec->sa[2]==ipsec->sa[3]))+gotoavoiding_sa_hang;+if(unlikely(ipsec->sa[2]==ipsec->sa[0]||+ipsec->sa[2]==ipsec->sa[1]))+gotoavoiding_sa_hang;+if(unlikely(ipsec->sa[3]&&(ipsec->sa[3]==ipsec->sa[0]||+ipsec->sa[3]==ipsec->sa[1])))+gotoavoiding_sa_hang;+}++if(unlikely(ipsec->sa[1]&&ipsec->sa[0]==ipsec->sa[1]))+gotoavoiding_sa_hang;++return0;++avoiding_sa_hang:+if(net_ratelimit())+printk(KERN_ERR"%s: failing attempted IPSEC offload with "+"duplicate SAs %08x %08x\n",tp->name,+sa[0],sa[1]);++bad_packet:+/* Any xfrm_offloads we've attached to this skb will be+*releasedforuswhentyphoon_start_tx()callsdev_kfree_skb_any()+*onit.+*+*Returnanerrortoindicatethispacketcannotbeoffloadedas+*specifiedandshouldnevermakeittothewire.+*/+return-EINVAL;+}+staticinttyphoon_start_tx(structsk_buff*skb,structnet_device*dev){structtyphoon*tp=netdev_priv(dev);structtransmit_ring*txRing;structtx_desc*txd,*first_txd;+u32origLastWrite;dma_addr_tskb_dma;intnumDesc;
@@ -811,6 +923,9 @@if(skb_tso_size(skb))numDesc++;+if(skb_has_xfrm_offload(skb))+numDesc++;+/* When checking for free space in the ring, we need to also*accountfortheinitialTxdescriptor,andwealwaysmustleave*atleastonedescriptorunusedintheringsothatitdoesn't
@@ -146,6 +146,9 @@/* Private data of this transformer, format is opaque,*interpretedbyxfrm_typemethods.*/void*data;++/* Intended direction of this state, used for offloading */+intdir;};enum{
Any specific reason to first define such a nice enum and then using int
in the struct?
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
@@ -146,6 +146,9 @@/* Private data of this transformer, format is opaque,*interpretedbyxfrm_typemethods.*/void*data;++/* Intended direction of this state, used for offloading */+intdir;};enum{
Any specific reason to first define such a nice enum and then using int
in the struct?
Just following the current style in net/xfrm.h, see xfrm_state.km.state
and XFRM_STATE_*.
Though, I probably should have used a u8; easily changed if it is an
issue.
--
Dave Dillow [off-list ref]
Any specific reason to first define such a nice enum and then using int
in the struct?
Just following the current style in net/xfrm.h, see xfrm_state.km.state
and XFRM_STATE_*.
Hmmm... Maybe I'd prepare patches then :)
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
If the driver knows the max depth which is allowed, why not have it
allocate its own bundle-like struct during initialization one for once ?
Instead of pushing the bundle list, dst is walked by the code of
the device's own xyz_xfrm_bundle_add into the said circular list,
entries get overwriten if the dst chain is longer and when the end of
dst is reached, the bundle-like list is walked in reverse order.
It avoids a few failure points imho.
--
Ueimor
If the driver knows the max depth which is allowed, why not have it
allocate its own bundle-like struct during initialization one for once ?
Instead of pushing the bundle list, dst is walked by the code of
the device's own xyz_xfrm_bundle_add into the said circular list,
entries get overwriten if the dst chain is longer and when the end of
dst is reached, the bundle-like list is walked in reverse order.
It avoids a few failure points imho.
Good idea! I'll see if I can't code it up. I definitely want to get rid
of that GFP_ATOMIC allocation.
--
David Dillow [off-list ref]
It's a shame we don't have an assembler optimized 3DES at least for
x86 in the crypto library of the kernel, even if just for tests such
as this.
I'll start reviewing your patches right now.
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-21 22:38:57
On Thu, 30 Dec 2004 03:48:34 -0500
David Dillow [off-list ref] wrote:
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:27:15-05:00 dave@thedillows.org
# Add direction information to xfrm_state. This will be needed to
# offload xfrm processing to the NIC.
And xfrm_state, when actually used, does not exist in a vacuum.
It is being used in terms of a particular xfrm_policy which has
a direction indicator already.
Given that, it seems the xfrm_state is not only unnecessary, but
also slightly illogical in fact. If you need the direction, you
should make sure that at those places you can get back to the
policy in order to obtain that piece of information.
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-21 22:40:45
On Thu, 30 Dec 2004 03:48:34 -0500
David Dillow [off-list ref] wrote:
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:28:25-05:00 dave@thedillows.org
# Add the xfrm offload management calls to struct netdevice.
#
# xfrm_state_add() is called for inbound xfrm states
# xfrm_bundle_add() is called for outbound xfrm bundles
# xfrm_state_del() is called for all offloaded xfrms,
# inbound or outbound.
#
# If a driver adds NETIF_F_IPSEC to its features, it must
# provide all three callbacks.
#
# Signed-off-by: David Dillow [off-list ref]
I have no problems with this interface. In fact, just about
whatever works for the card you have would be fine for an
initial implementation.
We know this interface will likely need to change over time
so we should explicitly mark these callbacks as in a state
of flux so that nobody who tries to play with this gets surprised
when we tweak it.
This whole scheme looks buggy. The intent is to 8-byte align
the object, but look at what the code is actually doing.
Whatever kmalloc() returns to xfrm_offload_alloc() is directly
used as the xfrm_offload pointer, and the members are initialized.
Then xfrm_offload_priv() does the alignments.
It is clear that kmalloc() is always giving you 8-byte aligned
data else the first time xfrm_offload_priv() is used you'd
get a bogus pointer since xfrm_offload_alloc() initialized
the object without first aligning the pointer.
We do something similar when we allocate netdevs, so have a look
at how net/core/dev.c:alloc_netdev() works.
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-21 23:21:27
On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow [off-list ref] wrote:
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:35:34-05:00 dave@thedillows.org
# Add a parameter to the decision function(s) used by
# xfrm_prune_bundles(). This will allow us to have more
# fine grained selection of bundles pruned (like, say,
# per device.)
#
# Signed-off-by: David Dillow [off-list ref]
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-21 23:23:58
On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow [off-list ref] wrote:
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:37:44-05:00 dave@thedillows.org
# Give device drivers a method to allow the use of crypto
# offload features for existing xfrm_states and bundles, as
# well as dynamically remove crypto offload capabilities.
#
# Signed-off-by: David Dillow [off-list ref]
This change looks fine, but it reminds me of something.
We have to make sure there is no way for xfrm_offloads to
refer to a device when it is brought down.
One way to ensure that, is to make each xfrm_offload attached
grab a reference to the device it is for. The refcounting will
help catch cases where stale xfrm_offload are around at
unregister_netdev() time.
From: David Dillow <dave@thedillows.org> Date: 2005-01-22 05:50:22
On Fri, 2005-01-21 at 14:38 -0800, David S. Miller wrote:
On Thu, 30 Dec 2004 03:48:34 -0500
David Dillow [off-list ref] wrote:
quoted
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:27:15-05:00 dave@thedillows.org
# Add direction information to xfrm_state. This will be needed to
# offload xfrm processing to the NIC.
And xfrm_state, when actually used, does not exist in a vacuum.
It is being used in terms of a particular xfrm_policy which has
a direction indicator already.
Given that, it seems the xfrm_state is not only unnecessary, but
also slightly illogical in fact. If you need the direction, you
should make sure that at those places you can get back to the
policy in order to obtain that piece of information.
Hmm, I can see having netdev->xfrm_bundle_add() assume a outbound state,
since it's only called for Tx.
Rx offloading could be interesting, but it looks like I could hook into
net/xfrm/xfrm_policy.c:__xfrm_policy_check(), and add the xfrm_states
seen there to a work queue for offloading. netdev->xfrm_state_add()
would then only see inbound states.
Sound sane?
--
David Dillow [off-list ref]
From: David Dillow <dave@thedillows.org> Date: 2005-01-22 05:52:13
On Fri, 2005-01-21 at 14:56 -0800, David S. Miller wrote:
On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow [off-list ref] wrote:
quoted
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:33:11-05:00 dave@thedillows.org
# Plumb in offloading of inbound xfrm_states.
#
# Signed-off-by: David Dillow [off-list ref]
Hmmm, this seems to deadlock. xfrm_state_add() is invoked
with the RTNL semaphore already acquired. For example, via
xfrm_user.c:xfrm_add_sa()
Are you sure? It worked just fine during testing, and I don't see where
in the call chain from xfrm_netlink_rcv() to xfrm_state_add() the RTNL
sema is taken. Perhaps you were thinking of xfrm_cfg_sem?
--
David Dillow [off-list ref]
netif_running() is only steady while the RTNL semaphore is held,
which is not necessarily true when xfrm_lookup() is invoked.
I've been reworking this area already to add the ability for the user to
control which devices/states combinations can be offloaded, so this code
will change. What do you think about just having the driver be
responsible for ignoring offload requests when they don't make sense,
like when the device is down, or in sleep mode?
BTW, xfrm_lookup() also calls stale_bundle(), which also relies on
netif_running(), among other tests.
--
David Dillow [off-list ref]
From: David Dillow <dave@thedillows.org> Date: 2005-01-22 05:53:36
On Fri, 2005-01-21 at 15:23 -0800, David S. Miller wrote:
On Thu, 30 Dec 2004 03:48:35 -0500
David Dillow [off-list ref] wrote:
quoted
# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
# 2004/12/30 00:37:44-05:00 dave@thedillows.org
# Give device drivers a method to allow the use of crypto
# offload features for existing xfrm_states and bundles, as
# well as dynamically remove crypto offload capabilities.
#
# Signed-off-by: David Dillow [off-list ref]
This change looks fine, but it reminds me of something.
We have to make sure there is no way for xfrm_offloads to
refer to a device when it is brought down.
One way to ensure that, is to make each xfrm_offload attached
grab a reference to the device it is for. The refcounting will
help catch cases where stale xfrm_offload are around at
unregister_netdev() time.
You're right, it would be a problem, and I have the oopsen to prove
it. :)
In patch 3 of the series, xfrm_offload_add() grabs a reference to the
device, and xfrm_offload_release() releases it when the offload's own
reference count hits zero.
--
David Dillow [off-list ref]
This whole scheme looks buggy. The intent is to 8-byte align
the object, but look at what the code is actually doing.
Whatever kmalloc() returns to xfrm_offload_alloc() is directly
used as the xfrm_offload pointer, and the members are initialized.
Then xfrm_offload_priv() does the alignments.
Right, I'm not worried about the alignment of the xfrm_offload, as I'm
assuming that kmalloc will give us something suitable. However, I do
want to make sure that the private section used by the driver is 8 byte
aligned in the face of a potentially oddly sized xfrm_offload struct.
Am I making a bad assumption that kmalloc() will give me a suitable
alignment for the base struct?
It is clear that kmalloc() is always giving you 8-byte aligned
data else the first time xfrm_offload_priv() is used you'd
get a bogus pointer since xfrm_offload_alloc() initialized
the object without first aligning the pointer.
The driver calls xfrm_offload_alloc(), then calls xfrm_offload_priv() to
get it's private struct, and inits that. I just want to make sure the
private struct is 8 byte aligned, which I assume (wrongly?) will be
sufficient to prevent misaligned accesses.
We do something similar when we allocate netdevs, so have a look
at how net/core/dev.c:alloc_netdev() works.
I copied part of that code to xfrm_offload_alloc(), removing the part
that aligned the base xfrm_offload struct. 32 byte alignment seemed a
bit much, do you think it is needed? I'll be happy to add that if you
want.
--
David Dillow [off-list ref]
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-26 06:11:06
On Sat, 22 Jan 2005 00:53:36 -0500
David Dillow [off-list ref] wrote:
In patch 3 of the series, xfrm_offload_add() grabs a reference to the
device, and xfrm_offload_release() releases it when the offload's own
reference count hits zero.
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-26 06:11:54
On Sat, 22 Jan 2005 00:53:01 -0500
David Dillow [off-list ref] wrote:
I've been reworking this area already to add the ability for the user to
control which devices/states combinations can be offloaded, so this code
will change. What do you think about just having the driver be
responsible for ignoring offload requests when they don't make sense,
like when the device is down, or in sleep mode?
That would be fine.
BTW, xfrm_lookup() also calls stale_bundle(), which also relies on
netif_running(), among other tests.
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-26 06:13:25
On Sat, 22 Jan 2005 00:52:13 -0500
David Dillow [off-list ref] wrote:
quoted
Hmmm, this seems to deadlock. xfrm_state_add() is invoked
with the RTNL semaphore already acquired. For example, via
xfrm_user.c:xfrm_add_sa()
Are you sure? It worked just fine during testing, and I don't see where
in the call chain from xfrm_netlink_rcv() to xfrm_state_add() the RTNL
sema is taken. Perhaps you were thinking of xfrm_cfg_sem?
From: "David S. Miller" <davem@davemloft.net> Date: 2005-01-26 06:17:26
On Sat, 22 Jan 2005 00:50:22 -0500
David Dillow [off-list ref] wrote:
Hmm, I can see having netdev->xfrm_bundle_add() assume a outbound state,
since it's only called for Tx.
Rx offloading could be interesting, but it looks like I could hook into
net/xfrm/xfrm_policy.c:__xfrm_policy_check(), and add the xfrm_states
seen there to a work queue for offloading. netdev->xfrm_state_add()
would then only see inbound states.
Sound sane?
You really can't get at the policies at these places somehow?
Even by passing down the xfrm_policy pointer into these
call chains?
If you could get at the xfrm_policy, then you could get the
direction.
From: David Dillow <dave@thedillows.org> Date: 2005-01-26 21:14:52
On Tue, 2005-01-25 at 22:17 -0800, David S. Miller wrote:
On Sat, 22 Jan 2005 00:50:22 -0500
David Dillow [off-list ref] wrote:
quoted
Hmm, I can see having netdev->xfrm_bundle_add() assume a outbound state,
since it's only called for Tx.
Rx offloading could be interesting, but it looks like I could hook into
net/xfrm/xfrm_policy.c:__xfrm_policy_check(), and add the xfrm_states
seen there to a work queue for offloading. netdev->xfrm_state_add()
would then only see inbound states.
Sound sane?
You really can't get at the policies at these places somehow?
Even by passing down the xfrm_policy pointer into these
call chains?
By these places, you mean xfrm_lookup() and __xfrm_policy_check()?
There I absolutely have access to the policy, but even better, I only
get to those places for one purpose -- xfrm_lookup() on output,
__xfrm_policy_check() on input.
Even in the first versions of the offload code, netdev->xfrm_state_add()
and netdev->xfrm_bundle_add() were solely called for inbound and
outbound xfrm_states, respectively. The problem was that I had no access
to the policy at xfrm_state.c::xfrm_state_add(), since it wouldn't be
called in the context of a policy.
By moving the offload calls/queuing to __xfrm_policy_check(), I have the
policy available, and I *know* it is inbound. If I offload SAs there, I
can use that knowledge, and I get the benefit of lazy offload -- I'll
only be offloading inbound SAs to netdev's that they've actually been
received on, rather than to all capable devices, as the first version
did.
And, by making the decision to offload at the policy check means I get
more info to use in that decision -- I'll probably change netdev-
xfrm_state_add() into netdev->xfrm_secpath_add(), and have it take the
inbound secpath as an argument. This will let the driver see a more
complete picture of the inbound xfrms, and allow it to not offload
problematic combinations (the typhoon firmware has issues with certain
combinations.)
Now I'm working on a nice cheap way to decide if the inbound packet is a
candidate for offloading, and avoid trying the same SA stack over and
over and over...
--
David Dillow [off-list ref]