From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:38
These patches implement the basic infrastructure to allow swap over networked
storage.
The basic idea is to reserve some memory up front to use when regular memory
runs out.
To bound network behaviour we accept only a limited number of concurrent
packets and drop those packets that are not aimed at the connection(s) servicing
the VM. Also all network paths that interact with userspace are to be avoided -
e.g. taps and NF_QUEUE.
PF_MEMALLOC is set when processing emergency skbs. This makes sense in that we
are indeed working on behalf of the swapper/VM. This allows us to use the
regular memory allocators for processing but requires that said processing have
bounded memory usage and has that accounted in the reserve.
I am particularly looking for comments on the design; is this acceptable?
Kind regards,
Peter
--
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:24
Provide means to reserve a specific amount pages.
The emergency pool is separated from the min watermark because ALLOC_HARDER
and ALLOC_HIGH modify the watermark in a relative way and thus do not ensure
a strict minimum.
Signed-off-by: Peter Zijlstra <redacted>
---
include/linux/mmzone.h | 3 +-
mm/page_alloc.c | 52 ++++++++++++++++++++++++++++++++++++++++---------
mm/vmstat.c | 6 ++---
3 files changed, 48 insertions(+), 13 deletions(-)
Index: linux-2.6-git/include/linux/mmzone.h
===================================================================
@@ -156,7 +156,7 @@ enum zone_type {structzone{/* Fields commonly accessed by the page allocator */unsignedlongfree_pages;-unsignedlongpages_min,pages_low,pages_high;+unsignedlongpages_emerg,pages_min,pages_low,pages_high;/**Wedon'tknowifthememorythatwe'regoingtoallocatewillbefreeable*or/anditwillbereleasedeventually,sotoavoidtotallywastingseveral
@@ -540,6 +540,7 @@ int sysctl_min_unmapped_ratio_sysctl_hanstructfile*,void__user*,size_t*,loff_t*);intsysctl_min_slab_ratio_sysctl_handler(structctl_table*,int,structfile*,void__user*,size_t*,loff_t*);+voidadjust_memalloc_reserve(intpages);#include<linux/topology.h>/* Returns the number of the current Node. */
@@ -991,7 +992,8 @@ int zone_watermark_ok(struct zone *z, inif(alloc_flags&ALLOC_HARDER)min-=min/4;-if(free_pages<=min+z->lowmem_reserve[classzone_idx])+if(free_pages<=min+z->lowmem_reserve[classzone_idx]++z->pages_emerg)return0;for(o=0;o<order;o++){/* At the next order, this order's pages become unavailable */
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:38
Provide a method to calculate the number of pages needed to store a given
number of slab objects (upper bound when considering possible partial and
free slabs).
Signed-off-by: Peter Zijlstra <redacted>
---
include/linux/slab.h | 1 +
mm/slab.c | 6 ++++++
2 files changed, 7 insertions(+)
Index: linux-2.6-git/include/linux/slab.h
===================================================================
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:38
Introduce page allocation rank.
This allocation rank is an measure of the 'hardness' of the page allocation.
Where hardness refers to how deep we have to reach (and thereby if reclaim
was activated) to obtain the page.
It basically is a mapping from the ALLOC_/gfp flags into a scalar quantity,
which allows for comparisons of the kind:
'would this allocation have succeeded using these gfp flags'.
For the gfp -> alloc_flags mapping we use the 'hardest' possible, those
used by __alloc_pages() right before going into direct reclaim.
The alloc_flags -> rank mapping is given by: 2*2^wmark - harder - 2*high
where wmark = { min = 1, low, high } and harder, high are booleans.
This gives:
0 is the hardest possible allocation - ALLOC_NO_WATERMARK,
1 is ALLOC_WMARK_MIN|ALLOC_HARDER|ALLOC_HIGH,
...
15 is ALLOC_WMARK_HIGH|ALLOC_HARDER,
16 is the softest allocation - ALLOC_WMARK_HIGH.
Rank <= 4 will have woke up kswapd and when also > 0 might have ran into
direct reclaim.
Rank > 8 rarely happens and means lots of memory free (due to parallel oom kill).
The allocation rank is stored in page->index for successful allocations.
'offline' testing of the rank is made impossible by direct reclaim and
fragmentation issues. That is, it is impossible to tell if a given allocation
will succeed without actually doing it.
The purpose of this measure is to introduce some fairness into the slab
allocator.
Signed-off-by: Peter Zijlstra <redacted>
---
mm/internal.h | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
mm/page_alloc.c | 58 ++++++++++--------------------------
2 files changed, 106 insertions(+), 41 deletions(-)
Index: linux-2.6-git/mm/internal.h
===================================================================
@@ -1259,48 +1252,27 @@ restart:*OK,we'rebelowthekswapdwatermarkandhavekickedbackground*reclaim.Nowthingsgetmorecomplex,sosetupalloc_flagsaccording*tohowwewanttoproceed.-*-*Thecallermaydipintopagereservesabitmoreifthecaller-*cannotrundirectreclaim,orifthecallerhasrealtimescheduling-*policyorisaskingfor__GFP_HIGHmemory.GFP_ATOMICrequestswill-*setbothALLOC_HARDER(!wait)andALLOC_HIGH(__GFP_HIGH).*/-alloc_flags=ALLOC_WMARK_MIN;-if((unlikely(rt_task(p))&&!in_interrupt())||!wait)-alloc_flags|=ALLOC_HARDER;-if(gfp_mask&__GFP_HIGH)-alloc_flags|=ALLOC_HIGH;-if(wait)-alloc_flags|=ALLOC_CPUSET;+alloc_flags=gfp_to_alloc_flags(gfp_mask);-/*-*Gothroughthezonelistagain.Let__GFP_HIGHandallocations-*comingfromrealtimetasksgodeeperintoreserves.-*-*Thisisthelastchance,ingeneral,beforethegotonopage.-*IgnorecpusetifGFP_ATOMIC(!wait)ratherthanfailalloc.-*Seealsocpuset_zone_allowed()commentinkernel/cpuset.c.-*/-page=get_page_from_freelist(gfp_mask,order,zonelist,alloc_flags);+/* This is the last chance, in general, before the goto nopage. */+page=get_page_from_freelist(gfp_mask,order,zonelist,+alloc_flags&~ALLOC_NO_WATERMARKS);if(page)gotogot_pg;/* This allocation should allow future memory freeing. */-rebalance:-if(((p->flags&PF_MEMALLOC)||unlikely(test_thread_flag(TIF_MEMDIE)))-&&!in_interrupt()){-if(!(gfp_mask&__GFP_NOMEMALLOC)){+if(alloc_flags&ALLOC_NO_WATERMARKS){nofail_alloc:-/* go through the zonelist yet again, ignoring mins */-page=get_page_from_freelist(gfp_mask,order,+/* go through the zonelist yet again, ignoring mins */+page=get_page_from_freelist(gfp_mask,order,zonelist,ALLOC_NO_WATERMARKS);-if(page)-gotogot_pg;-if(gfp_mask&__GFP_NOFAIL){-congestion_wait(WRITE,HZ/50);-gotonofail_alloc;-}+if(page)+gotogot_pg;+if(wait&&(gfp_mask&__GFP_NOFAIL)){+congestion_wait(WRITE,HZ/50);+gotonofail_alloc;}gotonopage;}
@@ -1309,6 +1281,10 @@ nofail_alloc:if(!wait)gotonopage;+/* Avoid recursion of direct reclaim */+if(p->flags&PF_MEMALLOC)+gotonopage;+cond_resched();/* We now go into synchronous reclaim */
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:38
Allow PF_MEMALLOC to be set in softirq context. When running softirqs from
a borrowed context save current->flags, ksoftirqd will have its own
task_struct.
Signed-off-by: Peter Zijlstra <redacted>
---
kernel/softirq.c | 3 +++
mm/internal.h | 14 ++++++++------
2 files changed, 11 insertions(+), 6 deletions(-)
Index: linux-2.6-git/mm/internal.h
===================================================================
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:39
Allow the mempool to use the memalloc reserves when all else fails and
the allocation context would otherwise allow it.
Signed-off-by: Peter Zijlstra <redacted>
---
mm/mempool.c | 10 ++++++++++
1 file changed, 10 insertions(+)
Index: linux-2.6-git/mm/mempool.c
===================================================================
@@ -229,6 +230,15 @@ repeat_alloc:}spin_unlock_irqrestore(&pool->lock,flags);+/* if we really had right to the emergency reserves try those */+if(gfp_to_alloc_flags(gfp_mask)&ALLOC_NO_WATERMARKS){+if(gfp_temp&__GFP_NOMEMALLOC){+gfp_temp&=~(__GFP_NOMEMALLOC|__GFP_NOWARN);+gotorepeat_alloc;+}else+gfp_temp|=__GFP_NOMEMALLOC|__GFP_NOWARN;+}+/* We must not sleep in the GFP_ATOMIC case */if(!(gfp_mask&__GFP_WAIT))returnNULL;
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:39
The slab allocator has some unfairness wrt gfp flags; when the slab cache is
grown the gfp flags are used to allocate more memory, however when there is
slab cache available (in partial or free slabs, per cpu caches or otherwise)
gfp flags are ignored.
Thus it is possible for less critical slab allocations to succeed and gobble
up precious memory when under memory pressure.
This patch solves that by using the newly introduced page allocation rank.
Page allocation rank is a scalar quantity connecting ALLOC_ and gfp flags which
represents how deep we had to reach into our reserves when allocating a page.
Rank 0 is the deepest we can reach (ALLOC_NO_WATERMARK) and 16 is the most
shallow allocation possible (ALLOC_WMARK_HIGH).
When the slab space is grown the rank of the page allocation is stored. For
each slab allocation we test the given gfp flags against this rank. Thereby
asking the question: would these flags have allowed the slab to grow.
If not so, we need to test the current situation. This is done by forcing the
growth of the slab space. (Just testing the free page limits will not work due
to direct reclaim) Failing this we need to fail the slab allocation.
Thus if we grew the slab under great duress while PF_MEMALLOC was set and we
really did access the memalloc reserve the rank would be set to 0. If the next
allocation to that slab would be GFP_NOFS|__GFP_NOMEMALLOC (which ordinarily
maps to rank 4 and always > 0) we'd want to make sure that memory pressure has
decreased enough to allow an allocation with the given gfp flags.
So in this case we try to force grow the slab cache and on failure we fail the
slab allocation. Thus preserving the available slab cache for more pressing
allocations.
If this newly allocated slab will be trimmed on the next kmem_cache_free
(not unlikely) this is no problem, since 1) it will free memory and 2) the
sole purpose of the allocation was to probe the allocation rank, we didn't
need the space itself.
[AIM9 results go here]
Signed-off-by: Peter Zijlstra <redacted>
---
mm/slab.c | 61 ++++++++++++++++++++++++++++++++++++++-----------------------
1 file changed, 38 insertions(+), 23 deletions(-)
Index: linux-2.6-git/mm/slab.c
===================================================================
@@ -3011,14 +3017,16 @@ must_grow:l3->free_objects-=ac->avail;alloc_done:spin_unlock(&l3->list_lock);-if(unlikely(!ac->avail)){intx;+force_grow:x=cache_grow(cachep,flags|GFP_THISNODE,node,NULL);/* cache_grow can reenable interrupts, then ac could change. */ac=cpu_cache_get(cachep);-if(!x&&ac->avail==0)/* no objects in sight? abort */++/* no objects in sight? abort */+if(!x&&(ac->avail==0||rank>cachep->rank))returnNULL;if(!ac->avail)/* objects refilled by interrupt? */
@@ -3370,13 +3383,14 @@ retry:must_grow:spin_unlock(&l3->list_lock);+force_grow:x=cache_grow(cachep,flags|GFP_THISNODE,nodeid,NULL);if(x)gotoretry;if(!(flags&__GFP_THISNODE))/* Unable to grow the cache. Fall back to other nodes. */-returnfallback_alloc(cachep,flags);+returnfallback_alloc(cachep,flags,rank);returnNULL;
@@ -3615,16 +3630,16 @@ __cache_alloc_node(struct kmem_cache *ca*toothernodes.Itmayfailwhilewestillhave*objectsonothernodesavailable.*/-ptr=____cache_alloc(cachep,flags);+ptr=____cache_alloc(cachep,flags,rank);}if(!ptr){/* ___cache_alloc_node can fall back to other nodes */-ptr=____cache_alloc_node(cachep,flags,nodeid);+ptr=____cache_alloc_node(cachep,flags,nodeid,rank);}}else{/* Node not bootstrapped yet */if(!(flags&__GFP_THISNODE))-ptr=fallback_alloc(cachep,flags);+ptr=fallback_alloc(cachep,flags,rank);}local_irq_restore(save_flags);
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:43
There is a small race between the procfs caller and the memory hotplug caller
of setup_per_zone_pages_min(). Not a big deal, but the next patch will add yet
another caller. Time to close the gap.
Signed-off-by: Peter Zijlstra <redacted>
---
mm/page_alloc.c | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
Index: linux-2.6-git/mm/page_alloc.c
===================================================================
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:56
__GFP_EMERGENCY will allow the allocation to disregard the watermarks,
much like PF_MEMALLOC.
Signed-off-by: Peter Zijlstra <redacted>
---
include/linux/gfp.h | 7 ++++++-
mm/internal.h | 10 +++++++---
2 files changed, 13 insertions(+), 4 deletions(-)
Index: linux-2.6-git/include/linux/gfp.h
===================================================================
@@ -54,7 +58,8 @@ struct vm_area_struct;#define GFP_LEVEL_MASK (__GFP_WAIT|__GFP_HIGH|__GFP_IO|__GFP_FS| \__GFP_COLD|__GFP_NOWARN|__GFP_REPEAT|\__GFP_NOFAIL|__GFP_NORETRY|__GFP_NO_GROW|__GFP_COMP|\-__GFP_NOMEMALLOC|__GFP_HARDWALL|__GFP_THISNODE)+__GFP_NOMEMALLOC|__GFP_HARDWALL|__GFP_THISNODE|\+__GFP_EMERGENCY)/* This equals 0, but use constants in case they ever change */#define GFP_NOWAIT (GFP_ATOMIC & ~__GFP_HIGH)
From: Peter Zijlstra <hidden> Date: 2007-01-16 10:28:56
In order to provide robust networked storage there must be a guarantee
of progress. That is, the storage device must never stall because of (physical)
OOM, because the device itself might be needed to get out of it (reclaim).
This means that the device must always find enough memory to build/send packets
over the network _and_ receive (level 7) ACKs for those packets.
The network stack has a huge capacity for buffering packets; waiting for
user-space to read them. There is a practical limit imposed to avoid DoS
scenarios. These two things make for a deadlock; what if the receive limit is
reached and all packets are buffered in non-critical sockets (those not serving
the network storage device waiting for an ACK to free a page).
Memory pressure will add to that; what if there is simply no memory left to
receive packets in.
This patch provides a service to register sockets as critical; SOCK_VMIO
is a promise the socket will never block on receive. Along with with a memory
reserve that will service a limited number of packets this can guarantee a
limited service to these critical sockets.
When we make sure that packets allocated from the reserve will only service
critical sockets we will not lose the memory and can guarantee progress.
The reserve is calculated to exceed the IP fragment caches and match the route
cache.
(Note on the name SOCK_VMIO; the basic problem is a circular dependency between
the network and virtual memory subsystems which needs to be broken. This does
make VM network IO - and only VM network IO - special, it does not generalize)
Signed-off-by: Peter Zijlstra <redacted>
---
include/linux/skbuff.h | 13 +++-
include/net/sock.h | 42 ++++++++++++++-
net/core/dev.c | 40 +++++++++++++-
net/core/skbuff.c | 50 ++++++++++++++++--
net/core/sock.c | 121 +++++++++++++++++++++++++++++++++++++++++++++
net/core/stream.c | 5 +
net/ipv4/ip_fragment.c | 1
net/ipv4/ipmr.c | 4 +
net/ipv4/route.c | 15 +++++
net/ipv4/sysctl_net_ipv4.c | 14 ++++-
net/ipv4/tcp_ipv4.c | 27 +++++++++-
net/ipv6/reassembly.c | 1
net/ipv6/route.c | 15 +++++
net/ipv6/sysctl_net_ipv6.c | 6 +-
net/ipv6/tcp_ipv6.c | 27 +++++++++-
net/netfilter/core.c | 5 +
security/selinux/avc.c | 2
17 files changed, 361 insertions(+), 27 deletions(-)
Index: linux-2.6-git/include/linux/skbuff.h
===================================================================
@@ -392,6 +392,7 @@ enum sock_flags {SOCK_RCVTSTAMP,/* %SO_TIMESTAMP setting */SOCK_LOCALROUTE,/* route locally only, %SO_DONTROUTE setting */SOCK_QUEUE_SHRUNK,/* write queue has been shrunk recently */+SOCK_VMIO,/* the VM depends on us - make sure we're serviced */};staticinlinevoidsock_copy_flags(structsock*nsk,structsock*osk)
@@ -722,13 +758,13 @@ static inline void sk_stream_writequeue_staticinlineintsk_stream_rmem_schedule(structsock*sk,structsk_buff*skb){return(int)skb->truesize<=sk->sk_forward_alloc||-sk_stream_mem_schedule(sk,skb->truesize,1);+sk_stream_mem_schedule(sk,skb,skb->truesize,1);}staticinlineintsk_stream_wmem_schedule(structsock*sk,intsize){returnsize<=sk->sk_forward_alloc||-sk_stream_mem_schedule(sk,size,0);+sk_stream_mem_schedule(sk,NULL,size,0);}/* Used by processes to "lock" a socket state, so that
@@ -1767,10 +1767,23 @@ int netif_receive_skb(struct sk_buff *skstructnet_device*orig_dev;intret=NET_RX_DROP;__be16type;+unsignedlongpflags=current->flags;++/* Emergency skb are special, they should+*-bedeliveredtoSOCK_VMIOsocketsonly+*-stayawayfromuserspace+*-haveboundedmemoryusage+*+*UsePF_MEMALLOCasapoormansmemorypool-thegroupingkind.+*Thissavesusfrompropagatingtheallocationcontextdowntoall+*allocationsites.+*/+if(unlikely(skb->emergency))+current->flags|=PF_MEMALLOC;/* if we've gotten here through NAPI, check netpoll */if(skb->dev->poll&&netpoll_rx(skb))-returnNET_RX_DROP;+gotoout;if(!skb->tstamp.off_sec)net_timestamp(skb);
@@ -1781,7 +1794,7 @@ int netif_receive_skb(struct sk_buff *skorig_dev=skb_bond(skb);if(!orig_dev)-returnNET_RX_DROP;+gotoout;__get_cpu_var(netdev_rx_stat).total++;
@@ -1798,6 +1811,8 @@ int netif_receive_skb(struct sk_buff *skgotoncls;}#endif+if(unlikely(skb->emergency))+gotoskip_taps;list_for_each_entry_rcu(ptype,&ptype_all,list){if(!ptype->dev||ptype->dev==skb->dev){
@@ -1807,6 +1822,7 @@ int netif_receive_skb(struct sk_buff *sk}}+skip_taps:#ifdef CONFIG_NET_CLS_ACTif(pt_prev){ret=deliver_skb(skb,pt_prev,orig_dev);
@@ -1819,15 +1835,26 @@ int netif_receive_skb(struct sk_buff *skif(ret==TC_ACT_SHOT||(ret==TC_ACT_STOLEN)){kfree_skb(skb);-gotoout;+gotounlock;}skb->tc_verd=0;ncls:#endif+if(unlikely(skb->emergency))+switch(skb->protocol){+case__constant_htons(ETH_P_ARP):+case__constant_htons(ETH_P_IP):+case__constant_htons(ETH_P_IPV6):+break;++default:+gotodrop;+}+if(handle_bridge(&skb,&pt_prev,&ret,orig_dev))-gotoout;+gotounlock;type=skb->protocol;list_for_each_entry_rcu(ptype,&ptype_base[ntohs(type)&15],list){
@@ -1842,6 +1869,7 @@ ncls:if(pt_prev){ret=pt_prev->func(skb,skb->dev,pt_prev,orig_dev);}else{+drop:kfree_skb(skb);/* Jamal, now you will not able to escape explaining*mehowyouweregoingtousethis.:-)
@@ -142,28 +142,34 @@ EXPORT_SYMBOL(skb_truesize_bug);*%GFP_ATOMIC.*/structsk_buff*__alloc_skb(unsignedintsize,gfp_tgfp_mask,-intfclone,intnode)+intflags,intnode){structkmem_cache*cache;structskb_shared_info*shinfo;structsk_buff*skb;u8*data;+intemergency=0;-cache=fclone?skbuff_fclone_cache:skbuff_head_cache;+size=SKB_DATA_ALIGN(size);+cache=(flags&SKB_ALLOC_FCLONE)+?skbuff_fclone_cache:skbuff_head_cache;+if(flags&SKB_ALLOC_RX)+gfp_mask|=__GFP_NOMEMALLOC|__GFP_NOWARN;+retry_alloc:/* Get the HEAD */skb=kmem_cache_alloc_node(cache,gfp_mask&~__GFP_DMA,node);if(!skb)-gotoout;+gotonoskb;/* Get the DATA. Size must match skb_add_mtu(). */-size=SKB_DATA_ALIGN(size);data=kmalloc_node_track_caller(size+sizeof(structskb_shared_info),gfp_mask,node);if(!data)gotonodata;memset(skb,0,offsetof(structsk_buff,truesize));+skb->emergency=emergency;skb->truesize=size+sizeof(structsk_buff);atomic_set(&skb->users,1);skb->head=data;
@@ -196,6 +196,120 @@ __u32 sysctl_rmem_default __read_mostly /* Maximal space eaten by iovec or ancilliary data plus some space */intsysctl_optmem_max__read_mostly=sizeof(unsignedlong)*(2*UIO_MAXIOV+512);+staticDEFINE_SPINLOCK(memalloc_lock);+staticintrx_net_reserve;++atomic_tvmio_socks;+atomic_temergency_rx_skbs;++staticintipfrag_threshold;++#define ipfrag_mtu() (1500) /* XXX: should be smallest mtu system wide */+#define ipfrag_skbs() (ipfrag_threshold / ipfrag_mtu())+#define ipfrag_pages() (ipfrag_threshold / (ipfrag_mtu() * (PAGE_SIZE / ipfrag_mtu())))++staticintiprt_pages;++/*+*isthereroomforanotheremergencyskb.+*/+intsk_emergency_skb_get(void)+{+intnr=atomic_add_return(1,&emergency_rx_skbs);+intthresh=(3*ipfrag_skbs())/2;+if(nr<thresh)+return1;++atomic_dec(&emergency_rx_skbs);+return0;+}++/**+*sk_adjust_memalloc-adjusttheglobalmemallocreserveforcriticalRX+*@socks:numberofnew%SOCK_VMIOsockets+*@tx_resserve_pages:numberofpagesto(un)reserveforTX+*+*Thisfunctionadjuststhememallocreservebasedonsystemdemand.+*TheRXreserveisalimit,andonlyaddedonce,notforeachsocket.+*+*NOTE:+*@tx_reserve_pagesisanupper-boundofmemoryusedforTXhence+*weneednotaccountthepageslikewedoforRXpages.+*/+voidsk_adjust_memalloc(intsocks,inttx_reserve_pages)+{+unsignedlongflags;+intreserve=tx_reserve_pages;+intnr_socks;++spin_lock_irqsave(&memalloc_lock,flags);+nr_socks=atomic_add_return(socks,&vmio_socks);+BUG_ON(nr_socks<0);++if(nr_socks){+intrx_pages=2*ipfrag_pages()+iprt_pages;+reserve+=rx_pages-rx_net_reserve;+rx_net_reserve=rx_pages;+}else{+reserve-=rx_net_reserve;+rx_net_reserve=0;+}++if(reserve)+adjust_memalloc_reserve(reserve);+spin_unlock_irqrestore(&memalloc_lock,flags);+}+EXPORT_SYMBOL_GPL(sk_adjust_memalloc);++/*+*tinyhelperfunctiontotrackthetotalipfragmentmemory+*neededbecauseofmodularipv6+*/+voidipfrag_reserve_memory(intfrags)+{+ipfrag_threshold+=frags;+sk_adjust_memalloc(0,0);+}+EXPORT_SYMBOL_GPL(ipfrag_reserve_memory);++voidiprt_reserve_memory(intpages)+{+iprt_pages+=pages;+sk_adjust_memalloc(0,0);+}+EXPORT_SYMBOL_GPL(iprt_reserve_memory);++/**+*sk_set_vmio-sets%SOCK_VMIO+*@sk:sockettosetiton+*+*Set%SOCK_VMIOonasocketandincreasethememallocreserve+*accordingly.+*/+intsk_set_vmio(structsock*sk)+{+intset=sock_flag(sk,SOCK_VMIO);+if(!set){+sk_adjust_memalloc(1,0);+sock_set_flag(sk,SOCK_VMIO);+sk->sk_allocation|=__GFP_EMERGENCY;+}+return!set;+}+EXPORT_SYMBOL_GPL(sk_set_vmio);++intsk_clear_vmio(structsock*sk)+{+intset=sock_flag(sk,SOCK_VMIO);+if(set){+sk_adjust_memalloc(-1,0);+sock_reset_flag(sk,SOCK_VMIO);+sk->sk_allocation&=~__GFP_EMERGENCY;+}+returnset;+}+EXPORT_SYMBOL_GPL(sk_clear_vmio);+staticintsock_set_timeout(long*timeo_p,char__user*optval,intoptlen){structtimevaltv;
@@ -239,6 +353,12 @@ int sock_queue_rcv_skb(struct sock *sk, interr=0;intskb_len;+if(unlikely(skb->emergency)){+if(!sk_has_vmio(sk)){+err=-ENOMEM;+gotoout;+}+}else/* Cast skb->rcvbuf to unsigned... It's pointless, but reducesnumberofwarningswhencompilingwith-W--ANK*/
@@ -1340,6 +1340,9 @@ int ip_mr_input(struct sk_buff *skb)structmfc_cache*cache;intlocal=((structrtable*)skb->dst)->rt_flags&RTCF_LOCAL;+if(unlikely(skb->emergency))+gotodrop;+/* Packet is looped back after forward, it should not beforwardedsecondtime,butstillcanbedeliveredlocally.*/
@@ -1411,6 +1414,7 @@ int ip_mr_input(struct sk_buff *skb)dont_forward:if(local)returnip_local_deliver(skb);+drop:kfree_skb(skb);return0;}
@@ -1604,6 +1604,22 @@ csum_err:gotodiscard;}+staticinttcp_v4_backlog_rcv(structsock*sk,structsk_buff*skb)+{+intret;+unsignedlongpflags=current->flags;+if(unlikely(skb->emergency)){+BUG_ON(!sk_has_vmio(sk));/* we dropped those before queueing */+if(!(pflags&PF_MEMALLOC))+current->flags|=PF_MEMALLOC;+}++ret=tcp_v4_do_rcv(sk,skb);++current->flags=pflags;+returnret;+}+/**Fromtcp_input.c*/
@@ -1654,6 +1670,15 @@ int tcp_v4_rcv(struct sk_buff *skb)if(!sk)gotono_tcp_socket;+if(unlikely(skb->emergency)){+if(!sk_has_vmio(sk))+gotodiscard_and_relse;+/*+decreasewindowsize..+tcp_enter_quickack_mode(sk);+*/+}+process:if(sk->sk_state==TCP_TIME_WAIT)gotodo_time_wait;
@@ -2429,7 +2454,7 @@ struct proto tcp_prot = {.getsockopt=tcp_getsockopt,.sendmsg=tcp_sendmsg,.recvmsg=tcp_recvmsg,-.backlog_rcv=tcp_v4_do_rcv,+.backlog_rcv=tcp_v4_backlog_rcv,.hash=tcp_v4_hash,.unhash=tcp_unhash,.get_port=tcp_v4_get_port,
@@ -1678,6 +1678,22 @@ ipv6_pktoptions:return0;}+staticinttcp_v6_backlog_rcv(structsock*sk,structsk_buff*skb)+{+intret;+unsignedlongpflags=current->flags;+if(unlikely(skb->emergency)){+BUG_ON(!sk_has_vmio(sk));/* we dropped those before queueing */+if(!(pflags&PF_MEMALLOC))+current->flags|=PF_MEMALLOC;+}++ret=tcp_v6_do_rcv(sk,skb);++current->flags=pflags;+returnret;+}+staticinttcp_v6_rcv(structsk_buff**pskb){structsk_buff*skb=*pskb;
@@ -1723,6 +1739,15 @@ static int tcp_v6_rcv(struct sk_buff **pif(!sk)gotono_tcp_socket;+if(unlikely(skb->emergency)){+if(!sk_has_vmio(sk))+gotodiscard_and_relse;+/*+decreasewindowsize..+tcp_enter_quickack_mode(sk);+*/+}+process:if(sk->sk_state==TCP_TIME_WAIT)gotodo_time_wait;
@@ -2127,7 +2152,7 @@ struct proto tcpv6_prot = {.getsockopt=tcp_getsockopt,.sendmsg=tcp_sendmsg,.recvmsg=tcp_recvmsg,-.backlog_rcv=tcp_v6_do_rcv,+.backlog_rcv=tcp_v6_backlog_rcv,.hash=tcp_v6_hash,.unhash=tcp_unhash,.get_port=tcp_v6_get_port,
@@ -224,7 +224,8 @@ int sk_stream_mem_schedule(struct sock */* Over hard limit. */if(atomic_read(sk->sk_prot->memory_allocated)>sk->sk_prot->sysctl_mem[2]){sk->sk_prot->enter_memory_pressure();-gotosuppress_allocation;+if(likely(!skb||!skb->emergency))+gotosuppress_allocation;}/* Under pressure. */
On Tue, Jan 16, 2007 at 10:46:06AM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
In order to provide robust networked storage there must be a guarantee
of progress. That is, the storage device must never stall because of (physical)
OOM, because the device itself might be needed to get out of it (reclaim).
quoted hunk
/* Used by processes to "lock" a socket state, so that
Index: linux-2.6-git/net/core/dev.c
===================================================================
@@ -1767,10 +1767,23 @@ int netif_receive_skb(struct sk_buff *skstructnet_device*orig_dev;intret=NET_RX_DROP;__be16type;+unsignedlongpflags=current->flags;++/* Emergency skb are special, they should+*-bedeliveredtoSOCK_VMIOsocketsonly+*-stayawayfromuserspace+*-haveboundedmemoryusage+*+*UsePF_MEMALLOCasapoormansmemorypool-thegroupingkind.+*Thissavesusfrompropagatingtheallocationcontextdowntoall+*allocationsites.+*/+if(unlikely(skb->emergency))+current->flags|=PF_MEMALLOC;
Access to 'current' in netif_receive_skb()???
Why do you want to work with, for example keventd?
quoted hunk
/* if we've gotten here through NAPI, check netpoll */
if (skb->dev->poll && netpoll_rx(skb))
- return NET_RX_DROP;
+ goto out;
if (!skb->tstamp.off_sec)
net_timestamp(skb);
@@ -1781,7 +1794,7 @@ int netif_receive_skb(struct sk_buff *sk orig_dev = skb_bond(skb); if (!orig_dev)- return NET_RX_DROP;+ goto out; __get_cpu_var(netdev_rx_stat).total++;
@@ -1798,6 +1811,8 @@ int netif_receive_skb(struct sk_buff *sk goto ncls; } #endif+ if (unlikely(skb->emergency))+ goto skip_taps; list_for_each_entry_rcu(ptype, &ptype_all, list) { if (!ptype->dev || ptype->dev == skb->dev) {
@@ -1807,6 +1822,7 @@ int netif_receive_skb(struct sk_buff *sk } }+skip_taps:
It is still a 'tap'.
quoted hunk
#ifdef CONFIG_NET_CLS_ACT
if (pt_prev) {
ret = deliver_skb(skb, pt_prev, orig_dev);
@@ -1819,15 +1835,26 @@ int netif_receive_skb(struct sk_buff *sk if (ret == TC_ACT_SHOT || (ret == TC_ACT_STOLEN)) { kfree_skb(skb);- goto out;+ goto unlock; } skb->tc_verd = 0; ncls: #endif+ if (unlikely(skb->emergency))+ switch(skb->protocol) {+ case __constant_htons(ETH_P_ARP):+ case __constant_htons(ETH_P_IP):+ case __constant_htons(ETH_P_IPV6):+ break;
@@ -1842,6 +1869,7 @@ ncls: if (pt_prev) { ret = pt_prev->func(skb, skb->dev, pt_prev, orig_dev); } else {+drop: kfree_skb(skb); /* Jamal, now you will not able to escape explaining * me how you were going to use this. :-)
@@ -142,28 +142,34 @@ EXPORT_SYMBOL(skb_truesize_bug);*%GFP_ATOMIC.*/structsk_buff*__alloc_skb(unsignedintsize,gfp_tgfp_mask,-intfclone,intnode)+intflags,intnode){structkmem_cache*cache;structskb_shared_info*shinfo;structsk_buff*skb;u8*data;+intemergency=0;-cache=fclone?skbuff_fclone_cache:skbuff_head_cache;+size=SKB_DATA_ALIGN(size);+cache=(flags&SKB_ALLOC_FCLONE)+?skbuff_fclone_cache:skbuff_head_cache;+if(flags&SKB_ALLOC_RX)+gfp_mask|=__GFP_NOMEMALLOC|__GFP_NOWARN;+retry_alloc:/* Get the HEAD */skb=kmem_cache_alloc_node(cache,gfp_mask&~__GFP_DMA,node);if(!skb)-gotoout;+gotonoskb;/* Get the DATA. Size must match skb_add_mtu(). */-size=SKB_DATA_ALIGN(size);data=kmalloc_node_track_caller(size+sizeof(structskb_shared_info),gfp_mask,node);if(!data)gotonodata;memset(skb,0,offsetof(structsk_buff,truesize));+skb->emergency=emergency;skb->truesize=size+sizeof(structsk_buff);atomic_set(&skb->users,1);skb->head=data;
@@ -196,6 +196,120 @@ __u32 sysctl_rmem_default __read_mostly /* Maximal space eaten by iovec or ancilliary data plus some space */intsysctl_optmem_max__read_mostly=sizeof(unsignedlong)*(2*UIO_MAXIOV+512);+staticDEFINE_SPINLOCK(memalloc_lock);+staticintrx_net_reserve;++atomic_tvmio_socks;+atomic_temergency_rx_skbs;++staticintipfrag_threshold;++#define ipfrag_mtu() (1500) /* XXX: should be smallest mtu system wide */+#define ipfrag_skbs() (ipfrag_threshold / ipfrag_mtu())+#define ipfrag_pages() (ipfrag_threshold / (ipfrag_mtu() * (PAGE_SIZE / ipfrag_mtu())))++staticintiprt_pages;++/*+*isthereroomforanotheremergencyskb.+*/+intsk_emergency_skb_get(void)+{+intnr=atomic_add_return(1,&emergency_rx_skbs);+intthresh=(3*ipfrag_skbs())/2;+if(nr<thresh)+return1;++atomic_dec(&emergency_rx_skbs);+return0;+}++/**+*sk_adjust_memalloc-adjusttheglobalmemallocreserveforcriticalRX+*@socks:numberofnew%SOCK_VMIOsockets+*@tx_resserve_pages:numberofpagesto(un)reserveforTX+*+*Thisfunctionadjuststhememallocreservebasedonsystemdemand.+*TheRXreserveisalimit,andonlyaddedonce,notforeachsocket.+*+*NOTE:+*@tx_reserve_pagesisanupper-boundofmemoryusedforTXhence+*weneednotaccountthepageslikewedoforRXpages.+*/+voidsk_adjust_memalloc(intsocks,inttx_reserve_pages)+{+unsignedlongflags;+intreserve=tx_reserve_pages;+intnr_socks;++spin_lock_irqsave(&memalloc_lock,flags);+nr_socks=atomic_add_return(socks,&vmio_socks);+BUG_ON(nr_socks<0);++if(nr_socks){+intrx_pages=2*ipfrag_pages()+iprt_pages;+reserve+=rx_pages-rx_net_reserve;+rx_net_reserve=rx_pages;+}else{+reserve-=rx_net_reserve;+rx_net_reserve=0;+}++if(reserve)+adjust_memalloc_reserve(reserve);+spin_unlock_irqrestore(&memalloc_lock,flags);+}+EXPORT_SYMBOL_GPL(sk_adjust_memalloc);++/*+*tinyhelperfunctiontotrackthetotalipfragmentmemory+*neededbecauseofmodularipv6+*/+voidipfrag_reserve_memory(intfrags)+{+ipfrag_threshold+=frags;+sk_adjust_memalloc(0,0);+}+EXPORT_SYMBOL_GPL(ipfrag_reserve_memory);++voidiprt_reserve_memory(intpages)+{+iprt_pages+=pages;+sk_adjust_memalloc(0,0);+}+EXPORT_SYMBOL_GPL(iprt_reserve_memory);++/**+*sk_set_vmio-sets%SOCK_VMIO+*@sk:sockettosetiton+*+*Set%SOCK_VMIOonasocketandincreasethememallocreserve+*accordingly.+*/+intsk_set_vmio(structsock*sk)+{+intset=sock_flag(sk,SOCK_VMIO);+if(!set){+sk_adjust_memalloc(1,0);+sock_set_flag(sk,SOCK_VMIO);+sk->sk_allocation|=__GFP_EMERGENCY;+}+return!set;+}+EXPORT_SYMBOL_GPL(sk_set_vmio);++intsk_clear_vmio(structsock*sk)+{+intset=sock_flag(sk,SOCK_VMIO);+if(set){+sk_adjust_memalloc(-1,0);+sock_reset_flag(sk,SOCK_VMIO);+sk->sk_allocation&=~__GFP_EMERGENCY;+}+returnset;+}+EXPORT_SYMBOL_GPL(sk_clear_vmio);+staticintsock_set_timeout(long*timeo_p,char__user*optval,intoptlen){structtimevaltv;
@@ -239,6 +353,12 @@ int sock_queue_rcv_skb(struct sock *sk, interr=0;intskb_len;+if(unlikely(skb->emergency)){+if(!sk_has_vmio(sk)){+err=-ENOMEM;+gotoout;+}+}else/* Cast skb->rcvbuf to unsigned... It's pointless, but reducesnumberofwarningswhencompilingwith-W--ANK*/
@@ -1340,6 +1340,9 @@ int ip_mr_input(struct sk_buff *skb)structmfc_cache*cache;intlocal=((structrtable*)skb->dst)->rt_flags&RTCF_LOCAL;+if(unlikely(skb->emergency))+gotodrop;+/* Packet is looped back after forward, it should not beforwardedsecondtime,butstillcanbedeliveredlocally.*/
@@ -1411,6 +1414,7 @@ int ip_mr_input(struct sk_buff *skb)dont_forward:if(local)returnip_local_deliver(skb);+drop:kfree_skb(skb);return0;}
@@ -1604,6 +1604,22 @@ csum_err:gotodiscard;}+staticinttcp_v4_backlog_rcv(structsock*sk,structsk_buff*skb)+{+intret;+unsignedlongpflags=current->flags;+if(unlikely(skb->emergency)){+BUG_ON(!sk_has_vmio(sk));/* we dropped those before queueing */+if(!(pflags&PF_MEMALLOC))+current->flags|=PF_MEMALLOC;+}++ret=tcp_v4_do_rcv(sk,skb);++current->flags=pflags;+returnret;
Why don't you want to just setup PF_MEMALLOC for the socket and all
related processes?
quoted hunk
+}
+
/*
* From tcp_input.c
*/
@@ -1654,6 +1670,15 @@ int tcp_v4_rcv(struct sk_buff *skb) if (!sk) goto no_tcp_socket;+ if (unlikely(skb->emergency)) {+ if (!sk_has_vmio(sk))+ goto discard_and_relse;+ /*+ decrease window size..+ tcp_enter_quickack_mode(sk);+ */
How does this decrease window size?
Maybe ack scheduling would be better handled by inet_csk_schedule_ack()
or just directly send an ack, which in turn requires allocation, which
can be bound to this received frame processing...
--
Evgeniy Polyakov
From: Peter Zijlstra <hidden> Date: 2007-01-16 13:50:28
On Tue, 2007-01-16 at 16:25 +0300, Evgeniy Polyakov wrote:
On Tue, Jan 16, 2007 at 10:46:06AM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
@@ -1767,10 +1767,23 @@ int netif_receive_skb(struct sk_buff *sk struct net_device *orig_dev; int ret = NET_RX_DROP; __be16 type;+ unsigned long pflags = current->flags;++ /* Emergency skb are special, they should+ * - be delivered to SOCK_VMIO sockets only+ * - stay away from userspace+ * - have bounded memory usage+ *+ * Use PF_MEMALLOC as a poor mans memory pool - the grouping kind.+ * This saves us from propagating the allocation context down to all+ * allocation sites.+ */+ if (unlikely(skb->emergency))+ current->flags |= PF_MEMALLOC;
Access to 'current' in netif_receive_skb()???
Why do you want to work with, for example keventd?
Can this run in keventd?
I thought this was softirq context and thus this would either run in a
borrowed context or in ksoftirqd. See patch 3/9.
quoted
@@ -1798,6 +1811,8 @@ int netif_receive_skb(struct sk_buff *sk goto ncls; } #endif+ if (unlikely(skb->emergency))+ goto skip_taps; list_for_each_entry_rcu(ptype, &ptype_all, list) { if (!ptype->dev || ptype->dev == skb->dev) {
@@ -1807,6 +1822,7 @@ int netif_receive_skb(struct sk_buff *sk } }+skip_taps:
It is still a 'tap'.
Not sure what you are saying, I thought this should stop delivery of
skbs to taps?
quoted
#ifdef CONFIG_NET_CLS_ACT
if (pt_prev) {
ret = deliver_skb(skb, pt_prev, orig_dev);
@@ -1819,15 +1835,26 @@ int netif_receive_skb(struct sk_buff *sk if (ret == TC_ACT_SHOT || (ret == TC_ACT_STOLEN)) { kfree_skb(skb);- goto out;+ goto unlock; } skb->tc_verd = 0; ncls: #endif+ if (unlikely(skb->emergency))+ switch(skb->protocol) {+ case __constant_htons(ETH_P_ARP):+ case __constant_htons(ETH_P_IP):+ case __constant_htons(ETH_P_IPV6):+ break;
@@ -1604,6 +1604,22 @@ csum_err:gotodiscard;}+staticinttcp_v4_backlog_rcv(structsock*sk,structsk_buff*skb)+{+intret;+unsignedlongpflags=current->flags;+if(unlikely(skb->emergency)){+BUG_ON(!sk_has_vmio(sk));/* we dropped those before queueing */+if(!(pflags&PF_MEMALLOC))+current->flags|=PF_MEMALLOC;+}++ret=tcp_v4_do_rcv(sk,skb);++current->flags=pflags;+returnret;
Why don't you want to just setup PF_MEMALLOC for the socket and all
related processes?
I'm not understanding what you're saying here.
I want grant the processing of skb->emergency packets access to the
memory reserves.
How would I set PF_MEMALLOC on a socket, its a process flag? And which
related processes?
quoted
+}
+
/*
* From tcp_input.c
*/
@@ -1654,6 +1670,15 @@ int tcp_v4_rcv(struct sk_buff *skb) if (!sk) goto no_tcp_socket;+ if (unlikely(skb->emergency)) {+ if (!sk_has_vmio(sk))+ goto discard_and_relse;+ /*+ decrease window size..+ tcp_enter_quickack_mode(sk);+ */
How does this decrease window size?
Maybe ack scheduling would be better handled by inet_csk_schedule_ack()
or just directly send an ack, which in turn requires allocation, which
can be bound to this received frame processing...
It doesn't, I thought that it might be a good idea doing that, but never
got around to actually figuring out how to do it.
On Tue, Jan 16, 2007 at 02:47:54PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
quoted
+ if (unlikely(skb->emergency))
+ current->flags |= PF_MEMALLOC;
Access to 'current' in netif_receive_skb()???
Why do you want to work with, for example keventd?
Can this run in keventd?
Initial netchannel implementation by Kelly Daly (IBM) worked in keventd
(or dedicated kernel thread, I do not recall).
I thought this was softirq context and thus this would either run in a
borrowed context or in ksoftirqd. See patch 3/9.
And how are you going to access 'current' in softirq?
netif_receive_skb() can also be called from a lot of other places
including keventd and/or different context - it is permitted to call it
everywhere to process packet.
I meant that you break the rule accessing 'current' in that context.
quoted
quoted
@@ -1798,6 +1811,8 @@ int netif_receive_skb(struct sk_buff *sk goto ncls; } #endif+ if (unlikely(skb->emergency))+ goto skip_taps; list_for_each_entry_rcu(ptype, &ptype_all, list) { if (!ptype->dev || ptype->dev == skb->dev) {
@@ -1807,6 +1822,7 @@ int netif_receive_skb(struct sk_buff *sk } }+skip_taps:
It is still a 'tap'.
Not sure what you are saying, I thought this should stop delivery of
skbs to taps?
Ingres filter can do whatever it wants with skb at that point, likely
you want to skip that hunk too.
quoted
quoted
#ifdef CONFIG_NET_CLS_ACT
if (pt_prev) {
ret = deliver_skb(skb, pt_prev, orig_dev);
@@ -1819,15 +1835,26 @@ int netif_receive_skb(struct sk_buff *sk if (ret == TC_ACT_SHOT || (ret == TC_ACT_STOLEN)) { kfree_skb(skb);- goto out;+ goto unlock; } skb->tc_verd = 0; ncls: #endif+ if (unlikely(skb->emergency))+ switch(skb->protocol) {+ case __constant_htons(ETH_P_ARP):+ case __constant_htons(ETH_P_IP):+ case __constant_htons(ETH_P_IPV6):+ break;
@@ -1604,6 +1604,22 @@ csum_err:gotodiscard;}+staticinttcp_v4_backlog_rcv(structsock*sk,structsk_buff*skb)+{+intret;+unsignedlongpflags=current->flags;+if(unlikely(skb->emergency)){+BUG_ON(!sk_has_vmio(sk));/* we dropped those before queueing */+if(!(pflags&PF_MEMALLOC))+current->flags|=PF_MEMALLOC;+}++ret=tcp_v4_do_rcv(sk,skb);++current->flags=pflags;+returnret;
Why don't you want to just setup PF_MEMALLOC for the socket and all
related processes?
I'm not understanding what you're saying here.
I want grant the processing of skb->emergency packets access to the
memory reserves.
How would I set PF_MEMALLOC on a socket, its a process flag? And which
related processes?
You use special flag for sockets to mark them as capable of
'reserve-eating', too many flags are a bit confusing.
I meant that you can just mark process which created such socket as
PF_MEMALLOC, and clone that flag on forks and other relatest calls without
all that checks for 'current' in different places.
quoted
quoted
+}
+
/*
* From tcp_input.c
*/
@@ -1654,6 +1670,15 @@ int tcp_v4_rcv(struct sk_buff *skb) if (!sk) goto no_tcp_socket;+ if (unlikely(skb->emergency)) {+ if (!sk_has_vmio(sk))+ goto discard_and_relse;+ /*+ decrease window size..+ tcp_enter_quickack_mode(sk);+ */
How does this decrease window size?
Maybe ack scheduling would be better handled by inet_csk_schedule_ack()
or just directly send an ack, which in turn requires allocation, which
can be bound to this received frame processing...
It doesn't, I thought that it might be a good idea doing that, but never
got around to actually figuring out how to do it.
From: Peter Zijlstra <hidden> Date: 2007-01-16 16:11:51
On Tue, 2007-01-16 at 18:33 +0300, Evgeniy Polyakov wrote:
On Tue, Jan 16, 2007 at 02:47:54PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
quoted
quoted
+ if (unlikely(skb->emergency))
+ current->flags |= PF_MEMALLOC;
Access to 'current' in netif_receive_skb()???
Why do you want to work with, for example keventd?
Can this run in keventd?
Initial netchannel implementation by Kelly Daly (IBM) worked in keventd
(or dedicated kernel thread, I do not recall).
quoted
I thought this was softirq context and thus this would either run in a
borrowed context or in ksoftirqd. See patch 3/9.
And how are you going to access 'current' in softirq?
netif_receive_skb() can also be called from a lot of other places
including keventd and/or different context - it is permitted to call it
everywhere to process packet.
I meant that you break the rule accessing 'current' in that context.
Yeah, I know, but as long as we're not actually in hard irq context
current does point to the task_struct in charge of current execution and
as long as we restore whatever was in the flags field before we started
poking, nothing can go wrong.
So, yes this is unconventional, but it does work as expected.
As for breaking, 3/9 makes it legal.
quoted
quoted
quoted
@@ -1798,6 +1811,8 @@ int netif_receive_skb(struct sk_buff *sk goto ncls; } #endif+ if (unlikely(skb->emergency))+ goto skip_taps; list_for_each_entry_rcu(ptype, &ptype_all, list) { if (!ptype->dev || ptype->dev == skb->dev) {
@@ -1807,6 +1822,7 @@ int netif_receive_skb(struct sk_buff *sk } }+skip_taps:
It is still a 'tap'.
Not sure what you are saying, I thought this should stop delivery of
skbs to taps?
Ingres filter can do whatever it wants with skb at that point, likely
you want to skip that hunk too.
Will look into Ingres filters, thanks for the pointer.
quoted
quoted
Why don't you want to just setup PF_MEMALLOC for the socket and all
related processes?
I'm not understanding what you're saying here.
I want grant the processing of skb->emergency packets access to the
memory reserves.
How would I set PF_MEMALLOC on a socket, its a process flag? And which
related processes?
You use special flag for sockets to mark them as capable of
'reserve-eating', too many flags are a bit confusing.
Right, and I use PF_MEMALLOC to implement that reserve-eating. There
must be a link between SOCK_VMIO and all allocations associated with
that socket.
I meant that you can just mark process which created such socket as
PF_MEMALLOC, and clone that flag on forks and other relatest calls without
all that checks for 'current' in different places.
Ah, thats the wrong level to think here, these processes never reach
user-space - nor should these sockets.
Also, I only want the processing of the actual network packet to be able
to eat the reserves, not any other thing that might happen in that
context.
And since network processing is mostly done in softirq context I must
mark these sections like I did.
How does this decrease window size?
Maybe ack scheduling would be better handled by inet_csk_schedule_ack()
or just directly send an ack, which in turn requires allocation, which
can be bound to this received frame processing...
It doesn't, I thought that it might be a good idea doing that, but never
got around to actually figuring out how to do it.
On Tue, Jan 16, 2007 at 05:08:15PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
On Tue, 2007-01-16 at 18:33 +0300, Evgeniy Polyakov wrote:
quoted
On Tue, Jan 16, 2007 at 02:47:54PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
quoted
quoted
+ if (unlikely(skb->emergency))
+ current->flags |= PF_MEMALLOC;
Access to 'current' in netif_receive_skb()???
Why do you want to work with, for example keventd?
Can this run in keventd?
Initial netchannel implementation by Kelly Daly (IBM) worked in keventd
(or dedicated kernel thread, I do not recall).
quoted
I thought this was softirq context and thus this would either run in a
borrowed context or in ksoftirqd. See patch 3/9.
And how are you going to access 'current' in softirq?
netif_receive_skb() can also be called from a lot of other places
including keventd and/or different context - it is permitted to call it
everywhere to process packet.
I meant that you break the rule accessing 'current' in that context.
Yeah, I know, but as long as we're not actually in hard irq context
current does point to the task_struct in charge of current execution and
as long as we restore whatever was in the flags field before we started
poking, nothing can go wrong.
So, yes this is unconventional, but it does work as expected.
As for breaking, 3/9 makes it legal.
You operate with 'current' in different contexts without any locks which
looks racy and even is not allowed. What will be 'current' for
netif_rx() case, which schedules softirq from hard irq context -
ksoftirqd, why do you want to set its flags?
quoted
I meant that you can just mark process which created such socket as
PF_MEMALLOC, and clone that flag on forks and other relatest calls without
all that checks for 'current' in different places.
Ah, thats the wrong level to think here, these processes never reach
user-space - nor should these sockets.
You limit this just to send an ack?
What about 'level-7' ack as you described in introduction?
Also, I only want the processing of the actual network packet to be able
to eat the reserves, not any other thing that might happen in that
context.
And since network processing is mostly done in softirq context I must
mark these sections like I did.
You artificially limit system to just add a reserve to generate one ack.
For that purpose you do not need to have all those flags - just reseve
some data in network core and use it when system is in OOM (or reclaim)
for critical data pathes.
How does this decrease window size?
Maybe ack scheduling would be better handled by inet_csk_schedule_ack()
or just directly send an ack, which in turn requires allocation, which
can be bound to this received frame processing...
It doesn't, I thought that it might be a good idea doing that, but never
got around to actually figuring out how to do it.
tcp_send_ack()?
does that shrink the window automagically?
Yes, it updates window, but having ack generated in that place is
actually very wrong. In that place system has not processed incoming
packet yet, so it can not generate correct ACK for received frame at
all. And it seems that the only purpose of the whole patchset is to
generate that poor ack - reseve 2007 ack packets (MAX_TCP_HEADER)
in system startup and reuse them when you are under memory pressure.
--
Evgeniy Polyakov
From: Peter Zijlstra <hidden> Date: 2007-01-17 09:10:21
On Wed, 2007-01-17 at 07:54 +0300, Evgeniy Polyakov wrote:
On Tue, Jan 16, 2007 at 05:08:15PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
On Tue, 2007-01-16 at 18:33 +0300, Evgeniy Polyakov wrote:
quoted
On Tue, Jan 16, 2007 at 02:47:54PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
quoted
quoted
+ if (unlikely(skb->emergency))
+ current->flags |= PF_MEMALLOC;
Access to 'current' in netif_receive_skb()???
Why do you want to work with, for example keventd?
Can this run in keventd?
Initial netchannel implementation by Kelly Daly (IBM) worked in keventd
(or dedicated kernel thread, I do not recall).
quoted
I thought this was softirq context and thus this would either run in a
borrowed context or in ksoftirqd. See patch 3/9.
And how are you going to access 'current' in softirq?
netif_receive_skb() can also be called from a lot of other places
including keventd and/or different context - it is permitted to call it
everywhere to process packet.
I meant that you break the rule accessing 'current' in that context.
Yeah, I know, but as long as we're not actually in hard irq context
current does point to the task_struct in charge of current execution and
as long as we restore whatever was in the flags field before we started
poking, nothing can go wrong.
So, yes this is unconventional, but it does work as expected.
As for breaking, 3/9 makes it legal.
You operate with 'current' in different contexts without any locks which
looks racy and even is not allowed. What will be 'current' for
netif_rx() case, which schedules softirq from hard irq context -
ksoftirqd, why do you want to set its flags?
I don't touch current in hardirq context, do I (if I did, that is indeed
a mistake)?
In all other contexts, current is valid.
quoted
quoted
I meant that you can just mark process which created such socket as
PF_MEMALLOC, and clone that flag on forks and other relatest calls without
all that checks for 'current' in different places.
Ah, thats the wrong level to think here, these processes never reach
user-space - nor should these sockets.
You limit this just to send an ack?
What about 'level-7' ack as you described in introduction?
Take NFS, it does full data traffic in kernel.
quoted
Also, I only want the processing of the actual network packet to be able
to eat the reserves, not any other thing that might happen in that
context.
And since network processing is mostly done in softirq context I must
mark these sections like I did.
You artificially limit system to just add a reserve to generate one ack.
For that purpose you do not need to have all those flags - just reseve
some data in network core and use it when system is in OOM (or reclaim)
for critical data pathes.
How would that end up being different, I would have to replace all
allocations done in the full network processing path.
This seems a much less invasive method, all the (allocation) code can
stay the way it is and use the normal allocation functions.
How does this decrease window size?
Maybe ack scheduling would be better handled by inet_csk_schedule_ack()
or just directly send an ack, which in turn requires allocation, which
can be bound to this received frame processing...
It doesn't, I thought that it might be a good idea doing that, but never
got around to actually figuring out how to do it.
tcp_send_ack()?
does that shrink the window automagically?
Yes, it updates window, but having ack generated in that place is
actually very wrong. In that place system has not processed incoming
packet yet, so it can not generate correct ACK for received frame at
all. And it seems that the only purpose of the whole patchset is to
generate that poor ack - reseve 2007 ack packets (MAX_TCP_HEADER)
in system startup and reuse them when you are under memory pressure.
Right, I suspected something like that; hence I wanted to just shrink
the window. Anyway, this is not a very important issue.
From: Pavel Machek <hidden> Date: 2007-01-17 09:12:26
Hi!
These patches implement the basic infrastructure to allow swap over networked
storage.
The basic idea is to reserve some memory up front to use when regular memory
runs out.
To bound network behaviour we accept only a limited number of concurrent
packets and drop those packets that are not aimed at the connection(s) servicing
the VM. Also all network paths that interact with userspace are to be avoided -
e.g. taps and NF_QUEUE.
PF_MEMALLOC is set when processing emergency skbs. This makes sense in that we
are indeed working on behalf of the swapper/VM. This allows us to use the
regular memory allocators for processing but requires that said processing have
bounded memory usage and has that accounted in the reserve.
From: Peter Zijlstra <hidden> Date: 2007-01-17 09:23:27
On Wed, 2007-01-17 at 10:12 +0100, Pavel Machek wrote:
Hi!
quoted
These patches implement the basic infrastructure to allow swap over networked
storage.
The basic idea is to reserve some memory up front to use when regular memory
runs out.
To bound network behaviour we accept only a limited number of concurrent
packets and drop those packets that are not aimed at the connection(s) servicing
the VM. Also all network paths that interact with userspace are to be avoided -
e.g. taps and NF_QUEUE.
PF_MEMALLOC is set when processing emergency skbs. This makes sense in that we
are indeed working on behalf of the swapper/VM. This allows us to use the
regular memory allocators for processing but requires that said processing have
bounded memory usage and has that accounted in the reserve.
How does it work with ARP, for example? You still need to reply to ARP
if you want to keep your ethernet connections.
ETH_P_ARP is fully processed (under PF_MEMALLOC).
ETH_P_IP{,V6} starts to drop packets not for selected sockets
(SOCK_VMIO) and processes the rest (under PF_MEMALLOC) with limitations;
the packet may never depend on user-space to complete processing.
On Wed, Jan 17, 2007 at 10:07:28AM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
You operate with 'current' in different contexts without any locks which
looks racy and even is not allowed. What will be 'current' for
netif_rx() case, which schedules softirq from hard irq context -
ksoftirqd, why do you want to set its flags?
I don't touch current in hardirq context, do I (if I did, that is indeed
a mistake)?
In all other contexts, current is valid.
Well, if you think that setting PF_MEMALLOC flag for keventd and
ksoftirqd is valid, then probably yes...
quoted
quoted
quoted
I meant that you can just mark process which created such socket as
PF_MEMALLOC, and clone that flag on forks and other relatest calls without
all that checks for 'current' in different places.
Ah, thats the wrong level to think here, these processes never reach
user-space - nor should these sockets.
You limit this just to send an ack?
What about 'level-7' ack as you described in introduction?
Take NFS, it does full data traffic in kernel.
NFS case is exactly the situation, when you only need to generate an ACK.
quoted
quoted
Also, I only want the processing of the actual network packet to be able
to eat the reserves, not any other thing that might happen in that
context.
And since network processing is mostly done in softirq context I must
mark these sections like I did.
You artificially limit system to just add a reserve to generate one ack.
For that purpose you do not need to have all those flags - just reseve
some data in network core and use it when system is in OOM (or reclaim)
for critical data pathes.
How would that end up being different, I would have to replace all
allocations done in the full network processing path.
This seems a much less invasive method, all the (allocation) code can
stay the way it is and use the normal allocation functions.
Ack is only generated in one place in TCP.
And acutally we are starting to talk about different approach - having
separated allocator for network, which will be turned on on OOM (reclaim
or at any other time). If you do not mind, I would likw to refresh a
discussion about network tree allocator, which utilizes own pool of
pages, performs self-defragmentation of the memeory, is very SMP
friendly in that regard that it is per-cpu like slab and never free
objects on different CPUs, so they always stay in the same cache.
Among other goodies it allows to have full sending/receiving zero-copy.
Here is a link:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=nta
How does this decrease window size?
Maybe ack scheduling would be better handled by inet_csk_schedule_ack()
or just directly send an ack, which in turn requires allocation, which
can be bound to this received frame processing...
It doesn't, I thought that it might be a good idea doing that, but never
got around to actually figuring out how to do it.
tcp_send_ack()?
does that shrink the window automagically?
Yes, it updates window, but having ack generated in that place is
actually very wrong. In that place system has not processed incoming
packet yet, so it can not generate correct ACK for received frame at
all. And it seems that the only purpose of the whole patchset is to
generate that poor ack - reseve 2007 ack packets (MAX_TCP_HEADER)
in system startup and reuse them when you are under memory pressure.
Right, I suspected something like that; hence I wanted to just shrink
the window. Anyway, this is not a very important issue.
tcp_enter_quickack_mode() does not update window, it allows to send ack
immediately after packet has been processed, window can be changed in
any way TCP state machine and congestion control want.
--
Evgeniy Polyakov
From: Peter Zijlstra <hidden> Date: 2007-01-18 12:20:37
On Thu, 2007-01-18 at 13:41 +0300, Evgeniy Polyakov wrote:
quoted
quoted
What about 'level-7' ack as you described in introduction?
Take NFS, it does full data traffic in kernel.
NFS case is exactly the situation, when you only need to generate an ACK.
No it is not, it needs the full RPC response.
quoted
quoted
You artificially limit system to just add a reserve to generate one ack.
For that purpose you do not need to have all those flags - just reseve
some data in network core and use it when system is in OOM (or reclaim)
for critical data pathes.
How would that end up being different, I would have to replace all
allocations done in the full network processing path.
This seems a much less invasive method, all the (allocation) code can
stay the way it is and use the normal allocation functions.
And acutally we are starting to talk about different approach - having
separated allocator for network, which will be turned on on OOM (reclaim
or at any other time).
I think we might be, I'm more talking about requirements on the
allocator, while you seem to talk about implementations.
Replacing the allocator, or splitting it in two based on a condition are
all fine as long as they observe the requirements.
The requirement I add is that there is a reserve nobody touches unless
given express permission.
You could implement this by modifying each reachable allocator call site
and stick a branch in and use an alternate allocator when the normal
route fails and we do have permission; much like:
foo = kmalloc(size, gfp_mask);
+ if (!foo && special)
+ foo = my_alloc(size)
And earlier versions of this work did something like that. But it
litters the code quite badly and its quite easy to miss spots. There can
be quite a few allocations in processing network data.
Hence my work on integrating this into the regular memory allocators.
FYI; 'special' evaluates to something like:
!(gfp_mask & __GFP_NOMEMALLOC) &&
((gfp_mask & __GFP_EMERGENCY) ||
(!in_irq() && (current->flags & PF_MEMALLOC)))
If you do not mind, I would likw to refresh a
discussion about network tree allocator,
which utilizes own pool of
pages,
very high order pages, no?
This means that you have to either allocate at boot time and cannot
resize/add pools; which means you waste all that memory if the network
load never comes near using the reserved amount.
Or, you get into all the same trouble the hugepages folks are trying so
very hard to solve.
performs self-defragmentation of the memeory,
Does it move memory about?
All it does is try to avoid fragmentation by policy - a problem
impossible to solve in general; but can achieve good results in view of
practical limitations on program behaviour.
Does your policy work for the given workload? we'll see.
Also, on what level, each level has both internal and external
fragmentation. I can argue that having large immovable objects in memory
adds to the fragmentation issues on the page-allocator level.
is very SMP
friendly in that regard that it is per-cpu like slab and never free
objects on different CPUs, so they always stay in the same cache.
This makes it very hard to guarantee a reserve limit. (Not impossible,
just more difficult)
Among other goodies it allows to have full sending/receiving zero-copy.
That won't ever work unless you have page aligned objects, otherwise you
cannot map them into user-space. Which seems to be at odds with your
tight packing/reduce internal fragmentation goals.
Zero-copy entails mapping the page the hardware writes the packet in
into user-space, right?
Since its impossible to predict to whoem the next packet is addressed
the packets must be written (by hardware) to different pages.
On Thu, Jan 18, 2007 at 01:18:44PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
quoted
How would that end up being different, I would have to replace all
allocations done in the full network processing path.
This seems a much less invasive method, all the (allocation) code can
stay the way it is and use the normal allocation functions.
quoted
And acutally we are starting to talk about different approach - having
separated allocator for network, which will be turned on on OOM (reclaim
or at any other time).
I think we might be, I'm more talking about requirements on the
allocator, while you seem to talk about implementations.
Replacing the allocator, or splitting it in two based on a condition are
all fine as long as they observe the requirements.
The requirement I add is that there is a reserve nobody touches unless
given express permission.
You could implement this by modifying each reachable allocator call site
and stick a branch in and use an alternate allocator when the normal
route fails and we do have permission; much like:
foo = kmalloc(size, gfp_mask);
+ if (!foo && special)
+ foo = my_alloc(size)
Network is special in this regard, since it only has one allocation path
(actually it has one cache for skb, and usual kmalloc, but they are
called from only two functions).
So it would become
ptr = network_alloc();
and network_alloc() would be usual kmalloc or call for own allocator in
case of deadlock.
And earlier versions of this work did something like that. But it
litters the code quite badly and its quite easy to miss spots. There can
be quite a few allocations in processing network data.
Hence my work on integrating this into the regular memory allocators.
FYI; 'special' evaluates to something like:
!(gfp_mask & __GFP_NOMEMALLOC) &&
((gfp_mask & __GFP_EMERGENCY) ||
(!in_irq() && (current->flags & PF_MEMALLOC)))
quoted
If you do not mind, I would likw to refresh a
discussion about network tree allocator,
quoted
which utilizes own pool of
pages,
very high order pages, no?
This means that you have to either allocate at boot time and cannot
resize/add pools; which means you waste all that memory if the network
load never comes near using the reserved amount.
Or, you get into all the same trouble the hugepages folks are trying so
very hard to solve.
It is configurable - by default it takes pool of 32k pages for allocations for
jumbo-frames (e1000 requires such allocations for 9k frames
unfortunately), without jumbo-frame support it works with pool of 0-order
pages, which grows dynamically when needed.
quoted
performs self-defragmentation of the memeory,
Does it move memory about?
It works in a page, not as pages - when neighbour regions are freed,
they are combined into single one with bigger size - it would be
extended to move pages around to combied them into bigger one though
too, but network stack requires high-order allocations in extremely rare
cases of broken design (Intel folks, sorry, but your hardware sucks in
that regard - jumbo frame of 9k should not require 16k of mem plu
network overhead).
NTA also does not align buffers to the power of two - extremely significant
win of that approach can be found on project's homepage with graps of
failed allocations and state of the mem for different sizes of
allocaions. Power-of-two overhead of SLAB is extremely high.
All it does is try to avoid fragmentation by policy - a problem
impossible to solve in general; but can achieve good results in view of
practical limitations on program behaviour.
Does your policy work for the given workload? we'll see.
Also, on what level, each level has both internal and external
fragmentation. I can argue that having large immovable objects in memory
adds to the fragmentation issues on the page-allocator level.
NTA works with pages, not with contiguous memory, it reduces
fragmentation inside pages, which can not be solved in SLAB, where
objects from the same page can live in different caches and thus _never_
can be combined. Thus, the only soultuin for SLAB is copy, which is not a
good one for big sizes and is just wrong for big pages.
It is not about page moving and VM tricks, which are generally described
as fragmentation avoidance technique, but about how fragmentation
problem is solved in one page.
quoted
is very SMP
friendly in that regard that it is per-cpu like slab and never free
objects on different CPUs, so they always stay in the same cache.
This makes it very hard to guarantee a reserve limit. (Not impossible,
just more difficult)
The whole pool of pages becomes reserve, since no one (and mainly VFS)
can consume that reserve.
quoted
Among other goodies it allows to have full sending/receiving zero-copy.
That won't ever work unless you have page aligned objects, otherwise you
cannot map them into user-space. Which seems to be at odds with your
tight packing/reduce internal fragmentation goals.
Zero-copy entails mapping the page the hardware writes the packet in
into user-space, right?
Since its impossible to predict to whoem the next packet is addressed
the packets must be written (by hardware) to different pages.
Yes, receiving zero-copy without appropriate hardware assist is
impossible, so either absence of such facility at all, or special overhead,
which forces object to lie in different pages. With hardware assist it
would be possible to select a flow in advance, so data would be packet
in the same page.
Sending zero-copy from userspace memory does not suffer with any such
problem.
--
Evgeniy Polyakov
From: Peter Zijlstra <hidden> Date: 2007-01-18 15:12:42
On Thu, 2007-01-18 at 16:58 +0300, Evgeniy Polyakov wrote:
Network is special in this regard, since it only has one allocation path
(actually it has one cache for skb, and usual kmalloc, but they are
called from only two functions).
So it would become
ptr = network_alloc();
and network_alloc() would be usual kmalloc or call for own allocator in
case of deadlock.
There is more to networking that skbs only, what about route cache,
there is quite a lot of allocs in this fib_* stuff, IGMP etc...
quoted
very high order pages, no?
This means that you have to either allocate at boot time and cannot
resize/add pools; which means you waste all that memory if the network
load never comes near using the reserved amount.
Or, you get into all the same trouble the hugepages folks are trying so
very hard to solve.
It is configurable - by default it takes pool of 32k pages for allocations for
jumbo-frames (e1000 requires such allocations for 9k frames
unfortunately), without jumbo-frame support it works with pool of 0-order
pages, which grows dynamically when needed.
With 0-order pages, you can only fit 2 1500 byte packets in there, you
could perhaps stick some small skb heads in there as well, but why
bother, the waste isn't _that_ high.
Esp if you would make a slab for 1500 mtu packets (5*1638 < 2*4096; and
1638 should be enough, right?)
It would make sense to pack related objects into a page so you could
free all together.
quoted
quoted
performs self-defragmentation of the memeory,
Does it move memory about?
It works in a page, not as pages - when neighbour regions are freed,
they are combined into single one with bigger size
Yeah, that is not defragmentation, defragmentation is moving active
regions about to create contiguous free space. What you do is free space
coalescence.
but network stack requires high-order allocations in extremely rare
cases of broken design (Intel folks, sorry, but your hardware sucks in
that regard - jumbo frame of 9k should not require 16k of mem plu
network overhead).
Well, if you have such hardware its not rare at all, But yeah that
sucks.
NTA also does not align buffers to the power of two - extremely significant
win of that approach can be found on project's homepage with graps of
failed allocations and state of the mem for different sizes of
allocaions. Power-of-two overhead of SLAB is extremely high.
Sure you can pack the page a little better(*), but I thought the main
advantage was a speed increase.
(*) memory is generally cheaper than engineering efforts, esp on this
scale. The only advantage in the manual packing is that (with the fancy
hardware stream engine mentioned below) you could ensure they are
grouped together (then again, the hardware stream engine would, together
with a SG-DMA engine, take care of that).
quoted
All it does is try to avoid fragmentation by policy - a problem
impossible to solve in general; but can achieve good results in view of
practical limitations on program behaviour.
Does your policy work for the given workload? we'll see.
Also, on what level, each level has both internal and external
fragmentation. I can argue that having large immovable objects in memory
adds to the fragmentation issues on the page-allocator level.
NTA works with pages, not with contiguous memory, it reduces
fragmentation inside pages, which can not be solved in SLAB, where
objects from the same page can live in different caches and thus _never_
can be combined. Thus, the only soultuin for SLAB is copy, which is not a
good one for big sizes and is just wrong for big pages.
By allocating, and never returning the page to the page-allocator you've
increased the fragmentation on the page-allocator level significantly.
It will avoid a super page ever forming around that page.
It is not about page moving and VM tricks, which are generally described
as fragmentation avoidance technique, but about how fragmentation
problem is solved in one page.
Short of defragmentation (move active regions about) fragmentation is an
unsolved problem. For any heuristic there is a pattern that will defeat
it.
Luckily program allocation behaviour is usually very regular (or
decomposable in well behaved groups).
quoted
quoted
is very SMP
friendly in that regard that it is per-cpu like slab and never free
objects on different CPUs, so they always stay in the same cache.
This makes it very hard to guarantee a reserve limit. (Not impossible,
just more difficult)
The whole pool of pages becomes reserve, since no one (and mainly VFS)
can consume that reserve.
Ah, but there you violate my requirement, any network allocation can
claim the last bit of memory. The whole idea was that the reserve is
explicitly managed.
It not only needs protection from other users but also from itself.
quoted
quoted
Among other goodies it allows to have full sending/receiving zero-copy.
That won't ever work unless you have page aligned objects, otherwise you
cannot map them into user-space. Which seems to be at odds with your
tight packing/reduce internal fragmentation goals.
Zero-copy entails mapping the page the hardware writes the packet in
into user-space, right?
Since its impossible to predict to whoem the next packet is addressed
the packets must be written (by hardware) to different pages.
Yes, receiving zero-copy without appropriate hardware assist is
impossible, so either absence of such facility at all, or special overhead,
which forces object to lie in different pages. With hardware assist it
would be possible to select a flow in advance, so data would be packet
in the same page.
I was not aware that hardware could order the packets in such a fashion.
Yes, if it can do that it becomes doable.
Sending zero-copy from userspace memory does not suffer with any such
problem.
True, that is properly ordered. But for that I'm not sure how NTA (you
really should change that name, there is no Tree anymore) helps here.
On Thu, Jan 18, 2007 at 04:10:52PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
On Thu, 2007-01-18 at 16:58 +0300, Evgeniy Polyakov wrote:
quoted
Network is special in this regard, since it only has one allocation path
(actually it has one cache for skb, and usual kmalloc, but they are
called from only two functions).
So it would become
ptr = network_alloc();
and network_alloc() would be usual kmalloc or call for own allocator in
case of deadlock.
There is more to networking that skbs only, what about route cache,
there is quite a lot of allocs in this fib_* stuff, IGMP etc...
skbs are the most extensively used path.
Actually the same is applied to route - dst_entries and rtable are
allocated through own wrappers.
quoted
quoted
very high order pages, no?
This means that you have to either allocate at boot time and cannot
resize/add pools; which means you waste all that memory if the network
load never comes near using the reserved amount.
Or, you get into all the same trouble the hugepages folks are trying so
very hard to solve.
It is configurable - by default it takes pool of 32k pages for allocations for
jumbo-frames (e1000 requires such allocations for 9k frames
unfortunately), without jumbo-frame support it works with pool of 0-order
pages, which grows dynamically when needed.
With 0-order pages, you can only fit 2 1500 byte packets in there, you
could perhaps stick some small skb heads in there as well, but why
bother, the waste isn't _that_ high.
Esp if you would make a slab for 1500 mtu packets (5*1638 < 2*4096; and
1638 should be enough, right?)
It would make sense to pack related objects into a page so you could
free all together.
With power-of-two allocation SLAB wastes 500 bytes for each 1500 MTU
packet (roughly), it is actaly one ACK packet - and I hear it from
person who develops a system, which is aimed to guarantee ACK
allocation in OOM :)
SLAB overhead is _very_ expensive for network - what if jumbo frame is
used? It becomes incredible in that case, although modern NICs allows
scatter-gather, which is aimed to fix the problem.
Cache misses for small packet flow due to the fact, that the same data
is allocated and freed and accessed on different CPUs will become an
issue soon, not right now, since two-four core CPUs are not yet to be
very popular and price for the cache miss is not _that_ high.
quoted
quoted
quoted
performs self-defragmentation of the memeory,
Does it move memory about?
It works in a page, not as pages - when neighbour regions are freed,
they are combined into single one with bigger size
Yeah, that is not defragmentation, defragmentation is moving active
regions about to create contiguous free space. What you do is free space
coalescence.
That is wrong definition just because no one developed different system.
Defragmentation is a result of broken system.
Existing design _does_not_ allow to have the situation when whole page
belongs to the same cache after it was actively used, the same is
applied to the situation when several pages, which create contiguous
region, are used by different users, so people start develop VM tricks
to move pages around so they would be placed near in address space.
Do not fix the result, fix the reason.
quoted
but network stack requires high-order allocations in extremely rare
cases of broken design (Intel folks, sorry, but your hardware sucks in
that regard - jumbo frame of 9k should not require 16k of mem plu
network overhead).
Well, if you have such hardware its not rare at all, But yeah that
sucks.
They do a good jop developing different approaches to workaround that
hardware 'feature', but this is still wrong situation.
quoted
NTA also does not align buffers to the power of two - extremely significant
win of that approach can be found on project's homepage with graps of
failed allocations and state of the mem for different sizes of
allocaions. Power-of-two overhead of SLAB is extremely high.
Sure you can pack the page a little better(*), but I thought the main
advantage was a speed increase.
(*) memory is generally cheaper than engineering efforts, esp on this
scale. The only advantage in the manual packing is that (with the fancy
hardware stream engine mentioned below) you could ensure they are
grouped together (then again, the hardware stream engine would, together
with a SG-DMA engine, take care of that).
Extensoin way of doing things.
That is wrong.
quoted
quoted
All it does is try to avoid fragmentation by policy - a problem
impossible to solve in general; but can achieve good results in view of
practical limitations on program behaviour.
Does your policy work for the given workload? we'll see.
Also, on what level, each level has both internal and external
fragmentation. I can argue that having large immovable objects in memory
adds to the fragmentation issues on the page-allocator level.
NTA works with pages, not with contiguous memory, it reduces
fragmentation inside pages, which can not be solved in SLAB, where
objects from the same page can live in different caches and thus _never_
can be combined. Thus, the only soultuin for SLAB is copy, which is not a
good one for big sizes and is just wrong for big pages.
By allocating, and never returning the page to the page-allocator you've
increased the fragmentation on the page-allocator level significantly.
It will avoid a super page ever forming around that page.
Not at all - SLAB fragmentation is so high, that stealing pages from its
highly fragmented pool does not result in any lose or win for SPAB
users. And it is possible to allocate at boot time.
NTA cache grows in _very_ rare cases, and it can be preallocated at
startup.
quoted
It is not about page moving and VM tricks, which are generally described
as fragmentation avoidance technique, but about how fragmentation
problem is solved in one page.
Short of defragmentation (move active regions about) fragmentation is an
unsolved problem. For any heuristic there is a pattern that will defeat
it.
Luckily program allocation behaviour is usually very regular (or
decomposable in well behaved groups).
We are talking about different approaces here.
Per-page defragmentation by playing games with memory management is one
approach. Run-time defragmentation by groupping neighbour regions is
another one.
Main issue is the fact, that with second one, requirement for the first
one becomes MUCH smaller, since when application, no matter how strange
its allocation pattern is, frees object, it will be groupped with
neighbours. In SLAB that will almost never happen, so situation with
memory tricks.
quoted
quoted
quoted
is very SMP
friendly in that regard that it is per-cpu like slab and never free
objects on different CPUs, so they always stay in the same cache.
This makes it very hard to guarantee a reserve limit. (Not impossible,
just more difficult)
The whole pool of pages becomes reserve, since no one (and mainly VFS)
can consume that reserve.
Ah, but there you violate my requirement, any network allocation can
claim the last bit of memory. The whole idea was that the reserve is
explicitly managed.
It not only needs protection from other users but also from itself.
Specifying some users as good and others as bad generally tends to very
bad behaviour. Your appwoach only covers some users, mine does not
differentiate between users, but prevents system from such situation at all.
quoted
quoted
quoted
Among other goodies it allows to have full sending/receiving zero-copy.
That won't ever work unless you have page aligned objects, otherwise you
cannot map them into user-space. Which seems to be at odds with your
tight packing/reduce internal fragmentation goals.
Zero-copy entails mapping the page the hardware writes the packet in
into user-space, right?
Since its impossible to predict to whoem the next packet is addressed
the packets must be written (by hardware) to different pages.
Yes, receiving zero-copy without appropriate hardware assist is
impossible, so either absence of such facility at all, or special overhead,
which forces object to lie in different pages. With hardware assist it
would be possible to select a flow in advance, so data would be packet
in the same page.
I was not aware that hardware could order the packets in such a fashion.
Yes, if it can do that it becomes doable.
Not hardware, but allocator, which can provide data with special
requirement like alignment and offset for given flow id.
Hardware just provides needed info and does DMA transfer into specified area.
You can find more on receiving zero-copy with _emulation_ of such
hardware (MMIO copy of the header) for example here:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=recv_zero_copy
where system perfomed data receiving of 1500 MTU sized frames directly
into VFS cache.
quoted
Sending zero-copy from userspace memory does not suffer with any such
problem.
True, that is properly ordered. But for that I'm not sure how NTA (you
really should change that name, there is no Tree anymore) helps here.
Because user has access to the memory which will be used directly by
hardware, it should not care about preallocation, although there are
problems with notification about completeness of operation, which can be
postponed in case of fancy egress filters used.
--
Evgeniy Polyakov
From: Peter Zijlstra <hidden> Date: 2007-01-18 17:33:45
On Thu, 2007-01-18 at 18:50 +0300, Evgeniy Polyakov wrote:
On Thu, Jan 18, 2007 at 04:10:52PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
On Thu, 2007-01-18 at 16:58 +0300, Evgeniy Polyakov wrote:
quoted
Network is special in this regard, since it only has one allocation path
(actually it has one cache for skb, and usual kmalloc, but they are
called from only two functions).
So it would become
ptr = network_alloc();
and network_alloc() would be usual kmalloc or call for own allocator in
case of deadlock.
There is more to networking that skbs only, what about route cache,
there is quite a lot of allocs in this fib_* stuff, IGMP etc...
skbs are the most extensively used path.
Actually the same is applied to route - dst_entries and rtable are
allocated through own wrappers.
Still, edit all places and perhaps forget one and make sure all new code
doesn't forget about it, or pick a solution that covers everything.
With power-of-two allocation SLAB wastes 500 bytes for each 1500 MTU
packet (roughly), it is actaly one ACK packet - and I hear it from
person who develops a system, which is aimed to guarantee ACK
allocation in OOM :)
I need full data traffic during OOM, not just a single ACK.
SLAB overhead is _very_ expensive for network - what if jumbo frame is
used? It becomes incredible in that case, although modern NICs allows
scatter-gather, which is aimed to fix the problem.
Jumbo frames are fine if the hardware can do SG-DMA..
Cache misses for small packet flow due to the fact, that the same data
is allocated and freed and accessed on different CPUs will become an
issue soon, not right now, since two-four core CPUs are not yet to be
very popular and price for the cache miss is not _that_ high.
SGI does networking too, right?
quoted
quoted
quoted
quoted
performs self-defragmentation of the memeory,
Does it move memory about?
It works in a page, not as pages - when neighbour regions are freed,
they are combined into single one with bigger size
Yeah, that is not defragmentation, defragmentation is moving active
regions about to create contiguous free space. What you do is free space
coalescence.
That is wrong definition just because no one developed different system.
Defragmentation is a result of broken system.
Existing design _does_not_ allow to have the situation when whole page
belongs to the same cache after it was actively used, the same is
applied to the situation when several pages, which create contiguous
region, are used by different users, so people start develop VM tricks
to move pages around so they would be placed near in address space.
Do not fix the result, fix the reason.
*plonk* 30+yrs of research ignored.
quoted
quoted
The whole pool of pages becomes reserve, since no one (and mainly VFS)
can consume that reserve.
Ah, but there you violate my requirement, any network allocation can
claim the last bit of memory. The whole idea was that the reserve is
explicitly managed.
It not only needs protection from other users but also from itself.
Specifying some users as good and others as bad generally tends to very
bad behaviour. Your appwoach only covers some users, mine does not
differentiate between users,
The kernel is special, right? It has priority over whatever user-land
does.
but prevents system from such situation at all.
I'm not seeing that, with your approach nobody stops the kernel from
filling up the memory with user-space network traffic.
swapping is not some random user process, its a fundamental kernel task,
if this fails the machine is history.
On Thu, Jan 18, 2007 at 06:31:53PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
skbs are the most extensively used path.
Actually the same is applied to route - dst_entries and rtable are
allocated through own wrappers.
Still, edit all places and perhaps forget one and make sure all new code
doesn't forget about it, or pick a solution that covers everything.
There is _one_ place for allocation of any kind of object.
skb path has two places.
quoted
With power-of-two allocation SLAB wastes 500 bytes for each 1500 MTU
packet (roughly), it is actaly one ACK packet - and I hear it from
person who develops a system, which is aimed to guarantee ACK
allocation in OOM :)
I need full data traffic during OOM, not just a single ACK.
But your code exactly limit codepath to several allocaions, which must
be ACK. You do not have enough reserve to support whole traffic.
So the right solution, IMO, is to _prevent_ such situation, which means
that allocation is not allowed to depend on external conditions like
VFS.
Actually my above sentences were about the case, when anly having
different allocator, it is possible to dramatically change memory usage
model, which supffers greatly from power-of-two allocations. OOM
condition is one of the results which has big SLAB overhead among other
roots. Actually all pathes which work with kmem_cache are safe against
it, since kernel cache packs objects, but thos who uses raw kmalloc has
problems.
quoted
SLAB overhead is _very_ expensive for network - what if jumbo frame is
used? It becomes incredible in that case, although modern NICs allows
scatter-gather, which is aimed to fix the problem.
Jumbo frames are fine if the hardware can do SG-DMA..
Notice word _IF_ in you sentence. e1000 for example can not (or it can,
but driver is not developed for such scenario).
quoted
Cache misses for small packet flow due to the fact, that the same data
is allocated and freed and accessed on different CPUs will become an
issue soon, not right now, since two-four core CPUs are not yet to be
very popular and price for the cache miss is not _that_ high.
SGI does networking too, right?
Yep, Cristoph Lameter developed own allocator too.
I agreee with you, that if that price is too high already, then it is a
dditional sign to look into network tree allocator (yep, name is bad)
again.
quoted
That is wrong definition just because no one developed different system.
Defragmentation is a result of broken system.
Existing design _does_not_ allow to have the situation when whole page
belongs to the same cache after it was actively used, the same is
applied to the situation when several pages, which create contiguous
region, are used by different users, so people start develop VM tricks
to move pages around so they would be placed near in address space.
Do not fix the result, fix the reason.
*plonk* 30+yrs of research ignored.
30 years to develop SLAB allocator? In what universe that is all about?
quoted
quoted
quoted
The whole pool of pages becomes reserve, since no one (and mainly VFS)
can consume that reserve.
Ah, but there you violate my requirement, any network allocation can
claim the last bit of memory. The whole idea was that the reserve is
explicitly managed.
It not only needs protection from other users but also from itself.
Specifying some users as good and others as bad generally tends to very
bad behaviour. Your appwoach only covers some users, mine does not
differentiate between users,
The kernel is special, right? It has priority over whatever user-land
does.
Kernel only does ACK generation and allocation for userspace.
Kernel does not know that some of users are potentially good or bad, and
if you will export this socket option to the userspace, everyone will
think that his application is good enough to use reserve.
So, for kernel-only side you just need to preallocate pool of packets
and use them when system is in OOM (reclaim). For the long direction,
new approach of memory allocaiton should be developed, and there are
different works in that direction - NTA is one of them and not the only
one, for the best resutlts it must be combined with vm-tricks
defragmentation too.
quoted
but prevents system from such situation at all.
I'm not seeing that, with your approach nobody stops the kernel from
filling up the memory with user-space network traffic.
swapping is not some random user process, its a fundamental kernel task,
if this fails the machine is history.
You completely misses the point. The main goal is to
1. reduce fragmentation and/or enable self defragmentation (which is
done in NTA), this also reduces memory usage.
2. perform correct recover steps in OOM - reduce memory usage, use
different allocator and/or reserve (which is the case, where NTA can be
used)
3. do not allow OOM condition - unfortunately it is not always possible,
but having separated allocation allows to not depend on external
conditions such as VFS memory usage, thus this approach reduces
condition when memory deadlock related to network path can happen.
Let me briefly describe your approach and possible drawbacks in it.
You start reserving some memory when systems is under memory pressure.
when system is in real trouble, you start using that reserve for special
tasks mainly for network path to allocate packets and process them in
order to get committed some memory swapping.
So, the problems I see here, are following:
1. it is possible that when you are starting to create a reserve, there
will not be enough memeory at all. So the solution is to reserve in
advance.
2. You differentiate by hand between critical and non-critical
allocations by specifying some kernel users as potentially possible to
allocate from reserve. This does not prevent from NVIDIA module to
allocate from that reserve too, does it? And you artificially limit
system to process only tiny bits of what it must do, thus potentially
leaking pathes which must use reserve too.
So, solution is to have a reserve in advance, and manage it using
special path when system is in OOM. So you will have network memory
reserve, which will be used when system is in trouble. It is very
similar to what you had.
But the whole reserve can never be used at all, so it should be used,
but not by those who can create OOM condition, thus it should be
exported to, for example, network only, and when system is in trouble,
network would be still functional (although only critical pathes).
Even further development of such idea is to prevent such OOM condition
at all - by starting swapping early (but wisely) and reduce memory
usage.
Network tree allocator does exactly above cases.
Here advertisement is over.
--
Evgeniy Polyakov
From: Peter Zijlstra <hidden> Date: 2007-01-19 12:55:24
Let me briefly describe your approach and possible drawbacks in it.
You start reserving some memory when systems is under memory pressure.
when system is in real trouble, you start using that reserve for special
tasks mainly for network path to allocate packets and process them in
order to get committed some memory swapping.
So, the problems I see here, are following:
1. it is possible that when you are starting to create a reserve, there
will not be enough memeory at all. So the solution is to reserve in
advance.
Swap is usually enabled at startup, but sure, if you want you can mess
this up.
2. You differentiate by hand between critical and non-critical
allocations by specifying some kernel users as potentially possible to
allocate from reserve.
True, all sockets that are needed for swap, no-one else.
This does not prevent from NVIDIA module to
allocate from that reserve too, does it?
All users of the NVidiot crap deserve all the pain they get.
If it breaks they get to keep both pieces.
And you artificially limit
system to process only tiny bits of what it must do, thus potentially
leaking pathes which must use reserve too.
How so? I cover pretty much every allocation needed to process an skb by
setting PF_MEMALLOC - the only drawback there is that the reserve might
not actually be large enough because it covers more allocations that
were considered. (thats one of the TODO items, validate the reserve
functions parameters)
So, solution is to have a reserve in advance, and manage it using
special path when system is in OOM. So you will have network memory
reserve, which will be used when system is in trouble. It is very
similar to what you had.
But the whole reserve can never be used at all, so it should be used,
but not by those who can create OOM condition, thus it should be
exported to, for example, network only, and when system is in trouble,
network would be still functional (although only critical pathes).
But the network can create OOM conditions for itself just fine.
Consider the remote storage disappearing for a while (it got rebooted,
someone tripped over the wire etc..). Now the rest of the network
traffic keeps coming and will queue up - because user-space is stalled,
waiting for more memory - and we run out of memory.
There must be a point where we start dropping packets that are not
critical to the survival of the machine.
Even further development of such idea is to prevent such OOM condition
at all - by starting swapping early (but wisely) and reduce memory
usage.
These just postpone execution but will not avoid it.
From: Christoph Lameter <hidden> Date: 2007-01-19 17:55:31
On Thu, 18 Jan 2007, Peter Zijlstra wrote:
quoted
Cache misses for small packet flow due to the fact, that the same data
is allocated and freed and accessed on different CPUs will become an
issue soon, not right now, since two-four core CPUs are not yet to be
very popular and price for the cache miss is not _that_ high.
SGI does networking too, right?
Sslab deals with those issues the right way. We have per processor
queues that attempt to keep the cache hot state. A special shared queue
exists between neighboring processors to facilitate exchange of objects
between then.
On Fri, Jan 19, 2007 at 01:53:15PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
2. You differentiate by hand between critical and non-critical
allocations by specifying some kernel users as potentially possible to
allocate from reserve.
True, all sockets that are needed for swap, no-one else.
quoted
This does not prevent from NVIDIA module to
allocate from that reserve too, does it?
All users of the NVidiot crap deserve all the pain they get.
If it breaks they get to keep both pieces.
I meant that pretty anyone can be those user, who can just add a bit
into own gfp_flags which are used for allocation.
quoted
And you artificially limit
system to process only tiny bits of what it must do, thus potentially
leaking pathes which must use reserve too.
How so? I cover pretty much every allocation needed to process an skb by
setting PF_MEMALLOC - the only drawback there is that the reserve might
not actually be large enough because it covers more allocations that
were considered. (thats one of the TODO items, validate the reserve
functions parameters)
You only covered ipv4/v6 and arp, maybe some route updates.
But it is very possible, that some allocations are missed like
multicast/broadcast. Selecting only special pathes out of the whole
possible network alocations tends to create a situation, when something
is missed or cross dependant on other pathes.
quoted
So, solution is to have a reserve in advance, and manage it using
special path when system is in OOM. So you will have network memory
reserve, which will be used when system is in trouble. It is very
similar to what you had.
But the whole reserve can never be used at all, so it should be used,
but not by those who can create OOM condition, thus it should be
exported to, for example, network only, and when system is in trouble,
network would be still functional (although only critical pathes).
But the network can create OOM conditions for itself just fine.
Consider the remote storage disappearing for a while (it got rebooted,
someone tripped over the wire etc..). Now the rest of the network
traffic keeps coming and will queue up - because user-space is stalled,
waiting for more memory - and we run out of memory.
Hmm... Neither UDP, nor TCP work that way actually.
There must be a point where we start dropping packets that are not
critical to the survival of the machine.
You still can drop them, the main point is that network allocations do
not depend on other allocations.
quoted
Even further development of such idea is to prevent such OOM condition
at all - by starting swapping early (but wisely) and reduce memory
usage.
These just postpone execution but will not avoid it.
No. If system allows to have such a condition, then
something is broken. It must be prevented, instead of creating special
hacks to recover from it.
--
Evgeniy Polyakov
From: Rik van Riel <riel@surriel.com> Date: 2007-01-20 22:58:50
Evgeniy Polyakov wrote:
On Fri, Jan 19, 2007 at 01:53:15PM +0100, Peter Zijlstra (a.p.zijlstra@chello.nl) wrote:
quoted
quoted
Even further development of such idea is to prevent such OOM condition
at all - by starting swapping early (but wisely) and reduce memory
usage.
These just postpone execution but will not avoid it.
No. If system allows to have such a condition, then
something is broken. It must be prevented, instead of creating special
hacks to recover from it.
Evgeniy, you may want to learn something about the VM before
stating that reality should not occur.
Due to the way everything in the kernel works, you cannot
prevent the memory allocator from allocating everything and
running out, except maybe by setting aside reserves to deal
with special subsystems.
As for your "swapping early and reduce memory usage", that is
just not possible in a system where a memory writeout may need
one or more memory allocations to succeed and other I/O paths
(eg. file writes) can take memory from the same pools.
With something like iscsi it may be _necessary_ for file writes
and swap to take memory from the same pools, because they can
share the same block device.
Please get out of your fantasy world and accept the constraints
the VM has to operate under. Maybe then you and Peter can agree
on something.
--
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is. Each group
calls the other unpatriotic.
On Sat, Jan 20, 2007 at 05:36:03PM -0500, Rik van Riel (riel@surriel.com) wrote:
Evgeniy Polyakov wrote:
quoted
On Fri, Jan 19, 2007 at 01:53:15PM +0100, Peter Zijlstra
(a.p.zijlstra@chello.nl) wrote:
quoted
quoted
quoted
Even further development of such idea is to prevent such OOM condition
at all - by starting swapping early (but wisely) and reduce memory
usage.
These just postpone execution but will not avoid it.
No. If system allows to have such a condition, then
something is broken. It must be prevented, instead of creating special
hacks to recover from it.
Evgeniy, you may want to learn something about the VM before
stating that reality should not occur.
I.e. I should start believing that OOM can not be prevented, bugs can
not be fixed and things can not be changed just because it happens right
now? That is why I'm not subscribed to lkml :)
Due to the way everything in the kernel works, you cannot
prevent the memory allocator from allocating everything and
running out, except maybe by setting aside reserves to deal
with special subsystems.
As for your "swapping early and reduce memory usage", that is
just not possible in a system where a memory writeout may need
one or more memory allocations to succeed and other I/O paths
(eg. file writes) can take memory from the same pools.
When system starts swapping only when it can not allocate new page,
then it is broken system. I bet you get warm closing way before you
hands are frostbitten, and you do not have a liter of alcohol in the
packet for such emergency. And to get warm closing you still need to
go over cold street into the shop, but you will do it before weather
becomes arctic.
With something like iscsi it may be _necessary_ for file writes
and swap to take memory from the same pools, because they can
share the same block device.
Of course swapping can require additional allocation, when it happens
over network it is quite obvious.
The main problem is the fact, that if system was put into the state,
when its life depends on the last possible allocation, then it is
broken.
There is a light connected to car's fuel tank which starts blinking,
when amount of fuel is less then predefined level. Car just does not
stop suddenly and starts to get fuel from reserve (well eventually it
stops, but it says about problem long before it dies).
Please get out of your fantasy world and accept the constraints
the VM has to operate under. Maybe then you and Peter can agree
on something.
I can not accept the situation, when problem is not fixed, but instead
recovery path is added. There must be both ways of dealing with it -
emergency force majeur recovery and preventive steps.
What we are talking about (except pointing to obvious things and sending
to school-classes), at least how I see this, is ways of dealing with
possible OOM condition. If OOM has happend, then there must be recovery
path, but OOM must be prevented, and ways to do this were described too.
--
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is. Each group
calls the other unpatriotic.
On Sat, Jan 20, 2007 at 05:36:03PM -0500, Rik van Riel (riel@surriel.com) wrote:
quoted
Due to the way everything in the kernel works, you cannot
prevent the memory allocator from allocating everything and
running out, except maybe by setting aside reserves to deal
with special subsystems.
As a technical side gets described, this is exactly the way I proposed -
there is special dedicated pool which does not depend on main system
allocator, so if the latter is empty, the former still _can_ work,
although it is possible that it will be empty too.
Separation.
It removes avalanche effect when one problem produces several different.
I do not say that some allocator is the best for dealing with such
situation, I just pointed that critical pathes were separated in NTA, so
they do not depend on each one's failure.
Actually that separation was introduced way too long ago with memory
pools, this is some kind of continuation, which adds a lot of additional
extremely useful features.
NTA used for network allocations is that pool, since in real life
packets can not be allocated in advance without memory overhead. For
simple situations like only ACK generatinos it is possible, which I
suggested first, but long-term solution is special allocator.
I selected NTA for this task because it has _additional_ features like
self-deragmentation, which is very useful part for networking, but if
only OOM recovery condition is concerned, then actually any other
allocator can be used of course.
--
Evgeniy Polyakov
From: Rik van Riel <riel@surriel.com> Date: 2007-01-21 16:31:15
Evgeniy Polyakov wrote:
On Sat, Jan 20, 2007 at 05:36:03PM -0500, Rik van Riel (riel@surriel.com) wrote:
quoted
Evgeniy Polyakov wrote:
quoted
On Fri, Jan 19, 2007 at 01:53:15PM +0100, Peter Zijlstra
(a.p.zijlstra@chello.nl) wrote:
quoted
quoted
Even further development of such idea is to prevent such OOM condition
at all - by starting swapping early (but wisely) and reduce memory
usage.
These just postpone execution but will not avoid it.
No. If system allows to have such a condition, then
something is broken. It must be prevented, instead of creating special
hacks to recover from it.
Evgeniy, you may want to learn something about the VM before
stating that reality should not occur.
I.e. I should start believing that OOM can not be prevented, bugs can
not be fixed and things can not be changed just because it happens right
now? That is why I'm not subscribed to lkml :)
The reasons for this are often not inside the VM itself,
but are due to the constraints imposed on the VM.
For example, with many of the journaled filesystems there
is no way to know in advance how much IO needs to be done
to complete a writeout of one dirty page (and consequently,
how much memory needs to be allocated to complete this one
writeout).
Parts of the VM could be changed to reduce the pressure
somewhat, eg. limiting the number of IOs in flight, but
that will probably have performance consequences that may
not be acceptable to Andrew and Linus and never get merged.
--
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is. Each group
calls the other unpatriotic.