Thread (34 messages) 34 messages, 12 authors, 2004-02-05

Re: Active Memory Defragmentation: Our implementation & problems

From: Alok Mooley <hidden>
Date: 2004-02-04 05:09:21
Also in: lkml

--- Dave Hansen <haveblue@us.ibm.com> wrote:
On Mon, 2004-02-02 at 20:46, Alok Mooley wrote:
quoted
	/*
	 * See that page is not file backed & flags are
as desired.
quoted
	 * PG_lru , PG_direct (currently) must be set
	 */
	if(!page->mapping && (flags==0x10020 ||
flags==0x10024 || 
quoted
flags==0x10060 || 
flags==0x10064) && page_count(page))
		return 1;
	/*Page is not movable*/
	return 0;
Why are these flags hard-coded?  
Will use the macros. Thanks.
quoted
static void update_to_ptes(struct page *to)
...
        pgprot.pgprot = (pte->pte_low) &
(PAGE_SIZE-1);

You probably want to wrap this up in a macro for
each arch.  I couldn't
find an analagous one that exists.  
quoted
static void free_allocated_page(struct page *page)
Is this really necessary?  Why don't the regular
buddy allocator
functions work?
 The regular buddy freeing function also increases the
number of free pages. Since we are not actually
freeing pages (we are just moving them), we do not
want the original freeing function. But then we could 
decrease the number of free pages by the same number &
use the buddy freeing function. Will do. Thanks.
quoted
/*
* Flush the cache and tlb entries corresponding to
the pte for the
quoted
* 'from' page
* Flush the tlb entries corresponding to the given
page and not the 
quoted
whole
* tlb.
*/
static void flush_from_caches(pte_addr_t paddr)
{
How does this function compare to
try_to_unmap_one()?  Can you use that
instead?  It looked like you cut and pasted some
code from there.
 Yes we have cut & pasted some code from there. But,
try_to_unmap_one also does lots of other stuff like
           /*
            * Store the swap location in the pte.
            * See handle_pte_fault() ...
            */
which we don't want. Hence we use a separate function.
I'll stop there for now.  There seems to be a lot of
code in the file
that's one-off from current kernel code.  I think a
close examination of
currently available kernel functions could drasticly
reduce the size of
your module.  
Will try to reduce the code. Thanks.


Could you also please comment & advise us on our
problems which are as below: -

We want to broaden our definition of a movable
page, & consider kernel
pages & file-backed pages also for movement (currently
we consider only userspace anonymous pages as
movable). Do
file-backed pages also obey the
3GB rule? In order to move such pages, we will have to
patch macros like "virt_to_phys"
& other related macros, so that the address
translation for pages moved by us will take place
vmalloc style, i.e., via page tables, instead of
direct +-3GB. Is it worth introducing such
an overhead for address translation (vmalloc does
it!)? If no, then is there another
way out, or is it better to stick to our current
definition of a movable page?
Identifying pages moved by us may involve introducing
a new page-flag. A new page-flag 
for per-cpu pages would be great, since we have to
traverse the per-cpu hot & cold lists
in order to identify if a page is on the pcp lists. 

As of now, we have adopted a failure based approach,
i.e, we defragment only when 
a higher order allocation failure has taken place
(just before kswapd starts swapping). 
We now want to defragment based on thresholds kept for
each allocation order. 
Instead of a daemon kicking in on a threshold 
violation (as proposed by Mr. Daniel Phillips), we
intend to capture idle cpu cycles
by inserting a new process just above the idle
process. Now, when we are scheduled,
we are sure that the cpu is idle, & this is when we
check for threshold violation & defragment.
One problem with this would be when to reschedule
ourselves (allow our preemption)? 
We do not want the memory state to change beneath us,
so right now we are not 
allowing our preemption.
This will ofcourse hog the cpu, & we may not be able
to reschedule just by checking
the need_resched flag. Any advice or suggestions
regarding this problem? Also, will
the idle cpu approach be better or will the daemon
based approach be better?

We will be thankful for any suggestions,advice &
comments. 

Thanking you,
-Alok & project group




__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free web site building tool. Try it!
http://webhosting.yahoo.com/ps/sb/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help