On Wed, 15 Sep 2010 18:21:47 -0700
Dave Hansen [off-list ref] wrote:

On Thu, 2010-09-16 at 09:12 +0900, KAMEZAWA Hiroyuki wrote:

quoted

I hear a customer's case. His server generates 3-80000+ new dentries per day
and dentries will be piled up to 1000000+ in a month. This makes open()'s 
performance very bad because Hash-lookup will be heavy. (He has very big memory.)

What we could ask him was
  - rewrite your application. or
  - reboot once in a month (and change hash size) or
  - drop_cache once in a month

Because their servers cannot stop, he used drop_caches once in a month
while his server is idle, at night. Changing HashSize cannot be a permanent
fix because he may not stop the server for years.

That is a really interesting case.

They must have a *ton* of completely extra memory laying around.  Do
they not have much page cache activity? 

I hear they have a ton of extra memory. Just open() slows down.

It usually balances out the  dentry/inode caches.

Would this user be better off with a smaller dentry hash in general? 

Maybe. I hear most of files were created-but-never-used data and logs.

Is it special hardware that should _have_ a lower default hash size?

I'm not sure. I think they have no boot option of hash size.

quoted

For rare users who have 10000000+ of files and tons of free memory, drop_cache
can be an emergency help. 

In this case, though, would a WARN_ON() in an emergency be such a bad
thing?  They evidently know what they're doing, and shouldn't be put off
by it.

Showing "Warning" means ", it's possibly bug." for almost all customers.
We'll get tons of "regression" report ;)

If you really want to add messages, please raise log level.
NOTICE or INFO sounds better(and moderate) to me because it's easy to explain
"Don't worry about the message, your kernel is stable and don't need to reboot.
 But please check the peformance, it tends to go bad. You lose cache.".
BTW, what(1or2or3) was writtern to "drop_cache" is important. please show.

Thanks,
-Kame







--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>