Just to be clear the "BUG...." line I quoted contains an error of my own.

It should say "kmalloc-8" twice rather than having "kmalloc-96" toward the
end.  This is the result of a cut-and-paste error (and brain already on
vacation).  The module ALSO has a larger struct the cache for which gets
merged with the kmalloc-96 cache.  I grabbed the first BUG line from the
dmesg output and replaced "96" with "8" when composing the email, but
missed the second occurrence.

Merry Christmas,
-Paul


On Mon, Dec 24, 2012 at 4:55 PM, Paul Hargrove [off-list ref] wrote:

I have a 3.7.1 kernel on x86-86
It is configured with
  CONFIG_SLUB=y
  CONFIG_SLUB_DEBUG=y

I have an out-of-tree module calling KMEM_CACHE for an 8-byte struct:
        cr_pdata_cachep = KMEM_CACHE(cr_pdata_s,0);
        if (!cr_pdata_cachep) goto no_pdata_cachep;
        printk(KERN_ERR "@ refcount = %d name = '%s'\n",
cr_pdata_cachep->refcount, cr_pdata_cachep->name);

The output of the printk, below, shows that the request has been merged
with the built-in 8-byte kmalloc pool, BUT the resulting refcount is 1,
rather than 2 (or more):
    @ refcount = 1 name = 'kmalloc-8'

This results in a very unhappy kernel when the module calls
    kmem_cache_destroy(cr_pdata_cachep);
at rmmod time, resulting is messages like
    BUG kmalloc-8 (Tainted: G           O): Objects remaining in
kmalloc-96 on kmem_cache_close()

A quick look through mm/slub.c appears to confirm my suspicion that
"s->refcount" is never incremented for the built-in kmalloc-* caches.
 However, I leave it to the experts to determine where the increment
belongs.

FWIW: I am currently passing SLAB_POISON for the flags argument to
KMEM_CACHE() as a work-around (it prevents merging and, if I understand
correctly, has no overhead in a non-debug build).

-Paul

--
Paul H. Hargrove                          PHHargrove@lbl.gov
Future Technologies Group
Computer and Data Sciences Department     Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory     Fax: +1-510-486-6900

-- 
Paul H. Hargrove                          PHHargrove@lbl.gov
Future Technologies Group
Computer and Data Sciences Department     Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory     Fax: +1-510-486-6900