Re: [PATCH 12/30] mm: memory reserve management

Hi Peter,

On Mon, 2008-07-28 at 12:17 +0200, Peter Zijlstra wrote:

On Mon, 2008-07-28 at 13:06 +0300, Pekka Enberg wrote:

quoted

Hi Peter,

On Thu, 2008-07-24 at 16:00 +0200, Peter Zijlstra wrote:

quoted

+/*
+ * alloc wrappers
+ */
+

Hmm, I'm not sure I like the use of __kmalloc_track_caller() (even
though you do add the wrappers for SLUB). The functions really are SLAB
internals so I'd prefer to see kmalloc_reserve() moved to the
allocators.

See below..

quoted

quoted

+void *___kmalloc_reserve(size_t size, gfp_t flags, int node, void *ip,
+			 struct mem_reserve *res, int *emerg)
+{

This function could use some comments...

Yes, my latest does have those.. let me paste the relevant bit:

+void *___kmalloc_reserve(size_t size, gfp_t flags, int node, void *ip,
+                        struct mem_reserve *res, int *emerg)
+{
+       void *obj;
+       gfp_t gfp;
+
+       /*
+        * Try a regular allocation, when that fails and we're not entitled
+        * to the reserves, fail.
+        */
+       gfp = flags | __GFP_NOMEMALLOC | __GFP_NOWARN;
+       obj = __kmalloc_node_track_caller(size, gfp, node, ip);
+
+       if (obj || !(gfp_to_alloc_flags(flags) & ALLOC_NO_WATERMARKS))
+               goto out;
+
+       /*
+        * If we were given a reserve to charge against, try that.
+        */
+       if (res && !mem_reserve_kmalloc_charge(res, size)) {
+               /*
+                * If we failed to charge and we're not allowed to wait for
+                * it to succeed, bail.
+                */
+               if (!(flags & __GFP_WAIT))
+                       goto out;
+
+               /*
+                * Wait for a successfull charge against the reserve. All
+                * uncharge operations against this reserve will wake us up.
+                */
+               wait_event(res->waitqueue,
+                               mem_reserve_kmalloc_charge(res, size));
+
+               /*
+                * After waiting for it, again try a regular allocation.
+                * Pressure could have lifted during our sleep. If this
+                * succeeds, uncharge the reserve.
+                */
+               obj = __kmalloc_node_track_caller(size, gfp, node, ip);
+               if (obj) {
+                       mem_reserve_kmalloc_charge(res, -size);
+                       goto out;
+               }
+       }
+
+       /*
+        * Regular allocation failed, and we've successfully charged our
+        * requested usage against the reserve. Do the emergency allocation.
+        */
+       obj = __kmalloc_node_track_caller(size, flags, node, ip);
+       WARN_ON(!obj);
+       if (emerg)
+               *emerg |= 1;
+
+out:
+       return obj;
+}

Heh, indeed, looks much better :-).

quoted

quoted

+	void *obj;
+	gfp_t gfp;
+
+	gfp = flags | __GFP_NOMEMALLOC | __GFP_NOWARN;
+	obj = __kmalloc_node_track_caller(size, gfp, node, ip);
+
+	if (obj || !(gfp_to_alloc_flags(flags) & ALLOC_NO_WATERMARKS))
+		goto out;
+
+	if (res && !mem_reserve_kmalloc_charge(res, size)) {
+		if (!(flags & __GFP_WAIT))
+			goto out;
+
+		wait_event(res->waitqueue,
+				mem_reserve_kmalloc_charge(res, size));
+
+		obj = __kmalloc_node_track_caller(size, gfp, node, ip);
+		if (obj) {
+			mem_reserve_kmalloc_charge(res, -size);

Why do we discharge here?

because a regular allocation succeeded.

quoted

quoted

+			goto out;
+		}

If the allocation fails, we try again (but nothing has changed, right?).
Why?

Note the different allocation flags for the two allocations.

Uhm, yeah. I missed that.

quoted

quoted

+	}
+
+	obj = __kmalloc_node_track_caller(size, flags, node, ip);
+	WARN_ON(!obj);

Why don't we discharge from the reserve here if !obj?

Well, this allocation should never fail:
  - we reserved memory
  - we accounted/throttle its usage

Thus this allocation should always succeed.

But if it *does* fail, it doesn't help that we mess up the reservation
counts, no?

Right, I guess we could just rename ksize() to something else then and
keep it internal to mm/.

quoted

quoted

+	/*
+	 * ksize gives the full allocated size vs the requested size we used to
+	 * charge; however since we round up to the nearest power of two, this
+	 * should all work nicely.
+	 */
+	mem_reserve_kmalloc_charge(res, -size);
+}