[PATCH V4,0/3] mm: compaction: proactive compaction trigger by user

These patches support triggering of proactive compaction by user on write
to the /proc/sys/vm/compaction_proactiveness.

Changes in V4:
  -- Changed the code as the 'proactive_defer' counter is removed.
  -- No changes in the logic of triggering the proactive compaction.
  -- Removed the 'proactive_defer' counter.

Changes in V3:
  -- Fixed review comments from Vlastimil and others.
  -- Fixed wake up logic when compaction_proactiveness is zero.
  -- https://lore.kernel.org/patchwork/patch/1438211/

Changes in V2:
  -- remove /proc/../proactive_compact_memory interface trigger for proactive compaction
  -- Intention is same that add a way to trigger proactive compaction by user.
  -- https://lore.kernel.org/patchwork/patch/1431283/

Changes in V1:
  -- Created the new /proc/sys/vm/proactive_compact_memory in
     interface to trigger proactive compaction from user 
  -- https://lore.kernel.org/lkml/1619098678-8501-1-git-send-email-charante@codeaurora.org/


Charan Teja Reddy (3):
  mm: compaction:  optimize proactive compaction deferrals
  mm: compaction: support triggering of proactive compaction by user
  mm: compaction: fix wakeup logic of proactive compaction

 Documentation/admin-guide/sysctl/vm.rst |  3 +-
 include/linux/compaction.h              |  2 ++
 include/linux/mmzone.h                  |  1 +
 kernel/sysctl.c                         |  2 +-
 mm/compaction.c                         | 61 +++++++++++++++++++++++++++------
 5 files changed, 56 insertions(+), 13 deletions(-)

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of the Code Aurora Forum, hosted by The Linux Foundation

The proactive compaction[1] gets triggered for every 500msec and run
compaction on the node for COMPACTION_HPAGE_ORDER (usually order-9)
pages based on the value set to sysctl.compaction_proactiveness.
Triggering the compaction for every 500msec in search of
COMPACTION_HPAGE_ORDER pages is not needed for all applications,
especially on the embedded system usecases which may have few MB's of
RAM. Enabling the proactive compaction in its state will endup in
running almost always on such systems.

Other side, proactive compaction can still be very much useful for
getting a set of higher order pages in some controllable
manner(controlled by using the sysctl.compaction_proactiveness). Thus on
systems where enabling the proactive compaction always may proove not
required, can trigger the same from user space on write to its sysctl
interface. As an example, say app launcher decide to launch the memory
heavy application which can be launched fast if it gets more higher
order pages thus launcher can prepare the system in advance by
triggering the proactive compaction from userspace.

This triggering of proactive compaction is done on a write to
sysctl.compaction_proactiveness by user.

[1]https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=facdaa917c4d5a376d09d25865f5a863f906234a

Signed-off-by: Charan Teja Reddy <redacted>
---
 Changes in V4:
	-- Changed the code as the 'proactive_defer' counter is removed.
	-- No changes in the logic of triggering the proactive compaction.

 Changes in V3:
        -- Fixed review comments from Valstimil and others.
        -- https://lore.kernel.org/patchwork/patch/1438211/

 Changes in V2:
	-- remove /proc/../proactive_compact_memory interface trigger for proactive compaction
        -- Intention is same that add a way to trigger proactive compaction by user.
        -- https://lore.kernel.org/patchwork/patch/1431283/

 changes in V1:
	-- Created the new /proc/sys/vm/proactive_compact_memory in
	   interface to trigger proactive compaction from user 
        -- https://lore.kernel.org/lkml/1619098678-8501-1-git-send-email-charante@codeaurora.org/

 Documentation/admin-guide/sysctl/vm.rst |  3 ++-
 include/linux/compaction.h              |  2 ++
 include/linux/mmzone.h                  |  1 +
 kernel/sysctl.c                         |  2 +-
 mm/compaction.c                         | 32 ++++++++++++++++++++++++++++++--
 5 files changed, 36 insertions(+), 4 deletions(-)

diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst
index 586cd4b..5e8097d 100644
--- a/Documentation/admin-guide/sysctl/vm.rst
+++ b/Documentation/admin-guide/sysctl/vm.rst

@@ -126,7 +126,8 @@ compaction_proactiveness
 
 This tunable takes a value in the range [0, 100] with a default value of
 20. This tunable determines how aggressively compaction is done in the
-background. Setting it to 0 disables proactive compaction.
+background. On write of non zero value to this tunable will immediately
+trigger the proactive compaction. Setting it to 0 disables proactive compaction.
 
 Note that compaction has a non-trivial system-wide impact as pages
 belonging to different processes are moved around, which could also lead

diff --git a/include/linux/compaction.h b/include/linux/compaction.h
index 4221888..04d5d9f 100644
--- a/include/linux/compaction.h
+++ b/include/linux/compaction.h

@@ -84,6 +84,8 @@ static inline unsigned long compact_gap(unsigned int order)
 extern unsigned int sysctl_compaction_proactiveness;
 extern int sysctl_compaction_handler(struct ctl_table *table, int write,
 			void *buffer, size_t *length, loff_t *ppos);
+extern int compaction_proactiveness_sysctl_handler(struct ctl_table *table,
+		int write, void *buffer, size_t *length, loff_t *ppos);
 extern int sysctl_extfrag_threshold;
 extern int sysctl_compact_unevictable_allowed;
 

diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index 0d53eba..9455809 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h

@@ -815,6 +815,7 @@ typedef struct pglist_data {
 	enum zone_type kcompactd_highest_zoneidx;
 	wait_queue_head_t kcompactd_wait;
 	struct task_struct *kcompactd;
+	bool proactive_compact_trigger;
 #endif
 	/*
 	 * This is a per-node reserve of pages that are not available

diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index d4a78e0..85b473e 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c

@@ -2859,7 +2859,7 @@ static struct ctl_table vm_table[] = {
 		.data		= &sysctl_compaction_proactiveness,
 		.maxlen		= sizeof(sysctl_compaction_proactiveness),
 		.mode		= 0644,
-		.proc_handler	= proc_dointvec_minmax,
+		.proc_handler	= compaction_proactiveness_sysctl_handler,
 		.extra1		= SYSCTL_ZERO,
 		.extra2		= &one_hundred,
 	},

diff --git a/mm/compaction.c b/mm/compaction.c
index bfbcb97..7672be3 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c

@@ -2708,6 +2708,30 @@ static void compact_nodes(void)
  */
 unsigned int __read_mostly sysctl_compaction_proactiveness = 20;
 
+int compaction_proactiveness_sysctl_handler(struct ctl_table *table, int write,
+		void *buffer, size_t *length, loff_t *ppos)
+{
+	int rc, nid;
+
+	rc = proc_dointvec_minmax(table, write, buffer, length, ppos);
+	if (rc)
+		return rc;
+
+	if (write && sysctl_compaction_proactiveness) {
+		for_each_online_node(nid) {
+			pg_data_t *pgdat = NODE_DATA(nid);
+
+			if (pgdat->proactive_compact_trigger)
+				continue;
+
+			pgdat->proactive_compact_trigger = true;
+			wake_up_interruptible(&pgdat->kcompactd_wait);
+		}
+	}
+
+	return 0;
+}
+
 /*
  * This is the entry point for compacting all nodes via
  * /proc/sys/vm/compact_memory

@@ -2752,7 +2776,8 @@ void compaction_unregister_node(struct node *node)
 
 static inline bool kcompactd_work_requested(pg_data_t *pgdat)
 {
-	return pgdat->kcompactd_max_order > 0 || kthread_should_stop();
+	return pgdat->kcompactd_max_order > 0 || kthread_should_stop() ||
+		pgdat->proactive_compact_trigger;
 }
 
 static bool kcompactd_node_suitable(pg_data_t *pgdat)

@@ -2905,7 +2930,8 @@ static int kcompactd(void *p)
 
 		trace_mm_compaction_kcompactd_sleep(pgdat->node_id);
 		if (wait_event_freezable_timeout(pgdat->kcompactd_wait,
-			kcompactd_work_requested(pgdat), timeout)) {
+			kcompactd_work_requested(pgdat), timeout) &&
+			!pgdat->proactive_compact_trigger) {
 
 			psi_memstall_enter(&pflags);
 			kcompactd_do_work(pgdat);

@@ -2940,6 +2966,8 @@ static int kcompactd(void *p)
 				timeout =
 				   default_timeout << COMPACT_MAX_DEFER_SHIFT;
 		}
+		if (unlikely(pgdat->proactive_compact_trigger))
+			pgdat->proactive_compact_trigger = false;
 	}
 
 	return 0;

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of the Code Aurora Forum, hosted by The Linux Foundation

When fragmentation score didn't go down across the proactive compaction
i.e. when no progress is made, next wake up for proactive compaction is
deferred for 1 << COMPACT_MAX_DEFER_SHIFT(=6) times, with each wakeup
interval of HPAGE_FRAG_CHECK_INTERVAL_MSEC(=500). In each of this
wakeup, it just decrement 'proactive_defer' counter and goes sleep i.e.
it is getting woken to just decrement a counter. The same deferral time
can also achieved by simply doing the HPAGE_FRAG_CHECK_INTERVAL_MSEC <<
COMPACT_MAX_DEFER_SHIFT thus unnecessary wakeup of kcompact thread is
avoided thus also removes the need of 'proactive_defer' thread counter.

Suggested-by: Vlastimil Babka <redacted>
Signed-off-by: Charan Teja Reddy <redacted>
---
 Changes in V4:
  -- Removed the 'proactive_defer' thread counter by optimizing proactive
     compaction deferrals.
  -- Changes from V1 through V3 doesn't exist.

 mm/compaction.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index 84fde27..bfbcb97 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c

@@ -2887,7 +2887,8 @@ static int kcompactd(void *p)
 {
 	pg_data_t *pgdat = (pg_data_t *)p;
 	struct task_struct *tsk = current;
-	unsigned int proactive_defer = 0;
+	long default_timeout = msecs_to_jiffies(HPAGE_FRAG_CHECK_INTERVAL_MSEC);
+	long timeout = default_timeout;
 
 	const struct cpumask *cpumask = cpumask_of_node(pgdat->node_id);
 

@@ -2904,23 +2905,30 @@ static int kcompactd(void *p)
 
 		trace_mm_compaction_kcompactd_sleep(pgdat->node_id);
 		if (wait_event_freezable_timeout(pgdat->kcompactd_wait,
-			kcompactd_work_requested(pgdat),
-			msecs_to_jiffies(HPAGE_FRAG_CHECK_INTERVAL_MSEC))) {
+			kcompactd_work_requested(pgdat), timeout)) {
 
 			psi_memstall_enter(&pflags);
 			kcompactd_do_work(pgdat);
 			psi_memstall_leave(&pflags);
+			/*
+			 * Reset the timeout value. The defer timeout by
+			 * proactive compaction can effectively lost
+			 * here but that is fine as the condition of the
+			 * zone changed substantionally and carrying on
+			 * with the previous defer is not useful.
+			 */
+			timeout = default_timeout;
 			continue;
 		}
 
-		/* kcompactd wait timeout */
+		/*
+		 * Start the proactive work with default timeout. Based
+		 * on the fragmentation score, this timeout is updated.
+		 */
+		timeout = default_timeout;
 		if (should_proactive_compact_node(pgdat)) {
 			unsigned int prev_score, score;
 
-			if (proactive_defer) {
-				proactive_defer--;
-				continue;
-			}
 			prev_score = fragmentation_score_node(pgdat);
 			proactive_compact_node(pgdat);
 			score = fragmentation_score_node(pgdat);

@@ -2928,8 +2936,9 @@ static int kcompactd(void *p)
 			 * Defer proactive compaction if the fragmentation
 			 * score did not go down i.e. no progress made.
 			 */
-			proactive_defer = score < prev_score ?
-					0 : 1 << COMPACT_MAX_DEFER_SHIFT;
+			if (unlikely(score >= prev_score))
+				timeout =
+				   default_timeout << COMPACT_MAX_DEFER_SHIFT;
 		}
 	}
 

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of the Code Aurora Forum, hosted by The Linux Foundation

Currently, proactive compaction tries to get triggered for every
HPAGE_FRAG_CHECK_INTERVAL_MSEC(=500msec) even when proactive compaction
is disabled with sysctl.compaction_proactiveness = 0. This results in
kcompactd thread wakes up and goes to sleep for every 500msec with out
the need of doing proactive compaction. Though this doesn't have any
overhead, few cpu cycles can be saved by avoid of waking up kcompactd
thread for proactive compaction when it is disabled.

Signed-off-by: Charan Teja Reddy <redacted>
---
 -- Changes in V4:
       o No functional changes from V3.
 -- Changes in V3:
       o Fix wake up logic in proactive compaction.
 -- Changes in V2 through V2 doesn't exist.

 mm/compaction.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/compaction.c b/mm/compaction.c
index 7672be3..44a232a 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c

@@ -2928,6 +2928,8 @@ static int kcompactd(void *p)
 	while (!kthread_should_stop()) {
 		unsigned long pflags;
 
+		if (!sysctl_compaction_proactiveness)
+			timeout = MAX_SCHEDULE_TIMEOUT;
 		trace_mm_compaction_kcompactd_sleep(pgdat->node_id);
 		if (wait_event_freezable_timeout(pgdat->kcompactd_wait,
 			kcompactd_work_requested(pgdat), timeout) &&

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of the Code Aurora Forum, hosted by The Linux Foundation

A gentle ping to have your valuable comments.

Thanks,
Charan

On 6/18/2021 8:48 PM, Charan Teja Reddy wrote:

These patches support triggering of proactive compaction by user on write
to the /proc/sys/vm/compaction_proactiveness.

Changes in V4:
  -- Changed the code as the 'proactive_defer' counter is removed.
  -- No changes in the logic of triggering the proactive compaction.
  -- Removed the 'proactive_defer' counter.

Changes in V3:
  -- Fixed review comments from Vlastimil and others.
  -- Fixed wake up logic when compaction_proactiveness is zero.
  -- https://lore.kernel.org/patchwork/patch/1438211/

Changes in V2:
  -- remove /proc/../proactive_compact_memory interface trigger for proactive compaction
  -- Intention is same that add a way to trigger proactive compaction by user.
  -- https://lore.kernel.org/patchwork/patch/1431283/

Changes in V1:
  -- Created the new /proc/sys/vm/proactive_compact_memory in
     interface to trigger proactive compaction from user 
  -- https://lore.kernel.org/lkml/1619098678-8501-1-git-send-email-charante@codeaurora.org/


Charan Teja Reddy (3):
  mm: compaction:  optimize proactive compaction deferrals
  mm: compaction: support triggering of proactive compaction by user
  mm: compaction: fix wakeup logic of proactive compaction

 Documentation/admin-guide/sysctl/vm.rst |  3 +-
 include/linux/compaction.h              |  2 ++
 include/linux/mmzone.h                  |  1 +
 kernel/sysctl.c                         |  2 +-
 mm/compaction.c                         | 61 +++++++++++++++++++++++++++------
 5 files changed, 56 insertions(+), 13 deletions(-)

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora
Forum, a Linux Foundation Collaborative Project

On Sat, 3 Jul 2021 15:52:10 +0530 Charan Teja Kalla [off-list ref] wrote:

A gentle ping to have your valuable comments.

Can we please have a resend?

The series has two fixes against the current code.  Please separate
that work out from the new feature.  So a 2-patch series to fix the bugs
followed by a single patch to add your new feature.

Thanks Andrew for the reply!!

On 7/16/2021 9:57 AM, Andrew Morton wrote:

On Sat, 3 Jul 2021 15:52:10 +0530 Charan Teja Kalla [off-list ref] wrote:

quoted

A gentle ping to have your valuable comments.

Can we please have a resend?

The series has two fixes against the current code.  Please separate
that work out from the new feature.  So a 2-patch series to fix the bugs
followed by a single patch to add your new feature.

https://lore.kernel.org/patchwork/patch/1448789/ -- Can go as a separate
bug fix.

https://lore.kernel.org/patchwork/patch/1448793/ -- is the second bug
fix which is tightly coupled with the feature of explicitly waking of
kcompactd on the event of change in compaction proactiveness, when it is
sleeping with MAX_SCHEDULE_TIMEOUT.

So, will make the changes with 1 patch bug fix and 2nd patch feature
where the second bug fix also clubbed.

I hope this is fine.

-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora
Forum, a Linux Foundation Collaborative Project