Thread (8 messages) 8 messages, 4 authors, 2025-10-15

Re: [PATCH v5] memcg: expose socket memory pressure in a cgroup

From: Daniel Sedlak <hidden>
Date: 2025-10-13 14:31:03
Also in: cgroups, linux-mm

Possibly related (same subject, not in this thread)

On 10/9/25 9:02 PM, Roman Gushchin wrote:
Shakeel Butt [off-list ref] writes:
quoted
On Thu, Oct 09, 2025 at 10:58:51AM -0700, Roman Gushchin wrote:
quoted
Shakeel Butt [off-list ref] writes:
quoted
On Thu, Oct 09, 2025 at 08:32:27AM -0700, Roman Gushchin wrote:
quoted
Daniel Sedlak [off-list ref] writes:
quoted
Hi Roman,

On 10/8/25 8:58 PM, Roman Gushchin wrote:
quoted
quoted
This patch exposes a new file for each cgroup in sysfs which is a
read-only single value file showing how many microseconds this cgroup
contributed to throttling the throughput of network sockets. The file is
accessible in the following path.

    /sys/fs/cgroup/**/<cgroup name>/memory.net.throttled_usec
Hi Daniel!
How this value is going to be used? In other words, do you need an
exact number or something like memory.events::net_throttled would be
enough for your case?
Just incrementing a counter each time the vmpressure() happens IMO
provides bad semantics of what is actually happening, because it can
hide important details, mainly the _time_ for how long the network
traffic was slowed down.

For example, when memory.events::net_throttled=1000, it can mean that
the network was slowed down for 1 second or 1000 seconds or something
between, and the memory.net.throttled_usec proposed by this patch
disambiguates it.

In addition, v1/v2 of this series started that way, then from v3 we
rewrote it to calculate the duration instead, which proved to be
better information for debugging, as it is easier to understand
implications.
But how are you planning to use this information? Is this just
"networking is under pressure for non-trivial amount of time ->
raise the memcg limit" or something more complicated?
We plan to use it mostly for observability purposes and to better 
understand which traffic patterns affect the socket pressure the most 
(so we can try to fix/delay/improve it). We do not know how commonly 
this issue appears in other deployments, but in our deployment, many of 
servers were affected by this slowdown, which varied in terms of 
hardware and software configuration. Currently, it is very hard to 
detect if the socket is under pressure without using tools like 
bpftrace, so we would like to expose this metric in a more accessible 
way. So in the end, we do not really care in which file this "socket 
pressure happened" notification will be stored.
quoted
quoted
quoted
quoted
I totally get it from the debugging perspective, but not sure about
usefulness of it as a permanent metric. This is why I'm asking if there
are lighter alternatives, e.g. memory.events or maybe even tracepoints.
If the combination of memory.events(.local) and tracepoint hook(s) is 
okay with you(?), we can use that and export the same information as in 
the current patch version. We can incorporate that into the next version.

Also, would it be possible to make the socket pressure signal 
configurable, e.g., allowing it to be configured via sysctl or per 
cgroup not to trigger the socket pressure signal? I cannot find the 
reasoning why this throttling cannot (maybe it can) be opt-out.
quoted
quoted
quoted
I also have a very similar opinion that if we expose the current
implementation detail through a stable interface, we might get stuck
with this implementation and I want to change this in future.

Coming back to what information should we expose that will be helpful
for Daniel & Matyas and will be beneficial in general. After giving some
thought, I think the time "network was slowed down" or more specifically
time window when mem_cgroup_sk_under_memory_pressure() returns true
might not be that useful without the actual network activity. Basically
if no one is calling mem_cgroup_sk_under_memory_pressure() and doing
some actions, the time window is not that useful.

How about we track the actions taken by the callers of
mem_cgroup_sk_under_memory_pressure()? Basically if network stack
reduces the buffer size or whatever the other actions it may take when
mem_cgroup_sk_under_memory_pressure() returns, tracking those actions
is what I think is needed here, at least for the debugging use-case.
I am not against it, but I feel that conveying those tracked actions (or 
how to represent them) to the user will be much harder. Are there 
already existing APIs to push this information to the user?

Thanks!
Daniel.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help