Hi,
A real workload has since run into the stall this series is about.
Mooncake, the KV cache store used with vLLM and SGLang, backs each
rank's host segment with MADV_HUGEPAGE memory and registers it with
the EFA NIC, which pins it long-term. Its commit message [1] says:
"under the default local NUMA policy every THP fault on a full node
runs direct compaction before falling back. With 8 ranks x 190GB
this made registration take ~38 minutes."
They worked around it by binding the segment MPOL_INTERLEAVE across
all allowed nodes, so no node fills before the others. That gives up
node-local placement, and a task that sets its own policy still gets
the stall.
[1] https://github.com/kvcache-ai/Mooncake/commit/c3fa13ecaf1038df933dcb784ff8ffe026381100
Thanks,
Qiliang