skb defer free casues a lot of memory not released in hugepages

Hello:
In my virtualization scenario, physical host enable hugepages feature and share hugepages to the virtual machine running on it.
A large amount of data is exchanged between the host and the vm through TCP local communication.Later, the vm is no longer needed and is destroyed, but the hugepages shared to the vm was not fully released back to the host. 
Tracing showed that the hugepages are occupied by skbs in host due to commit 68822bdf76f10 ("net: generalize skb freeing deferral to per-cpu lists").
it will remain some skbs in cpu skb_defer_list to defer free, it seems that may cause infinite delay if there is no triggering for net_rx_action(). 
A large amount of memory is not released because there are many CPUs and hugepages feature. Host can't recycle memory fast enough will affect the subsequent VM hugepages allocation.
Maybe we need a general method to release skb immediately or a timeout mechanism for skbs in defer list to release?
Best wishes!