Re: OOM killer changes
From: Michal Hocko <hidden>
Date: 2016-08-01 20:09:30
Possibly related (same subject, not in this thread)
- 2016-08-01 · Re: OOM killer changes · Michal Hocko <hidden>
On Mon 01-08-16 12:52:40, Ralf-Peter Rohbeck wrote:
On 01.08.2016 12:43, Michal Hocko wrote:quoted
On Mon 01-08-16 12:35:51, Ralf-Peter Rohbeck wrote:quoted
On 01.08.2016 12:26, Michal Hocko wrote:[...]quoted
quoted
the amount of dirty pages is much smaller as well as the anonymous memory. The biggest portion seems to be in the page cache. The memoryThe page cache will always be full if I'm writing at full steam to multiple drives, no?Yes, the memory full of page cache is not unusual. The large portion of that memory being dirty/writeback can be a problem. That is why we have a dirty memory throttling which slows down (throttles) writers to keep the amount reasonable. What is your dirty throttling setup? $ grep . /proc/sys/vm/dirty* and what is your storage setup?root@fs:~# grep . /proc/sys/vm/dirty* /proc/sys/vm/dirty_background_bytes:0 /proc/sys/vm/dirty_background_ratio:10 /proc/sys/vm/dirty_bytes:0 /proc/sys/vm/dirty_expire_centisecs:3000 /proc/sys/vm/dirty_ratio:20
With your 8G of RAM this can be quite a lot of dirty data at once. Is your storage able to write that back in a reasonable time? I mean this shouldn't cause the OOM killer but it can lead to some unexpected stalls especially when there are a lot of writers AFAIU. dirty_bytes knob should help to define a better cap.
/proc/sys/vm/dirtytime_expire_seconds:43200 /proc/sys/vm/dirty_writeback_centisecs:500 Storage setup: root@fs:~# lsscsi [0:2:0:0] disk LSI MR9271-8iCC 3.29 /dev/sda [0:2:1:0] disk LSI MR9271-8iCC 3.29 /dev/sdb [9:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdf [10:0:0:0] disk Seagate Backup+ Desk 050B /dev/sdc [11:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdd [12:0:0:0] disk Seagate Backup+ Desk 050B /dev/sde [13:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdg [14:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdl [15:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdh [16:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdi [17:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdm [18:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdj [19:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdk sda is a 6x 1TB RAID5 and sdb is a single 480GB SSD, both on a MegaRAID controller. The rest are 4TB USB drives that I'm experimenting with.
Which devices did you write when hitting the OOM killer? -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>