On Mon, 27 Oct 2014, Chris Friesen wrote:

There are details (stack traces, etc.) in the first message in the thread:
http://www.spinics.net/lists/linux-rt-users/msg12261.html


Originally we had thought that nfsd might have been implicated somehow, but it
seems like it was just a trigger (possibly by increasing the rate of sync
I/O).

In the interest of full disclosure I should point out that we're using a
modified kernel so there is a chance that we have introduced the problem
ourselves.  That said, we have not made significant changes to either ext4 or
jbd2.  (Just a couple of minor cherry-picked bugfixes.)

I don't think it's an ext4/jdb2 problem.
 

The relevant code paths are:

Journal commit.  The important thing here is that we set the PG_writeback on a
page, put the jbd2 journal head on BJ_Shadow list, then sleep waiting for page
writeback complete.  If the page writeback never completes, then the journal
head never comes off the BJ_Shadow list.

And that's what you need to investigate.

The rest of the threads being stuck waiting for the journal writeback
or inode->sem are just the consequence of it and have nothing to do
with the root cause of the problem.

ftrace with the block/writeback/jdb/ext4/sched tracepoints enabled
should provide a first insight into the issue.

Thanks,

	tglx