md:When opened /proc/mdstat, increase the refcount of

Started by majianpeng on Feb 26, 2012 · 7 messages from 2 authors · Last reply Feb 27, 2012

Subsystems (2)

Maintainers (3)

open the first message on its own page

Message details and related content
From 07a0eeb62a7bedec3f3312aff24ef62fbb36d820 Mon Sep 17 00:00:00 2001
From: majianpeng <redacted>
Date: Sun, 26 Feb 2012 20:43:05 +0800
Subject: [PATCH] md:When opened /proc/mdstat, increase the refcount of
 md-mod.ko

    If md configured module, polled /proc/mdstat and removed the md-mod
    can incur oops problem.

    Reproduce this bug by following step:
    1:configure md is module and insmod
    2:poll(/proc/mdstat,timeout)
    3:remove md-mod.ko
    4:oops occur

[  343.764057] BUG: unable to handle kernel paging request at
ffffffffa007f9e0
[  343.764101] IP: [<ffffffff8155578c>] _raw_spin_lock_irqsave+0xc/0x30
[  343.764133] PGD 1807067 PUD 180b063 PMD b7b72067 PTE 0
[  343.764167] Oops: 0002 [#1] SMP
[  343.764190] CPU 3
[  343.764205] Modules linked in: ext4 jbd2 crc16 async_pq async_xor xor
async_memcpy async_raid6_recov raid6_pq async_tx sata_sil e1000e mvsas
libsas scsi_transport_sas [last unloaded: md_mod]
[  343.764304]
[  343.764312] Pid: 5454, comm: udisks-daemon Not tainted 3.3.0-rc4+ #13
To Be Filled By O.E.M. To Be Filled By O.E.M./To be filled by O.E.M.
[  343.764366] RIP: 0010:[<ffffffff8155578c>]  [<ffffffff8155578c>]
_raw_spin_lock_irqsave+0xc/0x30
[  343.764403] RSP: 0018:ffff8800a514bab8  EFLAGS: 00010082
[  343.764423] RAX: 0000000000000282 RBX: ffffffffa007f9e0 RCX:
ffff8800ba0051a8
[  343.764448] RDX: 0000000000000100 RSI: ffff8800a514bc48 RDI:
ffffffffa007f9e0
[  343.764473] RBP: ffff8800a514bab8 R08: ffff8800a9bdb630 R09:
0000000000000001
[  343.764499] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8800a514bc48
[  343.764524] R13: ffff8800a514bb88 R14: ffff8800a514be2c R15:
0000000000000000

51 more lines

diff --git a/drivers/md/md.c b/drivers/md/md.c
index ce88755..105c7f9 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -6859,9 +6859,15 @@ static int md_seq_open(struct inode *inode, struct file *file)
 
 	seq = file->private_data;
 	seq->poll_event = atomic_read(&md_event_count);
+	__module_get(THIS_MODULE);
 	return error;
 }
 
+static int md_seq_release(struct inode *inode, struct file *file)
+{
+	module_put(THIS_MODULE);
+	return seq_release_private(inode, file);
+}
 static unsigned int mdstat_poll(struct file *filp, poll_table *wait)
 {
 	struct seq_file *seq = filp->private_data;
@@ -6882,7 +6888,7 @@ static const struct file_operations md_seq_fops = {
 	.open           = md_seq_open,
 	.read           = seq_read,
 	.llseek         = seq_lseek,
-	.release	= seq_release_private,
+	.release	= md_seq_release,
 	.poll		= mdstat_poll,
 };
 
-- 
1.7.5.4

 				
--------------
majianpeng
2012-02-26
Message details and related content
On Sun, 26 Feb 2012 12:55:03 +0800 "majianpeng" [off-list ref] wrote:
quoted
From 07a0eeb62a7bedec3f3312aff24ef62fbb36d820 Mon Sep 17 00:00:00 2001
From: majianpeng <redacted>
Date: Sun, 26 Feb 2012 20:43:05 +0800
Subject: [PATCH] md:When opened /proc/mdstat, increase the refcount of
 md-mod.ko

    If md configured module, polled /proc/mdstat and removed the md-mod
    can incur oops problem.

    Reproduce this bug by following step:
    1:configure md is module and insmod
    2:poll(/proc/mdstat,timeout)
    3:remove md-mod.ko
    4:oops occur

[  343.764057] BUG: unable to handle kernel paging request at
ffffffffa007f9e0
[  343.764101] IP: [<ffffffff8155578c>] _raw_spin_lock_irqsave+0xc/0x30
[  343.764133] PGD 1807067 PUD 180b063 PMD b7b72067 PTE 0
[  343.764167] Oops: 0002 [#1] SMP
[  343.764190] CPU 3
[  343.764205] Modules linked in: ext4 jbd2 crc16 async_pq async_xor xor
async_memcpy async_raid6_recov raid6_pq async_tx sata_sil e1000e mvsas
libsas scsi_transport_sas [last unloaded: md_mod]
[  343.764304]
[  343.764312] Pid: 5454, comm: udisks-daemon Not tainted 3.3.0-rc4+ #13
To Be Filled By O.E.M. To Be Filled By O.E.M./To be filled by O.E.M.
[  343.764366] RIP: 0010:[<ffffffff8155578c>]  [<ffffffff8155578c>]
_raw_spin_lock_irqsave+0xc/0x30
[  343.764403] RSP: 0018:ffff8800a514bab8  EFLAGS: 00010082
[  343.764423] RAX: 0000000000000282 RBX: ffffffffa007f9e0 RCX:
ffff8800ba0051a8
[  343.764448] RDX: 0000000000000100 RSI: ffff8800a514bc48 RDI:
ffffffffa007f9e0
[  343.764473] RBP: ffff8800a514bab8 R08: ffff8800a9bdb630 R09:
0000000000000001
[  343.764499] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8800a514bc48
[  343.764524] R13: ffff8800a514bb88 R14: ffff8800a514be2c R15:
0000000000000000

80 more quoted lines

Thanks for the report and the patch.
However I think the problem should be fixed in procfs.
md_set_fops contains
	.owner		= THIS_MODULE,

and procfs should be able to use this to keep a reference on the module
whenever the file is open.  It seems that it doesn't.
Fixing this in procfs is more general than just fixing it in md, and also it
is likely to be safe.  Having a module take a reference to itself is usually
racy and should be avoided.

Could you see if it can be fixed in procfs?

Thanks,
NeilBrown
Message details and related content
Yes,I know this.But I can't to fix procfs,so using this patch.


2012-02-26 



majianpeng 



发件人: NeilBrown 
发送时间: 2012-02-26  13:27:15 
收件人: majianpeng 
抄送: linux-raid 
主题: Re: md:When opened /proc/mdstat, increase the refcount of 
 
On Sun, 26 Feb 2012 12:55:03 +0800 "majianpeng" [off-list ref] wrote:
quoted
From 07a0eeb62a7bedec3f3312aff24ef62fbb36d820 Mon Sep 17 00:00:00 2001
From: majianpeng <redacted>
Date: Sun, 26 Feb 2012 20:43:05 +0800
Subject: [PATCH] md:When opened /proc/mdstat, increase the refcount of
 md-mod.ko

    If md configured module, polled /proc/mdstat and removed the md-mod
    can incur oops problem.

    Reproduce this bug by following step:
    1:configure md is module and insmod
    2:poll(/proc/mdstat,timeout)
    3:remove md-mod.ko
    4:oops occur

[  343.764057] BUG: unable to handle kernel paging request at
ffffffffa007f9e0
[  343.764101] IP: [<ffffffff8155578c>] _raw_spin_lock_irqsave+0xc/0x30
[  343.764133] PGD 1807067 PUD 180b063 PMD b7b72067 PTE 0
[  343.764167] Oops: 0002 [#1] SMP
[  343.764190] CPU 3
[  343.764205] Modules linked in: ext4 jbd2 crc16 async_pq async_xor xor
async_memcpy async_raid6_recov raid6_pq async_tx sata_sil e1000e mvsas
libsas scsi_transport_sas [last unloaded: md_mod]
[  343.764304]
[  343.764312] Pid: 5454, comm: udisks-daemon Not tainted 3.3.0-rc4+ #13
To Be Filled By O.E.M. To Be Filled By O.E.M./To be filled by O.E.M.
[  343.764366] RIP: 0010:[<ffffffff8155578c>]  [<ffffffff8155578c>]
_raw_spin_lock_irqsave+0xc/0x30
[  343.764403] RSP: 0018:ffff8800a514bab8  EFLAGS: 00010082
[  343.764423] RAX: 0000000000000282 RBX: ffffffffa007f9e0 RCX:
ffff8800ba0051a8
[  343.764448] RDX: 0000000000000100 RSI: ffff8800a514bc48 RDI:
ffffffffa007f9e0
[  343.764473] RBP: ffff8800a514bab8 R08: ffff8800a9bdb630 R09:
0000000000000001
[  343.764499] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8800a514bc48
[  343.764524] R13: ffff8800a514bb88 R14: ffff8800a514be2c R15:
0000000000000000

80 more quoted lines

Thanks for the report and the patch.
However I think the problem should be fixed in procfs.
md_set_fops contains
.owner = THIS_MODULE,
and procfs should be able to use this to keep a reference on the module
whenever the file is open.  It seems that it doesn't.
Fixing this in procfs is more general than just fixing it in md, and also it
is likely to be safe.  Having a module take a reference to itself is usually
racy and should be avoided.
Could you see if it can be fixed in procfs?
Thanks,
NeilBrown
Message details and related content
I wrote a letter to maintainer of procfs. He's answer is follow:
No.  Read remove_proc_entry(), especially the part under ->pde_unload_lock.
The whole damn point of that stuff is that opened file on procfs does *not*
pin the module down; IO in progress does, but that's it.
You can't deadlock rmmod foobar </proc/crap/foobar; with your patch we'll
be back to a pile of deadlocks in there.
IOW, NAK.
------------------				 
majianpeng
2012-02-26

-------------------------------------------------------------
发件人:NeilBrown
发送日期:2012-02-26 13:27:15
收件人:majianpeng
抄送:linux-raid
主题:Re: md:When opened /proc/mdstat, increase the refcount of

On Sun, 26 Feb 2012 12:55:03 +0800 "majianpeng" [off-list ref] wrote:
quoted
From 07a0eeb62a7bedec3f3312aff24ef62fbb36d820 Mon Sep 17 00:00:00 2001
From: majianpeng <redacted>
Date: Sun, 26 Feb 2012 20:43:05 +0800
Subject: [PATCH] md:When opened /proc/mdstat, increase the refcount of
 md-mod.ko

    If md configured module, polled /proc/mdstat and removed the md-mod
    can incur oops problem.

    Reproduce this bug by following step:
    1:configure md is module and insmod
    2:poll(/proc/mdstat,timeout)
    3:remove md-mod.ko
    4:oops occur

[  343.764057] BUG: unable to handle kernel paging request at
ffffffffa007f9e0
[  343.764101] IP: [<ffffffff8155578c>] _raw_spin_lock_irqsave+0xc/0x30
[  343.764133] PGD 1807067 PUD 180b063 PMD b7b72067 PTE 0
[  343.764167] Oops: 0002 [#1] SMP
[  343.764190] CPU 3
[  343.764205] Modules linked in: ext4 jbd2 crc16 async_pq async_xor xor
async_memcpy async_raid6_recov raid6_pq async_tx sata_sil e1000e mvsas
libsas scsi_transport_sas [last unloaded: md_mod]
[  343.764304]
[  343.764312] Pid: 5454, comm: udisks-daemon Not tainted 3.3.0-rc4+ #13
To Be Filled By O.E.M. To Be Filled By O.E.M./To be filled by O.E.M.
[  343.764366] RIP: 0010:[<ffffffff8155578c>]  [<ffffffff8155578c>]
_raw_spin_lock_irqsave+0xc/0x30
[  343.764403] RSP: 0018:ffff8800a514bab8  EFLAGS: 00010082
[  343.764423] RAX: 0000000000000282 RBX: ffffffffa007f9e0 RCX:
ffff8800ba0051a8
[  343.764448] RDX: 0000000000000100 RSI: ffff8800a514bc48 RDI:
ffffffffa007f9e0
[  343.764473] RBP: ffff8800a514bab8 R08: ffff8800a9bdb630 R09:
0000000000000001
[  343.764499] R10: 0000000000000000 R11: 0000000000000000 R12:
ffff8800a514bc48
[  343.764524] R13: ffff8800a514bb88 R14: ffff8800a514be2c R15:
0000000000000000

80 more quoted lines

Thanks for the report and the patch.
However I think the problem should be fixed in procfs.
md_set_fops contains
	.owner		= THIS_MODULE,

and procfs should be able to use this to keep a reference on the module
whenever the file is open.  It seems that it doesn't.
Fixing this in procfs is more general than just fixing it in md, and also it
is likely to be safe.  Having a module take a reference to itself is usually
racy and should be avoided.

Could you see if it can be fixed in procfs?

Thanks,
NeilBrown
Message details and related content
On Sun, 26 Feb 2012 13:39:57 +0800 "majianpeng" [off-list ref] wrote:
Yes,I know this.But I can't to fix procfs,so using this patch.
I wonder if this will fix it.

diff --git a/fs/proc/inode.c b/fs/proc/inode.c
index 7737c54..fc89842 100644
--- a/fs/proc/inode.c
+++ b/fs/proc/inode.c
@@ -449,7 +449,8 @@ struct inode *proc_get_inode(struct super_block *sb, struct proc_dir_entry *de)
 		if (de->proc_iops)
 			inode->i_op = de->proc_iops;
 		if (de->proc_fops) {
-			if (S_ISREG(inode->i_mode)) {
+			if (S_ISREG(inode->i_mode) &&
+			    !de->proc_fops->owner) {
 #ifdef CONFIG_COMPAT
 				if (!de->proc_fops->compat_ioctl)
 					inode->i_fop =

however I don't really understand the point of proc_ref_file_ops, so maybe
I'm missing something.

NeilBrown
Message details and related content
On Sun, 26 Feb 2012 16:33:25 +0800 "majianpeng" [off-list ref] wrote:
I wrote a letter to maintainer of procfs. He's answer is follow:
quoted
No.  Read remove_proc_entry(), especially the part under ->pde_unload_lock.
The whole damn point of that stuff is that opened file on procfs does *not*
pin the module down; IO in progress does, but that's it.
You can't deadlock rmmod foobar </proc/crap/foobar; with your patch we'll
be back to a pile of deadlocks in there.
IOW, NAK.
Hmmm... that sounds like Al Viro.

I see the point of that code now.  And your patch would be open to the same
problem wouldn't it. i.e.
    rmmod md_mod < /proc/mdstat

would deadlock??

So we still need to find a fix that actually works correctly.

That probably means moving the call to 'poll_wait' into procfs code, and using
a wait_queue_head which is managed by procfs.

So maybe:
 - put a wait_queue_head in 'struct proc_dir_entry'
 - have proc_reg_poll call poll_wait passing that wait_queue_head
 - write a function e.g. proc_poll_wake which calls wake_up on that
   wait_queue_head
 - have md.c save the return value from proc_create, and call the above
   function instead of calling "wake_up(&md_event_waiters)"

That should work.

NeilBrown
Message details and related content
my patch is:
---
 fs/proc/inode.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/fs/proc/inode.c b/fs/proc/inode.c
index 84fd323..78dbb03 100644
--- a/fs/proc/inode.c
+++ b/fs/proc/inode.c
@@ -317,6 +317,7 @@ static int proc_reg_open(struct inode *inode, struct file *file)
 	int (*open)(struct inode *, struct file *);
 	int (*release)(struct inode *, struct file *);
 	struct pde_opener *pdeo;
+	const struct file_operations *fops;
 
 	/*
 	 * What for, you ask? Well, we can have open, rmmod, remove_proc_entry
@@ -339,7 +340,12 @@ static int proc_reg_open(struct inode *inode, struct file *file)
 		return -ENOENT;
 	}
 	pde->pde_users++;
-	open = pde->proc_fops->open;
+	/*
+	 *If proc_fops defined in module and used .owner = THIS_MODULE,so open must
+	 *get a reference of module.
+	 */
+	fops = fops_get(pde->proc_fops);
+	open = fops->open;
 	release = pde->proc_fops->release;
 	spin_unlock(&pde->pde_unload_lock);
 
@@ -354,8 +360,10 @@ static int proc_reg_open(struct inode *inode, struct file *file)
 		/* Strictly for "too late" ->release in proc_reg_release(). */
 		pdeo->release = release;
 		list_add(&pdeo->lh, &pde->pde_openers);
-	} else
+	} else	{
+		fops_put(pde->proc_fops);
 		kfree(pdeo);
+	}
 	__pde_users_dec(pde);
 	spin_unlock(&pde->pde_unload_lock);
 	return rv;
@@ -402,6 +410,7 @@ static int proc_reg_release(struct inode *inode, struct file *file)
 	}
 	pde->pde_users++;
 	release = pde->proc_fops->release;
+	fops_put(pde->proc_fops);
 	if (pdeo) {
 		list_del(&pdeo->lh);
 		kfree(pdeo);
I tested ok,but Al Viro said deadlock, I did not tested.

------------------				 
majianpeng
2012-02-27

-------------------------------------------------------------
发件人:NeilBrown
发送日期:2012-02-26 19:01:32
收件人:majianpeng
抄送:linux-raid
主题:Re: md:When opened /proc/mdstat, increase the refcount of

On Sun, 26 Feb 2012 16:33:25 +0800 "majianpeng" [off-list ref] wrote:
I wrote a letter to maintainer of procfs. He's answer is follow:
quoted
No.  Read remove_proc_entry(), especially the part under ->pde_unload_lock.
The whole damn point of that stuff is that opened file on procfs does *not*
pin the module down; IO in progress does, but that's it.
You can't deadlock rmmod foobar </proc/crap/foobar; with your patch we'll
be back to a pile of deadlocks in there.
IOW, NAK.
Hmmm... that sounds like Al Viro.

I see the point of that code now.  And your patch would be open to the same
problem wouldn't it. i.e.
    rmmod md_mod < /proc/mdstat

would deadlock??

So we still need to find a fix that actually works correctly.

That probably means moving the call to 'poll_wait' into procfs code, and using
a wait_queue_head which is managed by procfs.

So maybe:
 - put a wait_queue_head in 'struct proc_dir_entry'
 - have proc_reg_poll call poll_wait passing that wait_queue_head
 - write a function e.g. proc_poll_wake which calls wake_up on that
   wait_queue_head
 - have md.c save the return value from proc_create, and call the above
   function instead of calling "wake_up(&md_event_waiters)"

That should work.

NeilBrown
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help