From: Dan Williams <hidden> Date: 2014-01-29 03:06:26
On Sun, Jan 26, 2014 at 4:03 PM, Francois Romieu [off-list ref] wrote:
Sander Eikelenboom [off-list ref] :
[...]
quoted
I have got a regression with a 3.14-mw kernel (last commit is 4ba9920e5e9c0e16b5ed24292d45322907bb9035):
It looks like it's related to the rtl8169 ...
--
Sander
Jan 26 11:36:26 serveerstertje kernel: [ 89.105537] ------------[ cut here ]------------
Jan 26 11:36:26 serveerstertje kernel: [ 89.116779] WARNING: CPU: 0 PID: 0 at lib/dma-debug.c:491 add_dma_entry+0x103/0x130()
Jan 26 11:36:26 serveerstertje kernel: [ 89.128148] DMA-API: exceeded 7 overlapping mappings of pfn 55ebe
Jan 26 11:36:26 serveerstertje kernel: [ 89.139397] Modules linked in:
Jan 26 11:36:26 serveerstertje kernel: [ 89.150535] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.13.0-20140125-mw-pcireset+ #1
Jan 26 11:36:26 serveerstertje kernel: [ 89.161784] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640) , BIOS V1.8B1 09/13/2010
Jan 26 11:36:26 serveerstertje kernel: [ 89.172965] 0000000000000009 ffff88005f603838 ffffffff81acbcfa ffffffff822134e0
Jan 26 11:36:26 serveerstertje kernel: [ 89.184156] ffff88005f603888 ffff88005f603878 ffffffff810bdf62 ffff880000000000
Jan 26 11:36:26 serveerstertje kernel: [ 89.195186] 0000000000055ebe 00000000ffffffef 0000000000000200 ffff8800592ea098
Jan 26 11:36:26 serveerstertje kernel: [ 89.206227] Call Trace:
Jan 26 11:36:26 serveerstertje kernel: [ 89.217027] <IRQ> [<ffffffff81acbcfa>] dump_stack+0x46/0x58
Jan 26 11:36:26 serveerstertje kernel: [ 89.227907] [<ffffffff810bdf62>] warn_slowpath_common+0x82/0xb0
Jan 26 11:36:26 serveerstertje kernel: [ 89.238678] [<ffffffff810be031>] warn_slowpath_fmt+0x41/0x50
Jan 26 11:36:26 serveerstertje kernel: [ 89.249336] [<ffffffff81471c5a>] ? active_pfn_read_overlap+0x3a/0x70
Jan 26 11:36:26 serveerstertje kernel: [ 89.259904] [<ffffffff814729e3>] add_dma_entry+0x103/0x130
Jan 26 11:36:26 serveerstertje kernel: [ 89.270416] [<ffffffff81472de6>] debug_dma_map_page+0x126/0x150
Jan 26 11:36:26 serveerstertje kernel: [ 89.280840] [<ffffffff81714686>] rtl8169_start_xmit+0x216/0xa20
[r8169 and xen stuff]
Dan, I miss the part of the debug code that tells where the mappings were
previously set.
In this case it was a facepalm mistake on my part. The mappings were
not being properly accounted in the last revision of the patch I sent.
I copied you on the fix [1].
--
Dan
[1]: http://marc.info/?l=linux-netdev&m=139096447627032&w=2
Hi Dan / Francois,
Didn't have time to test it before, but the patch doesn't seem to help.
I'm still getting the "DMA-API: exceeded 7 overlapping mappings of pfn 55ebe",
but i see now i forgot to mention i use r8169.use_dac=1 ...
Not using it seems to prevent the warning, but before 3.14 i have never seen this (with r8169.use_dac=1)
--
Sander
Wednesday, January 29, 2014, 4:06:24 AM, you wrote:
On Sun, Jan 26, 2014 at 4:03 PM, Francois Romieu [off-list ref] wrote:
quoted
Sander Eikelenboom [off-list ref] :
[...]
quoted
I have got a regression with a 3.14-mw kernel (last commit is 4ba9920e5e9c0e16b5ed24292d45322907bb9035):
It looks like it's related to the rtl8169 ...
--
Sander
Jan 26 11:36:26 serveerstertje kernel: [ 89.105537] ------------[ cut here ]------------
Jan 26 11:36:26 serveerstertje kernel: [ 89.116779] WARNING: CPU: 0 PID: 0 at lib/dma-debug.c:491 add_dma_entry+0x103/0x130()
Jan 26 11:36:26 serveerstertje kernel: [ 89.128148] DMA-API: exceeded 7 overlapping mappings of pfn 55ebe
Jan 26 11:36:26 serveerstertje kernel: [ 89.139397] Modules linked in:
Jan 26 11:36:26 serveerstertje kernel: [ 89.150535] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.13.0-20140125-mw-pcireset+ #1
Jan 26 11:36:26 serveerstertje kernel: [ 89.161784] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640) , BIOS V1.8B1 09/13/2010
Jan 26 11:36:26 serveerstertje kernel: [ 89.172965] 0000000000000009 ffff88005f603838 ffffffff81acbcfa ffffffff822134e0
Jan 26 11:36:26 serveerstertje kernel: [ 89.184156] ffff88005f603888 ffff88005f603878 ffffffff810bdf62 ffff880000000000
Jan 26 11:36:26 serveerstertje kernel: [ 89.195186] 0000000000055ebe 00000000ffffffef 0000000000000200 ffff8800592ea098
Jan 26 11:36:26 serveerstertje kernel: [ 89.206227] Call Trace:
Jan 26 11:36:26 serveerstertje kernel: [ 89.217027] <IRQ> [<ffffffff81acbcfa>] dump_stack+0x46/0x58
Jan 26 11:36:26 serveerstertje kernel: [ 89.227907] [<ffffffff810bdf62>] warn_slowpath_common+0x82/0xb0
Jan 26 11:36:26 serveerstertje kernel: [ 89.238678] [<ffffffff810be031>] warn_slowpath_fmt+0x41/0x50
Jan 26 11:36:26 serveerstertje kernel: [ 89.249336] [<ffffffff81471c5a>] ? active_pfn_read_overlap+0x3a/0x70
Jan 26 11:36:26 serveerstertje kernel: [ 89.259904] [<ffffffff814729e3>] add_dma_entry+0x103/0x130
Jan 26 11:36:26 serveerstertje kernel: [ 89.270416] [<ffffffff81472de6>] debug_dma_map_page+0x126/0x150
Jan 26 11:36:26 serveerstertje kernel: [ 89.280840] [<ffffffff81714686>] rtl8169_start_xmit+0x216/0xa20
[r8169 and xen stuff]
Dan, I miss the part of the debug code that tells where the mappings were
previously set.
In this case it was a facepalm mistake on my part. The mappings were
not being properly accounted in the last revision of the patch I sent.
I copied you on the fix [1].
Didn't have time to test it before, but the patch doesn't seem to help.
I'm still getting the "DMA-API: exceeded 7 overlapping mappings of pfn 55ebe",
but i see now i forgot to mention i use r8169.use_dac=1 ...
Not using it seems to prevent the warning, but before 3.14 i have never seen this (with r8169.use_dac=1)
--
Sander
Wednesday, January 29, 2014, 4:06:24 AM, you wrote:
quoted
On Sun, Jan 26, 2014 at 4:03 PM, Francois Romieu [off-list ref] wrote:
quoted
Sander Eikelenboom [off-list ref] :
[...]
quoted
I have got a regression with a 3.14-mw kernel (last commit is 4ba9920e5e9c0e16b5ed24292d45322907bb9035):
It looks like it's related to the rtl8169 ...
--
Sander
Jan 26 11:36:26 serveerstertje kernel: [ 89.105537] ------------[ cut here ]------------
Jan 26 11:36:26 serveerstertje kernel: [ 89.116779] WARNING: CPU: 0 PID: 0 at lib/dma-debug.c:491 add_dma_entry+0x103/0x130()
Jan 26 11:36:26 serveerstertje kernel: [ 89.128148] DMA-API: exceeded 7 overlapping mappings of pfn 55ebe
Jan 26 11:36:26 serveerstertje kernel: [ 89.139397] Modules linked in:
Jan 26 11:36:26 serveerstertje kernel: [ 89.150535] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.13.0-20140125-mw-pcireset+ #1
Jan 26 11:36:26 serveerstertje kernel: [ 89.161784] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640) , BIOS V1.8B1 09/13/2010
Jan 26 11:36:26 serveerstertje kernel: [ 89.172965] 0000000000000009 ffff88005f603838 ffffffff81acbcfa ffffffff822134e0
Jan 26 11:36:26 serveerstertje kernel: [ 89.184156] ffff88005f603888 ffff88005f603878 ffffffff810bdf62 ffff880000000000
Jan 26 11:36:26 serveerstertje kernel: [ 89.195186] 0000000000055ebe 00000000ffffffef 0000000000000200 ffff8800592ea098
Jan 26 11:36:26 serveerstertje kernel: [ 89.206227] Call Trace:
Jan 26 11:36:26 serveerstertje kernel: [ 89.217027] <IRQ> [<ffffffff81acbcfa>] dump_stack+0x46/0x58
Jan 26 11:36:26 serveerstertje kernel: [ 89.227907] [<ffffffff810bdf62>] warn_slowpath_common+0x82/0xb0
Jan 26 11:36:26 serveerstertje kernel: [ 89.238678] [<ffffffff810be031>] warn_slowpath_fmt+0x41/0x50
Jan 26 11:36:26 serveerstertje kernel: [ 89.249336] [<ffffffff81471c5a>] ? active_pfn_read_overlap+0x3a/0x70
Jan 26 11:36:26 serveerstertje kernel: [ 89.259904] [<ffffffff814729e3>] add_dma_entry+0x103/0x130
Jan 26 11:36:26 serveerstertje kernel: [ 89.270416] [<ffffffff81472de6>] debug_dma_map_page+0x126/0x150
Jan 26 11:36:26 serveerstertje kernel: [ 89.280840] [<ffffffff81714686>] rtl8169_start_xmit+0x216/0xa20
[r8169 and xen stuff]
Dan, I miss the part of the debug code that tells where the mappings were
previously set.
quoted
In this case it was a facepalm mistake on my part. The mappings were
not being properly accounted in the last revision of the patch I sent.
I copied you on the fix [1].
Didn't have time to test it before, but the patch doesn't seem to help.
I'm still getting the "DMA-API: exceeded 7 overlapping mappings of pfn 55ebe",
but i see now i forgot to mention i use r8169.use_dac=1 ...
quoted
Not using it seems to prevent the warning, but before 3.14 i have never seen this (with r8169.use_dac=1)
If you are still hitting this with the patch:
59f2e7df574c dma-debug: fix overlap detection
...then I'm more inclined to think it is an actual positive report.
If you don't mind I'll send some debug patches to narrow this down.
Didn't have time to test it before, but the patch doesn't seem to help.
I'm still getting the "DMA-API: exceeded 7 overlapping mappings of pfn 55ebe",
but i see now i forgot to mention i use r8169.use_dac=1 ...
quoted
Not using it seems to prevent the warning, but before 3.14 i have never seen this (with r8169.use_dac=1)
If you are still hitting this with the patch:
59f2e7df574c dma-debug: fix overlap detection
...then I'm more inclined to think it is an actual positive report.
If you don't mind I'll send some debug patches to narrow this down.
From: Dan Williams <hidden> Date: 2014-02-06 19:12:26
On Thu, Feb 6, 2014 at 6:27 AM, Sander Eikelenboom [off-list ref] wrote:
quoted
quoted
quoted
Not using it seems to prevent the warning, but before 3.14 i have never seen this (with r8169.use_dac=1)
quoted
If you are still hitting this with the patch:
quoted
59f2e7df574c dma-debug: fix overlap detection
quoted
...then I'm more inclined to think it is an actual positive report.
quoted
If you don't mind I'll send some debug patches to narrow this down.
Please do .. sounds better than bisecting :-)
Hi, attached is a patch that should give some insight whether the
driver is triggering many overlapping mappings. Try it on top of
3.14-rc1.
Thank you for the debug help!
Thursday, February 6, 2014, 8:12:15 PM, you wrote:
On Thu, Feb 6, 2014 at 6:27 AM, Sander Eikelenboom [off-list ref] wrote:
quoted
quoted
quoted
quoted
Not using it seems to prevent the warning, but before 3.14 i have never seen this (with r8169.use_dac=1)
quoted
If you are still hitting this with the patch:
quoted
59f2e7df574c dma-debug: fix overlap detection
quoted
...then I'm more inclined to think it is an actual positive report.
quoted
If you don't mind I'll send some debug patches to narrow this down.
Please do .. sounds better than bisecting :-)
Hi, attached is a patch that should give some insight whether the
driver is triggering many overlapping mappings. Try it on top of
3.14-rc1.
Thank you for the debug help!
Hi Dan,
Nifty feature the trace_printk .. however is there a way to limit the list it's spitting out
to what you are interesting in ?
At present the machine chokes while trying to spit out everything in one go and:
- it probably not of all of it is logged to disk because of all the rcu stalls and other problems it causes.
- the list on console at least looked a lot longer (and in the logs i don't see the original warn_on which should
be just before the dump.
However .. attached is what i have got ...
--
Sander
Hi Dan,
FYI just tested and put Xen out of the equation (booting baremetal) and it still persists.
I tried something else .. don't know if it gives you anymore insights, but it's worth the try:
@@ -474,11 +474,11 @@ static int active_pfn_set_overlap(unsigned long pfn, int overlap)returnoverlap;}-staticvoidactive_pfn_inc_overlap(unsignedlongpfn)+staticvoidactive_pfn_inc_overlap(structdma_debug_entry*ent){-intoverlap=active_pfn_read_overlap(pfn);+intoverlap=active_pfn_read_overlap(ent->pfn);-overlap=active_pfn_set_overlap(pfn,++overlap);+overlap=active_pfn_set_overlap(ent->pfn,++overlap);/* If we overflowed the overlap counter then we're potentially*leakingdma-mappings.Otherwise,ifmapsandunmapsare
@@ -486,15 +486,43 @@ static void active_pfn_inc_overlap(unsigned long pfn)*debug_dma_assert_idle()asthepfnmaybemarkedidle*prematurely.*/+WARN_ONCE(overlap>ACTIVE_PFN_MAX_OVERLAP,"DMA-API: exceeded %d overlapping mappings of pfn %lx\n",-ACTIVE_PFN_MAX_OVERLAP,pfn);+ACTIVE_PFN_MAX_OVERLAP,ent->pfn);++if(overlap>ACTIVE_PFN_MAX_OVERLAP){++dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. start dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+intidx;++for(idx=0;idx<HASH_SIZE;idx++){+structhash_bucket*bucket=&dma_entry_hash[idx];+structdma_debug_entry*entry;+unsignedlongflags;++list_for_each_entry(entry,&bucket->list,list){+if(entry->pfn==ent->pfn){+dev_info(entry->dev,"%s idx %d P=%Lx N=%lx D=%Lx L=%Lx %s %s\n",+type2name[entry->type],idx,+phys_addr(entry),entry->pfn,+entry->dev_addr,entry->size,+dir2name[entry->direction],+maperr2str[entry->map_err_type]);+}+}+}+dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. end of dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+}}
@@ -505,10 +533,10 @@ static int active_pfn_insert(struct dma_debug_entry *entry)spin_lock_irqsave(&radix_lock,flags);rc=radix_tree_insert(&dma_active_pfn,entry->pfn,entry);-if(rc==-EEXIST)-active_pfn_inc_overlap(entry->pfn);+if(rc==-EEXIST){+active_pfn_inc_overlap(entry);+}spin_unlock_irqrestore(&radix_lock,flags);-returnrc;}
Didn't have time to test it before, but the patch doesn't seem to help.
I'm still getting the "DMA-API: exceeded 7 overlapping mappings of pfn 55ebe",
but i see now i forgot to mention i use r8169.use_dac=1 ...
quoted
Not using it seems to prevent the warning, but before 3.14 i have never seen this (with r8169.use_dac=1)
If you are still hitting this with the patch:
59f2e7df574c dma-debug: fix overlap detection
...then I'm more inclined to think it is an actual positive report.
If you don't mind I'll send some debug patches to narrow this down.
From: Eric Dumazet <hidden> Date: 2014-02-11 21:28:56
On Tue, 2014-02-11 at 20:56 +0100, Sander Eikelenboom wrote:
quoted hunk
Hi Dan,
FYI just tested and put Xen out of the equation (booting baremetal) and it still persists.
I tried something else .. don't know if it gives you anymore insights, but it's worth the try:
@@ -474,11 +474,11 @@ static int active_pfn_set_overlap(unsigned long pfn, int overlap)returnoverlap;}-staticvoidactive_pfn_inc_overlap(unsignedlongpfn)+staticvoidactive_pfn_inc_overlap(structdma_debug_entry*ent){-intoverlap=active_pfn_read_overlap(pfn);+intoverlap=active_pfn_read_overlap(ent->pfn);-overlap=active_pfn_set_overlap(pfn,++overlap);+overlap=active_pfn_set_overlap(ent->pfn,++overlap);/* If we overflowed the overlap counter then we're potentially*leakingdma-mappings.Otherwise,ifmapsandunmapsare
@@ -486,15 +486,43 @@ static void active_pfn_inc_overlap(unsigned long pfn)*debug_dma_assert_idle()asthepfnmaybemarkedidle*prematurely.*/+WARN_ONCE(overlap>ACTIVE_PFN_MAX_OVERLAP,"DMA-API: exceeded %d overlapping mappings of pfn %lx\n",-ACTIVE_PFN_MAX_OVERLAP,pfn);+ACTIVE_PFN_MAX_OVERLAP,ent->pfn);++if(overlap>ACTIVE_PFN_MAX_OVERLAP){++dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. start dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+intidx;++for(idx=0;idx<HASH_SIZE;idx++){+structhash_bucket*bucket=&dma_entry_hash[idx];+structdma_debug_entry*entry;+unsignedlongflags;++list_for_each_entry(entry,&bucket->list,list){+if(entry->pfn==ent->pfn){+dev_info(entry->dev,"%s idx %d P=%Lx N=%lx D=%Lx L=%Lx %s %s\n",+type2name[entry->type],idx,+phys_addr(entry),entry->pfn,+entry->dev_addr,entry->size,+dir2name[entry->direction],+maperr2str[entry->map_err_type]);+}+}+}+dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. end of dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+}}
@@ -505,10 +533,10 @@ static int active_pfn_insert(struct dma_debug_entry *entry)spin_lock_irqsave(&radix_lock,flags);rc=radix_tree_insert(&dma_active_pfn,entry->pfn,entry);-if(rc==-EEXIST)-active_pfn_inc_overlap(entry->pfn);+if(rc==-EEXIST){+active_pfn_inc_overlap(entry);+}spin_unlock_irqrestore(&radix_lock,flags);-returnrc;}
Incoming frames might be taken out of order-3 pages.
With regular Ethernet frames, this is 21 frames per order-3 pages.
ACTIVE_PFN_MAX_OVERLAP seems too small.
Alternative would be to user order-0 only pages if CONFIG_DMA_API_DEBUG
is set. Not sure if it works if PAGE_SIZE=66536 ....
Tuesday, February 11, 2014, 10:28:52 PM, you wrote:
On Tue, 2014-02-11 at 20:56 +0100, Sander Eikelenboom wrote:
quoted
Hi Dan,
FYI just tested and put Xen out of the equation (booting baremetal) and it still persists.
I tried something else .. don't know if it gives you anymore insights, but it's worth the try:
@@ -474,11 +474,11 @@ static int active_pfn_set_overlap(unsigned long pfn, int overlap)returnoverlap;}-staticvoidactive_pfn_inc_overlap(unsignedlongpfn)+staticvoidactive_pfn_inc_overlap(structdma_debug_entry*ent){-intoverlap=active_pfn_read_overlap(pfn);+intoverlap=active_pfn_read_overlap(ent->pfn);-overlap=active_pfn_set_overlap(pfn,++overlap);+overlap=active_pfn_set_overlap(ent->pfn,++overlap);/* If we overflowed the overlap counter then we're potentially*leakingdma-mappings.Otherwise,ifmapsandunmapsare
@@ -486,15 +486,43 @@ static void active_pfn_inc_overlap(unsigned long pfn)*debug_dma_assert_idle()asthepfnmaybemarkedidle*prematurely.*/+WARN_ONCE(overlap>ACTIVE_PFN_MAX_OVERLAP,"DMA-API: exceeded %d overlapping mappings of pfn %lx\n",-ACTIVE_PFN_MAX_OVERLAP,pfn);+ACTIVE_PFN_MAX_OVERLAP,ent->pfn);++if(overlap>ACTIVE_PFN_MAX_OVERLAP){++dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. start dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+intidx;++for(idx=0;idx<HASH_SIZE;idx++){+structhash_bucket*bucket=&dma_entry_hash[idx];+structdma_debug_entry*entry;+unsignedlongflags;++list_for_each_entry(entry,&bucket->list,list){+if(entry->pfn==ent->pfn){+dev_info(entry->dev,"%s idx %d P=%Lx N=%lx D=%Lx L=%Lx %s %s\n",+type2name[entry->type],idx,+phys_addr(entry),entry->pfn,+entry->dev_addr,entry->size,+dir2name[entry->direction],+maperr2str[entry->map_err_type]);+}+}+}+dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. end of dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+}}
@@ -505,10 +533,10 @@ static int active_pfn_insert(struct dma_debug_entry *entry)spin_lock_irqsave(&radix_lock,flags);rc=radix_tree_insert(&dma_active_pfn,entry->pfn,entry);-if(rc==-EEXIST)-active_pfn_inc_overlap(entry->pfn);+if(rc==-EEXIST){+active_pfn_inc_overlap(entry);+}spin_unlock_irqrestore(&radix_lock,flags);-returnrc;}
From: Dan Williams <hidden> Date: 2014-02-12 02:07:15
On Tue, Feb 11, 2014 at 11:56 AM, Sander Eikelenboom
[off-list ref] wrote:
Hi Dan,
FYI just tested and put Xen out of the equation (booting baremetal) and it still persists.
I tried something else .. don't know if it gives you anymore insights, but it's worth the try:
@@ -474,11 +474,11 @@ static int active_pfn_set_overlap(unsigned long pfn, int overlap)returnoverlap;}-staticvoidactive_pfn_inc_overlap(unsignedlongpfn)+staticvoidactive_pfn_inc_overlap(structdma_debug_entry*ent){-intoverlap=active_pfn_read_overlap(pfn);+intoverlap=active_pfn_read_overlap(ent->pfn);-overlap=active_pfn_set_overlap(pfn,++overlap);+overlap=active_pfn_set_overlap(ent->pfn,++overlap);/* If we overflowed the overlap counter then we're potentially*leakingdma-mappings.Otherwise,ifmapsandunmapsare
@@ -486,15 +486,43 @@ static void active_pfn_inc_overlap(unsigned long pfn)*debug_dma_assert_idle()asthepfnmaybemarkedidle*prematurely.*/+WARN_ONCE(overlap>ACTIVE_PFN_MAX_OVERLAP,"DMA-API: exceeded %d overlapping mappings of pfn %lx\n",-ACTIVE_PFN_MAX_OVERLAP,pfn);+ACTIVE_PFN_MAX_OVERLAP,ent->pfn);++if(overlap>ACTIVE_PFN_MAX_OVERLAP){++dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. start dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+intidx;++for(idx=0;idx<HASH_SIZE;idx++){+structhash_bucket*bucket=&dma_entry_hash[idx];+structdma_debug_entry*entry;+unsignedlongflags;++list_for_each_entry(entry,&bucket->list,list){+if(entry->pfn==ent->pfn){+dev_info(entry->dev,"%s idx %d P=%Lx N=%lx D=%Lx L=%Lx %s %s\n",+type2name[entry->type],idx,+phys_addr(entry),entry->pfn,+entry->dev_addr,entry->size,+dir2name[entry->direction],+maperr2str[entry->map_err_type]);+}+}+}+dev_info(ent->dev,"DMA-API: exceeded %d overlapping mappings of pfn %lx .. end of dump\n",ACTIVE_PFN_MAX_OVERLAP,ent->pfn);+}}
@@ -505,10 +533,10 @@ static int active_pfn_insert(struct dma_debug_entry *entry)spin_lock_irqsave(&radix_lock,flags);rc=radix_tree_insert(&dma_active_pfn,entry->pfn,entry);-if(rc==-EEXIST)-active_pfn_inc_overlap(entry->pfn);+if(rc==-EEXIST){+active_pfn_inc_overlap(entry);+}spin_unlock_irqrestore(&radix_lock,flags);-returnrc;}
The overlap granularity is too large. Multiple dma_map_single
mappings are allowed to a given page as long as they don't collide on
the same cache line.
Please try the attached patch to see if it fixes this issue. Works ok for me.
From: Eric Dumazet <hidden> Date: 2014-02-12 04:17:46
On Tue, 2014-02-11 at 18:07 -0800, Dan Williams wrote:
The overlap granularity is too large. Multiple dma_map_single
mappings are allowed to a given page as long as they don't collide on
the same cache line.
I am not sure why you try number of mappings of a page.
Try launching 100 concurrent netperf -t TCP_SENFILE
Same page might be mapped more than 100 times, more than 10000 times in
some cases.
From: Dan Williams <hidden> Date: 2014-02-12 14:56:40
On Tue, Feb 11, 2014 at 8:17 PM, Eric Dumazet [off-list ref] wrote:
On Tue, 2014-02-11 at 18:07 -0800, Dan Williams wrote:
quoted
The overlap granularity is too large. Multiple dma_map_single
mappings are allowed to a given page as long as they don't collide on
the same cache line.
I am not sure why you try number of mappings of a page.
For this debug facility I am tracking whether dma has completed by
making sure there are no active dma_map entries in the address range
of a page being cow'd.
Try launching 100 concurrent netperf -t TCP_SENFILE
Same page might be mapped more than 100 times, more than 10000 times in
some cases.
Aren't these mapping serialized by the device to some extent?
Although multi-queue / multi-device would even defeat that...
Hmm, then I think at a minimum the activity tracking needs to be
constrained to overlapping DMA_FROM_DEVICE or DMA_BIDIRECTIONAL
mappings. However, I am still operating on the assumption that some
architectures (especially non-io-coherent or dmabounce architectures)
expect a dma mapping to reflect exclusive ownership of the buffer.
From the conversation I had with Russell, back in the day [1]:
"When we get to the second async_xor(), as we haven't started to run any
of these operations, the source and destination buffers are still mapped.
However, we ignore that and call dma_map_page() on them again - this is
illegal because the CPU does not own these buffers."
It might be the case that we can't have a general overlap detection
facility as it will flag stable use cases that nonetheless violate the
exclusivity expectation.
--
Dan
[1]: http://marc.info/?l=linux-arm-kernel&m=129389649101566&w=2
From: Ben Hutchings <hidden> Date: 2014-02-12 22:53:05
On Tue, 2014-02-11 at 13:28 -0800, Eric Dumazet wrote:
[...]
Incoming frames might be taken out of order-3 pages.
With regular Ethernet frames, this is 21 frames per order-3 pages.
ACTIVE_PFN_MAX_OVERLAP seems too small.
Alternative would be to user order-0 only pages if CONFIG_DMA_API_DEBUG
is set. Not sure if it works if PAGE_SIZE=66536 ....
Indeed, you can get a lot of packet buffers into a 64K page...
That may be useful for debugging this particular problem, but please
don't make debugging options change behaviour like this.
Ben.
--
Ben Hutchings
If more than one person is responsible for a bug, no one is at fault.
From: Dan Williams <hidden> Date: 2014-02-13 20:14:55
On Tue, 2014-02-11 at 20:17 -0800, Eric Dumazet wrote:
On Tue, 2014-02-11 at 18:07 -0800, Dan Williams wrote:
quoted
The overlap granularity is too large. Multiple dma_map_single
mappings are allowed to a given page as long as they don't collide on
the same cache line.
I am not sure why you try number of mappings of a page.
Try launching 100 concurrent netperf -t TCP_SENFILE
Same page might be mapped more than 100 times, more than 10000 times in
some cases.
Thanks for that test case.
I updated the fix patch with the following.
@@ -513,6 +513,13 @@ static int active_cln_insert(struct dma_debug_entry *entry)unsignedlongflags;intrc;+/* If the device is not writing memory then we don't have any+*concernsaboutthecpuconsumingstaledata.Thismitigates+*legitimateusagesofoverlappingmappings.+*/+if(entry->direction==DMA_TO_DEVICE)+return0;+spin_lock_irqsave(&radix_lock,flags);rc=radix_tree_insert(&dma_active_cacheline,to_cln(entry),entry);if(rc==-EEXIST)
@@ -526,6 +533,10 @@ static void active_cln_remove(struct dma_debug_entry *entry){unsignedlongflags;+/* ...mirror the insert case */+if(entry->direction==DMA_TO_DEVICE)+return;+spin_lock_irqsave(&radix_lock,flags);/* since we are counting overlaps the final put of the*cachelinewilloccurwhentheoverlapcountis0.
Sander, barring a negative test result from you I'll send the attached
patch to Andrew.
--
Dan
From: Dave Jones <hidden> Date: 2014-02-13 20:16:47
On Tue, Feb 11, 2014 at 06:07:10PM -0800, Dan Williams wrote:
> The overlap granularity is too large. Multiple dma_map_single
> mappings are allowed to a given page as long as they don't collide on
> the same cache line.
>
>
> Please try the attached patch to see if it fixes this issue. Works ok for me.
FWIW, since applying this, I haven't seen the 8169 warnings.
thanks,
Dave
Thursday, February 13, 2014, 9:14:47 PM, you wrote:
On Tue, 2014-02-11 at 20:17 -0800, Eric Dumazet wrote:
quoted
On Tue, 2014-02-11 at 18:07 -0800, Dan Williams wrote:
quoted
The overlap granularity is too large. Multiple dma_map_single
mappings are allowed to a given page as long as they don't collide on
the same cache line.
I am not sure why you try number of mappings of a page.
Try launching 100 concurrent netperf -t TCP_SENFILE
Same page might be mapped more than 100 times, more than 10000 times in
some cases.
@@ -513,6 +513,13 @@ static int active_cln_insert(struct dma_debug_entry *entry)unsignedlongflags;intrc;+/* If the device is not writing memory then we don't have any+*concernsaboutthecpuconsumingstaledata.Thismitigates+*legitimateusagesofoverlappingmappings.+*/
@@ -526,6 +533,10 @@ static void active_cln_remove(struct dma_debug_entry *entry) { unsigned long flags;+ /* ...mirror the insert case */
+ if (entry->>direction == DMA_TO_DEVICE)
+ return;
+
spin_lock_irqsave(&radix_lock, flags);
/* since we are counting overlaps the final put of the
* cacheline will occur when the overlap count is 0.
Sander, barring a negative test result from you I'll send the attached
patch to Andrew.
Hi Dan,
That seems to effectively suppress the warning, thanks and:
Tested-by; Sander Eikelenboom [off-list ref]
--
Sander
On Thu, Feb 13, 2014 at 4:49 PM, Sander Eikelenboom
[off-list ref] wrote:
Thursday, February 13, 2014, 9:14:47 PM, you wrote:
quoted
On Tue, 2014-02-11 at 20:17 -0800, Eric Dumazet wrote:
quoted
On Tue, 2014-02-11 at 18:07 -0800, Dan Williams wrote:
quoted
The overlap granularity is too large. Multiple dma_map_single
mappings are allowed to a given page as long as they don't collide on
the same cache line.
I am not sure why you try number of mappings of a page.
Try launching 100 concurrent netperf -t TCP_SENFILE
Same page might be mapped more than 100 times, more than 10000 times in
some cases.
@@ -513,6 +513,13 @@ static int active_cln_insert(struct dma_debug_entry *entry)unsignedlongflags;intrc;+/* If the device is not writing memory then we don't have any+*concernsaboutthecpuconsumingstaledata.Thismitigates+*legitimateusagesofoverlappingmappings.+*/
@@ -526,6 +533,10 @@ static void active_cln_remove(struct dma_debug_entry *entry) { unsigned long flags;+ /* ...mirror the insert case */
+ if (entry->>direction == DMA_TO_DEVICE)
quoted
+ return;
+
spin_lock_irqsave(&radix_lock, flags);
/* since we are counting overlaps the final put of the
* cacheline will occur when the overlap count is 0.
quoted
Sander, barring a negative test result from you I'll send the attached
patch to Andrew.
Hi Dan,
That seems to effectively suppress the warning, thanks and:
Tested-by; Sander Eikelenboom [off-list ref]
Is there a reason this isn't in Linus' tree yet?
josh
From: Dan Williams <hidden> Date: 2014-02-25 17:50:50
On Tue, Feb 25, 2014 at 9:45 AM, Josh Boyer [off-list ref] wrote:
On Thu, Feb 13, 2014 at 4:49 PM, Sander Eikelenboom
[off-list ref] wrote:
quoted
Thursday, February 13, 2014, 9:14:47 PM, you wrote:
quoted
On Tue, 2014-02-11 at 20:17 -0800, Eric Dumazet wrote:
quoted
On Tue, 2014-02-11 at 18:07 -0800, Dan Williams wrote:
quoted
The overlap granularity is too large. Multiple dma_map_single
mappings are allowed to a given page as long as they don't collide on
the same cache line.
I am not sure why you try number of mappings of a page.
Try launching 100 concurrent netperf -t TCP_SENFILE
Same page might be mapped more than 100 times, more than 10000 times in
some cases.
@@ -513,6 +513,13 @@ static int active_cln_insert(struct dma_debug_entry *entry)unsignedlongflags;intrc;+/* If the device is not writing memory then we don't have any+*concernsaboutthecpuconsumingstaledata.Thismitigates+*legitimateusagesofoverlappingmappings.+*/
@@ -526,6 +533,10 @@ static void active_cln_remove(struct dma_debug_entry *entry) { unsigned long flags;+ /* ...mirror the insert case */
+ if (entry->>direction == DMA_TO_DEVICE)
quoted
+ return;
+
spin_lock_irqsave(&radix_lock, flags);
/* since we are counting overlaps the final put of the
* cacheline will occur when the overlap count is 0.
quoted
Sander, barring a negative test result from you I'll send the attached
patch to Andrew.
Hi Dan,
That seems to effectively suppress the warning, thanks and:
Tested-by; Sander Eikelenboom [off-list ref]
Is there a reason this isn't in Linus' tree yet?
It's in -mm and now -next, I expect it will go upstream with akpm's next sync.
--
Dan