Hi Josef,

this still hapens with current 4.8-rc* releases. Anything i can do to
debug this? May be insert some code to check for an under or overflow in
the code?

Stefan

Am 14.08.2016 um 17:22 schrieb Stefan Priebe - Profihost AG:

Hi Josef,

anything i could do or test? Results with a vanilla next branch are the
same.

Stefan

Am 11.08.2016 um 08:09 schrieb Stefan Priebe - Profihost AG:

quoted

Hello,

the backtrace and info on umount looks the same:

[241910.341124] ------------[ cut here ]------------
[241910.379991] WARNING: CPU: 1 PID: 26664 at
fs/btrfs/extent-tree.c:5701 btrfs_free_block_groups+0x370/0x410 [btrfs]
[241910.422099] Modules linked in: netconsole mpt3sas ipt_REJECT
raid_class nf_reject_ipv4 scsi_transport_sas xt_multiport 8021q garp
iptable_filter ip_tables x_tables bonding coretemp loop usbhid ehci_pci
i2c_i801 ehci_hcd usbcore i2c_core shpchp usb_common ipmi_si
ipmi_msghandler button btrfs dm_mod raid1 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx xor raid6_pq md_mod sg sd_mod
ixgbe i40e mdio ptp ahci libahci pps_core megaraid_sas
[241910.616845] CPU: 1 PID: 26664 Comm: umount Not tainted
4.7.0-rc6-29043-g8b8b08c #1
[241910.669646] Hardware name: Supermicro X10DRH/X10DRH-iT, BIOS 1.0c
02/18/2015
[241910.723716]  0000000000000000 ffff8808d104bca8 ffffffffbd3d83cf
0000000000000000
[241910.779309]  0000000000000000 ffff8808d104bcf8 ffffffffbd085615
ffff8808d104bd08
[241910.835143]  000016455a3410a8 00000047a0000000 0000000000000000
ffff8808469e2088
[241910.891882] Call Trace:
[241910.947624]  [<ffffffffbd3d83cf>] dump_stack+0x63/0x84
[241911.003714]  [<ffffffffbd085615>] __warn+0xe5/0x100
[241911.060167]  [<ffffffffbd08564d>] warn_slowpath_null+0x1d/0x20
[241911.117422]  [<ffffffffc058ca90>]
btrfs_free_block_groups+0x370/0x410 [btrfs]
[241911.175975]  [<ffffffffc059e7ab>] close_ctree+0x15b/0x330 [btrfs]
[241911.235170]  [<ffffffffc056f089>] btrfs_put_super+0x19/0x20 [btrfs]
[241911.294638]  [<ffffffffbd1deaff>] generic_shutdown_super+0x6f/0x100
[241911.353005]  [<ffffffffbd1df026>] kill_anon_super+0x16/0x30
[241911.409832]  [<ffffffffc05720fa>] btrfs_kill_super+0x1a/0xb0 [btrfs]
[241911.466467]  [<ffffffffbd1df1f1>] deactivate_locked_super+0x51/0x90
[241911.522602]  [<ffffffffbd1dfb8e>] deactivate_super+0x4e/0x70
[241911.577979]  [<ffffffffbd1fba73>] cleanup_mnt+0x43/0x90
[241911.633188]  [<ffffffffbd1fbb12>] __cleanup_mnt+0x12/0x20
[241911.688146]  [<ffffffffbd0a1f61>] task_work_run+0x81/0xb0
[241911.742740]  [<ffffffffbd07ffcd>] exit_to_usermode_loop+0x66/0x95
[241911.797039]  [<ffffffffbd002a7d>] do_syscall_64+0x10d/0x150
[241911.850750]  [<ffffffffbd6d9ca1>] entry_SYSCALL64_slow_path+0x25/0x25
[241911.903564] ---[ end trace fae017546778f2b0 ]---
[241911.955332] ------------[ cut here ]------------
[241912.006262] WARNING: CPU: 1 PID: 26664 at
fs/btrfs/extent-tree.c:5702 btrfs_free_block_groups+0x40a/0x410 [btrfs]
[241912.059326] Modules linked in: netconsole mpt3sas ipt_REJECT
raid_class nf_reject_ipv4 scsi_transport_sas xt_multiport 8021q garp
iptable_filter ip_tables x_tables bonding coretemp loop usbhid ehci_pci
i2c_i801 ehci_hcd usbcore i2c_core shpchp usb_common ipmi_si
ipmi_msghandler button btrfs dm_mod raid1 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx xor raid6_pq md_mod sg sd_mod
ixgbe i40e mdio ptp ahci libahci pps_core megaraid_sas
[241912.298666] CPU: 1 PID: 26664 Comm: umount Tainted: G        W
4.7.0-rc6-29043-g8b8b08c #1
[241912.363401] Hardware name: Supermicro X10DRH/X10DRH-iT, BIOS 1.0c
02/18/2015
[241912.429395]  0000000000000000 ffff8808d104bca8 ffffffffbd3d83cf
0000000000000000
[241912.497080]  0000000000000000 ffff8808d104bcf8 ffffffffbd085615
ffff8808d104bd08
[241912.565113]  000016465a3410a8 00000047a0000000 0000000000000000
ffff8808469e2088
[241912.634105] Call Trace:
[241912.702992]  [<ffffffffbd3d83cf>] dump_stack+0x63/0x84
[241912.773473]  [<ffffffffbd085615>] __warn+0xe5/0x100
[241912.844339]  [<ffffffffbd08564d>] warn_slowpath_null+0x1d/0x20
[241912.916083]  [<ffffffffc058cb2a>]
btrfs_free_block_groups+0x40a/0x410 [btrfs]
[241912.989103]  [<ffffffffc059e7ab>] close_ctree+0x15b/0x330 [btrfs]
[241913.062672]  [<ffffffffc056f089>] btrfs_put_super+0x19/0x20 [btrfs]
[241913.136364]  [<ffffffffbd1deaff>] generic_shutdown_super+0x6f/0x100
[241913.208701]  [<ffffffffbd1df026>] kill_anon_super+0x16/0x30
[241913.279194]  [<ffffffffc05720fa>] btrfs_kill_super+0x1a/0xb0 [btrfs]
[241913.348065]  [<ffffffffbd1df1f1>] deactivate_locked_super+0x51/0x90
[241913.415082]  [<ffffffffbd1dfb8e>] deactivate_super+0x4e/0x70
[241913.479841]  [<ffffffffbd1fba73>] cleanup_mnt+0x43/0x90
[241913.543353]  [<ffffffffbd1fbb12>] __cleanup_mnt+0x12/0x20
[241913.605959]  [<ffffffffbd0a1f61>] task_work_run+0x81/0xb0
[241913.667542]  [<ffffffffbd07ffcd>] exit_to_usermode_loop+0x66/0x95
[241913.729612]  [<ffffffffbd002a7d>] do_syscall_64+0x10d/0x150
[241913.791203]  [<ffffffffbd6d9ca1>] entry_SYSCALL64_slow_path+0x25/0x25
[241913.852485] ---[ end trace fae017546778f2b1 ]---
[241913.913638] ------------[ cut here ]------------
[241913.974871] WARNING: CPU: 1 PID: 26664 at
fs/btrfs/extent-tree.c:10013 btrfs_free_block_groups+0x2ba/0x410 [btrfs]
[241914.039315] Modules linked in: netconsole mpt3sas ipt_REJECT
raid_class nf_reject_ipv4 scsi_transport_sas xt_multiport 8021q garp
iptable_filter ip_tables x_tables bonding coretemp loop usbhid ehci_pci
i2c_i801 ehci_hcd usbcore i2c_core shpchp usb_common ipmi_si
ipmi_msghandler button btrfs dm_mod raid1 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx xor raid6_pq md_mod sg sd_mod
ixgbe i40e mdio ptp ahci libahci pps_core megaraid_sas
[241914.315918] CPU: 1 PID: 26664 Comm: umount Tainted: G        W
4.7.0-rc6-29043-g8b8b08c #1
[241914.388096] Hardware name: Supermicro X10DRH/X10DRH-iT, BIOS 1.0c
02/18/2015
[241914.460679]  0000000000000000 ffff8808d104bca8 ffffffffbd3d83cf
0000000000000000
[241914.534126]  0000000000000000 ffff8808d104bcf8 ffffffffbd085615
ffff8808d104bce8
[241914.607523]  0000271dbd3dac8c ffff88085184aac8 0000000000000038
0000000000000000
[241914.681318] Call Trace:
[241914.754437]  [<ffffffffbd3d83cf>] dump_stack+0x63/0x84
[241914.828796]  [<ffffffffbd085615>] __warn+0xe5/0x100
[241914.902953]  [<ffffffffbd08564d>] warn_slowpath_null+0x1d/0x20
[241914.977271]  [<ffffffffc058c9da>]
btrfs_free_block_groups+0x2ba/0x410 [btrfs]
[241915.052041]  [<ffffffffc059e7ab>] close_ctree+0x15b/0x330 [btrfs]
[241915.126282]  [<ffffffffc056f089>] btrfs_put_super+0x19/0x20 [btrfs]
[241915.200758]  [<ffffffffbd1deaff>] generic_shutdown_super+0x6f/0x100
[241915.273872]  [<ffffffffbd1df026>] kill_anon_super+0x16/0x30
[241915.345132]  [<ffffffffc05720fa>] btrfs_kill_super+0x1a/0xb0 [btrfs]
[241915.414703]  [<ffffffffbd1df1f1>] deactivate_locked_super+0x51/0x90
[241915.482488]  [<ffffffffbd1dfb8e>] deactivate_super+0x4e/0x70
[241915.547994]  [<ffffffffbd1fba73>] cleanup_mnt+0x43/0x90
[241915.611962]  [<ffffffffbd1fbb12>] __cleanup_mnt+0x12/0x20
[241915.674717]  [<ffffffffbd0a1f61>] task_work_run+0x81/0xb0
[241915.736398]  [<ffffffffbd07ffcd>] exit_to_usermode_loop+0x66/0x95
[241915.798592]  [<ffffffffbd002a7d>] do_syscall_64+0x10d/0x150
[241915.860295]  [<ffffffffbd6d9ca1>] entry_SYSCALL64_slow_path+0x25/0x25
[241915.921642] ---[ end trace fae017546778f2b2 ]---
[241915.982893] BTRFS: space_info 4 has 114577997824 free, is not full
[241916.045103] BTRFS: space_info total=307627032576, used=193048903680,
pinned=0, reserved=0, may_use=688537059328, readonly=131072

Greets,
Stefan

Am 10.08.2016 um 23:31 schrieb Stefan Priebe - Profihost AG:

quoted

Hi Josef,

same again with chris next branch:

ERROR: error during balancing '/vmbackup/': No space left on device
There may be more info in syslog - try dmesg | tail
Dumping filters: flags 0x7, state 0x0, force is off
  DATA (flags 0x2): balancing, usage=5
  METADATA (flags 0x2): balancing, usage=5
  SYSTEM (flags 0x2): balancing, usage=5

dmesg:
[203784.411189] BTRFS info (device dm-0): 114 enospc errors during balance

uname -r 4.7.0-rc6-29043-g8b8b08c

Greets,
Stefan

Am 08.08.2016 um 08:17 schrieb Stefan Priebe - Profihost AG:

quoted

Am 04.08.2016 um 13:40 schrieb Stefan Priebe - Profihost AG:

quoted

Am 29.07.2016 um 23:03 schrieb Josef Bacik:

quoted

On 07/29/2016 03:14 PM, Omar Sandoval wrote:

quoted

On Fri, Jul 29, 2016 at 12:11:53PM -0700, Omar Sandoval wrote:

quoted

On Fri, Jul 29, 2016 at 08:40:26PM +0200, Stefan Priebe - Profihost
AG wrote:

quoted

Dear list,

i'm seeing btrfs no space messages frequently on big filesystems (>
30TB).

In all cases i'm getting a trace like this one a space_info warning.
(since commit [1]). Could someone please be so kind and help me
debugging / fixing this bug? I'm using space_cache=v2 on all those
systems.

Hm, so I think this indicates a bug in space accounting somewhere else
rather than the free space tree itself. I haven't debugged one of these
issues before, I'll see if I can reproduce it. Cc'ing Josef, too.

I should've asked, what sort of filesystem activity triggers this?

Chris just fixed this I think, try his next branch from his git tree

git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git

Thanks now running a 4.4 with those patches backported. If that still
shows an error i will try that vanilla tree.

OK this didn't work. I'll start / try using the linux-btrfs next branch
and look if this helps.

Greets,
Stefan

quoted

Thanks!

Stefan

quoted

and see if it still happens.  Thanks,

Josef