On Oct 23, 2012, at 4:16 PM, Goffredo Baroncelli [off-list ref] wrote:

$ sudo /mnt/home-ghigo/btrfs/btrfs-progs/btrfs fi sh
Label: 'test1'  uuid: 7ba72d6f-d226-4e8c-9a9c-92a7fd89cd99
	Total devices 4 FS bytes used 7.01GB
	devid    4 size 12.00GB used 3.21GB path /dev/vdf
	devid    3 size 3.00GB used 2.53GB path /dev/vdd
	devid    2 size 3.00GB used 2.95GB path /dev/vdc
	*** Some devices missing

Which is what you are reporting.

Yes. However I can reproduce this much more easily as:

[root@f18v ~]# mkfs.btrfs /dev/sd[bc]
[root@f18v ~]# mount /dev/sdb /mnt
[root@f18v ~]# btrfs device add /dev/sdd /mnt
[root@f18v ~]# btrfs device delete /dev/sdb /mnt
[root@f18v ~]# btrfs fi show

Label: none  uuid: 0daeada5-98c0-4a9a-8d0c-5a9dcfde2972
	Total devices 3 FS bytes used 796.00KB
	devid    3 size 80.00GB used 6.06GB path /dev/sdd
	devid    2 size 80.00GB used 6.06GB path /dev/sdc
	*** Some devices missing

4) If I touch the filesystem (eg creating a file or unmount the
filesystem) then the superblocks are update and the I get

Confirmed.

Conclusion:
- I was not able to reproduce your problem about removing the device. I
was able to remove the device after filling the filesystem and adding a
new device.

dd if=/dev/zero of=/mnt/btrfs1/bigfile bs=1M count=$((7*1024))

I think it needs to be bigger. I was at a bit over 8GB file size for a 9GB file system (3x 3GB drives). There was about 300MB of free space left according to df -h, which was for the whole volume, i.e. maybe around 100MB free space per device, and hence possibly not enough room to budge unless I added yet another drive. Then it was able to back out.

Chris Murphy