Bug in mkfs.btrfs?!

Hi,

I wanted to create a new btrfs fs for my backups.
When trying to mkfs.btrfs for that  device, I'm getting

"error checking /dev/loop2 mount status"


With strace I see where the problem is:

lstat("/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par",
0x7fffa30b3cf0) = -1 ENOENT (No such file or directory)



The problem is there is something missing at the end of the link, should be
something like "-part1", "-part2" etc.


I'll try the patch with the --foce option.

Regards,
Felix

I was able to create the fs with the --force options. But that bug should be fixed,
too :)

Is that link maybe too long? It is created by udev, and there was never a problem
using that link.


Regards,
Felix

On 22. January 2011 - 15:45, Felix Blanke wrote:

Date: Sat, 22 Jan 2011 15:45:13 +0100
From: Felix Blanke <redacted>
To: linux-btrfs@vger.kernel.org
Subject: Bug in mkfs.btrfs?!

Hi,

I wanted to create a new btrfs fs for my backups.
When trying to mkfs.btrfs for that  device, I'm getting

"error checking /dev/loop2 mount status"


With strace I see where the problem is:

lstat("/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par",
0x7fffa30b3cf0) = -1 ENOENT (No such file or directory)



The problem is there is something missing at the end of the link, should be
something like "-part1", "-part2" etc.


I'll try the patch with the --foce option.

Regards,
Felix

---end quoted text---

On Sat, Jan 22, 2011 at 03:52:22PM +0100, Felix Blanke wrote:

I was able to create the fs with the --force options. But that bug should be fixed,
too :)

Is that link maybe too long? It is created by udev, and there was never a problem
using that link.

   Yes, it certainly looks like it. There's 63 characters in the name
in your strace report, so it looks like it's being truncated somewhere.

   What was the exact command line you used to get the error?

   Hugo.

Regards,
Felix

On 22. January 2011 - 15:45, Felix Blanke wrote:

quoted

Date: Sat, 22 Jan 2011 15:45:13 +0100
From: Felix Blanke <redacted>
To: linux-btrfs@vger.kernel.org
Subject: Bug in mkfs.btrfs?!

Hi,

I wanted to create a new btrfs fs for my backups.
When trying to mkfs.btrfs for that  device, I'm getting

"error checking /dev/loop2 mount status"


With strace I see where the problem is:

lstat("/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par",
0x7fffa30b3cf0) = -1 ENOENT (No such file or directory)



The problem is there is something missing at the end of the link, should be
something like "-part1", "-part2" etc.


I'll try the patch with the --foce option.

Regards,
Felix

---end quoted text---

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Great oxymorons of the world, no. 9: Standard Deviation ---     

On Sat, Jan 22, 2011 at 03:11:24PM +0000, Hugo Mills wrote:

On Sat, Jan 22, 2011 at 03:52:22PM +0100, Felix Blanke wrote:

quoted

I was able to create the fs with the --force options. But that bug should be fixed,
too :)

Is that link maybe too long? It is created by udev, and there was never a problem
using that link.

   Yes, it certainly looks like it. There's 63 characters in the name
in your strace report, so it looks like it's being truncated somewhere.

   What was the exact command line you used to get the error?

   Also, can you post the full strace output? (Just so I can get a
better idea of where it's getting to in the code).

   I've just tried it myself:

$ sudo mkfs.btrfs /dev/disk/by-id/dm-uuid-LVM-XRQLHQNa0xEeIZL4ofuBGIcfkr1Dhry8YHhkjaw4bvZA4meDFQfEMy5elIsVNeWl 

WARNING! - Btrfs v0.19-36-g70c6c10 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

fs created label (null) on /dev/disk/by-id/dm-uuid-LVM-XRQLHQNa0xEeIZL4ofuBGIcfkr1Dhry8YHhkjaw4bvZA4meDFQfEMy5elIsVNeWl
	nodesize 4096 leafsize 4096 sectorsize 4096 size 1.00GB
Btrfs v0.19-36-g70c6c10

   Seems perfectly happy with a 92-character device name...

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Great oxymorons of the world, no. 9: Standard Deviation ---     

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of "losetup /dev/loop2":

/dev/loop2: [0010]:5324 
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128


Thanks you for looking into this!
While writing this I read your second mail. The strace output is attached.

Regards,
Felix

P.S.: The btrfs wiki is down. If someone reads this and have time to answer:

Means "-d single" that the data are not spread across the devices? The default is
raid0, right? I want some kind of security that if a device crashes only the
files on that damaged device are gone. Raid1 will be the next step, but act. I don't
have money for that :)


On 22. January 2011 - 15:11, Hugo Mills wrote:

Date: Sat, 22 Jan 2011 15:11:24 +0000
From: Hugo Mills <redacted>
To: Felix Blanke <redacted>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!
Mail-Followup-To: Hugo Mills [off-list ref], Felix Blanke
 [off-list ref], linux-btrfs@vger.kernel.org

On Sat, Jan 22, 2011 at 03:52:22PM +0100, Felix Blanke wrote:

quoted

I was able to create the fs with the --force options. But that bug should be fixed,
too :)

Is that link maybe too long? It is created by udev, and there was never a problem
using that link.

   Yes, it certainly looks like it. There's 63 characters in the name
in your strace report, so it looks like it's being truncated somewhere.

   What was the exact command line you used to get the error?

   Hugo.

quoted

Regards,
Felix

On 22. January 2011 - 15:45, Felix Blanke wrote:

quoted

Date: Sat, 22 Jan 2011 15:45:13 +0100
From: Felix Blanke <redacted>
To: linux-btrfs@vger.kernel.org
Subject: Bug in mkfs.btrfs?!

Hi,

I wanted to create a new btrfs fs for my backups.
When trying to mkfs.btrfs for that  device, I'm getting

"error checking /dev/loop2 mount status"


With strace I see where the problem is:

lstat("/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par",
0x7fffa30b3cf0) = -1 ENOENT (No such file or directory)



The problem is there is something missing at the end of the link, should be
something like "-part1", "-part2" etc.


I'll try the patch with the --foce option.

Regards,
Felix

---end quoted text---

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Great oxymorons of the world, no. 9: Standard Deviation ---     

---end quoted text---

On Sun, January 23, 2011 2:56 am, Felix Blanke wrote:

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of
"losetup /dev/loop2":

/dev/loop2: [0010]:5324
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3)
encryption=AES128

I'm not familiar with losetup, is /dev/loop2 a symlink ?

If so could you post an ls -l of /dev/loop2 please ?

cheers,
Chris
-- 
Chris Samuel, Melbourne, Australia
http://www.csamuel.org/

Hi,

no :) loop2 is the decrypted device.


A small example:

/dev/sda1 is encrypted. Then you run losetup with some options and /dev/sda1, enter
the password and after that /dev/loop1 will be created.

From now on you're using /dev/loop1 as your device node, e.g.

fdisk -l /dev/loop1


If you would use /dev/sda1 instead you would just see garbage, because of the
encryption :)



Regards,
Felix

On 23. January 2011 - 09:54, Chris Samuel wrote:

Date:	Sun, 23 Jan 2011 09:54:34 +1100
From: Chris Samuel <redacted>
To: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!


On Sun, January 23, 2011 2:56 am, Felix Blanke wrote:

quoted

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of
"losetup /dev/loop2":

/dev/loop2: [0010]:5324
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3)
encryption=AES128

I'm not familiar with losetup, is /dev/loop2 a symlink ?

If so could you post an ls -l of /dev/loop2 please ?

cheers,
Chris
-- 
Chris Samuel, Melbourne, Australia
http://www.csamuel.org/

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

---end quoted text---

   Hi, Felix,

On Sat, Jan 22, 2011 at 04:56:12PM +0100, Felix Blanke wrote:

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of "losetup /dev/loop2":

/dev/loop2: [0010]:5324 
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128


Thanks you for looking into this!
While writing this I read your second mail. The strace output is attached.

   OK, I've traced through the functions being called, and I really
can't see where it could be truncating the name, unless your system
has a stupidly small value of PATH_MAX.

   Can you apply the following patch (to the "next" branch of the
btrfs-progs git repo), rebuild, and try again? It's just adding some
debugging output to track what it's looking at.

   Hugo.

diff --git a/mkfs.c b/mkfs.c
index 2e99b95..51a5096 100644
--- a/mkfs.c
+++ b/mkfs.c

@@ -422,6 +422,7 @@ int main(int ac, char **av)
 	printf("WARNING! - see http://btrfs.wiki.kernel.org before using\n\n");
 
 	file = av[optind++];
+	printf("Checking whether %s is part of a mounted filesystem\n", file);
 	ret = check_mounted(file);
 	if (ret < 0) {
 		fprintf(stderr, "error checking %s mount status\n", file);

diff --git a/utils.c b/utils.c
index fd894f3..7fa3149 100644
--- a/utils.c
+++ b/utils.c

@@ -610,12 +610,16 @@ int resolve_loop_device(const char* loop_dev, char* loop_file, int max_len)
 	int ret_ioctl;
 	struct loop_info loopinfo;
 
+	printf("Resolving loop device %s (length %d)\n", loop_dev, max_len);
+
 	if ((loop_fd = open(loop_dev, O_RDONLY)) < 0)
 		return -errno;
 
 	ret_ioctl = ioctl(loop_fd, LOOP_GET_STATUS, &loopinfo);
 	close(loop_fd);
 
+	printf("Loop name = %s\n", loopinfo.lo_name);
+
 	if (ret_ioctl == 0)
 		strncpy(loop_file, loopinfo.lo_name, max_len);
 	else

@@ -639,6 +643,9 @@ int is_same_blk_file(const char* a, const char* b)
 		return -errno;
 	}
 
+	printf("Realpath of %s was %s\n", a, real_a);
+	printf("Realpath of %s was %s\n", b, real_b);
+
 	/* Identical path? */
 	if(strcmp(real_a, real_b) == 0)
 		return 1;

@@ -680,6 +687,9 @@ int is_same_loop_file(const char* a, const char* b)
 	const char* final_b;
 	int ret;
 
+	printf("is_same_loop_file: %s and %s\n", a, b);
+	printf("PATH_MAX = %d\n", PATH_MAX);
+
 	/* Resolve a if it is a loop device */
 	if((ret = is_loop_device(a)) < 0) {
 	   return ret;

@@ -784,8 +794,10 @@ int check_mounted(const char* file)
 			if(strcmp(mnt->mnt_type, "btrfs") != 0)
 				continue;
 
+			printf("Testing if btrfs device is in the dev list: %s\n", mnt->mnt_fsname);
 			ret = blk_file_in_dev_list(fs_devices_mnt, mnt->mnt_fsname);
 		} else {
+			printf("Testing if non-btrfs device is block or regular: %s\n", mnt->mnt_fsname);
 			/* ignore entries in the mount table that are not
 			   associated with a file*/
 			if((ret = is_existing_blk_or_reg_file(mnt->mnt_fsname)) < 0)

diff --git a/volumes.c b/volumes.c
index 7671855..2496fbd 100644
--- a/volumes.c
+++ b/volumes.c

@@ -130,6 +130,8 @@ static int device_list_add(const char *path,
 		device->fs_devices = fs_devices;
 	}
 
+	printf("Device added with name %s\n", device->name);
+
 	if (found_transid > fs_devices->latest_trans) {
 		fs_devices->latest_devid = devid;
 		fs_devices->latest_trans = found_transid;

@@ -223,6 +225,7 @@ int btrfs_scan_one_device(int fd, const char *path,
 		*total_devs = btrfs_super_num_devices(disk_super);
 	uuid_unparse(disk_super->fsid, uuidbuf);
 
+	printf("Adding device %s to list\n", path);
 	ret = device_list_add(path, disk_super, devid, fs_devices_ret);
 
 error_brelse:

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
               --- Doughnut furs ache me, Omar Dorlin. ---               

On 01/23/2011 07:18 PM, Hugo Mills wrote:

   Hi, Felix,

On Sat, Jan 22, 2011 at 04:56:12PM +0100, Felix Blanke wrote:

quoted

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of "losetup /dev/loop2":

/dev/loop2: [0010]:5324 
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128


Thanks you for looking into this!
While writing this I read your second mail. The strace output is attached.

   OK, I've traced through the functions being called, and I really
can't see where it could be truncating the name, unless your system
has a stupidly small value of PATH_MAX.

It seems that when mkfs.btrfs checks if the passed block device is
already mounted, uses the ioctl LOOP_GET_STATUS [1]. This ioctl has as
argument the struct loop_info.

This ioctl, should return the info about the back-end of the loop
device. The file name is returned via the "lo_name" field, which is an
array of 64 char...[2]

Felix, what is the output of the following command ?

	/sbin/losetup -a

If my analysis is correct, this command should return the filename
trunked at the 64th character too.

Goffredo

[1] file util.c, function resolve_loop_device
[2]
http://lxr.e2g.org/source/bionic/libc/kernel/common/linux/loop.h?a=ppc#L26
  and
http://lxr.e2g.org/source/bionic/libc/kernel/common/linux/loop.h?a=ppc#L15

Hi Goffredo,

you're damn right :)

scooter ~ # losetup -a
/dev/loop0: [0010]:3154
(/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par) encryption=AES128
/dev/loop1: [0010]:4552
(/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par) encryption=AES128
/dev/loop2: [0010]:4623
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128
/dev/loop3: [0010]:4604
(/dev/disk/by-id/ata-WDC_WD5000AAKS-65YGA0_WD-WCAS82035988-part1) encryption=AES128
/dev/loop4: [0010]:4586
(/dev/disk/by-id/ata-WDC_WD5000AAKS-65YGA0_WD-WCAS82030114-part1) encryption=AES128


Sorry that I didn't saw that earlier. If I would use /dev/sdxx instead of
/dev/disk/by-id/ for decrypting the devices it would be working. But those ids didn't
change if I remove/add a device, so it would be nice to be able to use those ids.


Do you know where  the right place to report that bug is?


Regards,
Felix



On 23. January 2011 - 23:02, Goffredo Baroncelli wrote:

Date: Sun, 23 Jan 2011 23:02:16 +0100
From: Goffredo Baroncelli <redacted>
To: Hugo Mills <redacted>, Felix Blanke
 [off-list ref]
CC: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

On 01/23/2011 07:18 PM, Hugo Mills wrote:

quoted

   Hi, Felix,

On Sat, Jan 22, 2011 at 04:56:12PM +0100, Felix Blanke wrote:

quoted

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of "losetup /dev/loop2":

/dev/loop2: [0010]:5324 
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128


Thanks you for looking into this!
While writing this I read your second mail. The strace output is attached.

   OK, I've traced through the functions being called, and I really
can't see where it could be truncating the name, unless your system
has a stupidly small value of PATH_MAX.

It seems that when mkfs.btrfs checks if the passed block device is
already mounted, uses the ioctl LOOP_GET_STATUS [1]. This ioctl has as
argument the struct loop_info.

This ioctl, should return the info about the back-end of the loop
device. The file name is returned via the "lo_name" field, which is an
array of 64 char...[2]

Felix, what is the output of the following command ?

	/sbin/losetup -a

If my analysis is correct, this command should return the filename
trunked at the 64th character too.

Goffredo

[1] file util.c, function resolve_loop_device
[2]
http://lxr.e2g.org/source/bionic/libc/kernel/common/linux/loop.h?a=ppc#L26
  and
http://lxr.e2g.org/source/bionic/libc/kernel/common/linux/loop.h?a=ppc#L15

quoted

   Can you apply the following patch (to the "next" branch of the
btrfs-progs git repo), rebuild, and try again? It's just adding some
debugging output to track what it's looking at.

   Hugo.

diff --git a/mkfs.c b/mkfs.c
index 2e99b95..51a5096 100644
--- a/mkfs.c
+++ b/mkfs.c

@@ -422,6 +422,7 @@ int main(int ac, char **av)
 	printf("WARNING! - see http://btrfs.wiki.kernel.org before using\n\n");
 
 	file = av[optind++];
+	printf("Checking whether %s is part of a mounted filesystem\n", file);
 	ret = check_mounted(file);
 	if (ret < 0) {
 		fprintf(stderr, "error checking %s mount status\n", file);

diff --git a/utils.c b/utils.c
index fd894f3..7fa3149 100644
--- a/utils.c
+++ b/utils.c

@@ -610,12 +610,16 @@ int resolve_loop_device(const char* loop_dev, char* loop_file, int max_len)
 	int ret_ioctl;
 	struct loop_info loopinfo;
 
+	printf("Resolving loop device %s (length %d)\n", loop_dev, max_len);
+
 	if ((loop_fd = open(loop_dev, O_RDONLY)) < 0)
 		return -errno;
 
 	ret_ioctl = ioctl(loop_fd, LOOP_GET_STATUS, &loopinfo);
 	close(loop_fd);
 
+	printf("Loop name = %s\n", loopinfo.lo_name);
+
 	if (ret_ioctl == 0)
 		strncpy(loop_file, loopinfo.lo_name, max_len);
 	else

@@ -639,6 +643,9 @@ int is_same_blk_file(const char* a, const char* b)
 		return -errno;
 	}
 
+	printf("Realpath of %s was %s\n", a, real_a);
+	printf("Realpath of %s was %s\n", b, real_b);
+
 	/* Identical path? */
 	if(strcmp(real_a, real_b) == 0)
 		return 1;

@@ -680,6 +687,9 @@ int is_same_loop_file(const char* a, const char* b)
 	const char* final_b;
 	int ret;
 
+	printf("is_same_loop_file: %s and %s\n", a, b);
+	printf("PATH_MAX = %d\n", PATH_MAX);
+
 	/* Resolve a if it is a loop device */
 	if((ret = is_loop_device(a)) < 0) {
 	   return ret;

@@ -784,8 +794,10 @@ int check_mounted(const char* file)
 			if(strcmp(mnt->mnt_type, "btrfs") != 0)
 				continue;
 
+			printf("Testing if btrfs device is in the dev list: %s\n", mnt->mnt_fsname);
 			ret = blk_file_in_dev_list(fs_devices_mnt, mnt->mnt_fsname);
 		} else {
+			printf("Testing if non-btrfs device is block or regular: %s\n", mnt->mnt_fsname);
 			/* ignore entries in the mount table that are not
 			   associated with a file*/
 			if((ret = is_existing_blk_or_reg_file(mnt->mnt_fsname)) < 0)

diff --git a/volumes.c b/volumes.c
index 7671855..2496fbd 100644
--- a/volumes.c
+++ b/volumes.c

@@ -130,6 +130,8 @@ static int device_list_add(const char *path,
 		device->fs_devices = fs_devices;
 	}
 
+	printf("Device added with name %s\n", device->name);
+
 	if (found_transid > fs_devices->latest_trans) {
 		fs_devices->latest_devid = devid;
 		fs_devices->latest_trans = found_transid;

@@ -223,6 +225,7 @@ int btrfs_scan_one_device(int fd, const char *path,
 		*total_devs = btrfs_super_num_devices(disk_super);
 	uuid_unparse(disk_super->fsid, uuidbuf);
 
+	printf("Adding device %s to list\n", path);
 	ret = device_list_add(path, disk_super, devid, fs_devices_ret);
 
 error_brelse:

---end quoted text---

Hallo, Felix,

Du meintest am 24.01.11:

Sorry that I didn't saw that earlier. If I would use /dev/sdxx
instead of /dev/disk/by-id/ for decrypting the devices it would be
working. But those ids didn't change if I remove/add a device, so it
would be nice to be able to use those ids.

I prefer working with LABELs. They can be short and self explaining.

Viele Gruesse!
Helmut

Lol :)

You can't use labels for the decryption, because an encrypted device has no label, it
doesn't even have a filesystem and you can't say if there are any kind of data on it
(in my case) :)

You can use labels after you have decrypted the device. Then I'm using labels, too.
But that doesn't help with  this problem :)


Regards,
Felix


On 24. January 2011 - 08:42, Helmut Hullen wrote:

Date:	24 Jan 2011 08:42:00 +0100
From: Helmut Hullen <redacted>
To: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

Hallo, Felix,

Du meintest am 24.01.11:

quoted

Sorry that I didn't saw that earlier. If I would use /dev/sdxx
instead of /dev/disk/by-id/ for decrypting the devices it would be
working. But those ids didn't change if I remove/add a device, so it
would be nice to be able to use those ids.

I prefer working with LABELs. They can be short and self explaining.

Viele Gruesse!
Helmut
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

---end quoted text---

On Sun, Jan 23, 2011 at 11:02:16PM +0100, Goffredo Baroncelli wrote:

On 01/23/2011 07:18 PM, Hugo Mills wrote:

quoted

   Hi, Felix,

On Sat, Jan 22, 2011 at 04:56:12PM +0100, Felix Blanke wrote:

quoted

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of "losetup /dev/loop2":

/dev/loop2: [0010]:5324 
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128


Thanks you for looking into this!
While writing this I read your second mail. The strace output is attached.

   OK, I've traced through the functions being called, and I really
can't see where it could be truncating the name, unless your system
has a stupidly small value of PATH_MAX.

It seems that when mkfs.btrfs checks if the passed block device is
already mounted, uses the ioctl LOOP_GET_STATUS [1]. This ioctl has as
argument the struct loop_info.

This ioctl, should return the info about the back-end of the loop
device. The file name is returned via the "lo_name" field, which is an
array of 64 char...[2]

   Good catch, Goffredo. I completely missed that.

   Interestingly, on my system, lo_name is indeed defined as 64 chars,
but I don't see Felix's problem. When I do losetup on the
/dev/disk/by-id/... link, my version of losetup seems to be following
the link:

# losetup /dev/loop1 /dev/disk/by-id/dm-uuid-LVM-XRQLHQNa0xEeIZL4ofuBGIcfkr1Dhry8YHhkjaw4bvZA4meDFQfEMy5elIsVNeWl 
# losetup -a
/dev/loop1: [0005]:1423915 (/dev/mapper/ruthven-btemp)

   I'm running Debian, and the mount package version 2.17.2-5 (losetup
is part of mount, it seems).

Felix, what is the output of the following command ?

	/sbin/losetup -a

If my analysis is correct, this command should return the filename
trunked at the 64th character too.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
           --- Sometimes, when I'm alone, I Google myself. ---           

Hi,

losetup is part of "util-linux":

http://www.kernel.org/pub/linux/utils/util-linux/


I'm using 2.17.2. Don't know if that is some kind of revision. 


I tried the newest version 2.19-rc1. There seems to be some kind of "fix":

scooter ~ # /home/fame/tmp/util-linux-2.19-rc1/mount/losetup -a
/dev/loop0: [0010]:3154
(/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-pa*), encryption  (type
16)
/dev/loop1: [0010]:4552
(/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-pa*), encryption  (type
16)
/dev/loop2: [0010]:4623
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part*), encryption  (type
16)
/dev/loop3: [0010]:4604
(/dev/disk/by-id/ata-WDC_WD5000AAKS-65YGA0_WD-WCAS82035988-part*), encryption  (type
16)
/dev/loop4: [0010]:4586
(/dev/disk/by-id/ata-WDC_WD5000AAKS-65YGA0_WD-WCAS82030114-part*), encryption  (type
16)



If the path is to long they add a '*' :) Do you think that will solve the problem?


I'll try it tomorrow.


Regards,
Felix


On 23. January 2011 - 23:27, Hugo Mills wrote:

Date: Sun, 23 Jan 2011 23:27:20 +0000
From: Hugo Mills <redacted>
To: kreijack@inwind.it
Cc: Hugo Mills <redacted>, Felix Blanke
 [off-list ref], linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!
Mail-Followup-To: Hugo Mills [off-list ref], kreijack@inwind.it, Hugo
 Mills [off-list ref], Felix Blanke [off-list ref],
 linux-btrfs@vger.kernel.org

On Sun, Jan 23, 2011 at 11:02:16PM +0100, Goffredo Baroncelli wrote:

quoted

On 01/23/2011 07:18 PM, Hugo Mills wrote:

quoted

   Hi, Felix,

On Sat, Jan 22, 2011 at 04:56:12PM +0100, Felix Blanke wrote:

quoted

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of "losetup /dev/loop2":

/dev/loop2: [0010]:5324 
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128


Thanks you for looking into this!
While writing this I read your second mail. The strace output is attached.

   OK, I've traced through the functions being called, and I really
can't see where it could be truncating the name, unless your system
has a stupidly small value of PATH_MAX.

It seems that when mkfs.btrfs checks if the passed block device is
already mounted, uses the ioctl LOOP_GET_STATUS [1]. This ioctl has as
argument the struct loop_info.

This ioctl, should return the info about the back-end of the loop
device. The file name is returned via the "lo_name" field, which is an
array of 64 char...[2]

   Good catch, Goffredo. I completely missed that.

   Interestingly, on my system, lo_name is indeed defined as 64 chars,
but I don't see Felix's problem. When I do losetup on the
/dev/disk/by-id/... link, my version of losetup seems to be following
the link:

# losetup /dev/loop1 /dev/disk/by-id/dm-uuid-LVM-XRQLHQNa0xEeIZL4ofuBGIcfkr1Dhry8YHhkjaw4bvZA4meDFQfEMy5elIsVNeWl 
# losetup -a
/dev/loop1: [0005]:1423915 (/dev/mapper/ruthven-btemp)

   I'm running Debian, and the mount package version 2.17.2-5 (losetup
is part of mount, it seems).

quoted

Felix, what is the output of the following command ?

	/sbin/losetup -a

If my analysis is correct, this command should return the filename
trunked at the 64th character too.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
           --- Sometimes, when I'm alone, I Google myself. ---           

---end quoted text---

On Mon, Jan 24, 2011 at 6:58 AM, Felix Blanke [off-list ref] w=
rote:

I tried the newest version 2.19-rc1. There seems to be some kind of "=

fix":

scooter ~ # /home/fame/tmp/util-linux-2.19-rc1/mount/losetup -a
/dev/loop0: [0010]:3154
(/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-pa*), en=

cryption =A0(type

16)

If the path is to long they add a '*' :) Do you think that will solve=

 the problem?

It shouldn't.

However, since you're using loopback for encryption, why not use
dm-crypt instead?

--=20
=46ajar
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Counter question: Why should I? :)


I used dm-crypt with luks some time ago, but it is really slooow. So I =
talked with
the dm-crypt devs. and they gave me the hint to test loop-aes, for more=
 speed. It is
about 4x faster for me :)

And I don't see any advantages using dm-crypt instead of loop-aes.



=46elix

On 24. January 2011 - 08:53, Fajar A. Nugraha wrote:

Date: Mon, 24 Jan 2011 08:53:41 +0700
From: "Fajar A. Nugraha" <redacted>
To: Felix Blanke <redacted>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!
=20
On Mon, Jan 24, 2011 at 6:58 AM, Felix Blanke [off-list ref]=

 wrote:

quoted

I tried the newest version 2.19-rc1. There seems to be some kind of=

 "fix":

quoted

scooter ~ # /home/fame/tmp/util-linux-2.19-rc1/mount/losetup -a
/dev/loop0: [0010]:3154
(/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-pa*), =

encryption =A0(type

quoted

16)

=20

quoted

If the path is to long they add a '*' :) Do you think that will sol=

ve the problem?

=20
It shouldn't.
=20
However, since you're using loopback for encryption, why not use
dm-crypt instead?
=20
--=20
Fajar

---end quoted text---
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi,

you were talking about the LOOP_GET_STATUS function. I'm not quite sure where does it
came from. Is it part of the kernel? Or does it come from the util-linux package?

I'm searching for the right location where do report that bug :)


Btw: I tested it with util-linux-2.19-rc1. The strace still contains the truncated
path, and no '*'. Therefore I think that ioctl is from the kernel.

Regards,
Felix

On 23. January 2011 - 23:02, Goffredo Baroncelli wrote:

Date: Sun, 23 Jan 2011 23:02:16 +0100
From: Goffredo Baroncelli <redacted>
To: Hugo Mills <redacted>, Felix Blanke
 [off-list ref]
CC: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

On 01/23/2011 07:18 PM, Hugo Mills wrote:

quoted

   Hi, Felix,

On Sat, Jan 22, 2011 at 04:56:12PM +0100, Felix Blanke wrote:

quoted

It was a simple:

mkfs.btrfs -L backup -d single /dev/loop2

But it also happens without the options, like:

mkfs.btrfs /dev/loop2


/dev/loop2 is a loop device, which is aes encrypted. The output of "losetup /dev/loop2":

/dev/loop2: [0010]:5324 
(/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3) encryption=AES128


Thanks you for looking into this!
While writing this I read your second mail. The strace output is attached.

   OK, I've traced through the functions being called, and I really
can't see where it could be truncating the name, unless your system
has a stupidly small value of PATH_MAX.

It seems that when mkfs.btrfs checks if the passed block device is
already mounted, uses the ioctl LOOP_GET_STATUS [1]. This ioctl has as
argument the struct loop_info.

This ioctl, should return the info about the back-end of the loop
device. The file name is returned via the "lo_name" field, which is an
array of 64 char...[2]

Felix, what is the output of the following command ?

	/sbin/losetup -a

If my analysis is correct, this command should return the filename
trunked at the 64th character too.

Goffredo

[1] file util.c, function resolve_loop_device
[2]
http://lxr.e2g.org/source/bionic/libc/kernel/common/linux/loop.h?a=ppc#L26
  and
http://lxr.e2g.org/source/bionic/libc/kernel/common/linux/loop.h?a=ppc#L15

quoted

   Can you apply the following patch (to the "next" branch of the
btrfs-progs git repo), rebuild, and try again? It's just adding some
debugging output to track what it's looking at.

   Hugo.

diff --git a/mkfs.c b/mkfs.c
index 2e99b95..51a5096 100644
--- a/mkfs.c
+++ b/mkfs.c

@@ -422,6 +422,7 @@ int main(int ac, char **av)
 	printf("WARNING! - see http://btrfs.wiki.kernel.org before using\n\n");
 
 	file = av[optind++];
+	printf("Checking whether %s is part of a mounted filesystem\n", file);
 	ret = check_mounted(file);
 	if (ret < 0) {
 		fprintf(stderr, "error checking %s mount status\n", file);

diff --git a/utils.c b/utils.c
index fd894f3..7fa3149 100644
--- a/utils.c
+++ b/utils.c

@@ -610,12 +610,16 @@ int resolve_loop_device(const char* loop_dev, char* loop_file, int max_len)
 	int ret_ioctl;
 	struct loop_info loopinfo;
 
+	printf("Resolving loop device %s (length %d)\n", loop_dev, max_len);
+
 	if ((loop_fd = open(loop_dev, O_RDONLY)) < 0)
 		return -errno;
 
 	ret_ioctl = ioctl(loop_fd, LOOP_GET_STATUS, &loopinfo);
 	close(loop_fd);
 
+	printf("Loop name = %s\n", loopinfo.lo_name);
+
 	if (ret_ioctl == 0)
 		strncpy(loop_file, loopinfo.lo_name, max_len);
 	else

@@ -639,6 +643,9 @@ int is_same_blk_file(const char* a, const char* b)
 		return -errno;
 	}
 
+	printf("Realpath of %s was %s\n", a, real_a);
+	printf("Realpath of %s was %s\n", b, real_b);
+
 	/* Identical path? */
 	if(strcmp(real_a, real_b) == 0)
 		return 1;

@@ -680,6 +687,9 @@ int is_same_loop_file(const char* a, const char* b)
 	const char* final_b;
 	int ret;
 
+	printf("is_same_loop_file: %s and %s\n", a, b);
+	printf("PATH_MAX = %d\n", PATH_MAX);
+
 	/* Resolve a if it is a loop device */
 	if((ret = is_loop_device(a)) < 0) {
 	   return ret;

@@ -784,8 +794,10 @@ int check_mounted(const char* file)
 			if(strcmp(mnt->mnt_type, "btrfs") != 0)
 				continue;
 
+			printf("Testing if btrfs device is in the dev list: %s\n", mnt->mnt_fsname);
 			ret = blk_file_in_dev_list(fs_devices_mnt, mnt->mnt_fsname);
 		} else {
+			printf("Testing if non-btrfs device is block or regular: %s\n", mnt->mnt_fsname);
 			/* ignore entries in the mount table that are not
 			   associated with a file*/
 			if((ret = is_existing_blk_or_reg_file(mnt->mnt_fsname)) < 0)

diff --git a/volumes.c b/volumes.c
index 7671855..2496fbd 100644
--- a/volumes.c
+++ b/volumes.c

@@ -130,6 +130,8 @@ static int device_list_add(const char *path,
 		device->fs_devices = fs_devices;
 	}
 
+	printf("Device added with name %s\n", device->name);
+
 	if (found_transid > fs_devices->latest_trans) {
 		fs_devices->latest_devid = devid;
 		fs_devices->latest_trans = found_transid;

@@ -223,6 +225,7 @@ int btrfs_scan_one_device(int fd, const char *path,
 		*total_devs = btrfs_super_num_devices(disk_super);
 	uuid_unparse(disk_super->fsid, uuidbuf);
 
+	printf("Adding device %s to list\n", path);
 	ret = device_list_add(path, disk_super, devid, fs_devices_ret);
 
 error_brelse:

---end quoted text---

On Mon, Jan 24, 2011 at 02:01:04PM +0100, Felix Blanke wrote:

Hi,

you were talking about the LOOP_GET_STATUS function. I'm not quite sure where does it
came from. Is it part of the kernel? Or does it come from the util-linux package?

   It's an ioctl (number 0x4c03) that works on loop devices, and
returns information about the loop device. Being an ioctl, it's
implemented in the kernel. Unfortunately, since it's part of the
kernel API, the size of the name field is probably fixed for the rest
of time, and so the bug can't be fixed.

I'm searching for the right location where do report that bug :)

   linux-kernel mailing list, I think.

Btw: I tested it with util-linux-2.19-rc1. The strace still contains
the truncated path, and no '*'. Therefore I think that ioctl is from
the kernel.

   Indeed.

   What I find interesting is that my copy of losetup follows symlinks
from the /dev/disk/by-id/... path back to the original device node
(/dev/dm-7 in my test case) before setting up the loop, whereas yours
seems not to.

   I think that that's probably the easiest solution to this problem:
modify losetup to use realpath(3) on the device node it's given.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
                      --- make bzImage, not war ---                      

On 24. January 2011 - 13:13, Hugo Mills wrote:

Date: Mon, 24 Jan 2011 13:13:41 +0000
From: Hugo Mills <redacted>
To: Felix Blanke <redacted>
Cc: kreijack@inwind.it, Hugo Mills <redacted>,
 linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!
Mail-Followup-To: Hugo Mills [off-list ref], Felix Blanke
 [off-list ref], kreijack@inwind.it, linux-btrfs@vger.kernel.org

On Mon, Jan 24, 2011 at 02:01:04PM +0100, Felix Blanke wrote:

quoted

Hi,

you were talking about the LOOP_GET_STATUS function. I'm not quite sure where does it
came from. Is it part of the kernel? Or does it come from the util-linux package?

   It's an ioctl (number 0x4c03) that works on loop devices, and
returns information about the loop device. Being an ioctl, it's
implemented in the kernel. Unfortunately, since it's part of the
kernel API, the size of the name field is probably fixed for the rest
of time, and so the bug can't be fixed.

That sounds great :/

quoted

I'm searching for the right location where do report that bug :)

   linux-kernel mailing list, I think.

quoted

Btw: I tested it with util-linux-2.19-rc1. The strace still contains
the truncated path, and no '*'. Therefore I think that ioctl is from
the kernel.

   Indeed.

   What I find interesting is that my copy of losetup follows symlinks
from the /dev/disk/by-id/... path back to the original device node
(/dev/dm-7 in my test case) before setting up the loop, whereas yours
seems not to.

   I think that that's probably the easiest solution to this problem:
modify losetup to use realpath(3) on the device node it's given.

I dont see where that helps with the problem. If I understand Goffredo correct
mkfs.btrfs is using the ioctl to get the path.

Letting losetup following the link will fix the output of "losetup /dev/loopX", but
it will not fix the truncated path from the ioctl and therefore it will not fix the
problem with mkfs.btrfs.


Felix

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
                      --- make bzImage, not war ---                      

---end quoted text---

On Mon, Jan 24, 2011 at 02:53:05PM +0100, Felix Blanke wrote:

On 24. January 2011 - 13:13, Hugo Mills wrote:

quoted

On Mon, Jan 24, 2011 at 02:01:04PM +0100, Felix Blanke wrote:

quoted

Hi,

you were talking about the LOOP_GET_STATUS function. I'm not quite sure where does it
came from. Is it part of the kernel? Or does it come from the util-linux package?

   It's an ioctl (number 0x4c03) that works on loop devices, and
returns information about the loop device. Being an ioctl, it's
implemented in the kernel. Unfortunately, since it's part of the
kernel API, the size of the name field is probably fixed for the rest
of time, and so the bug can't be fixed.

That sounds great :/

   If you change the layout of the structure, then you have two
incompatible APIs: the one before, and the one after the change. Code
that's compiled to work with one version won't work with the other.
This will cause all manner of pain, and the kernel developers tend
(quite rightly) to come down hard on any attempts to do that kind of
thing.

quoted

quoted

Btw: I tested it with util-linux-2.19-rc1. The strace still contains
the truncated path, and no '*'. Therefore I think that ioctl is from
the kernel.

   Indeed.

   What I find interesting is that my copy of losetup follows symlinks
from the /dev/disk/by-id/... path back to the original device node
(/dev/dm-7 in my test case) before setting up the loop, whereas yours
seems not to.

   I think that that's probably the easiest solution to this problem:
modify losetup to use realpath(3) on the device node it's given.

I dont see where that helps with the problem. If I understand
Goffredo correct mkfs.btrfs is using the ioctl to get the path.
Letting losetup following the link will fix the output of "losetup
/dev/loopX", but it will not fix the truncated path from the ioctl
and therefore it will not fix the problem with mkfs.btrfs.

   What seems to be happening is that at the point that losetup
creates the loop device, it stores the pathname of the underlying
device node in the kernel. When you run losetup -a or mkfs.btrfs,
these use the LOOP_GET_STATUS ioctl to retrieve that pathname. The
pathname that LOOP_GET_STATUS returns is truncated to 64 bytes (63
ASCII characters, plus the zero terminator). losetup -a simply prints
it; mkfs.btrfs tries to use it as a parameter to realpath(3). It is at
this point that the symptoms of breakage occur, because the truncated
name doesn't exist, so realpath returns an error.

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- But people have always eaten people,  / what else is there to ---  
         eat?  / If the Juju had meant us not to eat people / he         
                     wouldn't have made us of meat.                      

On Mon, Jan 24, 2011 at 02:29:36PM +0000, Hugo Mills wrote:

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

   Hmm... Just had a thought: is
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 on
your system a symlink or a device node? What does ls -l say?

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- But people have always eaten people,  / what else is there to ---  
         eat?  / If the Juju had meant us not to eat people / he         
                     wouldn't have made us of meat.                      

On Mon, Jan 24, 2011 at 02:29:36PM +0000, Hugo Mills wrote:

quoted

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

   Hmm... Just had a thought: is
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 on
your system a symlink or a device node? What does ls -l say?

It is a symlink created by udev:

ls -l /dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 
lrwxrwxrwx 1 root root 10 Jan 23 23:39
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 -> ../../sdb3

Following that link should work :)


Felix

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- But people have always eaten people,  / what else is there to ---  
         eat?  / If the Juju had meant us not to eat people / he         
                     wouldn't have made us of meat.                      

---end quoted text---

util-linux-2.18-r1 and still no symlink following.

I'll ask for that at the kernel mailing list in the next days. If your (Hugo)
util-linux doesn't include any kind of patches that behaviour is really strange.


Felix

On 24. January 2011 - 15:44, Felix Blanke wrote:

Date: Mon, 24 Jan 2011 15:44:14 +0100
From: Felix Blanke <redacted>
To: Hugo Mills <redacted>
Cc: kreijack@inwind.it, linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

quoted

On Mon, Jan 24, 2011 at 02:29:36PM +0000, Hugo Mills wrote:

quoted

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

   Hmm... Just had a thought: is
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 on
your system a symlink or a device node? What does ls -l say?

It is a symlink created by udev:

ls -l /dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 
lrwxrwxrwx 1 root root 10 Jan 23 23:39
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 -> ../../sdb3

Following that link should work :)


Felix

quoted

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- But people have always eaten people,  / what else is there to ---  
         eat?  / If the Juju had meant us not to eat people / he         
                     wouldn't have made us of meat.                      

---end quoted text---

---end quoted text---

On Mon, Jan 24, 2011 at 05:52:58PM +0100, Felix Blanke wrote:

util-linux-2.18-r1 and still no symlink following.

I'll ask for that at the kernel mailing list in the next days. If your (Hugo)
util-linux doesn't include any kind of patches that behaviour is really strange.

   Just for reference:

Package description page:
http://packages.debian.org/squeeze/util-linux

Original upstream sources Debian are using:
http://ftp.de.debian.org/debian/pool/main/u/util-linux/util-linux_2.17.2.orig.tar.gz

Debian patches on top of that source (mostly to the build system):
http://ftp.de.debian.org/debian/pool/main/u/util-linux/util-linux_2.17.2-5.diff.gz

   Hugo.

Felix

On 24. January 2011 - 15:44, Felix Blanke wrote:

quoted

Date: Mon, 24 Jan 2011 15:44:14 +0100
From: Felix Blanke <redacted>
To: Hugo Mills <redacted>
Cc: kreijack@inwind.it, linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

quoted

On Mon, Jan 24, 2011 at 02:29:36PM +0000, Hugo Mills wrote:

quoted

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

   Hmm... Just had a thought: is
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 on
your system a symlink or a device node? What does ls -l say?

It is a symlink created by udev:

ls -l /dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 
lrwxrwxrwx 1 root root 10 Jan 23 23:39
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 -> ../../sdb3

Following that link should work :)


Felix

quoted

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- Turning,  pages turning in the widening bath, / The spine ---    
        cannot bear the humidity. / Books fall apart; the binding        
            cannot hold. / Page 129 is loosed upon the world.            

Hi,

it is getting interesting :)


If I'm using the debian util-linux (with the patches) I'm getting an error while
executing losetup:

ioctl: LOOP_SET_STATUS: Invalid argument


Now the interesting part: The strace of the debian util-linux shows:

readlink("/dev", 0x7fffcea2a0a0, 4096)  = -1 EINVAL (Invalid argument)
readlink("/dev/disk", 0x7fffcea2a0a0, 4096) = -1 EINVAL (Invalid argument)
readlink("/dev/disk/by-id", 0x7fffcea2a0a0, 4096) = -1 EINVAL (Invalid argument)
readlink("/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3",
"../../sda3", 4096) = 10
readlink("/dev/sda3", 0x7fffcea2a0a0, 4096) = -1 EINVAL (Invalid argument)


But in the gentoo util-linux there is no readlink in the strace. There are also those
readlinks with the unpatched source from that debian package.

If I use the tarball of gentoo and compile it by my own there are readlinks. I'll
take a look if there are any gentoo-patches applied during the build.

Felix

On 24. January 2011 - 17:00, Hugo Mills wrote:

Date: Mon, 24 Jan 2011 17:00:24 +0000
From: Hugo Mills <redacted>
To: Felix Blanke <redacted>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!
Mail-Followup-To: Hugo Mills [off-list ref], Felix Blanke
 [off-list ref], linux-btrfs@vger.kernel.org

On Mon, Jan 24, 2011 at 05:52:58PM +0100, Felix Blanke wrote:

quoted

util-linux-2.18-r1 and still no symlink following.

I'll ask for that at the kernel mailing list in the next days. If your (Hugo)
util-linux doesn't include any kind of patches that behaviour is really strange.

   Just for reference:

Package description page:
http://packages.debian.org/squeeze/util-linux

Original upstream sources Debian are using:
http://ftp.de.debian.org/debian/pool/main/u/util-linux/util-linux_2.17.2.orig.tar.gz

Debian patches on top of that source (mostly to the build system):
http://ftp.de.debian.org/debian/pool/main/u/util-linux/util-linux_2.17.2-5.diff.gz

   Hugo.

quoted

Felix

On 24. January 2011 - 15:44, Felix Blanke wrote:

quoted

Date: Mon, 24 Jan 2011 15:44:14 +0100
From: Felix Blanke <redacted>
To: Hugo Mills <redacted>
Cc: kreijack@inwind.it, linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

quoted

On Mon, Jan 24, 2011 at 02:29:36PM +0000, Hugo Mills wrote:

quoted

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

   Hmm... Just had a thought: is
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 on
your system a symlink or a device node? What does ls -l say?

It is a symlink created by udev:

ls -l /dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 
lrwxrwxrwx 1 root root 10 Jan 23 23:39
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 -> ../../sdb3

Following that link should work :)


Felix

quoted

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- Turning,  pages turning in the widening bath, / The spine ---    
        cannot bear the humidity. / Books fall apart; the binding        
            cannot hold. / Page 129 is loosed upon the world.            

---end quoted text---

If gentoo is configured for using loop-aes, a patch is applied:

http://loop-aes.sourceforge.net/updates/util-linux-ng-2.17.1-20100308.diff.bz2

Debian doesn't seem to apply that patch, therefore I'm getting that ioctl error and
can't loop my encrypted devices.

Now everything does make sense :) The difference is somewhere in that patch. It's a
huge one. Maybe I'll talk to the loop-aes maintainer (I'm subscribed to that list)
and try to deep into that.

It is def. not a btrfs related problem. If anyone wants to keep on track with my
investigation just let me now, otherwise I'll stop posting about this in the
btrfs-list.


Thanks for your help!


Felix


On 24. January 2011 - 22:04, Felix Blanke wrote:

Date: Mon, 24 Jan 2011 22:04:54 +0100
From: Felix Blanke <redacted>
To: Hugo Mills <redacted>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

Hi,

it is getting interesting :)


If I'm using the debian util-linux (with the patches) I'm getting an error while
executing losetup:

ioctl: LOOP_SET_STATUS: Invalid argument


Now the interesting part: The strace of the debian util-linux shows:

readlink("/dev", 0x7fffcea2a0a0, 4096)  = -1 EINVAL (Invalid argument)
readlink("/dev/disk", 0x7fffcea2a0a0, 4096) = -1 EINVAL (Invalid argument)
readlink("/dev/disk/by-id", 0x7fffcea2a0a0, 4096) = -1 EINVAL (Invalid argument)
readlink("/dev/disk/by-id/ata-WDC_WD6400AAKS-22A7B2_WD-WCASY7780706-part3",
"../../sda3", 4096) = 10
readlink("/dev/sda3", 0x7fffcea2a0a0, 4096) = -1 EINVAL (Invalid argument)


But in the gentoo util-linux there is no readlink in the strace. There are also those
readlinks with the unpatched source from that debian package.

If I use the tarball of gentoo and compile it by my own there are readlinks. I'll
take a look if there are any gentoo-patches applied during the build.

Felix

On 24. January 2011 - 17:00, Hugo Mills wrote:

quoted

Date: Mon, 24 Jan 2011 17:00:24 +0000
From: Hugo Mills <redacted>
To: Felix Blanke <redacted>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!
Mail-Followup-To: Hugo Mills [off-list ref], Felix Blanke
 [off-list ref], linux-btrfs@vger.kernel.org

On Mon, Jan 24, 2011 at 05:52:58PM +0100, Felix Blanke wrote:

quoted

util-linux-2.18-r1 and still no symlink following.

I'll ask for that at the kernel mailing list in the next days. If your (Hugo)
util-linux doesn't include any kind of patches that behaviour is really strange.

   Just for reference:

Package description page:
http://packages.debian.org/squeeze/util-linux

Original upstream sources Debian are using:
http://ftp.de.debian.org/debian/pool/main/u/util-linux/util-linux_2.17.2.orig.tar.gz

Debian patches on top of that source (mostly to the build system):
http://ftp.de.debian.org/debian/pool/main/u/util-linux/util-linux_2.17.2-5.diff.gz

   Hugo.

quoted

Felix

On 24. January 2011 - 15:44, Felix Blanke wrote:

quoted

Date: Mon, 24 Jan 2011 15:44:14 +0100
From: Felix Blanke <redacted>
To: Hugo Mills <redacted>
Cc: kreijack@inwind.it, linux-btrfs@vger.kernel.org
Subject: Re: Bug in mkfs.btrfs?!

quoted

On Mon, Jan 24, 2011 at 02:29:36PM +0000, Hugo Mills wrote:

quoted

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

   Hmm... Just had a thought: is
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 on
your system a symlink or a device node? What does ls -l say?

It is a symlink created by udev:

ls -l /dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 
lrwxrwxrwx 1 root root 10 Jan 23 23:39
/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-part3 -> ../../sdb3

Following that link should work :)


Felix

quoted

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- Turning,  pages turning in the widening bath, / The spine ---    
        cannot bear the humidity. / Books fall apart; the binding        
            cannot hold. / Page 129 is loosed upon the world.            

---end quoted text---

---end quoted text---

On Mon, Jan 24, 2011 at 02:53:05PM +0100, Felix Blanke wrote:

quoted

On 24. January 2011 - 13:13, Hugo Mills wrote:

quoted

On Mon, Jan 24, 2011 at 02:01:04PM +0100, Felix Blanke wrote:

quoted

Hi,

you were talking about the LOOP_GET_STATUS function. I'm not quite sure where does it
came from. Is it part of the kernel? Or does it come from the util-linux package?

   It's an ioctl (number 0x4c03) that works on loop devices, and
returns information about the loop device. Being an ioctl, it's
implemented in the kernel. Unfortunately, since it's part of the
kernel API, the size of the name field is probably fixed for the rest
of time, and so the bug can't be fixed.

That sounds great :/

   If you change the layout of the structure, then you have two
incompatible APIs: the one before, and the one after the change. Code
that's compiled to work with one version won't work with the other.
This will cause all manner of pain, and the kernel developers tend
(quite rightly) to come down hard on any attempts to do that kind of
thing.

quoted

quoted

quoted

Btw: I tested it with util-linux-2.19-rc1. The strace still contains
the truncated path, and no '*'. Therefore I think that ioctl is from
the kernel.

   Indeed.

   What I find interesting is that my copy of losetup follows symlinks
from the /dev/disk/by-id/... path back to the original device node
(/dev/dm-7 in my test case) before setting up the loop, whereas yours
seems not to.

   I think that that's probably the easiest solution to this problem:
modify losetup to use realpath(3) on the device node it's given.

quoted

I dont see where that helps with the problem. If I understand
Goffredo correct mkfs.btrfs is using the ioctl to get the path.
Letting losetup following the link will fix the output of "losetup
/dev/loopX", but it will not fix the truncated path from the ioctl
and therefore it will not fix the problem with mkfs.btrfs.

   What seems to be happening is that at the point that losetup
creates the loop device, it stores the pathname of the underlying
device node in the kernel. When you run losetup -a or mkfs.btrfs,
these use the LOOP_GET_STATUS ioctl to retrieve that pathname. The
pathname that LOOP_GET_STATUS returns is truncated to 64 bytes (63
ASCII characters, plus the zero terminator). losetup -a simply prints
it; mkfs.btrfs tries to use it as a parameter to realpath(3). It is at
this point that the symptoms of breakage occur, because the truncated
name doesn't exist, so realpath returns an error.

   If, instead, the initial losetup call tracked the symlinks back to
the original device node (i.e. something like "/dev/sdb3", or
"/dev/mapper/ruthven-btest" in my example), then the name that's
stored in the kernel would be shorter, and we'd be less likely to see
the truncation. This is what my copy of losetup seems to be doing. I
can't see any distribution-specific patches in the source for
util-linux that would do this, though.

Ah, I see :)
It is strange that your util-linux does follow the link. 

I tried to read the informations with the newest (original) util-linux. But the
assignment of the /dev/loopX was done with an older util-linux. I'll try to use the
new util-linux to decrypt a device and look what losetup is reporting. Maybe that
will solve the problem and it's just my "old" version of util-linux which has that
problem.


Felix

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- But people have always eaten people,  / what else is there to ---  
         eat?  / If the Juju had meant us not to eat people / he         
                     wouldn't have made us of meat.                      

---end quoted text---

/*
 * CC'd to linux-kernel in case they have any feedback on this.
 *
 * Long thread, trying to work out why mkfs.btrfs failed to
 * make a filesystem on an encrypted loopback mount called
 * /dev/loop2. Cause turned out to be mkfs.btrfs calling
 * LOOP_GET_STATUS to find out if the block device was mounted
 * and getting a truncated device name back and so it later
 * fails when lstat() is called on the truncated device path.
 *
 * The long device name for the encrypted loopback mount was
 * because /dev/disk/by-id/$ID was used when Felix created it
 * to cope with devices moving around.
 */

On 25/01/11 00:01, Felix Blanke wrote:

you were talking about the LOOP_GET_STATUS function. I'm not
quite sure where does it came from. Is it part of the kernel?
Or does it come from the util-linux package?

It's in the kernel, and there is both LOOP_GET_STATUS (old
implementation) and LOOP_GET_STATUS64 (new implementation).

They return structures called loop_info and loop_info64
respectively and both are defined in include/linux/loop.h .

Sadly in both cases the lengths of paths are defined to be
LO_NAME_SIZE which is currently 64 and hence either
implementation will cause the problematic:

lstat("/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par",
0x7fffa30b3cf0) = -1 ENOENT (No such file or directory)

I've CC'd this to the LKML in case they have any feedback on
this apparent problem with the API.

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

On Tue, Jan 25, 2011 at 11:15:11AM +1100, Chris Samuel wrote:

/*
 * CC'd to linux-kernel in case they have any feedback on this.
 *
 * Long thread, trying to work out why mkfs.btrfs failed to
 * make a filesystem on an encrypted loopback mount called
 * /dev/loop2. Cause turned out to be mkfs.btrfs calling
 * LOOP_GET_STATUS to find out if the block device was mounted
 * and getting a truncated device name back and so it later
 * fails when lstat() is called on the truncated device path.
 *
 * The long device name for the encrypted loopback mount was
 * because /dev/disk/by-id/$ID was used when Felix created it
 * to cope with devices moving around.
 */

On 25/01/11 00:01, Felix Blanke wrote:

quoted

you were talking about the LOOP_GET_STATUS function. I'm not
quite sure where does it came from. Is it part of the kernel?
Or does it come from the util-linux package?

It's in the kernel, and there is both LOOP_GET_STATUS (old
implementation) and LOOP_GET_STATUS64 (new implementation).

They return structures called loop_info and loop_info64
respectively and both are defined in include/linux/loop.h .

Sadly in both cases the lengths of paths are defined to be
LO_NAME_SIZE which is currently 64 and hence either
implementation will cause the problematic:

lstat("/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par",
0x7fffa30b3cf0) = -1 ENOENT (No such file or directory)

I've CC'd this to the LKML in case they have any feedback on
this apparent problem with the API.

 
Since 2.6.37, you can get full path to the backing file from sys:
cat /sys/block/loopX/loop/backing_file

See
http://linux.derkeiler.com/Mailing-Lists/Kernel/2010-07/msg10996.html


HTH,

Petr

--
Petr Uzel
IRC: ptr_uzl @ freenode

Hi,

are you sure that patch is in the kernel?

I'm using 2.6.37 and don't have those attribues in my /sys.



Felix

On 10. February 2011 - 13:29, Petr Uzel wrote:

Date: Thu, 10 Feb 2011 13:29:27 +0100
From: Petr Uzel <redacted>
To: Chris Samuel <redacted>
Cc: Felix Blanke <redacted>, kreijack@inwind.it, Hugo Mills
 [off-list ref], linux-btrfs@vger.kernel.org, Linux Kernel
 [off-list ref]
Subject: Re: LOOP_GET_STATUS(64) truncates pathnames to 64 chars (was Re:
 Bug in mkfs.btrfs?!)
Mail-Followup-To: Chris Samuel [off-list ref], Felix Blanke
 [off-list ref], kreijack@inwind.it, Hugo Mills
 [off-list ref], linux-btrfs@vger.kernel.org, Linux Kernel
 [off-list ref]

On Tue, Jan 25, 2011 at 11:15:11AM +1100, Chris Samuel wrote:

quoted

/*
 * CC'd to linux-kernel in case they have any feedback on this.
 *
 * Long thread, trying to work out why mkfs.btrfs failed to
 * make a filesystem on an encrypted loopback mount called
 * /dev/loop2. Cause turned out to be mkfs.btrfs calling
 * LOOP_GET_STATUS to find out if the block device was mounted
 * and getting a truncated device name back and so it later
 * fails when lstat() is called on the truncated device path.
 *
 * The long device name for the encrypted loopback mount was
 * because /dev/disk/by-id/$ID was used when Felix created it
 * to cope with devices moving around.
 */

On 25/01/11 00:01, Felix Blanke wrote:

quoted

you were talking about the LOOP_GET_STATUS function. I'm not
quite sure where does it came from. Is it part of the kernel?
Or does it come from the util-linux package?

It's in the kernel, and there is both LOOP_GET_STATUS (old
implementation) and LOOP_GET_STATUS64 (new implementation).

They return structures called loop_info and loop_info64
respectively and both are defined in include/linux/loop.h .

Sadly in both cases the lengths of paths are defined to be
LO_NAME_SIZE which is currently 64 and hence either
implementation will cause the problematic:

lstat("/dev/disk/by-id/ata-INTEL_SSDSA2M160G2GC_CVPO939201JX160AGN-par",
0x7fffa30b3cf0) = -1 ENOENT (No such file or directory)

I've CC'd this to the LKML in case they have any feedback on
this apparent problem with the API.

 
Since 2.6.37, you can get full path to the backing file from sys:
cat /sys/block/loopX/loop/backing_file

See
http://linux.derkeiler.com/Mailing-Lists/Kernel/2010-07/msg10996.html


HTH,

Petr

--
Petr Uzel
IRC: ptr_uzl @ freenode

---end quoted text---

On 02/11/2011 02:04 PM, Felix Blanke wrote:

I'm using 2.6.37 and don't have those attribues in my /sys.

These attributes are there only when the loop device is configured,
IOW when backing file is attached.


Milan