[bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"
From: KOSAKI Motohiro <hidden>
Date: 2008-07-03 12:01:09
42 messages, 13 authors, 2008-07-05 · open the first message on its own page
From: KOSAKI Motohiro <hidden>
Date: 2008-07-03 12:01:09
Hi Michael, my server output following error message on 2.6.26-rc8-mm1. Is this a bug? ------------------------------------------------------------------ tg3.c:v3.93 (May 22, 2008) GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51 firmware: requesting tigon/tg3_tso.bin tg3: Failed to load firmware "tigon/tg3_tso.bin" tg3 0000:06:01.0: PCI INT A disabled GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered tg3: probe of 0000:06:01.0 failed with error -2 GSI 73 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52 firmware: requesting tigon/tg3_tso.bin
From: Jeff Garzik <hidden>
Date: 2008-07-03 12:21:40
KOSAKI Motohiro wrote:
Hi Michael, my server output following error message on 2.6.26-rc8-mm1. Is this a bug? ------------------------------------------------------------------ tg3.c:v3.93 (May 22, 2008) GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51 firmware: requesting tigon/tg3_tso.bin tg3: Failed to load firmware "tigon/tg3_tso.bin" tg3 0000:06:01.0: PCI INT A disabled GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered tg3: probe of 0000:06:01.0 failed with error -2 GSI 73 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52 firmware: requesting tigon/tg3_tso.bin
This change did not come from the network developers or Broadcom, so someone else broke tg3 in -mm... Jeff
From: Hugh Dickins <hidden>
Date: 2008-07-03 13:04:50
On Thu, 3 Jul 2008, Jeff Garzik wrote:
KOSAKI Motohiro wrote:quoted
Hi Michael, my server output following error message on 2.6.26-rc8-mm1. Is this a bug? ------------------------------------------------------------------ tg3.c:v3.93 (May 22, 2008) GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51 firmware: requesting tigon/tg3_tso.bin tg3: Failed to load firmware "tigon/tg3_tso.bin" tg3 0000:06:01.0: PCI INT A disabled GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered tg3: probe of 0000:06:01.0 failed with error -2 GSI 73 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52 firmware: requesting tigon/tg3_tso.binThis change did not come from the network developers or Broadcom, so someone else broke tg3 in -mm...
I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y. That caught me out on PowerMac G5 trying mmotm yesterday, it just hung for a few minutes in earlyish boot with a message about tg3_tso.bin, and then proceeded to boot up but without the network. I was unclear whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor. I avoid initrd, and have tigon3 built in, if that's of any relevance. I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5: mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y). Hugh
From: Jeff Garzik <hidden>
Date: 2008-07-03 13:11:32
Hugh Dickins wrote:
On Thu, 3 Jul 2008, Jeff Garzik wrote:quoted
KOSAKI Motohiro wrote:quoted
Hi Michael, my server output following error message on 2.6.26-rc8-mm1. Is this a bug? ------------------------------------------------------------------ tg3.c:v3.93 (May 22, 2008) GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51 firmware: requesting tigon/tg3_tso.bin tg3: Failed to load firmware "tigon/tg3_tso.bin" tg3 0000:06:01.0: PCI INT A disabled GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered tg3: probe of 0000:06:01.0 failed with error -2 GSI 73 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52 firmware: requesting tigon/tg3_tso.binThis change did not come from the network developers or Broadcom, so someone else broke tg3 in -mm...I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y. That caught me out on PowerMac G5 trying mmotm yesterday, it just hung for a few minutes in earlyish boot with a message about tg3_tso.bin, and then proceeded to boot up but without the network. I was unclear whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor. I avoid initrd, and have tigon3 built in, if that's of any relevance. I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5: mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y).
dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in. Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage. Jeff
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 13:33:46
On Thu, 2008-07-03 at 09:11 -0400, Jeff Garzik wrote:
Hugh Dickins wrote:quoted
On Thu, 3 Jul 2008, Jeff Garzik wrote:quoted
KOSAKI Motohiro wrote:quoted
Hi Michael, my server output following error message on 2.6.26-rc8-mm1. Is this a bug? ------------------------------------------------------------------ tg3.c:v3.93 (May 22, 2008) GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51 firmware: requesting tigon/tg3_tso.bin tg3: Failed to load firmware "tigon/tg3_tso.bin" tg3 0000:06:01.0: PCI INT A disabled GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered tg3: probe of 0000:06:01.0 failed with error -2 GSI 73 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52 firmware: requesting tigon/tg3_tso.binThis change did not come from the network developers or Broadcom, so someone else broke tg3 in -mm...I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y. That caught me out on PowerMac G5 trying mmotm yesterday, it just hung for a few minutes in earlyish boot with a message about tg3_tso.bin, and then proceeded to boot up but without the network. I was unclear whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor.
I shall respectfully refrain from commenting on the likelihood of the
former. With regard to the latter, here is the help text for the
FIRMWARE_IN_KERNEL option:
help
The kernel source tree includes a number of firmware 'blobs'
which are used by various drivers. The recommended way to
use these is to run "make firmware_install" and to copy the
resulting binary files created in usr/lib/firmware directory
of the kernel tree to the /lib/firmware on your system so
that they can be loaded by userspace helpers on request.
Enabling this option will build each required firmware blob
into the kernel directly, where request_firmware() will find
them without having to call out to userspace. This may be
useful if your root file system requires a device which uses
such firmware, and do not wish to use an initrd.
This single option controls the inclusion of firmware for
every driver which usees request_firmare() and ships its
firmware in the kernel source tree, to avoid a proliferation
of 'Include firmware for xxx device' options.
Say 'N' and let firmware be loaded from userspace.
If you think you can improve it, please let me have a revised attempt.
quoted
I avoid initrd, and have tigon3 built in, if that's of any relevance. I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5: mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y).dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in.
Complete nonsense. Setting CONFIG_FIRMWARE_IN_KERNEL isn't hard. But shouldn't be the _default_, either.
Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage.
They haven't even been reviewed. Nobody seems to have actually looked at the real changes (in particular, and commented on whether the device can run anyway without the TSO firmware being loaded, as some people seem to report). You're just throwing your toys out of the pram because of the 'default n' on a patch about 30 commits earlier in my tree. -- dwmw2
From: Jeff Garzik <hidden>
Date: 2008-07-03 13:39:00
David Woodhouse wrote:
quoted
dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in.Complete nonsense. Setting CONFIG_FIRMWARE_IN_KERNEL isn't hard. But shouldn't be the _default_, either.quoted
Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage.They haven't even been reviewed. Nobody seems to have actually looked at
Yes, they have. You just didn't like the answers you received. In particular, the Kconfig default for built-in tg3 firmware should result in the current behavior, without the user having to take extra steps. Because of your stubborn refusal on this Kconfig defaults issue, WE ALREADY HAVE DRIVER-DOES-NOT-WORK BREAKAGE, JUST AS PREDICTED. Wake up and smell reality. Please. Jeff
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 13:53:22
On Thu, 2008-07-03 at 09:38 -0400, Jeff Garzik wrote:
David Woodhouse wrote:quoted
quoted
dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in.Complete nonsense. Setting CONFIG_FIRMWARE_IN_KERNEL isn't hard. But shouldn't be the _default_, either.quoted
Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage.They haven't even been reviewed. Nobody seems to have actually looked atYes, they have. You just didn't like the answers you received.
I received no comment on any part of the changes within tg3.c; only whining about the default behaviour -- which isn't even _set_ as part of the patch in question, any more.
In particular, the Kconfig default for built-in tg3 firmware should result in the current behavior, without the user having to take extra steps.
After feedback from a number of people, there is no individual Kconfig option for the various firmwares; there is only one which controls them all -- CONFIG_FIRMWARE_IN_KERNEL. The thing you're whining about isn't even part of the patch which needs review.
Because of your stubborn refusal on this Kconfig defaults issue, WE ALREADY HAVE DRIVER-DOES-NOT-WORK BREAKAGE, JUST AS PREDICTED.
I strongly disagree that CONFIG_FIRMWARE_IN_KERNEL=y should be the default. But if I add this patch elsewhere in the kernel, will you quit your whining and actually review the patch you were asked to review? ...
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index 339c148..d47482f 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig@@ -37,6 +37,7 @@ config FW_LOADER config FIRMWARE_IN_KERNEL bool "Include in-kernel firmware blobs in kernel binary" depends on FW_LOADER + default y help The kernel source tree includes a number of firmware 'blobs' which are used by various drivers. The recommended way to
--
dwmw2
From: Chuck Lever <hidden>
Date: 2008-07-03 16:11:07
On Jul 3, 2008, at 7:59 AM, KOSAKI Motohiro wrote:
Hi Michael, my server output following error message on 2.6.26-rc8-mm1. Is this a bug? ------------------------------------------------------------------ tg3.c:v3.93 (May 22, 2008) GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51 firmware: requesting tigon/tg3_tso.bin tg3: Failed to load firmware "tigon/tg3_tso.bin" tg3 0000:06:01.0: PCI INT A disabled GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered tg3: probe of 0000:06:01.0 failed with error -2 GSI 73 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52 firmware: requesting tigon/tg3_tso.bin
Same problem here with linux-next on a Dell Latitude D620. -- Chuck Lever chuck[dot]lever[at]oracle[dot]com
From: Theodore Tso <tytso@mit.edu>
Date: 2008-07-03 17:33:20
On Thu, Jul 03, 2008 at 02:52:55PM +0100, David Woodhouse wrote:
After feedback from a number of people, there is no individual Kconfig option for the various firmwares; there is only one which controls them all -- CONFIG_FIRMWARE_IN_KERNEL. The thing you're whining about isn't even part of the patch which needs review.quoted
Because of your stubborn refusal on this Kconfig defaults issue, WE ALREADY HAVE DRIVER-DOES-NOT-WORK BREAKAGE, JUST AS PREDICTED.I strongly disagree that CONFIG_FIRMWARE_IN_KERNEL=y should be the default. But if I add this patch elsewhere in the kernel, will you quit your whining and actually review the patch you were asked to review? ...
I don't think it's whining. If your patch introduces changes which cause people .config to break by default after upgrading to a newer kernel and doing "make oldconfig" --- then that's a problem with your patch, and the missing hunk to enable CONFIG_FIRMWARE_IN_KERNEL=y is critically important. Linus has ruled this way in the past, when he's gotten screwed by this sort of issue in the past, and he was justifiably annoyed. We should treat the users who are willing to test and provide feedback on the latest kernel.org kernels with the same amount of regard. And if there are licensing religious fundamentalists who feel strongly about the firmware issue, then fine, they can change the .config. But the default should be to avoid users from having broken kernels, and a number of (quite clueful) users have already demonstrated that without setting CONFIG_FIRMWARE_IN_KERNEL=y as the default, your patches cause breakage. Regards, - Ted
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 18:56:33
On Thu, 2008-07-03 at 13:30 -0400, Theodore Tso wrote:
I don't think it's whining.
Neither is it an adequate review of the actual patch which was submitted.
If your patch introduces changes which cause people .config to break by default after upgrading to a newer kernel and doing "make oldconfig"
They had to 'make oldconfig' and then actually _choose_ to say 'no' to an option which is fairly clearly documented, that they are the relatively unusual position of wanting to have said 'yes' to. You're getting into Aunt Tillie territory, when you complain about that. Although it does make me wonder if it was better the way I had it originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL attached to each driver, rather than a single FIRMWARE_IN_KERNEL option which controls them all. Perhaps one way to help Aunt Tillie would be to tweak Kbuild to look at the MODULE_FIRMWARE() statements for in-kernel drivers, and to print a warning when the build finishes: "Your static kernel image may require the following firmware files, which are not included: ..." It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y', because the _normal_ setting for that option _really_ should be 'N'. Using request_firmware() satisfied from userspace is best practice these days, and almost all recent drivers do it that way _unconditionally_ anyway. What we're doing now is just cleaning up the older drivers which don't use request_firmware(), to conform to what is now common practice. And while we're retaining the _option_ to continue to build their firmware into the static kernel image, it isn't recommended and really shouldn't be the default configuration.
Linus has ruled this way in the past, when he's gotten screwed by this sort of issue in the past, and he was justifiably annoyed.
I am content to let Linus decide on what the default for the FIRMWARE_IN_KERNEL option will be. I am adamant that it _should_ be 'N', but it's easy enough for Linus to overrule me with a one-line change. In the meantime, it would be useful if Jeff would quit throwing his toys out of the pram on that issue and actually review the _code_ changes. In particular, are the reports correct that the device operates just fine without the TSO firmware loaded? Should we change the request_firmware() error path to just disable TSO and continue with the initialisation? I can understand why he might not want to answer that if the answer is affirmative, I suppose -- it detracts even _further_ from his already rather dubious argument about 'breaking' the driver, if it'll actually continue to work even when the firmware is completely absent. But it would be nice to get an honest and straightforward review of the code from _someone_ who actually knows the hardware.
And if there are licensing religious fundamentalists who feel strongly about the firmware issue, then fine, they can change the .config.
Less of the ad hominem, please. Especially when it's so misdirected. Updating these drivers to remove large blobs of static unswappable data from the kernel, and having it provided from userspace on demand as modern Linux drivers do, is a perfectly sensible technical goal all on its own. And given the GPL's explicit provisions with regard to collective works there are also entirely reasonable, non-"fundamentalist" grounds for believing that it _may_ pose a licensing problem, and for wanting to err on the side of caution in that respect too. Fedora is almost certain to ship with CONFIG_FIRMWARE_IN_KERNEL=n, and I'd be very surprised if Debian and other major distributions don't follow suit. It is the sensible, pragmatic, technically sound choice.
But the default should be to avoid users from having broken kernels, and a number of (quite clueful) users have already demonstrated that without setting CONFIG_FIRMWARE_IN_KERNEL=y as the default, your patches cause breakage.
By this argument, shouldn't we include images in the static kernel for _all_ drivers which currently use request_firmware()? Otherwise, it's possible for the user to 'break' them, right? -- dwmw2
From: David Miller <davem@davemloft.net>
Date: 2008-07-03 20:34:41
From: Jeff Garzik <redacted> Date: Thu, 03 Jul 2008 09:11:09 -0400
dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in. Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage.
I agree with Jeff, obviously. We both saw this song and dance coming. Now the reports are coming in from confused people who are losing their network. It is no surprise. And the person who introduced this swath of regressions acts like it's some kind of chore to enforce the obviously correct default behavior. Why is it such a big deal to make "obviously working" the default? In effect, you lied to us, in that you said that by default users wouldn't have to do anything to keep getting a working setup. But that is provably not true, look at all of these reports. Are you saying these people are idiots and don't know how to configure their kernel? Every single one of them? So don't be surprised how pissed off some of us are about these changes. You are inflicting pain on driver maintainers because now they have to sift through these "firmware not found" reports in addition to their normal workload. And David make it seem like it's inconvenient for him to implement the correct default, which in particular pisses me personally off the most. It's totally irresponsible, and I don't care what the legal or ideological motivation is. Given that, how in the world can you be surprised that the effected driver maintainers have no interest in reviewing the substance of these patches? You don't piss people off, then say "help me review this stuff." It doesn't work like that.
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 20:55:19
On Thu, 2008-07-03 at 13:34 -0700, David Miller wrote:
From: Jeff Garzik <redacted> Date: Thu, 03 Jul 2008 09:11:09 -0400quoted
dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in. Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage.I agree with Jeff, obviously. We both saw this song and dance coming. Now the reports are coming in from confused people who are losing their network. It is no surprise. And the person who introduced this swath of regressions acts like it's some kind of chore to enforce the obviously correct default behavior. Why is it such a big deal to make "obviously working" the default?
In effect, you lied to us, in that you said that by default users wouldn't have to do anything to keep getting a working setup.
No, I didn't lie to you. The conversation, in case you forgot, went like this... On Wed, 2008-06-18 at 16:23 -0700, David Miller wrote:
On Thu, 2008-06-19 at 00:16 +0100, David Woodhouse wrote:quoted
On Wed, 2008-06-18 at 16:05 -0700, David Miller wrote:quoted
Tell me this, how can I (with the default config option settings) netboot properly without an initial ramdisk after these tg3 patches and still get the proper firmware?I suppose the facetious answer is that you can't, just as you can't do it with a default config _before_ these patches -- because neither CONFIG_TIGON3 nor the various options you need for nfsroot are enabled by default. But if you _have_ a working nfsroot+tg3 config and you apply these patches, then all you need to do is say 'y' when you're asked if you want to include the firmware for it: CONFIG_TIGON3_FIRMWARE=y If you are competent enough to get nfsroot working, it isn't really very credible to claim you lack the wit to say 'y' when asked that question, surely? Solving that problem was step #1 in the process of converting drivers to use request_firmware(). You _have_ to be able to build the firmware into the kernel image, and you _can_.Fair enough. I'll step back and let you work out the objections Jeff raised.
Since then, I responded to feedback and changed the individual CONFIG_XXX_FIRMWARE options to a single CONFIG_FIRMWARE_IN_KERNEL which controls them all, but basically nothing has changed. What you accepted then is still true. On Thu, 2008-07-03 at 13:34 -0700, David Miller wrote: So don't be surprised how pissed off some of us are about these
changes. You are inflicting pain on driver maintainers because now they have to sift through these "firmware not found" reports in addition to their normal workload.
It also improves coverage testing, and found a real bug in the failure path...
And David make it seem like it's inconvenient for him to implement the correct default, which in particular pisses me personally off the most. It's totally irresponsible, and I don't care what the legal or ideological motivation is.
It's not inconvenient at all. It's just the _wrong_ default. But if we really can't get past it otherwise, then let's just set it to 'y' for now. I've committed the following, which will appear in linux-next tomorrow. Now, can someone _please_ give me a straight response to the allegation that the TSO firmware on the tg3 is _optional_ anyway, and that it can work without it? If that's true, we should fix the code path where request_firmware() fails, so it doesn't abort the initialisation. (And most of the whining about the driver being 'broken' is nonsense too.) ----
From 400f1b05a9707bd181a044877ca590e87c400749 Mon Sep 17 00:00:00 2001
From: David Woodhouse <dwmw2@infradead.org> Date: Thu, 3 Jul 2008 21:36:11 +0100 Subject: [PATCH] firmware: default CONFIG_FIRMWARE_IN_KERNEL=y This is obviously the wrong thing to do in the long (or even medium) term -- since the recommended way of handling firmware, as used _unconditionally_ by modern drivers, is to rely on request_firmware() being satisfied from userspace rather than keeping the firmware in unswappable static kernel memory. But this change preserves the property, for now, that the fixes to make older drivers use request_firmware() introduce _no_ "regressions" when Aunt Tillie runs 'make oldconfig' and accepts the defaults without looking at what she's doing. Signed-off-by: David Woodhouse <dwmw2@infradead.org> --- drivers/base/Kconfig | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index 339c148..d47482f 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig@@ -37,6 +37,7 @@ config FW_LOADER config FIRMWARE_IN_KERNEL bool "Include in-kernel firmware blobs in kernel binary" depends on FW_LOADER + default y help The kernel source tree includes a number of firmware 'blobs' which are used by various drivers. The recommended way to
--
1.5.5.1
--
dwmw2
From: Jeff Garzik <hidden>
Date: 2008-07-03 21:03:48
David Woodhouse wrote:
Although it does make me wonder if it was better the way I had it originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL attached to each driver, rather than a single FIRMWARE_IN_KERNEL option which controls them all.
IMO, individual options would be better. Plus, unless I am misunderstanding, the firmware is getting built into the kernel image not the tg3 module? If that is the case, then that creates problems by not moving with the driver. If that is not the case, all good. Jeff
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 21:33:59
Jeff Garzik wrote:
David Woodhouse wrote:quoted
Although it does make me wonder if it was better the way I had it originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL attached to each driver, rather than a single FIRMWARE_IN_KERNEL option which controls them all.IMO, individual options would be better.
They had individual options for a long time, but the consensus was that I should remove them -- a consensus which was probably right. It was moderately inconvenient going back through it all and recommitting it without that, but it was worth it to get it right...
Plus, unless I am misunderstanding, the firmware is getting built into the kernel image not the tg3 module?
That's right, although it doesn't really matter when they're both in the vmlinux. When it's actually a module, there really is no good reason not to let request_firmware() get satisfied from userspace. If you can load modules, then you can load firmware too -- the required udev stuff has been there as standard for a _long_ time, as most modern drivers _require_ it without even giving you the built-in-firmware option at all. It makes no more sense to object to that than it does to object to the module depending on _other_ modules. Both those other modules, and the required firmware, are _installed_ by the kernel Makefiles, after all. It wouldn't be _impossible_ to put firmware blobs into the foo.ko files themselves and find them there. The firmware blobs in the kernel are done in a separate section (like initcalls, exceptions tables, pci fixups, and a bunch of other stuff). It'd just take some work in module.c to link them into a global list, and some locking shenanigans in the lookups (and lifetime issues to think about). But it just isn't worth the added complexity, given that userspace is known to be alive and working. It's pointless not to just use request_firmware() normally, from a module. -- dwmw2
From: Rafael J. Wysocki <hidden>
Date: 2008-07-03 21:42:22
On Thursday, 3 of July 2008, David Woodhouse wrote:
Jeff Garzik wrote:quoted
David Woodhouse wrote:quoted
Although it does make me wonder if it was better the way I had it originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL attached to each driver, rather than a single FIRMWARE_IN_KERNEL option which controls them all.IMO, individual options would be better.They had individual options for a long time, but the consensus was that I should remove them -- a consensus which was probably right. It was moderately inconvenient going back through it all and recommitting it without that, but it was worth it to get it right...quoted
Plus, unless I am misunderstanding, the firmware is getting built into the kernel image not the tg3 module?That's right, although it doesn't really matter when they're both in the vmlinux. When it's actually a module, there really is no good reason not to let request_firmware() get satisfied from userspace. If you can load modules, then you can load firmware too -- the required udev stuff has been there as standard for a _long_ time, as most modern drivers _require_ it without even giving you the built-in-firmware option at all. It makes no more sense to object to that than it does to object to the module depending on _other_ modules. Both those other modules, and the required firmware, are _installed_ by the kernel Makefiles, after all. It wouldn't be _impossible_ to put firmware blobs into the foo.ko files themselves and find them there. The firmware blobs in the kernel are done in a separate section (like initcalls, exceptions tables, pci fixups, and a bunch of other stuff). It'd just take some work in module.c to link them into a global list, and some locking shenanigans in the lookups (and lifetime issues to think about). But it just isn't worth the added complexity, given that userspace is known to be alive and working. It's pointless not to just use request_firmware() normally, from a module.
Still, maybe we can add some kbuild magic to build the blobs along with their modules and to install them under /lib/firmware (by default) when the modules are installed in /lib/modules/... ? Rafael
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 21:44:17
Rafael J. Wysocki wrote:
Still, maybe we can add some kbuild magic to build the blobs along with their modules and to install them under /lib/firmware (by default) when the modules are installed in /lib/modules/... ?
Something like appending this to Makefile? firmware_and_modules_install: firmware_install modules_install (I'm still wondering if we should make 'firmware_install' install to /lib/firmware by default, instead of into the build tree as 'headers_install' does. The Aunt Tillie answer would definitely be 'yes', although that means it requires root privs; like modules_install does.) -- dwmw2
From: Rafael J. Wysocki <hidden>
Date: 2008-07-03 21:51:04
On Thursday, 3 of July 2008, David Woodhouse wrote:
Rafael J. Wysocki wrote:quoted
Still, maybe we can add some kbuild magic to build the blobs along with their modules and to install them under /lib/firmware (by default) when the modules are installed in /lib/modules/... ?Something like appending this to Makefile? firmware_and_modules_install: firmware_install modules_install (I'm still wondering if we should make 'firmware_install' install to /lib/firmware by default, instead of into the build tree as 'headers_install' does. The Aunt Tillie answer would definitely be 'yes', although that means it requires root privs; like modules_install does.)
I would prefer 'make firmware_install' to just copy the blobs into specific location in analogy with 'make modules_install', so that you can build the blobs as a normal user (for example, on an NFS server) and then put them into the right place as root (for example, on an NFS client that has no write privilege on the server).
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 21:55:20
Rafael J. Wysocki wrote:
On Thursday, 3 of July 2008, David Woodhouse wrote:quoted
Rafael J. Wysocki wrote:quoted
Still, maybe we can add some kbuild magic to build the blobs along with their modules and to install them under /lib/firmware (by default) when the modules are installed in /lib/modules/... ?Something like appending this to Makefile? firmware_and_modules_install: firmware_install modules_install (I'm still wondering if we should make 'firmware_install' install to /lib/firmware by default, instead of into the build tree as 'headers_install' does. The Aunt Tillie answer would definitely be 'yes', although that means it requires root privs; like modules_install does.)I would prefer 'make firmware_install' to just copy the blobs into specific location in analogy with 'make modules_install', so that you can build the blobs as a normal user (for example, on an NFS server) and then put them into the right place as root (for example, on an NFS client that has no write privilege on the server).
Not entirely sure which you mean. You _can't_ run 'make modules_install' as a normal user, unless you override $(INSTALL_MOD_PATH) on the command line. Do you want 'make firmware_install' to be the same? It isn't at the moment -- it installs to a subdirectory of the kernel build tree, like 'make headers_install' does. But I'm not sure which is better. -- dwmw2
From: Jeff Garzik <hidden>
Date: 2008-07-03 22:22:51
David Woodhouse wrote:
Jeff Garzik wrote:quoted
David Woodhouse wrote:quoted
Although it does make me wonder if it was better the way I had it originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL attached to each driver, rather than a single FIRMWARE_IN_KERNEL option which controls them all.IMO, individual options would be better.They had individual options for a long time, but the consensus was that I should remove them -- a consensus which was probably right. It was moderately inconvenient going back through it all and recommitting it without that, but it was worth it to get it right...quoted
Plus, unless I am misunderstanding, the firmware is getting built into the kernel image not the tg3 module?That's right, although it doesn't really matter when they're both in the vmlinux. When it's actually a module, there really is no good reason not to let request_firmware() get satisfied from userspace. If you can load modules, then you can load firmware too -- the required udev stuff has been there as standard for a _long_ time, as most modern drivers _require_ it without even giving you the built-in-firmware option at all. It makes no more sense to object to that than it does to object to the module depending on _other_ modules. Both those other modules, and the required firmware, are _installed_ by the kernel Makefiles, after all. It wouldn't be _impossible_ to put firmware blobs into the foo.ko files themselves and find them there. The firmware blobs in the kernel are done in a separate section (like initcalls, exceptions tables, pci fixups, and a bunch of other stuff). It'd just take some work in module.c to link them into a global list, and some locking shenanigans in the lookups (and lifetime issues to think about). But it just isn't worth the added complexity, given that userspace is known to be alive and working. It's pointless not to just use request_firmware() normally, from a module.
It is pointless -- if you assume everybody wants to run their distro and universe your way. If a firmware is built-in, then 'make firmware_install' is clearly optional and may be omitted. That invalidates several of your assumptions above. Further, all current kernel build and test etc. scripts are unaware of 'make firmware_install', and it is unfair to everybody to force a flag-day build process change on people, just to keep their drivers in the same working state today as it was yesterday. Conclusion - kernel build process must produce a working driver after your changes are applied, even in absence of 'make firmware_install'. That does not, repeat /not/ exclude the desired end goal of course -- separating the firmware from the driver, installing in /lib/firmware via 'make firmware_install', etc. I support your end goal, I really do. But I continue to feel there is a lack of regard for breakage and regressions. You are either ignoring or apparently just not seeing * how many new ways this can produce a non-working driver * how important it is in this specific case to fail-safe, and avoid a broken driver at all costs. As a concrete example, in the above quoted text you assume that a user will never copy kernel modules around. I can tell you that, with tg3.ko being nice and self-contained, yes it does get copied around (to multiple machines, etc.). With the firmware newly separated from tg3.ko, you have introduced breakage for any user that is unaware of this new requirement (kernel module == requires additional file now). Scripts that build install disks must be updated, otherwise a script that builds a boot image will include the drivers it knows about, but /not/ include the crucial firmware. None of this stuff is "pointless", none of this stuff may be dismissed as "making no sense". All these are real world examples where users FOLLOWING THEIR NORMAL, PROSCRIBED KERNEL PROCESSES will produce non-working drivers. The only valid assumption here is to assume that the user is /unaware/ of these new steps they must take in order to continue to have a working system. Regards, Jeff
From: Rafael J. Wysocki <hidden>
Date: 2008-07-03 22:25:54
On Thursday, 3 of July 2008, David Woodhouse wrote:
Rafael J. Wysocki wrote:quoted
On Thursday, 3 of July 2008, David Woodhouse wrote:quoted
Rafael J. Wysocki wrote:quoted
Still, maybe we can add some kbuild magic to build the blobs along with their modules and to install them under /lib/firmware (by default) when the modules are installed in /lib/modules/... ?Something like appending this to Makefile? firmware_and_modules_install: firmware_install modules_install (I'm still wondering if we should make 'firmware_install' install to /lib/firmware by default, instead of into the build tree as 'headers_install' does. The Aunt Tillie answer would definitely be 'yes', although that means it requires root privs; like modules_install does.)I would prefer 'make firmware_install' to just copy the blobs into specific location in analogy with 'make modules_install', so that you can build the blobs as a normal user (for example, on an NFS server) and then put them into the right place as root (for example, on an NFS client that has no write privilege on the server).Not entirely sure which you mean. You _can't_ run 'make modules_install' as a normal user, unless you override $(INSTALL_MOD_PATH) on the command line.
Yes, I know that.
Do you want 'make firmware_install' to be the same?
Yes, I'd prefer it to behave in the same way as 'make modules_install'. I'd use a config option like BUILD_FIRMWARE_BLOBS that, if set, would make the build system create firmware bin files in the same directories where the driver's .o files are located. Then, 'make firmware_install' would only copy those bin files to wherever was appropriate (eg. /lib/firmware/). Of course, there still would be a problem if there already were such firmware files at the destination, but that would have to be resolved anyway by the user wanting to install the new kernel along with the new firmware blobs.
It isn't at the moment -- it installs to a subdirectory of the kernel build tree, like 'make headers_install' does. But I'm not sure which is better.
IMO 'make headers_install' is used for a different purpose. You don't have to run it to be able to use the kernel in a usual way. OTOH, everyone is familiar with the 'make modules_install' mechanics and it seems natural to use analogous mechanics for firmware blobs.
From: Alan Cox <hidden>
Date: 2008-07-03 22:44:35
O
Further, all current kernel build and test etc. scripts are unaware of 'make firmware_install', and it is unfair to everybody to force a flag-day build process change on people, just to keep their drivers in the same working state today as it was yesterday.
IMHO we want firmware built in as the default for the moment. If the firmware model makes sense (as I think it does) then the distributions will catch up, turn it on and sort out the default behaviour - exactly as they did all those years ago with modules, more recently with "use an initrd" and so on.
as "making no sense". All these are real world examples where users FOLLOWING THEIR NORMAL, PROSCRIBED KERNEL PROCESSES will produce
I hope you mean "prescribed" ;)
The only valid assumption here is to assume that the user is /unaware/ of these new steps they must take in order to continue to have a working system.
To a large extent not the user but their distro - consider "make install"
From: Jeff Garzik <hidden>
Date: 2008-07-03 23:14:57
Alan Cox wrote:
quoted
Further, all current kernel build and test etc. scripts are unaware of 'make firmware_install', and it is unfair to everybody to force a flag-day build process change on people, just to keep their drivers in the same working state today as it was yesterday.IMHO we want firmware built in as the default for the moment. If the firmware model makes sense (as I think it does) then the distributions will catch up, turn it on and sort out the default behaviour - exactly as they did all those years ago with modules, more recently with "use an initrd" and so on.
Agreed.
quoted
as "making no sense". All these are real world examples where users FOLLOWING THEIR NORMAL, PROSCRIBED KERNEL PROCESSES will produceI hope you mean "prescribed" ;)
heh, *cough* yes
quoted
The only valid assumption here is to assume that the user is /unaware/ of these new steps they must take in order to continue to have a working system.To a large extent not the user but their distro - consider "make install"
Actually, I was tossing that about in my head: Is it a better idea to eliminate 'make firmware_install' completely, and instead implement it silently via 'make install'? 'make install' is already a big fat distro hook... Jeff
From: Alan Cox <hidden>
Date: 2008-07-03 23:20:23
Actually, I was tossing that about in my head: Is it a better idea to eliminate 'make firmware_install' completely, and instead implement it silently via 'make install'? 'make install' is already a big fat distro hook...
make firmware_install can encapsulate a lot of kernel specific knowledge so I think it belongs in the kernel tree to avoid problems in future. The use of make firmware_install belongs in the distro make install hooks. Otherwise we will mess up the distro stuff if we have to change the innards of make firmware_install in future, as may well occur.
From: David Miller <davem@davemloft.net>
Date: 2008-07-03 23:21:34
From: David Woodhouse <dwmw2@infradead.org> Date: Thu, 03 Jul 2008 19:56:02 +0100
It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y', because the _normal_ setting for that option _really_ should be 'N'.
On what basis? From a "obviously works" basis, the default should be 'y'.
What we're doing now is just cleaning up the older drivers which don't use request_firmware(), to conform to what is now common practice.
You say "conform" I say "break".
In the meantime, it would be useful if Jeff would quit throwing his toys out of the pram on that issue and actually review the _code_ changes. In particular, are the reports correct that the device operates just fine without the TSO firmware loaded? Should we change the request_firmware() error path to just disable TSO and continue with the initialisation?
No! The 5701 A0 firmware is necessary to load in order to work around hardware and existing firmware bugs on those cards. It's an issue of basic functionality, not just optimizations. 5701 A0 tg3 chips cannot operate at all without the firmware being present in the driver. Therefore, if you can't load the firmware, the card is not going to work.
Less of the ad hominem, please. Especially when it's so misdirected.
No, it is properly directed, you are breaking the tree for users.
Updating these drivers to remove large blobs of static unswappable data from the kernel, and having it provided from userspace on demand as modern Linux drivers do, is a perfectly sensible technical goal all on its own.
I disagree.
And given the GPL's explicit provisions with regard to collective works there are also entirely reasonable, non-"fundamentalist" grounds for believing that it _may_ pose a licensing problem, and for wanting to err on the side of caution in that respect too.
So now the real truth is revealed. You have no technical basis for this stuff you are ramming down everyone's throats. You want to choose a default based upon your legal agenda. That explains all of the bullshit that is attached to your work, and all of the bullshit arguments you make wrt. choosing defaults that break things for users. It's all about agendas rather than any real technical objectives. If it was purely technical, you wouldn't be choosing defaults that break things for users by default. Jeff and I warned you about this from day one, you did not listen, and now we have at least 10 reports just today of people with broken networking.
From: Theodore Tso <tytso@mit.edu>
Date: 2008-07-04 00:20:32
On Thu, Jul 03, 2008 at 04:21:20PM -0700, David Miller wrote:
quoted
And given the GPL's explicit provisions with regard to collective works there are also entirely reasonable, non-"fundamentalist" grounds for believing that it _may_ pose a licensing problem, and for wanting to err on the side of caution in that respect too.So now the real truth is revealed. You have no technical basis for this stuff you are ramming down everyone's throats. You want to choose a default based upon your legal agenda.
Yep, legal agenda. As I suspected, licensing religious fundamentalism. :-) People who care can change the defaults. People who are real religious nuts won't even let the firmware live in the same source tarball. But I hope you agree we clearly don't have consensus to take *that* step (rip out all firmware and make a whole bunch of drivers non-functional and forcing users to go on a treasure-hunt to find some new tarball they have to install on their existing system). So given that we're not ready to take that step, why not just leave the default as "yes" for now? The staged approach means that if you really want to do this ASAP, then start assembling the firmware tarball *now*, and for a while (read: at least 9-18 months) we can distribute firmware both in the kernel source tarball as well as separately in the licensing-religion-firmware tarball. See if you can get distros willing to ship it by default in most user's systems, and give people plenty of time to understand that we are trying to decouple firmware from the kernel sources. If we need to institute better versioning regimes between the drivers and firmware release levels, that will also give people a chance to get that all right. Then 6-9 months later, we can turn the default to 'no', and then maybe another 6-9 months after that, we can talk about removing the firmware modules. But it seems to me that you are skipping a few steps by arguing that the default should be changed here-and-now. We've been shipping firmware in the kernel for over a ***decade***; in fact, probably over 15 years. For people who are legal freaks/geeks, look up the legal terms "Estoppel" and "Laches". That provides a fairly large amount of protection right there. For people who aren't legal geeks, we've been doing this for well over a decade; another year or two really isn't a big deal. It certainly doesn't justify breaking users by default just to try to hurry up this process.
If it was purely technical, you wouldn't be choosing defaults that break things for users by default. Jeff and I warned you about this from day one, you did not listen, and now we have at least 10 reports just today of people with broken networking.
Not 15 minutes after David posted his note, we're now up to 11 reports; and this is only from an -mm patch series. Can you imagine the number of bug reports if this were allowed to ship in a mainline kernel.org release? One good thing is that we can definitely show that there people that are downloading, compiling and trying to build the -mm kernel. :-) - Ted
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 00:25:49
David Miller wrote:
From: David Woodhouse <dwmw2@infradead.org> Date: Thu, 03 Jul 2008 19:56:02 +0100quoted
It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y', because the _normal_ setting for that option _really_ should be 'N'.On what basis? From a "obviously works" basis, the default should be 'y'.
I already changed it to 'y'.
quoted
What we're doing now is just cleaning up the older drivers which don't use request_firmware(), to conform to what is now common practice.You say "conform" I say "break".
You mean... "What we're doing now is just cleaning up the older drivers which don't use request_firmware(), to break to what is now common practice." ? Doesn't really scan, does it? Common practice in modern Linux drivers is to use request_firmware(). I'm just going through and fixing up the older ones to do that too. (After making it possible to build that firmware _into_ the kernel so that we aren't forcing people to use an initrd where they didn't before, of course.) The word for that is definitely 'conform'. I know you don't _like_ the modern accepted practice, but that's _your_ windmill to tilt at. I have my own :)
quoted
In the meantime, it would be useful if Jeff would quit throwing his toys out of the pram on that issue and actually review the _code_ changes. In particular, are the reports correct that the device operates just fine without the TSO firmware loaded? Should we change the request_firmware() error path to just disable TSO and continue with the initialisation?No! The 5701 A0 firmware is necessary to load in order to work around hardware and existing firmware bugs on those cards. It's an issue of basic functionality, not just optimizations. 5701 A0 tg3 chips cannot operate at all without the firmware being present in the driver. Therefore, if you can't load the firmware, the card is not going to work.
Neat avoidance of question there... it was fairly clear that the 5701_A0 firmware was going to be mandatory; I was asking about the TSO firmware. Does anyone _else_ actually want to give a straight answer to a simple question? Someone who wouldn't have to follow it with an apology because of all their shouting about 'breakage' when the firmware in question is actually optional anyway, perhaps?
If it was purely technical, you wouldn't be choosing defaults that break things for users by default.
Actually, the beauty of Linux is that we _can_ change things where a minor short-term inconvenience leads to a better situation in the long term.
Jeff and I warned you about this from day one, you did not listen, and now we have at least 10 reports just today of people with broken networking.
Out of interest... of those, what proportion would be 'fixed' if they'd just paid attention when running 'make oldconfig', which is now addressed because I've changed the FIRMWARE_IN_KERNEL default to 'y'? And how many would be 'fixed' if someone had given me a straight answer when I asked about the TSO firmware, and that failure path no longer aborted the driver initialisation but instead just fell back to non-TSO? I'll look at making the requirement for 'make firmware_install' more obvious, or even making it happen automatically as part of 'modules_install'. -- dwmw2
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 01:10:01
Theodore Tso wrote:
On Thu, Jul 03, 2008 at 04:21:20PM -0700, David Miller wrote:quoted
You want to choose a default based upon your legal agenda.Yep, legal agenda. As I suspected, licensing religious fundamentalism. :-)
You are mistaken, and you're responding some words that someone _else_ put in my mouth.
The staged approach means that if you really want to do this ASAP, then start assembling the firmware tarball *now*,
That's easy enough. We can automatically generate a tree _from_ Linus' tree, with a scripted transform so that it includes only the firmware files (much like the kernel-headers tree automatically follows each commit in Linus' tree, but includes only the exported headers). And there are some hardware manufacturers who are willing to have their firmware included in such a firmware tarball, but who will _not_ give their permission to have it included in the Linux kernel because of the legal concerns you're so dismissive of. But that's OK -- we can pull from the automatically generated tree into a 'real' linux-firmware.git tree, which includes those extra firmware files. But there's no need to do it _now_. It can wait until the basic stuff is in Linus' tree and it can automatically derive from that. There's no particular rush, is there?
and for a while (read: at least 9-18 months) we can distribute firmware both in the kernel source tarball as well as separately
That makes a certain amount of sense.
in the licensing-religion-firmware tarball.
Please don't be gratuitously offensive, Ted. It's not polite, and it's not a particularly good debating technique either. I expect better from you.
See if you can get distros willing to ship it by default in most user's systems, and give people plenty of time to understand that we are trying to decouple firmware from the kernel sources.
The distros are certainly willing (and keen) to do it. The Fedora Engineering Steering Committee has already stated that it wants to do so, and the specfile changes to spit out a 'kernel-firmware' sub-package with the kernel build are ready to go right now. Fedora already modifies tarballs, for example 'openssh-5.0p1-noacss'. I think it's quite likely they'd end up using a 'linux-2.6.27-nofirmware' tarball too, and build the firmware package from the separate tree. Some other distributions have been doing that kind of thing _already_, even when it meant just ripping out certain drivers completely. That seems excessive to me; I prefer not to actually _break_ anything.
If we need to institute better versioning regimes between the drivers and firmware release levels, that will also give people a chance to get that all right. Then 6-9 months later, we can turn the default to 'no', and then maybe another 6-9 months after that, we can talk about removing the firmware modules. But it seems to me that you are skipping a few steps by arguing that the default should be changed here-and-now.
I disagree that the _default_ is such an issue -- largely because the normal case for modern drivers is not to include the firmware _anyway_, and the tools like 'mkinitrd' already cope with it just fine. But I've changed the default to 'y' now, as I already said. -- dwmw2
From: Grant Coady <hidden>
Date: 2008-07-04 01:29:19
On Fri, 04 Jul 2008 01:24:59 +0100, David Woodhouse [off-list ref] wrote: ...
I'll look at making the requirement for 'make firmware_install' more obvious, or even making it happen automatically as part of 'modules_install'.
I like this one: Automagically part of modules_install. No break existing kernel build scripts for the expected-by-user build sequence? And please put 'make firmware_install' into 'make help' if that's the way you go. And another point, at the moment I seem to get all sorts of odd things built I didn't know were there? root@pooh:/home/grant/linux/linux-2.6.26-rc8-mm1a# make firmware_install HOSTCC firmware/ihex2fw IHEX2FW firmware/atmsar11.fw INSTALL usr/lib/firmware/atmsar11.fw IHEX2FW firmware/dabusb/firmware.fw INSTALL usr/lib/firmware/dabusb/firmware.fw IHEX2FW firmware/emi26/loader.fw INSTALL usr/lib/firmware/emi26/loader.fw IHEX2FW firmware/emi26/firmware.fw INSTALL usr/lib/firmware/emi26/firmware.fw IHEX2FW firmware/emi26/bitstream.fw INSTALL usr/lib/firmware/emi26/bitstream.fw IHEX2FW firmware/emi62/loader.fw INSTALL usr/lib/firmware/emi62/loader.fw IHEX2FW firmware/emi62/bitstream.fw INSTALL usr/lib/firmware/emi62/bitstream.fw IHEX2FW firmware/emi62/spdif.fw INSTALL usr/lib/firmware/emi62/spdif.fw IHEX2FW firmware/emi62/midi.fw INSTALL usr/lib/firmware/emi62/midi.fw IHEX2FW firmware/keyspan/mpr.fw INSTALL usr/lib/firmware/keyspan/mpr.fw IHEX2FW firmware/keyspan/usa18x.fw INSTALL usr/lib/firmware/keyspan/usa18x.fw IHEX2FW firmware/keyspan/usa19.fw INSTALL usr/lib/firmware/keyspan/usa19.fw IHEX2FW firmware/keyspan/usa19qi.fw INSTALL usr/lib/firmware/keyspan/usa19qi.fw IHEX2FW firmware/keyspan/usa19qw.fw INSTALL usr/lib/firmware/keyspan/usa19qw.fw
From: Theodore Tso <tytso@mit.edu>
Date: 2008-07-04 01:48:41
On Fri, Jul 04, 2008 at 02:09:15AM +0100, David Woodhouse wrote:
But there's no need to do it _now_. It can wait until the basic stuff is in Linus' tree and it can automatically derive from that. There's no particular rush, is there?
The only rush if the main agenda is to extirpate all firmware from the mainline kernel sources. I don't think we can start the timer for doing that until the firmware tarball is created and it starts being used as the default way of delivering firmware for what you call "legacy" drivers. If there's no particular rush to finally rm the firmware for trivers such as tg3 from the source tree (and I personally don't think there should be any rush), or any rush to change the default for CONFIG_FIRMWARE_IN_KERNEL to "no", then I don't see any rush in creating the firmware tarball. If *you* think there is a rush in making CONFIG_FIRMWARE_IN_KERNEL default to "no", then you might want to decide to create the firmware tarball sooner, and get distro's everywhere to start using it, and to get everyone to understand that they should start including it in their systems. (Remember, not everyone uses the popular distributions like Fedora, Debian, Ubuntu, Open SuSE, etc.) But heck, that's up to you. :-)
quoted
and for a while (read: at least 9-18 months) we can distribute firmware both in the kernel source tarball as well as separatelyThat makes a certain amount of sense.
Glad we agree.
quoted
in the licensing-religion-firmware tarball.Please don't be gratuitously offensive, Ted. It's not polite, and it's not a particularly good debating technique either. I expect better from you.
Well, I think it's offensive to break users who have happily been using drivers that have been including firmware for a long, long, LONG time, and I expected better of you. So there. :-) - Ted
From: Mikael Pettersson <hidden>
Date: 2008-07-04 02:32:53
Alan Cox writes: > > The only valid assumption here is to assume that the user is /unaware/ > > of these new steps they must take in order to continue to have a working > > system. > > To a large extent not the user but their distro - consider "make install" > -- Last time I checked only x86 had 'make install'. I regularly build natively on ppc(32|64) and sparc64, and none of them implement 'make install' AFAIK. And on ARM I move the kernels over to a tftp server for network boots, again w/o 'make install'. Not that 'make install' is difficult. All it does it hand over to /sbin/installkernel or something like that. In the context of .config changes, 'make oldconfig' with 'select the default' must IMO result in a working kernel similar to the previous one. Anything else is madness or arrogance.
From: <hidden>
Date: 2008-07-04 02:42:21
On Fri, 4 Jul 2008, David Woodhouse wrote:
David Miller wrote:quoted
From: David Woodhouse <dwmw2@infradead.org> Date: Thu, 03 Jul 2008 19:56:02 +0100quoted
It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y', because the _normal_ setting for that option _really_ should be 'N'.On what basis? From a "obviously works" basis, the default should be 'y'.I already changed it to 'y'.quoted
quoted
What we're doing now is just cleaning up the older drivers which don't use request_firmware(), to conform to what is now common practice.You say "conform" I say "break".You mean... "What we're doing now is just cleaning up the older drivers which don't use request_firmware(), to break to what is now common practice." ? Doesn't really scan, does it? Common practice in modern Linux drivers is to use request_firmware(). I'm just going through and fixing up the older ones to do that too. (After making it possible to build that firmware _into_ the kernel so that we aren't forcing people to use an initrd where they didn't before, of course.)
has this taken place yet? (and if so, what kernel version first included this fix)
quoted
If it was purely technical, you wouldn't be choosing defaults that break things for users by default.Actually, the beauty of Linux is that we _can_ change things where a minor short-term inconvenience leads to a better situation in the long term.
but doing so should not be a easy and quick decision, and it needs to be made very clear exactly what breakage is going to take place and why (along with the explination of why the breakage couldn't be avoided)
quoted
Jeff and I warned you about this from day one, you did not listen, and now we have at least 10 reports just today of people with broken networking.Out of interest... of those, what proportion would be 'fixed' if they'd just paid attention when running 'make oldconfig', which is now addressed because I've changed the FIRMWARE_IN_KERNEL default to 'y'? And how many would be 'fixed' if someone had given me a straight answer when I asked about the TSO firmware, and that failure path no longer aborted the driver initialisation but instead just fell back to non-TSO? I'll look at making the requirement for 'make firmware_install' more obvious, or even making it happen automatically as part of 'modules_install'.
I won't mind this as long as I can get a working kernel without doing make firmware_install or make modules_install (I almost never use modules, my laptop is one of the few exceptions, and even there it's mostly becouse of the intel wireless driver needing userspace for firmware) David Lang
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 10:08:12
On Thu, 2008-07-03 at 19:42 -0700, david@lang.hm wrote:
quoted
(After making it possible to build that firmware _into_ the kernel so that we aren't forcing people to use an initrd where they didn't before, of course.)has this taken place yet? (and if so, what kernel version first included this fix)
It's in linux-next now. See the CONFIG_EXTRA_FIRMWARE option. That's one of the reasons Ted's ranting about 'religious fundamentalism' is so funny -- I've actually made it _easier_ for you to combine arbitrary firmware files into your kernel.
quoted
quoted
If it was purely technical, you wouldn't be choosing defaults that break things for users by default.Actually, the beauty of Linux is that we _can_ change things where a minor short-term inconvenience leads to a better situation in the long term.but doing so should not be a easy and quick decision, and it needs to be made very clear exactly what breakage is going to take place and why (along with the explination of why the breakage couldn't be avoided)
All forms of change introduce _some_ risk of breakage, of course. In this case, as usual, I've tried to be careful to avoid regressions. The most important part, obviously, was having a way to build firmware into the static kernel. When it comes to _modules_, doing that would introduce a certain amount of complexity which just doesn't seem necessary -- if you can load modules, then you have userspace, and you can use hotplug for firmware too. Especially given that so many modern drivers already _require_ you to do that, so the users understand it, and the tools like mkinitrd already cope with it -- checking MODULE_FIRMWARE() for the modules they include and including the appropriate files automatically. The alleged problem with modules seems to be _just_ about the fact that people need to run 'make firmware_install', and need to have their firmware installed. Something which all modern drivers require _anyway_, and people are used to in the general case already. It's not exactly hard; there's just the initial step of realising that the driver _you_ are using has now been updated to behave like all the others. That step is _minor_, and it doesn't actually get any easier _however_ long you postpone it. Yes, you might get 10 people in the first day tripping over it, and I'll look to see if I can make it clearer. But it's still a very minor, short-term thing. I suspect that the best option is just to hold off on updating the net drivers until later, when people already _know_ that they need to run 'make firmware_install', (or it happens automatically or something, if we go down that route). That way, Dave and Jeff shouldn't be affected by the initial transition period. There's plenty of other drivers which need updating, after all. And most maintainers are _happy_ to see the patches to bring their drivers in line with best current practice.
quoted
I'll look at making the requirement for 'make firmware_install' more obvious, or even making it happen automatically as part of 'modules_install'.I won't mind this as long as I can get a working kernel without doing make firmware_install or make modules_install
You can. You need to stay sober for long enough to say 'Y' when it asks you if you want to build the required firmware into the kernel. And we even made that the _default_ now, for the benefit of those who can't stay sober that long. (Perhaps we'll make 'allyesconfig' the default next?)
(I almost never use modules, my laptop is one of the few exceptions, and even there it's mostly becouse of the intel wireless driver needing userspace for firmware)
You don't need to do that any more :) -- dwmw2
From: Takashi Iwai <hidden>
Date: 2008-07-04 11:06:21
At Thu, 3 Jul 2008 14:04:36 +0100 (BST), Hugh Dickins wrote:
On Thu, 3 Jul 2008, Jeff Garzik wrote:quoted
KOSAKI Motohiro wrote:quoted
Hi Michael, my server output following error message on 2.6.26-rc8-mm1. Is this a bug? ------------------------------------------------------------------ tg3.c:v3.93 (May 22, 2008) GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51 firmware: requesting tigon/tg3_tso.bin tg3: Failed to load firmware "tigon/tg3_tso.bin" tg3 0000:06:01.0: PCI INT A disabled GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered tg3: probe of 0000:06:01.0 failed with error -2 GSI 73 (level, low) -> CPU 0 (0x0001) vector 51 tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52 firmware: requesting tigon/tg3_tso.binThis change did not come from the network developers or Broadcom, so someone else broke tg3 in -mm...I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y. That caught me out on PowerMac G5 trying mmotm yesterday, it just hung for a few minutes in earlyish boot with a message about tg3_tso.bin, and then proceeded to boot up but without the network. I was unclear whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor. I avoid initrd, and have tigon3 built in, if that's of any relevance. I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5: mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y).
Hmm, I got this error even with CONFIG_FIRMWARE_IN_KERNEL=y. Through a quick look at the code, the firmwares are not built indeed. I guess the fix like the following needed for building firmwares for modules. Now trying to build the kernel again to check this... Takashi
diff --git a/firmware/Makefile b/firmware/Makefile
index f88d746..e11fc2a 100644
--- a/firmware/Makefile
+++ b/firmware/Makefile@@ -55,6 +55,8 @@ fw-shipped-$(CONFIG_USB_SERIAL_XIRCOM) += keyspan_pda/xircom_pgs.fw fw-shipped-$(CONFIG_USB_VICAM) += vicam/firmware.fw fw-shipped-$(CONFIG_VIDEO_CPIA2) += cpia2/stv0672_vp4.bin +fw-shipped-y := $(fw-shipped-y) $(fw-shipped-m) + # If CONFIG_FIRMWARE_IN_KERNEL is not set, then don't include any firmware ifneq ($(CONFIG_FIRMWARE_IN_KERNEL),y) fw-shipped-y :=
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 13:18:20
On Fri, 2008-07-04 at 13:06 +0200, Takashi Iwai wrote:
Hmm, I got this error even with CONFIG_FIRMWARE_IN_KERNEL=y. Through a quick look at the code, the firmwares are not built indeed. I guess the fix like the following needed for building firmwares for modules. Now trying to build the kernel again to check this...
For modules, you just need run 'make INSTALL_FW_PATH=/lib/firmare firmware_install'. I should... 1. Change the default to /lib/firmware so that you don't have to set INSTALL_FW_PATH. 2. Add that to the 'make help' text. 3. Look at making 'make modules_install' installl the firmware required by the modules it's installing, so you don't even need to do _anything_ new. -- dwmw2
From: Takashi Iwai <hidden>
Date: 2008-07-04 13:27:16
At Fri, 04 Jul 2008 14:17:51 +0100, David Woodhouse wrote:
On Fri, 2008-07-04 at 13:06 +0200, Takashi Iwai wrote:quoted
Hmm, I got this error even with CONFIG_FIRMWARE_IN_KERNEL=y. Through a quick look at the code, the firmwares are not built indeed. I guess the fix like the following needed for building firmwares for modules. Now trying to build the kernel again to check this...For modules, you just need run 'make INSTALL_FW_PATH=/lib/firmare firmware_install'. I should... 1. Change the default to /lib/firmware so that you don't have to set INSTALL_FW_PATH. 2. Add that to the 'make help' text. 3. Look at making 'make modules_install' installl the firmware required by the modules it's installing, so you don't even need to do _anything_ new.
Ah I see. I thought you implemented the built-in firmware even for modules, but apparently it's not. Is mkinitrd clever enough to put all needed firmware files to initrd automatically? Otherwise this can still break the existing setup... thanks, Takashi
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 13:28:54
On Fri, 2008-07-04 at 15:26 +0200, Takashi Iwai wrote:
Ah I see. I thought you implemented the built-in firmware even for modules, but apparently it's not.
It isn't really necessary. If you can load modules, then you have userspace. And if you have userspace, you can load firmware too.
Is mkinitrd clever enough to put all needed firmware files to initrd automatically? Otherwise this can still break the existing setup...
Yes, it's had to be clever enough for that for a long time anyway -- most modern drivers have _only_ the 'request_firmware()' option and never gave you the choice of building it in. I'm just updating some of the older drivers to catch up. -- dwmw2
From: Jeff Garzik <hidden>
Date: 2008-07-04 13:29:17
David Woodhouse wrote:
3. Look at making 'make modules_install' installl the firmware required by the modules it's installing, so you don't even need to do _anything_ new.
Anything that works with existing build processes would be a positive step, preventing loads of build&test regressions. This does not change the need, of course, to default this firmware install stuff to 'off', and allow distros to turn it on when they're ready (as Alan, Ted, and others have suggested in this thread). Jeff
From: Jeff Garzik <hidden>
Date: 2008-07-04 13:42:51
Takashi Iwai wrote:
Ah I see. I thought you implemented the built-in firmware even for modules, but apparently it's not.
Correct.
Is mkinitrd clever enough to put all needed firmware files to initrd automatically? Otherwise this can still break the existing setup...
mkinitrd and similar scripts must be updated, so that drivers that worked prior to dwmw2's changes will continue to work after dwmw2's changes. If you fail to update some script somewhere, then the driver will be copied into the initramfs, but not the firmware, with obvious results. This is not a fail-safe system. This is an enforced-breakage, flag-day change that will cause a lot of additional work for a lot of people. Jeff
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 13:45:51
On Fri, 2008-07-04 at 09:42 -0400, Jeff Garzik wrote:
mkinitrd and similar scripts must be updated, so that drivers that worked prior to dwmw2's changes will continue to work after dwmw2's changes.
If you fail to update some script somewhere, then the driver will be copied into the initramfs, but not the firmware, with obvious results.
No, mkinitrd works fine, because a whole boatload of drivers _already_ require it to work that way and have done for a long time. Either you are severely mistaken, or you are being deliberately misleading. -- dwmw2
From: Jeff Garzik <hidden>
Date: 2008-07-04 14:11:13
David Woodhouse wrote:
On Fri, 2008-07-04 at 09:42 -0400, Jeff Garzik wrote:quoted
mkinitrd and similar scripts must be updated, so that drivers that worked prior to dwmw2's changes will continue to work after dwmw2's changes.quoted
If you fail to update some script somewhere, then the driver will be copied into the initramfs, but not the firmware, with obvious results.No, mkinitrd works fine, because a whole boatload of drivers _already_ require it to work that way and have done for a long time. Either you are severely mistaken, or you are being deliberately misleading.
It is a fact that mkinitrd, today, is unaware of your new system of obtaining firmware from the kernel source[or build] tree. Certainly it is aware of the need to copy firmware in general, but that doesn't change the fact that the tg3 firmware will not make it into initramfs, without additional steps taken. So, no, it doesn't "work fine" -- the firmware doesn't make it into the initramfs. Jeff
From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 14:13:47
On Fri, 2008-07-04 at 10:10 -0400, Jeff Garzik wrote:
David Woodhouse wrote:quoted
On Fri, 2008-07-04 at 09:42 -0400, Jeff Garzik wrote:quoted
mkinitrd and similar scripts must be updated, so that drivers that worked prior to dwmw2's changes will continue to work after dwmw2's changes.quoted
If you fail to update some script somewhere, then the driver will be copied into the initramfs, but not the firmware, with obvious results.No, mkinitrd works fine, because a whole boatload of drivers _already_ require it to work that way and have done for a long time. Either you are severely mistaken, or you are being deliberately misleading.It is a fact that mkinitrd, today, is unaware of your new system of obtaining firmware from the kernel source[or build] tree.
Of _course_ it is. Just as it's unaware of the need to download the b43 driver and b43-fwcutter and extract it.... what was your point, again? I'm working on making the required firmware get installed as part of 'make modules_install'. We already discussed that. There's no need for you to continue crying wolf with nonsense like 'mkinitrd won't work'. -- dwmw2
From: Jeff Garzik <hidden>
Date: 2008-07-05 06:14:25
David Woodhouse wrote:
I'm working on making the required firmware get installed as part of 'make modules_install'.
Great! That will definitely reduce silent regressions due to build process changes. Jeff