[bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

42 messages, 13 authors, 2008-07-05 · open the first message on its own page

[bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: KOSAKI Motohiro <hidden>
Date: 2008-07-03 12:01:09

Hi Michael,

my server output following error message on 2.6.26-rc8-mm1.
Is this a bug?

------------------------------------------------------------------
tg3.c:v3.93 (May 22, 2008)
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51
firmware: requesting tigon/tg3_tso.bin
tg3: Failed to load firmware "tigon/tg3_tso.bin"
tg3 0000:06:01.0: PCI INT A disabled
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered
tg3: probe of 0000:06:01.0 failed with error -2
GSI 73 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52
firmware: requesting tigon/tg3_tso.bin


Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-03 12:21:40

KOSAKI Motohiro wrote:
Hi Michael,

my server output following error message on 2.6.26-rc8-mm1.
Is this a bug?

------------------------------------------------------------------
tg3.c:v3.93 (May 22, 2008)
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51
firmware: requesting tigon/tg3_tso.bin
tg3: Failed to load firmware "tigon/tg3_tso.bin"
tg3 0000:06:01.0: PCI INT A disabled
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered
tg3: probe of 0000:06:01.0 failed with error -2
GSI 73 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52
firmware: requesting tigon/tg3_tso.bin
This change did not come from the network developers or Broadcom, so someone else broke tg3 in -mm...

	Jeff


Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Hugh Dickins <hidden>
Date: 2008-07-03 13:04:50

On Thu, 3 Jul 2008, Jeff Garzik wrote:
KOSAKI Motohiro wrote:
quoted
Hi Michael,

my server output following error message on 2.6.26-rc8-mm1.
Is this a bug?

------------------------------------------------------------------
tg3.c:v3.93 (May 22, 2008)
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51
firmware: requesting tigon/tg3_tso.bin
tg3: Failed to load firmware "tigon/tg3_tso.bin"
tg3 0000:06:01.0: PCI INT A disabled
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered
tg3: probe of 0000:06:01.0 failed with error -2
GSI 73 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52
firmware: requesting tigon/tg3_tso.bin
This change did not come from the network developers or Broadcom, so someone
else broke tg3 in -mm...
I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y.

That caught me out on PowerMac G5 trying mmotm yesterday, it just hung
for a few minutes in earlyish boot with a message about tg3_tso.bin,
and then proceeded to boot up but without the network.  I was unclear
whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor.

I avoid initrd, and have tigon3 built in, if that's of any relevance.

I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5:
mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y).

Hugh

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-03 13:11:32

Hugh Dickins wrote:
On Thu, 3 Jul 2008, Jeff Garzik wrote:
quoted
KOSAKI Motohiro wrote:
quoted
Hi Michael,

my server output following error message on 2.6.26-rc8-mm1.
Is this a bug?

------------------------------------------------------------------
tg3.c:v3.93 (May 22, 2008)
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51
firmware: requesting tigon/tg3_tso.bin
tg3: Failed to load firmware "tigon/tg3_tso.bin"
tg3 0000:06:01.0: PCI INT A disabled
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered
tg3: probe of 0000:06:01.0 failed with error -2
GSI 73 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52
firmware: requesting tigon/tg3_tso.bin
This change did not come from the network developers or Broadcom, so someone
else broke tg3 in -mm...
I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y.

That caught me out on PowerMac G5 trying mmotm yesterday, it just hung
for a few minutes in earlyish boot with a message about tg3_tso.bin,
and then proceeded to boot up but without the network.  I was unclear
whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor.

I avoid initrd, and have tigon3 built in, if that's of any relevance.

I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5:
mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y).

dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in.

Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage.

	Jeff

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 13:33:46

On Thu, 2008-07-03 at 09:11 -0400, Jeff Garzik wrote:
Hugh Dickins wrote:
quoted
On Thu, 3 Jul 2008, Jeff Garzik wrote:
quoted
KOSAKI Motohiro wrote:
quoted
Hi Michael,

my server output following error message on 2.6.26-rc8-mm1.
Is this a bug?

------------------------------------------------------------------
tg3.c:v3.93 (May 22, 2008)
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51
firmware: requesting tigon/tg3_tso.bin
tg3: Failed to load firmware "tigon/tg3_tso.bin"
tg3 0000:06:01.0: PCI INT A disabled
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered
tg3: probe of 0000:06:01.0 failed with error -2
GSI 73 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52
firmware: requesting tigon/tg3_tso.bin
This change did not come from the network developers or Broadcom, so someone
else broke tg3 in -mm...
I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y.

That caught me out on PowerMac G5 trying mmotm yesterday, it just hung
for a few minutes in earlyish boot with a message about tg3_tso.bin,
and then proceeded to boot up but without the network.  I was unclear
whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor.
I shall respectfully refrain from commenting on the likelihood of the
former. With regard to the latter, here is the help text for the
FIRMWARE_IN_KERNEL option:

        help
          The kernel source tree includes a number of firmware 'blobs'
          which are used by various drivers. The recommended way to
          use these is to run "make firmware_install" and to copy the
          resulting binary files created in usr/lib/firmware directory
          of the kernel tree to the /lib/firmware on your system so
          that they can be loaded by userspace helpers on request.

          Enabling this option will build each required firmware blob
          into the kernel directly, where request_firmware() will find
          them without having to call out to userspace. This may be
          useful if your root file system requires a device which uses
          such firmware, and do not wish to use an initrd.

          This single option controls the inclusion of firmware for
          every driver which usees request_firmare() and ships its
          firmware in the kernel source tree, to avoid a proliferation
          of 'Include firmware for xxx device' options.

          Say 'N' and let firmware be loaded from userspace.

If you think you can improve it, please let me have a revised attempt.
quoted
I avoid initrd, and have tigon3 built in, if that's of any relevance.

I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5:
mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y).

dwmw2 has been told repeatedly that his changes will cause PRECISELY 
these problems, but he refuses to take the simple steps necessary to 
ensure people can continue to boot their kernels after his changes go in.
Complete nonsense. Setting CONFIG_FIRMWARE_IN_KERNEL isn't hard. But
shouldn't be the _default_, either.
Presently his tg3 changes have been nak'd, in part, because of this 
obviously, forseeable, work-around-able breakage.
They haven't even been reviewed. Nobody seems to have actually looked at
the real changes (in particular, and commented on whether the device can
run anyway without the TSO firmware being loaded, as some people seem to
report). You're just throwing your toys out of the pram because of the
'default n' on a patch about 30 commits earlier in my tree.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-03 13:39:00

David Woodhouse wrote:
quoted
dwmw2 has been told repeatedly that his changes will cause PRECISELY these problems, but he refuses to take the simple steps necessary to ensure people can continue to boot their kernels after his changes go in.
Complete nonsense. Setting CONFIG_FIRMWARE_IN_KERNEL isn't hard. But
shouldn't be the _default_, either.
quoted
Presently his tg3 changes have been nak'd, in part, because of this obviously, forseeable, work-around-able breakage.
They haven't even been reviewed. Nobody seems to have actually looked at

Yes, they have.  You just didn't like the answers you received.

In particular, the Kconfig default for built-in tg3 firmware should result in the current behavior, without the user having to take extra steps.

Because of your stubborn refusal on this Kconfig defaults issue, WE ALREADY HAVE DRIVER-DOES-NOT-WORK BREAKAGE, JUST AS PREDICTED.

Wake up and smell reality.  Please.

	Jeff

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 13:53:22

On Thu, 2008-07-03 at 09:38 -0400, Jeff Garzik wrote:
David Woodhouse wrote:
quoted
quoted
dwmw2 has been told repeatedly that his changes will cause PRECISELY 
these problems, but he refuses to take the simple steps necessary to 
ensure people can continue to boot their kernels after his changes go in.
Complete nonsense. Setting CONFIG_FIRMWARE_IN_KERNEL isn't hard. But
shouldn't be the _default_, either.
quoted
Presently his tg3 changes have been nak'd, in part, because of this 
obviously, forseeable, work-around-able breakage.
They haven't even been reviewed. Nobody seems to have actually looked at

Yes, they have.  You just didn't like the answers you received.
I received no comment on any part of the changes within tg3.c; only
whining about the default behaviour -- which isn't even _set_ as part of
the patch in question, any more.
In particular, the Kconfig default for built-in tg3 firmware should 
result in the current behavior, without the user having to take extra steps.
After feedback from a number of people, there is no individual Kconfig
option for the various firmwares; there is only one which controls them
all -- CONFIG_FIRMWARE_IN_KERNEL. The thing you're whining about isn't
even part of the patch which needs review.
Because of your stubborn refusal on this Kconfig defaults issue, WE 
ALREADY HAVE DRIVER-DOES-NOT-WORK BREAKAGE, JUST AS PREDICTED.
I strongly disagree that CONFIG_FIRMWARE_IN_KERNEL=y should be the
default. But if I add this patch elsewhere in the kernel, will you quit
your whining and actually review the patch you were asked to review? ...
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index 339c148..d47482f 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig
@@ -37,6 +37,7 @@ config FW_LOADER
 config FIRMWARE_IN_KERNEL
 	bool "Include in-kernel firmware blobs in kernel binary"
 	depends on FW_LOADER
+	default y
 	help
 	  The kernel source tree includes a number of firmware 'blobs'
 	  which are used by various drivers. The recommended way to
-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Chuck Lever <hidden>
Date: 2008-07-03 16:11:07

On Jul 3, 2008, at 7:59 AM, KOSAKI Motohiro wrote:
Hi Michael,

my server output following error message on 2.6.26-rc8-mm1.
Is this a bug?

------------------------------------------------------------------
tg3.c:v3.93 (May 22, 2008)
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51
firmware: requesting tigon/tg3_tso.bin
tg3: Failed to load firmware "tigon/tg3_tso.bin"
tg3 0000:06:01.0: PCI INT A disabled
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered
tg3: probe of 0000:06:01.0 failed with error -2
GSI 73 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52
firmware: requesting tigon/tg3_tso.bin
Same problem here with linux-next on a Dell Latitude D620.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Theodore Tso <tytso@mit.edu>
Date: 2008-07-03 17:33:20

On Thu, Jul 03, 2008 at 02:52:55PM +0100, David Woodhouse wrote:
After feedback from a number of people, there is no individual Kconfig
option for the various firmwares; there is only one which controls them
all -- CONFIG_FIRMWARE_IN_KERNEL. The thing you're whining about isn't
even part of the patch which needs review.
quoted
Because of your stubborn refusal on this Kconfig defaults issue, WE 
ALREADY HAVE DRIVER-DOES-NOT-WORK BREAKAGE, JUST AS PREDICTED.
I strongly disagree that CONFIG_FIRMWARE_IN_KERNEL=y should be the
default. But if I add this patch elsewhere in the kernel, will you quit
your whining and actually review the patch you were asked to review? ...
I don't think it's whining.  If your patch introduces changes which
cause people .config to break by default after upgrading to a newer
kernel and doing "make oldconfig" --- then that's a problem with your
patch, and the missing hunk to enable CONFIG_FIRMWARE_IN_KERNEL=y is
critically important.

Linus has ruled this way in the past, when he's gotten screwed by this
sort of issue in the past, and he was justifiably annoyed.  We should
treat the users who are willing to test and provide feedback on the
latest kernel.org kernels with the same amount of regard.  And if
there are licensing religious fundamentalists who feel strongly about
the firmware issue, then fine, they can change the .config.  But the
default should be to avoid users from having broken kernels, and a
number of (quite clueful) users have already demonstrated that without
setting CONFIG_FIRMWARE_IN_KERNEL=y as the default, your patches cause
breakage.

Regards,

						- Ted

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 18:56:33

On Thu, 2008-07-03 at 13:30 -0400, Theodore Tso wrote:
I don't think it's whining.
Neither is it an adequate review of the actual patch which was
submitted.
  If your patch introduces changes which
cause people .config to break by default after upgrading to a newer
kernel and doing "make oldconfig" 
They had to 'make oldconfig' and then actually _choose_ to say 'no' to
an option which is fairly clearly documented, that they are the
relatively unusual position of wanting to have said 'yes' to. You're
getting into Aunt Tillie territory, when you complain about that.

Although it does make me wonder if it was better the way I had it
originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL
attached to each driver, rather than a single FIRMWARE_IN_KERNEL option
which controls them all.

Perhaps one way to help Aunt Tillie would be to tweak Kbuild to look at
the MODULE_FIRMWARE() statements for in-kernel drivers, and to print a
warning when the build finishes: "Your static kernel image may require
the following firmware files, which are not included: ..."

It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y',
because the _normal_ setting for that option _really_ should be 'N'.
Using request_firmware() satisfied from userspace is best practice these
days, and almost all recent drivers do it that way _unconditionally_
anyway.

What we're doing now is just cleaning up the older drivers which don't
use request_firmware(), to conform to what is now common practice. And
while we're retaining the _option_ to continue to build their firmware
into the static kernel image, it isn't recommended and really shouldn't
be the default configuration.
Linus has ruled this way in the past, when he's gotten screwed by this
sort of issue in the past, and he was justifiably annoyed. 
I am content to let Linus decide on what the default for the
FIRMWARE_IN_KERNEL option will be. I am adamant that it _should_ be 'N',
but it's easy enough for Linus to overrule me with a one-line change. 

In the meantime, it would be useful if Jeff would quit throwing his toys
out of the pram on that issue and actually review the _code_ changes. In
particular, are the reports correct that the device operates just fine
without the TSO firmware loaded? Should we change the request_firmware()
error path to just disable TSO and continue with the initialisation?

I can understand why he might not want to answer that if the answer is
affirmative, I suppose -- it detracts even _further_ from his already
rather dubious argument about 'breaking' the driver, if it'll actually
continue to work even when the firmware is completely absent. But it
would be nice to get an honest and straightforward review of the code
from _someone_ who actually knows the hardware.
 And if there are licensing religious fundamentalists who feel
strongly about the firmware issue, then fine, they can change
the .config. 
Less of the ad hominem, please. Especially when it's so misdirected.

Updating these drivers to remove large blobs of static unswappable data
from the kernel, and having it provided from userspace on demand as
modern Linux drivers do, is a perfectly sensible technical goal all on
its own.

And given the GPL's explicit provisions with regard to collective works
there are also entirely reasonable, non-"fundamentalist" grounds for
believing that it _may_ pose a licensing problem, and for wanting to err
on the side of caution in that respect too.

Fedora is almost certain to ship with CONFIG_FIRMWARE_IN_KERNEL=n, and
I'd be very surprised if Debian and other major distributions don't
follow suit. It is the sensible, pragmatic, technically sound choice.
 But the default should be to avoid users from having broken kernels,
and a number of (quite clueful) users have already demonstrated that
without setting CONFIG_FIRMWARE_IN_KERNEL=y as the default, your
patches cause breakage.
By this argument, shouldn't we include images in the static kernel for
_all_ drivers which currently use request_firmware()? Otherwise, it's
possible for the user to 'break' them, right?

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Miller <davem@davemloft.net>
Date: 2008-07-03 20:34:41

From: Jeff Garzik <redacted>
Date: Thu, 03 Jul 2008 09:11:09 -0400
dwmw2 has been told repeatedly that his changes will cause PRECISELY 
these problems, but he refuses to take the simple steps necessary to 
ensure people can continue to boot their kernels after his changes go in.

Presently his tg3 changes have been nak'd, in part, because of this 
obviously, forseeable, work-around-able breakage.
I agree with Jeff, obviously.

We both saw this song and dance coming.  Now the reports are coming in
from confused people who are losing their network.  It is no surprise.

And the person who introduced this swath of regressions acts like it's
some kind of chore to enforce the obviously correct default behavior.

Why is it such a big deal to make "obviously working" the default?

In effect, you lied to us, in that you said that by default users
wouldn't have to do anything to keep getting a working setup.  But
that is provably not true, look at all of these reports.  Are you
saying these people are idiots and don't know how to configure their
kernel?  Every single one of them?

So don't be surprised how pissed off some of us are about these
changes.  You are inflicting pain on driver maintainers because now
they have to sift through these "firmware not found" reports in
addition to their normal workload.

And David make it seem like it's inconvenient for him to implement the
correct default, which in particular pisses me personally off the
most.  It's totally irresponsible, and I don't care what the legal or
ideological motivation is.

Given that, how in the world can you be surprised that the effected
driver maintainers have no interest in reviewing the substance of
these patches?  You don't piss people off, then say "help me review
this stuff."  It doesn't work like that.

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 20:55:19

On Thu, 2008-07-03 at 13:34 -0700, David Miller wrote:
From: Jeff Garzik <redacted>
Date: Thu, 03 Jul 2008 09:11:09 -0400
quoted
dwmw2 has been told repeatedly that his changes will cause PRECISELY 
these problems, but he refuses to take the simple steps necessary to 
ensure people can continue to boot their kernels after his changes go in.

Presently his tg3 changes have been nak'd, in part, because of this 
obviously, forseeable, work-around-able breakage.
I agree with Jeff, obviously.

We both saw this song and dance coming.  Now the reports are coming in
from confused people who are losing their network.  It is no surprise.

And the person who introduced this swath of regressions acts like it's
some kind of chore to enforce the obviously correct default behavior.

Why is it such a big deal to make "obviously working" the default?
In effect, you lied to us, in that you said that by default users
wouldn't have to do anything to keep getting a working setup.
No, I didn't lie to you. The conversation, in case you forgot, went like
this...

On Wed, 2008-06-18 at 16:23 -0700, David Miller wrote:
On Thu, 2008-06-19 at 00:16 +0100, David Woodhouse wrote:
quoted
On Wed, 2008-06-18 at 16:05 -0700, David Miller wrote:
quoted
Tell me this, how can I (with the default config option settings)
netboot properly without an initial ramdisk after these tg3 patches
and still get the proper firmware?
I suppose the facetious answer is that you can't, just as you can't do
it with a default config _before_ these patches -- because neither
CONFIG_TIGON3 nor the various options you need for nfsroot are enabled
by default.

But if you _have_ a working nfsroot+tg3 config and you apply these
patches, then all you need to do is say 'y' when you're asked if you
want to include the firmware for it:
        CONFIG_TIGON3_FIRMWARE=y

If you are competent enough to get nfsroot working, it isn't really very
credible to claim you lack the wit to say 'y' when asked that question,
surely?

Solving that problem was step #1 in the process of converting drivers to
use request_firmware(). You _have_ to be able to build the firmware into
the kernel image, and you _can_.
Fair enough.

I'll step back and let you work out the objections Jeff raised.
Since then, I responded to feedback and changed the individual
CONFIG_XXX_FIRMWARE options to a single CONFIG_FIRMWARE_IN_KERNEL which
controls them all, but basically nothing has changed. What you accepted
then is still true.

On Thu, 2008-07-03 at 13:34 -0700, David Miller wrote:
So don't be surprised how pissed off some of us are about these
changes.  You are inflicting pain on driver maintainers because now
they have to sift through these "firmware not found" reports in
addition to their normal workload.
It also improves coverage testing, and found a real bug in the failure
path...
And David make it seem like it's inconvenient for him to implement the
correct default, which in particular pisses me personally off the
most.  It's totally irresponsible, and I don't care what the legal or
ideological motivation is.
It's not inconvenient at all. It's just the _wrong_ default. But if we
really can't get past it otherwise, then let's just set it to 'y' for
now. I've committed the following, which will appear in linux-next
tomorrow.

Now, can someone _please_ give me a straight response to the allegation
that the TSO firmware on the tg3 is _optional_ anyway, and that it can
work without it? If that's true, we should fix the code path where
request_firmware() fails, so it doesn't abort the initialisation. (And
most of the whining about the driver being 'broken' is nonsense too.)

----
From 400f1b05a9707bd181a044877ca590e87c400749 Mon Sep 17 00:00:00 2001
From: David Woodhouse <dwmw2@infradead.org>
Date: Thu, 3 Jul 2008 21:36:11 +0100
Subject: [PATCH] firmware: default CONFIG_FIRMWARE_IN_KERNEL=y

This is obviously the wrong thing to do in the long (or even medium)
term -- since the recommended way of handling firmware, as used
_unconditionally_ by modern drivers, is to rely on request_firmware()
being satisfied from userspace rather than keeping the firmware in
unswappable static kernel memory.

But this change preserves the property, for now, that the fixes to make
older drivers use request_firmware() introduce _no_ "regressions" when
Aunt Tillie runs 'make oldconfig' and accepts the defaults without
looking at what she's doing.

Signed-off-by: David Woodhouse <dwmw2@infradead.org>
---
 drivers/base/Kconfig |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index 339c148..d47482f 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig
@@ -37,6 +37,7 @@ config FW_LOADER
 config FIRMWARE_IN_KERNEL
 	bool "Include in-kernel firmware blobs in kernel binary"
 	depends on FW_LOADER
+	default y
 	help
 	  The kernel source tree includes a number of firmware 'blobs'
 	  which are used by various drivers. The recommended way to
-- 
1.5.5.1


-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-03 21:03:48

David Woodhouse wrote:
Although it does make me wonder if it was better the way I had it
originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL
attached to each driver, rather than a single FIRMWARE_IN_KERNEL option
which controls them all.
IMO, individual options would be better.

Plus, unless I am misunderstanding, the firmware is getting built into the kernel image not the tg3 module?

If that is the case, then that creates problems by not moving with the driver.

If that is not the case, all good.

	Jeff

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 21:33:59

Jeff Garzik wrote:
David Woodhouse wrote:
quoted
Although it does make me wonder if it was better the way I had it
originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL
attached to each driver, rather than a single FIRMWARE_IN_KERNEL option
which controls them all.
IMO, individual options would be better.
They had individual options for a long time, but the consensus was that I should remove them -- a consensus which was probably right. It was moderately inconvenient going back through it all and recommitting it without that, but it was worth it to get it right...
Plus, unless I am misunderstanding, the firmware is getting built into the kernel image not the tg3 module?
That's right, although it doesn't really matter when they're both in the vmlinux.

When it's actually a module, there really is no good reason not to let request_firmware() get satisfied from userspace. If you can load modules, then you can load firmware too -- the required udev stuff has been there as standard for a _long_ time, as most modern drivers _require_ it without even giving you the built-in-firmware option at all.

It makes no more sense to object to that than it does to object to the module depending on _other_ modules. Both those other modules, and the required firmware, are _installed_ by the kernel Makefiles, after all.

It wouldn't be _impossible_ to put firmware blobs into the foo.ko files themselves and find them there. The firmware blobs in the kernel are done in a separate section (like initcalls, exceptions tables, pci fixups, and a bunch of other stuff). It'd just take some work in module.c to link them into a global list, and some locking shenanigans in the lookups (and lifetime issues to think about). But it just isn't worth the added complexity, given that userspace is known to be alive and working. It's pointless not to just use request_firmware() normally, from a module.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Rafael J. Wysocki <hidden>
Date: 2008-07-03 21:42:22

On Thursday, 3 of July 2008, David Woodhouse wrote:
Jeff Garzik wrote:
quoted
David Woodhouse wrote:
quoted
Although it does make me wonder if it was better the way I had it
originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL
attached to each driver, rather than a single FIRMWARE_IN_KERNEL option
which controls them all.
IMO, individual options would be better.
They had individual options for a long time, but the consensus was that 
I should remove them -- a consensus which was probably right. It was 
moderately inconvenient going back through it all and recommitting it 
without that, but it was worth it to get it right...
quoted
Plus, unless I am misunderstanding, the firmware is getting built into 
the kernel image not the tg3 module?
That's right, although it doesn't really matter when they're both in the 
vmlinux.

When it's actually a module, there really is no good reason not to let 
request_firmware() get satisfied from userspace. If you can load 
modules, then you can load firmware too -- the required udev stuff has 
been there as standard for a _long_ time, as most modern drivers 
_require_ it without even giving you the built-in-firmware option at all.

It makes no more sense to object to that than it does to object to the 
module depending on _other_ modules. Both those other modules, and the 
required firmware, are _installed_ by the kernel Makefiles, after all.

It wouldn't be _impossible_ to put firmware blobs into the foo.ko files 
themselves and find them there. The firmware blobs in the kernel are 
done in a separate section (like initcalls, exceptions tables, pci 
fixups, and a bunch of other stuff). It'd just take some work in 
module.c to link them into a global list, and some locking shenanigans 
in the lookups (and lifetime issues to think about). But it just isn't 
worth the added complexity, given that userspace is known to be alive 
and working. It's pointless not to just use request_firmware() normally, 
from a module.
Still, maybe we can add some kbuild magic to build the blobs along with
their modules and to install them under /lib/firmware (by default) when the
modules are installed in /lib/modules/... ?

Rafael

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 21:44:17

Rafael J. Wysocki wrote:
Still, maybe we can add some kbuild magic to build the blobs along with
their modules and to install them under /lib/firmware (by default) when the
modules are installed in /lib/modules/... ?
Something like appending this to Makefile?

firmware_and_modules_install: firmware_install modules_install

(I'm still wondering if we should make 'firmware_install' install to /lib/firmware by default, instead of into the build tree as 'headers_install' does. The Aunt Tillie answer would definitely be 'yes', although that means it requires root privs; like modules_install does.)

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Rafael J. Wysocki <hidden>
Date: 2008-07-03 21:51:04

On Thursday, 3 of July 2008, David Woodhouse wrote:
Rafael J. Wysocki wrote:
quoted
Still, maybe we can add some kbuild magic to build the blobs along with
their modules and to install them under /lib/firmware (by default) when the
modules are installed in /lib/modules/... ?
Something like appending this to Makefile?

firmware_and_modules_install: firmware_install modules_install

(I'm still wondering if we should make 'firmware_install' install to 
/lib/firmware by default, instead of into the build tree as 
'headers_install' does. The Aunt Tillie answer would definitely be 
'yes', although that means it requires root privs; like modules_install 
does.)
I would prefer 'make firmware_install' to just copy the blobs into specific
location in analogy with 'make modules_install', so that you can build the
blobs as a normal user (for example, on an NFS server) and then put them
into the right place as root (for example, on an NFS client that has no write
privilege on the server).

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-03 21:55:20

Rafael J. Wysocki wrote:
On Thursday, 3 of July 2008, David Woodhouse wrote:
quoted
Rafael J. Wysocki wrote:
quoted
Still, maybe we can add some kbuild magic to build the blobs along with
their modules and to install them under /lib/firmware (by default) when the
modules are installed in /lib/modules/... ?
Something like appending this to Makefile?

firmware_and_modules_install: firmware_install modules_install

(I'm still wondering if we should make 'firmware_install' install to /lib/firmware by default, instead of into the build tree as 'headers_install' does. The Aunt Tillie answer would definitely be 'yes', although that means it requires root privs; like modules_install does.)
I would prefer 'make firmware_install' to just copy the blobs into specific
location in analogy with 'make modules_install', so that you can build the
blobs as a normal user (for example, on an NFS server) and then put them
into the right place as root (for example, on an NFS client that has no write
privilege on the server).
Not entirely sure which you mean. You _can't_ run 'make modules_install' as a normal user, unless you override $(INSTALL_MOD_PATH) on the command line.

Do you want 'make firmware_install' to be the same? It isn't at the moment -- it installs to a subdirectory of the kernel build tree, like 'make headers_install' does. But I'm not sure which is better.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-03 22:22:51

David Woodhouse wrote:
Jeff Garzik wrote:
quoted
David Woodhouse wrote:
quoted
Although it does make me wonder if it was better the way I had it
originally, with individual options like TIGON3_FIRMWARE_IN_KERNEL
attached to each driver, rather than a single FIRMWARE_IN_KERNEL option
which controls them all.
IMO, individual options would be better.
They had individual options for a long time, but the consensus was that I should remove them -- a consensus which was probably right. It was moderately inconvenient going back through it all and recommitting it without that, but it was worth it to get it right...
quoted
Plus, unless I am misunderstanding, the firmware is getting built into the kernel image not the tg3 module?
That's right, although it doesn't really matter when they're both in the vmlinux.

When it's actually a module, there really is no good reason not to let request_firmware() get satisfied from userspace. If you can load modules, then you can load firmware too -- the required udev stuff has been there as standard for a _long_ time, as most modern drivers _require_ it without even giving you the built-in-firmware option at all.

It makes no more sense to object to that than it does to object to the module depending on _other_ modules. Both those other modules, and the required firmware, are _installed_ by the kernel Makefiles, after all.

It wouldn't be _impossible_ to put firmware blobs into the foo.ko files themselves and find them there. The firmware blobs in the kernel are done in a separate section (like initcalls, exceptions tables, pci fixups, and a bunch of other stuff). It'd just take some work in module.c to link them into a global list, and some locking shenanigans in the lookups (and lifetime issues to think about). But it just isn't worth the added complexity, given that userspace is known to be alive and working. It's pointless not to just use request_firmware() normally, from a module.
It is pointless -- if you assume everybody wants to run their distro and universe your way.

If a firmware is built-in, then 'make firmware_install' is clearly optional and may be omitted.  That invalidates several of your assumptions above.

Further, all current kernel build and test etc. scripts are unaware of 'make firmware_install', and it is unfair to everybody to force a flag-day build process change on people, just to keep their drivers in the same working state today as it was yesterday.


Conclusion - kernel build process must produce a working driver after your changes are applied, even in absence of 'make firmware_install'.

That does not, repeat /not/ exclude the desired end goal of course -- separating the firmware from the driver, installing in /lib/firmware via 'make firmware_install', etc.


I support your end goal, I really do.  But I continue to feel there is a lack of regard for breakage and regressions.  You are either ignoring or apparently just not seeing
	* how many new ways this can produce a non-working driver
	* how important it is in this specific case to fail-safe,
	  and avoid a broken driver at all costs.

As a concrete example, in the above quoted text you assume that a user will never copy kernel modules around.  I can tell you that, with tg3.ko being nice and self-contained, yes it does get copied around (to multiple machines, etc.).  With the firmware newly separated from tg3.ko, you have introduced breakage for any user that is unaware of this new requirement (kernel module == requires additional file now).

Scripts that build install disks must be updated, otherwise a script that builds a boot image will include the drivers it knows about, but /not/ include the crucial firmware.

None of this stuff is "pointless", none of this stuff may be dismissed as "making no sense".  All these are real world examples where users FOLLOWING THEIR NORMAL, PROSCRIBED KERNEL PROCESSES will produce non-working drivers.

The only valid assumption here is to assume that the user is /unaware/ of these new steps they must take in order to continue to have a working system.

Regards,

	Jeff



Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Rafael J. Wysocki <hidden>
Date: 2008-07-03 22:25:54

On Thursday, 3 of July 2008, David Woodhouse wrote:
Rafael J. Wysocki wrote:
quoted
On Thursday, 3 of July 2008, David Woodhouse wrote:
quoted
Rafael J. Wysocki wrote:
quoted
Still, maybe we can add some kbuild magic to build the blobs along with
their modules and to install them under /lib/firmware (by default) when the
modules are installed in /lib/modules/... ?
Something like appending this to Makefile?

firmware_and_modules_install: firmware_install modules_install

(I'm still wondering if we should make 'firmware_install' install to 
/lib/firmware by default, instead of into the build tree as 
'headers_install' does. The Aunt Tillie answer would definitely be 
'yes', although that means it requires root privs; like modules_install 
does.)
I would prefer 'make firmware_install' to just copy the blobs into specific
location in analogy with 'make modules_install', so that you can build the
blobs as a normal user (for example, on an NFS server) and then put them
into the right place as root (for example, on an NFS client that has no write
privilege on the server).
Not entirely sure which you mean. You _can't_ run 'make modules_install' 
as a normal user, unless you override $(INSTALL_MOD_PATH) on the command 
line.
Yes, I know that.
Do you want 'make firmware_install' to be the same?
Yes, I'd prefer it to behave in the same way as 'make modules_install'.

I'd use a config option like BUILD_FIRMWARE_BLOBS that, if set, would make
the build system create firmware bin files in the same directories where the
driver's .o files are located.  Then, 'make firmware_install' would only copy
those bin files to wherever was appropriate (eg. /lib/firmware/).

Of course, there still would be a problem if there already were such firmware
files at the destination, but that would have to be resolved anyway by the user
wanting to install the new kernel along with the new firmware blobs.
It isn't at the moment -- it installs to a subdirectory of the kernel build tree, like 
'make headers_install' does. But I'm not sure which is better.
IMO 'make headers_install' is used for a different purpose.  You don't have to
run it to be able to use the kernel in a usual way.

OTOH, everyone is familiar with the 'make modules_install' mechanics and it
seems natural to use analogous mechanics for firmware blobs.

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Alan Cox <hidden>
Date: 2008-07-03 22:44:35

O
Further, all current kernel build and test etc. scripts are unaware of 
'make firmware_install', and it is unfair to everybody to force a 
flag-day build process change on people, just to keep their drivers in 
the same working state today as it was yesterday.
IMHO we want firmware built in as the default for the moment. If the
firmware model makes sense (as I think it does) then the distributions
will catch up, turn it on and sort out the default behaviour - exactly as
they did all those years ago with modules, more recently with "use an
initrd" and so on.
as "making no sense".  All these are real world examples where users 
FOLLOWING THEIR NORMAL, PROSCRIBED KERNEL PROCESSES will produce 
I hope you mean "prescribed" ;)
The only valid assumption here is to assume that the user is /unaware/ 
of these new steps they must take in order to continue to have a working 
system.
To a large extent not the user but their distro - consider "make install"

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-03 23:14:57

Alan Cox wrote:
quoted
Further, all current kernel build and test etc. scripts are unaware of 'make firmware_install', and it is unfair to everybody to force a flag-day build process change on people, just to keep their drivers in the same working state today as it was yesterday.
IMHO we want firmware built in as the default for the moment. If the
firmware model makes sense (as I think it does) then the distributions
will catch up, turn it on and sort out the default behaviour - exactly as
they did all those years ago with modules, more recently with "use an
initrd" and so on.
Agreed.

quoted
as "making no sense".  All these are real world examples where users FOLLOWING THEIR NORMAL, PROSCRIBED KERNEL PROCESSES will produce
I hope you mean "prescribed" ;)
heh, *cough* yes

quoted
The only valid assumption here is to assume that the user is /unaware/ of these new steps they must take in order to continue to have a working system.
To a large extent not the user but their distro - consider "make install"
Actually, I was tossing that about in my head:

Is it a better idea to eliminate 'make firmware_install' completely, and instead implement it silently via 'make install'?

'make install' is already a big fat distro hook...

	Jeff


Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Alan Cox <hidden>
Date: 2008-07-03 23:20:23

Actually, I was tossing that about in my head:

Is it a better idea to eliminate 'make firmware_install' completely, and 
instead implement it silently via 'make install'?

'make install' is already a big fat distro hook...
make firmware_install can encapsulate a lot of kernel specific knowledge
so I think it belongs in the kernel tree to avoid problems in future. The
use of make firmware_install belongs in the distro make install hooks.

Otherwise we will mess up the distro stuff if we have to change the
innards of make firmware_install in future, as may well occur.

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Miller <davem@davemloft.net>
Date: 2008-07-03 23:21:34

From: David Woodhouse <dwmw2@infradead.org>
Date: Thu, 03 Jul 2008 19:56:02 +0100
It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y',
because the _normal_ setting for that option _really_ should be 'N'.
On what basis?  From a "obviously works" basis, the default should be
'y'.
What we're doing now is just cleaning up the older drivers which don't
use request_firmware(), to conform to what is now common practice.
You say "conform" I say "break".
In the meantime, it would be useful if Jeff would quit throwing his toys
out of the pram on that issue and actually review the _code_ changes. In
particular, are the reports correct that the device operates just fine
without the TSO firmware loaded? Should we change the request_firmware()
error path to just disable TSO and continue with the initialisation?
No!

The 5701 A0 firmware is necessary to load in order to work around
hardware and existing firmware bugs on those cards.  It's an issue of
basic functionality, not just optimizations.

5701 A0 tg3 chips cannot operate at all without the firmware being
present in the driver.

Therefore, if you can't load the firmware, the card is not going to
work.
Less of the ad hominem, please. Especially when it's so misdirected.
No, it is properly directed, you are breaking the tree for users.
Updating these drivers to remove large blobs of static unswappable data
from the kernel, and having it provided from userspace on demand as
modern Linux drivers do, is a perfectly sensible technical goal all on
its own.
I disagree.
And given the GPL's explicit provisions with regard to collective works
there are also entirely reasonable, non-"fundamentalist" grounds for
believing that it _may_ pose a licensing problem, and for wanting to err
on the side of caution in that respect too.
So now the real truth is revealed.  You have no technical basis for
this stuff you are ramming down everyone's throats.

You want to choose a default based upon your legal agenda.

That explains all of the bullshit that is attached to your work, and
all of the bullshit arguments you make wrt. choosing defaults that
break things for users.

It's all about agendas rather than any real technical objectives.

If it was purely technical, you wouldn't be choosing defaults that
break things for users by default.  Jeff and I warned you about this
from day one, you did not listen, and now we have at least 10 reports
just today of people with broken networking.

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Theodore Tso <tytso@mit.edu>
Date: 2008-07-04 00:20:32

On Thu, Jul 03, 2008 at 04:21:20PM -0700, David Miller wrote:
quoted
And given the GPL's explicit provisions with regard to collective works
there are also entirely reasonable, non-"fundamentalist" grounds for
believing that it _may_ pose a licensing problem, and for wanting to err
on the side of caution in that respect too.
So now the real truth is revealed.  You have no technical basis for
this stuff you are ramming down everyone's throats.

You want to choose a default based upon your legal agenda.
Yep, legal agenda.  As I suspected, licensing religious fundamentalism.  :-)

People who care can change the defaults.  People who are real
religious nuts won't even let the firmware live in the same source
tarball.  But I hope you agree we clearly don't have consensus to take
*that* step (rip out all firmware and make a whole bunch of drivers
non-functional and forcing users to go on a treasure-hunt to find some
new tarball they have to install on their existing system).  So given
that we're not ready to take that step, why not just leave the default
as "yes" for now?

The staged approach means that if you really want to do this ASAP,
then start assembling the firmware tarball *now*, and for a while
(read: at least 9-18 months) we can distribute firmware both in the
kernel source tarball as well as separately in the
licensing-religion-firmware tarball.  See if you can get distros
willing to ship it by default in most user's systems, and give people
plenty of time to understand that we are trying to decouple firmware
from the kernel sources.  If we need to institute better versioning
regimes between the drivers and firmware release levels, that will
also give people a chance to get that all right.  Then 6-9 months
later, we can turn the default to 'no', and then maybe another 6-9
months after that, we can talk about removing the firmware modules.
But it seems to me that you are skipping a few steps by arguing that
the default should be changed here-and-now.

We've been shipping firmware in the kernel for over a ***decade***; in
fact, probably over 15 years.  For people who are legal freaks/geeks,
look up the legal terms "Estoppel" and "Laches".  That provides a
fairly large amount of protection right there.  For people who aren't
legal geeks, we've been doing this for well over a decade; another
year or two really isn't a big deal.  It certainly doesn't justify
breaking users by default just to try to hurry up this process.
If it was purely technical, you wouldn't be choosing defaults that
break things for users by default.  Jeff and I warned you about this
from day one, you did not listen, and now we have at least 10 reports
just today of people with broken networking.
Not 15 minutes after David posted his note, we're now up to 11
reports; and this is only from an -mm patch series.  Can you imagine
the number of bug reports if this were allowed to ship in a mainline
kernel.org release?  One good thing is that we can definitely show
that there people that are downloading, compiling and trying to build
the -mm kernel.  :-)

						- Ted

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 00:25:49

David Miller wrote:
From: David Woodhouse <dwmw2@infradead.org>
Date: Thu, 03 Jul 2008 19:56:02 +0100
quoted
It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y',
because the _normal_ setting for that option _really_ should be 'N'.
On what basis?  From a "obviously works" basis, the default should be
'y'.
I already changed it to 'y'.
quoted
What we're doing now is just cleaning up the older drivers which don't
use request_firmware(), to conform to what is now common practice.
You say "conform" I say "break".
You mean...
	"What we're doing now is just cleaning up the older drivers
	 which don't use request_firmware(), to break to what is now
	 common practice."
?

Doesn't really scan, does it?

Common practice in modern Linux drivers is to use request_firmware(). I'm just going through and fixing up the older ones to do that too.

(After making it possible to build that firmware _into_ the kernel so that we aren't forcing people to use an initrd where they didn't before, of course.)

The word for that is definitely 'conform'. I know you don't _like_ the modern accepted practice, but that's _your_ windmill to tilt at. I have my own :)
quoted
In the meantime, it would be useful if Jeff would quit throwing his toys
out of the pram on that issue and actually review the _code_ changes. In
particular, are the reports correct that the device operates just fine
without the TSO firmware loaded? Should we change the request_firmware()
error path to just disable TSO and continue with the initialisation?
No!

The 5701 A0 firmware is necessary to load in order to work around
hardware and existing firmware bugs on those cards.  It's an issue of
basic functionality, not just optimizations.

5701 A0 tg3 chips cannot operate at all without the firmware being
present in the driver.

Therefore, if you can't load the firmware, the card is not going to
work.
Neat avoidance of question there... it was fairly clear that the 5701_A0 firmware was going to be mandatory; I was asking about the TSO firmware.

Does anyone _else_ actually want to give a straight answer to a simple question? Someone who wouldn't have to follow it with an apology because of all their shouting about 'breakage' when the firmware in question is actually optional anyway, perhaps?

If it was purely technical, you wouldn't be choosing defaults that
break things for users by default.
Actually, the beauty of Linux is that we _can_ change things where a minor short-term inconvenience leads to a better situation in the long term.
Jeff and I warned you about this from day one, you did not listen, and
now we have at least 10 reports just today of people with broken
networking.
Out of interest... of those, what proportion would be 'fixed' if they'd just paid attention when running 'make oldconfig', which is now addressed because I've changed the FIRMWARE_IN_KERNEL default to 'y'?

And how many would be 'fixed' if someone had given me a straight answer when I asked about the TSO firmware, and that failure path no longer aborted the driver initialisation but instead just fell back to non-TSO?

I'll look at making the requirement for 'make firmware_install' more obvious, or even making it happen automatically as part of 'modules_install'.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 01:10:01

Theodore Tso wrote:
On Thu, Jul 03, 2008 at 04:21:20PM -0700, David Miller wrote:
quoted
You want to choose a default based upon your legal agenda.
Yep, legal agenda.  As I suspected, licensing religious fundamentalism.  :-)
You are mistaken, and you're responding some words that someone _else_ put in my mouth.
The staged approach means that if you really want to do this ASAP,
then start assembling the firmware tarball *now*,
That's easy enough. We can automatically generate a tree _from_ Linus' tree, with a scripted transform so that it includes only the firmware files (much like the kernel-headers tree automatically follows each commit in Linus' tree, but includes only the exported headers).

And there are some hardware manufacturers who are willing to have their firmware included in such a firmware tarball, but who will _not_ give their permission to have it included in the Linux kernel because of the legal concerns you're so dismissive of. But that's OK -- we can pull from the automatically generated tree into a 'real' linux-firmware.git tree, which includes those extra firmware files.

But there's no need to do it _now_. It can wait until the basic stuff is in Linus' tree and it can automatically derive from that. There's no particular rush, is there?
and for a while (read: at least 9-18 months) we can distribute firmware
both in the kernel source tarball as well as separately
That makes a certain amount of sense.
in the licensing-religion-firmware tarball.
Please don't be gratuitously offensive, Ted. It's not polite, and it's not a particularly good debating technique either. I expect better from you.
See if you can get distros willing to ship it by default in most
user's  systems, and give people plenty of time to understand that we
are trying to decouple firmware from the kernel sources.
The distros are certainly willing (and keen) to do it. The Fedora Engineering Steering Committee has already stated that it wants to do so, and the specfile changes to spit out a 'kernel-firmware' sub-package with the kernel build are ready to go right now.

Fedora already modifies tarballs, for example 'openssh-5.0p1-noacss'. I think it's quite likely they'd end up using a 'linux-2.6.27-nofirmware' tarball too, and build the firmware package from the separate tree.

Some other distributions have been doing that kind of thing _already_, even when it meant just ripping out certain drivers completely. That seems excessive to me; I prefer not to actually _break_ anything.
If we need to institute better versioning regimes between the drivers
and firmware release levels, that will also give people a chance to
get that all right.  Then 6-9 months later, we can turn the default
to 'no', and then maybe another 6-9 months after that, we can talk about removing the firmware modules.
But it seems to me that you are skipping a few steps by arguing that
the default should be changed here-and-now.
I disagree that the _default_ is such an issue -- largely because the normal case for modern drivers is not to include the firmware _anyway_, and the tools like 'mkinitrd' already cope with it just fine.

But I've changed the default to 'y' now, as I already said.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Grant Coady <hidden>
Date: 2008-07-04 01:29:19

On Fri, 04 Jul 2008 01:24:59 +0100, David Woodhouse [off-list ref] wrote:
...
I'll look at making the requirement for 'make firmware_install' more 
obvious, or even making it happen automatically as part of 
'modules_install'.
I like this one:  Automagically part of modules_install.  No break existing 
kernel build scripts for the expected-by-user build sequence? 

And please put 'make firmware_install' into 'make help' if that's the way 
you go.  

And another point, at the moment I seem to get all sorts of odd things 
built I didn't know were there?

root@pooh:/home/grant/linux/linux-2.6.26-rc8-mm1a# make firmware_install
  HOSTCC  firmware/ihex2fw
  IHEX2FW firmware/atmsar11.fw
  INSTALL usr/lib/firmware/atmsar11.fw
  IHEX2FW firmware/dabusb/firmware.fw
  INSTALL usr/lib/firmware/dabusb/firmware.fw
  IHEX2FW firmware/emi26/loader.fw
  INSTALL usr/lib/firmware/emi26/loader.fw
  IHEX2FW firmware/emi26/firmware.fw
  INSTALL usr/lib/firmware/emi26/firmware.fw
  IHEX2FW firmware/emi26/bitstream.fw
  INSTALL usr/lib/firmware/emi26/bitstream.fw
  IHEX2FW firmware/emi62/loader.fw
  INSTALL usr/lib/firmware/emi62/loader.fw
  IHEX2FW firmware/emi62/bitstream.fw
  INSTALL usr/lib/firmware/emi62/bitstream.fw
  IHEX2FW firmware/emi62/spdif.fw
  INSTALL usr/lib/firmware/emi62/spdif.fw
  IHEX2FW firmware/emi62/midi.fw
  INSTALL usr/lib/firmware/emi62/midi.fw
  IHEX2FW firmware/keyspan/mpr.fw
  INSTALL usr/lib/firmware/keyspan/mpr.fw
  IHEX2FW firmware/keyspan/usa18x.fw
  INSTALL usr/lib/firmware/keyspan/usa18x.fw
  IHEX2FW firmware/keyspan/usa19.fw
  INSTALL usr/lib/firmware/keyspan/usa19.fw
  IHEX2FW firmware/keyspan/usa19qi.fw
  INSTALL usr/lib/firmware/keyspan/usa19qi.fw
  IHEX2FW firmware/keyspan/usa19qw.fw
  INSTALL usr/lib/firmware/keyspan/usa19qw.fw

30 more lines

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Theodore Tso <tytso@mit.edu>
Date: 2008-07-04 01:48:41

On Fri, Jul 04, 2008 at 02:09:15AM +0100, David Woodhouse wrote:
But there's no need to do it _now_. It can wait until the basic stuff is  
in Linus' tree and it can automatically derive from that. There's no  
particular rush, is there?
The only rush if the main agenda is to extirpate all firmware from the
mainline kernel sources.  I don't think we can start the timer for
doing that until the firmware tarball is created and it starts being
used as the default way of delivering firmware for what you call
"legacy" drivers.  If there's no particular rush to finally rm the
firmware for trivers such as tg3 from the source tree (and I
personally don't think there should be any rush), or any rush to
change the default for CONFIG_FIRMWARE_IN_KERNEL to "no", then I don't
see any rush in creating the firmware tarball.  

If *you* think there is a rush in making CONFIG_FIRMWARE_IN_KERNEL
default to "no", then you might want to decide to create the firmware
tarball sooner, and get distro's everywhere to start using it, and to
get everyone to understand that they should start including it in
their systems.  (Remember, not everyone uses the popular distributions
like Fedora, Debian, Ubuntu, Open SuSE, etc.)

But heck, that's up to you.  :-)
quoted
and for a while (read: at least 9-18 months) we can distribute firmware
both in the kernel source tarball as well as separately
That makes a certain amount of sense.
Glad we agree.
quoted
in the licensing-religion-firmware tarball. 
Please don't be gratuitously offensive, Ted. It's not polite, and it's  
not a particularly good debating technique either. I expect better from 
you.
Well, I think it's offensive to break users who have happily been
using drivers that have been including firmware for a long, long, LONG
time, and I expected better of you.

So there.  :-)

						- Ted

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Mikael Pettersson <hidden>
Date: 2008-07-04 02:32:53

Alan Cox writes:
 > > The only valid assumption here is to assume that the user is /unaware/ 
 > > of these new steps they must take in order to continue to have a working 
 > > system.
 > 
 > To a large extent not the user but their distro - consider "make install"
 > --

Last time I checked only x86 had 'make install'. I regularly build
natively on ppc(32|64) and sparc64, and none of them implement
'make install' AFAIK. And on ARM I move the kernels over to a tftp
server for network boots, again w/o 'make install'.

Not that 'make install' is difficult. All it does it hand over to
/sbin/installkernel or something like that.

In the context of .config changes, 'make oldconfig' with 'select the
default' must IMO result in a working kernel similar to the previous
one. Anything else is madness or arrogance.

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: <hidden>
Date: 2008-07-04 02:42:21

On Fri, 4 Jul 2008, David Woodhouse wrote:
David Miller wrote:
quoted
From: David Woodhouse <dwmw2@infradead.org>
Date: Thu, 03 Jul 2008 19:56:02 +0100
quoted
It's wrong to change the CONFIG_FIRMWARE_IN_KERNEL default to 'Y',
because the _normal_ setting for that option _really_ should be 'N'.
On what basis?  From a "obviously works" basis, the default should be
'y'.
I already changed it to 'y'.
quoted
quoted
What we're doing now is just cleaning up the older drivers which don't
use request_firmware(), to conform to what is now common practice.
You say "conform" I say "break".
You mean...
	"What we're doing now is just cleaning up the older drivers
	 which don't use request_firmware(), to break to what is now
	 common practice."
?

Doesn't really scan, does it?

Common practice in modern Linux drivers is to use request_firmware(). I'm just going through and fixing up the older ones to do that too.

(After making it possible to build that firmware _into_ the kernel so that we aren't forcing people to use an initrd where they didn't before, of course.)
has this taken place yet? (and if so, what kernel version first included this fix)
quoted
If it was purely technical, you wouldn't be choosing defaults that
break things for users by default.
Actually, the beauty of Linux is that we _can_ change things where a minor short-term inconvenience leads to a better situation in the long term.
but doing so should not be a easy and quick decision, and it needs to be made very clear exactly what breakage is going to take place and why (along with the explination of why the breakage couldn't be avoided)
quoted
Jeff and I warned you about this from day one, you did not listen, and
now we have at least 10 reports just today of people with broken
networking.
Out of interest... of those, what proportion would be 'fixed' if they'd just paid attention when running 'make oldconfig', which is now addressed because I've changed the FIRMWARE_IN_KERNEL default to 'y'?

And how many would be 'fixed' if someone had given me a straight answer when I asked about the TSO firmware, and that failure path no longer aborted the driver initialisation but instead just fell back to non-TSO?

I'll look at making the requirement for 'make firmware_install' more obvious, or even making it happen automatically as part of 'modules_install'.
I won't mind this as long as I can get a working kernel without doing make firmware_install or make modules_install (I almost never use modules, my laptop is one of the few exceptions, and even there it's mostly becouse of the intel wireless driver needing userspace for firmware)

David Lang

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 10:08:12

On Thu, 2008-07-03 at 19:42 -0700, david@lang.hm wrote:
quoted
(After making it possible to build that firmware _into_ the kernel so that we 
aren't forcing people to use an initrd where they didn't before, of course.)
has this taken place yet? (and if so, what kernel version first included 
this fix)
It's in linux-next now. See the CONFIG_EXTRA_FIRMWARE option.

That's one of the reasons Ted's ranting about 'religious fundamentalism'
is so funny -- I've actually made it _easier_ for you to combine
arbitrary firmware files into your kernel.
quoted
quoted
If it was purely technical, you wouldn't be choosing defaults that
break things for users by default.
Actually, the beauty of Linux is that we _can_ change things where a minor 
short-term inconvenience leads to a better situation in the long term.
but doing so should not be a easy and quick decision, and it needs to be 
made very clear exactly what breakage is going to take place and why 
(along with the explination of why the breakage couldn't be avoided)
All forms of change introduce _some_ risk of breakage, of course. In
this case, as usual, I've tried to be careful to avoid regressions. The
most important part, obviously, was having a way to build firmware into
the static kernel.

When it comes to _modules_, doing that would introduce a certain amount
of complexity which just doesn't seem necessary -- if you can load
modules, then you have userspace, and you can use hotplug for firmware
too. Especially given that so many modern drivers already _require_ you
to do that, so the users understand it, and the tools like mkinitrd
already cope with it -- checking MODULE_FIRMWARE() for the modules they
include and including the appropriate files automatically.

The alleged problem with modules seems to be _just_ about the fact that
people need to run 'make firmware_install', and need to have their
firmware installed. Something which all modern drivers require _anyway_,
and people are used to in the general case already. It's not exactly
hard; there's just the initial step of realising that the driver _you_
are using has now been updated to behave like all the others.

That step is _minor_, and it doesn't actually get any easier _however_
long you postpone it. Yes, you might get 10 people in the first day
tripping over it, and I'll look to see if I can make it clearer. But
it's still a very minor, short-term thing.

I suspect that the best option is just to hold off on updating the net
drivers until later, when people already _know_ that they need to run
'make firmware_install', (or it happens automatically or something, if
we go down that route). That way, Dave and Jeff shouldn't be affected by
the initial transition period.

There's plenty of other drivers which need updating, after all. And most
maintainers are _happy_ to see the patches to bring their drivers in
line with best current practice.
quoted
I'll look at making the requirement for 'make firmware_install' more obvious, 
or even making it happen automatically as part of 'modules_install'.
I won't mind this as long as I can get a working kernel without doing make 
firmware_install or make modules_install
You can. You need to stay sober for long enough to say 'Y' when it asks
you if you want to build the required firmware into the kernel. And we
even made that the _default_ now, for the benefit of those who can't
stay sober that long. (Perhaps we'll make 'allyesconfig' the default
next?)
 (I almost never use modules, my laptop is one of the few exceptions,
and even there it's mostly becouse of the intel wireless driver
needing userspace for firmware)
You don't need to do that any more :)

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Takashi Iwai <hidden>
Date: 2008-07-04 11:06:21

At Thu, 3 Jul 2008 14:04:36 +0100 (BST),
Hugh Dickins wrote:
On Thu, 3 Jul 2008, Jeff Garzik wrote:
quoted
KOSAKI Motohiro wrote:
quoted
Hi Michael,

my server output following error message on 2.6.26-rc8-mm1.
Is this a bug?

------------------------------------------------------------------
tg3.c:v3.93 (May 22, 2008)
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.0: PCI INT A -> GSI 72 (level, low) -> IRQ 51
firmware: requesting tigon/tg3_tso.bin
tg3: Failed to load firmware "tigon/tg3_tso.bin"
tg3 0000:06:01.0: PCI INT A disabled
GSI 72 (level, low) -> CPU 0 (0x0001) vector 51 unregistered
tg3: probe of 0000:06:01.0 failed with error -2
GSI 73 (level, low) -> CPU 0 (0x0001) vector 51
tg3 0000:06:01.1: PCI INT B -> GSI 73 (level, low) -> IRQ 52
firmware: requesting tigon/tg3_tso.bin
This change did not come from the network developers or Broadcom, so someone
else broke tg3 in -mm...
I think it's a consequence of not choosing CONFIG_FIRMWARE_IN_KERNEL=y.

That caught me out on PowerMac G5 trying mmotm yesterday, it just hung
for a few minutes in earlyish boot with a message about tg3_tso.bin,
and then proceeded to boot up but without the network.  I was unclear
whether I'd been stupid, or the FIRMWARE_IN_KERNEL Kconfigery was poor.

I avoid initrd, and have tigon3 built in, if that's of any relevance.

I wonder if that's Andrew's problem with 2.6.26-rc8-mm1 on his G5:
mine here boots up fine (now I know to CONFIG_FIRMWARE_IN_KERNEL=y).
Hmm, I got this error even with CONFIG_FIRMWARE_IN_KERNEL=y.

Through a quick look at the code, the firmwares are not built indeed.
I guess the fix like the following needed for building firmwares for
modules.  Now trying to build the kernel again to check this...


Takashi

diff --git a/firmware/Makefile b/firmware/Makefile
index f88d746..e11fc2a 100644
--- a/firmware/Makefile
+++ b/firmware/Makefile
@@ -55,6 +55,8 @@ fw-shipped-$(CONFIG_USB_SERIAL_XIRCOM) += keyspan_pda/xircom_pgs.fw
 fw-shipped-$(CONFIG_USB_VICAM) += vicam/firmware.fw
 fw-shipped-$(CONFIG_VIDEO_CPIA2) += cpia2/stv0672_vp4.bin
 
+fw-shipped-y := $(fw-shipped-y) $(fw-shipped-m)
+
 # If CONFIG_FIRMWARE_IN_KERNEL is not set, then don't include any firmware
 ifneq ($(CONFIG_FIRMWARE_IN_KERNEL),y)
 fw-shipped-y :=

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 13:18:20

On Fri, 2008-07-04 at 13:06 +0200, Takashi Iwai wrote:
Hmm, I got this error even with CONFIG_FIRMWARE_IN_KERNEL=y.

Through a quick look at the code, the firmwares are not built indeed.
I guess the fix like the following needed for building firmwares for
modules.  Now trying to build the kernel again to check this...
For modules, you just need run
	'make INSTALL_FW_PATH=/lib/firmare firmware_install'.

I should...

1. Change the default to /lib/firmware so that you don't have to set
   INSTALL_FW_PATH.
2. Add that to the 'make help' text.
3. Look at making 'make modules_install' installl the firmware required
   by the modules it's installing, so you don't even need to do
   _anything_ new.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Takashi Iwai <hidden>
Date: 2008-07-04 13:27:16

At Fri, 04 Jul 2008 14:17:51 +0100,
David Woodhouse wrote:
On Fri, 2008-07-04 at 13:06 +0200, Takashi Iwai wrote:
quoted
Hmm, I got this error even with CONFIG_FIRMWARE_IN_KERNEL=y.

Through a quick look at the code, the firmwares are not built indeed.
I guess the fix like the following needed for building firmwares for
modules.  Now trying to build the kernel again to check this...
For modules, you just need run
	'make INSTALL_FW_PATH=/lib/firmare firmware_install'.

I should...

1. Change the default to /lib/firmware so that you don't have to set
   INSTALL_FW_PATH.
2. Add that to the 'make help' text.
3. Look at making 'make modules_install' installl the firmware required
   by the modules it's installing, so you don't even need to do
   _anything_ new.
Ah I see.  I thought you implemented the built-in firmware even for
modules, but apparently it's not.

Is mkinitrd clever enough to put all needed firmware files to initrd
automatically?  Otherwise this can still break the existing setup...


thanks,

Takashi

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 13:28:54

On Fri, 2008-07-04 at 15:26 +0200, Takashi Iwai wrote:
Ah I see.  I thought you implemented the built-in firmware even for
modules, but apparently it's not.
It isn't really necessary. If you can load modules, then you have
userspace. And if you have userspace, you can load firmware too.
Is mkinitrd clever enough to put all needed firmware files to initrd
automatically?  Otherwise this can still break the existing setup...
Yes, it's had to be clever enough for that for a long time anyway --
most modern drivers have _only_ the 'request_firmware()' option and
never gave you the choice of building it in. I'm just updating some of
the older drivers to catch up.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-04 13:29:17

David Woodhouse wrote:
3. Look at making 'make modules_install' installl the firmware required
   by the modules it's installing, so you don't even need to do
   _anything_ new.

Anything that works with existing build processes would be a positive step, preventing loads of build&test regressions.

This does not change the need, of course, to default this firmware install stuff to 'off', and allow distros to turn it on when they're ready (as Alan, Ted, and others have suggested in this thread).

	Jeff


Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-04 13:42:51

Takashi Iwai wrote:
Ah I see.  I thought you implemented the built-in firmware even for
modules, but apparently it's not.
Correct.

Is mkinitrd clever enough to put all needed firmware files to initrd
automatically?  Otherwise this can still break the existing setup...
mkinitrd and similar scripts must be updated, so that drivers that worked prior to dwmw2's changes will continue to work after dwmw2's changes.

If you fail to update some script somewhere, then the driver will be copied into the initramfs, but not the firmware, with obvious results.

This is not a fail-safe system.

This is an enforced-breakage, flag-day change that will cause a lot of additional work for a lot of people.

	Jeff


Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 13:45:51

On Fri, 2008-07-04 at 09:42 -0400, Jeff Garzik wrote:
mkinitrd and similar scripts must be updated, so that drivers that 
worked prior to dwmw2's changes will continue to work after dwmw2's
changes.
If you fail to update some script somewhere, then the driver will be 
copied into the initramfs, but not the firmware, with obvious results.
No, mkinitrd works fine, because a whole boatload of drivers _already_
require it to work that way and have done for a long time.

Either you are severely mistaken, or you are being deliberately
misleading.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-04 14:11:13

David Woodhouse wrote:
On Fri, 2008-07-04 at 09:42 -0400, Jeff Garzik wrote:
quoted
mkinitrd and similar scripts must be updated, so that drivers that worked prior to dwmw2's changes will continue to work after dwmw2's
changes.
quoted
If you fail to update some script somewhere, then the driver will be copied into the initramfs, but not the firmware, with obvious results.
No, mkinitrd works fine, because a whole boatload of drivers _already_
require it to work that way and have done for a long time.

Either you are severely mistaken, or you are being deliberately
misleading.
It is a fact that mkinitrd, today, is unaware of your new system of obtaining firmware from the kernel source[or build] tree.

Certainly it is aware of the need to copy firmware in general, but that doesn't change the fact that the tg3 firmware will not make it into initramfs, without additional steps taken.

So, no, it doesn't "work fine" -- the firmware doesn't make it into the initramfs.

	Jeff

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: David Woodhouse <dwmw2@infradead.org>
Date: 2008-07-04 14:13:47

On Fri, 2008-07-04 at 10:10 -0400, Jeff Garzik wrote:
David Woodhouse wrote:
quoted
On Fri, 2008-07-04 at 09:42 -0400, Jeff Garzik wrote:
quoted
mkinitrd and similar scripts must be updated, so that drivers that 
worked prior to dwmw2's changes will continue to work after dwmw2's
changes.
quoted
If you fail to update some script somewhere, then the driver will be 
copied into the initramfs, but not the firmware, with obvious results.
No, mkinitrd works fine, because a whole boatload of drivers _already_
require it to work that way and have done for a long time.

Either you are severely mistaken, or you are being deliberately
misleading.
It is a fact that mkinitrd, today, is unaware of your new system of 
obtaining firmware from the kernel source[or build] tree.
Of _course_ it is. Just as it's unaware of the need to download the b43
driver and b43-fwcutter and extract it.... what was your point, again?

I'm working on making the required firmware get installed as part of
'make modules_install'. We already discussed that. There's no need for
you to continue crying wolf with nonsense like 'mkinitrd won't work'.

-- 
dwmw2

Re: [bug?] tg3: Failed to load firmware "tigon/tg3_tso.bin"

From: Jeff Garzik <hidden>
Date: 2008-07-05 06:14:25

David Woodhouse wrote:
I'm working on making the required firmware get installed as part of
'make modules_install'.
Great!  That will definitely reduce silent regressions due to build process changes.

	Jeff

Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help