From: Martin Wilck <redacted>
Hello Martin, Himanshu, all,
here is v2 of the little series I first submitted on Nov 29, 2019.
Changes wrt v2:
- Added patch 3 to set the UNLOADING flag before waiting for sessions
to end (Roman)
- Use test_and_set_bit() for UNLOADING (Hannes)
Martin Wilck (3):
scsi: qla2xxx: avoid sending mailbox commands if firmware is stopped
scsi: qla2xxx: don't shut down firmware before closing sessions
scsi: qla2xxx: set UNLOADING before waiting for session deletion
drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
drivers/scsi/qla2xxx/qla_os.c | 39 +++++++++++++++++-----------------
2 files changed, 22 insertions(+), 20 deletions(-)
--
2.25.0
From: Martin Wilck <redacted>
Since 45235022da99, the firmware is shut down early in the controller
shutdown process. This causes commands sent to the firmware (such as LOGO)
to hang forever. Eventually one or more timeouts will be triggered.
Move the stopping of the firmware until after sessions have terminated.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
From: Martin Wilck <redacted>
Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"),
it is possible that FC commands are scheduled after the adapter firmware
has been shut down. IO sent to the firmware in this situation hangs
indefinitely. Avoid this for the LOGO code path that is typically taken
when adapters are shut down.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Signed-off-by: Martin Wilck <redacted>
Reviewed-by: Roman Bolshakov <redacted>
---
drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
drivers/scsi/qla2xxx/qla_os.c | 3 +++
2 files changed, 6 insertions(+)
From: Martin Wilck <redacted>
The purpose of the UNLOADING flag is to avoid port login procedures
to continue when a controller is in the process of shutting down.
It makes sense to set this flag before starting session teardown.
The only operations that must be able to continue are LOGO, PRLO,
and the like.
Furthermore, use atomic test_and_set_bit() to avoid the shutdown
being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(),
the test for UNLOADING is postponed until after the check for an already
disabled PCI board.
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 25 +++++++++++--------------
1 file changed, 11 insertions(+), 14 deletions(-)
From: Daniel Wagner <hidden> Date: 2020-02-06 12:33:33
Hi Martin,
On Wed, Feb 05, 2020 at 10:44:20PM +0100, mwilck@suse.com wrote:
From: Martin Wilck <redacted>
Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"),
it is possible that FC commands are scheduled after the adapter firmware
has been shut down. IO sent to the firmware in this situation hangs
indefinitely. Avoid this for the LOGO code path that is typically taken
when adapters are shut down.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Signed-off-by: Martin Wilck <redacted>
Reviewed-by: Roman Bolshakov <redacted>
Just to understand it correctly: 45235022da99 ("scsi: qla2xxx: Fix
driver unload by shutting down chip") is saying FW is not able to
shutdown propably and therefore we kill it first and still try to do
some cleanup.
Are you sure you got all the necessary places fixed up?
I tend to say we should add
if (!ha->flags.fw_started)
return QLA_FUNCTION_FAILED;
do qla2x00_mailbox_command() and handle the errors one by one. Just an
idea.
Thanks,
Daniel
From: Daniel Wagner <hidden> Date: 2020-02-06 12:42:34
On Wed, Feb 05, 2020 at 10:44:21PM +0100, mwilck@suse.com wrote:
From: Martin Wilck <redacted>
Since 45235022da99, the firmware is shut down early in the controller
shutdown process. This causes commands sent to the firmware (such as LOGO)
to hang forever. Eventually one or more timeouts will be triggered.
Move the stopping of the firmware until after sessions have terminated.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Signed-off-by: Martin Wilck <redacted>
From: Daniel Wagner <hidden> Date: 2020-02-06 12:46:46
On Wed, Feb 05, 2020 at 10:44:22PM +0100, mwilck@suse.com wrote:
From: Martin Wilck <redacted>
The purpose of the UNLOADING flag is to avoid port login procedures
to continue when a controller is in the process of shutting down.
It makes sense to set this flag before starting session teardown.
The only operations that must be able to continue are LOGO, PRLO,
and the like.
Furthermore, use atomic test_and_set_bit() to avoid the shutdown
being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(),
the test for UNLOADING is postponed until after the check for an already
disabled PCI board.
Signed-off-by: Martin Wilck <redacted>
From: Martin Wilck <hidden> Date: 2020-02-06 12:49:08
On Thu, 2020-02-06 at 13:33 +0100, Daniel Wagner wrote:
Hi Martin,
On Wed, Feb 05, 2020 at 10:44:20PM +0100, mwilck@suse.com wrote:
quoted
From: Martin Wilck <redacted>
Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by
shutting down chip"),
it is possible that FC commands are scheduled after the adapter
firmware
has been shut down. IO sent to the firmware in this situation hangs
indefinitely. Avoid this for the LOGO code path that is typically
taken
when adapters are shut down.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting
down chip")
Signed-off-by: Martin Wilck <redacted>
Reviewed-by: Roman Bolshakov <redacted>
Just to understand it correctly: 45235022da99 ("scsi: qla2xxx: Fix
driver unload by shutting down chip") is saying FW is not able to
shutdown propably and therefore we kill it first and still try to do
some cleanup.
Yes, that's what I observed. Mailbox access hangs in this case, as the
mailbox simply won't execute the command and the expected status change
will not happen.
Are you sure you got all the necessary places fixed up?
I tend to say we should add
if (!ha->flags.fw_started)
return QLA_FUNCTION_FAILED;
do qla2x00_mailbox_command() and handle the errors one by one. Just
an
idea.
I had that idea too. I even tried it out. IIRC it's not that easy. Some
commands need to be sent to the mailbox even in shut-down state
(MBC_EXECUTE_FIRMWARE, for example?). I admit I didn't dare going down
the path you're suggesting, I thought it had quite a potential to cause
regressions.
Did you mean this as a negative review, or rather an additional
suggestion?
Thanks
Martin
From: Daniel Wagner <hidden> Date: 2020-02-06 13:03:25
On Thu, Feb 06, 2020 at 01:50:32PM +0100, Martin Wilck wrote:
On Thu, 2020-02-06 at 13:33 +0100, Daniel Wagner wrote:
quoted
Are you sure you got all the necessary places fixed up?
I tend to say we should add
if (!ha->flags.fw_started)
return QLA_FUNCTION_FAILED;
do qla2x00_mailbox_command() and handle the errors one by one. Just
an
idea.
I had that idea too. I even tried it out. IIRC it's not that easy. Some
commands need to be sent to the mailbox even in shut-down state
(MBC_EXECUTE_FIRMWARE, for example?). I admit I didn't dare going down
the path you're suggesting, I thought it had quite a potential to cause
regressions.
Yes, this certainly causing regressions. This patch is using the
blacklist approach which is probably introducing less regression.
But I think for finding all the right spots the white listing approach
is better. Maybe only during development phase. Just an idea.
Did you mean this as a negative review, or rather an additional
suggestion?
This patch clearly improves things. So yes, I approve.
Reviewed-by: Daniel Wagner <redacted>
Thanks,
Daniel
From: Martin Wilck <hidden> Date: 2020-03-09 16:44:08
Hello Martin, Himanshu,
On Wed, 2020-02-05 at 22:44 +0100, mwilck@suse.com wrote:
From: Martin Wilck <redacted>
Hello Martin, Himanshu, all,
here is v2 of the little series I first submitted on Nov 29, 2019.
Changes wrt v2:
- Added patch 3 to set the UNLOADING flag before waiting for
sessions
to end (Roman)
- Use test_and_set_bit() for UNLOADING (Hannes)
Martin Wilck (3):
scsi: qla2xxx: avoid sending mailbox commands if firmware is
stopped
scsi: qla2xxx: don't shut down firmware before closing sessions
scsi: qla2xxx: set UNLOADING before waiting for session deletion
drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
drivers/scsi/qla2xxx/qla_os.c | 39 +++++++++++++++++---------------
--
2 files changed, 22 insertions(+), 20 deletions(-)
what about this series? Will it be merged for 5.7? It got positive
reviews from Daniel, and no negative ones so far, and it still applies
cleanly to 5.7/scsi-queue.
Thanks,
Martin W.
From: Roman Bolshakov <hidden> Date: 2020-03-13 14:55:48
On Wed, Feb 05, 2020 at 10:44:21PM +0100, mwilck@suse.com wrote:
From: Martin Wilck <redacted>
Since 45235022da99, the firmware is shut down early in the controller
shutdown process. This causes commands sent to the firmware (such as LOGO)
to hang forever. Eventually one or more timeouts will be triggered.
Move the stopping of the firmware until after sessions have terminated.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
Hi Martin,
Reviewed-by: Roman Bolshakov <redacted>
BTW, I think qla_nvme_delete() should be moved up to split the function into
two parts:
* session/port cleanup
* chip shutdown
Although invocation of the function after chip shutdown has no functional
implications at the moment.
Best regards,
Roman
From: Roman Bolshakov <hidden> Date: 2020-03-13 15:36:58
On Wed, Feb 05, 2020 at 10:44:22PM +0100, mwilck@suse.com wrote:
From: Martin Wilck <redacted>
The purpose of the UNLOADING flag is to avoid port login procedures
to continue when a controller is in the process of shutting down.
It makes sense to set this flag before starting session teardown.
The only operations that must be able to continue are LOGO, PRLO,
and the like.
Furthermore, use atomic test_and_set_bit() to avoid the shutdown
being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(),
the test for UNLOADING is postponed until after the check for an already
disabled PCI board.
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 25 +++++++++++--------------
1 file changed, 11 insertions(+), 14 deletions(-)
Hi Martin,
Reviewed-by: Roman Bolshakov <redacted>
Thanks,
Roman
On Wed, 5 Feb 2020, 1:44pm, mwilck@suse.com wrote:
quoted hunk
From: Martin Wilck <redacted>
Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip"),
it is possible that FC commands are scheduled after the adapter firmware
has been shut down. IO sent to the firmware in this situation hangs
indefinitely. Avoid this for the LOGO code path that is typically taken
when adapters are shut down.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Signed-off-by: Martin Wilck <redacted>
Reviewed-by: Roman Bolshakov <redacted>
---
drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
drivers/scsi/qla2xxx/qla_os.c | 3 +++
2 files changed, 6 insertions(+)
I'd probably not do it here; rather in qla2x00_get_sp()
/qla2x00_start_sp()/ qla2xxx_get_qpair_sp() time. Not all works are for
posting to firmware (QLA_EVT_IDC_ACK, QLA_EVT_UNMAP etc.).
Regards,
-Arun
On Wed, 5 Feb 2020, 1:44pm, mwilck@suse.com wrote:
quoted hunk
From: Martin Wilck <redacted>
Since 45235022da99, the firmware is shut down early in the controller
shutdown process. This causes commands sent to the firmware (such as LOGO)
to hang forever. Eventually one or more timeouts will be triggered.
Move the stopping of the firmware until after sessions have terminated.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting down chip")
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
NAK.
The fcport deletion was done after chip reset to minimize interference and
further action on fcports. We should not be sending out logouts during
unload (driver just implicitly logs out). If you experience any hangs,
please let us know.
Regards,
-Arun
On Wed, 5 Feb 2020, 1:44pm, mwilck@suse.com wrote:
quoted hunk
From: Martin Wilck <redacted>
The purpose of the UNLOADING flag is to avoid port login procedures
to continue when a controller is in the process of shutting down.
It makes sense to set this flag before starting session teardown.
The only operations that must be able to continue are LOGO, PRLO,
and the like.
Furthermore, use atomic test_and_set_bit() to avoid the shutdown
being run multiple times in parallel. In qla2x00_disable_board_on_pci_error(),
the test for UNLOADING is postponed until after the check for an already
disabled PCI board.
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 25 +++++++++++--------------
1 file changed, 11 insertions(+), 14 deletions(-)
From: Martin Wilck <hidden> Date: 2020-03-25 15:34:20
On Tue, 2020-03-24 at 17:36 -0700, Arun Easi wrote:
On Wed, 5 Feb 2020, 1:44pm, mwilck@suse.com wrote:
quoted
From: Martin Wilck <redacted>
Since 45235022da99, the firmware is shut down early in the
controller
shutdown process. This causes commands sent to the firmware (such
as LOGO)
to hang forever. Eventually one or more timeouts will be triggered.
Move the stopping of the firmware until after sessions have
terminated.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting
down chip")
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
...
NAK.
The fcport deletion was done after chip reset to minimize
interference and
further action on fcports. We should not be sending out logouts
during
unload (driver just implicitly logs out). If you experience any
hangs,
please let us know.
What about target mode? AFAIK target ports need to send explicit LOGO.
Regards
Martin
From: Martin Wilck <hidden> Date: 2020-03-25 16:28:45
On Tue, 2020-03-24 at 16:51 -0700, Arun Easi wrote:
On Wed, 5 Feb 2020, 1:44pm, mwilck@suse.com wrote:
quoted
From: Martin Wilck <redacted>
Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by
shutting down chip"),
it is possible that FC commands are scheduled after the adapter
firmware
has been shut down. IO sent to the firmware in this situation hangs
indefinitely. Avoid this for the LOGO code path that is typically
taken
when adapters are shut down.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting
down chip")
Signed-off-by: Martin Wilck <redacted>
Reviewed-by: Roman Bolshakov <redacted>
---
drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
drivers/scsi/qla2xxx/qla_os.c | 3 +++
2 files changed, 6 insertions(+)
struct qla_work_evt *e)
unsigned long flags;
bool q = false;
+ if (!vha->hw->flags.fw_started)
+ return QLA_FUNCTION_FAILED;
+
I'd probably not do it here; rather in qla2x00_get_sp()
/qla2x00_start_sp()/ qla2xxx_get_qpair_sp() time. Not all works are
for
posting to firmware (QLA_EVT_IDC_ACK, QLA_EVT_UNMAP etc.).
qla81xx_idc_ack() calls qla2x00_mailbox_command(), which should be
avoided IMO. But I'll review the various cases and re-post the patch.
Thanks,
Martin
From: Martin Wilck <hidden> Date: 2020-03-25 17:20:02
On Wed, 2020-03-25 at 17:28 +0100, Martin Wilck wrote:
On Tue, 2020-03-24 at 16:51 -0700, Arun Easi wrote:
quoted
On Wed, 5 Feb 2020, 1:44pm, mwilck@suse.com wrote:
quoted
From: Martin Wilck <redacted>
Since commit 45235022da99 ("scsi: qla2xxx: Fix driver unload by
shutting down chip"),
it is possible that FC commands are scheduled after the adapter
firmware
has been shut down. IO sent to the firmware in this situation
hangs
indefinitely. Avoid this for the LOGO code path that is typically
taken
when adapters are shut down.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by
shutting
down chip")
Signed-off-by: Martin Wilck <redacted>
Reviewed-by: Roman Bolshakov <redacted>
---
drivers/scsi/qla2xxx/qla_mbx.c | 3 +++
drivers/scsi/qla2xxx/qla_os.c | 3 +++
2 files changed, 6 insertions(+)
[...]
*vha,
struct qla_work_evt *e)
unsigned long flags;
bool q = false;
+ if (!vha->hw->flags.fw_started)
+ return QLA_FUNCTION_FAILED;
+
I'd probably not do it here; rather in qla2x00_get_sp()
/qla2x00_start_sp()/ qla2xxx_get_qpair_sp() time. Not all works are
for
posting to firmware (QLA_EVT_IDC_ACK, QLA_EVT_UNMAP etc.).
qla81xx_idc_ack() calls qla2x00_mailbox_command(), which should be
avoided IMO. But I'll review the various cases and re-post the patch.
Thinking about it once more, the approach is racy.
qla2x00_try_stop_firmware() can be called any time, and it sets
fw_started = 0 *after* actually stopping the firmware. Even if we check
fw_started, the firwmare might be stopped between our check and our
actual mailbox / FW register access, and we'd end up hanging.
At least the check should be as close as possible to the actual FW
access, e.g. in qla2x00_mailbox_command(), or before writing to the
request queue registers in qla2x00_start_sp() etc.
Perhaps the (!fw_started) condition should be treated like
ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if
is_rom_cmd() returns true?
I can re-post, but I feel this should really be done by someone who
knows exactly how the firmware operates, IOW Marvell staff.
Regards
Martin
From: Martin Wilck <hidden> Date: 2020-03-25 18:21:01
On Wed, 2020-03-25 at 18:20 +0100, Martin Wilck wrote:
Perhaps the (!fw_started) condition should be treated like
ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if
is_rom_cmd() returns true?
No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in
rom_cmds[], but has been found to hang (this is why I had the hunk in
qla24xx_fabric_logout()). The list of mailbox commands
that must be passed on when the FW is stopped has to be shorter than
rom_cmds[].
Better suggestions welcome.
Martin
On Tue, 2020-03-24 at 17:36 -0700, Arun Easi wrote:
quoted
On Wed, 5 Feb 2020, 1:44pm, mwilck@suse.com wrote:
quoted
From: Martin Wilck <redacted>
Since 45235022da99, the firmware is shut down early in the
controller
shutdown process. This causes commands sent to the firmware (such
as LOGO)
to hang forever. Eventually one or more timeouts will be triggered.
Move the stopping of the firmware until after sessions have
terminated.
Fixes: 45235022da99 ("scsi: qla2xxx: Fix driver unload by shutting
down chip")
Signed-off-by: Martin Wilck <redacted>
---
drivers/scsi/qla2xxx/qla_os.c | 21 ++++++++++-----------
1 file changed, 10 insertions(+), 11 deletions(-)
...
quoted
NAK.
The fcport deletion was done after chip reset to minimize interference
and further action on fcports. We should not be sending out logouts
during unload (driver just implicitly logs out). If you experience any
hangs, please let us know.
What about target mode? AFAIK target ports need to send explicit LOGO.
I do not see a hard requirement cited in spec, correct me if you see
otherwise. Other initiators should see a RSCN and see this device has
gone away.
Regards,
-Arun
On Wed, 2020-03-25 at 18:20 +0100, Martin Wilck wrote:
quoted
Perhaps the (!fw_started) condition should be treated like
ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if
is_rom_cmd() returns true?
No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in
rom_cmds[], but has been found to hang (this is why I had the hunk in
qla24xx_fabric_logout()). The list of mailbox commands
that must be passed on when the FW is stopped has to be shorter than
rom_cmds[].
With your patch 2/3, where UNLOADING is set prior to the reset of chip, in
place this issue should be largely addressed. Basically, paths that send
out request to wire check UNLOADING bit (if any path is missing that,
should add the check) before sending it out.
Now with UNLOADING set (with your patch 2/3), chip reset, and all
outstanding command's completion called (qla2x00_abort_isp_cleanup) I see
less chance of anything being sent out. If you see any issue with your
patches 1 & 2 (addressing my comments) applied, let me know and we can
tackle then. How about that?
Regards,
-Arun
From: Martin Wilck <hidden> Date: 2020-03-25 22:41:36
On Wed, 2020-03-25 at 15:13 -0700, Arun Easi wrote:
On Wed, 25 Mar 2020, 11:21am, Martin Wilck wrote:
quoted
On Wed, 2020-03-25 at 18:20 +0100, Martin Wilck wrote:
quoted
Perhaps the (!fw_started) condition should be treated like
ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if
is_rom_cmd() returns true?
No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in
rom_cmds[], but has been found to hang (this is why I had the hunk
in
qla24xx_fabric_logout()). The list of mailbox commands
that must be passed on when the FW is stopped has to be shorter
than
rom_cmds[].
With your patch 2/3, where UNLOADING is set prior to the reset of
chip, in
place this issue should be largely addressed. Basically, paths that
send
out request to wire check UNLOADING bit (if any path is missing
that,
should add the check) before sending it out.
Now with UNLOADING set (with your patch 2/3), chip reset, and all
outstanding command's completion called (qla2x00_abort_isp_cleanup) I
see
less chance of anything being sent out. If you see any issue with
your
patches 1 & 2 (addressing my comments) applied, let me know and we
can
tackle then. How about that?
It sounds like a plan. Although it means that I just wasted time trying
to figure out which mailbox commands need to be processed even if the
firmware is down :-)
Thanks,
Martin
----------------------------------------------------------------------
On Wed, 2020-03-25 at 15:13 -0700, Arun Easi wrote:
quoted
On Wed, 25 Mar 2020, 11:21am, Martin Wilck wrote:
quoted
On Wed, 2020-03-25 at 18:20 +0100, Martin Wilck wrote:
quoted
Perhaps the (!fw_started) condition should be treated like
ABORT_ISP_ACTIVE in qla2x00_mailbox_command, i.e. execute only if
is_rom_cmd() returns true?
No, this won't be sufficient, as e.g. MBC_IOCB_COMMAND_A64 is in
rom_cmds[], but has been found to hang (this is why I had the hunk
in
qla24xx_fabric_logout()). The list of mailbox commands
that must be passed on when the FW is stopped has to be shorter
than
rom_cmds[].
With your patch 2/3, where UNLOADING is set prior to the reset of
chip, in place this issue should be largely addressed. Basically,
paths that send out request to wire check UNLOADING bit (if any path
is missing that, should add the check) before sending it out.
Now with UNLOADING set (with your patch 2/3), chip reset, and all
outstanding command's completion called (qla2x00_abort_isp_cleanup) I
see less chance of anything being sent out. If you see any issue with
your patches 1 & 2 (addressing my comments) applied, let me know and
we can tackle then. How about that?
It sounds like a plan. Although it means that I just wasted time trying
to figure out which mailbox commands need to be processed even if the
firmware is down :-)
I hope it was not too much time. On the plus side, you know the mailbox
path very well now. :)
Regards,
-Arun