Applied the patch and tried with 10.2.4.70.54 firmware and it still crashes:

[  142.438377] ath10k_pci 0000:01:00.0: firmware crashed! (uuid 
a5499582-e220-46d2-9359-0b44219f69ea)
[  142.447512] ath10k_pci 0000:01:00.0: qca988x hw2.0 target 0x4100016c 
chip_id 0x043202ff sub 0000:0000
[  142.456879] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1 
tracing 0 dfs 1 testmode 1
[  142.469916] ath10k_pci 0000:01:00.0: firmware ver 10.2.4.70.54 api 5 
features no-p2p,raw-mode,mfp crc32 9d340dd9
[  142.480295] ath10k_pci 0000:01:00.0: board_file api 1 bmi_id N/A 
crc32 bebc7c08
[  142.487717] ath10k_pci 0000:01:00.0: htt-ver 2.1 wmi-op 5 htt-op 2 
cal file max-sta 128 raw 0 hwcrypto 1
[  142.499361] ath10k_pci 0000:01:00.0: firmware register dump:
[  142.505124] ath10k_pci 0000:01:00.0: [00]: 0x4100016C 0x000015B3 
0x009A4577 0x00955B31
[  142.513157] ath10k_pci 0000:01:00.0: [04]: 0x009A4577 0x00060130 
0x00000002 0x00439E98
[  142.521203] ath10k_pci 0000:01:00.0: [08]: 0x0044110C 0x00442074 
0x00407120 0x004436CC
[  142.529246] ath10k_pci 0000:01:00.0: [12]: 0x00000009 0x00000000 
0x009A3518 0x009A3526
[  142.537285] ath10k_pci 0000:01:00.0: [16]: 0x00958080 0x009A3EA6 
0x00000000 0x00000000
[  142.545324] ath10k_pci 0000:01:00.0: [20]: 0x409A4577 0x0040AAC4 
0x0040AC60 0x0040AC09
[  142.553356] ath10k_pci 0000:01:00.0: [24]: 0x809A44BA 0x0040AB24 
0x00400000 0xC09A4577
[  142.561400] ath10k_pci 0000:01:00.0: [28]: 0x809A39DE 0x0040AB84 
0x0044110C 0x00442074
[  142.569444] ath10k_pci 0000:01:00.0: [32]: 0x809A5FE2 0x0040ABB4 
0x0044110C 0x00407120
[  142.577483] ath10k_pci 0000:01:00.0: [36]: 0x809A2E6C 0x0040ABF4 
0x0040AC14 0x00001580
[  142.585522] ath10k_pci 0000:01:00.0: [40]: 0x80990F6F 0x0040AD04 
0x009C643C 0x004436CC
[  142.593554] ath10k_pci 0000:01:00.0: [44]: 0x80998510 0x0040AD64 
0x004208FC 0x00439E4C
[  142.601600] ath10k_pci 0000:01:00.0: [48]: 0x8099AE95 0x0040AD84 
0x004208FC 0x0042638C
[  142.609642] ath10k_pci 0000:01:00.0: [52]: 0x809BFC55 0x0040AEE4 
0x00424FE8 0x00000002
[  142.617681] ath10k_pci 0000:01:00.0: [56]: 0x80940F18 0x0040AF14 
0x00000004 0x004039D0
[  142.727220] ieee80211 phy0: Hardware restart was requested
[  142.732850] ath10k_pci 0000:01:00.0: failed to synchronize monitor 
vdev 1 stop: -143
[  142.740739] ath10k_pci 0000:01:00.0: failed to stop monitor vdev: -143


~Benjamin

On 12/03/2016 04:46 AM, Mohammed Shafi Shajakhan wrote:

https://patchwork.kernel.org/patch/9437519/
(sorry missed this in the previous thread)

On Sat, Dec 03, 2016 at 06:13:58PM +0530, Mohammed Shafi Shajakhan wrote:

quoted

Hi Benjamin,

On Fri, Dec 02, 2016 at 05:28:02PM -0800, Benjamin Morgan wrote:

quoted

Just tried 10.2.4.70.58 firmware that you linked to and it still crashes:

[  131.568989] ath10k_pci 0000:01:00.0: firmware crashed! (uuid
1838347e-9380-4a26-ac9d-2963ee95968b)
[  131.578124] ath10k_pci 0000:01:00.0: qca988x hw2.0 target
0x4100016c chip_id 0x043202ff sub 0000:0000
[  131.587491] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1
tracing 0 dfs 1 testmode 1
[  131.600521] ath10k_pci 0000:01:00.0: firmware ver 10.2.4.70.58
api 5 features no-p2p,raw-mode,mfp crc32 e1af076f
[  131.610899] ath10k_pci 0000:01:00.0: board_file api 1 bmi_id N/A
crc32 bebc7c08
[  131.618325] ath10k_pci 0000:01:00.0: htt-ver 2.1 wmi-op 5 htt-op
2 cal file max-sta 128 raw 0 hwcrypto 1
[  131.629965] ath10k_pci 0000:01:00.0: firmware register dump:
[  131.635728] ath10k_pci 0000:01:00.0: [00]: 0x4100016C 0x000015B3
0x009A45AF 0x00955B31
[  131.643761] ath10k_pci 0000:01:00.0: [04]: 0x009A45AF 0x00060130
0x00000002 0x00439E98
[  131.651806] ath10k_pci 0000:01:00.0: [08]: 0x0044110C 0x00442074
0x00407120 0x004436CC
[  131.659852] ath10k_pci 0000:01:00.0: [12]: 0x00000009 0x00000000
0x009A3550 0x009A355E
[  131.667892] ath10k_pci 0000:01:00.0: [16]: 0x00958080 0x009A31D6
0x00000000 0x00000000
[  131.675936] ath10k_pci 0000:01:00.0: [20]: 0x409A45AF 0x0040AAC4
0x0040AC60 0x0040AC09
[  131.683968] ath10k_pci 0000:01:00.0: [24]: 0x809A44F2 0x0040AB24
0x00400000 0xC09A45AF
[  131.692013] ath10k_pci 0000:01:00.0: [28]: 0x809A3A16 0x0040AB84
0x0044110C 0x00442074
[  131.700056] ath10k_pci 0000:01:00.0: [32]: 0x809A601A 0x0040ABB4
0x0044110C 0x00407120
[  131.708100] ath10k_pci 0000:01:00.0: [36]: 0x809A2EA4 0x0040ABF4
0x0040AC14 0x00001580
[  131.716143] ath10k_pci 0000:01:00.0: [40]: 0x80990F63 0x0040AD04
0x009C6458 0x004436CC
[  131.724175] ath10k_pci 0000:01:00.0: [44]: 0x80998520 0x0040AD64
0x004208FC 0x00439E4C
[  131.732220] ath10k_pci 0000:01:00.0: [48]: 0x8099AEA5 0x0040AD84
0x004208FC 0x00425874
[  131.740263] ath10k_pci 0000:01:00.0: [52]: 0x809BFC39 0x0040AEE4
0x00424FE8 0x00000002
[  131.748306] ath10k_pci 0000:01:00.0: [56]: 0x80940F18 0x0040AF14
0x00000004 0x004039D0
[  131.857076] ieee80211 phy0: Hardware restart was requested
[  131.862705] ath10k_pci 0000:01:00.0: failed to synchronize
monitor vdev 1 stop: -143
[  131.870594] ath10k_pci 0000:01:00.0: failed to stop monitor vdev: -143

[shafi] request to try with the latest backports and ath.git
Also worth trying this fix in ath10k (once the next firmware version)
is available. I will keep you posted. thank you !

quoted

~Benjamin

On 11/30/2016 11:09 PM, Mohammed Shafi Shajakhan wrote:

quoted

Hi,

On Tue, Nov 29, 2016 at 11:22:12AM -0800, Benjamin Morgan wrote:

quoted

When we try to transmit traffic (ping) between two meshed ath10k
devices running latest lede we keep experiencing ath10k firmware
crashes. This seems to only happen when running in 802.11n/ac mode
but not in 802.11a/g mode. Also, from the station dumps it appears
that management traffic is flowing between the devices, however when
we try to send unicast data traffic the firmware crashes
immediately.

[shafi] Did you get a chance to try with the below firmware as well
https://github.com/kvalo/ath10k-firmware/blob/master/QCA988X/hw2.0/10.2.4.70/firmware-5.bin_10.2.4.70.58

quoted

Platform: Archer C7 AC1750 v2
Software Image: LEDE (HEAD, r2299) Commit: https://github.com/lede-project/source/commit/d596c21ebd5a3e6ce933eff3e51989031e4b1d58

Crypto: wpa_supplicant
wpa_supplicant-wlan0.conf
network={
ssid="bmorgan_lede_mesh"
key_mgmt=SAE
mode=5
frequency=5180
psk="meshpassword"
}

Backports Verstion:
[    9.818007] Loading modules backported from Linux version
wt-2016-10-03-1-g6fcb1a6
[    9.825736] Backport generated by backports.git
backports-20160324-9-g0e38f5c

​​Ath10k Initialization on Station A (dmesg)
[    9.896715] PCI: Enabling device 0000:01:00.0 (0000 -> 0002)
[    9.902622] ath10k_pci 0000:01:00.0: pci irq legacy oper_irq_mode
1 irq_mode 0 reset_mode 0
[   10.123734] ath10k_pci 0000:01:00.0: Direct firmware load for
ath10k/pre-cal-pci-0000:01:00.0.bin failed with error -2
[   10.134620] ath10k_pci 0000:01:00.0: Falling back to user helper
[   10.287680] firmware ath10k!pre-cal-pci-0000:01:00.0.bin:
firmware_loading_store: map pages failed
[   10.622789] ath10k_pci 0000:01:00.0: qca988x hw2.0 target
0x4100016c chip_id 0x043202ff sub 0000:0000
[   10.632184] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1
tracing 0 dfs 1 testmode 1
[   10.645231] ath10k_pci 0000:01:00.0: firmware ver 10.2.4.70.54
api 5 features no-p2p,raw-mode,mfp crc32 9d340dd9
[   10.655660] ath10k_pci 0000:01:00.0: Direct firmware load for
ath10k/QCA988X/hw2.0/board-2.bin failed with error -2
[   10.666264] ath10k_pci 0000:01:00.0: Falling back to user helper
[   10.747925] firmware ath10k!QCA988X!hw2.0!board-2.bin:
firmware_loading_store: map pages failed
[   11.011123] ath10k_pci 0000:01:00.0: board_file api 1 bmi_id N/A
crc32 bebc7c08
[   12.155224] ath10k_pci 0000:01:00.0: htt-ver 2.1 wmi-op 5 htt-op
2 cal file max-sta 128 raw 0 hwcrypto 1

Station A (wlan0):
18:A6:F7:23:6E:66
10.230.5.41

Station B (wlan0):
18:a6:f7:26:0f:21
10.230.5.42

Station Dump on Station A before ping:
Station 18:a6:f7:26:0f:21 (on wlan0)
inactive time:340 ms
rx bytes:2472
rx packets:28
tx bytes:1204
tx packets:9
tx retries:0
tx failed:0
rx drop misc:1
signal: -14 dBm
signal avg:-14 dBm
Toffset:18142530 us
tx bitrate:6.0 MBit/s
rx bitrate:6.0 MBit/s
rx duration:1524 us
mesh llid:0
mesh plid:0
mesh plink:ESTAB
mesh local PS mode:ACTIVE
mesh peer PS mode:UNKNOWN
mesh non-peer PS mode:ACTIVE
authorized:yes
authenticated:yes
associated:yes
preamble:long
WMM/WME:yes
MFP:yes
TDLS peer:no
DTIM period:2
beacon interval:1000
connected time:10 seconds

​Crash Log on Station B (10.230.5.42)
[245.483888] ath10k_pci 0000:01:00.0: firmware crashed! (uuid
2bab5ee9-08ff-4a17-95b1-636d212acebc)
[245.493020] ath10k_pci 0000:01:00.0: qca988x hw2.0 target
0x4100016c chip_id 0x043202ff sub 0000:0000
[245.502384] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1
tracing 0 dfs 1 testmode 1
[245.515436] ath10k_pci 0000:01:00.0: firmware ver 10.2.4.70.54 api
5 features no-p2p,raw-mode,mfp crc32 9d340dd9
[245.525812] ath10k_pci 0000:01:00.0: board_file api 1 bmi_id N/A
crc32 bebc7c08
[245.533232] ath10k_pci 0000:01:00.0: htt-ver 2.1 wmi-op 5 htt-op 2
cal file max-sta 128 raw 0 hwcrypto 1
[245.544876] ath10k_pci 0000:01:00.0: firmware register dump:
[245.550633] ath10k_pci 0000:01:00.0: [00]: 0x4100016C 0x000015B3
0x009A4577 0x00955B31
[245.558676] ath10k_pci 0000:01:00.0: [04]: 0x009A4577 0x00060130
0x00000002 0x00439E98
[245.566715] ath10k_pci 0000:01:00.0: [08]: 0x0044110C 0x00442074
0x00407120 0x004436CC
[245.574749] ath10k_pci 0000:01:00.0: [12]: 0x00000009 0x00000000
0x009A3518 0x009A3526
[245.582793] ath10k_pci 0000:01:00.0: [16]: 0x00958080 0x0094085D
0x00000000 0x00000000
[245.590836] ath10k_pci 0000:01:00.0: [20]: 0x409A4577 0x0040AAC4
0x0040AC60 0x0040AC09
[245.598882] ath10k_pci 0000:01:00.0: [24]: 0x809A44BA 0x0040AB24
0x00400000 0xC09A4577
[245.606923] ath10k_pci 0000:01:00.0: [28]: 0x809A39DE 0x0040AB84
0x0044110C 0x00442074
[245.614955] ath10k_pci 0000:01:00.0: [32]: 0x809A5FE2 0x0040ABB4
0x0044110C 0x00407120
[245.623000] ath10k_pci 0000:01:00.0: [36]: 0x809A2E6C 0x0040ABF4
0x0040AC14 0x00001580
[245.631043] ath10k_pci 0000:01:00.0: [40]: 0x80990F6F 0x0040AD04
0x009C643C 0x004436CC
[245.639086] ath10k_pci 0000:01:00.0: [44]: 0x80998510 0x0040AD64
0x004208FC 0x00439E4C
[245.647129] ath10k_pci 0000:01:00.0: [48]: 0x8099AE95 0x0040AD84
0x004208FC 0x00425404
[245.655170] ath10k_pci 0000:01:00.0: [52]: 0x809BFC55 0x0040AEE4
0x00424FE8 0x00000002
[245.663198] ath10k_pci 0000:01:00.0: [56]: 0x80940F18 0x0040AF14
0x00000004 0x004039D0
[245.767023] ieee80211 phy0: Hardware restart was requested
[245.772655] ath10k_pci 0000:01:00.0: failed to synchronize monitor
vdev 1 stop: -143
[245.780542] ath10k_pci 0000:01:00.0: failed to stop monitor vdev: -143
​
With wpa_supplicant turned off:

Station Dump on Station A before ping:
Station 18:a6:f7:26:0f:21 (on wlan0)
inactive time:60 ms
rx bytes:3874
rx packets:67
tx bytes:707
tx packets:7
tx retries:0
tx failed:1
rx drop misc:3
signal: -14 dBm
signal avg:-14 dBm
Toffset:18446744073684250999 us
tx bitrate:6.0 MBit/s
rx bitrate:6.0 MBit/s
rx duration:1312 us
mesh llid:24953
mesh plid:59412
mesh plink:ESTAB
mesh local PS mode:ACTIVE
mesh peer PS mode:UNKNOWN
mesh non-peer PS mode:ACTIVE
authorized:yes
authenticated:yes
associated:yes
preamble:long
WMM/WME:yes
MFP:no
TDLS peer:no
DTIM period:2
beacon interval:1000
short slot time:yes
connected time:33 seconds

Firmware crash on Station A (10.230.5.41)
[ 1040.984599] ath10k_pci 0000:01:00.0: firmware crashed! (uuid
42d4f6dc-0e89-4505-aefb-58cbb70d0dce)
[ 1040.993732] ath10k_pci 0000:01:00.0: qca988x hw2.0 target
0x4100016c chip_id 0x043202ff sub 0000:0000
[ 1041.003100] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1
tracing 0 dfs 1 testmode 1
[ 1041.016144] ath10k_pci 0000:01:00.0: firmware ver 10.2.4.70.54
api 5 features no-p2p,raw-mode,mfp crc32 9d340dd9
[ 1041.026523] ath10k_pci 0000:01:00.0: board_file api 1 bmi_id N/A
crc32 bebc7c08
[ 1041.033940] ath10k_pci 0000:01:00.0: htt-ver 2.1 wmi-op 5 htt-op
2 cal file max-sta 128 raw 0 hwcrypto 1
[ 1041.045593] ath10k_pci 0000:01:00.0: firmware register dump:
[ 1041.051336] ath10k_pci 0000:01:00.0: [00]: 0x4100016C 0x000015B3
0x009A4577 0x00955B31
[ 1041.059383] ath10k_pci 0000:01:00.0: [04]: 0x009A4577 0x00060130
0x00000001 0x00435354
[ 1041.067424] ath10k_pci 0000:01:00.0: [08]: 0x0044110C 0x00442074
0x00407120 0x004436CC
[ 1041.075463] ath10k_pci 0000:01:00.0: [12]: 0x00000009 0x00000000
0x009A43A0 0x009A43DE
[ 1041.083490] ath10k_pci 0000:01:00.0: [16]: 0x00958080 0x0094085D
0x00000000 0x00000000
[ 1041.091535] ath10k_pci 0000:01:00.0: [20]: 0x409A4577 0x0040AAC4
0x0040AC60 0x0040AC09
[ 1041.099579] ath10k_pci 0000:01:00.0: [24]: 0x809A44BA 0x0040AB24
0x00955A00 0xC09A4577
[ 1041.107622] ath10k_pci 0000:01:00.0: [28]: 0x809A39DE 0x0040AB84
0x0044110C 0x00442074
[ 1041.115661] ath10k_pci 0000:01:00.0: [32]: 0x809A5FE2 0x0040ABB4
0x0044110C 0x00407120
[ 1041.123689] ath10k_pci 0000:01:00.0: [36]: 0x809A2E6C 0x0040ABF4
0x0040AC10 0x00001580
[ 1041.131733] ath10k_pci 0000:01:00.0: [40]: 0x80990F6F 0x0040AD04
0x009C643C 0x004436CC
[ 1041.139777] ath10k_pci 0000:01:00.0: [44]: 0x80998510 0x0040AD64
0x004208FC 0x00439E4C
[ 1041.147820] ath10k_pci 0000:01:00.0: [48]: 0x8099AE95 0x0040AD84
0x004208FC 0x00425758
[ 1041.155860] ath10k_pci 0000:01:00.0: [52]: 0x809BFC55 0x0040AEE4
0x00424FE8 0x00000002
[ 1041.163888] ath10k_pci 0000:01:00.0: [56]: 0x80940F18 0x0040AF14
0x00000004 0x004039D0
[ 1041.267025] ieee80211 phy0: Hardware restart was requested
[ 1041.272656] ath10k_pci 0000:01:00.0: failed to synchronize
monitor vdev 1 stop: -143
[ 1041.280545] ath10k_pci 0000:01:00.0: failed to stop monitor vdev: -143

​
Thanks in advance for your time and help,

~Benjamin