From: Saeed Mahameed <saeedm@nvidia.com>
Hi Dave, Jakub,
This series introduces some fixes to mlx5 driver.
Please pull and let me know if there is any problem.
Thanks,
Saeed.
---
The following changes since commit da5ac772cfe2a03058b0accfac03fad60c46c24d:
r8169: Avoid memcpy() over-reading of ETH_SS_STATS (2021-06-16 13:02:07 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/mlx5-fixes-2021-06-16
for you to fetch changes up to 0232fc2ddcf4ffe01069fd1aa07922652120f44a:
net/mlx5: Reset mkey index on creation (2021-06-16 15:36:49 -0700)
----------------------------------------------------------------
mlx5-fixes-2021-06-16
----------------------------------------------------------------
Alex Vesker (1):
net/mlx5: DR, Fix STEv1 incorrect L3 decapsulation padding
Aya Levin (1):
net/mlx5: Reset mkey index on creation
Dmytro Linkin (1):
net/mlx5e: Don't create devices during unload flow
Leon Romanovsky (2):
net/mlx5: Fix error path for set HCA defaults
net/mlx5: Check that driver was probed prior attaching the device
Parav Pandit (3):
net/mlx5: E-Switch, Read PF mac address
net/mlx5: E-Switch, Allow setting GUID for host PF vport
net/mlx5: SF_DEV, remove SF device on invalid state
drivers/net/ethernet/mellanox/mlx5/core/dev.c | 19 ++++++++++++++++
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 6 +++++
drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 ++-
drivers/net/ethernet/mellanox/mlx5/core/mr.c | 2 +-
.../net/ethernet/mellanox/mlx5/core/sf/dev/dev.c | 1 +
.../mellanox/mlx5/core/steering/dr_ste_v1.c | 26 +++++++++++++---------
drivers/net/ethernet/mellanox/mlx5/core/vport.c | 2 --
include/linux/mlx5/driver.h | 4 ++++
8 files changed, 49 insertions(+), 14 deletions(-)
@@ -1161,7 +1161,7 @@ static int mlx5_load(struct mlx5_core_dev *dev)err=mlx5_core_set_hca_defaults(dev);if(err){mlx5_core_err(dev,"Failed to set hca defaults\n");-gotoerr_sriov;+gotoerr_set_hca;}mlx5_vhca_event_start(dev);
@@ -1194,6 +1194,7 @@ static int mlx5_load(struct mlx5_core_dev *dev)mlx5_sf_hw_table_destroy(dev);err_vhca:mlx5_vhca_event_stop(dev);+err_set_hca:mlx5_cleanup_fs(dev);err_fs:mlx5_accel_tls_cleanup(dev);
@@ -320,6 +320,16 @@ int mlx5_attach_device(struct mlx5_core_dev *dev)}}else{adev=&priv->adev[i]->adev;++/* Pay attention that this is not PCI driver that+*mlx5_core_devisconnected,butauxiliarydriver.+*+*Herewecanraceofmoduleunloadwithdevlink+*reload,butwedon'tneedtotakeextralockbecause+*weareholdingglobalmlx5_intf_mutex.+*/+if(!adev->dev.driver)+continue;adrv=to_auxiliary_drv(adev->dev.driver);if(adrv->resume)
@@ -350,6 +360,10 @@ void mlx5_detach_device(struct mlx5_core_dev *dev)continue;adev=&priv->adev[i]->adev;+/* Auxiliary driver was unbind manually through sysfs */+if(!adev->dev.driver)+gotoskip_suspend;+adrv=to_auxiliary_drv(adev->dev.driver);if(adrv->suspend){
From: Parav Pandit <redacted>
External controller PF's MAC address is not read from the device during
vport setup. Fail to read this results in showing all zeros to user
while the factory programmed MAC is a valid value.
$ devlink port show eth1 -jp
{
"port": {
"pci/0000:03:00.0/196608": {
"type": "eth",
"netdev": "eth1",
"flavour": "pcipf",
"controller": 1,
"pfnum": 0,
"splittable": false,
"function": {
"hw_addr": "00:00:00:00:00:00"
}
}
}
}
Hence, read it when enabling a vport.
After the fix,
$ devlink port show eth1 -jp
{
"port": {
"pci/0000:03:00.0/196608": {
"type": "eth",
"netdev": "eth1",
"flavour": "pcipf",
"controller": 1,
"pfnum": 0,
"splittable": false,
"function": {
"hw_addr": "98:03:9b:a0:60:11"
}
}
}
}
Fixes: f099fde16db3 ("net/mlx5: E-switch, Support querying port function mac address")
Signed-off-by: Bodong Wang <redacted>
Signed-off-by: Parav Pandit <redacted>
Reviewed-by: Alaa Hleihel <redacted>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 6 ++++++
1 file changed, 6 insertions(+)
From: Parav Pandit <redacted>
E-switch should be able to set the GUID of host PF vport.
Currently it returns an error. This results in below error
when user attempts to configure MAC address of the PF of an
external controller.
$ devlink port function set pci/0000:03:00.0/196608 \
hw_addr 00:00:00:11:22:33
mlx5_core 0000:03:00.0: mlx5_esw_set_vport_mac_locked:1876:(pid 6715):\
"Failed to set vport 0 node guid, err = -22.
RDMA_CM will not function properly for this VF."
Check for zero vport is no longer needed.
Fixes: 330077d14de1 ("net/mlx5: E-switch, Supporting setting devlink port function mac address")
Signed-off-by: Yuval Avnery <redacted>
Signed-off-by: Parav Pandit <redacted>
Reviewed-by: Bodong Wang <redacted>
Reviewed-by: Alaa Hleihel <redacted>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/vport.c | 2 --
1 file changed, 2 deletions(-)
From: Alex Vesker <redacted>
Decapsulation L3 on small inner packets which are less than
64 Bytes was done incorrectly. In small packets there is an
extra padding added in L2 which should not be included in L3
length. The issue was that after decapL3 the extra L2 padding
caused an update on the L3 length.
To avoid this issue the new header is pushed to the beginning
of the packet (offset 0) which should not cause a HW reparse
and update the L3 length.
Fixes: c349b4137cfd ("net/mlx5: DR, Add STEv1 modify header logic")
Reviewed-by: Erez Shitrit <redacted>
Reviewed-by: Yevgeny Kliteynik <redacted>
Signed-off-by: Alex Vesker <redacted>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
.../mellanox/mlx5/core/steering/dr_ste_v1.c | 26 ++++++++++++-------
1 file changed, 16 insertions(+), 10 deletions(-)
@@ -694,7 +694,11 @@ static int dr_ste_v1_set_action_decap_l3_list(void *data,if(hw_action_sz/DR_STE_ACTION_DOUBLE_SZ<DR_STE_DECAP_L3_ACTION_NUM)return-EINVAL;-memcpy(padded_data,data,data_sz);+inline_data_sz=+MLX5_FLD_SZ_BYTES(ste_double_action_insert_with_inline_v1,inline_data);++/* Add an alignment padding */+memcpy(padded_data+data_sz%inline_data_sz,data,data_sz);/* Remove L2L3 outer headers */MLX5_SET(ste_single_action_remove_header_v1,hw_action,action_id,
@@ -706,32 +710,34 @@ static int dr_ste_v1_set_action_decap_l3_list(void *data,hw_action+=DR_STE_ACTION_DOUBLE_SZ;used_actions++;/* Remove and NOP are a single double action */-inline_data_sz=-MLX5_FLD_SZ_BYTES(ste_double_action_insert_with_inline_v1,inline_data);+/* Point to the last dword of the header */+data_ptr+=(data_sz/inline_data_sz)*inline_data_sz;-/* Add the new header inline + 2 extra bytes */+/* Add the new header using inline action 4Byte at a time, the header+*isaddedinreversedordertothebeginningofthepackettoavoid+*incorrectparsingbytheHW.Sinceheaderis14Bor18Banextra+*twobytesarepaddedandlaterremoved.+*/for(i=0;i<data_sz/inline_data_sz+1;i++){void*addr_inline;MLX5_SET(ste_double_action_insert_with_inline_v1,hw_action,action_id,DR_STE_V1_ACTION_ID_INSERT_INLINE);/* The hardware expects here offset to words (2 bytes) */-MLX5_SET(ste_double_action_insert_with_inline_v1,hw_action,start_offset,-i*2);+MLX5_SET(ste_double_action_insert_with_inline_v1,hw_action,start_offset,0);/* Copy bytes one by one to avoid endianness problem */addr_inline=MLX5_ADDR_OF(ste_double_action_insert_with_inline_v1,hw_action,inline_data);-memcpy(addr_inline,data_ptr,inline_data_sz);+memcpy(addr_inline,data_ptr-i*inline_data_sz,inline_data_sz);hw_action+=DR_STE_ACTION_DOUBLE_SZ;-data_ptr+=inline_data_sz;used_actions++;}-/* Remove 2 extra bytes */+/* Remove first 2 extra bytes */MLX5_SET(ste_single_action_remove_header_size_v1,hw_action,action_id,DR_STE_V1_ACTION_ID_REMOVE_BY_SIZE);-MLX5_SET(ste_single_action_remove_header_size_v1,hw_action,start_offset,data_sz/2);+MLX5_SET(ste_single_action_remove_header_size_v1,hw_action,start_offset,0);/* The hardware expects here size in words (2 bytes) */MLX5_SET(ste_single_action_remove_header_size_v1,hw_action,remove_size,1);used_actions++;
From: Parav Pandit <redacted>
When auxiliary bus autoprobe is disabled and SF is in ACTIVE state,
on SF port deletion it transitions from ACTIVE->ALLOCATED->INVALID.
When VHCA event handler queries the state, it is already transition
to INVALID state.
In this scenario, event handler missed to delete the SF device.
Fix it by deleting the SF when SF state is INVALID.
Fixes: 90d010b8634b ("net/mlx5: SF, Add auxiliary device support")
Signed-off-by: Parav Pandit <redacted>
Reviewed-by: Vu Pham <redacted>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.c | 1 +
1 file changed, 1 insertion(+)
From: Dmytro Linkin <redacted>
Running devlink reload command for port in switchdev mode cause
resources to corrupt: driver can't release allocated EQ and reclaim
memory pages, because "rdma" auxiliary device had add CQs which blocks
EQ from deletion.
Erroneous sequence happens during reload-down phase, and is following:
1. detach device - suspends auxiliary devices which support it, destroys
others. During this step "eth-rep" and "rdma-rep" are destroyed,
"eth" - suspended.
2. disable SRIOV - moves device to legacy mode; as part of disablement -
rescans drivers. This step adds "rdma" auxiliary device.
3. destroy EQ table - <failure>.
Driver shouldn't create any device during unload flows. To handle that
implement MLX5_PRIV_FLAGS_DETACH flag, set it on device detach and unset
on device attach. If flag is set do no-op on drivers rescan.
Fixes: a925b5e309c9 ("net/mlx5: Register mlx5 devices to auxiliary virtual bus")
Signed-off-by: Dmytro Linkin <redacted>
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Roi Dayan <redacted>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/dev.c | 4 ++++
include/linux/mlx5/driver.h | 4 ++++
2 files changed, 8 insertions(+)
@@ -542,6 +542,10 @@ struct mlx5_core_roce {enum{MLX5_PRIV_FLAGS_DISABLE_IB_ADEV=1<<0,MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV=1<<1,+/* Set during device detach to block any further devices+*creation/deletionondriversrescan.Unsetduringdeviceattach.+*/+MLX5_PRIV_FLAGS_DETACH=1<<2,};structmlx5_adev{
From: Aya Levin <redacted>
Reset only the index part of the mkey and keep the variant part. On
devlink reload, driver recreates mkeys, so the mkey index may change.
Trying to preserve the variant part of the mkey, driver mistakenly
merged the mkey index with current value. In case of a devlink reload,
current value of index part is dirty, so the index may be corrupted.
Fixes: 54c62e13ad76 ("{IB,net}/mlx5: Setup mkey variant before mr create command invocation")
Signed-off-by: Aya Levin <redacted>
Signed-off-by: Amir Tzin <redacted>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/mr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Hello:
This series was applied to netdev/net.git (refs/heads/master):
On Wed, 16 Jun 2021 15:40:08 -0700 you wrote:
From: Leon Romanovsky <leonro@nvidia.com>
In the case of the failure to execute mlx5_core_set_hca_defaults(),
we used wrong goto label to execute error unwind flow.
Fixes: 5bef709d76a2 ("net/mlx5: Enable host PF HCA after eswitch is initialized")
Reviewed-by: Saeed Mahameed <saeedm@nvidia.com>
Reviewed-by: Moshe Shemesh <redacted>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Parav Pandit <redacted>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
[...]