From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:32
Hi Jason, hi Doug,
Please consider to include following changes to the next merge window.
It contains:
- Patch 01 ~ 10: Typical code refactoring patches
- Patch 11: Requested by Jason
https://www.spinics.net/lists/linux-rdma/msg102009.html
- Patch 12 RDMA/rtrs-srv: Replace atomic_t with percpu_ref for ids_inflight, we
split the sysfs_emit conversion to a seperate patch 13.
- Patch 14 ~ 20: Bug fixes
V3->V2:
- ratelimit error message for first_con check. (Jason)
- split the sysfs_emit conversion to a new patch 13.
V2->V1:
- drop one patch "RDMA/rtrs-clt: No need to check queue_depth when
receiving" as requested by Leon
- (void) casting will be removed by next patch set
as requested by Leon.
https://www.spinics.net/lists/linux-rdma/msg102200.html
Dima Stepanov (1):
RDMA/rtrs: Use strscpy instead of strlcpy
Gioh Kim (7):
RDMA/rtrs-clt: Remove MAX_SESS_QUEUE_DEPTH from rtrs_send_sess_info
RDMA/rtrs: Change MAX_SESS_QUEUE_DEPTH
RDMA/rtrs: Define MIN_CHUNK_SIZE
RDMA/rtrs: Do not reset hb_missed_max after re-connection
RDMA/rtrs-srv: Duplicated session name is not allowed
RDMA/rtrs-srv: Fix memory leak of unfreed rtrs_srv_stats object
RDMA/rtrs-clt: Fix memory leak of not-freed sess->stats and
stats->pcpu_stats
Guoqing Jiang (6):
RDMA/rtrs-srv: Kill reject_w_econnreset label
RDMA/rtrs-srv: Clean up the code in __rtrs_srv_change_state
RDMA/rtrs-clt: Kill rtrs_clt_{start,stop}_hb
RDMA/rtrs-clt: Kill rtrs_clt_disconnect_from_sysfs
RDMA/rtrs-srv: Kill __rtrs_srv_change_state
RDMA/rtrs-clt: Remove redundant 'break'
Jack Wang (2):
RDMA/rtrs-srv: convert scnprintf to sysfs_emit
RDMA/rtrs-srv: Fix memory leak when having multiple sessions
Md Haris Iqbal (4):
RDMA/rtrs-srv: Add error messages for cases when failing RDMA
connection
RDMA/rtrs-clt: Check state of the rtrs_clt_sess before reading its
stats
RDMA/rtrs-srv: Replace atomic_t with percpu_ref for ids_inflight
RDMA/rtrs-clt: Check if the queue_depth has changed during a
reconnection
drivers/infiniband/ulp/rtrs/rtrs-clt-sysfs.c | 5 +-
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 58 +++----
drivers/infiniband/ulp/rtrs/rtrs-clt.h | 2 +-
drivers/infiniband/ulp/rtrs/rtrs-pri.h | 14 +-
drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c | 12 +-
drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c | 1 +
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 163 +++++++++++--------
drivers/infiniband/ulp/rtrs/rtrs-srv.h | 4 +-
drivers/infiniband/ulp/rtrs/rtrs.c | 1 -
9 files changed, 146 insertions(+), 114 deletions(-)
--
2.25.1
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:33
From: Gioh Kim <redacted>
Client receives queue_depth value from server. There is no need
to use MAX_SESS_QUEUE_DEPTH value.
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:34
From: Guoqing Jiang <redacted>
No need to use double switch to check the change of state everywhere,
let's change them to "if" to reduce size.
Signed-off-by: Guoqing Jiang <redacted>
Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Gioh Kim <redacted>
---
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 23 ++++-------------------
1 file changed, 4 insertions(+), 19 deletions(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:34
From: Md Haris Iqbal <redacted>
It was difficult to find out why it failed to establish RDMA
connection. This patch adds some messages to show which function
has failed why.
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
@@ -1356,8 +1356,10 @@ static struct rtrs_srv *get_or_create_srv(struct rtrs_srv_ctx *ctx,*Ifthisrequestisnotthefirstconnectionrequestfromthe*clientforthissessionthenfailandreturnerror.*/-if(!first_conn)+if(!first_conn){+pr_err_ratelimited("Error: Not the first connection request for this session\n");returnERR_PTR(-ENXIO);+}/* need to allocate a new srv */srv=kzalloc(sizeof(*srv),GFP_KERNEL);
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:36
From: Gioh Kim <redacted>
Max IB immediate data size is 2^28 (MAX_IMM_PAYL_BITS)
and the minimum chunk size is 4096 (2^12).
Therefore the maximum sess_queue_depth is 65536 (2^16).
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-pri.h | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:36
From: Gioh Kim <redacted>
Define MIN_CHUNK_SIZE to replace the hard-coding number.
We need 4k for metadata, so MIN_CHUNK_SIZE should be at least 8k.
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-pri.h | 1 +
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 4 ++--
2 files changed, 3 insertions(+), 2 deletions(-)
@@ -2165,9 +2165,9 @@ static int check_module_params(void)sess_queue_depth,1,MAX_SESS_QUEUE_DEPTH);return-EINVAL;}-if(max_chunk_size<4096||!is_power_of_2(max_chunk_size)){+if(max_chunk_size<MIN_CHUNK_SIZE||!is_power_of_2(max_chunk_size)){pr_err("Invalid max_chunk_size value %d, has to be >= %d and should be power of two.\n",-max_chunk_size,4096);+max_chunk_size,MIN_CHUNK_SIZE);return-EINVAL;}
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:37
From: Dima Stepanov <redacted>
During checkpatch analyzing the following warning message was found:
WARNING:STRLCPY: Prefer strscpy over strlcpy - see:
https://lore.kernel.org/r/CAHk-=wgfRnXz0W3D37d01q3JFkr_i_uTL=V6A6G1oUZcprmknw@mail.gmail.com/
Fix it by using strscpy calls instead of strlcpy.
Signed-off-by: Dima Stepanov <redacted>
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 4 ++--
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 6 +++---
2 files changed, 5 insertions(+), 5 deletions(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:38
From: Guoqing Jiang <redacted>
The two wrappers are not needed since we can call rtrs_{start,stop}_hb
directly.
Signed-off-by: Guoqing Jiang <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Gioh Kim <redacted>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 14 ++------------
1 file changed, 2 insertions(+), 12 deletions(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:40
From: Guoqing Jiang <redacted>
No need since the only user is rtrs_srv_change_state.
Signed-off-by: Guoqing Jiang <redacted>
Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Gioh Kim <redacted>
---
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 17 +++--------------
1 file changed, 3 insertions(+), 14 deletions(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:40
From: Guoqing Jiang <redacted>
It is duplicated with the very next line
Signed-off-by: Guoqing Jiang <redacted>
Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Gioh Kim <redacted>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 1 -
1 file changed, 1 deletion(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:41
From: Md Haris Iqbal <redacted>
When get_next_path_min_inflight is called to select the next path, it
iterates over the list of available rtrs_clt_sess (paths). It then reads
the number of inflight IOs for that path to select one which has the least
inflight IO.
But it may so happen that rtrs_clt_sess (path) is no longer in the
connected state because closing or error recovery paths can change the status
of the rtrs_clt_Sess.
For example, the client sent the heart-beat and did not get the
response, it would change the session status and stop IO processing.
The added checking of this patch can prevent accessing the broken path
and generating duplicated error messages.
It is ok if the status is changed after checking the status because
the error recovery path does not free memory and only tries to
reconnection. And also it is ok if the session is closed after checking
the status because closing the session changes the session status and
flush all IO beforing free memory. If the session is being accessed for
IO processing, the closing session will wait.
Fixes: 6a98d71daea18 ("RDMA/rtrs: client: main functionality")
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Reviewed-by: Gioh Kim <redacted>
Signed-off-by: Gioh Kim <redacted>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 3 +++
1 file changed, 3 insertions(+)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:44
From: Md Haris Iqbal <redacted>
ids_inflight is used to track the inflight IOs. But the use of atomic_t
variable can cause performance drops and can also become a performance
bottleneck.
This commit replaces the use of atomic_t with a percpu_ref structure. The
advantage it offers is, it doesn't check if the reference has fallen to 0,
until the user explicitly signals it to; and that is done by the
percpu_ref_kill() function call. After that, the percpu_ref structure
behaves like an atomic_t and for every put call, checks whether the
reference has fallen to 0 or not.
rtrs_srv_stats_rdma_to_str shows the count of ids_inflight as 0
for user-mode tools not to be confused.
Fixes: 9cb837480424e ("RDMA/rtrs: server: main functionality")
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Gioh Kim <redacted>
---
drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c | 12 +++---
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 43 +++++++++++++-------
drivers/infiniband/ulp/rtrs/rtrs-srv.h | 4 +-
3 files changed, 35 insertions(+), 24 deletions(-)
@@ -156,21 +169,14 @@ static int rtrs_srv_alloc_ops_ids(struct rtrs_srv_sess *sess)staticinlinevoidrtrs_srv_get_ops_ids(structrtrs_srv_sess*sess){-atomic_inc(&sess->ids_inflight);+percpu_ref_get(&sess->ids_inflight_ref);}staticinlinevoidrtrs_srv_put_ops_ids(structrtrs_srv_sess*sess){-if(atomic_dec_and_test(&sess->ids_inflight))-wake_up(&sess->ids_waitq);+percpu_ref_put(&sess->ids_inflight_ref);}-staticvoidrtrs_srv_wait_ops_ids(structrtrs_srv_sess*sess)-{-wait_event(sess->ids_waitq,!atomic_read(&sess->ids_inflight));-}--staticvoidrtrs_srv_reg_mr_done(structib_cq*cq,structib_wc*wc){structrtrs_srv_con*con=to_srv_con(wc->qp->qp_context);
@@ -1479,8 +1485,15 @@ static void rtrs_srv_close_work(struct work_struct *work)rdma_disconnect(con->c.cm_id);ib_drain_qp(con->c.qp);}-/* Wait for all inflights */-rtrs_srv_wait_ops_ids(sess);++/*+*Degraderefcounttotheusualmodelwithasingleshared+*atomic_tcounter+*/+percpu_ref_kill(&sess->ids_inflight_ref);++/* Wait for all completion */+wait_for_completion(&sess->complete_done);/* Notify upper layer if we are the last path */rtrs_srv_sess_down(sess);
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:47
From: Gioh Kim <redacted>
When re-connecting, it resets hb_missed_max to 0.
Before the first re-connecting, client will trigger re-connection
when it gets hb-ack more than 5 times. But after the first
re-connecting, clients will do re-connection whenever it does
not get hb-ack because hb_missed_max is 0.
There is no need to reset hb_missed_max when re-connecting.
hb_missed_max should be kept until closing the session.
Fixes: c0894b3ea69d3 ("RDMA/rtrs: core: lib functions shared between client and server modules")
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs.c | 1 -
1 file changed, 1 deletion(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:47
From: Gioh Kim <redacted>
If two clients try to use the same session name, rtrs-server generates a
kernel error that it failed to create the sysfs because the filename
is duplicated.
This patch adds code to check if there already exists the same session
name with the different UUID. If a client tries to add more session,
it sends the UUID and the session name. Therefore it is ok if there is
already same session name with the same UUID. The rtrs-server must fail
only-if there is the same session name with the different UUID.
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Aleksei Marov <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 42 +++++++++++++++++++++++++-
1 file changed, 41 insertions(+), 1 deletion(-)
@@ -753,7 +753,40 @@ static void rtrs_srv_sess_down(struct rtrs_srv_sess *sess)mutex_unlock(&srv->paths_ev_mutex);}+staticboolexist_sessname(structrtrs_srv_ctx*ctx,+constchar*sessname,constuuid_t*path_uuid)+{+structrtrs_srv*srv;+structrtrs_srv_sess*sess;+boolfound=false;++mutex_lock(&ctx->srv_mutex);+list_for_each_entry(srv,&ctx->srv_list,ctx_list){+mutex_lock(&srv->paths_mutex);++/* when a client with same uuid and same sessname tried to add a path */+if(uuid_equal(&srv->paths_uuid,path_uuid)){+mutex_unlock(&srv->paths_mutex);+continue;+}++list_for_each_entry(sess,&srv->paths_list,s.entry){+if(strlen(sess->s.sessname)==strlen(sessname)&&+!strcmp(sess->s.sessname,sessname)){+found=true;+break;+}+}+mutex_unlock(&srv->paths_mutex);+if(found)+break;+}+mutex_unlock(&ctx->srv_mutex);+returnfound;+}+staticintpost_recv_sess(structrtrs_srv_sess*sess);+staticintrtrs_rdma_do_reject(structrdma_cm_id*cm_id,interrno);staticintprocess_info_req(structrtrs_srv_con*con,structrtrs_msg_info_req*msg)
@@ -772,10 +805,17 @@ static int process_info_req(struct rtrs_srv_con *con,rtrs_err(s,"post_recv_sess(), err: %d\n",err);returnerr;}++if(exist_sessname(sess->srv->ctx,+msg->sessname,&sess->srv->paths_uuid)){+rtrs_err(s,"sessname is duplicated: %s\n",msg->sessname);+return-EPERM;+}+strscpy(sess->s.sessname,msg->sessname,sizeof(sess->s.sessname));+rwr=kcalloc(sess->mrs_num,sizeof(*rwr),GFP_KERNEL);if(unlikely(!rwr))return-ENOMEM;-strscpy(sess->s.sessname,msg->sessname,sizeof(sess->s.sessname));tx_sz=sizeof(*rsp);tx_sz+=sizeof(rsp->desc[0])*sess->mrs_num;
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:48
From: Gioh Kim <redacted>
When closing a session, currently the rtrs_srv_stats object in the
closing session is freed by kobject release. But if it failed
to create a session by various reasons, it must free the rtrs_srv_stats
object directly because kobject is not created yet.
This problem is found by kmemleak as below:
1. One client machine maps /dev/nullb0 with session name 'bla':
root@test1:~# echo "sessname=bla path=ip:192.168.122.190 \
device_path=/dev/nullb0" > /sys/devices/virtual/rnbd-client/ctl/map_device
2. Another machine failed to create a session with the same name 'bla':
root@test2:~# echo "sessname=bla path=ip:192.168.122.190 \
device_path=/dev/nullb1" > /sys/devices/virtual/rnbd-client/ctl/map_device
-bash: echo: write error: Connection reset by peer
3. The kmemleak on server machine reported an error:
unreferenced object 0xffff888033cdc800 (size 128):
comm "kworker/2:1", pid 83, jiffies 4295086585 (age 2508.680s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000a72903b2>] __alloc_sess+0x1d4/0x1250 [rtrs_server]
[<00000000d1e5321e>] rtrs_srv_rdma_cm_handler+0xc31/0xde0 [rtrs_server]
[<00000000bb2f6e7e>] cma_ib_req_handler+0xdc5/0x2b50 [rdma_cm]
[<00000000e896235d>] cm_process_work+0x2d/0x100 [ib_cm]
[<00000000b6866c5f>] cm_req_handler+0x11bc/0x1c40 [ib_cm]
[<000000005f5dd9aa>] cm_work_handler+0xe65/0x3cf2 [ib_cm]
[<00000000610151e7>] process_one_work+0x4bc/0x980
[<00000000541e0f77>] worker_thread+0x78/0x5c0
[<00000000423898ca>] kthread+0x191/0x1e0
[<000000005a24b239>] ret_from_fork+0x3a/0x50
Fixes: 39c2d639ca183 ("RDMA/rtrs-srv: Set .release function for rtrs srv device during device init")
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-srv.c | 1 +
1 file changed, 1 insertion(+)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:54
From: Md Haris Iqbal <redacted>
The queue_depth is a module parameter for rtrs_server. It is used on the
client side to determing the queue_depth of the request queue for the RNBD
virtual block device.
During a reconnection event for an already mapped device, in case the
rtrs_server module queue_depth has changed, fail the reconnect attempt.
Also stop further auto reconnection attempts. A manual reconnect via
sysfs has to be triggerred.
Fixes: 6a98d71daea18 ("RDMA/rtrs: client: main functionality")
Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com>
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 19 +++++++++++++++----
1 file changed, 15 insertions(+), 4 deletions(-)
From: Jack Wang <jinpu.wang@ionos.com> Date: 2021-05-28 11:30:55
From: Gioh Kim <redacted>
sess->stats and sess->stats->pcpu_stats objects are freed
when sysfs entry is removed. If something wrong happens and
session is closed before sysfs entry is created,
sess->stats and sess->stats->pcpu_stats objects are not freed.
This patch adds freeing of them at three places:
1. When client uses wrong address and session creation fails.
2. When client fails to create a sysfs entry.
3. When client adds wrong address via sysfs add_path.
Fixes: 215378b838df0 ("RDMA/rtrs: client: sysfs interface functions")
Signed-off-by: Gioh Kim <redacted>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
---
drivers/infiniband/ulp/rtrs/rtrs-clt.c | 6 ++++++
1 file changed, 6 insertions(+)
drivers/infiniband/ulp/rtrs/rtrs-clt.c:1786:19: warning: result of comparison of constant 'MAX_SESS_QUEUE_DEPTH' (65536) with expression of type 'u16' (aka 'unsigned short') is always false [-Wtautological-constant-out-of-range-compare]
drivers/infiniband/ulp/rtrs/rtrs-clt.c:1786:19: warning: result of comparison of constant 'MAX_SESS_QUEUE_DEPTH' (65536) with expression of type 'u16' (aka 'unsigned short') is always false [-Wtautological-constant-out-of-range-compare]
drivers/infiniband/ulp/rtrs/rtrs-clt.c:1786:19: warning: result of comparison of constant 'MAX_SESS_QUEUE_DEPTH' (65536) with expression of type 'u16' (aka 'unsigned short') is always false [-Wtautological-constant-out-of-range-compare]
if (queue_depth > MAX_SESS_QUEUE_DEPTH) {
~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~
include/linux/compiler.h:56:47: note: expanded from macro 'if'
#define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
^~~~
include/linux/compiler.h:58:61: note: expanded from macro '__trace_if_var'
#define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
^~~~
3 warnings generated.
vim +1786 drivers/infiniband/ulp/rtrs/rtrs-clt.c
6a98d71daea186 Jack Wang 2020-05-11 1750
6a98d71daea186 Jack Wang 2020-05-11 1751 static int rtrs_rdma_conn_established(struct rtrs_clt_con *con,
6a98d71daea186 Jack Wang 2020-05-11 1752 struct rdma_cm_event *ev)
6a98d71daea186 Jack Wang 2020-05-11 1753 {
6a98d71daea186 Jack Wang 2020-05-11 1754 struct rtrs_clt_sess *sess = to_clt_sess(con->c.sess);
6a98d71daea186 Jack Wang 2020-05-11 1755 struct rtrs_clt *clt = sess->clt;
6a98d71daea186 Jack Wang 2020-05-11 1756 const struct rtrs_msg_conn_rsp *msg;
6a98d71daea186 Jack Wang 2020-05-11 1757 u16 version, queue_depth;
6a98d71daea186 Jack Wang 2020-05-11 1758 int errno;
6a98d71daea186 Jack Wang 2020-05-11 1759 u8 len;
6a98d71daea186 Jack Wang 2020-05-11 1760
6a98d71daea186 Jack Wang 2020-05-11 1761 msg = ev->param.conn.private_data;
6a98d71daea186 Jack Wang 2020-05-11 1762 len = ev->param.conn.private_data_len;
6a98d71daea186 Jack Wang 2020-05-11 1763 if (len < sizeof(*msg)) {
6a98d71daea186 Jack Wang 2020-05-11 1764 rtrs_err(clt, "Invalid RTRS connection response\n");
6a98d71daea186 Jack Wang 2020-05-11 1765 return -ECONNRESET;
6a98d71daea186 Jack Wang 2020-05-11 1766 }
6a98d71daea186 Jack Wang 2020-05-11 1767 if (le16_to_cpu(msg->magic) != RTRS_MAGIC) {
6a98d71daea186 Jack Wang 2020-05-11 1768 rtrs_err(clt, "Invalid RTRS magic\n");
6a98d71daea186 Jack Wang 2020-05-11 1769 return -ECONNRESET;
6a98d71daea186 Jack Wang 2020-05-11 1770 }
6a98d71daea186 Jack Wang 2020-05-11 1771 version = le16_to_cpu(msg->version);
6a98d71daea186 Jack Wang 2020-05-11 1772 if (version >> 8 != RTRS_PROTO_VER_MAJOR) {
6a98d71daea186 Jack Wang 2020-05-11 1773 rtrs_err(clt, "Unsupported major RTRS version: %d, expected %d\n",
6a98d71daea186 Jack Wang 2020-05-11 1774 version >> 8, RTRS_PROTO_VER_MAJOR);
6a98d71daea186 Jack Wang 2020-05-11 1775 return -ECONNRESET;
6a98d71daea186 Jack Wang 2020-05-11 1776 }
6a98d71daea186 Jack Wang 2020-05-11 1777 errno = le16_to_cpu(msg->errno);
6a98d71daea186 Jack Wang 2020-05-11 1778 if (errno) {
6a98d71daea186 Jack Wang 2020-05-11 1779 rtrs_err(clt, "Invalid RTRS message: errno %d\n",
6a98d71daea186 Jack Wang 2020-05-11 1780 errno);
6a98d71daea186 Jack Wang 2020-05-11 1781 return -ECONNRESET;
6a98d71daea186 Jack Wang 2020-05-11 1782 }
6a98d71daea186 Jack Wang 2020-05-11 1783 if (con->c.cid == 0) {
6a98d71daea186 Jack Wang 2020-05-11 1784 queue_depth = le16_to_cpu(msg->queue_depth);
6a98d71daea186 Jack Wang 2020-05-11 1785
6a98d71daea186 Jack Wang 2020-05-11 @1786 if (queue_depth > MAX_SESS_QUEUE_DEPTH) {
6a98d71daea186 Jack Wang 2020-05-11 1787 rtrs_err(clt, "Invalid RTRS message: queue=%d\n",
6a98d71daea186 Jack Wang 2020-05-11 1788 queue_depth);
6a98d71daea186 Jack Wang 2020-05-11 1789 return -ECONNRESET;
6a98d71daea186 Jack Wang 2020-05-11 1790 }
6a98d71daea186 Jack Wang 2020-05-11 1791 if (!sess->rbufs || sess->queue_depth < queue_depth) {
6a98d71daea186 Jack Wang 2020-05-11 1792 kfree(sess->rbufs);
6a98d71daea186 Jack Wang 2020-05-11 1793 sess->rbufs = kcalloc(queue_depth, sizeof(*sess->rbufs),
6a98d71daea186 Jack Wang 2020-05-11 1794 GFP_KERNEL);
6a98d71daea186 Jack Wang 2020-05-11 1795 if (!sess->rbufs)
6a98d71daea186 Jack Wang 2020-05-11 1796 return -ENOMEM;
6a98d71daea186 Jack Wang 2020-05-11 1797 }
6a98d71daea186 Jack Wang 2020-05-11 1798 sess->queue_depth = queue_depth;
6a98d71daea186 Jack Wang 2020-05-11 1799 sess->max_hdr_size = le32_to_cpu(msg->max_hdr_size);
6a98d71daea186 Jack Wang 2020-05-11 1800 sess->max_io_size = le32_to_cpu(msg->max_io_size);
6a98d71daea186 Jack Wang 2020-05-11 1801 sess->flags = le32_to_cpu(msg->flags);
6a98d71daea186 Jack Wang 2020-05-11 1802 sess->chunk_size = sess->max_io_size + sess->max_hdr_size;
6a98d71daea186 Jack Wang 2020-05-11 1803
6a98d71daea186 Jack Wang 2020-05-11 1804 /*
6a98d71daea186 Jack Wang 2020-05-11 1805 * Global queue depth and IO size is always a minimum.
6a98d71daea186 Jack Wang 2020-05-11 1806 * If while a reconnection server sends us a value a bit
6a98d71daea186 Jack Wang 2020-05-11 1807 * higher - client does not care and uses cached minimum.
6a98d71daea186 Jack Wang 2020-05-11 1808 *
6a98d71daea186 Jack Wang 2020-05-11 1809 * Since we can have several sessions (paths) restablishing
6a98d71daea186 Jack Wang 2020-05-11 1810 * connections in parallel, use lock.
6a98d71daea186 Jack Wang 2020-05-11 1811 */
6a98d71daea186 Jack Wang 2020-05-11 1812 mutex_lock(&clt->paths_mutex);
6a98d71daea186 Jack Wang 2020-05-11 1813 clt->queue_depth = min_not_zero(sess->queue_depth,
6a98d71daea186 Jack Wang 2020-05-11 1814 clt->queue_depth);
6a98d71daea186 Jack Wang 2020-05-11 1815 clt->max_io_size = min_not_zero(sess->max_io_size,
6a98d71daea186 Jack Wang 2020-05-11 1816 clt->max_io_size);
6a98d71daea186 Jack Wang 2020-05-11 1817 mutex_unlock(&clt->paths_mutex);
6a98d71daea186 Jack Wang 2020-05-11 1818
6a98d71daea186 Jack Wang 2020-05-11 1819 /*
6a98d71daea186 Jack Wang 2020-05-11 1820 * Cache the hca_port and hca_name for sysfs
6a98d71daea186 Jack Wang 2020-05-11 1821 */
6a98d71daea186 Jack Wang 2020-05-11 1822 sess->hca_port = con->c.cm_id->port_num;
6a98d71daea186 Jack Wang 2020-05-11 1823 scnprintf(sess->hca_name, sizeof(sess->hca_name),
6a98d71daea186 Jack Wang 2020-05-11 1824 sess->s.dev->ib_dev->name);
6a98d71daea186 Jack Wang 2020-05-11 1825 sess->s.src_addr = con->c.cm_id->route.addr.src_addr;
03e9b33a0fd677 Md Haris Iqbal 2021-02-12 1826 /* set for_new_clt, to allow future reconnect on any path */
03e9b33a0fd677 Md Haris Iqbal 2021-02-12 1827 sess->for_new_clt = 1;
6a98d71daea186 Jack Wang 2020-05-11 1828 }
6a98d71daea186 Jack Wang 2020-05-11 1829
6a98d71daea186 Jack Wang 2020-05-11 1830 return 0;
6a98d71daea186 Jack Wang 2020-05-11 1831 }
6a98d71daea186 Jack Wang 2020-05-11 1832
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
From: Jason Gunthorpe <jgg@nvidia.com> Date: 2021-05-28 23:53:45
On Fri, May 28, 2021 at 01:29:58PM +0200, Jack Wang wrote:
Hi Jason, hi Doug,
Please consider to include following changes to the next merge window.
It contains:
- Patch 01 ~ 10: Typical code refactoring patches
- Patch 11: Requested by Jason
https://www.spinics.net/lists/linux-rdma/msg102009.html
- Patch 12 RDMA/rtrs-srv: Replace atomic_t with percpu_ref for ids_inflight, we
split the sysfs_emit conversion to a seperate patch 13.
- Patch 14 ~ 20: Bug fixes
V3->V2:
- ratelimit error message for first_con check. (Jason)
- split the sysfs_emit conversion to a new patch 13.
V2->V1:
- drop one patch "RDMA/rtrs-clt: No need to check queue_depth when
receiving" as requested by Leon
- (void) casting will be removed by next patch set
as requested by Leon.
https://www.spinics.net/lists/linux-rdma/msg102200.html
Dima Stepanov (1):
RDMA/rtrs: Use strscpy instead of strlcpy
Gioh Kim (7):
RDMA/rtrs-clt: Remove MAX_SESS_QUEUE_DEPTH from rtrs_send_sess_info
RDMA/rtrs: Change MAX_SESS_QUEUE_DEPTH
RDMA/rtrs: Define MIN_CHUNK_SIZE
RDMA/rtrs: Do not reset hb_missed_max after re-connection
RDMA/rtrs-srv: Duplicated session name is not allowed
RDMA/rtrs-srv: Fix memory leak of unfreed rtrs_srv_stats object
RDMA/rtrs-clt: Fix memory leak of not-freed sess->stats and
stats->pcpu_stats
Guoqing Jiang (6):
RDMA/rtrs-srv: Kill reject_w_econnreset label
RDMA/rtrs-srv: Clean up the code in __rtrs_srv_change_state
RDMA/rtrs-clt: Kill rtrs_clt_{start,stop}_hb
RDMA/rtrs-clt: Kill rtrs_clt_disconnect_from_sysfs
RDMA/rtrs-srv: Kill __rtrs_srv_change_state
RDMA/rtrs-clt: Remove redundant 'break'
Jack Wang (2):
RDMA/rtrs-srv: convert scnprintf to sysfs_emit
RDMA/rtrs-srv: Fix memory leak when having multiple sessions
Md Haris Iqbal (4):
RDMA/rtrs-srv: Add error messages for cases when failing RDMA
connection
RDMA/rtrs-clt: Check state of the rtrs_clt_sess before reading its
stats
RDMA/rtrs-srv: Replace atomic_t with percpu_ref for ids_inflight
RDMA/rtrs-clt: Check if the queue_depth has changed during a
drivers/infiniband/ulp/rtrs/rtrs-clt.c:1786:19: warning: result of comparison of constant 'MAX_SESS_QUEUE_DEPTH' (65536) with expression of type 'u16' (aka 'unsigned short') is always false [-Wtautological-constant-out-of-range-compare]
if (queue_depth > MAX_SESS_QUEUE_DEPTH) {
~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~
Thanks for the reporting.
As the check is checking against u16 max,I think we should reduce
MAX_SESS_QUEUE_DEPTH to 65535, and drop the check in line rtrs-clt:
1786
Jason, you mentioned v3 is applied in for-next, I guess I'll wait when
you push it out, and send the patch to fix this. is it ok?
Thanks!
drivers/infiniband/ulp/rtrs/rtrs-clt.c:1786:19: warning: result of comparison of constant 'MAX_SESS_QUEUE_DEPTH' (65536) with expression of type 'u16' (aka 'unsigned short') is always false [-Wtautological-constant-out-of-range-compare]
if (queue_depth > MAX_SESS_QUEUE_DEPTH) {
~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~
Thanks for the reporting.
As the check is checking against u16 max,I think we should reduce
MAX_SESS_QUEUE_DEPTH to 65535, and drop the check in line rtrs-clt:
1786
Jason, you mentioned v3 is applied in for-next, I guess I'll wait when
you push it out, and send the patch to fix this. is it ok?
Send me a fix right away and I'll fix the original patch
Jason
drivers/infiniband/ulp/rtrs/rtrs-clt.c:1786:19: warning: result of comparison of constant 'MAX_SESS_QUEUE_DEPTH' (65536) with expression of type 'u16' (aka 'unsigned short') is always false [-Wtautological-constant-out-of-range-compare]
if (queue_depth > MAX_SESS_QUEUE_DEPTH) {
~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~
Thanks for the reporting.
As the check is checking against u16 max,I think we should reduce
MAX_SESS_QUEUE_DEPTH to 65535, and drop the check in line rtrs-clt:
1786
Jason, you mentioned v3 is applied in for-next, I guess I'll wait when
you push it out, and send the patch to fix this. is it ok?
Send me a fix right away and I'll fix the original patch