From: Leon Romanovsky <leon@kernel.org> Date: 2016-06-20 06:06:28
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <zhaonenglong-C8/M+/jPZTeaMJb+Lgu22Q@public.gmane.org>
Signed-off-by: Lijun Ou <redacted>
---
+
+static void hns_roce_free_icm_pages(struct hns_roce_dev *hr_dev,
+ struct hns_roce_icm_chunk *chunk)
+{
+ int i;
+
+ if (chunk->nsg > 0)
+ dma_unmap_sg(&hr_dev->pdev->dev, chunk->mem, chunk->npages,
+ DMA_BIDIRECTIONAL);
+
+ for (i = 0; i < chunk->npages; ++i)
+ __free_pages(sg_page(&chunk->mem[i]),
+ get_order(chunk->mem[i].length));
You used alloc_pages for this allocation, so why are you using
__free_pages instead of free_pages?
Hi, Leon
The function prototype of these functions as below：
static inline struct page * alloc_pages(gfp_t gfp_mask, unsigned int
order);
void free_pages(unsigned long addr, unsigned int order);
void __free_pages(struct page *page, unsigned int order);
The type of the first parameter of free_pages is same with the type of
return value of alloc_pages.
Maybe it is better to call __free_pages to release memory that allocated
by calling alloc_pages.
From: Wei Hu (Xavier) <hidden> Date: 2016-06-20 07:55:43
On 2016/6/20 14:06, Leon Romanovsky wrote:
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <zhaonenglong-C8/M+/jPZTeaMJb+Lgu22Q@public.gmane.org>
Signed-off-by: Lijun Ou <redacted>
---
+
+static void hns_roce_free_icm_pages(struct hns_roce_dev *hr_dev,
+ struct hns_roce_icm_chunk *chunk)
+{
+ int i;
+
+ if (chunk->nsg > 0)
+ dma_unmap_sg(&hr_dev->pdev->dev, chunk->mem, chunk->npages,
+ DMA_BIDIRECTIONAL);
+
+ for (i = 0; i < chunk->npages; ++i)
+ __free_pages(sg_page(&chunk->mem[i]),
+ get_order(chunk->mem[i].length));
You used alloc_pages for this allocation, so why are you using
__free_pages instead of free_pages?
Hi, Leon
The function prototype of these functions as below：
static inline struct page * alloc_pages(gfp_t gfp_mask, unsigned int
order);
void free_pages(unsigned long addr, unsigned int order);
void __free_pages(struct page *page, unsigned int order);
The type of the first parameter of free_pages is same with the type of
return value of alloc_pages.
Maybe it is better to call __free_pages to release memory that allocated
by calling alloc_pages.
Hi, Leon
Now we haven't firmware.
But hardware still need memory for QPC\CQC\MTPT\mtt etc.
Thanks
Wei Hu
quoted
Regards
Wei Hu
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Leon Romanovsky <leon@kernel.org> Date: 2016-06-20 09:36:41
On Mon, Jun 20, 2016 at 03:49:24PM +0800, Wei Hu (Xavier) wrote:
On 2016/6/20 14:06, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <redacted>
Signed-off-by: Lijun Ou <redacted>
---
Hi, Leon
Now we haven't firmware.
But hardware still need memory for QPC\CQC\MTPT\mtt etc.
ICM stands for InfiniHost (Interconnect) Context Memory is a specific
memory place to share between host <-> FW and host <-> HW if HW is
aware of specific structures.
I assume that in your case, it is enough to allocate memory region and
supply it to HW. Am I right?
From: Wei Hu (Xavier) <hidden> Date: 2016-06-20 09:48:59
On 2016/6/20 17:27, Leon Romanovsky wrote:
On Mon, Jun 20, 2016 at 03:49:24PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 14:06, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <zhaonenglong-C8/M+/jPZTeaMJb+Lgu22Q@public.gmane.org>
Signed-off-by: Lijun Ou <redacted>
---
Hi, Leon
Now we haven't firmware.
But hardware still need memory for QPC\CQC\MTPT\mtt etc.
ICM stands for InfiniHost (Interconnect) Context Memory is a specific
memory place to share between host <-> FW and host <-> HW if HW is
aware of specific structures.
I assume that in your case, it is enough to allocate memory region and
supply it to HW. Am I right?
For Our hardware,
1. ICM has a memory management method, It's very good for
QPC\CQC\MTPT\mtt etc. we need it.
2. The meomry for QPC\CQC\MTPT\mtt only used for RoCE hardware and
driver, we don't want use MR.
3. Now we haven't firmware, maybe we need it next version.
Thanks
Wei Hu.
quoted
Thanks
Wei Hu
quoted
quoted
Regards
Wei Hu
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Leon Romanovsky <leon@kernel.org> Date: 2016-06-20 13:07:39
On Mon, Jun 20, 2016 at 05:48:15PM +0800, Wei Hu (Xavier) wrote:
On 2016/6/20 17:27, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 03:49:24PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 14:06, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <zhaonenglong-C8/M+/jPZTeaMJb+Lgu22Q@public.gmane.org>
Signed-off-by: Lijun Ou <redacted>
---
Hi, Leon
Now we haven't firmware.
But hardware still need memory for QPC\CQC\MTPT\mtt etc.
ICM stands for InfiniHost (Interconnect) Context Memory is a specific
memory place to share between host <-> FW and host <-> HW if HW is
aware of specific structures.
I assume that in your case, it is enough to allocate memory region and
supply it to HW. Am I right?
For Our hardware,
1. ICM has a memory management method, It's very good for QPC\CQC\MTPT\mtt
etc. we need it.
You need special HW to leverage its. AFAIK it is Mellanox specific.
2. The meomry for QPC\CQC\MTPT\mtt only used for RoCE hardware and driver,
we don't want use MR.
I didn't mean Infiniband MR, but memory region returned from standard
allocation functions (kmalloc, ...).
3. Now we haven't firmware, maybe we need it next version.
You are always invited to add support once it will be needed, no need to
add it in advance.
Thanks
From: Wei Hu (Xavier) <hidden> Date: 2016-06-21 04:40:36
On 2016/6/20 21:04, Leon Romanovsky wrote:
On Mon, Jun 20, 2016 at 05:48:15PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 17:27, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 03:49:24PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 14:06, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <zhaonenglong-C8/M+/jPZTeaMJb+Lgu22Q@public.gmane.org>
Signed-off-by: Lijun Ou <redacted>
---
Hi, Leon
Now we haven't firmware.
But hardware still need memory for QPC\CQC\MTPT\mtt etc.
ICM stands for InfiniHost (Interconnect) Context Memory is a specific
memory place to share between host <-> FW and host <-> HW if HW is
aware of specific structures.
I assume that in your case, it is enough to allocate memory region and
supply it to HW. Am I right?
For Our hardware,
1. ICM has a memory management method, It's very good for QPC\CQC\MTPT\mtt
etc. we need it.
You need special HW to leverage its. AFAIK it is Mellanox specific.
For our hardware, we use ICM to memory management, the memory shared
with host and HW.
QPC\CQC\MTPT\mtt has specific memory requirement.
QPC\CQC\MTPT need continuous memory. we use ICM to management the block
of memory. It's very good！
quoted
2. The meomry for QPC\CQC\MTPT\mtt only used for RoCE hardware and driver,
we don't want use MR.
I didn't mean Infiniband MR, but memory region returned from standard
allocation functions (kmalloc, ...).
quoted
3. Now we haven't firmware, maybe we need it next version.
You are always invited to add support once it will be needed, no need to
add it in advance.
Thanks
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
From: Leon Romanovsky <leon@kernel.org> Date: 2016-06-21 11:56:23
On Tue, Jun 21, 2016 at 12:37:39PM +0800, Wei Hu (Xavier) wrote:
On 2016/6/20 21:04, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 05:48:15PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 17:27, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 03:49:24PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 14:06, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <zhaonenglong-C8/M+/jPZTeaMJb+Lgu22Q@public.gmane.org>
Signed-off-by: Lijun Ou <redacted>
---
Hi, Leon
Now we haven't firmware.
But hardware still need memory for QPC\CQC\MTPT\mtt etc.
ICM stands for InfiniHost (Interconnect) Context Memory is a specific
memory place to share between host <-> FW and host <-> HW if HW is
aware of specific structures.
I assume that in your case, it is enough to allocate memory region and
supply it to HW. Am I right?
For Our hardware,
1. ICM has a memory management method, It's very good for QPC\CQC\MTPT\mtt
etc. we need it.
You need special HW to leverage its. AFAIK it is Mellanox specific.
For our hardware, we use ICM to memory management, the memory shared with
host and HW.
QPC\CQC\MTPT\mtt has specific memory requirement.
QPC\CQC\MTPT need continuous memory. we use ICM to management the block of
memory. It's very good！
I wasn't convinced why do you need to copy whole ICM logic which is
specific to Mellanox. Your requirements can be implemented by standard CMA
and/or DMA.
quoted
quoted
2. The meomry for QPC\CQC\MTPT\mtt only used for RoCE hardware and driver,
we don't want use MR.
I didn't mean Infiniband MR, but memory region returned from standard
allocation functions (kmalloc, ...).
quoted
3. Now we haven't firmware, maybe we need it next version.
You are always invited to add support once it will be needed, no need to
add it in advance.
Thanks
From: Wei Hu (Xavier) <hidden> Date: 2016-06-22 03:54:11
On 2016/6/21 19:55, Leon Romanovsky wrote:
On Tue, Jun 21, 2016 at 12:37:39PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 21:04, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 05:48:15PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 17:27, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 03:49:24PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/20 14:06, Leon Romanovsky wrote:
quoted
On Mon, Jun 20, 2016 at 12:37:40PM +0800, Wei Hu (Xavier) wrote:
quoted
On 2016/6/17 17:58, Leon Romanovsky wrote:
quoted
On Thu, Jun 16, 2016 at 10:35:16PM +0800, Lijun Ou wrote:
quoted
This patch mainly added icm support for RoCE. It initializes icm
which managers the relative memory blocks for RoCE. The data
structures of RoCE will be located in it. For example, CQ table,
QP table and MTPT table so on.
Signed-off-by: Wei Hu <redacted>
Signed-off-by: Nenglong Zhao <redacted>
Signed-off-by: Lijun Ou <redacted>
---
Hi, Leon
Now we haven't firmware.
But hardware still need memory for QPC\CQC\MTPT\mtt etc.
ICM stands for InfiniHost (Interconnect) Context Memory is a specific
memory place to share between host <-> FW and host <-> HW if HW is
aware of specific structures.
I assume that in your case, it is enough to allocate memory region and
supply it to HW. Am I right?
For Our hardware,
1. ICM has a memory management method, It's very good for QPC\CQC\MTPT\mtt
etc. we need it.
You need special HW to leverage its. AFAIK it is Mellanox specific.
For our hardware, we use ICM to memory management, the memory shared with
host and HW.
QPC\CQC\MTPT\mtt has specific memory requirement.
QPC\CQC\MTPT need continuous memory. we use ICM to management the block of
memory. It's very good！
I wasn't convinced why do you need to copy whole ICM logic which is
specific to Mellanox. Your requirements can be implemented by standard CMA
and/or DMA.
Hi, Leon
In hip06 soc,
Hardware need multiple memory blocks for QPC\CQC\MTPT, every block has
continuous memory xxKbyte (like 128Kbyte),
We need to configure the first address of 128Kbyte to hardware.
For example:
//------------------------------------------------------------------------
example 1:
In create qp,
1. If the xx Kbyte memory that include QPC related with qpn, has not
been allocated, do step 2.
else do step 3.
2. dma_alloc xx Kbyte memory for QPC, and configure the first address
of xx Kbyte to hardware.
3. find the QPC memory in xx Kbyte, get the dma_addr.
4. send mailbox command to hardware to create QP.
In step 2, we call xx_table_get function as below to perform logic.
int hns_roce_table_get(struct hns_roce_dev *hr_dev,
struct hns_roce_icm_table *table, unsigned long obj)
{
<snip>
//dma_alloc_coherent 128Kbyte memory
hns_roce_alloc_icm(hr_dev,
HNS_ROCE_TABLE_CHUNK_SIZE >> PAGE_SHIFT, xxxx);
<snip>
/*configure the first address of xx Kbyte to hardware*/
hns_roce_map_icm(hr_dev, table, obj);
<snip>
}
In step 3, we call xx_table_find function to perform logic.
void *hns_roce_table_find(struct hns_roce_icm_table *table, unsigned
long obj,
dma_addr_t *dma_handle);
example 2:
In modify qp:
1. find the QPC memory, get the virtual addr.
2. modify the fields of QPC.
3. send mailbox command to hardware to modify QP.
In step 1, we call xx_table_find function to perform logic.
//--------------------------------------------------------------------------
so, now we haven't a firmware, but ICM algorithm still suitable for
hip06 soc perfectly.
Regards
Wei Hu
quoted
quoted
quoted
2. The meomry for QPC\CQC\MTPT\mtt only used for RoCE hardware and driver,
we don't want use MR.
I didn't mean Infiniband MR, but memory region returned from standard
allocation functions (kmalloc, ...).
quoted
3. Now we haven't firmware, maybe we need it next version.
You are always invited to add support once it will be needed, no need to
add it in advance.
Thanks