Re: Problem running dpdk-qat example in SR-IOV mode

Hi Pankaj,
See below.

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Pankaj Joshi
Sent: Monday, February 20, 2017 11:12 AM
To: dev@dpdk.org; users@dpdk.org; qat-linux <redacted>
Subject: [dpdk-dev] Problem running dpdk-qat example in SR-IOV mode

Hello All,

I am trying to run dpdk-qat example code in SR-IOV mode. Steps done by me
so far are :

The dpdk-qat example app is an earlier mechanism of wrapping a QAT driver into DPDK - it's not using the cryptodev API, and depends on both a kernel driver and user-space libs from a QAT package.
Unless you have a really good reason for using this (like you wanted to use the older Cave Creek device) I would recommend you use the cryptodev API. 
The QAT PMD implementing the cryptodev API, is in drivers/crypto/qat. This supports Coleto and will support future QAT devices. It can be exercised using app/test/ and running tests like cryptodev_qat_....
Documentation on how to use it is here:
 http://dpdk.org/doc/guides/cryptodevs/qat.html

1. Enable SR-IOV, virtualization, VT-d in bios setup.

2. Boot file settings are like :
/boot/vmlinuz-3.13.0-24-generic.efi.signed
root=UUID=67d49074-1cca-490d-9626-bfda8b789ff7 ro
default_hugepagesz=1G
hugepagesz=1G hugepages=16 intel_iommu=on iommu=pt quiet splash
security=selinux selinux=1 vt.handoff=7 isolcpus=1-15

3. Install QAT1.6 software using below configuration :
d1  Set SRIOV Mode to "Host"

4. There is one coletocreek device and device state is up :
service qat_service start
There is 1 acceleration device(s) in the system:
 icp_dev0 - type=dh895xcc, inst_id=0, node_id=1,  bdf=83:00:0, #accel=6,
#engines=12, state=up

5. The output of the "lspci -d:443" command giving as below output :
83:01.0 Co-processor: Intel Corporation Device 0443
83:01.1 Co-processor: Intel Corporation Device 0443
83:01.2 Co-processor: Intel Corporation Device 0443
83:01.3 Co-processor: Intel Corporation Device 0443
....
....
83:04.7 Co-processor: Intel Corporation Device 0443

6. "lsmod | grep qat " gives no output.

The kernel module is called icp_qa_al.ko 
So lsmod | grep qa should show it.

7. "lspci -d :435" command gives output as :
83:00.0 Co-processor: Intel Corporation Coleto Creek PCIe Endpoint

8.The is no such folder :

/sys/bus/pci/drivers/dh895xcc/

This will only be there if using the driver built into the linux kernel since kernel 4.4.
If your kernel version is earlier than this you'll need to follow the steps at the above link for
10.4. Installation using 01.org QAT driver
Instead of
10.5. Installation using kernel.org driver

It looks like you're using a mix of both set of steps.

9. Output of the below command is 32.

cat /sys/bus/pci/drivers/qat_1_6_adf/0000:83:00.0/sriov_numvfs

10. After these steps, while running the dpdk-qat example in host m/c
(As The QAT1.6 is install as host with SR-IOV mode).

The number of crypto instances are getting as 0 and getting error as :

Crypto: get_crypto_instance_on_core failed with status: -1

Please suggest a way, how I can proceed for the same.

Regards,

Pankaj Joshi