On Wed, Apr 21, 2021 at 03:55:15PM -0700, James Smart wrote:

On 4/20/2021 5:38 AM, Benjamin Block wrote:
...

quoted

quoted

+	len = *((u32 *)(pcmd + 4));
+	len = be32_to_cpu(len);
+	memcpy(vport->qfpa_res, pcmd, len + 8);
+	len = len / LPFC_PRIORITY_RANGE_DESC_SIZE;
+
+	desc = (struct priority_range_desc *)(pcmd + 8);
+	vmid_range = vport->vmid_priority.vmid_range;
+	if (!vmid_range) {
+		vmid_range = kcalloc(MAX_PRIORITY_DESC, sizeof(*vmid_range),
+				     GFP_KERNEL);
+		if (!vmid_range) {
+			kfree(vport->qfpa_res);
+			goto out;
+		}
+		vport->vmid_priority.vmid_range = vmid_range;
+	}
+	vport->vmid_priority.num_descriptors = len;
+
+	for (i = 0; i < len; i++, vmid_range++, desc++) {
+		lpfc_printf_vlog(vport, KERN_DEBUG, LOG_ELS,
+				 "6539 vmid values low=%d, high=%d, qos=%d, "
+				 "local ve id=%d\n", desc->lo_range,
+				 desc->hi_range, desc->qos_priority,
+				 desc->local_ve_id);
+
+		vmid_range->low = desc->lo_range << 1;
+		if (desc->local_ve_id == QFPA_ODD_ONLY)
+			vmid_range->low++;
+		if (desc->qos_priority)
+			vport->vmid_flag |= LPFC_VMID_QOS_ENABLED;
+		vmid_range->qos = desc->qos_priority;

I'm curios, if the FC-switch signals it supports QoS for a range here, how
exactly interacts this with the VM IDs that you seem to allocate
dynamically during runtime for cgroups that request specific App IDs?
You don't seem to use `LPFC_VMID_QOS_ENABLED` anywhere else in the
series. >
Would different cgroups get different QoS classes/guarantees depending
on the selected VM ID (higher VM ID gets better QoS class, or something
like that?)? Would the tagged traffic be handled differently than the
ordinary traffic in the fabric?

The simple answer is there is no interaction w/ the cgroup on priority.
And no- we really don't look or use it.  The ranges don't really have hard
priority values. The way it works is that all values within a range is
equal; a value in the first range is "higher priority" than a value in the
second range; and a value in the second range is higher than those in the
third range, and so on. 

Ah. That's interesting. I thought it is like that, but wasn't sure from
the spec. Thanks for clarifying.

Doesn't really matter whether the range was marked
Best Effort or H/M/L. There's no real "weight".

What you see is the driver simply recording the different ranges so that it
knows what to allocate from later on. The driver creates a flat bitmap of
all possible values (max of 255) from all ranges - then will allocate values
on a first bit set basis.  I know at one point we were going to only
auto-assign if there was 1 range, and if multiple range was going to defer a
mgmt authority to tell us which range, but this obviously doesn't do that.

I was worrying a bit whether this would create some hard to debug
problems in the wild, when QoS essentially depends on the order in which
Applications/Containers are started and get IDs assigned accordingly -
assuming there is multiple priority ranges.

Also... although this is coded to support the full breadth of what the
standard allows, it may well be the switch only implements 1 range in
practice.

-- 
Best Regards, Benjamin Block  / Linux on IBM Z Kernel Development / IBM Systems
IBM Deutschland Research & Development GmbH    /    https://www.ibm.com/privacy
Vorsitz. AufsR.: Gregor Pillen         /        Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: AmtsG Stuttgart, HRB 243294