Re: [PATCH 4/6] irqchip: irq-mvebu-icu: new driver for Marvell ICU

On 30/05/17 13:05, Thomas Petazzoni wrote:

Hello,

On Tue, 30 May 2017 12:10:29 +0100, Marc Zyngier wrote:

quoted

Thanks for that, looks pretty interesting. A couple of comments below.

Thanks again for the review!

quoted

quoted

+/* GICP registers */
+#define GICP_SETSPI_NSR_OFFSET	0x0
+#define GICP_CLRSPI_NSR_OFFSET	0x8  

Shouldn't that live in some gicp-specific include file?

Would drivers/irqchip/irq-mvebu-gicp.h, included by both
irq-mvebu-gicp.c and irq-mvebu-icu.c be fine for you?

Sure, that'd be fine, assuming that it is necessary (see below).

quoted

quoted

+/* ICU definitions */
+#define ICU_MAX_IRQS		207
+#define IRQS_PER_ICU		64
+#define ICU_MAX_SPI_IRQ_IN_GIC	(2 * IRQS_PER_ICU)  

What's the relationship between ICU_MAX_IRQS and
IRQS_PER_ICU/ICU_MAX_SPI_IRQ_IN_GIC, if any? Is there only a single ICU?
Or can you have multiple ones?

There is one ICU per CP. The Armada 7K SoC has one CP, the Armada 8K
SoC has two CPs. Therefore on Armada 8K, you have two ICUs, one per CP.

OK. Is there any restriction on which SPI an ICU can generate?

But I see your point: there is in fact no direct relation between the
number of GICP SPI interrupts reserved and the number of ICUs and
interrupts per ICU.

Indeed. And maybe we should have an instance of the ICU device per CP.

The number of GICP SPI interrupts (128) should be moved into
irq-mvebu-gicp.h, together with the GICP register offsets.

quoted

quoted

+#define ICU_GIC_SPI_BASE0	64
+#define ICU_GIC_SPI_BASE1	288  

My own gut feeling is that there will be another version of this IP one
of these days, with different bases. Should we bite the bullet right
away and put those in DT?

Where should these properties go? Under the gicp DT node, or under the
ICU DT node?

If the ICU has no knowledge of the SPI it can generate, I'd rather put
that in the GICP node.

quoted

quoted

+static DEFINE_SPINLOCK(icu_lock);
+static DECLARE_BITMAP(icu_irq_alloc, ICU_MAX_SPI_IRQ_IN_GIC);
+
+static unsigned int
+mvebu_icu_icuidx_to_gicspi(unsigned int icuidx)
+{
+	if (icuidx < ICU_GIC_SPI_BASE0)
+		return ICU_GIC_SPI_BASE0 + icuidx;
+	else
+		return ICU_GIC_SPI_BASE1 + (icuidx - IRQS_PER_ICU);  

A small comment on how ICU indexes and GIC SPIs map would be good.

ACK.

quoted

Correct me if I'm wrong, but it really looks like you're dealing with
two devices here, each with their own SPI window.

We in fact don't really care about how many ICUs we have here. We have
128 GICP SPI interrupts available, in ranges:

 - ICU_GIC_SPI_BASE0 ; ICU_GIC_SPI_BASE0 + 64

 - ICU_GIC_SPI_BASE1 ; ICU_GIC_SPI_BASE1 + 64

The icu_irq_alloc bitmap is a global one, which allows to allocate one
GICP SPI interrupts amongst the available 128 interrupts, and this
function simply allows to map the index in this bitmap (from 0 to 127)
to the corresponding GICP SPI interrupt.

That makes a lot more sense now, thanks.

quoted

quoted

+	writel(icu_int, icu->base + ICU_INT_CFG(hwirq));  

You can use a writel_relaxed here.

ACK.

quoted

quoted

+	/*
+	 * The SATA unit has 2 ports, and a dedicated ICU entry per
+	 * port. The ahci sata driver supports only one irq interrupt
+	 * per SATA unit. To solve this conflict, we configure the 2
+	 * SATA wired interrupts in the south bridge into 1 GIC
+	 * interrupt in the north bridge. Even if only a single port
+	 * is enabled, if sata node is enabled, both interrupts are
+	 * configured (regardless of which port is actually in use).
+	 * The ICU index of SATA0 = 107, SATA1 = 109  

You can drop that last comment about the indexes, the #defines are good
enough.

Agreed.

quoted

quoted

+	 */
+	if (hwirq == ICU_SATA0_IRQ_INT || hwirq == ICU_SATA1_IRQ_INT) {
+		writel(icu_int, icu->base + ICU_INT_CFG(ICU_SATA0_IRQ_INT));
+		writel(icu_int, icu->base + ICU_INT_CFG(ICU_SATA1_IRQ_INT));
+	}  

Aren't you wasting an SPI here?

No: we allocate a single SPI, icu_int. What we're doing here is that we
have two different wired interrupts in the CP that are "connected" to
the same GICP SPI interrupt.

But if both ports are enabled, you're going to allocate one SPI per call
to this function, and the last one wins (you never "remember" that you
have configured one port already, and always allocate a new interrupt).

quoted

If you configure SATA0 first, then SATA1, SATA0's allocated SPI will
be wasted (not to mention the corresponding Linux interrupt too).
Can't this be worked around at the AHCI level? It is not like we
don't have any quirk there already...

This is something I wanted to look at, but at a follow-up work, as I
believe the proposed work around is reasonable, and does not affect the
DT binding.

Sure.

quoted

quoted

+	writel(0, icu->base + ICU_INT_CFG(irqd_to_hwirq(irq)));  

writel_relaxed (everywhere in this file).

ACK.

quoted

quoted

+	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	icu->base = devm_ioremap_resource(&pdev->dev, res);
+	if (IS_ERR(icu->base)) {
+		dev_err(&pdev->dev, "Failed to map icu base address.\n");
+		return PTR_ERR(icu->base);
+	}  

Careful here, you're leaking the memory from kstrdup above. Consider
using devm_kstrdup or/and moving all the potential failures (including
hooking on GICP) before doign any memory allocation.

Well spotted, will fix.

quoted

quoted

+	/* Set Clear/Set ICU SPI message address in AP */
+	writel(upper_32_bits(gicp_res->start + GICP_SETSPI_NSR_OFFSET),
+	       icu->base + ICU_SETSPI_NSR_AH);
+	writel(lower_32_bits(gicp_res->start + GICP_SETSPI_NSR_OFFSET),
+	       icu->base + ICU_SETSPI_NSR_AL);
+	writel(upper_32_bits(gicp_res->start + GICP_CLRSPI_NSR_OFFSET),
+	       icu->base + ICU_CLRSPI_NSR_AH);
+	writel(lower_32_bits(gicp_res->start + GICP_CLRSPI_NSR_OFFSET),
+	       icu->base + ICU_CLRSPI_NSR_AL);  

I wonder if it wouldn't be better to have a proper function call into
the GICP code to retrieve this information.

I've never been a big fan of random cross function calls between
drivers, but this can be done.

quoted

Or move the whole GICP probing in here, because even if it is a
separate piece of HW, it serves no real purpose on its own.

So you suggest to merge the whole irq-mvebu-gicp.c stuff inside
irq-mvebu-icu.c ?

Given how small that code is, it wouldn't be a big deal.

quoted

quoted

+	/*
+	 * Clean all ICU interrupts with type SPI_NSR, required to
+	 * avoid unpredictable SPI assignments done by firmware.
+	 */
+	for (i = 0 ; i < ICU_MAX_IRQS ; i++) {
+		icu_int = readl(icu->base + ICU_INT_CFG(i));
+		if ((icu_int >> ICU_GROUP_SHIFT) == ICU_GRP_NSR)
+			writel(0x0, icu->base + ICU_INT_CFG(i));  

Erm... Does it mean that non-secure can write to the configuration of
a secure interrupt? If that's the case, that's pretty... interesting.

I'll let Hannah and Yehuda answer this question, they know the ICU
details much better than I do.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html