On Mon, 2016-06-13 at 14:35 -0500, atull wrote:

quoted

quoted

+
+	/* Allow bridge to be visible to L3 masters or not */
+	if (priv->remap_mask) {
+		priv->l3_remap_value |= ALT_L3_REMAP_MPUZERO_MSK;

Doesn't seem like this belongs here.  I realize the write-only register
is a problem.  Maybe the syscon driver should be initializing this
value?

quoted

+
+		if (enable)
+			priv->l3_remap_value |= priv->remap_mask;
+		else
+			priv->l3_remap_value &= ~priv->remap_mask;
+
+		ret = regmap_write(priv->l3reg, ALT_L3_REMAP_OFST,
+				   priv->l3_remap_value);

This isn't going work if more than one bridge is used.  Each bridge has
its own priv and thus priv->l3_remap_value.  Each bridge's priv will
have just the bit for it's own remap set.  The 2nd bridge to be enabled
will turn off the 1st bridge when it re-write the l3 register.

If all the bridges shared a static global to cache the reg, then this
problem would be a replaced by a race, since nothing would be managing
concurrent access to that global from the independent bridge devices.

How about using the already existing regmap cache ability take care of
this?  Use regmap_update_bits() to update just the desired bit and let
remap take care of keeping track caching the register and protecting
access from multiple users.  It should support that and it should
support write-only registers, with the creator of the regmap (the syscon
driver in this case) supplying the initial value of the write-only reg.
Which is where ALT_L3_REMAP_MPUZERO_MSK could go in.

Please correct me if I'm wrong, but I think that regmap supports
the features you are talking about, but not syscon.

From my testing, it will work ok if the syscon driver were to set
syscon_config.cache_type one of the caches.  Since the l3 regs read back
as 0, rather than not being readable at all, making them write-only and
giving a default isn't strictly necessary.

It wouldn't be hard to add a write-only property to syscon.

One simple solution would be to take l3_remap_value out of the priv
and let it be shared by all h2f bridges.  That involves the least
amount of change.

You'll need a spin-lock to protect against concurrent access.

quoted

quoted

+
+static int alt_hps2fpga_enable_set(struct fpga_bridge *bridge, bool enable)
+{
+	return _alt_hps2fpga_enable_set(bridge->priv, enable);
+}
+
+static const struct fpga_bridge_ops altera_hps2fpga_br_ops = {
+	.enable_set = alt_hps2fpga_enable_set,
+	.enable_show = alt_hps2fpga_enable_show,
+};
+
+static struct altera_hps2fpga_data hps2fpga_data  = {
+	.name = HPS2FPGA_BRIDGE_NAME,
+	.remap_mask = ALT_L3_REMAP_H2F_MSK,
+};

Each of these data structs also includes space for all the private data
field of the drivers' state.  Seems a bit inefficient if only two of
them are configuration data.  It also means only one device of each type
can exists.  If one creates two bridges of the same type they'll
(silently) share a priv data struct and randomly break.  And the config
data structs can't be const.

Our hardware doesn't contain two devices of any of these three types.

Not yet, but doesn't Arria10 have three FPGA2SDRAM bridges?  But really,
it's just that I think having each device allocated state data for just
that device is more in line with the Linux device model than shared
static state pre-allocated for all devices of the same type to share.
 

quoted

What if these structs were a different altera_hps_config struct, which
the private data struct could then copy or point to?

struct altera_hpsbridge_config {
	const char *name;
	uint32_t remap_mask;
};

struct altera_hpsbridge_data {
	const struct altera_hpsbridge_config *config;
	...;
	struct clk *clk;
};

Yes, that sounds good and sensible.  I'll do that in v18.

Another thought, if the "altr,l3regs" binding included not just the
phandle to the l3regs device, but also the bit associated with the
bridge, then there wouldn't need to be bridge specific configuration for
the driver.  The same way the reset and clock properties point not just
to the reset and clock controller, but to the bit in the controller.

quoted

quoted

+
+static struct altera_hps2fpga_data lwhps2fpga_data  = {
+	.name = LWHPS2FPGA_BRIDGE_NAME,
+	.remap_mask = ALT_L3_REMAP_LWH2F_MSK,
+};
+
+static struct altera_hps2fpga_data fpga2hps_data  = {
+	.name = FPGA2HPS_BRIDGE_NAME,
+};
+
+static const struct of_device_id altera_fpga_of_match[] = {
+	{ .compatible = "altr,socfpga-hps2fpga-bridge",
+	  .data = &hps2fpga_data },
+	{ .compatible = "altr,socfpga-lwhps2fpga-bridge",
+	  .data = &lwhps2fpga_data },
+	{ .compatible = "altr,socfpga-fpga2hps-bridge",
+	  .data = &fpga2hps_data },
+	{},
+};
+
+static int alt_fpga_bridge_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct altera_hps2fpga_data *priv;
+	const struct of_device_id *of_id;
+	u32 enable;
+	int ret;
+
+	of_id = of_match_device(altera_fpga_of_match, dev);
+	priv = (struct altera_hps2fpga_data *)of_id->data;
+
+	priv->bridge_reset = devm_reset_control_get(dev, priv->name);
+	if (IS_ERR(priv->bridge_reset)) {
+		dev_err(dev, "Could not get %s reset control\n", priv->name);
+		return PTR_ERR(priv->bridge_reset);
+	}
+

quoted

+	priv->l3reg = syscon_regmap_lookup_by_compatible("altr,l3regs");
+	if (IS_ERR(priv->l3reg)) {
+		dev_err(dev, "regmap for altr,l3regs lookup failed\n");
+		return PTR_ERR(priv->l3reg);
+	}

Perhaps this could be wrapped in if(priv->remap_mask) { }.  The fpga2hps
bridge has no bits in the l3 remap register, so why should it need a
phandle to the l3 syscon?  This also prevents this driver from working
on Arria10, since it has no l3remap register at all, for any of the
bridges, so there's nothing for the phandle to point to.

Agreed.

quoted

quoted

+
+	priv->clk = of_clk_get(dev->of_node, 0);
+	if (IS_ERR(priv->clk)) {
+		dev_err(dev, "no clock specified\n");
+		return PTR_ERR(priv->clk);
+	}

devm_clk_get(dev, NULL); should get the 1st clock in the OF node, but
use the dev resource manager, so it doesn't need to be put.

Yes

quoted

quoted

+
+	ret = clk_prepare_enable(priv->clk);
+	if (ret) {
+		dev_err(dev, "could not enable clock\n");
+		return -EBUSY;

clk_put() on clk missing here and also the other error returns.

I'll use devm_clk_get() so that won't be needed.

quoted

quoted

+	}
+
+	ret = fpga_bridge_register(dev, priv->name, &altera_hps2fpga_br_ops,
+				   priv);
+	if (ret)
+		return ret;
+
+	if (!of_property_read_u32(dev->of_node, "bridge-enable", &enable)) {
+		if (enable > 1) {
+			dev_warn(dev, "invalid bridge-enable %u > 1\n", enable);
+		} else {
+			dev_info(dev, "%s bridge\n",
+				 (enable ? "enabling" : "disabling"));
+
+			ret = _alt_hps2fpga_enable_set(priv, enable);

Should this go through the bridge api, e.g. fpga_bridge_enable()?  Since
the bridge has already been registered.  Or is the bridge framework
supposed to be able to support bridges that might be enabled or disabled
behind its back?  If so, then isn't there a race here with
_alt_hps2fpga_enable_set() possible being called at the same time as
other operations on this bridge triggered by the code in fpga-bridge.c?

Alternatively, could the bridge be enabled or disabled before being
registered?

I'll do the enabling/disabling before registering the driver.

Thanks for your code review!

Alan