Hello Moritz,

-----Original Message-----
From: Moritz Fischer <mdf@kernel.org>
Sent: Sunday, February 21, 2021 10:40 AM
To: Tom Rix <trix@redhat.com>
Cc: Lizhi Hou <redacted>; linux-kernel@vger.kernel.org;
mdf@kernel.org; Lizhi Hou [off-list ref]; linux-fpga@vger.kernel.org;
Max Zhen [off-list ref]; Sonal Santan [off-list ref]; Michal
Simek [off-list ref]; Stefano Stabellini [off-list ref];
devicetree@vger.kernel.org; robh@kernel.org; Max Zhen [off-list ref]
Subject: Re: [PATCH V3 XRT Alveo 18/18] fpga: xrt: Kconfig and Makefile
updates for XRT drivers

On Sun, Feb 21, 2021 at 06:57:31AM -0800, Tom Rix wrote:

quoted

As I am looking through the files, I have this comment.

fpga/ is currently a single directory, while files could be organized
in subdirectories like

dfl/pci.c

instead have the possible subdir name as a prefix to the filename.

dfl-pci.c

For consistency,

xrt/metadata/metadata.c

should be

xrt-metadata.c

Agreed. Keep the prefix.

quoted

Likewise the build infra needs to integrated within the existing files
fpga/Kconfig,Makefile

This is a bigish refactor, so let's get a second opinion.

In what sense? You mean adding a subdirectory? Maybe something like this

drivers/fpga
  - dfl/
  - xilinx/
  - intel/
  - lattice/
  - xrt/
  ...

would generally make sense.

We didn't have enough drivers to prioritize that yet, but we can look into it.

If longer term we would like to reorganize the drivers/fpga directory structure 
should we keep the current directory structure for XRT as in the patch series?

quoted

Moritz ?

On 2/17/21 10:40 PM, Lizhi Hou wrote:

quoted

Update fpga Kconfig/Makefile and add Kconfig/Makefile for new drivers.

Expand the comment, there are several new configs that could use an
explanation

quoted

Signed-off-by: Sonal Santan <redacted>
Signed-off-by: Max Zhen <redacted>
Signed-off-by: Lizhi Hou <redacted>
---
 MAINTAINERS                        | 11 +++++++++++
 drivers/Makefile                   |  1 +
 drivers/fpga/Kconfig               |  2 ++
 drivers/fpga/Makefile              |  4 ++++
 drivers/fpga/xrt/Kconfig           |  8 ++++++++
 drivers/fpga/xrt/lib/Kconfig       | 16 ++++++++++++++++
 drivers/fpga/xrt/lib/Makefile      | 30 ++++++++++++++++++++++++++++++
 drivers/fpga/xrt/metadata/Kconfig  | 12 ++++++++++++
drivers/fpga/xrt/metadata/Makefile | 16 ++++++++++++++++
 drivers/fpga/xrt/mgmt/Kconfig      | 15 +++++++++++++++
 drivers/fpga/xrt/mgmt/Makefile     | 19 +++++++++++++++++++
 11 files changed, 134 insertions(+)  create mode 100644
drivers/fpga/xrt/Kconfig  create mode 100644
drivers/fpga/xrt/lib/Kconfig  create mode 100644
drivers/fpga/xrt/lib/Makefile  create mode 100644
drivers/fpga/xrt/metadata/Kconfig  create mode 100644
drivers/fpga/xrt/metadata/Makefile
 create mode 100644 drivers/fpga/xrt/mgmt/Kconfig  create mode
100644 drivers/fpga/xrt/mgmt/Makefile

diff --git a/MAINTAINERS b/MAINTAINERS index

d3e847f7f3dc..e6e147c2454c 100644

--- a/MAINTAINERS
+++ b/MAINTAINERS

@@ -6973,6 +6973,17 @@ F:	Documentation/fpga/
 F:	drivers/fpga/
 F:	include/linux/fpga/

+FPGA XRT DRIVERS
+M:	Lizhi Hou <lizhi.hou@xilinx.com>
+R:	Max Zhen <max.zhen@xilinx.com>
+R:	Sonal Santan <sonal.santan@xilinx.com>
+L:	linux-fpga@vger.kernel.org
+S:	Maintained
+W:	https://github.com/Xilinx/XRT
+F:	Documentation/fpga/xrt.rst
+F:	drivers/fpga/xrt/
+F:	include/uapi/linux/xrt/
+
 FPU EMULATOR
 M:	Bill Metzenthen <billm@melbpc.org.au>
 S:	Maintained

diff --git a/drivers/Makefile b/drivers/Makefile index

fd11b9ac4cc3..e03912af8e48 100644

--- a/drivers/Makefile
+++ b/drivers/Makefile

@@ -178,6 +178,7 @@ obj-$(CONFIG_STM)		+= hwtracing/stm/
 obj-$(CONFIG_ANDROID)		+= android/
 obj-$(CONFIG_NVMEM)		+= nvmem/
 obj-$(CONFIG_FPGA)		+= fpga/
+obj-y				+= fpga/xrt/metadata/

This is wrong.

Move metadata building to fpga/ Makefile and pick an appropriate config,

not just 'obj-y'

Will update.

quoted

quoted

 obj-$(CONFIG_FSI)		+= fsi/
 obj-$(CONFIG_TEE)		+= tee/
 obj-$(CONFIG_MULTIPLEXER)	+= mux/

diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig index

5645226ca3ce..aeca635b1f25 100644

--- a/drivers/fpga/Kconfig
+++ b/drivers/fpga/Kconfig

@@ -216,4 +216,6 @@ config FPGA_MGR_ZYNQMP_FPGA
 	  to configure the programmable logic(PL) through PS
 	  on ZynqMP SoC.

+source "drivers/fpga/xrt/Kconfig"
+
 endif # FPGA

diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile index

d8e21dfc6778..2b4453ff7c52 100644

--- a/drivers/fpga/Makefile
+++ b/drivers/fpga/Makefile

@@ -46,3 +46,7 @@ dfl-afu-objs += dfl-afu-error.o

 # Drivers for FPGAs which implement DFL
 obj-$(CONFIG_FPGA_DFL_PCI)		+= dfl-pci.o
+
+# XRT drivers for Alveo
+obj-$(CONFIG_FPGA_XRT_LIB)		+= xrt/lib/
+obj-$(CONFIG_FPGA_XRT_XMGMT)		+= xrt/mgmt/

I don't see how mgmnt would work without lib.  If that is so

these configs could collapse to CONFIG_FPGA_XRT

quoted

diff --git a/drivers/fpga/xrt/Kconfig b/drivers/fpga/xrt/Kconfig new

file mode 100644 index 000000000000..0e2c59589ddd

--- /dev/null
+++ b/drivers/fpga/xrt/Kconfig

@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only # # Xilinx Alveo FPGA
+device configuration #
+
+source "drivers/fpga/xrt/metadata/Kconfig"
+source "drivers/fpga/xrt/lib/Kconfig"
+source "drivers/fpga/xrt/mgmt/Kconfig"

diff --git a/drivers/fpga/xrt/lib/Kconfig

b/drivers/fpga/xrt/lib/Kconfig new file mode 100644 index
000000000000..eed5cb73f5e2

--- /dev/null
+++ b/drivers/fpga/xrt/lib/Kconfig

@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0-only # # XRT Alveo FPGA device
+configuration #
+
+config FPGA_XRT_LIB
+	tristate "XRT Alveo Driver Library"
+	depends on HWMON && PCI && HAS_IOMEM
+	select FPGA_XRT_METADATA
+	help
+	  Select this option to enable Xilinx XRT Alveo driver library. This
+	  library is core infrastructure of XRT Alveo FPGA drivers which
+	  provides functions for working with device nodes, iteration and
+	  lookup of platform devices, common interfaces for platform devices,
+	  plumbing of function call and ioctls between platform devices and
+	  parent partitions.

diff --git a/drivers/fpga/xrt/lib/Makefile

b/drivers/fpga/xrt/lib/Makefile new file mode 100644 index
000000000000..5641231b2a36

--- /dev/null
+++ b/drivers/fpga/xrt/lib/Makefile

@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2020-2021 Xilinx, Inc. All rights reserved.
+#
+# Authors: Sonal.Santan@xilinx.com
+#
+
+FULL_XRT_PATH=$(srctree)/$(src)/..
+FULL_DTC_PATH=$(srctree)/scripts/dtc/libfdt
+
+obj-$(CONFIG_FPGA_XRT_LIB) += xrt-lib.o
+
+xrt-lib-objs :=			\
+	main.o			\
+	xroot.o			\
+	xclbin.o		\
+	subdev.o		\
+	cdev.o			\
+	group.o			\
+	xleaf/vsec.o		\
+	xleaf/axigate.o		\
+	xleaf/devctl.o		\
+	xleaf/icap.o		\
+	xleaf/clock.o		\
+	xleaf/clkfreq.o		\
+	xleaf/ucs.o		\
+	xleaf/calib.o		\
+
+ccflags-y := -I$(FULL_XRT_PATH)/include	 \
+	-I$(FULL_DTC_PATH)

diff --git a/drivers/fpga/xrt/metadata/Kconfig

b/drivers/fpga/xrt/metadata/Kconfig
new file mode 100644
index 000000000000..5012c9c6584d

--- /dev/null
+++ b/drivers/fpga/xrt/metadata/Kconfig

@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0-only # # XRT Alveo FPGA device
+configuration #
+
+config FPGA_XRT_METADATA
+	bool "XRT Alveo Driver Metadata Parser"
+	select LIBFDT
+	help
+	  This option provides helper functions to parse Xilinx Alveo FPGA
+	  firmware metadata. The metadata is in device tree format and XRT

and the XRT

quoted

+	  driver uses it to discover HW subsystems behind PCIe BAR.

the HW

quoted

diff --git a/drivers/fpga/xrt/metadata/Makefile

b/drivers/fpga/xrt/metadata/Makefile
new file mode 100644
index 000000000000..14f65ef1595c

--- /dev/null
+++ b/drivers/fpga/xrt/metadata/Makefile

@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2020-2021 Xilinx, Inc. All rights reserved.
+#
+# Authors: Sonal.Santan@xilinx.com
+#
+
+FULL_XRT_PATH=$(srctree)/$(src)/..
+FULL_DTC_PATH=$(srctree)/scripts/dtc/libfdt
+
+obj-$(CONFIG_FPGA_XRT_METADATA) += xrt-md.o
+
+xrt-md-objs := metadata.o
+
+ccflags-y := -I$(FULL_XRT_PATH)/include	\
+	-I$(FULL_DTC_PATH)

diff --git a/drivers/fpga/xrt/mgmt/Kconfig

b/drivers/fpga/xrt/mgmt/Kconfig new file mode 100644 index
000000000000..2b2a2c34685c

--- /dev/null
+++ b/drivers/fpga/xrt/mgmt/Kconfig

@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only # # Xilinx XRT FPGA device
+configuration #
+
+config FPGA_XRT_XMGMT
+	tristate "Xilinx Alveo Management Driver"
+	depends on HWMON && PCI && FPGA_XRT_LIB

FPGA_XRT_LIB also depends on HWMON and PCI, so this could be minimized.

Tom

quoted

+	select FPGA_XRT_METADATA
+	select FPGA_BRIDGE
+	select FPGA_REGION
+	help
+	  Select this option to enable XRT PCIe driver for Xilinx Alveo FPGA.
+	  This driver provides interfaces for userspace application to access
+	  Alveo FPGA device.

diff --git a/drivers/fpga/xrt/mgmt/Makefile

b/drivers/fpga/xrt/mgmt/Makefile new file mode 100644 index
000000000000..8051708c361c

--- /dev/null
+++ b/drivers/fpga/xrt/mgmt/Makefile

@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2020-2021 Xilinx, Inc. All rights reserved.
+#
+# Authors: Sonal.Santan@xilinx.com
+#
+
+FULL_XRT_PATH=$(srctree)/$(src)/..
+FULL_DTC_PATH=$(srctree)/scripts/dtc/libfdt
+
+obj-$(CONFIG_FPGA_XRT_XMGMT)	+= xmgmt.o
+
+xmgmt-objs := root.o		\
+	   main.o		\
+	   fmgr-drv.o		\
+	   main-region.o
+
+ccflags-y := -I$(FULL_XRT_PATH)/include		\
+	-I$(FULL_DTC_PATH)

- Moritz

-Sonal