Re: [PATCH v9 13/27] cxl: prepare memdev creation for type2
flat view
From: Dan Williams <hidden>
Date: 2025-01-18 02:28:04
Also in:
linux-cxl
alejandro.lucero-palau@ wrote:
quoted hunk ↗ jump to hunk
From: Alejandro Lucero <redacted> Current cxl core is relying on a CXL_DEVTYPE_CLASSMEM type device when creating a memdev leading to problems when obtaining cxl_memdev_state references from a CXL_DEVTYPE_DEVMEM type. This last device type is managed by a specific vendor driver and does not need same sysfs files since not userspace intervention is expected. Create a new cxl_mem device type with no attributes for Type2. Avoid debugfs files relying on existence of cxl_memdev_state. Make devm_cxl_add_memdev accesible from a accel driver. Signed-off-by: Alejandro Lucero <redacted> --- drivers/cxl/core/cdat.c | 3 +++ drivers/cxl/core/memdev.c | 14 ++++++++++++-- drivers/cxl/core/region.c | 3 ++- drivers/cxl/cxlmem.h | 2 -- drivers/cxl/mem.c | 25 +++++++++++++++++++------ include/cxl/cxl.h | 2 ++ 6 files changed, 38 insertions(+), 11 deletions(-)diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index 8153f8d83a16..c57bc83e79ee 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c@@ -577,6 +577,9 @@ static struct cxl_dpa_perf *cxled_get_dpa_perf(struct cxl_endpoint_decoder *cxle struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); struct cxl_dpa_perf *perf; + if (!mds) + return ERR_PTR(-EINVAL); +
The references to @mds in cxled_get_dpa_perf() are gone after the DPA partition changes. If an accelerator has a CDAT it will get qos_class information for free. If it does not have a CDAT then I wonder how it is telling the BIOS the memory type for its CXL.mem?
quoted hunk ↗ jump to hunk
switch (mode) { case CXL_DECODER_RAM: perf = &mds->ram_perf;diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 836db4a462b3..f91feca586dd 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c@@ -547,9 +547,16 @@ static const struct device_type cxl_memdev_type = { .groups = cxl_memdev_attribute_groups, }; +static const struct device_type cxl_accel_memdev_type = { + .name = "cxl_accel_memdev", + .release = cxl_memdev_release, + .devnode = cxl_memdev_devnode, +}; + bool is_cxl_memdev(const struct device *dev) { - return dev->type == &cxl_memdev_type; + return (dev->type == &cxl_memdev_type || + dev->type == &cxl_accel_memdev_type);
At this point the game is over in terms of future code that trips over the assumption that all "is_cxl_memdev() == true" devices are created equal.
quoted hunk ↗ jump to hunk
} EXPORT_SYMBOL_NS_GPL(is_cxl_memdev, "CXL");@@ -660,7 +667,10 @@ static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds, dev->parent = cxlds->dev; dev->bus = &cxl_bus_type; dev->devt = MKDEV(cxl_mem_major, cxlmd->id); - dev->type = &cxl_memdev_type; + if (cxlds->type == CXL_DEVTYPE_DEVMEM) + dev->type = &cxl_accel_memdev_type; + else + dev->type = &cxl_memdev_type; device_set_pm_not_required(dev); INIT_WORK(&cxlmd->detach_work, detach_memdev);diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index d77899650798..967132b49832 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c@@ -1948,7 +1948,8 @@ static int cxl_region_attach(struct cxl_region *cxlr, return -EINVAL; } - cxl_region_perf_data_calculate(cxlr, cxled); + if (cxlr->type == CXL_DECODER_HOSTONLYMEM) + cxl_region_perf_data_calculate(cxlr, cxled);
Per-above no need to worry about @mds reference in this path.
quoted hunk ↗ jump to hunk
if (test_bit(CXL_REGION_F_AUTO, &cxlr->flags)) { int i;diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 4c1c53c29544..360d3728f492 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h@@ -87,8 +87,6 @@ static inline bool is_cxl_endpoint(struct cxl_port *port) return is_cxl_memdev(port->uport_dev); } -struct cxl_memdev *devm_cxl_add_memdev(struct device *host, - struct cxl_dev_state *cxlds); int devm_cxl_sanitize_setup_notifier(struct device *host, struct cxl_memdev *cxlmd); struct cxl_memdev_state;diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 2f03a4d5606e..93106a43990b 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c@@ -130,12 +130,18 @@ static int cxl_mem_probe(struct device *dev) dentry = cxl_debugfs_create_dir(dev_name(dev)); debugfs_create_devm_seqfile(dev, "dpamem", dentry, cxl_mem_dpa_show); - if (test_bit(CXL_POISON_ENABLED_INJECT, mds->poison.enabled_cmds)) - debugfs_create_file("inject_poison", 0200, dentry, cxlmd, - &cxl_poison_inject_fops); - if (test_bit(CXL_POISON_ENABLED_CLEAR, mds->poison.enabled_cmds)) - debugfs_create_file("clear_poison", 0200, dentry, cxlmd, - &cxl_poison_clear_fops); + /* + * Avoid poison debugfs files for Type2 devices as they rely on + * cxl_memdev_state. + */ + if (mds) { + if (test_bit(CXL_POISON_ENABLED_INJECT, mds->poison.enabled_cmds)) + debugfs_create_file("inject_poison", 0200, dentry, cxlmd, + &cxl_poison_inject_fops); + if (test_bit(CXL_POISON_ENABLED_CLEAR, mds->poison.enabled_cmds)) + debugfs_create_file("clear_poison", 0200, dentry, cxlmd, + &cxl_poison_clear_fops); + } rc = devm_add_action_or_reset(dev, remove_debugfs, dentry); if (rc)@@ -219,6 +225,13 @@ static umode_t cxl_mem_visible(struct kobject *kobj, struct attribute *a, int n) struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); + /* + * Avoid poison sysfs files for Type2 devices as they rely on + * cxl_memdev_state. + */ + if (!mds) + return 0;
It turns out that nothing in the poison handling path requires 'struct cxl_memdev_state'. So rather than key this rejection off of @mds == NULL, I would prefer to find a way to make this optional relative to data read from 'struct cxl_mailbox', or 'struct cxl_dev_state'. Conceptually there is nothing stopping a CXL accelertor from supporting poison management commands on its mailbox, it's just unlikely.