From: James Clark <james.clark@linaro.org> Date: 2024-12-17 11:56:21
Discard mode is a way to enable SPE related PMU events without the
overhead of recording any data. Add a format option, tests and docs for
it.
In theory we could make the driver drop calls to allocate the aux buffer
when discard mode is enabled. This would give a small memory saving,
but I think there is potential to interfere with any tools that don't
expect this so I left the aux allocation untouched. Even old tools that
don't know about discard mode will be able to use it because we publish
the format option. Not allocating the aux buffer will have to be added
to tools which I've done in Perf.
Tested on the FVP with SAMPLE_FEED_OP (0x812D):
$ perf stat -e armv8_pmuv3/event=0x812D/ -- true
Performance counter stats for 'true':
0 armv8_pmuv3/event=0x812D/
$ perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null &
$ perf stat -e armv8_pmuv3/event=0x812D/ -- true
Performance counter stats for 'true':
17350 armv8_pmuv3/event=0x812D/
James Clark (5):
perf: arm_spe: Add format option for discard mode
perf tool: arm-spe: Pull out functions for aux buffer and tracking
setup
perf tool: arm-spe: Don't allocate buffer or tracking event in discard
mode
perf test: arm_spe: Add test for discard mode
perf docs: arm_spe: Document new discard mode
drivers/perf/arm_spe_pmu.c | 23 ++++++
tools/perf/Documentation/perf-arm-spe.txt | 11 +++
tools/perf/arch/arm64/util/arm-spe.c | 90 +++++++++++++++--------
tools/perf/tests/shell/test_arm_spe.sh | 30 ++++++++
4 files changed, 122 insertions(+), 32 deletions(-)
--
2.34.1
From: James Clark <james.clark@linaro.org> Date: 2024-12-17 11:56:26
FEAT_SPEv1p2 (optional from Armv8.6) adds a discard mode that allows all
SPE data to be discarded rather than written to memory. Add a format
bit for this mode.
If the mode isn't supported, the format bit isn't published and attempts
to use it will result in -EOPNOTSUPP. Allocating an aux buffer is still
allowed even though it won't be written to so that old tools continue to
work, but updated tools can choose to skip this step.
Signed-off-by: James Clark <james.clark@linaro.org>
---
drivers/perf/arm_spe_pmu.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
@@ -502,6 +515,12 @@ static void arm_spe_perf_aux_output_begin(struct perf_output_handle *handle,u64base,limit;structarm_spe_pmu_buf*buf;+if(ATTR_CFG_GET_FLD(&event->attr,discard)){+limit=FIELD_PREP(PMBLIMITR_EL1_FM,PMBLIMITR_EL1_FM_DISCARD);+limit|=PMBLIMITR_EL1_E;+gotoout_write_limit;+}+/* Start a new aux session */buf=perf_aux_output_begin(handle,event);if(!buf){
@@ -743,6 +762,10 @@ static int arm_spe_pmu_event_init(struct perf_event *event)!(spe_pmu->features&SPE_PMU_FEAT_FILT_LAT))return-EOPNOTSUPP;+if(ATTR_CFG_GET_FLD(&event->attr,discard)&&+discard_unsupported(spe_pmu))+return-EOPNOTSUPP;+set_spe_event_has_cx(event);reg=arm_spe_event_to_pmscr(event);if(reg&(PMSCR_EL1_PA|PMSCR_EL1_PCT))
From: James Clark <james.clark@linaro.org> Date: 2024-12-17 11:56:30
These won't be used in the next commit in discard mode, so put them in
their own functions. No functional changes intended.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/arch/arm64/util/arm-spe.c | 83 +++++++++++++++++-----------
1 file changed, 51 insertions(+), 32 deletions(-)
@@ -330,6 +306,9 @@ static int arm_spe_recording_options(struct auxtrace_record *itr,pr_err("Failed to calculate default snapshot size and/or AUX area tracing mmap pages\n");return-EINVAL;}++pr_debug2("%sx snapshot size: %zu\n",ARM_SPE_PMU_NAME,+opts->auxtrace_snapshot_size);}/* We are in full trace mode but '-m,xyz' wasn't specified */
@@ -355,14 +334,15 @@ static int arm_spe_recording_options(struct auxtrace_record *itr,}}-if(opts->auxtrace_snapshot_mode)-pr_debug2("%sx snapshot size: %zu\n",ARM_SPE_PMU_NAME,-opts->auxtrace_snapshot_size);+return0;+}-evlist__for_each_entry_safe(evlist,tmp,evsel){-if(evsel__is_aux_event(evsel))-arm_spe_setup_evsel(evsel,cpus);-}+staticintarm_spe_setup_tracking_event(structevlist*evlist,+structrecord_opts*opts)+{+interr;+structevsel*tracking_evsel;+structperf_cpu_map*cpus=evlist->core.user_requested_cpus;/* Add dummy event to keep tracking */err=parse_event(evlist,"dummy:u");
From: James Clark <james.clark@linaro.org> Date: 2024-12-17 11:56:34
The buffer will never be written to so don't bother allocating it. The
tracking event is also not required.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/arch/arm64/util/arm-spe.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
From: James Clark <james.clark@linaro.org> Date: 2024-12-17 11:56:38
Add a test that checks that there were no AUX or AUXTRACE events
recorded when discard mode is used.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/tests/shell/test_arm_spe.sh | 30 ++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
@@ -107,7 +107,37 @@ arm_spe_system_wide_test() {arm_spe_report"SPE system-wide testing"$err}+arm_spe_discard_test(){+echo"SPE discard mode"++forfin/sys/bus/event_source/devices/arm_spe_*;do+if[-e"$f/format/discard"];then+cpu=$(cut-c-1"$f/cpumask")+break+fi+done++if[-z$cpu];then+arm_spe_report"SPE discard mode not present"2+return+fi++# Test can use wildcard SPE instance and Perf will only open the event+# on instances that have that format flag. But make sure the target+# runs on an instance with discard mode otherwise we're not testing+# anything.+perfrecord-o${perfdata}-earm_spe/discard/-N-B--no-bpf-event\+--taskset--cpu-list$cputrue++ifperfreport-i${perfdata}--stats|grep'AUX events\|AUXTRACE events';then+arm_spe_report"SPE discard mode found unexpected data"1+else+arm_spe_report"SPE discard mode"0+fi+}+ arm_spe_snapshot_test arm_spe_system_wide_test+arm_spe_discard_testexit$glb_err
From: James Clark <james.clark@linaro.org> Date: 2024-12-17 11:56:42
Document the flag, hint what it's used for and give an example with
other useful options to get minimal output.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/Documentation/perf-arm-spe.txt | 11 +++++++++++
1 file changed, 11 insertions(+)
@@ -150,6 +150,7 @@ arm_spe/load_filter=1,min_latency=10/' pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS)+ discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency.
@@ -220,6 +221,16 @@ Common errors Increase sampling interval (see above)+Discard mode+~~~~~~~~~~~~++SPE PMU events can be used without the overhead of collecting sample data if+discard mode is supported (optional from Armv8.6). First run a system wide SPE+session (or on the core of interest) using options to minimize output. Then run+perf stat:++ perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null &+ perf stat -e SAMPLE_FEED_LD SEE ALSO --------
From: Ian Rogers <irogers@google.com> Date: 2024-12-18 00:54:32
On Tue, Dec 17, 2024 at 3:56 AM James Clark [off-list ref] wrote:
quoted hunk
Document the flag, hint what it's used for and give an example with
other useful options to get minimal output.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/Documentation/perf-arm-spe.txt | 11 +++++++++++
1 file changed, 11 insertions(+)
@@ -150,6 +150,7 @@ arm_spe/load_filter=1,min_latency=10/' pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS)+ discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency.
@@ -220,6 +221,16 @@ Common errors Increase sampling interval (see above)+Discard mode+~~~~~~~~~~~~++SPE PMU events can be used without the overhead of collecting sample data if+discard mode is supported (optional from Armv8.6). First run a system wide SPE+session (or on the core of interest) using options to minimize output. Then run+perf stat:++ perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null &+ perf stat -e SAMPLE_FEED_LD
Perhaps clarify this should be an ARM SPE event? It seems strange to
have one perf command affect a later one, the purpose of things like
event multiplexing is to hide the hardware limits. I'd prefer if the
last bit was like:
From: James Clark <james.clark@linaro.org> Date: 2024-12-18 10:07:49
On 18/12/2024 12:54 am, Ian Rogers wrote:
On Tue, Dec 17, 2024 at 3:56 AM James Clark [off-list ref] wrote:
quoted
Document the flag, hint what it's used for and give an example with
other useful options to get minimal output.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/Documentation/perf-arm-spe.txt | 11 +++++++++++
1 file changed, 11 insertions(+)
@@ -150,6 +150,7 @@ arm_spe/load_filter=1,min_latency=10/' pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS)+ discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency.
@@ -220,6 +221,16 @@ Common errors Increase sampling interval (see above)+Discard mode+~~~~~~~~~~~~++SPE PMU events can be used without the overhead of collecting sample data if+discard mode is supported (optional from Armv8.6). First run a system wide SPE+session (or on the core of interest) using options to minimize output. Then run+perf stat:++ perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null &+ perf stat -e SAMPLE_FEED_LD
Perhaps clarify this should be an ARM SPE event? It seems strange to
have one perf command affect a later one, the purpose of things like
event multiplexing is to hide the hardware limits. I'd prefer if the
last bit was like:
Hi Ian,
Confusingly this isn't an SPE event, it is a normal PMU event. The fact that one Perf command affects the other is because these events only count when SPE is enabled. When it's enabled it has an effect on a per-core level which is why in the example I made it simpler by enabling SPE system wide.
SPE is an exclusive PMU like Coresight and some others so it can't be affected by multiplexing or anything like that. The SAMPLE_FEED_LD PMU would be, but as long as SPE stays enabled it will count the right thing regardless of multiplexing.
THanks
James
This patch series looks good to me.
Reviewed-by: Yeoreum Yun <redacted>
________________________________________
From: James Clark <james.clark@linaro.org>
Sent: 17 December 2024 11:56
To: linux-arm-kernel@lists.infradead.org; linux-perf-users@vger.kernel.org
Cc: James Clark; Will Deacon; Mark Rutland; Peter Zijlstra; Ingo Molnar; Arnaldo Carvalho de Melo; Namhyung Kim; Alexander Shishkin; Jiri Olsa; Ian Rogers; Adrian Hunter; Liang, Kan; John Garry; Mike Leach; Leo Yan; Graham Woodward; linux-kernel@vger.kernel.org; bpf@vger.kernel.org
Subject: [PATCH 0/5] perf: arm_spe: Add format option for discard mode
Discard mode is a way to enable SPE related PMU events without the
overhead of recording any data. Add a format option, tests and docs for
it.
In theory we could make the driver drop calls to allocate the aux buffer
when discard mode is enabled. This would give a small memory saving,
but I think there is potential to interfere with any tools that don't
expect this so I left the aux allocation untouched. Even old tools that
don't know about discard mode will be able to use it because we publish
the format option. Not allocating the aux buffer will have to be added
to tools which I've done in Perf.
Tested on the FVP with SAMPLE_FEED_OP (0x812D):
$ perf stat -e armv8_pmuv3/event=0x812D/ -- true
Performance counter stats for 'true':
0 armv8_pmuv3/event=0x812D/
$ perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null &
$ perf stat -e armv8_pmuv3/event=0x812D/ -- true
Performance counter stats for 'true':
17350 armv8_pmuv3/event=0x812D/
James Clark (5):
perf: arm_spe: Add format option for discard mode
perf tool: arm-spe: Pull out functions for aux buffer and tracking
setup
perf tool: arm-spe: Don't allocate buffer or tracking event in discard
mode
perf test: arm_spe: Add test for discard mode
perf docs: arm_spe: Document new discard mode
drivers/perf/arm_spe_pmu.c | 23 ++++++
tools/perf/Documentation/perf-arm-spe.txt | 11 +++
tools/perf/arch/arm64/util/arm-spe.c | 90 +++++++++++++++--------
tools/perf/tests/shell/test_arm_spe.sh | 30 ++++++++
4 files changed, 122 insertions(+), 32 deletions(-)
--
2.34.1
From: Ian Rogers <irogers@google.com> Date: 2024-12-18 19:47:21
On Wed, Dec 18, 2024 at 2:07 AM James Clark [off-list ref] wrote:
On 18/12/2024 12:54 am, Ian Rogers wrote:
quoted
On Tue, Dec 17, 2024 at 3:56 AM James Clark [off-list ref] wrote:
quoted
Document the flag, hint what it's used for and give an example with
other useful options to get minimal output.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/Documentation/perf-arm-spe.txt | 11 +++++++++++
1 file changed, 11 insertions(+)
@@ -150,6 +150,7 @@ arm_spe/load_filter=1,min_latency=10/' pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS)+ discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency.
@@ -220,6 +221,16 @@ Common errors Increase sampling interval (see above)+Discard mode+~~~~~~~~~~~~++SPE PMU events can be used without the overhead of collecting sample data if+discard mode is supported (optional from Armv8.6). First run a system wide SPE+session (or on the core of interest) using options to minimize output. Then run+perf stat:++ perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null &+ perf stat -e SAMPLE_FEED_LD
Perhaps clarify this should be an ARM SPE event? It seems strange to
have one perf command affect a later one, the purpose of things like
event multiplexing is to hide the hardware limits. I'd prefer if the
last bit was like:
Hi Ian,
Confusingly this isn't an SPE event, it is a normal PMU event. The fact
that one Perf command affects the other is because these events only
count when SPE is enabled. When it's enabled it has an effect on a
per-core level which is why in the example I made it simpler by enabling
SPE system wide.
SPE is an exclusive PMU like Coresight and some others so it can't be
affected by multiplexing or anything like that. The SAMPLE_FEED_LD PMU
would be, but as long as SPE stays enabled it will count the right thing
regardless of multiplexing.
Thanks James, sorry for my SPE ignorance. I'm smiling about the use of
the word exclusive. When I was trying to make the tests run in
parallel I used a file lock - so shared and exclusive. There were a
lot of issues with that, hence switching to 2 phases in the test,
parallel then sequential but I kept the "exclusive" tag for want of a
better word. Perhaps the notion of an exclusive PMU existed previously
but maybe I've accidentally invented the term by way of a failed file
lock experiment :-)
Presumably the two PMUs side-effecting each other is a known thing. I
wonder if we can capture this in the documentation. When you say
"normal PMU event" you mean core PMU events?
Thanks,
Ian
From: James Clark <james.clark@linaro.org> Date: 2024-12-19 10:11:02
On 18/12/2024 7:47 pm, Ian Rogers wrote:
On Wed, Dec 18, 2024 at 2:07 AM James Clark [off-list ref] wrote:
quoted
On 18/12/2024 12:54 am, Ian Rogers wrote:
quoted
On Tue, Dec 17, 2024 at 3:56 AM James Clark [off-list ref] wrote:
quoted
Document the flag, hint what it's used for and give an example with
other useful options to get minimal output.
Signed-off-by: James Clark <james.clark@linaro.org>
---
tools/perf/Documentation/perf-arm-spe.txt | 11 +++++++++++
1 file changed, 11 insertions(+)
@@ -150,6 +150,7 @@ arm_spe/load_filter=1,min_latency=10/' pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS)+ discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency.
@@ -220,6 +221,16 @@ Common errors Increase sampling interval (see above)+Discard mode+~~~~~~~~~~~~++SPE PMU events can be used without the overhead of collecting sample data if+discard mode is supported (optional from Armv8.6). First run a system wide SPE+session (or on the core of interest) using options to minimize output. Then run+perf stat:++ perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null &+ perf stat -e SAMPLE_FEED_LD
Perhaps clarify this should be an ARM SPE event? It seems strange to
have one perf command affect a later one, the purpose of things like
event multiplexing is to hide the hardware limits. I'd prefer if the
last bit was like:
Hi Ian,
Confusingly this isn't an SPE event, it is a normal PMU event. The fact
that one Perf command affects the other is because these events only
count when SPE is enabled. When it's enabled it has an effect on a
per-core level which is why in the example I made it simpler by enabling
SPE system wide.
SPE is an exclusive PMU like Coresight and some others so it can't be
affected by multiplexing or anything like that. The SAMPLE_FEED_LD PMU
would be, but as long as SPE stays enabled it will count the right thing
regardless of multiplexing.
Thanks James, sorry for my SPE ignorance. I'm smiling about the use of
the word exclusive. When I was trying to make the tests run in
parallel I used a file lock - so shared and exclusive. There were a
lot of issues with that, hence switching to 2 phases in the test,
parallel then sequential but I kept the "exclusive" tag for want of a
better word. Perhaps the notion of an exclusive PMU existed previously
Yeah, see PERF_PMU_CAP_EXCLUSIVE. Hopefully it doesn't cause too much confusion, the context of test vs PMU should make it clear.
but maybe I've accidentally invented the term by way of a failed file
lock experiment :-)
Presumably the two PMUs side-effecting each other is a known thing. I
wonder if we can capture this in the documentation. When you say
"normal PMU event" you mean core PMU events?
Thanks,
Ian
It should be a known thing yes, discard mode doesn't change this behavior anyway but just makes one use case of it better. I can add another section to this SPE manpage about it in a v2, that's probably the best place for it.
And yes, I meant core PMU event. I can clarify that the second example command is for a core PMU to avoid any doubt.
Thanks
James