On Thu, Oct 07, 2021 at 06:17:29AM -0300, Arnaldo Carvalho de Melo wrote:

On October 6, 2021 6:25:57 PM GMT-03:00, Renjith Ponnappan [off-list ref] wrote:

quoted

Hello Arnaldo & Jiri,

Thank you for your response.

The perf daemon is the closest implementation for what I was looking for.
Here we run perf in the back-ground, keep overwriting the samples being
collected and use the SIGUSR2 to signal to the perf daemon to collect the
perf data to a file. This fulfills the following requirements:

  1. Run perf in the background to collect data
  2. A method to signal perf to collect the samples for the current cycle.

The last part of the requirement which  was:

  1. The data-collection in step 2 above, rather than writing the data to
  file store it in in-memory datastructure via pointer manipulation. We can
  have a list of such samples stored in memory until the next step. This
  helps free up the CPU cycles used by perf for writing to file for
  applications.
  2. A method to signal perf to dump all the collected samples into
  separate files. This way the user can collect the stored samples when the
  CPU is relatively free.

Let me know whether we have support for storing samples as in-memory
samples.

Have you looked at the URL I pointed to you?

https://www.spinics.net/lists/linux-perf-users/msg11455.html

More specifically, read about 'perf record --overwrite"

also there's example with flight recorder in perf-daemon man page:
https://man7.org/linux/man-pages/man1/perf-daemon.1.html

jirka

- Arnaldo

quoted

Thanks in advance for your help!

*cheers,*
*Renjith*


On Tue, Oct 5, 2021 at 12:00 AM Jiri Olsa [off-list ref] wrote:

quoted

On Thu, Sep 30, 2021 at 10:39:21PM -0300, Arnaldo Carvalho de Melo wrote:

quoted

On September 30, 2021 10:28:28 PM GMT-03:00, Renjith Ponnappan <

renjithponnapps@gmail.com> wrote:

quoted

quoted

Hello Peter/Ingo/Arnaldo,

First of all, apologies if I bombarded you with an irrelevant question

in

quoted

quoted

your busy day and ignore this if the question is irrelevant.

I had a question about continuous background data-collection with perf

and

quoted

quoted

hope you are the right person to answer this. If not, it would be great

if

quoted

quoted

you can redirect me to the right person.

I am trying to build a CPU profiling system (on an embedded ARM Platform
with CPU/memory constraints) which has CPU Samples already collected

when

quoted

quoted

an application CPU starvation scenario occurs based on perf. The
implementation I am trying to use is:

  1. Run perf in the background collecting samples for the entire

system

quoted

This is already in perf:

https://www.spinics.net/lists/linux-perf-users/msg11455.html


Reply adding linux-perf-users@vger.kernel.org.

- Arnaldo

quoted

  with a sleep period of 60 seconds
  2. When an application CPU starvation scenario occurs (detected and
  raised by applications) notify the collection process to store the

last

quoted

quoted

  perf collection as data to be analyzed offline.

Have you come across such a scenario and any recommendations on this?

The following are the two implementations I have on the above:

  1. An external process which instructs perf to record the 60 seconds

by

quoted

quoted

  providing unique filenames each time. This approach was taking

around 40%

quoted

quoted

  CPU of a CPU core, everytime the perf record was getting written

(once for

quoted

quoted

  each 60 seconds cycle). This isn't okay as it could cause

aggravation of

quoted

quoted

  the CPU starvation situation.
  2. I tinkered with Perf Code to add the logic of looping and writing

the

quoted

quoted

  file incase of an event only. This did reduce the CPU to only the

case when

quoted

quoted

  an event was detected.

hi,
and there's also perf daemon to run perf sessions on background:
https://lore.kernel.org/lkml/20210130234856.271282-19-jolsa@kernel.org/ (local)

jirka

quoted

quoted

Would like to hear your opinion on whether approach 2 is the right way

here

quoted

quoted

and any suggestion/guidance you may have.

Thanks in advance for this help!

*cheers,*
*Renjith*