Thread (38 messages) 38 messages, 5 authors, 2019-10-04

Re: [RFC PATCH] trace2: don't overload target directories

From: Jeff Hostetler <hidden>
Date: 2019-07-30 18:00:59


On 7/29/2019 6:20 PM, Josh Steadmon wrote:
trace2 can write files into a target directory. With heavy usage, this
directory can fill up with files, causing difficulty for
trace-processing systems.
I'm routing data in my org to a daemon via a Named Pipe or UD Socket,
so I'm not seeing the thousands of files problems that you're seeing.
However, we were being overwhelmed with lots of "uninteresting" commands
and so I added some whitelisting to my post-processing daemon.  For
example, I want to know about checkout and push times -- I really don't
care about rev-parse or config times or other such minor commands.

I went one step further and allow either "(cmd_name)" or
the pair "(cmd_name, cmd_mode)".  This lets me select all checkouts
and limit checkouts to branch-changing ones, for example.  I drop
any events in my post-processor that does not match any of my whitelist
patterns.

Perhaps you could run a quick histogram and see if something would
be useful to pre-filter the data.  That is, if we had whitelisting
within git.exe itself, would you still have too much data and/or
would you still need the overload feature that you've proposed in
this RFC?

Thanks,
Jeff
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help