On Wed, Jun 28, 2017 at 3:12 AM, Mark Rutland [off-list ref] wrote:

On Tue, Jun 27, 2017 at 09:51:00PM -0700, Kyle Huey wrote:

quoted

On Tue, Jun 27, 2017 at 7:09 PM, Jin, Yao [off-list ref] wrote:

quoted

Hi,

In theory, the PMI interrupts in skid region should be dropped, right?

No, why would they be dropped?

My understanding of the situation is as follows:

There is some time, call it t_0, where the hardware counter overflows.
The PMU triggers an interrupt, but this is not instantaneous.  Call
the time when the interrupt is actually delivered t_1.  Then t_1 - t_0
is the "skid".

Note that if the counter is `exclude_kernel`, then at t_0 the CPU
*must* be running a userspace program.  But by t_1, the CPU may be
doing something else.  Your patch changed things so that if at t_1 the
CPU is in the kernel, then the interrupt is discarded.  But rr has
programmed the counter to deliver a signal on overflow (via F_SETSIG
on the fd returned by perf_event_open).  This change results in the
signal never being delivered, because the interrupt was ignored.
(More accurately, the signal is delivered the *next* time the counter
overflows, which is far past where we wanted to inject our
asynchronous event into our tracee.

Yes, this is a bug.

As we're trying to avoid smapling state, I think we can move the check
into perf_prepare_sample() or __perf_event_output(), where that state is
actually sampled. I'll take a look at that momentarily.

Just to clarify, you don't care about the sample state at all? i.e. you
don't need the user program counter?

Right. `sample_regs_user`, `sample_star_user`, `branch_sample_type`,
etc are all 0.
https://github.com/mozilla/rr/blob/cf594dd01f07d96a61409e9f41a29f78c8c51693/src/PerfCounters.cc#L194
is what we do use.

Is that signal delivered to the tracee, or to a different process that
traces it? If the latter, what ensures that the task is stopped
sufficiently quickly?

It's delivered to the tracee (via an F_SETOWN_EX with the tracee tid).
In practice we've found that on modern Intel hardware that the
interrupt and resulting signal delivery delay is bounded by a
relatively small number of counter events.

quoted

It seems to me that it might be reasonable to ignore the interrupt if
the purpose of the interrupt is to trigger sampling of the CPUs
register state.  But if the interrupt will trigger some other
operation, such as a signal on an fd, then there's no reason to drop
it.

Agreed. I'll try to have a patch for this soon.

I just need to figure out exactly where that overflow signal is
generated by the perf core.

Thanks,
Mark.

- Kyle