On Mon, Feb 08, 2021 at 10:02:06AM -0500, Steven Rostedt wrote:
On Sun, 7 Feb 2021 16:45:40 -0600
Josh Poimboeuf [off-list ref] wrote:
quoted
quoted
I do suspect involved people should start thinking about how they want
to deal with functions starting with
endbr64
call __fentry__
instead of the call being at the very top of the function.
FWIW, objtool's already fine with it (otherwise we would have discovered
the need to disable fcf-protection much sooner).
And this doesn't really affect tracing (note, another user that might be
affected is live kernel patching).
Good point, livepatch is indeed affected. Is there a better way to get
the "call __fentry__" address for a given function?
/*
* Convert a function address into the appropriate ftrace location.
*
* Usually this is just the address of the function, but on some architectures
* it's more complicated so allow them to provide a custom behaviour.
*/
#ifndef klp_get_ftrace_location
static unsigned long klp_get_ftrace_location(unsigned long faddr)
{
return faddr;
}
#endif
--
Josh
From: Peter Zijlstra <peterz@infradead.org> Date: 2021-02-08 18:23:35
On Mon, Feb 08, 2021 at 09:33:00AM -0600, Josh Poimboeuf wrote:
On Mon, Feb 08, 2021 at 10:02:06AM -0500, Steven Rostedt wrote:
quoted
On Sun, 7 Feb 2021 16:45:40 -0600
Josh Poimboeuf [off-list ref] wrote:
quoted
quoted
I do suspect involved people should start thinking about how they want
to deal with functions starting with
endbr64
call __fentry__
instead of the call being at the very top of the function.
FWIW, objtool's already fine with it (otherwise we would have discovered
the need to disable fcf-protection much sooner).
And this doesn't really affect tracing (note, another user that might be
affected is live kernel patching).
Good point, livepatch is indeed affected. Is there a better way to get
the "call __fentry__" address for a given function?
/*
* Convert a function address into the appropriate ftrace location.
*
* Usually this is just the address of the function, but on some architectures
* it's more complicated so allow them to provide a custom behaviour.
*/
#ifndef klp_get_ftrace_location
static unsigned long klp_get_ftrace_location(unsigned long faddr)
{
return faddr;
}
#endif
I suppose the trivial fix is to see if it points to endbr64 and if so,
increment the addr by the length of that.
From: Steven Rostedt <rostedt@goodmis.org> Date: 2021-02-08 18:33:23
On Mon, 8 Feb 2021 16:47:05 +0100
Peter Zijlstra [off-list ref] wrote:
quoted
/*
* Convert a function address into the appropriate ftrace location.
*
* Usually this is just the address of the function, but on some architectures
* it's more complicated so allow them to provide a custom behaviour.
*/
#ifndef klp_get_ftrace_location
static unsigned long klp_get_ftrace_location(unsigned long faddr)
{
return faddr;
}
#endif
I suppose the trivial fix is to see if it points to endbr64 and if so,
increment the addr by the length of that.
I thought of that too. But one thing that may be possible, is to use
kallsym. I believe you can get the range of a function (start and end of
the function) from kallsyms. Then ask ftrace for the addr in that range
(there should only be one).
-- Steve
From: Miroslav Benes <mbenes@suse.cz> Date: 2021-02-09 08:33:45
On Mon, 8 Feb 2021, Steven Rostedt wrote:
On Mon, 8 Feb 2021 16:47:05 +0100
Peter Zijlstra [off-list ref] wrote:
quoted
quoted
/*
* Convert a function address into the appropriate ftrace location.
*
* Usually this is just the address of the function, but on some architectures
* it's more complicated so allow them to provide a custom behaviour.
*/
#ifndef klp_get_ftrace_location
static unsigned long klp_get_ftrace_location(unsigned long faddr)
{
return faddr;
}
#endif
powerpc has this
static inline unsigned long klp_get_ftrace_location(unsigned long faddr)
{
/*
* Live patch works only with -mprofile-kernel on PPC. In this case,
* the ftrace location is always within the first 16 bytes.
*/
return ftrace_location_range(faddr, faddr + 16);
}
quoted
I suppose the trivial fix is to see if it points to endbr64 and if so,
increment the addr by the length of that.
I thought of that too. But one thing that may be possible, is to use
kallsym. I believe you can get the range of a function (start and end of
the function) from kallsyms. Then ask ftrace for the addr in that range
(there should only be one).
And we can do this if a hard-coded value live above is not welcome. If I
remember correctly, we used to have exactly this in the old versions of
kGraft. We walked through all ftrace records, called
kallsyms_lookup_size_offset() on every record's ip and if the offset+ip
matched faddr (in this case), we returned the ip.
Miroslav
From: Steven Rostedt <rostedt@goodmis.org> Date: 2021-02-09 14:51:06
On Tue, 9 Feb 2021 09:32:34 +0100 (CET)
Miroslav Benes [off-list ref] wrote:
powerpc has this
static inline unsigned long klp_get_ftrace_location(unsigned long faddr)
{
/*
* Live patch works only with -mprofile-kernel on PPC. In this case,
* the ftrace location is always within the first 16 bytes.
*/
return ftrace_location_range(faddr, faddr + 16);
}
quoted
quoted
I suppose the trivial fix is to see if it points to endbr64 and if so,
increment the addr by the length of that.
I thought of that too. But one thing that may be possible, is to use
kallsym. I believe you can get the range of a function (start and end of
the function) from kallsyms. Then ask ftrace for the addr in that range
(there should only be one).
And we can do this if a hard-coded value live above is not welcome. If I
remember correctly, we used to have exactly this in the old versions of
kGraft. We walked through all ftrace records, called
kallsyms_lookup_size_offset() on every record's ip and if the offset+ip
matched faddr (in this case), we returned the ip.
Either way is fine. Question is, should we just wait till CET is
implemented for the kernel before making any of these changes? Just knowing
that we have a solution to handle it may be good enough for now.
-- Steve
From: Miroslav Benes <mbenes@suse.cz> Date: 2021-02-09 15:16:50
On Tue, 9 Feb 2021, Steven Rostedt wrote:
On Tue, 9 Feb 2021 09:32:34 +0100 (CET)
Miroslav Benes [off-list ref] wrote:
quoted
powerpc has this
static inline unsigned long klp_get_ftrace_location(unsigned long faddr)
{
/*
* Live patch works only with -mprofile-kernel on PPC. In this case,
* the ftrace location is always within the first 16 bytes.
*/
return ftrace_location_range(faddr, faddr + 16);
}
quoted
quoted
I suppose the trivial fix is to see if it points to endbr64 and if so,
increment the addr by the length of that.
I thought of that too. But one thing that may be possible, is to use
kallsym. I believe you can get the range of a function (start and end of
the function) from kallsyms. Then ask ftrace for the addr in that range
(there should only be one).
And we can do this if a hard-coded value live above is not welcome. If I
remember correctly, we used to have exactly this in the old versions of
kGraft. We walked through all ftrace records, called
kallsyms_lookup_size_offset() on every record's ip and if the offset+ip
matched faddr (in this case), we returned the ip.
Either way is fine. Question is, should we just wait till CET is
implemented for the kernel before making any of these changes? Just knowing
that we have a solution to handle it may be good enough for now.
I'd prefer it to be a part of CET enablement patch set.
Miroslav
On Tue, Feb 9, 2021 at 6:49 AM Steven Rostedt [off-list ref] wrote:
On Tue, 9 Feb 2021 09:32:34 +0100 (CET)
Miroslav Benes [off-list ref] wrote:
quoted
powerpc has this
static inline unsigned long klp_get_ftrace_location(unsigned long faddr)
{
/*
* Live patch works only with -mprofile-kernel on PPC. In this case,
* the ftrace location is always within the first 16 bytes.
*/
return ftrace_location_range(faddr, faddr + 16);
}
quoted
quoted
I suppose the trivial fix is to see if it points to endbr64 and if so,
increment the addr by the length of that.
I thought of that too. But one thing that may be possible, is to use
kallsym. I believe you can get the range of a function (start and end of
the function) from kallsyms. Then ask ftrace for the addr in that range
(there should only be one).
And we can do this if a hard-coded value live above is not welcome. If I
remember correctly, we used to have exactly this in the old versions of
kGraft. We walked through all ftrace records, called
kallsyms_lookup_size_offset() on every record's ip and if the offset+ip
matched faddr (in this case), we returned the ip.
Either way is fine. Question is, should we just wait till CET is
implemented for the kernel before making any of these changes? Just knowing
that we have a solution to handle it may be good enough for now.
I think the issue is more fundamental than what appears on the surface.
According to endbr64 documentation it's not just any instruction.
The cpu will wait for it and if it's replaced with int3 or not seen at
the branch target the cpu will throw an exception.
If I understood the doc correctly it means that endbr64 can never be
replaced with a breakpoint. If that's the case text_poke_bp and kprobe
need to do extra safety checks.
From: Andy Lutomirski <luto@amacapital.net> Date: 2021-02-09 16:57:24
On Feb 9, 2021, at 8:45 AM, Alexei Starovoitov [off-list ref] wrote:
On Tue, Feb 9, 2021 at 6:49 AM Steven Rostedt [off-list ref] wrote:
quoted
quoted
On Tue, 9 Feb 2021 09:32:34 +0100 (CET)
Miroslav Benes [off-list ref] wrote:
powerpc has this
static inline unsigned long klp_get_ftrace_location(unsigned long faddr)
{
/*
* Live patch works only with -mprofile-kernel on PPC. In this case,
* the ftrace location is always within the first 16 bytes.
*/
return ftrace_location_range(faddr, faddr + 16);
}
quoted
quoted
I suppose the trivial fix is to see if it points to endbr64 and if so,
increment the addr by the length of that.
I thought of that too. But one thing that may be possible, is to use
kallsym. I believe you can get the range of a function (start and end of
the function) from kallsyms. Then ask ftrace for the addr in that range
(there should only be one).
And we can do this if a hard-coded value live above is not welcome. If I
remember correctly, we used to have exactly this in the old versions of
kGraft. We walked through all ftrace records, called
kallsyms_lookup_size_offset() on every record's ip and if the offset+ip
matched faddr (in this case), we returned the ip.
Either way is fine. Question is, should we just wait till CET is
implemented for the kernel before making any of these changes? Just knowing
that we have a solution to handle it may be good enough for now.
I think the issue is more fundamental than what appears on the surface.
According to endbr64 documentation it's not just any instruction.
The cpu will wait for it and if it's replaced with int3 or not seen at
the branch target the cpu will throw an exception.
If I understood the doc correctly it means that endbr64 can never be
replaced with a breakpoint. If that's the case text_poke_bp and kprobe
need to do extra safety checks.
Ugh.
Or we hack up #CP to handle this case. I don’t quite know how I feel about this.
On Tue, Feb 9, 2021 at 8:55 AM Andy Lutomirski [off-list ref] wrote:
Or we hack up #CP to handle this case. I don’t quite know how I feel about this.
I think that's the sane model - if we've replaced the instruction with
'int3', and we end up getting #CP due to that, just do the #BP
handling.
Anything else would just be insanely complicated, I feel.
Linus
From: Andy Lutomirski <luto@amacapital.net> Date: 2021-02-09 20:45:08
On Feb 9, 2021, at 10:09 AM, Linus Torvalds [off-list ref] wrote:
On Tue, Feb 9, 2021 at 8:55 AM Andy Lutomirski [off-list ref] wrote:
quoted
Or we hack up #CP to handle this case. I don’t quite know how I feel about this.
I think that's the sane model - if we've replaced the instruction with
'int3', and we end up getting #CP due to that, just do the #BP
handling.
Anything else would just be insanely complicated, I feel.
The other model is “don’t do that then.”
I suppose a nice property of patching ENDBR to INT3 is that, not only is it atomic, but ENDBR is sort of a NOP, so we don’t need to replace the ENDBR with anything.
On Tue, Feb 9, 2021 at 10:26 AM Andy Lutomirski [off-list ref] wrote:
quoted
Anything else would just be insanely complicated, I feel.
The other model is “don’t do that then.”
Hmm. I guess all the code that does int3 patching could just be taught
to always go to the next instruction instead.
I don't think advancing the rewriting is an option for the asm
alternative() logic or the static call infrastructure, but those
should never be about endbr anyway, so presumably that's not an issue.
So if it ends up being _only_ about kprobes, then the "don't do that
then" might work fine.
Linus