Re: GTE - The hardware timestamping engine

On Sat, Mar 20, 2021 at 12:56 PM Linus Walleij [off-list ref] wrote:

Hi Dipen,

thanks for your mail!

I involved some other kernel people to get some discussion.
I think Kent Gibson can be of great help because he is using
GPIOs with high precision.

We actually discussed this a bit when adding support for
realtime timestamps.

Adding Richard Cochran as well, for drivers/ptp/, he may be able to
identify whether this should be integrated into that framework in some
form.

fullquote below

On Wed, Mar 17, 2021 at 11:29 PM Dipen Patel [off-list ref] wrote:

quoted

Nvidia Tegra SoCs have generic timestamping engine (GTE) hardware module which
can monitor SoC signals like IRQ lines and GPIO lines for state change, upon
detecting the change, it can timestamp and store in its internal hardware FIFO.
The advantage of the GTE module can be realized in applications like robotics
or autonomous vehicle where it can help record events with precise timestamp.

That sounds very useful.

Certainly the kernel shall be able to handle this.

quoted

============
For GPIO:
============
1.  GPIO has to be configured as input and IRQ must be enabled.
2.  Ask GPIO controller driver to set corresponding timestamp bit in the
    specified GPIO config register.
3.  Translate GPIO specified by the client to its internal bitmap.
3.a For example, If client specifies GPIO line 31, it could be bit 13 of GTE
    register.
4.  Set internal bits to enable monitoring in GTE module
5.  Additionally GTE driver can open up lanes for the user space application
    as a client and can send timestamping events directly to the application.

I have some concerns:

1. GPIO should for all professional applications be used with the character
device /dev/gpiochipN, under no circumstances shall the old sysfs
ABI be used for this. In this case it is necessary because the
character device provides events in a FIFO to userspace, which is
what we need.

The timestamp provided to userspace is an opaque 64bit
unsigned value. I suppose we assume it is monotonic but
you can actually augment the semantics for your specific
stamp, as long as 64 bits is gonna work.

2. The timestamp for the chardev is currently obtained in
drivers/gpio/gpiolib-cdev.c like this:

42 more quoted lines

On Sat, Mar 20, 2021 at 01:44:20PM +0100, Arnd Bergmann wrote:

Adding Richard Cochran as well, for drivers/ptp/, he may be able to
identify whether this should be integrated into that framework in some
form.

I'm not familiar with the GTE, but it sounds like it is a (free
running?) clock with time stamping inputs.  If so, then it could
expose a PHC.  That gets you functionality:

- clock_gettime() and friends
- comparison ioctl between GTE clock and CLOCK_REALTIME
- time stamping channels with programmable input selection

The mentioned applications (robotics and autonomous vehicle, so near
and dear to my heart) surely already use the PHC API for dealing with
network and system time sources, and so exposing the GTE as a PHC
means that user space programs will have a consistent API.

[ The only drawback I can see is the naming of the C language
  identifiers in include/uapi/linux/ptp_clock.h.  If that bothers
  people, then these can be changed to something more generic while
  keeping compatibility aliases. ]

Thanks,
Richard

Hi Richard,

Thanks for your input and time. Please see below follow up.

On 3/20/21 8:38 AM, Richard Cochran wrote:

On Sat, Mar 20, 2021 at 01:44:20PM +0100, Arnd Bergmann wrote:

quoted

Adding Richard Cochran as well, for drivers/ptp/, he may be able to
identify whether this should be integrated into that framework in some
form.

I'm not familiar with the GTE, but it sounds like it is a (free
running?) clock with time stamping inputs.  If so, then it could
expose a PHC.  That gets you functionality:

- clock_gettime() and friends
- comparison ioctl between GTE clock and CLOCK_REALTIME
- time stamping channels with programmable input selection

GTE gets or rather records the timestamps from the TSC
(timestamp system coutner) so its not attached to GTE as any
one can access TSC, so not sure if we really need to implement PHC
and/or clock_* and friends for the GTE. I believe burden to find correlation
between various clock domains should be on the clients, consider below
example.

Networking client has access to both PTP and GTE, it would be its job
to find the correlations if that is at all needed based on whatever
use case that client serves. GTE in above may come in picture if said client
has some GPIO configured and wants timestamp on it.

Sorry if I misunderstood anything, you can elaborate more as I am also
interested in how GTE can fit in PTP framework and which usecase it can
help doing so.

The mentioned applications (robotics and autonomous vehicle, so near
and dear to my heart) surely already use the PHC API for dealing with
network and system time sources, and so exposing the GTE as a PHC
means that user space programs will have a consistent API.

[ The only drawback I can see is the naming of the C language
  identifiers in include/uapi/linux/ptp_clock.h.  If that bothers
  people, then these can be changed to something more generic while
  keeping compatibility aliases. ]

Thanks,
Richard

Thanks,
Best Regards,
Dipen Patel

On Mon, Mar 22, 2021 at 01:33:38PM -0700, Dipen Patel wrote:

Hi Richard,

Thanks for your input and time. Please see below follow up.

On 3/20/21 8:38 AM, Richard Cochran wrote:

quoted

On Sat, Mar 20, 2021 at 01:44:20PM +0100, Arnd Bergmann wrote:

quoted

Adding Richard Cochran as well, for drivers/ptp/, he may be able to
identify whether this should be integrated into that framework in some
form.

I'm not familiar with the GTE, but it sounds like it is a (free
running?) clock with time stamping inputs.  If so, then it could
expose a PHC.  That gets you functionality:

- clock_gettime() and friends
- comparison ioctl between GTE clock and CLOCK_REALTIME
- time stamping channels with programmable input selection

GTE gets or rather records the timestamps from the TSC
(timestamp system coutner) so its not attached to GTE as any
one can access TSC, so not sure if we really need to implement PHC
and/or clock_* and friends for the GTE. I believe burden to find correlation
between various clock domains should be on the clients, consider below
example.

I agree. My understanding is the the TSC is basically an SoC-wide clock
that can be (and is) used by several hardware blocks. There's an
interface for software to read out the value, but it's part of a block
called TKE (time-keeping engine, if I recall correctly) that implements
various clock sources and watchdog functionality.

As a matter of fact, I recall typing up a driver for that at some point
but I don't recall if I ever sent it out or what became of it. I can't
find it upstream at least.

Anyway, I think given that the GTE doesn't provide that clock itself but
rather just a means of taking a snapshot of that clock and stamping
certain events with that, it makes more sense to provide that clock from
the TKE driver.

Thierry

On Tue, Mar 23, 2021 at 10:03:18AM +0100, Thierry Reding wrote:

I agree. My understanding is the the TSC is basically an SoC-wide clock
that can be (and is) used by several hardware blocks. There's an
interface for software to read out the value, but it's part of a block
called TKE (time-keeping engine, if I recall correctly) that implements
various clock sources and watchdog functionality.

...

Anyway, I think given that the GTE doesn't provide that clock itself but
rather just a means of taking a snapshot of that clock and stamping
certain events with that, it makes more sense to provide that clock from
the TKE driver.

It sounds like TKE + GTE together act like a PHC, and GTE doesn't
need/want its own SW interface.

Thanks,
Richard