Re: Future Direction for rte_eth_stats_get()

+Jay, (@all, please keep everybody in the CCs :)

From: David Harton (dharton) [mailto:dharton@cisco.com]
To: Van Haaren, Harry <redacted>; Thomas Monjalon

quoted

quoted

xstats are driver agnostic and have a well-defined naming scheme.

Indeed, described here:
http://dpdk.org/doc/guides/prog_guide/poll_mode_drv.html#extended-
statistics-api

Thanks for sharing.  I do think what is in the link is well thought out but it also may
not match how the application would choose to represent that stat (capitalization,
abbreviation, etc).

Welcome. Sure, an application can match any xstats string to its "Display" counterpart,
or a few simple translations actually make (almost) all of them human readable. Translating
an _ to space, capitalize first letter of description items, and remove the unit at the end).
This use-case was considered when detailing the naming scheme.

quoted

The "rules" of encoding a statistic as an xstats string are pretty simple,
and if any PMD breaks the rules, it should be considered broken and fixed.

I understand it may be broken, but that doesn't help finding them and ensuring they aren't
broken to begin with.  What regression/automation is in place to ensure drivers aren't
broken?

Currently nothing automated - patch review on the mailing list. I'm open to ideas on this,
if you have suggestions.

quoted

Do you see a fundamental problem with parsing the strings to retrieve
values?

I think parsing strings is expensive CPU wise

Parsing the strings can be done once at startup, and the index of the statistics in the
array cached. When actually reading statistics, access of the array of statistics at the
previously cached index will return the same statistic. It is not needed to do strcmp()
per statistic per read.

That's why I was wondering if a binary id could be generated instead.

I've worked with such a system before, it dynamically registered string->int mappings at
runtime, and allowed "reverse" mapping them too. Its workable, and I'm not opposed to it
if the conclusion is that it really is necessary, but I'm not sure about that.

The API
would be very similar to the current xstats API except it would pass id/value pairs
instead of string/value pairs.  This avoids string comparisons to figure out while still
allowing flexibility between drivers.

Apart from a once-off scan at startup, xstats achieves this. 

 It would also 100% guarantee that any strings
returned by "const char* rte_eth_xstats_str_get(uint32_t stat_id)" are consistent across
devices.

Yes - that is a nice feature that xstats (in its current form) doesn't have.
But what price is there to pay for this?
We need to map an ID for each stat that exists.

How and where will this mapping happen? Perhaps would you expand a little on how
you see this working? 

I also think there is less chance for error if drivers assign their stats to ID versus
constructing a string. 

Have a look in how the mapping is done in the xstats implementation for ixgbe for example:
http://dpdk.org/browse/dpdk/tree/drivers/net/ixgbe/ixgbe_ethdev.c#n540

It's a pretty simple  {"string stat name" -> offsetof( stuct hw, register_var_name )}

I haven't checked my application, but I wonder if the binary size
would actually be smaller if all the stats strings were centrally located versus
distributed across binaries (highly dependent on the linker and optimization level).

Sounds like optimizing the wrong thing to me.

quoted

If you like I could code up a minimal sample-app that only pulls
statistics, and you can show "interest" in various statistics for printing
/ monitoring?

Appreciate the offer.  I actually understand what's is available.  And, BTW, I apologize
for being late to the game (looks like this was discussed last summer) but I'm just now
getting looped in and following the mailer but I'm just wondering if something that is
performance friendly for the user/application and flexible for the drivers is possible.

We're all looking for the same thing - just different approaches that's all :)


RE: Thomas asking about performance numbers:
I can scrape together some raw tsc data on Monday and post to list, and we can discuss it more then.


Regards, -Harry