Hi Alan,
here the missing answers.

Am 18.01.2016 um 23:32 schrieb H. Nikolaus Schaller [off-list ref]:

Just a first short answer (can't work 7/24 :).

Am 18.01.2016 um 23:03 schrieb One Thousand Gnomes [off-list ref]:

quoted

quoted

quoted

Your user space can do it (as most Android does).

How can it do it in automatically in a standardized way without need for daemon support?

You don't need to - it can be device specific. In Android it usually is.
I've never understood why low end devices don't also abstract user space
descriptions of power control into DT nodes as well as kernel properties ?

I don't know.

But what I know is that I don't want to solve someone else's problems...

quoted

quoted

* how can the daemon present another /dev/tty so that applications expecting such
an interface can attach to it (maybe through pty - but isn't that an overkill?)

You don't need to - you can monitor the rx/tx stats via procfs too. Not
pretty but given the daemon can do both the decoding and the monitoring
for your device it all seems a strawman. I'd argue given the sheer range
of ways people PM a GPS that you ought to have those power abstractions
in the user space daemon. They might use kernel methods, they might not.

The problem is that *I* have no control over user space. But I also don't want
to say to my users "that is not my problem - get it solved yourself". This does
not help them.

Especially as I can help them with the kernel based solutions we have.

quoted

quoted

* Who makes sure that this daemon is installed and running right after boot up on *any* Linux system
so that it can always react in case that the chip did start when it should be off?

Whoever puts the distribution together. The kernel runs init. Beyond that
we don't care. Not our problem. You can boot into emacs if you want.

Well, it is my big problem which goes contrary to our goal to have the best
supported platform... We would have to provide and maintain such things
so that they are compatible to a plethora of unknown runtime environments.

quoted

quoted

More generally: what is a kernel good for? Why do we need kernel drivers?

To maintain security, to manage hardware elements that cannot be managed
in user space,

Ahem. Most hardware elements can be manages in user space. Use FUSE for
everything and you will only need 10% of the kernel code.

So why are you always arguing with exaggerations?

quoted

and to provide abstractions where the abstraction is
necessary and meaningful to a large number of users (not a corner case
phone device

well, phone devices in general outnumber all other devices in sheer numbers.

Yes, the GTA04 has low quantities. But the main hurdle I always hear is that
the kernel is not good enough.

quoted

unless the abstraction can be made meaningful and widely
useful).

And, most device drivers are corner cases since they are special solutions
for singular platforms.

quoted

quoted

Just think about waking up the daemon process if a character is received. This is
much more costly than calling a notification function in the kernel driver which might
have to execute just 4 or 5 assembler instructions to decide what to do.

The logical continuation of that is of course not to have user space but
write your entire phone device in ring 0. Point being it is always a
trade off.

Your only way currently to do that is to open the tty and set a line
discipline which does your monitoring then hold it open. We can't stack
ldiscs either. If you want to monitor the line state with the physical
uart receiver powered down then this won't work at all.

quoted

So kernel drivers are sometimes the best solution and more efficient and controllable
than a user space daemon.

Sometimes but not always.

I am only interested in this case and here it obviously is.

quoted

And even if a kernel device is the right
solution to your device that doesn't mean it's general purpose enough to
justify being stuffed upstream. Android is full of "interesting" device
specific tweaks the sum of which would sink the kernel project.

This is of course an argument which we have to overcome.

quoted

quoted

I understand that. But what is Linux good for? For it's own sake or for
users and platforms using it? Isn't it that we take it from the community
and contribute improvements back to it?

Only when the community benefits overall.

Yes, it benefits because there are several requests for such tty/uart slaves/peers.

From this I deduce that there is community demand.

I brought up this discussion again, because Andrey and Tomeu recently asked
for progress.

quoted

quoted

So to me it appears that such a kernel feature is missing. Therefore we
are discussing it.

I'm glad - because it raises some hard questions and while I don't agree
with some of your starting points (like needing to "open" a uart without
user space

If have an idea how to turn off the device at boot time, before any user space
daemon is running, we can of course ignore that.

quoted

) I do agree that

- There isn't a nice way to bind extra non device specific behaviour to
open and close (but we have the right places to add one)

- There isn't a way to monitor rx data (and this is *really* hard to
sort especially when the uart is powered down)

Exactly. This is why we already work 3 years on this topic...

The solution is to optionally keep it powered up - as long as the peer
device asks for.

quoted

- Monitoring mctrl via a nice abstraction is going to need the mctrl
ops moving from tty to tty_port if you want to use that to trigger tty
slave behaviour. Doable but a change.

Yes, any new function or improvement is a change.

quoted

quoted

well I think most driver projects that have expressed that they want to have
such a solution just want an UART abstraction and not a general tty/serial
interface with all bells and whistles.

There isn't any difference at the proper abstraction layer. If you wanted
your monitor to power on and off when you open a console that's the same
problem space.

quoted

quoted

uart is just a helper library for some types of
port.

Yes, this is the type of port, our peer devices are directly connected to.

Think about the bigger picture. The high level abstraction is the tty and
the tty_port.

Yes, it is indeed.

quoted

But see below as I think your mental model is perhaps wrong
and this is a point of confusion ?

Maybe you do not accept that I want to keep as low level as reasonable (for me).

quoted

quoted

Another question if we discuss moving the hooks into the tty_port layer:

is it possible from tty layer to activate the UART without a user-space open()
and keep it activated after a close()?

No the entire tty layer from top to bottom assumes that you activate a
device by opening it from user space. uart, tty, the lot. The only
corner case weird exception is the serial console and that is a horrible
hack for output only - so anything that fixes that assumption should also
be made to fix the console case.

Reasonable.

quoted

It's never really mattered because the obscure corner cases where you
want a tty held open the cost is minimal because you just let systemd or
similar own the file handle along with everything else it is hanging on
to.

quoted

quoted

The ones I am familiar with either have the userspace managing it via
sysfs (which has some latency advantages when doing clever stuff) or
wired the power control to the carrier signal (or that is declared the
gpio that controls it to be the carrier).

How many of these special drivers are in mainline? Our target is to get full support
by mainline and not run our own kernel branch forever. Because we are not
yet-another-android-thing-that-just-needs-to-work-for-6-months-and-nobody-cares-
about-updates-and-GPL.

Both of those techniques work in mainline without kernel changes (at
least on devices where the right gpio sysfs nodes exist

they do not exist...

quoted

). Not pretty and
it would be better to abstract it - although ACPI often abstracts it by
having the kernel power down the uart which calls into ACPI which pokes a
GPIO line so presumably devicetree could learn the same tricks if it got
better at describing power management.

I do not want to wait until that happens.

quoted

quoted

In all mainlining attempts I know, the user-space control approach was rejected
because it was said that the kernel should take care of and not a /sysfs or some
other artificial protocol.

I'm not sure who said that but they obviously didn't look at the real
world 8)

quoted

But anyway, it would not solve the device RX data stream monitoring problem.
It just could be a solution to know when to turn the chip on or off.

This I think is actually the really hard and interesting part of the
problem. The "tell me about open and close" case is simple and can be
done via tty_port today with minimal extra hooks. There is a small
question about how you set those hooks from a DT binding

tty has no binding. An UART hardware has. Another reason for me to
start with UARTs.

quoted

- mostly because
we want the tty_port method table to be const for security reasons.

The data one is much harder because the abstractions in the tty_port
never see data. It goes direct from the hardware interface (possibly via
a support library liek uart but not always) on to the core of the tty
code. It's also a very hot path on things like older USB 3G modems that
use AT commands. Peter has done minor miracles on making it more
efficient but a USB modem can still really stress a small CPU without any
further callbacks appearing.

quoted

Neil's proposal to monitor the RX line was rejected because it was doing
nasty tricks with switching pinmux states and setting a temporary interrupt
on the UART RX line.

For some hardware that is the only way I know to do this because the
power hungry uart receiver is physically powered down. I would have to
check but I *think* that is true even on a modern x86 PC that supports
wakeups via serial - although it may be well hidden in ACPI and firmware.

Yes, agreed. But the gpio + interrupt solution was not mainlineable as well.

quoted

quoted

Therefore we now want to monitor the RX line "behind" the UART, i.e. on the
byte stream coming from the UART shift registers.

This did lead to tty/uart slaves concept and everybody wanted it. Now as we
have code proposals it should go back to become a user space daemon...

I'm not personally opoosed to the tty slave idea providing it ends up
attached to the tty_port not just uart.

Well if you can tell us how to handle the data path I have no problems with it
to attach to the tty level.

quoted

quoted

quoted

2. If not then hook tty_port_shutdown() and tty_port_open() because those
are the right abstraction point. Everything in the kernel that is a tty
is a tty_port.

What is in your view the right abstraction point for a peer device driver to get
notified about rx characters (even if the tty is currently not open)?

I don't think you have one. A lot of hardware has the receiver and
transceiver physically powered off when the tty is closed. You can't even
touch the registers in some cases (eg a PCI port in D3 state).

This is why I want to keep the UART open as long as the peer driver
tells to do so.

If there is no peer driver or the peer drivers does not need the UART
it is exactly the same as today.

quoted

I certainly have no good ideas for that specific case because apart from
things buried in stuff like ACPI I don't know of any general purpose way
to ask "how do I make some other piece of hardware peek at the level on
that line and/or trigger on rising or falling edges"

quoted

I think on a level of abstraction of all the different UART drivers. Which is sufficient
to access the UART by the peer driver. This is why I call it UART-peer (and
not serial or tty peer).

Not all uarts use the uart layer. It's an optional helper.

I have researched and it appears that struct uart_port originally came in 2002
in release 2.4 for the sa1100 uart. Since then it has been extended significantly
and is now used by some uart hardware drivers.

But this is again a black&white argument. You recommended to be either general
or patch the omap-serial driver only that we have. I said that i want it a little
more general - which does not mean that I want to rewrite everything. I.e.
for my purposes it would suffice to support those UARTs that already use the
uart_port layer. If a hardware doesn't use it, it could be useful to rewrite the
UART device driver as well. But that is not something I feel I must do and
not now.

quoted

quoted

You want to abstract from UART and add all tty bells and whistles. We need them
for the GPS chip's data stream (because our unknown GPS applications might
use them) - but other low level UART peer drivers do not even need them.

Ok I think the model you have may be wrong.

The lowest level physical interface abstraction in the whole tty stack is
the tty_port. Every object which can be opened as a tty contains a
tty_port, and the tty_port lifetime is the lifetime of the "physical"
interface to which it is attached.

tty_port requires the device provide a set of methods.

uart optionally sits on top of tty_port. It provides the tty_port methods
and provides a differentish set of low level abstractions more suited to
byte oriended serial port devices. Console doesn't use it, sdio serial
doesn't use it, some uarts don't use it, usb doesn't use it (including
on-board HSIC and SSIC devices), most jtag serials don't use it either.

The tty layer talks to the tty_port layer and also directly to tty
methods provided by the physical device.

Ok, I see. My picture was more a layer scheme based on the path how
data flows from the hardware specific uart driver to the uart layer and
then to the tty_port things.

quoted

Conveniently for you the
tty_port abstracts opening and closing the device. Inconveniently for you
it doesn't really mediate receiving and sending data although it does hold
the buffers used.

Yes.

quoted

quoted

quoted

Then all you need is the (possibly device specific) small patches to check
the device tree for the bindings on init, and if so set the port->ops
methods according to the binding.

For me it appears to only solves a small part of the whole problem.

1. it must be possible to start the UART before any user-space open()

There is no tty layer support at any level to do this, nor any reason
I've seen advanced to justify all the extra complexity needed.

This is why I want to do it on the uart_port.

quoted

quoted

2. the UART must be kept running as long as the peer driver wants to
monitor rx data

Correct, otherwise in the general case the uart may be physically powered
off and the kernel will in fact try aggressively to do so.

quoted

3. we need a hook to monitor: that (and which) data is incoming

That isn't tty_port related - but it's certainly hard.

Not, if we do it on uart_port level... It already works with two not very
big patches.

quoted

quoted

Another aspect is that on a physical RS232 the DTR line is usually
used to power on/off a remote device (the DCE). Therefore I prefer
to mimic that in software by intercepting the mctrl changes. This
additionally allows a client to turn off power of the chip through a "standard"
protocol, even while the tty file is kept open.

The mctrl currently goes via tty->ops->tiocmset()/tiocmget. That IMHO is
actually an oddity and moving it to the tty_port would make sense anyway
because it's state and behaviour that is persistent at hardware level.

I don't know what Peter thinks about that however.

quoted

quoted

Nobody wants to lose the ability to do so or to move stuff around.

Indeed. But why would you loose this ability at all?

Because all your hooks would break.

Only if you do big changes. That is why I ask if such big changes are planned
or can be anticipated. They probably don't come unexpectedly around the corner.

And if you reject other changes like ours, there won't be any in the next 20 years :)

quoted

quoted

If significant architectural changes are to be done, you can deprecate this
API and ask all users (driver authors) to replace it with something new.
And if they aren't adjusted within some time or no new interface is proposed,
they get removed. The *I* have the problem back on my table and not you :)

Isn't this a standard way to handle such situations?

Things are gradually changed every now and then. And other parts have to
follow. BTW: most of my work to update a production kernel to a new -rc1 is
to adjust non-mainline (or not-yet) drivers to such API changes.

What usually happens is the creator the tangle has despite best
intentions ended up somewhere else on another project, their users whine
if we break it, someone screams regression and it all gets dumped on the
maintainers.

That is of course something we want to avoid. This is why I persistently try
to discuss it with you every detail. Fortunately a good discussion has now started.

And it looks that we either need a kernel solution or a user space solution - but
none makes the responsible party happy... Although that is not a criterion for
a whole system architecture. There the more efficient solution should be done.
Efficient in terms of LOC, speed, maintenance requirements.

BR,
Nikolaus

--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html