Hi All,
I am currently working on building AMP systems using OpenMCAPI
(https://bitbucket.org/hollisb/openmcapi/wiki/Home) as the
inter-processor communication mechanism. With OpenMCAPI we, of course,
need a way to send messages to various cores. On some Freescale PPC
platforms (e.g. P1022DS, MPC8572DS), we have been using message
registers to do this work. Recently, I was looking at the OMAP4
mailboxes to gear up for moving into ARM based platforms.
With that, I noticed 'arch/arm/plat-omap/mailbox.c'. This is very
specific to the OMAP4 boards. I am looking at designing a new set of
drivers to expose a mailbox service to userspace that will be used
for inter-processor communication. This would entail the traditional
generic/specific driver split:
1. Hardware specific bits somewhere under '.../arch/*'. Drivers
for the MPIC message registers on Power and OMAP4 mailboxes, for
example.
2. A higher level driver under '.../drivers/mailbox/*'. That the
pieces in (1) would register with. This piece would expose the
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for example.
Now I have the following questions:
1. Do others see value in this?
2. Does something like this already exist?
3. Is someone else already working on this?
Any feedback will be greatly appreciated.
--
Meador Inge | meador_inge AT mentor.com
Mentor Embedded | http://www.mentor.com/embedded-software
Hi,
On Sat, Feb 12, 2011 at 2:49 AM, Meador Inge [off-list ref] wrote=
:
=A0 =A01. Hardware specific bits somewhere under '.../arch/*'. =A0Drivers
=A0 =A0 =A0 for the MPIC message registers on Power and OMAP4 mailboxes, =
for
=A0 =A0 =A0 example.
Yes; this can help.
=A0 =A02. A higher level driver under '.../drivers/mailbox/*'. =A0That th=
e
=A0 =A0 =A0 pieces in (1) would register with. =A0This piece would expose=
the
=A0 =A0 =A0 main kernel API.
A lot of mailboxes are too platform specific with regards to the communicat=
ion
with the main CPU and probably it depends on the mailbox too; you can
find polling and
interrupt supported mailbox support at times on the same platform.
APIs should probably be
generic enough to be able to operate in any context.
Now I have the following questions:
=A0 =A01. Do others see value in this?
At least I would like this; I wanted to generalize such mailbox IPCs
right from the
day when I was working on one, but coudnt really work on that.
=A0 =A02. Does something like this already exist?
Not generic as you say; but apart from the OMAP platforms,
you could refer to arch/arm/mach-ux500/prcmu for a mailbox based
IPC on the U8500 platform.
=A0 =A03. Is someone else already working on this?
Not sure of that too :), but I am CCing Linus W, the maintainer
of U8500 if he thinks it is a good idea to come up with a mailbox IPC
framework
Cheers!
--=20
---------
The views expressed in this email are personal and do not necessarily
echo my employers.
At least I would like this; I wanted to generalize such mailbox IPCs
right from the day when I was working on one, but coudnt really
work on that.
quoted
=A0 =A02. Does something like this already exist?
Not generic as you say; but apart from the OMAP platforms,
you could refer to arch/arm/mach-ux500/prcmu for a mailbox based
IPC on the U8500 platform.
We also have this thing:
arch/arm/mach-ux500/mbox-db5500.c
It's another mailbox driver, this one talks to the modem in the
U5500 (basically a physical transport for the CAIF protocol).
(For the U8500 I think modem IPC is instead handled with
a high-speed hardware FIFO, a bit different.)
quoted
=A0 =A03. Is someone else already working on this?
Not sure of that too :), but I am CCing Linus W, the maintainer
of U8500 if he thinks it is a good idea to come up with a mailbox IPC
framework
I don't know too much about the subject actually, I've not been
deeply into any such code. I don't think anyone is working
on something general from ST-Ericsson or Linaro.
Recently I saw that Texas Instruments are posting a "hardware
spinlock" framework though, this would be on a related tone,
but I think it's for shared data structures (control path) rather
than buffer passing (data path). I'm guessing this works like
that one CPU gets to spin waiting for another one to release
the lock.
Given that we may have a framework for hardware spinlock
and that we don't want to stockpile drivers into arch/*
or drivers/misc/* I would say it's intuitively a good idea,
but the question is what data types you would pass in?
In arch/arm/mach-ux500/include/mach/mbox-db5500.h
we have a struct like this:
/**
* struct mbox - Mailbox instance struct
* @list: Linked list head.
* @pdev: Pointer to device struct.
* @cb: Callback function. Will be called
* when new data is received.
* @client_data: Clients private data. Will be sent back
* in the callback function.
* @virtbase_peer: Virtual address for outgoing mailbox.
* @virtbase_local: Virtual address for incoming mailbox.
* @buffer: Then internal queue for outgoing messages.
* @name: Name of this mailbox.
* @buffer_available: Completion variable to achieve "blocking send".
* This variable will be signaled when there is
* internal buffer space available.
* @client_blocked: To keep track if any client is currently
* blocked.
* @lock: Spinlock to protect this mailbox instance.
* @write_index: Index in internal buffer to write to.
* @read_index: Index in internal buffer to read from.
* @allocated: Indicates whether this particular mailbox
* id has been allocated by someone.
*/
struct mbox {
struct list_head list;
struct platform_device *pdev;
mbox_recv_cb_t *cb;
void *client_data;
void __iomem *virtbase_peer;
void __iomem *virtbase_local;
u32 buffer[MBOX_BUF_SIZE];
char name[MBOX_NAME_SIZE];
struct completion buffer_available;
u8 client_blocked;
spinlock_t lock;
u8 write_index;
u8 read_index;
bool allocated;
};
Compare OMAPs mailboxes in
arch/arm/plat-omap/include/plat/mailbox.h:
typedef u32 mbox_msg_t;
truct omap_mbox_ops {
omap_mbox_type_t type;
int (*startup)(struct omap_mbox *mbox);
void (*shutdown)(struct omap_mbox *mbox);
/* fifo */
mbox_msg_t (*fifo_read)(struct omap_mbox *mbox);
void (*fifo_write)(struct omap_mbox *mbox, mbox_msg_t ms=
g);
int (*fifo_empty)(struct omap_mbox *mbox);
int (*fifo_full)(struct omap_mbox *mbox);
/* irq */
void (*enable_irq)(struct omap_mbox *mbox,
omap_mbox_irq_t irq);
void (*disable_irq)(struct omap_mbox *mbox,
omap_mbox_irq_t irq);
void (*ack_irq)(struct omap_mbox *mbox, omap_mbox_irq_t =
irq);
int (*is_irq)(struct omap_mbox *mbox, omap_mbox_irq_t i=
rq);
/* ctx */
void (*save_ctx)(struct omap_mbox *mbox);
void (*restore_ctx)(struct omap_mbox *mbox);
};
struct omap_mbox_queue {
spinlock_t lock;
struct kfifo fifo;
struct work_struct work;
struct tasklet_struct tasklet;
struct omap_mbox *mbox;
bool full;
};
struct omap_mbox {
char *name;
unsigned int irq;
struct omap_mbox_queue *txq, *rxq;
struct omap_mbox_ops *ops;
struct device *dev;
void *priv;
int use_count;
struct blocking_notifier_head notifier;
};
Some of this may be generalized? I dunno, they look quite
different but maybe queueing etc can actually be made general
enough to form a framework.
Yours,
Linus Walleij
=A0This would entail the traditional
generic/specific driver split:
=A0 =A01. Hardware specific bits somewhere under '.../arch/*'. =A0Drivers
=A0 =A0 =A0 for the MPIC message registers on Power and OMAP4 mailboxes, =
for
=A0 =A0 =A0 example.
Having any drivers under arch/* is no good tradition IMO.
Better move the whole shebang down to drivers/mailbox so
that the subsystem maintainer get the complete overview
of her/his driver family.
=A0 =A02. A higher level driver under '.../drivers/mailbox/*'. =A0That th=
e
=A0 =A0 =A0 pieces in (1) would register with. =A0This piece would expose=
the
=A0 =A0 =A0 main kernel API.
Cool...
=A0 =A03. Userspace interfaces for accessing the mailboxes. =A0A
=A0 =A0 =A0 '/dev/mailbox1', '/dev/mailbox2', etc... mapping, for example=
.
What kind of business does userspace have with directly using
mailboxes? Enlighten me so I get it... in our system these are
used by protocols, such as net/caif/* thru drivers/net/caif/*, and
we have similar kernelspace functionality for Phonet.
CAIF and Phonet on the other hand, have custom openings
down to the thing that exists on the other end of the mailbox.
Most of these systems tend to talk some funny protocol that
is often better handled by the kernel than by any userspace.
So is this for the situation when you have no intermediate
protocol between your userpace and the other CPU's
subsystem? Or are you thinking about handling that
protocol in userspace? That is generally not such a good idea
for efficiency reasons.
Yoursm
Linus Walleij
Hi Meador,
From: ext Meador Inge <redacted>
Subject: [RFC] Inter-processor Mailboxes Drivers
Date: Fri, 11 Feb 2011 15:19:51 -0600
Hi All,
I am currently working on building AMP systems using OpenMCAPI
(https://bitbucket.org/hollisb/openmcapi/wiki/Home) as the
inter-processor communication mechanism. With OpenMCAPI we, of
course,
need a way to send messages to various cores. On some Freescale PPC
platforms (e.g. P1022DS, MPC8572DS), we have been using message
registers to do this work. Recently, I was looking at the OMAP4
mailboxes to gear up for moving into ARM based platforms.
With that, I noticed 'arch/arm/plat-omap/mailbox.c'. This is very
specific to the OMAP4 boards. I am looking at designing a new set of
drivers to expose a mailbox service to userspace that will be used
for inter-processor communication. This would entail the traditional
generic/specific driver split:
1. Hardware specific bits somewhere under '.../arch/*'. Drivers
for the MPIC message registers on Power and OMAP4 mailboxes, for
example.
2. A higher level driver under '.../drivers/mailbox/*'. That the
pieces in (1) would register with. This piece would expose the
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for example.
Now I have the following questions:
1. Do others see value in this?
Yes.
I discussed this with TI(Hari) long time ago, but it didn't proceed
with some reason, not techinical one.
2. Does something like this already exist?
3. Is someone else already working on this?
Any feedback will be greatly appreciated.
Now the basic concpet can be devided into 3 parts, (1)HW dependent,
(2) generic driver, (3) character device interface. I guess that it
might be good to have one _pseudo_ H/W instance, then easier for
verification without real H/W and firmwares, across multiplatforms.
At least I would like this; I wanted to generalize such mailbox IPCs=
quoted
right from the day when I was working on one, but coudnt really
work on that.
quoted
=A0 =A02. Does something like this already exist?
Not generic as you say; but apart from the OMAP platforms,
you could refer to arch/arm/mach-ux500/prcmu for a mailbox based
IPC on the U8500 platform.
=
We also have this thing:
arch/arm/mach-ux500/mbox-db5500.c
=
It's another mailbox driver, this one talks to the modem in the
U5500 (basically a physical transport for the CAIF protocol).
(For the U8500 I think modem IPC is instead handled with
a high-speed hardware FIFO, a bit different.)
=
quoted
quoted
=A0 =A03. Is someone else already working on this?
Not sure of that too :), but I am CCing Linus W, the maintainer
of U8500 if he thinks it is a good idea to come up with a mailbox IP=
C
quoted
framework
=
I don't know too much about the subject actually, I've not been
deeply into any such code. I don't think anyone is working
on something general from ST-Ericsson or Linaro.
=
Recently I saw that Texas Instruments are posting a "hardware
spinlock" framework though, this would be on a related tone,
but I think it's for shared data structures (control path) rather
than buffer passing (data path). I'm guessing this works like
that one CPU gets to spin waiting for another one to release
the lock.
=
Given that we may have a framework for hardware spinlock
and that we don't want to stockpile drivers into arch/*
or drivers/misc/* I would say it's intuitively a good idea,
but the question is what data types you would pass in?
In arch/arm/mach-ux500/include/mach/mbox-db5500.h
we have a struct like this:
=
/**
* struct mbox - Mailbox instance struct
* @list: Linked list head.
* @pdev: Pointer to device struct.
* @cb: Callback function. Will be called
* when new data is received.
* @client_data: Clients private data. Will be sent back
* in the callback function.
* @virtbase_peer: Virtual address for outgoing mailbox.
* @virtbase_local: Virtual address for incoming mailbox.
* @buffer: Then internal queue for outgoing messages.
* @name: Name of this mailbox.
* @buffer_available: Completion variable to achieve "blocking send=
".
* This variable will be signaled when there is
* internal buffer space available.
* @client_blocked: To keep track if any client is currently
* blocked.
* @lock: Spinlock to protect this mailbox instance.
* @write_index: Index in internal buffer to write to.
* @read_index: Index in internal buffer to read from.
* @allocated: Indicates whether this particular mailbox
* id has been allocated by someone.
*/
struct mbox {
struct list_head list;
struct platform_device *pdev;
mbox_recv_cb_t *cb;
void *client_data;
void __iomem *virtbase_peer;
void __iomem *virtbase_local;
u32 buffer[MBOX_BUF_SIZE];
char name[MBOX_NAME_SIZE];
struct completion buffer_available;
u8 client_blocked;
spinlock_t lock;
u8 write_index;
u8 read_index;
bool allocated;
};
=
Compare OMAPs mailboxes in
arch/arm/plat-omap/include/plat/mailbox.h:
=
Some of this may be generalized? I dunno, they look quite
different but maybe queueing etc can actually be made general
enough to form a framework.
OMAP mailbox is the interrupt driven 32bit unit H/W FIFO to other
cores.
"struct omap_mbox_ops" was provided mainly to absorb the difference
between OMAP1 and OMAP2+ from H/W POV. So generally the layer could
be:
-----------------------
character device driver
-----------------------
generic mailbox driver
-----------------------
H/W registration
-----------------------
In OMAP case, in addition to the above, it could be exceptionally:
-----------------------
character device driver
-----------------------
generic mailbox driver
-----------------------
H/W registration
-----------------------
OMAP 1 | OMAP2+
-----------------------
So "character device driver"(interface) and "generic mailbox
driver"(queuing) may be able to abstructed/generalized.
On Mon, Feb 14, 2011 at 8:32 AM, Hiroshi DOYU [off-list ref] wrote:
OMAP mailbox is the interrupt driven 32bit unit H/W FIFO to other
cores.
How is it used? Is it a low-traffic (like single 32bit words etc) signal
control-path link while the actual high-throughput data-path is done
with shared memory? (That is how the db5500 mbox works anyways.)
Linus Walleij
On Mon, Feb 14, 2011 at 8:32 AM, Hiroshi DOYU [off-list ref] wrote:
quoted
OMAP mailbox is the interrupt driven 32bit unit H/W FIFO to other
cores.
How is it used? Is it a low-traffic (like single 32bit words etc) signal
control-path link while the actual high-throughput data-path is done
with shared memory? (That is how the db5500 mbox works anyways.)
Yes, maybe quite similar.
maibox is not single 32 bit but is 32 bit x 4(or 8?) slots fifo, IIRC,
and mainly it is used as notification between cores. And big amount of
data is transftered with sahred memory, which has been mapped onto the
virtual address space of the other side of core, in advance.
For example, typical usage of DSP, mp3 decoding,
1, ARM maps 2 shared memory area(input/output) onto DSP virtual
address space.
2, ARM fills mp3 data in input buffer.
3, ARM notifies DSP that data is ready in input buffer.
4, DSP decodes input data and put output data on output buffer.
5, DSP notifies that output buffer is ready.
Roughly something like the above. DSP S/W is multi-tasking, though.
Does db5500 use IOMMU for mapping shared memories?
On Mon, Feb 14, 2011 at 8:32 AM, Hiroshi DOYU [off-list ref] wrote:
quoted
OMAP mailbox is the interrupt driven 32bit unit H/W FIFO to other
cores.
How is it used? Is it a low-traffic (like single 32bit words etc) signal
control-path link while the actual high-throughput data-path is done
with shared memory? (That is how the db5500 mbox works anyways.)
Yes, maybe quite similar.
maibox is not single 32 bit but is 32 bit x 4(or 8?) slots fifo, IIRC,
and mainly it is used as notification between cores. And big amount of
data is transftered with sahred memory, which has been mapped onto the
virtual address space of the other side of core, in advance.
For example, typical usage of DSP, mp3 decoding,
1, ARM maps 2 shared memory area(input/output) onto DSP virtual
address space.
2, ARM fills mp3 data in input buffer.
3, ARM notifies DSP that data is ready in input buffer.
4, DSP decodes input data and put output data on output buffer.
5, DSP notifies that output buffer is ready.
Roughly something like the above. DSP S/W is multi-tasking, though.
Here, ARM side process talks to DSP site process in their own way, and
also there are another cores, talking with their own protocols. So I
think that at least, protocol part should be pluggable anyway although
it doesn't have to be always in userland.
On Fri, Feb 11, 2011 at 03:19:51PM -0600, Meador Inge wrote:
1. Hardware specific bits somewhere under '.../arch/*'. Drivers
for the MPIC message registers on Power and OMAP4 mailboxes, for
example.
2. A higher level driver under '.../drivers/mailbox/*'. That the
pieces in (1) would register with. This piece would expose the
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for example.
How about using virtio for all of this and having the mailbox as a
notification/message passing driver for the virtio backend? There are
already virtio console and network drivers that could be useful for the
userspace part of it. drivers/virtio/virtio_ring.c might be a good
starting point if you thought there was some mileage in this approach.
Jamie
From: Ira W. Snyder <hidden> Date: 2011-02-14 16:53:27
On Mon, Feb 14, 2011 at 12:03:59PM +0200, Ohad Ben-Cohen wrote:
On Mon, Feb 14, 2011 at 12:01 PM, Jamie Iles [off-list ref] wrote:
quoted
On Fri, Feb 11, 2011 at 03:19:51PM -0600, Meador Inge wrote:
quoted
1. Hardware specific bits somewhere under '.../arch/*'. Drivers
for the MPIC message registers on Power and OMAP4 mailboxes, for
example.
2. A higher level driver under '.../drivers/mailbox/*'. That the
pieces in (1) would register with. This piece would expose the
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for example.
How about using virtio for all of this and having the mailbox as a
notification/message passing driver for the virtio backend?
This is exactly what we are doing now, and it looks promising. expect
patches soon.
I'll be happy to examine the feasibility of doing a port to mpc83xx as
soon as I see the code. :-) I have been using the message registers to
create a software "network card" over PCI (between a host system and an
mpc83xx in a PCI slot). I have wanted to use virtio for this task for a
long time.
I think a uniform interface for the mailbox registers would be a very
useful API.
Thanks,
Ira
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for =
example.
What kind of business does userspace have with directly using
mailboxes? Enlighten me so I get it... in our system these are
used by protocols, such as net/caif/* thru drivers/net/caif/*, and
we have similar kernelspace functionality for Phonet.
CAIF and Phonet on the other hand, have custom openings
down to the thing that exists on the other end of the mailbox.
Most of these systems tend to talk some funny protocol that
is often better handled by the kernel than by any userspace.
So is this for the situation when you have no intermediate
protocol between your userpace and the other CPU's
subsystem? Or are you thinking about handling that
protocol in userspace? That is generally not such a good idea
for efficiency reasons.
OpenMCAPI (http://openmcapi.org) implements the MCAPI specification,=20
which is a simple application-level communication API that uses shared=20
memory. The API could be layered over any protocol, but was more or less =
designed for simple shared-memory systems, e.g. fixed topology, no=20
retransmission, etc.
Currently, we implement almost all of this as a shared library, plus a=20
very small kernel driver. The only requirements on the kernel are to=20
allow userspace to map the shared memory area, and provide an IPI=20
mechanism (and allow the process to sleep while waiting). Applications=20
sync with each other using normal atomic memory operations.
We're now trying to optimize the transfer of scalars on platforms that=20
provide mailboxes (beyond simple IPIs), which is why we're looking at=20
defining a user-facing API to such hardware.
I'll add that we haven't done serious optimization yet, but the numbers=20
we do have seem reasonable. What are the "efficiency" issues you're=20
worried about?
Hollis Blanchard
Mentor Graphics, Embedded Systems Division
On Fri, Feb 11, 2011 at 03:19:51PM -0600, Meador Inge wrote:
quoted
1. Hardware specific bits somewhere under '.../arch/*'. Drivers
for the MPIC message registers on Power and OMAP4 mailboxes, for
example.
2. A higher level driver under '.../drivers/mailbox/*'. That the
pieces in (1) would register with. This piece would expose the
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for example.
How about using virtio for all of this and having the mailbox as a
notification/message passing driver for the virtio backend? There are
already virtio console and network drivers that could be useful for the
userspace part of it. drivers/virtio/virtio_ring.c might be a good
starting point if you thought there was some mileage in this approach.
To be honest, I am not that familiar with 'virtio', but I will take a look. Thanks for the pointer. Maybe Hollis can speak to this idea more.
On Fri, Feb 11, 2011 at 03:19:51PM -0600, Meador Inge wrote:
quoted
1. Hardware specific bits somewhere under '.../arch/*'. =
Drivers
quoted
quoted
for the MPIC message registers on Power and OMAP4 mailboxes, =
quoted
quoted
for
example.
2. A higher level driver under '.../drivers/mailbox/*'. That =
the
quoted
quoted
pieces in (1) would register with. This piece would expose =
the
quoted
quoted
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for =
example.
quoted
How about using virtio for all of this and having the mailbox as a
notification/message passing driver for the virtio backend? There =
are
quoted
already virtio console and network drivers that could be useful for =
the
quoted
userspace part of it. drivers/virtio/virtio_ring.c might be a good
starting point if you thought there was some mileage in this =
approach.
To be honest, I am not that familiar with 'virtio', but I will take a=20
look. Thanks for the pointer. Maybe Hollis can speak to this idea =
more.
My opinion is that virtio is (over?) complicated.
I've looked into it in the past, and I'm definitely open to using it if=20
somebody can demonstrate how easy it is, but adopting it wouldn't have=20
helped OpenMCAPI with our use cases, and would have incurred extra pain, =
so we didn't.
Hollis Blanchard
Mentor Graphics, Embedded Systems Division
On Fri, Feb 11, 2011 at 03:19:51PM -0600, Meador Inge wrote:
quoted
1. Hardware specific bits somewhere under '.../arch/*'. Drivers
for the MPIC message registers on Power and OMAP4 mailboxes,
for
example.
2. A higher level driver under '.../drivers/mailbox/*'. That the
pieces in (1) would register with. This piece would expose the
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for example.
How about using virtio for all of this and having the mailbox as a
notification/message passing driver for the virtio backend? There are
already virtio console and network drivers that could be useful for the
userspace part of it. drivers/virtio/virtio_ring.c might be a good
starting point if you thought there was some mileage in this approach.
To be honest, I am not that familiar with 'virtio', but I will take a
look. Thanks for the pointer. Maybe Hollis can speak to this idea more.
My opinion is that virtio is (over?) complicated.
Considering the case of omap mailbox H/W, it is just a simple one way
4 slot x 32bit H/W FIFO, I also may think that this may be a bit too
much...
I've looked into it in the past, and I'm definitely open to using it if
somebody can demonstrate how easy it is, but adopting it wouldn't have
helped OpenMCAPI with our use cases, and would have incurred extra pain,
so we didn't.
Hollis Blanchard
Mentor Graphics, Embedded Systems Division
On Fri, Feb 11, 2011 at 03:19:51PM -0600, Meador Inge wrote:
quoted
1. Hardware specific bits somewhere under '.../arch/*'. =
Drivers
quoted
quoted
quoted
quoted
for the MPIC message registers on Power and OMAP4 =
mailboxes,
quoted
quoted
quoted
quoted
for
example.
2. A higher level driver under '.../drivers/mailbox/*'. =
That the
quoted
quoted
quoted
quoted
pieces in (1) would register with. This piece would =
expose the
quoted
quoted
quoted
quoted
main kernel API.
3. Userspace interfaces for accessing the mailboxes. A
'/dev/mailbox1', '/dev/mailbox2', etc... mapping, for =
example.
quoted
quoted
quoted
How about using virtio for all of this and having the mailbox as a
notification/message passing driver for the virtio backend? There =
are
quoted
quoted
quoted
already virtio console and network drivers that could be useful for =
the
quoted
quoted
quoted
userspace part of it. drivers/virtio/virtio_ring.c might be a good
starting point if you thought there was some mileage in this =
approach.
quoted
quoted
To be honest, I am not that familiar with 'virtio', but I will take =
a
quoted
quoted
look. Thanks for the pointer. Maybe Hollis can speak to this idea =
more.
quoted
My opinion is that virtio is (over?) complicated.
Considering the case of omap mailbox H/W, it is just a simple one way
4 slot x 32bit H/W FIFO, I also may think that this may be a bit too
much...
I think the proposal is to implement a virtio link using the OMAP=20
mailboxes as the interrupt mechanism, and shared memory to carry the=20
data and descriptor rings.
Hollis Blanchard
Mentor Graphics, Embedded Systems Division
OpenMCAPI (http://openmcapi.org) implements the MCAPI specification,
which is a simple application-level communication API that uses shared
memory. The API could be layered over any protocol, but was more or less
designed for simple shared-memory systems, e.g. fixed topology, no
retransmission, etc.
Cool...
Currently, we implement almost all of this as a shared library, plus a
very small kernel driver. The only requirements on the kernel are to
allow userspace to map the shared memory area, and provide an IPI
mechanism (and allow the process to sleep while waiting). Applications
sync with each other using normal atomic memory operations.
Can't this real small kernel driver take care of the mailbox
business as well?
It seems a bit backward if you have say /dev/mcapi0, /dev/mcapi1
etc (or however you expose this to userspace) and /dev/mailbox0
/dev/mailbox1 etc on top of that. One device node per communication
channel instead of this would certainly be nicer? Then you would
have some ioctl() on the /dev/mcapi0 etc node to trigger the
transport and need not worry that it's a mailbox doing the sync.
What I'm after is that whatever datapath you have should include
the control mechanism, now it's like you're opening two interfaces
into the kernel, one for mapping in data pages, one for synchronizing
the transfers, or am I getting things wrong?
I think nominally all mailbox users would be in-kernel like the
MCAPI driver, so they don't need a userspace interface, to me
it feels like say /dev/mutex0, /dev/mutex1 for some other
shared memory opening into the kernel (such as the framebuffer),
and that would look a bit funny.
I'll add that we haven't done serious optimization yet, but the numbers
we do have seem reasonable. What are the "efficiency" issues you're
worried about?
For huge data flows I think you may get into trouble, needing things
like queueing, descriptor pools etc. But if you're convinced this will
work, do go ahead.
Linus Walleij