On Sun, Jan 20, 2019 at 06:49:56PM -0800, hpa@zytor.com wrote:

On January 20, 2019 5:45:53 PM PST, Joel Fernandes [off-list ref] wrote:

quoted

On Sun, Jan 20, 2019 at 01:58:15PM -0800, hpa@zytor.com wrote:

quoted

On January 20, 2019 8:10:03 AM PST, Joel Fernandes

[off-list ref] wrote:

quoted

quoted

On Sat, Jan 19, 2019 at 11:01:13PM -0800, hpa@zytor.com wrote:

quoted

On January 19, 2019 2:36:06 AM PST, Greg KH

[off-list ref] wrote:

quoted

quoted

On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig

wrote:

quoted

quoted

quoted

quoted

quoted

This seems like a pretty horrible idea and waste of kernel

memory.

quoted

quoted

quoted

quoted

It's only a waste if you want it to be a waste, i.e. if you load

the

quoted

quoted

quoted

quoted

kernel module.

This really isn't any different from how /proc/config.gz works.

quoted

Just add support to kbuild to store a compressed archive in

initramfs

quoted

quoted

quoted

and unpack it in the right place.

I think the issue is that some devices do not use initramfs, or

switch

quoted

quoted

away from it after init happens or something like that.  Joel has

all

quoted

quoted

of
the looney details that he can provide.

thanks,

greg k-h

Yeah, well... but it is kind of a losing game... the more

in-kernel

quoted

quoted

stuff there is the less smiley are things to actually be supported.

It is better than nothing, and if this makes things a bit easier and
solves
real-world issues people have been having, and is optional, then I
don't see
why not.

quoted

Modularizing is it in some ways even crazier in the sense that at

that point you are relying on the filesystem containing the module,
which has to be loaded into the kernel by a root user. One could

even

quoted

quoted

wonder if a better way to do this would be to have "make
modules_install" park an archive file – or even a directory as

opposed

quoted

quoted

to a symlink – with this stuff in /lib/modules. We could even

provide a

quoted

quoted

tmpfs shim which autoloads such an archive via the firmware loader;
this might even be generically useful, who knows.

All this seems to assume where the modules are located. In Android,

we

quoted

quoted

don't
have /lib/modules. This patch generically fits into the grand scheme
things
and I think is just better made a part of the kernel since it is not
that
huge once compressed, as Dan also pointed. The more complex, and the
more
assumptions we make, the less likely people writing tools will get

it

quoted

quoted

right
and be able to easily use it.

quoted

Note also that initramfs contents can be built into the kernel.

Extracting such content into a single-instance tmpfs would again be

a

quoted

quoted

possibility

Such an approach would bloat the kernel image size though, which may
not work
for everyone. The module based approach, on the other hand, gives an
option
to the user to enable the feature, but not have it loaded into

memory

quoted

quoted

or used
until it is really needed.

thanks,

- Joel

Well, where are the modules? They must exist in the filesystem.

The scheme of loading a module doesn't depend on _where_ the module is
on the
filesystem. As long as a distro knows how to load a module in its own
way (by
looking into whichever paths it cares about), that's all that matters. 
And
the module contains compressed headers which saves space, vs storing it
uncompressed on the file system.

To remove complete reliance on the filesystem, there is an option of
not
building it as a module, and making it as a built-in.

I think I see your point now - you're saying if its built-in, then it
becomes kernel memory that is lost and unswappable. Did I get that
right?
I am saying that if that's a major concern, then:
1. Don't make it a built-in, make it a module.
2. Don't enable it at for your distro, and use a linux-headers package
or
whatever else you have been using so far that works for you.

thanks,

- Joel

My point is that if we're going to actually solve a problem, we need to make it so that the distro won't just disable it anyway, and it ought to be something scalable; otherwise nothing is gained.

I am *not* disagreeing with the problem statement!

Now, /proc isn't something that will autoload modules. A filesystem *will*, although you need to be able to mount it; furthermore, it makes it trivially to extend it (and the firmware interface provides an . easy way to feed the data to such a filesystem without having to muck with anything magic.)

Heck, we could even make it a squashfs image that can just be mounted.

So, first of all, where does Android keep its modules, and what is actually included? Is /sbin/modprobe used to load the modules, as is normal? We might even be able to address this with some fairly trivial enhancements to modprobe; specifically to search in the module paths for something that isn't a module per se.

FWIW, 'modprobe' does exist on Android. Although most of the times,
Init's builtin insmod function[1] gets used.

The module locations are /{system, vendor,odm}/lib/modules and yes,
modprobe can be made to load modules from there too. [2]

- ssp

The best scenario would be if we could simply have the tools find the location equivalent of /lib/modules/$version/source...

[1] https://android.googlesource.com/platform/system/core/+/master/init/builtins.cpp#230
[2] https://source.android.com/devices/architecture/kernel/modular-kernels

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.