Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial recipe for repo 2.17.3

From: Jasper Orschulko <hidden>
Date: 2021-11-11 12:10:33
Also in: bitbake-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Peter,

However, this means 
that only the source for the repo wrapper is available in DL_DIR. So 
when the repo wrapper executes it will try to go on the Internet to 
fetch the rest of the repo source, which will fail.

just to clearify: Are we talking about installing repo on a target, or
are we talking about the repo fetcher?

When installing repo on the target, we do not call any "repo" command
within the bitbake build process. We just clone the git repo containing
the wrapper script and install the wrapper script into the sysroot. At
that point the bitbake build process is finished and the user ends up
with just the wrapper script on his device. Just as he would when
installing repo from the system repositories on debian, arch, etc.

When using the fetcher, the repo sources are fetched during the initial
do_fetch (this also works with `bitbake --runonly=do_fetch`) and placed
into the DL_DIR (in the .repo folder, next to the sources as described
in the repo manifest). So unless the user changes something fetcher
related in the recipe or deletes his DL_DIR, he can use this build
environment for an offline build.

You can try this for yourself, if you like:

1) clone https://github.com/Jasper-Ben/demo-kas
2) (with docker and docker-compose installed) run `make offline-build`

- -- 
With best regards

Jasper Orschulko
DevOps Engineer

Tel. +49 30 58 58 14 265
Fax +49 30 58 58 14 999
Jasper.Orschulko@iris-sensing.com

• • • • • • • • • • • • • • • • • • • • • • • • • •

iris-GmbH
infrared & intelligent sensors
Schnellerstraße 1-5 | 12439 Berlin

https://iris-sensing.com/





On Thu, 2021-11-11 at 11:34 +0000, Peter Kjellerstedt wrote:

quoted

-----Original Message-----
From: Jasper Orschulko <redacted>
Sent: den 11 november 2021 11:05
To: richard.purdie@linuxfoundation.org; openembedded-
core@lists.openembedded.org; kweihmann@outlook.com; Peter
Kjellerstedt
[off-list ref]; jasper@fancydomain.eu
Cc: martin@mko.dev; Daniel Baumgart
[off-list ref];
bitbake-devel@lists.openembedded.org
Subject: Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools: Initial
recipe
for repo 2.17.3

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Peter,

quoted

How do you avoid the repo wrapper fetching the repo source itself
and instead fetch it using the bitbake fetcher? I have seen nothing
to that extent in the patches so far.

we don't. This recipe only installs the repo wrapper. The source code
is downloaded and installed when running `repo init`.

However, in our opinion this is not an issue. When installing repo as
a
package to a target, this is the expected behaviour. The target would
only have the repo wrapper installed, just like on any other linux
distrobution. The actual usage of repo on the target is decoupled
from
the bitbake build process.

It might be how repo is designed, but it will still break the bitbake
expectations. I.e., if I have an environment with the layers available,
a populated DL_DIR and BB_NO_NETWORK = "1", and then disconnect the 
build host from the Internet, I should still be able to source 
oe-init-build-env for a new machine and build it. However, this means
that only the source for the repo wrapper is available in DL_DIR. So 
when the repo wrapper executes it will try to go on the Internet to 
fetch the rest of the repo source, which will fail.

quoted

When using the repo fetcher, the repo source is fetched during the
do_fetch stage by running `repo init` (when SRCREV = AUTOREV, changes
to the recipe or no previous sources available in DL_DIR). By
executing
this within the "runfetchcmd" function, this also works with the
usual
network features bitbake provides, e.g. proxy.
If the recipe has not changed and sources are already available from
a
previous run, repo will not be rerun. As such, reproducing a build
offline is also possible.

- --
With best regards

Jasper Orschulko
DevOps Engineer

Tel. +49 30 58 58 14 265
Fax +49 30 58 58 14 999
Jasper.Orschulko@iris-sensing.com

• • • • • • • • • • • • • • • • • • • • • • • • • •

iris-GmbH
infrared & intelligent sensors
Schnellerstraße 1-5 | 12439 Berlin

https://iris-sensing.com/





On Wed, 2021-11-10 at 23:55 +0000, Peter Kjellerstedt wrote:

quoted

-----Original Message-----
From: bitbake-devel@lists.openembedded.org <bitbake-
devel@lists.openembedded.org> On Behalf Of Jasper Orschulko
Sent: den 9 november 2021 12:26
To: richard.purdie@linuxfoundation.org; openembedded-
core@lists.openembedded.org; kweihmann@outlook.com;
jasper@fancydomain.eu
Cc: martin@mko.dev; Daniel.Baumgart@iris-sensing.net; bitbake-
devel@lists.openembedded.org
Subject: Re: [bitbake-devel] [oe-core][PATCH 1/2] devtools:
Initial
recipe
for repo 2.17.3

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Richard,

I think our implementation of the repo fetcher checks the (what I
believe to be) most relevant parts of your checklist (thanks for
the
write-up). Martin, please corrent me if I'm missing something:

quoted

a) network access for sources is only expected to happen in the
do_fetch step.
This is not enforced or tested but is required so that we can:

 i) audit the sources used (i.e. for license/manifest reasons)
 ii) support offline builds with a suitable cache
 iii) allow work to continue even with downtime upstream
 iv) allow for changes upstream in incompatible ways
 v) allow rebuilding of the software in X years time

check

quoted

b) network access is not expected in do_unpack

check

How do you avoid the repo wrapper fetching the repo source itself
and instead fetch it using the bitbake fetcher? I have seen nothing
to that extent in the patches so far.

//Peter

`h`	back out one level
`j`	next message in thread
`k`	previous message in thread
`l`	drill in
`Esc`	close help / fold thread tree
`?`	toggle this help