With recipe specific sysroots this should now be deterministic. The log
above suggests you were not using recipe specific sysroots? This would
therefore only be applicable to morty and earlier?
Cheers,
Richard
Comments below. thanks
-----Original Message-----
From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org]
Sent: 2017年12月11日 6:48
To: Bai, Haiqing; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] gmp: depends on flex-native to fix parallel building issue
On Wed, 2017-11-29 at 14:54 +0800, Haiqing Bai wrote:
quoted hunk
fix below parallel building issue:
configure:27365: result: flex
configure:27403: flex conftest.l
.../sysroots/x86_64-linux/usr/bin/flex.real: No such file or directory
configure:27407: $? = 127
configure:27409: checking lex output file root
configure:27420: error: cannot find output from flex; giving up
Signed-off-by: Haiqing Bai <redacted>
---
meta/recipes-support/gmp/gmp.inc | 2 ++
1 file changed, 2 insertions(+)
With recipe specific sysroots this should now be deterministic. The log above suggests you were not using recipe specific sysroots? This would therefore only be applicable to morty and earlier?
[This issue is founded on x86-64 building, but it does not mean it is only related with x86. Actually this is caused by the defect of the traditional probe mechanism of 'configure', the package configure
script try to probe whether has package 'flex' , then some optional actions will be done by it. In this issue, when this probe happens, /usr/bin/flex exists but '/usr/bin/flex.real' has not created for the
parallel building, then configure reports the error and exits from building. Since there are no atomic guarantee for the package output during parallel building, so here add this depends]
Cheers,
Richard
From: Richard Purdie <hidden> Date: 2017-12-12 09:22:33
Let me make this simpler. Which release of the project did you see this
issue with?
I believe this issue would only occur with morty and early, pyro, rocko
and master are not affected.
Cheers,
Richard
On Tue, 2017-12-12 at 07:13 +0000, Bai, Haiqing wrote:
Comments below. thanks
-----Original Message-----
From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org]
Sent: 2017年12月11日 6:48
To: Bai, Haiqing; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] gmp: depends on flex-native to fix
parallel building issue
On Wed, 2017-11-29 at 14:54 +0800, Haiqing Bai wrote:
quoted
fix below parallel building issue:
configure:27365: result: flex
configure:27403: flex conftest.l
.../sysroots/x86_64-linux/usr/bin/flex.real: No such file or
directory
configure:27407: $? = 127
configure:27409: checking lex output file root
configure:27420: error: cannot find output from flex; giving up
Signed-off-by: Haiqing Bai <redacted>
---
meta/recipes-support/gmp/gmp.inc | 2 ++
1 file changed, 2 insertions(+)
With recipe specific sysroots this should now be deterministic. The
log above suggests you were not using recipe specific sysroots? This
would therefore only be applicable to morty and earlier?
[This issue is founded on x86-64 building, but it does not mean it
is only related with x86. Actually this is caused by the defect of
the traditional probe mechanism of 'configure', the package
configure
script try to probe whether has package 'flex' , then some optional
actions will be done by it. In this issue, when this probe
happens, /usr/bin/flex exists but '/usr/bin/flex.real' has not
created for the
parallel building, then configure reports the error and exits from
building. Since there are no atomic guarantee for the package
output during parallel building, so here add this depends]
Cheers,
Richard
Yes, sir, I did not reproduce this on master. Even on morty, I only saw
twice. Maybe we can hold on this and let me keep monitoring. Thanks.
-----Original Message-----
From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org]
Sent: 2017年12月12日 17:23
To: Bai, Haiqing; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] gmp: depends on flex-native to fix parallel building issue
Let me make this simpler. Which release of the project did you see this issue with?
I believe this issue would only occur with morty and early, pyro, rocko and master are not affected.
Cheers,
Richard
On Tue, 2017-12-12 at 07:13 +0000, Bai, Haiqing wrote:
Comments below. thanks
-----Original Message-----
From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org]
Sent: 2017年12月11日 6:48
To: Bai, Haiqing; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] gmp: depends on flex-native to fix
parallel building issue
On Wed, 2017-11-29 at 14:54 +0800, Haiqing Bai wrote:
quoted
fix below parallel building issue:
configure:27365: result: flex
configure:27403: flex conftest.l
.../sysroots/x86_64-linux/usr/bin/flex.real: No such file or
directory
configure:27407: $? = 127
configure:27409: checking lex output file root
configure:27420: error: cannot find output from flex; giving up
Signed-off-by: Haiqing Bai <redacted>
---
meta/recipes-support/gmp/gmp.inc | 2 ++
1 file changed, 2 insertions(+)
With recipe specific sysroots this should now be deterministic. The
log above suggests you were not using recipe specific sysroots? This
would therefore only be applicable to morty and earlier?
[This issue is founded on x86-64 building, but it does not mean it is
only related with x86. Actually this is caused by the defect of the
traditional probe mechanism of 'configure', the package configure
script try to probe whether has package 'flex' , then some optional
actions will be done by it. In this issue, when this probe happens,
/usr/bin/flex exists but '/usr/bin/flex.real' has not created for the
parallel building, then configure reports the error and exits from
building. Since there are no atomic guarantee for the package output
during parallel building, so here add this depends]
Cheers,
Richard
From: Randy MacLeod <hidden> Date: 2018-01-03 21:35:39
On 2017-12-12 04:22 AM, Richard Purdie wrote:
Let me make this simpler. Which release of the project did you see this
issue with?
I believe this issue would only occur with morty and early, pyro, rocko
and master are not affected.
I expect I'm missing an undocumented recipe rule here but...
Yes, the race seems to only occur on morty and earlier, but
the dependency is real and not yet listed explicitly in the recipe.
$ grep flex tmp/work/i586-poky-linux/gmp/6.1.2-r0/temp/log.do_configure
checking for flex... flex
It's resolved transitively through:
gmp -> something -> binutils-cross-foo -> flex-native
I was on the fence about whether to be explicit or to rely on
most/all recipes pulling in binutils-cross-foo via default
rules. In looking at the guidance in 4.3.9 Dependencies:
http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#new-dependencies
and briefly checking out LLVM's lld loader recipe where it seems
that flex-native might not be needed so it's best to be explicit.
Unless there's an objection, Bai will re-submit the simple
fix with an updated commit log. If anyone has time, they
can check if the LLVM lld build really does NOT pull in flex-native
and if not, then add explicit dependencies as required.
../Randy
Cheers,
Richard
On Tue, 2017-12-12 at 07:13 +0000, Bai, Haiqing wrote:
quoted
Comments below. thanks
-----Original Message-----
From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org]
Sent: 2017年12月11日 6:48
To: Bai, Haiqing; openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] gmp: depends on flex-native to fix
parallel building issue
On Wed, 2017-11-29 at 14:54 +0800, Haiqing Bai wrote:
quoted
fix below parallel building issue:
configure:27365: result: flex
configure:27403: flex conftest.l
.../sysroots/x86_64-linux/usr/bin/flex.real: No such file or
directory
configure:27407: $? = 127
configure:27409: checking lex output file root
configure:27420: error: cannot find output from flex; giving up
Signed-off-by: Haiqing Bai <redacted>
---
meta/recipes-support/gmp/gmp.inc | 2 ++
1 file changed, 2 insertions(+)
With recipe specific sysroots this should now be deterministic. The
log above suggests you were not using recipe specific sysroots? This
would therefore only be applicable to morty and earlier?
[This issue is founded on x86-64 building, but it does not mean it
is only related with x86. Actually this is caused by the defect of
the traditional probe mechanism of 'configure', the package
configure
script try to probe whether has package 'flex' , then some optional
actions will be done by it. In this issue, when this probe
happens, /usr/bin/flex exists but '/usr/bin/flex.real' has not
created for the
parallel building, then configure reports the error and exits from
building. Since there are no atomic guarantee for the package
output during parallel building, so here add this depends]
Cheers,
Richard
--
# Randy MacLeod. WR Linux
# Wind River an Intel Company
From: Richard Purdie <hidden> Date: 2018-01-03 21:51:05
On Wed, 2018-01-03 at 16:35 -0500, Randy MacLeod wrote:
I expect I'm missing an undocumented recipe rule here but...
Yes, the race seems to only occur on morty and earlier, but
the dependency is real and not yet listed explicitly in the recipe.
$ grep flex tmp/work/i586-poky-linux/gmp/6.1.2-
r0/temp/log.do_configure
checking for flex... flex
It's resolved transitively through:
gmp -> something -> binutils-cross-foo -> flex-native
I think it can happily build without flex by not re-generating certain
files. Whilst on target gmp does seem to have an indirect dependency,
our cross compiler does not:
$ grep flex tmp/work/x86_64-linux/gmp-native/6.1.2-r0/temp/log.do_configure
checking for flex... no
due to recipe specific sysroots and HOSTTOOLS, this is all at least
deterministic and it either builds with or without it deterministically
in all cases.
I was on the fence about whether to be explicit or to rely on
most/all recipes pulling in binutils-cross-foo via default
rules. In looking at the guidance in 4.3.9 Dependencies:
http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#n
ew-dependencies
and briefly checking out LLVM's lld loader recipe where it seems
that flex-native might not be needed so it's best to be explicit.
Unless there's an objection, Bai will re-submit the simple
fix with an updated commit log. If anyone has time, they
can check if the LLVM lld build really does NOT pull in flex-native
and if not, then add explicit dependencies as required.
Actually, I do strongly object to adding in a flex-native dependency to
gmp-native and lengthening an already long/cumbersome toolchain
bootstrap process.
Cheers,
Richard