On Sat, Mar 31, 2018 at 11:20:22AM +0900, Masahiro Yamada wrote:

2018-03-31 7:21 GMT+09:00 Andrei Vagin [off-list ref]:

quoted

On Fri, Mar 30, 2018 at 10:40:22AM -0700, Andrei Vagin wrote:

quoted

On Fri, Mar 23, 2018 at 10:04:32PM +0900, Masahiro Yamada wrote:

quoted

Now that the kernel build supports flex and bison, remove the _shipped
files and generate them during the build instead.

There are no more shipped lexer and parser, so I ripped off the rules
in scripts/Malefile.lib that were used for REGENERATE_PARSERS.

The genksyms parser has ambiguous grammar, which would emit warnings:

 scripts/genksyms/parse.y: warning: 9 shift/reduce conflicts [-Wconflicts-sr]
 scripts/genksyms/parse.y: warning: 5 reduce/reduce conflicts [-Wconflicts-rr]

They are normally suppressed, but displayed when W=1 is given.

Signed-off-by: Masahiro Yamada <redacted>
---

 scripts/Makefile.lib                 |   24 +-
 scripts/genksyms/Makefile            |   23 +
 scripts/genksyms/lex.lex.c_shipped   | 2291 --------------------------------
 scripts/genksyms/parse.tab.c_shipped | 2394 ----------------------------------
 scripts/genksyms/parse.tab.h_shipped |  119 --
 5 files changed, 26 insertions(+), 4825 deletions(-)
 delete mode 100644 scripts/genksyms/lex.lex.c_shipped
 delete mode 100644 scripts/genksyms/parse.tab.c_shipped
 delete mode 100644 scripts/genksyms/parse.tab.h_shipped

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 2fde810..b7d2c97 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib

@@ -183,14 +183,8 @@ endef
 quiet_cmd_flex = LEX     $@
       cmd_flex = $(LEX) -o$@ -L $<

-ifdef REGENERATE_PARSERS
-.PRECIOUS: $(src)/%.lex.c_shipped
-$(src)/%.lex.c_shipped: $(src)/%.l
-   $(call cmd,flex)
-endif
-
 .PRECIOUS: $(obj)/%.lex.c
-$(filter %.lex.c,$(targets)): $(obj)/%.lex.c: $(src)/%.l FORCE
+$(obj)/%.lex.c: $(src)/%.l FORCE
    $(call if_changed,flex)

 # YACC

@@ -198,27 +192,15 @@ $(filter %.lex.c,$(targets)): $(obj)/%.lex.c: $(src)/%.l FORCE
 quiet_cmd_bison = YACC    $@
       cmd_bison = $(YACC) -o$@ -t -l $<

-ifdef REGENERATE_PARSERS
-.PRECIOUS: $(src)/%.tab.c_shipped
-$(src)/%.tab.c_shipped: $(src)/%.y
-   $(call cmd,bison)
-endif
-
 .PRECIOUS: $(obj)/%.tab.c
-$(filter %.tab.c,$(targets)): $(obj)/%.tab.c: $(src)/%.y FORCE
+$(obj)/%.tab.c: $(src)/%.y FORCE
    $(call if_changed,bison)

 quiet_cmd_bison_h = YACC    $@
       cmd_bison_h = bison -o/dev/null --defines=$@ -t -l $<

-ifdef REGENERATE_PARSERS
-.PRECIOUS: $(src)/%.tab.h_shipped
-$(src)/%.tab.h_shipped: $(src)/%.y
-   $(call cmd,bison_h)
-endif
-
 .PRECIOUS: $(obj)/%.tab.h
-$(filter %.tab.h,$(targets)): $(obj)/%.tab.h: $(src)/%.y FORCE
+$(obj)/%.tab.h: $(src)/%.y FORCE
    $(call if_changed,bison_h)

 # Shipped files

diff --git a/scripts/genksyms/Makefile b/scripts/genksyms/Makefile
index 0ccac51..f4749e8 100644
--- a/scripts/genksyms/Makefile
+++ b/scripts/genksyms/Makefile

@@ -5,9 +5,32 @@ always             := $(hostprogs-y)

 genksyms-objs      := genksyms.o parse.tab.o lex.lex.o

+# FIXME: fix the ambiguous grammar in parse.y and delete this hack
+#
+# Suppress shift/reduce, reduce/reduce conflicts warnings
+# unless W=1 is specified.
+ifeq ($(findstring 1,$(KBUILD_ENABLE_EXTRA_GCC_CHECKS)),)
+SUPPRESS_BISON_WARNING := 2>/dev/null

We have a robot which runs CRIU tests on linux-next.
Yesterday it failed with this error:

  HOSTCC  scripts/genksyms/genksyms.o
make[2]: *** [scripts/genksyms/parse.tab.c] Error 127

cripts/genksyms/Makefile:20: recipe for target 'scripts/genksyms/parse.tab.c' failed
scripts/Makefile.build:559: recipe for target 'scripts/genksyms' failed
Makefile:1073: recipe for target 'scripts' failed
make[1]: *** [scripts/genksyms] Error 2
make: *** [scripts] Error 2
make: *** Waiting for unfinished jobs....

https://travis-ci.org/avagin/linux/jobs/360056903

From this output, it is very hard to understand what was going wrong.

The reason was that bison and fles were not installed, but I think the
error message should be more clear.

quoted

Thanks,
Andrei

Thanks for the report.


OK, I will apply the fix-up attached below.

If bison is not installed, it will fail with clear message.

Thank you!

  HOSTCC  scripts/genksyms/genksyms.o
/bin/sh: 1: bison: not found
make[2]: *** [scripts/genksyms/Makefile:18:
scripts/genksyms/parse.tab.c] Error 127
make[1]: *** [scripts/Makefile.build:559: scripts/genksyms] Error 2
make: *** [Makefile:1073: scripts] Error 2


BTW, without flex and bison, how did you build Kconfig?

A config is created on the host, but a kernel is compiled in a docker
container. Maybe bison was installed on the host.

We do something like this:

$ make olddefconfig
$ make localyesconfig
$ docker run -v /home/travis/build/avagin/linux:/mnt/kernel -v
/home/travis/.ccache:/mnt/ccache -w /mnt/kernel criu-kernel make -j 4