[PATCH bpf-next v2] samples/bpf: Update README.rst and Makefile for manually compiling LLVM and clang

The current llvm/clang build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed

The llvm community has adopted new ways to build the compiler. There are
different ways to build llvm/clang, the Clang Getting Started page [1] has
one way. As Yonghong said, it is better to just copy the build procedure
in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

[1] https://clang.llvm.org/get_started.html

Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Acked-by: Yonghong Song <redacted>
---

v2: Update the commit message suggested by Yonghong,
    thank you very much.

 samples/bpf/Makefile   |  2 +-
 samples/bpf/README.rst | 17 ++++++++++-------
 2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 26fc96c..d061446 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile

@@ -208,7 +208,7 @@ TPROGLDLIBS_xdpsock		+= -pthread -lcap
 TPROGLDLIBS_xsk_fwd		+= -pthread
 
 # Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline:
-#  make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+# make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
 LLC ?= llc
 CLANG ?= clang
 OPT ?= opt

diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index dd34b2d..d1be438 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst

@@ -65,17 +65,20 @@ To generate a smaller llc binary one can use::
 Quick sniplet for manually compiling LLVM and clang
 (build dependencies are cmake and gcc-c++)::
 
- $ git clone http://llvm.org/git/llvm.git
- $ cd llvm/tools
- $ git clone --depth 1 http://llvm.org/git/clang.git
- $ cd ..; mkdir build; cd build
- $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
- $ make -j $(getconf _NPROCESSORS_ONLN)
+ $ git clone https://github.com/llvm/llvm-project.git
+ $ mkdir -p llvm-project/llvm/build/install
+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+            -DLLVM_ENABLE_PROJECTS="clang"    \
+            -DBUILD_SHARED_LIBS=OFF           \
+            -DCMAKE_BUILD_TYPE=Release        \
+            -DLLVM_BUILD_RUNTIME=OFF
+ $ ninja
 
 It is also possible to point make to the newly compiled 'llc' or
 'clang' command via redefining LLC or CLANG on the make command line::
 
- make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
 
 Cross compiling samples
 -----------------------

-- 
2.1.0

On 2021-01-19, Tiezhu Yang wrote:

llvm-project/llvm/build/install is not used.

+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+            -DLLVM_ENABLE_PROJECTS="clang"    \
+            -DBUILD_SHARED_LIBS=OFF           \

-DBUILD_SHARED_LIBS=OFF is the default. It can be omitted.

+            -DCMAKE_BUILD_TYPE=Release        \
+            -DLLVM_BUILD_RUNTIME=OFF

-DLLVM_BUILD_RUNTIME=OFF can be omitted if none of
compiler-rt/libc++/libc++abi is built.

+ $ ninja

It is also possible to point make to the newly compiled 'llc' or
'clang' command via redefining LLC or CLANG on the make command line::

- make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang

Cross compiling samples
-----------------------
-- 
2.1.0

-- 
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/1611042978-21473-1-git-send-email-yangtiezhu%40loongson.cn.

On 01/20/2021 05:58 AM, Fangrui Song wrote:

On 2021-01-19, Tiezhu Yang wrote:

quoted

The current llvm/clang build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) 
redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum 
(20) redirects followed

The llvm community has adopted new ways to build the compiler. There are
different ways to build llvm/clang, the Clang Getting Started page 
[1] has
one way. As Yonghong said, it is better to just copy the build procedure
in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

[1] https://clang.llvm.org/get_started.html

Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Acked-by: Yonghong Song <redacted>
---

v2: Update the commit message suggested by Yonghong,
   thank you very much.

samples/bpf/Makefile   |  2 +-
samples/bpf/README.rst | 17 ++++++++++-------
2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 26fc96c..d061446 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile

@@ -208,7 +208,7 @@ TPROGLDLIBS_xdpsock        += -pthread -lcap

TPROGLDLIBS_xsk_fwd        += -pthread

# Allows pointing LLC/CLANG to a LLVM backend with bpf support, 
redefine on cmdline:
-#  make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc 
CLANG=~/git/llvm/build/bin/clang
+# make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc 
CLANG=~/git/llvm-project/llvm/build/bin/clang
LLC ?= llc
CLANG ?= clang
OPT ?= opt

diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index dd34b2d..d1be438 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst

@@ -65,17 +65,20 @@ To generate a smaller llc binary one can use::

Quick sniplet for manually compiling LLVM and clang
(build dependencies are cmake and gcc-c++)::

- $ git clone http://llvm.org/git/llvm.git
- $ cd llvm/tools
- $ git clone --depth 1 http://llvm.org/git/clang.git
- $ cd ..; mkdir build; cd build
- $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
- $ make -j $(getconf _NPROCESSORS_ONLN)
+ $ git clone https://github.com/llvm/llvm-project.git
+ $ mkdir -p llvm-project/llvm/build/install

llvm-project/llvm/build/install is not used.

Yes, just mkdir -p llvm-project/llvm/build is OK.

quoted

+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+            -DLLVM_ENABLE_PROJECTS="clang"    \
+            -DBUILD_SHARED_LIBS=OFF           \

-DBUILD_SHARED_LIBS=OFF is the default. It can be omitted.

I search the related doc [1] [2], remove this option is OK for me.

BUILD_SHARED_LIBS:BOOL

     Flag indicating if each LLVM component (e.g. Support) is built as a 
shared library (ON) or as a static library (OFF). Its default value is OFF.

[1] https://www.llvm.org/docs/CMake.html
[2] https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html

quoted

+ -DCMAKE_BUILD_TYPE=Release        \
+            -DLLVM_BUILD_RUNTIME=OFF

-DLLVM_BUILD_RUNTIME=OFF can be omitted if none of
compiler-rt/libc++/libc++abi is built.

I am not very sure about it because the default value of
LLVM_BUILD_RUNTIME is ON? [3]

option(LLVM_BUILD_RUNTIME
"Build the LLVM runtime libraries." ON)

[3] https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt

If anyone has any more suggestions, please let me know.
I will send v3 after waiting for other feedback.

By the way, Documentation/bpf/bpf_devel_QA.rst maybe need a separate
patch to remove some cmake options?

Thanks,
Tiezhu

quoted

+ $ ninja

It is also possible to point make to the newly compiled 'llc' or
'clang' command via redefining LLC or CLANG on the make command line::

- make M=samples/bpf LLC=~/git/llvm/build/bin/llc 
CLANG=~/git/llvm/build/bin/clang
+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc 
CLANG=~/git/llvm-project/llvm/build/bin/clang

Cross compiling samples
-----------------------
-- 
2.1.0

-- 
You received this message because you are subscribed to the Google 
Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, 
send an email to clang-built-linux+unsubscribe@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clang-built-linux/1611042978-21473-1-git-send-email-yangtiezhu%40loongson.cn.

On 01/20/2021 10:43 AM, Nick Desaulniers wrote:

On Mon, Jan 18, 2021 at 11:56 PM Tiezhu Yang [off-list ref] wrote:

quoted

The current llvm/clang build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed

The llvm community has adopted new ways to build the compiler. There are
different ways to build llvm/clang, the Clang Getting Started page [1] has
one way. As Yonghong said, it is better to just copy the build procedure
in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

[1] https://clang.llvm.org/get_started.html

There's also https://www.kernel.org/doc/html/latest/kbuild/llvm.html#getting-llvm
(could cross link in rst/sphinx).

quoted

Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Acked-by: Yonghong Song <redacted>
---

v2: Update the commit message suggested by Yonghong,
     thank you very much.

  samples/bpf/Makefile   |  2 +-
  samples/bpf/README.rst | 17 ++++++++++-------
  2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 26fc96c..d061446 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile

@@ -208,7 +208,7 @@ TPROGLDLIBS_xdpsock         += -pthread -lcap
  TPROGLDLIBS_xsk_fwd            += -pthread

  # Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline:
-#  make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+# make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang
  LLC ?= llc
  CLANG ?= clang
  OPT ?= opt

diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index dd34b2d..d1be438 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst

@@ -65,17 +65,20 @@ To generate a smaller llc binary one can use::
  Quick sniplet for manually compiling LLVM and clang
  (build dependencies are cmake and gcc-c++)::

- $ git clone http://llvm.org/git/llvm.git
- $ cd llvm/tools
- $ git clone --depth 1 http://llvm.org/git/clang.git
- $ cd ..; mkdir build; cd build
- $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"

Is the BPF target not yet on by default?

The default value includes BPF.

  I frown upon disabling other backends.

I think the initial aim of the build procedure in samples/bpf/README.rst [1]
is a simple and quick start guide:

[ To generate a smaller llc binary one can use:

  -DLLVM_TARGETS_TO_BUILD="BPF" ]

[1] https://github.com/torvalds/linux/blob/master/samples/bpf/README.rst

quoted

- $ make -j $(getconf _NPROCESSORS_ONLN)
+ $ git clone https://github.com/llvm/llvm-project.git
+ $ mkdir -p llvm-project/llvm/build/install
+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+            -DLLVM_ENABLE_PROJECTS="clang"    \
+            -DBUILD_SHARED_LIBS=OFF           \
+            -DCMAKE_BUILD_TYPE=Release        \
+            -DLLVM_BUILD_RUNTIME=OFF
+ $ ninja

  It is also possible to point make to the newly compiled 'llc' or
  'clang' command via redefining LLC or CLANG on the make command line::

- make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang

  Cross compiling samples
  -----------------------
--
2.1.0

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/1611042978-21473-1-git-send-email-yangtiezhu%40loongson.cn.

On 1/19/21 6:32 PM, Tiezhu Yang wrote:

On 01/20/2021 05:58 AM, Fangrui Song wrote:

quoted

On 2021-01-19, Tiezhu Yang wrote:

quoted

The current llvm/clang build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

$ git clone 
http://llvm.org/git/llvm.git 
Cloning into 'llvm'...
fatal: unable to access 
'http://llvm.org/git/llvm.git/ 
': Maximum (20) redirects followed
$ git clone --depth 1 
http://llvm.org/git/clang.git 
Cloning into 'clang'...
fatal: unable to access 
'http://llvm.org/git/clang.git/ 
': Maximum (20) redirects followed

The llvm community has adopted new ways to build the compiler. There are
different ways to build llvm/clang, the Clang Getting Started page 
[1] has
one way. As Yonghong said, it is better to just copy the build procedure
in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

[1] 
https://clang.llvm.org/get_started.html 

Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Acked-by: Yonghong Song <redacted>
---

v2: Update the commit message suggested by Yonghong,
   thank you very much.

samples/bpf/Makefile   |  2 +-
samples/bpf/README.rst | 17 ++++++++++-------
2 files changed, 11 insertions(+), 8 deletions(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 26fc96c..d061446 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile

@@ -208,7 +208,7 @@ TPROGLDLIBS_xdpsock        += -pthread -lcap

TPROGLDLIBS_xsk_fwd        += -pthread

# Allows pointing LLC/CLANG to a LLVM backend with bpf support, 
redefine on cmdline:
-#  make M=samples/bpf/ LLC=~/git/llvm/build/bin/llc 
CLANG=~/git/llvm/build/bin/clang
+# make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc 
CLANG=~/git/llvm-project/llvm/build/bin/clang
LLC ?= llc
CLANG ?= clang
OPT ?= opt

diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index dd34b2d..d1be438 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst

@@ -65,17 +65,20 @@ To generate a smaller llc binary one can use::

Quick sniplet for manually compiling LLVM and clang
(build dependencies are cmake and gcc-c++)::

- $ git clone 
http://llvm.org/git/llvm.git 
- $ cd llvm/tools
- $ git clone --depth 1 
http://llvm.org/git/clang.git 
- $ cd ..; mkdir build; cd build
- $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
- $ make -j $(getconf _NPROCESSORS_ONLN)
+ $ git clone https://github.com/llvm/llvm-project.git
+ $ mkdir -p llvm-project/llvm/build/install

llvm-project/llvm/build/install is not used.

Yes, just mkdir -p llvm-project/llvm/build is OK.

quoted

quoted

+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+            -DLLVM_ENABLE_PROJECTS="clang"    \
+            -DBUILD_SHARED_LIBS=OFF           \

-DBUILD_SHARED_LIBS=OFF is the default. It can be omitted.

I search the related doc [1] [2], remove this option is OK for me.

BUILD_SHARED_LIBS:BOOL

     Flag indicating if each LLVM component (e.g. Support) is built as a 
shared library (ON) or as a static library (OFF). Its default value is OFF.

[1] 
https://www.llvm.org/docs/CMake.html 
[2] 
https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html 

quoted

quoted

+ -DCMAKE_BUILD_TYPE=Release        \
+            -DLLVM_BUILD_RUNTIME=OFF

-DLLVM_BUILD_RUNTIME=OFF can be omitted if none of
compiler-rt/libc++/libc++abi is built.

I am not very sure about it because the default value of
LLVM_BUILD_RUNTIME is ON? [3]

option(LLVM_BUILD_RUNTIME
"Build the LLVM runtime libraries." ON)

[3] https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt

If anyone has any more suggestions, please let me know.
I will send v3 after waiting for other feedback.

By the way, Documentation/bpf/bpf_devel_QA.rst maybe need a separate
patch to remove some cmake options?

Please submit updated this patch and Documentation/bpf/bpf_devel_QA.rst
together. This way, it is easy to cross check. Thanks.

Thanks,
Tiezhu

[...]

On Mon, Jan 18, 2021 at 11:56 PM Tiezhu Yang [off-list ref] wrote:

The current llvm/clang build procedure in samples/bpf/README.rst is
out of date. See below that the links are not accessible any more.

$ git clone http://llvm.org/git/llvm.git
Cloning into 'llvm'...
fatal: unable to access 'http://llvm.org/git/llvm.git/': Maximum (20) redirects followed
$ git clone --depth 1 http://llvm.org/git/clang.git
Cloning into 'clang'...
fatal: unable to access 'http://llvm.org/git/clang.git/': Maximum (20) redirects followed

The llvm community has adopted new ways to build the compiler. There are
different ways to build llvm/clang, the Clang Getting Started page [1] has
one way. As Yonghong said, it is better to just copy the build procedure
in Documentation/bpf/bpf_devel_QA.rst to keep consistent.

I verified the procedure and it is proved to be feasible, so we should
update README.rst to reflect the reality. At the same time, update the
related comment in Makefile.

[1] https://clang.llvm.org/get_started.html

There's also https://www.kernel.org/doc/html/latest/kbuild/llvm.html#getting-llvm
(could cross link in rst/sphinx).

Is the BPF target not yet on by default?  I frown upon disabling other backends.

- $ make -j $(getconf _NPROCESSORS_ONLN)
+ $ git clone https://github.com/llvm/llvm-project.git
+ $ mkdir -p llvm-project/llvm/build/install
+ $ cd llvm-project/llvm/build
+ $ cmake .. -G "Ninja" -DLLVM_TARGETS_TO_BUILD="BPF;X86" \
+            -DLLVM_ENABLE_PROJECTS="clang"    \
+            -DBUILD_SHARED_LIBS=OFF           \
+            -DCMAKE_BUILD_TYPE=Release        \
+            -DLLVM_BUILD_RUNTIME=OFF
+ $ ninja

 It is also possible to point make to the newly compiled 'llc' or
 'clang' command via redefining LLC or CLANG on the make command line::

- make M=samples/bpf LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
+ make M=samples/bpf LLC=~/git/llvm-project/llvm/build/bin/llc CLANG=~/git/llvm-project/llvm/build/bin/clang

 Cross compiling samples
 -----------------------
--
2.1.0

--
You received this message because you are subscribed to the Google Groups "Clang Built Linux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clang-built-linux+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clang-built-linux/1611042978-21473-1-git-send-email-yangtiezhu%40loongson.cn.

-- 
Thanks,
~Nick Desaulniers