Commit Graph

53 Commits

Author SHA1 Message Date
Masahiro Yamada
cb08a02659 kbuild: rpm-pkg: disable kernel-devel package when cross-compiling
Since commit f1d87664b8 ("kbuild: cross-compile linux-headers package
when possible"), 'make binrpm-pkg' may attempt to cross-compile the
kernel-devel package, but it fails under certain circumstances.

For example, when CONFIG_MODULE_SIG_FORMAT is enabled on openSUSE
Tumbleweed, the following command fails:

  $ make ARCH=arm64 CROSS_COMPILE=aarch64-suse-linux- binrpm-pkg
      [ snip ]
  Rebuilding host programs with aarch64-suse-linux-gcc...
    HOSTCC  /home/masahiro/ref/linux/rpmbuild/BUILDROOT/kernel-6.12.0_rc4-1.aarch64/usr/src/kernels/6.12.0-rc4/scripts/kallsyms
    HOSTCC  /home/masahiro/ref/linux/rpmbuild/BUILDROOT/kernel-6.12.0_rc4-1.aarch64/usr/src/kernels/6.12.0-rc4/scripts/sorttable
    HOSTCC  /home/masahiro/ref/linux/rpmbuild/BUILDROOT/kernel-6.12.0_rc4-1.aarch64/usr/src/kernels/6.12.0-rc4/scripts/asn1_compiler
    HOSTCC  /home/masahiro/ref/linux/rpmbuild/BUILDROOT/kernel-6.12.0_rc4-1.aarch64/usr/src/kernels/6.12.0-rc4/scripts/sign-file
  /home/masahiro/ref/linux/rpmbuild/BUILDROOT/kernel-6.12.0_rc4-1.aarch64/usr/src/kernels/6.12.0-rc4/scripts/sign-file.c:25:10: fatal error: openssl/opensslv.h: No such file or directory
     25 | #include <openssl/opensslv.h>
        |          ^~~~~~~~~~~~~~~~~~~~
  compilation terminated.

I believe this issue is less common on Fedora because the disto's cross-
compilier cannot link user-space programs. Hence, CONFIG_CC_CAN_LINK is
unset.

On Fedora 40, the package information explains this limitation clearly:

  $ dnf info gcc-aarch64-linux-gnu
      [ snip ]
  Description  : Cross-build GNU C compiler.
               :
               : Only building kernels is currently supported.  Support for cross-building
               : user space programs is not currently provided as that would massively multiply
               : the number of packages.

Anyway, cross-compiling RPM packages is somewhat challenging.

This commit disables the kernel-devel package when cross-compiling
because I did not come up with a better solution.

Fixes: f1d87664b8 ("kbuild: cross-compile linux-headers package when possible")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
2024-10-31 21:40:46 +09:00
Masahiro Yamada
5b000f3cbb kbuild: pacman-pkg: do not override objtree
objtree is defined and exported by the top-level Makefile. I prefer
not to override it.

There is no need to pass the absolute path of objtree. PKGBUILD can
detect it by itself.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by:  Thomas Weißschuh <linux@weissschuh.net>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Christian Heusel <christian@heusel.eu>
2024-09-01 20:34:49 +09:00
Thomas Weißschuh
c8578539de kbuild: add script and target to generate pacman package
pacman is the package manager used by Arch Linux and its derivates.
Creating native packages from the kernel tree has multiple advantages:

* The package triggers the correct hooks for initramfs generation and
  bootloader configuration
* Uninstallation is complete and also invokes the relevant hooks
* New UAPI headers can be installed without any manual bookkeeping

The PKGBUILD file is a modified version of the one used for the
downstream Arch Linux "linux" package.
Extra steps that should not be necessary for a development kernel have
been removed and an UAPI header package has been added.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2024-07-22 01:24:22 +09:00
Thayne Harbaugh
c615665389 kbuild: Fix build target deb-pkg: ln: failed to create hard link
The make deb-pkg target calls debian-orig which attempts to either
hard link the source .tar to the build-output location or copy the
source .tar to the build-output location.  The test to determine
whether to ln or cp is incorrectly expanded by Make and consequently
always attempts to ln the source .tar.  This fix corrects the escaping
of '$' so that the test is expanded by the shell rather than by Make
and appropriately selects between ln and cp.

Fixes: b44aa8c96e ("kbuild: deb-pkg: make .orig tarball a hard link if possible")
Signed-off-by: Thayne Harbaugh <thayne@mastodonlabs.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2024-06-26 00:18:57 +09:00
Masahiro Yamada
e2bad142bb kbuild: unexport abs_srctree and abs_objtree
Commit 25b146c5b8 ("kbuild: allow Kbuild to start from any directory")
exported abs_srctree and abs_objtree to avoid recomputation after the
sub-make. However, this approach turned out to be fragile.

Commit 5fa94ceb79 ("kbuild: set correct abs_srctree and abs_objtree
for package builds") moved them above "ifneq ($(sub_make_done),1)",
eliminating the need for exporting them.

These are only needed in the top Makefile. If an absolute path is
required in sub-directories, you can use $(abspath ) or $(realpath )
as needed.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2024-03-10 17:27:17 +09:00
Masahiro Yamada
53243e0983 kbuild: deb-pkg: remove the fakeroot builds support
In 2017, the dpkg suite introduced the rootless builds support with the
following commits:

  - 2436807c87b0 ("dpkg-deb: Add support for rootless builds")
  - fca1bfe84068 ("dpkg-buildpackage: Add support for rootless builds")

This feature is available in the default dpkg on Debian 10 and Ubuntu
20.04.

Remove the old method.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-12-10 15:34:37 +09:00
Masahiro Yamada
ef6609adf1 kbuild: remove the last use of old cmd_src_tar rule in packaging
The rpm-pkg and deb-pkg targets have transitioned to using 'git archive'
for tarball creation.

Although the old cmd_src_tar is still used by snap-pkg, there is no need
to pack and unpack a tarball solely for passing the source to snapcraft.

Instead, you can use 'source-type: local' to tell the source location to
snapcraft.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-11-28 11:38:32 +09:00
Dmitrii Bundin
259b8bd13d kbuild: deb-pkg: apply short -R and -j options
The long version --rules-file and --jobs are available since 1.18.8
while their short analogues -R and -j have been added since 1.14.7.

The option --rules-file the way it works currently was introduced in the
commit 5cd52673aabdf5eaa58181972119a41041fc85f2 of dpkg dated 23.07.18
with the following changelog entry:

* Fix dpkg-buildpackage option --rules-file parsing. It was trying to parse
  it as --rules-target, which due to the ordering was a no-op.

The current behavior of the long version --rules-file is guaranteed to
be in use starting 1.19.1 and might cause build failures for some
versions newer than 1.18.8 even in spite of being documented that way.

Signed-off-by: Dmitrii Bundin <dmitrii.bundin.a@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-11-28 11:22:51 +09:00
Masahiro Yamada
a55d4aee76 kbuild: make binrpm-pkg always produce kernel-devel package
The generation of the kernel-devel package is disabled for binrpm-pkg
presumably because it was quite big (>= 200MB) and took a long time to
package.

Commit fe66b5d2ae ("kbuild: refactor kernel-devel RPM package and
linux-headers Deb package") reduced the package size to 12MB, and now
it is quick to build. It won't hurt to have binrpm-pkg generate it by
default.

If you want to skip the kernel-devel package generation, you can pass
RPMOPTS='--without devel':

  $ make binrpm-pkg RPMOPTS='--without devel'

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
2023-10-18 17:16:09 +09:00
Masahiro Yamada
ffa46bbc58 kbuild: rpm-pkg: generate kernel.spec in rpmbuild/SPECS/
kernel.spec is the last piece that resides outside the rpmbuild/
directory. Move all the RPM-related files to rpmbuild/ consistently.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
2023-10-03 20:49:09 +09:00
Masahiro Yamada
ed79c34d3c kbuild: deb-pkg: support DEB_BUILD_OPTIONS=parallel=N in debian/rules
'make srcdeb-pkg' generates a source package, which you can build
later by using dpkg-buildpackage.

In older dpkg versions, 'dpkg-buildpackage --jobs=N' sets not only
DEB_BUILD_OPTIONS but also MAKEFLAGS. Hence, passing -j or --jobs
to dpkg-buildpackage was enough for kicking the parallel execution.

The behavior was changed by commit 1d0ea9b2ba3f ("dpkg-buildpackage:
Change -j, --jobs semantics to non-force mode") of dpkg project. [1]

Since then, 'dpkg-buildpackage --jobs=N' sets only DEB_BUILD_OPTIONS,
which is not parsed by the current debian/rules. To build the package
in parallel, you need to pass the alternative --jobs-force option or
set the MAKEFLAGS environment variable.

Debian policy [2] suggests the following code snippet for debian/rules.

  ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
      NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
      MAKEFLAGS += -j$(NUMJOBS)
  endif

I tweaked the code to filter out parallel=1 and passed --jobs=1 to
dpkg-buildpackage from scripts/Makefile.package. It is needed to force
'make deb-pkg' without the -j option to run in serial. Please note that
dpkg-buildpackage sets parallel=<nproc> in DEB_BUILD_OPTIONS by default
(that is, --jobs=auto is the default) and --jobs=1 is needed to restore
the serial execution. When dpkg-buildpackage is invoked from Kbuild,
the number of jobs is inherited from the top level Makefile. Passing
--jobs=1 to dpkg-buildpackage allows debian/rules to skip parsing
DEB_BUILD_OPTIONS.

[1] 1d0ea9b2ba
[2] https://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules-options

Reported-by: Bastian Germann <bage@linutronix.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-08-29 22:29:35 +09:00
Masahiro Yamada
4b970e4365 kbuild: deb-pkg: use Debian compliant shebang for debian/rules
Debian Policy "4.9. Main building script: debian/rules" requires
"debian/rules must start with the line #!/usr/bin/make -f". [1]

Currently, Kbuild does not follow this policy.

When Kbuild generates debian/rules, "#!$(command -v $MAKE) -f" is
expanded by shell. The resuling string may not be "#!/usr/bin/make -f".

There was a reason to opt out the Debian policy.

If you run '/path/to/my/custom/make deb-pkg', debian/rules must also be
invoked by the same Make program. If #!/usr/bin/make were hard-coded in
debian/rules, the sub-make would be executed by a possibly different
Make version.

This is problematic due to the MAKEFLAGS incompatibility, especially the
job server flag. Old Make versions used --jobserver-fds to propagate job
server file descriptors, but Make >= 4.2 uses --jobserver-auth. The flag
disagreement between the parent/child Makes would result in a process
fork explosion.

However, having a non-standard path in the shebang causes another issue;
the generated source package is not portable as such a path does not
exist in other build environments.

This commit solves those conflicting demands.

Hard-code '#!/usr/bin/make -f' in debian/rules to create a portable and
Debian-compliant source package.

Pass '--rules-file=$(MAKE) -f debian/rules' when dpkg-buildpackage is
invoked from Makefile so that debian/rules is executed by the same Make
program as used to start Kbuild.

[1] https://www.debian.org/doc/debian-policy/ch-source.html#main-building-script-debian-rules

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-08-08 01:08:54 +09:00
Masahiro Yamada
783c55ae7a kbuild: rpm-pkg: skip build dependency check on non-rpm systems
Commit 8818039f95 ("kbuild: add ability to make source rpm buildable
using koji") added the BuildRequires: field.

Checking the build dependency is fine, but one annoyance is that
'make (bin)rpm-pkg' fails on non-rpm systems [1]. For example, Debian
provides rpmbuild via 'apt install rpm', but of course cannot meet the
requirement listed in the BuildRequires: field.

It is possible to pass RPMOPTS=--nodeps to work around it, but it is
reasonable to do it automatically.

If 'rpm -q rpm' fails, it is not an RPM-managed system. (The command
'rpm' is not installed at all, or was installed by other means.)

In that case, pass --nodeps to skip the build dependency check.

[1]: https://lore.kernel.org/linux-kbuild/Y6mkdYQYmjUz7bqV@li-4a3a4a4c-28e5-11b2-a85c-a8d192c6f089.ibm.com/

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:33 +09:00
Masahiro Yamada
37477496d6 kbuild: rpm-pkg: refactor *rpm-pkg targets
Merge the similar build targets.

Also, make the output location consistent.

Previously, source packages were created in the build directory,
while binary packages under ~/rpmbuild/RPMS/.

Now, Kbuild creates the rpmbuild/ directory in the build directory,
and saves all packages under it.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:33 +09:00
Masahiro Yamada
6db9ced464 kbuild: rpm-pkg: build the kernel in-place for rpm-pkg
Currently, 'make rpm-pkg' always builds the kernel from the pristine
source tree in the ~/rpmbuild/BUILD/ directory.

Build the kernel incrementally just like 'make binrpm-pkg'.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:33 +09:00
Masahiro Yamada
975667d02d kbuild: rpm-pkg: rename binkernel.spec to kernel.spec
Now kernel.spec and binkernel.spec have the exactly same contents.

Use kernel.spec for binrpm-pkg as well.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:33 +09:00
Masahiro Yamada
2a291fc315 kbuild: rpm-pkg: introduce %{with_devel} switch to select devel package
scripts/package/mkspec preprocesses the spec file by sed, but it is
unreadable. This commit removes the last portion of the sed scripting.

Remove the $S$M prefixes from the conditionally generated lines.
Instead, surround the code with %if %{with_devel} ... %endif.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:33 +09:00
Masahiro Yamada
1789fc9125 kbuild: rpm-pkg: invoke the kernel build from rpmbuild for binrpm-pkg
To reduce the preprocess of the spec file, invoke the kernel build
from rpmbuild.

Run init/build-version to increment the release number not only for
binrpm-pkg but also for srcrpm-pkg and rpm-pkg.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:33 +09:00
Masahiro Yamada
76a48b8ffb kbuild: add a phony target to run a command with Kbuild env vars
There are some cases where we want to run a command with the same
environment variables as Kbuild uses. For example, 'make coccicheck'
invokes scripts/coccicheck from the top Makefile so that the script can
reference to ${LINUXINCLUDE}, ${KBUILD_EXTMOD}, etc. The top Makefile
defines several phony targets that run a script.

We do it also for an internally used script, which results in a somewhat
complex call graph.

One example:

 debian/rules binary-arch
   -> make intdeb-pkg
      -> scripts/package/builddeb

It is also tedious to add a dedicated target like 'intdeb-pkg' for each
use case.

Add a generic target 'run-command' to run an arbitrary command in an
environment with all Kbuild variables set.

The usage is:

  $ make run-command KBUILD_RUN_COMMAND=<command>

The concept is similar to:

  $ dpkg-architecture -c <command>

This executes <command> in an environment which has all DEB_* variables
defined.

Convert the existing 'make intdeb-pkg'.

Another possible usage is to interrogate a Make variable.

  $ make run-command KBUILD_RUN_COMMAND='echo $(KBUILD_CFLAGS)'

might be useful to see KBUILD_CFLAGS set by the top Makefile.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:32 +09:00
Masahiro Yamada
192868258d kbuild: rpm-pkg: do not hard-code $MAKE in spec file
Currently, $MAKE will expand to the GNU Make program that created the
source RPM. This is problematic if you carry it to a different build
host to run 'rpmbuild' there.

Consider this command:

  $ /path/to/my/custom/make srcrpm-pkg

The spec file in the SRPM will record '/path/to/my/custom/make', which
exists only on that build environment.

To create a portable SRPM, the spec file should avoid hard-coding $MAKE.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-07-25 00:59:32 +09:00
Linus Torvalds
d55571c008 Kbuild updates for v6.4
- Refactor scripts/kallsyms to make it faster and easier to maintain
 
  - Clean up menuconfig
 
  - Provide Clang with hard-coded target triple instead of CROSS_COMPILE
 
  - Use -z pack-relative-relocs flags instead of --use-android-relr-tags
    for arm64 CONFIG_RELR
 
  - Add srcdeb-pkg target to build only a Debian source package
 
  - Add KDEB_SOURCE_COMPRESS option to specify the compression for a
    Debian source package
 
  - Misc cleanups and fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQJJBAABCgAzFiEEbmPs18K1szRHjPqEPYsBB53g2wYFAmRM9usVHG1hc2FoaXJv
 eUBrZXJuZWwub3JnAAoJED2LAQed4NsGg+UQAKbDv/rlBLsdFOWnFgh8IUE58uUb
 UF/lmbyPQPZS1+z8WS/ec4ONigPd0gcfjI8klaZuYJ2j5xK0Nf+J8KRQlG9vLCDJ
 O0jPV77mKRMkz6UIaa5Dd8MNje8Pn+6T1qev1I9+0dBhyS2csHds3NuDuI/kDehB
 V1097moJcTEUGbNWTgnIA3LckIP51OMUgmT94gntcWcTvuGahRmn+Vvn4pijd6rn
 r3081wejYl+gZ9DGobRT7yXTsGNkHBHJcmK7VaocAL29haNt+rEULvAC1ABEUkaE
 tJMLodybbBiNH441EuUHJBA90yWUxL2EDmTz7kvvYrwem5FsFx6dpuKUXcJpCxW7
 x66qOQHrHC7ZY98heMs07TLz+5yMhAseBSGgf6QDSvX7CP20kCgf7q0IikfhByMo
 ZWrxj5Zg+P6qoqP5NuoZM4waOarz4wN96OAdB4ao3tbMb0kmzdpvI7W9ZuLaBmEP
 iTYm5r6if++a4a09V4WfWOyE0+7TYwSWwl271pIeovZy0p77ulgiYFo8/a9XqKlS
 CrUP1r85zkDHYN/RS4DcbXkqa4RnfYUBd/9h/5zAMLOWwGLRets+/xA4sD2QE9nV
 xzel3AhVrE3uX0QEPOZnFWTUb5TXkYuhrDxkqRGit61XUeP0Ohy6P1ri8f1dI74J
 2huTmg+CZkyfXEff
 =UuFY
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

 - Refactor scripts/kallsyms to make it faster and easier to maintain

 - Clean up menuconfig

 - Provide Clang with hard-coded target triple instead of CROSS_COMPILE

 - Use -z pack-relative-relocs flags instead of --use-android-relr-tags
   for arm64 CONFIG_RELR

 - Add srcdeb-pkg target to build only a Debian source package

 - Add KDEB_SOURCE_COMPRESS option to specify the compression for a
   Debian source package

 - Misc cleanups and fixes

* tag 'kbuild-v6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
  kbuild: deb-pkg: specify targets in debian/rules as .PHONY
  sparc: unify sparc32/sparc64 archhelp
  kbuild: rpm-pkg: remove kernel-drm PROVIDES
  kbuild: deb-pkg: add KDEB_SOURCE_COMPRESS to specify source compression
  kbuild: add srcdeb-pkg target
  Makefile: use -z pack-relative-relocs
  kbuild: clang: do not use CROSS_COMPILE for target triple
  kconfig: menuconfig: reorder functions to remove forward declarations
  kconfig: menuconfig: remove unused M_EVENT macro
  kconfig: menuconfig: remove OLD_NCURSES macro
  kbuild: builddeb: Eliminate debian/arch use
  scripts/kallsyms: update the usage in the comment block
  scripts/kallsyms: decrease expand_symbol() / cleanup_symbol_name() calls
  scripts/kallsyms: change the output order
  scripts/kallsyms: move compiler-generated symbol patterns to mksysmap
  scripts/kallsyms: exclude symbols generated by itself dynamically
  scripts/mksysmap: use sed with in-line comments
  scripts/mksysmap: remove comments described in nm(1)
  scripts/kallsyms: remove redundant code for omitting U and N
  kallsyms: expand symbol name into comment for debugging
2023-04-30 11:32:53 -07:00
Masahiro Yamada
1d29b4c223 kbuild: deb-pkg: add KDEB_SOURCE_COMPRESS to specify source compression
Add KDEB_SOURCE_COMPRESS to specify the compression for the orig and
debian tarballs. (cf. the existing KDEB_COMPRESS is used to specify
the compression for binary packages.)

Supported algorithms are gzip, bzip2, lzma, and xz, all of which are
supported by dpkg-source.

The current default is gzip. You can change it via the environment
variable, for example, 'KDEB_SOURCE_COMPRESS=xz make deb-pkg'.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-04-25 10:04:19 +09:00
Masahiro Yamada
31f735c65d kbuild: add srcdeb-pkg target
This new target builds only the debian source package.

Unify the build rules of deb-pkg, srcdeb-pkg, bindeb-pkg to avoid
code duplication.

--no-check-builddeps is added to srcdeb-pkg so that build dependencies
will not be checked.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-04-23 22:46:46 +09:00
Masahiro Yamada
9cedc5e89a kbuild: use proper prefix for tarballs to fix rpm-pkg build error
Since commit f8d94c4e40 ("kbuild: do not create intermediate *.tar
for source tarballs"), 'make rpm-pkg' fails because the prefix of the
source tarball is 'linux.tar/' instead of 'linux/'. $(basename $@)
strips only '.gz' from the filename linux.tar.gz.

You need to strip two suffixes from compressed tarballs and one suffix
from uncompressed tarballs (for example 'perf-6.3.0.tar' generated by
'make perf-tar-src-pkg').

One tricky fix might be --prefix=$(firstword $(subst .tar, ,$@))/
but I think it is better to hard-code the prefix.

Fixes: f8d94c4e40 ("kbuild: do not create intermediate *.tar for source tarballs")
Reported-by: Jiwei Sun <sunjw10@lenovo.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-04-23 21:23:10 +09:00
Masahiro Yamada
3c65a2704c kbuild: do not create intermediate *.tar for tar packages
Commit 05e96e96a3 ("kbuild: use git-archive for source package
creation") split the compression as a separate step to factor out
the common build rules.

With the previous commit, we got back to the situation where source
tarballs are compressed on-the-fly.
There is no reason to keep the separate compression rules.

Generate the comressed tar packages directly.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
2023-04-16 17:38:41 +09:00
Masahiro Yamada
f8d94c4e40 kbuild: do not create intermediate *.tar for source tarballs
Since commit 05e96e96a3 ("kbuild: use git-archive for source package
creation"), a source tarball is created in two steps; create *.tar file
then compress it. I split the compression as a separate rule because I
just thought 'git archive' supported only gzip.

For other compression algorithms, I could pipe the two commands:

  $ git archive HEAD | xz > linux.tar.xz

I read git-archive(1) carefully, and I realized GIT had provided a
more elegant way:

  $ git -c tar.tar.xz.command=xz archive -o linux.tar.xz HEAD

This commit uses 'tar.tar.*.command' configuration to specify the
compression backend so we can compress a source tarball on-the-fly.

GIT commit 767cf4579f0e ("archive: implement configurable tar filters")
is more than a decade old, so it should be available on almost all build
environments.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
2023-04-16 17:38:16 +09:00
Masahiro Yamada
f6d8283549 kbuild: merge cmd_archive_linux and cmd_archive_perf
The two commands, cmd_archive_linux and cmd_archive_perf, are similar.
Merge them to make it easier to add more changes to the git-archive
command.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
2023-04-16 17:37:01 +09:00
Masahiro Yamada
aa7d233f45 kbuild: give up untracked files for source package builds
When the source tree is dirty and contains untracked files, package
builds may fail, for example, when a broken symlink exists, a file
path contains whitespaces, etc.

Since commit 05e96e96a3 ("kbuild: use git-archive for source package
creation"), the source tarball only contains committed files because
it is created by 'git archive'. scripts/package/gen-diff-patch tries
to address the diff from HEAD, but including untracked files by the
hand-crafted script introduces more complexity. I wrote a patch [1] to
make it work in most cases, but still wonder if this is what we should
aim for.

To simplify the code, this patch just gives up untracked files. Going
forward, it is your responsibility to do 'git add' for what you want in
the source package. The script shows a warning just in case you forgot
to do so. It should be checked only when building source packages.

[1]: https://lore.kernel.org/all/CAK7LNAShbZ56gSh9PrbLnBDYKnjtTkHMoCXeGrhcxMvqXGq9=g@mail.gmail.com/2-0001-kbuild-make-package-builds-more-robust.patch

Fixes: 05e96e96a3 ("kbuild: use git-archive for source package creation")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-04-11 08:58:45 +09:00
Masahiro Yamada
05e96e96a3 kbuild: use git-archive for source package creation
Commit 5c3d1d0abb ("kbuild: add a tool to list files ignored by git")
added a new tool, scripts/list-gitignored. My intention was to create
source packages without cleaning the source tree, without relying on git.

Linus strongly objected to it, and suggested using 'git archive' instead.
[1] [2] [3]

This commit goes in that direction - Remove scripts/list-gitignored.c
and rewrites Makefiles and scripts to use 'git archive' for building
Debian and RPM source packages. It also makes 'make perf-tar*-src-pkg'
use 'git archive' again.

Going forward, building source packages is only possible in a git-managed
tree. Building binary packages does not require git.

[1]: https://lore.kernel.org/lkml/CAHk-=wi49sMaC7vY1yMagk7eqLK=1jHeHQ=yZ_k45P=xBccnmA@mail.gmail.com/
[2]: https://lore.kernel.org/lkml/CAHk-=wh5AixGsLeT0qH2oZHKq0FLUTbyTw4qY921L=PwYgoGVw@mail.gmail.com/
[3]: https://lore.kernel.org/lkml/CAHk-=wgM-W6Fu==EoAVCabxyX8eYBz9kNC88-tm9ExRQwA79UQ@mail.gmail.com/

Fixes: 5c3d1d0abb ("kbuild: add a tool to list files ignored by git")
Fixes: e0ca16749a ("kbuild: make perf-tar*-src-pkg work without relying on git")
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-03-16 22:46:12 +09:00
Masahiro Yamada
81f59a26f3 kbuild: rpm-pkg: move source components to rpmbuild/SOURCES
Prepare to add more files to the source RPM.

Also, fix the build error when KCONFIG_CONFIG is set:
  error: Bad file: ./.config: No such file or directory

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-03-16 22:45:56 +09:00
Masahiro Yamada
1fc9095846 kbuild: tar-pkg: use tar rules in scripts/Makefile.package
Use %.tar, %.tar.gz, %.tar.bz2, %.tar.xz, %.tar.zst rules in
scripts/Makefile.package.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:23:30 +09:00
Masahiro Yamada
e0ca16749a kbuild: make perf-tar*-src-pkg work without relying on git
Currently, perf-tar*-src-pkg only uses 'git archive', but it is better
to make it work without relying on git.

The file, HEAD, which saves the commit hash, will be included in the
tarball only when the source tree is managed by git. The git tree is
more precisely checked; it has been copied from scripts/setlocalversion.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:23:30 +09:00
Masahiro Yamada
e785399559 kbuild: deb-pkg: switch over to source format 3.0 (quilt)
Change the source format from "1.0" to "3.0 (quilt)" because it works
more cleanly.

All files except .config and debian/ go into the orig tarball.
Add a single patch, debian/patches/config, and delete the ugly
extend-diff-ignore patterns.

The debian tarball will be compressed into *.debian.tar.xz by default.
If you like to use a different compression mode, you can pass the
command line option, DPKG_FLAGS=-Zgzip, for example.

The orig tarball only supports gzip for now. The combination of
gzip and xz is somewhat clumsy, but it is not a practical problem.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:22:37 +09:00
Masahiro Yamada
b44aa8c96e kbuild: deb-pkg: make .orig tarball a hard link if possible
If '..' belongs to the same filesystem, create a hard link instead of
a copy. In most cases, you can save disk space.

I do not want to use 'mv' because keeping linux.tar.gz is useful to
avoid unneeded rebuilding of the tarball.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:20:38 +09:00
Masahiro Yamada
6eabebb1b6 kbuild: deb-pkg: hide KDEB_SOURCENAME from Makefile
scripts/Makefile.package does not need to know the value of
KDEB_SOURCENAME because the source name can be taken from
debian/changelog by using dpkg-parsechangelog.

Move the default of KDEB_SOURCENAME (i.e. linux-upstream) to
scripts/package/mkdebian.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:19:38 +09:00
Masahiro Yamada
6fc91752d7 kbuild: srcrpm-pkg: create source package without cleaning
If you run 'make (src)rpm-pkg', all objects are lost due to 'make clean',
which makes the incremental builds impossible.

Instead of cleaning, pass the exclude list to tar's --exclude-from
option.

Previously, the .config was contained in the source tarball.

With this commit, the source rpm consists of separate linux.tar.gz
and .config.

Remove stale comments. Now, 'make (src)rpm-pkg' works with O= option.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:18:47 +09:00
Masahiro Yamada
1ec9bb704f kbuild: rpm-pkg: build binary packages from source rpm
The build rules of rpm-pkg and srcrpm-pkg are almost the same.
Remove the code duplication.

Change rpm-pkg to build binary packages from the source package generated
by srcrpm-pkg.

This changes the output directory of the srpm generated by 'make rpm-pkg'
because srcrpm-pkg overrides _srcrpmdir.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-26 15:18:47 +09:00
Masahiro Yamada
7bf4582d7a kbuild: deb-pkg: create source package without cleaning
If you run 'make deb-pkg', all objects are lost due to 'make clean',
which makes the incremental builds impossible.

Instead of cleaning, pass the exclude list to tar's --exclude-from
option.

Previously, *.diff.gz contained some check-in files such as
.clang-format, .cocciconfig.

With this commit, *.diff.gz will only contain the .config and debian/.
The other source files will go into the .orig tarball.

linux.tar.gz is rebuilt only when the source files that would go into
the tarball are changed.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-02-26 15:18:07 +09:00
Masahiro Yamada
58e0e5c85e kbuild: deb-pkg: add --source-option=-sP
If the source package fails to build, ../linux.orig is left over.
In the next run of 'make deb-pkg', you will get the following error:

  dpkg-source: error: orig directory 'linux.orig' already exists, not overwriting, giving up; use -sA, -sK or -sP to override

You can manually remove ../linux.orig, but it is annoying.

Pass -sP down to dpkg-source.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2023-02-05 18:51:21 +09:00
Masahiro Yamada
f6e09b07cc kbuild: do not put .scmversion into the source tarball
.scmversion is used by (src)rpm-pkg and deb-pkg to carry KERNELRELEASE.

In fact, deb-pkg does not rely on it any more because the generated
debian/rules specifies KERNELRELEASE from the command line.

Do likwise for (src)rpm-pkg, and remove this feature.
For the same reason, you do not need to save LOCALVERSION in the
spec file.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-30 13:00:30 +09:00
Jun ASAKA
9c9b55a594 kbuild: add a missing line for help message
The help message line for building the source RPM package was missing.
Added it.

Signed-off-by: Jun ASAKA <JunASAKA@zzy040330.moe>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-12-30 17:22:14 +09:00
Linus Torvalds
6feb57c2fd Kbuild updates for v6.2
- Support zstd-compressed debug info
 
  - Allow W=1 builds to detect objects shared among multiple modules
 
  - Add srcrpm-pkg target to generate a source RPM package
 
  - Make the -s option detection work for future GNU Make versions
 
  - Add -Werror to KBUILD_CPPFLAGS when CONFIG_WERROR=y
 
  - Allow W=1 builds to detect -Wundef warnings in any preprocessed files
 
  - Raise the minimum supported version of binutils to 2.25
 
  - Use $(intcmp ...) to compare integers if GNU Make >= 4.4 is used
 
  - Use $(file ...) to read a file if GNU Make >= 4.2 is used
 
  - Print error if GNU Make older than 3.82 is used
 
  - Allow modpost to detect section mismatches with Clang LTO
 
  - Include vmlinuz.efi into kernel tarballs for arm64 CONFIG_EFI_ZBOOT=y
 -----BEGIN PGP SIGNATURE-----
 
 iQJJBAABCgAzFiEEbmPs18K1szRHjPqEPYsBB53g2wYFAmOeImsVHG1hc2FoaXJv
 eUBrZXJuZWwub3JnAAoJED2LAQed4NsG06IP/iVjuWFvnjDZT4X8X6zN8aKp1vtR
 EMkmoRtt5cD4CLb1MG4N7irYHgedQSx4rYceP45MyW1I3egl6Ct14RDyeQ1xSIZb
 XFTLDCZvfl/up3MdiqNAqKRS7x5lk9++7F0t+2SoQxKQyJvm735XreX+VhZ1FeLB
 qcHrmzJ5veky5Ry/3OkNUgKFBjKEAL+qKMc55uvkXqfTb3KoBa2r4VC1OaoYGRru
 R8oF9qQRnGVQAl/LbBVchmgSjxryxPrCvBGiKlK03VkXdzEMHMimEJh3BQ6e0PGo
 gajdk+4liy7z+jQnI7jFhvJjGKzkEP/Bc99M/uS92QX5MgpH6mqpHMoqqPiqW87K
 RmZH37FqRu1Vo8dpibmH6r2K6YD/HHRjaDHk1VuuCQYEn0dsNmokPXOqd/1v0I1i
 TXPjWOw1AID5vMJWllqxFhpeVvf0vx5BT/UNrh68MLqlJZzv2eMVJb4fNy6640ml
 U0NclMnOa3eOmf5z1T7/LqDRTa63Q0kpanRrBpcmVOaqW+ZpQ3SQjh4uBN1PyJHL
 cX3Skc341DyRlFiT54QhGKlm57MEb2gjhBZ3Z4J+b7sEFgvjXH/W8vcOGIKlppmA
 CfYMyres4OV+fJc89ONkWsvLiOP1OeUGPvytm33J5QMKXc8SzOLP0D/F8kjrDflm
 EROKuZ4EA5ej/rOy
 =Ig/Y
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild

Pull Kbuild updates from Masahiro Yamada:

 - Support zstd-compressed debug info

 - Allow W=1 builds to detect objects shared among multiple modules

 - Add srcrpm-pkg target to generate a source RPM package

 - Make the -s option detection work for future GNU Make versions

 - Add -Werror to KBUILD_CPPFLAGS when CONFIG_WERROR=y

 - Allow W=1 builds to detect -Wundef warnings in any preprocessed files

 - Raise the minimum supported version of binutils to 2.25

 - Use $(intcmp ...) to compare integers if GNU Make >= 4.4 is used

 - Use $(file ...) to read a file if GNU Make >= 4.2 is used

 - Print error if GNU Make older than 3.82 is used

 - Allow modpost to detect section mismatches with Clang LTO

 - Include vmlinuz.efi into kernel tarballs for arm64 CONFIG_EFI_ZBOOT=y

* tag 'kbuild-v6.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (29 commits)
  buildtar: fix tarballs with EFI_ZBOOT enabled
  modpost: Include '.text.*' in TEXT_SECTIONS
  padata: Mark padata_work_init() as __ref
  kbuild: ensure Make >= 3.82 is used
  kbuild: refactor the prerequisites of the modpost rule
  kbuild: change module.order to list *.o instead of *.ko
  kbuild: use .NOTINTERMEDIATE for future GNU Make versions
  kconfig: refactor Makefile to reduce process forks
  kbuild: add read-file macro
  kbuild: do not sort after reading modules.order
  kbuild: add test-{ge,gt,le,lt} macros
  Documentation: raise minimum supported version of binutils to 2.25
  kbuild: add -Wundef to KBUILD_CPPFLAGS for W=1 builds
  kbuild: move -Werror from KBUILD_CFLAGS to KBUILD_CPPFLAGS
  kbuild: Port silent mode detection to future gnu make.
  init/version.c: remove #include <generated/utsrelease.h>
  firmware_loader: remove #include <generated/utsrelease.h>
  modpost: Mark uuid_le type to be suitable only for MEI
  kbuild: add ability to make source rpm buildable using koji
  kbuild: warn objects shared among multiple modules
  ...
2022-12-19 12:33:32 -06:00
Paran Lee
df1f1ea956 scripts: add rust in scripts/Makefile.package
Add rust argument at TAR_CONTENT in
scripts/Makefile.package script with alphabetical order.

Signed-off-by: Paran Lee <p4ranlee@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-11-24 09:27:57 +09:00
Ivan Vecera
8818039f95 kbuild: add ability to make source rpm buildable using koji
Changes:
- added new target 'srcrpm-pkg' to generate source rpm
- added required build tools to spec file
- removed locally compiled host tools to force their re-compile

Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Signed-off-by: Jonathan Toppins <jtoppins@redhat.com>
Acked-by: Íñigo Huguet <ihuguet@redhat.com>
Tested-by: Ivan Vecera <ivecera@redhat.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-11-24 00:23:44 +09:00
Masahiro Yamada
a3c4d4abaa kbuild: hard-code KBUILD_ALLDIRS in scripts/Makefile.package
My future plan is to list subdirectories in ./Kbuild. When it occurs,
$(vmlinux-alldirs) will not contain all subdirectories.

Let's hard-code the directory list until I get around to implementing
a more sophisticated way for generating a source tarball.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2022-09-29 04:38:57 +09:00
Masahiro Yamada
6105e4f651 kbuild: rpm-pkg: pass 'linux' to --target option of rpmbuild
Presumably, _target_os is defined even if the --target flag does not
specify it, but it is better to make it explicit.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2022-07-27 21:18:00 +09:00
Paweł Jasiak
88f5e1e662 kbuild: Add make tarzst-pkg build option
Add tarzst-pkg and perf-tarzst-src-pkg targets to build zstd compressed
tarballs.

Signed-off-by: Paweł Jasiak <pawel@jasiak.dev>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-10-12 11:50:46 +09:00
Masahiro Yamada
82526ef433 kbuild: deb-pkg: change the source package name to linux-upstream
Change the source package name from 'linux-$(KERNELRELEASE)' to
'linux-upstream'.

Initially, I tried to use 'linux' to be aligned with the Debian
kernel package, but Ben suggested 'linux-upstream' so that it is
clearly distinguished from distribution packages. [1]

The filenames will be changed as follows:

[Before]
  linux-5.12.0-rc3+_5.12.0-rc3+-1.dsc
  linux-5.12.0-rc3+_5.12.0-rc3+.orig.tar.gz
  linux-5.12.0-rc3+_5.12.0-rc3+-1.diff.gz

[After]
  linux-upstream_5.12.0-rc3+-1.dsc
  linux-upstream_5.12.0-rc3+.orig.tar.gz
  linux-upstream_5.12.0-rc3+-1.diff.gz

Commit 3716001bcb ("deb-pkg: add source package") introduced
KDEB_SOURCENAME. If you are unhappy with the default name, you can
override it via KDEB_SOURCENAME.

[1]: https://lore.kernel.org/linux-kbuild/06ffa2a690d57f867b4bc1b42f0026917b1dd3cd.camel@decadent.org.uk/T/#m2c4afa0eca5ced5e57795b002f2dbcb05d7a4a44

Suggested-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2021-04-25 05:30:45 +09:00
Denis Efremov
e4a42c82e9 kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables
Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp.
GZIP, BZIP2, LZOP env variables are reserved by the tools. The original
attempt to redefine them internally doesn't work in makefiles/scripts
intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in
broken builds. There can be other broken build commands because of this,
so the universal solution is to use non-reserved env variables for the
compression tools.

Fixes: 8dfb61dcba ("kbuild: add variables for compression tools")
Signed-off-by: Denis Efremov <efremov@linux.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-11 20:14:41 +09:00
Denis Efremov
8dfb61dcba kbuild: add variables for compression tools
Allow user to use alternative implementations of compression tools,
such as pigz, pbzip2, pxz. For example, multi-threaded tools to
speed up the build:
$ make GZIP=pigz BZIP2=pbzip2

Variables _GZIP, _BZIP2, _LZOP are used internally because original env
vars are reserved by the tools. The use of GZIP in gzip tool is obsolete
since 2015. However, alternative implementations (e.g., pigz) still rely
on it. BZIP2, BZIP, LZOP vars are not obsolescent.

The credit goes to @grsecurity.

As a sidenote, for multi-threaded lzma, xz compression one can use:
$ export XZ_OPT="--threads=0"

Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-06-06 23:42:01 +09:00