mirror of
https://github.com/torvalds/linux.git
synced 2024-11-22 04:02:20 +00:00
c6f371bab2
XZ_EXTERN was used to make internal functions static in the preboot code. However, in other decompressors this hasn't been done. On x86-64, this makes no difference to the kernel image size. Omit XZ_EXTERN and let some of the internal functions be extern in the preboot code. Omitting XZ_EXTERN from include/linux/xz.h fixes warnings in "make htmldocs" and makes the intradocument links to xz_dec functions work in Documentation/staging/xz.rst. The alternative would have been to add "XZ_EXTERN" to c_id_attributes in Documentation/conf.py but omitting XZ_EXTERN seemed cleaner. Link: https://lore.kernel.org/lkml/20240723205437.3c0664b0@kaneli/ Link: https://lkml.kernel.org/r/20240724110544.16430-1-lasse.collin@tukaani.org Signed-off-by: Lasse Collin <lasse.collin@tukaani.org> Tested-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) Cc: Jonathan Corbet <corbet@lwn.net> Cc: Sam James <sam@gentoo.org> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Emil Renner Berthing <emil.renner.berthing@canonical.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Joel Stanley <joel@jms.id.au> Cc: Jubin Zhong <zhongjubin@huawei.com> Cc: Jules Maselbas <jmaselbas@zdiv.net> Cc: Krzysztof Kozlowski <krzk@kernel.org> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Rui Li <me@lirui.org> Cc: Simon Glass <sjg@chromium.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Will Deacon <will@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
99 lines
4.0 KiB
ReStructuredText
99 lines
4.0 KiB
ReStructuredText
.. SPDX-License-Identifier: 0BSD
|
|
|
|
============================
|
|
XZ data compression in Linux
|
|
============================
|
|
|
|
Introduction
|
|
============
|
|
|
|
XZ is a general purpose data compression format with high compression
|
|
ratio. The XZ decompressor in Linux is called XZ Embedded. It supports
|
|
the LZMA2 filter and optionally also Branch/Call/Jump (BCJ) filters
|
|
for executable code. CRC32 is supported for integrity checking.
|
|
|
|
See the `XZ Embedded`_ home page for the latest version which includes
|
|
a few optional extra features that aren't required in the Linux kernel
|
|
and information about using the code outside the Linux kernel.
|
|
|
|
For userspace, `XZ Utils`_ provide a zlib-like compression library
|
|
and a gzip-like command line tool.
|
|
|
|
.. _XZ Embedded: https://tukaani.org/xz/embedded.html
|
|
.. _XZ Utils: https://tukaani.org/xz/
|
|
|
|
XZ related components in the kernel
|
|
===================================
|
|
|
|
The xz_dec module provides XZ decompressor with single-call (buffer
|
|
to buffer) and multi-call (stateful) APIs in include/linux/xz.h.
|
|
|
|
For decompressing the kernel image, initramfs, and initrd, there
|
|
is a wrapper function in lib/decompress_unxz.c. Its API is the
|
|
same as in other decompress_*.c files, which is defined in
|
|
include/linux/decompress/generic.h.
|
|
|
|
For kernel makefiles, three commands are provided for use with
|
|
``$(call if_changed)``. They require the xz tool from XZ Utils.
|
|
|
|
- ``$(call if_changed,xzkern)`` is for compressing the kernel image.
|
|
It runs the script scripts/xz_wrap.sh which uses arch-optimized
|
|
options and a big LZMA2 dictionary.
|
|
|
|
- ``$(call if_changed,xzkern_with_size)`` is like ``xzkern`` above but
|
|
this also appends a four-byte trailer containing the uncompressed size
|
|
of the file. The trailer is needed by the boot code on some archs.
|
|
|
|
- Other things can be compressed with ``$(call if_needed,xzmisc)``
|
|
which will use no BCJ filter and 1 MiB LZMA2 dictionary.
|
|
|
|
Notes on compression options
|
|
============================
|
|
|
|
Since the XZ Embedded supports only streams with CRC32 or no integrity
|
|
check, make sure that you don't use some other integrity check type
|
|
when encoding files that are supposed to be decoded by the kernel.
|
|
With liblzma from XZ Utils, you need to use either ``LZMA_CHECK_CRC32``
|
|
or ``LZMA_CHECK_NONE`` when encoding. With the ``xz`` command line tool,
|
|
use ``--check=crc32`` or ``--check=none`` to override the default
|
|
``--check=crc64``.
|
|
|
|
Using CRC32 is strongly recommended unless there is some other layer
|
|
which will verify the integrity of the uncompressed data anyway.
|
|
Double checking the integrity would probably be waste of CPU cycles.
|
|
Note that the headers will always have a CRC32 which will be validated
|
|
by the decoder; you can only change the integrity check type (or
|
|
disable it) for the actual uncompressed data.
|
|
|
|
In userspace, LZMA2 is typically used with dictionary sizes of several
|
|
megabytes. The decoder needs to have the dictionary in RAM:
|
|
|
|
- In multi-call mode the dictionary is allocated as part of the
|
|
decoder state. The reasonable maximum dictionary size for in-kernel
|
|
use will depend on the target hardware: a few megabytes is fine for
|
|
desktop systems while 64 KiB to 1 MiB might be more appropriate on
|
|
some embedded systems.
|
|
|
|
- In single-call mode the output buffer is used as the dictionary
|
|
buffer. That is, the size of the dictionary doesn't affect the
|
|
decompressor memory usage at all. Only the base data structures
|
|
are allocated which take a little less than 30 KiB of memory.
|
|
For the best compression, the dictionary should be at least
|
|
as big as the uncompressed data. A notable example of single-call
|
|
mode is decompressing the kernel itself (except on PowerPC).
|
|
|
|
The compression presets in XZ Utils may not be optimal when creating
|
|
files for the kernel, so don't hesitate to use custom settings to,
|
|
for example, set the dictionary size. Also, xz may produce a smaller
|
|
file in single-threaded mode so setting that explicitly is recommended.
|
|
Example::
|
|
|
|
xz --threads=1 --check=crc32 --lzma2=dict=512KiB inputfile
|
|
|
|
xz_dec API
|
|
==========
|
|
|
|
This is available with ``#include <linux/xz.h>``.
|
|
|
|
.. kernel-doc:: include/linux/xz.h
|