mirror of
https://github.com/torvalds/linux.git
synced 2024-11-30 16:11:38 +00:00
5c1aab1dd5
Add zstd compression and decompression support to BtrFS. zstd at its
fastest level compresses almost as well as zlib, while offering much
faster compression and decompression, approaching lzo speeds.
I benchmarked btrfs with zstd compression against no compression, lzo
compression, and zlib compression. I benchmarked two scenarios. Copying
a set of files to btrfs, and then reading the files. Copying a tarball
to btrfs, extracting it to btrfs, and then reading the extracted files.
After every operation, I call `sync` and include the sync time.
Between every pair of operations I unmount and remount the filesystem
to avoid caching. The benchmark files can be found in the upstream
zstd source repository under
`contrib/linux-kernel/{btrfs-benchmark.sh,btrfs-extract-benchmark.sh}`
[1] [2].
I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD.
The first compression benchmark is copying 10 copies of the unzipped
Silesia corpus [3] into a BtrFS filesystem mounted with
`-o compress-force=Method`. The decompression benchmark times how long
it takes to `tar` all 10 copies into `/dev/null`. The compression ratio is
measured by comparing the output of `df` and `du`. See the benchmark file
[1] for details. I benchmarked multiple zstd compression levels, although
the patch uses zstd level 1.
| Method | Ratio | Compression MB/s | Decompression speed |
|---------|-------|------------------|---------------------|
| None | 0.99 | 504 | 686 |
| lzo | 1.66 | 398 | 442 |
| zlib | 2.58 | 65 | 241 |
| zstd 1 | 2.57 | 260 | 383 |
| zstd 3 | 2.71 | 174 | 408 |
| zstd 6 | 2.87 | 70 | 398 |
| zstd 9 | 2.92 | 43 | 406 |
| zstd 12 | 2.93 | 21 | 408 |
| zstd 15 | 3.01 | 11 | 354 |
The next benchmark first copies `linux-4.11.6.tar` [4] to btrfs. Then it
measures the compression ratio, extracts the tar, and deletes the tar.
Then it measures the compression ratio again, and `tar`s the extracted
files into `/dev/null`. See the benchmark file [2] for details.
| Method | Tar Ratio | Extract Ratio | Copy (s) | Extract (s)| Read (s) |
|--------|-----------|---------------|----------|------------|----------|
| None | 0.97 | 0.78 | 0.981 | 5.501 | 8.807 |
| lzo | 2.06 | 1.38 | 1.631 | 8.458 | 8.585 |
| zlib | 3.40 | 1.86 | 7.750 | 21.544 | 11.744 |
| zstd 1 | 3.57 | 1.85 | 2.579 | 11.479 | 9.389 |
[1] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/btrfs-benchmark.sh
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/btrfs-extract-benchmark.sh
[3] http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia
[4] https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.11.6.tar.xz
zstd source repository: https://github.com/facebook/zstd
Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>
94 lines
3.0 KiB
Plaintext
94 lines
3.0 KiB
Plaintext
config BTRFS_FS
|
|
tristate "Btrfs filesystem support"
|
|
select CRYPTO
|
|
select CRYPTO_CRC32C
|
|
select ZLIB_INFLATE
|
|
select ZLIB_DEFLATE
|
|
select LZO_COMPRESS
|
|
select LZO_DECOMPRESS
|
|
select ZSTD_COMPRESS
|
|
select ZSTD_DECOMPRESS
|
|
select RAID6_PQ
|
|
select XOR_BLOCKS
|
|
select SRCU
|
|
|
|
help
|
|
Btrfs is a general purpose copy-on-write filesystem with extents,
|
|
writable snapshotting, support for multiple devices and many more
|
|
features focused on fault tolerance, repair and easy administration.
|
|
|
|
The filesystem disk format is no longer unstable, and it's not
|
|
expected to change unless there are strong reasons to do so. If there
|
|
is a format change, file systems with a unchanged format will
|
|
continue to be mountable and usable by newer kernels.
|
|
|
|
For more information, please see the web pages at
|
|
http://btrfs.wiki.kernel.org.
|
|
|
|
To compile this file system support as a module, choose M here. The
|
|
module will be called btrfs.
|
|
|
|
If unsure, say N.
|
|
|
|
config BTRFS_FS_POSIX_ACL
|
|
bool "Btrfs POSIX Access Control Lists"
|
|
depends on BTRFS_FS
|
|
select FS_POSIX_ACL
|
|
help
|
|
POSIX Access Control Lists (ACLs) support permissions for users and
|
|
groups beyond the owner/group/world scheme.
|
|
|
|
To learn more about Access Control Lists, visit the POSIX ACLs for
|
|
Linux website <http://acl.bestbits.at/>.
|
|
|
|
If you don't know what Access Control Lists are, say N
|
|
|
|
config BTRFS_FS_CHECK_INTEGRITY
|
|
bool "Btrfs with integrity check tool compiled in (DANGEROUS)"
|
|
depends on BTRFS_FS
|
|
help
|
|
Adds code that examines all block write requests (including
|
|
writes of the super block). The goal is to verify that the
|
|
state of the filesystem on disk is always consistent, i.e.,
|
|
after a power-loss or kernel panic event the filesystem is
|
|
in a consistent state.
|
|
|
|
If the integrity check tool is included and activated in
|
|
the mount options, plenty of kernel memory is used, and
|
|
plenty of additional CPU cycles are spent. Enabling this
|
|
functionality is not intended for normal use.
|
|
|
|
In most cases, unless you are a btrfs developer who needs
|
|
to verify the integrity of (super)-block write requests
|
|
during the run of a regression test, say N
|
|
|
|
config BTRFS_FS_RUN_SANITY_TESTS
|
|
bool "Btrfs will run sanity tests upon loading"
|
|
depends on BTRFS_FS
|
|
help
|
|
This will run some basic sanity tests on the free space cache
|
|
code to make sure it is acting as it should. These are mostly
|
|
regression tests and are only really interesting to btrfs
|
|
developers.
|
|
|
|
If unsure, say N.
|
|
|
|
config BTRFS_DEBUG
|
|
bool "Btrfs debugging support"
|
|
depends on BTRFS_FS
|
|
help
|
|
Enable run-time debugging support for the btrfs filesystem. This may
|
|
enable additional and expensive checks with negative impact on
|
|
performance, or export extra information via sysfs.
|
|
|
|
If unsure, say N.
|
|
|
|
config BTRFS_ASSERT
|
|
bool "Btrfs assert support"
|
|
depends on BTRFS_FS
|
|
help
|
|
Enable run-time assertion checking. This will result in panics if
|
|
any of the assertions trip. This is meant for btrfs developers only.
|
|
|
|
If unsure, say N.
|