mirror of
https://github.com/torvalds/linux.git
synced 2024-12-11 05:33:09 +00:00
511fb5bafe
-----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCZOXpbgAKCRCRxhvAZXjc oi8PAQCtXelGZHmTcmevsO8p4Qz7hFpkonZ/TnxKf+RdnlNgPgD+NWi+LoRBpaAj xk4z8SqJaTTP4WXrG5JZ6o7EQkUL8gE= =2e9I -----END PGP SIGNATURE----- Merge tag 'v6.6-vfs.super' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs Pull superblock updates from Christian Brauner: "This contains the super rework that was ready for this cycle. The first part changes the order of how we open block devices and allocate superblocks, contains various cleanups, simplifications, and a new mechanism to wait on superblock state changes. This unblocks work to ultimately limit the number of writers to a block device. Jan has already scheduled follow-up work that will be ready for v6.7 and allows us to restrict the number of writers to a given block device. That series builds on this work right here. The second part contains filesystem freezing updates. Overview: The generic superblock changes are rougly organized as follows (ignoring additional minor cleanups): (1) Removal of the bd_super member from struct block_device. This was a very odd back pointer to struct super_block with unclear rules. For all relevant places we have other means to get the same information so just get rid of this. (2) Simplify rules for superblock cleanup. Roughly, everything that is allocated during fs_context initialization and that's stored in fs_context->s_fs_info needs to be cleaned up by the fs_context->free() implementation before the superblock allocation function has been called successfully. After sget_fc() returned fs_context->s_fs_info has been transferred to sb->s_fs_info at which point sb->kill_sb() if fully responsible for cleanup. Adhering to these rules means that cleanup of sb->s_fs_info in fill_super() is to be avoided as it's brittle and inconsistent. Cleanup shouldn't be duplicated between sb->put_super() as sb->put_super() is only called if sb->s_root has been set aka when the filesystem has been successfully born (SB_BORN). That complexity should be avoided. This also means that block devices are to be closed in sb->kill_sb() instead of sb->put_super(). More details in the lower section. (3) Make it possible to lookup or create a superblock before opening block devices There's a subtle dependency on (2) as some filesystems did rely on fill_super() to be called in order to correctly clean up sb->s_fs_info. All these filesystems have been fixed. (4) Switch most filesystem to follow the same logic as the generic mount code now does as outlined in (3). (5) Use the superblock as the holder of the block device. We can now easily go back from block device to owning superblock. (6) Export and extend the generic fs_holder_ops and use them as holder ops everywhere and remove the filesystem specific holder ops. (7) Call from the block layer up into the filesystem layer when the block device is removed, allowing to shut down the filesystem without risk of deadlocks. (8) Get rid of get_super(). We can now easily go back from the block device to owning superblock and can call up from the block layer into the filesystem layer when the device is removed. So no need to wade through all registered superblock to find the owning superblock anymore" Link: https://lore.kernel.org/lkml/20230824-prall-intakt-95dbffdee4a0@brauner/ * tag 'v6.6-vfs.super' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (47 commits) super: use higher-level helper for {freeze,thaw} super: wait until we passed kill super super: wait for nascent superblocks super: make locking naming consistent super: use locking helpers fs: simplify invalidate_inodes fs: remove get_super block: call into the file system for ioctl BLKFLSBUF block: call into the file system for bdev_mark_dead block: consolidate __invalidate_device and fsync_bdev block: drop the "busy inodes on changed media" log message dasd: also call __invalidate_device when setting the device offline amiflop: don't call fsync_bdev in FDFMTBEG floppy: call disk_force_media_change when changing the format block: simplify the disk_force_media_change interface nbd: call blk_mark_disk_dead in nbd_clear_sock_ioctl xfs use fs_holder_ops for the log and RT devices xfs: drop s_umount over opening the log and RT devices ext4: use fs_holder_ops for the log device ext4: drop s_umount over opening the log device ... |
||
---|---|---|
.. | ||
inode.c | ||
internal.h | ||
Kconfig | ||
Makefile | ||
README | ||
uncompress.c |
Notes on Filesystem Layout -------------------------- These notes describe what mkcramfs generates. Kernel requirements are a bit looser, e.g. it doesn't care if the <file_data> items are swapped around (though it does care that directory entries (inodes) in a given directory are contiguous, as this is used by readdir). All data is currently in host-endian format; neither mkcramfs nor the kernel ever do swabbing. (See section `Block Size' below.) <filesystem>: <superblock> <directory_structure> <data> <superblock>: struct cramfs_super (see cramfs_fs.h). <directory_structure>: For each file: struct cramfs_inode (see cramfs_fs.h). Filename. Not generally null-terminated, but it is null-padded to a multiple of 4 bytes. The order of inode traversal is described as "width-first" (not to be confused with breadth-first); i.e. like depth-first but listing all of a directory's entries before recursing down its subdirectories: the same order as `ls -AUR' (but without the /^\..*:$/ directory header lines); put another way, the same order as `find -type d -exec ls -AU1 {} \;'. Beginning in 2.4.7, directory entries are sorted. This optimization allows cramfs_lookup to return more quickly when a filename does not exist, speeds up user-space directory sorts, etc. <data>: One <file_data> for each file that's either a symlink or a regular file of non-zero st_size. <file_data>: nblocks * <block_pointer> (where nblocks = (st_size - 1) / blksize + 1) nblocks * <block> padding to multiple of 4 bytes The i'th <block_pointer> for a file stores the byte offset of the *end* of the i'th <block> (i.e. one past the last byte, which is the same as the start of the (i+1)'th <block> if there is one). The first <block> immediately follows the last <block_pointer> for the file. <block_pointer>s are each 32 bits long. When the CRAMFS_FLAG_EXT_BLOCK_POINTERS capability bit is set, each <block_pointer>'s top bits may contain special flags as follows: CRAMFS_BLK_FLAG_UNCOMPRESSED (bit 31): The block data is not compressed and should be copied verbatim. CRAMFS_BLK_FLAG_DIRECT_PTR (bit 30): The <block_pointer> stores the actual block start offset and not its end, shifted right by 2 bits. The block must therefore be aligned to a 4-byte boundary. The block size is either blksize if CRAMFS_BLK_FLAG_UNCOMPRESSED is also specified, otherwise the compressed data length is included in the first 2 bytes of the block data. This is used to allow discontiguous data layout and specific data block alignments e.g. for XIP applications. The order of <file_data>'s is a depth-first descent of the directory tree, i.e. the same order as `find -size +0 \( -type f -o -type l \) -print'. <block>: The i'th <block> is the output of zlib's compress function applied to the i'th blksize-sized chunk of the input data if the corresponding CRAMFS_BLK_FLAG_UNCOMPRESSED <block_ptr> bit is not set, otherwise it is the input data directly. (For the last <block> of the file, the input may of course be smaller.) Each <block> may be a different size. (See <block_pointer> above.) <block>s are merely byte-aligned, not generally u32-aligned. When CRAMFS_BLK_FLAG_DIRECT_PTR is specified then the corresponding <block> may be located anywhere and not necessarily contiguous with the previous/next blocks. In that case it is minimally u32-aligned. If CRAMFS_BLK_FLAG_UNCOMPRESSED is also specified then the size is always blksize except for the last block which is limited by the file length. If CRAMFS_BLK_FLAG_DIRECT_PTR is set and CRAMFS_BLK_FLAG_UNCOMPRESSED is not set then the first 2 bytes of the block contains the size of the remaining block data as this cannot be determined from the placement of logically adjacent blocks. Holes ----- This kernel supports cramfs holes (i.e. [efficient representation of] blocks in uncompressed data consisting entirely of NUL bytes), but by default mkcramfs doesn't test for & create holes, since cramfs in kernels up to at least 2.3.39 didn't support holes. Run mkcramfs with -z if you want it to create files that can have holes in them. Tools ----- The cramfs user-space tools, including mkcramfs and cramfsck, are located at <http://sourceforge.net/projects/cramfs/>. Future Development ================== Block Size ---------- (Block size in cramfs refers to the size of input data that is compressed at a time. It's intended to be somewhere around PAGE_SIZE for cramfs_read_folio's convenience.) The superblock ought to indicate the block size that the fs was written for, since comments in <linux/pagemap.h> indicate that PAGE_SIZE may grow in future (if I interpret the comment correctly). Currently, mkcramfs #define's PAGE_SIZE as 4096 and uses that for blksize, whereas Linux-2.3.39 uses its PAGE_SIZE, which in turn is defined as PAGE_SIZE (which can be as large as 32KB on arm). This discrepancy is a bug, though it's not clear which should be changed. One option is to change mkcramfs to take its PAGE_SIZE from <asm/page.h>. Personally I don't like this option, but it does require the least amount of change: just change `#define PAGE_SIZE (4096)' to `#include <asm/page.h>'. The disadvantage is that the generated cramfs cannot always be shared between different kernels, not even necessarily kernels of the same architecture if PAGE_SIZE is subject to change between kernel versions (currently possible with arm and ia64). The remaining options try to make cramfs more sharable. One part of that is addressing endianness. The two options here are `always use little-endian' (like ext2fs) or `writer chooses endianness; kernel adapts at runtime'. Little-endian wins because of code simplicity and little CPU overhead even on big-endian machines. The cost of swabbing is changing the code to use the le32_to_cpu etc. macros as used by ext2fs. We don't need to swab the compressed data, only the superblock, inodes and block pointers. The other part of making cramfs more sharable is choosing a block size. The options are: 1. Always 4096 bytes. 2. Writer chooses blocksize; kernel adapts but rejects blocksize > PAGE_SIZE. 3. Writer chooses blocksize; kernel adapts even to blocksize > PAGE_SIZE. It's easy enough to change the kernel to use a smaller value than PAGE_SIZE: just make cramfs_read_folio read multiple blocks. The cost of option 1 is that kernels with a larger PAGE_SIZE value don't get as good compression as they can. The cost of option 2 relative to option 1 is that the code uses variables instead of #define'd constants. The gain is that people with kernels having larger PAGE_SIZE can make use of that if they don't mind their cramfs being inaccessible to kernels with smaller PAGE_SIZE values. Option 3 is easy to implement if we don't mind being CPU-inefficient: e.g. get read_folio to decompress to a buffer of size MAX_BLKSIZE (which must be no larger than 32KB) and discard what it doesn't need. Getting read_folio to read into all the covered pages is harder. The main advantage of option 3 over 1, 2, is better compression. The cost is greater complexity. Probably not worth it, but I hope someone will disagree. (If it is implemented, then I'll re-use that code in e2compr.) Another cost of 2 and 3 over 1 is making mkcramfs use a different block size, but that just means adding and parsing a -b option. Inode Size ---------- Given that cramfs will probably be used for CDs etc. as well as just silicon ROMs, it might make sense to expand the inode a little from its current 12 bytes. Inodes other than the root inode are followed by filename, so the expansion doesn't even have to be a multiple of 4 bytes.