linux/fs
Linus Torvalds 6c32978414 Notifications over pipes + Keyring notifications
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEqG5UsNXhtOCrfGQP+7dXa6fLC2sFAl7U/i8ACgkQ+7dXa6fL
 C2u2eg/+Oy6ybq0hPovYVkFI9WIG7ZCz7w9Q6BEnfYMqqn3dnfJxKQ3l4pnQEOWw
 f4QfvpvevsYfMtOJkYcG6s66rQgbFdqc5TEyBBy0QNp3acRolN7IXkcopvv9xOpQ
 JxedpbFG1PTFLWjvBpyjlrUPouwLzq2FXAf1Ox0ZIMw6165mYOMWoli1VL8dh0A0
 Ai7JUB0WrvTNbrwhV413obIzXT/rPCdcrgbQcgrrLPex8lQ47ZAE9bq6k4q5HiwK
 KRzEqkQgnzId6cCNTFBfkTWsx89zZunz7jkfM5yx30MvdAtPSxvvpfIPdZRZkXsP
 E2K9Fk1/6OQZTC0Op3Pi/bt+hVG/mD1p0sQUDgo2MO3qlSS+5mMkR8h3mJEgwK12
 72P4YfOJkuAy2z3v4lL0GYdUDAZY6i6G8TMxERKu/a9O3VjTWICDOyBUS6F8YEAK
 C7HlbZxAEOKTVK0BTDTeEUBwSeDrBbvH6MnRlZCG5g1Fos2aWP0udhjiX8IfZLO7
 GN6nWBvK1fYzfsUczdhgnoCzQs3suoDo04HnsTPGJ8De52T4x2RsjV+gPx0nrNAq
 eWChl1JvMWsY2B3GLnl9XQz4NNN+EreKEkk+PULDGllrArrPsp5Vnhb9FJO1PVCU
 hMDJHohPiXnKbc8f4Bd78OhIvnuoGfJPdM5MtNe2flUKy2a2ops=
 =YTGf
 -----END PGP SIGNATURE-----

Merge tag 'notifications-20200601' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs

Pull notification queue from David Howells:
 "This adds a general notification queue concept and adds an event
  source for keys/keyrings, such as linking and unlinking keys and
  changing their attributes.

  Thanks to Debarshi Ray, we do have a pull request to use this to fix a
  problem with gnome-online-accounts - as mentioned last time:

     https://gitlab.gnome.org/GNOME/gnome-online-accounts/merge_requests/47

  Without this, g-o-a has to constantly poll a keyring-based kerberos
  cache to find out if kinit has changed anything.

  [ There are other notification pending: mount/sb fsinfo notifications
    for libmount that Karel Zak and Ian Kent have been working on, and
    Christian Brauner would like to use them in lxc, but let's see how
    this one works first ]

  LSM hooks are included:

   - A set of hooks are provided that allow an LSM to rule on whether or
     not a watch may be set. Each of these hooks takes a different
     "watched object" parameter, so they're not really shareable. The
     LSM should use current's credentials. [Wanted by SELinux & Smack]

   - A hook is provided to allow an LSM to rule on whether or not a
     particular message may be posted to a particular queue. This is
     given the credentials from the event generator (which may be the
     system) and the watch setter. [Wanted by Smack]

  I've provided SELinux and Smack with implementations of some of these
  hooks.

  WHY
  ===

  Key/keyring notifications are desirable because if you have your
  kerberos tickets in a file/directory, your Gnome desktop will monitor
  that using something like fanotify and tell you if your credentials
  cache changes.

  However, we also have the ability to cache your kerberos tickets in
  the session, user or persistent keyring so that it isn't left around
  on disk across a reboot or logout. Keyrings, however, cannot currently
  be monitored asynchronously, so the desktop has to poll for it - not
  so good on a laptop. This facility will allow the desktop to avoid the
  need to poll.

  DESIGN DECISIONS
  ================

   - The notification queue is built on top of a standard pipe. Messages
     are effectively spliced in. The pipe is opened with a special flag:

        pipe2(fds, O_NOTIFICATION_PIPE);

     The special flag has the same value as O_EXCL (which doesn't seem
     like it will ever be applicable in this context)[?]. It is given up
     front to make it a lot easier to prohibit splice&co from accessing
     the pipe.

     [?] Should this be done some other way?  I'd rather not use up a new
         O_* flag if I can avoid it - should I add a pipe3() system call
         instead?

     The pipe is then configured::

        ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, queue_depth);
        ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter);

     Messages are then read out of the pipe using read().

   - It should be possible to allow write() to insert data into the
     notification pipes too, but this is currently disabled as the
     kernel has to be able to insert messages into the pipe *without*
     holding pipe->mutex and the code to make this work needs careful
     auditing.

   - sendfile(), splice() and vmsplice() are disabled on notification
     pipes because of the pipe->mutex issue and also because they
     sometimes want to revert what they just did - but one or more
     notification messages might've been interleaved in the ring.

   - The kernel inserts messages with the wait queue spinlock held. This
     means that pipe_read() and pipe_write() have to take the spinlock
     to update the queue pointers.

   - Records in the buffer are binary, typed and have a length so that
     they can be of varying size.

     This allows multiple heterogeneous sources to share a common
     buffer; there are 16 million types available, of which I've used
     just a few, so there is scope for others to be used. Tags may be
     specified when a watchpoint is created to help distinguish the
     sources.

   - Records are filterable as types have up to 256 subtypes that can be
     individually filtered. Other filtration is also available.

   - Notification pipes don't interfere with each other; each may be
     bound to a different set of watches. Any particular notification
     will be copied to all the queues that are currently watching for it
     - and only those that are watching for it.

   - When recording a notification, the kernel will not sleep, but will
     rather mark a queue as having lost a message if there's
     insufficient space. read() will fabricate a loss notification
     message at an appropriate point later.

   - The notification pipe is created and then watchpoints are attached
     to it, using one of:

        keyctl_watch_key(KEY_SPEC_SESSION_KEYRING, fds[1], 0x01);
        watch_mount(AT_FDCWD, "/", 0, fd, 0x02);
        watch_sb(AT_FDCWD, "/mnt", 0, fd, 0x03);

     where in both cases, fd indicates the queue and the number after is
     a tag between 0 and 255.

   - Watches are removed if either the notification pipe is destroyed or
     the watched object is destroyed. In the latter case, a message will
     be generated indicating the enforced watch removal.

  Things I want to avoid:

   - Introducing features that make the core VFS dependent on the
     network stack or networking namespaces (ie. usage of netlink).

   - Dumping all this stuff into dmesg and having a daemon that sits
     there parsing the output and distributing it as this then puts the
     responsibility for security into userspace and makes handling
     namespaces tricky. Further, dmesg might not exist or might be
     inaccessible inside a container.

   - Letting users see events they shouldn't be able to see.

  TESTING AND MANPAGES
  ====================

   - The keyutils tree has a pipe-watch branch that has keyctl commands
     for making use of notifications. Proposed manual pages can also be
     found on this branch, though a couple of them really need to go to
     the main manpages repository instead.

     If the kernel supports the watching of keys, then running "make
     test" on that branch will cause the testing infrastructure to spawn
     a monitoring process on the side that monitors a notifications pipe
     for all the key/keyring changes induced by the tests and they'll
     all be checked off to make sure they happened.

        https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/keyutils.git/log/?h=pipe-watch

   - A test program is provided (samples/watch_queue/watch_test) that
     can be used to monitor for keyrings, mount and superblock events.
     Information on the notifications is simply logged to stdout"

* tag 'notifications-20200601' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs:
  smack: Implement the watch_key and post_notification hooks
  selinux: Implement the watch_key security hook
  keys: Make the KEY_NEED_* perms an enum rather than a mask
  pipe: Add notification lossage handling
  pipe: Allow buffers to be marked read-whole-or-error for notifications
  Add sample notification program
  watch_queue: Add a key/keyring notification facility
  security: Add hooks to rule on setting a watch
  pipe: Add general notification queue support
  pipe: Add O_NOTIFICATION_PIPE
  security: Add a hook for the point of notification insertion
  uapi: General notification queue definitions
2020-06-13 09:56:21 -07:00
..
9p 9p: read only once on O_NONBLOCK 2020-03-27 09:29:56 +00:00
adfs docs: filesystems: fix renamed references 2020-04-20 15:45:22 -06:00
affs docs: filesystems: fix renamed references 2020-04-20 15:45:22 -06:00
afs afs: Fix afs_store_data() to set mtime in new operation descriptor 2020-06-11 16:04:30 -07:00
autofs
befs
bfs docs: filesystems: fix renamed references 2020-04-20 15:45:22 -06:00
btrfs A lot of bug fixes and cleanups for ext4, including: 2020-06-05 16:19:28 -07:00
cachefiles A fair amount of stuff this time around, dominated by yet another massive 2020-06-01 15:45:27 -07:00
ceph ceph: skip checking caps when session reconnecting and releasing reqs 2020-06-01 13:22:53 +02:00
cifs 22 changesets, 2 for stable. Includes big performance improvement for large i/o when using multichannel, also includes DFS fixes 2020-06-05 16:40:53 -07:00
coda docs: filesystems: convert coda.txt to ReST 2020-05-05 09:22:21 -06:00
configfs A fair amount of stuff this time around, dominated by yet another massive 2020-06-01 15:45:27 -07:00
cramfs docs: filesystems: fix renamed references 2020-04-20 15:45:22 -06:00
crypto fscrypt updates for 5.8 2020-06-01 12:10:17 -07:00
debugfs Merge 5.7-rc3 into driver-core-next 2020-04-27 09:34:55 +02:00
devpts
dlm dlm for 5.8 2020-06-05 16:43:16 -07:00
ecryptfs A fair amount of stuff this time around, dominated by yet another massive 2020-06-01 15:45:27 -07:00
efivarfs
efs
erofs Changes since last update: 2020-06-02 20:16:55 -07:00
exfat Description for this pull request: 2020-06-09 11:24:59 -07:00
exportfs
ext2 mmap locking API: convert mmap_sem comments 2020-06-09 09:39:14 -07:00
ext4 linux-kselftest-kunit-5.8-rc1 2020-06-09 10:04:47 -07:00
f2fs f2fs-for-5.8-rc1 2020-06-09 11:28:59 -07:00
fat fat: improve the readahead for FAT entries 2020-06-04 19:06:25 -07:00
freevxfs
fscache Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next 2020-06-03 16:27:18 -07:00
fuse fuse update for 5.8 2020-06-09 15:48:24 -07:00
gfs2 Changes in gfs2: 2020-06-08 12:47:09 -07:00
hfs for-5.8/block-2020-06-01 2020-06-02 15:29:19 -07:00
hfsplus block: remove the error_sector argument to blkdev_issue_flush 2020-05-22 08:45:46 -06:00
hostfs hostfs: Use kasprintf() instead of fixed buffer formatting 2020-03-29 23:23:00 +02:00
hpfs hpfs: fix warning due to superfluous semicolon 2020-06-06 10:08:17 -07:00
hugetlbfs mmap locking API: convert mmap_sem API comments 2020-06-09 09:39:14 -07:00
iomap A lot of bug fixes and cleanups for ext4, including: 2020-06-05 16:19:28 -07:00
isofs for-5.8/block-2020-06-01 2020-06-02 15:29:19 -07:00
jbd2 A lot of bug fixes and cleanups for ext4, including: 2020-06-05 16:19:28 -07:00
jffs2
jfs Replace zero-length array in JFS 2020-06-02 20:11:35 -07:00
kernfs mmap locking API: convert mmap_sem comments 2020-06-09 09:39:14 -07:00
lockd
minix
nfs NFS Client Updates for Linux 5.8 2020-06-11 12:22:41 -07:00
nfs_common
nfsd Highlights: 2020-06-11 10:33:13 -07:00
nilfs2 nilfs2: fix null pointer dereference at nilfs_segctor_do_construct() 2020-06-10 19:14:17 -07:00
nls
notify \n 2020-06-04 13:51:54 -07:00
ntfs Merge branch 'akpm' (patches from Andrew) 2020-06-02 12:21:36 -07:00
ocfs2 ocfs2: fix build failure when TCP/IP is disabled 2020-06-11 18:17:47 -07:00
omfs fs: convert mpage_readpages to mpage_readahead 2020-06-02 10:59:07 -07:00
openpromfs
orangefs orangefs: a conversion and a cleanup... 2020-06-05 16:44:36 -07:00
overlayfs overlayfs update for 5.8 2020-06-09 15:40:50 -07:00
proc Merge branch 'proc-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2020-06-12 12:38:18 -07:00
pstore Merge branch 'uaccess.__copy_from_user' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-06-01 16:18:46 -07:00
qnx4
qnx6 fs: convert mpage_readpages to mpage_readahead 2020-06-02 10:59:07 -07:00
quota sysctl: pass kernel pointers to ->proc_handler 2020-04-27 02:07:40 -04:00
ramfs
reiserfs \n 2020-06-04 13:53:10 -07:00
romfs docs: filesystems: fix renamed references 2020-04-20 15:45:22 -06:00
squashfs Merge branch 'akpm' (patches from Andrew) 2020-06-02 12:21:36 -07:00
sysfs RDMA 5.8 merge window pull request 2020-06-05 14:05:57 -07:00
sysv docs: filesystems: fix renamed references 2020-04-20 15:45:22 -06:00
tracefs
ubifs mm: remove the pgprot argument to __vmalloc 2020-06-02 10:59:11 -07:00
udf for-5.8/block-2020-06-01 2020-06-02 15:29:19 -07:00
ufs
unicode .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
vboxsf vboxsf: don't use the source name in the bdi name 2020-05-07 08:45:47 -06:00
verity fs-verity: remove unnecessary extern keywords 2020-05-12 16:44:00 -07:00
xfs Third part of new DAX code for 5.8: 2020-06-11 10:48:12 -07:00
zonefs zonefs changes for 5.8 2020-06-04 13:50:13 -07:00
aio.c kernel: move use_mm/unuse_mm to kthread.c 2020-06-10 19:14:18 -07:00
anon_inodes.c
attr.c
bad_inode.c fs: move the fiemap definitions out of fs.h 2020-06-03 23:16:55 -04:00
binfmt_aout.c exec: Rename flush_old_exec begin_new_exec 2020-05-07 16:55:47 -05:00
binfmt_elf_fdpic.c Merge branch 'uaccess.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-06-10 16:02:54 -07:00
binfmt_elf.c Merge branch 'uaccess.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-06-10 16:02:54 -07:00
binfmt_em86.c Merge branch 'akpm' (patches from Andrew) 2020-06-04 19:18:29 -07:00
binfmt_flat.c Merge branch 'uaccess.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-06-10 16:02:54 -07:00
binfmt_misc.c Merge branch 'akpm' (patches from Andrew) 2020-06-04 19:18:29 -07:00
binfmt_script.c Merge branch 'akpm' (patches from Andrew) 2020-06-04 19:18:29 -07:00
block_dev.c for-5.8/drivers-2020-06-01 2020-06-02 15:37:03 -07:00
buffer.c fs/buffer.c: use attach/detach_page_private 2020-06-02 10:59:07 -07:00
char_dev.c vfs: allow unprivileged whiteout creation 2020-05-14 16:44:23 +02:00
compat_binfmt_elf.c Split the old READ_IMPLIES_EXEC workaround from executable PT_GNU_STACK 2020-06-05 13:45:21 -07:00
compat.c
coredump.c mmap locking API: convert mmap_sem comments 2020-06-09 09:39:14 -07:00
d_path.c
dax.c dax,iomap: Add helper dax_iomap_zero() to zero a range 2020-04-02 19:15:03 -07:00
dcache.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next 2020-06-03 16:27:18 -07:00
dcookies.c
direct-io.c for-5.8-tag 2020-06-02 19:59:25 -07:00
drop_caches.c sysctl: pass kernel pointers to ->proc_handler 2020-04-27 02:07:40 -04:00
eventfd.c eventfd: convert to f_op->read_iter() 2020-05-06 22:33:43 -04:00
eventpoll.c epoll: call final ep_events_available() check under the lock 2020-05-14 10:00:35 -07:00
exec.c mmap locking API: convert mmap_sem comments 2020-06-09 09:39:14 -07:00
fcntl.c
fhandle.c
file_table.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next 2020-06-03 16:27:18 -07:00
file.c fix multiplication overflow in copy_fdtable() 2020-05-19 18:29:36 -04:00
filesystems.c fs/filesystems.c: downgrade user-reachable WARN_ONCE() to pr_warn_once() 2020-04-10 15:36:22 -07:00
fs_context.c vfs: don't parse "silent" option 2020-05-14 16:44:25 +02:00
fs_parser.c fs_parse: remove pr_notice() about each validation 2020-04-02 09:35:26 -07:00
fs_pin.c
fs_struct.c
fs_types.c
fs-writeback.c A lot of bug fixes and cleanups for ext4, including: 2020-06-05 16:19:28 -07:00
fsopen.c
inode.c AFS Changes 2020-06-05 16:26:36 -07:00
internal.h A lot of bug fixes and cleanups for ext4, including: 2020-06-05 16:19:28 -07:00
io_uring.c io_uring-5.8-2020-06-11 2020-06-11 16:10:08 -07:00
io-wq.c io_uring-5.8-2020-06-11 2020-06-11 16:10:08 -07:00
io-wq.h io_uring: avoid whole io_wq_work copy for requests completed inline 2020-06-10 17:58:46 -06:00
ioctl.c fs: remove the access_ok() check in ioctl_fiemap 2020-06-03 23:16:55 -04:00
Kconfig docs: filesystems: fix renamed references 2020-04-20 15:45:22 -06:00
Kconfig.binfmt Fix for arch/sh build regression with newer binutils, removal of SH5, 2020-06-06 15:22:01 -07:00
libfs.c block: remove the error_sector argument to blkdev_issue_flush 2020-05-22 08:45:46 -06:00
locks.c Highlights: 2020-06-11 10:33:13 -07:00
Makefile
mbcache.c
mount.h proc/mounts: add cursor 2020-05-14 16:44:24 +02:00
mpage.c fs: convert mpage_readpages to mpage_readahead 2020-06-02 10:59:07 -07:00
namei.c vfs: clean up posix_acl_permission() logic aroudn MAY_NOT_BLOCK 2020-06-08 11:04:19 -07:00
namespace.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-06-10 16:09:11 -07:00
no-block.c
nsfs.c nsproxy: attach to namespaces via pidfds 2020-05-13 11:41:22 +02:00
open.c Merge branch 'akpm' (patches from Andrew) 2020-06-02 12:21:36 -07:00
pipe.c Notifications over pipes + Keyring notifications 2020-06-13 09:56:21 -07:00
pnode.c propagate_one(): mnt_set_mountpoint() needs mount_lock 2020-04-27 10:37:14 -04:00
pnode.h
posix_acl.c vfs: clean up posix_acl_permission() logic aroudn MAY_NOT_BLOCK 2020-06-08 11:04:19 -07:00
proc_namespace.c Merge branch 'proc-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2020-06-04 13:54:34 -07:00
read_write.c powerpc: Add back __ARCH_WANT_SYS_LLSEEK macro 2020-04-03 00:09:59 +11:00
readdir.c readdir.c: get rid of the last __put_user(), drop now-useless access_ok() 2020-05-01 20:29:54 -04:00
select.c pselect6() and friends: take handling the combined 6th/7th args into helper 2020-05-29 19:10:42 -04:00
seq_file.c fs/seq_file.c: seq_read: Update pr_info_ratelimited 2020-06-04 19:06:25 -07:00
signalfd.c
splice.c Notifications over pipes + Keyring notifications 2020-06-13 09:56:21 -07:00
stack.c
stat.c New code for 5.8: 2020-06-02 19:45:12 -07:00
statfs.c
super.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2020-06-10 16:09:11 -07:00
sync.c overlayfs update for 5.8 2020-06-09 15:40:50 -07:00
timerfd.c
userfaultfd.c mmap locking API: convert mmap_sem comments 2020-06-09 09:39:14 -07:00
utimes.c utimensat: AT_EMPTY_PATH support 2020-05-14 16:44:24 +02:00
xattr.c xattr: fix uninitialized out-param 2020-04-09 15:33:09 -04:00