linux/fs
David Howells 248f219cb8 rxrpc: Rewrite the data and ack handling code
Rewrite the data and ack handling code such that:

 (1) Parsing of received ACK and ABORT packets and the distribution and the
     filing of DATA packets happens entirely within the data_ready context
     called from the UDP socket.  This allows us to process and discard ACK
     and ABORT packets much more quickly (they're no longer stashed on a
     queue for a background thread to process).

 (2) We avoid calling skb_clone(), pskb_pull() and pskb_trim().  We instead
     keep track of the offset and length of the content of each packet in
     the sk_buff metadata.  This means we don't do any allocation in the
     receive path.

 (3) Jumbo DATA packet parsing is now done in data_ready context.  Rather
     than cloning the packet once for each subpacket and pulling/trimming
     it, we file the packet multiple times with an annotation for each
     indicating which subpacket is there.  From that we can directly
     calculate the offset and length.

 (4) A call's receive queue can be accessed without taking locks (memory
     barriers do have to be used, though).

 (5) Incoming calls are set up from preallocated resources and immediately
     made live.  They can than have packets queued upon them and ACKs
     generated.  If insufficient resources exist, DATA packet #1 is given a
     BUSY reply and other DATA packets are discarded).

 (6) sk_buffs no longer take a ref on their parent call.

To make this work, the following changes are made:

 (1) Each call's receive buffer is now a circular buffer of sk_buff
     pointers (rxtx_buffer) rather than a number of sk_buff_heads spread
     between the call and the socket.  This permits each sk_buff to be in
     the buffer multiple times.  The receive buffer is reused for the
     transmit buffer.

 (2) A circular buffer of annotations (rxtx_annotations) is kept parallel
     to the data buffer.  Transmission phase annotations indicate whether a
     buffered packet has been ACK'd or not and whether it needs
     retransmission.

     Receive phase annotations indicate whether a slot holds a whole packet
     or a jumbo subpacket and, if the latter, which subpacket.  They also
     note whether the packet has been decrypted in place.

 (3) DATA packet window tracking is much simplified.  Each phase has just
     two numbers representing the window (rx_hard_ack/rx_top and
     tx_hard_ack/tx_top).

     The hard_ack number is the sequence number before base of the window,
     representing the last packet the other side says it has consumed.
     hard_ack starts from 0 and the first packet is sequence number 1.

     The top number is the sequence number of the highest-numbered packet
     residing in the buffer.  Packets between hard_ack+1 and top are
     soft-ACK'd to indicate they've been received, but not yet consumed.

     Four macros, before(), before_eq(), after() and after_eq() are added
     to compare sequence numbers within the window.  This allows for the
     top of the window to wrap when the hard-ack sequence number gets close
     to the limit.

     Two flags, RXRPC_CALL_RX_LAST and RXRPC_CALL_TX_LAST, are added also
     to indicate when rx_top and tx_top point at the packets with the
     LAST_PACKET bit set, indicating the end of the phase.

 (4) Calls are queued on the socket 'receive queue' rather than packets.
     This means that we don't need have to invent dummy packets to queue to
     indicate abnormal/terminal states and we don't have to keep metadata
     packets (such as ABORTs) around

 (5) The offset and length of a (sub)packet's content are now passed to
     the verify_packet security op.  This is currently expected to decrypt
     the packet in place and validate it.

     However, there's now nowhere to store the revised offset and length of
     the actual data within the decrypted blob (there may be a header and
     padding to skip) because an sk_buff may represent multiple packets, so
     a locate_data security op is added to retrieve these details from the
     sk_buff content when needed.

 (6) recvmsg() now has to handle jumbo subpackets, where each subpacket is
     individually secured and needs to be individually decrypted.  The code
     to do this is broken out into rxrpc_recvmsg_data() and shared with the
     kernel API.  It now iterates over the call's receive buffer rather
     than walking the socket receive queue.

Additional changes:

 (1) The timers are condensed to a single timer that is set for the soonest
     of three timeouts (delayed ACK generation, DATA retransmission and
     call lifespan).

 (2) Transmission of ACK and ABORT packets is effected immediately from
     process-context socket ops/kernel API calls that cause them instead of
     them being punted off to a background work item.  The data_ready
     handler still has to defer to the background, though.

 (3) A shutdown op is added to the AF_RXRPC socket so that the AFS
     filesystem can shut down the socket and flush its own work items
     before closing the socket to deal with any in-progress service calls.

Future additional changes that will need to be considered:

 (1) Make sure that a call doesn't hog the front of the queue by receiving
     data from the network as fast as userspace is consuming it to the
     exclusion of other calls.

 (2) Transmit delayed ACKs from within recvmsg() when we've consumed
     sufficiently more packets to avoid the background work item needing to
     run.

Signed-off-by: David Howells <dhowells@redhat.com>
2016-09-08 11:10:12 +01:00
..
9p Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-07 10:01:14 -04:00
adfs Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-07 10:01:14 -04:00
affs get rid of 'parent' argument of ->d_compare() 2016-07-31 16:37:25 -04:00
afs rxrpc: Rewrite the data and ack handling code 2016-09-08 11:10:12 +01:00
autofs4 Merge branch 'work.const-qstr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-06 09:49:02 -04:00
befs fs/befs/io.c:befs_bread(): remove unneeded initialization to NULL 2016-05-23 17:04:14 -07:00
bfs more trivial ->iterate_shared conversions 2016-05-09 11:41:14 -04:00
btrfs Merge branch 'for-linus-4.8' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs 2016-08-26 20:22:01 -07:00
cachefiles cachefiles: Fix race between inactivating and culling a cache object 2016-08-03 13:33:26 -04:00
ceph ceph: initialize pathbase in the !dentry case in encode_caps_cb() 2016-08-09 17:26:56 +02:00
cifs get rid of 'parent' argument of ->d_compare() 2016-07-31 16:37:25 -04:00
coda drop redundant ->owner initializations 2016-05-29 19:08:00 -04:00
configfs configfs: don't set buffer_needs_fill to zero if show() returns error 2016-07-10 21:02:18 +09:00
cramfs more trivial ->iterate_shared conversions 2016-05-09 11:41:14 -04:00
crypto block, fs, mm, drivers: use bio set/get op accessors 2016-06-07 13:41:38 -06:00
debugfs Merge branch 'd_real' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs into work.misc 2016-06-30 23:34:49 -04:00
devpts userns: Remove the now unnecessary FS_USERNS_DEV_MOUNT flag 2016-06-23 15:47:31 -05:00
dlm dlm: fix malfunction of dlm_tool caused by debugfs changes 2016-08-26 13:22:14 -05:00
ecryptfs ecryptfs: don't allow mmap when the lower fs doesn't support it 2016-07-08 10:35:28 -05:00
efivarfs get rid of 'parent' argument of ->d_compare() 2016-07-31 16:37:25 -04:00
efs fs/efs/super.c: fix return value 2016-05-20 17:58:30 -07:00
exofs block, fs, mm, drivers: use bio set/get op accessors 2016-06-07 13:41:38 -06:00
exportfs introduce a parallel variant of ->iterate() 2016-05-02 19:49:29 -04:00
ext2 Merge branch 'work.const-qstr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-06 09:49:02 -04:00
ext4 Fix bugs that could cause kernel deadlocks or file system corruption 2016-08-29 12:37:11 -07:00
f2fs f2fs: avoid potential deadlock in f2fs_move_file_range 2016-08-19 11:15:08 +09:00
fat Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-07 10:01:14 -04:00
freevxfs freevxfs: update Kconfig information 2016-06-13 10:20:39 +02:00
fscache Merge branch 'd_real' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs into work.misc 2016-06-30 23:34:49 -04:00
fuse Merge branch 'work.const-qstr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-06 09:49:02 -04:00
gfs2 fs: return EPERM on immutable inode 2016-08-07 10:03:31 -04:00
hfs Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-07 10:01:14 -04:00
hfsplus Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-07 10:01:14 -04:00
hostfs hostfs: Freeing an ERR_PTR in hostfs_fill_sb_common() 2016-08-04 00:18:10 +02:00
hpfs get rid of 'parent' argument of ->d_compare() 2016-07-31 16:37:25 -04:00
hugetlbfs mm, fs: remove remaining PAGE_CACHE_* and page_cache_{get,release} usage 2016-04-04 10:41:08 -07:00
isofs get rid of 'parent' argument of ->d_compare() 2016-07-31 16:37:25 -04:00
jbd2 The major change this cycle is deleting ext4's copy of the file system 2016-07-26 18:35:55 -07:00
jffs2 vfs: make the string hashes salt the hash 2016-06-10 20:21:46 -07:00
jfs get rid of 'parent' argument of ->d_compare() 2016-07-31 16:37:25 -04:00
kernfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2016-07-29 15:54:19 -07:00
lockd Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-07-28 12:59:05 -07:00
logfs Merge branch 'work.const-qstr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-06 09:49:02 -04:00
minix simple local filesystems: switch to ->iterate_shared() 2016-05-02 19:49:32 -04:00
ncpfs get rid of 'parent' argument of ->d_compare() 2016-07-31 16:37:25 -04:00
nfs NFS client bugfixes for Linux 4.8 2016-08-12 12:32:24 -07:00
nfs_common
nfsd nfsd: don't return an unhashed lock stateid after taking mutex 2016-08-12 16:10:25 -04:00
nilfs2 nilfs2: move ioctl interface and disk layout to uapi separately 2016-08-02 19:35:21 -04:00
nls
notify fsnotify: avoid spurious EMFILE errors from inotify_init() 2016-05-19 19:12:14 -07:00
ntfs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-07-28 12:59:05 -07:00
ocfs2 Merge branch 'work.const-qstr' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-06 09:49:02 -04:00
omfs more trivial ->iterate_shared conversions 2016-05-09 11:41:14 -04:00
openpromfs more trivial ->iterate_shared conversions 2016-05-09 11:41:14 -04:00
orangefs orangefs: Account for jiffies wraparound. 2016-08-02 15:39:13 -04:00
overlayfs ovl: simplify empty checking 2016-07-29 12:05:25 +02:00
proc Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-08-18 01:17:32 -04:00
pstore ramoops: use persistent_ram_free() instead of kfree() for freeing prz 2016-08-05 11:21:46 -07:00
qnx4 more trivial ->iterate_shared conversions 2016-05-09 11:41:14 -04:00
qnx6 more trivial ->iterate_shared conversions 2016-05-09 11:41:14 -04:00
quota Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2016-07-29 15:54:19 -07:00
ramfs tmpfs/ramfs: fix VM_MAYSHARE mappings for NOMMU 2016-05-20 17:58:30 -07:00
reiserfs reiserfs: fix "new_insert_key may be used uninitialized ..." 2016-08-02 19:35:22 -04:00
romfs romfs, squashfs: switch to ->iterate_shared() 2016-05-09 11:41:15 -04:00
squashfs fs: have ll_rw_block users pass in op and flags separately 2016-06-07 13:41:38 -06:00
sysfs kernfs: The cgroup filesystem also benefits from SB_I_NOEXEC 2016-06-23 15:41:56 -05:00
sysv vfs: make the string hashes salt the hash 2016-06-10 20:21:46 -07:00
tracefs tracefs: ->d_parent is never NULL or negative... 2016-05-29 16:22:07 -04:00
ubifs ubifs: Fix xattr generic handler usage 2016-08-23 23:02:52 +02:00
udf Merge branch 'for-4.8/drivers' of git://git.kernel.dk/linux-block 2016-07-26 15:37:51 -07:00
ufs Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-07-28 12:59:05 -07:00
xfs Merge branch 'iomap-fixes-4.8-rc3' into for-next 2016-08-17 11:13:37 +10:00
aio.c aio: make aio_setup_ring killable 2016-05-23 17:04:14 -07:00
anon_inodes.c
attr.c vfs: Don't modify inodes with a uid or gid unknown to the vfs 2016-07-05 15:06:46 -05:00
bad_inode.c switch ->setxattr() to passing dentry and inode separately 2016-05-27 20:09:16 -04:00
binfmt_aout.c fs: fix binfmt_aout.c build error 2016-05-28 16:34:59 -07:00
binfmt_elf_fdpic.c elf_fdpic_transfer_args_to_stack(): make it generic 2016-07-25 16:51:49 +10:00
binfmt_elf.c binfmt_elf: fix calculations for bss padding 2016-08-02 19:35:14 -04:00
binfmt_em86.c fs/binfmt_em86.c: fix incompatible pointer type 2016-08-02 19:35:15 -04:00
binfmt_flat.c binfmt_flat: allow compressed flat binary format to work on MMU systems 2016-07-28 13:29:12 +10:00
binfmt_misc.c binfmt_misc for-linus on 20160727 2016-08-07 10:13:14 -04:00
binfmt_script.c
block_dev.c fs/block_dev: fix potential NULL ptr deref in freeze_bdev() 2016-08-25 08:38:26 -06:00
buffer.c xfs: update for 4.8-rc1 2016-07-27 09:53:35 -07:00
char_dev.c chardev: add missing line break in pr_warn 2016-07-14 16:21:53 +09:00
compat_binfmt_elf.c
compat_ioctl.c [media] cec: add compat32 ioctl support 2016-06-28 10:00:13 -03:00
compat.c Fix a number of bugs, most notably a potential stale data exposure 2016-05-24 12:55:26 -07:00
coredump.c coredump: fix dumping through pipes 2016-06-07 22:07:09 -04:00
dax.c libnvdimm for 4.8 2016-07-28 17:38:16 -07:00
dcache.c Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-07 10:01:14 -04:00
dcookies.c
direct-io.c direct-io: use bio set/get op accessors 2016-06-07 13:41:38 -06:00
drop_caches.c
eventfd.c eventfd: document lockless access in eventfd_poll 2016-03-22 15:36:02 -07:00
eventpoll.c fs: poll/select/recvmmsg: use timespec64 for timeout events 2016-05-19 19:12:14 -07:00
exec.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu 2016-08-04 18:04:44 -04:00
fcntl.c
fhandle.c fs/coredump: prevent fsuid=0 dumps into user-controlled directories 2016-03-22 15:36:02 -07:00
file_table.c
file.c give readdir(2)/getdents(2)/etc. uniform exclusion with lseek() 2016-05-02 19:49:28 -04:00
filesystems.c
fs_pin.c
fs_struct.c
fs-writeback.c mm, writeback: flush plugged IO in wakeup_flusher_threads() 2016-08-09 19:58:06 -06:00
inode.c Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-08-07 10:01:14 -04:00
internal.h binfmt_misc for-linus on 20160727 2016-08-07 10:13:14 -04:00
ioctl.c vfs: ioctl: prevent double-fetch in dedupe ioctl 2016-07-28 15:23:12 -07:00
iomap.c iomap: mark ->iomap_end as optional 2016-08-17 08:42:34 +10:00
Kconfig Highlights: 2016-08-04 19:59:06 -04:00
Kconfig.binfmt m68k: enable binfmt_flat on systems with an MMU 2016-07-28 13:29:13 +10:00
libfs.c lockless next_positive() 2016-06-20 17:11:29 -04:00
locks.c locks: use file_inode() 2016-07-01 10:24:18 -04:00
Makefile fs: introduce iomap infrastructure 2016-06-21 09:23:11 +10:00
mbcache.c mbcache: add reusable flag to cache entries 2016-02-22 22:44:04 -05:00
mount.h
mpage.c block/mm: make bdev_ops->rw_page() take a bool for read/write 2016-08-07 14:41:02 -06:00
namei.c fs: return EPERM on immutable inode 2016-08-07 10:03:31 -04:00
namespace.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2016-07-29 15:54:19 -07:00
no-block.c
nsfs.c
open.c binfmt_misc for-linus on 20160727 2016-08-07 10:13:14 -04:00
pipe.c mm: memcontrol: only mark charged pages with PageKmemcg 2016-08-09 10:14:10 -07:00
pnode.c propogate_mnt: Handle the first propogated copy being a slave 2016-05-05 09:54:45 -05:00
pnode.h
posix_acl.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2016-07-29 15:54:19 -07:00
proc_namespace.c vfs: show_vfsstat: do not ignore errors from show_devname method 2016-03-16 13:09:08 -04:00
read_write.c x86/syscalls: Add compat_sys_preadv64v2/compat_sys_pwritev64v2 2016-07-15 10:30:26 +02:00
readdir.c restore killability of old mutex_lock_killable(&inode->i_mutex) users 2016-05-26 00:13:25 -04:00
select.c fs: poll/select/recvmmsg: use timespec64 for timeout events 2016-05-19 19:12:14 -07:00
seq_file.c fs/seq_file: fix out-of-bounds read 2016-08-26 17:39:35 -07:00
signalfd.c
splice.c Merge branch 'ovl-fixes' into for-linus 2016-05-11 00:00:29 -04:00
stack.c
stat.c
statfs.c
super.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2016-07-29 15:54:19 -07:00
sync.c mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros 2016-04-04 10:41:08 -07:00
timerfd.c timerfd: Reject ALARM timerfds without CAP_WAKE_ALARM 2016-06-09 23:42:38 +02:00
userfaultfd.c mm: introduce fault_env 2016-07-26 16:19:19 -07:00
utimes.c fs: return EPERM on immutable inode 2016-08-07 10:03:31 -04:00
xattr.c vfs: Don't modify inodes with a uid or gid unknown to the vfs 2016-07-05 15:06:46 -05:00