leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity
df0108c5da
linux/fs
History
Jason Baron df0108c5da epoll: add EPOLLEXCLUSIVE flag 
Currently, epoll file descriptors or epfds (the fd returned from
epoll_create[1]()) that are added to a shared wakeup source are always
added in a non-exclusive manner.  This means that when we have multiple
epfds attached to a shared fd source they are all woken up.  This creates
thundering herd type behavior.

Introduce a new 'EPOLLEXCLUSIVE' flag that can be passed as part of the
'event' argument during an epoll_ctl() EPOLL_CTL_ADD operation.  This new
flag allows for exclusive wakeups when there are multiple epfds attached
to a shared fd event source.

The implementation walks the list of exclusive waiters, and queues an
event to each epfd, until it finds the first waiter that has threads
blocked on it via epoll_wait().  The idea is to search for threads which
are idle and ready to process the wakeup events.  Thus, we queue an event
to at least 1 epfd, but may still potentially queue an event to all epfds
that are attached to the shared fd source.

Performance testing was done by Madars Vitolins using a modified version
of Enduro/X.  The use of the 'EPOLLEXCLUSIVE' flag reduce the length of
this particular workload from 860s down to 24s.

Sample epoll_clt text:

EPOLLEXCLUSIVE

  Sets an exclusive wakeup mode for the epfd file descriptor that is
  being attached to the target file descriptor, fd.  Thus, when an event
  occurs and multiple epfd file descriptors are attached to the same
  target file using EPOLLEXCLUSIVE, one or more epfds will receive an
  event with epoll_wait(2).  The default in this scenario (when
  EPOLLEXCLUSIVE is not set) is for all epfds to receive an event.
  EPOLLEXCLUSIVE may only be specified with the op EPOLL_CTL_ADD.

Signed-off-by: Jason Baron <jbaron@akamai.com>
Tested-by: Madars Vitolins <m@silodev.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Al Viro <viro@ftp.linux.org.uk>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: Eric Wong <normalperson@yhbt.net>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Hagen Paul Pfeifer <hagen@jauu.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-20 17:09:18 -08:00
..
9p kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
adfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
affs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
afs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
autofs4 switch ->get_link() to delayed_call, kill ->put_link() 2015-12-30 13:01:03 -05:00
befs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
bfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
btrfs Merge branch 'for-linus-4.5' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs 2016-01-18 12:44:40 -08:00
cachefiles convert a bunch of open-coded instances of memdup_user_nul() 2016-01-04 10:26:58 -05:00
ceph kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
cifs page-flags: define PG_locked behavior on compound pages 2016-01-15 17:56:32 -08:00
coda kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
configfs Configfs changes for the 4.5 merge window: 2016-01-12 18:15:34 -08:00
cramfs don't put symlink bodies in pagecache into highmem 2015-12-08 22:41:36 -05:00
debugfs debugfs: fix refcount imbalance in start_creating 2015-11-11 02:04:44 -05:00
devpts
dlm convert a bunch of open-coded instances of memdup_user_nul() 2016-01-04 10:26:58 -05:00
ecryptfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
efivarfs
efs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
exofs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
exportfs
ext2 kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
ext4 kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
f2fs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
fat kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
freevxfs don't put symlink bodies in pagecache into highmem 2015-12-08 22:41:36 -05:00
fscache FS-Cache: Handle a write to the page immediately beyond the EOF marker 2015-11-11 02:11:02 -05:00
fuse kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
gfs2 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security 2016-01-17 19:13:15 -08:00
hfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
hfsplus kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
hostfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
hpfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
hugetlbfs mm/hugetlbfs: unmap pages if page fault raced with hole punch 2016-01-15 17:56:32 -08:00
isofs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
jbd2 fs: use block_device name vsprintf helper 2016-01-06 13:03:18 -05:00
jffs2 kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
jfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
kernfs Revert "kernfs: do not account ino_ida allocations to memcg" 2016-01-14 16:00:49 -08:00
lockd lockd: constify nlmsvc_binding structure 2016-01-07 10:10:50 -05:00
logfs Merge branch 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild 2016-01-20 09:45:43 -08:00
minix kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
ncpfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
nfs Merge branch 'akpm' (patches from Andrew) 2016-01-15 11:41:44 -08:00
nfs_common lockd: NLM grace period shouldn't block NFSv4 opens 2015-08-13 10:22:06 -04:00
nfsd Smaller bugfixes and cleanup, including a fix for a failures of 2016-01-15 12:49:44 -08:00
nilfs2 kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
nls
notify fsnotify: destroy marks with call_srcu instead of dedicated thread 2016-01-14 16:00:49 -08:00
ntfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
ocfs2 Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs 2016-01-15 11:51:51 -08:00
omfs
openpromfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
overlayfs switch ->get_link() to delayed_call, kill ->put_link() 2015-12-30 13:01:03 -05:00
proc fs/proc/task_mmu.c: add workaround for old compilers 2016-01-20 17:09:18 -08:00
pstore pstore: fix code comment to match code 2015-11-02 13:41:52 -08:00
qnx4 kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
qnx6 kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
quota quota: constify qtree_fmt_operations structures 2016-01-04 10:58:35 +01:00
ramfs don't put symlink bodies in pagecache into highmem 2015-12-08 22:41:36 -05:00
reiserfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
romfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
squashfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
sysfs platform/chrome: Branch for v4.4 2015-11-13 21:53:18 -08:00
sysv kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
tracefs tracefs: Fix refcount imbalance in start_creating() 2015-11-04 22:13:45 -05:00
ubifs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
udf Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs 2016-01-15 11:51:51 -08:00
ufs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
xfs kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
aio.c mm: move ->mremap() from file_operations to vm_operations_struct 2015-09-04 16:54:41 -07:00
anon_inodes.c
attr.c
bad_inode.c fs/bad_inode.c: is_bad_inode can be boolean 2015-12-06 21:17:14 -05:00
binfmt_aout.c
binfmt_elf_fdpic.c libnvdimm for 4.4: 2015-11-10 12:07:22 -08:00
binfmt_elf.c Merge branch 'for-linus-2' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2015-11-11 09:45:24 -08:00
binfmt_em86.c
binfmt_flat.c
binfmt_misc.c
binfmt_script.c
block_dev.c Merge branch 'for-4.5/core' of git://git.kernel.dk/linux-block 2016-01-19 15:03:34 -08:00
buffer.c fs: use block_device name vsprintf helper 2016-01-06 13:03:18 -05:00
char_dev.c fs/char_dev.c: fix incorrect documentation for unregister_chrdev_region 2015-08-05 13:49:35 -07:00
compat_binfmt_elf.c
compat_ioctl.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-01-12 17:11:47 -08:00
compat.c saner calling conventions for copy_mount_options() 2016-01-04 10:28:32 -05:00
coredump.c coredump: Use 64bit time for unix time of coredump 2015-12-06 21:17:17 -05:00
dax.c dax: re-enable dax pmd mappings 2016-01-15 17:56:32 -08:00
dcache.c kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
dcookies.c
direct-io.c fix the regression from "direct-io: Fix negative return from dio read beyond eof" 2015-12-08 15:02:42 -05:00
drop_caches.c inode: convert inode_sb_list_lock to per-sb 2015-08-17 18:39:46 -04:00
eventfd.c Documentation: filesystem: Fix typo in fs/eventfd.c 2015-12-08 14:52:03 +01:00
eventpoll.c epoll: add EPOLLEXCLUSIVE flag 2016-01-20 17:09:18 -08:00
exec.c don't carry MAY_OPEN in op->acc_mode 2016-01-04 10:28:40 -05:00
fcntl.c fcntl: allow to set O_DIRECT flag on pipe 2016-01-09 02:55:37 -05:00
fhandle.c
file_table.c fs, file table: reinit files_stat.max_files after deferred memory initialisation 2015-08-07 04:39:40 +03:00
file.c kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
filesystems.c
fs_pin.c
fs_struct.c
fs-writeback.c cgroup, memcg, writeback: drop spurious rcu locking around mem_cgroup_css_from_page() 2016-01-15 17:56:32 -08:00
inode.c kmemcg: account certain kmem allocations to memcg 2016-01-14 16:00:49 -08:00
internal.h Merge branch 'for-linus' into work.misc 2016-01-08 21:20:11 -05:00
ioctl.c Merge branch 'work.copy_file_range' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-01-12 16:30:34 -08:00
Kconfig dax: re-enable dax pmd mappings 2016-01-15 17:56:32 -08:00
Kconfig.binfmt
libfs.c switch ->get_link() to delayed_call, kill ->put_link() 2015-12-30 13:01:03 -05:00
locks.c Merge branch 'work.copy_file_range' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-01-12 16:30:34 -08:00
Makefile ext4: promote ext4 over ext2 in the default probe order 2015-10-15 10:33:21 -04:00
mbcache.c
mount.h
mpage.c mm, fs: introduce mapping_gfp_constraint() 2015-11-06 17:50:42 -08:00
namei.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-01-12 17:11:47 -08:00
namespace.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-01-12 17:11:47 -08:00
no-block.c
nsfs.c fs/seq_file: convert int seq_vprint/seq_printf/etc... returns to void 2015-09-11 15:21:34 -07:00
open.c don't carry MAY_OPEN in op->acc_mode 2016-01-04 10:28:40 -05:00
pipe.c fs/pipe.c: return error code rather than 0 in pipe_write() 2015-11-11 02:18:26 -05:00
pnode.c
pnode.h mnt: Clarify and correct the disconnect logic in umount_tree 2015-07-22 20:33:27 -05:00
posix_acl.c xattr handlers: Simplify list operation 2015-12-13 19:46:12 -05:00
proc_namespace.c vfs: show_vfsstat: remove redundant initialization and check of error code 2015-12-06 21:17:16 -05:00
read_write.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-01-12 17:11:47 -08:00
readdir.c
select.c poll: plug an unused argument to do_poll 2016-01-06 08:26:52 -05:00
seq_file.c fs, seqfile: always allow oom killer 2015-11-06 17:50:42 -08:00
signalfd.c signalfd: fix information leak in signalfd_copyinfo 2015-08-07 04:39:40 +03:00
splice.c fs: __generic_file_splice_read retry lookup on AOP_TRUNCATED_PAGE 2016-01-09 02:55:35 -05:00
stack.c
stat.c fs/stat.c: drop the last new_valid_dev check 2016-01-16 11:17:23 -08:00
statfs.c
super.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2016-01-14 17:04:19 -08:00
sync.c fs/sync.c: make sync_file_range(2) use WB_SYNC_NONE writeback 2015-11-06 17:50:42 -08:00
timerfd.c
userfaultfd.c userfaultfd: revert "userfaultfd: waitqueue: add nr wake parameter to __wake_up_locked_key" 2015-09-22 15:09:53 -07:00
utimes.c
xattr.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2016-01-12 17:11:47 -08:00