License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 14:07:57 +00:00
|
|
|
// SPDX-License-Identifier: GPL-2.0
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
|
|
|
* linux/fs/pipe.c
|
|
|
|
*
|
|
|
|
* Copyright (C) 1991, 1992, 1999 Linus Torvalds
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <linux/mm.h>
|
|
|
|
#include <linux/file.h>
|
|
|
|
#include <linux/poll.h>
|
|
|
|
#include <linux/slab.h>
|
|
|
|
#include <linux/module.h>
|
|
|
|
#include <linux/init.h>
|
|
|
|
#include <linux/fs.h>
|
2010-05-20 08:43:18 +00:00
|
|
|
#include <linux/log2.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <linux/mount.h>
|
2019-03-25 16:38:23 +00:00
|
|
|
#include <linux/pseudo_fs.h>
|
2012-03-23 22:01:50 +00:00
|
|
|
#include <linux/magic.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <linux/pipe_fs_i.h>
|
|
|
|
#include <linux/uio.h>
|
|
|
|
#include <linux/highmem.h>
|
2006-03-30 13:15:30 +00:00
|
|
|
#include <linux/pagemap.h>
|
2007-02-07 06:48:00 +00:00
|
|
|
#include <linux/audit.h>
|
2008-05-07 03:42:38 +00:00
|
|
|
#include <linux/syscalls.h>
|
2010-05-19 19:03:16 +00:00
|
|
|
#include <linux/fcntl.h>
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
#include <linux/memcontrol.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2016-12-24 19:46:01 +00:00
|
|
|
#include <linux/uaccess.h>
|
2005-04-16 22:20:36 +00:00
|
|
|
#include <asm/ioctls.h>
|
|
|
|
|
2013-03-12 13:58:10 +00:00
|
|
|
#include "internal.h"
|
|
|
|
|
2010-05-19 19:03:16 +00:00
|
|
|
/*
|
|
|
|
* The max size that a non-root user is allowed to grow the pipe. Can
|
2010-06-03 12:54:39 +00:00
|
|
|
* be set by root in /proc/sys/fs/pipe-max-size
|
2010-05-19 19:03:16 +00:00
|
|
|
*/
|
2010-06-03 12:54:39 +00:00
|
|
|
unsigned int pipe_max_size = 1048576;
|
|
|
|
|
2016-01-18 15:36:09 +00:00
|
|
|
/* Maximum allocatable pages per user. Hard limit is unset by default, soft
|
|
|
|
* matches default values.
|
|
|
|
*/
|
|
|
|
unsigned long pipe_user_pages_hard;
|
|
|
|
unsigned long pipe_user_pages_soft = PIPE_DEF_BUFFERS * INR_OPEN_CUR;
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
2019-11-15 13:30:32 +00:00
|
|
|
* We use head and tail indices that aren't masked off, except at the point of
|
|
|
|
* dereference, but rather they're allowed to wrap naturally. This means there
|
|
|
|
* isn't a dead spot in the buffer, but the ring has to be a power of two and
|
|
|
|
* <= 2^31.
|
|
|
|
* -- David Howells 2019-09-23.
|
|
|
|
*
|
2005-04-16 22:20:36 +00:00
|
|
|
* Reads with count = 0 should always return 0.
|
|
|
|
* -- Julian Bradfield 1999-06-07.
|
|
|
|
*
|
|
|
|
* FIFOs and Pipes now generate SIGIO for both readers and writers.
|
|
|
|
* -- Jeremy Elson <jelson@circlemud.org> 2001-08-16
|
|
|
|
*
|
|
|
|
* pipe_read & write cleanup
|
|
|
|
* -- Manfred Spraul <manfred@colorfullife.com> 2002-05-09
|
|
|
|
*/
|
|
|
|
|
2009-04-14 17:48:41 +00:00
|
|
|
static void pipe_lock_nested(struct pipe_inode_info *pipe, int subclass)
|
|
|
|
{
|
2013-03-21 15:01:38 +00:00
|
|
|
if (pipe->files)
|
2013-03-21 06:32:24 +00:00
|
|
|
mutex_lock_nested(&pipe->mutex, subclass);
|
2009-04-14 17:48:41 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
void pipe_lock(struct pipe_inode_info *pipe)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* pipe_lock() nests non-pipe inode locks (for writing to a file)
|
|
|
|
*/
|
|
|
|
pipe_lock_nested(pipe, I_MUTEX_PARENT);
|
|
|
|
}
|
|
|
|
EXPORT_SYMBOL(pipe_lock);
|
|
|
|
|
|
|
|
void pipe_unlock(struct pipe_inode_info *pipe)
|
|
|
|
{
|
2013-03-21 15:01:38 +00:00
|
|
|
if (pipe->files)
|
2013-03-21 06:32:24 +00:00
|
|
|
mutex_unlock(&pipe->mutex);
|
2009-04-14 17:48:41 +00:00
|
|
|
}
|
|
|
|
EXPORT_SYMBOL(pipe_unlock);
|
|
|
|
|
2013-03-21 16:24:01 +00:00
|
|
|
static inline void __pipe_lock(struct pipe_inode_info *pipe)
|
|
|
|
{
|
|
|
|
mutex_lock_nested(&pipe->mutex, I_MUTEX_PARENT);
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void __pipe_unlock(struct pipe_inode_info *pipe)
|
|
|
|
{
|
|
|
|
mutex_unlock(&pipe->mutex);
|
|
|
|
}
|
|
|
|
|
2009-04-14 17:48:41 +00:00
|
|
|
void pipe_double_lock(struct pipe_inode_info *pipe1,
|
|
|
|
struct pipe_inode_info *pipe2)
|
|
|
|
{
|
|
|
|
BUG_ON(pipe1 == pipe2);
|
|
|
|
|
|
|
|
if (pipe1 < pipe2) {
|
|
|
|
pipe_lock_nested(pipe1, I_MUTEX_PARENT);
|
|
|
|
pipe_lock_nested(pipe2, I_MUTEX_CHILD);
|
|
|
|
} else {
|
2009-07-21 08:09:23 +00:00
|
|
|
pipe_lock_nested(pipe2, I_MUTEX_PARENT);
|
|
|
|
pipe_lock_nested(pipe1, I_MUTEX_CHILD);
|
2009-04-14 17:48:41 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/* Drop the inode semaphore and wait for a pipe event, atomically */
|
2006-04-10 13:18:35 +00:00
|
|
|
void pipe_wait(struct pipe_inode_info *pipe)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
|
|
|
DEFINE_WAIT(wait);
|
|
|
|
|
2005-09-10 07:26:12 +00:00
|
|
|
/*
|
|
|
|
* Pipes are system-local resources, so sleeping on them
|
|
|
|
* is considered a noninteractive wait:
|
|
|
|
*/
|
2007-10-15 15:00:13 +00:00
|
|
|
prepare_to_wait(&pipe->wait, &wait, TASK_INTERRUPTIBLE);
|
2009-04-14 17:48:41 +00:00
|
|
|
pipe_unlock(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
schedule();
|
2006-04-10 13:18:35 +00:00
|
|
|
finish_wait(&pipe->wait, &wait);
|
2009-04-14 17:48:41 +00:00
|
|
|
pipe_lock(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2006-04-11 11:57:45 +00:00
|
|
|
static void anon_pipe_buf_release(struct pipe_inode_info *pipe,
|
|
|
|
struct pipe_buffer *buf)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
|
|
|
struct page *page = buf->page;
|
|
|
|
|
2006-03-30 13:15:30 +00:00
|
|
|
/*
|
|
|
|
* If nobody else uses this page, and we don't already have a
|
|
|
|
* temporary page, let's keep track of it as a one-deep
|
2006-04-11 11:57:45 +00:00
|
|
|
* allocation cache. (Otherwise just release our reference to it)
|
2006-03-30 13:15:30 +00:00
|
|
|
*/
|
2006-04-11 11:57:45 +00:00
|
|
|
if (page_count(page) == 1 && !pipe->tmp_page)
|
2006-04-11 11:53:33 +00:00
|
|
|
pipe->tmp_page = page;
|
2006-04-11 11:57:45 +00:00
|
|
|
else
|
mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros
PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.
This promise never materialized. And unlikely will.
We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.
Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.
Let's stop pretending that pages in page cache are special. They are
not.
The changes are pretty straight-forward:
- <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
- page_cache_get() -> get_page();
- page_cache_release() -> put_page();
This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.
The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.
There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.
virtual patch
@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT
@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE
@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK
@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)
@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)
@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-04-01 12:29:47 +00:00
|
|
|
put_page(page);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
static int anon_pipe_buf_steal(struct pipe_inode_info *pipe,
|
|
|
|
struct pipe_buffer *buf)
|
|
|
|
{
|
|
|
|
struct page *page = buf->page;
|
|
|
|
|
|
|
|
if (page_count(page) == 1) {
|
2019-03-05 23:43:13 +00:00
|
|
|
memcg_kmem_uncharge(page, 0);
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
__SetPageLocked(page);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
2007-06-12 18:51:32 +00:00
|
|
|
/**
|
2008-02-13 23:03:22 +00:00
|
|
|
* generic_pipe_buf_steal - attempt to take ownership of a &pipe_buffer
|
2007-06-12 18:51:32 +00:00
|
|
|
* @pipe: the pipe that the buffer belongs to
|
|
|
|
* @buf: the buffer to attempt to steal
|
|
|
|
*
|
|
|
|
* Description:
|
2008-02-13 23:03:22 +00:00
|
|
|
* This function attempts to steal the &struct page attached to
|
2007-06-12 18:51:32 +00:00
|
|
|
* @buf. If successful, this function returns 0 and returns with
|
|
|
|
* the page locked. The caller may then reuse the page for whatever
|
2008-02-13 23:03:22 +00:00
|
|
|
* he wishes; the typical use is insertion into a different file
|
2007-06-12 18:51:32 +00:00
|
|
|
* page cache.
|
|
|
|
*/
|
2006-05-02 13:29:57 +00:00
|
|
|
int generic_pipe_buf_steal(struct pipe_inode_info *pipe,
|
|
|
|
struct pipe_buffer *buf)
|
2006-03-30 13:16:46 +00:00
|
|
|
{
|
2006-04-30 14:36:32 +00:00
|
|
|
struct page *page = buf->page;
|
|
|
|
|
2007-06-12 18:51:32 +00:00
|
|
|
/*
|
|
|
|
* A reference of one is golden, that means that the owner of this
|
|
|
|
* page is the only one holding a reference to it. lock the page
|
|
|
|
* and return OK.
|
|
|
|
*/
|
2006-04-30 14:36:32 +00:00
|
|
|
if (page_count(page) == 1) {
|
|
|
|
lock_page(page);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
return 1;
|
2006-03-30 13:16:46 +00:00
|
|
|
}
|
2010-05-26 06:44:22 +00:00
|
|
|
EXPORT_SYMBOL(generic_pipe_buf_steal);
|
2006-03-30 13:16:46 +00:00
|
|
|
|
2007-06-12 18:51:32 +00:00
|
|
|
/**
|
2008-02-13 23:03:22 +00:00
|
|
|
* generic_pipe_buf_get - get a reference to a &struct pipe_buffer
|
2007-06-12 18:51:32 +00:00
|
|
|
* @pipe: the pipe that the buffer belongs to
|
|
|
|
* @buf: the buffer to get a reference to
|
|
|
|
*
|
|
|
|
* Description:
|
|
|
|
* This function grabs an extra reference to @buf. It's used in
|
|
|
|
* in the tee() system call, when we duplicate the buffers in one
|
|
|
|
* pipe into another.
|
|
|
|
*/
|
2019-04-05 21:02:10 +00:00
|
|
|
bool generic_pipe_buf_get(struct pipe_inode_info *pipe, struct pipe_buffer *buf)
|
2006-04-11 13:51:17 +00:00
|
|
|
{
|
2019-04-05 21:02:10 +00:00
|
|
|
return try_get_page(buf->page);
|
2006-04-11 13:51:17 +00:00
|
|
|
}
|
2010-05-26 06:44:22 +00:00
|
|
|
EXPORT_SYMBOL(generic_pipe_buf_get);
|
2006-04-11 13:51:17 +00:00
|
|
|
|
2007-06-12 18:51:32 +00:00
|
|
|
/**
|
|
|
|
* generic_pipe_buf_confirm - verify contents of the pipe buffer
|
2007-07-27 06:08:51 +00:00
|
|
|
* @info: the pipe that the buffer belongs to
|
2007-06-12 18:51:32 +00:00
|
|
|
* @buf: the buffer to confirm
|
|
|
|
*
|
|
|
|
* Description:
|
|
|
|
* This function does nothing, because the generic pipe code uses
|
|
|
|
* pages that are always good when inserted into the pipe.
|
|
|
|
*/
|
2007-06-14 11:10:48 +00:00
|
|
|
int generic_pipe_buf_confirm(struct pipe_inode_info *info,
|
|
|
|
struct pipe_buffer *buf)
|
2006-05-01 17:59:03 +00:00
|
|
|
{
|
|
|
|
return 0;
|
|
|
|
}
|
2010-05-26 06:44:22 +00:00
|
|
|
EXPORT_SYMBOL(generic_pipe_buf_confirm);
|
2006-05-01 17:59:03 +00:00
|
|
|
|
2009-05-07 13:37:36 +00:00
|
|
|
/**
|
|
|
|
* generic_pipe_buf_release - put a reference to a &struct pipe_buffer
|
|
|
|
* @pipe: the pipe that the buffer belongs to
|
|
|
|
* @buf: the buffer to put a reference to
|
|
|
|
*
|
|
|
|
* Description:
|
|
|
|
* This function releases a reference to @buf.
|
|
|
|
*/
|
|
|
|
void generic_pipe_buf_release(struct pipe_inode_info *pipe,
|
|
|
|
struct pipe_buffer *buf)
|
|
|
|
{
|
mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros
PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.
This promise never materialized. And unlikely will.
We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.
Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.
Let's stop pretending that pages in page cache are special. They are
not.
The changes are pretty straight-forward:
- <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
- page_cache_get() -> get_page();
- page_cache_release() -> put_page();
This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.
The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.
There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.
virtual patch
@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT
@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE
@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK
@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)
@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)
@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-04-01 12:29:47 +00:00
|
|
|
put_page(buf->page);
|
2009-05-07 13:37:36 +00:00
|
|
|
}
|
2010-05-26 06:44:22 +00:00
|
|
|
EXPORT_SYMBOL(generic_pipe_buf_release);
|
2009-05-07 13:37:36 +00:00
|
|
|
|
2019-01-23 14:19:18 +00:00
|
|
|
/* New data written to a pipe may be appended to a buffer with this type. */
|
2006-12-13 08:34:04 +00:00
|
|
|
static const struct pipe_buf_operations anon_pipe_buf_ops = {
|
2007-06-14 11:10:48 +00:00
|
|
|
.confirm = generic_pipe_buf_confirm,
|
2005-04-16 22:20:36 +00:00
|
|
|
.release = anon_pipe_buf_release,
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
.steal = anon_pipe_buf_steal,
|
2006-05-01 17:59:03 +00:00
|
|
|
.get = generic_pipe_buf_get,
|
2005-04-16 22:20:36 +00:00
|
|
|
};
|
|
|
|
|
splice: don't merge into linked buffers
Before this patch, it was possible for two pipes to affect each other after
data had been transferred between them with tee():
============
$ cat tee_test.c
int main(void) {
int pipe_a[2];
if (pipe(pipe_a)) err(1, "pipe");
int pipe_b[2];
if (pipe(pipe_b)) err(1, "pipe");
if (write(pipe_a[1], "abcd", 4) != 4) err(1, "write");
if (tee(pipe_a[0], pipe_b[1], 2, 0) != 2) err(1, "tee");
if (write(pipe_b[1], "xx", 2) != 2) err(1, "write");
char buf[5];
if (read(pipe_a[0], buf, 4) != 4) err(1, "read");
buf[4] = 0;
printf("got back: '%s'\n", buf);
}
$ gcc -o tee_test tee_test.c
$ ./tee_test
got back: 'abxx'
$
============
As suggested by Al Viro, fix it by creating a separate type for
non-mergeable pipe buffers, then changing the types of buffers in
splice_pipe_to_pipe() and link_pipe().
Cc: <stable@vger.kernel.org>
Fixes: 7c77f0b3f920 ("splice: implement pipe to pipe splicing")
Fixes: 70524490ee2e ("[PATCH] splice: add support for sys_tee()")
Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2019-01-23 14:19:17 +00:00
|
|
|
static const struct pipe_buf_operations anon_pipe_buf_nomerge_ops = {
|
2007-06-14 11:10:48 +00:00
|
|
|
.confirm = generic_pipe_buf_confirm,
|
2005-04-16 22:20:36 +00:00
|
|
|
.release = anon_pipe_buf_release,
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
.steal = anon_pipe_buf_steal,
|
2006-05-01 17:59:03 +00:00
|
|
|
.get = generic_pipe_buf_get,
|
2005-04-16 22:20:36 +00:00
|
|
|
};
|
|
|
|
|
pipes: add a "packetized pipe" mode for writing
The actual internal pipe implementation is already really about
individual packets (called "pipe buffers"), and this simply exposes that
as a special packetized mode.
When we are in the packetized mode (marked by O_DIRECT as suggested by
Alan Cox), a write() on a pipe will not merge the new data with previous
writes, so each write will get a pipe buffer of its own. The pipe
buffer is then marked with the PIPE_BUF_FLAG_PACKET flag, which in turn
will tell the reader side to break the read at that boundary (and throw
away any partial packet contents that do not fit in the read buffer).
End result: as long as you do writes less than PIPE_BUF in size (so that
the pipe doesn't have to split them up), you can now treat the pipe as a
packet interface, where each read() system call will read one packet at
a time. You can just use a sufficiently big read buffer (PIPE_BUF is
sufficient, since bigger than that doesn't guarantee atomicity anyway),
and the return value of the read() will naturally give you the size of
the packet.
NOTE! We do not support zero-sized packets, and zero-sized reads and
writes to a pipe continue to be no-ops. Also note that big packets will
currently be split at write time, but that the size at which that
happens is not really specified (except that it's bigger than PIPE_BUF).
Currently that limit is the system page size, but we might want to
explicitly support bigger packets some day.
The main user for this is going to be the autofs packet interface,
allowing us to stop having to care so deeply about exact packet sizes
(which have had bugs with 32/64-bit compatibility modes). But user
space can create packetized pipes with "pipe2(fd, O_DIRECT)", which will
fail with an EINVAL on kernels that do not support this interface.
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: David Miller <davem@davemloft.net>
Cc: Ian Kent <raven@themaw.net>
Cc: Thomas Meyer <thomas@m3y3r.de>
Cc: stable@kernel.org # needed for systemd/autofs interaction fix
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-04-29 20:12:42 +00:00
|
|
|
static const struct pipe_buf_operations packet_pipe_buf_ops = {
|
|
|
|
.confirm = generic_pipe_buf_confirm,
|
|
|
|
.release = anon_pipe_buf_release,
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
.steal = anon_pipe_buf_steal,
|
pipes: add a "packetized pipe" mode for writing
The actual internal pipe implementation is already really about
individual packets (called "pipe buffers"), and this simply exposes that
as a special packetized mode.
When we are in the packetized mode (marked by O_DIRECT as suggested by
Alan Cox), a write() on a pipe will not merge the new data with previous
writes, so each write will get a pipe buffer of its own. The pipe
buffer is then marked with the PIPE_BUF_FLAG_PACKET flag, which in turn
will tell the reader side to break the read at that boundary (and throw
away any partial packet contents that do not fit in the read buffer).
End result: as long as you do writes less than PIPE_BUF in size (so that
the pipe doesn't have to split them up), you can now treat the pipe as a
packet interface, where each read() system call will read one packet at
a time. You can just use a sufficiently big read buffer (PIPE_BUF is
sufficient, since bigger than that doesn't guarantee atomicity anyway),
and the return value of the read() will naturally give you the size of
the packet.
NOTE! We do not support zero-sized packets, and zero-sized reads and
writes to a pipe continue to be no-ops. Also note that big packets will
currently be split at write time, but that the size at which that
happens is not really specified (except that it's bigger than PIPE_BUF).
Currently that limit is the system page size, but we might want to
explicitly support bigger packets some day.
The main user for this is going to be the autofs packet interface,
allowing us to stop having to care so deeply about exact packet sizes
(which have had bugs with 32/64-bit compatibility modes). But user
space can create packetized pipes with "pipe2(fd, O_DIRECT)", which will
fail with an EINVAL on kernels that do not support this interface.
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: David Miller <davem@davemloft.net>
Cc: Ian Kent <raven@themaw.net>
Cc: Thomas Meyer <thomas@m3y3r.de>
Cc: stable@kernel.org # needed for systemd/autofs interaction fix
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-04-29 20:12:42 +00:00
|
|
|
.get = generic_pipe_buf_get,
|
|
|
|
};
|
|
|
|
|
2019-01-23 14:19:18 +00:00
|
|
|
/**
|
|
|
|
* pipe_buf_mark_unmergeable - mark a &struct pipe_buffer as unmergeable
|
|
|
|
* @buf: the buffer to mark
|
|
|
|
*
|
|
|
|
* Description:
|
|
|
|
* This function ensures that no future writes will be merged into the
|
|
|
|
* given &struct pipe_buffer. This is necessary when multiple pipe buffers
|
|
|
|
* share the same backing page.
|
|
|
|
*/
|
splice: don't merge into linked buffers
Before this patch, it was possible for two pipes to affect each other after
data had been transferred between them with tee():
============
$ cat tee_test.c
int main(void) {
int pipe_a[2];
if (pipe(pipe_a)) err(1, "pipe");
int pipe_b[2];
if (pipe(pipe_b)) err(1, "pipe");
if (write(pipe_a[1], "abcd", 4) != 4) err(1, "write");
if (tee(pipe_a[0], pipe_b[1], 2, 0) != 2) err(1, "tee");
if (write(pipe_b[1], "xx", 2) != 2) err(1, "write");
char buf[5];
if (read(pipe_a[0], buf, 4) != 4) err(1, "read");
buf[4] = 0;
printf("got back: '%s'\n", buf);
}
$ gcc -o tee_test tee_test.c
$ ./tee_test
got back: 'abxx'
$
============
As suggested by Al Viro, fix it by creating a separate type for
non-mergeable pipe buffers, then changing the types of buffers in
splice_pipe_to_pipe() and link_pipe().
Cc: <stable@vger.kernel.org>
Fixes: 7c77f0b3f920 ("splice: implement pipe to pipe splicing")
Fixes: 70524490ee2e ("[PATCH] splice: add support for sys_tee()")
Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2019-01-23 14:19:17 +00:00
|
|
|
void pipe_buf_mark_unmergeable(struct pipe_buffer *buf)
|
|
|
|
{
|
|
|
|
if (buf->ops == &anon_pipe_buf_ops)
|
|
|
|
buf->ops = &anon_pipe_buf_nomerge_ops;
|
|
|
|
}
|
|
|
|
|
2019-01-23 14:19:18 +00:00
|
|
|
static bool pipe_buf_can_merge(struct pipe_buffer *buf)
|
|
|
|
{
|
|
|
|
return buf->ops == &anon_pipe_buf_ops;
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
static ssize_t
|
2014-04-02 23:56:54 +00:00
|
|
|
pipe_read(struct kiocb *iocb, struct iov_iter *to)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2014-04-02 23:56:54 +00:00
|
|
|
size_t total_len = iov_iter_count(to);
|
2006-10-01 06:28:47 +00:00
|
|
|
struct file *filp = iocb->ki_filp;
|
2013-03-21 15:16:56 +00:00
|
|
|
struct pipe_inode_info *pipe = filp->private_data;
|
2005-04-16 22:20:36 +00:00
|
|
|
int do_wakeup;
|
|
|
|
ssize_t ret;
|
|
|
|
|
|
|
|
/* Null read succeeds. */
|
|
|
|
if (unlikely(total_len == 0))
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
do_wakeup = 0;
|
|
|
|
ret = 0;
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_lock(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
for (;;) {
|
2019-11-15 13:30:32 +00:00
|
|
|
unsigned int head = pipe->head;
|
|
|
|
unsigned int tail = pipe->tail;
|
|
|
|
unsigned int mask = pipe->ring_size - 1;
|
|
|
|
|
|
|
|
if (!pipe_empty(head, tail)) {
|
|
|
|
struct pipe_buffer *buf = &pipe->bufs[tail & mask];
|
2005-04-16 22:20:36 +00:00
|
|
|
size_t chars = buf->len;
|
2014-02-04 00:11:42 +00:00
|
|
|
size_t written;
|
|
|
|
int error;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
if (chars > total_len)
|
|
|
|
chars = total_len;
|
|
|
|
|
2016-09-27 08:45:12 +00:00
|
|
|
error = pipe_buf_confirm(pipe, buf);
|
2006-05-01 17:59:03 +00:00
|
|
|
if (error) {
|
2006-03-30 13:15:30 +00:00
|
|
|
if (!ret)
|
2010-10-21 12:56:00 +00:00
|
|
|
ret = error;
|
2006-03-30 13:15:30 +00:00
|
|
|
break;
|
|
|
|
}
|
2006-05-01 17:59:03 +00:00
|
|
|
|
2014-04-02 23:56:54 +00:00
|
|
|
written = copy_page_to_iter(buf->page, buf->offset, chars, to);
|
2014-02-04 00:11:42 +00:00
|
|
|
if (unlikely(written < chars)) {
|
2006-04-11 11:57:45 +00:00
|
|
|
if (!ret)
|
2014-02-04 00:11:42 +00:00
|
|
|
ret = -EFAULT;
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
ret += chars;
|
|
|
|
buf->offset += chars;
|
|
|
|
buf->len -= chars;
|
pipes: add a "packetized pipe" mode for writing
The actual internal pipe implementation is already really about
individual packets (called "pipe buffers"), and this simply exposes that
as a special packetized mode.
When we are in the packetized mode (marked by O_DIRECT as suggested by
Alan Cox), a write() on a pipe will not merge the new data with previous
writes, so each write will get a pipe buffer of its own. The pipe
buffer is then marked with the PIPE_BUF_FLAG_PACKET flag, which in turn
will tell the reader side to break the read at that boundary (and throw
away any partial packet contents that do not fit in the read buffer).
End result: as long as you do writes less than PIPE_BUF in size (so that
the pipe doesn't have to split them up), you can now treat the pipe as a
packet interface, where each read() system call will read one packet at
a time. You can just use a sufficiently big read buffer (PIPE_BUF is
sufficient, since bigger than that doesn't guarantee atomicity anyway),
and the return value of the read() will naturally give you the size of
the packet.
NOTE! We do not support zero-sized packets, and zero-sized reads and
writes to a pipe continue to be no-ops. Also note that big packets will
currently be split at write time, but that the size at which that
happens is not really specified (except that it's bigger than PIPE_BUF).
Currently that limit is the system page size, but we might want to
explicitly support bigger packets some day.
The main user for this is going to be the autofs packet interface,
allowing us to stop having to care so deeply about exact packet sizes
(which have had bugs with 32/64-bit compatibility modes). But user
space can create packetized pipes with "pipe2(fd, O_DIRECT)", which will
fail with an EINVAL on kernels that do not support this interface.
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: David Miller <davem@davemloft.net>
Cc: Ian Kent <raven@themaw.net>
Cc: Thomas Meyer <thomas@m3y3r.de>
Cc: stable@kernel.org # needed for systemd/autofs interaction fix
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-04-29 20:12:42 +00:00
|
|
|
|
|
|
|
/* Was it a packet buffer? Clean up and exit */
|
|
|
|
if (buf->flags & PIPE_BUF_FLAG_PACKET) {
|
|
|
|
total_len = chars;
|
|
|
|
buf->len = 0;
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
if (!buf->len) {
|
2019-10-31 15:59:24 +00:00
|
|
|
bool wake;
|
2016-09-27 08:45:12 +00:00
|
|
|
pipe_buf_release(pipe, buf);
|
2019-09-24 15:09:04 +00:00
|
|
|
spin_lock_irq(&pipe->wait.lock);
|
2019-11-15 13:30:32 +00:00
|
|
|
tail++;
|
|
|
|
pipe->tail = tail;
|
2019-09-24 15:07:04 +00:00
|
|
|
do_wakeup = 1;
|
2019-10-31 15:59:24 +00:00
|
|
|
wake = head - (tail - 1) == pipe->max_usage / 2;
|
|
|
|
if (wake)
|
2019-11-01 20:52:24 +00:00
|
|
|
wake_up_locked_poll(
|
2019-09-24 15:07:04 +00:00
|
|
|
&pipe->wait, EPOLLOUT | EPOLLWRNORM);
|
2019-09-24 15:09:04 +00:00
|
|
|
spin_unlock_irq(&pipe->wait.lock);
|
2019-10-31 15:59:24 +00:00
|
|
|
if (wake)
|
2019-09-24 15:07:04 +00:00
|
|
|
kill_fasync(&pipe->fasync_writers, SIGIO, POLL_OUT);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
total_len -= chars;
|
|
|
|
if (!total_len)
|
|
|
|
break; /* common path: read succeeded */
|
2019-11-15 13:30:32 +00:00
|
|
|
if (!pipe_empty(head, tail)) /* More to do? */
|
|
|
|
continue;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2019-11-15 13:30:32 +00:00
|
|
|
|
2006-04-11 11:53:33 +00:00
|
|
|
if (!pipe->writers)
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
2006-04-11 11:53:33 +00:00
|
|
|
if (!pipe->waiting_writers) {
|
2005-04-16 22:20:36 +00:00
|
|
|
/* syscall merging: Usually we must not sleep
|
|
|
|
* if O_NONBLOCK is set, or if we got some data.
|
|
|
|
* But if a writer sleeps in kernel space, then
|
|
|
|
* we can wait for that data without violating POSIX.
|
|
|
|
*/
|
|
|
|
if (ret)
|
|
|
|
break;
|
|
|
|
if (filp->f_flags & O_NONBLOCK) {
|
|
|
|
ret = -EAGAIN;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if (signal_pending(current)) {
|
2006-04-11 11:57:45 +00:00
|
|
|
if (!ret)
|
|
|
|
ret = -ERESTARTSYS;
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
|
|
|
}
|
2006-04-11 11:53:33 +00:00
|
|
|
pipe_wait(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
2006-04-11 11:57:45 +00:00
|
|
|
|
|
|
|
/* Signal writers asynchronously that there is more room. */
|
2005-04-16 22:20:36 +00:00
|
|
|
if (do_wakeup) {
|
2019-11-01 20:52:24 +00:00
|
|
|
wake_up_interruptible_poll(&pipe->wait, EPOLLOUT | EPOLLWRNORM);
|
2006-04-11 11:53:33 +00:00
|
|
|
kill_fasync(&pipe->fasync_writers, SIGIO, POLL_OUT);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
if (ret > 0)
|
|
|
|
file_accessed(filp);
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
pipes: add a "packetized pipe" mode for writing
The actual internal pipe implementation is already really about
individual packets (called "pipe buffers"), and this simply exposes that
as a special packetized mode.
When we are in the packetized mode (marked by O_DIRECT as suggested by
Alan Cox), a write() on a pipe will not merge the new data with previous
writes, so each write will get a pipe buffer of its own. The pipe
buffer is then marked with the PIPE_BUF_FLAG_PACKET flag, which in turn
will tell the reader side to break the read at that boundary (and throw
away any partial packet contents that do not fit in the read buffer).
End result: as long as you do writes less than PIPE_BUF in size (so that
the pipe doesn't have to split them up), you can now treat the pipe as a
packet interface, where each read() system call will read one packet at
a time. You can just use a sufficiently big read buffer (PIPE_BUF is
sufficient, since bigger than that doesn't guarantee atomicity anyway),
and the return value of the read() will naturally give you the size of
the packet.
NOTE! We do not support zero-sized packets, and zero-sized reads and
writes to a pipe continue to be no-ops. Also note that big packets will
currently be split at write time, but that the size at which that
happens is not really specified (except that it's bigger than PIPE_BUF).
Currently that limit is the system page size, but we might want to
explicitly support bigger packets some day.
The main user for this is going to be the autofs packet interface,
allowing us to stop having to care so deeply about exact packet sizes
(which have had bugs with 32/64-bit compatibility modes). But user
space can create packetized pipes with "pipe2(fd, O_DIRECT)", which will
fail with an EINVAL on kernels that do not support this interface.
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: David Miller <davem@davemloft.net>
Cc: Ian Kent <raven@themaw.net>
Cc: Thomas Meyer <thomas@m3y3r.de>
Cc: stable@kernel.org # needed for systemd/autofs interaction fix
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-04-29 20:12:42 +00:00
|
|
|
static inline int is_packetized(struct file *file)
|
|
|
|
{
|
|
|
|
return (file->f_flags & O_DIRECT) != 0;
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
static ssize_t
|
2014-04-03 19:05:18 +00:00
|
|
|
pipe_write(struct kiocb *iocb, struct iov_iter *from)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2006-10-01 06:28:47 +00:00
|
|
|
struct file *filp = iocb->ki_filp;
|
2013-03-21 15:16:56 +00:00
|
|
|
struct pipe_inode_info *pipe = filp->private_data;
|
2019-09-20 15:32:19 +00:00
|
|
|
unsigned int head, max_usage, mask;
|
2014-04-03 19:05:18 +00:00
|
|
|
ssize_t ret = 0;
|
|
|
|
int do_wakeup = 0;
|
|
|
|
size_t total_len = iov_iter_count(from);
|
2005-04-16 22:20:36 +00:00
|
|
|
ssize_t chars;
|
|
|
|
|
|
|
|
/* Null write succeeds. */
|
|
|
|
if (unlikely(total_len == 0))
|
|
|
|
return 0;
|
|
|
|
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_lock(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2006-04-11 11:53:33 +00:00
|
|
|
if (!pipe->readers) {
|
2005-04-16 22:20:36 +00:00
|
|
|
send_sig(SIGPIPE, current, 0);
|
|
|
|
ret = -EPIPE;
|
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
head = pipe->head;
|
2019-10-16 15:47:32 +00:00
|
|
|
max_usage = pipe->max_usage;
|
2019-11-15 13:30:32 +00:00
|
|
|
mask = pipe->ring_size - 1;
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/* We try to merge small writes */
|
|
|
|
chars = total_len & (PAGE_SIZE-1); /* size of the last buffer */
|
2019-09-20 15:32:19 +00:00
|
|
|
if (!pipe_empty(head, pipe->tail) && chars != 0) {
|
2019-11-15 13:30:32 +00:00
|
|
|
struct pipe_buffer *buf = &pipe->bufs[(head - 1) & mask];
|
2005-04-16 22:20:36 +00:00
|
|
|
int offset = buf->offset + buf->len;
|
2006-04-11 11:57:45 +00:00
|
|
|
|
2019-01-23 14:19:18 +00:00
|
|
|
if (pipe_buf_can_merge(buf) && offset + chars <= PAGE_SIZE) {
|
2016-09-27 08:45:12 +00:00
|
|
|
ret = pipe_buf_confirm(pipe, buf);
|
2015-10-17 21:26:09 +00:00
|
|
|
if (ret)
|
2006-03-30 13:15:30 +00:00
|
|
|
goto out;
|
2006-05-01 17:59:03 +00:00
|
|
|
|
2014-04-03 19:05:18 +00:00
|
|
|
ret = copy_page_from_iter(buf->page, offset, chars, from);
|
|
|
|
if (unlikely(ret < chars)) {
|
2015-10-17 21:26:09 +00:00
|
|
|
ret = -EFAULT;
|
2005-04-16 22:20:36 +00:00
|
|
|
goto out;
|
2006-05-01 18:02:05 +00:00
|
|
|
}
|
2014-04-03 19:05:18 +00:00
|
|
|
do_wakeup = 1;
|
2015-10-17 21:26:09 +00:00
|
|
|
buf->len += ret;
|
2014-04-03 19:05:18 +00:00
|
|
|
if (!iov_iter_count(from))
|
2005-04-16 22:20:36 +00:00
|
|
|
goto out;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
for (;;) {
|
2006-04-11 11:53:33 +00:00
|
|
|
if (!pipe->readers) {
|
2005-04-16 22:20:36 +00:00
|
|
|
send_sig(SIGPIPE, current, 0);
|
2006-04-11 11:57:45 +00:00
|
|
|
if (!ret)
|
|
|
|
ret = -EPIPE;
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
|
|
|
}
|
2019-11-15 13:30:32 +00:00
|
|
|
|
2019-09-20 15:32:19 +00:00
|
|
|
head = pipe->head;
|
|
|
|
if (!pipe_full(head, pipe->tail, max_usage)) {
|
2019-11-15 13:30:32 +00:00
|
|
|
struct pipe_buffer *buf = &pipe->bufs[head & mask];
|
2006-04-11 11:53:33 +00:00
|
|
|
struct page *page = pipe->tmp_page;
|
2014-04-03 19:05:18 +00:00
|
|
|
int copied;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
if (!page) {
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
page = alloc_page(GFP_HIGHUSER | __GFP_ACCOUNT);
|
2005-04-16 22:20:36 +00:00
|
|
|
if (unlikely(!page)) {
|
|
|
|
ret = ret ? : -ENOMEM;
|
|
|
|
break;
|
|
|
|
}
|
2006-04-11 11:53:33 +00:00
|
|
|
pipe->tmp_page = page;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2019-09-20 15:32:19 +00:00
|
|
|
|
|
|
|
/* Allocate a slot in the ring in advance and attach an
|
|
|
|
* empty buffer. If we fault or otherwise fail to use
|
|
|
|
* it, either the reader will consume it or it'll still
|
|
|
|
* be there for the next write.
|
|
|
|
*/
|
|
|
|
spin_lock_irq(&pipe->wait.lock);
|
|
|
|
|
|
|
|
head = pipe->head;
|
2019-10-07 15:30:51 +00:00
|
|
|
if (pipe_full(head, pipe->tail, max_usage)) {
|
|
|
|
spin_unlock_irq(&pipe->wait.lock);
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
2019-09-20 15:32:19 +00:00
|
|
|
pipe->head = head + 1;
|
|
|
|
|
2006-04-11 11:57:45 +00:00
|
|
|
/* Always wake up, even if the copy fails. Otherwise
|
2005-04-16 22:20:36 +00:00
|
|
|
* we lock up (O_NONBLOCK-)readers that sleep due to
|
|
|
|
* syscall merging.
|
|
|
|
* FIXME! Is this really true?
|
|
|
|
*/
|
2019-11-01 20:52:24 +00:00
|
|
|
wake_up_locked_poll(
|
2019-09-20 15:32:19 +00:00
|
|
|
&pipe->wait, EPOLLIN | EPOLLRDNORM);
|
|
|
|
|
|
|
|
spin_unlock_irq(&pipe->wait.lock);
|
|
|
|
kill_fasync(&pipe->fasync_readers, SIGIO, POLL_IN);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
/* Insert it into the buffer array */
|
2019-09-20 15:32:19 +00:00
|
|
|
buf = &pipe->bufs[head & mask];
|
2005-04-16 22:20:36 +00:00
|
|
|
buf->page = page;
|
|
|
|
buf->ops = &anon_pipe_buf_ops;
|
|
|
|
buf->offset = 0;
|
2019-09-20 15:32:19 +00:00
|
|
|
buf->len = 0;
|
pipes: add a "packetized pipe" mode for writing
The actual internal pipe implementation is already really about
individual packets (called "pipe buffers"), and this simply exposes that
as a special packetized mode.
When we are in the packetized mode (marked by O_DIRECT as suggested by
Alan Cox), a write() on a pipe will not merge the new data with previous
writes, so each write will get a pipe buffer of its own. The pipe
buffer is then marked with the PIPE_BUF_FLAG_PACKET flag, which in turn
will tell the reader side to break the read at that boundary (and throw
away any partial packet contents that do not fit in the read buffer).
End result: as long as you do writes less than PIPE_BUF in size (so that
the pipe doesn't have to split them up), you can now treat the pipe as a
packet interface, where each read() system call will read one packet at
a time. You can just use a sufficiently big read buffer (PIPE_BUF is
sufficient, since bigger than that doesn't guarantee atomicity anyway),
and the return value of the read() will naturally give you the size of
the packet.
NOTE! We do not support zero-sized packets, and zero-sized reads and
writes to a pipe continue to be no-ops. Also note that big packets will
currently be split at write time, but that the size at which that
happens is not really specified (except that it's bigger than PIPE_BUF).
Currently that limit is the system page size, but we might want to
explicitly support bigger packets some day.
The main user for this is going to be the autofs packet interface,
allowing us to stop having to care so deeply about exact packet sizes
(which have had bugs with 32/64-bit compatibility modes). But user
space can create packetized pipes with "pipe2(fd, O_DIRECT)", which will
fail with an EINVAL on kernels that do not support this interface.
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: David Miller <davem@davemloft.net>
Cc: Ian Kent <raven@themaw.net>
Cc: Thomas Meyer <thomas@m3y3r.de>
Cc: stable@kernel.org # needed for systemd/autofs interaction fix
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-04-29 20:12:42 +00:00
|
|
|
buf->flags = 0;
|
|
|
|
if (is_packetized(filp)) {
|
|
|
|
buf->ops = &packet_pipe_buf_ops;
|
|
|
|
buf->flags = PIPE_BUF_FLAG_PACKET;
|
|
|
|
}
|
2006-04-11 11:53:33 +00:00
|
|
|
pipe->tmp_page = NULL;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-09-20 15:32:19 +00:00
|
|
|
copied = copy_page_from_iter(page, 0, PAGE_SIZE, from);
|
|
|
|
if (unlikely(copied < PAGE_SIZE && iov_iter_count(from))) {
|
|
|
|
if (!ret)
|
|
|
|
ret = -EFAULT;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
ret += copied;
|
|
|
|
buf->offset = 0;
|
|
|
|
buf->len = copied;
|
|
|
|
|
2014-04-03 19:05:18 +00:00
|
|
|
if (!iov_iter_count(from))
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
|
|
|
}
|
2019-11-15 13:30:32 +00:00
|
|
|
|
2019-09-20 15:32:19 +00:00
|
|
|
if (!pipe_full(head, pipe->tail, max_usage))
|
2005-04-16 22:20:36 +00:00
|
|
|
continue;
|
2019-11-15 13:30:32 +00:00
|
|
|
|
|
|
|
/* Wait for buffer space to become available. */
|
2005-04-16 22:20:36 +00:00
|
|
|
if (filp->f_flags & O_NONBLOCK) {
|
2006-04-11 11:57:45 +00:00
|
|
|
if (!ret)
|
|
|
|
ret = -EAGAIN;
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
if (signal_pending(current)) {
|
2006-04-11 11:57:45 +00:00
|
|
|
if (!ret)
|
|
|
|
ret = -ERESTARTSYS;
|
2005-04-16 22:20:36 +00:00
|
|
|
break;
|
|
|
|
}
|
2006-04-11 11:53:33 +00:00
|
|
|
pipe->waiting_writers++;
|
|
|
|
pipe_wait(pipe);
|
|
|
|
pipe->waiting_writers--;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
out:
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
if (do_wakeup) {
|
2019-11-01 20:52:24 +00:00
|
|
|
wake_up_interruptible_poll(&pipe->wait, EPOLLIN | EPOLLRDNORM);
|
2006-04-11 11:53:33 +00:00
|
|
|
kill_fasync(&pipe->fasync_readers, SIGIO, POLL_IN);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2014-01-23 23:55:21 +00:00
|
|
|
if (ret > 0 && sb_start_write_trylock(file_inode(filp)->i_sb)) {
|
2012-03-26 13:59:21 +00:00
|
|
|
int err = file_update_time(filp);
|
|
|
|
if (err)
|
|
|
|
ret = err;
|
2014-01-23 23:55:21 +00:00
|
|
|
sb_end_write(file_inode(filp)->i_sb);
|
2012-03-26 13:59:21 +00:00
|
|
|
}
|
2005-04-16 22:20:36 +00:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2008-02-08 12:21:23 +00:00
|
|
|
static long pipe_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2013-03-21 15:16:56 +00:00
|
|
|
struct pipe_inode_info *pipe = filp->private_data;
|
2019-11-15 13:30:32 +00:00
|
|
|
int count, head, tail, mask;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
switch (cmd) {
|
|
|
|
case FIONREAD:
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_lock(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
count = 0;
|
2019-11-15 13:30:32 +00:00
|
|
|
head = pipe->head;
|
|
|
|
tail = pipe->tail;
|
|
|
|
mask = pipe->ring_size - 1;
|
|
|
|
|
|
|
|
while (tail != head) {
|
|
|
|
count += pipe->bufs[tail & mask].len;
|
|
|
|
tail++;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
2006-04-11 11:53:33 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
return put_user(count, (int __user *)arg);
|
|
|
|
default:
|
2012-05-25 10:39:13 +00:00
|
|
|
return -ENOIOCTLCMD;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-12-31 15:42:12 +00:00
|
|
|
/* No kernel lock held - fine */
|
2018-06-28 16:43:44 +00:00
|
|
|
static __poll_t
|
|
|
|
pipe_poll(struct file *filp, poll_table *wait)
|
2017-12-31 15:42:12 +00:00
|
|
|
{
|
2018-06-28 16:43:44 +00:00
|
|
|
__poll_t mask;
|
2017-12-31 15:42:12 +00:00
|
|
|
struct pipe_inode_info *pipe = filp->private_data;
|
2019-11-15 13:30:32 +00:00
|
|
|
unsigned int head = READ_ONCE(pipe->head);
|
|
|
|
unsigned int tail = READ_ONCE(pipe->tail);
|
2018-06-28 16:43:44 +00:00
|
|
|
|
|
|
|
poll_wait(filp, &pipe->wait, wait);
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
BUG_ON(pipe_occupancy(head, tail) > pipe->ring_size);
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/* Reading only -- no need for acquiring the semaphore. */
|
2018-06-28 16:43:44 +00:00
|
|
|
mask = 0;
|
2005-04-16 22:20:36 +00:00
|
|
|
if (filp->f_mode & FMODE_READ) {
|
2019-11-15 13:30:32 +00:00
|
|
|
if (!pipe_empty(head, tail))
|
|
|
|
mask |= EPOLLIN | EPOLLRDNORM;
|
2006-04-11 11:53:33 +00:00
|
|
|
if (!pipe->writers && filp->f_version != pipe->w_counter)
|
2018-02-11 22:34:03 +00:00
|
|
|
mask |= EPOLLHUP;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if (filp->f_mode & FMODE_WRITE) {
|
2019-10-16 15:47:32 +00:00
|
|
|
if (!pipe_full(head, tail, pipe->max_usage))
|
2019-11-15 13:30:32 +00:00
|
|
|
mask |= EPOLLOUT | EPOLLWRNORM;
|
2005-09-06 22:17:48 +00:00
|
|
|
/*
|
2018-02-11 22:34:03 +00:00
|
|
|
* Most Unices do not set EPOLLERR for FIFOs but on Linux they
|
2005-09-06 22:17:48 +00:00
|
|
|
* behave exactly like pipes for poll().
|
|
|
|
*/
|
2006-04-11 11:53:33 +00:00
|
|
|
if (!pipe->readers)
|
2018-02-11 22:34:03 +00:00
|
|
|
mask |= EPOLLERR;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
return mask;
|
|
|
|
}
|
|
|
|
|
vfs: fix subtle use-after-free of pipe_inode_info
The pipe code was trying (and failing) to be very careful about freeing
the pipe info only after the last access, with a pattern like:
spin_lock(&inode->i_lock);
if (!--pipe->files) {
inode->i_pipe = NULL;
kill = 1;
}
spin_unlock(&inode->i_lock);
__pipe_unlock(pipe);
if (kill)
free_pipe_info(pipe);
where the final freeing is done last.
HOWEVER. The above is actually broken, because while the freeing is
done at the end, if we have two racing processes releasing the pipe
inode info, the one that *doesn't* free it will decrement the ->files
count, and unlock the inode i_lock, but then still use the
"pipe_inode_info" afterwards when it does the "__pipe_unlock(pipe)".
This is *very* hard to trigger in practice, since the race window is
very small, and adding debug options seems to just hide it by slowing
things down.
Simon originally reported this way back in July as an Oops in
kmem_cache_allocate due to a single bit corruption (due to the final
"spin_unlock(pipe->mutex.wait_lock)" incrementing a field in a different
allocation that had re-used the free'd pipe-info), it's taken this long
to figure out.
Since the 'pipe->files' accesses aren't even protected by the pipe lock
(we very much use the inode lock for that), the simple solution is to
just drop the pipe lock early. And since there were two users of this
pattern, create a helper function for it.
Introduced commit ba5bb147330a ("pipe: take allocation and freeing of
pipe_inode_info out of ->i_mutex").
Reported-by: Simon Kirby <sim@hostway.ca>
Reported-by: Ian Applegate <ia@cloudflare.com>
Acked-by: Al Viro <viro@zeniv.linux.org.uk>
Cc: stable@kernel.org # v3.10+
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-12-02 17:44:51 +00:00
|
|
|
static void put_pipe_info(struct inode *inode, struct pipe_inode_info *pipe)
|
|
|
|
{
|
|
|
|
int kill = 0;
|
|
|
|
|
|
|
|
spin_lock(&inode->i_lock);
|
|
|
|
if (!--pipe->files) {
|
|
|
|
inode->i_pipe = NULL;
|
|
|
|
kill = 1;
|
|
|
|
}
|
|
|
|
spin_unlock(&inode->i_lock);
|
|
|
|
|
|
|
|
if (kill)
|
|
|
|
free_pipe_info(pipe);
|
|
|
|
}
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
static int
|
2013-03-12 13:58:10 +00:00
|
|
|
pipe_release(struct inode *inode, struct file *file)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
vfs: fix subtle use-after-free of pipe_inode_info
The pipe code was trying (and failing) to be very careful about freeing
the pipe info only after the last access, with a pattern like:
spin_lock(&inode->i_lock);
if (!--pipe->files) {
inode->i_pipe = NULL;
kill = 1;
}
spin_unlock(&inode->i_lock);
__pipe_unlock(pipe);
if (kill)
free_pipe_info(pipe);
where the final freeing is done last.
HOWEVER. The above is actually broken, because while the freeing is
done at the end, if we have two racing processes releasing the pipe
inode info, the one that *doesn't* free it will decrement the ->files
count, and unlock the inode i_lock, but then still use the
"pipe_inode_info" afterwards when it does the "__pipe_unlock(pipe)".
This is *very* hard to trigger in practice, since the race window is
very small, and adding debug options seems to just hide it by slowing
things down.
Simon originally reported this way back in July as an Oops in
kmem_cache_allocate due to a single bit corruption (due to the final
"spin_unlock(pipe->mutex.wait_lock)" incrementing a field in a different
allocation that had re-used the free'd pipe-info), it's taken this long
to figure out.
Since the 'pipe->files' accesses aren't even protected by the pipe lock
(we very much use the inode lock for that), the simple solution is to
just drop the pipe lock early. And since there were two users of this
pattern, create a helper function for it.
Introduced commit ba5bb147330a ("pipe: take allocation and freeing of
pipe_inode_info out of ->i_mutex").
Reported-by: Simon Kirby <sim@hostway.ca>
Reported-by: Ian Applegate <ia@cloudflare.com>
Acked-by: Al Viro <viro@zeniv.linux.org.uk>
Cc: stable@kernel.org # v3.10+
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-12-02 17:44:51 +00:00
|
|
|
struct pipe_inode_info *pipe = file->private_data;
|
2006-04-11 11:53:33 +00:00
|
|
|
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_lock(pipe);
|
2013-03-12 13:58:10 +00:00
|
|
|
if (file->f_mode & FMODE_READ)
|
|
|
|
pipe->readers--;
|
|
|
|
if (file->f_mode & FMODE_WRITE)
|
|
|
|
pipe->writers--;
|
2006-04-11 11:57:45 +00:00
|
|
|
|
2013-03-21 06:21:19 +00:00
|
|
|
if (pipe->readers || pipe->writers) {
|
2018-02-11 22:34:03 +00:00
|
|
|
wake_up_interruptible_sync_poll(&pipe->wait, EPOLLIN | EPOLLOUT | EPOLLRDNORM | EPOLLWRNORM | EPOLLERR | EPOLLHUP);
|
2006-04-11 11:53:33 +00:00
|
|
|
kill_fasync(&pipe->fasync_readers, SIGIO, POLL_IN);
|
|
|
|
kill_fasync(&pipe->fasync_writers, SIGIO, POLL_OUT);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
2013-03-21 06:21:19 +00:00
|
|
|
|
vfs: fix subtle use-after-free of pipe_inode_info
The pipe code was trying (and failing) to be very careful about freeing
the pipe info only after the last access, with a pattern like:
spin_lock(&inode->i_lock);
if (!--pipe->files) {
inode->i_pipe = NULL;
kill = 1;
}
spin_unlock(&inode->i_lock);
__pipe_unlock(pipe);
if (kill)
free_pipe_info(pipe);
where the final freeing is done last.
HOWEVER. The above is actually broken, because while the freeing is
done at the end, if we have two racing processes releasing the pipe
inode info, the one that *doesn't* free it will decrement the ->files
count, and unlock the inode i_lock, but then still use the
"pipe_inode_info" afterwards when it does the "__pipe_unlock(pipe)".
This is *very* hard to trigger in practice, since the race window is
very small, and adding debug options seems to just hide it by slowing
things down.
Simon originally reported this way back in July as an Oops in
kmem_cache_allocate due to a single bit corruption (due to the final
"spin_unlock(pipe->mutex.wait_lock)" incrementing a field in a different
allocation that had re-used the free'd pipe-info), it's taken this long
to figure out.
Since the 'pipe->files' accesses aren't even protected by the pipe lock
(we very much use the inode lock for that), the simple solution is to
just drop the pipe lock early. And since there were two users of this
pattern, create a helper function for it.
Introduced commit ba5bb147330a ("pipe: take allocation and freeing of
pipe_inode_info out of ->i_mutex").
Reported-by: Simon Kirby <sim@hostway.ca>
Reported-by: Ian Applegate <ia@cloudflare.com>
Acked-by: Al Viro <viro@zeniv.linux.org.uk>
Cc: stable@kernel.org # v3.10+
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-12-02 17:44:51 +00:00
|
|
|
put_pipe_info(inode, pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int
|
2013-03-12 13:58:10 +00:00
|
|
|
pipe_fasync(int fd, struct file *filp, int on)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2013-03-21 15:16:56 +00:00
|
|
|
struct pipe_inode_info *pipe = filp->private_data;
|
2013-03-12 13:58:10 +00:00
|
|
|
int retval = 0;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_lock(pipe);
|
2013-03-12 13:58:10 +00:00
|
|
|
if (filp->f_mode & FMODE_READ)
|
|
|
|
retval = fasync_helper(fd, filp, on, &pipe->fasync_readers);
|
|
|
|
if ((filp->f_mode & FMODE_WRITE) && retval >= 0) {
|
2006-04-11 11:57:45 +00:00
|
|
|
retval = fasync_helper(fd, filp, on, &pipe->fasync_writers);
|
2013-03-12 13:58:10 +00:00
|
|
|
if (retval < 0 && (filp->f_mode & FMODE_READ))
|
|
|
|
/* this can happen only if on == T */
|
2009-03-12 21:31:28 +00:00
|
|
|
fasync_helper(-1, filp, 0, &pipe->fasync_readers);
|
|
|
|
}
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
2009-02-01 21:52:56 +00:00
|
|
|
return retval;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2016-10-11 20:53:40 +00:00
|
|
|
static unsigned long account_pipe_buffers(struct user_struct *user,
|
2016-01-18 15:36:09 +00:00
|
|
|
unsigned long old, unsigned long new)
|
|
|
|
{
|
2016-10-11 20:53:40 +00:00
|
|
|
return atomic_long_add_return(new - old, &user->pipe_bufs);
|
2016-01-18 15:36:09 +00:00
|
|
|
}
|
|
|
|
|
2016-10-11 20:53:40 +00:00
|
|
|
static bool too_many_pipe_buffers_soft(unsigned long user_bufs)
|
2016-01-18 15:36:09 +00:00
|
|
|
{
|
2018-02-06 23:42:08 +00:00
|
|
|
unsigned long soft_limit = READ_ONCE(pipe_user_pages_soft);
|
|
|
|
|
|
|
|
return soft_limit && user_bufs > soft_limit;
|
2016-01-18 15:36:09 +00:00
|
|
|
}
|
|
|
|
|
2016-10-11 20:53:40 +00:00
|
|
|
static bool too_many_pipe_buffers_hard(unsigned long user_bufs)
|
2016-01-18 15:36:09 +00:00
|
|
|
{
|
2018-02-06 23:42:08 +00:00
|
|
|
unsigned long hard_limit = READ_ONCE(pipe_user_pages_hard);
|
|
|
|
|
|
|
|
return hard_limit && user_bufs > hard_limit;
|
2016-01-18 15:36:09 +00:00
|
|
|
}
|
|
|
|
|
2018-02-06 23:41:53 +00:00
|
|
|
static bool is_unprivileged_user(void)
|
|
|
|
{
|
|
|
|
return !capable(CAP_SYS_RESOURCE) && !capable(CAP_SYS_ADMIN);
|
|
|
|
}
|
|
|
|
|
2013-03-21 15:04:15 +00:00
|
|
|
struct pipe_inode_info *alloc_pipe_info(void)
|
2006-04-10 13:18:35 +00:00
|
|
|
{
|
2006-04-11 11:53:33 +00:00
|
|
|
struct pipe_inode_info *pipe;
|
2016-10-11 20:53:34 +00:00
|
|
|
unsigned long pipe_bufs = PIPE_DEF_BUFFERS;
|
|
|
|
struct user_struct *user = get_current_user();
|
2016-10-11 20:53:40 +00:00
|
|
|
unsigned long user_bufs;
|
2018-02-06 23:42:08 +00:00
|
|
|
unsigned int max_size = READ_ONCE(pipe_max_size);
|
2006-04-10 13:18:35 +00:00
|
|
|
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
pipe = kzalloc(sizeof(struct pipe_inode_info), GFP_KERNEL_ACCOUNT);
|
2016-10-11 20:53:34 +00:00
|
|
|
if (pipe == NULL)
|
|
|
|
goto out_free_uid;
|
|
|
|
|
2018-02-06 23:42:08 +00:00
|
|
|
if (pipe_bufs * PAGE_SIZE > max_size && !capable(CAP_SYS_RESOURCE))
|
|
|
|
pipe_bufs = max_size >> PAGE_SHIFT;
|
2016-10-11 20:53:43 +00:00
|
|
|
|
2016-10-11 20:53:40 +00:00
|
|
|
user_bufs = account_pipe_buffers(user, 0, pipe_bufs);
|
2016-10-11 20:53:37 +00:00
|
|
|
|
2018-02-06 23:41:53 +00:00
|
|
|
if (too_many_pipe_buffers_soft(user_bufs) && is_unprivileged_user()) {
|
2016-10-11 20:53:40 +00:00
|
|
|
user_bufs = account_pipe_buffers(user, pipe_bufs, 1);
|
2016-10-11 20:53:37 +00:00
|
|
|
pipe_bufs = 1;
|
2016-10-11 20:53:34 +00:00
|
|
|
}
|
2016-01-18 15:36:09 +00:00
|
|
|
|
2018-02-06 23:41:53 +00:00
|
|
|
if (too_many_pipe_buffers_hard(user_bufs) && is_unprivileged_user())
|
2016-10-11 20:53:37 +00:00
|
|
|
goto out_revert_acct;
|
|
|
|
|
|
|
|
pipe->bufs = kcalloc(pipe_bufs, sizeof(struct pipe_buffer),
|
|
|
|
GFP_KERNEL_ACCOUNT);
|
|
|
|
|
2016-10-11 20:53:34 +00:00
|
|
|
if (pipe->bufs) {
|
|
|
|
init_waitqueue_head(&pipe->wait);
|
|
|
|
pipe->r_counter = pipe->w_counter = 1;
|
2019-10-16 15:47:32 +00:00
|
|
|
pipe->max_usage = pipe_bufs;
|
2019-11-15 13:30:32 +00:00
|
|
|
pipe->ring_size = pipe_bufs;
|
2016-10-11 20:53:34 +00:00
|
|
|
pipe->user = user;
|
|
|
|
mutex_init(&pipe->mutex);
|
|
|
|
return pipe;
|
2006-04-10 13:18:35 +00:00
|
|
|
}
|
|
|
|
|
2016-10-11 20:53:37 +00:00
|
|
|
out_revert_acct:
|
2016-10-11 20:53:40 +00:00
|
|
|
(void) account_pipe_buffers(user, pipe_bufs, 0);
|
2016-10-11 20:53:34 +00:00
|
|
|
kfree(pipe);
|
|
|
|
out_free_uid:
|
|
|
|
free_uid(user);
|
2010-05-20 08:43:18 +00:00
|
|
|
return NULL;
|
2006-04-10 13:18:35 +00:00
|
|
|
}
|
|
|
|
|
2013-03-21 15:06:46 +00:00
|
|
|
void free_pipe_info(struct pipe_inode_info *pipe)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
(void) account_pipe_buffers(pipe->user, pipe->ring_size, 0);
|
2016-01-18 15:36:09 +00:00
|
|
|
free_uid(pipe->user);
|
2019-11-15 13:30:32 +00:00
|
|
|
for (i = 0; i < pipe->ring_size; i++) {
|
2006-04-11 11:53:33 +00:00
|
|
|
struct pipe_buffer *buf = pipe->bufs + i;
|
2005-04-16 22:20:36 +00:00
|
|
|
if (buf->ops)
|
2016-09-27 08:45:12 +00:00
|
|
|
pipe_buf_release(pipe, buf);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
2006-04-11 11:53:33 +00:00
|
|
|
if (pipe->tmp_page)
|
|
|
|
__free_page(pipe->tmp_page);
|
2010-05-20 08:43:18 +00:00
|
|
|
kfree(pipe->bufs);
|
2006-04-11 11:53:33 +00:00
|
|
|
kfree(pipe);
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2006-03-26 09:37:24 +00:00
|
|
|
static struct vfsmount *pipe_mnt __read_mostly;
|
2006-04-11 11:57:45 +00:00
|
|
|
|
2007-05-08 07:26:18 +00:00
|
|
|
/*
|
|
|
|
* pipefs_dname() is called from d_path().
|
|
|
|
*/
|
|
|
|
static char *pipefs_dname(struct dentry *dentry, char *buffer, int buflen)
|
|
|
|
{
|
|
|
|
return dynamic_dname(dentry, buffer, buflen, "pipe:[%lu]",
|
2015-03-17 22:26:12 +00:00
|
|
|
d_inode(dentry)->i_ino);
|
2007-05-08 07:26:18 +00:00
|
|
|
}
|
|
|
|
|
2009-02-20 06:02:22 +00:00
|
|
|
static const struct dentry_operations pipefs_dentry_operations = {
|
2007-05-08 07:26:18 +00:00
|
|
|
.d_dname = pipefs_dname,
|
2005-04-16 22:20:36 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
static struct inode * get_pipe_inode(void)
|
|
|
|
{
|
2011-07-26 09:36:34 +00:00
|
|
|
struct inode *inode = new_inode_pseudo(pipe_mnt->mnt_sb);
|
2006-04-11 11:53:33 +00:00
|
|
|
struct pipe_inode_info *pipe;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
if (!inode)
|
|
|
|
goto fail_inode;
|
|
|
|
|
2010-10-23 15:19:54 +00:00
|
|
|
inode->i_ino = get_next_ino();
|
|
|
|
|
2013-03-21 15:04:15 +00:00
|
|
|
pipe = alloc_pipe_info();
|
2006-04-11 11:53:33 +00:00
|
|
|
if (!pipe)
|
2005-04-16 22:20:36 +00:00
|
|
|
goto fail_iput;
|
2006-04-10 13:18:35 +00:00
|
|
|
|
2013-03-21 06:21:19 +00:00
|
|
|
inode->i_pipe = pipe;
|
|
|
|
pipe->files = 2;
|
2006-04-11 11:53:33 +00:00
|
|
|
pipe->readers = pipe->writers = 1;
|
2013-03-12 13:58:10 +00:00
|
|
|
inode->i_fop = &pipefifo_fops;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Mark the inode dirty from the very beginning,
|
|
|
|
* that way it will never be moved to the dirty
|
|
|
|
* list because "mark_inode_dirty()" will think
|
|
|
|
* that it already _is_ on the dirty list.
|
|
|
|
*/
|
|
|
|
inode->i_state = I_DIRTY;
|
|
|
|
inode->i_mode = S_IFIFO | S_IRUSR | S_IWUSR;
|
2008-11-13 23:39:05 +00:00
|
|
|
inode->i_uid = current_fsuid();
|
|
|
|
inode->i_gid = current_fsgid();
|
2016-09-14 14:48:04 +00:00
|
|
|
inode->i_atime = inode->i_mtime = inode->i_ctime = current_time(inode);
|
2006-04-11 11:53:33 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
return inode;
|
|
|
|
|
|
|
|
fail_iput:
|
|
|
|
iput(inode);
|
2006-04-11 11:57:45 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
fail_inode:
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2012-07-21 11:33:25 +00:00
|
|
|
int create_pipe_files(struct file **res, int flags)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2012-07-21 11:33:25 +00:00
|
|
|
struct inode *inode = get_pipe_inode();
|
2006-10-01 06:29:26 +00:00
|
|
|
struct file *f;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
|
|
|
if (!inode)
|
2012-07-21 11:33:25 +00:00
|
|
|
return -ENFILE;
|
2005-04-16 22:20:36 +00:00
|
|
|
|
2018-06-09 14:05:18 +00:00
|
|
|
f = alloc_file_pseudo(inode, pipe_mnt, "",
|
|
|
|
O_WRONLY | (flags & (O_NONBLOCK | O_DIRECT)),
|
|
|
|
&pipefifo_fops);
|
fs/pipe.c: preserve alloc_file() error code
If sys_pipe() was unable to allocate a 'struct file', it always failed
with ENFILE, which means "The number of simultaneously open files in the
system would exceed a system-imposed limit." However, alloc_file()
actually returns an ERR_PTR value and might fail with other error codes.
Currently, in addition to ENFILE, it can fail with ENOMEM, potentially
when there are few open files in the system. Update sys_pipe() to
preserve this error code.
In a prior submission of a similar patch (1) some concern was raised
about introducing a new error code for sys_pipe(). However, for most
system calls, programs cannot assume that new error codes will never be
introduced. In addition, ENOMEM was, in fact, already a possible error
code for sys_pipe(), in the case where the file descriptor table could
not be expanded due to insufficient memory.
(1) http://comments.gmane.org/gmane.linux.kernel/1357942
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-10-17 21:26:08 +00:00
|
|
|
if (IS_ERR(f)) {
|
2018-06-09 14:05:18 +00:00
|
|
|
free_pipe_info(inode->i_pipe);
|
|
|
|
iput(inode);
|
|
|
|
return PTR_ERR(f);
|
fs/pipe.c: preserve alloc_file() error code
If sys_pipe() was unable to allocate a 'struct file', it always failed
with ENFILE, which means "The number of simultaneously open files in the
system would exceed a system-imposed limit." However, alloc_file()
actually returns an ERR_PTR value and might fail with other error codes.
Currently, in addition to ENFILE, it can fail with ENOMEM, potentially
when there are few open files in the system. Update sys_pipe() to
preserve this error code.
In a prior submission of a similar patch (1) some concern was raised
about introducing a new error code for sys_pipe(). However, for most
system calls, programs cannot assume that new error codes will never be
introduced. In addition, ENOMEM was, in fact, already a possible error
code for sys_pipe(), in the case where the file descriptor table could
not be expanded due to insufficient memory.
(1) http://comments.gmane.org/gmane.linux.kernel/1357942
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-10-17 21:26:08 +00:00
|
|
|
}
|
2006-04-11 11:57:45 +00:00
|
|
|
|
2013-03-21 15:16:56 +00:00
|
|
|
f->private_data = inode->i_pipe;
|
2006-10-01 06:29:26 +00:00
|
|
|
|
2018-06-17 18:15:10 +00:00
|
|
|
res[0] = alloc_file_clone(f, O_RDONLY | (flags & O_NONBLOCK),
|
|
|
|
&pipefifo_fops);
|
fs/pipe.c: preserve alloc_file() error code
If sys_pipe() was unable to allocate a 'struct file', it always failed
with ENFILE, which means "The number of simultaneously open files in the
system would exceed a system-imposed limit." However, alloc_file()
actually returns an ERR_PTR value and might fail with other error codes.
Currently, in addition to ENFILE, it can fail with ENOMEM, potentially
when there are few open files in the system. Update sys_pipe() to
preserve this error code.
In a prior submission of a similar patch (1) some concern was raised
about introducing a new error code for sys_pipe(). However, for most
system calls, programs cannot assume that new error codes will never be
introduced. In addition, ENOMEM was, in fact, already a possible error
code for sys_pipe(), in the case where the file descriptor table could
not be expanded due to insufficient memory.
(1) http://comments.gmane.org/gmane.linux.kernel/1357942
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-10-17 21:26:08 +00:00
|
|
|
if (IS_ERR(res[0])) {
|
2018-07-09 06:29:58 +00:00
|
|
|
put_pipe_info(inode, inode->i_pipe);
|
|
|
|
fput(f);
|
|
|
|
return PTR_ERR(res[0]);
|
fs/pipe.c: preserve alloc_file() error code
If sys_pipe() was unable to allocate a 'struct file', it always failed
with ENFILE, which means "The number of simultaneously open files in the
system would exceed a system-imposed limit." However, alloc_file()
actually returns an ERR_PTR value and might fail with other error codes.
Currently, in addition to ENFILE, it can fail with ENOMEM, potentially
when there are few open files in the system. Update sys_pipe() to
preserve this error code.
In a prior submission of a similar patch (1) some concern was raised
about introducing a new error code for sys_pipe(). However, for most
system calls, programs cannot assume that new error codes will never be
introduced. In addition, ENOMEM was, in fact, already a possible error
code for sys_pipe(), in the case where the file descriptor table could
not be expanded due to insufficient memory.
(1) http://comments.gmane.org/gmane.linux.kernel/1357942
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-10-17 21:26:08 +00:00
|
|
|
}
|
2013-03-21 15:16:56 +00:00
|
|
|
res[0]->private_data = inode->i_pipe;
|
2012-07-21 11:33:25 +00:00
|
|
|
res[1] = f;
|
|
|
|
return 0;
|
2006-10-01 06:29:26 +00:00
|
|
|
}
|
|
|
|
|
2012-08-19 16:17:29 +00:00
|
|
|
static int __do_pipe_flags(int *fd, struct file **files, int flags)
|
2006-10-01 06:29:26 +00:00
|
|
|
{
|
|
|
|
int error;
|
|
|
|
int fdw, fdr;
|
|
|
|
|
pipes: add a "packetized pipe" mode for writing
The actual internal pipe implementation is already really about
individual packets (called "pipe buffers"), and this simply exposes that
as a special packetized mode.
When we are in the packetized mode (marked by O_DIRECT as suggested by
Alan Cox), a write() on a pipe will not merge the new data with previous
writes, so each write will get a pipe buffer of its own. The pipe
buffer is then marked with the PIPE_BUF_FLAG_PACKET flag, which in turn
will tell the reader side to break the read at that boundary (and throw
away any partial packet contents that do not fit in the read buffer).
End result: as long as you do writes less than PIPE_BUF in size (so that
the pipe doesn't have to split them up), you can now treat the pipe as a
packet interface, where each read() system call will read one packet at
a time. You can just use a sufficiently big read buffer (PIPE_BUF is
sufficient, since bigger than that doesn't guarantee atomicity anyway),
and the return value of the read() will naturally give you the size of
the packet.
NOTE! We do not support zero-sized packets, and zero-sized reads and
writes to a pipe continue to be no-ops. Also note that big packets will
currently be split at write time, but that the size at which that
happens is not really specified (except that it's bigger than PIPE_BUF).
Currently that limit is the system page size, but we might want to
explicitly support bigger packets some day.
The main user for this is going to be the autofs packet interface,
allowing us to stop having to care so deeply about exact packet sizes
(which have had bugs with 32/64-bit compatibility modes). But user
space can create packetized pipes with "pipe2(fd, O_DIRECT)", which will
fail with an EINVAL on kernels that do not support this interface.
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: David Miller <davem@davemloft.net>
Cc: Ian Kent <raven@themaw.net>
Cc: Thomas Meyer <thomas@m3y3r.de>
Cc: stable@kernel.org # needed for systemd/autofs interaction fix
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-04-29 20:12:42 +00:00
|
|
|
if (flags & ~(O_CLOEXEC | O_NONBLOCK | O_DIRECT))
|
2008-07-24 04:29:30 +00:00
|
|
|
return -EINVAL;
|
|
|
|
|
2012-07-21 11:33:25 +00:00
|
|
|
error = create_pipe_files(files, flags);
|
|
|
|
if (error)
|
|
|
|
return error;
|
2006-10-01 06:29:26 +00:00
|
|
|
|
2008-07-24 04:29:30 +00:00
|
|
|
error = get_unused_fd_flags(flags);
|
2006-10-01 06:29:26 +00:00
|
|
|
if (error < 0)
|
|
|
|
goto err_read_pipe;
|
|
|
|
fdr = error;
|
|
|
|
|
2008-07-24 04:29:30 +00:00
|
|
|
error = get_unused_fd_flags(flags);
|
2006-10-01 06:29:26 +00:00
|
|
|
if (error < 0)
|
|
|
|
goto err_fdr;
|
|
|
|
fdw = error;
|
|
|
|
|
2008-12-14 09:57:47 +00:00
|
|
|
audit_fd_pair(fdr, fdw);
|
2006-10-01 06:29:26 +00:00
|
|
|
fd[0] = fdr;
|
|
|
|
fd[1] = fdw;
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
err_fdr:
|
|
|
|
put_unused_fd(fdr);
|
|
|
|
err_read_pipe:
|
2012-07-21 11:33:25 +00:00
|
|
|
fput(files[0]);
|
|
|
|
fput(files[1]);
|
2006-10-01 06:29:26 +00:00
|
|
|
return error;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
2012-08-19 16:17:29 +00:00
|
|
|
int do_pipe_flags(int *fd, int flags)
|
|
|
|
{
|
|
|
|
struct file *files[2];
|
|
|
|
int error = __do_pipe_flags(fd, files, flags);
|
|
|
|
if (!error) {
|
|
|
|
fd_install(fd[0], files[0]);
|
|
|
|
fd_install(fd[1], files[1]);
|
|
|
|
}
|
|
|
|
return error;
|
|
|
|
}
|
|
|
|
|
2008-05-03 19:10:37 +00:00
|
|
|
/*
|
|
|
|
* sys_pipe() is the normal C calling standard for creating
|
|
|
|
* a pipe. It's not the way Unix traditionally does this, though.
|
|
|
|
*/
|
2018-03-11 10:34:28 +00:00
|
|
|
static int do_pipe2(int __user *fildes, int flags)
|
2008-05-03 19:10:37 +00:00
|
|
|
{
|
2012-08-19 16:17:29 +00:00
|
|
|
struct file *files[2];
|
2008-05-03 19:10:37 +00:00
|
|
|
int fd[2];
|
|
|
|
int error;
|
|
|
|
|
2012-08-19 16:17:29 +00:00
|
|
|
error = __do_pipe_flags(fd, files, flags);
|
2008-05-03 19:10:37 +00:00
|
|
|
if (!error) {
|
2012-08-19 16:17:29 +00:00
|
|
|
if (unlikely(copy_to_user(fildes, fd, sizeof(fd)))) {
|
|
|
|
fput(files[0]);
|
|
|
|
fput(files[1]);
|
|
|
|
put_unused_fd(fd[0]);
|
|
|
|
put_unused_fd(fd[1]);
|
2008-05-03 19:10:37 +00:00
|
|
|
error = -EFAULT;
|
2012-08-19 16:17:29 +00:00
|
|
|
} else {
|
|
|
|
fd_install(fd[0], files[0]);
|
|
|
|
fd_install(fd[1], files[1]);
|
2008-05-07 03:42:38 +00:00
|
|
|
}
|
2008-05-03 19:10:37 +00:00
|
|
|
}
|
|
|
|
return error;
|
|
|
|
}
|
|
|
|
|
2018-03-11 10:34:28 +00:00
|
|
|
SYSCALL_DEFINE2(pipe2, int __user *, fildes, int, flags)
|
|
|
|
{
|
|
|
|
return do_pipe2(fildes, flags);
|
|
|
|
}
|
|
|
|
|
2009-01-14 13:14:35 +00:00
|
|
|
SYSCALL_DEFINE1(pipe, int __user *, fildes)
|
2008-07-24 04:29:30 +00:00
|
|
|
{
|
2018-03-11 10:34:28 +00:00
|
|
|
return do_pipe2(fildes, 0);
|
2008-07-24 04:29:30 +00:00
|
|
|
}
|
|
|
|
|
2013-03-21 06:07:59 +00:00
|
|
|
static int wait_for_partner(struct pipe_inode_info *pipe, unsigned int *cnt)
|
2013-03-12 13:46:27 +00:00
|
|
|
{
|
2019-11-15 13:30:32 +00:00
|
|
|
int cur = *cnt;
|
2013-03-12 13:46:27 +00:00
|
|
|
|
|
|
|
while (cur == *cnt) {
|
2013-03-21 06:07:59 +00:00
|
|
|
pipe_wait(pipe);
|
2013-03-12 13:46:27 +00:00
|
|
|
if (signal_pending(current))
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
return cur == *cnt ? -ERESTARTSYS : 0;
|
|
|
|
}
|
|
|
|
|
2013-03-21 06:07:59 +00:00
|
|
|
static void wake_up_partner(struct pipe_inode_info *pipe)
|
2013-03-12 13:46:27 +00:00
|
|
|
{
|
2013-03-21 06:07:59 +00:00
|
|
|
wake_up_interruptible(&pipe->wait);
|
2013-03-12 13:46:27 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static int fifo_open(struct inode *inode, struct file *filp)
|
|
|
|
{
|
|
|
|
struct pipe_inode_info *pipe;
|
2013-03-12 13:58:10 +00:00
|
|
|
bool is_pipe = inode->i_sb->s_magic == PIPEFS_MAGIC;
|
2013-03-12 13:46:27 +00:00
|
|
|
int ret;
|
|
|
|
|
2013-03-21 06:21:19 +00:00
|
|
|
filp->f_version = 0;
|
|
|
|
|
|
|
|
spin_lock(&inode->i_lock);
|
|
|
|
if (inode->i_pipe) {
|
|
|
|
pipe = inode->i_pipe;
|
|
|
|
pipe->files++;
|
|
|
|
spin_unlock(&inode->i_lock);
|
|
|
|
} else {
|
|
|
|
spin_unlock(&inode->i_lock);
|
2013-03-21 15:04:15 +00:00
|
|
|
pipe = alloc_pipe_info();
|
2013-03-12 13:46:27 +00:00
|
|
|
if (!pipe)
|
2013-03-21 06:21:19 +00:00
|
|
|
return -ENOMEM;
|
|
|
|
pipe->files = 1;
|
|
|
|
spin_lock(&inode->i_lock);
|
|
|
|
if (unlikely(inode->i_pipe)) {
|
|
|
|
inode->i_pipe->files++;
|
|
|
|
spin_unlock(&inode->i_lock);
|
2013-03-21 15:06:46 +00:00
|
|
|
free_pipe_info(pipe);
|
2013-03-21 06:21:19 +00:00
|
|
|
pipe = inode->i_pipe;
|
|
|
|
} else {
|
|
|
|
inode->i_pipe = pipe;
|
|
|
|
spin_unlock(&inode->i_lock);
|
|
|
|
}
|
2013-03-12 13:46:27 +00:00
|
|
|
}
|
2013-03-21 15:16:56 +00:00
|
|
|
filp->private_data = pipe;
|
2013-03-21 06:21:19 +00:00
|
|
|
/* OK, we have a pipe and it's pinned down */
|
|
|
|
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_lock(pipe);
|
2013-03-12 13:46:27 +00:00
|
|
|
|
|
|
|
/* We can only do regular read/write on fifos */
|
|
|
|
filp->f_mode &= (FMODE_READ | FMODE_WRITE);
|
|
|
|
|
|
|
|
switch (filp->f_mode) {
|
|
|
|
case FMODE_READ:
|
|
|
|
/*
|
|
|
|
* O_RDONLY
|
|
|
|
* POSIX.1 says that O_NONBLOCK means return with the FIFO
|
|
|
|
* opened, even when there is no process writing the FIFO.
|
|
|
|
*/
|
|
|
|
pipe->r_counter++;
|
|
|
|
if (pipe->readers++ == 0)
|
2013-03-21 06:07:59 +00:00
|
|
|
wake_up_partner(pipe);
|
2013-03-12 13:46:27 +00:00
|
|
|
|
2013-03-12 13:58:10 +00:00
|
|
|
if (!is_pipe && !pipe->writers) {
|
2013-03-12 13:46:27 +00:00
|
|
|
if ((filp->f_flags & O_NONBLOCK)) {
|
2018-02-11 22:34:03 +00:00
|
|
|
/* suppress EPOLLHUP until we have
|
2013-03-12 13:46:27 +00:00
|
|
|
* seen a writer */
|
|
|
|
filp->f_version = pipe->w_counter;
|
|
|
|
} else {
|
2013-03-21 06:07:59 +00:00
|
|
|
if (wait_for_partner(pipe, &pipe->w_counter))
|
2013-03-12 13:46:27 +00:00
|
|
|
goto err_rd;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
break;
|
2019-11-15 13:30:32 +00:00
|
|
|
|
2013-03-12 13:46:27 +00:00
|
|
|
case FMODE_WRITE:
|
|
|
|
/*
|
|
|
|
* O_WRONLY
|
|
|
|
* POSIX.1 says that O_NONBLOCK means return -1 with
|
|
|
|
* errno=ENXIO when there is no process reading the FIFO.
|
|
|
|
*/
|
|
|
|
ret = -ENXIO;
|
2013-03-12 13:58:10 +00:00
|
|
|
if (!is_pipe && (filp->f_flags & O_NONBLOCK) && !pipe->readers)
|
2013-03-12 13:46:27 +00:00
|
|
|
goto err;
|
|
|
|
|
|
|
|
pipe->w_counter++;
|
|
|
|
if (!pipe->writers++)
|
2013-03-21 06:07:59 +00:00
|
|
|
wake_up_partner(pipe);
|
2013-03-12 13:46:27 +00:00
|
|
|
|
2013-03-12 13:58:10 +00:00
|
|
|
if (!is_pipe && !pipe->readers) {
|
2013-03-21 06:07:59 +00:00
|
|
|
if (wait_for_partner(pipe, &pipe->r_counter))
|
2013-03-12 13:46:27 +00:00
|
|
|
goto err_wr;
|
|
|
|
}
|
|
|
|
break;
|
2019-11-15 13:30:32 +00:00
|
|
|
|
2013-03-12 13:46:27 +00:00
|
|
|
case FMODE_READ | FMODE_WRITE:
|
|
|
|
/*
|
|
|
|
* O_RDWR
|
|
|
|
* POSIX.1 leaves this case "undefined" when O_NONBLOCK is set.
|
|
|
|
* This implementation will NEVER block on a O_RDWR open, since
|
|
|
|
* the process can at least talk to itself.
|
|
|
|
*/
|
|
|
|
|
|
|
|
pipe->readers++;
|
|
|
|
pipe->writers++;
|
|
|
|
pipe->r_counter++;
|
|
|
|
pipe->w_counter++;
|
|
|
|
if (pipe->readers == 1 || pipe->writers == 1)
|
2013-03-21 06:07:59 +00:00
|
|
|
wake_up_partner(pipe);
|
2013-03-12 13:46:27 +00:00
|
|
|
break;
|
|
|
|
|
|
|
|
default:
|
|
|
|
ret = -EINVAL;
|
|
|
|
goto err;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Ok! */
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
2013-03-12 13:46:27 +00:00
|
|
|
return 0;
|
|
|
|
|
|
|
|
err_rd:
|
|
|
|
if (!--pipe->readers)
|
|
|
|
wake_up_interruptible(&pipe->wait);
|
|
|
|
ret = -ERESTARTSYS;
|
|
|
|
goto err;
|
|
|
|
|
|
|
|
err_wr:
|
|
|
|
if (!--pipe->writers)
|
|
|
|
wake_up_interruptible(&pipe->wait);
|
|
|
|
ret = -ERESTARTSYS;
|
|
|
|
goto err;
|
|
|
|
|
|
|
|
err:
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
vfs: fix subtle use-after-free of pipe_inode_info
The pipe code was trying (and failing) to be very careful about freeing
the pipe info only after the last access, with a pattern like:
spin_lock(&inode->i_lock);
if (!--pipe->files) {
inode->i_pipe = NULL;
kill = 1;
}
spin_unlock(&inode->i_lock);
__pipe_unlock(pipe);
if (kill)
free_pipe_info(pipe);
where the final freeing is done last.
HOWEVER. The above is actually broken, because while the freeing is
done at the end, if we have two racing processes releasing the pipe
inode info, the one that *doesn't* free it will decrement the ->files
count, and unlock the inode i_lock, but then still use the
"pipe_inode_info" afterwards when it does the "__pipe_unlock(pipe)".
This is *very* hard to trigger in practice, since the race window is
very small, and adding debug options seems to just hide it by slowing
things down.
Simon originally reported this way back in July as an Oops in
kmem_cache_allocate due to a single bit corruption (due to the final
"spin_unlock(pipe->mutex.wait_lock)" incrementing a field in a different
allocation that had re-used the free'd pipe-info), it's taken this long
to figure out.
Since the 'pipe->files' accesses aren't even protected by the pipe lock
(we very much use the inode lock for that), the simple solution is to
just drop the pipe lock early. And since there were two users of this
pattern, create a helper function for it.
Introduced commit ba5bb147330a ("pipe: take allocation and freeing of
pipe_inode_info out of ->i_mutex").
Reported-by: Simon Kirby <sim@hostway.ca>
Reported-by: Ian Applegate <ia@cloudflare.com>
Acked-by: Al Viro <viro@zeniv.linux.org.uk>
Cc: stable@kernel.org # v3.10+
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-12-02 17:44:51 +00:00
|
|
|
|
|
|
|
put_pipe_info(inode, pipe);
|
2013-03-12 13:46:27 +00:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2013-03-12 13:58:10 +00:00
|
|
|
const struct file_operations pipefifo_fops = {
|
|
|
|
.open = fifo_open,
|
|
|
|
.llseek = no_llseek,
|
2014-04-02 23:56:54 +00:00
|
|
|
.read_iter = pipe_read,
|
2014-04-03 19:05:18 +00:00
|
|
|
.write_iter = pipe_write,
|
2018-06-28 16:43:44 +00:00
|
|
|
.poll = pipe_poll,
|
2013-03-12 13:58:10 +00:00
|
|
|
.unlocked_ioctl = pipe_ioctl,
|
|
|
|
.release = pipe_release,
|
|
|
|
.fasync = pipe_fasync,
|
2013-03-12 13:46:27 +00:00
|
|
|
};
|
|
|
|
|
pipe: relocate round_pipe_size() above pipe_set_size()
Patch series "pipe: fix limit handling", v2.
When changing a pipe's capacity with fcntl(F_SETPIPE_SZ), various limits
defined by /proc/sys/fs/pipe-* files are checked to see if unprivileged
users are exceeding limits on memory consumption.
While documenting and testing the operation of these limits I noticed
that, as currently implemented, these checks have a number of problems:
(1) When increasing the pipe capacity, the checks against the limits
in /proc/sys/fs/pipe-user-pages-{soft,hard} are made against
existing consumption, and exclude the memory required for the
increased pipe capacity. The new increase in pipe capacity can then
push the total memory used by the user for pipes (possibly far) over
a limit. This can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity
is less than the existing pipe capacity. This can lead to problems
if a user sets a large pipe capacity, and then the limits are
lowered, with the result that the user will no longer be able to
decrease the pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a) simultaneously,
and then allocate pipe buffers that are accounted for only in step
(c). The race means that the user's pipe buffer allocation could be
pushed over the limit (by an arbitrary amount, depending on how
unlucky we were in the race). [Thanks to Vegard Nossum for spotting
this point, which I had missed.]
This patch series addresses these three problems.
This patch (of 8):
This is a minor preparatory patch. After subsequent patches,
round_pipe_size() will be called from pipe_set_size(), so place
round_pipe_size() above pipe_set_size().
Link: http://lkml.kernel.org/r/91a91fdb-a959-ba7f-b551-b62477cc98a1@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:22 +00:00
|
|
|
/*
|
|
|
|
* Currently we rely on the pipe array holding a power-of-2 number
|
2017-11-17 23:29:21 +00:00
|
|
|
* of pages. Returns 0 on error.
|
pipe: relocate round_pipe_size() above pipe_set_size()
Patch series "pipe: fix limit handling", v2.
When changing a pipe's capacity with fcntl(F_SETPIPE_SZ), various limits
defined by /proc/sys/fs/pipe-* files are checked to see if unprivileged
users are exceeding limits on memory consumption.
While documenting and testing the operation of these limits I noticed
that, as currently implemented, these checks have a number of problems:
(1) When increasing the pipe capacity, the checks against the limits
in /proc/sys/fs/pipe-user-pages-{soft,hard} are made against
existing consumption, and exclude the memory required for the
increased pipe capacity. The new increase in pipe capacity can then
push the total memory used by the user for pipes (possibly far) over
a limit. This can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity
is less than the existing pipe capacity. This can lead to problems
if a user sets a large pipe capacity, and then the limits are
lowered, with the result that the user will no longer be able to
decrease the pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a) simultaneously,
and then allocate pipe buffers that are accounted for only in step
(c). The race means that the user's pipe buffer allocation could be
pushed over the limit (by an arbitrary amount, depending on how
unlucky we were in the race). [Thanks to Vegard Nossum for spotting
this point, which I had missed.]
This patch series addresses these three problems.
This patch (of 8):
This is a minor preparatory patch. After subsequent patches,
round_pipe_size() will be called from pipe_set_size(), so place
round_pipe_size() above pipe_set_size().
Link: http://lkml.kernel.org/r/91a91fdb-a959-ba7f-b551-b62477cc98a1@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:22 +00:00
|
|
|
*/
|
2018-02-06 23:42:00 +00:00
|
|
|
unsigned int round_pipe_size(unsigned long size)
|
pipe: relocate round_pipe_size() above pipe_set_size()
Patch series "pipe: fix limit handling", v2.
When changing a pipe's capacity with fcntl(F_SETPIPE_SZ), various limits
defined by /proc/sys/fs/pipe-* files are checked to see if unprivileged
users are exceeding limits on memory consumption.
While documenting and testing the operation of these limits I noticed
that, as currently implemented, these checks have a number of problems:
(1) When increasing the pipe capacity, the checks against the limits
in /proc/sys/fs/pipe-user-pages-{soft,hard} are made against
existing consumption, and exclude the memory required for the
increased pipe capacity. The new increase in pipe capacity can then
push the total memory used by the user for pipes (possibly far) over
a limit. This can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity
is less than the existing pipe capacity. This can lead to problems
if a user sets a large pipe capacity, and then the limits are
lowered, with the result that the user will no longer be able to
decrease the pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a) simultaneously,
and then allocate pipe buffers that are accounted for only in step
(c). The race means that the user's pipe buffer allocation could be
pushed over the limit (by an arbitrary amount, depending on how
unlucky we were in the race). [Thanks to Vegard Nossum for spotting
this point, which I had missed.]
This patch series addresses these three problems.
This patch (of 8):
This is a minor preparatory patch. After subsequent patches,
round_pipe_size() will be called from pipe_set_size(), so place
round_pipe_size() above pipe_set_size().
Link: http://lkml.kernel.org/r/91a91fdb-a959-ba7f-b551-b62477cc98a1@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:22 +00:00
|
|
|
{
|
2018-02-06 23:42:05 +00:00
|
|
|
if (size > (1U << 31))
|
2018-02-06 23:42:00 +00:00
|
|
|
return 0;
|
|
|
|
|
2018-02-06 23:41:45 +00:00
|
|
|
/* Minimum pipe size, as required by POSIX */
|
|
|
|
if (size < PAGE_SIZE)
|
2018-02-06 23:42:05 +00:00
|
|
|
return PAGE_SIZE;
|
2017-11-17 23:29:21 +00:00
|
|
|
|
2018-02-06 23:42:05 +00:00
|
|
|
return roundup_pow_of_two(size);
|
pipe: relocate round_pipe_size() above pipe_set_size()
Patch series "pipe: fix limit handling", v2.
When changing a pipe's capacity with fcntl(F_SETPIPE_SZ), various limits
defined by /proc/sys/fs/pipe-* files are checked to see if unprivileged
users are exceeding limits on memory consumption.
While documenting and testing the operation of these limits I noticed
that, as currently implemented, these checks have a number of problems:
(1) When increasing the pipe capacity, the checks against the limits
in /proc/sys/fs/pipe-user-pages-{soft,hard} are made against
existing consumption, and exclude the memory required for the
increased pipe capacity. The new increase in pipe capacity can then
push the total memory used by the user for pipes (possibly far) over
a limit. This can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity
is less than the existing pipe capacity. This can lead to problems
if a user sets a large pipe capacity, and then the limits are
lowered, with the result that the user will no longer be able to
decrease the pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a) simultaneously,
and then allocate pipe buffers that are accounted for only in step
(c). The race means that the user's pipe buffer allocation could be
pushed over the limit (by an arbitrary amount, depending on how
unlucky we were in the race). [Thanks to Vegard Nossum for spotting
this point, which I had missed.]
This patch series addresses these three problems.
This patch (of 8):
This is a minor preparatory patch. After subsequent patches,
round_pipe_size() will be called from pipe_set_size(), so place
round_pipe_size() above pipe_set_size().
Link: http://lkml.kernel.org/r/91a91fdb-a959-ba7f-b551-b62477cc98a1@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:22 +00:00
|
|
|
}
|
|
|
|
|
2010-05-20 08:43:18 +00:00
|
|
|
/*
|
|
|
|
* Allocate a new array of pipe buffers and copy the info over. Returns the
|
|
|
|
* pipe size if successful, or return -ERROR on error.
|
|
|
|
*/
|
2016-10-11 20:53:25 +00:00
|
|
|
static long pipe_set_size(struct pipe_inode_info *pipe, unsigned long arg)
|
2010-05-20 08:43:18 +00:00
|
|
|
{
|
|
|
|
struct pipe_buffer *bufs;
|
2019-11-15 13:30:32 +00:00
|
|
|
unsigned int size, nr_slots, head, tail, mask, n;
|
2016-10-11 20:53:40 +00:00
|
|
|
unsigned long user_bufs;
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
long ret = 0;
|
2016-10-11 20:53:25 +00:00
|
|
|
|
|
|
|
size = round_pipe_size(arg);
|
2019-11-15 13:30:32 +00:00
|
|
|
nr_slots = size >> PAGE_SHIFT;
|
2016-10-11 20:53:25 +00:00
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
if (!nr_slots)
|
2016-10-11 20:53:25 +00:00
|
|
|
return -EINVAL;
|
|
|
|
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
/*
|
|
|
|
* If trying to increase the pipe capacity, check that an
|
|
|
|
* unprivileged user is not trying to exceed various limits
|
|
|
|
* (soft limit check here, hard limit check just below).
|
|
|
|
* Decreasing the pipe capacity is always permitted, even
|
|
|
|
* if the user is currently over a limit.
|
|
|
|
*/
|
2019-11-15 13:30:32 +00:00
|
|
|
if (nr_slots > pipe->ring_size &&
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
size > pipe_max_size && !capable(CAP_SYS_RESOURCE))
|
2016-10-11 20:53:25 +00:00
|
|
|
return -EPERM;
|
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
user_bufs = account_pipe_buffers(pipe->user, pipe->ring_size, nr_slots);
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
if (nr_slots > pipe->ring_size &&
|
2016-10-11 20:53:40 +00:00
|
|
|
(too_many_pipe_buffers_hard(user_bufs) ||
|
|
|
|
too_many_pipe_buffers_soft(user_bufs)) &&
|
2018-02-06 23:41:53 +00:00
|
|
|
is_unprivileged_user()) {
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
ret = -EPERM;
|
|
|
|
goto out_revert_acct;
|
|
|
|
}
|
2010-05-20 08:43:18 +00:00
|
|
|
|
|
|
|
/*
|
2019-11-15 13:30:32 +00:00
|
|
|
* We can shrink the pipe, if arg is greater than the ring occupancy.
|
|
|
|
* Since we don't expect a lot of shrink+grow operations, just free and
|
|
|
|
* allocate again like we would do for growing. If the pipe currently
|
2010-05-20 08:43:18 +00:00
|
|
|
* contains more buffers than arg, then return busy.
|
|
|
|
*/
|
2019-11-15 13:30:32 +00:00
|
|
|
mask = pipe->ring_size - 1;
|
|
|
|
head = pipe->head;
|
|
|
|
tail = pipe->tail;
|
|
|
|
n = pipe_occupancy(pipe->head, pipe->tail);
|
|
|
|
if (nr_slots < n) {
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
ret = -EBUSY;
|
|
|
|
goto out_revert_acct;
|
|
|
|
}
|
2010-05-20 08:43:18 +00:00
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
bufs = kcalloc(nr_slots, sizeof(*bufs),
|
pipe: account to kmemcg
Pipes can consume a significant amount of system memory, hence they
should be accounted to kmemcg.
This patch marks pipe_inode_info and anonymous pipe buffer page
allocations as __GFP_ACCOUNT so that they would be charged to kmemcg.
Note, since a pipe buffer page can be "stolen" and get reused for other
purposes, including mapping to userspace, we clear PageKmemcg thus
resetting page->_mapcount and uncharge it in anon_pipe_buf_steal, which
is introduced by this patch.
A note regarding anon_pipe_buf_steal implementation. We allow to steal
the page if its ref count equals 1. It looks racy, but it is correct
for anonymous pipe buffer pages, because:
- We lock out all other pipe users, because ->steal is called with
pipe_lock held, so the page can't be spliced to another pipe from
under us.
- The page is not on LRU and it never was.
- Thus a parallel thread can access it only by PFN. Although this is
quite possible (e.g. see page_idle_get_page and balloon_page_isolate)
this is not dangerous, because all such functions do is increase page
ref count, check if the page is the one they are looking for, and
decrease ref count if it isn't. Since our page is clean except for
PageKmemcg mark, which doesn't conflict with other _mapcount users,
the worst that can happen is we see page_count > 2 due to a transient
ref, in which case we false-positively abort ->steal, which is still
fine, because ->steal is not guaranteed to succeed.
Link: http://lkml.kernel.org/r/20160527150313.GD26059@esperanza
Signed-off-by: Vladimir Davydov <vdavydov@virtuozzo.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-07-26 22:24:33 +00:00
|
|
|
GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
if (unlikely(!bufs)) {
|
|
|
|
ret = -ENOMEM;
|
|
|
|
goto out_revert_acct;
|
|
|
|
}
|
2010-05-20 08:43:18 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* The pipe array wraps around, so just start the new one at zero
|
2019-11-15 13:30:32 +00:00
|
|
|
* and adjust the indices.
|
2010-05-20 08:43:18 +00:00
|
|
|
*/
|
2019-11-15 13:30:32 +00:00
|
|
|
if (n > 0) {
|
|
|
|
unsigned int h = head & mask;
|
|
|
|
unsigned int t = tail & mask;
|
|
|
|
if (h > t) {
|
|
|
|
memcpy(bufs, pipe->bufs + t,
|
|
|
|
n * sizeof(struct pipe_buffer));
|
|
|
|
} else {
|
|
|
|
unsigned int tsize = pipe->ring_size - t;
|
|
|
|
if (h > 0)
|
|
|
|
memcpy(bufs + tsize, pipe->bufs,
|
|
|
|
h * sizeof(struct pipe_buffer));
|
|
|
|
memcpy(bufs, pipe->bufs + t,
|
|
|
|
tsize * sizeof(struct pipe_buffer));
|
|
|
|
}
|
2010-05-20 08:43:18 +00:00
|
|
|
}
|
|
|
|
|
2019-11-15 13:30:32 +00:00
|
|
|
head = n;
|
|
|
|
tail = 0;
|
|
|
|
|
2010-05-20 08:43:18 +00:00
|
|
|
kfree(pipe->bufs);
|
|
|
|
pipe->bufs = bufs;
|
2019-11-15 13:30:32 +00:00
|
|
|
pipe->ring_size = nr_slots;
|
2019-10-16 15:47:32 +00:00
|
|
|
pipe->max_usage = nr_slots;
|
2019-11-15 13:30:32 +00:00
|
|
|
pipe->tail = tail;
|
|
|
|
pipe->head = head;
|
2019-10-16 15:47:32 +00:00
|
|
|
return pipe->max_usage * PAGE_SIZE;
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
|
|
|
|
out_revert_acct:
|
2019-11-15 13:30:32 +00:00
|
|
|
(void) account_pipe_buffers(pipe->user, nr_slots, pipe->ring_size);
|
pipe: fix limit checking in pipe_set_size()
The limit checking in pipe_set_size() (used by fcntl(F_SETPIPE_SZ))
has the following problems:
(1) When increasing the pipe capacity, the checks against the limits in
/proc/sys/fs/pipe-user-pages-{soft,hard} are made against existing
consumption, and exclude the memory required for the increased pipe
capacity. The new increase in pipe capacity can then push the total
memory used by the user for pipes (possibly far) over a limit. This
can also trigger the problem described next.
(2) The limit checks are performed even when the new pipe capacity is
less than the existing pipe capacity. This can lead to problems if a
user sets a large pipe capacity, and then the limits are lowered,
with the result that the user will no longer be able to decrease the
pipe capacity.
(3) As currently implemented, accounting and checking against the
limits is done as follows:
(a) Test whether the user has exceeded the limit.
(b) Make new pipe buffer allocation.
(c) Account new allocation against the limits.
This is racey. Multiple processes may pass point (a)
simultaneously, and then allocate pipe buffers that are accounted
for only in step (c). The race means that the user's pipe buffer
allocation could be pushed over the limit (by an arbitrary amount,
depending on how unlucky we were in the race). [Thanks to Vegard
Nossum for spotting this point, which I had missed.]
This patch addresses the above problems as follows:
* Perform checks against the limits only when increasing a pipe's
capacity; an unprivileged user can always decrease a pipe's capacity.
* Alter the checks against limits to include the memory required for
the new pipe capacity.
* Re-order the accounting step so that it precedes the buffer
allocation. If the accounting step determines that a limit has
been reached, revert the accounting and cause the operation to fail.
The program below can be used to demonstrate problems 1 and 2, and the
effect of the fix. The program takes one or more command-line arguments.
The first argument specifies the number of pipes that the program should
create. The remaining arguments are, alternately, pipe capacities that
should be set using fcntl(F_SETPIPE_SZ), and sleep intervals (in
seconds) between the fcntl() operations. (The sleep intervals allow the
possibility to change the limits between fcntl() operations.)
Problem 1
=========
Using the test program on an unpatched kernel, we first set some
limits:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Then show that we can set a pipe with capacity (100MB) that is
over the hard limit
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Now set the capacity to 100MB twice. The second call fails (which is
probably surprising to most users, since it seems like a no-op):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000 0 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
F_SETPIPE_SZ returned 134217728
Loop 2: set pipe capacity to 100000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
With a patched kernel, setting a capacity over the limit fails at the
first attempt:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 100000000
Initial pipe capacity: 65536
Loop 1: set pipe capacity to 100000000 bytes
Loop 1, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
There is a small chance that the change to fix this problem could
break user-space, since there are cases where fcntl(F_SETPIPE_SZ)
calls that previously succeeded might fail. However, the chances are
small, since (a) the pipe-user-pages-{soft,hard} limits are new (in
4.5), and the default soft/hard limits are high/unlimited. Therefore,
it seems warranted to make these limits operate more precisely (and
behave more like what users probably expect).
Problem 2
=========
Running the test program on an unpatched kernel, we first set some limits:
# getconf PAGESIZE
4096
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard # 40.96 MB
Now perform two fcntl(F_SETPIPE_SZ) operations on a single pipe,
first setting a pipe capacity (10MB), sleeping for a few seconds,
during which time the hard limit is lowered, and then set pipe
capacity to a smaller amount (5MB):
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 748
# Initial pipe capacity: 65536
Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard # 4.096 MB
# Loop 2: set pipe capacity to 5000000 bytes
Loop 2, pipe 0: F_SETPIPE_SZ failed: fcntl: Operation not permitted
In this case, the user should be able to lower the limit.
With a kernel that has the patch below, the second fcntl()
succeeds:
# echo 0 > /proc/sys/fs/pipe-user-pages-soft
# echo 1000000000 > /proc/sys/fs/pipe-max-size
# echo 10000 > /proc/sys/fs/pipe-user-pages-hard
# sudo -u mtk ./test_F_SETPIPE_SZ 1 10000000 15 5000000 &
[1] 3215
# Initial pipe capacity: 65536
# Loop 1: set pipe capacity to 10000000 bytes
F_SETPIPE_SZ returned 16777216
Sleeping 15 seconds
# echo 1000 > /proc/sys/fs/pipe-user-pages-hard
# Loop 2: set pipe capacity to 5000000 bytes
F_SETPIPE_SZ returned 8388608
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
/* test_F_SETPIPE_SZ.c
(C) 2016, Michael Kerrisk; licensed under GNU GPL version 2 or later
Test operation of fcntl(F_SETPIPE_SZ) for setting pipe capacity
and interactions with limits defined by /proc/sys/fs/pipe-* files.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
int
main(int argc, char *argv[])
{
int (*pfd)[2];
int npipes;
int pcap, rcap;
int j, p, s, stime, loop;
if (argc < 2) {
fprintf(stderr, "Usage: %s num-pipes "
"[pipe-capacity sleep-time]...\n", argv[0]);
exit(EXIT_FAILURE);
}
npipes = atoi(argv[1]);
pfd = calloc(npipes, sizeof (int [2]));
if (pfd == NULL) {
perror("calloc");
exit(EXIT_FAILURE);
}
for (j = 0; j < npipes; j++) {
if (pipe(pfd[j]) == -1) {
fprintf(stderr, "Loop %d: pipe() failed: ", j);
perror("pipe");
exit(EXIT_FAILURE);
}
}
printf("Initial pipe capacity: %d\n", fcntl(pfd[0][0], F_GETPIPE_SZ));
for (j = 2; j < argc; j += 2 ) {
loop = j / 2;
pcap = atoi(argv[j]);
printf(" Loop %d: set pipe capacity to %d bytes\n", loop, pcap);
for (p = 0; p < npipes; p++) {
s = fcntl(pfd[p][0], F_SETPIPE_SZ, pcap);
if (s == -1) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"failed: ", loop, p);
perror("fcntl");
exit(EXIT_FAILURE);
}
if (p == 0) {
printf(" F_SETPIPE_SZ returned %d\n", s);
rcap = s;
} else {
if (s != rcap) {
fprintf(stderr, " Loop %d, pipe %d: F_SETPIPE_SZ "
"unexpected return: %d\n", loop, p, s);
exit(EXIT_FAILURE);
}
}
stime = (j + 1 < argc) ? atoi(argv[j + 1]) : 0;
if (stime > 0) {
printf(" Sleeping %d seconds\n", stime);
sleep(stime);
}
}
}
exit(EXIT_SUCCESS);
}
8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---8x---
Patch history:
v2
* Switch order of test in 'if' statement to avoid function call
(to capability()) in normal path. [This is a fix to a preexisting
wart in the code. Thanks to Willy Tarreau]
* Perform (size > pipe_max_size) check before calling
account_pipe_buffers(). [Thanks to Vegard Nossum]
Quoting Vegard:
The potential problem happens if the user passes a very large number
which will overflow pipe->user->pipe_bufs.
On 32-bit, sizeof(int) == sizeof(long), so if they pass arg = INT_MAX
then round_pipe_size() returns INT_MAX. Although it's true that the
accounting is done in terms of pages and not bytes, so you'd need on
the order of (1 << 13) = 8192 processes hitting the limit at the same
time in order to make it overflow, which seems a bit unlikely.
(See https://lkml.org/lkml/2016/8/12/215 for another discussion on the
limit checking)
Link: http://lkml.kernel.org/r/1e464945-536b-2420-798b-e77b9c7e8593@gmail.com
Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>
Reviewed-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Willy Tarreau <w@1wt.eu>
Cc: <socketpair@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: Jens Axboe <axboe@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-10-11 20:53:31 +00:00
|
|
|
return ret;
|
2010-05-20 08:43:18 +00:00
|
|
|
}
|
|
|
|
|
2010-11-29 00:27:19 +00:00
|
|
|
/*
|
|
|
|
* After the inode slimming patch, i_pipe/i_bdev/i_cdev share the same
|
|
|
|
* location, so checking ->i_pipe is not enough to verify that this is a
|
|
|
|
* pipe.
|
|
|
|
*/
|
|
|
|
struct pipe_inode_info *get_pipe_info(struct file *file)
|
|
|
|
{
|
2013-03-21 15:16:56 +00:00
|
|
|
return file->f_op == &pipefifo_fops ? file->private_data : NULL;
|
2010-11-29 00:27:19 +00:00
|
|
|
}
|
|
|
|
|
2010-05-20 08:43:18 +00:00
|
|
|
long pipe_fcntl(struct file *file, unsigned int cmd, unsigned long arg)
|
|
|
|
{
|
|
|
|
struct pipe_inode_info *pipe;
|
|
|
|
long ret;
|
|
|
|
|
2010-11-28 22:09:57 +00:00
|
|
|
pipe = get_pipe_info(file);
|
2010-05-20 08:43:18 +00:00
|
|
|
if (!pipe)
|
|
|
|
return -EBADF;
|
|
|
|
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_lock(pipe);
|
2010-05-20 08:43:18 +00:00
|
|
|
|
|
|
|
switch (cmd) {
|
2016-10-11 20:53:25 +00:00
|
|
|
case F_SETPIPE_SZ:
|
|
|
|
ret = pipe_set_size(pipe, arg);
|
2010-05-20 08:43:18 +00:00
|
|
|
break;
|
|
|
|
case F_GETPIPE_SZ:
|
2019-10-16 15:47:32 +00:00
|
|
|
ret = pipe->max_usage * PAGE_SIZE;
|
2010-05-20 08:43:18 +00:00
|
|
|
break;
|
|
|
|
default:
|
|
|
|
ret = -EINVAL;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
2013-03-21 16:24:01 +00:00
|
|
|
__pipe_unlock(pipe);
|
2010-05-20 08:43:18 +00:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2011-01-07 06:49:50 +00:00
|
|
|
static const struct super_operations pipefs_ops = {
|
|
|
|
.destroy_inode = free_inode_nonrcu,
|
2011-11-01 00:10:04 +00:00
|
|
|
.statfs = simple_statfs,
|
2011-01-07 06:49:50 +00:00
|
|
|
};
|
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
/*
|
|
|
|
* pipefs should _never_ be mounted by userland - too much of security hassle,
|
|
|
|
* no real gain from having the whole whorehouse mounted. So we don't need
|
|
|
|
* any operations on the root directory. However, we need a non-trivial
|
|
|
|
* d_name - pipe: will go nicely and kill the special-casing in procfs.
|
|
|
|
*/
|
2019-03-25 16:38:23 +00:00
|
|
|
|
|
|
|
static int pipefs_init_fs_context(struct fs_context *fc)
|
2005-04-16 22:20:36 +00:00
|
|
|
{
|
2019-03-25 16:38:23 +00:00
|
|
|
struct pseudo_fs_context *ctx = init_pseudo(fc, PIPEFS_MAGIC);
|
|
|
|
if (!ctx)
|
|
|
|
return -ENOMEM;
|
|
|
|
ctx->ops = &pipefs_ops;
|
|
|
|
ctx->dops = &pipefs_dentry_operations;
|
|
|
|
return 0;
|
2005-04-16 22:20:36 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
static struct file_system_type pipe_fs_type = {
|
|
|
|
.name = "pipefs",
|
2019-03-25 16:38:23 +00:00
|
|
|
.init_fs_context = pipefs_init_fs_context,
|
2005-04-16 22:20:36 +00:00
|
|
|
.kill_sb = kill_anon_super,
|
|
|
|
};
|
|
|
|
|
|
|
|
static int __init init_pipe_fs(void)
|
|
|
|
{
|
|
|
|
int err = register_filesystem(&pipe_fs_type);
|
2006-04-11 11:57:45 +00:00
|
|
|
|
2005-04-16 22:20:36 +00:00
|
|
|
if (!err) {
|
|
|
|
pipe_mnt = kern_mount(&pipe_fs_type);
|
|
|
|
if (IS_ERR(pipe_mnt)) {
|
|
|
|
err = PTR_ERR(pipe_mnt);
|
|
|
|
unregister_filesystem(&pipe_fs_type);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return err;
|
|
|
|
}
|
|
|
|
|
|
|
|
fs_initcall(init_pipe_fs);
|