| .\" Copyright (C) 2005 Michael Kerrisk <mtk.manpages@gmail.com> |
| .\" |
| .\" %%%LICENSE_START(VERBATIM) |
| .\" Permission is granted to make and distribute verbatim copies of this |
| .\" manual provided the copyright notice and this permission notice are |
| .\" preserved on all copies. |
| .\" |
| .\" Permission is granted to copy and distribute modified versions of this |
| .\" manual under the conditions for verbatim copying, provided that the |
| .\" entire resulting derived work is distributed under the terms of a |
| .\" permission notice identical to this one. |
| .\" |
| .\" Since the Linux kernel and libraries are constantly changing, this |
| .\" manual page may be incorrect or out-of-date. The author(s) assume no |
| .\" responsibility for errors or omissions, or for damages resulting from |
| .\" the use of the information contained herein. The author(s) may not |
| .\" have taken the same level of care in the production of this manual, |
| .\" which is licensed free of charge, as they might when working |
| .\" professionally. |
| .\" |
| .\" Formatted or processed versions of this manual, if unaccompanied by |
| .\" the source, must acknowledge the copyright and authors of this work. |
| .\" %%%LICENSE_END |
| .\" |
| .TH PIPE 7 2016-10-08 "Linux" "Linux Programmer's Manual" |
| .SH NAME |
| pipe \- overview of pipes and FIFOs |
| .SH DESCRIPTION |
| Pipes and FIFOs (also known as named pipes) |
| provide a unidirectional interprocess communication channel. |
| A pipe has a |
| .I read end |
| and a |
| .IR "write end" . |
| Data written to the write end of a pipe can be read |
| from the read end of the pipe. |
| |
| A pipe is created using |
| .BR pipe (2), |
| which creates a new pipe and returns two file descriptors, |
| one referring to the read end of the pipe, |
| the other referring to the write end. |
| Pipes can be used to create a communication channel between related |
| processes; see |
| .BR pipe (2) |
| for an example. |
| |
| A FIFO (short for First In First Out) has a name within the filesystem |
| (created using |
| .BR mkfifo (3)), |
| and is opened using |
| .BR open (2). |
| Any process may open a FIFO, assuming the file permissions allow it. |
| The read end is opened using the |
| .B O_RDONLY |
| flag; the write end is opened using the |
| .B O_WRONLY |
| flag. |
| See |
| .BR fifo (7) |
| for further details. |
| .IR Note : |
| although FIFOs have a pathname in the filesystem, |
| I/O on FIFOs does not involve operations on the underlying device |
| (if there is one). |
| .SS I/O on pipes and FIFOs |
| The only difference between pipes and FIFOs is the manner in which |
| they are created and opened. |
| Once these tasks have been accomplished, |
| I/O on pipes and FIFOs has exactly the same semantics. |
| |
| If a process attempts to read from an empty pipe, then |
| .BR read (2) |
| will block until data is available. |
| If a process attempts to write to a full pipe (see below), then |
| .BR write (2) |
| blocks until sufficient data has been read from the pipe |
| to allow the write to complete. |
| Nonblocking I/O is possible by using the |
| .BR fcntl (2) |
| .B F_SETFL |
| operation to enable the |
| .B O_NONBLOCK |
| open file status flag. |
| |
| The communication channel provided by a pipe is a |
| .IR "byte stream" : |
| there is no concept of message boundaries. |
| |
| If all file descriptors referring to the write end of a pipe |
| have been closed, then an attempt to |
| .BR read (2) |
| from the pipe will see end-of-file |
| .RB ( read (2) |
| will return 0). |
| If all file descriptors referring to the read end of a pipe |
| have been closed, then a |
| .BR write (2) |
| will cause a |
| .B SIGPIPE |
| signal to be generated for the calling process. |
| If the calling process is ignoring this signal, then |
| .BR write (2) |
| fails with the error |
| .BR EPIPE . |
| An application that uses |
| .BR pipe (2) |
| and |
| .BR fork (2) |
| should use suitable |
| .BR close (2) |
| calls to close unnecessary duplicate file descriptors; |
| this ensures that end-of-file and |
| .BR SIGPIPE / EPIPE |
| are delivered when appropriate. |
| |
| It is not possible to apply |
| .BR lseek (2) |
| to a pipe. |
| .SS Pipe capacity |
| A pipe has a limited capacity. |
| If the pipe is full, then a |
| .BR write (2) |
| will block or fail, depending on whether the |
| .B O_NONBLOCK |
| flag is set (see below). |
| Different implementations have different limits for the pipe capacity. |
| Applications should not rely on a particular capacity: |
| an application should be designed so that a reading process consumes data |
| as soon as it is available, |
| so that a writing process does not remain blocked. |
| |
| In Linux versions before 2.6.11, the capacity of a pipe was the same as |
| the system page size (e.g., 4096 bytes on i386). |
| Since Linux 2.6.11, the pipe capacity is 65536 bytes. |
| Since Linux 2.6.35, the default pipe capacity is 65536 bytes, |
| but the capacity can be queried and set using the |
| .BR fcntl (2) |
| .BR F_GETPIPE_SZ |
| and |
| .BR F_SETPIPE_SZ |
| operations. |
| See |
| .BR fcntl (2) |
| for more information. |
| |
| The following |
| .BR ioctl (2) |
| operation, which can be applied to a file descriptor |
| that refers to either end of a pipe, |
| places a count of the number of unread bytes in the pipe in the |
| .I int |
| buffer pointed to by the final argument of the call: |
| |
| ioctl(fd, FIONREAD, &nbytes); |
| |
| The |
| .B FIONREAD |
| operation is not specified in any standard, |
| but is provided on many implementations. |
| .\" |
| .SS /proc files |
| On Linux, the following files control how much memory can be used for pipes: |
| .TP |
| .IR /proc/sys/fs/pipe-max-pages " (only in Linux 2.6.34)" |
| .\" commit b492e95be0ae672922f4734acf3f5d35c30be948 |
| An upper limit, in pages, on the capacity that an unprivileged user |
| (one without the |
| .BR CAP_SYS_RESOURCE |
| capability) |
| can set for a pipe. |
| |
| The default value for this limit is 16 times the default pipe capacity |
| (see above); the lower limit is two pages. |
| |
| This interface was removed in Linux 2.6.35, in favor of |
| .IR /proc/sys/fs/pipe-max-size . |
| .TP |
| .IR /proc/sys/fs/pipe-max-size " (since Linux 2.6.35)" |
| .\" commit ff9da691c0498ff81fdd014e7a0731dab2337dac |
| The maximum size (in bytes) of individual pipes that can be set |
| .\" This limit is not checked on pipe creation, where the capacity is |
| .\" always PIPE_DEF_BUFS, regardless of pipe-max-size |
| by users without the |
| .B CAP_SYS_RESOURCE |
| capability. |
| The value assigned to this file may be rounded upward, |
| to reflect the value actually employed for a convenient implementation. |
| To determine the rounded-up value, |
| display the contents of this file after assigning a value to it. |
| |
| The default value for this file is 1048576 (1 MiB). |
| The minimum value that can be assigned to this file is the system page size. |
| Attempts to set a limit less than the page size cause |
| .BR write (2) |
| to fail with the error |
| .BR EINVAL . |
| .TP |
| .IR /proc/sys/fs/pipe-user-pages-hard " (since Linux 4.5)" |
| .\" commit 759c01142a5d0f364a462346168a56de28a80f52 |
| The hard limit on the total size (in pages) of all pipes created or set by |
| a single unprivileged user (i.e., one with neither the |
| .B CAP_SYS_RESOURCE |
| nor the |
| .B CAP_SYS_ADMIN |
| capability). |
| So long as the total number of pages allocated to pipe buffers |
| for this user is at this limit, |
| attempts to create new pipes will be denied, |
| and attempts to increase a pipe's capacity will be denied. |
| |
| When the value of this limit is zero (which is the default), |
| no hard limit is applied. |
| .\" The default was chosen to avoid breaking existing applications that |
| .\" make intensive use of pipes (e.g., for splicing). |
| .TP |
| .IR /proc/sys/fs/pipe-user-pages-soft " (since Linux 4.5)" |
| .\" commit 759c01142a5d0f364a462346168a56de28a80f52 |
| The soft limit on the total size (in pages) of all pipes created or set by |
| a single unprivileged user (i.e., one with neither the |
| .B CAP_SYS_RESOURCE |
| nor the |
| .B CAP_SYS_ADMIN |
| capability). |
| So long as the total number of pages allocated to pipe buffers |
| for this user is at this limit, |
| individual pipes created by a user will be limited to one page, |
| and attempts to increase a pipe's capacity will be denied. |
| |
| When the value of this limit is zero, no soft limit is applied. |
| The default value for this file is 16384, |
| which permits creating up to 1024 pipes with the default capacity. |
| .PP |
| Before Linux 4.9, some bugs affected the handling of the |
| .IR pipe-user-pages-soft |
| and |
| .IR pipe-user-pages-hard |
| limits; see BUGS. |
| .\" |
| .SS PIPE_BUF |
| POSIX.1 says that |
| .BR write (2)s |
| of less than |
| .B PIPE_BUF |
| bytes must be atomic: the output data is written to the pipe as a |
| contiguous sequence. |
| Writes of more than |
| .B PIPE_BUF |
| bytes may be nonatomic: the kernel may interleave the data |
| with data written by other processes. |
| POSIX.1 requires |
| .B PIPE_BUF |
| to be at least 512 bytes. |
| (On Linux, |
| .B PIPE_BUF |
| is 4096 bytes.) |
| The precise semantics depend on whether the file descriptor is nonblocking |
| .RB ( O_NONBLOCK ), |
| whether there are multiple writers to the pipe, and on |
| .IR n , |
| the number of bytes to be written: |
| .TP |
| \fBO_NONBLOCK\fP disabled, \fIn\fP <= \fBPIPE_BUF\fP |
| All |
| .I n |
| bytes are written atomically; |
| .BR write (2) |
| may block if there is not room for |
| .I n |
| bytes to be written immediately |
| .TP |
| \fBO_NONBLOCK\fP enabled, \fIn\fP <= \fBPIPE_BUF\fP |
| If there is room to write |
| .I n |
| bytes to the pipe, then |
| .BR write (2) |
| succeeds immediately, writing all |
| .I n |
| bytes; otherwise |
| .BR write (2) |
| fails, with |
| .I errno |
| set to |
| .BR EAGAIN . |
| .TP |
| \fBO_NONBLOCK\fP disabled, \fIn\fP > \fBPIPE_BUF\fP |
| The write is nonatomic: the data given to |
| .BR write (2) |
| may be interleaved with |
| .BR write (2)s |
| by other process; |
| the |
| .BR write (2) |
| blocks until |
| .I n |
| bytes have been written. |
| .TP |
| \fBO_NONBLOCK\fP enabled, \fIn\fP > \fBPIPE_BUF\fP |
| If the pipe is full, then |
| .BR write (2) |
| fails, with |
| .I errno |
| set to |
| .BR EAGAIN . |
| Otherwise, from 1 to |
| .I n |
| bytes may be written (i.e., a "partial write" may occur; |
| the caller should check the return value from |
| .BR write (2) |
| to see how many bytes were actually written), |
| and these bytes may be interleaved with writes by other processes. |
| .SS Open file status flags |
| The only open file status flags that can be meaningfully applied to |
| a pipe or FIFO are |
| .B O_NONBLOCK |
| and |
| .BR O_ASYNC . |
| |
| Setting the |
| .B O_ASYNC |
| flag for the read end of a pipe causes a signal |
| .RB ( SIGIO |
| by default) to be generated when new input becomes available on the pipe. |
| The target for delivery of signals must be set using the |
| .BR fcntl (2) |
| .B F_SETOWN |
| command. |
| On Linux, |
| .B O_ASYNC |
| is supported for pipes and FIFOs only since kernel 2.6. |
| .SS Portability notes |
| On some systems (but not Linux), pipes are bidirectional: |
| data can be transmitted in both directions between the pipe ends. |
| POSIX.1 requires only unidirectional pipes. |
| Portable applications should avoid reliance on |
| bidirectional pipe semantics. |
| .SS BUGS |
| Before Linux 4.9, some bugs affected the handling of the |
| .IR pipe-user-pages-soft |
| and |
| .IR pipe-user-pages-hard |
| limits when using the |
| .BR fcntl (2) |
| .BR F_SETPIPE_SZ |
| operation to change a pipe's capacity: |
| .\" These bugs where remedied by a series of patches, in particular, |
| .\" commit b0b91d18e2e97b741b294af9333824ecc3fadfd8 and |
| .\" commit a005ca0e6813e1d796a7422a7e31d8b8d6555df1 |
| .IP (1) 5 |
| When increasing the pipe capacity, the checks against the soft and |
| hard limits were made against existing consumption, |
| and excluded the memory required for the increased pipe capacity. |
| The new increase in pipe capacity could then push the total |
| memory used by the user for pipes (possibly far) over a limit. |
| (This could also trigger the problem described next.) |
| |
| Starting with Linux 4.9, |
| the limit checking includes the memory required for the new pipe capacity. |
| .IP (2) |
| The limit checks were performed even when the new pipe capacity was |
| less than the existing pipe capacity. |
| This could lead to problems if a user set a large pipe capacity, |
| and then the limits were lowered, with the result that the user could |
| no longer decrease the pipe capacity. |
| |
| Starting with Linux 4.9, checks against the limits |
| are performed only when increasing a pipe's capacity; |
| an unprivileged user can always decrease a pipe's capacity. |
| .IP (3) |
| The accounting and checking against the limits were done as follows: |
| |
| .RS |
| .PD 0 |
| .IP (a) 4 |
| Test whether the user has exceeded the limit. |
| .IP (b) |
| Make the new pipe buffer allocation. |
| .IP (c) |
| Account new allocation against the limits. |
| .PD |
| .RE |
| .IP |
| This was racey. |
| Multiple processes could pass point (a) simultaneously, |
| and then allocate pipe buffers that were accounted for only in step (c), |
| with the result that the user's pipe buffer |
| allocation could be pushed over the limit. |
| |
| Starting with Linux 4.9, |
| the accounting step is performed before doing the allocation, |
| and the operation fails if the limit would be exceeded. |
| .PP |
| Before Linux 4.9, bugs similar to points (1) and (3) could also occur |
| when the kernel allocated memory for a new pipe buffer; |
| that is, when calling |
| .BR pipe (2) |
| and when opening a previously unopened FIFO. |
| .SH SEE ALSO |
| .BR mkfifo (1), |
| .BR dup (2), |
| .BR fcntl (2), |
| .BR open (2), |
| .BR pipe (2), |
| .BR poll (2), |
| .BR select (2), |
| .BR socketpair (2), |
| .BR splice (2), |
| .BR stat (2), |
| .BR mkfifo (3), |
| .BR epoll (7), |
| .BR fifo (7) |