pv/docs/pv.1

.TH PV 1 "June 2024" Linux "User Manuals"
.SH NAME
pv \- monitor the progress of data through a pipe
.SH SYNOPSIS
.B pv
[\fIOPTION\fR]
[\fIFILE\fR]...
.br
.B pv
[\fI\-h\fR|\fI\-V\fR]


.SH DESCRIPTION
.B pv
shows the progress of data through a pipeline by giving information such as
time elapsed, percentage completed (with progress bar), current throughput
rate, total data transferred, and ETA.

To use it, insert it in a pipeline between two processes, with the
appropriate options.  Its standard input will be passed through to its
standard output and progress will be shown on standard error.

.B pv
will copy each supplied
.I FILE
in turn to standard output
.BR "" "(" -
means standard input), or if no
.IR FILE s
are specified just standard input is copied. This is the same behaviour
as
.BR cat (1).

A simple example to watch how quickly a file is transferred using
.BR nc (1):

.RS
.B pv file | nc -w 1 somewhere.com 3000
.RE

A similar example, transferring a file from another process and passing the
expected size to
.BR pv :

.RS
.B cat file | pv -s 12345 | nc -w 1 somewhere.com 3000
.RE

A more complicated example using numeric output to feed into the
.BR dialog (1)
program for a full-screen progress display:
 
.RS
.B (tar cf - . \e
.br
.B " | pv -n -s $(du -sb . | awk '{print $1}') \e"
.br
.B " | gzip -9 > out.tgz) 2>&1 \e"
.br
.B | dialog --gauge 'Progress' 7 70
.RE

Taking an image of a disk, skipping errors:

.RS
.B pv -EE /dev/your/disk/device > disk-image.img
.RE

Writing an image back to a disk:

.RS
.B pv disk-image.img > /dev/your/disk/device
.RE

Zeroing a disk:

.RS
.B pv < /dev/zero > /dev/your/disk/device
.RE

Note that if the input size cannot be calculated, and the output is a block
device, then the size of the block device will be used and
.B pv
will automatically stop at that size as if
.B \-S
had been given.

(Linux only): Watching file descriptor 3 opened by another process 1234:

.RS
.B pv -d 1234:3
.RE

(Linux only): Watching all file descriptors used by process 1234:

.RS
.B pv -d 1234
.RE


.SH OPTIONS
.B pv
takes many options, which are divided into display switches, output
modifiers, and general options.


.SH DISPLAY SWITCHES
If no display switches are specified,
.B pv
behaves as if
.BR \-p ", " \-t ", " \-e ", " \-r ", and " \-b
had been given (i.e. everything except average rate is switched on).
Otherwise, only those display types that are explicitly switched on will be
shown.
.TP
.B \-p, \-\-progress
Turn the progress bar on.  If any inputs are not files, or are
unreadable, and no size was explicitly given (with the
.B \-s
modifier), the progress bar cannot indicate how close to completion the
transfer is, so it will just move left and right to indicate that data is
moving.
.TP
.B \-t, \-\-timer
Turn the timer on.  This will display the total elapsed time that
.B pv
has been running for.
.TP
.B \-e, \-\-eta
Turn the ETA timer on.  This will attempt to guess, based on current
transfer rates and the total data size, how long it will be before
completion.  This option will have no effect if the total data size cannot
be determined.
.TP
.B \-I, \-\-fineta
Turn the ETA timer on, but display the estimated local time of arrival
instead of time left.  When the estimated time is more than 6 hours in the
future, the date is shown as well.
.TP
.B \-r, \-\-rate
Turn the rate counter on.  This will display the current rate of data
transfer.
.TP
.B \-a, \-\-average\-rate
Turn the average rate counter on.  This will display the current average
rate of data transfer (default: last 30s, see
.BR \-m ).
.TP
.B \-b, \-\-bytes
Turn the total byte counter on.  This will display the total amount of
data transferred so far.
.TP
.B \-8, \-\-bits
Display the total bits instead of the total bytes.  The output suffix will
be "b" instead of "B".
.TP
.B \-k, \-\-si
Display and interpret suffixes as multiples of 1000 rather than the default
of 1024.  Note that this only takes effect on options after this one, so for
consistency, specify this option first.
.TP
.B \-T, \-\-buffer\-percent
Turn on the transfer buffer percentage display.  This will show the
percentage of the transfer buffer in use - but see the caveat under
.B %T
in the
.B FORMATTING
section below.  Implies
.BR \-C .
.TP
.BI \-A\  NUM \fR,\ \fB\-\-last\-written\  NUM
Show the last
.I NUM
bytes written - but see the caveat under
.B %nA
in the
.B FORMATTING
section below.  Implies
.BR \-C .
.TP
.BI \-F\  FORMAT \fR,\ \fB\-\-format\  FORMAT
Ignore the options
.BR \-p ,
.BR \-t ,
.BR \-e ,
.BR \-r ,
.BR \-a ,
.BR \-b ,
.BR \-T ,
and
.BR \-A ,
and instead use the format string
.I FORMAT
to determine the output format.  See the
.B FORMATTING
section below.
.TP
.B \-n, \-\-numeric
Numeric output.  Instead of giving a visual indication of progress,
.B pv
will give an integer percentage, one per line, on standard error, suitable
for piping (via convoluted redirection) into
.BR dialog (1).
Note that
.B \-f
is not required if
.B \-n
is being used.
.TP
.B ""
Note that if
.B \-\-numeric
is in use, then adding
.B \-\-bytes
will cause the number of bytes processed so far to be output instead of a
percentage; if
.B \-\-line\-mode
is also in use as well as
.B \-\-bytes
and
.BR \-\-numeric ,
then instead of bytes or a percentage, the number of lines
so far is output.  And finally, if
.B \-\-timer
is added to
.BR \-\-numeric ,
then each output line is prefixed with the elapsed time 
so far, as a decimal number of seconds.
.TP
.B \-q, \-\-quiet
No output.  Useful if the
.B \-L
option is being used on its own to just limit the transfer rate of a pipe.


.SH OUTPUT MODIFIERS
.TP
.B \-W, \-\-wait
Wait until the first byte has been transferred before showing any progress
information or calculating any ETAs.  Useful if the program you are piping to
or from requires extra information before it starts, eg piping data into
.BR gpg (1)
or
.BR mcrypt (1)
which require a passphrase before data can be processed.
.TP
.BI \-D\  SEC \fR,\ \fB\-\-delay\-start\  SEC
Wait until 
.I SEC
seconds have passed before showing any progress information, for example in
a script where you only want to show a progress bar if it starts taking a
long time.  Note that this can be a decimal such as 0.5.
.TP
.BI \-s\  SIZE \fR,\ \fB\-\-size\  SIZE
Assume the total amount of data to be transferred is
.I SIZE
bytes when calculating percentages and ETAs.  The same suffixes of "k", "m"
etc can be used as with
.BR -L .
.TP
.B ""
If
.I SIZE
starts with
.BR "@" ,
the size of file whose name follows the
.B @
will be used.
.TP
.B ""
Note that
.B \-\-size
has no effect if used with
.BI \-d\  PID
to watch all file descriptors of a process, but will work with
.BR "-d PID:FD" .
.TP
.B \-l, \-\-line\-mode
Instead of counting bytes, count lines (newline characters). The progress
bar will only move when a new line is found, and the value passed to the
.B \-s
option will be interpreted as a line count.
.TP
.B ""
If this option is used without
.BR \-s ,
the "total size" (in this case, total line count) is calculated by reading
through all input files once before transfer starts.  If any inputs are
pipes or non-regular files, or are unreadable, the total size will not be
calculated.
.TP
.B \-0, \-\-null
Count lines as terminated with a zero byte instead of with a newline.
This option implies \-\-line\-mode.
.TP
.BI \-i\  SEC \fR,\ \fB\-\-interval\  SEC
Wait
.I SEC
seconds between updates.  The default is to update every second.
Note that this can be a decimal such as 0.1.
.TP
.BI \-m\  SEC \fR,\ \fB\-\-average-rate-window\  SEC
Compute current average rate over a
.I SEC
seconds window for average rate and ETA calculations (default 30 seconds).
.TP
.BI \-w\  WIDTH \fR,\ \fB\-\-width\  WIDTH
Assume the terminal is
.I WIDTH
characters wide, instead of trying to work it out (or assuming 80 if it
cannot be guessed).  If this option is used, the output width will not be
adjusted if the width of the terminal changes while the transfer is running.
.TP
.BI \-H\  HEIGHT \fR,\ \fB\-\-height\  HEIGHT
Assume the terminal is
.I HEIGHT
rows high, instead of trying to work it out (or assuming 25 if it
cannot be guessed).  If this option is used, the output height will not be
adjusted if the height of the terminal changes while the transfer is
running.
.TP
.BI \-N\  NAME \fR,\ \fB\-\-name\  NAME
Prefix the output information with
.IR NAME .
Useful in conjunction with
.B \-c
if you have a complicated pipeline and you want to be able to tell different
parts of it apart.
.TP
.B \-f, \-\-force
Force output.  Normally,
.B pv
will not output any visual display if standard error is not a terminal. 
This option forces it to do so.
.TP
.B \-c, \-\-cursor
Use cursor positioning escape sequences instead of just using carriage
returns.  This is useful in conjunction with
.B \-N
(name) if you are using multiple
.B pv
invocations in a single, long, pipeline.


.SH DATA TRANSFER MODIFIERS
.TP
.BI \-o\  FILE \fR,\ \fB\-\-output\  FILE
Write data to
.I FILE
rather than standard output.  If the file already exists, it will be
truncated.
.TP
.BI \-L\  RATE \fR,\ \fB\-\-rate-limit\  RATE
Limit the transfer to a maximum of
.I RATE
bytes per second.  A suffix of "K", "M", "G", or "T" can be added to denote
kibibytes (*1024), mebibytes, and so on.  If
.B \-\-si
was also passed, suffixes will denote kilobytes (*1000), megabytes, etc. 
Note the caveat about the positioning of
.B \-\-si .
.TP
.BI \-B\  BYTES \fR,\ \fB\-\-buffer-size\  BYTES
Use a transfer buffer size of
.I BYTES
bytes.  A suffix of "K", "M", "G", or "T" can be added to denote
kibibytes (*1024), mebibytes, and so on.  The default buffer size is the
block size of the input file's filesystem multiplied by 32 (512KiB max), or
400KiB if the block size cannot be determined.  This can be useful on
platforms like MacOS which perform better in pipelines with specific buffer
sizes such as 1024.  Implies
.BR \-C .
.TP
.B \-C, \-\-no-splice
Never use
.BR splice (2),
even if it would normally be possible.  The
.BR splice (2)
system call is a more efficient way of transferring data from or to a pipe
than regular
.BR read (2)
and
.BR write (2),
but means that the transfer buffer may not be used.  This prevents
.B \-A
and
.B \-T
from working, cannot work with
.BR \-X ,
and makes
.B \-B
redundant, so using
.BR \-A ,
.BR \-T ,
.BR \-X ,
or
.B \-B
automatically switches on
.BR \-C .
Switching on
.B \-C
results in a small loss of transfer efficiency.
(This option has no effect on systems where
.BR splice (2)
is unavailable).
.TP
.B \-E, \-\-skip-errors
Ignore read errors by attempting to skip past the offending sections.  The
corresponding parts of the output will be null bytes.  At first only a few
bytes will be skipped, but if there are many errors in a row then the skips
will move up to chunks of 512.  This is intended to be similar to
.B dd conv=sync,noerror
but has not been as thoroughly tested.
.TP
.B ""
Specify
.B \-E
twice to only report a read error once per file, instead of reporting each
byte range skipped.
.TP
.BI \-Z\  BYTES \fR,\ \fB\-\-error\-skip\-block\  BYTES
When ignoring read errors with
.BR \-E ,
instead of trying to adaptively skip by reading small amounts and skipping
progressively larger sections until a read succeeds, move to the next file
block of
.I BYTES
bytes as soon as an error occurs.  There may still be some shorter skips
where the block being skipped coincides with the end of the transfer buffer.
.TP
.B ""
This option can only be used with
.B \-E
and is intended for use when reading from a block device, such as
.B \-E\ \-Z\ 4K
to skip in 4 kibibyte blocks.  This will speed up reads from faulty media,
at the expense of potentially losing more data.
.TP
.B \-S, \-\-stop-at-size
If a size was specified with
.BR \-s ,
stop transferring data once that many bytes have been written, instead of
continuing to the end of input.
.TP
.B \-Y, \-\-sync
After every write operation, synchronise the buffer caches to disk - see
.BR fdatasync (2).
This has no effect when the output is a pipe.  Using
.B \-Y
may improve the accuracy of the progress bar when writing to a slow disk.
.TP
.B \-K, \-\-direct-io
Set the
.B O_DIRECT
flag on all inputs and outputs, if it is available.  This will minimise
the effect of caches, at the cost of performance.  Due to memory alignment
requirements, it also may cause read or write failures with an error of
"Invalid argument", especially if reading and writing files across a variety
of filesystems in a single
.B pv
call.  Use this option with caution.
.TP
.B \-X, \-\-discard
Instead of transferring input data to standard output, discard it.  This is
equivalent to redirecting standard output to
.IR /dev/null ,
except that
.BR write (2)
is never called.  Implies
.BR \-C .
.TP
.BI \-d\  PID\fR[\fB:\fR\fIFD\fR],\  \fB\-\-watchfd\  PID\fR[\fB:\fR\fIFD\fR]
Instead of transferring data, watch file descriptor
.I FD
of process
.IR PID ,
and show its progress.  The
.B pv
process will exit when
.I FD
either changes to a different file, changes read/write mode, or is closed;
other data transfer modifiers - and remote control - may not be used with
this option.
.TP
.B ""
If only a
.I PID
is specified, then that process will be watched, and all regular files and
block devices it opens will be shown with a progress bar.  The
.B pv
process will exit when process
.I PID
exits.
.TP
.BI \-R\  PID \fR,\ \fB\-\-remote\  PID
If
.I PID
is an instance of
.B pv
that is already running,
.BI \-R\  PID
will cause that instance to act as though it had been given
this instance's command line instead.  For example, if
.B pv -L 123K
is running with process ID 9876, then running
.B pv -R 9876 -L 321K
will cause it to start using a rate limit of 321KiB instead of 123KiB.
Note that some options cannot be changed while running, such as
.BR \-c ,
.BR \-l ,
.BR \-f ,
.BR \-D ,
.BR \-E ,
and
.BR \-S .

.SH GENERAL OPTIONS
.TP
.BI \-P\  FILE \fR,\ \fB\-\-pidfile\  FILE
Save the process ID of
.B pv
in
.IR FILE .
The file will be replaced if it already exists, and will be removed when
.B pv
exits.  While
.B pv
is running, it will contain a single number - the process ID of
.B pv
- followed by a newline.
.TP
.B \-h, \-\-help
Print a usage message on standard output and exit successfully.
.TP
.B \-V, \-\-version         
Print version information on standard output and exit successfully.


.SH FORMATTING
If the
.B \-F
option is given, then the output format is determined by the given format
string.  Within that string, the following sequences can be used:
.TP
.B %p
Progress bar.
Expands to fill the remaining space. Should only be specified once.
Equivalent to
.BR \-p .
.TP
.B %t
Elapsed time.  Equivalent to
.BR \-t .
.TP
.B %e
ETA as time remaining.  Equivalent to
.BR \-e .
.TP
.B %I
ETA as local time of completion.  Equivalent to
.BR \-I .
.TP
.B %r
Current data transfer rate.  Equivalent to
.BR \-r .
.TP
.B %a
Average data transfer rate.  Equivalent to
.BR \-a .
.TP
.B %b
Bytes transferred so far (or lines if
.B \-l
was specified).  Equivalent to
.BR \-b .
If
.B \-\-bits
was specified, 
.B %b
shows the bits transferred so far, not bytes.
.TP
.B %T
Percentage of the transfer buffer in use.  Equivalent to
.BR \-T .
Shows "{----}" if the transfer is being done with
.BR splice (2),
since splicing to or from pipes does not use the buffer.
.TP
.B %nA
Show the last 
.B n
bytes written (e.g.
.B %16A
for the last 16 bytes).
Shows only dots if the transfer is being done with
.BR splice (2),
since splicing to or from pipes does not use the buffer.
.TP
.B %N
Name prefix given by
.BR -N .
Padded to 9 characters with spaces, and suffixed with :.
.TP
.B %%
A single %.

.P
The format string equivalent of turning on all display switches is
.BR "\`%N %b %T %t %r %a %p %e'" .


.SH COMMON SWITCHES
Some suggested common switch combinations:

.TP
.B pv -ptebar
Show a progress bar, elapsed time, estimated completion time, byte counter,
average rate, and current rate.
.TP
.B pv -betlap
Show a progress bar, elapsed time, estimated completion time, line counter,
and average rate, counting lines instead of bytes.
.TP
.B pv -t
Show only the elapsed time - useful as a simple timer, e.g.
.BR "sleep 10m | pv -t" .
.TP
.B pv -pterb
The default behaviour: progress bar, elapsed time, estimated completion
time, current rate, and byte counter.

.P
On MacOS, it may be useful to specify
.B -B 1024
in a pipeline, as this may improve performance.


.SH EXIT STATUS
An exit status of 1 indicates a problem with the
.B \-R
or
.B \-P
options.

Any other exit status is a bitmask of the following:

.TP
.B 2
One or more files could not be accessed, 
.BR stat (2)ed,
or opened.
.TP
.B 4
An input file was the same as the output file.
.TP
.B 8
Internal error with closing a file or moving to the next file.
.TP
.B 16
There was an error while transferring data from one or more input files.
.TP
.B 32
A signal was caught that caused an early exit.
.TP
.B 64
Memory allocation failed.

.P
A zero exit status indicates no problems.


.SH ENVIRONMENT
The following environment variables may affect
.BR pv :

.TP
.B HOME
The current user's home directory.  This may be used by the remote control
mechanism
(the
.B \-\-remote
option)
to exchange messages between
.B pv
instances: if the
.I /run/user/UID/
directory does not exist (where
.I UID
is the current user ID), then
.I $HOME/.pv/
will be used instead.
.TP
.BR TMPDIR ", " TMP
The directory to create per-tty lock files for the terminal when using the
.B \-\-cursor
option.  If
.B TMPDIR
is set to a non-empty value, it is the directory under which lock files are
created.  Otherwise, if
.B TMP
is set, then it is used; and if neither are set, then
.I /tmp
is used.


.SH AUTHOR
Written by Andrew Wood, with patches submitted by various other people. 
Please see the package's ACKNOWLEDGEMENTS file for a complete list of
contributors.


.SH KNOWN PROBLEMS
The following problems are known to exist in
.BR pv :
.IP \(bu 3
In some versions of
.BR bash (1)
and
.BR zsh (1),
the construct
.B <(pv filename)
will not output any progress to the terminal when run from an interactive
shell, due to the subprocess being run in a separate process group from the
one that owns the terminal.  In these cases, use
.BR \-\-force .
.P
.IP \(bu 3
The
.B -c
option does not work properly on Cygwin without
.B cygserver
running, if started near the bottom of the screen (IPC is needed to handle
the terminal scrolling).  To fix this, start
.B cygserver
before using
.BR "pv -c" .
.P
.IP \(bu 3
The
.B -R
option requires that either
.B /run/user/<uid>/
or
.B $HOME/
can be written to, for inter-process communication.
.P
If you find any other problems, please report them.


.SH REPORTING BUGS
Please report any bugs to
.BR pv@ivarch.com .

Alternatively, use the issue tracker linked from the
.B pv
home page: <https://www.ivarch.com/programs/pv.shtml>


.SH "SEE ALSO"
.BR cat (1),
.BR dialog (1),
.BR splice (2),
.BR open (2)
(for
.BR O_DIRECT )

.SH COPYRIGHT
Copyright \(co 2002-2008, 2010, 2012-2015, 2017, 2021, 2023-2024 Andrew Wood.

License GPLv3+: GNU GPL version 3 or later <https://www.gnu.org/licenses/gpl-3.0.html>.

This is free software: you are free to change and redistribute it.  There is
NO WARRANTY, to the extent permitted by law.