New --store-and-forward option (#100), including refactoring several areas to allow PV to run in two passes, and the ability to look at the output pipe buffer utilisation to more accurately reflect the total data transferred to the receiving end, rather than just the total amount written to the output buffer.

docs/NEWS.md

@@ -1,5 +1,6 @@
 ### UNRELEASED
 
+ * feature: new "`--store-and-forward`" option to read input to a file first, then write it to the output ([#100](https://codeberg.org/a-j-wood/pv/issues/100))
  * feature: new "`--stats`" option to show transfer stats at the end, like "`ping`" ([#49](https://codeberg.org/a-j-wood/pv/issues/49))
  * feature: "`--rate`" can now be used with "`--numeric`" ([#17](https://codeberg.org/a-j-wood/pv/issues/17))
  * feature: "`--gauge`" with "`--progress`" to show rate gauge when size is unknown ([#46](https://codeberg.org/a-j-wood/pv/issues/46))

docs/pv.1

@@ -491,6 +491,30 @@ except that
 is never called.  Implies
 .BR \-C .
 .TP
+.BI \-U\  FILE \fR,\ \fB\-\-store\-and\-forward\  FILE
+Instead of passing data through immediately, do it in two stages - first
+read all input and write it to
+.IR FILE ,
+and then once the input is exhausted, read all of
+.I FILE
+and write it to the output.
+.I FILE
+remains in place afterwards, unless it is
+"\fB-\fR", in which case
+.B pv
+creates a temporary file for this purpose, and automatically removes it
+afterwards.
+.TP
+.B ""
+This can be useful if you have a pipeline which generates data (your
+input) quickly but you don't know the size, and you wish to pass it to some
+slower process, once all of the input has been generated and you know its
+size, so you can see its progress.  Note that when doing this with
+relatively small amounts of data,
+.B \-\-no-splice
+may be preferable so that pipe buffering doesn't affect the progress
+display.
+.TP
 .BI \-d\  PID\fR[\fB:\fR\fIFD\fR],\  \fB\-\-watchfd\  PID\fR[\fB:\fR\fIFD\fR]
 Instead of transferring data, watch file descriptor
 .I FD

docs/pv.1.md

@@ -364,6 +364,23 @@ display types that are explicitly switched on will be shown.
     This is equivalent to redirecting standard output to */dev/null*,
     except that **write**(2) is never called. Implies **-C**.
 
+**-U FILE, \--store-and-forward FILE**
+
+:   Instead of passing data through immediately, do it in two stages -
+    first read all input and write it to *FILE*, and then once the input
+    is exhausted, read all of *FILE* and write it to the output. *FILE*
+    remains in place afterwards, unless it is \"**-**\", in which case
+    **pv** creates a temporary file for this purpose, and automatically
+    removes it afterwards.
+
+:   This can be useful if you have a pipeline which generates data (your
+    input) quickly but you don\'t know the size, and you wish to pass it
+    to some slower process, once all of the input has been generated and
+    you know its size, so you can see its progress. Note that when doing
+    this with relatively small amounts of data, **\--no-splice** may be
+    preferable so that pipe buffering doesn\'t affect the progress
+    display.
+
 **-d PID\[:FD\], \--watchfd PID\[:FD\]**
 
 :   Instead of transferring data, watch file descriptor *FD* of process