Original problem reported for HP-UX LVM v2.2 by Michael White in
<http://lists.gnu.org/archive/html/bug-tar/2012-10/msg00000.html>.
This patch fixes some other gotchas that I noticed.
* gnulib.modules: Add extern-inline.
* src/common.h: Use _GL_INLINE_HEADER_BEGIN, _GL_INLINE_HEADER_END.
(COMMON_INLINE, max, min): New macros.
(represent_uintmax, valid_timespec): New inline functions.
(SYSINT_BUFSIZE): New constant.
(sysinttostr, strtosysint, decode_timespec): New decls.
* src/create.c (start_private_header): Silently bring the time_t
value into range; it is now the caller's responsibility to deal
with any overflow error. Use uid 0 and gid 0 rather than the
user's uid/gid, since the faked header isn't "owned" by the user
and the uid/gid could in theory be out of range. Leave major and
minor zeroed.
(FILL): Remove.
(write_gnu_long_link): Let start_private_header zero things out.
* src/create.c (write_gnu_long_link, write_extended):
* src/xheader.c (xheader_write_global):
Use start_time, not current time; no point hammering on the clock.
* src/compare.c (diff_multivol): Check that offset, size are in range.
* src/incremen.c (read_incr_db_01, write_directory_file_entry):
Allow negative time_t, dev_t, and ino_t.
* src/list.c (max): Remove (moved to common.h).
(read_header): Check that size is in range.
(from_header): Return intmax_t, not uintmax_t, to allow negative.
All callers changed. At compile time, check assumptions about
intmax_t and uintmax_t. Use bool for booleans. Avoid overflow
hassles on picky hosts.
(mode_from_header): Last arg is now bool *, not unsigned *.
All callers changed.
(simple_print_header): Do not assume UID, GID fit in 'long'.
* src/list.c (from_header):
* src/xheader.c (out_of_range_header):
Arg is now a plain minimum value, not minus minval converted to
uintmax_t. All callers changed.
* src/misc.c (COMMON_INLINE): New macro.
(sysinttostr, strtosysint, decode_timespec): New functions.
* src/sparse.c (oldgnu_add_sparse, oldgnu_fixup_header)
(star_fixup_header):
Check for offset overflow.
(decode_num): Clear errno before calling strtoumax.
* src/tar.c (expand_pax_option): Don't discard nanoseconds.
* src/xheader.c (assign_time_option): Allow negative time_t.
(decode_record): Simplify, since out-of-range string is guaranteed
to produce a value exceeding len_max.
(xheader_read): Last arg is off_t, not size_t.
Caller should diagnose negative arg, as needed.
Check that it's in range.
(enum decode_time_status): Remove.
(_decode_time): Remove, folding into decode_time.
(decode_time): Return bool, not enum decode_time_status.
Rely on decode_timespec to do most of the work.
(code_signed_num): New function.
(code_num): Use it.
(decode_signed_num): New function.
(decode_num): Use it.
(gid_coder, gid_decoder, uid_coder, uid_decoder, sparse_map_decoder)
(sparse_map_decoder): Code and decode negative values.
(sparse_map_decoder): Improve check for out-of-range values.
* tests/time01.at: New file.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* tests/testsuite.at: Include it.
This has a similar meaning as in other GNU applications
such as coreutils and Emacs.
* NEWS: Document it.
* .gitignore: Remove redundant build-aux.
Remove gnu, since gnu/Makefile.am is now in git.
Add gnu/.gitignore, gnu/charset.alias, gnu/*.h, gnu/*/ to cover
autogenerated files.
* bootstrap.conf (gnulib_mk): Remove.
* configure.ac: Add support for --enable-gcc-warnings,
taken from coreutils and simplified.
* gnu/Makefile.am: New file. Formerly this was autogenerated,
but the autogenerated file has been renamed to gnulib.mk,
its usual name when bootstrapping from gnulib.
This way, AM_CFLAGS can incorporate warning options.
* gnulib.modules: Add manywarnings.
* lib/Makefile.am, src/Makefile.am (AM_CFLAGS): New macro,
incorporating warning options.
* lib/attr-xattr.in.h (ENOATTR): New macro, if not already defined.
* src/buffer.c (magic): Don't rely on incomplete initializers.
* src/common.h (report_difference): Add printf format attribute.
* src/system.c (sys_exec_command, sys_exec_info_script)
(sys_exec_checkpoint_script):
* src/tar.c (update_argv):
Add casts to char * to pacify GCC warnings about using string
literals in a char * context.
* src/xattrs.c, src/xattrs.h (xattrs_clear_setup):
Declare parameters as (void), not ().
* src/xheader.c (xheader_format_name): Initialize pptr to null,
to pacify GCC. Remove unnecessary test of nptr versus null.
* src/xheader.c (xattr_decode_keyword)
(xattr_encode_keyword): New functions.
(xheader_print_n,xattr_decoder): Use them.
* tests/xattr05.at: New test case.
* tests/Makefile.am: Add new test case.
* tests/testsuite.at: Likewise.
* configure.ac: Check whether ACLs are available on the
host system.
* gnulib.modules: Add acl
* src/create.c (start_header): Store ACLs when creating
a POSIX archive.
(dump_file0): Handle ACLs.
* src/extract.c (delayed_set_stat) <acls_a_ptr, acls_a_len>
<acls_d_ptr, acls_d_len>: New members.
(delayed_link): Likewise.
(set_stat,delay_set_stat)
(apply_nonancestor_delayed_set_stat): Set ACLs.
* src/tar.c: New options: "--acls", "--no-acls"
(tar_stat_destroy): Free acls_a_ptr and acls_d_ptr fields.
* src/tar.h (tar_stat_info) <acls_a_ptr, acls_a_len>
<acls_d_ptr, acls_d_len>: New members.
* src/xattrs.c (xattrs_acls_get, xattrs_acls_set): New functions.
* src/xheader.c: Support new keywors: "SCHILY.acl.access" and
"SCHILY.acl.default".
* tests/Makefile.am: Add new tests.
* tests/testsuite.at: Likewise.
* tests/acls01.at: New test.
* tests/acls02.at: New test.
* src/Makefile.am: Add xattrs.[ch]
* src/xattrs.c: New file.
* src/xattrs.h: New file.
* src/common.h (READ_LIKE_SUBCOMMAND): New define.
(selinux_context_option, acls_option, xattrs_option): New globals.
(xheader_xattr_init, xheader_xattr_free)
(xheader_xattr_copy, xheader_xattr_add): New protos.
(WARN_XATTR_WRITE): New mask.
* src/create.c (start_header): Handle xattrs pairs if in POSIX format.
(dump_file0): Handle extended attributes.
* src/extract.c (delayed_set_stat,delayed_link) <xattr_map_size>
<xattr_map>: New members.
(set_xattr): New static function.
(open_output_file): Accept an additional argument indicating
whether the file has already been created.
(set_stat,delay_set_stat)
(apply_nonancestor_delayed_set_stat)
(extract_file): Handle extended attributes.
* src/list.c (decode_header, simple_print_header): Display
extended attributes.
* src/tar.c: New options --xattrs, --no-xattrs, --xattrs-include,
--xattrs-exclude
(tar_stat_destroy): Free the xattr_map storage.
* src/tar.h (xattr_array): New struct.
(tar_stat_info) <xattr_map_size, xattr_map>: New members.
* src/warning.c: New warning control keyword "xattr-write".
* src/xheader.c (xheader_xattr_init)
(xheader_xattr_free, xheader_xattr_add)
(xheader_xattr_copy): New functions.
(struct xhdr_tab) <prefix>: New member.
(locate_handler): Permit selecting the keyword based on its
prefix.
(xheader_protected_pattern_p)
(xheader_protected_keyword_p): Likewise.
(xattr_coder, xattr_decoder): New functions.
(xhdr_tab): Reflect the changes to struct xhdr_tab.
New keyword "SCHILY.xattr".
* tests/Makefile.am: Add new tests.
* tests/testsuite.at: Likewise.
(AT_CHECK_UTIL, AT_XATTRS_UTILS_PREREQ)
(AT_CAPABILITIES_UTILS_PREREQ, AT_XATTRS_PREREQ): New defuns.
* tests/xattr01.at: New test.
* tests/xattr02.at: New test.
* tests/xattr03.at: New test.
* tests/xattr04.at: New test.
* tests/capabs_raw01.at: New test.
* src/buffer.c (match_volume_label): Call set_volume_label.
(check_label_pattern): Get label string
as argument.
(match_volume_label): Handle volume labels stored in
global PAX headers.
* src/common.c (print_header,read_header): Change signature.
(read_header_primitive): Remove prototype.
* src/list.c (recent_global_header): New static.
(list_archive): Always print volume labels.
(read_header_primitive): Remove.
(read_header): Change the signature (all callers updated)
Save the recent global header.
(volume_label_printed): New static.
(simple_print_header): New function (ex-print_header).
(print_header): Change the signature (all callers updated).
For POSIX formats, print first volume header (if set).
* src/xheader.c (xheader_write_global): Write the data
accumulated in xhdr->stk even if keyword_global_override_list
is empty.
(xheader_read): On unexpected EOF, report error instead of
coredumping.
(XHDR_PROTECTED, XHDR_GLOBAL): New defines.
(struct xhdr_tab): Remove `protected' with `flags'. All uses updated.
(decg): If XHDR_GLOBAL bit is set, call the keyword's decode
method instead of adding it to `kwl'.
* src/compare.c: Update calls to read_header.
* src/create.c: Likewise.
* src/delete.c: Likewise.
* src/update.c: Likewise.
* src/extract.c: Likewise.
(extract_volhdr): Do not print "Reading <label>" statement, because
it is inconsistent: it is not printed if the volume begins with a
member continued from the previous volume.
* tests/label01.at: New testcase.
* tests/label02.at: New testcase.
* tests/Makefile.am, tests/testsuite.at: Add new testcases.
* src/tar.c (struct textual_date): ts is a copy of the structure,
not a pointer to it. Date is a copy as well, hence the `const' is
taken away.
(get_date_or_file): Return 0/1 depending on success/failure.
Copy timestamp to the `ts' member. Store a copy of the string
in `date'.
(report_textual_dates): Report only if verbose_option is set,
but always free the list.
(expand_pax_option): New function.
(parse_opt): Preprocess the argument to xheader_set_option with
expand_pax_option.
(decode_options): Call report_textual_dates unconditionally.
* src/xheader.c (exthdr_mtime_option, exthdr_mtime)
(globexthdr_mtime_option, globexthdr_mtime): New statics.
(xheader_set_keyword_equal): handle exthdr.mtime and globexthdr.mtime.
(xheader_write): Override `t' argument if a corresponding
exthdr.mtime or globexthdr.mtime option is set.
* NEWS: Update
* doc/tar.texi: Document the changes.
This makes two pax archives binary equivalent if they
have the same contents and care is taken to make extended
headers otherwise reproducible, e.g. by using:
--pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0
Proposed by Michael D. Adams <mdmkolbe@gmail.com>.
* src/common.h (start_private_header): Take time_t as 3rd param.
(xheader_write): Likewise.
* src/create.c (start_private_header): Take time_t as 3rd param.
All callers updated.
(write_extended): Use file's mtime as mtime for its extended header,
Use current time stamp as mtime for global headers.
(xheader_write): Take time_t as 3rd param.
(xheader_decode_global,xheader_store,xheader_read)
(xheader_write_global,xheader_write,xheader_string_begin)
(xheader_string_add,xheader_string_end): Take xhdr as first
argument.
* src/common.h (add_exclude_tag): Add decl; C99 requires this
and Forte warns about it.
* src/incremen.c: Include <mkdtemp.h> for mkdtemp prototype,
for same reason.
* src/misc.c (get_max_open_files): Rewrite to avoid code that
Forte C complains about as being unreachable.
* src/xheader.c (mtime_code): Rewrite to avoid Forte error
reported by Trond Hasle Amundsen.
* src/incremen.c (compare_dirnames): Rewrite to avoid casts.
* src/utf8.c (string_ascii_p): Likewise.
* src/xheader.c (mtime_coder, volume_size_coder, volume_offset_coder):
Likewise.
SCALAR(0x8283218)
argument. All callers changed.
(decode_record): Check for numeric overflow
(xheader_string_end): Return boolean value. Check for possible
numeric overflow
qualifier. Change last argument.
Correct buffer size calculation (allocated too much space).
(xheader_write): Increase global_header_count here ...
(xheader_write_global): ... instead of here
qualifier.
(struct xhdr_tab.decoder): Change prototype. POSIX allows string
values to contain embedded nulls, so take an extra argument
specifying the length of the string.
(decx,decg,dummy_decoder,atime_decoder,gid_decoder)
(gname_decoder,linkpath_decoder,ctime_decoder,mtime_decoder)
(path_decoder,size_decoder,uid_decoder,uname_decoder)
(sparse_size_decoder,sparse_numblocks_decoder)
(sparse_offset_decoder,sparse_numbytes_decoder): Likewise.
(decode_record): Pass value length to the handler
(run_override_list): Pass value length to the decoder
(xheader_print_n): New function
(xheader_print): Rewritten using xheader_print_n
(xheader_finish): Do not rely om strlen to compute the length of
the collected string: it can contain embedded nulls
(xheader_string_begin,xheader_string_add,xheader_string_end): New
functions.
(sparse_map_decoder,dumpdir_coder,dumpdir_decoder): New
functions. Handle GNU.sparse.map and GNU.dumpdir variables.
(xhdr_tab): Add new variables.
by struct stat; keep them to full nanosecond resolution.
This affects behavior only on older hosts or file systems
that have lower-resolution time stamps.
* src/common.h (OLDER_STAT_TIME): Parenthesize arg.
(OLDER_TAR_STAT_TIME): New macro.
(code_timespec): New function.
(BILLION, LOG10_BILLION, TIMESPEC_STRSIZE_BOUND): New constants.
* src/compare.c (diff_file): Use full time stamp resolution.
* src/create.c (start_header, dump_file0): Likewise.
(start_header, dump_file0): Adjust to new structure layout.
(dump_regular_finish): Simplify by using timespec_cmp.
* src/extract.c (struct delayed_set_stat): Don't store stat info
that we don't need, to save space. All uses changed.
(struct delayed_set_stat, struct delayed_link, file_newer_p):
(create_placeholder_file, extract_link, apply_delayed_links):
Use full time stamp resolution.
(check_time): Use code_timespec rather than rolling our own code.
(set_stat, delay_set_stat): Arg now points to tar_stat_info to
avoid losing time information. All callers changed.
* src/list.c (read_and, decode_header, print_heaeder):
Use full time stamp resolution.
* src/misc.c (code_timespec): New function.
* src/tar.h (struct tar_stat_info): Record atime, mtime, ctime
separately, for benefit of hosts with lower resolution.
* src/update.c (update_archive): Use full time stamp resolution.
* src/xheader.c (code_time): Use new code_timespec function
to simplify code.
(atime_coder, atime_decoder, ctime_coder, ctime_decoder):
(mtime_coder, mtime_decoder): Use full time stamp resolution.
Report time stamps to full resolution in environment.
Report memory allocation failures rather than ignoring them.
* src/system.c (time_to_env): New function.
(oct_to_env, str_to_env, chr_to_env): Report memory allocation failures.
(stat_to_env): Report full resolution in time stamps.
* src/sparse.c: Include <inttostr.h>.
(struct tar_sparse_file): offset and dumped_size are off_t, not
size_t. optab is now const *.
(dump_zeros): Return bool success flag, not off_t.
All callers changed.
Use a constant-zero buffer rather than clearing a buffer each time.
Don't mess up if write fails.
(dump_zeros, check_sparse_region):
Don't assume off_t is no wider than size_t.
(tar_sparse_init): Don't bother clearing a field that is already clear.
(zero_block_p): First arg is const *, not *.
(clear_block, SPARSES_INIT_COUNT): Remove.
(sparse_add_map): First arg is now struct start_stat_info *, not
struct tar_sparse_file *. All callers changed.
Use x2nrealloc to check for size_t overflow.
(parse_scan_file): Cache commonly-used parts of file.
Use an auto buffer, not a static one.
Don't bother clearing the buffer; not needed.
Don't bother clearing items that are already clear.
(oldgnu_optab, star_optab, pax_optab): Now const.
(sparse_dump_region): Don't bother clearing the buffer before
reading into it; just clear the parts that aren't read into.
(sparse_dump_file): Clear the whole local variable 'file'.
(diff_buffer): Remove; now a local var.
(check_sparse_region): Don't bother clearing buffer before
reading into it. Don't assume off_t is promoted to long.
(oldgnu_get_sparse_info, star_get_sparse_info):
Use an auto status, not static.
* src/tar.h (struct tar_stat_info): had_trailing_slash is
now bool, not int.
* src/xheader.c (sparse_offset_coder, sparse_numbytes_coder):
Rewrite to avoid cast.
(sparse_offset_decoder, sparse_numbytes_decoder):
Diagnose excess entries rather than crashing.
Invalid header fields go undiagnosed.
Some valid time strings are ignored.
* NEWS: Better support for full-resolution time stamps.
The -v option now prints time stamps only to 1-minute resolution.
* gnulib.modules: Add utimens.
* lib/.cvsignore: Add imaxtostr.c, inttostr.c, inttostr.h,
offtostr.c, umaxtostr.c, utimens.c, utimens.h. Remove paxconvert.c.
* lib/Makefile.tmpl (libtar_a_SOURCES): Remove paxconvert.c.
* lib/paxconvert.c: Remove; superseded by umaxtostr.c.
* po/POTFILES.in: Remove lib/paxconvert.c. Add lib/xalloc-die.c,
lib/obstack.c.
* src/buffer.c (set_start_time, compute_duration, start_time):
Use gettime rather than rolling our own code.
* src/common.h (OLDGNU_NAME_FIELD_SIZE, MAXOCTAL11, MAXOCTAL7): Remove.
(newer_ctime_option): Remove.
(timespec_lt): New function.
(OLDER_STAT_TIME): Use it.
(string_to_chars): First arg is char const *, not char *.
(tartime): Time arg is now struct timespec. New bool arg.
All callers changed.
(code_ns_fraction): New decl.
(sys_stat_nanoseconds): Remove decl.
(get_stat_atime, get_stat_ctime, get_stat_mtime): New functions.
(set_stat_atime, set_stat_ctime, set_stat_mtime): New functions.
* src/compare.c: Include utimens.h rather than rolling our own.
(diff_dir, diff_file, diff_link, diff_symlink, diff_special):
Prototype.
(diff_dumpdir, diff_multivol): Prototype.
(diff_file): Support higher-resolution time stamps.
* src/create.c: Include utimens.h rather than rolling our own.
(MAX_OCTAL_VAL): New macro.
(tar_copy_str, string_to_chars): Don't bother to zero-fill;
the destination is already zeroed.
(string_to_chars): First arg is char const *.
(start_private_header): Use MINOR_TO_CHARS, not MAJOR_TO_CHARS,
for minor device number.
(write_header_name, dump_hard_link, dump_file0):
Simplify test for old GNU format.
(start_header): Put in placeholders for uid, etc., even when
using extended headers, for benefit of older "tar" implementations.
Don't assume uintmax_t is wider than 32 bits.
Output extended header for mtime if needed.
(dump_regular_finish, dump_file0):
Support extended time stamp resolution.
* src/extract.c: Include utimens.h rather than rolling our own.
(check_time): Support extended time stamp resolution.
* src/list.c: Include <inttostr.h>.
(tartime): Use umaxtostr rather than stringify_uintmax_t_backwards.
* src/xheader.c: Include <inttostr.h>.
Do not include <xstrtol.h>.
(strtoimax) [!HAVE_DECL_STRTOIMAX && !defined strtoimax]: New decl.
(strtoumax) [!HAVE_DECL_STRTOUMAX && !defined strtoumax]: New decl.
(BILLION, LOG10_BILLION): New constants.
(to_decimal): Remove; superseded by inttostr. All callers changed
to use umaxtostr.
(xheader_format_name): Don't assume pids and uintmax_t values
fit in 63 bytes (!) when printed.
(decode_record): Don't bother to check for ERANGE; an out of range
value must be treater than len_max anyway.
If the length is out of range, output it in the diagnostic.
(format_uintmax): Remove; all callers changed to use umaxtostr.
(xheader_print): Don't assume sizes can be printed in 99 bytes (!).
(out_of_range_header): New function.
(decode_time): Use it.
(code_time): Accept struct timespec, not time_t and unsigned long.
All callers changed. Size sbuf properly, and remove unnecessary check.
Don't assume time stamps can fit in 199 bytes.
Handle negative time stamps. Handle fractional time stamps
more consistently. Don't output unnecessary trailing zeros.
(decode_time): Yield struct timespec, not time_t and unsigned long.
All callers changed.
Handle negative time stamps. Truncate towards minus infinity
consistently. Improve overflow checks, and output a better
diagnostic on overflow.
(code_num): Don't assume uintmax_t can be printed in 99 bytes (!).
(decode_num): New function, for better diagnostics.
(atime_coder, atime_decoder, gid_decoder, ctime_coder):
(ctime_decoder, mtime_coder, mtime_decoder, size_decoder):
(uid_decoder, sparse_size_decoder, sparse_numblocks_decoder):
(sparse_offset_decoder, sparse_numbytes_decoder):
Use decode_num, etc., instead of xstrtoumax, etc.
* src/xheader.c (sparse_numblocks_decoder): Remove unchecked use
of `calloc'. Use xcalloc instead.
(decode_time, gid_decoder, size_decoder, uid_decoder):
(sparse_size_decoder, sparse_offset_decoder, sparse_numblocks_decoder):
Ensure that the result of calling xstrtoumax is no larger than
the maximum value for the target type. Upon any failure, exit with
a diagnostic.
(sparse_numblocks_decoder): Avoid buffer overrun/heap corruption:
use x2nrealloc, rather than `n *= 2' and xrealloc(p, n,....
(decode_time): Rewrite to accept time strings like
1119018481.000000000. Before, such strings were always ignored.
a corrupted extended header. Problem reported by Jim Meyering.
Also, check for other ways that the header might be invalid,
e.g., missing newline at end. Do not allow keys with nulls.
Allow blanks before and after length, as POSIX requires.
Do not allow leading "-" in length. Check for length overflow.
(xheader_decode, xheader_decode_global): Let decode_record
check for exhaustion of record.
(xheader_read): Null-terminate the extended record;
decode_record relies on this.
