This patch fixes a race condition in the --atime-preserve=replace
option, which might cause tar to improperly follow a symbolic link.
It also drops the use of the _FIOSATIME ioctl of Solaris 2.x
and later, which loses resolution on time stamps. Modern Solaris
systems support full-resolution time stamps in the kernel, and
it's not worth the hassle of testing this call, useful only in
no-longer-supported Solaris variants.
Also, it undoes a change I recently introduced to the --compare
option, which caused it to not follow symbolic links unless the
--dereference option was also used. Quite possibly this change is
a good idea, but the old behavior was documented and the change
should not have been installed casually.
* configure.ac: Don't check for stropts.h and sys/filio.h.
* gnulib.modules: Add futimens, utimensat. Remove futimens.
* src/common.h (fd_utimensat): New decl.
* src/compare.c (diff_file, diff_multivol):
Don't use open_read_flags: those are for --create only.
* src/create.c (dump_file0): Adjust to set_file_atime changes.
Pass fstatat_flags to set_file_atime, so that symbolic links are
not followed inadvertantly.
* src/extract.c: Don't include utimens.h.
(set_stat): Use fd_utimensat ant UTIME_NOW rather than fdutimens.
* src/misc.c: Don't include utimens.h, stropts.h, sys/filio.h.
(fd_utimensat): New function.
(set_file_atime): Use it. New arg atflag, controlling symlink
handling. All callers changed.
* configure.ac: Check for fchmod and fchown. Don't check for utimes.
* src/extract.c (fdchmod, fdchown, fdstat): New functions.
(set_mode, set_stat): New arg FD. All callers changed.
This avoids some race conditions between closing a regular file
and setting its metadata, and it's a bit faster.
* NEWS: Don't mention dirfd; no longer needed.
* gnulib.modules: Remove dirfd.
* src/create.c (get_directory_entries): Remove the code dealing
with dirfd failures, as the new fdopendir replacement doesn't
close its argument so we don't need to call dirfd. See
<http://lists.gnu.org/archive/html/bug-gnulib/2010-09/msg00208.html>
and gnulib commit 970c9038e4cca46e1b037ae0a6d574dfae6a7327.
* NEWS: Note the change. Mention dirfd and fdopendir.
* gnulib.modules: Add dirfd and fdopendir. The code was already
using fdopendir; dirfd is a new need.
* src/common.h (open_searchdir_flags, get_directory_entries):
(subfile_open, restore_parent_fd, tar_stat_close): New decls.
(check_exclusion_tags): Adjust signature to match code change.
* src/create.c (IMPOSTOR_ERRNO): New constant.
(check_exclusion_tags): First arg is now a struct tar_stat_info
const *, not an fd. All callers changed.
(dump_regular_file, dump_file0): A zero fd represents an unused
slot, so play it safe if the fd member is zero here. A negative
fd represents the negation of an errno value, so play it safe and
do not assign -1 to fd merely because an open fails.
(open_failure_recover, get_directory_entries, restore_parent_fd):
(subfile_open): New functions. These help to recover from file
descriptor exhaustion.
(dump_dir, dump_file0): Use them.
(dump_file0): Use tar_stat_close instead of rolling our own close.
* src/incremen.c (scan_directory): Use get_directory_entries,
subfile_open, etc., to recover from file descriptor exhaustion.
* src/names.c (add_hierarchy_to_namelist): Likewise.
(collect_and_sort_names): A negative fd represents the negation
of an errno value, so play it safe and do not assign -1 to fd.
* src/tar.c (decode_options): Set open_searchdir_flags.
Add O_CLOEXEC to all the open flags.
(tar_stat_close): New function, which knows how to deal with
new convention for directory streams and file descriptors.
Diagnose 'close' failures.
(tar_stat_destroy): Use it.
* src/tar.h (struct tar_stat_info): New member dirstream.
fd now has the negative of an errno value, not merely -1, if
the file could not be opened, so that failures to reopen directories
are better-diagnosed later.
* tests/Makefile.am (TESTSUITE_AT): Add extrac11.at.
* tests/testsuite.at: Likewise.
* tests/extrac11.at: New file.
* doc/tar.texi (Reliability and security, Reliability):
(Permissions problems, Data corruption and repair, Race conditions):
(Security, Privacy, Integrity, Live untrusted data):
(Security rules of thumb): New nodes.
* NEWS: Document this.
* gnulib.modules: Add openat, readlinkat.
* src/common.h (open_read_flags, fstatat_flags): New global variables.
(cachedir_file_p, dump_file, check_exclusion_tags, scan_directory):
Adjust to new signatures, described below.
(name_fill_directory): Remove.
* src/compare.c (diff_file, diff_multivol): Use open_read_flags.
* src/create.c (struct exclusion_tag): Exclusion predicates now take
a file descriptor, not a file name.
(add_exclusion_tag): Likewise. All uses changed.
(cachedir_file_p): Likewise.
(check_exclusion_tags): The directory is now a file descriptor,
not a file name. All uses changed. Use openat for better traversal.
(file_dumpable_p): Arg is now a struct stat, not a struct
tar_stat_info. All uses changed. Check the arg's file types too.
(dump_dir0, dump_dir, dump_file0, dump_file): Omit top_level and
parent_device args, since st->parent tells us that now. All uses
changed.
(dump_dir): Likewise. Also, omit fd arg for similar reasons.
Apply fdsavedir to a dup of the file descriptor, since we need a
file descriptor for openat etc. as well, and fdsavedir (perhaps
unwisely) consumes its file descriptor when successful.
Do not consume st->fd when successful; this simplifies the caller.
(create_archive): Allocate a file descriptor when retraversing
a directory, during incremental dumps.
(dump_file0): Use fstatat, openat, and readlinkat for better traversal.
When opening a file, use the result of fstat on the file descriptor
rather than the fstatat on the directory entry, to avoid some race
conditions. No need to reopen the directory since we now no longer
close it. Change "did we open the file?" test from 0 <= fd to
0 < fd since fd == 0 now represents uninitialized.
(dump_file): Now accepts struct tar_stat_info describing parent,
not parent_device. Also, accept basename and fullname of entry.
All uses changed.
* src/incremen.c (update_parent_directory): Accept struct
tar_stat_info for parent, not name. All callers changed.
Use fstatat for safer directory traversal.
(procdir): Accept struct tar_stat_info, not struct stat and
dev_t, for info about directory. All callers changed.
(scan_directory): Accept struct tar_stat_info, not name,
device, and cmdline, for info about directory. All callers
changed. Do not consume the file descriptor, since caller
might need it. Use fstatat and openat for safer directory
traversal; also, use fstat after opening to double-check.
(name_fill_directory): Remove.
* src/names.c (add_hierarchy_to_namelist): Accept struct
tar_stat_info instead of device and cmdline. All callers changed.
When descending into a subdirectory, use openat and fstat for
safer directory traversal.
(collect_and_sort_names): Use open and fstat for safer directory
traversal. Set up struct tar_stat_info for callee's new API.
* src/tar.c (decode_options): Initialize open_read_flags
and fstatat_flags.
(tar_stat_destroy): Close st->fd if it is positive (not zero!).
* src/tar.h (struct tar_stat_info): New members parent, fd.
* src/update.c (update_archive): Adjust to dump_file's API change.
* tests/filerem02.at: Ignore stderr since its contents now depend
on the file system implementation.
* src/common.h (WARN_ALL): Don't mask with 0xffffffff; on a 32-bit
host, 0xffffffff is of type 'unsigned int', which makes WARN_ALL
an unsigned int value that is too large to fit into an int, and
the C standard says that this has undefined behavior. The mask is
not needed, so omit it.
* src/tar.c (LOW_DENSITY_NUM, MID_DENSITY_NUM, HIGH_DENSITY_NUM):
Restore these macros, undoing the previous change to this file.
The macros are used after all, in some cases. Sorry about that.
* src/create.c (UINTMAX_TO_CHARS): Remove; no longer used.
* src/tar.c (LOW_DENSITY_NUM, MID_DENSITY_NUM, HIGH_DENSITY_NUM):
Likewise.
* src/incremen.c (DIR_IS_NEW): Comment out; not used.
Mark this with a FIXME, since it looks like it should be used.
* tests/extrac07.at, tests/extrac09.at, tests/listed03.at: Use
AT_UNPRIVILEGED_PREREQ, since this test requires non-root
privileges.
* tests/extrac07.at: Don't use "chmod -w", as POSIX says it's not
portable to start a chmod permissions-list with "-" as it may be
confused with an option. Use "chmod a-w" instead.
* src/common.h (chdir_current): New decl.
* src/extract.c (struct delayed_set_stat, struct delayed_link):
New member change_dir.
(delay_set_stat, create_placeholder_file): Set it.
(apply_nonancestor_delayed_set_stat, apply_delayed_links): Use it.
(extract_link): Check that the links are all relative to the same
directory.
(extract_archive): Restore the current directory after
apply_nonancestor_delayed_set_stat has possibly changed it.
* src/misc.c (chdir_current): New external var; this used to
be the private static variable 'previous' inside chdir_dir.
All uses changed.
* tests/Makefile.am (TESTSUITE_AT): New test extrac10.at.
* tests/extrac10.at: New file.
* tests/testsuite.at: Include it.
* src/sparse.c (sparse_scan_file): If the file is entirely sparse,
that is, if ST_NBLOCKS is zero, don't bother scanning for nonzero
blocks. Idea by Kit Westneat, communicated by Bernd Schubert in
<http://lists.gnu.org/archive/html/bug-tar/2010-08/msg00038.html>.
Also, omit unnecessary lseek at start of file.
The idea was suggested by Petr Sumbera in the thread starting here:
http://lists.gnu.org/archive/html/bug-tar/2010-08/msg00000.html
* src/extract.c (set_mode): Save the errno of the chmod that
failed, for the benefit of chmod_error_details. Do not bother
retrying chmod unless the mode suggests setuid is the issue.
(extract_archive): Remove redundant call to priv_set_remove_linkdir.
* src/system.c: Include priv-set.h.
(sys_spawn_shell, sys_child_open_for_compress):
(sys_child_open_for_uncompress, sys_exec_command):
(sys_exec_info_script, sys_exec_checkpoint_script):
Invoke priv_set_restore_linkdir before execv or execlp, so that
the subprocess has the same privileges that tar originally did.
This patch was inspired by the following patch that addressed a
similar problem in GNU coreutils du:
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;h=efe53cc72b599979ea292754ecfe8abf7c839d22
* src/common.h (name_count): New decl.
* src/create.c (trivial_link_count): New static var.
(create_archive): Initialize it.
(dump_hard_link, file_count_links): Use it, so that files with
link count 1 are handled correctly when they are found multiple times.
* src/names.c (allocated_entries): Renamed from allocated_names,
since the identifier's name was misleading. All uses changed.
(entries): Renamed from names. All uses changed.
(scanned): Renamed from name_index. All uses changed.
(name_count): New var.
(name_add_name): Increment it.
* tests/link04.at: New file.
* tests/testsuite.at: Add it.
* tests/Makefile.am (TESTSUITE_AT): Likewise.
* src/extract.c (maybe_recoverable, create_placeholder_file):
Change interdir_made from int * to bool *, since the flag has just
two values 0 and 1. All uses changed. This does not affect tar's
behavior.
* COPYING: Update to latest version; this is just minor formatting.
* doc/fdl.texi: Update from GFDL 1.2 to 1.3.
* doc/tar.texi: Adjust to new format of fdl.texi. Omit trailing
white space.
* src/misc.c (struct wd): Rename 'saved' to 'err', with new semantics.
(chdir_arg, chdir_do): Adjust to new semantics. Do not report an
error merely because save_cwd fails; report an error only if
save_cwd's result is needed later.
* tests/extrac09.at: New file, to test for bug that was fixed.
* tests/testsuite.at: Include it.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* src/misc.c (normalize_filename): For now, go back to making
filenames absolute, even though this causes 'tar' to fail when
getcwd fails. However, do not attempt to resolve ".." as this
does not work with symlinks. Also, do the right thing with
leading file system prefixes and on hosts where // != /.
* src/tar.c (TAR_SIZE_SUFFIXES): New define.
(parse_opt): Allow for size suffixes in arguments to
-L and --record-size options.
* NEWS, doc/tar.texi: Update.
* src/extract.c: Don't include xgetcwd.h.
(extract_dir): stat "." rather than statting getcwd's output.
* src/misc.c (normalize_filename_x): Rewrite so as not to resolve
/../, which can't be done reliably in the presence of symlinks.
Don't reject valid names such as ".".
(normalize_filename): Don't make it absolute; that way, we don't
have to invoke xgetcwd which might fail. Don't bother to realloc
at the end, since that uses time and now saves little space.
(chdir_do): Don't crash if xgetcwd fails.
* tests/Makefile.am (TESTSUITE_AT): Add listed03.at.
* tests/listed03.at: New file.
* tests/testsuite.at: Include listed03.at.
* tests/extrac07.at: Fix a typo (invalid number
of arguments before format list).
* tests/link02.at: Use `ln' instead of `link'.
* tests/link03.at: Likewise.
* src/extract.c (create_placeholder_file)
(extract_link, extract_symlink)
(extract_node, extract_fifo): Handle all possible
return values from maybe_recoverable. This complements
8f390db92f. Reported by Ico Doornekamp <bug-tar@zevv.nl>.
* NEWS: Update.
Tar --remove-files relied on canonicalize_file_name,
which replaces symlinks in file name components with
the directories they point to. Due to this, tar
effectively ignored existence of symbolic links and
was unable to remove a directory that contained any
(Alexander Kozlov <akozlov@nada.kth.se>, 2010-03-15).
* gnulib.modules: Remove canonicalize.
* src/misc.c (normalize_filename): Rewrite
from scratch. The function operates only on
its input string, it makes no attempt to test
components for existence or to resolve symbolic
links.
* tests/Makefile.am (TESTSUITE_AT): Add remfiles03.at.
* tests/testsuite.at: Likewise.
* tests/remfiles03.at: New test case.
* NEWS: Update.
* doc/tar.texi (Including a Label in the Archive): Revise
the section.
* NEWS: Update
* src/buffer.c (open_archive): Check volume label on
ACCESS_UPDATE as well.
* src/list.c (test_archive_label): Rewrite to match the
documentation.
* src/names.c (regex_usage_warning): Return int.
(names_notfound): Rewrite the conditional.
(label_notfound): New function.
* tests/label03.at: New testcase.
* tests/label04.at: New testcase.
* tests/label05.at: New testcase.
* tests/Makefile.am: Add new testcases.
* tests/testsuite.at: Likewise.
