* src/common.h (must_be_dot_or_slash): New decl.
* src/extract.c (mark_after_links): New function, taking code
that used to be in create_placeholder_file.
(create_placeholder_file): Use it.
(delay_set_stat): Always delay setting status for . and /.
* src/misc.c (must_be_dot_or_slash): Now extern.
* tests/extrac12.at: New file.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* tests/testsuite.at: Likewise.
* NEWS: Note the change. Mention dirfd and fdopendir.
* gnulib.modules: Add dirfd and fdopendir. The code was already
using fdopendir; dirfd is a new need.
* src/common.h (open_searchdir_flags, get_directory_entries):
(subfile_open, restore_parent_fd, tar_stat_close): New decls.
(check_exclusion_tags): Adjust signature to match code change.
* src/create.c (IMPOSTOR_ERRNO): New constant.
(check_exclusion_tags): First arg is now a struct tar_stat_info
const *, not an fd. All callers changed.
(dump_regular_file, dump_file0): A zero fd represents an unused
slot, so play it safe if the fd member is zero here. A negative
fd represents the negation of an errno value, so play it safe and
do not assign -1 to fd merely because an open fails.
(open_failure_recover, get_directory_entries, restore_parent_fd):
(subfile_open): New functions. These help to recover from file
descriptor exhaustion.
(dump_dir, dump_file0): Use them.
(dump_file0): Use tar_stat_close instead of rolling our own close.
* src/incremen.c (scan_directory): Use get_directory_entries,
subfile_open, etc., to recover from file descriptor exhaustion.
* src/names.c (add_hierarchy_to_namelist): Likewise.
(collect_and_sort_names): A negative fd represents the negation
of an errno value, so play it safe and do not assign -1 to fd.
* src/tar.c (decode_options): Set open_searchdir_flags.
Add O_CLOEXEC to all the open flags.
(tar_stat_close): New function, which knows how to deal with
new convention for directory streams and file descriptors.
Diagnose 'close' failures.
(tar_stat_destroy): Use it.
* src/tar.h (struct tar_stat_info): New member dirstream.
fd now has the negative of an errno value, not merely -1, if
the file could not be opened, so that failures to reopen directories
are better-diagnosed later.
* tests/Makefile.am (TESTSUITE_AT): Add extrac11.at.
* tests/testsuite.at: Likewise.
* tests/extrac11.at: New file.
* NEWS: Document this.
* gnulib.modules: Add openat, readlinkat.
* src/common.h (open_read_flags, fstatat_flags): New global variables.
(cachedir_file_p, dump_file, check_exclusion_tags, scan_directory):
Adjust to new signatures, described below.
(name_fill_directory): Remove.
* src/compare.c (diff_file, diff_multivol): Use open_read_flags.
* src/create.c (struct exclusion_tag): Exclusion predicates now take
a file descriptor, not a file name.
(add_exclusion_tag): Likewise. All uses changed.
(cachedir_file_p): Likewise.
(check_exclusion_tags): The directory is now a file descriptor,
not a file name. All uses changed. Use openat for better traversal.
(file_dumpable_p): Arg is now a struct stat, not a struct
tar_stat_info. All uses changed. Check the arg's file types too.
(dump_dir0, dump_dir, dump_file0, dump_file): Omit top_level and
parent_device args, since st->parent tells us that now. All uses
changed.
(dump_dir): Likewise. Also, omit fd arg for similar reasons.
Apply fdsavedir to a dup of the file descriptor, since we need a
file descriptor for openat etc. as well, and fdsavedir (perhaps
unwisely) consumes its file descriptor when successful.
Do not consume st->fd when successful; this simplifies the caller.
(create_archive): Allocate a file descriptor when retraversing
a directory, during incremental dumps.
(dump_file0): Use fstatat, openat, and readlinkat for better traversal.
When opening a file, use the result of fstat on the file descriptor
rather than the fstatat on the directory entry, to avoid some race
conditions. No need to reopen the directory since we now no longer
close it. Change "did we open the file?" test from 0 <= fd to
0 < fd since fd == 0 now represents uninitialized.
(dump_file): Now accepts struct tar_stat_info describing parent,
not parent_device. Also, accept basename and fullname of entry.
All uses changed.
* src/incremen.c (update_parent_directory): Accept struct
tar_stat_info for parent, not name. All callers changed.
Use fstatat for safer directory traversal.
(procdir): Accept struct tar_stat_info, not struct stat and
dev_t, for info about directory. All callers changed.
(scan_directory): Accept struct tar_stat_info, not name,
device, and cmdline, for info about directory. All callers
changed. Do not consume the file descriptor, since caller
might need it. Use fstatat and openat for safer directory
traversal; also, use fstat after opening to double-check.
(name_fill_directory): Remove.
* src/names.c (add_hierarchy_to_namelist): Accept struct
tar_stat_info instead of device and cmdline. All callers changed.
When descending into a subdirectory, use openat and fstat for
safer directory traversal.
(collect_and_sort_names): Use open and fstat for safer directory
traversal. Set up struct tar_stat_info for callee's new API.
* src/tar.c (decode_options): Initialize open_read_flags
and fstatat_flags.
(tar_stat_destroy): Close st->fd if it is positive (not zero!).
* src/tar.h (struct tar_stat_info): New members parent, fd.
* src/update.c (update_archive): Adjust to dump_file's API change.
* tests/filerem02.at: Ignore stderr since its contents now depend
on the file system implementation.
* tests/extrac07.at, tests/extrac09.at, tests/listed03.at: Use
AT_UNPRIVILEGED_PREREQ, since this test requires non-root
privileges.
* tests/extrac07.at: Don't use "chmod -w", as POSIX says it's not
portable to start a chmod permissions-list with "-" as it may be
confused with an option. Use "chmod a-w" instead.
* src/common.h (chdir_current): New decl.
* src/extract.c (struct delayed_set_stat, struct delayed_link):
New member change_dir.
(delay_set_stat, create_placeholder_file): Set it.
(apply_nonancestor_delayed_set_stat, apply_delayed_links): Use it.
(extract_link): Check that the links are all relative to the same
directory.
(extract_archive): Restore the current directory after
apply_nonancestor_delayed_set_stat has possibly changed it.
* src/misc.c (chdir_current): New external var; this used to
be the private static variable 'previous' inside chdir_dir.
All uses changed.
* tests/Makefile.am (TESTSUITE_AT): New test extrac10.at.
* tests/extrac10.at: New file.
* tests/testsuite.at: Include it.
This patch was inspired by the following patch that addressed a
similar problem in GNU coreutils du:
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;h=efe53cc72b599979ea292754ecfe8abf7c839d22
* src/common.h (name_count): New decl.
* src/create.c (trivial_link_count): New static var.
(create_archive): Initialize it.
(dump_hard_link, file_count_links): Use it, so that files with
link count 1 are handled correctly when they are found multiple times.
* src/names.c (allocated_entries): Renamed from allocated_names,
since the identifier's name was misleading. All uses changed.
(entries): Renamed from names. All uses changed.
(scanned): Renamed from name_index. All uses changed.
(name_count): New var.
(name_add_name): Increment it.
* tests/link04.at: New file.
* tests/testsuite.at: Add it.
* tests/Makefile.am (TESTSUITE_AT): Likewise.
* src/misc.c (struct wd): Rename 'saved' to 'err', with new semantics.
(chdir_arg, chdir_do): Adjust to new semantics. Do not report an
error merely because save_cwd fails; report an error only if
save_cwd's result is needed later.
* tests/extrac09.at: New file, to test for bug that was fixed.
* tests/testsuite.at: Include it.
* tests/Makefile.am (TESTSUITE_AT): Add it.
* src/extract.c: Don't include xgetcwd.h.
(extract_dir): stat "." rather than statting getcwd's output.
* src/misc.c (normalize_filename_x): Rewrite so as not to resolve
/../, which can't be done reliably in the presence of symlinks.
Don't reject valid names such as ".".
(normalize_filename): Don't make it absolute; that way, we don't
have to invoke xgetcwd which might fail. Don't bother to realloc
at the end, since that uses time and now saves little space.
(chdir_do): Don't crash if xgetcwd fails.
* tests/Makefile.am (TESTSUITE_AT): Add listed03.at.
* tests/listed03.at: New file.
* tests/testsuite.at: Include listed03.at.
* tests/extrac07.at: Fix a typo (invalid number
of arguments before format list).
* tests/link02.at: Use `ln' instead of `link'.
* tests/link03.at: Likewise.
Tar --remove-files relied on canonicalize_file_name,
which replaces symlinks in file name components with
the directories they point to. Due to this, tar
effectively ignored existence of symbolic links and
was unable to remove a directory that contained any
(Alexander Kozlov <akozlov@nada.kth.se>, 2010-03-15).
* gnulib.modules: Remove canonicalize.
* src/misc.c (normalize_filename): Rewrite
from scratch. The function operates only on
its input string, it makes no attempt to test
components for existence or to resolve symbolic
links.
* tests/Makefile.am (TESTSUITE_AT): Add remfiles03.at.
* tests/testsuite.at: Likewise.
* tests/remfiles03.at: New test case.
* NEWS: Update.
* doc/tar.texi (Including a Label in the Archive): Revise
the section.
* NEWS: Update
* src/buffer.c (open_archive): Check volume label on
ACCESS_UPDATE as well.
* src/list.c (test_archive_label): Rewrite to match the
documentation.
* src/names.c (regex_usage_warning): Return int.
(names_notfound): Rewrite the conditional.
(label_notfound): New function.
* tests/label03.at: New testcase.
* tests/label04.at: New testcase.
* tests/label05.at: New testcase.
* tests/Makefile.am: Add new testcases.
* tests/testsuite.at: Likewise.
* src/common.h (read_header_mode): New enum.
(read_header): Change type of the 3rd argument.
* src/list.c (read_header): Change type of the 3rd argument.
All callers updated.
* src/buffer.c (try_new_volume): Allow for volumes split at the
extended/ustar header boundary. This is against POSIX specs, but
we must be able to read such archives anyway.
* tests/multiv07.at: New test case.
* tests/Makefile.am: Add multiv07.at
* tests/testsuite.at: Likewise.
* src/compare.c: Update calls to read_header.
* src/delete.c: Likewise.
* src/update.c: Likewise.
* src/buffer.c (match_volume_label): Call set_volume_label.
(check_label_pattern): Get label string
as argument.
(match_volume_label): Handle volume labels stored in
global PAX headers.
* src/common.c (print_header,read_header): Change signature.
(read_header_primitive): Remove prototype.
* src/list.c (recent_global_header): New static.
(list_archive): Always print volume labels.
(read_header_primitive): Remove.
(read_header): Change the signature (all callers updated)
Save the recent global header.
(volume_label_printed): New static.
(simple_print_header): New function (ex-print_header).
(print_header): Change the signature (all callers updated).
For POSIX formats, print first volume header (if set).
* src/xheader.c (xheader_write_global): Write the data
accumulated in xhdr->stk even if keyword_global_override_list
is empty.
(xheader_read): On unexpected EOF, report error instead of
coredumping.
(XHDR_PROTECTED, XHDR_GLOBAL): New defines.
(struct xhdr_tab): Remove `protected' with `flags'. All uses updated.
(decg): If XHDR_GLOBAL bit is set, call the keyword's decode
method instead of adding it to `kwl'.
* src/compare.c: Update calls to read_header.
* src/create.c: Likewise.
* src/delete.c: Likewise.
* src/update.c: Likewise.
* src/extract.c: Likewise.
(extract_volhdr): Do not print "Reading <label>" statement, because
it is inconsistent: it is not printed if the volume begins with a
member continued from the previous volume.
* tests/label01.at: New testcase.
* tests/label02.at: New testcase.
* tests/Makefile.am, tests/testsuite.at: Add new testcases.
* src/common.h (namebuf_t): New typedef.
(namebuf_create, namebuf_free)
(namebuf_name): New prototypes.
(remname): New prototype.
* src/misc.c (struct namebuf): New structure.
(namebuf_create, namebuf_free)
(namebuf_name): New functions.
* src/create.c (dup_dir0): Remove is_avoided_name
checks. This is taken care of in update_archive.
* src/incremen.c (scan_directory): Use namebuf
to produce full file names.
* src/names.c (nametail): Remove extra level of
indirection. All uses updated.
(avoided_name_table, add_avoided_name)
(is_avoided_name): Remove.
* src/update.c (update_archive): Change algorithm.
Instead of adding unmodified files to the avoided_name
table, create namelist so that it contains only
modified files.
* tests/Makefile.am: Add update01.at, update02.at
* tests/testsuite.at: Likewise.
* tests/update.at (AT_KEYWORDS): Add update00.
Make sure the files are deleted only if they were succesfully stored
to the archive.
* src/exit.c: New file.
* src/unlink.c: New file.
* src/Makefile.am (tar_SOURCES): Add exit.c and unlink.c.
* src/common.h: Include progname.h
(program_name): Remove global.
(records_written): New extern.
(queue_deferred_unlink, finish_deferred_unlinks): New prototypes.
(fatal_exit_hook): New extern.
* src/create.c (create_archive): Call finish_deferred_unlinks.
(dump_hard_link, dump_file0): Don't actually unlink the file,
queue it to deferred_unlinks instead.
* src/delete.c (records_written): Remove extern: declared in
common.h.
* src/extract.c (extract_archive): Set fatal_exit_hook.
(fatal_exit, xalloc_die): Move to exit.c
* src/system.c (sys_wait_for_child): Exit immediately
if the child dies or exits with a non-zero status.
(sys_child_open_for_compress)
(sys_child_open_for_uncompress): Use set_program_name,
instead of setting program_name directly.
* src/tar.c (main): Use set_program_name,
instead of setting program_name directly.
* tests/Makefile.am (TESTSUITE_AT): Add remfiles01.at
and remfiles02.at.
* tests/testsuite.at: Likewise.
* tests/gzip.at: Reflect the above changes.
See tests/append02.at for a detailed description
* src/common.h (MODE_FROM_HEADER): Take additional argument.
(mode_from_header): Likewise.
* src/create.c (mode_to_chars): Store all mode bits if
using OLDGNU_FORMAT. This reverses f4e4adea80.
* src/list.c (decode_header): Use header mode field
to discern between GNU and OLDGNU formats.
(mode_from_header): Store unrecognized mode bits (from 10th up)
in the location pointed to by the third parameter.
* tests/append02.at: Update documentation and references.
* src/incremen.c (read_directory_file): Execute eventual -C dir
after opening the snapshot file.
(collect_and_sort_names): Remove call to chdir_do
* tests/incr05.at, tests/incr06.at: Use relative file names for
snapshot files.
* src/names.c (is_pattern): Remove.
(regex_usage_warning): Use fnmatch_pattern_has_wildcards instead of
is_pattern.
* src/tar.c: New option --exclude-backups.
(vcs_file_table, backup_file_table): New globals.
(add_exclude_array): New function.
* tests/exclude01.at, tests/exclude02.at,
tests/exclude03.at, tests/exclude04.at,
tests/exclude05.at: New testcases. Supplied by Phil Proudman.
* tests/Makefile.am (TESTSUITE_AT): Add new tests.
* tests/testsuite.at: Add new tests.
* THANKS: Update.
Changes to src/create.c and src/incremen.c are partially
based on patch from Alexander Peslyak <solar at openwall.com>.
The new testcases require paxutils commit f653a2b or later.
* src/common.h (struct name): New member `cmdline'.
(dump_file): Change type of the 2nd argument to bool.
(file_removed_diag, dir_removed_diag): New prototypes.
(addname): New argument `cmdline'.
(name_from_list): Change return value.
* src/create.c (dump_dir0, dump_dir): top_level is bool.
(create_archive): Update calls to name_from_list.
Take advantage of the name->cmdline to set top_level argument
during incremental backups.
(dump_file0): top_level is bool.
Do not bail out if a no-top-level file disappears during incremental
backup, use file_removed_diag instead.
(dump_filed): top_level is bool.
* src/incremen.c (update_parent_directory): Silently ignore
ENOENT. It should have already been reported elsewhere.
(scan_directory): Use dir_removed_diag to report missing directories.
* src/misc.c (file_removed_diag, dir_removed_diag): New functions.
* src/names.c (name_gather): Set ->cmdname.
(addname): Likewise. All uses updated.
(name_from_list): Return struct name const *. All uses updated.
* tests/filerem01.at: New testcase.
* tests/filerem02.at: New testcase.
* tests/Makefile.am, tests/testsuite.at: Add filerem01.at, filerem02.at
* tests/grow.at, test/truncate.at: Use new syntax for genfile --run.
* NEWS: Update.
* doc/tar.texi: Minor fix.
The modified algorithm tries to avoid dumping the same
directory twice and ensures the order of the directories
in the resulting archive is the same, whatever their order
on the command line. It also fixes the operation of
--listed-incremental -C.
* gnulib.modules: Add canonicalize
* src/common.h (incremental_level): New global.
(check_exclusion_tags): first argument is const.
(get_directory_contents): Add third argument.
(zap_slashes, normalize_filename): New prototypes.
(chdir_count): New prototype.
(WARN_VERBOSE_WARNINGS): New define.
(WARN_ALL): Exclude WARN_VERBOSE_WARNINGS.
* src/compare.c (diff_dumpdir): Update the call to get_directory_contents.
* src/create.c (check_exclusion_tags): First argument is const.
Use ISSLASH and DIRECTORY_SEPARATOR instead of referring to '/'.
* src/incremen.c (struct directory): New member `caname'.
(hash_directory_name): Rename to hash_directory_canonical_name. Operate
on the canonical name.
(compare_directory_names): Rename to compare_directory_canonical_names.
Operate on the canonical name.
(make_directory): Take two arguments.
(free_directory): Free caname.
(attach_directory): Create caname.
(find_directory): Use caname for lookups.
(PD_VERBOSE): Remove.
(PD_FORCE_INIT): New define.
(procdir): First argument is const.
Reinitialize directory if PD_FORCE_INIT bit is set.
Do not use PD_VERBOSE or verbose_option for issuing warnings.
Rely on WARNOPT instead.
Always set *entry.
(scan_directory): Take three arguments. The third one is a boolean
which is true if the directory is explicitly mentioned on the command
line.
(get_directory_contents): Remove. Use scan_directory instead.
All callers updated.
(read_directory_file): Truncate the file if --level=0 is given.
* src/misc.c: Include canonicalize.h
(zap_slashes, normalize_filename): New functions.
(chdir_count): New function.
* src/names.c (add_hierarchy_to_namelist): Take three arguments, as
get_directory_contents and scan_directory.
(collect_and_sort_names): Allow at most one -C, before file name
arguments.
Read directory file after eventual changing to another directory.
Avoid adding the same directory under different pathnames to
the list.
* src/tar.c: New option --level.
* tests/incr03.at, tests/incr04.at, tests/listed01.at,
tests/listed02.at, tests/rename01.at, tests/rename02.at,
tests/rename03.at: Update for new tar behavior.
* tests/multiv01.at: Do not use --listed-incremental.
* NEWS, THANKS: Update
* src/extract.c (extract_dir): reset status to 0 if the
directory already exists.
* src/misc.c (maybe_backup_file): Assign before_backup_name
and clear after_backup_name before checking if we really need
to backup the file.
* tests/backup01.at: New testcase.
* tests/extrac08.at: New testcase.
* tests/Makefile.am, tests/testsuite.at: Add extrac08.at and
backup01.at
* src/create.c (dump_hard_link): Always look up in the link table
if remove_files_option is set. Patch suggested by Theodore Ts'o
<tytso@mit.edu>.
(check_links): Remove extra newline from the warning message.
* tests/link02.at, tests/link03.at: New testcases.
* tests/Makefile.am (TESTSUITE_AT): Add link02.at and link03.at
* tests/testsuite.at: Include link02.at and link03.at
* src/create.c (file_count_links): Transform link target
before the look up.
* tests/xform-h.at: New test case.
* tests/Makefile.am (TESTSUITE_AT): Add xform-h.at.
* tests/testsuite.at: Add xform-h.at
* Makefile.am (INCLUDES): Add ../lib, for rmt-command.h
* src/buffer.c (short_read): Report the actual record size
only if the archive descriptor refers to a device.
* tests/pipe.at, tests/shortrec.at,
tests/sparsemvp.at: Reflect the above change.
* NEWS, configure.ac: Version 1.22.90
* bootstrap.conf: Include size_max.
* gnulib.modules: Remove memset, rmdir. Replace strdup with
strdup-posix. Patch by Eric Blake.
* src/tar.c: Implement -I as a shorthand for --use-compress-program.
* doc/tar.texi: Document -I.
* tests/pipe.at, tests/shortrec.at: Account for eventual 'Record
size' output.
* tests/testsuite.at (AT_TAR_CHECK_HOOK): New define
(AT_TAR_WITH_HOOK, TAR_IGNREC_HOOK): New macros.
which was always false on a first record and thus disabled record
size autodetection. Thanks Ed Leaver for the patch.
(_gnu_flush_read): Handle blocking_factor == 1.
* tests/sparsemv.at: Reflect changes to buffer.c.
* tests/sparsemvp.at: Likewise.
* tests/volsize.at: Likewise.
* NEWS: Update.
* THANKS: Add Ed Leaver.
* src/incremen.c (struct directory): New member `next'. Change
type of `name'.
(dirhead, dirtail): New statics.
(make_directory): Reflect changes to struct directory.
(free_directory, attach_directory): New functions.
(dirlist_replace_prefix): New function.
(note_directory): Use attach_directory, instead of make_directory,
(find_directory, find_directory_meta): Use free_directory.
(procdir): Replace directory prefixes in directory list to avoid
marking subdirectories as renamed after renaming their parent
directory.
(append_incremental_renames): Iterate over directory list, not
hash table, to preserve logical ordering of renames.
* tests/rename04.at, tests/rename05.at: New test cases.
* tests/Makefile.am, tests/testsuite.at: Add rename04.at and
rename05.at.
* tests/atlocal.in (decho): New function.
* tests/multiv06.at: Use decho instead of echo2.
* tests/incremental.at: Raise wait interval to 2 seconds.
* src/tar.c (decode_options): Do not allow volume length less
than record size.
* src/buffer.c (_gnu_flush_write): Compensate for the effect
of eventual flush_archive occurring in the middle of buffer
move.
Increment records_written only if _flush_write was able to write
something.
* tests/multiv06.at: New testcase.
* tests/Makefile.am, test/testsuite.at: Add tests/multiv06.at
* src/buffer.c (try_new_volume): Print more information with error
diagnostics.
(_gnu_flush_write): Improve error checking. Adjust
real_s_sizeleft before calling new_volume to avoid creating
malformed multivolume headers.
* tests/delete05.at, tests/gzip.at, tests/ignfail.at,
tests/longv7.at, tests/lustar01.at, tests/lustar02.at,
tests/shortfile.at: Update to match new diagnostic wording
(see 2008-05-06).
* NEWS: Update.
* configure.ac: Raise version number to 1.20
* src/compare.c (diff_dumpdir): const.
* src/common.h (dumpdir_t,dumpdir_iter_t): New data types.
(dumpdir_create0,dumpdir_create,dumpdir_free,dumpdir_locate)
(dumpdir_first,dumpdir_next): New functions.
* src/incremen.c (dumpdir_create0,dumpdir_create,dumpdir_free)
(dumpdir_first,dumpdir_next): New functions.
(dumpdir_locate): Rewrite using binary search.
(struct directory): Change members char *contents, *icontents to
struct dumpdir *dump, *idump. All references updated.
(note_directory): Last arg is const.
* src/names.c (add_hierarchy_to_namelist): buffer is const.
* tests/incr03.at, tests/incr04.at, tests/rename02.at,
tests/rename03.at: Insert calls to sleep between creation of files
and adding them to the archive.
