* src/extract.c (struct delayed_set_stat, struct delayed_link):
* src/misc.c (normalize_filename, wd_count, chdir_count)
(chdir_arg, tar_getcdpath):
* src/names.c (name_gather, addname, add_hierarchy_to_namelist):
* src/unlink.c (struct deferred_unlink, flush_deferred_unlinks):
Use idx_t, not int, for directory indexes, so as to not
limit their number to INT_MAX; this is theoretically possible
if -T is used.
* src/names.c (name_next_elt, name_next):
Use bool for boolean.
* src/common.h (GLOBAL): Remove this macro, and all its uses.
It collides with GCC 14 and -Wmissing-variable-declarations.
Change all uses of GLOBAL to use extern instead,
and declare the variables in their respective .c files.
Move .c file’s extern declarations here, so that they
appear only once and are checked against definitions.
* src/names.c (unconsumed_option_tail): Now static.
* configure.ac: Omit stuff no longer needed now that Gnulib or
paxlib does it, or the code no longer needs the configure-time checks.
Do not use AC_SYS_LARGEFILE (Gnulib largefile does this) or check
for fcntl.h, memory.h, net/errno.h, sgtty.h, string.h,
sys/param.h, sys/device.h, sys/gentape.h, sys/inet.h,
sys/io/trioctl.h, sys/time.h, sys/tprintf.h, sys/tape.h, unistd.h,
locale.h, netdb.h; these are all now standard, or old ways of getting
at magtapes are no longer needed and we now have only sys/mtio.h.
Do not check for lstat, readlink, symlink, and check only for
waitpid’s existence rather than attempting to replace it.
Do not check for decls of getgrgid, getpwuid, or time.
Check just once for iconv.h.
* gnulib.modules: Add largefile.
* lib/.gitignore, lib/Makefile.am (noinst_HEADERS, libtar_a_SOURCES):
Remove system-ioctl.h, which is no longer in paxlib.
All includes now changed to just check HAVE_SYS_MTIO_H directly.
* lib/wordsplit.c (wordsplit_c_escape_tab, wordsplit_errstr)
(wordsplit_nerrs):
Now static or an enum, and without any leading "_" in the name.
* src/buffer.c (record_start, record_end, current_block, records_read):
* src/delete.c (records_skipped): Add extern decl to pacify GCC.
* src/compare.c, src/create.c, src/extract.c: Omit uses of
HAVE_READLINK and HAVE_SYMLINK since we now let Gnulib deal with
platforms lacking readlinkat and symlinkat.
* src/system.c: Use "#if !HAVE_WAITPID" instead of "#if MSDOS".
Bug reported in https://savannah.gnu.org/bugs/?65838.
Bug introduced by 79d1ac38c1.
* src/extract.c (make_directories): Restore second argument. This
reverts the change made in 79d1ac38c1.
(maybe_recoverable, rename_directory): Update calls to make_directories.
* tests/extrac27.at: New file.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
* src/extract.c (delay_set_stat): Simplify hash lookup;
no need to initialize members other than file_name.
Avoid assignment in ‘if’ when it’s easy.
(extract_finish): Do not bother to free when we are about to exit.
delayed_set_stat avoids inserting duplicate entries into
delayed_set_stat_head. It was doing this by scanning the entire
list.
Normally this list is small, but if --delay-directory-restore is
used (including automatically for incremental archives), this list
grows with the total number of directories in the archive.
The entire scan takes O(n) time. Extracting an archive with n
directories could therefore take O(n^2) time.
The included test uses AT_SKIP_LARGE_FILES, allowing it to optionally be
skipped. It may execute slowly on certain filesystems or disks, as it
creates thousands of directories.
There are still potentially problematic O(n) scans in
find_direct_ancestor and remove_delayed_set_stat, which this patch does
not attempt to fix.
* NEWS: Update.
* src/extract.c (delayed_set_stat_table): Create a table for O(1)
lookups of entries in the delayed_set_stat_head list. The list
remains, as tracking insertion order is important.
(dl_hash, dl_compare): New hash table helper functions.
(delay_set_stat): Create the hash table, replace the O(n) list scan
with a hash_lookup, insert new entries into the hash table.
(remove_delayed_set_stat): Also remove entry from hash table.
(apply_nonancestor_delayed_set_stat): Also remove entry from hash
table.
(extract_finish): Free the (empty) hash table.
* tests/extrac26.at: New file.
* tests/Makefile.am (TESTSUITE_AT): Include extrac26.at.
* tests/testsuite.at: Include extrac26.at.
Extract delayed links in tar file order, rather than
in hash table order with modifications.
This is simpler and more likely to use the kernel’s
cached filesystem data, assuming related delayed links
are nearby in the tar file.
* src/extract.c (struct delayed_link.has_predecessor):
Remove. All uses removed.
(delayed_link_head, delayed_link_tail): New static vars.
This resurrects delayed_link_head’s old function
except that the linked list is now in forward order, not reverse.
(find_delayed_link_source): Now simply returns bool,
since the callers no longer need the pointer.
(create_placeholder_file):
Put the delayed link at the end of the linked list.
Omit no-longer-needed last arg. All callers changed.
(apply_delayed_links): Simplify now that we can just iterate
through the delayed_link_head list.
This also ports to C23 [[maybe_unused]].
* configure.ac (WARN_CFLAGS): Do not add -Wno-unused-parameter.
Add MAYBE_UNUSED where needed in source code.
Also, put it at the front where C23 requires it.
* src/extract.c (create_placeholder_file): Use FLEXNSIZEOF (overlooked
by c542d3d0c8)
(apply_delayed_links): Don't follow the "next" chain after its entries
have been applied.
Portability bug caught by GCC 13 -fstrict-flex-arrays.
* gnulib.modules: Add flexmember.
* src/create.c (struct link):
* src/exclist.c (struct excfile):
* src/extract.c (struct delayed_link, struct string_list):
Include <flexmember.h>. Use FLEXIBLE_ARRAY_MEMBER, for
portability to strict C99 or later. All storage
allocations changed to use FLEXNSIZEOF.
* src/extract.c (maybe_recoverable): If make_directories indicates
success, suppose some intermediate directories have been made, even
if in fact they have not. That's necessary to avoid dead loops when
maybe_recoverable is called with the same arguments again.
The bug was introduced by commit 79d1ac38c1, which didn't take into
account all the consequences of returning RECOVER_OK on EEXIST, in
particular interactions with the delayed_set_stat logic.
The commit 79d1ac38c1 is reverted (the bug it was intended to fix
was actually fixed by 79a442d7b0). Instead:
* src/extract.c (maybe_recoverable): Don't call maybe_recoverable
if EEXIST is reported when UNLINK_FIRST_OLD_FILES option is set.
This bug was introduced by the recent lseek-related changes.
* src/delete.c (delete_archive_members):
* src/update.c (update_archive):
Copy the member if acting as a filter, rather than lseeking over
it, which is possible if stdin is a regular file.
* src/list.c (skim_file, skim_member):
* src/sparse.c (sparse_skim_file):
New functions, for copying when a filter.
* src/list.c (skip_file): Remove; replaced with skim_file.
All callers changed.
(skip_member): Reimplement in terms of skim_member.
* src/sparse.c (sparse_skip_file):
Remove; replaced with sparse_skim_file. All callers changed.
* src/update.c (acting_as_filter): New static var.
(update_archive): Set it; this is like delete.c.
* tests/delete01.at (deleting a member after a big one):
* tests/delete02.at (deleting a member from stdin archive):
Also test filter case.
* src/extract.c (set_xattr): Simplify, by having it do only
the mknodat and xattrs_xattrs_set, rather than also
trying to recover from failure. Caller simplified too.
* tests/xattr07.at (xattrs: xattrs and --skip-old-files):
Adjust test to match fixed behavior.
Problem reported by Kevin Raymond in:
https://bugzilla.redhat.com/show_bug.cgi?id=1886540
* src/extract.c (open_output_file): If we already created the
empty file, do not open with O_EXCL, or with O_CREAT or O_TRUNC
for that matter. Instead, use only O_NOFOLLOW to avoid some
races. When estimating current mode, use openflag & O_EXCL rather
than overwriting_old_files.
(extract_file): Also invert S_IWUSR if it’s not set.
* tests/xattr08.at: New test.
* tests/Makefile.am, tests/testsuite.at: Add it.
Do this by searching a hash table instead of a linked list.
Problem reported by Martin Dørum in https://mort.coffee/home/tar/
via Gavin Smith in:
https://lists.gnu.org/r/bug-tar/2022-07/msg00003.html
* src/extract.c: Include hash.h.
Improve performance a bit on non-birthtime hosts
(struct delayed_link.has_predecessor): New member.
(delayed_link_head): Remove, replacing with ...
(delayed_link_table): ... this new variable. All uses
of linked list replaced with hash table.
(dl_hash, dl_compare): New functions for hash table.
(create_placeholder_file): Initialize has_predecessor.
(apply_delayed_link): New function, with body taken from
most of the old apply_delayed_link.
(apply_delayed_links): Use it. Respect has_predecessor.
Don’t bother freeing as we are about to exit.
* src/extract.c (HAVE_BIRTHTIME, BIRTHTIME_EQ): New macros.
(struct delayed_link, create_placeholder_file, extract_link)
(apply_delayed_links): Avoid unnecessary work on platforms
like GNU/Linux that lack birthtime.
* src/extract.c (is_directory_link): New arg ST. Caller changed.
(is_directory_link, open_output_file):
Use readlinkat, not fstatat, to determine whether a string
names a symlink. This avoids EOVERFLOW issues.
(extract_dir): Avoid duplicate calls to fstatat when
keep_directory_symlink_option && fstatat_flags == 0
and the file is a symlink to an existing file.
Problem reported in:
https://lists.gnu.org/r/bug-tar/2022-03/msg00000.html
* src/extract.c (make_directories): Retry the file creation as
long as the directory exists, regardless of whether tar itself
created the directory.
Copyright-paperwork-exempt: Yes
Problem reported by James Abbatiello in:
https://lists.gnu.org/r/bug-tar/2022-03/msg00000.html
* src/extract.c (make_directories): Do not assume that when
mkdirat fails with errno == EEXIST that there is an existing file
that can be statted. It could be a dangling symlink. Instead,
wait until the end and stat it.
* src/common.h (xheader_xattr_free,xheader_xattr_copy): Remove protos.
(xattr_map_init,xattr_map_copy)
(xattr_map_add,xattr_map_free): New protos.
* src/tar.h (xattr_map): New struct.
(tar_stat_info): Replace xattr_map_size and xattr_map with one
field: xattr_map.
* src/xattrs.c (XATTRS_PREFIX,XATTRS_PREFIX_LEN): New defines.
(xheader_xattr_init,xattr_map_init)
(xattr_map_free,xattr_map_add)
(xheader_xattr_add,xattr_map_copy): New functions.
All uses changed.
* src/create.c (start_header): Update to use struct xattr_map.
* src/extract.c: Update to use struct xattr_map.
* src/tar.c: Likewise.
* src/xheader.c (xheader_xattr_init,xheader_xattr_free)
(xheader_xattr_add,xheader_xattr_copy): Remove.
(xattr_coder,xattr_decoder): Use xattr_map_ functions.
With today’s compilers ‘inline’ is typically not needed for
performance (at least the way GNU Tar uses it) and it gets in the
way of portability.
* configure.ac: Omit AC_C_INLINE; no longer needed here.
* lib/attr-xattr.in.h (setxattr, lsetxattr, fsetxattr, getxattr)
(lgetxattr, fgetxattr, listxattr, llistxattr, flistxattr):
* lib/wordsplit.c (skip_delim_internal, skip_delim)
(skip_delim_real, exptab_matches):
* src/delete.c (flush_file):
* src/extract.c (safe_dir_mode):
* src/misc.c (ptr_align):
Now just static, not static inline.
* lib/wordsplit.h (wordsplit_getwords): Remove; no longer used.
* src/common.h (name_more_files): Now COMMON_INLINE, not
extern inline - which is not portable according to C99,
the way we were using it.
Fedora 33 uses GCC 10.2.1, which is a bit pickier.
* configure.ac: Do not use -Wsystem-headers, as this
runs afoul of netdb.h on Fedora 33.
* gnulib.modules: Add ‘attribute’.
* lib/wordsplit.c (wsnode_new): Return the newly allocated
pointer instead of a boolean, to pacify GCC 10.2.1 which otherwise
complains about use of possibly-null pointers. All uses changed.
* src/buffer.c (try_new_volume): Don’t assume find_next_block succeeds.
(_write_volume_label): Pacify GCC 10.2.1 with an ‘assume’, since
LABEL must be nonnull here.
* src/common.h (FALLTHROUGH): Remove; now in attribute.h.
Include attribute.h, for ATTRIBUTE_NONNULL.
* src/misc.c (assign_string_or_null): New function,
taking over the old role of assign_string.
(assign_string): Assume VALUE is non-null.
(assign_null): New function, taking over the old
role of assign_string when its VALUE was nonnull.
All callers of assign_string changed to use these functions.
(assign_string_n): Clear *STRING if VALUE is null,
to fix a potential double-free.
If the archive being extracted contains multiple copies
of the same symlink, extract only the first of them and
skip the rest. The use case is described in
https://lists.gnu.org/archive/html/bug-tar/2021-01/msg00026.html
The bug was introduced by 2d3396c3ea.
* src/extract.c (create_placeholder_file): If placeholder already
exists and is registered in the delayed_link list, return immediately.
Bug was introduced by commit 34d15af1.
* src/extract.c (prepare_to_extract): When extracting over pipe,
process only regular files.
* tests/extrac24.at: New test case.
* tests/Makefile.am: Add new test case.
* tests/testsuite.at: Likewise.
* src/extract.c (prepare_to_extract): Return true to proceed with
the extraction, and false to skip the current member. If extracting
over a pipe, skip unlinking logic.
(extract_archive): Update accordingly.
Given this option, tar failed to preserve permissions of empty directories
and to create files under directories owned by the current user that did
not have the S_IWUSR bit set.
* src/extract.c (fd_chmod): Rename to fd_i_chmod.
(fd_chmod): New function.
(safe_dir_mode): New function.
(extract_dir): Special handling for existing directories in
--no-overwrite-dir mode.
* tests/extrac23.at: New file.
* tests/Makefile.am: Add new test case.
* tests/testsuite.at: Likewise.
* src/extract.c (create_placeholder_file): Take additional argument:
the delayed_link entry after which to the newly created one.
(extract_link): Create a placeholder file if the target link name
exists in the delayed_link list.
* src/extract.c (find_direct_ancestor): Remove useless test.
(delay_set_stat): If the file name being added is already in
the list, update stored data instead of creating a new entry.
This works for archives with reversed order of members.
* tests/extrac22.at: New testcase.
* tests/Makefile.am: Add new testcase.
* tests/testsuite.at: Include new testcase.
The call was introduced by commit ccef8581. It caused tar to abort
on perfectly normal operations, like untarring archives containing
./ with the -U option,
See http://lists.gnu.org/archive/html/bug-tar/2019-01/msg00019.html
for details.
* src/extract.c (maybe_recoverable): Remove misplaced call to abort().
* src/transform.c (_single_transform_name_to_obstack): Mark with
FALLTHROUGH statement rather than /* FALL THROUGH */ comment.
Only the former works with gcc-8.
* src/extract.c (maybe_recoverable): Call abort to tell gcc-8 that
this code is unreachable.
* src/common.h (FALLTHROUGH): New macro, for use with gcc
-Wimplicit-fallthrough=5, which is now the default when used with
Gnulib after commit 2017-05-16T16:23:52!eggert@cs.ucla.edu
and with --enable-gcc-warnings
* src/extract.c (find_direct_ancestor): New function.
(create_placeholder_file): Set after_links member on delayed_set_stat
entries starting from the direct ancestor of the placeholder file.
* tests/extrac21.at: New testcase.
* tests/testsuite.at: Add extrac21
* tests/Makefile.am: Likewise.
* NEWS: Update.
* src/common.h (remove_delayed_set_stat): New proto.
* src/extract.c (free_delayed_set_stat)
(remove_delayed_set_stat): New function.
(apply_nonancestor_delayed_set_stat): Use free_delayed_set_stat.
* src/misc.c (safer_rmdir): Remove delayed_set_stat entry
corresponding to the removed directory.
* tests/incr10.at: New test case.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
* src/extract.c: Move one_top_level stuff to tar.c (decode_options).
* src/tar.c (option_conflict_error): New function.
(decode_options): Use option_conflict_error to complain about
conflicting options in a uniform manner.
Process one_top_level options here.
(request_stdin): Fix error message.
* tests/onetop04.at: New testcase: check --one-top-level with --list.
* tests/Makefile.am: Add new testcase.
* tests/testsuite.at: Add new testcase.
