Problem and fix reported by Pavel Cahyna in
https://lists.gnu.org/r/bug-tar/2025-01/msg00000.html
* src/extract.c (extract_dir): With --no-overwrite-dir,
skip the chmod if the directory already exists.
* tests/extrac23.at (--no-overwrite-dir on empty directory):
Move the part of the test that looks at a nonempty directory ...
* tests/extrac30.at: ... to this new file, because the test now
must be run as non-root. Adjust the test to match the new behavior.
* tests/Makefile.am (TESTSUITE_AT), tests/testsuite.at: Add it.
* src/names.c (namelist_match_from): New function.
(namelist_match): Rewrite as a wrapper over it.
(register_match): New function.
(name_match)" Update all possible matches in the name list.
* tests/extrac29.at: New test.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
* src/list.c (skim_member): Recognize directory members using
the same rules as during extraction.
* tests/skipdir.at: New testcase.
* tests/testsuite.at: Add new test.
* tests/Makefile.am: Likewise.
Detailed bug report: https://savannah.gnu.org/bugs/index.php?66774
* src/extract.c (update_interdir_set_stat): New function.
(extract_dir): If the directory already exists, check if it
has been created as intermediate directory earlier. If so,
update its delayed_set_stat data from archive.
* tests/Makefile.am: Add new testcase.
* tests/testsuite.at: Add new testcase.
* tests/extrac28.at: New file.
Bug reported in https://savannah.gnu.org/bugs/?65838.
Bug introduced by 79d1ac38c1.
* src/extract.c (make_directories): Restore second argument. This
reverts the change made in 79d1ac38c1.
(maybe_recoverable, rename_directory): Update calls to make_directories.
* tests/extrac27.at: New file.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
This fixes https://savannah.gnu.org/bugs/?64387.
* src/exclist.c (info_attach_exclist): Always use file->flags.
(exclude_vcs_ignores): Pass flags from struct vcs_ignore_file
to excfile_add.
* tests/exclude19.at: New test.
* tests/exclude20.at: New test.
* tests/Makefile.am: Add new tests.
* tests/testsuite.at: Add new tests.
Support upcasing and downcasing in multi-byte locales.
* gnulib.modules: Add c32rtomb, c32tolower, c32toupper,
mbrtoc32-regular.
* src/transform.c: Do not include ctype.h. Include mcel.h.
(stk, stk_init): Move up.
(run_case_conv): Return void, not char *. Append result to
stk directly; this avoids the need for a separate allocation.
All callers changed. Do not assume a single-byte locale.
* tests/xform04.at: New test.
* tests/Makefile.am (TESTSUITE_AT):
* tests/testsuite.at: Add it.
delayed_set_stat avoids inserting duplicate entries into
delayed_set_stat_head. It was doing this by scanning the entire
list.
Normally this list is small, but if --delay-directory-restore is
used (including automatically for incremental archives), this list
grows with the total number of directories in the archive.
The entire scan takes O(n) time. Extracting an archive with n
directories could therefore take O(n^2) time.
The included test uses AT_SKIP_LARGE_FILES, allowing it to optionally be
skipped. It may execute slowly on certain filesystems or disks, as it
creates thousands of directories.
There are still potentially problematic O(n) scans in
find_direct_ancestor and remove_delayed_set_stat, which this patch does
not attempt to fix.
* NEWS: Update.
* src/extract.c (delayed_set_stat_table): Create a table for O(1)
lookups of entries in the delayed_set_stat_head list. The list
remains, as tracking insertion order is important.
(dl_hash, dl_compare): New hash table helper functions.
(delay_set_stat): Create the hash table, replace the O(n) list scan
with a hash_lookup, insert new entries into the hash table.
(remove_delayed_set_stat): Also remove entry from hash table.
(apply_nonancestor_delayed_set_stat): Also remove entry from hash
table.
(extract_finish): Free the (empty) hash table.
* tests/extrac26.at: New file.
* tests/Makefile.am (TESTSUITE_AT): Include extrac26.at.
* tests/testsuite.at: Include extrac26.at.
&> is a bashism and causes various tests to fail with /bin/sh as non-bash
(e.g. dash). Use the same pattern the rest of the file uses instead of &>.
Copyright-paperwork-exempt: true
* src/common.h (name): New field: is_wildcard.
(name_scan): Change protoype.
* src/delete.c: Update calls to name_scan.
* src/names.c (addname, add_starting_file): Initialize is_wildcard.
(namelist_match): Take two arguments. If second one is true, return
only exact matches.
(name_scan): Likewise. All callers updated.
(name_from_list): Skip patterns.
* src/update.c (remove_exact_name): New function.
(update_archive): Do not remove matching name, if it is a pattern.
Instead, add a new entry with the matching file name.
* tests/update04.at: New test.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Include new test.
* NEWS: Update.
See https://savannah.gnu.org/bugs/?63823
* src/delete.c (flush_file): Break the loop on EOF.
* tests/delete06.at: New test.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Likewise.
Problem reported by Kevin Raymond in:
https://bugzilla.redhat.com/show_bug.cgi?id=1886540
* src/extract.c (open_output_file): If we already created the
empty file, do not open with O_EXCL, or with O_CREAT or O_TRUNC
for that matter. Instead, use only O_NOFOLLOW to avoid some
races. When estimating current mode, use openflag & O_EXCL rather
than overwriting_old_files.
(extract_file): Also invert S_IWUSR if it’s not set.
* tests/xattr08.at: New test.
* tests/Makefile.am, tests/testsuite.at: Add it.
Bug was introduced by commit 34d15af1.
* src/extract.c (prepare_to_extract): When extracting over pipe,
process only regular files.
* tests/extrac24.at: New test case.
* tests/Makefile.am: Add new test case.
* tests/testsuite.at: Likewise.
* src/incremen.c: Change the meaning of the DIRF_RENAMED flag. Now it
marks a directory which is the last one in a chain of renames.
Regular renamed directories are recognized by their orig member being
non-NULL. Directories marked with DIRF_RENAMED start encoding of renames.
(procdir): Clear DIRF_RENAMED flag on directories which are origins for
renames.
(makedumpdir): Use the orig member to check if the directory is a
result of a rename.
(store_rename): Move the check for DIR_IS_RENAMED to the caller. Don't
clear the DIRF_RENAMED, it is not needed any more.
* tests/rename06.at: New test.
* tests/Makefile.am: Add rename06.at
* tests/testsuite.at: Likewise.
Given this option, tar failed to preserve permissions of empty directories
and to create files under directories owned by the current user that did
not have the S_IWUSR bit set.
* src/extract.c (fd_chmod): Rename to fd_i_chmod.
(fd_chmod): New function.
(safe_dir_mode): New function.
(extract_dir): Special handling for existing directories in
--no-overwrite-dir mode.
* tests/extrac23.at: New file.
* tests/Makefile.am: Add new test case.
* tests/testsuite.at: Likewise.
* src/extract.c (find_direct_ancestor): Remove useless test.
(delay_set_stat): If the file name being added is already in
the list, update stored data instead of creating a new entry.
This works for archives with reversed order of members.
* tests/extrac22.at: New testcase.
* tests/Makefile.am: Add new testcase.
* tests/testsuite.at: Include new testcase.
Existing compression tests used tar with compiled-in defaults. However,
some of the defaults are sure to create archives that are not byte-to-byte
reproducible (e.g. DEFAULT_ARCHIVE_FORMAT=POSIX, because the name field
in posix extended headers uses PID of the creating process by default).
Moreover, some compressors (e.g. gzip) store current timestamp in
the file header when compressing from stdin, so that using cmp on the
two created archives as the tests did is error-prone. Another problem
is that the tests implicitly assumed that tar uses archive suffix to
recognize its format when extracting, which isn't the case. Finally,
there's hardly any reason in using sed to create m4 sources, when
everything can be achieved by m4 itself.
* tests/Makefile.am: Remove generation of compress-*.at files.
* tests/compress.at.in: Remove.
* tests/compress.m4: New file.
* tests/testsuite.at: Include compress.m4, use TAR_CHECK_COMPRESS to
check compression options.
* src/checkpoint.c (checkpoint_finish_compile): Set default action,
if necessary.
* tests/checkpoint/defaults.at: New testcase.
* tests/checkpoint/dot-compat.at: New testcase.
* tests/checkpoint/dot-int.at: New testcase.
* tests/checkpoint/dot.at: New testcase.
* tests/checkpoint/interval.at: New testcase.
* tests/Makefile.am: Add new testcases.
* tests/testsuite.at Include new testcases.
* tests/compress.at.in: New file: template from which each
per-compression-tool test is derived.
* tests/Makefile.am (TESTSUITE_AT): Add it.
(EXTRA_DIST): Add compress.at.in.
(compress-*.at): New rules to generate a file/tests for each compression tool.
* tests/testsuite.at (Compression): Add each of these generated
files in a new section.
* tests/.gitignore: Ignore the new generated files.
Define environment variable TARTEST_SKIP_LARGE_FILES=1 in order
to skip tests that require lots of disk space.
* tests/testsuite.at (AT_SKIP_LARGE_FILES): New macro.
* tests/sparse03.at: Mark test with AT_SKIP_LARGE_FILES.
* tests/sparse05.at: Likewise.
* tests/star/pax-big-10g.at: Likewise.
* tests/star/ustar-big-2g.at: Likewise.
* tests/star/ustar-big-8g.at: Likewise.
* src/extract.c (find_direct_ancestor): New function.
(create_placeholder_file): Set after_links member on delayed_set_stat
entries starting from the direct ancestor of the placeholder file.
* tests/extrac21.at: New testcase.
* tests/testsuite.at: Add extrac21
* tests/Makefile.am: Likewise.
* NEWS: Update.
* src/create.c (file_count_links): Apply safer_name_suffix to the
hard link name prior to transforming it.
* tests/xform03.at: New test case.
* tests/Makefile.am: Add xform03.at
* tests/testsuite.at: Likewise.
When creating multivolume archives, the bufmap code in buffer.c
implicitly assumed that the members are stored in the archive
contiguously, ignoring the member (and eventual extended) headers
between them. This worked until the member header happened to be
at the very beginning of the volume, in which case its length was
included in the calculation of the stored size and size left to
store. Due to this, the GNUFileParts extended header contained
invalid GNU.volume.offset value, and the resulting archive failed
to extract properly.
This patch also eliminates improper listing of file part headers
as regular files, when creating multivolume posix archives with -v.
* src/buffer.c (bufmap): New member nblocks. Counts number of blocks
of file data written since reset.
(bufmap_reset): Reset nblocks to 0.
(_flush_write): Update nblocks. When computing offset difference for
bufmap_reset, count only data blocks, not headers.
(close_archive): Flush archive until all blocks are written.
(add_chunk_header): Use simple_finish_header instead of finish_header
to avoid listing chunk header as regular file in verbose mode.
* tests/multiv10.at: New test case.
* tests/Makefile.am: Add new test.
* tests/testsuite.at: Add new test.
* src/common.h (name_more_files): New proto.
(files_from_option): Remove.
* src/names.c (name_more_files): New file.
(names_options): Fix declaration of the
add-file option.
(names_parse_opt): Handle --add-file.
* src/tar.c (struct tar_args): Remove the input_files member.
Change all uses: use name_more_files() instead.
* tests/Makefile.am: Add new test.
* tests/add-file.at: New testcase.
* tests/testsuite.at: Add new test.
Make sure that 'GNU.sparse.name' header has higher priority than
(for sparse-purposes artificially modified) 'path' pax header.
Historically, the 'GNU.sparse.name' header comes before 'path';
this caused that modified 'path' header won and that is not what
we want in sparse "capable" tar implementation.
* src/tar.h (tar_stat_info): New argument sparse_name_done.
* src/xheader.c (raw_path_decoder): Move here the unconditional
code from path_decoder.
(path_decoder): Apply raw_path_decoder only if sparse_path_decoder
was not yet called.
(sparse_path_decoder): New wrapper around raw_path_decoder.
* tests/sparse07.at: New testcase.
* tests/testsuite.at: Mention new testcase.
* tests/Makefile.am: Likewise.
This makes archive create behavior consistent with the
documentation. Without this change xattr include/exclude options
are accepted when creating an archive but are silently ignored.
* src/xattrs.c (xattrs_xattrs_get): Apply exclude/include mask
when fetching extended attributes
* tests/Makefile.am: Add new test case.
* tests/testsuite.at: Likewise.
* src/names.c (file_selection_option)
(file_selection_option_name): New functions.
(unconsumed_option_push, unconsumed_option_free)
(unconsumed_option_report): New functions.
(name_list_advance): Maintain a list
of eventually unconsumed options during archive creation.
Report unconsumed options, if any.
* tests/positional01.at: New test case.
* tests/positional02.at: New test case.
* tests/positional03.at: New test case.
* tests/Makefile.am: Add new test cases.
* tests/testsuite.at: Likewise.
* NEWS: Document the changes.
* configure.ac: Version 1.29.90
* doc/tar.texi: Document the changes.
The auxiliary utility ttyemu proved to be unreliable. Given existing
differences between pty implementations and termios ioctls on various
platforms, writing it in a portable way requires effort disproportional
to its actual purpose.
* configure.ac: Remove check for grantpt
* gnulib.modules: Remove posix_openpt, ptsname, and unlockpt
* tests/Makefile.am (TESTSUITE_AT): Remove iotty.at
(check_PROGRAMS): Remove ttyemu.
* tests/testsuite.at: Remove iotty.at
* tests/iotty.at: Remove.
* tests/ttyemu.c: Remove.
The new `--clamp-mtime` option will change the behavior of `--mtime` to only
use the time specified if the file mtime is newer than the given time.
The `--clamp-mtime` option can only be used together with `--mtime`.
Typical use case is to make builds reproducible: to loose less
information, it's better to keep the original date of an archive, except for
files modified during the build process. In that case, using a reference
(and thus reproducible) timestamps for the latter is good enough. See
<https://wiki.debian.org/ReproducibleBuilds> for more information.
Patch submitted by Jeremy Bobbio and
Daniel Kahn Gillmor <dkg@fifthhorseman.net>
* doc/tar.1: Document --clamp-mtime
* doc/tar.texi: Likewise.
* src/common.h (set_mtime_option_mode): New enum
(set_mtime_option): Change type to enum set_mtime_option_mode.
(NEWER_OPTION_INITIALIZED): Rename to NEWER_OPTION_INITIALIZED.
* src/create.c (start_header): Set mtime depending on set_mtime_option.
* src/tar.c (options,parse_opt): New option --clamp-mtime
(decode_options): Initialize mtime_option
* tests/time02.at: New testcase.
* tests/Makefile.am: Add new testcase
* tests/testsuite.at: Likewise.
