This merely refactors.
* src/misc.c: Don’t include assert.h.
(struct wd.one_top_level): Remove. All uses changed to say that
it’s a one_top_level directory iff its index is odd and
one_top_level_dir is nonnull. This simplifies the code and
removes the need for a couple of ‘assert’s. This code
uses ~+X instead of ~X to pacify gcc -Wbool-operation.
Prefer faccessat to fstatat for dir checking.
Also, don’t bother doing either check if caller
is about to check the rough equivalent anyway.
* src/extract.c (make_directories): Now static.
2nd arg is now a three-way flag. All uses changed.
Use faccessat instead of fstatat, to avoid EOVERFLOW issues.
(create_dir): New function, for use outside this module.
External caller changed.
(maybe_recoverable): Set *INTERDIR_MADE if make_directories
said we made the parent.
* src/extract.c (make_directories): New arg JUST_PARENT.
All callers changed.
* src/misc.c (struct wd.name, add_wd, chdir_arg):
Use char *, not char const *, to let make_directories
temporarily alter slashes. All uses changed.
(chdir_do): Do not create a new name with trailing "/."
because make_directories no longer needs this.
* src/extract.c (make_directories): Return a 3-way int rather than
an int (actually a bool) and another optional bool via a pointer.
All callers changed. In comment, make it clearer what the
return value means. Also, improve behavior in a small way: when
issuing a diagnostic on MS-Windows-like systems, use the original
directory name rather than translating '\\' to '/'.
* src/misc.c (add_wd, ensure_wd): New functions, replacing ...
(grow_wd): .. this function, for clarity. All uses changed.
(chdir_arg): Omit 2nd arg, to keep callers simpler.
All uses changed. Instead of being recursive, be iterative;
this is part of the simplification.
It turns out that make_directories can do what we need (create all
directories in a path) if we append a dummy "." component, as it creates
directories up to and excluding the last component of the path.
Also, do not avoid using delay_set_stat on the newly created
directories. This matches the rest of the code and avoids potentially
leaving the newly-created directories with too open permissions.
This requires some workarounds to cope with
apply_nonancestor_delayed_set_stat and mark_metadata_set problems: do it
as in the rest of the code - apply the stats before proceeding with
extraction of anything else.
The patch leverages the existing -C code. In order to do that, every
entry in the wd[] table gets another companion entry in the table that
represents the --one-top-level directory. There is one additional field
in each entry that allows skipping the companion entries if they are not
desired.
The actual directory is created lazily by chdir_do() if needed, as you
requested, to avoid empty "a/foo" after --one-top-level=foo -C a -C b.
The patch "by the way" fixes also extraction of hardlinks with
--one-top-level which currently is broken in the typical case (the
transform is not applied to the target, so the hardlink is wrong).
The patch does not yet handle the --show-transformed case with
--one-top-level that you discussed in another subthread. As a result, two
tests now fail (onetop02.at and onetop04.at). I suppose that this would
be quite easy to fix.
Another issue that I am aware of is that I am not sure whether to call
repair_delayed_set_stat and/or delay_set_stat on the newly created
directories like extract_dir() does (the whole delay_set code is abit
mysterious to me).
Use of --create together with --one-top-level should probably be
forbidden, as unlink.c uses wd[] in a way that will likely break in the
presence of companion entries (the chdir_do call in
flush_deferred_unlinks).
Show the one-top-level arg as a transformation
* src/misc.c (file_removed_diag): Guard set_exit_status call
with !ignore_failed_read_option, consistent with stat_diag and
other diagnostic functions.
* tests/filerem03.at: New test.
* tests/testsuite.at: Include it.
* tests/Makefile.am: Add it.
Copyright-paperwork-exempt: yes
Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
This addresses CVE-2025-45582.
* gnulib.modules: Add openat2.
* src/misc.c (open_subdir): New static function.
(fdbase_opendir): Use it.
* src/tar.c (open_searchdir_how): New var, replacing and
augmenting open_searchdir_flags. All uses changed.
* tests/extrac31.at: New file.
* tests/Makefile (TESTSUITE_AT), tests/testuite.at: Add it.
* src/misc.c (fdbase_opendir): When the new directory is a
subdirectory of the old one, open relative to the old one rather
than relative all the way back to chdir_fd, and if that open fails
for a non-EMFILE reason, keep rather than discard the old directory.
Although this might help (or hurt) performance, the main
motivation is to make it easier in future commits
to prevent tarballs from escaping the extraction directory.
* src/common.h: (BADFD): New constant.
(struct fdbase): New type.
* src/create.c (dump_file0): Use parent->fd instead of caching
it into a local, as the latter approach is now awkward.
* src/extract.c (extract_link): Don’t save errno unless needed.
* src/misc.c (safer_rmdir): New arg F. All callers changed.
(maybe_backup_file): Construct full after_backup_name, now
that find_backup_file_name no longer does that for us.
(chdir_fd): Now static not extern, as other modules now use fdbase.
(fdbase_cache): New static var.
(fdbase_clear): New function. Call it whenever removing
or renaming directories or symlinks to directories.
(fdbase_opendir): New static function.
(fdbase, fdbase1): New functions. Call them whenever the
code formerly passed chdir_fd to a syscall.
* src/exclist.c (excluded_name):
* src/misc.c (normalize_filename_x, must_be_dot_or_slash)
(chdir_arg):
Use dotslash or dotslashlen instead of doing things by hand.
* src/misc.c (slashlen, dotslashlen): New functions.
(safer_rmdir): Do not worry about unlinkat with AT_REMOVEDIR
succeeding on ".", as POSIX prohibits it, and it does not succeed
on any known platform. This simplifies the file name test.
Continue to worry about "/" though, as POSIX does allow
it to be removed.
This prepares for future changes that need directory IDs.
* src/common.h (struct chdir_id): New struct.
* src/extract.c (extract_dir): Use chdir_id to avoid duplicate stats.
* src/misc.c (struct wd): New member ID.
(grow_wd): New function, extracted from chdir_arg and that
also initializes id.err.
(chdir_arg): Use it. Initialize id.err.
(chdir_id): New function.
* src/misc.c (quote_copy_string, tar_savedir):
Use bool for booleans. All uses changed.
(quote_copy_string): Use char for chars.
(unquote_string): Return void, since nobody uses return value.
(unquote_string): Check for overflow in escapes like \777.
(wdcache): Now array of idx_t not int, since in theory it
might contain values greater than INT_MAX. All uses changed.
The 2024-08-09 Gnulib changes that caused some modules prefer
signed types to size_t means that Tar should follow suit.
* src/buffer.c (short_read):
* src/system.c (sys_child_open_for_compress)
(sys_child_open_for_uncompress):
rmtread and safe_read return ptrdiff_t not idx_t;
don’t rely on implementation defined conversion.
* src/misc.c (blocking_read): Never return a negative number.
Return idx_t, not ptrdiff_t, with the same convention for EOF
and error as the new full_read. All callers changed.
* src/sparse.c (sparse_dump_region, check_sparse_region)
(check_data_region):
* src/update.c (append_file):
full_read no longer returns SAFE_READ_ERROR for I/O error; instead it
returns the number of bytes successfully read, and sets errno.
Adjust to this.
* src/system.c (sys_child_open_for_uncompress):
Rewrite to avoid need for goto and label.
In common.h, replace macros with constants or functions when that
is easy. This makes code a bit more reliable (functions evaluate
their args exactly once) and easier to debug (many debugging
environments cannot access macros).
* src/common.h (CHKBLANKS): Remove. All uses removed.
(NAME_FIELD_SIZE, PREFIX_FIELD_SIZE, UNAME_FIELD_SIZE)
(GNAME_FIELD_SIZE, TAREXIT_SUCCESS, TAREXIT_DIFFERS)
(TAREXIT_FAILURE, LG_8, LG_256, DEFAULT_CHECKPOINT)
(MAX_OLD_FILES, TF_READ, TF_WRITE, TF_DELETED, XFORM_REGFILE)
(XFORM_LINK, XFORM_SYMLINK, XFORM_ALL, WARN_ALONE_ZERO_BLOCK)
(WARN_BAD_DUMPDIR, WARN_CACHEDIR, WARN_CONTIGUOUS_CAST)
(WARN_FILE_CHANGED, WARN_FILE_IGNORED, WARN_FILE_REMOVED)
(WARN_FILE_SHRANK, WARN_FILE_UNCHANGED, WARN_FILENAME_WITH_NULS)
(WARN_IGNORE_ARCHIVE, WARN_IGNORE_NEWER, WARN_NEW_DIRECTORY)
(WARN_RENAME_DIRECTORY, WARN_SYMLINK_CAST, WARN_TIMESTAMP)
(WARN_UNKNOWN_CAST, WARN_UNKNOWN_KEYWORD, WARN_XDEV)
(WARN_DECOMPRESS_PROGRAM, WARN_EXISTING_FILE, WARN_XATTR_WRITE)
(WARN_RECORD_SIZE, WARN_FAILED_READ, WARN_MISSING_ZERO_BLOCKS)
(WARN_VERBOSE_WARNINGS, WARN_ALL, EXCL_DEFAULT, EXCL_RECURSIVE)
(EXCL_NON_RECURSIVE): Now enum constants rather than macros.
(time_option_initialized, isfound, wasfound, warning_enabled):
Now functions rather than macros TIME_OPTION_INITIALIZED, ISFOUND,
WASFOUND, WARNING_ENABLED. All uses changed.
(OLDER_STAT_TIME, OLDER_TAR_STAT_TIME, EXTRACT_OVER_PIPE)
(TAR_ARGS_INITIALIZER): Remove. All uses replaced with their
definiens or equivalent.
When parsing numbers prefer using strtosysint (renamed stoint)
to using strtoul and its variants.
This is simpler and faster and likely more reliable than
relying on quirks of the system strtoul etc,
and it standardizes how tar deals with parsing integers.
Among other things, the C standard and POSIX don’t specify
what strtol does to errno when conversions cannot be performed,
and it requires strtoul to support "-" before unsigned numbers.
* gnulib.modules (strtoimax, strtol, strtoumax, xstrtoimax):
Remove.
* src/checkpoint.c (checkpoint_compile_action, getwidth)
(format_checkpoint_string):
* src/incremen.c (read_incr_db_01, read_num)
* src/map.c (parse_id):
* src/misc.c (decode_timespec):
* src/sparse.c (decode_num):
* src/tar.c (parse_owner_group, parse_opt):
* src/transform.c (parse_transform_expr):
* src/xheader.c (decode_record, decode_signed_num)
(sparse_map_decoder):
Prefer stoint to strtol etc.
Don’t rely on errno == EINVAL as the standards don’t guarantee it.
* src/checkpoint.c (getwidth, format_checkpoint_string):
Check for invalid string suffix.
* src/checkpoint.c (getwidth):
Return intmax_t, not long. All callers changed.
* src/incremen.c (read_directory_file):
It’s just a one-digit number, so just subtract '0'.
* src/map.c (parse_id): Return bool not int. All callers changed.
* src/misc.c (stoint): Rename from strtosysint, and add
a bool * argument for reporting overflow. All callers changed.
(decode_timespec): Simplify by using ckd_sub rather than
checking for overflow by hand.
* src/tar.c (incremental_level): Now signed char to
emphasize that it can be only -1, 0, 1. All uses changed.
* src/xheader.c (decode_record): Avoid giant diagnostics.
* src/extract.c (struct delayed_set_stat, struct delayed_link):
* src/misc.c (normalize_filename, wd_count, chdir_count)
(chdir_arg, tar_getcdpath):
* src/names.c (name_gather, addname, add_hierarchy_to_namelist):
* src/unlink.c (struct deferred_unlink, flush_deferred_unlinks):
Use idx_t, not int, for directory indexes, so as to not
limit their number to INT_MAX; this is theoretically possible
if -T is used.
* src/names.c (name_next_elt, name_next):
Use bool for boolean.
It ports around issues that our handwritten code does not.
* gnulib.modules: Add xalignalloc.
* src/misc.c (ptr_align, page_aligned_alloc): Remove.
All page_aligned_alloc callers changed to use xalignalloc.
It’s now safe to assume support for C99 formats like %jd, so remove
some of the longwinded formatting code put in only to be portable to
pre-C99 platforms.
* gnulib.modules: Add intprops.
* src/buffer.c (format_total_stats, try_new_volume)
(write_volume_label):
* src/checkpoint.c (format_checkpoint_string):
* src/compare.c (verify_volume):
* src/create.c (to_chars_subst, dump_regular_file):
* src/incremen.c (read_num):
* src/list.c (read_and, from_header, simple_print_header)
(print_for_mkdir):
* src/sparse.c (sparse_dump_region):
* src/system.c (dec_to_env, sys_exec_info_script)
(sys_exec_checkpoint_script):
* src/xheader.c (out_of_range_header):
Prefer C99 formats like %jd and %ju to STRINGIFY_BIGINT.
* src/common.h: Sort includes.
Include intprops.h, verify.h. All other includes of verify.h
removed.
(intmax, uintmax): New functions and macros.
(STRINGIFY_BIGINT): Remove; no longer used.
(TIMESPEC_STRSIZE_BOUND): Make it 1 byte bigger, for negatives.
* src/create.c (MAX_VAL_WITH_DIGITS, to_base256):
Use *_WIDTH macros rather than assuming no padding bits.
Prefer UINTMAX_MAX to (uintmax_t) -1.
* src/list.c (tartime): Use strftime result rather
than running strlen later.
* src/misc.c (timetostr): New function. Prefer it when
printing time_t values.
* gnulib.modules: Add assert-h, for static_assert.
* src/common.h, src/list.c, src/misc.c:
Prefer static_assert to #if + #error. This doesn’t fix any bugs; it’s
just that in general it’s better to avoid the preprocessor.
update submodules to latest
* gnulib.modules: Add c-ctype.
* lib/wordsplit.c, src/buffer.c, src/exclist.c, src/incremen.c:
* src/list.c, src/misc.c, src/names.c, src/sparse.c, src/tar.c:
* src/xheader.c:
Include c-ctype.h, and use its API rather than ctype.h’s.
This is more likely to work when oddball locales are used.
* src/transform.c: Include ctype.h, since this module still uses
tolower and toupper (this is probably wrong - should be multi-byte).
With today’s compilers ‘inline’ is typically not needed for
performance (at least the way GNU Tar uses it) and it gets in the
way of portability.
* configure.ac: Omit AC_C_INLINE; no longer needed here.
* lib/attr-xattr.in.h (setxattr, lsetxattr, fsetxattr, getxattr)
(lgetxattr, fgetxattr, listxattr, llistxattr, flistxattr):
* lib/wordsplit.c (skip_delim_internal, skip_delim)
(skip_delim_real, exptab_matches):
* src/delete.c (flush_file):
* src/extract.c (safe_dir_mode):
* src/misc.c (ptr_align):
Now just static, not static inline.
* lib/wordsplit.h (wordsplit_getwords): Remove; no longer used.
* src/common.h (name_more_files): Now COMMON_INLINE, not
extern inline - which is not portable according to C99,
the way we were using it.
Fedora 33 uses GCC 10.2.1, which is a bit pickier.
* configure.ac: Do not use -Wsystem-headers, as this
runs afoul of netdb.h on Fedora 33.
* gnulib.modules: Add ‘attribute’.
* lib/wordsplit.c (wsnode_new): Return the newly allocated
pointer instead of a boolean, to pacify GCC 10.2.1 which otherwise
complains about use of possibly-null pointers. All uses changed.
* src/buffer.c (try_new_volume): Don’t assume find_next_block succeeds.
(_write_volume_label): Pacify GCC 10.2.1 with an ‘assume’, since
LABEL must be nonnull here.
* src/common.h (FALLTHROUGH): Remove; now in attribute.h.
Include attribute.h, for ATTRIBUTE_NONNULL.
* src/misc.c (assign_string_or_null): New function,
taking over the old role of assign_string.
(assign_string): Assume VALUE is non-null.
(assign_null): New function, taking over the old
role of assign_string when its VALUE was nonnull.
All callers of assign_string changed to use these functions.
(assign_string_n): Clear *STRING if VALUE is null,
to fix a potential double-free.
* src/misc.c (maybe_backup_file): Pass chdir_fd, not AT_FDCWD,
to find_backup_file_name. This fixes a bug whereby the wrong
backup file name was chosen for numbered backups.
Paul Eggert
Pavel CahynaandPaul Eggert
Marco NenciariniandPaul Eggert
Sergey Poznyakoff
Sergey Poznyakoff