tar: live within system-supplied limits on file descriptors

* NEWS: Note the change.  Mention dirfd and fdopendir.
* gnulib.modules: Add dirfd and fdopendir.  The code was already
using fdopendir; dirfd is a new need.
* src/common.h (open_searchdir_flags, get_directory_entries):
(subfile_open, restore_parent_fd, tar_stat_close): New decls.
(check_exclusion_tags): Adjust signature to match code change.
* src/create.c (IMPOSTOR_ERRNO): New constant.
(check_exclusion_tags): First arg is now a struct tar_stat_info
const *, not an fd.  All callers changed.
(dump_regular_file, dump_file0): A zero fd represents an unused
slot, so play it safe if the fd member is zero here.  A negative
fd represents the negation of an errno value, so play it safe and
do not assign -1 to fd merely because an open fails.
(open_failure_recover, get_directory_entries, restore_parent_fd):
(subfile_open): New functions.  These help to recover from file
descriptor exhaustion.
(dump_dir, dump_file0): Use them.
(dump_file0): Use tar_stat_close instead of rolling our own close.
* src/incremen.c (scan_directory): Use get_directory_entries,
subfile_open, etc., to recover from file descriptor exhaustion.
* src/names.c (add_hierarchy_to_namelist): Likewise.
(collect_and_sort_names): A negative fd represents the negation
of an errno value, so play it safe and do not assign -1 to fd.
* src/tar.c (decode_options): Set open_searchdir_flags.
Add O_CLOEXEC to all the open flags.
(tar_stat_close): New function, which knows how to deal with
new convention for directory streams and file descriptors.
Diagnose 'close' failures.
(tar_stat_destroy): Use it.
* src/tar.h (struct tar_stat_info): New member dirstream.
fd now has the negative of an errno value, not merely -1, if
the file could not be opened, so that failures to reopen directories
are better-diagnosed later.
* tests/Makefile.am (TESTSUITE_AT): Add extrac11.at.
* tests/testsuite.at: Likewise.
* tests/extrac11.at: New file.

tests/Makefile.am

@@ -77,6 +77,7 @@ TESTSUITE_AT = \
  extrac08.at\
  extrac09.at\
  extrac10.at\
+ extrac11.at\
  filerem01.at\
  filerem02.at\
  gzip.at\

tests/extrac11.at

@@ -0,0 +1,77 @@
+# Process this file with autom4te to create testsuite. -*- Autotest -*-
+
+# Test suite for GNU tar.
+# Copyright (C) 2010 Free Software Foundation, Inc.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3, or (at your option)
+# any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# written by Paul Eggert
+
+# Check that 'tar' works even in a file-descriptor-limited environment.
+
+AT_SETUP([scarce file descriptors])
+AT_KEYWORDS([extract extrac11])
+
+AT_TAR_CHECK([
+dirs='a
+      a/b
+      a/b/c
+      a/b/c/d
+      a/b/c/d/e
+      a/b/c/d/e/f
+      a/b/c/d/e/f/g
+      a/b/c/d/e/f/g/h
+      a/b/c/d/e/f/g/h/i
+      a/b/c/d/e/f/g/h/i/j
+      a/b/c/d/e/f/g/h/i/j/k
+'
+files=
+mkdir $dirs dest1 dest2 dest3 || exit
+for dir in $dirs; do
+  for file in X Y Z; do
+    echo $file >$dir/$file || exit
+    files="$files $file"
+  done
+done
+
+# Check that "ulimit" itself works.
+((ulimit -n 100 &&
+  tar -cf archive1.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
+  tar -xf archive1.tar -C dest1 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
+ ) &&
+ diff -r a dest1/a
+) >/dev/null 2>&1 ||
+   AT_SKIP_TEST
+
+# Another test that "ulimit" itself works:
+# tar should fail when completely starved of file descriptors.
+((ulimit -n 4 &&
+  tar -cf archive2.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
+  tar -xf archive2.tar -C dest2 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
+ ) &&
+ diff -r a dest2/a
+) >/dev/null 2>&1 &&
+   AT_SKIP_TEST
+
+# Tar should work when there are few, but enough, file descriptors.
+((ulimit -n 10 &&
+  tar -cf archive3.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
+  tar -xf archive3.tar -C dest3 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
+ ) &&
+ diff -r a dest3/a >/dev/null 2>&1
+) || { diff -r a dest3/a; exit 1; }
+],
+[0],[],[],[],[],[gnu])
+
+AT_CLEANUP

tests/testsuite.at

@@ -149,6 +149,7 @@ m4_include([extrac07.at])
 m4_include([extrac08.at])
 m4_include([extrac09.at])
 m4_include([extrac10.at])
+m4_include([extrac11.at])
 
 m4_include([label01.at])
 m4_include([label02.at])