Tests: read known data from the test volumes without mounting (#1907)

The Linux workflow checks the content of the Tests/*.hc volumes by
mounting them, which needs root, FUSE and the installed package, and so
only runs on x86_64. Add a test that recovers the same data through the
volume layer instead, so it also runs on other architectures, e.g. on the
big-endian cross build under qemu-user.

Tests/volume_read_test.cpp opens a volume read-only and reads its data
area with Volume::ReadSectors: in one call, as multi-sector ranges at
nonzero offsets with odd sector counts, again with the encryption thread
pool running, and one sector at a time; every read must match the first.
CPU features are detected as in the application, and the character set
is taken from the environment (LC_CTYPE) so that paths with non-ASCII
characters work; if that locale is unavailable, the test continues in the
C locale and says so.

Tests/test_volume_read.py builds it against Volume.a and Platform.a of a
finished standard Linux build (CXX and NM can be set for a cross build)
and checks the plaintext of every Tests/test.*.hc volume: the FAT boot
sector, DUMMY.TXT with the content "Dummy" at the cluster given by the
root directory, and a known SHA-256 digest of the whole data area. A
volume without a digest, or a digest without a volume, is an error.

It runs in the Linux workflow after the .deb build and in the build job
of the big-endian workflow (test.sha512.hc only in the automatic check,
all volumes in the full run), and requires the thread pool there.

Signed-off-by: Ville Takio <ville+git@takio.fi>

.github/workflows/build-linux-big-endian.yml

@@ -1,12 +1,14 @@
 name: Build and test Linux big-endian
 
 # Cross-builds the console version for s390x (big-endian) and runs it under
-# qemu-user: the algorithm self-tests and one of the Tests/*.hc volumes
-# (created on little-endian).
+# qemu-user: the algorithm self-tests, a read of known data from one of the
+# Tests/*.hc volumes without mounting, and a mount of one of them (all
+# created on little-endian).
 #
 # The full suite (manual run with "full" selected) also builds amd64, then
-# tests the Tests/*.hc volumes on both builds and volume compatibility
-# between them in both directions, in separate jobs that run concurrently.
+# reads and tests all Tests/*.hc volumes on both builds and volume
+# compatibility between them in both directions, in separate jobs that run
+# concurrently.
 
 on:
   push:
@@ -25,6 +27,8 @@ on:
       - 'src/Makefile'
       - 'Tests/test.*.hc'
       - 'Tests/test_byte_order_interop.py'
+      - 'Tests/test_volume_read.py'
+      - 'Tests/volume_read_test.cpp'
       - '.github/workflows/build-linux-big-endian.yml'
   pull_request:
     branches: [ "master" ]
@@ -42,6 +46,8 @@ on:
       - 'src/Makefile'
       - 'Tests/test.*.hc'
       - 'Tests/test_byte_order_interop.py'
+      - 'Tests/test_volume_read.py'
+      - 'Tests/volume_read_test.cpp'
       - '.github/workflows/build-linux-big-endian.yml'
   # Manual runs, e.g. on a fork before opening a pull request.
   workflow_dispatch:
@@ -128,6 +134,19 @@ jobs:
     - name: Algorithm self-tests
       run: src/Main/veracrypt --text --test
 
+    # Needs Volume.a and Platform.a, so it runs here and not in the jobs that
+    # only receive the binary. One volume keeps the automatic check short.
+    - name: Read known data from the test volumes
+      run: |
+        set -- --build-dir src --require-thread-pool
+        if [ "${{ matrix.arch }}" = s390x ]; then
+          export CXX=s390x-linux-gnu-g++ NM=s390x-linux-gnu-nm
+        fi
+        if [ "${{ env.FULL }}" != true ]; then
+          set -- "$@" --fixture test.sha512.hc
+        fi
+        python3 Tests/test_volume_read.py "$@"
+
     # One volume keeps this check short; the full suite opens all of them.
     - name: Volume created on little-endian
       if: env.FULL != 'true'

.github/workflows/build-linux.yml

@@ -22,6 +22,9 @@ on:
       - 'src/Volume/**'
       - 'src/Makefile'
       - 'Tests/test_linux_privilege_helpers.py'
+      - 'Tests/test.*.hc'
+      - 'Tests/test_volume_read.py'
+      - 'Tests/volume_read_test.cpp'
       - '.github/workflows/build-linux.yml'
   pull_request:
     branches: [ "master" ]
@@ -44,6 +47,9 @@ on:
       - 'src/Volume/**'
       - 'src/Makefile'
       - 'Tests/test_linux_privilege_helpers.py'
+      - 'Tests/test.*.hc'
+      - 'Tests/test_volume_read.py'
+      - 'Tests/volume_read_test.cpp'
       - '.github/workflows/build-linux.yml'
 
 concurrency:
@@ -129,6 +135,12 @@ jobs:
         name: veracrypt-console-debs-${{ matrix.os }}
         path: /tmp/VeraCrypt_Packaging/Console/Packaging/veracrypt-console-*.*
 
+    # Decrypts the Tests/*.hc volumes through Volume::ReadSectors, without
+    # mounting, using the archives that the last build above (the console
+    # version) left in src, and checks known data.
+    - name: Read known data from the test volumes
+      run: python3 Tests/test_volume_read.py --build-dir src --require-thread-pool
+
     - name: Install and test VeraCrypt GUI .deb packages
       run: |
         sudo apt install -y /tmp/VeraCrypt_Packaging/GUI/Packaging/veracrypt-*.deb