mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-24 02:20:37 +00:00
f55dc71c3f
With commitsed7d352e7dandbb1867c7c7, we now have input streams for both compressed and uncompressed SSTables that provide seamless checksum and digest checking. The code for these was based on `validate_checksums()`, which implements its own validation logic over raw streams. This has led to some duplicate code. This PR deduplicates the uncompressed case by modifying `validate_checksums()` to use a checksummed input stream instead of a raw stream. The same cannot be done for compressed SSTables though. The reason is that `validate_checksums()` needs to examine the whole data file, even if an invalid chunk is encountered. In the checksummed case we support that by offloading the error handling logic from the data source via a function parameter. In the compressed data source we cannot do that because it needs to return decompressed data and decompression may fail if the data are invalid. This PR also enables `validate_checksums()` to partially verify SSTables with just the per-chunk checksums if the digest is missing. In more detail, this PR consists of: * Port of some integrity checks from `do_validate_uncompressed()` to the checksummed data source. It should now be able to detect corruption due to truncated or appended chunks (expected number of chunks is retrieved from the CRC component). * Introduction of `error_handler` parameter in checksummed data source and `data_stream()`. * Refactoring of `validate_checksums()`. The JSON response of `sstable validate-checksums` was also modified to report a missing digest. * Tests for `validate_checksums()` against SSTables with truncated data, appended data, invalid digests, or no digest. Refs #19058. This PR is a hybrid of cleanup and feature. No backport is needed. Closes scylladb/scylladb#20933 * github.com:scylladb/scylladb: tools/scylla-sstable: Rename valid_checksums -> valid test: Check validate_checksums() with missing digest sstables: Allow validate_checksums() to report missing digests sstables: Refactor validate_checksums() to use checksummed data stream sstables: Add error_handler parameter to data_stream() sstables: Add error handler in checksummed data source sstables: Check for excessive chunks in checksummed data source sstables: Check for premature EOF in checksummed data source test: test_validate_checksums: Check SSTable with invalid digest test: test_validate_checksums: Check SSTable with appended data test: test_validate_checksums: Complement test for truncated SSTable
