mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-20 06:54:24 +00:00
docs(parquet-design): filter position-delete rows by file_path before merge
Position-delete files come in two shapes: single-target (one data file named via ReferencedDataFile) and multi-target (one delete file covers many data files; each row carries its own file_path column). The previous prose said "merges that set into a per-data-file roaring bitmap" without spelling out the multi-target case, which would over-delete by attributing every row in a multi-target delete file to every applicable data file. Add a bullet pair making the row-level file_path filter explicit: single-target rows all contribute, multi-target rows only contribute when file_path == data_file.Path.
This commit is contained in:
@@ -749,7 +749,12 @@ Position deletes name `(data_file, row_position)` pairs. The set of position-del
|
||||
}
|
||||
```
|
||||
|
||||
Pushdown merges that set into a per-data-file roaring bitmap and caches it as a side index:
|
||||
For each applicable delete file, the bitmap builder reads its rows and adds row positions to the per-data-file bitmap as follows:
|
||||
|
||||
- when `pdf.ReferencedDataFile == data_file.Path` (single-target), every row contributes its `position` value;
|
||||
- when `pdf.ReferencedDataFile` is empty (multi-target file), each row carries its own `(file_path, position)` pair, and only rows where `file_path == data_file.Path` contribute. Skipping the row-level `file_path` filter would over-delete by attributing every row in the delete file to every applicable data file.
|
||||
|
||||
Pushdown merges the filtered rows into a per-data-file roaring bitmap and caches it as a side index:
|
||||
|
||||
```text
|
||||
<parquet_parent>/.index/<file_name>/<identity>/deletes.position.bitmap
|
||||
|
||||
Reference in New Issue
Block a user