From 651a72e803ee911b692f3393e725502742262f33 Mon Sep 17 00:00:00 2001 From: Chris Lu Date: Sat, 25 Apr 2026 15:04:57 -0700 Subject: [PATCH] docs(parquet-design): filter position-delete rows by file_path before merge Position-delete files come in two shapes: single-target (one data file named via ReferencedDataFile) and multi-target (one delete file covers many data files; each row carries its own file_path column). The previous prose said "merges that set into a per-data-file roaring bitmap" without spelling out the multi-target case, which would over-delete by attributing every row in a multi-target delete file to every applicable data file. Add a bullet pair making the row-level file_path filter explicit: single-target rows all contribute, multi-target rows only contribute when file_path == data_file.Path. --- PARQUET_PUSHDOWN_DESIGN.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/PARQUET_PUSHDOWN_DESIGN.md b/PARQUET_PUSHDOWN_DESIGN.md index b121aa34c..ee3858a86 100644 --- a/PARQUET_PUSHDOWN_DESIGN.md +++ b/PARQUET_PUSHDOWN_DESIGN.md @@ -749,7 +749,12 @@ Position deletes name `(data_file, row_position)` pairs. The set of position-del } ``` -Pushdown merges that set into a per-data-file roaring bitmap and caches it as a side index: +For each applicable delete file, the bitmap builder reads its rows and adds row positions to the per-data-file bitmap as follows: + +- when `pdf.ReferencedDataFile == data_file.Path` (single-target), every row contributes its `position` value; +- when `pdf.ReferencedDataFile` is empty (multi-target file), each row carries its own `(file_path, position)` pair, and only rows where `file_path == data_file.Path` contribute. Skipping the row-level `file_path` filter would over-delete by attributing every row in the delete file to every applicable data file. + +Pushdown merges the filtered rows into a per-data-file roaring bitmap and caches it as a side index: ```text /.index///deletes.position.bitmap