mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-08-28 11:56:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(parquet-design): equality deletes are key-row sets, not predicates

Browse Source 
The previous "Equality deletes carry a predicate (e.g. id = 42)"
framing is misleading. Iceberg equality-delete files are not
serialized predicates — they are stored row sets keyed on a fixed
set of columns named by the manifest's equality_ids list. A data row
is deleted if its values on those columns match any delete-file row.

Replace the predicate framing with an execution model that:

- reads only the equality-key columns from the data file;
- loads the equality-key rows from each applicable delete file;
- joins on the key tuple using Iceberg's equality semantics
  (NULL matches NULL, Iceberg-defined NaN handling, canonical
  encoded comparison for decimals / timestamps / dates).

Forward-link to a not-yet-added cache-reusability subsection that
the next commit will make explicit.
This commit is contained in:
Chris Lu
2026-04-25 15:05:21 -07:00
parent 651a72e803
commit ac49cfd59b
1 changed files with 10 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PARQUET_PUSHDOWN_DESIGN.md
+10 -1
View File
@@ -802,7 +802,16 @@ When pushdown receives a request, it computes the key from the position-delete e
### Equality deletes (must evaluate at query time)
Equality deletes carry a predicate (e.g. `id = 42`). The applicability rule is:
An equality-delete file is *not* a serialized SQL predicate. It is a list of rows, each carrying values for a fixed set of equality-key columns (the file's `equality_ids`). A data row in scope is deleted if its values for those columns match any row in the delete file.
Execution model:
1. Read the equality-key columns from the data file (using projection / page-level pruning over those columns only).
2. Read the equality-key rows from each applicable delete file. The set of these rows can be loaded once and reused across queries against the same data file (see [cache reusability](#equality-delete-bitmap-cache-reusability) below).
3. For each data row, look up its key tuple in the delete-file key set. Use Iceberg's equality semantics: `NULL` values match `NULL` (unlike SQL `NULL = NULL`), and floating-point comparisons follow Iceberg's documented rules (`NaN` is a single class for equality matching). Decimal, timestamp, and date types compare by canonical encoded value.
4. Mark matching data rows for deletion.
The applicability rule for which equality-delete files apply is:
```text
{ edf :