mirrors/nomulus
1
0
Fork 0
mirror of https://github.com/google/nomulus synced 2026-09-20 06:54:43 +00:00
Code Issues Releases Wiki Activity

Refactor contact history PII wipeout logic into a Beam pipeline (#1994)

Browse Source 
Because we need to check if a contact history is the most recent for its
underlying contact resource, the query-wipe out-repeat loop no longer works
ideally due to the added overhead with the query.

Instead, we refactor the logic into a Beam pipeline where the query only
needs to be performed once and history entries eligible for wipe out are
handled individually in their own transforms. Because history entries
are otherwise immutable, we can run the pipeline in relatively relaxed
repeatable read isolation level. We also do not worry about batching for
performance, as we do not anticipate this operation to put a lot of
strains on the particular table.
This commit is contained in:
Lai Jiang
2023-04-19 13:04:45 -04:00
committed by GitHub
parent b474e50e87
commit 5ec73f3809
13 changed files with 625 additions and 409 deletions
Download Patch File Download Diff File Expand all files Collapse all files
release/cloudbuild-nomulus.yaml
+3 -1
View File
@@ -97,7 +97,9 @@ steps:
google.registry.beam.rde.RdePipeline \
google/registry/beam/rde_pipeline_metadata.json \
google.registry.beam.resave.ResaveAllEppResourcesPipeline \
google/registry/beam/resave_all_epp_resources_pipeline_metadata.json
google/registry/beam/resave_all_epp_resources_pipeline_metadata.json \
google.registry.beam.wipeout.WipeOutContactHistoryPiiPipeline \
google/registry/beam/wipe_out_contact_history_pii_pipeline_metadata.json
# Tentatively build and publish Cloud SQL schema jar here, before schema release
# process is finalized. Also publish nomulus:core jars that are needed for
# server/schema compatibility tests.