Use shared jar to stage BEAM pipeline if possible (#1008)

* Use shared jar to stage BEAM pipeline if possible

Allow multiple BEAM pipelines with the same classes and dependencies to
share one Uber jar.

Added metadata for BulkDeleteDatastorePipeline.

Updated shell and Cloud Build scripts to stage all pipelines in one
step.
This commit is contained in:
Weimin Yu
2021-03-16 13:19:30 -04:00
committed by GitHub
parent bae5dacbae
commit eb2e1c60ca
6 changed files with 88 additions and 49 deletions
+5 -3
View File
@@ -83,11 +83,13 @@ steps:
- -c
- |
./release/stage_beam_pipeline.sh \
init_sql_pipeline \
beam_pipeline_common \
${TAG_NAME} \
${PROJECT_ID} \
google.registry.beam.initsql.InitSqlPipeline \
google/registry/beam/init_sql_pipeline_metadata.json \
${TAG_NAME} \
${PROJECT_ID}
google.registry.beam.datastore.BulkDeleteDatastorePipeline \
google/registry/beam/bulk_delete_datastore_pipeline_metadata.json
# Tentatively build and publish Cloud SQL schema jar here, before schema release
# process is finalized. Also publish nomulus:core jars that are needed for
# server/schema compatibility tests.