Add a no-async actions DB migration phase (#1579 )

* Add a no-async actions DB migration phase This needs to be set several hours prior to entering the READONLY stage. This is not a read-only stage; all synchronous actions under Datastore (such as domain creates) will continue to succeed. The only thing that will fail is host deletes, host renames, and contact deletes, as these three actions require a mapreduce to run before they are complete, and we don't want mapreduces hanging around and executing during what is supposed to be a short duration READONLY period.
Use UrlFetch for RDE and default TLS (1.2) for other URL connections (#1578 )
2026-05-20 14:51:48 +00:00 · 2022-04-01 16:55:51 -04:00 · 2022-03-31 14:08:28 -04:00 · 2022-03-31 12:27:32 -04:00 · 2022-03-30 15:17:38 -04:00 · 2022-03-30 11:36:24 -04:00
271 changed files with 12098 additions and 10226 deletions
--- a/build.gradle
+++ b/build.gradle
@@ -55,7 +55,7 @@ plugins {

 node {
  download = true
-  version = "14.15.5"
+  version = "16.14.0"
  npmVersion = "6.14.11"
 }

--- a/common/gradle/dependency-locks/testCompile.lockfile
+++ b/common/gradle/dependency-locks/testCompile.lockfile
@@ -4,7 +4,7 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/common/gradle/dependency-locks/testCompileClasspath.lockfile
+++ b/common/gradle/dependency-locks/testCompileClasspath.lockfile
@@ -4,7 +4,7 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/common/gradle/dependency-locks/testRuntime.lockfile
+++ b/common/gradle/dependency-locks/testRuntime.lockfile
@@ -4,8 +4,8 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/common/gradle/dependency-locks/testRuntimeClasspath.lockfile
+++ b/common/gradle/dependency-locks/testRuntimeClasspath.lockfile
@@ -4,8 +4,8 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/common/gradle/dependency-locks/testingCompile.lockfile
+++ b/common/gradle/dependency-locks/testingCompile.lockfile
@@ -4,7 +4,7 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/common/gradle/dependency-locks/testingCompileClasspath.lockfile
+++ b/common/gradle/dependency-locks/testingCompileClasspath.lockfile
@@ -4,7 +4,7 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/common/gradle/dependency-locks/testingRuntime.lockfile
+++ b/common/gradle/dependency-locks/testingRuntime.lockfile
@@ -4,8 +4,8 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/common/gradle/dependency-locks/testingRuntimeClasspath.lockfile
+++ b/common/gradle/dependency-locks/testingRuntimeClasspath.lockfile
@@ -4,8 +4,8 @@
 com.google.auto.value:auto-value-annotations:1.7.4
 com.google.code.findbugs:jsr305:3.0.2
 com.google.errorprone:error_prone_annotations:2.5.1
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava:30.1.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
--- a/core/gradle/dependency-locks/compile.lockfile
+++ b/core/gradle/dependency-locks/compile.lockfile
@@ -7,11 +7,11 @@ args4j:args4j:2.0.23
 cglib:cglib-nodep:2.2
 com.101tec:zkclient:0.10
 com.beust:jcommander:1.60
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -26,113 +26,114 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.6
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -143,9 +144,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -165,31 +166,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -213,33 +217,33 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -249,7 +253,7 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
+org.hamcrest:hamcrest-core:1.3
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -278,11 +282,11 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/compileClasspath.lockfile
+++ b/core/gradle/dependency-locks/compileClasspath.lockfile
@@ -7,11 +7,11 @@ args4j:args4j:2.0.23
 cglib:cglib-nodep:2.2
 com.101tec:zkclient:0.10
 com.beust:jcommander:1.60
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -25,113 +25,114 @@ com.github.jnr:jnr-posix:3.1.4
 com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.6
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -142,9 +143,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -164,28 +165,28 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -208,33 +209,33 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.conscrypt:conscrypt-openjdk-uber:2.5.1
@@ -243,7 +244,6 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -272,11 +272,11 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/default.lockfile
+++ b/core/gradle/dependency-locks/default.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,114 +30,115 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
 com.google.cloud.sql:postgres-socket-factory:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -148,9 +149,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -173,31 +174,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -222,34 +226,34 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -259,7 +263,7 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
+org.hamcrest:hamcrest-core:1.3
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -291,12 +295,12 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/deploy_jar.lockfile
+++ b/core/gradle/dependency-locks/deploy_jar.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,114 +30,115 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
 com.google.cloud.sql:postgres-socket-factory:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -148,9 +149,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -173,31 +174,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -221,34 +225,34 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -258,7 +262,6 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -290,12 +293,12 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/nonprodCompile.lockfile
+++ b/core/gradle/dependency-locks/nonprodCompile.lockfile
@@ -7,11 +7,11 @@ args4j:args4j:2.0.23
 cglib:cglib-nodep:2.2
 com.101tec:zkclient:0.10
 com.beust:jcommander:1.60
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -26,113 +26,114 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.6
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -143,9 +144,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -165,31 +166,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -213,33 +217,33 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -249,7 +253,7 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
+org.hamcrest:hamcrest-core:1.3
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -278,11 +282,11 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/nonprodCompileClasspath.lockfile
+++ b/core/gradle/dependency-locks/nonprodCompileClasspath.lockfile
@@ -7,11 +7,11 @@ args4j:args4j:2.0.23
 cglib:cglib-nodep:2.2
 com.101tec:zkclient:0.10
 com.beust:jcommander:1.60
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -25,113 +25,114 @@ com.github.jnr:jnr-posix:3.1.4
 com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.6
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -142,9 +143,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -164,28 +165,28 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -209,33 +210,33 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.conscrypt:conscrypt-openjdk-uber:2.5.1
@@ -244,7 +245,7 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
+org.hamcrest:hamcrest-core:1.3
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -273,11 +274,11 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/nonprodRuntime.lockfile
+++ b/core/gradle/dependency-locks/nonprodRuntime.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,113 +30,114 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -147,9 +148,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -172,31 +173,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -221,34 +225,34 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -258,7 +262,7 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
+org.hamcrest:hamcrest-core:1.3
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -289,12 +293,12 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/nonprodRuntimeClasspath.lockfile
+++ b/core/gradle/dependency-locks/nonprodRuntimeClasspath.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,113 +30,114 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -147,9 +148,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -172,31 +173,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -221,34 +225,34 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -258,7 +262,7 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
+org.hamcrest:hamcrest-core:1.3
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -289,12 +293,12 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/runtime.lockfile
+++ b/core/gradle/dependency-locks/runtime.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,113 +30,114 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -147,9 +148,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -172,31 +173,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -221,34 +225,34 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -258,7 +262,7 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
+org.hamcrest:hamcrest-core:1.3
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -289,12 +293,12 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/runtimeClasspath.lockfile
+++ b/core/gradle/dependency-locks/runtimeClasspath.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,114 +30,115 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
 com.google.cloud.sql:postgres-socket-factory:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.113.12
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -148,9 +149,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -173,31 +174,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -221,34 +225,34 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
 org.apache.commons:commons-lang3:3.8.1
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.apache.httpcomponents:httpcore:4.4.15
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -258,7 +262,6 @@ org.easymock:easymock:3.0
 org.flywaydb:flyway-core:5.2.4
 org.glassfish.jaxb:jaxb-runtime:2.3.1
 org.glassfish.jaxb:txw2:2.3.1
-org.hamcrest:hamcrest-core:2.1
 org.hamcrest:hamcrest:2.1
 org.hibernate.common:hibernate-commons-annotations:5.1.2.Final
 org.hibernate:hibernate-core:5.4.23.Final
@@ -290,12 +293,12 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/testCompile.lockfile
+++ b/core/gradle/dependency-locks/testCompile.lockfile
@@ -7,11 +7,11 @@ args4j:args4j:2.0.23
 cglib:cglib-nodep:2.2
 com.101tec:zkclient:0.10
 com.beust:jcommander:1.60
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -26,61 +26,63 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -88,54 +90,53 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-api-stubs:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
 com.google.cloud:google-cloud-nio:0.123.4
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.118.0
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.6
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava-testlib:30.1.1-jre
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -147,9 +148,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -174,31 +175,34 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -223,24 +227,24 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-direct-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-direct-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
@@ -249,18 +253,18 @@ org.apache.commons:commons-text:1.6
 org.apache.ftpserver:ftplet-api:1.0.6
 org.apache.ftpserver:ftpserver-core:1.0.6
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
+org.apache.httpcomponents:httpcore:4.4.15
 org.apache.mina:mina-core:2.0.4
 org.apache.sshd:sshd-core:2.0.0
 org.apache.sshd:sshd-scp:2.0.0
 org.apache.sshd:sshd-sftp:2.0.0
 org.apache.tomcat:tomcat-annotations-api:8.0.5
 org.apiguardian:apiguardian-api:1.1.0
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -328,11 +332,11 @@ org.testcontainers:junit-jupiter:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:selenium:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/testCompileClasspath.lockfile
+++ b/core/gradle/dependency-locks/testCompileClasspath.lockfile
@@ -7,11 +7,11 @@ args4j:args4j:2.0.23
 cglib:cglib-nodep:2.2
 com.101tec:zkclient:0.10
 com.beust:jcommander:1.60
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -25,61 +25,63 @@ com.github.jnr:jnr-posix:3.1.4
 com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -87,54 +89,53 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-api-stubs:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
 com.google.cloud:google-cloud-nio:0.123.4
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.118.0
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.6
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava-testlib:30.1.1-jre
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -146,9 +147,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -173,28 +174,28 @@ io.confluent:kafka-avro-serializer:5.3.2
 io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -219,24 +220,24 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-direct-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-direct-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
@@ -245,18 +246,18 @@ org.apache.commons:commons-text:1.6
 org.apache.ftpserver:ftplet-api:1.0.6
 org.apache.ftpserver:ftpserver-core:1.0.6
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
+org.apache.httpcomponents:httpcore:4.4.15
 org.apache.mina:mina-core:2.0.4
 org.apache.sshd:sshd-core:2.0.0
 org.apache.sshd:sshd-scp:2.0.0
 org.apache.sshd:sshd-sftp:2.0.0
 org.apache.tomcat:tomcat-annotations-api:8.0.5
 org.apiguardian:apiguardian-api:1.1.0
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.conscrypt:conscrypt-openjdk-uber:2.5.1
@@ -323,11 +324,11 @@ org.testcontainers:junit-jupiter:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:selenium:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/testRuntime.lockfile
+++ b/core/gradle/dependency-locks/testRuntime.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,61 +30,63 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -92,55 +94,54 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-api-stubs:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
 com.google.cloud.sql:postgres-socket-factory:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
 com.google.cloud:google-cloud-nio:0.123.4
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.118.0
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava-testlib:30.1.1-jre
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -152,9 +153,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -183,31 +184,34 @@ io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
 io.github.java-diff-utils:java-diff-utils:4.9
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -233,24 +237,24 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-direct-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-direct-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
@@ -259,18 +263,18 @@ org.apache.commons:commons-text:1.6
 org.apache.ftpserver:ftplet-api:1.0.6
 org.apache.ftpserver:ftpserver-core:1.0.6
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
+org.apache.httpcomponents:httpcore:4.4.15
 org.apache.mina:mina-core:2.0.4
 org.apache.sshd:sshd-core:2.0.0
 org.apache.sshd:sshd-scp:2.0.0
 org.apache.sshd:sshd-sftp:2.0.0
 org.apache.tomcat:tomcat-annotations-api:8.0.5
 org.apiguardian:apiguardian-api:1.1.0
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -340,12 +344,12 @@ org.testcontainers:junit-jupiter:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:selenium:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/gradle/dependency-locks/testRuntimeClasspath.lockfile
+++ b/core/gradle/dependency-locks/testRuntimeClasspath.lockfile
@@ -11,11 +11,11 @@ com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0
 com.eclipsesource.j2v8:j2v8_win32_x86_64:4.6.0
-com.fasterxml.jackson.core:jackson-annotations:2.12.4
-com.fasterxml.jackson.core:jackson-core:2.12.4
-com.fasterxml.jackson.core:jackson-databind:2.12.4
-com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.12.4
-com.fasterxml.jackson:jackson-bom:2.12.4
+com.fasterxml.jackson.core:jackson-annotations:2.13.0
+com.fasterxml.jackson.core:jackson-core:2.13.0
+com.fasterxml.jackson.core:jackson-databind:2.13.0
+com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.13.0
+com.fasterxml.jackson:jackson-bom:2.13.0
 com.fasterxml:classmate:1.5.1
 com.github.docker-java:docker-java-api:3.2.7
 com.github.docker-java:docker-java-transport-zerodep:3.2.7
@@ -30,61 +30,63 @@ com.github.jnr:jnr-unixsocket:0.38.5
 com.github.jnr:jnr-x86asm:1.0.2
 com.google.android:annotations:4.1.1.4
 com.google.api-client:google-api-client-appengine:1.31.3
-com.google.api-client:google-api-client-jackson2:1.31.3
+com.google.api-client:google-api-client-jackson2:1.32.2
 com.google.api-client:google-api-client-java6:1.31.3
 com.google.api-client:google-api-client-servlet:1.31.3
-com.google.api-client:google-api-client:1.32.1
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.125.0
+com.google.api-client:google-api-client:1.32.2
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.132.1
 com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:1.27.1
 com.google.api.grpc:grpc-google-cloud-bigtable-v2:1.27.1
-com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.2.0
-com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:grpc-google-cloud-spanner-v1:6.12.1
-com.google.api.grpc:grpc-google-common-protos:2.3.2
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.1.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.125.0
-com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.125.0
-com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-bigtable-v2:2.1.0
-com.google.api.grpc:proto-google-cloud-datastore-v1:0.91.3
-com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.2
-com.google.api.grpc:proto-google-cloud-pubsub-v1:1.96.2
-com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.2.0
+com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.4.8
+com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:grpc-google-cloud-storage-v2:2.0.1-alpha
+com.google.api.grpc:grpc-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.8.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.132.1
+com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.132.1
+com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-bigtable-v2:2.5.1
+com.google.api.grpc:proto-google-cloud-datastore-v1:0.93.2
+com.google.api.grpc:proto-google-cloud-firestore-v1:3.0.10
+com.google.api.grpc:proto-google-cloud-pubsub-v1:1.97.1
+com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.4.8
 com.google.api.grpc:proto-google-cloud-secretmanager-v1:1.4.0
 com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
-com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.12.1
-com.google.api.grpc:proto-google-cloud-spanner-v1:6.12.1
+com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-spanner-v1:6.17.4
+com.google.api.grpc:proto-google-cloud-storage-v2:2.0.1-alpha
 com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
 com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
-com.google.api.grpc:proto-google-common-protos:2.5.0
-com.google.api.grpc:proto-google-iam-v1:1.1.0
-com.google.api:api-common:2.0.2
-com.google.api:gax-grpc:2.4.1
-com.google.api:gax-httpjson:0.88.0
-com.google.api:gax:2.4.1
+com.google.api.grpc:proto-google-common-protos:2.7.1
+com.google.api.grpc:proto-google-iam-v1:1.2.0
+com.google.api:api-common:2.1.2
+com.google.api:gax-grpc:2.8.1
+com.google.api:gax-httpjson:0.93.1
+com.google.api:gax:2.8.1
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
-com.google.apis:google-api-services-bigquery:v2-rev20210813-1.32.1
+com.google.apis:google-api-services-bigquery:v2-rev20211129-1.32.1
 com.google.apis:google-api-services-clouddebugger:v2-rev20210813-1.32.1
 com.google.apis:google-api-services-cloudkms:v1-rev108-1.25.0
-com.google.apis:google-api-services-cloudresourcemanager:v1-rev20210815-1.32.1
+com.google.apis:google-api-services-cloudresourcemanager:v1-rev20211017-1.32.1
 com.google.apis:google-api-services-dataflow:v1b3-rev20210818-1.32.1
 com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0
 com.google.apis:google-api-services-drive:v2-rev393-1.25.0
 com.google.apis:google-api-services-groupssettings:v1-rev82-1.25.0
-com.google.apis:google-api-services-healthcare:v1-rev20210806-1.32.1
-com.google.apis:google-api-services-iamcredentials:v1-rev20201022-1.31.0
+com.google.apis:google-api-services-healthcare:v1-rev20211016-1.32.1
+com.google.apis:google-api-services-iamcredentials:v1-rev20210326-1.32.1
 com.google.apis:google-api-services-monitoring:v3-rev540-1.25.0
-com.google.apis:google-api-services-pubsub:v1-rev20210809-1.32.1
+com.google.apis:google-api-services-pubsub:v1-rev20211130-1.32.1
 com.google.apis:google-api-services-sheets:v4-rev612-1.25.0
 com.google.apis:google-api-services-sqladmin:v1beta4-rev20210119-1.31.0
-com.google.apis:google-api-services-storage:v1-rev20210127-1.32.1
+com.google.apis:google-api-services-storage:v1-rev20211201-1.32.1
 com.google.appengine.tools:appengine-gcs-client:0.8.1
 com.google.appengine.tools:appengine-mapreduce:0.9
 com.google.appengine.tools:appengine-pipeline:0.2.13
@@ -92,55 +94,54 @@ com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-api-stubs:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:1.1.0
-com.google.auth:google-auth-library-oauth2-http:1.1.0
+com.google.auth:google-auth-library-credentials:1.3.0
+com.google.auth:google-auth-library-oauth2-http:1.3.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.8.2
+com.google.auto.value:auto-value-annotations:1.9
 com.google.auto.value:auto-value:1.7.4
-com.google.cloud.bigdataoss:gcsio:2.2.2
-com.google.cloud.bigdataoss:util:2.2.2
-com.google.cloud.bigtable:bigtable-client-core:1.23.1
-com.google.cloud.bigtable:bigtable-metrics-api:1.23.1
-com.google.cloud.datastore:datastore-v1-proto-client:1.6.3
+com.google.cloud.bigdataoss:gcsio:2.2.4
+com.google.cloud.bigdataoss:util:2.2.4
+com.google.cloud.bigtable:bigtable-client-core:1.25.1
+com.google.cloud.bigtable:bigtable-metrics-api:1.25.1
+com.google.cloud.datastore:datastore-v1-proto-client:2.1.3
 com.google.cloud.sql:jdbc-socket-factory-core:1.2.1
 com.google.cloud.sql:postgres-socket-factory:1.2.1
-com.google.cloud:google-cloud-bigquery:2.1.2
-com.google.cloud:google-cloud-bigquerystorage:2.1.0
+com.google.cloud:google-cloud-bigquerystorage:2.8.1
 com.google.cloud:google-cloud-bigtable:1.27.1
-com.google.cloud:google-cloud-core-grpc:2.1.0
-com.google.cloud:google-cloud-core-http:2.1.0
-com.google.cloud:google-cloud-core:2.1.0
-com.google.cloud:google-cloud-firestore:3.0.2
+com.google.cloud:google-cloud-core-grpc:2.3.5
+com.google.cloud:google-cloud-core-http:1.95.4
+com.google.cloud:google-cloud-core:2.3.5
+com.google.cloud:google-cloud-firestore:3.0.10
 com.google.cloud:google-cloud-nio:0.123.4
-com.google.cloud:google-cloud-pubsub:1.114.2
-com.google.cloud:google-cloud-pubsublite:1.2.0
+com.google.cloud:google-cloud-pubsub:1.115.1
+com.google.cloud:google-cloud-pubsublite:1.4.8
 com.google.cloud:google-cloud-secretmanager:1.4.0
-com.google.cloud:google-cloud-spanner:6.12.1
+com.google.cloud:google-cloud-spanner:6.17.4
 com.google.cloud:google-cloud-storage:1.118.0
 com.google.cloud:google-cloud-tasks:1.33.2
 com.google.cloud:grpc-gcp:1.1.0
-com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.2
+com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.0.10
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.8
+com.google.code.gson:gson:2.8.9
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.9.0
+com.google.errorprone:error_prone_annotations:2.10.0
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flatbuffers:flatbuffers-java:1.12.0
-com.google.flogger:flogger-system-backend:0.7.3
-com.google.flogger:flogger:0.7.3
-com.google.flogger:google-extensions:0.6
+com.google.flogger:flogger-system-backend:0.7.4
+com.google.flogger:flogger:0.7.4
+com.google.flogger:google-extensions:0.7.4
 com.google.guava:failureaccess:1.0.1
 com.google.guava:guava-testlib:30.1.1-jre
-com.google.guava:guava:30.1.1-jre
+com.google.guava:guava:31.0.1-jre
 com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
-com.google.http-client:google-http-client-apache-v2:1.39.2
+com.google.http-client:google-http-client-apache-v2:1.40.1
 com.google.http-client:google-http-client-appengine:1.39.2
-com.google.http-client:google-http-client-gson:1.40.0
-com.google.http-client:google-http-client-jackson2:1.39.2
-com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.40.0
+com.google.http-client:google-http-client-gson:1.41.0
+com.google.http-client:google-http-client-jackson2:1.41.0
+com.google.http-client:google-http-client-protobuf:1.40.1
+com.google.http-client:google-http-client:1.41.0
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -152,9 +153,9 @@ com.google.oauth-client:google-oauth-client-appengine:1.31.4
 com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
-com.google.oauth-client:google-oauth-client:1.31.5
-com.google.protobuf:protobuf-java-util:3.17.3
-com.google.protobuf:protobuf-java:3.17.3
+com.google.oauth-client:google-oauth-client:1.32.1
+com.google.protobuf:protobuf-java-util:3.19.2
+com.google.protobuf:protobuf-java:3.19.2
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.google.truth.extensions:truth-java8-extension:1.1.2
@@ -183,31 +184,34 @@ io.confluent:kafka-schema-registry-client:5.3.2
 io.dropwizard.metrics:metrics-core:3.1.2
 io.github.classgraph:classgraph:4.8.104
 io.github.java-diff-utils:java-diff-utils:4.9
-io.grpc:grpc-alts:1.40.1
-io.grpc:grpc-api:1.40.1
-io.grpc:grpc-auth:1.40.1
-io.grpc:grpc-context:1.40.1
-io.grpc:grpc-core:1.40.1
-io.grpc:grpc-grpclb:1.40.1
-io.grpc:grpc-netty-shaded:1.40.1
-io.grpc:grpc-netty:1.40.0
-io.grpc:grpc-protobuf-lite:1.40.1
-io.grpc:grpc-protobuf:1.40.1
-io.grpc:grpc-stub:1.40.1
-io.netty:netty-buffer:4.1.52.Final
-io.netty:netty-codec-http2:4.1.52.Final
-io.netty:netty-codec-http:4.1.52.Final
-io.netty:netty-codec-socks:4.1.52.Final
-io.netty:netty-codec:4.1.52.Final
-io.netty:netty-common:4.1.52.Final
-io.netty:netty-handler-proxy:4.1.52.Final
-io.netty:netty-handler:4.1.52.Final
-io.netty:netty-resolver:4.1.52.Final
+io.grpc:grpc-alts:1.43.2
+io.grpc:grpc-api:1.43.2
+io.grpc:grpc-auth:1.43.2
+io.grpc:grpc-context:1.43.2
+io.grpc:grpc-core:1.43.2
+io.grpc:grpc-grpclb:1.43.2
+io.grpc:grpc-netty-shaded:1.43.2
+io.grpc:grpc-netty:1.43.2
+io.grpc:grpc-protobuf-lite:1.43.2
+io.grpc:grpc-protobuf:1.43.2
+io.grpc:grpc-services:1.43.2
+io.grpc:grpc-stub:1.43.2
+io.grpc:grpc-xds:1.43.2
+io.netty:netty-buffer:4.1.63.Final
+io.netty:netty-codec-http2:4.1.63.Final
+io.netty:netty-codec-http:4.1.63.Final
+io.netty:netty-codec-socks:4.1.63.Final
+io.netty:netty-codec:4.1.63.Final
+io.netty:netty-common:4.1.63.Final
+io.netty:netty-handler-proxy:4.1.63.Final
+io.netty:netty-handler:4.1.63.Final
+io.netty:netty-resolver:4.1.63.Final
 io.netty:netty-tcnative-boringssl-static:2.0.33.Final
-io.netty:netty-transport:4.1.52.Final
-io.opencensus:opencensus-api:0.28.0
-io.opencensus:opencensus-contrib-grpc-util:0.28.0
+io.netty:netty-transport:4.1.63.Final
+io.opencensus:opencensus-api:0.30.0
+io.opencensus:opencensus-contrib-grpc-util:0.30.0
 io.opencensus:opencensus-contrib-http-util:0.28.0
+io.opencensus:opencensus-proto:0.2.0
 io.perfmark:perfmark-api:0.23.0
 it.unimi.dsi:fastutil:6.5.16
 javax.activation:activation:1.1
@@ -233,24 +237,24 @@ org.apache.arrow:arrow-format:5.0.0
 org.apache.arrow:arrow-memory-core:5.0.0
 org.apache.arrow:arrow-vector:5.0.0
 org.apache.avro:avro:1.8.2
-org.apache.beam:beam-model-fn-execution:2.34.0
-org.apache.beam:beam-model-job-management:2.34.0
-org.apache.beam:beam-model-pipeline:2.34.0
-org.apache.beam:beam-runners-core-construction-java:2.34.0
-org.apache.beam:beam-runners-core-java:2.34.0
-org.apache.beam:beam-runners-direct-java:2.34.0
-org.apache.beam:beam-runners-google-cloud-dataflow-java:2.34.0
-org.apache.beam:beam-runners-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-core:2.34.0
-org.apache.beam:beam-sdks-java-expansion-service:2.34.0
-org.apache.beam:beam-sdks-java-extensions-arrow:2.34.0
-org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.34.0
-org.apache.beam:beam-sdks-java-extensions-protobuf:2.34.0
-org.apache.beam:beam-sdks-java-fn-execution:2.34.0
-org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.34.0
-org.apache.beam:beam-sdks-java-io-kafka:2.34.0
+org.apache.beam:beam-model-fn-execution:2.37.0
+org.apache.beam:beam-model-job-management:2.37.0
+org.apache.beam:beam-model-pipeline:2.37.0
+org.apache.beam:beam-runners-core-construction-java:2.37.0
+org.apache.beam:beam-runners-core-java:2.37.0
+org.apache.beam:beam-runners-direct-java:2.37.0
+org.apache.beam:beam-runners-google-cloud-dataflow-java:2.37.0
+org.apache.beam:beam-runners-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-core:2.37.0
+org.apache.beam:beam-sdks-java-expansion-service:2.37.0
+org.apache.beam:beam-sdks-java-extensions-arrow:2.37.0
+org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.37.0
+org.apache.beam:beam-sdks-java-extensions-protobuf:2.37.0
+org.apache.beam:beam-sdks-java-fn-execution:2.37.0
+org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.37.0
+org.apache.beam:beam-sdks-java-io-kafka:2.37.0
 org.apache.beam:beam-vendor-bytebuddy-1_11_0:0.1
-org.apache.beam:beam-vendor-grpc-1_36_0:0.2
+org.apache.beam:beam-vendor-grpc-1_43_2:0.1
 org.apache.beam:beam-vendor-guava-26_0-jre:0.1
 org.apache.commons:commons-compress:1.20
 org.apache.commons:commons-exec:1.3
@@ -259,18 +263,18 @@ org.apache.commons:commons-text:1.6
 org.apache.ftpserver:ftplet-api:1.0.6
 org.apache.ftpserver:ftpserver-core:1.0.6
 org.apache.httpcomponents:httpclient:4.5.13
-org.apache.httpcomponents:httpcore:4.4.14
+org.apache.httpcomponents:httpcore:4.4.15
 org.apache.mina:mina-core:2.0.4
 org.apache.sshd:sshd-core:2.0.0
 org.apache.sshd:sshd-scp:2.0.0
 org.apache.sshd:sshd-sftp:2.0.0
 org.apache.tomcat:tomcat-annotations-api:8.0.5
 org.apiguardian:apiguardian-api:1.1.0
-org.bouncycastle:bcpg-jdk15on:1.61
-org.bouncycastle:bcpkix-jdk15on:1.61
-org.bouncycastle:bcprov-jdk15on:1.61
+org.bouncycastle:bcpg-jdk15on:1.67
+org.bouncycastle:bcpkix-jdk15on:1.67
+org.bouncycastle:bcprov-jdk15on:1.67
 org.checkerframework:checker-compat-qual:2.5.5
-org.checkerframework:checker-qual:3.18.0
+org.checkerframework:checker-qual:3.21.0
 org.codehaus.jackson:jackson-core-asl:1.9.13
 org.codehaus.jackson:jackson-mapper-asl:1.9.13
 org.codehaus.mojo:animal-sniffer-annotations:1.20
@@ -341,12 +345,12 @@ org.testcontainers:junit-jupiter:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:selenium:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.1
+org.threeten:threetenbp:1.5.2
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
 org.xerial.snappy:snappy-java:1.1.8.4
-org.yaml:snakeyaml:1.27
+org.yaml:snakeyaml:1.28
 us.fatehi:schemacrawler-api:16.10.1
 us.fatehi:schemacrawler-diagram:16.10.1
 us.fatehi:schemacrawler-tools:16.10.1
--- a/core/src/main/java/google/registry/backup/CommitLogCheckpointAction.java
+++ b/core/src/main/java/google/registry/backup/CommitLogCheckpointAction.java
@@ -28,11 +28,11 @@ import google.registry.model.ofy.CommitLogCheckpointRoot;
 import google.registry.request.Action;
 import google.registry.request.Action.Service;
 import google.registry.request.auth.Auth;
-import google.registry.util.Clock;
 import google.registry.util.CloudTasksUtils;
 import java.util.Optional;
 import javax.inject.Inject;
 import org.joda.time.DateTime;
+import org.joda.time.Duration;

 /**
 * Action that saves commit log checkpoints to Datastore and kicks off a diff export task.
@@ -57,7 +57,22 @@ public final class CommitLogCheckpointAction implements Runnable {

  private static final String QUEUE_NAME = "export-commits";

-  @Inject Clock clock;
+  /**
+   * The amount of time enqueueing should be delayed.
+   *
+   * <p>The {@link ExportCommitLogDiffAction} is enqueued in {@link CommitLogCheckpointAction},
+   * which is inside a Datastore transaction that persists the checkpoint to be exported. After the
+   * switch to CloudTasks API, the task may be invoked before the Datastore transaction commits.
+   * When this happens, the checkpoint is not found which leads to {@link
+   * com.google.common.base.VerifyException}.
+   *
+   * <p>In order to invoke the task after the transaction commits, a reasonable delay should be
+   * added to each task. The latency of the request is mostly in the range of 4-6 seconds; Choosing
+   * a value 30% greater than the upper bound should solve the issue invoking a task before the
+   * transaction commits.
+   */
+  static final Duration ENQUEUE_DELAY_SECONDS = Duration.standardSeconds(8);
+
  @Inject CommitLogCheckpointStrategy strategy;
  @Inject CloudTasksUtils cloudTasksUtils;

@@ -96,14 +111,15 @@ public final class CommitLogCheckpointAction implements Runnable {
                  // Enqueue a diff task between previous and current checkpoints.
                  cloudTasksUtils.enqueue(
                      QUEUE_NAME,
-                      cloudTasksUtils.createPostTask(
+                      cloudTasksUtils.createPostTaskWithDelay(
                          ExportCommitLogDiffAction.PATH,
                          Service.BACKEND.toString(),
                          ImmutableMultimap.of(
                              LOWER_CHECKPOINT_TIME_PARAM,
                              lastWrittenTime.toString(),
                              UPPER_CHECKPOINT_TIME_PARAM,
-                              checkpoint.getCheckpointTime().toString())));
+                              checkpoint.getCheckpointTime().toString()),
+                          ENQUEUE_DELAY_SECONDS));
                  return true;
                });
    return isCheckPointPersisted ? Optional.of(checkpoint) : Optional.empty();
--- a/core/src/main/java/google/registry/backup/SyncDatastoreToSqlSnapshotAction.java
+++ b/core/src/main/java/google/registry/backup/SyncDatastoreToSqlSnapshotAction.java
@@ -29,6 +29,8 @@ import google.registry.request.Parameter;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
 import google.registry.util.Sleeper;
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
 import java.util.Optional;
 import javax.inject.Inject;
 import org.joda.time.DateTime;
@@ -114,9 +116,22 @@ public class SyncDatastoreToSqlSnapshotAction implements Runnable {
      response.setPayload(
          String.format(SUCCESS_RESPONSE_TEMPLATE, sqlSnapshotId, checkpoint.getCheckpointTime()));
      return;
-    } catch (Exception e) {
+    } catch (Throwable e) {
+      logger.atSevere().withCause(e).log("Failed to sync Datastore to SQL.");
      response.setStatus(SC_INTERNAL_SERVER_ERROR);
-      response.setPayload(e.getMessage());
+      response.setPayload(getStackTrace(e));
+    }
+  }
+
+  private static String getStackTrace(Throwable e) {
+    try {
+      ByteArrayOutputStream bis = new ByteArrayOutputStream();
+      PrintStream printStream = new PrintStream(bis);
+      e.printStackTrace(printStream);
+      printStream.close();
+      return bis.toString();
+    } catch (RuntimeException re) {
+      return re.getMessage();
    }
  }

--- a/core/src/main/java/google/registry/batch/AsyncTaskEnqueuer.java
+++ b/core/src/main/java/google/registry/batch/AsyncTaskEnqueuer.java
@@ -22,14 +22,17 @@ import com.google.appengine.api.taskqueue.TaskOptions;
 import com.google.appengine.api.taskqueue.TaskOptions.Method;
 import com.google.appengine.api.taskqueue.TransientFailureException;
 import com.google.common.base.Joiner;
+import com.google.common.collect.ArrayListMultimap;
 import com.google.common.collect.ImmutableSortedSet;
+import com.google.common.collect.Multimap;
 import com.google.common.flogger.FluentLogger;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.model.EppResource;
 import google.registry.model.eppcommon.Trid;
 import google.registry.model.host.HostResource;
 import google.registry.persistence.VKey;
-import google.registry.util.AppEngineServiceUtils;
+import google.registry.request.Action.Service;
+import google.registry.util.CloudTasksUtils;
 import google.registry.util.Retrier;
 import javax.inject.Inject;
 import javax.inject.Named;
@@ -58,25 +61,23 @@ public final class AsyncTaskEnqueuer {
  private static final Duration MAX_ASYNC_ETA = Duration.standardDays(30);

  private final Duration asyncDeleteDelay;
-  private final Queue asyncActionsPushQueue;
  private final Queue asyncDeletePullQueue;
  private final Queue asyncDnsRefreshPullQueue;
-  private final AppEngineServiceUtils appEngineServiceUtils;
  private final Retrier retrier;

+  private CloudTasksUtils cloudTasksUtils;
+
  @Inject
  public AsyncTaskEnqueuer(
-      @Named(QUEUE_ASYNC_ACTIONS) Queue asyncActionsPushQueue,
      @Named(QUEUE_ASYNC_DELETE) Queue asyncDeletePullQueue,
      @Named(QUEUE_ASYNC_HOST_RENAME) Queue asyncDnsRefreshPullQueue,
      @Config("asyncDeleteFlowMapreduceDelay") Duration asyncDeleteDelay,
-      AppEngineServiceUtils appEngineServiceUtils,
+      CloudTasksUtils cloudTasksUtils,
      Retrier retrier) {
-    this.asyncActionsPushQueue = asyncActionsPushQueue;
    this.asyncDeletePullQueue = asyncDeletePullQueue;
    this.asyncDnsRefreshPullQueue = asyncDnsRefreshPullQueue;
    this.asyncDeleteDelay = asyncDeleteDelay;
-    this.appEngineServiceUtils = appEngineServiceUtils;
+    this.cloudTasksUtils = cloudTasksUtils;
    this.retrier = retrier;
  }

@@ -102,19 +103,17 @@ public final class AsyncTaskEnqueuer {
          entityKey, firstResave, MAX_ASYNC_ETA);
      return;
    }
-    logger.atInfo().log("Enqueuing async re-save of %s to run at %s.", entityKey, whenToResave);
-    String backendHostname = appEngineServiceUtils.getServiceHostname("backend");
-    TaskOptions task =
-        TaskOptions.Builder.withUrl(ResaveEntityAction.PATH)
-            .method(Method.POST)
-            .header("Host", backendHostname)
-            .countdownMillis(etaDuration.getMillis())
-            .param(PARAM_RESOURCE_KEY, entityKey.stringify())
-            .param(PARAM_REQUESTED_TIME, now.toString());
+    Multimap<String, String> params = ArrayListMultimap.create();
+    params.put(PARAM_RESOURCE_KEY, entityKey.stringify());
+    params.put(PARAM_REQUESTED_TIME, now.toString());
    if (whenToResave.size() > 1) {
-      task.param(PARAM_RESAVE_TIMES, Joiner.on(',').join(whenToResave.tailSet(firstResave, false)));
+      params.put(PARAM_RESAVE_TIMES, Joiner.on(',').join(whenToResave.tailSet(firstResave, false)));
    }
-    addTaskToQueueWithRetry(asyncActionsPushQueue, task);
+    logger.atInfo().log("Enqueuing async re-save of %s to run at %s.", entityKey, whenToResave);
+    cloudTasksUtils.enqueue(
+        QUEUE_ASYNC_ACTIONS,
+        cloudTasksUtils.createPostTaskWithDelay(
+            ResaveEntityAction.PATH, Service.BACKEND.toString(), params, etaDuration));
  }

  /** Enqueues a task to asynchronously delete a contact or host, by key. */
--- a/core/src/main/java/google/registry/beam/common/RegistryJpaIO.java
+++ b/core/src/main/java/google/registry/beam/common/RegistryJpaIO.java
@@ -23,6 +23,7 @@ import com.google.common.collect.Streams;
 import google.registry.beam.common.RegistryQuery.CriteriaQuerySupplier;
 import google.registry.model.UpdateAutoTimestamp;
 import google.registry.model.UpdateAutoTimestamp.DisableAutoUpdateResource;
+import google.registry.model.common.DatabaseMigrationStateSchedule;
 import google.registry.model.replay.SqlEntity;
 import google.registry.persistence.transaction.JpaTransactionManager;
 import google.registry.persistence.transaction.TransactionManagerFactory;
@@ -234,6 +235,10 @@ public final class RegistryJpaIO {

      @ProcessElement
      public void processElement(OutputReceiver<T> outputReceiver) {
+        // Preload the migration schedule into cache, otherwise the cache loading query may happen
+        // before the setDatabaseSnapshot call in the transaction below, causing it to fail.
+        DatabaseMigrationStateSchedule.get();
+
        jpaTm()
            .transactNoRetry(
                () -> {
--- a/core/src/main/java/google/registry/beam/comparedb/ValidateSqlUtils.java
+++ b/core/src/main/java/google/registry/beam/comparedb/ValidateSqlUtils.java
@@ -20,6 +20,8 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
 import com.google.common.base.Preconditions;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Maps.EntryTransformer;
 import google.registry.beam.initsql.Transforms;
 import google.registry.config.RegistryEnvironment;
 import google.registry.model.EppResource;
@@ -37,19 +39,24 @@ import google.registry.model.registrar.Registrar;
 import google.registry.model.replay.SqlEntity;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.util.DiffUtils;
+import java.io.Serializable;
 import java.lang.reflect.Field;
 import java.math.BigInteger;
+import java.util.Collection;
 import java.util.HashMap;
+import java.util.LinkedHashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
+import java.util.stream.Collectors;
 import org.apache.beam.sdk.metrics.Counter;
 import org.apache.beam.sdk.metrics.Metrics;
 import org.apache.beam.sdk.transforms.DoFn;
 import org.apache.beam.sdk.values.KV;
 import org.apache.beam.sdk.values.TupleTag;
+import org.checkerframework.checker.nullness.qual.Nullable;

 /** Helpers for use by {@link ValidateDatabasePipeline}. */
@DeleteAfterMigration
@@ -194,13 +201,52 @@ final class ValidateSqlUtils {
        return;
      }

-      if (!Objects.equals(entity0, entity1)) {
+      Map<String, Object> fields0 =
+          Maps.transformEntries(
+              ((ImmutableObject) entity0).toDiffableFieldMap(), new DiffableFieldNormalizer());
+      Map<String, Object> fields1 =
+          Maps.transformEntries(
+              ((ImmutableObject) entity1).toDiffableFieldMap(), new DiffableFieldNormalizer());
+      if (!Objects.equals(fields0, fields1)) {
        unequalCounters.get(counterKey).inc();
-        out.output(unEqualEntityLog(kv.getKey(), entities.get(0), entities.get(1)));
+        out.output(kv.getKey() + "  " + DiffUtils.prettyPrintEntityDeepDiff(fields0, fields1));
      }
    }
  }

+  /**
+   * Normalizes trivial differences between objects persisted in Datastore and SQL.
+   *
+   * <p>This class works on a map generated by {@link ImmutableObject#toDiffableFieldMap}, and
+   * performs the following changes:
+   *
+   * <ul>
+   *   <li>If a value is an empty {@link Collection}, it is converted to null
+   *   <li>For each {@link google.registry.model.eppcommon.Address} object, empty strings are
+   *       removed from its {@code string} field, which is a {@link List}.
+   * </ul>
+   */
+  static class DiffableFieldNormalizer
+      implements EntryTransformer<String, Object, Object>, Serializable {
+
+    @Override
+    public Object transformEntry(String key, @Nullable Object value) {
+      if (value instanceof Collection && ((Collection<?>) value).isEmpty()) {
+        return null;
+      }
+      if (key.equals("street") && value instanceof List) {
+        return ((List<?>) value)
+            .stream().filter(v -> v != null && !Objects.equals("", v)).collect(Collectors.toList());
+      }
+      // Short-term hack: LinkedHashMap<String, ?> represents a child ImmutableObject instance.
+      if (value instanceof LinkedHashMap
+          && ((LinkedHashMap<?, ?>) value).keySet().stream().anyMatch(e -> e instanceof String)) {
+        return Maps.transformEntries((Map<String, Object>) value, this);
+      }
+      return value;
+    }
+  }
+
  static SqlEntity normalizeEntity(SqlEntity sqlEntity) {
    if (sqlEntity instanceof EppResource) {
      return normalizeEppResource(sqlEntity);
--- a/core/src/main/java/google/registry/beam/rde/RdeIO.java
+++ b/core/src/main/java/google/registry/beam/rde/RdeIO.java
@@ -68,6 +68,7 @@ import org.apache.beam.sdk.values.PDone;
 import org.bouncycastle.jce.provider.BouncyCastleProvider;
 import org.bouncycastle.openpgp.PGPPublicKey;
 import org.joda.time.DateTime;
+import org.joda.time.Duration;

 public class RdeIO {

@@ -261,6 +262,7 @@ public class RdeIO {

    private static final FluentLogger logger = FluentLogger.forEnclosingClass();
    private static final long serialVersionUID = 5822176227753327224L;
+    private static final Duration ENQUEUE_DELAY = Duration.standardMinutes(1);

    private final CloudTasksUtils cloudTasksUtils;

@@ -271,10 +273,10 @@ public class RdeIO {
    @ProcessElement
    public void processElement(
        @Element KV<PendingDeposit, Integer> input, PipelineOptions options) {
+
      tm().transact(
              () -> {
                PendingDeposit key = input.getKey();
-                int revision = input.getValue();
                Registry registry = Registry.get(key.tld());
                Optional<Cursor> cursor =
                    transactIfJpaTm(
@@ -296,26 +298,30 @@ public class RdeIO {
                tm().put(Cursor.create(key.cursor(), newPosition, registry));
                logger.atInfo().log(
                    "Rolled forward %s on %s cursor to %s.", key.cursor(), key.tld(), newPosition);
-                RdeRevision.saveRevision(key.tld(), key.watermark(), key.mode(), revision);
+                RdeRevision.saveRevision(key.tld(), key.watermark(), key.mode(), input.getValue());
                // Enqueueing a task is a side effect that is not undone if the transaction rolls
                // back. So this may result in multiple copies of the same task being processed.
                // This is fine because the RdeUploadAction is guarded by a lock and tracks progress
-                // by cursor. The BrdaCopyAction writes a file to GCS, which is an atomic action.
+                // by cursor. The BrdaCopyAction writes a file to GCS, which is an atomic action. It
+                // is also guarded by a cursor to not run before the cursor is updated. We also
+                // include a delay to minimize the chance that the enqueued job executes before the
+                // transaction is committed, which triggers a retry.
                if (key.mode() == RdeMode.FULL) {
                  cloudTasksUtils.enqueue(
                      RDE_UPLOAD_QUEUE,
-                      cloudTasksUtils.createPostTask(
+                      cloudTasksUtils.createPostTaskWithDelay(
                          RdeUploadAction.PATH,
                          Service.BACKEND.getServiceId(),
                          ImmutableMultimap.of(
                              RequestParameters.PARAM_TLD,
                              key.tld(),
                              RdeModule.PARAM_PREFIX,
-                              options.getJobName() + '/')));
+                              options.getJobName() + '/'),
+                          ENQUEUE_DELAY));
                } else {
                  cloudTasksUtils.enqueue(
                      BRDA_QUEUE,
-                      cloudTasksUtils.createPostTask(
+                      cloudTasksUtils.createPostTaskWithDelay(
                          BrdaCopyAction.PATH,
                          Service.BACKEND.getServiceId(),
                          ImmutableMultimap.of(
@@ -324,7 +330,8 @@ public class RdeIO {
                              RdeModule.PARAM_WATERMARK,
                              key.watermark().toString(),
                              RdeModule.PARAM_PREFIX,
-                              options.getJobName() + '/')));
+                              options.getJobName() + '/'),
+                          ENQUEUE_DELAY));
                }
              });
    }
--- a/core/src/main/java/google/registry/config/RegistryConfig.java
+++ b/core/src/main/java/google/registry/config/RegistryConfig.java
@@ -576,6 +576,19 @@ public final class RegistryConfig {
      return config.beam.highPerformanceMachineType;
    }

+    /**
+     * Returns initial number of workers used for a Beam pipeline. Autoscaling can still in effect.
+     *
+     * @see <a
+     *     href=https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#horizontal-autoscaling>
+     *     Horizontal Autoscaling </a>
+     */
+    @Provides
+    @Config("initialWorkerCount")
+    public static int provideInitialWorkerCount(RegistryConfigSettings config) {
+      return config.beam.initialWorkerCount;
+    }
+
    /**
     * Returns the default job region to run Apache Beam (Cloud Dataflow) jobs in.
     *
--- a/core/src/main/java/google/registry/config/RegistryConfigSettings.java
+++ b/core/src/main/java/google/registry/config/RegistryConfigSettings.java
@@ -137,6 +137,7 @@ public class RegistryConfigSettings {
  public static class Beam {
    public String defaultJobRegion;
    public String highPerformanceMachineType;
+    public int initialWorkerCount;
    public String stagingBucketUrl;
  }

--- a/core/src/main/java/google/registry/config/files/default-config.yaml
+++ b/core/src/main/java/google/registry/config/files/default-config.yaml
@@ -442,6 +442,11 @@ beam:
  # core count per machine may be preferable in order to preserve IP addresses.
  # See: https://cloud.google.com/compute/quotas#cpu_quota
  highPerformanceMachineType: n2-standard-4
+  # The initial number of workers requested. This can help speed up the pipeline
+  # which otherwise would take some time to spin up the necessary number of
+  # works. Autoscaling is still in effect to reduce the number of workers if
+  # not in use.
+  initialWorkerCount: 24
  stagingBucketUrl: gcs-bucket-with-staged-templates

 keyring:
@@ -486,9 +491,9 @@ sslCertificateValidation:
  # The minimum number of days between two successive expiring notification emails.
  expirationWarningIntervalDays: 15
  # Text for expiring certificate notification email subject.
-  expirationWarningEmailSubjectText: "[Important] Expiring SSL certificate for Google Registry EPP connection"
-  # Text for expiring certificate notification email body that accepts 3 parameters:
-  # registrar name, certificate type, and expiration date, respectively.
+  expirationWarningEmailSubjectText: "Expiring SSL certificate for Example Registry EPP connection"
+  # Text for expiring certificate notification email body that accepts 4 parameters:
+  # registrar name, certificate type, expiration date and registrar id, respectively.
  expirationWarningEmailBodyText: |
    Dear %1$s,

@@ -496,25 +501,15 @@ sslCertificateValidation:

    Kindly update your production account certificate within the support console using the following steps:

-      1. Navigate to support.registry.google and login using your %4$s@registry.google credentials.
-          * If this is your first time logging in, you will be prompted to reset your password, so please keep your new password safe.
-          * If you are already logged in with some other Google account(s) but not your %4$s@registry.google account, you need to click on
-          “Add Account” and login using your %4$s@registry.google credentials.
+      1. Navigate to support.registry.example and login using your %4$s@registry.example credentials.
      2. Select “Settings > Security” from the left navigation bar.
      3. Click “Edit” on the top left corner.
-      4. Enter your full certificate string (including lines -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----) in the box.
-      5. Click “Save”. If there are validation issues with the form, you will be prompted to fix them and click “Save” again.
+      4. Enter your full certificate string in the box.
+      5. Click “Save”.

-    A failover SSL certificate can also be added in order to prevent connection issues once your main certificate expires. Connecting with either of the certificates will work with our production EPP server.
-
-    Further information about our EPP connection requirements can be found in section 9.2 in the updated Technical Guide in your Google Drive folder.
-
-    Note that account certificate changes take a few minutes to become effective and that the existing connections will remain unaffected by the change.
-
-    If you also would like to update your OT&E account certificate, please send an email from your primary or technical contact to registry-support@google.com and include the full certificate string (including lines -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----).

    Regards,
-    Google Registry
+    Example Registry

  # The minimum number of bits an RSA key must contain.
  minimumRsaKeyLength: 2048
--- a/core/src/main/java/google/registry/export/BackupDatastoreAction.java
+++ b/core/src/main/java/google/registry/export/BackupDatastoreAction.java
@@ -14,18 +14,22 @@

 package google.registry.export;

-import static google.registry.export.CheckBackupAction.enqueuePollTask;
 import static google.registry.request.Action.Method.POST;

+import com.google.common.base.Joiner;
+import com.google.common.collect.ImmutableMultimap;
 import com.google.common.flogger.FluentLogger;
 import google.registry.config.RegistryConfig;
 import google.registry.export.datastore.DatastoreAdmin;
 import google.registry.export.datastore.Operation;
 import google.registry.model.annotations.DeleteAfterMigration;
 import google.registry.request.Action;
+import google.registry.request.Action.Service;
 import google.registry.request.HttpException.InternalServerErrorException;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
+import google.registry.util.Clock;
+import google.registry.util.CloudTasksUtils;
 import javax.inject.Inject;

 /**
@@ -59,6 +63,8 @@ public class BackupDatastoreAction implements Runnable {

  @Inject DatastoreAdmin datastoreAdmin;
  @Inject Response response;
+  @Inject Clock clock;
+  @Inject CloudTasksUtils cloudTasksUtils;

  @Inject
  BackupDatastoreAction() {}
@@ -73,8 +79,19 @@ public class BackupDatastoreAction implements Runnable {
              .execute();

      String backupName = backup.getName();
-      // Enqueue a poll task to monitor the backup and load REPORTING-related kinds into bigquery.
-      enqueuePollTask(backupName, AnnotatedEntities.getReportingKinds());
+      // Enqueue a poll task to monitor the backup for completion and load reporting-related kinds
+      // into bigquery.
+      cloudTasksUtils.enqueue(
+          CheckBackupAction.QUEUE,
+          cloudTasksUtils.createPostTaskWithDelay(
+              CheckBackupAction.PATH,
+              Service.BACKEND.toString(),
+              ImmutableMultimap.of(
+                  CheckBackupAction.CHECK_BACKUP_NAME_PARAM,
+                  backupName,
+                  CheckBackupAction.CHECK_BACKUP_KINDS_TO_LOAD_PARAM,
+                  Joiner.on(',').join(AnnotatedEntities.getReportingKinds())),
+              CheckBackupAction.POLL_COUNTDOWN));
      String message =
          String.format(
              "Datastore backup started with name: %s\nSaving to %s",
--- a/core/src/main/java/google/registry/export/BigqueryPollJobAction.java
+++ b/core/src/main/java/google/registry/export/BigqueryPollJobAction.java
@@ -14,18 +14,14 @@

 package google.registry.export;

-import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
-import static com.google.appengine.api.taskqueue.TaskOptions.Builder.withUrl;
 import static google.registry.bigquery.BigqueryUtils.toJobReferenceString;

 import com.google.api.services.bigquery.Bigquery;
 import com.google.api.services.bigquery.model.Job;
 import com.google.api.services.bigquery.model.JobReference;
-import com.google.appengine.api.taskqueue.Queue;
-import com.google.appengine.api.taskqueue.TaskHandle;
-import com.google.appengine.api.taskqueue.TaskOptions;
-import com.google.appengine.api.taskqueue.TaskOptions.Method;
+import com.google.cloud.tasks.v2.Task;
 import com.google.common.flogger.FluentLogger;
+import com.google.protobuf.ByteString;
 import dagger.Lazy;
 import google.registry.request.Action;
 import google.registry.request.Header;
@@ -33,12 +29,10 @@ import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.HttpException.NotModifiedException;
 import google.registry.request.Payload;
 import google.registry.request.auth.Auth;
-import google.registry.util.TaskQueueUtils;
+import google.registry.util.CloudTasksUtils;
 import java.io.ByteArrayInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.IOException;
 import java.io.ObjectInputStream;
-import java.io.ObjectOutputStream;
 import javax.inject.Inject;
 import org.joda.time.Duration;

@@ -67,11 +61,14 @@ public class BigqueryPollJobAction implements Runnable {
  static final Duration POLL_COUNTDOWN = Duration.standardSeconds(20);

  @Inject Bigquery bigquery;
-  @Inject TaskQueueUtils taskQueueUtils;
+  @Inject CloudTasksUtils cloudTasksUtils;
+
  @Inject @Header(CHAINED_TASK_QUEUE_HEADER) Lazy<String> chainedQueueName;
  @Inject @Header(PROJECT_ID_HEADER) String projectId;
  @Inject @Header(JOB_ID_HEADER) String jobId;
-  @Inject @Payload byte[] payload;
+
+  @Inject @Payload ByteString payload;
+
  @Inject BigqueryPollJobAction() {}

  @Override
@@ -79,20 +76,25 @@ public class BigqueryPollJobAction implements Runnable {
    boolean jobOutcome =
        checkJobOutcome(); // Throws a NotModifiedException if the job hasn't completed.
    // If the job failed, do not enqueue the next step.
-    if (!jobOutcome || payload == null || payload.length == 0) {
+    if (!jobOutcome || payload == null || payload.size() == 0) {
      return;
    }
    // If there is a payload, it's a chained task, so enqueue it.
-    TaskOptions task;
+    Task task;
    try {
-      task = (TaskOptions) new ObjectInputStream(new ByteArrayInputStream(payload)).readObject();
+      task =
+          (Task)
+              new ObjectInputStream(new ByteArrayInputStream(payload.toByteArray())).readObject();
    } catch (ClassNotFoundException | IOException e) {
      throw new BadRequestException("Cannot deserialize task from payload", e);
    }
-    String taskName = taskQueueUtils.enqueue(getQueue(chainedQueueName.get()), task).getName();
+    Task enqueuedTask = cloudTasksUtils.enqueue(chainedQueueName.get(), task);
    logger.atInfo().log(
        "Added chained task %s for %s to queue %s: %s",
-        taskName, task.getUrl(), chainedQueueName.get(), task);
+        enqueuedTask.getName(),
+        enqueuedTask.getAppEngineHttpRequest().getRelativeUri(),
+        chainedQueueName.get(),
+        enqueuedTask);
  }

  /**
@@ -124,51 +126,4 @@ public class BigqueryPollJobAction implements Runnable {
    logger.atInfo().log("Bigquery job succeeded - %s.", jobRefString);
    return true;
  }
-
-
-  /** Helper class to enqueue a bigquery poll job. */
-  public static class BigqueryPollJobEnqueuer {
-
-    private final TaskQueueUtils taskQueueUtils;
-
-    @Inject
-    BigqueryPollJobEnqueuer(TaskQueueUtils taskQueueUtils) {
-      this.taskQueueUtils = taskQueueUtils;
-    }
-
-    /** Enqueue a task to poll for the success or failure of the referenced BigQuery job. */
-    public TaskHandle enqueuePollTask(JobReference jobRef) {
-      return taskQueueUtils.enqueue(
-          getQueue(QUEUE), createCommonPollTask(jobRef).method(Method.GET));
-    }
-
-    /**
-     * Enqueue a task to poll for the success or failure of the referenced BigQuery job and to
-     * launch the provided task in the specified queue if the job succeeds.
-     */
-    public TaskHandle enqueuePollTask(
-        JobReference jobRef, TaskOptions chainedTask, Queue chainedTaskQueue) throws IOException {
-      // Serialize the chainedTask into a byte array to put in the task payload.
-      ByteArrayOutputStream taskBytes = new ByteArrayOutputStream();
-      new ObjectOutputStream(taskBytes).writeObject(chainedTask);
-      return taskQueueUtils.enqueue(
-          getQueue(QUEUE),
-          createCommonPollTask(jobRef)
-              .method(Method.POST)
-              .header(CHAINED_TASK_QUEUE_HEADER, chainedTaskQueue.getQueueName())
-              .payload(taskBytes.toByteArray()));
-    }
-
-    /**
-     * Enqueue a task to poll for the success or failure of the referenced BigQuery job and to
-     * launch the provided task in the specified queue if the job succeeds.
-     */
-    private static TaskOptions createCommonPollTask(JobReference jobRef) {
-      // Omit host header so that task will be run on the current backend/module.
-      return withUrl(PATH)
-          .countdownMillis(POLL_COUNTDOWN.getMillis())
-          .header(PROJECT_ID_HEADER, jobRef.getProjectId())
-          .header(JOB_ID_HEADER, jobRef.getJobId());
-    }
-  }
 }
--- a/core/src/main/java/google/registry/export/CheckBackupAction.java
+++ b/core/src/main/java/google/registry/export/CheckBackupAction.java
@@ -16,18 +16,14 @@ package google.registry.export;

 import static com.google.common.base.Preconditions.checkArgument;
 import static com.google.common.collect.Sets.intersection;
-import static google.registry.export.UploadDatastoreBackupAction.enqueueUploadBackupTask;
 import static google.registry.request.Action.Method.GET;
 import static google.registry.request.Action.Method.POST;
 import static javax.servlet.http.HttpServletResponse.SC_NOT_FOUND;

 import com.google.api.client.googleapis.json.GoogleJsonResponseException;
-import com.google.appengine.api.taskqueue.QueueFactory;
-import com.google.appengine.api.taskqueue.TaskHandle;
-import com.google.appengine.api.taskqueue.TaskOptions;
-import com.google.appengine.api.taskqueue.TaskOptions.Method;
 import com.google.common.base.Joiner;
 import com.google.common.base.Splitter;
+import com.google.common.collect.ImmutableMultimap;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.Sets;
 import com.google.common.flogger.FluentLogger;
@@ -35,6 +31,7 @@ import google.registry.export.datastore.DatastoreAdmin;
 import google.registry.export.datastore.Operation;
 import google.registry.model.annotations.DeleteAfterMigration;
 import google.registry.request.Action;
+import google.registry.request.Action.Service;
 import google.registry.request.HttpException;
 import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.HttpException.InternalServerErrorException;
@@ -45,6 +42,7 @@ import google.registry.request.RequestMethod;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
 import google.registry.util.Clock;
+import google.registry.util.CloudTasksUtils;
 import java.io.IOException;
 import java.util.Set;
 import javax.inject.Inject;
@@ -83,6 +81,7 @@ public class CheckBackupAction implements Runnable {
  @Inject DatastoreAdmin datastoreAdmin;
  @Inject Clock clock;
  @Inject Response response;
+  @Inject CloudTasksUtils cloudTasksUtils;
  @Inject @RequestMethod Action.Method requestMethod;

  @Inject
@@ -175,21 +174,22 @@ public class CheckBackupAction implements Runnable {
    if (exportedKindsToLoad.isEmpty()) {
      message += "no kinds to load into BigQuery.";
    } else {
-      enqueueUploadBackupTask(backupId, backup.getExportFolderUrl(), exportedKindsToLoad);
+      /** Enqueue a task for starting a backup load. */
+      cloudTasksUtils.enqueue(
+          UploadDatastoreBackupAction.QUEUE,
+          cloudTasksUtils.createPostTask(
+              UploadDatastoreBackupAction.PATH,
+              Service.BACKEND.toString(),
+              ImmutableMultimap.of(
+                  UploadDatastoreBackupAction.UPLOAD_BACKUP_ID_PARAM,
+                  backupId,
+                  UploadDatastoreBackupAction.UPLOAD_BACKUP_FOLDER_PARAM,
+                  backup.getExportFolderUrl(),
+                  UploadDatastoreBackupAction.UPLOAD_BACKUP_KINDS_PARAM,
+                  Joiner.on(',').join(exportedKindsToLoad))));
      message += "BigQuery load task enqueued.";
    }
    logger.atInfo().log(message);
    response.setPayload(message);
  }
-
-  /** Enqueue a poll task to monitor the named backup for completion. */
-  static TaskHandle enqueuePollTask(String backupId, ImmutableSet<String> kindsToLoad) {
-    return QueueFactory.getQueue(QUEUE)
-        .add(
-            TaskOptions.Builder.withUrl(PATH)
-                .method(Method.POST)
-                .countdownMillis(POLL_COUNTDOWN.getMillis())
-                .param(CHECK_BACKUP_NAME_PARAM, backupId)
-                .param(CHECK_BACKUP_KINDS_TO_LOAD_PARAM, Joiner.on(',').join(kindsToLoad)));
-  }
 }
--- a/core/src/main/java/google/registry/export/UpdateSnapshotViewAction.java
+++ b/core/src/main/java/google/registry/export/UpdateSnapshotViewAction.java
@@ -21,8 +21,6 @@ import com.google.api.services.bigquery.Bigquery;
 import com.google.api.services.bigquery.model.Table;
 import com.google.api.services.bigquery.model.TableReference;
 import com.google.api.services.bigquery.model.ViewDefinition;
-import com.google.appengine.api.taskqueue.TaskOptions;
-import com.google.appengine.api.taskqueue.TaskOptions.Method;
 import com.google.common.flogger.FluentLogger;
 import google.registry.bigquery.CheckedBigquery;
 import google.registry.config.RegistryConfig.Config;
@@ -85,17 +83,6 @@ public class UpdateSnapshotViewAction implements Runnable {
  @Inject
  UpdateSnapshotViewAction() {}

-  /** Create a task for updating a snapshot view. */
-  static TaskOptions createViewUpdateTask(
-      String datasetId, String tableId, String kindName, String viewName) {
-    return TaskOptions.Builder.withUrl(PATH)
-        .method(Method.POST)
-        .param(UPDATE_SNAPSHOT_DATASET_ID_PARAM, datasetId)
-        .param(UPDATE_SNAPSHOT_TABLE_ID_PARAM, tableId)
-        .param(UPDATE_SNAPSHOT_KIND_PARAM, kindName)
-        .param(UPDATE_SNAPSHOT_VIEWNAME_PARAM, viewName);
-  }
-
  @Override
  public void run() {
    try {
--- a/core/src/main/java/google/registry/export/UploadDatastoreBackupAction.java
+++ b/core/src/main/java/google/registry/export/UploadDatastoreBackupAction.java
@@ -14,9 +14,7 @@

 package google.registry.export;

-import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
 import static com.google.common.base.MoreObjects.firstNonNull;
-import static google.registry.export.UpdateSnapshotViewAction.createViewUpdateTask;
 import static google.registry.request.Action.Method.POST;

 import com.google.api.services.bigquery.Bigquery;
@@ -25,27 +23,33 @@ import com.google.api.services.bigquery.model.JobConfiguration;
 import com.google.api.services.bigquery.model.JobConfigurationLoad;
 import com.google.api.services.bigquery.model.JobReference;
 import com.google.api.services.bigquery.model.TableReference;
-import com.google.appengine.api.taskqueue.TaskHandle;
-import com.google.appengine.api.taskqueue.TaskOptions;
-import com.google.appengine.api.taskqueue.TaskOptions.Method;
+import com.google.cloud.tasks.v2.Task;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Joiner;
 import com.google.common.base.Splitter;
 import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.ImmutableMultimap;
 import com.google.common.flogger.FluentLogger;
+import com.google.common.net.HttpHeaders;
+import com.google.common.net.MediaType;
+import com.google.protobuf.ByteString;
+import com.google.protobuf.util.Timestamps;
 import google.registry.bigquery.BigqueryUtils.SourceFormat;
 import google.registry.bigquery.BigqueryUtils.WriteDisposition;
 import google.registry.bigquery.CheckedBigquery;
 import google.registry.config.RegistryConfig.Config;
-import google.registry.export.BigqueryPollJobAction.BigqueryPollJobEnqueuer;
 import google.registry.model.annotations.DeleteAfterMigration;
 import google.registry.request.Action;
+import google.registry.request.Action.Service;
 import google.registry.request.HttpException.BadRequestException;
 import google.registry.request.HttpException.InternalServerErrorException;
 import google.registry.request.Parameter;
 import google.registry.request.auth.Auth;
+import google.registry.util.Clock;
+import google.registry.util.CloudTasksUtils;
+import java.io.ByteArrayOutputStream;
 import java.io.IOException;
+import java.io.ObjectOutputStream;
 import javax.inject.Inject;

 /** Action to load a Datastore backup from Google Cloud Storage into BigQuery. */
@@ -75,7 +79,9 @@ public class UploadDatastoreBackupAction implements Runnable {
  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

  @Inject CheckedBigquery checkedBigquery;
-  @Inject BigqueryPollJobEnqueuer bigqueryPollEnqueuer;
+  @Inject CloudTasksUtils cloudTasksUtils;
+  @Inject Clock clock;
+
  @Inject @Config("projectId") String projectId;

  @Inject
@@ -93,18 +99,6 @@ public class UploadDatastoreBackupAction implements Runnable {
  @Inject
  UploadDatastoreBackupAction() {}

-  /** Enqueue a task for starting a backup load. */
-  public static TaskHandle enqueueUploadBackupTask(
-      String backupId, String gcsFile, ImmutableSet<String> kinds) {
-    return getQueue(QUEUE)
-        .add(
-            TaskOptions.Builder.withUrl(PATH)
-                .method(Method.POST)
-                .param(UPLOAD_BACKUP_ID_PARAM, backupId)
-                .param(UPLOAD_BACKUP_FOLDER_PARAM, gcsFile)
-                .param(UPLOAD_BACKUP_KINDS_PARAM, Joiner.on(',').join(kinds)));
-  }
-
  @Override
  public void run() {
    try {
@@ -142,12 +136,46 @@ public class UploadDatastoreBackupAction implements Runnable {
      Job job = makeLoadJob(jobRef, sourceUri, tableId);
      bigquery.jobs().insert(projectId, job).execute();

-      // Enqueue a task to check on the load job's completion, and if it succeeds, to update a
-      // well-known view in BigQuery to point at the newly loaded backup table for this kind.
-      bigqueryPollEnqueuer.enqueuePollTask(
-          jobRef,
-          createViewUpdateTask(BACKUP_DATASET, tableId, kindName, LATEST_BACKUP_VIEW_NAME),
-          getQueue(UpdateSnapshotViewAction.QUEUE));
+      // Serialize the chainedTask into a byte array to put in the task payload.
+      ByteArrayOutputStream taskBytes = new ByteArrayOutputStream();
+      new ObjectOutputStream(taskBytes)
+          .writeObject(
+              cloudTasksUtils.createPostTask(
+                  UpdateSnapshotViewAction.PATH,
+                  Service.BACKEND.toString(),
+                  ImmutableMultimap.of(
+                      UpdateSnapshotViewAction.UPDATE_SNAPSHOT_DATASET_ID_PARAM,
+                      BACKUP_DATASET,
+                      UpdateSnapshotViewAction.UPDATE_SNAPSHOT_TABLE_ID_PARAM,
+                      tableId,
+                      UpdateSnapshotViewAction.UPDATE_SNAPSHOT_KIND_PARAM,
+                      kindName,
+                      UpdateSnapshotViewAction.UPDATE_SNAPSHOT_VIEWNAME_PARAM,
+                      LATEST_BACKUP_VIEW_NAME)));
+
+      // Enqueues a task to poll for the success or failure of the referenced BigQuery job and to
+      // launch the provided task in the specified queue if the job succeeds.
+      cloudTasksUtils.enqueue(
+          BigqueryPollJobAction.QUEUE,
+          Task.newBuilder()
+              .setAppEngineHttpRequest(
+                  cloudTasksUtils
+                      .createPostTask(BigqueryPollJobAction.PATH, Service.BACKEND.toString(), null)
+                      .getAppEngineHttpRequest()
+                      .toBuilder()
+                      .putHeaders(BigqueryPollJobAction.PROJECT_ID_HEADER, jobRef.getProjectId())
+                      .putHeaders(BigqueryPollJobAction.JOB_ID_HEADER, jobRef.getJobId())
+                      .putHeaders(
+                          BigqueryPollJobAction.CHAINED_TASK_QUEUE_HEADER,
+                          UpdateSnapshotViewAction.QUEUE)
+                      // need to include CONTENT_TYPE in header when body is not empty
+                      .putHeaders(HttpHeaders.CONTENT_TYPE, MediaType.FORM_DATA.toString())
+                      .setBody(ByteString.copyFrom(taskBytes.toByteArray()))
+                      .build())
+              .setScheduleTime(
+                  Timestamps.fromMillis(
+                      clock.nowUtc().plus(BigqueryPollJobAction.POLL_COUNTDOWN).getMillis()))
+              .build());

      builder.append(String.format(" - %s:%s\n", projectId, jobId));
      logger.atInfo().log("Submitted load job %s:%s.", projectId, jobId);
--- a/core/src/main/java/google/registry/flows/certs/CertificateChecker.java
+++ b/core/src/main/java/google/registry/flows/certs/CertificateChecker.java
@@ -107,7 +107,7 @@ public class CertificateChecker {
    // https://stackoverflow.com/questions/49895713/how-to-find-the-matching-curve-name-from-an-ecpublickey.
    if (key instanceof ECPublicKey) {
      ECPublicKey ecKey = (ECPublicKey) key;
-      params = EC5Util.convertSpec(ecKey.getParams(), false);
+      params = EC5Util.convertSpec(ecKey.getParams());
    } else if (key instanceof org.bouncycastle.jce.interfaces.ECPublicKey) {
      org.bouncycastle.jce.interfaces.ECPublicKey ecKey =
          (org.bouncycastle.jce.interfaces.ECPublicKey) key;
--- a/core/src/main/java/google/registry/flows/contact/ContactDeleteFlow.java
+++ b/core/src/main/java/google/registry/flows/contact/ContactDeleteFlow.java
@@ -25,6 +25,7 @@ import static google.registry.model.ResourceTransferUtils.handlePendingTransferO
 import static google.registry.model.eppoutput.Result.Code.SUCCESS;
 import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
 import static google.registry.model.transfer.TransferStatus.SERVER_CANCELLED;
+import static google.registry.persistence.transaction.TransactionManagerFactory.assertAsyncActionsAreAllowed;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;

 import com.google.common.collect.ImmutableSet;
@@ -94,6 +95,7 @@ public final class ContactDeleteFlow implements TransactionalFlow {
    extensionManager.register(MetadataExtension.class);
    validateRegistrarIsLoggedIn(registrarId);
    extensionManager.validate();
+    assertAsyncActionsAreAllowed();
    DateTime now = tm().getTransactionTime();
    checkLinkedDomains(targetId, now, ContactResource.class, DomainBase::getReferencedContacts);
    ContactResource existingContact = loadAndVerifyExistence(ContactResource.class, targetId, now);
--- a/core/src/main/java/google/registry/flows/domain/DomainFlowUtils.java
+++ b/core/src/main/java/google/registry/flows/domain/DomainFlowUtils.java
@@ -301,7 +301,6 @@ public class DomainFlowUtils {
            String.format(
                "A maximum of %s DS records are allowed per domain.", MAX_DS_RECORDS_PER_DOMAIN));
      }
-      // TODO(sarahbot@): Add signature length verification
      ImmutableList<DelegationSignerData> invalidAlgorithms =
          dsData.stream()
              .filter(ds -> !validateAlgorithm(ds.getAlgorithm()))
@@ -322,6 +321,20 @@ public class DomainFlowUtils {
                "Domain contains DS record(s) with an invalid digest type: %s",
                invalidDigestTypes));
      }
+      ImmutableList<DelegationSignerData> digestsWithInvalidDigestLength =
+          dsData.stream()
+              .filter(
+                  ds ->
+                      DigestType.fromWireValue(ds.getDigestType()).isPresent()
+                          && (ds.getDigest().length
+                              != DigestType.fromWireValue(ds.getDigestType()).get().getBytes()))
+              .collect(toImmutableList());
+      if (!digestsWithInvalidDigestLength.isEmpty()) {
+        throw new InvalidDsRecordException(
+            String.format(
+                "Domain contains DS record(s) with an invalid digest length: %s",
+                digestsWithInvalidDigestLength));
+      }
    }
  }

--- a/core/src/main/java/google/registry/flows/host/HostDeleteFlow.java
+++ b/core/src/main/java/google/registry/flows/host/HostDeleteFlow.java
@@ -22,6 +22,7 @@ import static google.registry.flows.ResourceFlowUtils.verifyResourceOwnership;
 import static google.registry.flows.host.HostFlowUtils.validateHostName;
 import static google.registry.model.eppoutput.Result.Code.SUCCESS;
 import static google.registry.model.eppoutput.Result.Code.SUCCESS_WITH_ACTION_PENDING;
+import static google.registry.persistence.transaction.TransactionManagerFactory.assertAsyncActionsAreAllowed;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;

 import com.google.common.collect.ImmutableSet;
@@ -96,6 +97,7 @@ public final class HostDeleteFlow implements TransactionalFlow {
    extensionManager.register(MetadataExtension.class);
    validateRegistrarIsLoggedIn(registrarId);
    extensionManager.validate();
+    assertAsyncActionsAreAllowed();
    DateTime now = tm().getTransactionTime();
    validateHostName(targetId);
    checkLinkedDomains(targetId, now, HostResource.class, DomainBase::getNameservers);
--- a/core/src/main/java/google/registry/flows/host/HostUpdateFlow.java
+++ b/core/src/main/java/google/registry/flows/host/HostUpdateFlow.java
@@ -28,6 +28,7 @@ import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomain
 import static google.registry.flows.host.HostFlowUtils.verifySuperordinateDomainOwnership;
 import static google.registry.model.index.ForeignKeyIndex.loadAndGetKey;
 import static google.registry.model.reporting.HistoryEntry.Type.HOST_UPDATE;
+import static google.registry.persistence.transaction.TransactionManagerFactory.assertAsyncActionsAreAllowed;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.util.CollectionUtils.isNullOrEmpty;

@@ -136,6 +137,9 @@ public final class HostUpdateFlow implements TransactionalFlow {
    validateHostName(targetId);
    HostResource existingHost = loadAndVerifyExistence(HostResource.class, targetId, now);
    boolean isHostRename = suppliedNewHostName != null;
+    if (isHostRename) {
+      assertAsyncActionsAreAllowed();
+    }
    String oldHostName = targetId;
    String newHostName = firstNonNull(suppliedNewHostName, oldHostName);
    DomainBase oldSuperordinateDomain =
--- a/core/src/main/java/google/registry/model/EntityClasses.java
+++ b/core/src/main/java/google/registry/model/EntityClasses.java
@@ -40,6 +40,7 @@ import google.registry.model.rde.RdeRevision;
 import google.registry.model.registrar.Registrar;
 import google.registry.model.registrar.RegistrarContact;
 import google.registry.model.replay.LastSqlTransaction;
+import google.registry.model.replay.ReplayGap;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.model.server.Lock;
 import google.registry.model.server.ServerSecret;
@@ -86,6 +87,7 @@ public final class EntityClasses {
          Registrar.class,
          RegistrarContact.class,
          Registry.class,
+          ReplayGap.class,
          ServerSecret.class);

  private EntityClasses() {}
--- a/core/src/main/java/google/registry/model/billing/BillingEvent.java
+++ b/core/src/main/java/google/registry/model/billing/BillingEvent.java
@@ -308,7 +308,9 @@ public abstract class BillingEvent extends ImmutableObject
        @javax.persistence.Index(columnList = "eventTime"),
        @javax.persistence.Index(columnList = "billingTime"),
        @javax.persistence.Index(columnList = "syntheticCreationTime"),
-        @javax.persistence.Index(columnList = "allocationToken")
+        @javax.persistence.Index(columnList = "domainRepoId"),
+        @javax.persistence.Index(columnList = "allocationToken"),
+        @javax.persistence.Index(columnList = "cancellation_matching_billing_recurrence_id")
      })
  @AttributeOverride(name = "id", column = @Column(name = "billing_event_id"))
  @WithLongVKey(compositeKey = true)
@@ -518,6 +520,7 @@ public abstract class BillingEvent extends ImmutableObject
      indexes = {
        @javax.persistence.Index(columnList = "registrarId"),
        @javax.persistence.Index(columnList = "eventTime"),
+        @javax.persistence.Index(columnList = "domainRepoId"),
        @javax.persistence.Index(columnList = "recurrenceEndTime"),
        @javax.persistence.Index(columnList = "recurrence_time_of_year")
      })
@@ -614,7 +617,10 @@ public abstract class BillingEvent extends ImmutableObject
      indexes = {
        @javax.persistence.Index(columnList = "registrarId"),
        @javax.persistence.Index(columnList = "eventTime"),
-        @javax.persistence.Index(columnList = "billingTime")
+        @javax.persistence.Index(columnList = "domainRepoId"),
+        @javax.persistence.Index(columnList = "billingTime"),
+        @javax.persistence.Index(columnList = "billing_event_id"),
+        @javax.persistence.Index(columnList = "billing_recurrence_id")
      })
  @AttributeOverride(name = "id", column = @Column(name = "billing_cancellation_id"))
  @WithLongVKey(compositeKey = true)
--- a/core/src/main/java/google/registry/model/common/DatabaseMigrationStateSchedule.java
+++ b/core/src/main/java/google/registry/model/common/DatabaseMigrationStateSchedule.java
@@ -30,6 +30,7 @@ import google.registry.model.annotations.DeleteAfterMigration;
 import google.registry.model.common.TimedTransitionProperty.TimedTransition;
 import google.registry.model.replay.SqlOnlyEntity;
 import java.time.Duration;
+import java.util.Arrays;
 import javax.persistence.Entity;
 import javax.persistence.PersistenceException;
 import org.joda.time.DateTime;
@@ -62,11 +63,28 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton implements
   * not the phase is read-only.
   */
  public enum MigrationState {
+    /** Datastore is the only DB being used. */
    DATASTORE_ONLY(PrimaryDatabase.DATASTORE, false, ReplayDirection.NO_REPLAY),
+
+    /** Datastore is the primary DB, with changes replicated to Cloud SQL. */
    DATASTORE_PRIMARY(PrimaryDatabase.DATASTORE, false, ReplayDirection.DATASTORE_TO_SQL),
+
+    /** Datastore is the primary DB, with replication, and async actions are disallowed. */
+    DATASTORE_PRIMARY_NO_ASYNC(PrimaryDatabase.DATASTORE, false, ReplayDirection.DATASTORE_TO_SQL),
+
+    /** Datastore is the primary DB, with replication, and all mutating actions are disallowed. */
    DATASTORE_PRIMARY_READ_ONLY(PrimaryDatabase.DATASTORE, true, ReplayDirection.DATASTORE_TO_SQL),
+
+    /**
+     * Cloud SQL is the primary DB, with replication back to Datastore, and all mutating actions are
+     * disallowed.
+     */
    SQL_PRIMARY_READ_ONLY(PrimaryDatabase.CLOUD_SQL, true, ReplayDirection.SQL_TO_DATASTORE),
+
+    /** Cloud SQL is the primary DB, with changes replicated to Datastore. */
    SQL_PRIMARY(PrimaryDatabase.CLOUD_SQL, false, ReplayDirection.SQL_TO_DATASTORE),
+
+    /** Cloud SQL is the only DB being used. */
    SQL_ONLY(PrimaryDatabase.CLOUD_SQL, false, ReplayDirection.NO_REPLAY);

    private final PrimaryDatabase primaryDatabase;
@@ -146,11 +164,17 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton implements
            .putAll(
                MigrationState.DATASTORE_PRIMARY,
                MigrationState.DATASTORE_ONLY,
+                MigrationState.DATASTORE_PRIMARY_NO_ASYNC)
+            .putAll(
+                MigrationState.DATASTORE_PRIMARY_NO_ASYNC,
+                MigrationState.DATASTORE_ONLY,
+                MigrationState.DATASTORE_PRIMARY,
                MigrationState.DATASTORE_PRIMARY_READ_ONLY)
            .putAll(
                MigrationState.DATASTORE_PRIMARY_READ_ONLY,
                MigrationState.DATASTORE_ONLY,
                MigrationState.DATASTORE_PRIMARY,
+                MigrationState.DATASTORE_PRIMARY_NO_ASYNC,
                MigrationState.SQL_PRIMARY_READ_ONLY,
                MigrationState.SQL_PRIMARY)
            .putAll(
@@ -165,10 +189,9 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton implements
                MigrationState.SQL_ONLY,
                MigrationState.SQL_PRIMARY_READ_ONLY,
                MigrationState.SQL_PRIMARY);
+
    // In addition, we can always transition from a state to itself (useful when updating the map).
-    for (MigrationState migrationState : MigrationState.values()) {
-      builder.put(migrationState, migrationState);
-    }
+    Arrays.stream(MigrationState.values()).forEach(state -> builder.put(state, state));
    return builder.build();
  }

@@ -246,7 +269,7 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton implements
   * A provided map of transitions may be valid by itself (i.e. it shifts states properly, doesn't
   * skip states, and doesn't backtrack incorrectly) while still being invalid. In addition to the
   * transitions in the map being valid, the single transition from the current map at the current
-   * time to the new map at the current time time must also be valid.
+   * time to the new map at the current time must also be valid.
   */
  private static void validateTransitionAtCurrentTime(
      TimedTransitionProperty<MigrationState, MigrationStateTransition> newTransitions) {
--- a/core/src/main/java/google/registry/model/domain/DomainBase.java
+++ b/core/src/main/java/google/registry/model/domain/DomainBase.java
@@ -69,7 +69,10 @@ import org.joda.time.DateTime;
      @Index(columnList = "techContact"),
      @Index(columnList = "tld"),
      @Index(columnList = "registrantContact"),
-      @Index(columnList = "dnsRefreshRequestTime")
+      @Index(columnList = "dnsRefreshRequestTime"),
+      @Index(columnList = "billing_recurrence_id"),
+      @Index(columnList = "transfer_billing_event_id"),
+      @Index(columnList = "transfer_billing_recurrence_id")
    })
@WithStringVKey
@ExternalMessagingName("domain")
@@ -89,7 +92,10 @@ public class DomainBase extends DomainContent
  @ElementCollection
  @JoinTable(
      name = "DomainHost",
-      indexes = {@Index(columnList = "domain_repo_id,host_repo_id", unique = true)})
+      indexes = {
+        @Index(columnList = "domain_repo_id,host_repo_id", unique = true),
+        @Index(columnList = "host_repo_id")
+      })
  @Access(AccessType.PROPERTY)
  @Column(name = "host_repo_id")
  public Set<VKey<HostResource>> getNsHosts() {
--- a/core/src/main/java/google/registry/model/domain/GracePeriod.java
+++ b/core/src/main/java/google/registry/model/domain/GracePeriod.java
@@ -46,7 +46,12 @@ import org.joda.time.DateTime;
 */
@Embed
@Entity
-@Table(indexes = @Index(columnList = "domainRepoId"))
+@Table(
+    indexes = {
+      @Index(columnList = "domainRepoId"),
+      @Index(columnList = "billing_event_id"),
+      @Index(columnList = "billing_recurrence_id")
+    })
 public class GracePeriod extends GracePeriodBase implements DatastoreAndSqlEntity {

  @Id
@@ -189,18 +194,6 @@ public class GracePeriod extends GracePeriodBase implements DatastoreAndSqlEntit
    return clone;
  }

-  /**
-   * Returns a clone of this {@link GracePeriod} with {@link #billingEventRecurring} set to the
-   * given value.
-   *
-   * <p>TODO(b/162231099): Remove this function after duplicate id issue is solved.
-   */
-  public GracePeriod cloneWithRecurringBillingEvent(VKey<BillingEvent.Recurring> recurring) {
-    GracePeriod clone = clone(this);
-    clone.billingEventRecurring = BillingRecurrenceVKey.create(recurring);
-    return clone;
-  }
-
  /** Entity class to represent a historic {@link GracePeriod}. */
  @Entity(name = "GracePeriodHistory")
  @Table(indexes = @Index(columnList = "domainRepoId"))
--- a/core/src/main/java/google/registry/model/domain/token/AllocationToken.java
+++ b/core/src/main/java/google/registry/model/domain/token/AllocationToken.java
@@ -76,6 +76,8 @@ import org.joda.time.DateTime;
      @javax.persistence.Index(
          columnList = "domainName",
          name = "allocation_token_domain_name_idx"),
+      @javax.persistence.Index(columnList = "tokenType"),
+      @javax.persistence.Index(columnList = "redemption_domain_repo_id")
    })
 public class AllocationToken extends BackupGroupRoot implements Buildable, DatastoreAndSqlEntity {

--- a/core/src/main/java/google/registry/model/host/HostResource.java
+++ b/core/src/main/java/google/registry/model/host/HostResource.java
@@ -36,7 +36,23 @@ import javax.persistence.AccessType;
@javax.persistence.Entity(name = "Host")
@javax.persistence.Table(
    name = "Host",
-    indexes = {@javax.persistence.Index(columnList = "hostName")})
+    /**
+     * A gin index defined on the inet_addresses field ({@link HostBase#inetAddresses} cannot be
+     * declared here because JPA/Hibernate does not support index type specification. As a result,
+     * the hibernate-generated schema (which is for reference only) does not have this index.
+     *
+     * <p>There are Hibernate-specific solutions for adding this index to Hibernate's domain model.
+     * We could either declare the index in hibernate.cfg.xml or add it to the {@link
+     * org.hibernate.cfg.Configuration} instance for {@link SessionFactory} instantiation (which
+     * would prevent us from using JPA standard bootstrapping). For now, there is no obvious benefit
+     * doing either.
+     */
+    indexes = {
+      @javax.persistence.Index(columnList = "hostName"),
+      @javax.persistence.Index(columnList = "creationTime"),
+      @javax.persistence.Index(columnList = "deletionTime"),
+      @javax.persistence.Index(columnList = "currentSponsorRegistrarId")
+    })
@ExternalMessagingName("host")
@WithStringVKey
@Access(AccessType.FIELD) // otherwise it'll use the default if the repoId (property)
--- a/core/src/main/java/google/registry/model/replay/ReplayGap.java
+++ b/core/src/main/java/google/registry/model/replay/ReplayGap.java
@@ -0,0 +1,60 @@
+// Copyright 2022 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.model.replay;
+
+import static google.registry.model.annotations.NotBackedUp.Reason.TRANSIENT;
+
+import com.googlecode.objectify.annotation.Entity;
+import com.googlecode.objectify.annotation.Id;
+import google.registry.model.ImmutableObject;
+import google.registry.model.annotations.DeleteAfterMigration;
+import google.registry.model.annotations.NotBackedUp;
+import org.joda.time.DateTime;
+
+/**
+ * Tracks gaps in transaction ids when replicating from SQL to datastore.
+ *
+ * <p>SQL -&gt; DS replication uses a Transaction table indexed by a SEQUENCE column, which normally
+ * increments monotonically for each committed transaction. Gaps in this sequence can occur when a
+ * transaction is rolled back or when a transaction has been initiated but not committed to the
+ * table at the time of a query. To protect us from the latter scenario, we need to keep track of
+ * these gaps and replay any of them that have been filled in since we processed their batch.
+ */
+@DeleteAfterMigration
+@NotBackedUp(reason = TRANSIENT)
+@Entity
+public class ReplayGap extends ImmutableObject implements DatastoreOnlyEntity {
+  @Id long transactionId;
+
+  // We can't use a CreateAutoTimestamp here because this ends up getting persisted in an ofy
+  // transaction that happens in JPA mode, so we don't end up getting an active transaction manager
+  // when the timestamp needs to be set.
+  DateTime timestamp;
+
+  ReplayGap() {}
+
+  ReplayGap(DateTime timestamp, long transactionId) {
+    this.timestamp = timestamp;
+    this.transactionId = transactionId;
+  }
+
+  long getTransactionId() {
+    return transactionId;
+  }
+
+  DateTime getTimestamp() {
+    return timestamp;
+  }
+}
--- a/core/src/main/java/google/registry/model/replay/ReplicateToDatastoreAction.java
+++ b/core/src/main/java/google/registry/model/replay/ReplicateToDatastoreAction.java
@@ -14,6 +14,8 @@

 package google.registry.model.replay;

+import static com.google.common.base.Preconditions.checkState;
+import static com.google.common.collect.ImmutableList.toImmutableList;
 import static google.registry.model.ofy.ObjectifyService.auditedOfy;
 import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.ofyTm;
@@ -40,6 +42,7 @@ import google.registry.request.auth.Auth;
 import google.registry.util.Clock;
 import google.registry.util.RequestStatusChecker;
 import java.io.IOException;
+import java.util.ArrayList;
 import java.util.List;
 import java.util.Optional;
 import javax.inject.Inject;
@@ -70,6 +73,20 @@ public class ReplicateToDatastoreAction implements Runnable {
   */
  public static final int BATCH_SIZE = 200;

+  /**
+   * The longest time that we'll keep trying to resolve a gap in the Transaction table in
+   * milliseconds, after which, the gap record will be deleted.
+   */
+  public static final long MAX_GAP_RETENTION_MILLIS = 300000;
+
+  /**
+   * The maximum number of entitities to be mutated per transaction. For our purposes, the entities
+   * that we're keeping track of are each in their own entity group. Per datastore documentation, we
+   * should be allowed to update up to 25 of them. In practice, we get an error if we go beyond 24
+   * (possibly due to something in our own infrastructure).
+   */
+  private static final int MAX_ENTITIES_PER_TXN = 24;
+
  public static final Duration REPLICATE_TO_DATASTORE_LOCK_LEASE_LENGTH = standardHours(1);

  private final Clock clock;
@@ -89,6 +106,12 @@ public class ReplicateToDatastoreAction implements Runnable {
    return getTransactionBatchAtSnapshot(Optional.empty());
  }

+  /**
+   * Get the next batch of transactions, optionally from a specific SQL database snapshot.
+   *
+   * <p>Note that this method may also apply transactions from previous batches that had not yet
+   * been committed at the time the previous batch was retrieved.
+   */
  static List<TransactionEntity> getTransactionBatchAtSnapshot(Optional<String> snapshotId) {
    // Get the next batch of transactions that we haven't replicated.
    LastSqlTransaction lastSqlTxnBeforeBatch = ofyTm().transact(LastSqlTransaction::load);
@@ -97,6 +120,11 @@ public class ReplicateToDatastoreAction implements Runnable {
          .transactWithoutBackup(
              () -> {
                snapshotId.ifPresent(jpaTm()::setDatabaseSnapshot);
+
+                // Fill in any gaps in the transaction log that have since become available before
+                // processing the next batch.
+                applyMissingTransactions();
+
                return jpaTm()
                    .query(
                        "SELECT txn FROM TransactionEntity txn WHERE id >" + " :lastId ORDER BY id",
@@ -111,60 +139,175 @@ public class ReplicateToDatastoreAction implements Runnable {
  }

  /**
-   * Apply a transaction to Datastore, returns true if there was a fatal error and the batch should
-   * be aborted.
+   * Iterate over the recent gaps in the Transaction table and apply any that have been filled in.
   *
-   * <p>Throws an exception if a fatal error occurred and the batch should be aborted
+   * <p>Must be called from within a JPA transaction.
+   *
+   * <p>Gap rewriting is a complicated matter, and the algorithm is the product of some very deep
+   * consideration by mmuller and weiminyu. Basically, the constraints are:
+   *
+   * <ol>
+   *   <li>Replay has to work against a database snapshot (gap replay would break this, so we don't
+   *       call this method when replaying against a snapshot)
+   * </ol>
+   */
+  private static void applyMissingTransactions() {
+    long now = jpaTm().getTransactionTime().getMillis();
+    ImmutableList<ReplayGap> gaps = ofyTm().loadAllOf(ReplayGap.class);
+    jpaTm()
+        .query("SELECT txn from TransactionEntity txn WHERE id IN :gapIds", TransactionEntity.class)
+        .setParameter(
+            "gapIds", gaps.stream().map(gap -> gap.getTransactionId()).collect(toImmutableList()))
+        .getResultStream()
+        .forEach(
+            txn -> {
+              // Transcribe the transaction and delete the gap record in the same ofy transaction.
+              ofyTm()
+                  .transact(
+                      () -> {
+                        // Write the transaction to datastore.
+                        try {
+                          Transaction.deserialize(txn.getContents()).writeToDatastore();
+                        } catch (IOException e) {
+                          throw new RuntimeException("Error during transaction deserialization", e);
+                        }
+
+                        // Find and delete the gap record.
+                        ImmutableList<ReplayGap> filledGaps =
+                            gaps.stream()
+                                .filter(gap -> gap.getTransactionId() == txn.getId())
+                                .collect(toImmutableList());
+                        checkState(
+                            filledGaps.size() == 1,
+                            "Bad list of gaps for discovered id: %s",
+                            filledGaps);
+                        auditedOfy().deleteIgnoringReadOnlyWithoutBackup().entity(gaps.get(0));
+                      });
+              logger.atInfo().log("Applied missing transaction %s", txn.getId());
+            });
+
+    // Clean up any gaps that have expired (in batches because they're each in their own entity
+    // group).
+    ArrayList<ReplayGap> gapBatch = new ArrayList<>();
+    gaps.stream()
+        .forEach(
+            gap -> {
+              if (now - gap.getTimestamp().getMillis() > MAX_GAP_RETENTION_MILLIS) {
+                gapBatch.add(gap);
+              }
+              if (gapBatch.size() == MAX_ENTITIES_PER_TXN) {
+                deleteReplayGaps(gapBatch);
+                gapBatch.clear();
+              }
+            });
+    if (!gapBatch.isEmpty()) {
+      deleteReplayGaps(gapBatch);
+    }
+  }
+
+  private static void deleteReplayGaps(ArrayList<ReplayGap> gapsToDelete) {
+    logger.atInfo().log(
+        "deleting gap records for %s",
+        gapsToDelete.stream().map(g -> g.getTransactionId()).collect(toImmutableList()));
+    ofyTm()
+        .transact(() -> auditedOfy().deleteIgnoringReadOnlyWithoutBackup().entities(gapsToDelete));
+  }
+
+  /**
+   * Apply a transaction to Datastore.
+   *
+   * <p>Throws an exception if a fatal error occurred and the batch should be aborted.
   */
  @VisibleForTesting
  public static void applyTransaction(TransactionEntity txnEntity) {
    logger.atInfo().log("Applying a single transaction Cloud SQL -> Cloud Datastore.");
+    boolean done = false;
    try (UpdateAutoTimestamp.DisableAutoUpdateResource disabler =
        UpdateAutoTimestamp.disableAutoUpdate()) {
-      ofyTm()
-          .transact(
-              () -> {
-                // Reload the last transaction id, which could possibly have changed.
-                LastSqlTransaction lastSqlTxn = LastSqlTransaction.load();
-                long nextTxnId = lastSqlTxn.getTransactionId() + 1;
-                if (nextTxnId < txnEntity.getId()) {
-                  // We're missing a transaction.  This is bad.  Transaction ids are supposed to
-                  // increase monotonically, so we abort rather than applying anything out of
-                  // order.
-                  throw new IllegalStateException(
-                      String.format(
-                          "Missing transaction: last txn id = %s, next available txn = %s",
-                          nextTxnId - 1, txnEntity.getId()));
-                } else if (nextTxnId > txnEntity.getId()) {
-                  // We've already replayed this transaction.  This shouldn't happen, as GAE cron
-                  // is supposed to avoid overruns and this action shouldn't be executed from any
-                  // other context, but it's not harmful as we can just ignore the transaction.  Log
-                  // it so that we know about it and move on.
-                  logger.atWarning().log(
-                      "Ignoring transaction %s, which appears to have already been applied.",
-                      txnEntity.getId());
-                  return;
-                }

-                logger.atInfo().log(
-                    "Applying transaction %s to Cloud Datastore.", txnEntity.getId());
+      // We put this in a do/while loop because we can potentially clean out some range of gaps
+      // first and we want to do those in their own transaction so as not to run up the entity group
+      // count.  (This is a highly pathological case: consecutive gaps are rare, but the fact that
+      // they can occur potentially increases the entity group count to beyond what we can
+      // accommodate in a single transaction.)
+      do {
+        done =
+            ofyTm()
+                .transact(
+                    () -> {
+                      // Reload the last transaction id, which could possibly have changed.
+                      LastSqlTransaction lastSqlTxn = LastSqlTransaction.load();
+                      long nextTxnId = lastSqlTxn.getTransactionId() + 1;

-                // At this point, we know txnEntity is the correct next transaction, so write it
-                // to Datastore.
-                try {
-                  Transaction.deserialize(txnEntity.getContents()).writeToDatastore();
-                } catch (IOException e) {
-                  throw new RuntimeException("Error during transaction deserialization", e);
-                }
+                      // Skip missing transactions.  Missed transactions can happen normally.  If a
+                      // transaction gets rolled back, the sequence counter doesn't.
+                      int gapCount = 0;
+                      while (nextTxnId < txnEntity.getId()) {
+                        logger.atWarning().log(
+                            "Ignoring transaction %s, which does not exist.", nextTxnId);
+                        auditedOfy()
+                            .saveIgnoringReadOnlyWithoutBackup()
+                            .entity(new ReplayGap(ofyTm().getTransactionTime(), nextTxnId));
+                        ++nextTxnId;

-                // Write the updated last transaction id to Datastore as part of this Datastore
-                // transaction.
-                auditedOfy()
-                    .saveIgnoringReadOnlyWithoutBackup()
-                    .entity(lastSqlTxn.cloneWithNewTransactionId(nextTxnId));
-                logger.atInfo().log(
-                    "Finished applying single transaction Cloud SQL -> Cloud Datastore.");
-              });
+                        // Don't exceed the entity group count trying to clean these up (we stop at
+                        // max
+                        // - 1 because we also want to save the lastSqlTransaction).
+                        if (++gapCount == MAX_ENTITIES_PER_TXN - 1) {
+                          break;
+                        }
+                      }
+
+                      // Don't write gap records in the same transaction as the SQL transaction that
+                      // we're replaying.  Return false to force us to go through and repeat in a
+                      // new
+                      // transaction.
+                      if (gapCount > 0) {
+                        // We haven't replayed the transaction, but we've determined that we can
+                        // ignore everything before it so update lastSqlTransaction accordingly.
+                        auditedOfy()
+                            .saveIgnoringReadOnlyWithoutBackup()
+                            .entity(lastSqlTxn.cloneWithNewTransactionId(nextTxnId - 1));
+                        return false;
+                      }
+
+                      if (nextTxnId > txnEntity.getId()) {
+                        // We've already replayed this transaction.  This shouldn't happen, as GAE
+                        // cron
+                        // is supposed to avoid overruns and this action shouldn't be executed from
+                        // any
+                        // other context, but it's not harmful as we can just ignore the
+                        // transaction.
+                        // Log it so that we know about it and move on.
+                        logger.atWarning().log(
+                            "Ignoring transaction %s, which appears to have already been applied.",
+                            txnEntity.getId());
+                        return true;
+                      }
+
+                      logger.atInfo().log(
+                          "Applying transaction %s to Cloud Datastore.", txnEntity.getId());
+
+                      // At this point, we know txnEntity is the correct next transaction, so write
+                      // it
+                      // to Datastore.
+                      try {
+                        Transaction.deserialize(txnEntity.getContents()).writeToDatastore();
+                      } catch (IOException e) {
+                        throw new RuntimeException("Error during transaction deserialization", e);
+                      }
+
+                      // Write the updated last transaction id to Datastore as part of this
+                      // Datastore
+                      // transaction.
+                      auditedOfy()
+                          .saveIgnoringReadOnlyWithoutBackup()
+                          .entity(lastSqlTxn.cloneWithNewTransactionId(nextTxnId));
+                      logger.atInfo().log(
+                          "Finished applying single transaction Cloud SQL -> Cloud Datastore.");
+                      return true;
+                    });
+      } while (!done);
    }
  }

--- a/core/src/main/java/google/registry/model/tld/Registries.java
+++ b/core/src/main/java/google/registry/model/tld/Registries.java
@@ -24,6 +24,7 @@ import static com.google.common.collect.Maps.filterValues;
 import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
 import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
 import static google.registry.model.ofy.ObjectifyService.auditedOfy;
+import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.util.CollectionUtils.entriesToImmutableMap;
 import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
@@ -38,6 +39,8 @@ import com.google.common.net.InternetDomainName;
 import com.googlecode.objectify.Key;
 import google.registry.model.tld.Registry.TldType;
 import java.util.Optional;
+import java.util.stream.Stream;
+import javax.persistence.EntityManager;

 /** Utilities for finding and listing {@link Registry} entities. */
 public final class Registries {
@@ -58,23 +61,31 @@ public final class Registries {
        () ->
            tm().doTransactionless(
                    () -> {
-                      ImmutableSet<String> tlds =
-                          tm().isOfy()
-                              ? auditedOfy()
-                                  .load()
-                                  .type(Registry.class)
-                                  .ancestor(getCrossTldKey())
-                                  .keys()
-                                  .list()
-                                  .stream()
-                                  .map(Key::getName)
-                                  .collect(toImmutableSet())
-                              : tm().loadAllOf(Registry.class).stream()
-                                  .map(Registry::getTldStr)
-                                  .collect(toImmutableSet());
-                      return Registry.getAll(tlds).stream()
-                          .map(e -> Maps.immutableEntry(e.getTldStr(), e.getTldType()))
-                          .collect(entriesToImmutableMap());
+                      if (tm().isOfy()) {
+                        ImmutableSet<String> tlds =
+                            auditedOfy()
+                                .load()
+                                .type(Registry.class)
+                                .ancestor(getCrossTldKey())
+                                .keys()
+                                .list()
+                                .stream()
+                                .map(Key::getName)
+                                .collect(toImmutableSet());
+                        return Registry.getAll(tlds).stream()
+                            .map(e -> Maps.immutableEntry(e.getTldStr(), e.getTldType()))
+                            .collect(entriesToImmutableMap());
+                      } else {
+                        EntityManager entityManager = jpaTm().getEntityManager();
+                        Stream<?> resultStream =
+                            entityManager
+                                .createQuery("SELECT tldStrId, tldType FROM Tld")
+                                .getResultStream();
+                        return resultStream
+                            .map(e -> ((Object[]) e))
+                            .map(e -> Maps.immutableEntry((String) e[0], ((TldType) e[1])))
+                            .collect(entriesToImmutableMap());
+                      }
                    }));
  }

--- a/core/src/main/java/google/registry/model/tmch/ClaimsList.java
+++ b/core/src/main/java/google/registry/model/tmch/ClaimsList.java
@@ -70,8 +70,7 @@ public class ClaimsList extends ImmutableObject implements SqlOnlyEntity {
   *
   * <p>Note that the value of this field is parsed from the claims list file(See this <a
   * href="https://tools.ietf.org/html/draft-lozano-tmch-func-spec-08#section-6.1">RFC</>), it is
-   * the DNL List creation datetime from the rfc. Since this field has been used by Datastore, we
-   * cannot change its name until we finish the migration.
+   * the DNL List creation datetime from the rfc.
   *
   * <p>TODO(b/177567432): Rename this field to tmdbGenerationTime.
   */
--- a/core/src/main/java/google/registry/model/tmch/TmchCrl.java
+++ b/core/src/main/java/google/registry/model/tmch/TmchCrl.java
@@ -47,7 +47,7 @@ public final class TmchCrl extends CrossTldSingleton implements SqlOnlyEntity {
   * Change the singleton to a new ASCII-armored X.509 CRL.
   *
   * <p>Please do not call this function unless your CRL is properly formatted, signed by the root,
-   * and actually newer than the one currently in Datastore.
+   * and actually newer than the one currently in the database.
   */
  public static void set(final String crl, final String url) {
    jpaTm()
--- a/core/src/main/java/google/registry/model/transfer/DomainTransferData.java
+++ b/core/src/main/java/google/registry/model/transfer/DomainTransferData.java
@@ -94,8 +94,6 @@ public class DomainTransferData extends TransferData<DomainTransferData.Builder>
   *
   * <p>This field should be null if there is not currently a pending transfer or if the object
   * being transferred is not a domain.
-   *
-   * <p>TODO(b/158230654) Remove unused columns for TransferData in Contact table.
   */
  @IgnoreSave(IfNull.class)
  @Column(name = "transfer_billing_event_id")
--- a/core/src/main/java/google/registry/module/backend/BackendComponent.java
+++ b/core/src/main/java/google/registry/module/backend/BackendComponent.java
@@ -43,7 +43,8 @@ import google.registry.rde.JSchModule;
 import google.registry.request.Modules.DatastoreServiceModule;
 import google.registry.request.Modules.Jackson2Module;
 import google.registry.request.Modules.NetHttpTransportModule;
-import google.registry.request.Modules.URLFetchServiceModule;
+import google.registry.request.Modules.UrlConnectionServiceModule;
+import google.registry.request.Modules.UrlFetchServiceModule;
 import google.registry.request.Modules.UrlFetchTransportModule;
 import google.registry.request.Modules.UserServiceModule;
 import google.registry.request.auth.AuthModule;
@@ -80,7 +81,8 @@ import javax.inject.Singleton;
      ServerTridProviderModule.class,
      SheetsServiceModule.class,
      StackdriverModule.class,
-      URLFetchServiceModule.class,
+      UrlConnectionServiceModule.class,
+      UrlFetchServiceModule.class,
      UrlFetchTransportModule.class,
      UserServiceModule.class,
      VoidDnsWriterModule.class,
--- a/core/src/main/java/google/registry/module/frontend/FrontendComponent.java
+++ b/core/src/main/java/google/registry/module/frontend/FrontendComponent.java
@@ -34,7 +34,6 @@ import google.registry.monitoring.whitebox.StackdriverModule;
 import google.registry.privileges.secretmanager.SecretManagerModule;
 import google.registry.request.Modules.Jackson2Module;
 import google.registry.request.Modules.NetHttpTransportModule;
-import google.registry.request.Modules.UrlFetchTransportModule;
 import google.registry.request.Modules.UserServiceModule;
 import google.registry.request.auth.AuthModule;
 import google.registry.ui.ConsoleDebug.ConsoleConfigModule;
@@ -65,7 +64,6 @@ import javax.inject.Singleton;
      SecretManagerModule.class,
      ServerTridProviderModule.class,
      StackdriverModule.class,
-      UrlFetchTransportModule.class,
      UserServiceModule.class,
      UtilsModule.class
    })
--- a/core/src/main/java/google/registry/module/pubapi/PubApiComponent.java
+++ b/core/src/main/java/google/registry/module/pubapi/PubApiComponent.java
@@ -34,7 +34,6 @@ import google.registry.persistence.PersistenceModule;
 import google.registry.privileges.secretmanager.SecretManagerModule;
 import google.registry.request.Modules.Jackson2Module;
 import google.registry.request.Modules.NetHttpTransportModule;
-import google.registry.request.Modules.UrlFetchTransportModule;
 import google.registry.request.Modules.UserServiceModule;
 import google.registry.request.auth.AuthModule;
 import google.registry.util.UtilsModule;
@@ -62,7 +61,6 @@ import javax.inject.Singleton;
      SecretManagerModule.class,
      ServerTridProviderModule.class,
      StackdriverModule.class,
-      UrlFetchTransportModule.class,
      UserServiceModule.class,
      UtilsModule.class
    })
--- a/core/src/main/java/google/registry/module/tools/ToolsComponent.java
+++ b/core/src/main/java/google/registry/module/tools/ToolsComponent.java
@@ -36,7 +36,6 @@ import google.registry.privileges.secretmanager.SecretManagerModule;
 import google.registry.request.Modules.DatastoreServiceModule;
 import google.registry.request.Modules.Jackson2Module;
 import google.registry.request.Modules.NetHttpTransportModule;
-import google.registry.request.Modules.UrlFetchTransportModule;
 import google.registry.request.Modules.UserServiceModule;
 import google.registry.request.auth.AuthModule;
 import google.registry.util.UtilsModule;
@@ -66,7 +65,6 @@ import javax.inject.Singleton;
      ServerTridProviderModule.class,
      StackdriverModule.class,
      ToolsRequestComponentModule.class,
-      UrlFetchTransportModule.class,
      UserServiceModule.class,
      UtilsModule.class
    })
--- a/core/src/main/java/google/registry/persistence/transaction/TransactionEntity.java
+++ b/core/src/main/java/google/registry/persistence/transaction/TransactionEntity.java
@@ -14,6 +14,7 @@

 package google.registry.persistence.transaction;

+import com.google.common.annotations.VisibleForTesting;
 import google.registry.model.ImmutableObject;
 import google.registry.model.replay.SqlOnlyEntity;
 import javax.persistence.Entity;
@@ -39,7 +40,8 @@ public class TransactionEntity extends ImmutableObject implements SqlOnlyEntity

  TransactionEntity() {}

-  TransactionEntity(byte[] contents) {
+  @VisibleForTesting
+  public TransactionEntity(byte[] contents) {
    this.contents = contents;
  }

--- a/core/src/main/java/google/registry/persistence/transaction/TransactionManagerFactory.java
+++ b/core/src/main/java/google/registry/persistence/transaction/TransactionManagerFactory.java
@@ -15,6 +15,7 @@
 package google.registry.persistence.transaction;

 import static com.google.common.base.Preconditions.checkState;
+import static google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState.DATASTORE_PRIMARY_NO_ASYNC;
 import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
 import static org.joda.time.DateTimeZone.UTC;

@@ -23,6 +24,7 @@ import com.google.appengine.api.utils.SystemProperty.Environment.Value;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Suppliers;
 import google.registry.config.RegistryEnvironment;
+import google.registry.model.annotations.DeleteAfterMigration;
 import google.registry.model.common.DatabaseMigrationStateSchedule;
 import google.registry.model.common.DatabaseMigrationStateSchedule.PrimaryDatabase;
 import google.registry.model.ofy.DatastoreTransactionManager;
@@ -198,6 +200,22 @@ public final class TransactionManagerFactory {
    }
  }

+  /**
+   * Asserts that async actions (contact/host deletes and host renames) are allowed.
+   *
+   * <p>These are allowed at all times except during the {@link
+   * DatabaseMigrationStateSchedule.MigrationState#DATASTORE_PRIMARY_NO_ASYNC} stage. Note that
+   * {@link ReadOnlyModeException} may well be thrown during other read-only stages inside the
+   * transaction manager; this method specifically checks only async actions.
+   */
+  @DeleteAfterMigration
+  public static void assertAsyncActionsAreAllowed() {
+    if (DatabaseMigrationStateSchedule.getValueAtTime(DateTime.now(UTC))
+        .equals(DATASTORE_PRIMARY_NO_ASYNC)) {
+      throw new ReadOnlyModeException();
+    }
+  }
+
  /** Registry is currently undergoing maintenance and is in read-only mode. */
  public static class ReadOnlyModeException extends IllegalStateException {
    public ReadOnlyModeException() {
--- a/core/src/main/java/google/registry/rde/BrdaCopyAction.java
+++ b/core/src/main/java/google/registry/rde/BrdaCopyAction.java
@@ -14,8 +14,13 @@

 package google.registry.rde;

+import static google.registry.model.common.Cursor.CursorType.BRDA;
+import static google.registry.model.common.Cursor.getCursorTimeOrStartOfTime;
 import static google.registry.model.rde.RdeMode.THIN;
+import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
+import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
 import static google.registry.request.Action.Method.POST;
+import static google.registry.util.DateTimeUtils.isBeforeOrAt;

 import com.google.cloud.storage.BlobId;
 import com.google.common.flogger.FluentLogger;
@@ -23,9 +28,11 @@ import com.google.common.io.ByteStreams;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.gcs.GcsUtils;
 import google.registry.keyring.api.KeyModule.Key;
+import google.registry.model.common.Cursor;
 import google.registry.model.rde.RdeNamingUtils;
 import google.registry.model.rde.RdeRevision;
 import google.registry.request.Action;
+import google.registry.request.HttpException.NoContentException;
 import google.registry.request.Parameter;
 import google.registry.request.RequestParameters;
 import google.registry.request.auth.Auth;
@@ -89,6 +96,16 @@ public final class BrdaCopyAction implements Runnable {
  private void copyAsRyde() throws IOException {
    // TODO(b/217772483): consider guarding this action with a lock and check if there is work.
    // Not urgent since file writes on GCS are atomic.
+    Optional<Cursor> cursor =
+        transactIfJpaTm(() -> tm().loadByKeyIfPresent(Cursor.createVKey(BRDA, tld)));
+    DateTime brdaCursorTime = getCursorTimeOrStartOfTime(cursor);
+    if (isBeforeOrAt(brdaCursorTime, watermark)) {
+      throw new NoContentException(
+          String.format(
+              "Waiting on RdeStagingAction for TLD %s to copy BRDA deposit for %s to GCS; "
+                  + "last BRDA staging completion was before %s",
+              tld, watermark, brdaCursorTime));
+    }
    int revision =
        RdeRevision.getCurrentRevision(tld, watermark, THIN)
            .orElseThrow(
--- a/core/src/main/java/google/registry/rde/RdeReporter.java
+++ b/core/src/main/java/google/registry/rde/RdeReporter.java
@@ -33,7 +33,6 @@ import google.registry.config.RegistryConfig.Config;
 import google.registry.keyring.api.KeyModule.Key;
 import google.registry.request.HttpException.InternalServerErrorException;
 import google.registry.util.Retrier;
-import google.registry.util.UrlFetchException;
 import google.registry.xjc.XjcXmlTransformer;
 import google.registry.xjc.iirdea.XjcIirdeaResponseElement;
 import google.registry.xjc.iirdea.XjcIirdeaResult;
@@ -55,12 +54,15 @@ public class RdeReporter {

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

-  /** @see <a href="http://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-05#section-4">
-   *     ICANN Registry Interfaces - Interface details</a>*/
+  /**
+   * @see <a href="http://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-05#section-4">
+   *     ICANN Registry Interfaces - Interface details</a>
+   */
  private static final String REPORT_MIME = "text/xml";

  @Inject Retrier retrier;
  @Inject URLFetchService urlFetchService;
+
  @Inject @Config("rdeReportUrlPrefix") String reportUrlPrefix;
  @Inject @Key("icannReportingPassword") String password;
  @Inject RdeReporter() {}
@@ -89,14 +91,14 @@ public class RdeReporter {
                case SC_BAD_REQUEST:
                  break;
                default:
-                  throw new UrlFetchException("PUT failed", req, rsp1);
+                  throw new RuntimeException("PUT failed");
              }
              return rsp1;
            },
            SocketTimeoutException.class);

    // Ensure the XML response is valid.
-    XjcIirdeaResult result = parseResult(rsp);
+    XjcIirdeaResult result = parseResult(rsp.getContent());
    if (result.getCode().getValue() != 1000) {
      logger.atWarning().log(
          "PUT rejected: %d %s\n%s",
@@ -108,11 +110,11 @@ public class RdeReporter {
  /**
   * Unmarshals IIRDEA XML result object from {@link HTTPResponse} payload.
   *
-   * @see <a href="http://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-05#section-4.1">
+   * @see <a
+   *     href="http://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-05#section-4.1">
   *     ICANN Registry Interfaces - IIRDEA Result Object</a>
   */
-  private XjcIirdeaResult parseResult(HTTPResponse rsp) throws XmlException {
-    byte[] responseBytes = rsp.getContent();
+  private XjcIirdeaResult parseResult(byte[] responseBytes) throws XmlException {
    logger.atInfo().log("Received response:\n%s", new String(responseBytes, UTF_8));
    XjcIirdeaResponseElement response = XjcXmlTransformer.unmarshal(
        XjcIirdeaResponseElement.class, new ByteArrayInputStream(responseBytes));
--- a/core/src/main/java/google/registry/rde/RdeStagingAction.java
+++ b/core/src/main/java/google/registry/rde/RdeStagingAction.java
@@ -261,6 +261,10 @@ public final class RdeStagingAction implements Runnable {
  @Config("highPerformanceMachineType")
  String machineType;

+  @Inject
+  @Config("initialWorkerCount")
+  int numWorkers;
+
  @Inject @Config("transactionCooldown") Duration transactionCooldown;
  @Inject @Config("beamStagingBucketUrl") String stagingBucketUrl;
  @Inject @Config("rdeBucket") String rdeBucket;
@@ -341,6 +345,7 @@ public final class RdeStagingAction implements Runnable {
                                          .encode(stagingKeyBytes))
                                  .put("registryEnvironment", RegistryEnvironment.get().name())
                                  .put("workerMachineType", machineType)
+                                  .put("numWorkers", String.valueOf(numWorkers))
                                  .put(
                                      "jpaTransactionManagerType",
                                      JpaTransactionManagerType.READ_ONLY_REPLICA.toString())
--- a/core/src/main/java/google/registry/rde/RdeStagingReducer.java
+++ b/core/src/main/java/google/registry/rde/RdeStagingReducer.java
@@ -64,6 +64,7 @@ public final class RdeStagingReducer extends Reducer<PendingDeposit, DepositFrag
  private static final long serialVersionUID = 60326234579091203L;

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();
+  private static final Duration ENQUEUE_DELAY = Duration.standardMinutes(1);

  private final CloudTasksUtils cloudTasksUtils;
  private final LockHandler lockHandler;
@@ -202,6 +203,14 @@ public final class RdeStagingReducer extends Reducer<PendingDeposit, DepositFrag
      logger.atInfo().log("Manual operation; not advancing cursor or enqueuing upload task.");
      return;
    }
+    // We need to save the revision in a separate transaction because the subsequent upload/copy
+    // action reads the most current revision from the database. If it is done in the same
+    // transaction with the enqueueing, the action might start running before the transaction is
+    // committed, due to Cloud Tasks not being transaction aware, unlike Task Queue. The downside
+    // is that if for some reason the second transaction is rolled back, the revision update is not
+    // undone. But this should be fine since the next run will just increment the revision and start
+    // over.
+    tm().transact(() -> RdeRevision.saveRevision(tld, watermark, mode, revision));
    tm().transact(
            () -> {
              Registry registry = Registry.get(tld);
@@ -225,29 +234,33 @@ public final class RdeStagingReducer extends Reducer<PendingDeposit, DepositFrag
              tm().put(Cursor.create(key.cursor(), newPosition, registry));
              logger.atInfo().log(
                  "Rolled forward %s on %s cursor to %s.", key.cursor(), tld, newPosition);
-              RdeRevision.saveRevision(tld, watermark, mode, revision);
              // Enqueueing a task is a side effect that is not undone if the transaction rolls
-              // back. So this may result in multiple copies of the same task being processed. This
-              // is fine because the RdeUploadAction is guarded by a lock and tracks progress by
-              // cursor. The BrdaCopyAction writes a file to GCS, which is an atomic action.
+              // back. So this may result in multiple copies of the same task being processed.
+              // This is fine because the RdeUploadAction is guarded by a lock and tracks progress
+              // by cursor. The BrdaCopyAction writes a file to GCS, which is an atomic action. It
+              // is also guarded by a cursor to not run before the cursor is updated. We also
+              // include a delay to minimize the chance that the enqueued job executes before the
+              // transaction is committed, which triggers a retry.
              if (mode == RdeMode.FULL) {
                cloudTasksUtils.enqueue(
                    "rde-upload",
-                    cloudTasksUtils.createPostTask(
+                    cloudTasksUtils.createPostTaskWithDelay(
                        RdeUploadAction.PATH,
                        Service.BACKEND.toString(),
-                        ImmutableMultimap.of(RequestParameters.PARAM_TLD, tld)));
+                        ImmutableMultimap.of(RequestParameters.PARAM_TLD, tld),
+                        ENQUEUE_DELAY));
              } else {
                cloudTasksUtils.enqueue(
                    "brda",
-                    cloudTasksUtils.createPostTask(
+                    cloudTasksUtils.createPostTaskWithDelay(
                        BrdaCopyAction.PATH,
                        Service.BACKEND.toString(),
                        ImmutableMultimap.of(
                            RequestParameters.PARAM_TLD,
                            tld,
                            RdeModule.PARAM_WATERMARK,
-                            watermark.toString())));
+                            watermark.toString()),
+                        ENQUEUE_DELAY));
              }
            });
  }
--- a/core/src/main/java/google/registry/rde/RdeUploadAction.java
+++ b/core/src/main/java/google/registry/rde/RdeUploadAction.java
@@ -32,6 +32,7 @@ import static java.util.Arrays.asList;
 import com.google.cloud.storage.BlobId;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.collect.HashMultimap;
+import com.google.common.collect.Ordering;
 import com.google.common.flogger.FluentLogger;
 import com.google.common.io.ByteStreams;
 import com.jcraft.jsch.JSch;
@@ -122,7 +123,6 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
  @Inject @Key("rdeSigningKey") PGPKeyPair signingKey;
  @Inject @Key("rdeStagingDecryptionKey") PGPPrivateKey stagingDecryptionKey;
  @Inject RdeUploadAction() {}
-
  @Override
  public void run() {
    logger.atInfo().log("Attempting to acquire RDE upload lock for TLD '%s'.", tld);
@@ -140,6 +140,27 @@ public final class RdeUploadAction implements Runnable, EscrowTask {

  @Override
  public void runWithLock(final DateTime watermark) throws Exception {
+    // If a prefix is not provided, but we are in SQL mode, try to determine the prefix. This should
+    // only happen when the RDE upload cron job runs to catch up any un-retried (i. e. expected)
+    // RDE failures.
+    if (prefix.isEmpty() && !tm().isOfy()) {
+      // The prefix is always in the format of: rde-2022-02-21t00-00-00z-2022-02-21t00-07-33z, where
+      // the first datetime is the watermark and the second one is the time when the RDE beam job
+      // launched. We search for the latest folder that starts with "rde-[watermark]".
+      String partialPrefix =
+          String.format("rde-%s", watermark.toString("yyyy-MM-dd't'HH-mm-ss'z'"));
+      String latestFilenameSuffix =
+          gcsUtils.listFolderObjects(bucket, partialPrefix).stream()
+              .max(Ordering.natural())
+              .orElse(null);
+      if (latestFilenameSuffix == null) {
+        throw new NoContentException(
+            String.format("RDE deposit for TLD %s on %s does not exist", tld, watermark));
+      }
+      int firstSlashPosition = latestFilenameSuffix.indexOf('/');
+      prefix =
+          Optional.of(partialPrefix + latestFilenameSuffix.substring(0, firstSlashPosition + 1));
+    }
    logger.atInfo().log("Verifying readiness to upload the RDE deposit.");
    Optional<Cursor> cursor =
        transactIfJpaTm(() -> tm().loadByKeyIfPresent(Cursor.createVKey(RDE_STAGING, tld)));
@@ -148,7 +169,7 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
      throw new NoContentException(
          String.format(
              "Waiting on RdeStagingAction for TLD %s to send %s upload; "
-                  + "last RDE staging completion was at %s",
+                  + "last RDE staging completion was before %s",
              tld, watermark, stagingCursorTime));
    }
    DateTime sftpCursorTime =
@@ -241,9 +262,9 @@ public final class RdeUploadAction implements Runnable, EscrowTask {
                    .setSignatureOutput(sigOut, signingKey)
                    .setFileMetadata(nameWithoutPrefix, xmlLength, watermark)
                    .build()) {
-            long bytesCopied = ByteStreams.copy(ghostrydeDecoder, rydeEncoder);
+          long bytesCopied = ByteStreams.copy(ghostrydeDecoder, rydeEncoder);
          logger.atInfo().log("Uploaded %,d bytes to path '%s'.", bytesCopied, rydeFilename);
-          }
+        }
        String sigFilename = nameWithoutPrefix + ".sig";
        BlobId sigGcsFilename = BlobId.of(bucket, name + ".sig");
        byte[] signature = sigOut.toByteArray();
--- a/core/src/main/java/google/registry/request/Modules.java
+++ b/core/src/main/java/google/registry/request/Modules.java
@@ -29,6 +29,7 @@ import com.google.appengine.api.users.UserService;
 import com.google.appengine.api.users.UserServiceFactory;
 import dagger.Module;
 import dagger.Provides;
+import java.net.HttpURLConnection;
 import javax.inject.Singleton;

 /** Dagger modules for App Engine services and other vendor classes. */
@@ -45,13 +46,22 @@ public final class Modules {
    }
  }

+  /** Dagger module for {@link UrlConnectionService}. */
+  @Module
+  public static final class UrlConnectionServiceModule {
+    @Provides
+    static UrlConnectionService provideUrlConnectionService() {
+      return url -> (HttpURLConnection) url.openConnection();
+    }
+  }
+
  /** Dagger module for {@link URLFetchService}. */
  @Module
-  public static final class URLFetchServiceModule {
+  public static final class UrlFetchServiceModule {
    private static final URLFetchService fetchService = URLFetchServiceFactory.getURLFetchService();

    @Provides
-    static URLFetchService provideURLFetchService() {
+    static URLFetchService provideUrlFetchService() {
      return fetchService;
    }
  }
--- a/core/src/main/java/google/registry/request/RequestModule.java
+++ b/core/src/main/java/google/registry/request/RequestModule.java
@@ -30,6 +30,7 @@ import com.google.common.collect.ImmutableSet;
 import com.google.common.io.ByteStreams;
 import com.google.common.io.CharStreams;
 import com.google.common.net.MediaType;
+import com.google.protobuf.ByteString;
 import dagger.Module;
 import dagger.Provides;
 import google.registry.model.common.DatabaseMigrationStateSchedule.PrimaryDatabase;
@@ -184,6 +185,16 @@ public final class RequestModule {
    }
  }

+  @Provides
+  @Payload
+  static ByteString providePayloadAsByteString(HttpServletRequest req) {
+    try {
+      return ByteString.copyFrom(ByteStreams.toByteArray(req.getInputStream()));
+    } catch (IOException e) {
+      throw new RuntimeException(e);
+    }
+  }
+
  @Provides
  static LockHandler provideLockHandler(LockHandlerImpl lockHandler) {
    return lockHandler;
--- a/core/src/main/java/google/registry/request/UrlConnectionService.java
+++ b/core/src/main/java/google/registry/request/UrlConnectionService.java
@@ -0,0 +1,26 @@
+// Copyright 2022 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.request;
+
+import java.io.IOException;
+import java.net.HttpURLConnection;
+import java.net.URL;
+import java.security.GeneralSecurityException;
+
+/** Functional interface for opening a connection from a URL, injectable for testing. */
+public interface UrlConnectionService {
+
+  HttpURLConnection createConnection(URL url) throws IOException, GeneralSecurityException;
+}
--- a/core/src/main/java/google/registry/request/UrlConnectionUtils.java
+++ b/core/src/main/java/google/registry/request/UrlConnectionUtils.java
@@ -1,4 +1,4 @@
-// Copyright 2017 The Nomulus Authors. All Rights Reserved.
+// Copyright 2022 The Nomulus Authors. All Rights Reserved.
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-package google.registry.util;
+package google.registry.request;

 import static com.google.common.base.Preconditions.checkState;
 import static com.google.common.io.BaseEncoding.base64;
@@ -22,36 +22,41 @@ import static com.google.common.net.HttpHeaders.CONTENT_LENGTH;
 import static com.google.common.net.HttpHeaders.CONTENT_TYPE;
 import static java.nio.charset.StandardCharsets.UTF_8;

-import com.google.appengine.api.urlfetch.HTTPHeader;
-import com.google.appengine.api.urlfetch.HTTPRequest;
-import com.google.appengine.api.urlfetch.HTTPResponse;
-import com.google.common.base.Ascii;
 import com.google.common.base.Strings;
+import com.google.common.io.ByteStreams;
 import com.google.common.net.MediaType;
-import java.util.Optional;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.net.URLConnection;
 import java.util.Random;

-/** Helper methods for the App Engine URL fetch service. */
-public final class UrlFetchUtils {
+/** Utilities for common functionality relating to {@link java.net.URLConnection}s. */
+public class UrlConnectionUtils {

-  /** Returns value of first header matching {@code name}. */
-  public static Optional<String> getHeaderFirst(HTTPResponse rsp, String name) {
-    return getHeaderFirstInternal(rsp.getHeadersUncombined(), name);
+  /** Retrieves the response from the given connection as a byte array. */
+  public static byte[] getResponseBytes(URLConnection connection) throws IOException {
+    return ByteStreams.toByteArray(connection.getInputStream());
  }

-  /** Returns value of first header matching {@code name}. */
-  public static Optional<String> getHeaderFirst(HTTPRequest req, String name) {
-    return getHeaderFirstInternal(req.getHeaders(), name);
+  /** Sets auth on the given connection with the given username/password. */
+  public static void setBasicAuth(URLConnection connection, String username, String password) {
+    setBasicAuth(connection, String.format("%s:%s", username, password));
  }

-  private static Optional<String> getHeaderFirstInternal(Iterable<HTTPHeader> hdrs, String name) {
-    name = Ascii.toLowerCase(name);
-    for (HTTPHeader header : hdrs) {
-      if (Ascii.toLowerCase(header.getName()).equals(name)) {
-        return Optional.of(header.getValue());
-      }
+  /** Sets auth on the given connection with the given string, formatted "username:password". */
+  public static void setBasicAuth(URLConnection connection, String usernameAndPassword) {
+    String token = base64().encode(usernameAndPassword.getBytes(UTF_8));
+    connection.setRequestProperty(AUTHORIZATION, "Basic " + token);
+  }
+
+  /** Sets the given byte[] payload on the given connection with a particular content type. */
+  public static void setPayload(URLConnection connection, byte[] bytes, String contentType)
+      throws IOException {
+    connection.setRequestProperty(CONTENT_TYPE, contentType);
+    connection.setDoOutput(true);
+    try (DataOutputStream dataStream = new DataOutputStream(connection.getOutputStream())) {
+      dataStream.write(bytes);
    }
-    return Optional.empty();
  }

  /**
@@ -62,16 +67,16 @@ public final class UrlFetchUtils {
   * @see <a href="http://www.ietf.org/rfc/rfc2388.txt">RFC2388 - Returning Values from Forms</a>
   */
  public static void setPayloadMultipart(
-      HTTPRequest request,
+      URLConnection connection,
      String name,
      String filename,
      MediaType contentType,
      String data,
-      Random random) {
+      Random random)
+      throws IOException {
    String boundary = createMultipartBoundary(random);
    checkState(
-        !data.contains(boundary),
-        "Multipart data contains autogenerated boundary: %s", boundary);
+        !data.contains(boundary), "Multipart data contains autogenerated boundary: %s", boundary);
    String multipart =
        String.format("--%s\r\n", boundary)
            + String.format(
@@ -83,11 +88,9 @@ public final class UrlFetchUtils {
            + "\r\n"
            + String.format("--%s--\r\n", boundary);
    byte[] payload = multipart.getBytes(UTF_8);
-    request.addHeader(
-        new HTTPHeader(
-            CONTENT_TYPE, String.format("multipart/form-data;" + " boundary=\"%s\"", boundary)));
-    request.addHeader(new HTTPHeader(CONTENT_LENGTH, Integer.toString(payload.length)));
-    request.setPayload(payload);
+    connection.setRequestProperty(CONTENT_LENGTH, Integer.toString(payload.length));
+    setPayload(
+        connection, payload, String.format("multipart/form-data;" + " boundary=\"%s\"", boundary));
  }

  private static String createMultipartBoundary(Random random) {
@@ -98,12 +101,4 @@ public final class UrlFetchUtils {
    // See https://tools.ietf.org/html/rfc2046#section-5.1.1
    return Strings.repeat("-", 30) + base64().encode(rand);
  }
-
-  /** Sets the HTTP Basic Authentication header on an {@link HTTPRequest}. */
-  public static void setAuthorizationHeader(HTTPRequest req, Optional<String> login) {
-    if (login.isPresent()) {
-      String token = base64().encode(login.get().getBytes(UTF_8));
-      req.addHeader(new HTTPHeader(AUTHORIZATION, "Basic " + token));
-    }
-  }
 }
--- a/core/src/main/java/google/registry/tmch/LordnRequestInitializer.java
+++ b/core/src/main/java/google/registry/tmch/LordnRequestInitializer.java
@@ -15,12 +15,12 @@
 package google.registry.tmch;

 import static com.google.common.base.Verify.verifyNotNull;
-import static google.registry.util.UrlFetchUtils.setAuthorizationHeader;

-import com.google.appengine.api.urlfetch.HTTPRequest;
 import com.google.common.flogger.FluentLogger;
 import google.registry.keyring.api.KeyModule.Key;
 import google.registry.model.tld.Registry;
+import google.registry.request.UrlConnectionUtils;
+import java.net.HttpURLConnection;
 import java.util.Optional;
 import javax.inject.Inject;

@@ -37,8 +37,9 @@ final class LordnRequestInitializer {
  }

  /** Initializes a URL fetch request for talking to the MarksDB server. */
-  void initialize(HTTPRequest request, String tld) {
-    setAuthorizationHeader(request, getMarksDbLordnCredentials(tld));
+  void initialize(HttpURLConnection connection, String tld) {
+    getMarksDbLordnCredentials(tld)
+        .ifPresent(login -> UrlConnectionUtils.setBasicAuth(connection, login));
  }

  /** Returns the username and password for the current TLD to login to the MarksDB server. */
--- a/core/src/main/java/google/registry/tmch/Marksdb.java
+++ b/core/src/main/java/google/registry/tmch/Marksdb.java
@@ -14,26 +14,25 @@

 package google.registry.tmch;

-import static com.google.appengine.api.urlfetch.FetchOptions.Builder.validateCertificate;
-import static com.google.appengine.api.urlfetch.HTTPMethod.GET;
 import static com.google.common.base.Preconditions.checkArgument;
+import static google.registry.request.UrlConnectionUtils.getResponseBytes;
+import static google.registry.request.UrlConnectionUtils.setBasicAuth;
 import static google.registry.util.HexDumper.dumpHex;
-import static google.registry.util.UrlFetchUtils.setAuthorizationHeader;
 import static java.nio.charset.StandardCharsets.US_ASCII;
 import static javax.servlet.http.HttpServletResponse.SC_OK;

-import com.google.appengine.api.urlfetch.HTTPRequest;
-import com.google.appengine.api.urlfetch.HTTPResponse;
-import com.google.appengine.api.urlfetch.URLFetchService;
 import com.google.common.collect.ImmutableList;
 import com.google.common.flogger.FluentLogger;
 import com.google.common.io.ByteSource;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.keyring.api.KeyModule.Key;
-import google.registry.util.UrlFetchException;
+import google.registry.request.UrlConnectionService;
+import google.registry.util.UrlConnectionException;
 import java.io.ByteArrayInputStream;
 import java.io.IOException;
+import java.net.HttpURLConnection;
 import java.net.URL;
+import java.security.GeneralSecurityException;
 import java.security.Security;
 import java.security.SignatureException;
 import java.util.Arrays;
@@ -57,7 +56,8 @@ public final class Marksdb {
  private static final FluentLogger logger = FluentLogger.forEnclosingClass();
  private static final int MAX_DNL_LOGGING_LENGTH = 500;

-  @Inject URLFetchService fetchService;
+  @Inject UrlConnectionService urlConnectionService;
+
  @Inject @Config("tmchMarksdbUrl") String tmchMarksdbUrl;
  @Inject @Key("marksdbPublicKey") PGPPublicKey marksdbPublicKey;
  @Inject Marksdb() {}
@@ -111,24 +111,22 @@ public final class Marksdb {
    }
  }

-  byte[] fetch(URL url, Optional<String> loginAndPassword) throws IOException {
-    HTTPRequest req = new HTTPRequest(url, GET, validateCertificate().setDeadline(60d));
-    setAuthorizationHeader(req, loginAndPassword);
-    HTTPResponse rsp;
+  byte[] fetch(URL url, Optional<String> loginAndPassword)
+      throws IOException, GeneralSecurityException {
+    HttpURLConnection connection = urlConnectionService.createConnection(url);
+    loginAndPassword.ifPresent(auth -> setBasicAuth(connection, auth));
    try {
-      rsp = fetchService.fetch(req);
+      if (connection.getResponseCode() != SC_OK) {
+        throw new UrlConnectionException("Failed to fetch from MarksDB", connection);
+      }
+      return getResponseBytes(connection);
    } catch (IOException e) {
-      throw new IOException(
-          String.format("Error connecting to MarksDB at URL %s", url), e);
+      throw new IOException(String.format("Error connecting to MarksDB at URL %s", url), e);
    }
-    if (rsp.getResponseCode() != SC_OK) {
-      throw new UrlFetchException("Failed to fetch from MarksDB", req, rsp);
-    }
-    return rsp.getContent();
  }

  List<String> fetchSignedCsv(Optional<String> loginAndPassword, String csvPath, String sigPath)
-      throws IOException, SignatureException, PGPException {
+      throws IOException, GeneralSecurityException, PGPException {
    checkArgument(
        loginAndPassword.isPresent(), "Cannot fetch from MarksDB without login credentials");

--- a/core/src/main/java/google/registry/tmch/NordnUploadAction.java
+++ b/core/src/main/java/google/registry/tmch/NordnUploadAction.java
@@ -16,28 +16,23 @@ package google.registry.tmch;

 import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
 import static com.google.appengine.api.taskqueue.TaskOptions.Builder.withUrl;
-import static com.google.appengine.api.urlfetch.FetchOptions.Builder.validateCertificate;
-import static com.google.appengine.api.urlfetch.HTTPMethod.POST;
 import static com.google.common.base.Preconditions.checkArgument;
 import static com.google.common.base.Preconditions.checkNotNull;
 import static com.google.common.net.HttpHeaders.LOCATION;
 import static com.google.common.net.MediaType.CSV_UTF_8;
+import static google.registry.request.UrlConnectionUtils.getResponseBytes;
 import static google.registry.tmch.LordnTaskUtils.COLUMNS_CLAIMS;
 import static google.registry.tmch.LordnTaskUtils.COLUMNS_SUNRISE;
-import static google.registry.util.UrlFetchUtils.getHeaderFirst;
-import static google.registry.util.UrlFetchUtils.setPayloadMultipart;
 import static java.nio.charset.StandardCharsets.US_ASCII;
 import static java.nio.charset.StandardCharsets.UTF_8;
 import static javax.servlet.http.HttpServletResponse.SC_ACCEPTED;

+import com.google.api.client.http.HttpMethods;
 import com.google.appengine.api.taskqueue.LeaseOptions;
 import com.google.appengine.api.taskqueue.Queue;
 import com.google.appengine.api.taskqueue.TaskHandle;
 import com.google.appengine.api.taskqueue.TaskOptions;
 import com.google.appengine.api.taskqueue.TransientFailureException;
-import com.google.appengine.api.urlfetch.HTTPRequest;
-import com.google.appengine.api.urlfetch.HTTPResponse;
-import com.google.appengine.api.urlfetch.URLFetchService;
 import com.google.apphosting.api.DeadlineExceededException;
 import com.google.common.base.Joiner;
 import com.google.common.base.Strings;
@@ -49,16 +44,19 @@ import google.registry.config.RegistryConfig.Config;
 import google.registry.request.Action;
 import google.registry.request.Parameter;
 import google.registry.request.RequestParameters;
+import google.registry.request.UrlConnectionService;
+import google.registry.request.UrlConnectionUtils;
 import google.registry.request.auth.Auth;
 import google.registry.util.Clock;
 import google.registry.util.Retrier;
 import google.registry.util.TaskQueueUtils;
-import google.registry.util.UrlFetchException;
+import google.registry.util.UrlConnectionException;
 import java.io.IOException;
+import java.net.HttpURLConnection;
 import java.net.URL;
+import java.security.GeneralSecurityException;
 import java.security.SecureRandom;
 import java.util.List;
-import java.util.Optional;
 import java.util.Random;
 import java.util.concurrent.TimeUnit;
 import javax.inject.Inject;
@@ -97,7 +95,8 @@ public final class NordnUploadAction implements Runnable {
  @Inject Retrier retrier;
  @Inject SecureRandom random;
  @Inject LordnRequestInitializer lordnRequestInitializer;
-  @Inject URLFetchService fetchService;
+  @Inject UrlConnectionService urlConnectionService;
+
  @Inject @Config("tmchMarksdbUrl") String tmchMarksdbUrl;
  @Inject @Parameter(LORDN_PHASE_PARAM) String phase;
  @Inject @Parameter(RequestParameters.PARAM_TLD) String tld;
@@ -118,7 +117,7 @@ public final class NordnUploadAction implements Runnable {
  public void run() {
    try {
      processLordnTasks();
-    } catch (IOException e) {
+    } catch (IOException | GeneralSecurityException e) {
      throw new RuntimeException(e);
    }
  }
@@ -163,7 +162,7 @@ public final class NordnUploadAction implements Runnable {
    }
  }

-  private void processLordnTasks() throws IOException {
+  private void processLordnTasks() throws IOException, GeneralSecurityException {
    checkArgument(phase.equals(PARAM_LORDN_PHASE_SUNRISE)
        || phase.equals(PARAM_LORDN_PHASE_CLAIMS),
        "Invalid phase specified to Nordn servlet: %s.", phase);
@@ -193,47 +192,49 @@ public final class NordnUploadAction implements Runnable {
   * <p>Idempotency: If the exact same LORDN report is uploaded twice, the MarksDB server will
   * return the same confirmation number.
   *
-   * @see <a href="http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08#section-6.3">
-   *     TMCH functional specifications - LORDN File</a>
+   * @see <a href="http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08#section-6.3">TMCH
+   *     functional specifications - LORDN File</a>
   */
-  private void uploadCsvToLordn(String urlPath, String csvData) throws IOException {
+  private void uploadCsvToLordn(String urlPath, String csvData)
+      throws IOException, GeneralSecurityException {
    String url = tmchMarksdbUrl + urlPath;
    logger.atInfo().log(
        "LORDN upload task %s: Sending to URL: %s ; data: %s", actionLogId, url, csvData);
-    HTTPRequest req = new HTTPRequest(new URL(url), POST, validateCertificate().setDeadline(60d));
-    lordnRequestInitializer.initialize(req, tld);
-    setPayloadMultipart(req, "file", "claims.csv", CSV_UTF_8, csvData, random);
-    HTTPResponse rsp;
+    HttpURLConnection connection = urlConnectionService.createConnection(new URL(url));
+    connection.setRequestMethod(HttpMethods.POST);
+    lordnRequestInitializer.initialize(connection, tld);
+    UrlConnectionUtils.setPayloadMultipart(
+        connection, "file", "claims.csv", CSV_UTF_8, csvData, random);
    try {
-      rsp = fetchService.fetch(req);
+      int responseCode = connection.getResponseCode();
+      if (logger.atInfo().isEnabled()) {
+        String responseContent = new String(getResponseBytes(connection), US_ASCII);
+        if (responseContent.isEmpty()) {
+          responseContent = "(null)";
+        }
+        logger.atInfo().log(
+            "LORDN upload task %s response: HTTP response code %d, response data: %s",
+            actionLogId, responseCode, responseContent);
+      }
+      if (responseCode != SC_ACCEPTED) {
+        throw new UrlConnectionException(
+            String.format(
+                "LORDN upload task %s error: Failed to upload LORDN claims to MarksDB",
+                actionLogId),
+            connection);
+      }
+      String location = connection.getHeaderField(LOCATION);
+      if (location == null) {
+        throw new UrlConnectionException(
+            String.format(
+                "LORDN upload task %s error: MarksDB failed to provide a Location header",
+                actionLogId),
+            connection);
+      }
+      getQueue(NordnVerifyAction.QUEUE).add(makeVerifyTask(new URL(location)));
    } catch (IOException e) {
-      throw new IOException(
-          String.format("Error connecting to MarksDB at URL %s", url), e);
+      throw new IOException(String.format("Error connecting to MarksDB at URL %s", url), e);
    }
-    if (logger.atInfo().isEnabled()) {
-      String response =
-          (rsp.getContent() == null) ? "(null)" : new String(rsp.getContent(), US_ASCII);
-      logger.atInfo().log(
-          "LORDN upload task %s response: HTTP response code %d, response data: %s",
-          actionLogId, rsp.getResponseCode(), response);
-    }
-    if (rsp.getResponseCode() != SC_ACCEPTED) {
-      throw new UrlFetchException(
-          String.format(
-              "LORDN upload task %s error: Failed to upload LORDN claims to MarksDB", actionLogId),
-          req,
-          rsp);
-    }
-    Optional<String> location = getHeaderFirst(rsp, LOCATION);
-    if (!location.isPresent()) {
-      throw new UrlFetchException(
-          String.format(
-              "LORDN upload task %s error: MarksDB failed to provide a Location header",
-              actionLogId),
-          req,
-          rsp);
-    }
-    getQueue(NordnVerifyAction.QUEUE).add(makeVerifyTask(new URL(location.get())));
  }

  private TaskOptions makeVerifyTask(URL url) {
--- a/core/src/main/java/google/registry/tmch/NordnVerifyAction.java
+++ b/core/src/main/java/google/registry/tmch/NordnVerifyAction.java
@@ -14,15 +14,11 @@

 package google.registry.tmch;

-import static com.google.appengine.api.urlfetch.FetchOptions.Builder.validateCertificate;
-import static com.google.appengine.api.urlfetch.HTTPMethod.GET;
+import static google.registry.request.UrlConnectionUtils.getResponseBytes;
 import static java.nio.charset.StandardCharsets.UTF_8;
 import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;
 import static javax.servlet.http.HttpServletResponse.SC_OK;

-import com.google.appengine.api.urlfetch.HTTPRequest;
-import com.google.appengine.api.urlfetch.HTTPResponse;
-import com.google.appengine.api.urlfetch.URLFetchService;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.flogger.FluentLogger;
 import com.google.common.io.ByteSource;
@@ -32,10 +28,13 @@ import google.registry.request.HttpException.ConflictException;
 import google.registry.request.Parameter;
 import google.registry.request.RequestParameters;
 import google.registry.request.Response;
+import google.registry.request.UrlConnectionService;
 import google.registry.request.auth.Auth;
-import google.registry.util.UrlFetchException;
+import google.registry.util.UrlConnectionException;
 import java.io.IOException;
+import java.net.HttpURLConnection;
 import java.net.URL;
+import java.security.GeneralSecurityException;
 import java.util.Map.Entry;
 import javax.inject.Inject;

@@ -68,7 +67,8 @@ public final class NordnVerifyAction implements Runnable {

  @Inject LordnRequestInitializer lordnRequestInitializer;
  @Inject Response response;
-  @Inject URLFetchService fetchService;
+  @Inject UrlConnectionService urlConnectionService;
+
  @Inject @Header(URL_HEADER) URL url;
  @Inject @Header(HEADER_ACTION_LOG_ID) String actionLogId;
  @Inject @Parameter(RequestParameters.PARAM_TLD) String tld;
@@ -78,7 +78,7 @@ public final class NordnVerifyAction implements Runnable {
  public void run() {
    try {
      verify();
-    } catch (IOException e) {
+    } catch (IOException | GeneralSecurityException e) {
      throw new RuntimeException(e);
    }
  }
@@ -90,57 +90,55 @@ public final class NordnVerifyAction implements Runnable {
   * available.
   *
   * @throws ConflictException if MarksDB has not yet finished processing the LORDN upload
-   * @see <a href="http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08#section-6.3.1">
-   *     TMCH functional specifications LORDN Log File</a>
+   * @see <a href="http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08#section-6.3.1">TMCH
+   *     functional specifications LORDN Log File</a>
   */
  @VisibleForTesting
-  LordnLog verify() throws IOException {
+  LordnLog verify() throws IOException, GeneralSecurityException {
    logger.atInfo().log("LORDN verify task %s: Sending request to URL %s", actionLogId, url);
-    HTTPRequest req = new HTTPRequest(url, GET, validateCertificate().setDeadline(60d));
-    lordnRequestInitializer.initialize(req, tld);
-    HTTPResponse rsp;
+    HttpURLConnection connection = urlConnectionService.createConnection(url);
+    lordnRequestInitializer.initialize(connection, tld);
    try {
-      rsp = fetchService.fetch(req);
-    } catch (IOException e) {
-      throw new IOException(
-          String.format("Error connecting to MarksDB at URL %s", url), e);
-    }
-    logger.atInfo().log(
-        "LORDN verify task %s response: HTTP response code %d, response data: %s",
-        actionLogId, rsp.getResponseCode(), rsp.getContent());
-    if (rsp.getResponseCode() == SC_NO_CONTENT) {
-      // Send a 400+ status code so App Engine will retry the task.
-      throw new ConflictException("Not ready");
-    }
-    if (rsp.getResponseCode() != SC_OK) {
-      throw new UrlFetchException(
-          String.format("LORDN verify task %s: Failed to verify LORDN upload to MarksDB.",
-              actionLogId),
-          req, rsp);
-    }
-    LordnLog log =
-        LordnLog.parse(ByteSource.wrap(rsp.getContent()).asCharSource(UTF_8).readLines());
-    if (log.getStatus() == LordnLog.Status.ACCEPTED) {
-      logger.atInfo().log("LORDN verify task %s: Upload accepted.", actionLogId);
-    } else {
-      logger.atSevere().log(
-          "LORDN verify task %s: Upload rejected with reason: %s", actionLogId, log);
-    }
-    for (Entry<String, LordnLog.Result> result : log) {
-      switch (result.getValue().getOutcome()) {
-        case OK:
-          break;
-        case WARNING:
-          // fall through
-        case ERROR:
-          logger.atWarning().log(result.toString());
-          break;
-        default:
-          logger.atWarning().log(
-              "LORDN verify task %s: Unexpected outcome: %s", actionLogId, result);
-          break;
+      int responseCode = connection.getResponseCode();
+      logger.atInfo().log(
+          "LORDN verify task %s response: HTTP response code %d", actionLogId, responseCode);
+      if (responseCode == SC_NO_CONTENT) {
+        // Send a 400+ status code so App Engine will retry the task.
+        throw new ConflictException("Not ready");
      }
+      if (responseCode != SC_OK) {
+        throw new UrlConnectionException(
+            String.format(
+                "LORDN verify task %s: Failed to verify LORDN upload to MarksDB.", actionLogId),
+            connection);
+      }
+      LordnLog log =
+          LordnLog.parse(
+              ByteSource.wrap(getResponseBytes(connection)).asCharSource(UTF_8).readLines());
+      if (log.getStatus() == LordnLog.Status.ACCEPTED) {
+        logger.atInfo().log("LORDN verify task %s: Upload accepted.", actionLogId);
+      } else {
+        logger.atSevere().log(
+            "LORDN verify task %s: Upload rejected with reason: %s", actionLogId, log);
+      }
+      for (Entry<String, LordnLog.Result> result : log) {
+        switch (result.getValue().getOutcome()) {
+          case OK:
+            break;
+          case WARNING:
+            // fall through
+          case ERROR:
+            logger.atWarning().log(result.toString());
+            break;
+          default:
+            logger.atWarning().log(
+                "LORDN verify task %s: Unexpected outcome: %s", actionLogId, result);
+            break;
+        }
+      }
+      return log;
+    } catch (IOException e) {
+      throw new IOException(String.format("Error connecting to MarksDB at URL %s", url), e);
    }
-    return log;
  }
 }
--- a/core/src/main/java/google/registry/tmch/TmchCertificateAuthority.java
+++ b/core/src/main/java/google/registry/tmch/TmchCertificateAuthority.java
@@ -41,8 +41,8 @@ import javax.inject.Inject;
 * Helper methods for accessing ICANN's TMCH root certificate and revocation list.
 *
 * <p>There are two CRLs, a real one for the production environment and a pilot one for
- * non-production environments. The Datastore singleton {@link TmchCrl} entity is used to cache this
- * CRL once loaded and will always contain the proper one corresponding to the environment.
+ * non-production environments. The singleton {@link TmchCrl} entity is used to cache this CRL once
+ * loaded and will always contain the proper one corresponding to the environment.
 *
 * <p>The CRTs do not change and are included as files in the codebase that are not refreshed. They
 * were downloaded from https://ca.icann.org/tmch.crt and https://ca.icann.org/tmch_pilot.crt
@@ -66,7 +66,7 @@ public final class TmchCertificateAuthority {
  }

  /**
-   * A cached supplier that loads the CRL from Datastore or chooses a default value.
+   * A cached supplier that loads the CRL from the database or chooses a default value.
   *
   * <p>We keep the cache here rather than caching TmchCrl in the model, because loading the CRL
   * string into an X509CRL instance is expensive and should itself be cached.
@@ -132,7 +132,7 @@ public final class TmchCertificateAuthority {
  }

  /**
-   * Update to the latest TMCH X.509 certificate revocation list and save it to Datastore.
+   * Update to the latest TMCH X.509 certificate revocation list and save it to the database.
   *
   * <p>Your ASCII-armored CRL must be signed by the current ICANN root certificate.
   *
--- a/core/src/main/java/google/registry/tmch/TmchCrlAction.java
+++ b/core/src/main/java/google/registry/tmch/TmchCrlAction.java
@@ -40,7 +40,7 @@ public final class TmchCrlAction implements Runnable {
  @Inject TmchCertificateAuthority tmchCertificateAuthority;
  @Inject TmchCrlAction() {}

-  /** Synchronously fetches latest ICANN TMCH CRL and saves it to Datastore. */
+  /** Synchronously fetches latest ICANN TMCH CRL and saves it to the database. */
  @Override
  public void run() {
    try {
--- a/core/src/main/java/google/registry/tmch/TmchDnlAction.java
+++ b/core/src/main/java/google/registry/tmch/TmchDnlAction.java
@@ -23,7 +23,7 @@ import google.registry.model.tmch.ClaimsListDao;
 import google.registry.request.Action;
 import google.registry.request.auth.Auth;
 import java.io.IOException;
-import java.security.SignatureException;
+import java.security.GeneralSecurityException;
 import java.util.List;
 import java.util.Optional;
 import javax.inject.Inject;
@@ -52,7 +52,7 @@ public final class TmchDnlAction implements Runnable {
    List<String> lines;
    try {
      lines = marksdb.fetchSignedCsv(marksdbDnlLoginAndPassword, DNL_CSV_PATH, DNL_SIG_PATH);
-    } catch (SignatureException | IOException | PGPException e) {
+    } catch (GeneralSecurityException | IOException | PGPException e) {
      throw new RuntimeException(e);
    }
    ClaimsList claims = ClaimsListParser.parse(lines);
--- a/core/src/main/java/google/registry/tmch/TmchSmdrlAction.java
+++ b/core/src/main/java/google/registry/tmch/TmchSmdrlAction.java
@@ -22,7 +22,7 @@ import google.registry.model.smd.SignedMarkRevocationList;
 import google.registry.request.Action;
 import google.registry.request.auth.Auth;
 import java.io.IOException;
-import java.security.SignatureException;
+import java.security.GeneralSecurityException;
 import java.util.List;
 import java.util.Optional;
 import javax.inject.Inject;
@@ -51,7 +51,7 @@ public final class TmchSmdrlAction implements Runnable {
    List<String> lines;
    try {
      lines = marksdb.fetchSignedCsv(marksdbSmdrlLoginAndPassword, SMDRL_CSV_PATH, SMDRL_SIG_PATH);
-    } catch (SignatureException | IOException | PGPException e) {
+    } catch (GeneralSecurityException | IOException | PGPException e) {
      throw new RuntimeException(e);
    }
    SignedMarkRevocationList smdrl = SmdrlCsvParser.parse(lines);
--- a/core/src/main/java/google/registry/tools/DigestType.java
+++ b/core/src/main/java/google/registry/tools/DigestType.java
@@ -14,6 +14,9 @@

 package google.registry.tools;

+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Maps;
+import java.util.Arrays;
 import java.util.Optional;

 /**
@@ -26,33 +29,38 @@ import java.util.Optional;
 * https://www.iana.org/assignments/ds-rr-types/ds-rr-types.xhtml
 */
 public enum DigestType {
-  SHA1(1),
-  SHA256(2),
+  SHA1(1, 20),
+  SHA256(2, 32),
  // Algorithm number 3 is GOST R 34.11-94 and is deliberately NOT SUPPORTED.
  // This algorithm was reviewed by ise-crypto and deemed academically broken (b/207029800).
  // In addition, RFC 8624 specifies that this algorithm MUST NOT be used for DNSSEC delegations.
  // TODO(sarhabot@): Add note in Cloud DNS code to notify the Registry of any new changes to
  // supported digest types.
-  SHA384(4);
+  SHA384(4, 48);

  private final int wireValue;
+  private final int bytes;

-  DigestType(int wireValue) {
+  DigestType(int wireValue, int bytes) {
    this.wireValue = wireValue;
+    this.bytes = bytes;
  }

+  private static final ImmutableMap<Integer, DigestType> WIRE_VALUE_TO_DIGEST_TYPE =
+      Maps.uniqueIndex(Arrays.stream(DigestType.values()).iterator(), DigestType::getWireValue);
+
  /** Fetches a DigestType enumeration constant by its IANA assigned value. */
  public static Optional<DigestType> fromWireValue(int wireValue) {
-    for (DigestType alg : DigestType.values()) {
-      if (alg.getWireValue() == wireValue) {
-        return Optional.of(alg);
-      }
-    }
-    return Optional.empty();
+    return Optional.ofNullable(WIRE_VALUE_TO_DIGEST_TYPE.get(wireValue));
  }

  /** Fetches a value in the range [0, 255] that encodes this DS digest type on the wire. */
  public int getWireValue() {
    return wireValue;
  }
+
+  /** Returns the expected length in bytes of the signature. */
+  public int getBytes() {
+    return bytes;
+  }
 }
--- a/core/src/main/java/google/registry/tools/DsRecord.java
+++ b/core/src/main/java/google/registry/tools/DsRecord.java
@@ -51,6 +51,11 @@ abstract class DsRecord {
    checkArgumentPresent(
        DigestType.fromWireValue(digestType),
        String.format("DS record uses an unrecognized digest type: %d", digestType));
+    if (DigestType.fromWireValue(digestType).get().getBytes()
+        != BaseEncoding.base16().decode(digest).length) {
+      throw new IllegalArgumentException(
+          String.format("DS record has an invalid digest length: %s", digest));
+    }

    if (!DomainFlowUtils.validateAlgorithm(alg)) {
      throw new IllegalArgumentException(
--- a/core/src/main/java/google/registry/tools/GenerateEscrowDepositCommand.java
+++ b/core/src/main/java/google/registry/tools/GenerateEscrowDepositCommand.java
@@ -33,7 +33,6 @@ import google.registry.model.rde.RdeMode;
 import google.registry.rde.RdeStagingAction;
 import google.registry.request.Action.Service;
 import google.registry.tools.params.DateTimeParameter;
-import google.registry.util.AppEngineServiceUtils;
 import google.registry.util.CloudTasksUtils;
 import java.util.List;
 import java.util.stream.Collectors;
@@ -90,8 +89,6 @@ final class GenerateEscrowDepositCommand implements CommandWithRemoteApi {
      required = true)
  private String outdir;

-  @Inject AppEngineServiceUtils appEngineServiceUtils;
-
  @Inject CloudTasksUtils cloudTasksUtils;

  @Override
--- a/core/src/main/java/google/registry/tools/ListTxnsCommand.java
+++ b/core/src/main/java/google/registry/tools/ListTxnsCommand.java
@@ -0,0 +1,116 @@
+// Copyright 2022 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.tools;
+
+import static google.registry.persistence.transaction.TransactionManagerFactory.replicaJpaTm;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import com.beust.jcommander.Parameter;
+import com.beust.jcommander.Parameters;
+import com.google.common.base.Splitter;
+import com.google.common.io.BaseEncoding;
+import google.registry.model.common.Cursor;
+import google.registry.persistence.transaction.Transaction;
+import google.registry.persistence.transaction.TransactionEntity;
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.util.List;
+
+/** Lists {@link Cursor} timestamps used by locking rolling cursor tasks, like in RDE. */
+@Parameters(separators = " =", commandDescription = "Lists the contents of the Transaction table.")
+final class ListTxnsCommand implements CommandWithRemoteApi {
+
+  @Parameter(names = "--infile", description = "Parse an input file instead of reading from db.")
+  private String infile;
+
+  @Parameter(
+      names = "--full_dump",
+      description =
+          "Do a full dump of the contents of the transaction.  Without this, "
+              + "just write transactions as CSV lines suitable for ingestion via --infile.")
+  private boolean fullDump = false;
+
+  @Override
+  public void run() {
+    if (infile == null) {
+      fetchFromDb();
+    } else {
+      parseCsvFile();
+    }
+  }
+
+  private void parseCsvFile() {
+    try {
+      BufferedReader src = Files.newBufferedReader(Paths.get(infile), UTF_8);
+      String line;
+      while ((line = src.readLine()) != null) {
+        List<String> cols = Splitter.on(",").splitToList(line);
+        writeRecord(Integer.parseInt(cols.get(0)), BaseEncoding.base64().decode(cols.get(1)));
+      }
+    } catch (IOException e) {
+      throw new RuntimeException(e);
+    }
+  }
+
+  private void fetchFromDb() {
+    long lastTransactionId = 0;
+    List<TransactionEntity> results;
+
+    do {
+      final long txnId = lastTransactionId; // For use in the lambda.
+      results =
+          replicaJpaTm()
+              .transact(
+                  () ->
+                      replicaJpaTm()
+                          .query(
+                              "select t from TransactionEntity t where id > :lastTransactionId",
+                              TransactionEntity.class)
+                          .setParameter("lastTransactionId", txnId)
+                          .setMaxResults(1000)
+                          .getResultList());
+
+      for (TransactionEntity txn : results) {
+        writeRecord(txn.getId(), txn.getContents());
+        lastTransactionId = txn.getId();
+      }
+    } while (results.size() > 0);
+  }
+
+  private void writeRecord(long id, byte[] contents) {
+    if (fullDump) {
+      Transaction txn;
+      try {
+        txn = Transaction.deserialize(contents);
+      } catch (IOException ex) {
+        System.err.printf("Error deserializing transaction %s\n", id);
+        return;
+      }
+      System.out.printf("transaction %s <<<\n", id);
+      for (Transaction.Mutation mut : txn.getMutations()) {
+        if (mut instanceof Transaction.Update) {
+          System.out.println("updating: " + ((Transaction.Update) mut).getEntity());
+        } else {
+          System.out.println("deleting: " + ((Transaction.Delete) mut).getKey());
+        }
+      }
+      System.out.println(">>>");
+    } else {
+      System.out.printf("%s,%s\n", id, BaseEncoding.base64().encode(contents));
+    }
+  }
+}
--- a/core/src/main/java/google/registry/tools/RegistryCli.java
+++ b/core/src/main/java/google/registry/tools/RegistryCli.java
@@ -250,6 +250,8 @@ final class RegistryCli implements AutoCloseable, CommandRunner {
      // Cloud SQL after the database migration. Note that the DB password is stored in Datastore
      // and it is already initialized above.
      TransactionManagerFactory.setJpaTm(() -> component.nomulusToolJpaTransactionManager().get());
+      TransactionManagerFactory.setReplicaJpaTm(
+          () -> component.nomulusToolReplicaJpaTransactionManager().get());
    }

    command.run();
--- a/core/src/main/java/google/registry/tools/RegistryTool.java
+++ b/core/src/main/java/google/registry/tools/RegistryTool.java
@@ -92,6 +92,7 @@ public final class RegistryTool {
          .put("list_registrars", ListRegistrarsCommand.class)
          .put("list_reserved_lists", ListReservedListsCommand.class)
          .put("list_tlds", ListTldsCommand.class)
+          .put("list_txns", ListTxnsCommand.class)
          .put("load_snapshot", LoadSnapshotCommand.class)
          .put("load_test", LoadTestCommand.class)
          .put("lock_domain", LockDomainCommand.class)
--- a/core/src/main/java/google/registry/tools/RegistryToolComponent.java
+++ b/core/src/main/java/google/registry/tools/RegistryToolComponent.java
@@ -33,13 +33,14 @@ import google.registry.keyring.api.KeyModule;
 import google.registry.keyring.kms.KmsModule;
 import google.registry.persistence.PersistenceModule;
 import google.registry.persistence.PersistenceModule.NomulusToolJpaTm;
+import google.registry.persistence.PersistenceModule.ReadOnlyReplicaJpaTm;
 import google.registry.persistence.transaction.JpaTransactionManager;
 import google.registry.privileges.secretmanager.SecretManagerModule;
 import google.registry.rde.RdeModule;
 import google.registry.request.Modules.DatastoreServiceModule;
 import google.registry.request.Modules.Jackson2Module;
-import google.registry.request.Modules.URLFetchServiceModule;
-import google.registry.request.Modules.UrlFetchTransportModule;
+import google.registry.request.Modules.UrlConnectionServiceModule;
+import google.registry.request.Modules.UrlFetchServiceModule;
 import google.registry.request.Modules.UserServiceModule;
 import google.registry.tools.AuthModule.LocalCredentialModule;
 import google.registry.tools.javascrap.CompareEscrowDepositsCommand;
@@ -79,8 +80,8 @@ import javax.inject.Singleton;
      RegistryToolDataflowModule.class,
      RequestFactoryModule.class,
      SecretManagerModule.class,
-      URLFetchServiceModule.class,
-      UrlFetchTransportModule.class,
+      UrlConnectionServiceModule.class,
+      UrlFetchServiceModule.class,
      UserServiceModule.class,
      UtilsModule.class,
      VoidDnsWriterModule.class,
@@ -189,6 +190,9 @@ interface RegistryToolComponent {
  @NomulusToolJpaTm
  Lazy<JpaTransactionManager> nomulusToolJpaTransactionManager();

+  @ReadOnlyReplicaJpaTm
+  Lazy<JpaTransactionManager> nomulusToolReplicaJpaTransactionManager();
+
  @Component.Builder
  interface Builder {
    @BindsInstance
--- a/core/src/main/java/google/registry/tools/SetNumInstancesCommand.java
+++ b/core/src/main/java/google/registry/tools/SetNumInstancesCommand.java
@@ -15,18 +15,14 @@
 package google.registry.tools;

 import static com.google.common.base.Preconditions.checkArgument;
-import static com.google.common.collect.ImmutableSet.toImmutableSet;
 import static com.google.common.collect.ImmutableSetMultimap.flatteningToImmutableSetMultimap;
 import static google.registry.util.CollectionUtils.nullToEmpty;
-import static google.registry.util.PreconditionsUtils.checkArgumentPresent;

 import com.beust.jcommander.Parameter;
 import com.beust.jcommander.Parameters;
 import com.google.api.services.appengine.v1.Appengine;
-import com.google.common.base.Enums;
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
-import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.ImmutableSetMultimap;
 import com.google.common.collect.Maps;
 import com.google.common.flogger.FluentLogger;
@@ -35,8 +31,6 @@ import google.registry.request.Action.Service;
 import google.registry.util.AppEngineServiceUtils;
 import java.io.IOException;
 import java.util.List;
-import java.util.Locale;
-import java.util.Set;
 import javax.inject.Inject;

 /** A command to set the number of instances for an App Engine service. */
@@ -49,19 +43,18 @@ final class SetNumInstancesCommand implements CommandWithRemoteApi {

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

-  private static final ImmutableSet<Service> ALL_DEPLOYED_SERVICES =
-      ImmutableSet.copyOf(Service.values());
+  private static final ImmutableList<Service> ALL_DEPLOYED_SERVICES =
+      ImmutableList.copyOf(Service.values());

  private static final ImmutableMap<String, Service> SERVICE_ID_TO_SERVICE =
      Maps.uniqueIndex(ALL_DEPLOYED_SERVICES, Service::getServiceId);

-  // TODO(b/119629679): Use List<Service> after upgrading jcommander to latest version.
  @Parameter(
      names = {"-s", "--services"},
      description =
          "Comma-delimited list of App Engine services to set. "
              + "Allowed values: [DEFAULT, TOOLS, BACKEND, PUBAPI]")
-  private List<String> serviceNames = ImmutableList.of();
+  private List<Service> services = ImmutableList.of();

  @Parameter(
      names = {"-v", "--versions"},
@@ -93,19 +86,6 @@ final class SetNumInstancesCommand implements CommandWithRemoteApi {

  @Override
  public void run() {
-
-    ImmutableSet<Service> services =
-        serviceNames.stream()
-            .map(s -> s.toUpperCase(Locale.US))
-            .map(
-                name ->
-                    checkArgumentPresent(
-                        Enums.getIfPresent(Service.class, name).toJavaUtil(),
-                        "Invalid service '%s'. Allowed values are %s",
-                        name,
-                        ALL_DEPLOYED_SERVICES))
-            .collect(toImmutableSet());
-
    if (nonLiveVersions) {
      checkArgument(versions.isEmpty(), "--versions cannot be set if --non_live_versions is set");

@@ -149,7 +129,7 @@ final class SetNumInstancesCommand implements CommandWithRemoteApi {
        version, service, numInstances);
  }

-  private ImmutableSetMultimap<Service, String> getAllLiveVersionsMap(Set<Service> services) {
+  private ImmutableSetMultimap<Service, String> getAllLiveVersionsMap(List<Service> services) {
    try {
      return nullToEmpty(appengine.apps().services().list(projectId).execute().getServices())
          .stream()
@@ -165,7 +145,8 @@ final class SetNumInstancesCommand implements CommandWithRemoteApi {
    }
  }

-  private ImmutableSetMultimap<Service, String> getManualScalingVersionsMap(Set<Service> services) {
+  private ImmutableSetMultimap<Service, String> getManualScalingVersionsMap(
+      List<Service> services) {
    return services.stream()
        .collect(
            flatteningToImmutableSetMultimap(
--- a/core/src/main/java/google/registry/tools/ValidateLoginCredentialsCommand.java
+++ b/core/src/main/java/google/registry/tools/ValidateLoginCredentialsCommand.java
@@ -56,7 +56,6 @@ final class ValidateLoginCredentialsCommand implements CommandWithRemoteApi {
      validateWith = PathParameter.InputFile.class)
  private Path clientCertificatePath;

-  // TODO(sarahbot@): Remove this after hash fallback is removed
  @Nullable
  @Parameter(
      names = {"-h", "--cert_hash"},
--- a/core/src/main/java/google/registry/ui/server/registrar/RegistrarSettingsAction.java
+++ b/core/src/main/java/google/registry/ui/server/registrar/RegistrarSettingsAction.java
@@ -59,7 +59,6 @@ import google.registry.ui.forms.FormException;
 import google.registry.ui.forms.FormFieldException;
 import google.registry.ui.server.RegistrarFormFields;
 import google.registry.ui.server.SendEmailUtils;
-import google.registry.util.AppEngineServiceUtils;
 import google.registry.util.CloudTasksUtils;
 import google.registry.util.CollectionUtils;
 import google.registry.util.DiffUtils;
@@ -108,7 +107,6 @@ public class RegistrarSettingsAction implements Runnable, JsonActionRunner.JsonA
  private static ThreadLocal<Boolean> isInTestDriver = ThreadLocal.withInitial(() -> false);

  @Inject JsonActionRunner jsonActionRunner;
-  @Inject AppEngineServiceUtils appEngineServiceUtils;
  @Inject RegistrarConsoleMetrics registrarConsoleMetrics;
  @Inject SendEmailUtils sendEmailUtils;
  @Inject AuthenticatedRegistrarAccessor registrarAccessor;
--- a/core/src/main/java/google/registry/whois/NameserverLookupByIpCommand.java
+++ b/core/src/main/java/google/registry/whois/NameserverLookupByIpCommand.java
@@ -64,11 +64,17 @@ final class NameserverLookupByIpCommand implements WhoisCommand {
          jpaTm()
              .transact(
                  () ->
-                      // We cannot query @Convert-ed fields in HQL so we must use native Postgres
+                      // We cannot query @Convert-ed fields in HQL so we must use native Postgres.
                      jpaTm()
                          .getEntityManager()
+                          /**
+                           * Using array_operator <@ (contained-by) with gin index on inet_address.
+                           * Without gin index, this is slightly slower than the alternative form of
+                           * ':address = ANY(inet_address)'.
+                           */
                          .createNativeQuery(
-                              "SELECT * From \"Host\" WHERE :address = ANY(inet_addresses) AND "
+                              "SELECT * From \"Host\" WHERE "
+                                  + "ARRAY[ CAST(:address AS TEXT) ] <@ inet_addresses AND "
                                  + "deletion_time > CAST(:now AS timestamptz)",
                              HostResource.class)
                          .setParameter("address", InetAddresses.toAddrString(ipAddress))
--- a/core/src/main/resources/google/registry/beam/rde_pipeline_metadata.json
+++ b/core/src/main/resources/google/registry/beam/rde_pipeline_metadata.json
@@ -49,22 +49,6 @@
      "regexes": [
        "[A-Za-z0-9\\-_]+"
      ]
-    },
-    {
-      "name": "workerMachineType",
-      "label": "The GCE machine type for the dataflow job workers.",
-      "helpText": "See https://cloud.google.com/dataflow/quotas#compute-engine-quotas for available machine types.",
-      "regexes": [
-        "[a-z0-9\\-]+"
-      ]
-    },
-    {
-      "name": "usePublicIps",
-      "label": "Whether the GCE workers are assigned public IPs",
-      "helpText": "Public IPs have an associated cost and there's a quota per region on the total number of public IPs assigned at a given time. If the service only needs to access GCP APIs, it's better to not use public IP, but one needs to configure the network accordingly. See https://cloud.google.com/dataflow/docs/guides/routes-firewall.",
-      "regexes": [
-        "true|false"
-      ]
    }
  ]
 }
--- a/core/src/test/java/google/registry/backup/CommitLogCheckpointActionTest.java
+++ b/core/src/test/java/google/registry/backup/CommitLogCheckpointActionTest.java
@@ -47,11 +47,10 @@ public class CommitLogCheckpointActionTest {

  private DateTime now = DateTime.now(UTC);
  private CommitLogCheckpointAction task = new CommitLogCheckpointAction();
-  private final CloudTasksHelper cloudTasksHelper = new CloudTasksHelper();
+  private final CloudTasksHelper cloudTasksHelper = new CloudTasksHelper(new FakeClock(now));

  @BeforeEach
  void beforeEach() {
-    task.clock = new FakeClock(now);
    task.strategy = strategy;
    task.cloudTasksUtils = cloudTasksHelper.getTestCloudTasksUtils();
    when(strategy.computeCheckpoint())
@@ -68,7 +67,8 @@ public class CommitLogCheckpointActionTest {
        new TaskMatcher()
            .url(ExportCommitLogDiffAction.PATH)
            .param(ExportCommitLogDiffAction.LOWER_CHECKPOINT_TIME_PARAM, START_OF_TIME.toString())
-            .param(ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM, now.toString()));
+            .param(ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM, now.toString())
+            .scheduleTime(now.plus(CommitLogCheckpointAction.ENQUEUE_DELAY_SECONDS)));
    assertThat(loadRoot().getLastWrittenTime()).isEqualTo(now);
  }

@@ -82,7 +82,8 @@ public class CommitLogCheckpointActionTest {
        new TaskMatcher()
            .url(ExportCommitLogDiffAction.PATH)
            .param(ExportCommitLogDiffAction.LOWER_CHECKPOINT_TIME_PARAM, oneMinuteAgo.toString())
-            .param(ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM, now.toString()));
+            .param(ExportCommitLogDiffAction.UPPER_CHECKPOINT_TIME_PARAM, now.toString())
+            .scheduleTime(now.plus(CommitLogCheckpointAction.ENQUEUE_DELAY_SECONDS)));
    assertThat(loadRoot().getLastWrittenTime()).isEqualTo(now);
  }

--- a/core/src/test/java/google/registry/batch/AsyncTaskEnqueuerTest.java
+++ b/core/src/test/java/google/registry/batch/AsyncTaskEnqueuerTest.java
@@ -22,23 +22,20 @@ import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_ACTIONS;
 import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_DELETE;
 import static google.registry.batch.AsyncTaskEnqueuer.QUEUE_ASYNC_HOST_RENAME;
 import static google.registry.testing.DatabaseHelper.persistActiveContact;
-import static google.registry.testing.TaskQueueHelper.assertNoTasksEnqueued;
-import static google.registry.testing.TaskQueueHelper.assertTasksEnqueued;
 import static google.registry.testing.TestLogHandlerUtils.assertLogMessage;
-import static org.joda.time.Duration.standardDays;
-import static org.joda.time.Duration.standardHours;
 import static org.joda.time.Duration.standardSeconds;
-import static org.mockito.Mockito.when;

+import com.google.cloud.tasks.v2.HttpMethod;
 import com.google.common.collect.ImmutableSortedSet;
 import google.registry.model.contact.ContactResource;
 import google.registry.testing.AppEngineExtension;
+import google.registry.testing.CloudTasksHelper;
+import google.registry.testing.CloudTasksHelper.TaskMatcher;
 import google.registry.testing.FakeClock;
 import google.registry.testing.FakeSleeper;
 import google.registry.testing.InjectExtension;
-import google.registry.testing.TaskQueueHelper.TaskMatcher;
-import google.registry.util.AppEngineServiceUtils;
 import google.registry.util.CapturingLogHandler;
+import google.registry.util.CloudTasksUtils;
 import google.registry.util.JdkLoggerConfig;
 import google.registry.util.Retrier;
 import java.util.logging.Level;
@@ -48,7 +45,6 @@ import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.ExtendWith;
 import org.junit.jupiter.api.extension.RegisterExtension;
-import org.mockito.Mock;
 import org.mockito.junit.jupiter.MockitoExtension;
 import org.mockito.junit.jupiter.MockitoSettings;
 import org.mockito.quality.Strictness;
@@ -63,27 +59,25 @@ public class AsyncTaskEnqueuerTest {

  @RegisterExtension public final InjectExtension inject = new InjectExtension();

-  @Mock private AppEngineServiceUtils appEngineServiceUtils;
-
  private AsyncTaskEnqueuer asyncTaskEnqueuer;
  private final CapturingLogHandler logHandler = new CapturingLogHandler();
  private final FakeClock clock = new FakeClock(DateTime.parse("2015-05-18T12:34:56Z"));
+  private CloudTasksHelper cloudTasksHelper = new CloudTasksHelper(clock);

  @BeforeEach
  void beforeEach() {
    JdkLoggerConfig.getConfig(AsyncTaskEnqueuer.class).addHandler(logHandler);
-    when(appEngineServiceUtils.getServiceHostname("backend")).thenReturn("backend.hostname.fake");
-    asyncTaskEnqueuer = createForTesting(appEngineServiceUtils, clock, standardSeconds(90));
+    asyncTaskEnqueuer =
+        createForTesting(cloudTasksHelper.getTestCloudTasksUtils(), clock, standardSeconds(90));
  }

  public static AsyncTaskEnqueuer createForTesting(
-      AppEngineServiceUtils appEngineServiceUtils, FakeClock clock, Duration asyncDeleteDelay) {
+      CloudTasksUtils cloudTasksUtils, FakeClock clock, Duration asyncDeleteDelay) {
    return new AsyncTaskEnqueuer(
-        getQueue(QUEUE_ASYNC_ACTIONS),
        getQueue(QUEUE_ASYNC_DELETE),
        getQueue(QUEUE_ASYNC_HOST_RENAME),
        asyncDeleteDelay,
-        appEngineServiceUtils,
+        cloudTasksUtils,
        new Retrier(new FakeSleeper(clock), 1));
  }

@@ -92,18 +86,16 @@ public class AsyncTaskEnqueuerTest {
    ContactResource contact = persistActiveContact("jd23456");
    asyncTaskEnqueuer.enqueueAsyncResave(
        contact.createVKey(), clock.nowUtc(), clock.nowUtc().plusDays(5));
-    assertTasksEnqueued(
+    cloudTasksHelper.assertTasksEnqueued(
        QUEUE_ASYNC_ACTIONS,
-        new TaskMatcher()
+        new CloudTasksHelper.TaskMatcher()
            .url(ResaveEntityAction.PATH)
-            .method("POST")
-            .header("Host", "backend.hostname.fake")
+            .method(HttpMethod.POST)
+            .service("backend")
            .header("content-type", "application/x-www-form-urlencoded")
            .param(PARAM_RESOURCE_KEY, contact.createVKey().stringify())
            .param(PARAM_REQUESTED_TIME, clock.nowUtc().toString())
-            .etaDelta(
-                standardDays(5).minus(standardSeconds(30)),
-                standardDays(5).plus(standardSeconds(30))));
+            .scheduleTime(clock.nowUtc().plus(Duration.standardDays(5))));
  }

  @Test
@@ -114,19 +106,17 @@ public class AsyncTaskEnqueuerTest {
        contact.createVKey(),
        now,
        ImmutableSortedSet.of(now.plusHours(24), now.plusHours(50), now.plusHours(75)));
-    assertTasksEnqueued(
+    cloudTasksHelper.assertTasksEnqueued(
        QUEUE_ASYNC_ACTIONS,
        new TaskMatcher()
            .url(ResaveEntityAction.PATH)
-            .method("POST")
-            .header("Host", "backend.hostname.fake")
+            .method(HttpMethod.POST)
+            .service("backend")
            .header("content-type", "application/x-www-form-urlencoded")
            .param(PARAM_RESOURCE_KEY, contact.createVKey().stringify())
            .param(PARAM_REQUESTED_TIME, now.toString())
            .param(PARAM_RESAVE_TIMES, "2015-05-20T14:34:56.000Z,2015-05-21T15:34:56.000Z")
-            .etaDelta(
-                standardHours(24).minus(standardSeconds(30)),
-                standardHours(24).plus(standardSeconds(30))));
+            .scheduleTime(clock.nowUtc().plus(Duration.standardHours(24))));
  }

  @MockitoSettings(strictness = Strictness.LENIENT)
@@ -135,7 +125,7 @@ public class AsyncTaskEnqueuerTest {
    ContactResource contact = persistActiveContact("jd23456");
    asyncTaskEnqueuer.enqueueAsyncResave(
        contact.createVKey(), clock.nowUtc(), clock.nowUtc().plusDays(31));
-    assertNoTasksEnqueued(QUEUE_ASYNC_ACTIONS);
+    cloudTasksHelper.assertNoTasksEnqueued(QUEUE_ASYNC_ACTIONS);
    assertLogMessage(logHandler, Level.INFO, "Ignoring async re-save");
  }
 }
--- a/core/src/test/java/google/registry/batch/DeleteContactsAndHostsActionTest.java
+++ b/core/src/test/java/google/registry/batch/DeleteContactsAndHostsActionTest.java
@@ -91,13 +91,13 @@ import google.registry.model.transfer.ContactTransferData;
 import google.registry.model.transfer.TransferData;
 import google.registry.model.transfer.TransferResponse;
 import google.registry.model.transfer.TransferStatus;
+import google.registry.testing.CloudTasksHelper;
 import google.registry.testing.FakeClock;
 import google.registry.testing.FakeResponse;
 import google.registry.testing.FakeSleeper;
 import google.registry.testing.InjectExtension;
 import google.registry.testing.TaskQueueHelper.TaskMatcher;
 import google.registry.testing.mapreduce.MapreduceTestCase;
-import google.registry.util.AppEngineServiceUtils;
 import google.registry.util.RequestStatusChecker;
 import google.registry.util.Retrier;
 import google.registry.util.Sleeper;
@@ -148,7 +148,7 @@ public class DeleteContactsAndHostsActionTest
    inject.setStaticField(Ofy.class, "clock", clock);
    enqueuer =
        AsyncTaskEnqueuerTest.createForTesting(
-            mock(AppEngineServiceUtils.class), clock, Duration.ZERO);
+            new CloudTasksHelper(clock).getTestCloudTasksUtils(), clock, Duration.ZERO);
    AsyncTaskMetrics asyncTaskMetricsMock = mock(AsyncTaskMetrics.class);
    action = new DeleteContactsAndHostsAction();
    action.asyncTaskMetrics = asyncTaskMetricsMock;
--- a/core/src/test/java/google/registry/batch/DeleteExpiredDomainsActionTest.java
+++ b/core/src/test/java/google/registry/batch/DeleteExpiredDomainsActionTest.java
@@ -39,7 +39,6 @@ import google.registry.model.domain.DomainHistory;
 import google.registry.model.ofy.Ofy;
 import google.registry.model.poll.PollMessage;
 import google.registry.model.reporting.HistoryEntry;
-import google.registry.monitoring.whitebox.EppMetric;
 import google.registry.persistence.transaction.QueryComposer.Comparator;
 import google.registry.testing.AppEngineExtension;
 import google.registry.testing.DualDatabaseTest;
@@ -78,8 +77,7 @@ class DeleteExpiredDomainsActionTest {
    createTld("tld");
    EppController eppController =
        DaggerEppTestComponent.builder()
-            .fakesAndMocksModule(
-                FakesAndMocksModule.create(clock, EppMetric.builderForRequest(clock)))
+            .fakesAndMocksModule(FakesAndMocksModule.create(clock))
            .build()
            .startRequest()
            .eppController();
--- a/core/src/test/java/google/registry/batch/RefreshDnsOnHostRenameActionTest.java
+++ b/core/src/test/java/google/registry/batch/RefreshDnsOnHostRenameActionTest.java
@@ -49,6 +49,7 @@ import google.registry.batch.RefreshDnsOnHostRenameAction.RefreshDnsOnHostRename
 import google.registry.dns.DnsQueue;
 import google.registry.model.host.HostResource;
 import google.registry.model.server.Lock;
+import google.registry.testing.CloudTasksHelper;
 import google.registry.testing.DualDatabaseTest;
 import google.registry.testing.FakeClock;
 import google.registry.testing.FakeResponse;
@@ -59,7 +60,6 @@ import google.registry.testing.TestOfyAndSql;
 import google.registry.testing.TestOfyOnly;
 import google.registry.testing.TestSqlOnly;
 import google.registry.testing.mapreduce.MapreduceTestCase;
-import google.registry.util.AppEngineServiceUtils;
 import google.registry.util.RequestStatusChecker;
 import google.registry.util.Retrier;
 import google.registry.util.Sleeper;
@@ -89,7 +89,7 @@ public class RefreshDnsOnHostRenameActionTest
    createTld("tld");
    enqueuer =
        AsyncTaskEnqueuerTest.createForTesting(
-            mock(AppEngineServiceUtils.class), clock, Duration.ZERO);
+            new CloudTasksHelper(clock).getTestCloudTasksUtils(), clock, Duration.ZERO);
    AsyncTaskMetrics asyncTaskMetricsMock = mock(AsyncTaskMetrics.class);
    action = new RefreshDnsOnHostRenameAction();
    action.asyncTaskMetrics = asyncTaskMetricsMock;
--- a/core/src/test/java/google/registry/batch/RelockDomainActionTest.java
+++ b/core/src/test/java/google/registry/batch/RelockDomainActionTest.java
@@ -31,8 +31,6 @@ import static google.registry.testing.SqlHelper.saveRegistryLock;
 import static google.registry.tools.LockOrUnlockDomainCommand.REGISTRY_LOCK_STATUSES;
 import static javax.servlet.http.HttpServletResponse.SC_NO_CONTENT;
 import static javax.servlet.http.HttpServletResponse.SC_OK;
-import static org.mockito.Mockito.lenient;
-import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.verify;
 import static org.mockito.Mockito.verifyNoMoreInteractions;

@@ -51,7 +49,6 @@ import google.registry.testing.FakeResponse;
 import google.registry.testing.TestOfyAndSql;
 import google.registry.testing.UserInfo;
 import google.registry.tools.DomainLockUtils;
-import google.registry.util.AppEngineServiceUtils;
 import google.registry.util.EmailMessage;
 import google.registry.util.SendEmailService;
 import google.registry.util.StringGenerator.Alphabets;
@@ -111,11 +108,6 @@ public class RelockDomainActionTest {
            DOMAIN_NAME, CLIENT_ID, false, Optional.empty());
    assertThat(loadByEntity(domain).getStatusValues()).containsNoneIn(REGISTRY_LOCK_STATUSES);

-    AppEngineServiceUtils appEngineServiceUtils = mock(AppEngineServiceUtils.class);
-    lenient()
-        .when(appEngineServiceUtils.getServiceHostname("backend"))
-        .thenReturn("backend.hostname.fake");
-
    action = createAction(oldLock.getRevisionId());
  }

--- a/core/src/test/java/google/registry/batch/ResaveEntityActionTest.java
+++ b/core/src/test/java/google/registry/batch/ResaveEntityActionTest.java
@@ -25,12 +25,10 @@ import static google.registry.testing.DatabaseHelper.persistActiveContact;
 import static google.registry.testing.DatabaseHelper.persistDomainWithDependentResources;
 import static google.registry.testing.DatabaseHelper.persistDomainWithPendingTransfer;
 import static google.registry.testing.DatabaseHelper.persistResource;
-import static google.registry.testing.TaskQueueHelper.assertTasksEnqueued;
 import static org.joda.time.Duration.standardDays;
-import static org.joda.time.Duration.standardSeconds;
 import static org.mockito.Mockito.verify;
-import static org.mockito.Mockito.when;

+import com.google.cloud.tasks.v2.HttpMethod;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.ImmutableSortedSet;
 import google.registry.model.domain.DomainBase;
@@ -40,12 +38,12 @@ import google.registry.model.eppcommon.StatusValue;
 import google.registry.model.ofy.Ofy;
 import google.registry.request.Response;
 import google.registry.testing.AppEngineExtension;
+import google.registry.testing.CloudTasksHelper;
+import google.registry.testing.CloudTasksHelper.TaskMatcher;
 import google.registry.testing.DualDatabaseTest;
 import google.registry.testing.FakeClock;
 import google.registry.testing.InjectExtension;
-import google.registry.testing.TaskQueueHelper.TaskMatcher;
 import google.registry.testing.TestOfyAndSql;
-import google.registry.util.AppEngineServiceUtils;
 import org.joda.time.DateTime;
 import org.joda.time.Duration;
 import org.junit.jupiter.api.BeforeEach;
@@ -67,17 +65,17 @@ public class ResaveEntityActionTest {

  @RegisterExtension public final InjectExtension inject = new InjectExtension();

-  @Mock private AppEngineServiceUtils appEngineServiceUtils;
  @Mock private Response response;
  private final FakeClock clock = new FakeClock(DateTime.parse("2016-02-11T10:00:00Z"));
  private AsyncTaskEnqueuer asyncTaskEnqueuer;
+  private CloudTasksHelper cloudTasksHelper = new CloudTasksHelper(clock);

  @BeforeEach
  void beforeEach() {
    inject.setStaticField(Ofy.class, "clock", clock);
-    when(appEngineServiceUtils.getServiceHostname("backend")).thenReturn("backend.hostname.fake");
    asyncTaskEnqueuer =
-        AsyncTaskEnqueuerTest.createForTesting(appEngineServiceUtils, clock, Duration.ZERO);
+        AsyncTaskEnqueuerTest.createForTesting(
+            cloudTasksHelper.getTestCloudTasksUtils(), clock, Duration.ZERO);
    createTld("tld");
  }

@@ -143,17 +141,15 @@ public class ResaveEntityActionTest {
    DomainBase resavedDomain = loadByEntity(domain);
    assertThat(resavedDomain.getGracePeriods()).isEmpty();

-    assertTasksEnqueued(
+    cloudTasksHelper.assertTasksEnqueued(
        QUEUE_ASYNC_ACTIONS,
        new TaskMatcher()
            .url(ResaveEntityAction.PATH)
-            .method("POST")
-            .header("Host", "backend.hostname.fake")
+            .method(HttpMethod.POST)
+            .service("backend")
            .header("content-type", "application/x-www-form-urlencoded")
            .param(PARAM_RESOURCE_KEY, resavedDomain.createVKey().stringify())
            .param(PARAM_REQUESTED_TIME, requestedTime.toString())
-            .etaDelta(
-                standardDays(5).minus(standardSeconds(30)),
-                standardDays(5).plus(standardSeconds(30))));
+            .scheduleTime(clock.nowUtc().plus(standardDays(5))));
  }
 }
--- a/core/src/test/java/google/registry/batch/SendExpiringCertificateNotificationEmailActionTest.java
+++ b/core/src/test/java/google/registry/batch/SendExpiringCertificateNotificationEmailActionTest.java
@@ -59,49 +59,18 @@ class SendExpiringCertificateNotificationEmailActionTest {
  private static final String EXPIRATION_WARNING_EMAIL_BODY_TEXT =
      " Dear %1$s,\n"
          + "\n"
-          + "    We would like to inform you that your %2$s SSL certificate will expire at\n"
-          + "    %3$s. Please take note that using expired certificates will prevent\n"
-          + "    successful Registry login.\n"
+          + "We would like to inform you that your %2$s certificate will expire at %3$s."
          + "\n"
-          + "    Kindly update your production account certificate within the support\n"
-          + "    console using the following steps:\n"
+          + " Kind update your account using the following steps: "
          + "\n"
-          + "      1. Navigate to support.registry.google and login using your\n"
-          + "    %4$s@registry.google credentials.\n"
-          + "          * If this is your first time logging in, you will be prompted to\n"
-          + "          reset your password, so please keep your new password safe.\n"
-          + "          * If you are already logged in with some other Google account(s) but\n"
-          + "          not your %4$s@registry.google account, you need to click on\n"
-          + "          “Add Account” and login using your %4$s@registry.google credentials.\n"
-          + "      2. Select “Settings > Security” from the left navigation bar.\n"
-          + "      3. Click “Edit” on the top left corner.\n"
-          + "      4. Enter your full certificate string\n"
-          + "         (including lines -----BEGIN CERTIFICATE----- and\n"
-          + "         -----END CERTIFICATE-----) in the box.\n"
-          + "      5. Click “Save”. If there are validation issues with the form, you will\n"
-          + "         be prompted to fix them and click “Save” again.\n"
-          + "\n"
-          + "    A failover SSL certificate can also be added in order to prevent connection\n"
-          + "    issues once your main certificate expires. Connecting with either of the\n"
-          + "    certificates will work with our production EPP server.\n"
-          + "\n"
-          + "    Further information about our EPP connection requirements can be found in\n"
-          + "    section 9.2 in the updated Technical Guide in your Google Drive folder.\n"
-          + "\n"
-          + "    Note that account certificate changes take a few minutes to become\n"
-          + "    effective and that the existing connections will remain unaffected by\n"
-          + "    the change.\n"
-          + "\n"
-          + "    If you also would like to update your OT&E account certificate, please send\n"
-          + "    an email from your primary or technical contact to\n"
-          + "    registry-support@google.com and include the full certificate string\n"
-          + "    (including lines -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----).\n"
-          + "\n"
-          + "    Regards,\n"
-          + "    Google Registry\n";
+          + "  1. Navigate to support and login using your %4$s@registry.example credentials.\n"
+          + "  2. Click Settings -> Privacy on the top left corner.\n"
+          + "  3. Click Edit and enter certificate string."
+          + "  3. Click Save"
+          + "Regards,"
+          + "Example Registry";

-  private static final String EXPIRATION_WARNING_EMAIL_SUBJECT_TEXT =
-      "[Important] Expiring SSL certificate for Google " + "Registry EPP connection";
+  private static final String EXPIRATION_WARNING_EMAIL_SUBJECT_TEXT = "Expiration Warning Email";

  @RegisterExtension
  public final AppEngineExtension appEngine =
@@ -623,11 +592,11 @@ class SendExpiringCertificateNotificationEmailActionTest {
    assertThat(emailBody).contains(registrarName);
    assertThat(emailBody).contains(certificateType.getDisplayName());
    assertThat(emailBody).contains(certExpirationDateStr);
-    assertThat(emailBody).contains(registrarId + "@registry.google");
-    assertThat(emailBody).doesNotContain("%1$s@registry.google");
-    assertThat(emailBody).doesNotContain("%2$s@registry.google");
-    assertThat(emailBody).doesNotContain("%3$s@registry.google");
-    assertThat(emailBody).doesNotContain("%4$s@registry.google");
+    assertThat(emailBody).contains(registrarId + "@registry.example");
+    assertThat(emailBody).doesNotContain("%1$s");
+    assertThat(emailBody).doesNotContain("%2$s");
+    assertThat(emailBody).doesNotContain("%3$s");
+    assertThat(emailBody).doesNotContain("%4$s");
  }

  @TestOfyAndSql
--- a/core/src/test/java/google/registry/beam/common/DatabaseSnapshotTest.java
+++ b/core/src/test/java/google/registry/beam/common/DatabaseSnapshotTest.java
@@ -37,6 +37,7 @@ import org.junit.jupiter.api.AfterAll;
 import org.junit.jupiter.api.BeforeAll;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
+import org.junitpioneer.jupiter.RetryingTest;
 import org.testcontainers.containers.PostgreSQLContainer;
 import org.testcontainers.junit.jupiter.Container;
 import org.testcontainers.junit.jupiter.Testcontainers;
@@ -106,7 +107,7 @@ public class DatabaseSnapshotTest {
    try (DatabaseSnapshot databaseSnapshot = DatabaseSnapshot.createSnapshot()) {}
  }

-  @Test
+  @RetryingTest(2) // TODO(b/226945065) revert to @Test when DatbaseMigrationSchedule is removed.
  void readSnapshot() {
    try (DatabaseSnapshot databaseSnapshot = DatabaseSnapshot.createSnapshot()) {
      Registry snapshotRegistry =
@@ -120,7 +121,7 @@ public class DatabaseSnapshotTest {
    }
  }

-  @Test
+  @RetryingTest(2) // TODO(b/226945065) revert to @Test when DatbaseMigrationSchedule is removed.
  void readSnapshot_withSubsequentChange() {
    try (DatabaseSnapshot databaseSnapshot = DatabaseSnapshot.createSnapshot()) {
      Registry updated =
--- a/core/src/test/java/google/registry/beam/comparedb/ValidateSqlUtilsTest.java
+++ b/core/src/test/java/google/registry/beam/comparedb/ValidateSqlUtilsTest.java
@@ -19,11 +19,19 @@ import static google.registry.beam.comparedb.ValidateSqlUtils.getMedianIdForHist
 import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static org.joda.time.DateTimeZone.UTC;

+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Maps;
 import com.google.common.truth.Truth;
+import google.registry.beam.comparedb.ValidateSqlUtils.DiffableFieldNormalizer;
 import google.registry.model.bulkquery.TestSetupHelper;
+import google.registry.model.contact.ContactAddress;
+import google.registry.model.contact.ContactResource;
+import google.registry.model.contact.PostalInfo;
 import google.registry.model.domain.DomainHistory;
 import google.registry.testing.AppEngineExtension;
 import google.registry.testing.FakeClock;
+import google.registry.util.DiffUtils;
+import java.util.Map;
 import org.joda.time.DateTime;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.extension.RegisterExtension;
@@ -60,4 +68,29 @@ class ValidateSqlUtilsTest {
    assertThat(getMedianIdForHistoryTable("DomainHistory"))
        .hasValue(setupHelper.domainHistory.getId());
  }
+
+  @Test
+  void diffableFieldNormalizer() {
+    ContactResource contactResource =
+        new ContactResource.Builder()
+            .setLocalizedPostalInfo(
+                new PostalInfo.Builder()
+                    .setType(PostalInfo.Type.LOCALIZED)
+                    .setAddress(
+                        new ContactAddress.Builder()
+                            .setStreet(ImmutableList.of("111 8th Ave", ""))
+                            .setCity("New York")
+                            .setState("NY")
+                            .setZip("10011")
+                            .setCountryCode("US")
+                            .build())
+                    .build())
+            .build();
+    Map<String, Object> origMap = contactResource.toDiffableFieldMap();
+    Map<String, Object> trimmedMap = Maps.transformEntries(origMap, new DiffableFieldNormalizer());
+    // In the trimmed map, localizedPostalInfo.address.street only has one element in the list,
+    // thus the output: 'null -> '
+    Truth.assertThat(DiffUtils.prettyPrintEntityDeepDiff(trimmedMap, origMap))
+        .isEqualTo("localizedPostalInfo.address.street.1: null -> \n");
+  }
 }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Ben McIlwain	e4312322dc	Add a no-async actions DB migration phase (#1579 ) * Add a no-async actions DB migration phase This needs to be set several hours prior to entering the READONLY stage. This is not a read-only stage; all synchronous actions under Datastore (such as domain creates) will continue to succeed. The only thing that will fail is host deletes, host renames, and contact deletes, as these three actions require a mapreduce to run before they are complete, and we don't want mapreduces hanging around and executing during what is supposed to be a short duration READONLY period.	2022-04-01 16:55:51 -04:00
gbrodman	24dfaf6406	Use UrlFetch for RDE and default TLS (1.2) for other URL connections (#1578 ) * Use UrlFetch for RDE and default TLS (1.2) for other URL connections This removes the TLS 1.3-settings in the module providers and, essentially, reverts the changes in #1535 only to the RdeReporter and RdeReportActionTest	2022-03-31 14:08:28 -04:00
Rachel Guan	7afb8fa343	Add default value to renewal_price_behavior (#1575 ) * Add default value to renewal_price_behavior * Change DEFAULT_PRICE to DEFAULT	2022-03-31 12:27:32 -04:00
Michael Muller	02b3f7b505	Fix a few references to "Datastore" in comments (#1576 ) * Fix a few references to "Datastore" in comments Fix references to Datastore in the comments of classes that are now SQL-only.	2022-03-30 15:17:38 -04:00
Lai Jiang	25342aa480	Make a best effort guess on the RDE folder name (prefix) when not provided. (#1574 ) We have a cron job that runs the RDE upload action every 4 hours for all TLD. Normally this should be a no-op beacuse a RDE upload is scheduled after RDE staging is completed, and when it fails with non-2XX status it will retry. However if for some reason it failed due to 20X status (like waiting for the SFTP cursor), it will not retry but rely on the cron job to catch up. With the BEAM RDE pipeline every staging job saves all its deposits in a uniquely named folder to avoid the need to use a lock, which is not practical in BEAM. However the cron job has no way of knowing what the prefixes are for each TLD so it will fail in SQL mode. In this PR we implemented a logic to guess what the prefix should be and use it, if we are in SQL mode and a prefix is not provided. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1574) <!-- Reviewable:end -->	2022-03-30 11:36:24 -04:00
Rachel Guan	3ef1e6c6a4	Add renewal columns in BillingRecurrence (#1568 ) * Add renewal columns in BillingRecurrence * Change from event to recurrence in file name	2022-03-28 17:42:01 -04:00
Lai Jiang	9363b30b3e	Set the initial worker count for the RDE beam pipeline at 24 (#1572 ) * Set the initial worker count for the RDE beam pipeline at 24 This likely will speed up the pipeline by skipping the initially slow process of spinning up instances.	2022-03-27 22:51:58 -04:00
Weimin Yu	342f7d72a2	Fix sporadic SQL Snapshot failure (#1571 ) * Fix sporadic SQL Snapshot failure The Postgresql set-snapshot statement (called in JpaTransactionManager.setDatabaseSnapshot() method) must be the first statement in the SQL transaction. Currenty the JpaTransaction.transact() method may insert a query for DatabaseMigrationStateSchedule before the user query when the cache is empty or the cached value expires. This PR proactively preloads the cache in RegistryJpaIO to prevent cache loading inside the transaction. This PR also changes some DatabaseSnapshotTest tests to be retrying, in case they run just after the cache expires. (This has happened before in CI).	2022-03-25 15:55:00 -04:00
Michael Muller	3592877210	Terminate with exit code of 1 on format check fail (#1570 ) The format check script currently outputs "true" if there were files that need reformatting and "false" if not, which is useful for gradle but less so for other applications (notably commit hooks). Terminate with an exit code of 1 if the format check fails. TESTED: Tried this from both a pre-commit hook and from the gradle build.	2022-03-25 12:50:27 -04:00
Michael Muller	3006ca39ca	Add a "list_txns" to dump Transaction table (#1569 ) * Add a "list_txns" to dump Transaction table Add the list_txns command which can dump the entire contents of the Transaction table, either in csv format or as human readable transactions. The CSV format is useful for storing the transaction table at a specific point in time for later reference without requiring us to repeatedly hit the replica. Creating this without tests because this command has a very short shelf-life and is really only intended to be run by developers. Tested all features locally. * Reformatted	2022-03-25 12:41:10 -04:00
gbrodman	63adfa77ed	Use TLS v1.3 explicitly in RDE reporting (#1564 ) * Use TLS v1.3 explicitly in RDE reporting The default Java 1.8 TLS version is 1.2 which isn't supported by the ICANN upload site.	2022-03-25 12:09:46 -04:00
Weimin Yu	fd5e5bf6f1	Ignore trivial differences when comparing DB (#1567 ) * Ignore trivial differences when comparing DB Some data difference are due to entity model differences and also harmless. We should igore them when comparing Datastore and SQL. This PR ignores the following diffs: - null vs empty collection - the empty string in Address.stree field, which is a list	2022-03-23 13:52:31 -04:00
sarahcaseybot	2495167215	Address some tiny TODOs (#1566 ) * Address some tiny TODOs * Format fix	2022-03-23 12:23:29 -04:00
gbrodman	0c6f399533	Bump flogger and beam dependency versions (#1562 ) * Bump flogger and beam dependency versions Beam 2.34.0 -> 2.37.0 Flogger 0.7.3 -> 0.7.4 Intellij keeps getting confused about which version of Flogger we're bringing in. Even though we had previously locked Flogger to 0.7.3, for some reason it was still bringing in the Beam transitive dependency of 0.6.0 which was causing the a bunch of class initialization errors. Bumping Beam to 2.34.0 bumps the transitive dependency to 0.7.4 so we can always use that.	2022-03-22 16:08:32 -04:00
Michael Muller	075ea23f1d	Fix build warning (#1565 )	2022-03-22 10:34:36 -04:00
Lai Jiang	3c19d4cbf6	Some code health fixes (#1563 ) 1. testRun_withPrefix() in RdeUploadActionTest does calls a mock lock handler and does not actually try to read from the fake GCS implementation. Therefore there's no point settig it up. 2. Remove an unused field in UploadDatastoreBackupActionTest. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1563) <!-- Reviewable:end -->	2022-03-22 09:49:14 -04:00
Rachel Guan	ea4d60c830	Remove static methods in back up actions (#1481 ) * Remove static methods in back up actions * Remove BigqueryPollJob helper class * Add schedule time in task comparison * Change payload type from byte[] to ByteString	2022-03-17 17:54:20 -04:00
Lai Jiang	c24e0053c8	Fix a subtle issue in BRDA copy caused by Cloud Tasks (#1556 ) * Fix a subtle issue in BRDA copy caused by Cloud Tasks After the Cloud Tasks migration and #1508, the BRDA copy job now routinely fail on the first try because the revision update is not commited by the time the Cloud Tasks job enqueued in the same transaction runs for the first time. This is because the enqueueing is a side effect and not part of the transaction. The job eventually succeeds because of retries. This PR attempts to mitigate the initial failure by adding a delay to the enqueued job, and checking the cursor in the job itself to prevent it from running before the transaction is commited.	2022-03-17 16:32:07 -04:00
Michael Muller	537a6e4466	Fix issues with saving and deleting gap records (#1561 ) * Fix issues with saving and deleting gap records Datastore limits us to mutating up to 25 records per transaction. We sometimes exceed that when deleting expired gap records. In addition, it is theoretically possible for us to accumulate enough continuous gap records to exceed this count while replaying the original transaction. Deal with deletion by breaking up the gap records to be deleted into a batch size that is small enough to be deleted transactionally (in practice, we don't much care about the transactionality but it doesn't seem like we can delete batches without it). Deal with the possibility of too many additions by always breaking out gap record storage and last transaction number updates into their own transaction(s) (separate from the replay of the original SQL transaction).	2022-03-17 15:09:59 -04:00
Ben McIlwain	6c20d39a2d	Add domain repo ID and token type indexes to AllocationToken table (#1560 ) These are useful for the purposes of filtering by one-time/multi-use tokens, and for determining which one-time tokens have been used (and if so, for which domain).	2022-03-17 13:58:45 -04:00
Michael Muller	60c156c061	Track and replay Transaction table gaps (#1557 ) * Track and replay Transaction table gaps Id gaps in the Transaction table can be the result of a transactions committed out of order. To deal with this, keep track of gaps for up to five minutes and check to see if they've been back-filled prior to applying the next batch of transactions during reply. * Changes for review * Calculate gap expiration time before gap queries * Reformat.	2022-03-16 11:08:45 -04:00
Ben McIlwain	742ad0b37c	Add 3 more SQL indexes to the Host table (#1559 ) * Add 3 more SQL indexes to the Host table These indexes on creationTime, deletionTime, and currentSponsorRegistrarId are present on the other two EPP resource tables (Domain and Contact), and are useful for a wide variety of operations/analytics queries.	2022-03-15 22:16:36 -04:00
Weimin Yu	6dd6ebce75	Improve cache loading in Registries.java (#1558 ) * Improve cache loading in Registries.java The loader for the TLD cache in Registries.java unnecessarily reads from another cache when running with SQL, potentially triggering additional database access. This code runs in the whois query path, and contributes to the high latency in sandbox.	2022-03-15 22:02:05 -04:00
Ben McIlwain	767e3935af	Make DigestType.fromWireValue() more performant (#1555 ) * Make DigestType.fromWireValue() more performant	2022-03-15 13:00:54 -04:00
Rachel Guan	86aa420773	Add a a delay to task in CommitLogCheckpointAction (#1554 ) * Add delay to task	2022-03-15 10:49:35 -04:00
Ben McIlwain	61b38569e2	Add domainRepoId indexes to billing events (#1544 ) The query analyzer identified this is a missing index on the BillingEvent table, and I added it for recurrences and cancellations as well as it's likely to be a problem for them too. "Give me all the billing events associated with a given domain by its repo ID" seems like a pretty common use case for the DB (and does appear to be used by our invoicing pipeline). This is a follow-up to PR #1545.	2022-03-14 17:20:58 -04:00
Ben McIlwain	4bfa19a90c	Add 9 more indexes to SQL schema (#1540 ) These indexes were identified as missing by PostgreSQL's query analyzer in our sandbox environment (where we get enough realistic EPP traffic to identify these deficiencies). Note that a lot of the new indexes being named have to use the DB representation of the column name because they are either embedded or subclassed entities, whereas most of the existing ones are able to simply refer to Java field names. This is the Java schema follow-up PR to PR #1541, which is what added the actual DB changes through Flyway scripts.	2022-03-14 15:59:05 -04:00
Rachel Guan	a001df6d7a	Remove AppEngineServiceUtils dependency (#1534 )	2022-03-14 14:10:30 -04:00
Rachel Guan	6caf7819ed	Replace email content with placeholders (#1530 ) * Replace email content with placeholders * Improve sample email wording	2022-03-14 11:30:58 -04:00
Weimin Yu	757803e985	Revise host.inet_addresses query to use gin index (#1550 ) * Revise host.inet_addresses query to use gin index	2022-03-09 23:32:17 -05:00
Weimin Yu	4b1f4f96e3	Reorganize new schema changes (#1551 ) * Reorganize new schema changes Reorganized new schema changes and make each flyway script update a single table. Each flyway script is executed in a single database transaction so that the script can be rolled back in one shot. It acquires a shared lock on all tables touched by the script. This is deadlock-prone because in a busy database, there may be user queries that attempt to lock the same set of tables, but in different order. By limiting each script to one table, we avoid the problem. We should have some a presubmit check to enforce this rule. All changes have been deployed to Sandbox out-of-band. When doing so, we changed all CREATE INDEX statements to CREATE INDEX IF NOT EXISTS. Future deployments should be able to proceed normally.	2022-03-09 20:47:24 -05:00
Ben McIlwain	bd49e8b238	Add anti-deadlock instructions to DB update README (#1552 )	2022-03-09 18:50:15 -05:00
Weimin Yu	6249a8e118	Revise Host index on inet_addresses (#1549 ) * Revise Host index on inet_addresses The index on the 'inet_addresses array column should be of gin or gist type, which index individual array elements. We use gin for now since host updates are not often, and gin has better accuracy. Since flyway script V108__... has not been deployed, we edit the file in place instead of adding a new script. This will be followed up with a modified query that can take advantage of the gin index. Until then we don't expect to see performance improvement. The suspected bottlenect query in the whois path is: select * from "Host" where 'non-ip-string' = any(inet_address) and deletion_time < now(); It needs to be revised into: select * from "Host" where array['non-ip-string'] <@ inet_address and deletion_time < now(); The combined change reduces the query time from 90ms to 30ms in Sandbox, and from 150ms to 40ms in production. It is unclear if this solves all problem with whois latency.	2022-03-08 14:22:01 -05:00
Ben McIlwain	9b7bb12cd1	Add deletionTime/inetAddresses indexes to Host table to support WHOIS (#1548 ) * Add deletionTime/inetAddresses indexes to Host table to support WHOIS Weimin identified these as missing, and being the cause of slowdowns in NameserverLookupByIpCommand that we're seeing in sandbox. This is the first of two PRs, adding just the Flyway/schema changes. The second PR adding the Java object model changes is #1547.	2022-03-07 16:07:11 -05:00
Michael Muller	71d13bab71	Improve Transaction gap processing (#1546 ) Skip multiple gaps in one pass and write the correct transaction id to datastore.	2022-03-07 13:26:18 -05:00
Ben McIlwain	8db28b7e61	Add domainRepoId indexes to billing events (#1545 ) * Add domainRepoId indexes to billing events #1544 The query analyzer identified this is a missing index on the BillingEvent table, and I added it for recurrences and cancellations as well as it's likely to be a problem for them too. "Give me all the billing events associated with a given domain by its repo ID" seems like a pretty common use case for the DB (and does appear to be used by our invoicing pipeline). This is the first of two PRs that makes just the DB changes. The second PR (#1544) will add the Java code changes, and will be committed after this one is deployed.	2022-03-07 12:21:40 -05:00
Lai Jiang	5a7dc307c5	Update the the latest LTS Node version (#1543 ) <!-- Reviewable:start --> This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1543) <!-- Reviewable:end -->	2022-03-07 11:49:18 -05:00
Ben McIlwain	67278af3cb	Add 9 more indexes to SQL schema (#1541 ) * Add 9 more indexes to SQL schema This indexes were identified as missing by PostgreSQL's query analyzer in our sandbox environment (where we get enough realistic EPP traffic to identify these deficiencies). This is the first of two PRs -- the second PR (#1540) will be merged only after this one is live in production. Note that this is the PR that actually modifies the database though, so once this one is deployed we will already have the benefit of the new indexes.	2022-03-05 17:57:36 -05:00
sarahcaseybot	1eafc983ab	Check signature length in DS records (#1538 ) * Check signature length in DS records * Small fixes * Add unit tests * Formatting fix	2022-03-04 15:18:14 -05:00
gbrodman	f1bbdc5a0b	Use built-in Java URL connections instead of UrlFetchService (#1535 ) - Use the standard HttpsURLConnection to write/read data - Rewrite RdeReporter, Nordn*Action, and Marksdb classes and related tests to conform to the new format - Remove FakeURLFetchService and ForwardingUrlFetchService as they weren't used - Refactor UrlFetchException to UrlConnectionException - Refactor UrlFetchUtils to UrlConnectionUtils I will need to test this on Alpha. Fortunately the connections that don't require auth (e.g. TMDB downloading) should be testable.	2022-03-04 14:16:22 -05:00
Michael Muller	b146301495	Allow replicateToDatastore to skip gaps (#1539 ) * Allow replicateToDatastore to skip gaps As it turns out, gaps in the transaction id sequence number are expected because rollbacks do not rollback sequence numbers. To deal with this, stop checking these. This change is not adequate in and of itself, as it is possible for a gap to be introduced if two transactions are committed out of order of their sequence number. We are currently discussing several strategies to mitigate this. * Remove println, add a record verification	2022-03-04 09:04:13 -05:00
Weimin Yu	437a747eae	Pass stack trace to validate_datastore user (#1537 ) * Pass stack trace to validate_datastore user	2022-03-03 16:10:31 -05:00
Weimin Yu	a620b37c80	Fix hanging test (#1536 ) * Fix hanging test Tests using the TestServerExtension may hang forever if an underlying component (e.g., testcontainer for psql) fails. This may be the cause of the some kokoro runs that timeed out after three hours.	2022-03-03 14:43:32 -05:00
Rachel Guan	267cbeb95b	Inject CloudTasksUtil to AsyncTaskEnqueuer (#1522 ) * Inject CloudTasksUtil to AsyncTasksEnqueuer * Rebase * Remove QUEUE_ASYNC_DELETE from AsyncTasksEnqueuer * Refactor create() * Remove AppEngineServiceUtil depdendency from AsyncTaskEnqueuer	2022-03-02 11:31:45 -05:00