Remove VKeyTranslatorFactory.createVKey(String) (#1312 )

* Remove VKeyTranslatorFactory.createVKey(String) This method serves the same function as VKey.fromWebsafeKey(), and isn't used anywhere. Move the test for it into VKeyTest and use it to instead test fromWebsafeKey() (which didn't previously have a test).
Keep new lines/spaces as the way it shows in config yaml (#1311 )
2026-07-09 01:27:09 +00:00 · 2021-09-09 12:24:02 -04:00 · 2021-09-08 22:00:32 -04:00 · 2021-09-03 16:56:57 -04:00 · 2021-09-03 11:25:53 -04:00 · 2021-09-03 09:39:55 -04:00
502 changed files with 10505 additions and 6928 deletions
@@ -215,6 +215,7 @@ PRESUBMITS = {
         "RdapDomainSearchAction.java",
         "RdapNameserverSearchAction.java",
         "RdapSearchActionBase.java",
+         "ReadOnlyCheckingEntityManager.java",
         "RegistryQuery",
         },
    ):
@@ -44,7 +44,6 @@ def outcastTestPatterns = [
    "google/registry/flows/domain/DomainCreateFlowTest.*",
    "google/registry/flows/domain/DomainUpdateFlowTest.*",
    "google/registry/tools/CreateDomainCommandTest.*",
-    "google/registry/tools/server/CreatePremiumListActionTest.*",
 ]

 // Tests that fail when running Gradle in a docker container, e. g. when
@@ -70,8 +69,6 @@ def dockerIncompatibleTestPatterns = [
 // Nomulus classes, e.g., threads and objects retained by frameworks.
 // TODO(weiminyu): identify cause and fix offending tests.
 def fragileTestPatterns = [
-    // Problem seems to lie with AppEngine TaskQueue for test.
-    "google/registry/cron/TldFanoutActionTest.*",
    // Test Datastore inexplicably aborts transaction.
    "google/registry/model/tmch/ClaimsListShardTest.*",
    // Changes cache timeouts and for some reason appears to have contention
@@ -185,6 +182,7 @@ dependencies {
  compile deps['com.google.monitoring-client:metrics']
  compile deps['com.google.monitoring-client:stackdriver']
  compile deps['com.google.api-client:google-api-client-java6']
+  compile deps['com.google.api.grpc:proto-google-cloud-tasks-v2']
  compile deps['com.google.apis:google-api-services-admin-directory']
  compile deps['com.google.apis:google-api-services-appengine']
  compile deps['com.google.apis:google-api-services-bigquery']
@@ -217,11 +215,13 @@ dependencies {
  compile deps['com.google.flogger:flogger']
  runtime deps['com.google.flogger:flogger-system-backend']
  compile deps['com.google.guava:guava']
+  compile deps['com.google.protobuf:protobuf-java']
  gradleLint.ignore('unused-dependency') {
    compile deps['com.google.gwt:gwt-user']
  }
  compile deps['com.google.cloud:google-cloud-core']
  compile deps['com.google.cloud:google-cloud-storage']
+  compile deps['com.google.cloud:google-cloud-tasks']
  compile deps['com.google.http-client:google-http-client']
  compile deps['com.google.http-client:google-http-client-appengine']
  compile deps['com.google.http-client:google-http-client-jackson2']
@@ -54,12 +54,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -83,10 +86,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -104,11 +107,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -119,10 +123,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -134,8 +138,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -151,17 +155,17 @@ commons-logging:commons-logging:1.2
 dnsjava:dnsjava:3.3.1
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -259,7 +263,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
@@ -53,12 +53,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -82,10 +85,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -103,11 +106,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger:0.5.1
 com.google.flogger:google-extensions:0.5.1
@@ -117,10 +121,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -133,7 +137,7 @@ com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
 com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -149,17 +153,17 @@ commons-logging:commons-logging:1.2
 dnsjava:dnsjava:3.3.1
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -252,7 +256,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
@@ -58,12 +58,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -87,10 +90,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -109,11 +112,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -124,10 +128,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -139,8 +143,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -159,17 +163,17 @@ guru.nidi:graphviz-java-all-j2v8:0.17.0
 guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -273,7 +277,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
@@ -58,12 +58,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -87,10 +90,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -109,11 +112,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -124,10 +128,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -139,8 +143,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -159,17 +163,17 @@ guru.nidi:graphviz-java-all-j2v8:0.17.0
 guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -272,7 +276,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
@@ -54,12 +54,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -83,10 +86,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -104,11 +107,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -119,10 +123,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -134,8 +138,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -151,17 +155,17 @@ commons-logging:commons-logging:1.2
 dnsjava:dnsjava:3.3.1
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -259,7 +263,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
@@ -53,12 +53,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -82,10 +85,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -103,11 +106,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger:0.5.1
 com.google.flogger:google-extensions:0.5.1
@@ -117,10 +121,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -133,7 +137,7 @@ com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
 com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -149,17 +153,17 @@ commons-logging:commons-logging:1.2
 dnsjava:dnsjava:3.3.1
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -253,7 +257,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.xerial.snappy:snappy-java:1.1.4
@@ -58,12 +58,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -87,10 +90,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -108,11 +111,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -123,10 +127,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -138,8 +142,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -158,17 +162,17 @@ guru.nidi:graphviz-java-all-j2v8:0.17.0
 guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -271,7 +275,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
@@ -58,12 +58,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -87,10 +90,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -108,11 +111,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -123,10 +127,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -138,8 +142,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -158,17 +162,17 @@ guru.nidi:graphviz-java-all-j2v8:0.17.0
 guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -271,7 +275,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
@@ -58,12 +58,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -87,10 +90,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -108,11 +111,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -123,10 +127,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -138,8 +142,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -158,17 +162,17 @@ guru.nidi:graphviz-java-all-j2v8:0.17.0
 guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -271,7 +275,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
@@ -58,12 +58,15 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
-com.google.api.grpc:proto-google-common-protos:2.1.0
-com.google.api.grpc:proto-google-iam-v1:1.0.10
-com.google.api:api-common:1.10.1
-com.google.api:gax-grpc:1.62.0
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
+com.google.api.grpc:proto-google-common-protos:2.3.2
+com.google.api.grpc:proto-google-iam-v1:1.0.14
+com.google.api:api-common:1.10.4
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.79.0
-com.google.api:gax:1.62.0
+com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
 com.google.apis:google-api-services-appengine:v1-rev130-1.25.0
 com.google.apis:google-api-services-bigquery:v2-rev20200916-1.30.10
@@ -87,10 +90,10 @@ com.google.appengine.tools:appengine-pipeline:0.2.13
 com.google.appengine:appengine-api-1.0-sdk:1.9.86
 com.google.appengine:appengine-remote-api:1.9.86
 com.google.appengine:appengine-testing:1.9.86
-com.google.auth:google-auth-library-credentials:0.24.1
-com.google.auth:google-auth-library-oauth2-http:0.24.1
+com.google.auth:google-auth-library-credentials:0.26.0
+com.google.auth:google-auth-library-oauth2-http:0.26.0
 com.google.auto.service:auto-service-annotations:1.0-rc7
-com.google.auto.value:auto-value-annotations:1.7.4
+com.google.auto.value:auto-value-annotations:1.8.1
 com.google.auto.value:auto-value:1.7.4
 com.google.cloud.bigdataoss:gcsio:2.1.6
 com.google.cloud.bigdataoss:util:2.1.6
@@ -109,11 +112,12 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.113.12
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
-com.google.code.gson:gson:2.8.6
+com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
 com.google.dagger:dagger:2.33
-com.google.errorprone:error_prone_annotations:2.5.1
+com.google.errorprone:error_prone_annotations:2.7.1
 com.google.escapevelocity:escapevelocity:0.9.1
 com.google.flogger:flogger-system-backend:0.5.1
 com.google.flogger:flogger:0.5.1
@@ -124,10 +128,10 @@ com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
 com.google.gwt:gwt-user:2.9.0
 com.google.http-client:google-http-client-apache-v2:1.39.0
 com.google.http-client:google-http-client-appengine:1.39.0
-com.google.http-client:google-http-client-gson:1.39.0
+com.google.http-client:google-http-client-gson:1.39.2
 com.google.http-client:google-http-client-jackson2:1.39.0
 com.google.http-client:google-http-client-protobuf:1.33.0
-com.google.http-client:google-http-client:1.39.0
+com.google.http-client:google-http-client:1.39.2
 com.google.inject.extensions:guice-multibindings:4.1.0
 com.google.inject:guice:4.1.0
 com.google.j2objc:j2objc-annotations:1.3
@@ -139,8 +143,8 @@ com.google.oauth-client:google-oauth-client-java6:1.31.4
 com.google.oauth-client:google-oauth-client-jetty:1.31.4
 com.google.oauth-client:google-oauth-client-servlet:1.31.4
 com.google.oauth-client:google-oauth-client:1.31.4
-com.google.protobuf:protobuf-java-util:3.15.3
-com.google.protobuf:protobuf-java:3.15.3
+com.google.protobuf:protobuf-java-util:3.17.3
+com.google.protobuf:protobuf-java:3.17.3
 com.google.re2j:re2j:1.6
 com.google.template:soy:2021-02-01
 com.googlecode.charts4j:charts4j:1.3
@@ -159,17 +163,17 @@ guru.nidi:graphviz-java-all-j2v8:0.17.0
 guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.65
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
-io.grpc:grpc-context:1.36.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
+io.grpc:grpc-context:1.39.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -272,7 +276,7 @@ org.testcontainers:database-commons:1.15.2
 org.testcontainers:jdbc:1.15.2
 org.testcontainers:postgresql:1.15.2
 org.testcontainers:testcontainers:1.15.2
-org.threeten:threetenbp:1.5.0
+org.threeten:threetenbp:1.5.1
 org.tukaani:xz:1.5
 org.w3c.css:sac:1.3
 org.webjars.npm:viz.js-for-graphviz-java:2.1.3
@@ -54,10 +54,13 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
 com.google.api.grpc:proto-google-common-protos:2.3.2
 com.google.api.grpc:proto-google-iam-v1:1.0.14
 com.google.api:api-common:1.10.4
-com.google.api:gax-grpc:1.62.0
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.83.0
 com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
@@ -106,6 +109,7 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.118.0
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
 com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
@@ -160,17 +164,17 @@ commons-logging:commons-logging:1.2
 dnsjava:dnsjava:3.3.1
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.102
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
 io.grpc:grpc-context:1.39.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -53,10 +53,13 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
 com.google.api.grpc:proto-google-common-protos:2.3.2
 com.google.api.grpc:proto-google-iam-v1:1.0.14
 com.google.api:api-common:1.10.4
-com.google.api:gax-grpc:1.62.0
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.83.0
 com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
@@ -105,6 +108,7 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.118.0
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
 com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
@@ -158,17 +162,17 @@ commons-logging:commons-logging:1.2
 dnsjava:dnsjava:3.3.1
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.102
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
 io.grpc:grpc-context:1.39.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -58,10 +58,13 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
 com.google.api.grpc:proto-google-common-protos:2.3.2
 com.google.api.grpc:proto-google-iam-v1:1.0.14
 com.google.api:api-common:1.10.4
-com.google.api:gax-grpc:1.62.0
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.83.0
 com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
@@ -111,6 +114,7 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.118.0
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
 com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
@@ -169,17 +173,17 @@ guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.102
 io.github.java-diff-utils:java-diff-utils:4.9
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
 io.grpc:grpc-context:1.39.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -58,10 +58,13 @@ com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:1.4.0
 com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:2.0.2
 com.google.api.grpc:proto-google-cloud-spanner-v1:2.0.2
+com.google.api.grpc:proto-google-cloud-tasks-v2:1.33.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.89.2
+com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.89.2
 com.google.api.grpc:proto-google-common-protos:2.3.2
 com.google.api.grpc:proto-google-iam-v1:1.0.14
 com.google.api:api-common:1.10.4
-com.google.api:gax-grpc:1.62.0
+com.google.api:gax-grpc:1.66.0
 com.google.api:gax-httpjson:0.83.0
 com.google.api:gax:1.66.0
 com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0
@@ -111,6 +114,7 @@ com.google.cloud:google-cloud-pubsublite:0.7.0
 com.google.cloud:google-cloud-secretmanager:1.4.0
 com.google.cloud:google-cloud-spanner:2.0.2
 com.google.cloud:google-cloud-storage:1.118.0
+com.google.cloud:google-cloud-tasks:1.33.2
 com.google.code.findbugs:jsr305:3.0.2
 com.google.code.gson:gson:2.8.7
 com.google.common.html.types:types:1.0.6
@@ -169,17 +173,17 @@ guru.nidi:graphviz-java:0.17.0
 io.dropwizard.metrics:metrics-core:3.2.6
 io.github.classgraph:classgraph:4.8.102
 io.github.java-diff-utils:java-diff-utils:4.9
-io.grpc:grpc-alts:1.36.0
-io.grpc:grpc-api:1.36.0
-io.grpc:grpc-auth:1.36.0
+io.grpc:grpc-alts:1.39.0
+io.grpc:grpc-api:1.39.0
+io.grpc:grpc-auth:1.39.0
 io.grpc:grpc-context:1.39.0
-io.grpc:grpc-core:1.36.0
-io.grpc:grpc-grpclb:1.36.0
-io.grpc:grpc-netty-shaded:1.36.0
+io.grpc:grpc-core:1.39.0
+io.grpc:grpc-grpclb:1.39.0
+io.grpc:grpc-netty-shaded:1.39.0
 io.grpc:grpc-netty:1.32.2
-io.grpc:grpc-protobuf-lite:1.36.0
-io.grpc:grpc-protobuf:1.36.0
-io.grpc:grpc-stub:1.36.0
+io.grpc:grpc-protobuf-lite:1.39.0
+io.grpc:grpc-protobuf:1.39.0
+io.grpc:grpc-stub:1.39.0
 io.netty:netty-buffer:4.1.51.Final
 io.netty:netty-codec-http2:4.1.51.Final
 io.netty:netty-codec-http:4.1.51.Final
@@ -36,6 +36,11 @@ import google.registry.gcs.GcsUtils;
 import google.registry.model.common.DatabaseMigrationStateSchedule;
 import google.registry.model.common.DatabaseMigrationStateSchedule.MigrationState;
 import google.registry.model.common.DatabaseMigrationStateSchedule.ReplayDirection;
+import google.registry.model.replay.DatastoreEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.NonReplicatedEntity;
+import google.registry.model.replay.ReplaySpecializer;
+import google.registry.model.replay.SqlReplayCheckpoint;
 import google.registry.model.server.Lock;
 import google.registry.model.translators.VKeyTranslatorFactory;
 import google.registry.persistence.VKey;
@@ -44,11 +49,6 @@ import google.registry.request.Action.Method;
 import google.registry.request.Parameter;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
-import google.registry.schema.replay.DatastoreEntity;
-import google.registry.schema.replay.DatastoreOnlyEntity;
-import google.registry.schema.replay.NonReplicatedEntity;
-import google.registry.schema.replay.ReplaySpecializer;
-import google.registry.schema.replay.SqlReplayCheckpoint;
 import google.registry.util.Clock;
 import google.registry.util.RequestStatusChecker;
 import java.io.IOException;
@@ -58,6 +58,7 @@ import javax.inject.Inject;
 import javax.servlet.http.HttpServletResponse;
 import org.joda.time.DateTime;
 import org.joda.time.Duration;
+import org.joda.time.Seconds;

 /** Action that replays commit logs to Cloud SQL to keep it up to date. */
@Action(
@@ -97,7 +98,8 @@ public class ReplayCommitLogsToSqlAction implements Runnable {

  @Override
  public void run() {
-    MigrationState state = DatabaseMigrationStateSchedule.getValueAtTime(clock.nowUtc());
+    DateTime startTime = clock.nowUtc();
+    MigrationState state = DatabaseMigrationStateSchedule.getValueAtTime(startTime);
    if (!state.getReplayDirection().equals(ReplayDirection.DATASTORE_TO_SQL)) {
      String message =
          String.format(
@@ -126,7 +128,7 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
      if (dryRun) {
        resultMessage = executeDryRun();
      } else {
-        resultMessage = replayFiles();
+        resultMessage = replayFiles(startTime);
      }
      response.setStatus(SC_OK);
      response.setPayload(resultMessage);
@@ -160,10 +162,11 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
        fileBatch.stream().limit(10).collect(toImmutableList()));
  }

-  private String replayFiles() {
-    DateTime replayTimeoutTime = clock.nowUtc().plus(REPLAY_TIMEOUT_DURATION);
+  private String replayFiles(DateTime startTime) {
+    DateTime replayTimeoutTime = startTime.plus(REPLAY_TIMEOUT_DURATION);
    DateTime searchStartTime = jpaTm().transact(() -> SqlReplayCheckpoint.get().plusMillis(1));
    int filesProcessed = 0;
+    int transactionsProcessed = 0;
    // Starting from one millisecond after the last file we processed, search for and import files
    // one hour at a time until we catch up to the current time or we hit the replay timeout (in
    // which case the next run will pick up from where we leave off).
@@ -171,12 +174,12 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
    // We use hour-long batches because GCS supports filename prefix-based searches.
    while (true) {
      if (isAtOrAfter(clock.nowUtc(), replayTimeoutTime)) {
-        return String.format(
-            "Reached max execution time after replaying %d file(s).", filesProcessed);
+        return createResponseString(
+            "Reached max execution time", startTime, filesProcessed, transactionsProcessed);
      }
      if (isBeforeOrAt(clock.nowUtc(), searchStartTime)) {
-        return String.format(
-            "Caught up to current time after replaying %d file(s).", filesProcessed);
+        return createResponseString(
+            "Caught up to current time", startTime, filesProcessed, transactionsProcessed);
      }
      // Search through the end of the hour
      DateTime searchEndTime =
@@ -189,18 +192,32 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
            searchStartTime.toString("yyyy-MM-dd HH"));
      }
      for (BlobInfo file : fileBatch) {
-        processFile(file);
+        transactionsProcessed += processFile(file);
        filesProcessed++;
        if (clock.nowUtc().isAfter(replayTimeoutTime)) {
-          return String.format(
-              "Reached max execution time after replaying %d file(s).", filesProcessed);
+          return createResponseString(
+              "Reached max execution time", startTime, filesProcessed, transactionsProcessed);
        }
      }
      searchStartTime = searchEndTime.plusMillis(1);
    }
  }

-  private void processFile(BlobInfo metadata) {
+  private String createResponseString(
+      String msg, DateTime startTime, int filesProcessed, int transactionsProcessed) {
+    double tps =
+        (double) transactionsProcessed
+            / (double) Seconds.secondsBetween(startTime, clock.nowUtc()).getSeconds();
+    return String.format(
+        "%s after replaying %d file(s) containing %d total transaction(s) (%.2f tx/s).",
+        msg, filesProcessed, transactionsProcessed, tps);
+  }
+
+  /**
+   * Replays the commit logs in the given commit log file and returns the number of transactions
+   * committed.
+   */
+  private int processFile(BlobInfo metadata) {
    try (InputStream input = gcsUtils.openInputStream(metadata.getBlobId())) {
      // Load and process the Datastore transactions one at a time
      ImmutableList<ImmutableList<VersionedEntity>> allTransactions =
@@ -213,6 +230,7 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
      logger.atInfo().log(
          "Replayed %d transactions from commit log file %s with size %d B.",
          allTransactions.size(), metadata.getName(), metadata.getSize());
+      return allTransactions.size();
    } catch (IOException e) {
      throw new RuntimeException(
          "Errored out while replaying commit log file " + metadata.getName(), e);
@@ -242,7 +260,7 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
            .ifPresent(
                sqlEntity -> {
                  sqlEntity.beforeSqlSaveOnReplay();
-                  jpaTm().put(sqlEntity);
+                  jpaTm().putIgnoringReadOnly(sqlEntity);
                });
      } else {
        // this should never happen, but we shouldn't fail on it
@@ -275,7 +293,7 @@ public class ReplayCommitLogsToSqlAction implements Runnable {
          && !DatastoreOnlyEntity.class.isAssignableFrom(entityClass)
          && entityClass.getAnnotation(javax.persistence.Entity.class) != null) {
        ReplaySpecializer.beforeSqlDelete(entityVKey);
-        jpaTm().delete(entityVKey);
+        jpaTm().deleteIgnoringReadOnly(entityVKey);
      }
    } catch (Throwable t) {
      logger.atSevere().log("Error when deleting key %s", entityVKey);
@@ -28,10 +28,10 @@ import com.googlecode.objectify.Key;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.model.EppResource;
 import google.registry.model.ImmutableObject;
+import google.registry.model.domain.RegistryLock;
 import google.registry.model.eppcommon.Trid;
 import google.registry.model.host.HostResource;
 import google.registry.persistence.VKey;
-import google.registry.schema.domain.RegistryLock;
 import google.registry.util.AppEngineServiceUtils;
 import google.registry.util.Retrier;
 import javax.inject.Inject;
@@ -21,8 +21,9 @@ import static google.registry.config.RegistryEnvironment.PRODUCTION;
 import static google.registry.mapreduce.MapreduceRunner.PARAM_DRY_RUN;
 import static google.registry.model.ResourceTransferUtils.updateForeignKeyIndexDeletionTime;
 import static google.registry.model.ofy.ObjectifyService.auditedOfy;
-import static google.registry.model.registry.Registries.getTldsOfType;
 import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
+import static google.registry.model.tld.Registries.getTldsOfType;
+import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.request.Action.Method.POST;
 import static google.registry.request.RequestParameters.PARAM_TLDS;
@@ -42,27 +43,31 @@ import google.registry.config.RegistryEnvironment;
 import google.registry.dns.DnsQueue;
 import google.registry.mapreduce.MapreduceRunner;
 import google.registry.mapreduce.inputs.EppResourceInputs;
+import google.registry.model.CreateAutoTimestamp;
 import google.registry.model.EppResourceUtils;
 import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.DomainHistory;
 import google.registry.model.index.EppResourceIndex;
 import google.registry.model.index.ForeignKeyIndex;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.Registry.TldType;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.Registry.TldType;
 import google.registry.request.Action;
 import google.registry.request.Parameter;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
 import java.util.List;
+import java.util.concurrent.atomic.AtomicInteger;
 import javax.inject.Inject;
+import org.hibernate.CacheMode;
+import org.hibernate.ScrollMode;
+import org.hibernate.ScrollableResults;
+import org.hibernate.query.Query;
 import org.joda.time.DateTime;
 import org.joda.time.Duration;

 /**
- * Deletes all prober DomainBases and their subordinate history entries, poll messages, and
- * billing events, along with their ForeignKeyDomainIndex and EppResourceIndex entities.
- *
- * <p>See: https://www.youtube.com/watch?v=xuuv0syoHnM
+ * Deletes all prober DomainBases and their subordinate history entries, poll messages, and billing
+ * events, along with their ForeignKeyDomainIndex and EppResourceIndex entities.
 */
@Action(
    service = Action.Service.BACKEND,
@@ -73,10 +78,51 @@ public class DeleteProberDataAction implements Runnable {

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

+  /**
+   * The maximum amount of time we allow a prober domain to be in use.
+   *
+   * <p>In practice, the prober's connection will time out well before this duration. This includes
+   * a decent buffer.
+   */
+  private static final Duration DOMAIN_USED_DURATION = Duration.standardHours(1);
+
+  /**
+   * The minimum amount of time we want a domain to be "soft deleted".
+   *
+   * <p>The domain has to remain soft deleted for at least enough time for the DNS task to run and
+   * remove it from DNS itself. This is probably on the order of minutes.
+   */
+  private static final Duration SOFT_DELETE_DELAY = Duration.standardHours(1);
+
+  private static final DnsQueue dnsQueue = DnsQueue.create();
+
+  // Domains to delete must:
+  // 1. Be in one of the prober TLDs
+  // 2. Not be a nic domain
+  // 3. Have no subordinate hosts
+  // 4. Not still be used (within an hour of creation time)
+  // 5. Either be active (creationTime <= now < deletionTime) or have been deleted a while ago (this
+  //    prevents accidental double-map with the same key from immediately deleting active domains)
+  //
+  // Note: creationTime must be compared to a Java object (CreateAutoTimestamp) but deletionTime can
+  // be compared directly to the SQL timestamp (it's a DateTime)
+  private static final String DOMAIN_QUERY_STRING =
+      "FROM Domain d WHERE d.tld IN :tlds AND d.fullyQualifiedDomainName NOT LIKE 'nic.%' AND"
+          + " (d.subordinateHosts IS EMPTY OR d.subordinateHosts IS NULL) AND d.creationTime <"
+          + " :creationTimeCutoff AND ((d.creationTime <= :nowAutoTimestamp AND d.deletionTime >"
+          + " current_timestamp()) OR d.deletionTime < :nowMinusSoftDeleteDelay) ORDER BY d.repoId";
+
+  /** Number of domains to retrieve and delete per SQL transaction. */
+  private static final int BATCH_SIZE = 1000;
+
  @Inject @Parameter(PARAM_DRY_RUN) boolean isDryRun;
  /** List of TLDs to work on. If empty - will work on all TLDs that end with .test. */
  @Inject @Parameter(PARAM_TLDS) ImmutableSet<String> tlds;
-  @Inject @Config("registryAdminClientId") String registryAdminClientId;
+
+  @Inject
+  @Config("registryAdminClientId")
+  String registryAdminRegistrarId;
+
  @Inject MapreduceRunner mrRunner;
  @Inject Response response;
  @Inject DeleteProberDataAction() {}
@@ -84,25 +130,14 @@ public class DeleteProberDataAction implements Runnable {
  @Override
  public void run() {
    checkState(
-        !Strings.isNullOrEmpty(registryAdminClientId),
+        !Strings.isNullOrEmpty(registryAdminRegistrarId),
        "Registry admin client ID must be configured for prober data deletion to work");
-    mrRunner
-        .setJobName("Delete prober data")
-        .setModuleName("backend")
-        .runMapOnly(
-            new DeleteProberDataMapper(getProberRoidSuffixes(), isDryRun, registryAdminClientId),
-            ImmutableList.of(EppResourceInputs.createKeyInput(DomainBase.class)))
-        .sendLinkToMapreduceConsole(response);
-  }
-
-  private ImmutableSet<String> getProberRoidSuffixes() {
    checkArgument(
        !PRODUCTION.equals(RegistryEnvironment.get())
            || tlds.stream().allMatch(tld -> tld.endsWith(".test")),
        "On production, can only work on TLDs that end with .test");
    ImmutableSet<String> deletableTlds =
-        getTldsOfType(TldType.TEST)
-            .stream()
+        getTldsOfType(TldType.TEST).stream()
            .filter(tld -> tlds.isEmpty() ? tld.endsWith(".test") : tlds.contains(tld))
            .collect(toImmutableSet());
    checkArgument(
@@ -110,10 +145,161 @@ public class DeleteProberDataAction implements Runnable {
        "If tlds are given, they must all exist and be TEST tlds. Given: %s, not found: %s",
        tlds,
        Sets.difference(tlds, deletableTlds));
-    return deletableTlds
-        .stream()
-        .map(tld -> Registry.get(tld).getRoidSuffix())
-        .collect(toImmutableSet());
+    ImmutableSet<String> proberRoidSuffixes =
+        deletableTlds.stream()
+            .map(tld -> Registry.get(tld).getRoidSuffix())
+            .collect(toImmutableSet());
+    if (tm().isOfy()) {
+      mrRunner
+          .setJobName("Delete prober data")
+          .setModuleName("backend")
+          .runMapOnly(
+              new DeleteProberDataMapper(proberRoidSuffixes, isDryRun, registryAdminRegistrarId),
+              ImmutableList.of(EppResourceInputs.createKeyInput(DomainBase.class)))
+          .sendLinkToMapreduceConsole(response);
+    } else {
+      runSqlJob(deletableTlds);
+    }
+  }
+
+  private void runSqlJob(ImmutableSet<String> deletableTlds) {
+    AtomicInteger softDeletedDomains = new AtomicInteger();
+    AtomicInteger hardDeletedDomains = new AtomicInteger();
+    jpaTm().transact(() -> processDomains(deletableTlds, softDeletedDomains, hardDeletedDomains));
+    logger.atInfo().log(
+        "%s %d domains.",
+        isDryRun ? "Would have soft-deleted" : "Soft-deleted", softDeletedDomains.get());
+    logger.atInfo().log(
+        "%s %d domains.",
+        isDryRun ? "Would have hard-deleted" : "Hard-deleted", hardDeletedDomains.get());
+  }
+
+  private void processDomains(
+      ImmutableSet<String> deletableTlds,
+      AtomicInteger softDeletedDomains,
+      AtomicInteger hardDeletedDomains) {
+    DateTime now = tm().getTransactionTime();
+    // Scroll through domains, soft-deleting as necessary (very few will be soft-deleted) and
+    // keeping track of which domains to hard-delete (there can be many, so we batch them up)
+    ScrollableResults scrollableResult =
+        jpaTm()
+            .query(DOMAIN_QUERY_STRING, DomainBase.class)
+            .setParameter("tlds", deletableTlds)
+            .setParameter(
+                "creationTimeCutoff", CreateAutoTimestamp.create(now.minus(DOMAIN_USED_DURATION)))
+            .setParameter("nowMinusSoftDeleteDelay", now.minus(SOFT_DELETE_DELAY))
+            .setParameter("nowAutoTimestamp", CreateAutoTimestamp.create(now))
+            .unwrap(Query.class)
+            .setCacheMode(CacheMode.IGNORE)
+            .scroll(ScrollMode.FORWARD_ONLY);
+    ImmutableList.Builder<String> domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
+    ImmutableList.Builder<String> hostNamesToHardDelete = new ImmutableList.Builder<>();
+    for (int i = 1; scrollableResult.next(); i = (i + 1) % BATCH_SIZE) {
+      DomainBase domain = (DomainBase) scrollableResult.get(0);
+      processDomain(
+          domain,
+          domainRepoIdsToHardDelete,
+          hostNamesToHardDelete,
+          softDeletedDomains,
+          hardDeletedDomains);
+      // Batch the deletion and DB flush + session clearing so we don't OOM
+      if (i == 0) {
+        hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
+        domainRepoIdsToHardDelete = new ImmutableList.Builder<>();
+        hostNamesToHardDelete = new ImmutableList.Builder<>();
+        jpaTm().getEntityManager().flush();
+        jpaTm().getEntityManager().clear();
+      }
+    }
+    // process the remainder
+    hardDeleteDomainsAndHosts(domainRepoIdsToHardDelete.build(), hostNamesToHardDelete.build());
+  }
+
+  private void processDomain(
+      DomainBase domain,
+      ImmutableList.Builder<String> domainRepoIdsToHardDelete,
+      ImmutableList.Builder<String> hostNamesToHardDelete,
+      AtomicInteger softDeletedDomains,
+      AtomicInteger hardDeletedDomains) {
+    // If the domain is still active, that means that the prober encountered a failure and did not
+    // successfully soft-delete the domain (thus leaving its DNS entry published). We soft-delete
+    // it now so that the DNS entry can be handled. The domain will then be hard-deleted the next
+    // time the job is run.
+    if (EppResourceUtils.isActive(domain, tm().getTransactionTime())) {
+      if (isDryRun) {
+        logger.atInfo().log(
+            "Would soft-delete the active domain: %s (%s)",
+            domain.getDomainName(), domain.getRepoId());
+      } else {
+        softDeleteDomain(domain, registryAdminRegistrarId, dnsQueue);
+      }
+      softDeletedDomains.incrementAndGet();
+    } else {
+      if (isDryRun) {
+        logger.atInfo().log(
+            "Would hard-delete the non-active domain: %s (%s) and its dependents",
+            domain.getDomainName(), domain.getRepoId());
+      } else {
+        domainRepoIdsToHardDelete.add(domain.getRepoId());
+        hostNamesToHardDelete.addAll(domain.getSubordinateHosts());
+      }
+      hardDeletedDomains.incrementAndGet();
+    }
+  }
+
+  private void hardDeleteDomainsAndHosts(
+      ImmutableList<String> domainRepoIds, ImmutableList<String> hostNames) {
+    jpaTm()
+        .query("DELETE FROM Host WHERE fullyQualifiedHostName IN :hostNames")
+        .setParameter("hostNames", hostNames)
+        .executeUpdate();
+    jpaTm()
+        .query("DELETE FROM BillingEvent WHERE domainRepoId IN :repoIds")
+        .setParameter("repoIds", domainRepoIds)
+        .executeUpdate();
+    jpaTm()
+        .query("DELETE FROM BillingRecurrence WHERE domainRepoId IN :repoIds")
+        .setParameter("repoIds", domainRepoIds)
+        .executeUpdate();
+    jpaTm()
+        .query("DELETE FROM BillingCancellation WHERE domainRepoId IN :repoIds")
+        .setParameter("repoIds", domainRepoIds)
+        .executeUpdate();
+    jpaTm()
+        .query("DELETE FROM DomainHistory WHERE domainRepoId IN :repoIds")
+        .setParameter("repoIds", domainRepoIds)
+        .executeUpdate();
+    jpaTm()
+        .query("DELETE FROM PollMessage WHERE domainRepoId IN :repoIds")
+        .setParameter("repoIds", domainRepoIds)
+        .executeUpdate();
+    jpaTm()
+        .query("DELETE FROM Domain WHERE repoId IN :repoIds")
+        .setParameter("repoIds", domainRepoIds)
+        .executeUpdate();
+  }
+
+  // Take a DNS queue + admin registrar id as input so that it can be called from the mapper as well
+  private static void softDeleteDomain(
+      DomainBase domain, String registryAdminRegistrarId, DnsQueue localDnsQueue) {
+    DomainBase deletedDomain =
+        domain.asBuilder().setDeletionTime(tm().getTransactionTime()).setStatusValues(null).build();
+    DomainHistory historyEntry =
+        new DomainHistory.Builder()
+            .setDomain(domain)
+            .setType(DOMAIN_DELETE)
+            .setModificationTime(tm().getTransactionTime())
+            .setBySuperuser(true)
+            .setReason("Deletion of prober data")
+            .setClientId(registryAdminRegistrarId)
+            .build();
+    // Note that we don't bother handling grace periods, billing events, pending transfers, poll
+    // messages, or auto-renews because those will all be hard-deleted the next time the job runs
+    // anyway.
+    tm().putAllWithoutBackup(ImmutableList.of(deletedDomain, historyEntry));
+    // updating foreign keys is a no-op in SQL
+    updateForeignKeyIndexDeletionTime(deletedDomain);
+    localDnsQueue.addDomainRefreshTask(deletedDomain.getDomainName());
  }

  /** Provides the map method that runs for each existing DomainBase entity. */
@@ -122,32 +308,17 @@ public class DeleteProberDataAction implements Runnable {
    private static final DnsQueue dnsQueue = DnsQueue.create();
    private static final long serialVersionUID = -7724537393697576369L;

-    /**
-     * The maximum amount of time we allow a prober domain to be in use.
-     *
-     * In practice, the prober's connection will time out well before this duration. This includes a
-     * decent buffer.
-     *
-     */
-    private static final Duration DOMAIN_USED_DURATION = Duration.standardHours(1);
-
-    /**
-     * The minimum amount of time we want a domain to be "soft deleted".
-     *
-     * The domain has to remain soft deleted for at least enough time for the DNS task to run and
-     * remove it from DNS itself. This is probably on the order of minutes.
-     */
-    private static final Duration SOFT_DELETE_DELAY = Duration.standardHours(1);
-
    private final ImmutableSet<String> proberRoidSuffixes;
    private final Boolean isDryRun;
-    private final String registryAdminClientId;
+    private final String registryAdminRegistrarId;

    public DeleteProberDataMapper(
-        ImmutableSet<String> proberRoidSuffixes, Boolean isDryRun, String registryAdminClientId) {
+        ImmutableSet<String> proberRoidSuffixes,
+        Boolean isDryRun,
+        String registryAdminRegistrarId) {
      this.proberRoidSuffixes = proberRoidSuffixes;
      this.isDryRun = isDryRun;
-      this.registryAdminClientId = registryAdminClientId;
+      this.registryAdminRegistrarId = registryAdminRegistrarId;
    }

    @Override
@@ -203,7 +374,7 @@ public class DeleteProberDataAction implements Runnable {
          logger.atInfo().log(
              "Would soft-delete the active domain: %s (%s)", domainName, domainKey);
        } else {
-          softDeleteDomain(domain);
+          tm().transact(() -> softDeleteDomain(domain, registryAdminRegistrarId, dnsQueue));
        }
        getContext().incrementCounter("domains soft-deleted");
        return;
@@ -223,8 +394,7 @@ public class DeleteProberDataAction implements Runnable {
          tm().transact(
                  () -> {
                    // This ancestor query selects all descendant HistoryEntries, BillingEvents,
-                    // PollMessages,
-                    // and TLD-specific entities, as well as the domain itself.
+                    // PollMessages, and TLD-specific entities, as well as the domain itself.
                    List<Key<Object>> domainAndDependentKeys =
                        auditedOfy().load().ancestor(domainKey).keys().list();
                    ImmutableSet<Key<?>> allKeys =
@@ -243,32 +413,5 @@ public class DeleteProberDataAction implements Runnable {
      getContext().incrementCounter("domains hard-deleted");
      getContext().incrementCounter("total entities hard-deleted", entitiesDeleted);
    }
-
-    private void softDeleteDomain(final DomainBase domain) {
-      tm().transactNew(
-              () -> {
-                DomainBase deletedDomain =
-                    domain
-                        .asBuilder()
-                        .setDeletionTime(tm().getTransactionTime())
-                        .setStatusValues(null)
-                        .build();
-                DomainHistory historyEntry =
-                    new DomainHistory.Builder()
-                        .setDomain(domain)
-                        .setType(DOMAIN_DELETE)
-                        .setModificationTime(tm().getTransactionTime())
-                        .setBySuperuser(true)
-                        .setReason("Deletion of prober data")
-                        .setClientId(registryAdminClientId)
-                        .build();
-                // Note that we don't bother handling grace periods, billing events, pending
-                // transfers, poll messages, or auto-renews because these will all be hard-deleted
-                // the next time the mapreduce runs anyway.
-                tm().putAll(deletedDomain, historyEntry);
-                updateForeignKeyIndexDeletionTime(deletedDomain);
-                dnsQueue.addDomainRefreshTask(deletedDomain.getDomainName());
-              });
-    }
  }
 }
@@ -52,9 +52,9 @@ import google.registry.model.common.Cursor;
 import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.DomainHistory;
 import google.registry.model.domain.Period;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
+import google.registry.model.tld.Registry;
 import google.registry.persistence.VKey;
 import google.registry.request.Action;
 import google.registry.request.Parameter;
@@ -28,16 +28,16 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.model.domain.DomainBase;
+import google.registry.model.domain.RegistryLock;
 import google.registry.model.eppcommon.StatusValue;
 import google.registry.model.registrar.Registrar;
 import google.registry.model.registrar.RegistrarContact;
-import google.registry.model.registry.RegistryLockDao;
+import google.registry.model.tld.RegistryLockDao;
 import google.registry.persistence.VKey;
 import google.registry.request.Action;
 import google.registry.request.Parameter;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
-import google.registry.schema.domain.RegistryLock;
 import google.registry.tools.DomainLockUtils;
 import google.registry.util.DateTimeUtils;
 import google.registry.util.EmailMessage;
@@ -0,0 +1,342 @@
+// Copyright 2021 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.batch;
+
+import static com.google.common.collect.ImmutableList.toImmutableList;
+import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
+import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
+import static org.apache.http.HttpStatus.SC_INTERNAL_SERVER_ERROR;
+import static org.apache.http.HttpStatus.SC_OK;
+import static org.joda.time.DateTimeZone.UTC;
+
+import com.google.auto.value.AutoValue;
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.ImmutableSortedSet;
+import com.google.common.collect.Streams;
+import com.google.common.flogger.FluentLogger;
+import com.google.common.net.MediaType;
+import google.registry.config.RegistryConfig.Config;
+import google.registry.flows.certs.CertificateChecker;
+import google.registry.model.registrar.Registrar;
+import google.registry.model.registrar.RegistrarContact;
+import google.registry.model.registrar.RegistrarContact.Type;
+import google.registry.request.Action;
+import google.registry.request.Response;
+import google.registry.request.auth.Auth;
+import google.registry.util.EmailMessage;
+import google.registry.util.SendEmailService;
+import java.util.Date;
+import java.util.Optional;
+import javax.inject.Inject;
+import javax.mail.internet.AddressException;
+import javax.mail.internet.InternetAddress;
+import org.joda.time.DateTime;
+import org.joda.time.Duration;
+import org.joda.time.format.DateTimeFormat;
+import org.joda.time.format.DateTimeFormatter;
+
+/** An action that sends notification emails to registrars whose certificates are expiring soon. */
+@Action(
+    service = Action.Service.BACKEND,
+    path = SendExpiringCertificateNotificationEmailAction.PATH,
+    auth = Auth.AUTH_INTERNAL_OR_ADMIN)
+public class SendExpiringCertificateNotificationEmailAction implements Runnable {
+  public static final String PATH = "/_dr/task/sendExpiringCertificateNotificationEmail";
+  /**
+   * Used as an offset when storing the last notification email sent date.
+   *
+   * <p>This is used to handle edges cases when the update happens in between the day switch. For
+   * instance,if the job starts at 2:00 am every day and it finishes at 2:03 of the same day, then
+   * next day at 2am, the date difference will be less than a day, which will lead to the date
+   * difference between two successive email sent date being the expected email interval days + 1;
+   */
+  protected static final Duration UPDATE_TIME_OFFSET = Duration.standardMinutes(10);
+
+  private static final FluentLogger logger = FluentLogger.forEnclosingClass();
+  private static final DateTimeFormatter DATE_FORMATTER = DateTimeFormat.forPattern("yyyy-MM-dd");
+
+  private final CertificateChecker certificateChecker;
+  private final String expirationWarningEmailBodyText;
+  private final SendEmailService sendEmailService;
+  private final String expirationWarningEmailSubjectText;
+  private final InternetAddress gSuiteOutgoingEmailAddress;
+  private final Response response;
+
+  @Inject
+  public SendExpiringCertificateNotificationEmailAction(
+      @Config("expirationWarningEmailBodyText") String expirationWarningEmailBodyText,
+      @Config("expirationWarningEmailSubjectText") String expirationWarningEmailSubjectText,
+      @Config("gSuiteOutgoingEmailAddress") InternetAddress gSuiteOutgoingEmailAddress,
+      SendEmailService sendEmailService,
+      CertificateChecker certificateChecker,
+      Response response) {
+    this.certificateChecker = certificateChecker;
+    this.expirationWarningEmailSubjectText = expirationWarningEmailSubjectText;
+    this.sendEmailService = sendEmailService;
+    this.gSuiteOutgoingEmailAddress = gSuiteOutgoingEmailAddress;
+    this.expirationWarningEmailBodyText = expirationWarningEmailBodyText;
+    this.response = response;
+  }
+
+  @Override
+  public void run() {
+    response.setContentType(MediaType.PLAIN_TEXT_UTF_8);
+    try {
+      sendNotificationEmails();
+      response.setStatus(SC_OK);
+    } catch (Exception e) {
+      logger.atWarning().withCause(e).log(
+          "Exception thrown when sending expiring certificate notification emails.");
+      response.setStatus(SC_INTERNAL_SERVER_ERROR);
+      response.setPayload(String.format("Exception thrown with cause: %s", e));
+    }
+  }
+
+  /**
+   * Returns a list of registrars that should receive expiring notification emails. There are two
+   * certificates that should be considered (the main certificate and failOver certificate). The
+   * registrars should receive notifications if one of the certificate checks returns true.
+   */
+  @VisibleForTesting
+  ImmutableList<RegistrarInfo> getRegistrarsWithExpiringCertificates() {
+    logger.atInfo().log(
+        "Getting a list of registrars that should receive expiring notification emails.");
+    return Streams.stream(Registrar.loadAllCached())
+        .map(
+            registrar ->
+                RegistrarInfo.create(
+                    registrar,
+                    registrar.getClientCertificate().isPresent()
+                        && certificateChecker.shouldReceiveExpiringNotification(
+                            registrar.getLastExpiringCertNotificationSentDate(),
+                            registrar.getClientCertificate().get()),
+                    registrar.getFailoverClientCertificate().isPresent()
+                        && certificateChecker.shouldReceiveExpiringNotification(
+                            registrar.getLastExpiringFailoverCertNotificationSentDate(),
+                            registrar.getFailoverClientCertificate().get())))
+        .filter(
+            registrarInfo ->
+                registrarInfo.isCertExpiring() || registrarInfo.isFailOverCertExpiring())
+        .collect(toImmutableList());
+  }
+
+  /**
+   * Sends a notification email to the registrar regarding the expiring certificate and returns true
+   * if it's sent successfully.
+   */
+  @VisibleForTesting
+  boolean sendNotificationEmail(
+      Registrar registrar,
+      DateTime lastExpiringCertNotificationSentDate,
+      CertificateType certificateType,
+      Optional<String> certificate) {
+    if (!certificate.isPresent()
+        || !certificateChecker.shouldReceiveExpiringNotification(
+            lastExpiringCertNotificationSentDate, certificate.get())) {
+      return false;
+    }
+    try {
+      ImmutableSet<InternetAddress> recipients = getEmailAddresses(registrar, Type.TECH);
+      Date expirationDate = certificateChecker.getCertificate(certificate.get()).getNotAfter();
+      logger.atInfo().log(
+          "Registrar %s should receive an email that its %s SSL certificate will expire on %s.",
+          registrar.getRegistrarName(),
+          certificateType.getDisplayName(),
+          expirationDate.toString());
+      if (recipients.isEmpty()) {
+        logger.atWarning().log(
+            "Registrar %s contains no email addresses to receive notification email.",
+            registrar.getRegistrarName());
+        return false;
+      }
+      sendEmailService.sendEmail(
+          EmailMessage.newBuilder()
+              .setFrom(gSuiteOutgoingEmailAddress)
+              .setSubject(expirationWarningEmailSubjectText)
+              .setBody(
+                  getEmailBody(
+                      registrar.getRegistrarName(),
+                      certificateType,
+                      expirationDate,
+                      registrar.getClientId()))
+              .setRecipients(recipients)
+              .setCcs(getEmailAddresses(registrar, Type.ADMIN))
+              .build());
+      /*
+       * A duration time offset is used here to ensure that date comparison between two
+       * successive dates is always greater than 1 day. This date is set as last updated date,
+       * for applicable certificate.
+       */
+      updateLastNotificationSentDate(
+          registrar,
+          DateTime.now(UTC).minusMinutes((int) UPDATE_TIME_OFFSET.getStandardMinutes()),
+          certificateType);
+      return true;
+    } catch (Exception e) {
+      throw new RuntimeException(
+          String.format(
+              "Failed to send expiring certificate notification email to registrar %s.",
+              registrar.getRegistrarName()));
+    }
+  }
+
+  /** Updates the last notification sent date in database. */
+  @VisibleForTesting
+  void updateLastNotificationSentDate(
+      Registrar registrar, DateTime now, CertificateType certificateType) {
+    try {
+      tm().transact(
+              () -> {
+                Registrar.Builder newRegistrar = tm().loadByEntity(registrar).asBuilder();
+                switch (certificateType) {
+                  case PRIMARY:
+                    newRegistrar.setLastExpiringCertNotificationSentDate(now);
+                    tm().put(newRegistrar.build());
+                    logger.atInfo().log(
+                        "Updated last notification email sent date to %s for %s certificate of "
+                            + "registrar %s.",
+                        DATE_FORMATTER.print(now),
+                        certificateType.getDisplayName(),
+                        registrar.getRegistrarName());
+                    break;
+                  case FAILOVER:
+                    newRegistrar.setLastExpiringFailoverCertNotificationSentDate(now);
+                    tm().put(newRegistrar.build());
+                    logger.atInfo().log(
+                        "Updated last notification email sent date to %s for %s certificate of "
+                            + "registrar %s.",
+                        DATE_FORMATTER.print(now),
+                        certificateType.getDisplayName(),
+                        registrar.getRegistrarName());
+                    break;
+                  default:
+                    throw new IllegalArgumentException(
+                        String.format(
+                            "Unsupported certificate type: %s being passed in when updating "
+                                + "the last notification sent date to registrar %s.",
+                            certificateType.toString(), registrar.getRegistrarName()));
+                }
+              });
+    } catch (Exception e) {
+      throw new RuntimeException(
+          String.format(
+              "Failed to update the last notification sent date to Registrar %s for the %s "
+                  + "certificate.",
+              registrar.getRegistrarName(), certificateType.getDisplayName()));
+    }
+  }
+
+  /** Sends notification emails to registrars with expiring certificates. */
+  @VisibleForTesting
+  int sendNotificationEmails() {
+    int emailsSent = 0;
+    for (RegistrarInfo registrarInfo : getRegistrarsWithExpiringCertificates()) {
+      Registrar registrar = registrarInfo.registrar();
+      if (registrarInfo.isCertExpiring()) {
+        sendNotificationEmail(
+            registrar,
+            registrar.getLastExpiringCertNotificationSentDate(),
+            CertificateType.PRIMARY,
+            registrar.getClientCertificate());
+        emailsSent++;
+      }
+      if (registrarInfo.isFailOverCertExpiring()) {
+        sendNotificationEmail(
+            registrar,
+            registrar.getLastExpiringFailoverCertNotificationSentDate(),
+            CertificateType.FAILOVER,
+            registrar.getFailoverClientCertificate());
+        emailsSent++;
+      }
+    }
+    logger.atInfo().log(
+        "Attempted to send %d expiring certificate notification emails.", emailsSent);
+    return emailsSent;
+  }
+
+  /** Returns a list of email addresses of the registrar that should receive a notification email */
+  @VisibleForTesting
+  ImmutableSet<InternetAddress> getEmailAddresses(Registrar registrar, Type contactType) {
+    ImmutableSortedSet<RegistrarContact> contacts = registrar.getContactsOfType(contactType);
+    ImmutableSet.Builder<InternetAddress> recipientEmails = new ImmutableSet.Builder<>();
+    for (RegistrarContact contact : contacts) {
+      try {
+        recipientEmails.add(new InternetAddress(contact.getEmailAddress()));
+      } catch (AddressException e) {
+        logger.atWarning().withCause(e).log(
+            "Registrar Contact email address %s of Registrar %s is invalid; skipping.",
+            contact.getEmailAddress(), registrar.getRegistrarName());
+      }
+    }
+    return recipientEmails.build();
+  }
+
+  /**
+   * Generates email content by taking registrar name, certificate type and expiration date as
+   * parameters.
+   */
+  @VisibleForTesting
+  @SuppressWarnings("lgtm[java/dereferenced-value-may-be-null]")
+  String getEmailBody(
+      String registrarName, CertificateType type, Date expirationDate, String registrarId) {
+    checkArgumentNotNull(expirationDate, "Expiration date cannot be null");
+    checkArgumentNotNull(type, "Certificate type cannot be null");
+    checkArgumentNotNull(registrarId, "Registrar Id cannot be null");
+    return String.format(
+        expirationWarningEmailBodyText,
+        registrarName,
+        type.getDisplayName(),
+        DATE_FORMATTER.print(new DateTime(expirationDate)),
+        registrarId);
+  }
+
+  /**
+   * Certificate types for X509Certificate.
+   *
+   * <p><b>Note:</b> These types are only used to indicate the type of expiring certificate in
+   * notification emails.
+   */
+  protected enum CertificateType {
+    PRIMARY("primary"),
+    FAILOVER("fail-over");
+
+    private final String displayName;
+
+    CertificateType(String displayName) {
+      this.displayName = displayName;
+    }
+
+    public String getDisplayName() {
+      return displayName;
+    }
+  }
+
+  @AutoValue
+  public abstract static class RegistrarInfo {
+    static RegistrarInfo create(
+        Registrar registrar, boolean isCertExpiring, boolean isFailOverCertExpiring) {
+      return new AutoValue_SendExpiringCertificateNotificationEmailAction_RegistrarInfo(
+          registrar, isCertExpiring, isFailOverCertExpiring);
+    }
+
+    public abstract Registrar registrar();
+
+    public abstract boolean isCertExpiring();
+
+    public abstract boolean isFailOverCertExpiring();
+  }
+}
@@ -23,6 +23,7 @@ import com.google.common.collect.Streams;
 import google.registry.backup.AppEngineEnvironment;
 import google.registry.beam.common.RegistryQuery.CriteriaQuerySupplier;
 import google.registry.model.ofy.ObjectifyService;
+import google.registry.model.replay.SqlEntity;
 import google.registry.persistence.transaction.JpaTransactionManager;
 import google.registry.persistence.transaction.TransactionManagerFactory;
 import java.io.Serializable;
@@ -358,17 +359,17 @@ public final class RegistryJpaIO {
    @ProcessElement
    public void processElement(@Element KV<ShardedKey<Integer>, Iterable<T>> kv) {
      try (AppEngineEnvironment env = new AppEngineEnvironment()) {
-        ImmutableList<Object> ofyEntities =
+        ImmutableList<Object> entities =
            Streams.stream(kv.getValue())
                .map(this.jpaConverter::apply)
                // TODO(b/177340730): post migration delete the line below.
                .filter(Objects::nonNull)
                .collect(ImmutableList.toImmutableList());
        try {
-          jpaTm().transact(() -> jpaTm().putAll(ofyEntities));
-          counter.inc(ofyEntities.size());
+          jpaTm().transact(() -> jpaTm().putAll(entities));
+          counter.inc(entities.size());
        } catch (RuntimeException e) {
-          processSingly(ofyEntities);
+          processSingly(entities);
        }
      }
    }
@@ -377,19 +378,22 @@ public final class RegistryJpaIO {
     * Writes entities in a failed batch one by one to identify the first bad entity and throws a
     * {@link RuntimeException} on it.
     */
-    private void processSingly(ImmutableList<Object> ofyEntities) {
-      for (Object ofyEntity : ofyEntities) {
+    private void processSingly(ImmutableList<Object> entities) {
+      for (Object entity : entities) {
        try {
-          jpaTm().transact(() -> jpaTm().put(ofyEntity));
+          jpaTm().transact(() -> jpaTm().put(entity));
          counter.inc();
        } catch (RuntimeException e) {
-          throw new RuntimeException(toOfyKey(ofyEntity).toString(), e);
+          throw new RuntimeException(toEntityKeyString(entity), e);
        }
      }
    }

-    private com.googlecode.objectify.Key<?> toOfyKey(Object ofyEntity) {
-      return com.googlecode.objectify.Key.create(ofyEntity);
+    private String toEntityKeyString(Object entity) {
+      if (entity instanceof SqlEntity) {
+        return ((SqlEntity) entity).getPrimaryKeyString();
+      }
+      return "Non-SqlEntity: " + String.valueOf(entity);
    }
  }
 }
@@ -32,8 +32,8 @@ import google.registry.model.host.HostResource;
 import google.registry.model.poll.PollMessage;
 import google.registry.model.registrar.Registrar;
 import google.registry.model.registrar.RegistrarContact;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.HistoryEntry;
+import google.registry.model.tld.Registry;
 import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
 import java.io.Serializable;
 import java.util.Collection;
@@ -39,9 +39,9 @@ import google.registry.backup.VersionedEntity;
 import google.registry.model.billing.BillingEvent.Flag;
 import google.registry.model.billing.BillingEvent.Reason;
 import google.registry.model.domain.DomainBase;
+import google.registry.model.replay.DatastoreAndSqlEntity;
+import google.registry.model.replay.SqlEntity;
 import google.registry.model.reporting.HistoryEntry;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
-import google.registry.schema.replay.SqlEntity;
 import google.registry.tools.LevelDbLogReader;
 import java.io.Serializable;
 import java.util.Collection;
@@ -30,7 +30,7 @@ import google.registry.model.common.Cursor;
 import google.registry.model.rde.RdeMode;
 import google.registry.model.rde.RdeNamingUtils;
 import google.registry.model.rde.RdeRevision;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import google.registry.rde.DepositFragment;
 import google.registry.rde.Ghostryde;
 import google.registry.rde.PendingDeposit;
@@ -0,0 +1,65 @@
+// Copyright 2021 The Nomulus Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package google.registry.config;
+
+import com.google.api.gax.core.FixedCredentialsProvider;
+import com.google.cloud.tasks.v2.CloudTasksClient;
+import com.google.cloud.tasks.v2.CloudTasksSettings;
+import dagger.Module;
+import dagger.Provides;
+import google.registry.config.CredentialModule.DefaultCredential;
+import google.registry.config.RegistryConfig.Config;
+import google.registry.util.CloudTasksUtils;
+import google.registry.util.GoogleCredentialsBundle;
+import google.registry.util.Retrier;
+import java.io.IOException;
+import javax.inject.Provider;
+import javax.inject.Singleton;
+
+/**
+ * A {@link Module} that provides {@link CloudTasksUtils}.
+ *
+ * <p>The class itself cannot be annotated as {@code Inject} because its requested dependencies use
+ * the {@link Config} qualifier which is not available in the {@code util} package.
+ */
+@Module
+public abstract class CloudTasksUtilsModule {
+
+  @Singleton
+  @Provides
+  public static CloudTasksUtils provideCloudTasksUtils(
+      @Config("projectId") String projectId,
+      @Config("locationId") String locationId,
+      // Use a provider so that we can use try-with-resources with the client, which implements
+      // Autocloseable.
+      Provider<CloudTasksClient> clientProvider,
+      Retrier retrier) {
+    return new CloudTasksUtils(retrier, projectId, locationId, clientProvider);
+  }
+
+  @Provides
+  public static CloudTasksClient provideCloudTasksClient(
+      @DefaultCredential GoogleCredentialsBundle credentials) {
+    try {
+      return CloudTasksClient.create(
+          CloudTasksSettings.newBuilder()
+              .setCredentialsProvider(
+                  FixedCredentialsProvider.create(credentials.getGoogleCredentials()))
+              .build());
+    } catch (IOException e) {
+      throw new RuntimeException(e);
+    }
+  }
+}
@@ -60,7 +60,7 @@ import org.joda.time.Duration;
 * <p>This class does not represent the total configuration of the Nomulus service. It's <b>only
 * meant for settings that need to be configured <i>once</i></b>. Settings which may be subject to
 * change in the future, should instead be retrieved from Datastore. The {@link
- * google.registry.model.registry.Registry Registry} class is one such example of this.
+ * google.registry.model.tld.Registry Registry} class is one such example of this.
 *
 * <p>Note: Only settings that are actually configurable belong in this file. It's not a catch-all
 * for anything widely used throughout the code base.
@@ -105,6 +105,12 @@ public final class RegistryConfig {
      return config.appEngine.projectId;
    }

+    @Provides
+    @Config("locationId")
+    public static String provideLocationId(RegistryConfigSettings config) {
+      return config.appEngine.locationId;
+    }
+
    /**
     * The filename of the logo to be displayed in the header of the registrar console.
     *
@@ -1394,8 +1400,8 @@ public final class RegistryConfig {
  /**
   * Returns the amount of time a domain label list should be cached in memory before expiring.
   *
-   * @see google.registry.model.registry.label.ReservedList
-   * @see google.registry.model.registry.label.PremiumList
+   * @see google.registry.model.tld.label.ReservedList
+   * @see google.registry.model.tld.label.PremiumList
   */
  public static Duration getDomainLabelListCacheDuration() {
    return Duration.standardSeconds(CONFIG_SETTINGS.get().caching.domainLabelCachingSeconds);
@@ -45,6 +45,7 @@ public class RegistryConfigSettings {
  /** Configuration options that apply to the entire App Engine project. */
  public static class AppEngine {
    public String projectId;
+    public String locationId;
    public boolean isLocal;
    public String defaultServiceUrl;
    public String backendServiceUrl;
@@ -8,6 +8,10 @@
 appEngine:
  # Globally unique App Engine project ID
  projectId: registry-project-id
+  # Location of the App engine project, note that us-central1 and europe-west1 are special in that
+  # they are used without the trailing number in App Engine commands and Google Cloud Console.
+  # See: https://cloud.google.com/appengine/docs/locations
+  locationId: registry-location-id

  # whether to use local/test credentials when connecting to the servers
  isLocal: true
@@ -452,12 +456,36 @@ sslCertificateValidation:
  # The minimum number of days between two successive expiring notification emails.
  expirationWarningIntervalDays: 15
  # Text for expiring certificate notification email subject.
-  expirationWarningEmailSubjectText: Certificate Expring Within 30 Days.
+  expirationWarningEmailSubjectText: "[Important] Expiring SSL certificate for Google Registry EPP connection"
  # Text for expiring certificate notification email body that accepts 3 parameters:
  # registrar name, certificate type, and expiration date, respectively.
  expirationWarningEmailBodyText: |
-    Hello Registrar %s,
-       The %s certificate is expiring on %s.
+    Dear %1$s,
+
+    We would like to inform you that your %2$s SSL certificate will expire at %3$s. Please take note that using expired certificates will prevent successful Registry login.
+
+    Kindly update your production account certificate within the support console using the following steps:
+
+      1. Navigate to support.registry.google and login using your %4$s@registry.google credentials.
+          * If this is your first time logging in, you will be prompted to reset your password, so please keep your new password safe.
+          * If you are already logged in with some other Google account(s) but not your %4$s@registry.google account, you need to click on
+          “Add Account” and login using your %4$s@registry.google credentials.
+      2. Select “Settings > Security” from the left navigation bar.
+      3. Click “Edit” on the top left corner.
+      4. Enter your full certificate string (including lines -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----) in the box.
+      5. Click “Save”. If there are validation issues with the form, you will be prompted to fix them and click “Save” again.
+
+    A failover SSL certificate can also be added in order to prevent connection issues once your main certificate expires. Connecting with either of the certificates will work with our production EPP server.
+
+    Further information about our EPP connection requirements can be found in section 9.2 in the updated Technical Guide in your Google Drive folder.
+
+    Note that account certificate changes take a few minutes to become effective and that the existing connections will remain unaffected by the change.
+
+    If you also would like to update your OT&E account certificate, please send an email from your primary or technical contact to registry-support@google.com and include the full certificate string (including lines -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----).
+
+    Regards,
+    Google Registry
+
  # The minimum number of bits an RSA key must contain.
  minimumRsaKeyLength: 2048
  # The ECDSA curves that are allowed for public keys.
@@ -14,14 +14,10 @@

 package google.registry.cron;

-import static com.google.appengine.api.taskqueue.QueueFactory.getQueue;
-import static com.google.appengine.api.taskqueue.TaskOptions.Builder.withUrl;
 import static com.google.common.base.Preconditions.checkArgument;
 import static com.google.common.base.Predicates.in;
 import static com.google.common.base.Predicates.not;
-import static com.google.common.base.Strings.nullToEmpty;
 import static com.google.common.collect.ImmutableSet.toImmutableSet;
-import static com.google.common.collect.Iterables.getFirst;
 import static com.google.common.collect.Multimaps.filterKeys;
 import static com.google.common.net.MediaType.PLAIN_TEXT_UTF_8;
 import static google.registry.cron.CronModule.ENDPOINT_PARAM;
@@ -31,26 +27,29 @@ import static google.registry.cron.CronModule.FOR_EACH_TEST_TLD_PARAM;
 import static google.registry.cron.CronModule.JITTER_SECONDS_PARAM;
 import static google.registry.cron.CronModule.QUEUE_PARAM;
 import static google.registry.cron.CronModule.RUN_IN_EMPTY_PARAM;
-import static google.registry.model.registry.Registries.getTldsOfType;
-import static google.registry.model.registry.Registry.TldType.REAL;
-import static google.registry.model.registry.Registry.TldType.TEST;
+import static google.registry.model.tld.Registries.getTldsOfType;
+import static google.registry.model.tld.Registry.TldType.REAL;
+import static google.registry.model.tld.Registry.TldType.TEST;
 import static java.util.concurrent.TimeUnit.SECONDS;

-import com.google.appengine.api.taskqueue.Queue;
-import com.google.appengine.api.taskqueue.TaskHandle;
-import com.google.appengine.api.taskqueue.TaskOptions;
+import com.google.cloud.tasks.v2.Task;
+import com.google.common.collect.ArrayListMultimap;
 import com.google.common.collect.ImmutableListMultimap;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.Multimap;
 import com.google.common.collect.Streams;
 import com.google.common.flogger.FluentLogger;
+import com.google.protobuf.Timestamp;
 import google.registry.request.Action;
+import google.registry.request.Action.Service;
 import google.registry.request.Parameter;
 import google.registry.request.ParameterMap;
 import google.registry.request.RequestParameters;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
-import google.registry.util.TaskQueueUtils;
+import google.registry.util.Clock;
+import google.registry.util.CloudTasksUtils;
+import java.time.Instant;
 import java.util.Optional;
 import java.util.Random;
 import java.util.stream.Stream;
@@ -105,7 +104,8 @@ public final class TldFanoutAction implements Runnable {

  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

-  @Inject TaskQueueUtils taskQueueUtils;
+  @Inject Clock clock;
+  @Inject CloudTasksUtils cloudTasksUtils;
  @Inject Response response;
  @Inject @Parameter(ENDPOINT_PARAM) String endpoint;
  @Inject @Parameter(QUEUE_PARAM) String queue;
@@ -115,7 +115,9 @@ public final class TldFanoutAction implements Runnable {
  @Inject @Parameter(EXCLUDE_PARAM) ImmutableSet<String> excludes;
  @Inject @Parameter(JITTER_SECONDS_PARAM) Optional<Integer> jitterSeconds;
  @Inject @ParameterMap ImmutableListMultimap<String, String> params;
-  @Inject TldFanoutAction() {}
+
+  @Inject
+  TldFanoutAction() {}

  @Override
  public void run() {
@@ -126,8 +128,7 @@ public final class TldFanoutAction implements Runnable {
        runInEmpty || forEachTestTld || forEachRealTld,
        "At least one of runInEmpty, forEachTestTld, forEachRealTld must be given");
    checkArgument(
-        !(runInEmpty && !excludes.isEmpty()),
-        "Can't specify 'exclude' with 'runInEmpty'");
+        !(runInEmpty && !excludes.isEmpty()), "Can't specify 'exclude' with 'runInEmpty'");
    ImmutableSet<String> tlds =
        runInEmpty
            ? ImmutableSet.of("")
@@ -137,7 +138,6 @@ public final class TldFanoutAction implements Runnable {
                .filter(not(in(excludes)))
                .collect(toImmutableSet());
    Multimap<String, String> flowThruParams = filterKeys(params, not(in(CONTROL_PARAMS)));
-    Queue taskQueue = getQueue(queue);
    StringBuilder outputPayload =
        new StringBuilder(
            String.format("OK: Launched the following %d tasks in queue %s\n", tlds.size(), queue));
@@ -146,33 +146,41 @@ public final class TldFanoutAction implements Runnable {
      logger.atWarning().log("No TLDs to fan-out!");
    }
    for (String tld : tlds) {
-      TaskOptions taskOptions = createTaskOptions(tld, flowThruParams);
-      TaskHandle taskHandle = taskQueueUtils.enqueue(taskQueue, taskOptions);
+      Task task = createTask(tld, flowThruParams);
+      Task createdTask = cloudTasksUtils.enqueue(queue, task);
      outputPayload.append(
          String.format(
              "- Task: '%s', tld: '%s', endpoint: '%s'\n",
-              taskHandle.getName(), tld, taskOptions.getUrl()));
+              createdTask.getName(), tld, createdTask.getAppEngineHttpRequest().getRelativeUri()));
      logger.atInfo().log(
-          "Task: '%s', tld: '%s', endpoint: '%s'", taskHandle.getName(), tld, taskOptions.getUrl());
+          "Task: '%s', tld: '%s', endpoint: '%s'",
+          createdTask.getName(), tld, createdTask.getAppEngineHttpRequest().getRelativeUri());
    }
    response.setContentType(PLAIN_TEXT_UTF_8);
    response.setPayload(outputPayload.toString());
  }

-  private TaskOptions createTaskOptions(String tld, Multimap<String, String> params) {
-    TaskOptions options =
-        withUrl(endpoint)
-            .countdownMillis(
-                jitterSeconds
-                    .map(seconds -> random.nextInt((int) SECONDS.toMillis(seconds)))
-                    .orElse(0));
+  private Task createTask(String tld, Multimap<String, String> params) {
    if (!tld.isEmpty()) {
-      options.param(RequestParameters.PARAM_TLD, tld);
+      params = ArrayListMultimap.create(params);
+      params.put(RequestParameters.PARAM_TLD, tld);
    }
-    for (String param : params.keySet()) {
-      // TaskOptions.param() does not accept null values.
-      options.param(param, nullToEmpty(getFirst(params.get(param), null)));
-    }
-    return options;
+    Instant scheduleTime =
+        Instant.ofEpochMilli(
+            clock
+                .nowUtc()
+                .plusMillis(
+                    jitterSeconds
+                        .map(seconds -> random.nextInt((int) SECONDS.toMillis(seconds)))
+                        .orElse(0))
+                .getMillis());
+    return Task.newBuilder(
+            CloudTasksUtils.createPostTask(endpoint, Service.BACKEND.toString(), params))
+        .setScheduleTime(
+            Timestamp.newBuilder()
+                .setSeconds(scheduleTime.getEpochSecond())
+                .setNanos(scheduleTime.getNano())
+                .build())
+        .build();
  }
 }
@@ -20,7 +20,7 @@ import static google.registry.dns.DnsConstants.DNS_PULL_QUEUE_NAME;
 import static google.registry.dns.DnsConstants.DNS_TARGET_CREATE_TIME_PARAM;
 import static google.registry.dns.DnsConstants.DNS_TARGET_NAME_PARAM;
 import static google.registry.dns.DnsConstants.DNS_TARGET_TYPE_PARAM;
-import static google.registry.model.registry.Registries.assertTldExists;
+import static google.registry.model.tld.Registries.assertTldExists;
 import static google.registry.request.RequestParameters.PARAM_TLD;
 import static google.registry.util.DomainNameUtils.getTldFromDomainName;
 import static java.util.concurrent.TimeUnit.MILLISECONDS;
@@ -38,7 +38,7 @@ import com.google.common.flogger.FluentLogger;
 import com.google.common.net.InternetDomainName;
 import com.google.common.util.concurrent.RateLimiter;
 import google.registry.dns.DnsConstants.TargetType;
-import google.registry.model.registry.Registries;
+import google.registry.model.tld.Registries;
 import google.registry.util.Clock;
 import google.registry.util.NonFinalForTesting;
 import google.registry.util.SystemClock;
@@ -19,7 +19,7 @@ import static com.google.common.base.Preconditions.checkState;
 import com.google.common.collect.ImmutableMap;
 import com.google.common.flogger.FluentLogger;
 import google.registry.dns.writer.DnsWriter;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import java.util.Map;
 import javax.inject.Inject;

@@ -32,7 +32,7 @@ import google.registry.dns.DnsMetrics.ActionStatus;
 import google.registry.dns.DnsMetrics.CommitStatus;
 import google.registry.dns.DnsMetrics.PublishStatus;
 import google.registry.dns.writer.DnsWriter;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import google.registry.request.Action;
 import google.registry.request.HttpException.ServiceUnavailableException;
 import google.registry.request.Parameter;
@@ -47,8 +47,8 @@ import com.google.common.hash.HashFunction;
 import com.google.common.hash.Hashing;
 import google.registry.config.RegistryConfig.Config;
 import google.registry.dns.DnsConstants.TargetType;
-import google.registry.model.registry.Registries;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registries;
+import google.registry.model.tld.Registry;
 import google.registry.request.Action;
 import google.registry.request.Parameter;
 import google.registry.request.auth.Auth;
@@ -39,7 +39,7 @@ import google.registry.dns.writer.DnsWriterZone;
 import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.host.HostResource;
-import google.registry.model.registry.Registries;
+import google.registry.model.tld.Registries;
 import google.registry.util.Clock;
 import google.registry.util.Concurrent;
 import google.registry.util.Retrier;
@@ -30,7 +30,7 @@ import google.registry.dns.writer.DnsWriterZone;
 import google.registry.model.domain.DomainBase;
 import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.host.HostResource;
-import google.registry.model.registry.Registries;
+import google.registry.model.tld.Registries;
 import google.registry.util.Clock;
 import java.io.IOException;
 import java.net.Inet4Address;
@@ -355,6 +355,12 @@
    <url-pattern>/_dr/task/deleteExpiredDomains</url-pattern>
  </servlet-mapping>

+  <!-- Background action to send notification emails to registrars with expiring certificate. -->
+  <servlet-mapping>
+    <servlet-name>backend-servlet</servlet-name>
+    <url-pattern>/_dr/task/sendExpiringCertificateNotificationEmail</url-pattern>
+  </servlet-mapping>
+
  <!-- Mapreduce to import contacts from escrow file -->
  <servlet-mapping>
    <servlet-name>backend-servlet</servlet-name>
@@ -102,6 +102,7 @@
    <target>backend</target>
  </cron>

+  <!-- Disabled for sql-only tests.
  <cron>
    <url><![CDATA[/_dr/cron/commitLogCheckpoint]]></url>
    <description>
@@ -110,6 +111,7 @@
    <schedule>every 3 minutes synchronized</schedule>
    <target>backend</target>
  </cron>
+  -->

  <cron>
    <url><![CDATA[/_dr/task/deleteContactsAndHosts]]></url>
@@ -133,6 +135,7 @@
    <target>backend</target>
  </cron>

+  <!-- Disabled for sql-only tests
  <cron>
    <url><![CDATA[/_dr/cron/fanout?queue=export-snapshot&endpoint=/_dr/task/backupDatastore&runInEmpty]]></url>
    <description>
@@ -140,16 +143,14 @@
      It also enqueues a new task to wait on the completion of that job and then load the resulting
      snapshot into bigquery.
    </description>
-    <!--
+    <!- -
      Keep google.registry.export.CheckBackupAction.MAXIMUM_BACKUP_RUNNING_TIME less than
-      this interval. -->
+      this interval. - ->
    <schedule>every day 06:00</schedule>
    <target>backend</target>
  </cron>
+  -->

-  <!--
-    Removed for the duration of load testing
-    TODO(b/71607184): Restore after loadtesting is done
  <cron>
    <url><![CDATA[/_dr/cron/fanout?queue=retryable-cron-tasks&endpoint=/_dr/task/deleteProberData&runInEmpty]]></url>
    <description>
@@ -159,7 +160,6 @@
    <timezone>UTC</timezone>
    <target>backend</target>
  </cron>
-  -->

  <!-- TODO: Add borgmon job to check that these files are created and updated successfully. -->
  <cron>
@@ -200,6 +200,7 @@
    <target>backend</target>
  </cron>

+  <!-- Disabled for sql-only tests.
  <cron>
    <url><![CDATA[/_dr/cron/fanout?queue=replay-commit-logs-to-sql&endpoint=/_dr/task/replayCommitLogsToSql&runInEmpty]]></url>
    <description>
@@ -208,4 +209,26 @@
    <schedule>every 3 minutes</schedule>
    <target>backend</target>
  </cron>
+  -->
+
+  <!--
+    The next two wipeout jobs are required when crash has production data.
+   -->
+  <cron>
+    <url><![CDATA[/_dr/task/wipeOutCloudSql]]></url>
+    <description>
+      This job runs an action that deletes all data in Cloud SQL.
+    </description>
+    <schedule>every saturday 03:07</schedule>
+    <target>backend</target>
+  </cron>
+
+  <cron>
+    <url><![CDATA[/_dr/task/wipeOutDatastore]]></url>
+    <description>
+      This job runs an action that deletes all data in Cloud Datastore.
+    </description>
+    <schedule>every saturday 03:07</schedule>
+    <target>backend</target>
+  </cron>
 </cronentries>
@@ -330,4 +330,13 @@
    <schedule>every day 15:00</schedule>
    <target>backend</target>
  </cron>
+
+  <cron>
+    <url><![CDATA[/_dr/cron/fanout?queue=replay-commit-logs-to-sql&endpoint=/_dr/task/replayCommitLogsToSql&runInEmpty]]></url>
+    <description>
+      Replays recent commit logs from Datastore to the SQL secondary backend.
+    </description>
+    <schedule>every 3 minutes</schedule>
+    <target>backend</target>
+  </cron>
 </cronentries>
@@ -168,6 +168,15 @@
    <target>backend</target>
  </cron>

+  <cron>
+    <url><![CDATA[/_dr/task/sendExpiringCertificateNotificationEmail]]></url>
+    <description>
+      This job runs an action that sends emails to partners if their certificates are expiring soon.
+    </description>
+    <schedule>every day 04:30</schedule>
+    <target>backend</target>
+  </cron>
+
  <cron>
    <url><![CDATA[/_dr/cron/fanout?queue=export-snapshot&endpoint=/_dr/task/backupDatastore&runInEmpty]]></url>
    <description>
@@ -230,7 +239,7 @@
  </cron>

  <cron>
-    <url><![CDATA[/_dr/cron/fanout?queue=replay-commit-logs-to-sql&endpoint=/_dr/task/replayCommitLogsToSql&runInEmpty&dryRun=true]]></url>
+    <url><![CDATA[/_dr/cron/fanout?queue=replay-commit-logs-to-sql&endpoint=/_dr/task/replayCommitLogsToSql&runInEmpty]]></url>
    <description>
      Replays recent commit logs from Datastore to the SQL secondary backend.
    </description>
@@ -17,7 +17,7 @@ package google.registry.export;
 import static com.google.common.base.Verify.verifyNotNull;
 import static google.registry.mapreduce.inputs.EppResourceInputs.createEntityInput;
 import static google.registry.model.EppResourceUtils.isActive;
-import static google.registry.model.registry.Registries.getTldsOfType;
+import static google.registry.model.tld.Registries.getTldsOfType;
 import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.request.Action.Method.POST;
@@ -38,8 +38,8 @@ import google.registry.config.RegistryConfig.Config;
 import google.registry.gcs.GcsUtils;
 import google.registry.mapreduce.MapreduceRunner;
 import google.registry.model.domain.DomainBase;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.Registry.TldType;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.Registry.TldType;
 import google.registry.request.Action;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
@@ -30,14 +30,14 @@ import com.google.common.collect.Streams;
 import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
 import google.registry.config.RegistryConfig.Config;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.label.PremiumList.PremiumEntry;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.label.PremiumList.PremiumEntry;
+import google.registry.model.tld.label.PremiumListDao;
 import google.registry.request.Action;
 import google.registry.request.Parameter;
 import google.registry.request.RequestParameters;
 import google.registry.request.Response;
 import google.registry.request.auth.Auth;
-import google.registry.schema.tld.PremiumListDao;
 import google.registry.storage.drive.DriveConnection;
 import java.io.IOException;
 import java.util.Optional;
@@ -23,7 +23,7 @@ import static javax.servlet.http.HttpServletResponse.SC_OK;

 import com.google.common.flogger.FluentLogger;
 import com.google.common.net.MediaType;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import google.registry.request.Action;
 import google.registry.request.Parameter;
 import google.registry.request.RequestParameters;
@@ -16,10 +16,10 @@ package google.registry.export;

 import com.google.common.base.Joiner;
 import google.registry.config.RegistryConfig.Config;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.label.ReservedList;
-import google.registry.model.registry.label.ReservedList.ReservedListEntry;
-import google.registry.model.registry.label.ReservedListDao;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.label.ReservedList;
+import google.registry.model.tld.label.ReservedList.ReservedListEntry;
+import google.registry.model.tld.label.ReservedListDao;
 import java.util.Set;
 import java.util.TreeSet;
 import javax.inject.Inject;
@@ -19,8 +19,8 @@ import static com.google.common.net.HttpHeaders.ACCESS_CONTROL_ALLOW_ORIGIN;
 import static google.registry.flows.domain.DomainFlowUtils.validateDomainName;
 import static google.registry.flows.domain.DomainFlowUtils.validateDomainNameWithIdnTables;
 import static google.registry.flows.domain.DomainFlowUtils.verifyNotInPredelegation;
-import static google.registry.model.registry.label.ReservationType.getTypeOfHighestSeverity;
-import static google.registry.model.registry.label.ReservedList.getReservationTypes;
+import static google.registry.model.tld.label.ReservationType.getTypeOfHighestSeverity;
+import static google.registry.model.tld.label.ReservedList.getReservationTypes;
 import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.AVAILABLE;
 import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.REGISTERED;
 import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.RESERVED;
@@ -46,8 +46,8 @@ import google.registry.flows.domain.DomainFlowUtils.BadCommandForRegistryPhaseEx
 import google.registry.flows.domain.DomainFlowUtils.InvalidIdnDomainLabelException;
 import google.registry.model.domain.DomainBase;
 import google.registry.model.index.ForeignKeyIndex;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.label.ReservationType;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.label.ReservationType;
 import google.registry.monitoring.whitebox.CheckApiMetric;
 import google.registry.monitoring.whitebox.CheckApiMetric.Availability;
 import google.registry.request.Action;
@@ -23,7 +23,7 @@ import google.registry.flows.domain.DomainPricingLogic;
 import google.registry.flows.domain.FeesAndCredits;
 import google.registry.model.ImmutableObject;
 import google.registry.model.eppinput.EppInput;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import org.joda.time.DateTime;

 /**
@@ -28,8 +28,8 @@ import static google.registry.flows.domain.DomainFlowUtils.isValidReservedCreate
 import static google.registry.flows.domain.DomainFlowUtils.validateDomainName;
 import static google.registry.flows.domain.DomainFlowUtils.validateDomainNameWithIdnTables;
 import static google.registry.flows.domain.DomainFlowUtils.verifyNotInPredelegation;
-import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
-import static google.registry.model.registry.label.ReservationType.getTypeOfHighestSeverity;
+import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
+import static google.registry.model.tld.label.ReservationType.getTypeOfHighestSeverity;

 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.ImmutableMap;
@@ -67,10 +67,10 @@ import google.registry.model.eppoutput.CheckData.DomainCheckData;
 import google.registry.model.eppoutput.EppResponse;
 import google.registry.model.eppoutput.EppResponse.ResponseExtension;
 import google.registry.model.index.ForeignKeyIndex;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.Registry.TldState;
-import google.registry.model.registry.label.ReservationType;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.Registry.TldState;
+import google.registry.model.tld.label.ReservationType;
 import google.registry.persistence.VKey;
 import google.registry.util.Clock;
 import java.util.HashSet;
@@ -43,8 +43,8 @@ import google.registry.model.domain.token.AllocationTokenExtension;
 import google.registry.model.eppinput.EppInput;
 import google.registry.model.eppinput.ResourceCommand;
 import google.registry.model.eppoutput.EppResponse;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import google.registry.model.tmch.ClaimsListDao;
 import google.registry.util.Clock;
 import java.util.HashSet;
@@ -44,11 +44,11 @@ import static google.registry.flows.domain.DomainFlowUtils.verifyUnitIsYears;
 import static google.registry.model.EppResourceUtils.createDomainRepoId;
 import static google.registry.model.IdService.allocateId;
 import static google.registry.model.eppcommon.StatusValue.SERVER_HOLD;
-import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
-import static google.registry.model.registry.Registry.TldState.QUIET_PERIOD;
-import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
-import static google.registry.model.registry.label.ReservationType.NAME_COLLISION;
 import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_CREATE;
+import static google.registry.model.tld.Registry.TldState.GENERAL_AVAILABILITY;
+import static google.registry.model.tld.Registry.TldState.QUIET_PERIOD;
+import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
+import static google.registry.model.tld.label.ReservationType.NAME_COLLISION;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.util.DateTimeUtils.END_OF_TIME;
 import static google.registry.util.DateTimeUtils.leapSafeAddYears;
@@ -104,14 +104,14 @@ import google.registry.model.index.ForeignKeyIndex;
 import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
 import google.registry.model.poll.PollMessage;
 import google.registry.model.poll.PollMessage.Autorenew;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.Registry.TldState;
-import google.registry.model.registry.Registry.TldType;
-import google.registry.model.registry.label.ReservationType;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.Registry.TldState;
+import google.registry.model.tld.Registry.TldType;
+import google.registry.model.tld.label.ReservationType;
 import google.registry.tmch.LordnTaskUtils;
 import java.util.Optional;
 import javax.inject.Inject;
@@ -85,11 +85,11 @@ import google.registry.model.eppinput.EppInput;
 import google.registry.model.eppoutput.EppResponse;
 import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
 import google.registry.model.poll.PollMessage;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.Registry.TldType;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.Registry.TldType;
 import google.registry.model.transfer.TransferStatus;
 import java.util.Collections;
 import java.util.Optional;
@@ -27,17 +27,17 @@ import static com.google.common.collect.Sets.intersection;
 import static com.google.common.collect.Sets.union;
 import static google.registry.model.domain.DomainBase.MAX_REGISTRATION_YEARS;
 import static google.registry.model.ofy.ObjectifyService.auditedOfy;
-import static google.registry.model.registry.Registries.findTldForName;
-import static google.registry.model.registry.Registries.getTlds;
-import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
-import static google.registry.model.registry.Registry.TldState.PREDELEGATION;
-import static google.registry.model.registry.Registry.TldState.QUIET_PERIOD;
-import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
-import static google.registry.model.registry.label.ReservationType.ALLOWED_IN_SUNRISE;
-import static google.registry.model.registry.label.ReservationType.FULLY_BLOCKED;
-import static google.registry.model.registry.label.ReservationType.NAME_COLLISION;
-import static google.registry.model.registry.label.ReservationType.RESERVED_FOR_ANCHOR_TENANT;
-import static google.registry.model.registry.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
+import static google.registry.model.tld.Registries.findTldForName;
+import static google.registry.model.tld.Registries.getTlds;
+import static google.registry.model.tld.Registry.TldState.GENERAL_AVAILABILITY;
+import static google.registry.model.tld.Registry.TldState.PREDELEGATION;
+import static google.registry.model.tld.Registry.TldState.QUIET_PERIOD;
+import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
+import static google.registry.model.tld.label.ReservationType.ALLOWED_IN_SUNRISE;
+import static google.registry.model.tld.label.ReservationType.FULLY_BLOCKED;
+import static google.registry.model.tld.label.ReservationType.NAME_COLLISION;
+import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_ANCHOR_TENANT;
+import static google.registry.model.tld.label.ReservationType.RESERVED_FOR_SPECIFIC_USE;
 import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.pricing.PricingEngineProxy.isDomainPremium;
@@ -119,13 +119,13 @@ import google.registry.model.host.HostResource;
 import google.registry.model.poll.PollMessage;
 import google.registry.model.registrar.Registrar;
 import google.registry.model.registrar.Registrar.State;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.Registry.TldState;
-import google.registry.model.registry.label.ReservationType;
-import google.registry.model.registry.label.ReservedList;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
 import google.registry.model.reporting.HistoryEntry;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.Registry.TldState;
+import google.registry.model.tld.label.ReservationType;
+import google.registry.model.tld.label.ReservedList;
 import google.registry.model.tmch.ClaimsListDao;
 import google.registry.persistence.VKey;
 import google.registry.tldconfig.idn.IdnLabelValidator;
@@ -1097,10 +1097,11 @@ public class DomainFlowUtils {
    } else {
      return jpaTm()
          .query(
-              "FROM DomainHistory WHERE modificationTime >= :beginning "
-                  + "ORDER BY modificationTime ASC",
+              "FROM DomainHistory WHERE modificationTime >= :beginning AND domainRepoId = "
+                  + ":repoId ORDER BY modificationTime ASC",
              DomainHistory.class)
          .setParameter("beginning", now.minus(maxSearchPeriod))
+          .setParameter("repoId", domainBase.getRepoId())
          .getResultList();
    }
  }
@@ -32,7 +32,7 @@ import google.registry.model.domain.fee.BaseFee.FeeType;
 import google.registry.model.domain.fee.Fee;
 import google.registry.model.domain.token.AllocationToken;
 import google.registry.model.pricing.PremiumPricingEngine.DomainPrices;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import java.math.BigDecimal;
 import java.math.RoundingMode;
 import java.util.Optional;
@@ -71,10 +71,10 @@ import google.registry.model.eppinput.EppInput;
 import google.registry.model.eppinput.ResourceCommand;
 import google.registry.model.eppoutput.EppResponse;
 import google.registry.model.poll.PollMessage;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import java.util.Optional;
 import javax.inject.Inject;
 import org.joda.money.Money;
@@ -64,10 +64,10 @@ import google.registry.model.eppinput.EppInput;
 import google.registry.model.eppinput.ResourceCommand;
 import google.registry.model.eppoutput.EppResponse;
 import google.registry.model.poll.PollMessage;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import java.util.Optional;
 import javax.inject.Inject;
 import org.joda.money.Money;
@@ -57,9 +57,9 @@ import google.registry.model.domain.rgp.GracePeriodStatus;
 import google.registry.model.eppcommon.AuthInfo;
 import google.registry.model.eppoutput.EppResponse;
 import google.registry.model.poll.PollMessage;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import google.registry.model.transfer.DomainTransferData;
 import google.registry.model.transfer.TransferStatus;
 import java.util.Optional;
@@ -45,9 +45,9 @@ import google.registry.model.domain.DomainHistory;
 import google.registry.model.domain.metadata.MetadataExtension;
 import google.registry.model.eppcommon.AuthInfo;
 import google.registry.model.eppoutput.EppResponse;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import google.registry.model.transfer.TransferStatus;
 import java.util.Optional;
 import javax.inject.Inject;
@@ -47,9 +47,9 @@ import google.registry.model.domain.DomainHistory;
 import google.registry.model.domain.metadata.MetadataExtension;
 import google.registry.model.eppcommon.AuthInfo;
 import google.registry.model.eppoutput.EppResponse;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import google.registry.model.transfer.TransferStatus;
 import java.util.Optional;
 import javax.inject.Inject;
@@ -66,10 +66,10 @@ import google.registry.model.eppinput.EppInput;
 import google.registry.model.eppinput.ResourceCommand;
 import google.registry.model.eppoutput.EppResponse;
 import google.registry.model.poll.PollMessage;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.DomainTransactionRecord.TransactionReportField;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import google.registry.model.transfer.DomainTransferData;
 import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
 import google.registry.model.transfer.TransferResponse.DomainTransferResponse;
@@ -32,7 +32,7 @@ import google.registry.model.domain.rgp.GracePeriodStatus;
 import google.registry.model.eppcommon.Trid;
 import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
 import google.registry.model.poll.PollMessage;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import google.registry.model.transfer.DomainTransferData;
 import google.registry.model.transfer.TransferData;
 import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
@@ -76,8 +76,8 @@ import google.registry.model.eppcommon.StatusValue;
 import google.registry.model.eppinput.EppInput;
 import google.registry.model.eppinput.ResourceCommand;
 import google.registry.model.eppoutput.EppResponse;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.IcannReportingTypes.ActivityReportField;
+import google.registry.model.tld.Registry;
 import java.util.Optional;
 import javax.inject.Inject;
 import org.joda.time.DateTime;
@@ -21,7 +21,7 @@ import com.google.common.net.InternetDomainName;
 import google.registry.flows.EppException;
 import google.registry.model.domain.DomainCommand;
 import google.registry.model.domain.token.AllocationToken;
-import google.registry.model.registry.Registry;
+import google.registry.model.tld.Registry;
 import org.joda.time.DateTime;

 /**
@@ -31,8 +31,8 @@ import google.registry.model.domain.DomainCommand;
 import google.registry.model.domain.token.AllocationToken;
 import google.registry.model.domain.token.AllocationToken.TokenStatus;
 import google.registry.model.domain.token.AllocationToken.TokenType;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.HistoryEntry;
+import google.registry.model.tld.Registry;
 import google.registry.persistence.VKey;
 import java.util.List;
 import java.util.Optional;
@@ -16,7 +16,7 @@ package google.registry.flows.host;

 import static google.registry.model.EppResourceUtils.isActive;
 import static google.registry.model.EppResourceUtils.loadByForeignKey;
-import static google.registry.model.registry.Registries.findTldForName;
+import static google.registry.model.tld.Registries.findTldForName;
 import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
 import static java.util.stream.Collectors.joining;

@@ -17,7 +17,6 @@ package google.registry.model;
 import com.google.common.collect.ImmutableSet;
 import google.registry.model.billing.BillingEvent;
 import google.registry.model.common.Cursor;
-import google.registry.model.common.DatabaseMigrationStateSchedule;
 import google.registry.model.common.EntityGroupRoot;
 import google.registry.model.common.GaeUserIdConverter;
 import google.registry.model.contact.ContactHistory;
@@ -39,16 +38,12 @@ import google.registry.model.poll.PollMessage;
 import google.registry.model.rde.RdeRevision;
 import google.registry.model.registrar.Registrar;
 import google.registry.model.registrar.RegistrarContact;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.label.ReservedList;
+import google.registry.model.replay.LastSqlTransaction;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.model.server.Lock;
 import google.registry.model.server.ServerSecret;
-import google.registry.model.tmch.ClaimsList;
-import google.registry.model.tmch.ClaimsList.ClaimsListRevision;
-import google.registry.model.tmch.ClaimsList.ClaimsListSingleton;
+import google.registry.model.tld.Registry;
 import google.registry.model.tmch.TmchCrl;
-import google.registry.schema.replay.LastSqlTransaction;

 /** Sets of classes of the Objectify-registered entities in use throughout the model. */
 public final class EntityClasses {
@@ -61,9 +56,6 @@ public final class EntityClasses {
          BillingEvent.Modification.class,
          BillingEvent.OneTime.class,
          BillingEvent.Recurring.class,
-          ClaimsList.class,
-          ClaimsListRevision.class,
-          ClaimsListSingleton.class,
          CommitLogBucket.class,
          CommitLogCheckpoint.class,
          CommitLogCheckpointRoot.class,
@@ -72,7 +64,6 @@ public final class EntityClasses {
          ContactHistory.class,
          ContactResource.class,
          Cursor.class,
-          DatabaseMigrationStateSchedule.class,
          DomainBase.class,
          DomainHistory.class,
          EntityGroupRoot.class,
@@ -94,7 +85,6 @@ public final class EntityClasses {
          Registrar.class,
          RegistrarContact.class,
          Registry.class,
-          ReservedList.class,
          ServerSecret.class,
          TmchCrl.class);

@@ -20,6 +20,7 @@ import static com.google.common.collect.Sets.difference;
 import static com.google.common.collect.Sets.union;
 import static google.registry.config.RegistryConfig.getEppResourceCachingDuration;
 import static google.registry.config.RegistryConfig.getEppResourceMaxCachedEntries;
+import static google.registry.persistence.transaction.TransactionManagerFactory.ofyTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.util.CollectionUtils.nullToEmpty;
 import static google.registry.util.CollectionUtils.nullToEmptyImmutableCopy;
@@ -37,6 +38,8 @@ import com.googlecode.objectify.annotation.Id;
 import com.googlecode.objectify.annotation.Index;
 import google.registry.config.RegistryConfig;
 import google.registry.model.eppcommon.StatusValue;
+import google.registry.model.index.EppResourceIndex;
+import google.registry.model.index.ForeignKeyIndex;
 import google.registry.model.ofy.CommitLogManifest;
 import google.registry.model.transfer.TransferData;
 import google.registry.persistence.VKey;
@@ -219,6 +222,14 @@ public abstract class EppResource extends BackupGroupRoot implements Buildable {
  @Override
  public abstract Builder<?, ?> asBuilder();

+  /** Used when replaying from SQL to DS to populate the Datastore indexes. */
+  protected void saveIndexesToDatastore() {
+    ofyTm()
+        .putAll(
+            ForeignKeyIndex.create(this, getDeletionTime()),
+            EppResourceIndex.create(Key.create(this)));
+  }
+
  /** EppResources that are loaded via foreign keys should implement this marker interface. */
  public interface ForeignKeyedEppResource {}

@@ -40,9 +40,9 @@ import google.registry.model.host.HostResource;
 import google.registry.model.index.ForeignKeyIndex;
 import google.registry.model.ofy.CommitLogManifest;
 import google.registry.model.ofy.CommitLogMutation;
-import google.registry.model.registry.Registry;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.model.reporting.HistoryEntryDao;
+import google.registry.model.tld.Registry;
 import google.registry.model.transfer.DomainTransferData;
 import google.registry.model.transfer.TransferData;
 import google.registry.model.transfer.TransferStatus;
@@ -18,8 +18,8 @@ import static com.google.common.base.Preconditions.checkArgument;
 import static com.google.common.base.Preconditions.checkNotNull;
 import static com.google.common.base.Preconditions.checkState;
 import static com.google.common.collect.ImmutableList.toImmutableList;
-import static google.registry.model.registry.Registry.TldState.GENERAL_AVAILABILITY;
-import static google.registry.model.registry.Registry.TldState.START_DATE_SUNRISE;
+import static google.registry.model.tld.Registry.TldState.GENERAL_AVAILABILITY;
+import static google.registry.model.tld.Registry.TldState.START_DATE_SUNRISE;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.util.DateTimeUtils.START_OF_TIME;

@@ -36,11 +36,11 @@ import google.registry.model.pricing.StaticPremiumListPricingEngine;
 import google.registry.model.registrar.Registrar;
 import google.registry.model.registrar.RegistrarAddress;
 import google.registry.model.registrar.RegistrarContact;
-import google.registry.model.registry.Registry;
-import google.registry.model.registry.Registry.TldState;
-import google.registry.model.registry.label.PremiumList;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.Registry.TldState;
+import google.registry.model.tld.label.PremiumList;
+import google.registry.model.tld.label.PremiumListDao;
 import google.registry.persistence.VKey;
-import google.registry.schema.tld.PremiumListDao;
 import google.registry.util.CidrAddressBlock;
 import java.util.Collection;
 import java.util.Optional;
@@ -46,13 +46,13 @@ import google.registry.model.domain.DomainHistory;
 import google.registry.model.domain.GracePeriod;
 import google.registry.model.domain.rgp.GracePeriodStatus;
 import google.registry.model.domain.token.AllocationToken;
+import google.registry.model.replay.DatastoreAndSqlEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;
 import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
 import google.registry.persistence.BillingVKey.BillingEventVKey;
 import google.registry.persistence.BillingVKey.BillingRecurrenceVKey;
 import google.registry.persistence.VKey;
 import google.registry.persistence.WithLongVKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
-import google.registry.schema.replay.DatastoreOnlyEntity;
 import java.util.Objects;
 import java.util.Optional;
 import java.util.Set;
@@ -30,9 +30,9 @@ import google.registry.model.ImmutableObject;
 import google.registry.model.UpdateAutoTimestamp;
 import google.registry.model.annotations.InCrossTld;
 import google.registry.model.common.Cursor.CursorId;
-import google.registry.model.registry.Registry;
+import google.registry.model.replay.DatastoreAndSqlEntity;
+import google.registry.model.tld.Registry;
 import google.registry.persistence.VKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import java.io.Serializable;
 import java.util.List;
 import java.util.Optional;
@@ -15,9 +15,7 @@
 package google.registry.model.common;

 import static com.google.common.base.Preconditions.checkArgument;
-import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
-import static google.registry.model.ofy.ObjectifyService.auditedOfy;
-import static google.registry.persistence.transaction.TransactionManagerFactory.ofyTm;
+import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.util.DateTimeUtils.START_OF_TIME;

 import com.google.common.annotations.VisibleForTesting;
@@ -26,28 +24,25 @@ import com.google.common.cache.CacheLoader;
 import com.google.common.cache.LoadingCache;
 import com.google.common.collect.ImmutableMultimap;
 import com.google.common.collect.ImmutableSortedMap;
-import com.googlecode.objectify.Key;
-import com.googlecode.objectify.annotation.Embed;
-import com.googlecode.objectify.annotation.Entity;
-import com.googlecode.objectify.annotation.Mapify;
-import google.registry.model.annotations.InCrossTld;
-import google.registry.model.common.TimedTransitionProperty.TimeMapper;
+import com.google.common.flogger.FluentLogger;
+import google.registry.config.RegistryEnvironment;
 import google.registry.model.common.TimedTransitionProperty.TimedTransition;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.SqlOnlyEntity;
 import java.time.Duration;
-import java.util.Optional;
+import javax.persistence.Entity;
+import javax.persistence.PersistenceException;
 import org.joda.time.DateTime;

 /**
 * A wrapper object representing the stage-to-time mapping of the Registry 3.0 Cloud SQL migration.
 *
- * <p>The entity is stored in Datastore throughout the entire migration so as to have a single point
- * of access.
+ * <p>The entity is stored in SQL throughout the entire migration so as to have a single point of
+ * access.
 */
@Entity
-@InCrossTld
-public class DatabaseMigrationStateSchedule extends CrossTldSingleton
-    implements DatastoreOnlyEntity {
+public class DatabaseMigrationStateSchedule extends CrossTldSingleton implements SqlOnlyEntity {
+
+  private static final FluentLogger logger = FluentLogger.forEnclosingClass();

  public enum PrimaryDatabase {
    CLOUD_SQL,
@@ -96,12 +91,11 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
    }
  }

-  @Embed
  public static class MigrationStateTransition extends TimedTransition<MigrationState> {
    private MigrationState migrationState;

    @Override
-    protected MigrationState getValue() {
+    public MigrationState getValue() {
      return migrationState;
    }

@@ -185,7 +179,6 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
              MigrationStateTransition.class);

  @VisibleForTesting
-  @Mapify(TimeMapper.class)
  public TimedTransitionProperty<MigrationState, MigrationStateTransition> migrationTransitions =
      TimedTransitionProperty.forMapify(
          MigrationState.DATASTORE_ONLY, MigrationStateTransition.class);
@@ -194,14 +187,14 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
  private DatabaseMigrationStateSchedule() {}

  @VisibleForTesting
-  DatabaseMigrationStateSchedule(
+  public DatabaseMigrationStateSchedule(
      TimedTransitionProperty<MigrationState, MigrationStateTransition> migrationTransitions) {
    this.migrationTransitions = migrationTransitions;
  }

-  /** Sets and persists to Datastore the provided migration transition schedule. */
+  /** Sets and persists to SQL the provided migration transition schedule. */
  public static void set(ImmutableSortedMap<DateTime, MigrationState> migrationTransitionMap) {
-    ofyTm().assertInTransaction();
+    jpaTm().assertInTransaction();
    TimedTransitionProperty<MigrationState, MigrationStateTransition> transitions =
        TimedTransitionProperty.make(
            migrationTransitionMap,
@@ -211,7 +204,7 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
            MigrationState.DATASTORE_ONLY,
            "migrationTransitionMap must start with DATASTORE_ONLY");
    validateTransitionAtCurrentTime(transitions);
-    ofyTm().put(new DatabaseMigrationStateSchedule(transitions));
+    jpaTm().putIgnoringReadOnly(new DatabaseMigrationStateSchedule(transitions));
    CACHE.invalidateAll();
  }

@@ -225,23 +218,26 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
    return get().getValueAtTime(dateTime);
  }

-  /** Loads the currently-set migration schedule from Datastore, or the default if none exists. */
+  /** Loads the currently-set migration schedule from SQL, or the default if none exists. */
  @VisibleForTesting
  static TimedTransitionProperty<MigrationState, MigrationStateTransition> getUncached() {
-    return Optional.ofNullable(
-            auditedOfy()
-                .doTransactionless(
-                    () ->
-                        auditedOfy()
-                            .load()
-                            .key(
-                                Key.create(
-                                    getCrossTldKey(),
-                                    DatabaseMigrationStateSchedule.class,
-                                    CrossTldSingleton.SINGLETON_ID))
-                            .now()))
-        .map(s -> s.migrationTransitions)
-        .orElse(DEFAULT_TRANSITION_MAP);
+    return jpaTm()
+        .transactWithoutBackup(
+            () -> {
+              try {
+                return jpaTm()
+                    .loadSingleton(DatabaseMigrationStateSchedule.class)
+                    .map(s -> s.migrationTransitions)
+                    .orElse(DEFAULT_TRANSITION_MAP);
+              } catch (PersistenceException e) {
+                if (!RegistryEnvironment.get().equals(RegistryEnvironment.UNITTEST)) {
+                  throw e;
+                }
+                logger.atWarning().withCause(e).log(
+                    "Error when retrieving migration schedule; this should only happen in tests.");
+                return DEFAULT_TRANSITION_MAP;
+              }
+            });
  }

  /**
@@ -252,8 +248,8 @@ public class DatabaseMigrationStateSchedule extends CrossTldSingleton
   */
  private static void validateTransitionAtCurrentTime(
      TimedTransitionProperty<MigrationState, MigrationStateTransition> newTransitions) {
-    MigrationState currentValue = getUncached().getValueAtTime(ofyTm().getTransactionTime());
-    MigrationState nextCurrentValue = newTransitions.getValueAtTime(ofyTm().getTransactionTime());
+    MigrationState currentValue = getUncached().getValueAtTime(jpaTm().getTransactionTime());
+    MigrationState nextCurrentValue = newTransitions.getValueAtTime(jpaTm().getTransactionTime());
    checkArgument(
        VALID_STATE_TRANSITIONS.get(currentValue).contains(nextCurrentValue),
        "Cannot transition from current state-as-of-now %s to new state-as-of-now %s",
@@ -19,7 +19,7 @@ import com.googlecode.objectify.Key;
 import com.googlecode.objectify.annotation.Entity;
 import com.googlecode.objectify.annotation.Id;
 import google.registry.model.BackupGroupRoot;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;
 import javax.annotation.Nullable;

 /**
@@ -21,10 +21,10 @@ import com.googlecode.objectify.annotation.EntitySubclass;
 import google.registry.model.EppResource;
 import google.registry.model.ImmutableObject;
 import google.registry.model.contact.ContactHistory.ContactHistoryId;
+import google.registry.model.replay.DatastoreEntity;
+import google.registry.model.replay.SqlEntity;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
-import google.registry.schema.replay.DatastoreEntity;
-import google.registry.schema.replay.SqlEntity;
 import java.io.Serializable;
 import java.util.Optional;
 import javax.annotation.Nullable;
@@ -19,9 +19,9 @@ import com.googlecode.objectify.annotation.Entity;
 import google.registry.model.EppResource.ForeignKeyedEppResource;
 import google.registry.model.annotations.ExternalMessagingName;
 import google.registry.model.annotations.ReportedOn;
+import google.registry.model.replay.DatastoreAndSqlEntity;
 import google.registry.persistence.VKey;
 import google.registry.persistence.WithStringVKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
 import org.joda.time.DateTime;
@@ -66,6 +66,11 @@ public class ContactResource extends ContactBase
    return ContactBase.cloneContactProjectedAtTime(this, now);
  }

+  @Override
+  public void beforeDatastoreSaveOnReplay() {
+    saveIndexesToDatastore();
+  }
+
  @Override
  public Builder asBuilder() {
    return new Builder(clone(this));
@@ -21,9 +21,9 @@ import google.registry.model.annotations.ExternalMessagingName;
 import google.registry.model.annotations.ReportedOn;
 import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.host.HostResource;
+import google.registry.model.replay.DatastoreAndSqlEntity;
 import google.registry.persistence.VKey;
 import google.registry.persistence.WithStringVKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import java.util.Set;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
@@ -163,6 +163,11 @@ public class DomainBase extends DomainContent
    return cloneDomainProjectedAtTime(this, now);
  }

+  @Override
+  public void beforeDatastoreSaveOnReplay() {
+    saveIndexesToDatastore();
+  }
+
  public static VKey<DomainBase> createVKey(Key<DomainBase> key) {
    return VKey.create(DomainBase.class, key.getName(), key);
  }
@@ -62,8 +62,9 @@ import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.eppcommon.StatusValue;
 import google.registry.model.host.HostResource;
 import google.registry.model.poll.PollMessage;
-import google.registry.model.registry.Registry;
+import google.registry.model.replay.ReplaySpecializer;
 import google.registry.model.reporting.HistoryEntry;
+import google.registry.model.tld.Registry;
 import google.registry.model.transfer.DomainTransferData;
 import google.registry.model.transfer.TransferStatus;
 import google.registry.persistence.VKey;
@@ -374,7 +375,7 @@ public class DomainContent extends EppResource
  /**
   * Callback to delete grace periods and DelegationSignerData records prior to domain delete.
   *
-   * <p>See {@link google.registry.schema.replay.ReplaySpecializer}.
+   * <p>See {@link ReplaySpecializer}.
   */
  public static void beforeSqlDelete(VKey<DomainBase> key) {
    // Delete all grace periods associated with the domain.
@@ -28,11 +28,11 @@ import google.registry.model.domain.GracePeriod.GracePeriodHistory;
 import google.registry.model.domain.secdns.DelegationSignerData;
 import google.registry.model.domain.secdns.DomainDsDataHistory;
 import google.registry.model.host.HostResource;
+import google.registry.model.replay.DatastoreEntity;
+import google.registry.model.replay.SqlEntity;
 import google.registry.model.reporting.DomainTransactionRecord;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
-import google.registry.schema.replay.DatastoreEntity;
-import google.registry.schema.replay.SqlEntity;
 import java.io.Serializable;
 import java.util.HashSet;
 import java.util.Optional;
@@ -23,10 +23,10 @@ import com.googlecode.objectify.annotation.Embed;
 import google.registry.model.billing.BillingEvent;
 import google.registry.model.billing.BillingEvent.Recurring;
 import google.registry.model.domain.rgp.GracePeriodStatus;
+import google.registry.model.replay.DatastoreAndSqlEntity;
 import google.registry.persistence.BillingVKey.BillingEventVKey;
 import google.registry.persistence.BillingVKey.BillingRecurrenceVKey;
 import google.registry.persistence.VKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import javax.annotation.Nullable;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
@@ -12,7 +12,7 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-package google.registry.schema.domain;
+package google.registry.model.domain;

 import static com.google.common.base.Preconditions.checkArgument;
 import static google.registry.util.DateTimeUtils.isBeforeOrAt;
@@ -23,7 +23,7 @@ import google.registry.model.Buildable;
 import google.registry.model.CreateAutoTimestamp;
 import google.registry.model.ImmutableObject;
 import google.registry.model.UpdateAutoTimestamp;
-import google.registry.schema.replay.SqlOnlyEntity;
+import google.registry.model.replay.SqlOnlyEntity;
 import google.registry.util.DateTimeUtils;
 import java.time.ZonedDateTime;
 import java.util.Optional;
@@ -232,7 +232,7 @@ public final class RegistryLock extends ImmutableObject implements Buildable, Sq
    return new Builder(clone(this));
  }

-  /** Builder for {@link google.registry.schema.domain.RegistryLock}. */
+  /** Builder for {@link RegistryLock}. */
  public static class Builder extends Buildable.Builder<RegistryLock> {
    public Builder() {}

@@ -17,7 +17,7 @@ package google.registry.model.domain.secdns;
 import static google.registry.model.IdService.allocateId;

 import google.registry.model.domain.DomainHistory;
-import google.registry.schema.replay.SqlOnlyEntity;
+import google.registry.model.replay.SqlOnlyEntity;
 import javax.persistence.Access;
 import javax.persistence.AccessType;
 import javax.persistence.Column;
@@ -46,11 +46,11 @@ import google.registry.model.annotations.ReportedOn;
 import google.registry.model.common.TimedTransitionProperty;
 import google.registry.model.common.TimedTransitionProperty.TimeMapper;
 import google.registry.model.common.TimedTransitionProperty.TimedTransition;
+import google.registry.model.replay.DatastoreAndSqlEntity;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.DomainHistoryVKey;
 import google.registry.persistence.VKey;
 import google.registry.persistence.WithStringVKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import java.util.Optional;
 import java.util.Set;
 import javax.annotation.Nullable;
@@ -21,10 +21,10 @@ import com.googlecode.objectify.annotation.EntitySubclass;
 import google.registry.model.EppResource;
 import google.registry.model.ImmutableObject;
 import google.registry.model.host.HostHistory.HostHistoryId;
+import google.registry.model.replay.DatastoreEntity;
+import google.registry.model.replay.SqlEntity;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
-import google.registry.schema.replay.DatastoreEntity;
-import google.registry.schema.replay.SqlEntity;
 import java.io.Serializable;
 import java.util.Optional;
 import javax.annotation.Nullable;
@@ -19,9 +19,9 @@ import com.googlecode.objectify.annotation.Entity;
 import google.registry.model.EppResource.ForeignKeyedEppResource;
 import google.registry.model.annotations.ExternalMessagingName;
 import google.registry.model.annotations.ReportedOn;
+import google.registry.model.replay.DatastoreAndSqlEntity;
 import google.registry.persistence.VKey;
 import google.registry.persistence.WithStringVKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import javax.persistence.Access;
 import javax.persistence.AccessType;

@@ -51,6 +51,11 @@ public class HostResource extends HostBase
    return VKey.create(HostResource.class, getRepoId(), Key.create(this));
  }

+  @Override
+  public void beforeDatastoreSaveOnReplay() {
+    saveIndexesToDatastore();
+  }
+
  @Override
  public Builder asBuilder() {
    return new Builder(clone(this));
@@ -25,7 +25,7 @@ import com.googlecode.objectify.annotation.Parent;
 import google.registry.model.BackupGroupRoot;
 import google.registry.model.EppResource;
 import google.registry.model.annotations.ReportedOn;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;

 /** An index that allows for quick enumeration of all EppResource entities (e.g. via map reduce). */
@ReportedOn
@@ -24,7 +24,7 @@ import com.googlecode.objectify.annotation.Id;
 import google.registry.model.EppResource;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.VirtualEntity;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;

 /** A virtual entity to represent buckets to which EppResourceIndex objects are randomly added. */
@Entity
@@ -47,9 +47,9 @@ import google.registry.model.annotations.ReportedOn;
 import google.registry.model.contact.ContactResource;
 import google.registry.model.domain.DomainBase;
 import google.registry.model.host.HostResource;
+import google.registry.model.replay.DatastoreOnlyEntity;
 import google.registry.persistence.VKey;
 import google.registry.persistence.transaction.CriteriaQueryBuilder;
-import google.registry.schema.replay.DatastoreOnlyEntity;
 import google.registry.util.NonFinalForTesting;
 import java.util.Collection;
 import java.util.Comparator;
@@ -33,7 +33,7 @@ import google.registry.model.Buildable;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.NotBackedUp;
 import google.registry.model.annotations.NotBackedUp.Reason;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;
 import google.registry.util.NonFinalForTesting;
 import java.util.Random;
 import java.util.function.Supplier;
@@ -27,7 +27,7 @@ import com.googlecode.objectify.annotation.Parent;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.NotBackedUp;
 import google.registry.model.annotations.NotBackedUp.Reason;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.Objects;
@@ -23,7 +23,7 @@ import com.googlecode.objectify.annotation.Id;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.NotBackedUp;
 import google.registry.model.annotations.NotBackedUp.Reason;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;
 import org.joda.time.DateTime;

 /** Singleton parent entity for all commit log checkpoints. */
@@ -25,7 +25,7 @@ import com.googlecode.objectify.annotation.Parent;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.NotBackedUp;
 import google.registry.model.annotations.NotBackedUp.Reason;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;
 import java.util.LinkedHashSet;
 import java.util.Set;
 import org.joda.time.DateTime;
@@ -29,7 +29,7 @@ import com.googlecode.objectify.annotation.Parent;
 import google.registry.model.ImmutableObject;
 import google.registry.model.annotations.NotBackedUp;
 import google.registry.model.annotations.NotBackedUp.Reason;
-import google.registry.schema.replay.DatastoreOnlyEntity;
+import google.registry.model.replay.DatastoreOnlyEntity;

 /** Representation of a saved entity in a {@link CommitLogManifest} (not deletes). */
@Entity
@@ -36,12 +36,12 @@ import google.registry.model.annotations.InCrossTld;
 import google.registry.model.contact.ContactHistory;
 import google.registry.model.domain.DomainHistory;
 import google.registry.model.host.HostHistory;
+import google.registry.model.replay.DatastoreEntity;
+import google.registry.model.replay.SqlEntity;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.persistence.VKey;
 import google.registry.persistence.transaction.QueryComposer;
 import google.registry.persistence.transaction.TransactionManager;
-import google.registry.schema.replay.DatastoreEntity;
-import google.registry.schema.replay.SqlEntity;
 import java.util.List;
 import java.util.NoSuchElementException;
 import java.util.Optional;
@@ -336,7 +336,7 @@ public class DatastoreTransactionManager implements TransactionManager {

  @Override
  public <T> QueryComposer<T> createQueryComposer(Class<T> entity) {
-    return new DatastoreQueryComposerImpl(entity);
+    return new DatastoreQueryComposerImpl<>(entity);
  }

  @Override
@@ -349,6 +349,16 @@ public class DatastoreTransactionManager implements TransactionManager {
    return true;
  }

+  @Override
+  public void putIgnoringReadOnly(Object entity) {
+    syncIfTransactionless(getOfy().saveIgnoringReadOnly().entities(toDatastoreEntity(entity)));
+  }
+
+  @Override
+  public void deleteIgnoringReadOnly(VKey<?> key) {
+    syncIfTransactionless(getOfy().deleteIgnoringReadOnly().key(key.getOfyKey()));
+  }
+
  /**
   * Executes the given {@link Result} instance synchronously if not in a transaction.
   *
@@ -19,6 +19,7 @@ import static com.google.common.base.Preconditions.checkState;
 import static com.google.common.collect.Maps.uniqueIndex;
 import static com.googlecode.objectify.ObjectifyService.ofy;
 import static google.registry.config.RegistryConfig.getBaseOfyRetryDuration;
+import static google.registry.persistence.transaction.TransactionManagerFactory.assertNotReadOnlyMode;
 import static google.registry.util.CollectionUtils.union;

 import com.google.appengine.api.datastore.DatastoreFailureException;
@@ -131,6 +132,7 @@ public class Ofy {
   * <p>We only allow this in transactions so commit logs can be written in tandem with the delete.
   */
  public Deleter delete() {
+    assertNotReadOnlyMode();
    return new AugmentedDeleter() {
      @Override
      protected void handleDeletion(Iterable<Key<?>> keys) {
@@ -148,12 +150,8 @@ public class Ofy {
   * <p>No backups get written.
   */
  public Deleter deleteWithoutBackup() {
-    return new AugmentedDeleter() {
-      @Override
-      protected void handleDeletion(Iterable<Key<?>> keys) {
-        checkProhibitedAnnotations(keys, VirtualEntity.class);
-      }
-    };
+    assertNotReadOnlyMode();
+    return deleteIgnoringReadOnly();
  }

  /**
@@ -163,6 +161,7 @@ public class Ofy {
   * <p>We only allow this in transactions so commit logs can be written in tandem with the save.
   */
  public Saver save() {
+    assertNotReadOnlyMode();
    return new AugmentedSaver() {
      @Override
      protected void handleSave(Iterable<?> entities) {
@@ -182,6 +181,12 @@ public class Ofy {
   * <p>No backups get written.
   */
  public Saver saveWithoutBackup() {
+    assertNotReadOnlyMode();
+    return saveIgnoringReadOnly();
+  }
+
+  /** Save, ignoring any backups or any read-only settings. */
+  public Saver saveIgnoringReadOnly() {
    return new AugmentedSaver() {
      @Override
      protected void handleSave(Iterable<?> entities) {
@@ -190,6 +195,16 @@ public class Ofy {
    };
  }

+  /** Delete, ignoring any backups or any read-only settings. */
+  public Deleter deleteIgnoringReadOnly() {
+    return new AugmentedDeleter() {
+      @Override
+      protected void handleDeletion(Iterable<Key<?>> keys) {
+        checkProhibitedAnnotations(keys, VirtualEntity.class);
+      }
+    };
+  }
+
  private Clock getClock() {
    return injectedClock == null ? clock : injectedClock;
  }
@@ -22,9 +22,9 @@ import com.google.common.collect.ImmutableMap;
 import com.googlecode.objectify.Key;
 import google.registry.config.RegistryEnvironment;
 import google.registry.model.UpdateAutoTimestamp;
+import google.registry.model.replay.DatastoreEntity;
+import google.registry.model.replay.ReplaySpecializer;
 import google.registry.persistence.VKey;
-import google.registry.schema.replay.DatastoreEntity;
-import google.registry.schema.replay.ReplaySpecializer;
 import java.util.HashMap;
 import java.util.Map;
 import java.util.concurrent.ConcurrentLinkedQueue;
@@ -43,6 +43,7 @@ import google.registry.model.host.HostResource;
 import google.registry.model.poll.PendingActionNotificationResponse.ContactPendingActionNotificationResponse;
 import google.registry.model.poll.PendingActionNotificationResponse.DomainPendingActionNotificationResponse;
 import google.registry.model.poll.PendingActionNotificationResponse.HostPendingActionNotificationResponse;
+import google.registry.model.replay.DatastoreAndSqlEntity;
 import google.registry.model.reporting.HistoryEntry;
 import google.registry.model.transfer.TransferData.TransferServerApproveEntity;
 import google.registry.model.transfer.TransferResponse;
@@ -50,7 +51,6 @@ import google.registry.model.transfer.TransferResponse.ContactTransferResponse;
 import google.registry.model.transfer.TransferResponse.DomainTransferResponse;
 import google.registry.persistence.VKey;
 import google.registry.persistence.WithLongVKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import java.util.List;
 import java.util.Optional;
 import javax.annotation.Nullable;
@@ -18,8 +18,8 @@ import static com.google.common.base.Preconditions.checkNotNull;
 import static google.registry.util.DomainNameUtils.getTldFromDomainName;

 import com.google.common.net.InternetDomainName;
-import google.registry.model.registry.Registry;
-import google.registry.schema.tld.PremiumListDao;
+import google.registry.model.tld.Registry;
+import google.registry.model.tld.label.PremiumListDao;
 import java.util.Optional;
 import javax.inject.Inject;
 import org.joda.money.Money;
@@ -39,7 +39,7 @@ public final class RdeNamingUtils {
  }

  /** Returns same thing as {@link #makeRydeFilename} except without the series and revision. */
-  static String makePartialName(String tld, DateTime date, RdeMode mode) {
+  public static String makePartialName(String tld, DateTime date, RdeMode mode) {
    return String.format("%s_%s_%s",
        checkNotNull(tld), formatDate(date), mode.getFilenameComponent());
  }
@@ -27,9 +27,9 @@ import com.googlecode.objectify.annotation.Ignore;
 import google.registry.model.BackupGroupRoot;
 import google.registry.model.ImmutableObject;
 import google.registry.model.rde.RdeRevision.RdeRevisionId;
+import google.registry.model.replay.NonReplicatedEntity;
 import google.registry.persistence.VKey;
 import google.registry.persistence.converter.LocalDateConverter;
-import google.registry.schema.replay.NonReplicatedEntity;
 import java.io.Serializable;
 import java.util.Optional;
 import javax.persistence.Column;
@@ -31,7 +31,7 @@ import static google.registry.config.RegistryConfig.getDefaultRegistrarWhoisServ
 import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
 import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
 import static google.registry.model.ofy.ObjectifyService.auditedOfy;
-import static google.registry.model.registry.Registries.assertTldsExist;
+import static google.registry.model.tld.Registries.assertTldsExist;
 import static google.registry.persistence.transaction.TransactionManagerFactory.jpaTm;
 import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
 import static google.registry.persistence.transaction.TransactionManagerUtil.transactIfJpaTm;
@@ -77,9 +77,9 @@ import google.registry.model.annotations.InCrossTld;
 import google.registry.model.annotations.ReportedOn;
 import google.registry.model.common.EntityGroupRoot;
 import google.registry.model.registrar.Registrar.BillingAccountEntry.CurrencyMapper;
-import google.registry.model.registry.Registry;
+import google.registry.model.replay.DatastoreAndSqlEntity;
+import google.registry.model.tld.Registry;
 import google.registry.persistence.VKey;
-import google.registry.schema.replay.DatastoreAndSqlEntity;
 import google.registry.util.CidrAddressBlock;
 import java.security.cert.CertificateParsingException;
 import java.util.Comparator;
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Michael Muller	031f4ea063	Remove VKeyTranslatorFactory.createVKey(String) (#1312 ) * Remove VKeyTranslatorFactory.createVKey(String) This method serves the same function as VKey.fromWebsafeKey(), and isn't used anywhere. Move the test for it into VKeyTest and use it to instead test fromWebsafeKey() (which didn't previously have a test).	2021-09-09 12:24:02 -04:00
Rachel Guan	0a25182fea	Keep new lines/spaces as the way it shows in config yaml (#1311 ) * Paste the email content as the way it is in doc.	2021-09-08 22:00:32 -04:00
Lai Jiang	f1dcb1299f	Reformat a file (#1305 ) * Reformat a file Constants should be SNAKE_CASE named and static fields usually go before instance fields.	2021-09-03 16:56:57 -04:00
gbrodman	d46594c610	Select correct history entries when creating canceling records (#1309 )	2021-09-03 11:25:53 -04:00
gbrodman	0a9fa8cf23	Add RetryingTest annotation to flaky RDE test (#1306 ) I'm not sure why this test is failing. It's failing saying that the listObjects call is failing to include "soy_2000-01-01_thin_S1_R1.xml.ghostryde" in the results, however the verifyFiles method that we call right beforehand verifies that file and its contents	2021-09-03 09:39:55 -04:00
sarahcaseybot	db4bf90538	Remove Datastore references from DomainLabelEntry (#1307 )	2021-09-02 17:55:55 -04:00
Rachel Guan	d6127e4c0c	Log registrar and certificate info before sending an email (#1308 ) * Log registrar and certificate info before sending an email	2021-09-02 16:58:58 -04:00
sarahcaseybot	447bfa162b	Remove Datastore references in BaseDomainLabelList (#1304 )	2021-09-02 13:21:45 -04:00
Rachel Guan	c9efa61198	Update expiring certificate notification email content (#1294 ) * Update expiring certificate notification email content * Improve test cases	2021-08-30 11:51:05 -04:00
gbrodman	054c0625a8	Add SQL functionality to DeleteProberDataAction (#1218 ) This includes a change to how the JPA transaction manager handles existence and load checks for entities with compound IDs. Previously, we relied on the fields all being named the same in the ID entity and the parent entity. This didn't work for History objects (e.g. DomainHistory) so existence checks were broken. Now, we use the methods the same way that Hibernate does (if possible). Note as well that there's a bit of semi-duplicated logic in DeleteProberDataAction (between the mapper and the SQL logic). The mapper code will be deleted once we've shifted to SQL, and for now it's better to keep it in place for logging purposes.	2021-08-27 21:09:08 -04:00
gbrodman	b03639d7fc	Implement read-only transaction manager modes for R3.0 migration (#1241 ) This involves: - Altering both transaction managers to check for a read-only mode at the start of standard write actions (e.g. delete, put). - Altering both raw layers (entity manager, ofy) to throw exceptions on write actions as well - Implementing bypass routes for reading / setting / removing the schedule itself so that we don't get "stuck"	2021-08-27 15:59:16 -04:00
Rachel Guan	bd9af0de84	Improve logging for SendExpiringCertificateNotificationEmailAction.java (#1302 ) * Improve logging for SendExpiringCertificateNotificationEmailAction.java	2021-08-27 13:11:54 -04:00
gbrodman	ae911a5280	Fix semantic merge conflict accidentally introduced (#1301 )	2021-08-26 16:15:56 -04:00
gbrodman	d57597f40f	Clean up ReplicateToDatastoreAction and tests (#1299 ) * Clean up ReplicateToDatastoreAction and tests 1. applyTransaction should throw an error if it fails; this allows us to have more information in the caller (and it shouldn't usually happen) 2. Set a response code + payload now, since this is an action that is called by cron 3. Add a method to the test log subject that allows us to check if a severe log with a particular Throwable cause was logged (since the cause isn't contained in the log message itself directly)	2021-08-25 14:45:05 -06:00
gbrodman	2641d0d462	Save indexes when replaying EppResources SQL->DS (#1300 ) * Save indexes when replaying EppResources SQL->DS We implement this similarly to how we implement the beforeSqlSaveOnReplay callback in the other direction -- a beforeDatastoreSaveOnReplay method that is called when replaying a Mutation to Datastore. This means that the asynchronous replay will create the relevant ForeignKeyIndex and EppResourceIndex objects for EppResources saved when SQL is primary.	2021-08-25 14:44:44 -06:00
sarahcaseybot	5b41f0b9b6	Remove ClaimsList from Datastore Schema (#1298 ) * Remove ClaimsList from Datastore schema * Remove some Datastore references * Remove unnecessary annotations	2021-08-25 11:58:44 -04:00
Lai Jiang	1a26677d72	Implement a util class to manage push queues using Cloud Tasks API (#1290 ) * Implement a util class to manage push queues using Cloud Tasks API Push queues were part of App Engine when they debuted. As a result the Task Queue API were part of the App Engine SDK and can only be used in App Engine classic runtime. The new Cloud Tasks API can be used in any runtime but it only supports push queues. In this PR we implement a util class (CloudTasksUtils) like TaskQueueUtils to handle enqueuing tasks to push queues using Cloud Tasks. One action (TldFanoutAction) was converted to use the new API as a demo. Mass migration of other call sites of the old API will follow in a separate PR. TESTED=deployed to alpha and verified that tasks are corrected enqueued and executed.	2021-08-24 21:13:54 -04:00
gbrodman	f1beeb4016	Add double-replay to remaining existing ReplayExtension calls (#1297 ) The only other change is that we need to reconstitute serverApproveEntities for DomainTransferData in more situations (to fill out the ofy keys)	2021-08-23 15:08:09 -04:00
gbrodman	5c33286056	Compare SQL and Datastore objects in SQL->DS replay testing (#1291 ) Add double-replay to the Host*Flow tests to show how this works. The only change to the double replay itself is that now we store the Datastore entity in the TransactionEntity object -- this is because we use Objectify to serialize the objects into bytes and we need it to know about the entity in question.	2021-08-23 11:05:14 -04:00
gbrodman	603a95d719	Add DS->SQL replay cron job to production (#1292 ) * Add DS->SQL replay cron job to production This won't do anything until we set the migration schedule to DATASTORE_PRIMARY. Actions in order: 1. Add this cron job (it'll be a no-op) 2. Run the init-sql-pipeline to populate production's SQL DB 3. Set the SqlReplayCheckpoint to a time before the smear backup that was used in step #1 (maybe 30 minutes) 4. Set the database migration schedule to transition to DATASTORE_PRIMARY at some point	2021-08-23 07:59:51 -06:00
gbrodman	0a3774d3f7	Add withDsAndCloudSql to flow test (#1293 ) * Add withDsAndCloudSql to flow test Not sure why this wasn't failing before	2021-08-20 09:07:38 -06:00
Rachel Guan	cc60b27dd3	Add sending notification email mechanism for expiring certificates (#1179 ) * Resolve rebase conflict * Fix and imporove based on feedback.	2021-08-19 12:49:45 -04:00
Rachel Guan	52c18f9967	Remove files that are not longer used for create/update premium list (#1288 ) * Remove files that are not longer used for create/update premium list * Remove comments/notes related to create/update premium list action files	2021-08-18 14:04:57 -04:00
gbrodman	5339b3cb6c	Remove -- from crash cron comment (#1289 ) This is causing the release build to fail, see https://pantheon.corp.google.com/cloud-build/builds;region=global/22ec980b-c2b6-43fe-994a-aa98c0dbc9d4?project=domain-registry-dev	2021-08-18 11:30:01 -04:00
sarahcaseybot	d18dab3327	Remove ReservedList from Datastore schema (#1285 ) * Remove ReservedList from Datastore schema * Remove some Datastore references * Add a different non-replicated entity to ReplayCommitLogsToSqlActionTest	2021-08-17 16:56:00 -04:00
gbrodman	61932c1809	Use direct ofyTm reference when clearing cache in tool (#1287 ) We shouldn't reference tm() at all before initializing the JPA transaction manager, since tm() looks at the database migration schedule when figuring out which transaction manager to use.	2021-08-17 13:20:17 -06:00
sarahcaseybot	8eb8c810e8	Remove DeleteEntityAction (#1282 )	2021-08-16 13:21:00 -04:00
Weimin Yu	c03a7b0b33	Update cron jobs in crash (#1284 ) * Update cron jobs in crash Add wipeout cron jobs for the duration of migration testing with production data. * Disable Datastore-related cron jobs	2021-08-16 12:03:45 -04:00
gbrodman	7a4c109b36	Remove recursive load in DBMSS cache (#1286 ) * Remove recursive load in DBMSS cache This occurs because if we do a standard transaction, the JpaTxnManager checks to see if we should be doing backups, which involves loading the migration state schedule (causing the recursion). When starting the transaction to load the schedule, we should explicitly transactWithoutBackup so there's no need to check. This wasn't hit in tests because we previously manually set the replication to not occur in the JpaTransactionManagerExtension -- we remove that and related setters.	2021-08-14 12:34:23 -06:00
Ben McIlwain	22b1b8d21a	Add instructions for two-step DB schema updates (#1283 ) * Add instructions for two-step DB schema updates These expanded steps are required by the recent enabling of the SQL integration test suite.	2021-08-13 17:21:38 -04:00
Weimin Yu	5bbabadafd	Generate string to uniquely identify a SqlEntity (#1271 ) * Generate string to uniquely identify a SqlEntity Add a method to SqlEntity that returns a string built from the entity's primary key(s). This string can be used in logging.	2021-08-13 16:22:54 -04:00
Ben McIlwain	6c73161ff8	Add the domain DNS refresh request time field to the DB schema (#1280 ) * Add the domain DNS refresh request time field to the DB schema This isn't used yet, but it will eventually be the replacement for the dns-pull task queue once we get further in the migration. * Remove index	2021-08-13 15:32:18 -04:00
Rachel Guan	7faee04422	Modify class name to remove checkstyleTest warning (#1281 )	2021-08-13 14:16:58 -04:00
Ben McIlwain	b340b2b5e9	Add tx/s instrumentation to replay action and re-enable it on sandbox (#1276 )	2021-08-12 18:33:47 -04:00
gbrodman	7f733cd16d	Store DatabaseMigrationSchedule in SQL instead of Datastore (#1269 ) * Store DatabaseMigrationSchedule in SQL instead of Datastore This requires messing around with some of the JPA unit test rule creation since it requires saving / retrieving the schedule pretty much always (which itself includes the hstore extension).	2021-08-12 15:57:31 -06:00
Ben McIlwain	60469479a4	Consolidate all remaining schema classes into model package (#1278 )	2021-08-12 13:38:50 -04:00
Ben McIlwain	5158673f21	Consolidate all Registry/TLD-related classes into google.registry.model.tld (#1277 )	2021-08-11 18:04:51 -04:00