1
0
mirror of https://github.com/google/nomulus synced 2026-05-21 23:31:51 +00:00

Compare commits

...

23 Commits

Author SHA1 Message Date
Weimin Yu
1dcf34ccc2 Report BSA block status in DomainCheckFlow (#2288)
- Registered names are not affected.

- Reserved names are not affected.

- Names that are none of the above and match some BSA labels are
  reported as blocked.
2024-01-12 17:17:51 -05:00
Weimin Yu
9273d2bf15 Remove deleted BSA labels from database (#2286)
Fixed the bug that retains deleted BSA labels in the database.

Added a few simple end-to-end tests for BSA download.
2024-01-12 14:20:56 -05:00
Ben McIlwain
036d35c11a Make the BSA upload unvailable domains task work with HTTP GET as well (#2287)
Apparently Google Cloud Scheduler can only do GET, not POST, for some reason.
2024-01-12 12:17:52 -05:00
Ben McIlwain
a8ce34586d Add production cronjobs for BSA download/upload actions (#2285)
* Add production cronjob for uploading BSA unavailable names

* Add production cronjob for BSA download action
2024-01-11 18:38:52 -05:00
Ben McIlwain
26fb04f00c Add sandbox cronjob for upload BSA unavailable names (#2284) 2024-01-11 12:21:40 -05:00
Ben McIlwain
9d4c38684a Add a cron schedule for the BSA upload unavailable domains task (#2280)
Also fixes the action taken in the case where zero unavailable domains are
found, and temporarily changes over to using the primary DB (because the replica
transaction was timing out at 30 seconds on large databases). I'll switch this
over to use batching and move it back to replica afterwards, but this should
unblock us temporarily.
2024-01-10 14:34:06 -05:00
Pavlo Tkach
d7edd27cdd Add support for Ubuntu20 on kokoro (#2279) 2024-01-10 14:32:34 -05:00
Ben McIlwain
265d69051b Map /_dr/task/uploadBsaUnavailableNames in BSA service's web.xml (#2276)
This should have been part of PR #2265 but we all missed it.

Also adds a couple of useful logging statements.
2024-01-09 18:51:23 -05:00
Lai Jiang
b5d2b56426 Build Nomulus with Java 17 (#2255)
This PR makes it possible to build the Nomulus code base using Java 17.
Building with Java 11 continue to be possible and the resulting bytecodes are
still at Java 8 level. Also upgraded Gradle to 8.5.

There are several necessary changes to make this happen:

1. Some Gradle plugins need to be upgraded to support Java 17, notably
errorprone. As a result, a lot more "errors" were caught and corrected.

2. All test code are now built and run at Java 8 level. Previously it was left
undefined (which defaults to the version of the compiler) and had led to
situations where we inadvertently called Java 8+ features in production that
are not caught by tests. The change also made the java8compatibility subproject
obsolete, which is therefore removed.

3. Removed the docs subproject. Its main use is to generate flows.md, but it
relies heavily on Java internal APIs that have changed significant with each
version. Upgrading to Java 11 required extensive refactoring of the code there,
and Java 17 again removed many APIs that were used. I don't think it is worth
the maintenance effort just to have a tool to generate flows.md which no one
actually reads.

4. Capped a few GCP dependencies because the latest version depends on
 grpc-java >= 1.59.0, which includes a runtime incompatibility
 (https://github.com/grpc/grpc-java/releases/tag/v1.59.0).
2024-01-09 15:56:37 -05:00
Ben McIlwain
e79c63142a Add a daily batch action to upload unavailable domains to BSA (#2265) 2024-01-09 14:52:07 -05:00
Weimin Yu
f8ac7afc33 Check BSA block status in CheckApi (#2271)
* Check BSA block status in CheckApi

Checks for and reports BSA block status if the name is not registered or
reserved.

Also moves CheckApiActionTest to standardTest. Whatever problem forcing
it to another suite has apparently disappeared.
2024-01-09 13:19:07 -05:00
Ben McIlwain
e56e751652 Fix build warnings (#2274)
All of these were causing warnings to appear during a build of the codebase.
2024-01-09 13:15:54 -05:00
Ben McIlwain
cfdf12aa7d Add OkHttp as a library used by the core Nomulus project (#2272)
This will be used in a subsequent PR (see #2265) to upload the unavailable
domain names list to the BSA endpoint.
2024-01-09 12:53:15 -05:00
Weimin Yu
811b385544 Add cron config for the bsaDownload job in Sandbox (#2267)
* Add cron config for the bsaDownload job in Sandbox
2024-01-05 11:10:48 -05:00
Weimin Yu
3f5c9d1246 BSA for integration test (#2256)
Supports the full blocklist download cycle (download, diffing, diff-apply, and order-status reporting) and the refreshing of unblockable domains.

Submitted due to tight deadline. We will conduct post-submit review and refactoring.
2024-01-05 11:09:40 -05:00
Pavlo Tkach
5315752bc0 Add ICANN csv response GZIP decoding (#2269) 2024-01-04 18:35:21 -05:00
Pavlo Tkach
4eee7b8c0d Add support for bsa service to cloud tasks config (#2268) 2024-01-03 17:38:42 -05:00
Weimin Yu
ecb39d5899 Use custom whois message for bsa-blocked domain (#2241)
* Use custom whois message for bsa-blocked domain
2024-01-02 14:40:34 -05:00
Lai Jiang
42b508427b Bypass SCRYPT hashing in tests (#2262)
SCRYPT is much computationally heavier than SHA265 (by design), which
resulted in test run time doubling due to most tests initializing canned
data that uses hashing.

Since out tests are not verifying the correctness of a specific hashing
algorithm anyway, this PR makes it so that simple concatenation is used
in tests.

Also moved RegistryEnvironment to the util subproject so it can be called by
PasswordUtils, which makes sense as it is a utility class.
2023-12-21 16:17:37 -05:00
sarahcaseybot
20b5b43501 Add type conversion to TimedTransitionProperty<Money> deserializer to handle JPY currency (#2258)
* Add BigInt conversion to TimedTransitionProperty<Money> deserializer to handle JPY currency

* Remove unnecessary lines in test

* Add eap schedule check

* Don't use raw LinkedHashMap type

* add timezone
2023-12-21 12:59:54 -05:00
Lai Jiang
08285f5de7 Greatly increase the upper limit of proxy instances in production (#2259)
From our investigation, the Monday night WHOIS storm does not cause any
strain to the backend system. The backend latency metrics are all well within
the limits. The latency measured from the proxy matches observed latency
by the prober, and we see that the "used" CPU is 1.5x of "requested" CPU
during the time when the latency is above the threshold.

Making this change hopefully removes the proxy as the bottleneck and
ameliorate the pages.
2023-12-20 15:37:29 -05:00
Pavlo Tkach
fb4c5b457d Prevent reusing ianaId for real registrars (#2257) 2023-12-20 15:20:04 -05:00
Pavlo Tkach
781c212275 Add IcannHttpReporter failed response logging (#2252) 2023-12-18 11:03:33 -05:00
262 changed files with 8234 additions and 5215 deletions

View File

@@ -29,7 +29,7 @@ buildscript {
dependencies {
classpath 'com.google.cloud.tools:appengine-gradle-plugin:2.4.1'
classpath 'net.ltgt.gradle:gradle-errorprone-plugin:2.0.2'
classpath 'net.ltgt.gradle:gradle-errorprone-plugin:3.1.0'
classpath 'org.sonatype.aether:aether-api:1.13.1'
classpath 'org.sonatype.aether:aether-impl:1.13.1'
}
@@ -41,9 +41,9 @@ plugins {
// Re-enable when compatible with Gradle 8
// id 'nebula.lint' version '16.0.2'
id 'net.ltgt.errorprone' version '2.0.2'
id 'net.ltgt.errorprone' version '3.1.0'
id 'checkstyle'
id 'com.github.johnrengelman.shadow' version '5.1.0'
id 'com.github.johnrengelman.shadow' version '8.1.1'
// NodeJs plugin
id "com.github.node-gradle.node" version "3.0.1"
@@ -307,8 +307,7 @@ subprojects {
afterEvaluate {
if (rootProject.enableDependencyLocking.toBoolean()
&& project.name != 'integration'
&& project.name != 'java8compatibility') {
&& project.name != 'integration') {
// The ':integration' project runs server/schema integration tests using
// dynamically specified jars with no transitive dependency. Therefore
// dependency-locking does not make sense. Furthermore, during
@@ -316,9 +315,6 @@ subprojects {
// immutable. Locking activation would trigger an invalid operation
// exception.
//
// The ':java8compatibility' project is test-only. Its source does not go
// into production.
//
// For all other projects, due to problem with the gradle-license-report
// plugin, the dependencyLicenseReport configuration must opt out of
// dependency-locking. See dependency_lic.gradle for the reason why.
@@ -362,9 +358,13 @@ subprojects {
apply from: "${rootDir.path}/java_common.gradle"
if (project.name != 'docs') {
if (!rootProject.enableCrossReferencing.toBoolean()) {
compileJava {
// TODO: Remove this once we migrate off AppEngine.
// TODO: Remove this once we migrate off of AppEngine.
options.release = 8
}
compileTestJava {
// TODO: Remove this once we migrate off of AppEngine.
options.release = 8
}
}

View File

@@ -36,7 +36,7 @@ plugins {
// We don't anticipate enabling the Gradle lint plugin because they will not support Kotlin
// See https://github.com/nebula-plugins/gradle-lint-plugin/issues/166
// id 'nebula.lint' version '16.0.2'
id("net.ltgt.errorprone") version "2.0.2"
id("net.ltgt.errorprone") version "3.1.0"
checkstyle
id("com.diffplug.spotless") version "6.20.0"
}

View File

@@ -15,14 +15,14 @@ com.squareup.okhttp3:okhttp:4.10.0=classpath
com.squareup.okio:okio-jvm:3.0.0=classpath
com.squareup.okio:okio:3.0.0=classpath
dev.equo.ide:solstice:1.3.1=classpath
net.ltgt.errorprone:net.ltgt.errorprone.gradle.plugin:2.0.2=classpath
net.ltgt.gradle:gradle-errorprone-plugin:2.0.2=classpath
net.ltgt.errorprone:net.ltgt.errorprone.gradle.plugin:3.1.0=classpath
net.ltgt.gradle:gradle-errorprone-plugin:3.1.0=classpath
org.eclipse.jgit:org.eclipse.jgit:6.6.0.202305301015-r=classpath
org.eclipse.platform:org.eclipse.osgi:3.18.300=classpath
org.jetbrains.kotlin:kotlin-stdlib-common:1.9.0=classpath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.9.0=classpath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.0=classpath
org.jetbrains.kotlin:kotlin-stdlib:1.9.0=classpath
org.jetbrains.kotlin:kotlin-stdlib-common:1.9.20=classpath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.8.0=classpath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.0=classpath
org.jetbrains.kotlin:kotlin-stdlib:1.9.20=classpath
org.jetbrains:annotations:13.0=classpath
org.slf4j:slf4j-api:1.7.36=classpath
org.tukaani:xz:1.9=classpath

View File

@@ -1,11 +1,11 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
aopalliance:aopalliance:1.0=buildScriptClasspath,compileClasspath
aopalliance:aopalliance:1.0=annotationProcessor,buildScriptClasspath,compileClasspath
args4j:args4j:2.0.23=buildScriptClasspath,compileClasspath
com.fasterxml.jackson.core:jackson-core:2.14.2=buildScriptClasspath,compileClasspath
com.fasterxml.jackson:jackson-bom:2.14.2=buildScriptClasspath,compileClasspath
com.github.ben-manes.caffeine:caffeine:2.7.0=annotationProcessor
com.github.ben-manes.caffeine:caffeine:3.0.5=annotationProcessor
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor
com.google.android:annotations:4.1.1.4=buildScriptClasspath
com.google.api-client:google-api-client:2.2.0=buildScriptClasspath,compileClasspath
@@ -21,29 +21,30 @@ com.google.api:gax:2.31.0=buildScriptClasspath,compileClasspath
com.google.apis:google-api-services-storage:v1-rev20230301-2.0.0=buildScriptClasspath,compileClasspath
com.google.auth:google-auth-library-credentials:1.19.0=buildScriptClasspath,compileClasspath
com.google.auth:google-auth-library-oauth2-http:1.19.0=buildScriptClasspath,compileClasspath
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor
com.google.auto.value:auto-value-annotations:1.10.1=buildScriptClasspath,compileClasspath
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor
com.google.auto.value:auto-value:1.10.4=annotationProcessor
com.google.auto:auto-common:0.10=annotationProcessor
com.google.auto:auto-common:1.2.1=annotationProcessor
com.google.cloud:google-cloud-core-grpc:2.21.0=buildScriptClasspath,compileClasspath
com.google.cloud:google-cloud-core-http:2.21.0=buildScriptClasspath,compileClasspath
com.google.cloud:google-cloud-core:2.21.0=buildScriptClasspath,compileClasspath
com.google.cloud:google-cloud-storage:2.22.6=buildScriptClasspath,compileClasspath
com.google.code.findbugs:jFormatString:3.0.0=annotationProcessor
com.google.code.findbugs:jsr305:3.0.2=annotationProcessor,buildScriptClasspath,compileClasspath
com.google.code.gson:gson:2.10.1=buildScriptClasspath,compileClasspath
com.google.common.html.types:types:1.0.6=buildScriptClasspath,compileClasspath
com.google.errorprone:error_prone_annotation:2.3.4=annotationProcessor
com.google.errorprone:error_prone_annotations:2.18.0=buildScriptClasspath,compileClasspath
com.google.errorprone:error_prone_annotations:2.3.4=annotationProcessor
com.google.errorprone:error_prone_check_api:2.3.4=annotationProcessor
com.google.errorprone:error_prone_core:2.3.4=annotationProcessor
com.google.errorprone:error_prone_type_annotations:2.3.4=annotationProcessor
com.google.errorprone:error_prone_annotation:2.23.0=annotationProcessor
com.google.errorprone:error_prone_annotations:2.23.0=annotationProcessor,buildScriptClasspath,compileClasspath
com.google.errorprone:error_prone_check_api:2.23.0=annotationProcessor
com.google.errorprone:error_prone_core:2.23.0=annotationProcessor
com.google.errorprone:error_prone_type_annotations:2.23.0=annotationProcessor
com.google.escapevelocity:escapevelocity:0.9.1=buildScriptClasspath,compileClasspath
com.google.guava:failureaccess:1.0.1=annotationProcessor,buildScriptClasspath,compileClasspath
com.google.guava:guava-parent:32.1.1-jre=buildScriptClasspath,compileClasspath
com.google.guava:guava:27.0.1-jre=annotationProcessor
com.google.guava:guava:32.1.1-jre=buildScriptClasspath,compileClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor
com.google.guava:failureaccess:1.0.1=annotationProcessor
com.google.guava:failureaccess:1.0.2=buildScriptClasspath,compileClasspath
com.google.guava:guava-parent:32.1.1-jre=annotationProcessor
com.google.guava:guava:32.1.1-jre=annotationProcessor
com.google.guava:guava:33.0.0-jre=buildScriptClasspath,compileClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=buildScriptClasspath,compileClasspath
com.google.http-client:google-http-client-apache-v2:1.43.3=buildScriptClasspath,compileClasspath
com.google.http-client:google-http-client-appengine:1.43.3=buildScriptClasspath,compileClasspath
com.google.http-client:google-http-client-gson:1.43.3=buildScriptClasspath,compileClasspath
@@ -51,19 +52,20 @@ com.google.http-client:google-http-client-jackson2:1.43.3=buildScriptClasspath,c
com.google.http-client:google-http-client:1.43.3=buildScriptClasspath,compileClasspath
com.google.inject.extensions:guice-multibindings:4.1.0=buildScriptClasspath,compileClasspath
com.google.inject:guice:4.1.0=buildScriptClasspath,compileClasspath
com.google.j2objc:j2objc-annotations:1.1=annotationProcessor
com.google.inject:guice:5.1.0=annotationProcessor
com.google.j2objc:j2objc-annotations:2.8=buildScriptClasspath,compileClasspath
com.google.jsinterop:jsinterop-annotations:1.0.1=buildScriptClasspath,compileClasspath
com.google.oauth-client:google-oauth-client:1.34.1=buildScriptClasspath,compileClasspath
com.google.protobuf:protobuf-java-util:3.23.2=buildScriptClasspath,compileClasspath
com.google.protobuf:protobuf-java:3.19.6=annotationProcessor
com.google.protobuf:protobuf-java:3.23.2=buildScriptClasspath,compileClasspath
com.google.protobuf:protobuf-java:3.4.0=annotationProcessor
com.google.re2j:re2j:1.6=buildScriptClasspath
com.google.template:soy:2021-02-01=buildScriptClasspath,compileClasspath
com.googlecode.java-diff-utils:diffutils:1.3.0=annotationProcessor
com.ibm.icu:icu4j:57.1=buildScriptClasspath,compileClasspath
commons-codec:commons-codec:1.15=buildScriptClasspath,compileClasspath
commons-logging:commons-logging:1.2=buildScriptClasspath,compileClasspath
io.github.eisop:dataflow-errorprone:3.34.0-eisop1=annotationProcessor
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor
io.grpc:grpc-alts:1.55.3=buildScriptClasspath,compileClasspath
io.grpc:grpc-api:1.55.3=buildScriptClasspath,compileClasspath
io.grpc:grpc-auth:1.55.3=buildScriptClasspath,compileClasspath
@@ -84,16 +86,13 @@ io.opencensus:opencensus-proto:0.2.0=buildScriptClasspath
io.perfmark:perfmark-api:0.26.0=buildScriptClasspath
javax.annotation:javax.annotation-api:1.3.2=buildScriptClasspath,compileClasspath
javax.annotation:jsr250-api:1.0=buildScriptClasspath,compileClasspath
javax.inject:javax.inject:1=buildScriptClasspath,compileClasspath
org.apache.commons:commons-lang3:3.12.0=buildScriptClasspath,compileClasspath
org.apache.commons:commons-text:1.10.0=buildScriptClasspath,compileClasspath
javax.inject:javax.inject:1=annotationProcessor,buildScriptClasspath,compileClasspath
org.apache.commons:commons-lang3:3.13.0=buildScriptClasspath,compileClasspath
org.apache.commons:commons-text:1.11.0=buildScriptClasspath,compileClasspath
org.apache.httpcomponents:httpclient:4.5.14=buildScriptClasspath,compileClasspath
org.apache.httpcomponents:httpcore:4.4.16=buildScriptClasspath,compileClasspath
org.checkerframework:checker-qual:3.0.0=annotationProcessor
org.checkerframework:checker-qual:3.33.0=buildScriptClasspath,compileClasspath
org.checkerframework:dataflow:3.0.0=annotationProcessor
org.checkerframework:javacutil:3.0.0=annotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.17=annotationProcessor
org.checkerframework:checker-qual:3.33.0=annotationProcessor
org.checkerframework:checker-qual:3.41.0=buildScriptClasspath,compileClasspath
org.codehaus.mojo:animal-sniffer-annotations:1.23=buildScriptClasspath
org.conscrypt:conscrypt-openjdk-uber:2.5.2=buildScriptClasspath,compileClasspath
org.json:json:20160212=buildScriptClasspath,compileClasspath
@@ -102,9 +101,6 @@ org.ow2.asm:asm-commons:7.0=buildScriptClasspath,compileClasspath
org.ow2.asm:asm-tree:7.0=buildScriptClasspath,compileClasspath
org.ow2.asm:asm-util:7.0=buildScriptClasspath,compileClasspath
org.ow2.asm:asm:7.0=buildScriptClasspath,compileClasspath
org.pcollections:pcollections:2.1.2=annotationProcessor
org.plumelib:plume-util:1.0.6=annotationProcessor
org.plumelib:reflection-util:0.0.2=annotationProcessor
org.plumelib:require-javadoc:0.1.0=annotationProcessor
org.pcollections:pcollections:3.1.4=annotationProcessor
org.threeten:threetenbp:1.6.8=buildScriptClasspath,compileClasspath
empty=

View File

@@ -35,7 +35,7 @@ import java.util.function.Supplier;
import java.util.stream.Collectors;
/**
* Creates the files for a web-page summary of a given {@Link ProjectData}.
* Creates the files for a web-page summary of a given {@link ProjectData}.
*
* <p>The main job of this class is rendering a tailored cover page that includes information about
* the project and any task that ran.

View File

@@ -11,8 +11,8 @@ com.diffplug.spotless:spotless-lib:2.40.0=classpath
com.diffplug.spotless:spotless-plugin-gradle:6.20.0=classpath
com.dorongold.plugins:task-tree:2.1.0=classpath
com.dorongold.task-tree:com.dorongold.task-tree.gradle.plugin:2.1.0=classpath
com.github.jengelman.gradle.plugins:shadow:5.1.0=classpath
com.github.johnrengelman.shadow:com.github.johnrengelman.shadow.gradle.plugin:5.1.0=classpath
com.github.johnrengelman.shadow:com.github.johnrengelman.shadow.gradle.plugin:8.1.1=classpath
com.github.johnrengelman:shadow:8.1.1=classpath
com.github.node-gradle.node:com.github.node-gradle.node.gradle.plugin:3.0.1=classpath
com.github.node-gradle:gradle-node-plugin:3.0.1=classpath
com.google.cloud.tools:appengine-gradle-plugin:2.4.1=classpath
@@ -29,35 +29,34 @@ com.googlecode.javaewah:JavaEWAH:1.2.3=classpath
com.squareup.okhttp3:okhttp:4.10.0=classpath
com.squareup.okio:okio-jvm:3.0.0=classpath
com.squareup.okio:okio:3.0.0=classpath
commons-io:commons-io:2.6=classpath
commons-io:commons-io:2.11.0=classpath
dev.equo.ide:solstice:1.3.1=classpath
net.ltgt.errorprone:net.ltgt.errorprone.gradle.plugin:2.0.2=classpath
net.ltgt.gradle:gradle-errorprone-plugin:2.0.2=classpath
org.apache.ant:ant-launcher:1.9.7=classpath
org.apache.ant:ant:1.9.7=classpath
net.ltgt.errorprone:net.ltgt.errorprone.gradle.plugin:3.1.0=classpath
net.ltgt.gradle:gradle-errorprone-plugin:3.1.0=classpath
org.apache.ant:ant-launcher:1.10.13=classpath
org.apache.ant:ant:1.10.13=classpath
org.apache.commons:commons-compress:1.20=classpath
org.apache.commons:commons-lang3:3.5=classpath
org.checkerframework:checker-qual:2.10.0=classpath
org.codehaus.plexus:plexus-utils:3.0.24=classpath
org.codehaus.plexus:plexus-utils:3.5.1=classpath
org.eclipse.jgit:org.eclipse.jgit:6.6.0.202305301015-r=classpath
org.eclipse.platform:org.eclipse.osgi:3.18.300=classpath
org.glassfish:javax.json:1.0.4=classpath
org.jdom:jdom2:2.0.6=classpath
org.jdom:jdom2:2.0.6.1=classpath
org.jetbrains.kotlin:kotlin-stdlib-common:1.6.20=classpath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.5.31=classpath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.5.31=classpath
org.jetbrains.kotlin:kotlin-stdlib:1.6.20=classpath
org.jetbrains:annotations:13.0=classpath
org.ow2.asm:asm-analysis:7.0-beta=classpath
org.ow2.asm:asm-commons:7.0-beta=classpath
org.ow2.asm:asm-tree:7.0-beta=classpath
org.ow2.asm:asm:7.0-beta=classpath
org.ow2.asm:asm-commons:9.4=classpath
org.ow2.asm:asm-tree:9.4=classpath
org.ow2.asm:asm:9.4=classpath
org.slf4j:slf4j-api:1.7.36=classpath
org.sonatype.aether:aether-api:1.13.1=classpath
org.sonatype.aether:aether-impl:1.13.1=classpath
org.sonatype.aether:aether-spi:1.13.1=classpath
org.sonatype.aether:aether-util:1.13.1=classpath
org.tukaani:xz:1.9=classpath
org.vafer:jdependency:2.1.1=classpath
org.vafer:jdependency:2.8.0=classpath
org.yaml:snakeyaml:1.21=classpath
empty=

View File

@@ -1,69 +1,67 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.7.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
aopalliance:aopalliance:1.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.github.ben-manes.caffeine:caffeine:2.9.3=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.github.ben-manes.caffeine:caffeine:3.0.5=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.auto.value:auto-value-annotations:1.10.1=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.auto:auto-common:0.10=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.code.findbugs:jFormatString:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.auto.value:auto-value-annotations:1.10.4=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.auto:auto-common:1.2.1=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.code.findbugs:jsr305:3.0.2=annotationProcessor,checkstyle,compileClasspath,deploy_jar,errorprone,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath,testing,testingAnnotationProcessor,testingCompileClasspath
com.google.errorprone:error_prone_annotation:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.19.1=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.errorprone:error_prone_annotations:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_annotation:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.23.0=annotationProcessor,compileClasspath,deploy_jar,errorprone,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath,testing,testingAnnotationProcessor,testingCompileClasspath
com.google.errorprone:error_prone_annotations:2.7.1=checkstyle
com.google.errorprone:error_prone_check_api:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_core:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_type_annotations:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.flogger:flogger:0.7.4=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.guava:failureaccess:1.0.1=annotationProcessor,checkstyle,compileClasspath,deploy_jar,errorprone,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath,testing,testingAnnotationProcessor,testingCompileClasspath
com.google.guava:guava-parent:32.1.2-jre=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.guava:guava:27.0.1-jre=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_core:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:error_prone_type_annotations:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.errorprone:javac:9+181-r4173-1=errorproneJavac
com.google.flogger:flogger:0.8=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.guava:failureaccess:1.0.1=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.guava:failureaccess:1.0.2=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.guava:guava-parent:32.1.1-jre=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.guava:guava:31.0.1-jre=checkstyle
com.google.guava:guava:32.1.2-jre=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,compileClasspath,deploy_jar,errorprone,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath,testing,testingAnnotationProcessor,testingCompileClasspath
com.google.j2objc:j2objc-annotations:1.1=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.guava:guava:32.1.1-jre=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.guava:guava:33.0.0-jre=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=checkstyle,compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.google.inject:guice:5.1.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.j2objc:j2objc-annotations:1.3=checkstyle
com.google.j2objc:j2objc-annotations:2.8=compileClasspath,testCompileClasspath,testingCompileClasspath
com.google.protobuf:protobuf-java:3.4.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.truth:truth:1.1.5=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.googlecode.java-diff-utils:diffutils:1.3.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.protobuf:protobuf-java:3.19.6=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
com.google.truth:truth:1.2.0=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
com.puppycrawl.tools:checkstyle:9.3=checkstyle
commons-beanutils:commons-beanutils:1.9.4=checkstyle
commons-collections:commons-collections:3.2.2=checkstyle
info.picocli:picocli:4.6.2=checkstyle
io.github.java-diff-utils:java-diff-utils:4.12=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
javax.inject:javax.inject:1=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
joda-time:joda-time:2.12.5=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
io.github.eisop:dataflow-errorprone:3.34.0-eisop1=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,compileClasspath,deploy_jar,errorprone,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath,testing,testingAnnotationProcessor,testingCompileClasspath
javax.inject:javax.inject:1=annotationProcessor,compileClasspath,deploy_jar,errorprone,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath,testing,testingAnnotationProcessor,testingCompileClasspath
joda-time:joda-time:2.12.6=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
junit:junit:4.13.2=testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
net.sf.saxon:Saxon-HE:10.6=checkstyle
org.antlr:antlr4-runtime:4.9.3=checkstyle
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
org.checkerframework:checker-compat-qual:2.5.3=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
org.checkerframework:checker-qual:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.checkerframework:checker-qual:3.12.0=checkstyle
org.checkerframework:checker-qual:3.35.0=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
org.checkerframework:dataflow:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.checkerframework:javacutil:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.17=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.checkerframework:checker-qual:3.33.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.checkerframework:checker-qual:3.42.0=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
org.hamcrest:hamcrest-core:1.3=testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
org.jacoco:org.jacoco.agent:0.8.9=jacocoAgent,jacocoAnt
org.jacoco:org.jacoco.ant:0.8.9=jacocoAnt
org.jacoco:org.jacoco.core:0.8.9=jacocoAnt
org.jacoco:org.jacoco.report:0.8.9=jacocoAnt
org.javassist:javassist:3.28.0-GA=checkstyle
org.junit.jupiter:junit-jupiter-api:5.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.10.0=testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:5.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.10.1=testCompileClasspath,testRuntimeClasspath
org.junit:junit-bom:5.10.1=testCompileClasspath,testRuntimeClasspath
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-commons:9.5=jacocoAnt
org.ow2.asm:asm-tree:9.5=jacocoAnt
org.ow2.asm:asm:9.5=compileClasspath,deploy_jar,jacocoAnt,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
org.pcollections:pcollections:2.1.2=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.plumelib:plume-util:1.0.6=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.plumelib:reflection-util:0.0.2=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.plumelib:require-javadoc:0.1.0=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.ow2.asm:asm:9.5=jacocoAnt
org.ow2.asm:asm:9.6=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath,testing,testingCompileClasspath
org.pcollections:pcollections:3.1.4=annotationProcessor,errorprone,testAnnotationProcessor,testingAnnotationProcessor
org.reflections:reflections:0.10.2=checkstyle
empty=errorproneJavac,testingCompile,testingRuntime,testingRuntimeClasspath
empty=testingCompile,testingRuntime,testingRuntimeClasspath

View File

@@ -0,0 +1,43 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.util;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.Iterators.partition;
import static com.google.common.collect.Iterators.transform;
import static com.google.common.collect.Streams.stream;
import static java.lang.Math.min;
import com.google.common.collect.ImmutableList;
import java.util.stream.Stream;
/** Utilities for breaking up a {@link Stream} into batches. */
public final class BatchedStreams {
static final int MAX_BATCH = 1024 * 1024;
private BatchedStreams() {}
/**
* Transform a flat {@link Stream} into a {@code Stream} of batches.
*
* <p>Closing the returned stream does not close the original stream.
*/
public static <T> Stream<ImmutableList<T>> toBatches(Stream<T> stream, int batchSize) {
checkArgument(batchSize > 0, "batchSize must be a positive integer.");
return stream(
transform(partition(stream.iterator(), min(MAX_BATCH, batchSize)), ImmutableList::copyOf));
}
}

View File

@@ -0,0 +1,65 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.util;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.util.BatchedStreams.toBatches;
import static java.util.stream.Collectors.counting;
import static java.util.stream.Collectors.groupingBy;
import static org.junit.Assert.assertThrows;
import com.google.common.collect.ImmutableList;
import java.util.stream.IntStream;
import java.util.stream.Stream;
import org.junit.jupiter.api.Test;
/** Unit tests for {@link BatchedStreams}. */
public class BatchedStreamsTest {
@Test
void invalidBatchSize() {
assertThat(assertThrows(IllegalArgumentException.class, () -> toBatches(Stream.of(), 0)))
.hasMessageThat()
.contains("must be a positive integer");
}
@Test
void batch_success() {
// 900_002 elements -> 900 1K-batches + 1 2-element-batch
Stream<Integer> data = IntStream.rangeClosed(0, 900_001).boxed();
assertThat(
toBatches(data, 1000).map(ImmutableList::size).collect(groupingBy(x -> x, counting())))
.containsExactly(1000, 900L, 2, 1L);
}
@Test
void batch_partialBatch() {
Stream<Integer> data = Stream.of(1, 2, 3);
assertThat(
toBatches(data, 1000).map(ImmutableList::size).collect(groupingBy(x -> x, counting())))
.containsExactly(3, 1L);
}
@Test
void batch_truncateBatchSize() {
// 2M elements -> 2 1M-batches despite the user-specified 2M batch size.
Stream<Integer> data = IntStream.range(0, 1024 * 2048).boxed();
assertThat(
toBatches(data, 2_000_000)
.map(ImmutableList::size)
.collect(groupingBy(x -> x, counting())))
.containsExactly(1024 * 1024, 2L);
}
}

View File

@@ -49,13 +49,11 @@ public final class SystemInfo {
/**
* Returns {@code true} if system command can be run from path.
*
* <p><b>Warning:</b> The command is actually run! So there could be side-effects. You might
* need to specify a version flag or something. Return code is ignored.
* <p><b>Warning:</b> The command is actually run! So there could be side-effects. You might need
* to specify a version flag or something. Return code is ignored.
*
* <p>This result is a memoized. If multiple therads try to get the same result at once, the
* heavy lifting will only be performed by the first thread and the rest will wait.
*
* @throws ExecutionException
* <p>This result is a memoized. If multiple therads try to get the same result at once, the heavy
* lifting will only be performed by the first thread and the rest will wait.
*/
public static boolean hasCommand(String cmd) throws ExecutionException {
return hasCommandCache.get(cmd);

View File

@@ -307,6 +307,24 @@
},
{
"moduleLicense": "The W3C Software License"
},
{
// "Apache License, Version 2.0".
"moduleLicense": null,
"moduleName": "com.squareup.okhttp3:okhttp"
},
{
// "Apache License, Version 2.0".
"moduleLicense": null,
"moduleName": "com.squareup.okio:okio"
},
{
"moduleLicense": "(GPL-2.0-only WITH Classpath-exception-2.0)",
"moduleName": "io.github.eisop:dataflow-errorprone"
},
{
"moduleLicense": "GNU General Public License, version 2 (GPL2), with the classpath exception",
"moduleName": "io.github.eisop:dataflow-errorprone"
}
]
}

View File

@@ -99,8 +99,6 @@ PROPERTIES = [
'If true, show all test output in near-realtime.',
'false',
bool),
Property('flowDocsFile',
'Output filename for the flowDocsTool command.'),
Property('enableDependencyLocking',
'Enables dependency locking.',
'true',

View File

@@ -1,36 +1,39 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.ben-manes.caffeine:caffeine:2.7.0=annotationProcessor,errorprone,testAnnotationProcessor
aopalliance:aopalliance:1.0=annotationProcessor,errorprone,testAnnotationProcessor
com.github.ben-manes.caffeine:caffeine:3.0.5=annotationProcessor,errorprone,testAnnotationProcessor
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.auto:auto-common:0.10=annotationProcessor,errorprone,testAnnotationProcessor
com.google.code.findbugs:jFormatString:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.auto.service:auto-service-annotations:1.0.1=annotationProcessor,errorprone,testAnnotationProcessor
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,errorprone,testAnnotationProcessor
com.google.auto:auto-common:1.2.1=annotationProcessor,errorprone,testAnnotationProcessor
com.google.code.findbugs:jsr305:3.0.2=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_annotation:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_annotation:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.7.1=checkstyle
com.google.errorprone:error_prone_check_api:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_type_annotations:2.3.4=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:error_prone_type_annotations:2.23.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.errorprone:javac:9+181-r4173-1=errorproneJavac
com.google.guava:failureaccess:1.0.1=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor
com.google.guava:guava:27.0.1-jre=annotationProcessor,errorprone,testAnnotationProcessor
com.google.guava:guava-parent:32.1.1-jre=annotationProcessor,errorprone,testAnnotationProcessor
com.google.guava:guava:31.0.1-jre=checkstyle
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,errorprone,testAnnotationProcessor
com.google.j2objc:j2objc-annotations:1.1=annotationProcessor,errorprone,testAnnotationProcessor
com.google.guava:guava:32.1.1-jre=annotationProcessor,errorprone,testAnnotationProcessor
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=checkstyle
com.google.inject:guice:5.1.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.j2objc:j2objc-annotations:1.3=checkstyle
com.google.protobuf:protobuf-java:3.4.0=annotationProcessor,errorprone,testAnnotationProcessor
com.googlecode.java-diff-utils:diffutils:1.3.0=annotationProcessor,errorprone,testAnnotationProcessor
com.google.protobuf:protobuf-java:3.19.6=annotationProcessor,errorprone,testAnnotationProcessor
com.puppycrawl.tools:checkstyle:9.3=checkstyle
commons-beanutils:commons-beanutils:1.9.4=checkstyle
commons-collections:commons-collections:3.2.2=checkstyle
info.picocli:picocli:4.6.2=checkstyle
io.github.eisop:dataflow-errorprone:3.34.0-eisop1=annotationProcessor,errorprone,testAnnotationProcessor
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,errorprone,testAnnotationProcessor
javax.inject:javax.inject:1=annotationProcessor,errorprone,testAnnotationProcessor
net.sf.saxon:Saxon-HE:10.6=checkstyle
org.antlr:antlr4-runtime:4.9.3=checkstyle
org.checkerframework:checker-qual:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
org.checkerframework:checker-qual:3.12.0=checkstyle
org.checkerframework:dataflow:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
org.checkerframework:javacutil:3.0.0=annotationProcessor,errorprone,testAnnotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.17=annotationProcessor,errorprone,testAnnotationProcessor
org.checkerframework:checker-qual:3.33.0=annotationProcessor,errorprone,testAnnotationProcessor
org.jacoco:org.jacoco.agent:0.8.9=jacocoAgent,jacocoAnt
org.jacoco:org.jacoco.ant:0.8.9=jacocoAnt
org.jacoco:org.jacoco.core:0.8.9=jacocoAnt
@@ -39,9 +42,6 @@ org.javassist:javassist:3.28.0-GA=checkstyle
org.ow2.asm:asm-commons:9.5=jacocoAnt
org.ow2.asm:asm-tree:9.5=jacocoAnt
org.ow2.asm:asm:9.5=jacocoAnt
org.pcollections:pcollections:2.1.2=annotationProcessor,errorprone,testAnnotationProcessor
org.plumelib:plume-util:1.0.6=annotationProcessor,errorprone,testAnnotationProcessor
org.plumelib:reflection-util:0.0.2=annotationProcessor,errorprone,testAnnotationProcessor
org.plumelib:require-javadoc:0.1.0=annotationProcessor,errorprone,testAnnotationProcessor
org.pcollections:pcollections:3.1.4=annotationProcessor,errorprone,testAnnotationProcessor
org.reflections:reflections:0.10.2=checkstyle
empty=compileClasspath,deploy_jar,errorproneJavac,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
empty=compileClasspath,deploy_jar,runtimeClasspath,testCompileClasspath,testRuntimeClasspath

View File

@@ -38,7 +38,6 @@ def jsDir = "${project.projectDir}/src/main/javascript"
def outcastTestPatterns = [
// Problem seems to lie with AppEngine TaskQueue for test.
"google/registry/batch/RefreshDnsOnHostRenameActionTest.*",
"google/registry/flows/CheckApiActionTest.*",
"google/registry/flows/EppLifecycleHostTest.*",
"google/registry/flows/domain/DomainCreateFlowTest.*",
"google/registry/flows/domain/DomainUpdateFlowTest.*",
@@ -213,6 +212,7 @@ dependencies {
implementation deps['com.jcraft:jsch']
testImplementation deps['com.thoughtworks.qdox:qdox']
implementation deps['com.zaxxer:HikariCP']
implementation deps['com.squareup.okhttp3:okhttp']
implementation deps['dnsjava:dnsjava']
runtime deps['guru.nidi:graphviz-java-all-j2v8']
testImplementation deps['io.github.classgraph:classgraph']

View File

@@ -2,9 +2,10 @@
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
antlr:antlr:2.7.7=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
aopalliance:aopalliance:1.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
args4j:args4j:2.0.23=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
aopalliance:aopalliance:1.0=annotationProcessor,compileClasspath,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
args4j:args4j:2.0.23=soy
args4j:args4j:2.0.26=css
args4j:args4j:2.33=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
cglib:cglib-nodep:2.2=css
com.101tec:zkclient:0.10=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.beust:jcommander:1.60=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -17,18 +18,18 @@ com.fasterxml.jackson.datatype:jackson-datatype-joda:2.15.2=compileClasspath,dep
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.15.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml.jackson:jackson-bom:2.15.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.fasterxml:classmate:1.5.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.ben-manes.caffeine:caffeine:2.7.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.github.ben-manes.caffeine:caffeine:2.9.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.docker-java:docker-java-api:3.3.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.docker-java:docker-java-transport-zerodep:3.3.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.docker-java:docker-java-transport:3.3.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jffi:1.3.11=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.ben-manes.caffeine:caffeine:3.0.5=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.github.docker-java:docker-java-api:3.3.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.docker-java:docker-java-transport-zerodep:3.3.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.docker-java:docker-java-transport:3.3.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jffi:1.3.12=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-a64asm:1.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-constants:0.10.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-enxio:0.32.15=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-ffi:2.2.14=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-posix:3.1.17=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-unixsocket:0.38.20=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-enxio:0.32.16=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-ffi:2.2.15=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-posix:3.1.18=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-unixsocket:0.38.21=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.jnr:jnr-x86asm:1.0.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.github.kevinstern:software-and-algorithms:1.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.android:annotations:4.1.1.4=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
@@ -38,131 +39,129 @@ com.google.api-client:google-api-client-java6:1.35.2=compileClasspath,deploy_jar
com.google.api-client:google-api-client-servlet:1.35.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api-client:google-api-client:1.35.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:gapic-google-cloud-storage-v2:2.22.6-alpha=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.41.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.165.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.165.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.26.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.106.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.12.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.45.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.45.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.45.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1:2.47.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.171.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta2:0.171.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-bigtable-v2:2.30.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.107.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-pubsublite-v1:1.12.19=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:6.55.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:6.55.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-spanner-v1:6.55.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-cloud-storage-v2:2.23.0-alpha=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-common-protos:2.23.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.41.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.165.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.165.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.26.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.26.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-datastore-v1:0.107.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-firestore-v1:3.14.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:grpc-google-common-protos:2.29.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1:2.47.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.171.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta2:0.171.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:2.30.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-bigtable-v2:2.30.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-datastore-v1:0.108.6=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-firestore-v1:3.15.7=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-monitoring-v3:1.64.0=compileClasspath,nonprodCompileClasspath,testCompileClasspath
com.google.api.grpc:proto-google-cloud-monitoring-v3:3.24.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.106.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.12.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-monitoring-v3:3.32.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.107.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-pubsublite-v1:1.12.19=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-secretmanager-v1:2.23.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-secretmanager-v1beta1:2.23.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.45.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.45.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-v1:6.45.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:6.55.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:6.55.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-spanner-v1:6.55.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-storage-v2:2.23.0-alpha=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2:2.23.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2beta2:0.113.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-cloud-tasks-v2beta3:0.113.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-common-protos:2.24.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-iam-v1:1.18.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:api-common:2.16.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-grpc:2.32.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-httpjson:2.32.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax:2.33.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-common-protos:2.29.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api.grpc:proto-google-iam-v1:1.24.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:api-common:2.21.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-grpc:2.38.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax-httpjson:2.38.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.api:gax:2.38.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-admin-directory:directory_v1-rev118-1.25.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-appengine:v1-rev20230831-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-bigquery:v2-rev20230520-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20230129-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-appengine:v1-rev20231107-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-bigquery:v2-rev20230812-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20230806-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-dataflow:v1b3-rev20220920-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-dns:v2beta1-rev99-1.25.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-drive:v2-rev393-1.25.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-gmail:v1-rev20220404-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-groupssettings:v1-rev20210624-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-healthcare:v1-rev20230510-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-healthcare:v1-rev20231101-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-iamcredentials:v1-rev20211203-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-monitoring:v3-rev20230806-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-monitoring:v3-rev20231215-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-pubsub:v1-rev20220904-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sheets:v4-rev20230815-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sqladmin:v1beta4-rev20230831-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-storage:v1-rev20230617-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-sqladmin:v1beta4-rev20231208-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.apis:google-api-services-storage:v1-rev20231202-2.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-1.0-sdk:1.9.86=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
com.google.appengine:appengine-api-1.0-sdk:2.0.19=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-stubs:2.0.19=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-1.0-sdk:2.0.24=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-api-stubs:2.0.24=testCompileClasspath,testRuntimeClasspath
com.google.appengine:appengine-testing:1.9.86=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-credentials:1.19.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-oauth2-http:1.19.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-credentials:1.20.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auth:google-auth-library-oauth2-http:1.20.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.service:auto-service-annotations:1.0.1=errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.auto.service:auto-service-annotations:1.1.1=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.service:auto-service:1.1.1=annotationProcessor
com.google.auto.value:auto-value-annotations:1.10.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value:1.10.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value:1.10.4=annotationProcessor,testAnnotationProcessor
com.google.auto:auto-common:0.10=errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.auto:auto-common:1.2.1=annotationProcessor
com.google.auto.value:auto-value-annotations:1.10.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.auto.value:auto-value-annotations:1.9=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.auto.value:auto-value:1.10.4=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.auto:auto-common:1.2.1=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.closure-stylesheets:closure-stylesheets:1.5.0=css
com.google.cloud.bigdataoss:gcsio:2.2.16=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.bigdataoss:util:2.2.16=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.bigtable:bigtable-client-core-config:1.28.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.datastore:datastore-v1-proto-client:2.16.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:jdbc-socket-factory-core:1.14.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:postgres-socket-factory:1.14.0=deploy_jar,runtimeClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigquerystorage:2.41.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigtable-stats:2.26.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigtable:2.26.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-grpc:2.22.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.datastore:datastore-v1-proto-client:2.17.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:jdbc-socket-factory-core:1.15.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud.sql:postgres-socket-factory:1.15.1=deploy_jar,runtimeClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigquerystorage:2.47.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigtable-stats:2.30.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-bigtable:2.30.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-grpc:2.28.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core-http:2.21.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core:2.22.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-firestore:3.14.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-core:2.28.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-firestore:3.15.7=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-monitoring:1.82.0=compileClasspath,nonprodCompileClasspath,testCompileClasspath
com.google.cloud:google-cloud-monitoring:3.24.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-monitoring:3.32.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-nio:0.126.19=testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsub:1.124.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsublite:1.12.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsub:1.125.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-pubsublite:1.12.19=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-secretmanager:2.23.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-spanner:6.45.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-spanner:6.55.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-storage:2.22.6=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:google-cloud-tasks:2.23.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:grpc-gcp:1.4.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.14.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.code.findbugs:jFormatString:3.0.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.cloud:grpc-gcp:1.5.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.cloud:proto-google-cloud-firestore-bundle-v1:3.15.7=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.code.findbugs:jsr305:3.0.1=css
com.google.code.findbugs:jsr305:3.0.2=annotationProcessor,checkstyle,compileClasspath,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,soy,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.code.gson:gson:2.10.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.code.gson:gson:2.7=css,soy
com.google.common.html.types:types:1.0.6=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
com.google.dagger:dagger-compiler:2.48=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger-producers:2.48=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger-spi:2.48=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger:2.48=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.devtools.ksp:symbol-processing-api:1.9.0-1.0.12=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotation:2.3.4=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.18.0=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.21.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.errorprone:error_prone_annotations:2.3.4=errorprone,nonprodAnnotationProcessor
com.google.dagger:dagger-compiler:2.50=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger-spi:2.50=annotationProcessor,testAnnotationProcessor
com.google.dagger:dagger:2.50=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.devtools.ksp:symbol-processing-api:1.9.20-1.0.14=annotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotation:2.23.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_annotations:2.23.0=annotationProcessor,compileClasspath,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.errorprone:error_prone_annotations:2.7.1=checkstyle,soy
com.google.errorprone:error_prone_check_api:2.3.4=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.3.4=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_type_annotations:2.3.4=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_check_api:2.23.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_core:2.23.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:error_prone_type_annotations:2.23.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.errorprone:javac-shaded:9-dev-r4023-3=annotationProcessor,testAnnotationProcessor
com.google.errorprone:javac:9+181-r4173-1=errorproneJavac
com.google.escapevelocity:escapevelocity:0.9.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
com.google.flatbuffers:flatbuffers-java:1.12.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.flogger:flogger-system-backend:0.7.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.flogger:flogger:0.7.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.flogger:google-extensions:0.7.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.flogger:flogger-system-backend:0.8=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.flogger:flogger:0.8=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.flogger:google-extensions:0.8=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.googlejavaformat:google-java-format:1.5=annotationProcessor,testAnnotationProcessor
com.google.guava:failureaccess:1.0.1=annotationProcessor,checkstyle,compileClasspath,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.guava:guava-parent:32.1.2-jre=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.guava:guava-testlib:32.1.2-jre=testCompileClasspath,testRuntimeClasspath
com.google.guava:failureaccess:1.0.1=checkstyle,errorprone,nonprodAnnotationProcessor,soy
com.google.guava:failureaccess:1.0.2=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.guava:guava-parent:32.1.1-jre=errorprone,nonprodAnnotationProcessor
com.google.guava:guava-testlib:33.0.0-jre=testCompileClasspath,testRuntimeClasspath
com.google.guava:guava:20.0=css
com.google.guava:guava:27.0.1-jre=errorprone,nonprodAnnotationProcessor
com.google.guava:guava:31.0.1-jre=checkstyle,soy
com.google.guava:guava:32.1.2-jre=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,compileClasspath,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.guava:guava:32.1.1-jre=errorprone,nonprodAnnotationProcessor
com.google.guava:guava:33.0.0-jre=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava=annotationProcessor,checkstyle,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
com.google.gwt:gwt-user:2.10.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-apache-v2:1.43.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.http-client:google-http-client-appengine:1.43.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -172,8 +171,8 @@ com.google.http-client:google-http-client-protobuf:1.43.3=compileClasspath,deplo
com.google.http-client:google-http-client:1.43.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.inject.extensions:guice-multibindings:4.1.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
com.google.inject:guice:4.1.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.inject:guice:5.1.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.inject:guice:7.0.0=soy
com.google.j2objc:j2objc-annotations:1.1=errorprone,nonprodAnnotationProcessor
com.google.j2objc:j2objc-annotations:1.3=checkstyle,soy
com.google.j2objc:j2objc-annotations:2.8=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.javascript:closure-compiler-externs:v20160713=css
@@ -189,24 +188,24 @@ com.google.oauth-client:google-oauth-client-java6:1.34.1=compileClasspath,deploy
com.google.oauth-client:google-oauth-client-jetty:1.34.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.oauth-client:google-oauth-client-servlet:1.34.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.oauth-client:google-oauth-client:1.34.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java-util:3.23.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java-util:3.25.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:2.5.0=css
com.google.protobuf:protobuf-java:3.23.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:3.4.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.protobuf:protobuf-java:3.19.6=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.protobuf:protobuf-java:3.25.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.protobuf:protobuf-java:4.0.0-rc-2=soy
com.google.re2j:re2j:1.7=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.google.template:soy:2021-02-01=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
com.google.truth.extensions:truth-java8-extension:1.1.5=testCompileClasspath,testRuntimeClasspath
com.google.truth:truth:1.1.5=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.googlecode.java-diff-utils:diffutils:1.3.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
com.google.truth.extensions:truth-java8-extension:1.2.0=testCompileClasspath,testRuntimeClasspath
com.google.truth:truth:1.2.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.googlecode.json-simple:json-simple:1.1.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.ibm.icu:icu4j:57.1=compileClasspath,nonprodCompileClasspath,soy,testCompileClasspath
com.ibm.icu:icu4j:73.2=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.ibm.icu:icu4j:74.2=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
com.jcraft:jsch:0.1.55=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.lmax:disruptor:3.4.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.puppycrawl.tools:checkstyle:9.3=checkstyle
com.squareup.okhttp3:okhttp:3.11.0=testCompileClasspath,testRuntimeClasspath
com.squareup.okio:okio:1.14.0=testCompileClasspath,testRuntimeClasspath
com.squareup.okhttp3:okhttp:4.12.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.squareup.okio:okio-jvm:3.6.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.squareup.okio:okio:3.6.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.squareup:javapoet:1.13.0=annotationProcessor,testAnnotationProcessor
com.squareup:kotlinpoet:1.11.0=annotationProcessor,testAnnotationProcessor
com.sun.activation:jakarta.activation:1.2.2=jaxb
@@ -214,17 +213,17 @@ com.sun.activation:javax.activation:1.2.0=jaxb
com.sun.istack:istack-commons-runtime:3.0.7=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.sun.istack:istack-commons-runtime:4.1.2=nonprodRuntime,runtime
com.sun.xml.bind:jaxb-impl:2.3.3=jaxb
com.sun.xml.bind:jaxb-osgi:4.0.3=jaxb
com.sun.xml.bind:jaxb-osgi:4.0.4=jaxb
com.sun.xml.bind:jaxb-xjc:2.3.3=jaxb
com.sun.xml.fastinfoset:FastInfoset:1.2.15=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.thoughtworks.paranamer:paranamer:2.7=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
com.thoughtworks.qdox:qdox:1.12.1=testCompileClasspath,testRuntimeClasspath
com.zaxxer:HikariCP:3.4.5=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
commons-beanutils:commons-beanutils:1.9.4=checkstyle
commons-codec:commons-codec:1.15=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
commons-codec:commons-codec:1.16.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
commons-collections:commons-collections:3.2.2=checkstyle
commons-logging:commons-logging:1.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
dnsjava:dnsjava:3.5.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
dnsjava:dnsjava:3.5.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
guru.nidi.com.eclipsesource.j2v8:j2v8_linux_x86_64:4.6.0=nonprodRuntime,runtime,testRuntimeClasspath
guru.nidi.com.eclipsesource.j2v8:j2v8_macosx_x86_64:4.6.0=nonprodRuntime,runtime,testRuntimeClasspath
guru.nidi.com.eclipsesource.j2v8:j2v8_win32_x86:4.6.0=nonprodRuntime,runtime,testRuntimeClasspath
@@ -237,37 +236,40 @@ io.confluent:common-config:5.3.2=compileClasspath,deploy_jar,nonprodCompileClass
io.confluent:common-utils:5.3.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.confluent:kafka-avro-serializer:5.3.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.confluent:kafka-schema-registry-client:5.3.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.github.classgraph:classgraph:4.8.104=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.github.java-diff-utils:java-diff-utils:4.12=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-alts:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-api:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-auth:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-census:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-context:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-core:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-googleapis:1.56.1=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-grpclb:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-netty-shaded:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-netty:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf-lite:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-rls:1.56.1=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-services:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-stub:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-xds:1.56.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-buffer:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http2:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-socks:4.1.87.Final=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-codec:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-common:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-handler-proxy:4.1.87.Final=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-handler:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-resolver:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.github.classgraph:classgraph:4.8.162=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.github.eisop:dataflow-errorprone:3.34.0-eisop1=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
io.github.java-diff-utils:java-diff-utils:4.12=annotationProcessor,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-alts:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-api:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-auth:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-census:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-context:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-core:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-googleapis:1.59.1=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-grpclb:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-inprocess:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-netty-shaded:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-netty:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf-lite:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-protobuf:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-rls:1.59.1=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.grpc:grpc-services:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-stub:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-util:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.grpc:grpc-xds:1.59.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-buffer:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http2:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-http:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-codec-socks:4.1.97.Final=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-codec:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-common:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-handler-proxy:4.1.97.Final=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
io.netty:netty-handler:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-resolver:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-tcnative-boringssl-static:2.0.52.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-tcnative-classes:2.0.52.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-transport-native-unix-common:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-transport:4.1.87.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-transport-native-unix-common:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.netty:netty-transport:4.1.97.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-api:0.31.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-exemplar-util:0.31.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-contrib-grpc-metrics:0.31.0=compileClasspath,nonprodCompileClasspath,testCompileClasspath
@@ -281,16 +283,15 @@ io.opencensus:opencensus-impl-core:0.31.0=compileClasspath,deploy_jar,nonprodCom
io.opencensus:opencensus-impl:0.31.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.opencensus:opencensus-proto:0.2.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
io.perfmark:perfmark-api:0.26.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
jakarta.activation:jakarta.activation-api:2.1.0=jaxb
jakarta.activation:jakarta.activation-api:2.1.2=nonprodRuntime,runtime
jakarta.activation:jakarta.activation-api:2.1.2=jaxb,nonprodRuntime,runtime
jakarta.inject:jakarta.inject-api:2.0.1=soy
jakarta.xml.bind:jakarta.xml.bind-api:4.0.0=jaxb,nonprodRuntime,runtime
jakarta.xml.bind:jakarta.xml.bind-api:4.0.1=jaxb,nonprodRuntime,runtime
javacc:javacc:4.1=css
javax.activation:activation:1.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
javax.activation:javax.activation-api:1.2.0=compileClasspath,deploy_jar,jaxb,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
javax.annotation:javax.annotation-api:1.3.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
javax.annotation:jsr250-api:1.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,soy,testCompileClasspath,testRuntimeClasspath
javax.inject:javax.inject:1=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
javax.inject:javax.inject:1=annotationProcessor,compileClasspath,deploy_jar,errorprone,nonprodAnnotationProcessor,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
javax.jdo:jdo2-api:2.3-20090302111651=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
javax.mail:mail:1.5.0-b01=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
javax.persistence:javax.persistence-api:2.2=annotationProcessor,compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testCompileClasspath,testRuntimeClasspath
@@ -302,11 +303,11 @@ jline:jline:1.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRunti
joda-time:joda-time:2.10.14=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
junit:junit:4.13.2=nonprodCompileClasspath,nonprodRuntimeClasspath,testCompileClasspath,testRuntimeClasspath
net.arnx:nashorn-promise:0.1.1=nonprodRuntime,runtime,testRuntimeClasspath
net.bytebuddy:byte-buddy-agent:1.14.6=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy-agent:1.14.10=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.12.18=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
net.bytebuddy:byte-buddy:1.14.6=testCompileClasspath,testRuntimeClasspath
net.bytebuddy:byte-buddy:1.14.10=testCompileClasspath,testRuntimeClasspath
net.java.dev.javacc:javacc:4.1=css
net.java.dev.jna:jna:5.12.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
net.java.dev.jna:jna:5.13.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
net.ltgt.gradle.incap:incap:0.2=annotationProcessor,testAnnotationProcessor
net.sf.saxon:Saxon-HE:10.6=checkstyle
org.antlr:antlr4-runtime:4.9.3=checkstyle
@@ -314,32 +315,33 @@ org.apache.arrow:arrow-format:5.0.0=compileClasspath,deploy_jar,nonprodCompileCl
org.apache.arrow:arrow-memory-core:5.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.arrow:arrow-vector:5.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.avro:avro:1.8.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-fn-execution:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-job-management:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-pipeline:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-construction-java:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-java:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-direct-java:2.50.0=testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-java-fn-execution:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-core:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-expansion-service:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-arrow:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-avro:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-protobuf:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-fn-execution:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-harness:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-kafka:2.50.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-fn-execution:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-job-management:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-model-pipeline:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-construction-java:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-core-java:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-direct-java:2.53.0=testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-runners-java-fn-execution:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-core:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-expansion-service:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-arrow:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-avro:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-extensions-protobuf:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-fn-execution:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-harness:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-io-kafka:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-sdks-java-transform-service-launcher:2.53.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-vendor-grpc-1_54_0:0.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.beam:beam-vendor-guava-32_1_2-jre:0.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-compress:1.23.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-compress:1.24.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-csv:1.10.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-exec:1.3=nonprodRuntime,runtime,testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.12.0=testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.13.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
org.apache.commons:commons-text:1.10.0=testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.13.0=testCompileClasspath,testRuntimeClasspath
org.apache.commons:commons-lang3:3.14.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath
org.apache.commons:commons-text:1.11.0=testCompileClasspath,testRuntimeClasspath
org.apache.ftpserver:ftplet-api:1.2.0=testCompileClasspath,testRuntimeClasspath
org.apache.ftpserver:ftpserver-core:1.2.0=testCompileClasspath,testRuntimeClasspath
org.apache.httpcomponents:httpclient:4.5.14=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -348,23 +350,20 @@ org.apache.mina:mina-core:2.1.6=testCompileClasspath,testRuntimeClasspath
org.apache.sshd:sshd-core:2.0.0=testCompileClasspath,testRuntimeClasspath
org.apache.sshd:sshd-scp:2.0.0=testCompileClasspath,testRuntimeClasspath
org.apache.sshd:sshd-sftp:2.0.0=testCompileClasspath,testRuntimeClasspath
org.apache.tomcat:tomcat-annotations-api:11.0.0-M11=testCompileClasspath,testRuntimeClasspath
org.apache.tomcat:tomcat-annotations-api:11.0.0-M16=testCompileClasspath,testRuntimeClasspath
org.apiguardian:apiguardian-api:1.1.2=testCompileClasspath
org.bouncycastle:bcpg-jdk15on:1.67=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.bouncycastle:bcpkix-jdk15on:1.67=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.bouncycastle:bcprov-jdk15on:1.67=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.checkerframework:checker-compat-qual:2.5.3=compileClasspath,nonprodCompileClasspath,nonprodRuntime,runtime,testCompileClasspath
org.checkerframework:checker-compat-qual:2.5.5=annotationProcessor,deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testAnnotationProcessor,testRuntimeClasspath
org.checkerframework:checker-qual:3.0.0=errorprone,nonprodAnnotationProcessor
org.checkerframework:checker-compat-qual:2.5.5=annotationProcessor,testAnnotationProcessor
org.checkerframework:checker-compat-qual:2.5.6=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
org.checkerframework:checker-qual:3.12.0=checkstyle,soy
org.checkerframework:checker-qual:3.31.0=nonprodRuntime,runtime
org.checkerframework:checker-qual:3.33.0=annotationProcessor,compileClasspath,nonprodCompileClasspath,testAnnotationProcessor
org.checkerframework:checker-qual:3.35.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.checkerframework:dataflow:3.0.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.checkerframework:javacutil:3.0.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.checkerframework:checker-qual:3.33.0=errorprone,nonprodAnnotationProcessor
org.checkerframework:checker-qual:3.41.0=annotationProcessor,compileClasspath,nonprodCompileClasspath,nonprodRuntime,runtime,testAnnotationProcessor
org.checkerframework:checker-qual:3.42.0=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.codehaus.jackson:jackson-core-asl:1.9.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.codehaus.jackson:jackson-mapper-asl:1.9.13=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.codehaus.mojo:animal-sniffer-annotations:1.17=errorprone,nonprodAnnotationProcessor
org.codehaus.mojo:animal-sniffer-annotations:1.23=deploy_jar,nonprodRuntimeClasspath,runtimeClasspath,testRuntimeClasspath
org.conscrypt:conscrypt-openjdk-uber:2.5.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.easymock:easymock:3.0=css
@@ -376,12 +375,12 @@ org.eclipse.jetty:jetty-server:9.4.49.v20220914=compileClasspath,deploy_jar,nonp
org.eclipse.jetty:jetty-servlet:9.4.49.v20220914=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-util-ajax:9.4.49.v20220914=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.eclipse.jetty:jetty-util:9.4.49.v20220914=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.flywaydb:flyway-core:9.22.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:jaxb-core:4.0.3=nonprodRuntime,runtime
org.flywaydb:flyway-core:9.22.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:jaxb-core:4.0.4=nonprodRuntime,runtime
org.glassfish.jaxb:jaxb-runtime:2.3.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:jaxb-runtime:4.0.3=nonprodRuntime,runtime
org.glassfish.jaxb:jaxb-runtime:4.0.4=nonprodRuntime,runtime
org.glassfish.jaxb:txw2:2.3.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.glassfish.jaxb:txw2:4.0.3=nonprodRuntime,runtime
org.glassfish.jaxb:txw2:4.0.4=nonprodRuntime,runtime
org.gwtproject:gwt-user:2.10.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.hamcrest:hamcrest-core:1.1=css
org.hamcrest:hamcrest-core:1.3=nonprodCompileClasspath,nonprodRuntimeClasspath
@@ -401,53 +400,55 @@ org.jboss.logging:jboss-logging:3.4.3.Final=compileClasspath,deploy_jar,nonprodC
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jboss:jandex:2.4.2.Final=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-reflect:1.6.10=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-common:1.9.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.6.10=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.10=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib:1.9.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-common:1.9.10=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.9.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.9.10=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.0=annotationProcessor,testAnnotationProcessor
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib:1.9.10=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jetbrains.kotlin:kotlin-stdlib:1.9.20=annotationProcessor,testAnnotationProcessor
org.jetbrains:annotations:13.0=annotationProcessor,testAnnotationProcessor
org.jetbrains:annotations:17.0.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.joda:joda-money:1.0.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.joda:joda-money:1.0.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.json:json:20160212=soy
org.json:json:20230618=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jsoup:jsoup:1.16.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit-pioneer:junit-pioneer:2.1.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-migrationsupport:5.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-params:5.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-launcher:1.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-runner:1.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-api:1.10.0=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-commons:1.10.0=testRuntimeClasspath
org.junit:junit-bom:5.10.0=testCompileClasspath,testRuntimeClasspath
org.json:json:20231013=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.jsoup:jsoup:1.17.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.junit-pioneer:junit-pioneer:1.9.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-api:5.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-engine:5.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-migrationsupport:5.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.jupiter:junit-jupiter-params:5.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-commons:1.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-engine:1.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-launcher:1.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-runner:1.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-api:1.10.1=testCompileClasspath,testRuntimeClasspath
org.junit.platform:junit-platform-suite-commons:1.10.1=testRuntimeClasspath
org.junit:junit-bom:5.10.1=testCompileClasspath,testRuntimeClasspath
org.jvnet.staxex:stax-ex:1.8=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-core:1.10.19=css
org.mockito:mockito-core:5.5.0=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-junit-jupiter:5.5.0=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-core:5.8.0=testCompileClasspath,testRuntimeClasspath
org.mockito:mockito-junit-jupiter:5.8.0=testCompileClasspath,testRuntimeClasspath
org.mortbay.jetty:jetty-util:6.1.26=testCompileClasspath,testRuntimeClasspath
org.mortbay.jetty:jetty:6.1.26=testCompileClasspath,testRuntimeClasspath
org.objenesis:objenesis:2.1=css
org.objenesis:objenesis:3.3=testRuntimeClasspath
org.opentest4j:opentest4j:1.3.0=testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:7.0=soy
org.ow2.asm:asm-analysis:9.5=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-analysis:9.6=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-commons:7.0=soy
org.ow2.asm:asm-commons:9.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-commons:9.5=jacocoAnt
org.ow2.asm:asm-tree:7.0=soy
org.ow2.asm:asm-tree:9.5=compileClasspath,deploy_jar,jacocoAnt,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-tree:9.5=jacocoAnt
org.ow2.asm:asm-tree:9.6=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-util:7.0=soy
org.ow2.asm:asm-util:9.5=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm-util:9.6=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm:7.0=soy
org.ow2.asm:asm:9.5=compileClasspath,deploy_jar,jacocoAnt,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.pcollections:pcollections:2.1.2=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.plumelib:plume-util:1.0.6=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.plumelib:reflection-util:0.0.2=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.plumelib:require-javadoc:0.1.0=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.postgresql:postgresql:42.6.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.ow2.asm:asm:9.5=jacocoAnt
org.ow2.asm:asm:9.6=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.pcollections:pcollections:3.1.4=annotationProcessor,errorprone,nonprodAnnotationProcessor,testAnnotationProcessor
org.postgresql:postgresql:42.7.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntime,nonprodRuntimeClasspath,runtime,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.reflections:reflections:0.10.2=checkstyle
org.rnorth.duct-tape:duct-tape:1.0.8=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.seleniumhq.selenium:selenium-api:3.141.59=testCompileClasspath,testRuntimeClasspath
@@ -463,23 +464,23 @@ org.seleniumhq.selenium:selenium-support:3.141.59=testCompileClasspath,testRunti
org.slf4j:jcl-over-slf4j:1.7.30=nonprodRuntime,runtime,testRuntimeClasspath
org.slf4j:jul-to-slf4j:1.7.30=nonprodRuntime,runtime,testRuntimeClasspath
org.slf4j:slf4j-api:1.7.30=nonprodRuntime,runtime
org.slf4j:slf4j-api:1.7.36=compileClasspath,nonprodCompileClasspath,nonprodRuntimeClasspath,testCompileClasspath
org.slf4j:slf4j-api:2.0.9=deploy_jar,runtimeClasspath,testRuntimeClasspath
org.slf4j:slf4j-jdk14:2.0.9=deploy_jar,runtimeClasspath,testRuntimeClasspath
org.slf4j:slf4j-api:2.0.9=compileClasspath,nonprodCompileClasspath,nonprodRuntimeClasspath,testCompileClasspath
org.slf4j:slf4j-api:2.1.0-alpha1=deploy_jar,runtimeClasspath,testRuntimeClasspath
org.slf4j:slf4j-jdk14:2.1.0-alpha1=deploy_jar,runtimeClasspath,testRuntimeClasspath
org.springframework:spring-core:5.3.27=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.springframework:spring-expression:5.3.27=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.springframework:spring-jcl:5.3.27=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:database-commons:1.19.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:jdbc:1.19.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:junit-jupiter:1.19.0=testCompileClasspath,testRuntimeClasspath
org.testcontainers:postgresql:1.19.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:selenium:1.19.0=testCompileClasspath,testRuntimeClasspath
org.testcontainers:testcontainers:1.19.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:database-commons:1.19.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:jdbc:1.19.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:junit-jupiter:1.19.3=testCompileClasspath,testRuntimeClasspath
org.testcontainers:postgresql:1.19.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.testcontainers:selenium:1.19.3=testCompileClasspath,testRuntimeClasspath
org.testcontainers:testcontainers:1.19.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.threeten:threetenbp:1.6.8=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.tukaani:xz:1.5=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.w3c.css:sac:1.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.webjars.npm:viz.js-graphviz-java:2.1.3=nonprodRuntime,runtime,testRuntimeClasspath
org.xerial.snappy:snappy-java:1.1.10.3=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.xerial.snappy:snappy-java:1.1.10.4=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
org.yaml:snakeyaml:2.0=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
us.fatehi:schemacrawler-api:16.10.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
us.fatehi:schemacrawler-diagram:16.10.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
@@ -488,4 +489,4 @@ us.fatehi:schemacrawler-utility:16.10.1=compileClasspath,deploy_jar,nonprodCompi
us.fatehi:schemacrawler:16.10.1=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
xerces:xmlParserAPIs:2.6.2=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
xpp3:xpp3:1.1.4c=compileClasspath,deploy_jar,nonprodCompileClasspath,nonprodRuntimeClasspath,runtimeClasspath,testCompileClasspath,testRuntimeClasspath
empty=devtool,errorproneJavac,nomulus_test
empty=devtool,nomulus_test

View File

@@ -17,15 +17,14 @@ package google.registry.batch;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.util.DateTimeUtils.END_OF_TIME;
import static google.registry.util.RegistryEnvironment.PRODUCTION;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryEnvironment;
import google.registry.flows.poll.PollFlowUtils;
import google.registry.model.EppResource;
import google.registry.model.EppResourceUtils;
@@ -40,6 +39,7 @@ import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import google.registry.util.RegistryEnvironment;
import javax.inject.Inject;
/**

View File

@@ -18,13 +18,13 @@ import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.batch.BatchModule.PARAM_DRY_RUN;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.dns.DnsUtils.requestDomainDnsRefresh;
import static google.registry.model.reporting.HistoryEntry.Type.DOMAIN_DELETE;
import static google.registry.model.tld.Tlds.getTldsOfType;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.POST;
import static google.registry.request.RequestParameters.PARAM_TLDS;
import static google.registry.util.RegistryEnvironment.PRODUCTION;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
@@ -32,7 +32,6 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import com.google.common.flogger.FluentLogger;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.EppResourceUtils;
import google.registry.model.domain.Domain;
@@ -41,6 +40,7 @@ import google.registry.model.tld.Tld.TldType;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.util.RegistryEnvironment;
import java.util.concurrent.atomic.AtomicInteger;
import javax.inject.Inject;
import org.hibernate.CacheMode;

View File

@@ -31,7 +31,6 @@ import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import google.registry.beam.billing.ExpandBillingRecurrencesPipeline;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.model.billing.BillingEvent;
import google.registry.model.billing.BillingRecurrence;
import google.registry.model.common.Cursor;
@@ -40,6 +39,7 @@ import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import google.registry.util.RegistryEnvironment;
import java.io.IOException;
import java.util.Optional;
import javax.inject.Inject;

View File

@@ -27,12 +27,12 @@ import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.request.Action;
import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import google.registry.util.RegistryEnvironment;
import javax.inject.Inject;
/**

View File

@@ -39,7 +39,6 @@ import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.EmailMessage;
import java.util.Date;
import java.util.Optional;
import javax.inject.Inject;
import javax.mail.internet.AddressException;
@@ -160,12 +159,11 @@ public class SendExpiringCertificateNotificationEmailAction implements Runnable
try {
ImmutableSet<InternetAddress> recipients = getEmailAddresses(registrar, Type.TECH);
ImmutableSet<InternetAddress> ccs = getEmailAddresses(registrar, Type.ADMIN);
Date expirationDate = certificateChecker.getCertificate(certificate.get()).getNotAfter();
DateTime expirationDate =
new DateTime(certificateChecker.getCertificate(certificate.get()).getNotAfter());
logger.atInfo().log(
" %s SSL certificate of registrar '%s' will expire on %s.",
certificateType.getDisplayName(),
registrar.getRegistrarName(),
expirationDate.toString());
certificateType.getDisplayName(), registrar.getRegistrarName(), expirationDate);
if (recipients.isEmpty() && ccs.isEmpty()) {
logger.atWarning().log(
"Registrar %s contains no TECH nor ADMIN email addresses to receive notification"
@@ -302,7 +300,7 @@ public class SendExpiringCertificateNotificationEmailAction implements Runnable
@VisibleForTesting
@SuppressWarnings("lgtm[java/dereferenced-value-may-be-null]")
String getEmailBody(
String registrarName, CertificateType type, Date expirationDate, String registrarId) {
String registrarName, CertificateType type, DateTime expirationDate, String registrarId) {
checkArgumentNotNull(expirationDate, "Expiration date cannot be null");
checkArgumentNotNull(type, "Certificate type cannot be null");
checkArgumentNotNull(registrarId, "Registrar Id cannot be null");
@@ -310,7 +308,7 @@ public class SendExpiringCertificateNotificationEmailAction implements Runnable
expirationWarningEmailBodyText,
registrarName,
type.getDisplayName(),
DATE_FORMATTER.print(new DateTime(expirationDate)),
DATE_FORMATTER.print(expirationDate),
registrarId);
}

View File

@@ -28,7 +28,6 @@ import com.google.common.flogger.FluentLogger;
import com.google.common.net.MediaType;
import google.registry.beam.wipeout.WipeOutContactHistoryPiiPipeline;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.model.contact.ContactHistory;
import google.registry.request.Action;
import google.registry.request.Action.Service;
@@ -36,6 +35,7 @@ import google.registry.request.Parameter;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import google.registry.util.RegistryEnvironment;
import java.io.IOException;
import java.util.Optional;
import javax.inject.Inject;

View File

@@ -14,9 +14,9 @@
package google.registry.beam.common;
import google.registry.config.RegistryEnvironment;
import google.registry.persistence.PersistenceModule.JpaTransactionManagerType;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.util.RegistryEnvironment;
import java.util.Objects;
import javax.annotation.Nullable;
import org.apache.beam.sdk.extensions.gcp.options.GcpOptions;
@@ -63,7 +63,7 @@ public interface RegistryPipelineOptions extends GcpOptions {
}
/**
* Validates the GCP project and Registry environment settings in {@code option}. If project is
* Validates the GCP project and Registry environment settings in {@code options}. If project is
* undefined, it is set according to the Registry environment; if project is defined but
* inconsistent with the Registry environment, an {@link IllegalArgumentException} will be thrown.
*

View File

@@ -19,10 +19,10 @@ import static google.registry.beam.common.RegistryPipelineOptions.toRegistryPipe
import com.google.auto.service.AutoService;
import com.google.common.flogger.FluentLogger;
import dagger.Lazy;
import google.registry.config.RegistryEnvironment;
import google.registry.config.SystemPropertySetter;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.persistence.transaction.TransactionManagerFactory;
import google.registry.util.RegistryEnvironment;
import google.registry.util.SystemPropertySetter;
import org.apache.beam.sdk.harness.JvmInitializer;
import org.apache.beam.sdk.options.PipelineOptions;

View File

@@ -144,6 +144,7 @@ public class RdeIO {
this.marshaller = new RdeMarshaller(validationMode);
}
@SuppressWarnings("unused")
@Setup
public void setup() {
Security.addProvider(new BouncyCastleProvider());

View File

@@ -116,14 +116,14 @@ import org.joda.time.DateTime;
* <p>The pipeline is broadly divided into two parts -- creating the {@link DepositFragment}s, and
* processing them.
*
* <h1>Creating {@link DepositFragment}</h1>
* <h2>Creating {@link DepositFragment}</h2>
*
* <h2>{@link Registrar}</h2>
* <h3>{@link Registrar}</h3>
*
* Non-test registrar entities are loaded from Cloud SQL and marshalled into deposit fragments. They
* are <b>NOT</b> rewound to the watermark.
*
* <h2>{@link EppResource}</h2>
* <h3>{@link EppResource}</h3>
*
* All EPP resources are loaded from the corresponding {@link HistoryEntry}, which has the resource
* embedded. In general, we find most recent history entry before watermark and filter out the ones
@@ -158,7 +158,7 @@ import org.joda.time.DateTime;
* watermark to the domain at watermark. We then proceed to create the (pending deposit: deposit
* fragment) pair for subordinate hosts using the added domain information.
*
* <h1>Processing {@link DepositFragment}</h1>
* <h2>Processing {@link DepositFragment}</h2>
*
* The (pending deposit: deposit fragment) pairs from different resources are combined and grouped
* by pending deposit. For each pending deposit, all the relevant deposit fragments are written into

View File

@@ -225,7 +225,7 @@ public class SafeBrowsingTransforms {
new JSONObject(
CharStreams.toString(
new InputStreamReader(response.getEntity().getContent(), UTF_8)));
logger.atInfo().log("Got response: %s", responseBody.toString());
logger.atInfo().log("Got response: %s", responseBody);
if (responseBody.length() == 0) {
logger.atInfo().log("Response was empty, no threats detected.");
} else {

View File

@@ -144,7 +144,8 @@ public class BigqueryConnection implements AutoCloseable {
public Builder setPollInterval(Duration pollInterval) {
checkArgument(
!pollInterval.isShorterThan(MIN_POLL_INTERVAL),
"poll interval must be at least %ldms", MIN_POLL_INTERVAL.getMillis());
"poll interval must be at least %s ms",
MIN_POLL_INTERVAL.getMillis());
instance.pollInterval = pollInterval;
return this;
}
@@ -216,7 +217,7 @@ public class BigqueryConnection implements AutoCloseable {
}
public Builder timeToLive(Duration duration) {
this.table.setExpirationTime(new DateTime(UTC).plus(duration).getMillis());
this.table.setExpirationTime(DateTime.now(UTC).plus(duration).getMillis());
return this;
}
@@ -556,7 +557,6 @@ public class BigqueryConnection implements AutoCloseable {
/**
* Launch a job, but do not wait for it to complete.
*
* @throws BigqueryJobFailureException
*/
private Job launchJob(Job job, @Nullable AbstractInputStreamContent data) {
verify(job.getStatus() == null);
@@ -572,7 +572,6 @@ public class BigqueryConnection implements AutoCloseable {
/**
* Synchronously waits for a job to complete that's already been launched.
*
* @throws BigqueryJobFailureException
*/
private Job waitForJob(Job job) {
verify(job.getStatus() != null);
@@ -591,7 +590,6 @@ public class BigqueryConnection implements AutoCloseable {
/**
* Checks completed job for errors.
*
* @throws BigqueryJobFailureException
*/
private static Job checkJob(Job job) {
verify(job.getStatus() != null);

View File

@@ -72,7 +72,7 @@ public final class BigqueryJobFailureException extends RuntimeException {
/**
* Returns a short error code describing why this job failed.
*
* <h3>Sample Reasons</h3>
* <p>Sample Reasons:
*
* <ul>
* <li>{@code "duplicate"}: The table you're trying to create already exists.
@@ -80,8 +80,8 @@ public final class BigqueryJobFailureException extends RuntimeException {
* <li>{@code "unknown"}: Non-Bigquery errors.
* </ul>
*
* @see <a href="https://cloud.google.com/bigquery/troubleshooting-errors">
* Troubleshooting Errors</a>
* @see <a href="https://cloud.google.com/bigquery/troubleshooting-errors">Troubleshooting
* Errors</a>
*/
public String getReason() {
if (jobStatus != null) {

View File

@@ -0,0 +1,164 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static java.nio.charset.StandardCharsets.UTF_8;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.api.client.http.HttpMethods;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import com.google.common.io.ByteStreams;
import google.registry.bsa.api.BsaCredential;
import google.registry.bsa.api.BsaException;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.UrlConnectionService;
import google.registry.util.Retrier;
import java.io.BufferedInputStream;
import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.security.GeneralSecurityException;
import java.util.function.BiConsumer;
import javax.inject.Inject;
import javax.net.ssl.HttpsURLConnection;
/** Fetches data from the BSA API. */
public class BlockListFetcher {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private final UrlConnectionService urlConnectionService;
private final BsaCredential credential;
private final ImmutableMap<String, String> blockListUrls;
private final Retrier retrier;
@Inject
BlockListFetcher(
UrlConnectionService urlConnectionService,
BsaCredential credential,
@Config("bsaDataUrls") ImmutableMap<String, String> blockListUrls,
Retrier retrier) {
this.urlConnectionService = urlConnectionService;
this.credential = credential;
this.blockListUrls = blockListUrls;
this.retrier = retrier;
}
LazyBlockList fetch(BlockListType blockListType) {
// TODO: use more informative exceptions to describe retriable errors
return retrier.callWithRetry(
() -> tryFetch(blockListType),
e -> e instanceof BsaException && ((BsaException) e).isRetriable());
}
LazyBlockList tryFetch(BlockListType blockListType) {
try {
URL dataUrl = new URL(blockListUrls.get(blockListType.name()));
logger.atInfo().log("Downloading from %s", dataUrl);
HttpsURLConnection connection =
(HttpsURLConnection) urlConnectionService.createConnection(dataUrl);
connection.setRequestMethod(HttpMethods.GET);
connection.setRequestProperty("Authorization", "Bearer " + credential.getAuthToken());
int code = connection.getResponseCode();
if (code != SC_OK) {
String errorDetails = "";
try (InputStream errorStream = connection.getErrorStream()) {
errorDetails = new String(ByteStreams.toByteArray(errorStream), UTF_8);
} catch (NullPointerException e) {
// No error message.
} catch (Exception e) {
errorDetails = "Failed to retrieve error message: " + e.getMessage();
}
throw new BsaException(
String.format(
"Status code: [%s], error: [%s], details: [%s]",
code, connection.getResponseMessage(), errorDetails),
/* retriable= */ true);
}
return new LazyBlockList(blockListType, connection);
} catch (IOException e) {
throw new BsaException(e, /* retriable= */ true);
} catch (GeneralSecurityException e) {
throw new BsaException(e, /* retriable= */ false);
}
}
static class LazyBlockList implements Closeable {
private final BlockListType blockListType;
private final HttpsURLConnection connection;
private final BufferedInputStream inputStream;
private final String checksum;
LazyBlockList(BlockListType blockListType, HttpsURLConnection connection) throws IOException {
this.blockListType = blockListType;
this.connection = connection;
this.inputStream = new BufferedInputStream(connection.getInputStream());
this.checksum = readChecksum();
}
/** Reads the BSA-generated checksum, which is the first line of the input. */
private String readChecksum() throws IOException {
StringBuilder checksum = new StringBuilder();
char ch;
while ((ch = peekInputStream()) != Character.MAX_VALUE && !Character.isWhitespace(ch)) {
checksum.append((char) inputStream.read());
}
while ((ch = peekInputStream()) != Character.MAX_VALUE && Character.isWhitespace(ch)) {
inputStream.read();
}
return checksum.toString();
}
char peekInputStream() throws IOException {
inputStream.mark(1);
int byteValue = inputStream.read();
inputStream.reset();
return (char) byteValue;
}
BlockListType getName() {
return blockListType;
}
String checksum() {
return checksum;
}
void consumeAll(BiConsumer<byte[], Integer> consumer) throws IOException {
byte[] buffer = new byte[1024];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
consumer.accept(buffer, bytesRead);
}
}
@Override
public void close() {
if (inputStream != null) {
try {
inputStream.close();
} catch (IOException e) {
// Fall through to close the connection.
}
}
connection.disconnect();
}
}
}

View File

@@ -14,8 +14,10 @@
package google.registry.bsa;
/** Identifiers of the BSA lists with blocking labels. */
public enum BlockList {
/**
* The product types of the block lists, which determines the http endpoint that serves the data.
*/
public enum BlockListType {
BLOCK,
BLOCK_PLUS;
}

View File

@@ -0,0 +1,269 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Maps.newHashMap;
import static com.google.common.collect.Multimaps.newListMultimap;
import static com.google.common.collect.Multimaps.toMultimap;
import com.google.auto.value.AutoValue;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import com.google.common.collect.Multimap;
import com.google.common.collect.Sets;
import google.registry.bsa.api.BlockLabel;
import google.registry.bsa.api.BlockLabel.LabelType;
import google.registry.bsa.api.BlockOrder;
import google.registry.bsa.api.BlockOrder.OrderType;
import google.registry.bsa.persistence.DownloadSchedule;
import google.registry.bsa.persistence.DownloadSchedule.CompletedJob;
import google.registry.tldconfig.idn.IdnTableEnum;
import java.util.HashMap;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.stream.Stream;
import javax.inject.Inject;
/** Creates diffs between the most recent download and the previous one. */
class BsaDiffCreator {
private static final Splitter LINE_SPLITTER = Splitter.on(',').trimResults();
private static final Splitter ORDER_SPLITTER = Splitter.on(';').trimResults();
private static final String BSA_CSV_HEADER = "domainLabel,orderIDs";
/** An impossible value for order ID. See {@link #createDiff} for usage. */
static final Long ORDER_ID_SENTINEL = Long.MIN_VALUE;
private final GcsClient gcsClient;
@Inject
BsaDiffCreator(GcsClient gcsClient) {
this.gcsClient = gcsClient;
}
private <K, V extends Comparable> Multimap<K, V> listBackedMultiMap() {
return newListMultimap(newHashMap(), Lists::newArrayList);
}
BsaDiff createDiff(DownloadSchedule schedule, IdnChecker idnChecker) {
String currentJobName = schedule.jobName();
Optional<String> previousJobName = schedule.latestCompleted().map(CompletedJob::jobName);
/*
* Memory usage is a concern when creating a diff, when the newest download needs to be held in
* memory in its entirety. The top-grade AppEngine VM has 3GB of memory, leaving less than 1.5GB
* to application memory footprint after subtracting overheads due to copying garbage collection
* and non-heap data etc. Assuming 400K labels, each of which on average included in 5 orders,
* the memory footprint is at least 300MB when loaded into a Hashset-backed Multimap (64-bit
* JVM, with 12-byte object header, 16-byte array header, and 16-byte alignment).
*
* <p>The memory footprint can be reduced in two ways, by using a canonical instance for each
* order ID value, and by using a ArrayList-backed Multimap. Together they reduce memory size to
* well below 100MB for the scenario above.
*
* <p>We need to watch out for the download sizes even after the migration to GKE. However, at
* that point we will have a wider selection of hardware.
*
* <p>Beam pipeline is not a good option. It has to be launched as a separate, asynchronous job,
* and there is no guaranteed limit to launch delay. Both issues would increase code complexity.
*/
Canonicals<Long> canonicals = new Canonicals<>();
try (Stream<Line> currentStream = loadBlockLists(currentJobName);
Stream<Line> previousStream =
previousJobName.map(this::loadBlockLists).orElseGet(Stream::of)) {
/*
* Load current label/order pairs into a multimap, which will contain both new labels and
* those that stay on when processing is done.
*/
Multimap<String, Long> newAndRemaining =
currentStream
.map(line -> line.labelOrderPairs(canonicals))
.flatMap(x -> x)
.collect(
toMultimap(
LabelOrderPair::label, LabelOrderPair::orderId, this::listBackedMultiMap));
Multimap<String, Long> deleted =
previousStream
.map(
line -> {
// Mark labels that exist in both downloads with the SENTINEL id. This helps
// distinguish existing label with new order from new labels.
if (newAndRemaining.containsKey(line.label())
&& !newAndRemaining.containsEntry(line.label(), ORDER_ID_SENTINEL)) {
newAndRemaining.put(line.label(), ORDER_ID_SENTINEL);
}
return line;
})
.map(line -> line.labelOrderPairs(canonicals))
.flatMap(x -> x)
.filter(kv -> !newAndRemaining.remove(kv.label(), kv.orderId()))
.collect(
toMultimap(
LabelOrderPair::label, LabelOrderPair::orderId, this::listBackedMultiMap));
/*
* Labels in `newAndRemaining`:
*
* <ul>
* <li>Mapped to `sentinel` only: Labels without change, ignore
* <li>Mapped to `sentinel` and some orders: Existing labels with new order mapping. Those
* orders are new orders.
* <li>Mapped to some orders but not `sentinel`: New labels and new orders.
* </ul>
*
* <p>The `deleted` map has
*
* <ul>
* <li>Deleted labels: the keyset of deleted minus the keyset of the newAndRemaining
* <li>Deleted orders: the union of values.
* </ul>
*/
return new BsaDiff(
ImmutableMultimap.copyOf(newAndRemaining), ImmutableMultimap.copyOf(deleted), idnChecker);
}
}
Stream<Line> loadBlockLists(String jobName) {
return Stream.of(BlockListType.values())
.map(blockList -> gcsClient.readBlockList(jobName, blockList))
.flatMap(x -> x)
.filter(line -> !line.startsWith(BSA_CSV_HEADER))
.map(BsaDiffCreator::parseLine);
}
static Line parseLine(String line) {
List<String> columns = LINE_SPLITTER.splitToList(line);
checkArgument(columns.size() == 2, "Invalid line: [%s]", line);
checkArgument(!Strings.isNullOrEmpty(columns.get(0)), "Missing label in line: [%s]", line);
try {
ImmutableList<Long> orderIds =
ORDER_SPLITTER
.splitToStream(columns.get(1))
.map(Long::valueOf)
.collect(toImmutableList());
checkArgument(!orderIds.isEmpty(), "Missing orders in line: [%s]", line);
checkArgument(
!orderIds.contains(ORDER_ID_SENTINEL), "Invalid order id %s", ORDER_ID_SENTINEL);
return Line.of(columns.get(0), orderIds);
} catch (NumberFormatException e) {
throw new IllegalArgumentException(line, e);
}
}
static class BsaDiff {
private final ImmutableMultimap<String, Long> newAndRemaining;
private final ImmutableMultimap<String, Long> deleted;
private final IdnChecker idnChecker;
BsaDiff(
ImmutableMultimap<String, Long> newAndRemaining,
ImmutableMultimap<String, Long> deleted,
IdnChecker idnChecker) {
this.newAndRemaining = newAndRemaining;
this.deleted = deleted;
this.idnChecker = idnChecker;
}
Stream<BlockOrder> getOrders() {
return Stream.concat(
newAndRemaining.values().stream()
.filter(value -> !Objects.equals(ORDER_ID_SENTINEL, value))
.distinct()
.map(id -> BlockOrder.of(id, OrderType.CREATE)),
deleted.values().stream().distinct().map(id -> BlockOrder.of(id, OrderType.DELETE)));
}
Stream<BlockLabel> getLabels() {
return Stream.of(
newAndRemaining.asMap().entrySet().stream()
.filter(e -> e.getValue().size() > 1 || !e.getValue().contains(ORDER_ID_SENTINEL))
.filter(entry -> entry.getValue().contains(ORDER_ID_SENTINEL))
.map(
entry ->
BlockLabel.of(
entry.getKey(),
LabelType.NEW_ORDER_ASSOCIATION,
getAllValidIdnNames(entry.getKey()))),
newAndRemaining.asMap().entrySet().stream()
.filter(e -> e.getValue().size() > 1 || !e.getValue().contains(ORDER_ID_SENTINEL))
.filter(entry -> !entry.getValue().contains(ORDER_ID_SENTINEL))
.map(
entry ->
BlockLabel.of(
entry.getKey(),
LabelType.CREATE,
getAllValidIdnNames(entry.getKey()))),
Sets.difference(deleted.keySet(), newAndRemaining.keySet()).stream()
.map(label -> BlockLabel.of(label, LabelType.DELETE, getAllValidIdnNames(label))))
.flatMap(x -> x);
}
ImmutableSet<String> getAllValidIdnNames(String label) {
return idnChecker.getAllValidIdns(label).stream()
.map(IdnTableEnum::name)
.collect(toImmutableSet());
}
}
static class Canonicals<T> {
private final HashMap<T, T> cache;
Canonicals() {
cache = Maps.newHashMap();
}
T get(T value) {
cache.putIfAbsent(value, value);
return cache.get(value);
}
}
@AutoValue
abstract static class LabelOrderPair {
abstract String label();
abstract Long orderId();
static LabelOrderPair of(String key, Long value) {
return new AutoValue_BsaDiffCreator_LabelOrderPair(key, value);
}
}
@AutoValue
abstract static class Line {
abstract String label();
abstract ImmutableList<Long> orderIds();
Stream<LabelOrderPair> labelOrderPairs(Canonicals<Long> canonicals) {
return orderIds().stream().map(id -> LabelOrderPair.of(label(), canonicals.get(id)));
}
static Line of(String label, ImmutableList<Long> orderIds) {
return new AutoValue_BsaDiffCreator_Line(label, orderIds);
}
}
}

View File

@@ -0,0 +1,238 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static google.registry.bsa.BlockListType.BLOCK;
import static google.registry.bsa.BlockListType.BLOCK_PLUS;
import static google.registry.bsa.api.JsonSerializations.toCompletedOrdersReport;
import static google.registry.bsa.api.JsonSerializations.toInProgressOrdersReport;
import static google.registry.bsa.api.JsonSerializations.toUnblockableDomainsReport;
import static google.registry.bsa.persistence.LabelDiffUpdates.applyLabelDiff;
import static google.registry.request.Action.Method.GET;
import static google.registry.request.Action.Method.POST;
import static google.registry.util.BatchedStreams.toBatches;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.flogger.FluentLogger;
import dagger.Lazy;
import google.registry.bsa.BlockListFetcher.LazyBlockList;
import google.registry.bsa.BsaDiffCreator.BsaDiff;
import google.registry.bsa.api.BlockLabel;
import google.registry.bsa.api.BlockOrder;
import google.registry.bsa.api.BsaReportSender;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.persistence.DownloadSchedule;
import google.registry.bsa.persistence.DownloadScheduler;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.tld.Tlds;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import java.util.Objects;
import java.util.Optional;
import java.util.stream.Stream;
import javax.inject.Inject;
@Action(
service = Action.Service.BSA,
path = BsaDownloadAction.PATH,
method = {GET, POST},
auth = Auth.AUTH_API_ADMIN)
public class BsaDownloadAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
static final String PATH = "/_dr/task/bsaDownload";
private final DownloadScheduler downloadScheduler;
private final BlockListFetcher blockListFetcher;
private final BsaDiffCreator diffCreator;
private final BsaReportSender bsaReportSender;
private final GcsClient gcsClient;
private final Lazy<IdnChecker> lazyIdnChecker;
private final BsaLock bsaLock;
private final Clock clock;
private final int transactionBatchSize;
private final Response response;
@Inject
BsaDownloadAction(
DownloadScheduler downloadScheduler,
BlockListFetcher blockListFetcher,
BsaDiffCreator diffCreator,
BsaReportSender bsaReportSender,
GcsClient gcsClient,
Lazy<IdnChecker> lazyIdnChecker,
BsaLock bsaLock,
Clock clock,
@Config("bsaTxnBatchSize") int transactionBatchSize,
Response response) {
this.downloadScheduler = downloadScheduler;
this.blockListFetcher = blockListFetcher;
this.diffCreator = diffCreator;
this.bsaReportSender = bsaReportSender;
this.gcsClient = gcsClient;
this.lazyIdnChecker = lazyIdnChecker;
this.bsaLock = bsaLock;
this.clock = clock;
this.transactionBatchSize = transactionBatchSize;
this.response = response;
}
@Override
public void run() {
try {
if (!bsaLock.executeWithLock(this::runWithinLock)) {
logger.atInfo().log("Job is being executed by another worker.");
}
} catch (Throwable throwable) {
// TODO(12/31/2023): consider sending an alert email.
// TODO: if unretriable errors, log at severe and send email.
logger.atWarning().withCause(throwable).log("Failed to update block lists.");
}
// Always return OK. Let the next cron job retry.
response.setStatus(SC_OK);
}
Void runWithinLock() {
// Cannot enroll new TLDs after download starts. This may change if b/309175410 is fixed.
if (!Tlds.hasActiveBsaEnrollment(clock.nowUtc())) {
logger.atInfo().log("No TLDs enrolled with BSA. Quitting.");
return null;
}
Optional<DownloadSchedule> scheduleOptional = downloadScheduler.schedule();
if (!scheduleOptional.isPresent()) {
logger.atInfo().log("Nothing to do.");
return null;
}
BsaDiff diff = null;
DownloadSchedule schedule = scheduleOptional.get();
switch (schedule.stage()) {
case DOWNLOAD_BLOCK_LISTS:
try (LazyBlockList block = blockListFetcher.fetch(BLOCK);
LazyBlockList blockPlus = blockListFetcher.fetch(BLOCK_PLUS)) {
ImmutableMap<BlockListType, String> fetchedChecksums =
ImmutableMap.of(BLOCK, block.checksum(), BLOCK_PLUS, blockPlus.checksum());
ImmutableMap<BlockListType, String> prevChecksums =
schedule
.latestCompleted()
.map(DownloadSchedule.CompletedJob::checksums)
.orElseGet(ImmutableMap::of);
boolean checksumsMatch = Objects.equals(fetchedChecksums, prevChecksums);
if (!schedule.alwaysDownload() && checksumsMatch) {
logger.atInfo().log(
"Skipping download b/c block list checksums have not changed: [%s]",
fetchedChecksums);
schedule.updateJobStage(DownloadStage.NOP, fetchedChecksums);
return null;
} else if (checksumsMatch) {
logger.atInfo().log(
"Checksums match but download anyway: elapsed time since last download exceeds"
+ " configured limit.");
}
// When downloading, always fetch both lists so that whole data set is in one GCS folder.
ImmutableMap<BlockListType, String> actualChecksum =
gcsClient.saveAndChecksumBlockList(
schedule.jobName(), ImmutableList.of(block, blockPlus));
if (!Objects.equals(fetchedChecksums, actualChecksum)) {
logger.atSevere().log(
"Inlined checksums do not match those calculated by us. Theirs: [%s]; ours: [%s]",
fetchedChecksums, actualChecksum);
schedule.updateJobStage(DownloadStage.CHECKSUMS_DO_NOT_MATCH, fetchedChecksums);
// TODO(01/15/24): add email alert.
return null;
}
schedule.updateJobStage(DownloadStage.MAKE_ORDER_AND_LABEL_DIFF, actualChecksum);
}
// Fall through
case MAKE_ORDER_AND_LABEL_DIFF:
diff = diffCreator.createDiff(schedule, lazyIdnChecker.get());
// TODO(weiminyu): log the diff stats
gcsClient.writeOrderDiffs(schedule.jobName(), diff.getOrders());
gcsClient.writeLabelDiffs(schedule.jobName(), diff.getLabels());
schedule.updateJobStage(DownloadStage.APPLY_ORDER_AND_LABEL_DIFF);
// Fall through
case APPLY_ORDER_AND_LABEL_DIFF:
try (Stream<BlockLabel> labels =
diff != null ? diff.getLabels() : gcsClient.readLabelDiffs(schedule.jobName())) {
Stream<ImmutableList<BlockLabel>> batches = toBatches(labels, transactionBatchSize);
gcsClient.writeUnblockableDomains(
schedule.jobName(),
batches
.map(
batch ->
applyLabelDiff(batch, lazyIdnChecker.get(), schedule, clock.nowUtc()))
.flatMap(ImmutableList::stream));
}
schedule.updateJobStage(DownloadStage.REPORT_START_OF_ORDER_PROCESSING);
// Fall through
case REPORT_START_OF_ORDER_PROCESSING:
try (Stream<BlockOrder> orders = gcsClient.readOrderDiffs(schedule.jobName())) {
// We expect that all order instances and the json string can fit in memory.
Optional<String> report = toInProgressOrdersReport(orders);
if (report.isPresent()) {
// Log report data
gcsClient.logInProgressOrderReport(
schedule.jobName(), BsaStringUtils.LINE_SPLITTER.splitToStream(report.get()));
bsaReportSender.sendOrderStatusReport(report.get());
} else {
logger.atInfo().log("No new or deleted orders in this round.");
}
}
schedule.updateJobStage(DownloadStage.UPLOAD_UNBLOCKABLE_DOMAINS_FOR_NEW_ORDERS);
// Fall through
case UPLOAD_UNBLOCKABLE_DOMAINS_FOR_NEW_ORDERS:
try (Stream<UnblockableDomain> unblockables =
gcsClient.readUnblockableDomains(schedule.jobName())) {
/* The number of unblockable domains may be huge in theory (label x ~50 tlds), but in
* practice should be relatively small (tens of thousands?). Batches can be introduced
* if size becomes a problem.
*/
Optional<String> report = toUnblockableDomainsReport(unblockables);
if (report.isPresent()) {
gcsClient.logAddedUnblockableDomainsReport(
schedule.jobName(), BsaStringUtils.LINE_SPLITTER.splitToStream(report.get()));
// During downloads, unblockable domains are only added, not removed.
bsaReportSender.addUnblockableDomainsUpdates(report.get());
} else {
logger.atInfo().log("No changes in the set of unblockable domains in this round.");
}
}
schedule.updateJobStage(DownloadStage.REPORT_END_OF_ORDER_PROCESSING);
// Fall through
case REPORT_END_OF_ORDER_PROCESSING:
try (Stream<BlockOrder> orders = gcsClient.readOrderDiffs(schedule.jobName())) {
// Orders are expected to be few, so the report can be kept in memory.
Optional<String> report = toCompletedOrdersReport(orders);
if (report.isPresent()) {
gcsClient.logCompletedOrderReport(
schedule.jobName(), BsaStringUtils.LINE_SPLITTER.splitToStream(report.get()));
bsaReportSender.sendOrderStatusReport(report.get());
}
}
schedule.updateJobStage(DownloadStage.DONE);
return null;
case DONE:
case NOP:
case CHECKSUMS_DO_NOT_MATCH:
logger.atWarning().log("Unexpectedly reached the %s stage.", schedule.stage());
break;
}
return null;
}
}

View File

@@ -14,32 +14,27 @@
package google.registry.bsa;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.lock.LockHandler;
import java.util.concurrent.Callable;
import javax.inject.Inject;
import org.joda.time.Duration;
@Action(
service = Service.BSA,
path = PlaceholderAction.PATH,
method = Action.Method.GET,
auth = Auth.AUTH_API_ADMIN)
public class PlaceholderAction implements Runnable {
private final Response response;
/** Helper for guarding all BSA related work with a common lock. */
public class BsaLock {
static final String PATH = "/_dr/task/bsaDownload";
private static final String LOCK_NAME = "all-bsa-jobs";
private final LockHandler lockHandler;
private final Duration leaseExpiry;
@Inject
public PlaceholderAction(Response response) {
this.response = response;
BsaLock(LockHandler lockHandler, @Config("bsaLockLeaseExpiry") Duration leaseExpiry) {
this.lockHandler = lockHandler;
this.leaseExpiry = leaseExpiry;
}
@Override
public void run() {
response.setStatus(SC_OK);
response.setPayload("Hello World");
boolean executeWithLock(Callable<Void> callable) {
return lockHandler.executeWithLocks(callable, null, leaseExpiry, LOCK_NAME);
}
}

View File

@@ -0,0 +1,174 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static google.registry.bsa.BsaStringUtils.LINE_SPLITTER;
import static google.registry.request.Action.Method.GET;
import static google.registry.request.Action.Method.POST;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.common.collect.ImmutableList;
import com.google.common.flogger.FluentLogger;
import google.registry.bsa.api.BsaReportSender;
import google.registry.bsa.api.JsonSerializations;
import google.registry.bsa.api.UnblockableDomainChange;
import google.registry.bsa.persistence.DomainsRefresher;
import google.registry.bsa.persistence.RefreshSchedule;
import google.registry.bsa.persistence.RefreshScheduler;
import google.registry.config.RegistryConfig.Config;
import google.registry.model.tld.Tlds;
import google.registry.request.Action;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.BatchedStreams;
import google.registry.util.Clock;
import java.util.Optional;
import java.util.stream.Stream;
import javax.inject.Inject;
import org.joda.time.Duration;
@Action(
service = Action.Service.BSA,
path = BsaRefreshAction.PATH,
method = {GET, POST},
auth = Auth.AUTH_API_ADMIN)
public class BsaRefreshAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
static final String PATH = "/_dr/task/bsaRefresh";
private final RefreshScheduler scheduler;
private final GcsClient gcsClient;
private final BsaReportSender bsaReportSender;
private final int transactionBatchSize;
private final Duration domainTxnMaxDuration;
private final BsaLock bsaLock;
private final Clock clock;
private final Response response;
@Inject
BsaRefreshAction(
RefreshScheduler scheduler,
GcsClient gcsClient,
BsaReportSender bsaReportSender,
@Config("bsaTxnBatchSize") int transactionBatchSize,
@Config("domainTxnMaxDuration") Duration domainTxnMaxDuration,
BsaLock bsaLock,
Clock clock,
Response response) {
this.scheduler = scheduler;
this.gcsClient = gcsClient;
this.bsaReportSender = bsaReportSender;
this.transactionBatchSize = transactionBatchSize;
this.domainTxnMaxDuration = domainTxnMaxDuration;
this.bsaLock = bsaLock;
this.clock = clock;
this.response = response;
}
@Override
public void run() {
try {
if (!bsaLock.executeWithLock(this::runWithinLock)) {
logger.atInfo().log("Job is being executed by another worker.");
}
} catch (Throwable throwable) {
// TODO(12/31/2023): consider sending an alert email.
logger.atWarning().withCause(throwable).log("Failed to update block lists.");
}
// Always return OK. No need to use a retrier on `runWithinLock`. Its individual steps are
// implicitly retried. If action fails, the next cron will continue at checkpoint.
response.setStatus(SC_OK);
}
/** Executes the refresh action while holding the BSA lock. */
Void runWithinLock() {
// Cannot enroll new TLDs after download starts. This may change if b/309175410 is fixed.
if (!Tlds.hasActiveBsaEnrollment(clock.nowUtc())) {
logger.atInfo().log("No TLDs enrolled with BSA. Quitting.");
return null;
}
Optional<RefreshSchedule> maybeSchedule = scheduler.schedule();
if (!maybeSchedule.isPresent()) {
logger.atInfo().log("No completed downloads yet. Exiting.");
return null;
}
RefreshSchedule schedule = maybeSchedule.get();
DomainsRefresher refresher =
new DomainsRefresher(
schedule.prevRefreshTime(), clock.nowUtc(), domainTxnMaxDuration, transactionBatchSize);
switch (schedule.stage()) {
case CHECK_FOR_CHANGES:
ImmutableList<UnblockableDomainChange> blockabilityChanges =
refresher.checkForBlockabilityChanges();
if (blockabilityChanges.isEmpty()) {
logger.atInfo().log("No change to Unblockable domains found.");
schedule.updateJobStage(RefreshStage.DONE);
return null;
}
gcsClient.writeRefreshChanges(schedule.jobName(), blockabilityChanges.stream());
schedule.updateJobStage(RefreshStage.APPLY_CHANGES);
// Fall through
case APPLY_CHANGES:
try (Stream<UnblockableDomainChange> changes =
gcsClient.readRefreshChanges(schedule.jobName())) {
BatchedStreams.toBatches(changes, 500).forEach(refresher::applyUnblockableChanges);
}
schedule.updateJobStage(RefreshStage.UPLOAD_REMOVALS);
// Fall through
case UPLOAD_REMOVALS:
try (Stream<UnblockableDomainChange> changes =
gcsClient.readRefreshChanges(schedule.jobName())) {
Optional<String> report =
JsonSerializations.toUnblockableDomainsRemovalReport(
changes
.filter(UnblockableDomainChange::isDelete)
.map(UnblockableDomainChange::domainName));
if (report.isPresent()) {
gcsClient.logRemovedUnblockableDomainsReport(
schedule.jobName(), LINE_SPLITTER.splitToStream(report.get()));
bsaReportSender.removeUnblockableDomainsUpdates(report.get());
} else {
logger.atInfo().log("No Unblockable domains to remove.");
}
}
schedule.updateJobStage(RefreshStage.UPLOAD_ADDITIONS);
// Fall through
case UPLOAD_ADDITIONS:
try (Stream<UnblockableDomainChange> changes =
gcsClient.readRefreshChanges(schedule.jobName())) {
Optional<String> report =
JsonSerializations.toUnblockableDomainsReport(
changes
.filter(UnblockableDomainChange::isAddOrChange)
.map(UnblockableDomainChange::newValue));
if (report.isPresent()) {
gcsClient.logRemovedUnblockableDomainsReport(
schedule.jobName(), LINE_SPLITTER.splitToStream(report.get()));
bsaReportSender.removeUnblockableDomainsUpdates(report.get());
} else {
logger.atInfo().log("No new Unblockable domains to add.");
}
}
schedule.updateJobStage(RefreshStage.DONE);
break;
case DONE:
logger.atInfo().log("Unexpectedly reaching the `DONE` stage.");
break;
}
return null;
}
}

View File

@@ -0,0 +1,45 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static com.google.common.base.Preconditions.checkArgument;
import com.google.common.base.Joiner;
import com.google.common.base.Splitter;
import java.util.List;
/** Helpers for domain name manipulation and string serialization of Java objects. */
public class BsaStringUtils {
public static final Joiner DOMAIN_JOINER = Joiner.on('.');
public static final Joiner PROPERTY_JOINER = Joiner.on(',');
public static final Splitter DOMAIN_SPLITTER = Splitter.on('.');
public static final Splitter PROPERTY_SPLITTER = Splitter.on(',');
public static final Splitter LINE_SPLITTER = Splitter.on('\n');
public static String getLabelInDomain(String domainName) {
List<String> parts = DOMAIN_SPLITTER.limit(1).splitToList(domainName);
checkArgument(!parts.isEmpty(), "Not a valid domain: [%s]", domainName);
return parts.get(0);
}
public static String getTldInDomain(String domainName) {
List<String> parts = DOMAIN_SPLITTER.splitToList(domainName);
checkArgument(parts.size() == 2, "Not a valid domain: [%s]", domainName);
return parts.get(1);
}
private BsaStringUtils() {}
}

View File

@@ -0,0 +1,42 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static google.registry.persistence.PersistenceModule.TransactionIsolationLevel.TRANSACTION_REPEATABLE_READ;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.errorprone.annotations.CanIgnoreReturnValue;
import java.util.concurrent.Callable;
/**
* Helpers for executing JPA transactions for BSA processing.
*
* <p>All mutating transactions for BSA may be executed at the {@code TRANSACTION_REPEATABLE_READ}
* level.
*/
public final class BsaTransactions {
@CanIgnoreReturnValue
public static <T> T bsaTransact(Callable<T> work) {
return tm().transact(work, TRANSACTION_REPEATABLE_READ);
}
@CanIgnoreReturnValue
public static <T> T bsaQuery(Callable<T> work) {
return tm().transact(work, TRANSACTION_REPEATABLE_READ);
}
private BsaTransactions() {}
}

View File

@@ -14,23 +14,32 @@
package google.registry.bsa;
import google.registry.bsa.api.BlockLabel;
import google.registry.bsa.api.BlockOrder;
/** The processing stages of a download. */
public enum DownloadStage {
/** Downloads BSA block list files. */
DOWNLOAD,
/** Generates block list diffs with the previous download. */
MAKE_DIFF,
/** Applies the label diffs to the database tables. */
APPLY_DIFF,
DOWNLOAD_BLOCK_LISTS,
/**
* Generates block list diffs against the previous download. The diffs consist of a stream of
* {@link BlockOrder orders} and a stream of {@link BlockLabel labels}.
*/
MAKE_ORDER_AND_LABEL_DIFF,
/** Applies the diffs to the database. */
APPLY_ORDER_AND_LABEL_DIFF,
/**
* Makes a REST API call to BSA endpoint, declaring that processing starts for new orders in the
* diffs.
*/
START_UPLOADING,
/** Makes a REST API call to BSA endpoint, sending the domains that cannot be blocked. */
UPLOAD_DOMAINS_IN_USE,
REPORT_START_OF_ORDER_PROCESSING,
/**
* Makes a REST API call to BSA endpoint, uploading unblockable domains that match labels in the
* diff.
*/
UPLOAD_UNBLOCKABLE_DOMAINS_FOR_NEW_ORDERS,
/** Makes a REST API call to BSA endpoint, declaring the completion of order processing. */
FINISH_UPLOADING,
REPORT_END_OF_ORDER_PROCESSING,
/** The terminal stage after processing succeeds. */
DONE,
/**
@@ -42,5 +51,5 @@ public enum DownloadStage {
* The terminal stage indicating that the downloads are not processed because their BSA-generated
* checksums do not match those calculated by us.
*/
CHECKSUMS_NOT_MATCH;
CHECKSUMS_DO_NOT_MATCH;
}

View File

@@ -0,0 +1,229 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static com.google.common.io.BaseEncoding.base16;
import com.google.cloud.storage.BlobId;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import google.registry.bsa.BlockListFetcher.LazyBlockList;
import google.registry.bsa.api.BlockLabel;
import google.registry.bsa.api.BlockOrder;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.api.UnblockableDomainChange;
import google.registry.config.RegistryConfig.Config;
import google.registry.gcs.GcsUtils;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.stream.Stream;
import javax.inject.Inject;
/** Stores and accesses BSA-related data, including original downloads and processed data. */
public class GcsClient {
// Intermediate data files:
static final String LABELS_DIFF_FILE = "labels_diff.csv";
static final String ORDERS_DIFF_FILE = "orders_diff.csv";
static final String UNBLOCKABLE_DOMAINS_FILE = "unblockable_domains.csv";
static final String REFRESHED_UNBLOCKABLE_DOMAINS_FILE = "refreshed_unblockable_domains.csv";
// Logged report data sent to BSA.
static final String IN_PROGRESS_ORDERS_REPORT = "in_progress_orders.json";
static final String COMPLETED_ORDERS_REPORT = "completed_orders.json";
static final String ADDED_UNBLOCKABLE_DOMAINS_REPORT = "added_unblockable_domains.json";
static final String REMOVED_UNBLOCKABLE_DOMAINS_REPORT = "removed_unblockable_domains.json";
private final GcsUtils gcsUtils;
private final String bucketName;
private final String checksumAlgorithm;
@Inject
GcsClient(
GcsUtils gcsUtils,
@Config("bsaGcsBucket") String bucketName,
@Config("bsaChecksumAlgorithm") String checksumAlgorithm) {
this.gcsUtils = gcsUtils;
this.bucketName = bucketName;
this.checksumAlgorithm = checksumAlgorithm;
}
static String getBlockListFileName(BlockListType blockListType) {
return blockListType.name() + ".csv";
}
ImmutableMap<BlockListType, String> saveAndChecksumBlockList(
String jobName, ImmutableList<LazyBlockList> blockLists) {
// Downloading sequentially, since one is expected to be much smaller than the other.
return blockLists.stream()
.collect(
ImmutableMap.toImmutableMap(
LazyBlockList::getName, blockList -> saveAndChecksumBlockList(jobName, blockList)));
}
private String saveAndChecksumBlockList(String jobName, LazyBlockList blockList) {
BlobId blobId = getBlobId(jobName, getBlockListFileName(blockList.getName()));
try (BufferedOutputStream gcsWriter =
new BufferedOutputStream(gcsUtils.openOutputStream(blobId))) {
MessageDigest messageDigest = MessageDigest.getInstance(checksumAlgorithm);
blockList.consumeAll(
(byteArray, length) -> {
try {
gcsWriter.write(byteArray, 0, length);
} catch (IOException e) {
throw new RuntimeException(e);
}
messageDigest.update(byteArray, 0, length);
});
return base16().lowerCase().encode(messageDigest.digest());
} catch (IOException | NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
}
private static void writeWithNewline(BufferedWriter writer, String line) {
try {
writer.write(line);
if (!line.endsWith("\n")) {
writer.write('\n');
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Stream<String> readBlockList(String jobName, BlockListType blockListType) {
return readStream(getBlobId(jobName, getBlockListFileName(blockListType)));
}
Stream<BlockOrder> readOrderDiffs(String jobName) {
BlobId blobId = getBlobId(jobName, ORDERS_DIFF_FILE);
return readStream(blobId).map(BlockOrder::deserialize);
}
void writeOrderDiffs(String jobName, Stream<BlockOrder> orders) {
BlobId blobId = getBlobId(jobName, ORDERS_DIFF_FILE);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
orders.map(BlockOrder::serialize).forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Stream<BlockLabel> readLabelDiffs(String jobName) {
BlobId blobId = getBlobId(jobName, LABELS_DIFF_FILE);
return readStream(blobId).map(BlockLabel::deserialize);
}
void writeLabelDiffs(String jobName, Stream<BlockLabel> labels) {
BlobId blobId = getBlobId(jobName, LABELS_DIFF_FILE);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
labels.map(BlockLabel::serialize).forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Stream<UnblockableDomain> readUnblockableDomains(String jobName) {
BlobId blobId = getBlobId(jobName, UNBLOCKABLE_DOMAINS_FILE);
return readStream(blobId).map(UnblockableDomain::deserialize);
}
void writeUnblockableDomains(String jobName, Stream<UnblockableDomain> unblockables) {
BlobId blobId = getBlobId(jobName, UNBLOCKABLE_DOMAINS_FILE);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
unblockables
.map(UnblockableDomain::serialize)
.forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Stream<UnblockableDomainChange> readRefreshChanges(String jobName) {
BlobId blobId = getBlobId(jobName, UNBLOCKABLE_DOMAINS_FILE);
return readStream(blobId).map(UnblockableDomainChange::deserialize);
}
void writeRefreshChanges(String jobName, Stream<UnblockableDomainChange> changes) {
BlobId blobId = getBlobId(jobName, REFRESHED_UNBLOCKABLE_DOMAINS_FILE);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
changes
.map(UnblockableDomainChange::serialize)
.forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
void logInProgressOrderReport(String jobName, Stream<String> lines) {
BlobId blobId = getBlobId(jobName, IN_PROGRESS_ORDERS_REPORT);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
lines.forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
void logCompletedOrderReport(String jobName, Stream<String> lines) {
BlobId blobId = getBlobId(jobName, COMPLETED_ORDERS_REPORT);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
lines.forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
void logAddedUnblockableDomainsReport(String jobName, Stream<String> lines) {
BlobId blobId = getBlobId(jobName, ADDED_UNBLOCKABLE_DOMAINS_REPORT);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
lines.forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
void logRemovedUnblockableDomainsReport(String jobName, Stream<String> lines) {
BlobId blobId = getBlobId(jobName, REMOVED_UNBLOCKABLE_DOMAINS_REPORT);
try (BufferedWriter gcsWriter = getWriter(blobId)) {
lines.forEach(line -> writeWithNewline(gcsWriter, line));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
BlobId getBlobId(String folder, String name) {
return BlobId.of(bucketName, String.format("%s/%s", folder, name));
}
Stream<String> readStream(BlobId blobId) {
return new BufferedReader(
new InputStreamReader(gcsUtils.openInputStream(blobId), StandardCharsets.UTF_8))
.lines();
}
BufferedWriter getWriter(BlobId blobId) {
return new BufferedWriter(
new OutputStreamWriter(gcsUtils.openOutputStream(blobId), StandardCharsets.UTF_8));
}
}

View File

@@ -22,7 +22,6 @@ import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import com.google.common.collect.Sets.SetView;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.model.tld.Tlds;
@@ -76,8 +75,8 @@ public class IdnChecker {
*
* @param idnTables String names of {@link IdnTableEnum} values
*/
public SetView<Tld> getForbiddingTlds(ImmutableSet<String> idnTables) {
return Sets.difference(allTlds, getSupportingTlds(idnTables));
public ImmutableSet<Tld> getForbiddingTlds(ImmutableSet<String> idnTables) {
return Sets.difference(allTlds, getSupportingTlds(idnTables)).immutableCopy();
}
private static ImmutableMap<IdnTableEnum, ImmutableSet<Tld>> getIdnToTldMap(DateTime now) {

View File

@@ -0,0 +1,30 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
public enum RefreshStage {
/**
* Checks for stale unblockable domains. The output is a stream of {@link
* google.registry.bsa.api.UnblockableDomainChange} objects that describe the stale domains.
*/
CHECK_FOR_CHANGES,
/** Fixes the stale domains in the database. */
APPLY_CHANGES,
/** Reports the unblockable domains to be removed to BSA. */
UPLOAD_REMOVALS,
/** Reports the newly found unblockable domains to BSA. */
UPLOAD_ADDITIONS,
DONE;
}

View File

@@ -0,0 +1,80 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static com.google.common.base.Verify.verify;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.bsa.BsaStringUtils.DOMAIN_JOINER;
import static google.registry.flows.domain.DomainFlowUtils.isReserved;
import static google.registry.model.tld.Tlds.findTldForName;
import com.google.common.collect.ImmutableSet;
import com.google.common.net.InternetDomainName;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldState;
import google.registry.model.tld.Tld.TldType;
import google.registry.model.tld.Tlds;
import google.registry.model.tld.label.ReservedList;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Stream;
import org.joda.time.DateTime;
/**
* Utility for looking up reserved domain names.
*
* <p>This utility is only concerned with reserved domains that can be created (with appropriate
* tokens).
*/
public final class ReservedDomainsUtils {
private ReservedDomainsUtils() {}
public static Stream<String> getAllReservedNames(DateTime now) {
return Tlds.getTldEntitiesOfType(TldType.REAL).stream()
.filter(tld -> Tld.isEnrolledWithBsa(tld, now))
.map(tld -> getAllReservedDomainsInTld(tld, now))
.flatMap(ImmutableSet::stream);
}
/** Returns */
static ImmutableSet<String> getAllReservedDomainsInTld(Tld tld, DateTime now) {
return tld.getReservedListNames().stream()
.map(ReservedList::get)
.filter(Optional::isPresent)
.map(Optional::get)
.map(ReservedList::getReservedListEntries)
.map(Map::keySet)
.flatMap(Set::stream)
.map(label -> DOMAIN_JOINER.join(label, tld.getTldStr()))
.filter(domain -> isReservedDomain(domain, now))
.collect(toImmutableSet());
}
/**
* Returns true if {@code domain} is a reserved name that can be registered right now (e.g.,
* during sunrise or with allocation token), therefore unblockable.
*/
public static boolean isReservedDomain(String domain, DateTime now) {
Optional<InternetDomainName> tldStr = findTldForName(InternetDomainName.from(domain));
verify(tldStr.isPresent(), "Tld for domain [%s] unexpectedly missing.", domain);
Tld tld = Tld.get(tldStr.get().toString());
return isReserved(
InternetDomainName.from(domain),
Objects.equals(tld.getTldState(now), TldState.START_DATE_SUNRISE));
}
}

View File

@@ -0,0 +1,237 @@
// Copyright 2024 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.model.tld.Tld.isEnrolledWithBsa;
import static google.registry.model.tld.Tlds.getTldEntitiesOfType;
import static google.registry.model.tld.label.ReservedList.loadReservedLists;
import static google.registry.persistence.PersistenceModule.TransactionIsolationLevel.TRANSACTION_REPEATABLE_READ;
import static google.registry.persistence.transaction.TransactionManagerFactory.replicaTm;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.request.Action.Method.GET;
import static google.registry.request.Action.Method.POST;
import static java.nio.charset.StandardCharsets.US_ASCII;
import com.google.api.client.http.HttpStatusCodes;
import com.google.cloud.storage.BlobId;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Ordering;
import com.google.common.flogger.FluentLogger;
import com.google.common.hash.Hashing;
import com.google.common.io.ByteSource;
import google.registry.bsa.api.BsaCredential;
import google.registry.config.RegistryConfig.Config;
import google.registry.gcs.GcsUtils;
import google.registry.model.tld.Tld;
import google.registry.model.tld.Tld.TldType;
import google.registry.model.tld.label.ReservedList;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.util.zip.GZIPOutputStream;
import javax.inject.Inject;
import okhttp3.MediaType;
import okhttp3.MultipartBody;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
import org.joda.time.DateTime;
/**
* Daily action that uploads unavailable domain names on applicable TLDs to BSA.
*
* <p>The upload is a single zipped text file containing combined details for all BSA-enrolled TLDs.
* The text is a newline-delimited list of punycoded fully qualified domain names, and contains all
* domains on each TLD that are registered and/or reserved.
*
* <p>The file is also uploaded to GCS to preserve it as a record for ourselves.
*/
@Action(
service = Service.BSA,
path = "/_dr/task/uploadBsaUnavailableNames",
method = {GET, POST},
auth = Auth.AUTH_API_ADMIN)
public class UploadBsaUnavailableDomainsAction implements Runnable {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
Clock clock;
BsaCredential bsaCredential;
GcsUtils gcsUtils;
String gcsBucket;
String apiUrl;
google.registry.request.Response response;
@Inject
public UploadBsaUnavailableDomainsAction(
Clock clock,
BsaCredential bsaCredential,
GcsUtils gcsUtils,
@Config("bsaUnavailableDomainsGcsBucket") String gcsBucket,
@Config("bsaUploadUnavailableDomainsUrl") String apiUrl,
google.registry.request.Response response) {
this.clock = clock;
this.bsaCredential = bsaCredential;
this.gcsUtils = gcsUtils;
this.gcsBucket = gcsBucket;
this.apiUrl = apiUrl;
this.response = response;
}
@Override
public void run() {
// TODO(mcilwain): Implement a date Cursor, have the cronjob run frequently, and short-circuit
// the run if the daily upload is already completed.
DateTime runTime = clock.nowUtc();
// TODO(mcilwain): Batch this.
String unavailableDomains =
Joiner.on("\n")
.join(tm().transact(() -> getUnavailableDomains(runTime), TRANSACTION_REPEATABLE_READ));
if (unavailableDomains.isEmpty()) {
logger.atWarning().log("No unavailable domains found; terminating.");
} else {
uploadToGcs(unavailableDomains, runTime);
uploadToBsa(unavailableDomains, runTime);
}
}
/** Uploads the unavailable domains list to GCS in the unavailable domains bucket. */
void uploadToGcs(String unavailableDomains, DateTime runTime) {
logger.atInfo().log("Uploading unavailable names file to GCS in bucket %s", gcsBucket);
BlobId blobId = BlobId.of(gcsBucket, createFilename(runTime));
try (OutputStream gcsOutput = gcsUtils.openOutputStream(blobId);
Writer osWriter = new OutputStreamWriter(gcsOutput, US_ASCII)) {
osWriter.write(unavailableDomains);
} catch (Exception e) {
logger.atSevere().withCause(e).log(
"Error writing BSA unavailable domains to GCS; skipping to BSA upload ...");
}
}
void uploadToBsa(String unavailableDomains, DateTime runTime) {
try {
byte[] gzippedContents = gzipUnavailableDomains(unavailableDomains);
String sha512Hash = ByteSource.wrap(gzippedContents).hash(Hashing.sha512()).toString();
String filename = createFilename(runTime);
OkHttpClient client = new OkHttpClient().newBuilder().build();
RequestBody body =
new MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart(
"zone",
null,
RequestBody.create(
String.format("{\"checkSum\": \"%s\"}", sha512Hash).getBytes(US_ASCII),
MediaType.parse("application/json")))
.addFormDataPart(
"file",
String.format("%s.gz", filename),
RequestBody.create(gzippedContents, MediaType.parse("application/octet-stream")))
.build();
Request request =
new Request.Builder()
.url(apiUrl)
.method("POST", body)
.addHeader("Authorization", "Bearer " + bsaCredential.getAuthToken())
.build();
logger.atInfo().log(
"Uploading unavailable domains list %s to %s with hash %s", filename, apiUrl, sha512Hash);
try (Response uploadResponse = client.newCall(request).execute()) {
logger.atInfo().log(
"Received response with code %s from server: %s",
uploadResponse.code(),
uploadResponse.body() == null ? "(none)" : uploadResponse.body().string());
}
} catch (IOException e) {
logger.atSevere().withCause(e).log("Error while attempting to upload to BSA, aborting.");
response.setStatus(HttpStatusCodes.STATUS_CODE_SERVER_ERROR);
response.setPayload("Error while attempting to upload to BSA: " + e.getMessage());
}
}
private byte[] gzipUnavailableDomains(String unavailableDomains) throws IOException {
try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
try (GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream)) {
gzipOutputStream.write(unavailableDomains.getBytes(US_ASCII));
}
return byteArrayOutputStream.toByteArray();
}
}
private static String createFilename(DateTime runTime) {
return String.format("unavailable_domains_%s.txt", runTime.toString());
}
private ImmutableSortedSet<String> getUnavailableDomains(DateTime runTime) {
ImmutableSet<Tld> bsaEnabledTlds =
getTldEntitiesOfType(TldType.REAL).stream()
.filter(tld -> isEnrolledWithBsa(tld, runTime))
.collect(toImmutableSet());
logger.atInfo().log(
"Getting unavailable domains in TLDs: %s ...",
bsaEnabledTlds.stream().map(Tld::getTldStr).collect(toImmutableSet()));
ImmutableSortedSet.Builder<String> unavailableDomains =
new ImmutableSortedSet.Builder<>(Ordering.natural());
for (Tld tld : bsaEnabledTlds) {
for (ReservedList reservedList : loadReservedLists(tld.getReservedListNames())) {
if (reservedList.getShouldPublish()) {
unavailableDomains.addAll(
reservedList.getReservedListEntries().keySet().stream()
.map(label -> toDomain(label, tld))
.collect(toImmutableSet()));
}
}
}
unavailableDomains.addAll(
replicaTm()
.query(
"SELECT domainName FROM Domain "
+ "WHERE tld IN :tlds "
+ "AND deletionTime > :now ",
String.class)
.setParameter(
"tlds", bsaEnabledTlds.stream().map(Tld::getTldStr).collect(toImmutableSet()))
.setParameter("now", runTime)
.getResultList());
ImmutableSortedSet<String> result = unavailableDomains.build();
logger.atInfo().log("Found %d total unavailable domains.", result.size());
return result;
}
private static String toDomain(String domainLabel, Tld tld) {
return String.format("%s.%s", domainLabel, tld.getTldStr());
}
}

View File

@@ -0,0 +1,64 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
import com.google.auto.value.AutoValue;
import com.google.common.base.Joiner;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableSet;
import java.util.List;
/**
* A BSA label to block. New domains with matching second-level domain (SLD) will be denied
* registration in TLDs enrolled with BSA.
*/
@AutoValue
public abstract class BlockLabel {
static final Joiner JOINER = Joiner.on(',');
static final Splitter SPLITTER = Splitter.on(',').trimResults();
public abstract String label();
public abstract LabelType labelType();
public abstract ImmutableSet<String> idnTables();
public String serialize() {
return JOINER.join(label(), labelType().name(), idnTables().stream().sorted().toArray());
}
public static BlockLabel deserialize(String text) {
List<String> items = SPLITTER.splitToList(text);
try {
return of(
items.get(0),
LabelType.valueOf(items.get(1)),
ImmutableSet.copyOf(items.subList(2, items.size())));
} catch (NumberFormatException ne) {
throw new IllegalArgumentException(text);
}
}
public static BlockLabel of(String label, LabelType type, ImmutableSet<String> idnTables) {
return new AutoValue_BlockLabel(label, type, idnTables);
}
public enum LabelType {
CREATE,
NEW_ORDER_ASSOCIATION,
DELETE;
}
}

View File

@@ -0,0 +1,57 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
import com.google.auto.value.AutoValue;
import com.google.common.base.Joiner;
import com.google.common.base.Splitter;
import java.util.List;
/**
* A BSA order, which are needed when communicating with the BSA API while processing downloaded
* block lists.
*/
@AutoValue
public abstract class BlockOrder {
public abstract long orderId();
public abstract OrderType orderType();
static final Joiner JOINER = Joiner.on(',');
static final Splitter SPLITTER = Splitter.on(',');
public String serialize() {
return JOINER.join(orderId(), orderType().name());
}
public static BlockOrder deserialize(String text) {
List<String> items = SPLITTER.splitToList(text);
try {
return of(Long.valueOf(items.get(0)), OrderType.valueOf(items.get(1)));
} catch (NumberFormatException ne) {
throw new IllegalArgumentException(text);
}
}
public static BlockOrder of(long orderId, OrderType orderType) {
return new AutoValue_BlockOrder(orderId, orderType);
}
public enum OrderType {
CREATE,
DELETE;
}
}

View File

@@ -0,0 +1,160 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
import static google.registry.request.UrlConnectionUtils.getResponseBytes;
import static java.nio.charset.StandardCharsets.UTF_8;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.api.client.http.HttpMethods;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.flogger.FluentLogger;
import com.google.gson.Gson;
import google.registry.config.RegistryConfig.Config;
import google.registry.keyring.api.Keyring;
import google.registry.request.UrlConnectionService;
import google.registry.request.UrlConnectionUtils;
import google.registry.util.Clock;
import java.io.IOException;
import java.net.URL;
import java.security.GeneralSecurityException;
import java.util.Map;
import javax.annotation.Nullable;
import javax.annotation.concurrent.ThreadSafe;
import javax.inject.Inject;
import javax.net.ssl.HttpsURLConnection;
import org.joda.time.Duration;
import org.joda.time.Instant;
/**
* A credential for accessing the BSA API.
*
* <p>Fetches on-demand an auth token from BSA's auth http endpoint and caches it for repeated use
* until the token expires (expiry set by BSA and recorded in the configuration file). An expired
* token is refreshed only when requested. Token refreshing is blocking but thread-safe.
*
* <p>The token-fetching request authenticates itself with an API key, which is stored in the Secret
* Manager.
*/
@ThreadSafe
public class BsaCredential {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** Content type of the auth http request. */
private static final String CONTENT_TYPE = "application/x-www-form-urlencoded";
/** Template of the auth http request's payload. User must provide an API key. */
private static final String AUTH_REQ_BODY_TEMPLATE = "apiKey=%s&space=BSA";
/** The variable name for the auth token in the returned json response. */
public static final String ID_TOKEN = "id_token";
private final UrlConnectionService urlConnectionService;
private final String authUrl;
private final Duration authTokenExpiry;
private final Keyring keyring;
private final Clock clock;
@Nullable private String authToken;
private Instant lastRefreshTime;
@Inject
BsaCredential(
UrlConnectionService urlConnectionService,
@Config("bsaAuthUrl") String authUrl,
@Config("bsaAuthTokenExpiry") Duration authTokenExpiry,
Keyring keyring,
Clock clock) {
this.urlConnectionService = urlConnectionService;
this.authUrl = authUrl;
this.authTokenExpiry = authTokenExpiry;
this.keyring = keyring;
this.clock = clock;
}
/**
* Returns the auth token for accessing the BSA API.
*
* <p>This method refreshes the token if it is expired, and is thread-safe..
*/
public String getAuthToken() {
try {
ensureAuthTokenValid();
} catch (IOException e) {
throw new BsaException(e, /* retriable= */ true);
} catch (GeneralSecurityException e) {
throw new BsaException(e, /* retriable= */ false);
}
return this.authToken;
}
private void ensureAuthTokenValid() throws IOException, GeneralSecurityException {
Instant now = Instant.ofEpochMilli(clock.nowUtc().getMillis());
if (authToken != null && lastRefreshTime.plus(authTokenExpiry).isAfter(now)) {
logger.atInfo().log("AuthToken still valid, reusing.");
return;
}
synchronized (this) {
authToken = fetchNewAuthToken();
lastRefreshTime = now;
logger.atInfo().log("AuthToken refreshed at %s.", now);
}
}
@VisibleForTesting
String fetchNewAuthToken() throws IOException, GeneralSecurityException {
String payload = String.format(AUTH_REQ_BODY_TEMPLATE, keyring.getBsaApiKey());
URL url = new URL(authUrl);
logger.atInfo().log("Fetching auth token from %s", url);
HttpsURLConnection connection = null;
try {
connection = (HttpsURLConnection) urlConnectionService.createConnection(url);
connection.setRequestMethod(HttpMethods.POST);
UrlConnectionUtils.setPayload(connection, payload.getBytes(UTF_8), CONTENT_TYPE);
int code = connection.getResponseCode();
if (code != SC_OK) {
String errorDetails;
try {
errorDetails = new String(getResponseBytes(connection), UTF_8);
} catch (Exception e) {
errorDetails = "Failed to retrieve error message: " + e.getMessage();
}
throw new BsaException(
String.format(
"Status code: [%s], error: [%s], details: [%s]",
code, connection.getResponseMessage(), errorDetails),
/* retriable= */ true);
}
// TODO: catch json syntax exception
@SuppressWarnings("unchecked")
String idToken =
new Gson()
.fromJson(new String(getResponseBytes(connection), UTF_8), Map.class)
.getOrDefault(ID_TOKEN, "")
.toString();
if (idToken.isEmpty()) {
throw new BsaException("Response missing ID token", /* retriable= */ false);
}
return idToken;
} finally {
if (connection != null) {
connection.disconnect();
}
}
}
}

View File

@@ -0,0 +1,34 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
public class BsaException extends RuntimeException {
private final boolean retriable;
public BsaException(Throwable cause, boolean retriable) {
super(cause);
this.retriable = retriable;
}
public BsaException(String message, boolean retriable) {
super(message);
this.retriable = retriable;
}
public boolean isRetriable() {
return this.retriable;
}
}

View File

@@ -0,0 +1,127 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
import static java.nio.charset.StandardCharsets.UTF_8;
import static javax.servlet.http.HttpServletResponse.SC_ACCEPTED;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import com.google.api.client.http.HttpMethods;
import com.google.common.flogger.FluentLogger;
import com.google.common.io.ByteStreams;
import com.google.common.net.MediaType;
import google.registry.config.RegistryConfig.Config;
import google.registry.request.UrlConnectionService;
import google.registry.request.UrlConnectionUtils;
import google.registry.util.Retrier;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.security.GeneralSecurityException;
import javax.inject.Inject;
import javax.net.ssl.HttpsURLConnection;
/**
* Sends order processing reports to BSA.
*
* <p>Senders are responsible for keeping payloads at reasonable sizes.
*/
public class BsaReportSender {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final MediaType CONTENT_TYPE = MediaType.JSON_UTF_8;
private final UrlConnectionService urlConnectionService;
private final BsaCredential credential;
private final String orderStatusUrl;
private final String addUnblockableDomainsUrl;
private final String removeUnblockableDomainsUrl;
private final Retrier retrier;
@Inject
BsaReportSender(
UrlConnectionService urlConnectionService,
BsaCredential credential,
@Config("bsaOrderStatusUrl") String orderStatusUrl,
@Config("bsaAddUnblockableDomainsUrl") String addUnblockableDomainsUrl,
@Config("bsaRemoveUnblockableDomainsUrl") String removeUnblockableDomainsUrl,
Retrier retrier) {
this.urlConnectionService = urlConnectionService;
this.credential = credential;
this.orderStatusUrl = orderStatusUrl;
this.addUnblockableDomainsUrl = addUnblockableDomainsUrl;
this.removeUnblockableDomainsUrl = removeUnblockableDomainsUrl;
this.retrier = retrier;
}
public void sendOrderStatusReport(String payload) {
retrier.callWithRetry(
() -> trySendData(this.orderStatusUrl, payload),
e -> e instanceof BsaException && ((BsaException) e).isRetriable());
}
public void addUnblockableDomainsUpdates(String payload) {
retrier.callWithRetry(
() -> trySendData(this.addUnblockableDomainsUrl, payload),
e -> e instanceof BsaException && ((BsaException) e).isRetriable());
}
public void removeUnblockableDomainsUpdates(String payload) {
retrier.callWithRetry(
() -> trySendData(this.removeUnblockableDomainsUrl, payload),
e -> e instanceof BsaException && ((BsaException) e).isRetriable());
}
Void trySendData(String urlString, String payload) {
try {
URL url = new URL(urlString);
HttpsURLConnection connection =
(HttpsURLConnection) urlConnectionService.createConnection(url);
connection.setRequestMethod(HttpMethods.POST);
connection.setRequestProperty("Authorization", "Bearer " + credential.getAuthToken());
UrlConnectionUtils.setPayload(connection, payload.getBytes(UTF_8), CONTENT_TYPE.toString());
int code = connection.getResponseCode();
if (code != SC_OK && code != SC_ACCEPTED) {
String errorDetails = "";
try (InputStream errorStream = connection.getErrorStream()) {
errorDetails = new String(ByteStreams.toByteArray(errorStream), UTF_8);
} catch (NullPointerException e) {
// No error message.
} catch (Exception e) {
errorDetails = "Failed to retrieve error message: " + e.getMessage();
}
// TODO(b/318404541): sanitize errorDetails to prevent log injection attack.
throw new BsaException(
String.format(
"Status code: [%s], error: [%s], details: [%s]",
code, connection.getResponseMessage(), errorDetails),
/* retriable= */ true);
}
try (InputStream errorStream = connection.getInputStream()) {
String responseMessage = new String(ByteStreams.toByteArray(errorStream), UTF_8);
logger.atInfo().log("Received response: [%s]", responseMessage);
} catch (Exception e) {
logger.atInfo().withCause(e).log("Failed to retrieve response message.");
}
return null;
} catch (IOException e) {
throw new BsaException(e, /* retriable= */ true);
} catch (GeneralSecurityException e) {
throw new BsaException(e, /* retriable= */ false);
}
}
}

View File

@@ -0,0 +1,91 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.Maps.newTreeMap;
import static com.google.common.collect.Multimaps.newListMultimap;
import static com.google.common.collect.Multimaps.toMultimap;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableMultimap;
import com.google.common.collect.Lists;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import google.registry.bsa.api.BlockOrder.OrderType;
import java.util.Locale;
import java.util.Optional;
import java.util.stream.Stream;
/** Helpers for generating {@link BlockOrder} and {@link UnblockableDomain} reports. */
public final class JsonSerializations {
private static final Gson GSON = new GsonBuilder().setPrettyPrinting().create();
private JsonSerializations() {}
public static Optional<String> toInProgressOrdersReport(Stream<BlockOrder> orders) {
ImmutableList<ImmutableMap<String, Object>> maps =
orders.map(JsonSerializations::asInProgressOrder).collect(toImmutableList());
if (maps.isEmpty()) {
return Optional.empty();
}
return Optional.of(GSON.toJson(maps));
}
public static Optional<String> toCompletedOrdersReport(Stream<BlockOrder> orders) {
ImmutableList<ImmutableMap<String, Object>> maps =
orders.map(JsonSerializations::asCompletedOrder).collect(toImmutableList());
if (maps.isEmpty()) {
return Optional.empty();
}
return Optional.of(GSON.toJson(maps));
}
public static Optional<String> toUnblockableDomainsReport(Stream<UnblockableDomain> domains) {
ImmutableMultimap<String, String> reasonToNames =
ImmutableMultimap.copyOf(
domains.collect(
toMultimap(
domain -> domain.reason().name().toLowerCase(Locale.ROOT),
UnblockableDomain::domainName,
() -> newListMultimap(newTreeMap(), Lists::newArrayList))));
if (reasonToNames.isEmpty()) {
return Optional.empty();
}
return Optional.of(GSON.toJson(reasonToNames.asMap()));
}
public static Optional<String> toUnblockableDomainsRemovalReport(Stream<String> domainNames) {
ImmutableList<String> domainsList = domainNames.collect(toImmutableList());
if (domainsList.isEmpty()) {
return Optional.empty();
}
return Optional.of(GSON.toJson(domainsList));
}
private static ImmutableMap<String, Object> asInProgressOrder(BlockOrder order) {
String status =
order.orderType().equals(OrderType.CREATE) ? "ActivationInProgress" : "ReleaseInProgress";
return ImmutableMap.of("blockOrderId", order.orderId(), "status", status);
}
private static ImmutableMap<String, Object> asCompletedOrder(BlockOrder order) {
String status = order.orderType().equals(OrderType.CREATE) ? "Active" : "Closed";
return ImmutableMap.of("blockOrderId", order.orderId(), "status", status);
}
}

View File

@@ -0,0 +1,58 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
import static google.registry.bsa.BsaStringUtils.DOMAIN_JOINER;
import static google.registry.bsa.BsaStringUtils.PROPERTY_JOINER;
import static google.registry.bsa.BsaStringUtils.PROPERTY_SPLITTER;
import com.google.auto.value.AutoValue;
import java.util.List;
/**
* A domain name whose second-level domain (SLD) matches a BSA label but is not blocked. It may be
* already registered, or on the TLD's reserve list.
*/
// TODO(1/15/2024): rename to UnblockableDomain.
@AutoValue
public abstract class UnblockableDomain {
abstract String domainName();
abstract Reason reason();
/** Reasons why a valid domain name cannot be blocked. */
public enum Reason {
REGISTERED,
RESERVED,
INVALID;
}
public String serialize() {
return PROPERTY_JOINER.join(domainName(), reason().name());
}
public static UnblockableDomain deserialize(String text) {
List<String> items = PROPERTY_SPLITTER.splitToList(text);
return of(items.get(0), Reason.valueOf(items.get(1)));
}
public static UnblockableDomain of(String domainName, Reason reason) {
return new AutoValue_UnblockableDomain(domainName, reason);
}
public static UnblockableDomain of(String label, String tld, Reason reason) {
return of(DOMAIN_JOINER.join(label, tld), reason);
}
}

View File

@@ -0,0 +1,99 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.api;
import static com.google.common.base.Verify.verify;
import static google.registry.bsa.BsaStringUtils.PROPERTY_JOINER;
import com.google.auto.value.AutoValue;
import com.google.auto.value.extension.memoized.Memoized;
import google.registry.bsa.BsaStringUtils;
import google.registry.bsa.api.UnblockableDomain.Reason;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
/** Change record of an {@link UnblockableDomain}. */
@AutoValue
public abstract class UnblockableDomainChange {
/**
* The text used in place of an empty {@link #newReason()} when an instance is serialized to
* string.
*
* <p>This value helps manual inspection of the change files, making it easier to `grep` for
* deletions in BSA reports.
*/
private static final String DELETE_REASON_PLACEHOLDER = "IS_DELETE";
abstract UnblockableDomain unblockable();
abstract Optional<Reason> newReason();
public String domainName() {
return unblockable().domainName();
}
@Memoized
public UnblockableDomain newValue() {
verify(newReason().isPresent(), "Removed unblockable does not have new value.");
return UnblockableDomain.of(unblockable().domainName(), newReason().get());
}
public boolean isAddOrChange() {
return newReason().isPresent();
}
public boolean isDelete() {
return !this.isAddOrChange();
}
public boolean isNew() {
return newReason().filter(unblockable().reason()::equals).isPresent();
}
public String serialize() {
return PROPERTY_JOINER.join(
unblockable().domainName(),
unblockable().reason(),
newReason().map(Reason::name).orElse(DELETE_REASON_PLACEHOLDER));
}
public static UnblockableDomainChange deserialize(String text) {
List<String> items = BsaStringUtils.PROPERTY_SPLITTER.splitToList(text);
return of(
UnblockableDomain.of(items.get(0), Reason.valueOf(items.get(1))),
Objects.equals(items.get(2), DELETE_REASON_PLACEHOLDER)
? Optional.empty()
: Optional.of(Reason.valueOf(items.get(2))));
}
public static UnblockableDomainChange ofNew(UnblockableDomain unblockable) {
return of(unblockable, Optional.of(unblockable.reason()));
}
public static UnblockableDomainChange ofDeleted(UnblockableDomain unblockable) {
return of(unblockable, Optional.empty());
}
public static UnblockableDomainChange ofChanged(UnblockableDomain unblockable, Reason newReason) {
return of(unblockable, Optional.of(newReason));
}
private static UnblockableDomainChange of(
UnblockableDomain unblockable, Optional<Reason> newReason) {
return new AutoValue_UnblockableDomainChange(unblockable, newReason);
}
}

View File

@@ -1,102 +0,0 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.persistence;
import com.google.common.base.Objects;
import google.registry.bsa.persistence.BsaDomainInUse.BsaDomainInUseId;
import google.registry.model.CreateAutoTimestamp;
import google.registry.persistence.VKey;
import java.io.Serializable;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.IdClass;
/** A domain matching a BSA label but is in use (registered or reserved), so cannot be blocked. */
@Entity
@IdClass(BsaDomainInUseId.class)
public class BsaDomainInUse {
@Id String label;
@Id String tld;
@Column(nullable = false)
@Enumerated(EnumType.STRING)
Reason reason;
/**
* Creation time of this record, which is the most recent time when the domain was detected to be
* in use wrt BSA. It may be during the processing of a download, or during some other job that
* refreshes the state.
*
* <p>This field is for information only.
*/
@SuppressWarnings("unused")
@Column(nullable = false)
CreateAutoTimestamp createTime = CreateAutoTimestamp.create(null);
// For Hibernate
BsaDomainInUse() {}
public BsaDomainInUse(String label, String tld, Reason reason) {
this.label = label;
this.tld = tld;
this.reason = reason;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof BsaDomainInUse)) {
return false;
}
BsaDomainInUse that = (BsaDomainInUse) o;
return Objects.equal(label, that.label)
&& Objects.equal(tld, that.tld)
&& reason == that.reason
&& Objects.equal(createTime, that.createTime);
}
@Override
public int hashCode() {
return Objects.hashCode(label, tld, reason, createTime);
}
enum Reason {
REGISTERED,
RESERVED;
}
static class BsaDomainInUseId implements Serializable {
private String label;
private String tld;
// For Hibernate
BsaDomainInUseId() {}
BsaDomainInUseId(String label, String tld) {
this.label = label;
this.tld = tld;
}
}
static VKey<BsaDomainInUse> vKey(String label, String tld) {
return VKey.create(BsaDomainInUse.class, new BsaDomainInUseId(label, tld));
}
}

View File

@@ -14,9 +14,11 @@
package google.registry.bsa.persistence;
import static google.registry.bsa.persistence.BsaDomainRefresh.Stage.MAKE_DIFF;
import static google.registry.bsa.RefreshStage.CHECK_FOR_CHANGES;
import static google.registry.bsa.RefreshStage.DONE;
import com.google.common.base.Objects;
import google.registry.bsa.RefreshStage;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.persistence.VKey;
@@ -37,7 +39,7 @@ import org.joda.time.DateTime;
* change status when the IDN tables change, and will be handled by a separate tool when it happens.
*/
@Entity
public class BsaDomainRefresh {
class BsaDomainRefresh {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@@ -51,7 +53,7 @@ public class BsaDomainRefresh {
@Column(nullable = false)
@Enumerated(EnumType.STRING)
Stage stage = MAKE_DIFF;
RefreshStage stage = CHECK_FOR_CHANGES;
BsaDomainRefresh() {}
@@ -67,21 +69,26 @@ public class BsaDomainRefresh {
* Returns the starting time of this job as a string, which can be used as folder name on GCS when
* storing download data.
*/
public String getJobName() {
return "refresh-" + getCreationTime().toString();
String getJobName() {
return getCreationTime().toString() + "-refresh";
}
public Stage getStage() {
boolean isDone() {
return java.util.Objects.equals(stage, DONE);
}
RefreshStage getStage() {
return this.stage;
}
BsaDomainRefresh setStage(Stage stage) {
this.stage = stage;
BsaDomainRefresh setStage(RefreshStage refreshStage) {
this.stage = refreshStage;
return this;
}
@SuppressWarnings("AmbiguousMethodReference")
VKey<BsaDomainRefresh> vKey() {
return vKey(this);
return vKey(jobId);
}
@Override
@@ -104,14 +111,7 @@ public class BsaDomainRefresh {
return Objects.hashCode(jobId, creationTime, updateTime, stage);
}
static VKey vKey(BsaDomainRefresh bsaDomainRefresh) {
return VKey.create(BsaDomainRefresh.class, bsaDomainRefresh.jobId);
}
enum Stage {
MAKE_DIFF,
APPLY_DIFF,
REPORT_REMOVALS,
REPORT_ADDITIONS;
static VKey<BsaDomainRefresh> vKey(long jobId) {
return VKey.create(BsaDomainRefresh.class, jobId);
}
}

View File

@@ -15,18 +15,20 @@
package google.registry.bsa.persistence;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static google.registry.bsa.DownloadStage.DOWNLOAD;
import static google.registry.bsa.DownloadStage.DONE;
import static google.registry.bsa.DownloadStage.DOWNLOAD_BLOCK_LISTS;
import com.google.common.base.Joiner;
import com.google.common.base.Objects;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSortedMap;
import google.registry.bsa.BlockList;
import google.registry.bsa.BlockListType;
import google.registry.bsa.DownloadStage;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.persistence.VKey;
import java.util.Locale;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
@@ -41,7 +43,7 @@ import org.joda.time.DateTime;
/** Records of ongoing and completed download jobs. */
@Entity
@Table(indexes = {@Index(columnList = "creationTime")})
public class BsaDownload {
class BsaDownload {
private static final Joiner CSV_JOINER = Joiner.on(',');
private static final Splitter CSV_SPLITTER = Splitter.on(',');
@@ -61,7 +63,7 @@ public class BsaDownload {
@Column(nullable = false)
@Enumerated(EnumType.STRING)
DownloadStage stage = DOWNLOAD;
DownloadStage stage = DOWNLOAD_BLOCK_LISTS;
BsaDownload() {}
@@ -74,14 +76,21 @@ public class BsaDownload {
}
/**
* Returns the starting time of this job as a string, which can be used as folder name on GCS when
* storing download data.
* Returns a unique name of the job.
*
* <p>The returned value should be a valid GCS folder name, consisting of only lower case
* alphanumerics, underscore, hyphen and dot.
*/
public String getJobName() {
return getCreationTime().toString();
String getJobName() {
// Return a value based on job start time, which is unique.
return getCreationTime().toString().toLowerCase(Locale.ROOT).replace(":", "");
}
public DownloadStage getStage() {
boolean isDone() {
return java.util.Objects.equals(stage, DONE);
}
DownloadStage getStage() {
return this.stage;
}
@@ -90,19 +99,20 @@ public class BsaDownload {
return this;
}
BsaDownload setChecksums(ImmutableMap<BlockList, String> checksums) {
BsaDownload setChecksums(ImmutableMap<BlockListType, String> checksums) {
blockListChecksums =
CSV_JOINER.withKeyValueSeparator("=").join(ImmutableSortedMap.copyOf(checksums));
return this;
}
ImmutableMap<BlockList, String> getChecksums() {
ImmutableMap<BlockListType, String> getChecksums() {
if (blockListChecksums.isEmpty()) {
return ImmutableMap.of();
}
return CSV_SPLITTER.withKeyValueSeparator('=').split(blockListChecksums).entrySet().stream()
.collect(
toImmutableMap(entry -> BlockList.valueOf(entry.getKey()), entry -> entry.getValue()));
toImmutableMap(
entry -> BlockListType.valueOf(entry.getKey()), entry -> entry.getValue()));
}
@Override

View File

@@ -44,7 +44,7 @@ final class BsaLabel {
DateTime creationTime;
// For Hibernate.
BsaLabel() {}
private BsaLabel() {}
BsaLabel(String label, DateTime creationTime) {
this.label = label;

View File

@@ -14,6 +14,9 @@
package google.registry.bsa.persistence;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.config.RegistryConfig.getEppResourceCachingDuration;
import static google.registry.config.RegistryConfig.getEppResourceMaxCachedEntries;
import static google.registry.model.CacheUtils.newCacheBuilder;
@@ -22,10 +25,15 @@ import static google.registry.persistence.transaction.TransactionManagerFactory.
import com.github.benmanes.caffeine.cache.CacheLoader;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableCollection;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import google.registry.persistence.VKey;
import java.time.Duration;
import java.util.Map;
import java.util.Optional;
import org.apache.commons.lang3.stream.Streams;
/** Helpers for {@link BsaLabel}. */
public final class BsaLabelUtils {
@@ -43,9 +51,11 @@ public final class BsaLabelUtils {
@Override
public Map<VKey<BsaLabel>, Optional<BsaLabel>> loadAll(
Iterable<? extends VKey<BsaLabel>> keys) {
// TODO(b/309173359): need this for DomainCheckFlow
throw new UnsupportedOperationException(
"LoadAll not supported by the BsaLabel cache loader.");
ImmutableMap<VKey<? extends BsaLabel>, BsaLabel> existingLabels =
replicaTm().reTransact(() -> replicaTm().loadByKeysIfPresent(keys));
return Streams.of(keys)
.collect(
toImmutableMap(key -> key, key -> Optional.ofNullable(existingLabels.get(key))));
}
};
@@ -84,4 +94,15 @@ public final class BsaLabelUtils {
public static boolean isLabelBlocked(String domainLabel) {
return cacheBsaLabels.get(BsaLabel.vKey(domainLabel)).isPresent();
}
/** Returns the elements in {@code domainLabels} that are blocked by BSA. */
public static ImmutableSet<String> getBlockedLabels(ImmutableCollection<String> domainLabels) {
ImmutableList<VKey<BsaLabel>> queriedLabels =
domainLabels.stream().map(BsaLabel::vKey).collect(toImmutableList());
return cacheBsaLabels.getAll(queriedLabels).values().stream()
.filter(Optional::isPresent)
.map(Optional::get)
.map(BsaLabel::getLabel)
.collect(toImmutableSet());
}
}

View File

@@ -0,0 +1,153 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.persistence;
import static com.google.common.base.Verify.verify;
import static google.registry.bsa.BsaStringUtils.DOMAIN_JOINER;
import static google.registry.bsa.BsaStringUtils.DOMAIN_SPLITTER;
import com.google.common.base.Objects;
import com.google.common.collect.ImmutableList;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.persistence.BsaUnblockableDomain.BsaUnblockableDomainId;
import google.registry.model.CreateAutoTimestamp;
import google.registry.persistence.VKey;
import java.io.Serializable;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.IdClass;
/** A domain matching a BSA label but is in use (registered or reserved), so cannot be blocked. */
@Entity
@IdClass(BsaUnblockableDomainId.class)
class BsaUnblockableDomain {
@Id String label;
@Id String tld;
@Column(nullable = false)
@Enumerated(EnumType.STRING)
Reason reason;
/**
* Creation time of this record, which is the most recent time when the domain was detected to be
* in use wrt BSA. It may be during the processing of a download, or during some other job that
* refreshes the state.
*
* <p>This field is for information only.
*/
@SuppressWarnings("unused")
@Column(nullable = false)
CreateAutoTimestamp createTime = CreateAutoTimestamp.create(null);
// For Hibernate
BsaUnblockableDomain() {}
BsaUnblockableDomain(String label, String tld, Reason reason) {
this.label = label;
this.tld = tld;
this.reason = reason;
}
String domainName() {
return DOMAIN_JOINER.join(label, tld);
}
/**
* Returns the equivalent {@link UnblockableDomain} instance, for use by communication with the
* BSA API.
*/
UnblockableDomain toUnblockableDomain() {
return UnblockableDomain.of(label, tld, UnblockableDomain.Reason.valueOf(reason.name()));
}
VKey<BsaUnblockableDomain> toVkey() {
return vKey(this.label, this.tld);
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof BsaUnblockableDomain)) {
return false;
}
BsaUnblockableDomain that = (BsaUnblockableDomain) o;
return Objects.equal(label, that.label)
&& Objects.equal(tld, that.tld)
&& reason == that.reason
&& Objects.equal(createTime, that.createTime);
}
@Override
public int hashCode() {
return Objects.hashCode(label, tld, reason, createTime);
}
static BsaUnblockableDomain of(String domainName, Reason reason) {
ImmutableList<String> parts = ImmutableList.copyOf(DOMAIN_SPLITTER.splitToList(domainName));
verify(parts.size() == 2, "Invalid domain name: %s", domainName);
return new BsaUnblockableDomain(parts.get(0), parts.get(1), reason);
}
static VKey<BsaUnblockableDomain> vKey(String domainName) {
ImmutableList<String> parts = ImmutableList.copyOf(DOMAIN_SPLITTER.splitToList(domainName));
verify(parts.size() == 2, "Invalid domain name: %s", domainName);
return vKey(parts.get(0), parts.get(1));
}
static VKey<BsaUnblockableDomain> vKey(String label, String tld) {
return VKey.create(BsaUnblockableDomain.class, new BsaUnblockableDomainId(label, tld));
}
enum Reason {
REGISTERED,
RESERVED;
}
static class BsaUnblockableDomainId implements Serializable {
private String label;
private String tld;
@SuppressWarnings("unused") // For Hibernate
BsaUnblockableDomainId() {}
BsaUnblockableDomainId(String label, String tld) {
this.label = label;
this.tld = tld;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof BsaUnblockableDomainId)) {
return false;
}
BsaUnblockableDomainId that = (BsaUnblockableDomainId) o;
return Objects.equal(label, that.label) && Objects.equal(tld, that.tld);
}
@Override
public int hashCode() {
return Objects.hashCode(label, tld);
}
}
}

View File

@@ -0,0 +1,258 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.persistence;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.bsa.ReservedDomainsUtils.getAllReservedNames;
import static google.registry.bsa.ReservedDomainsUtils.isReservedDomain;
import static google.registry.bsa.persistence.Queries.queryLivesDomains;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static java.util.stream.Collectors.groupingBy;
import com.google.common.collect.ImmutableCollection;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import com.google.common.collect.Sets.SetView;
import com.google.common.collect.Streams;
import google.registry.bsa.BsaStringUtils;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.api.UnblockableDomain.Reason;
import google.registry.bsa.api.UnblockableDomainChange;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import google.registry.util.BatchedStreams;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.stream.Stream;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
* Rechecks {@link BsaUnblockableDomain the registered/reserved domain names} in the database for
* changes.
*
* <p>A registered/reserved domain name may change status in the following cases:
*
* <ul>
* <li>A domain whose reason for being unblockable is `REGISTERED` will become blockable when the
* domain is deregistered.
* <li>A domain whose reason for being unblockable is `REGISTERED` will have its reason changed to
* `RESERVED` if the domain is also on the reserved list.
* <li>A domain whose reason for being unblockable is `RESERVED` will become blockable when the
* domain is removed from the reserve list.
* <li>A domain whose reason for being unblockable is `RESERVED` will have its reason changed to
* `REGISTERED` if the domain is also on the reserved list.
* <li>A blockable domain becomes unblockable when it is added to the reserve list.
* <li>A blockable domain becomes unblockable when it is registered (with admin override).
* </ul>
*
* <p>As a reminder, invalid domain names are not stored in the database. They change status only
* when IDNs change in the TLDs, which rarely happens, and will be handled by dedicated procedures.
*
* <p>Domain blockability changes must be reported to BSA as follows:
*
* <ul>
* <li>A blockable domain becoming unblockable: an addition
* <li>An unblockable domain becoming blockable: a removal
* <li>An unblockable domain with reason change: a removal followed by an insertion.
* </ul>
*
* <p>Since BSA has separate endpoints for receiving blockability changes, removals must be sent
* before additions.
*/
public final class DomainsRefresher {
private final DateTime prevRefreshStartTime;
private final int transactionBatchSize;
private final DateTime now;
public DomainsRefresher(
DateTime prevRefreshStartTime,
DateTime now,
Duration domainTxnMaxDuration,
int transactionBatchSize) {
this.prevRefreshStartTime = prevRefreshStartTime.minus(domainTxnMaxDuration);
this.now = now;
this.transactionBatchSize = transactionBatchSize;
}
public ImmutableList<UnblockableDomainChange> checkForBlockabilityChanges() {
ImmutableList<UnblockableDomainChange> downgrades = refreshStaleUnblockables();
ImmutableList<UnblockableDomainChange> upgrades = getNewUnblockables();
ImmutableSet<String> upgradedDomains =
upgrades.stream().map(UnblockableDomainChange::domainName).collect(toImmutableSet());
ImmutableList<UnblockableDomainChange> trueDowngrades =
downgrades.stream()
.filter(c -> !upgradedDomains.contains(c.domainName()))
.collect(toImmutableList());
return new ImmutableList.Builder<UnblockableDomainChange>()
.addAll(upgrades)
.addAll(trueDowngrades)
.build();
}
/**
* Returns all changes to unblockable domains that have been reported to BSA. Please see {@link
* UnblockableDomainChange} for types of possible changes. Note that invalid domain names are not
* covered by this class and will be handled separately.
*
* <p>The number of changes are expected to be small for now. It is limited by the number of
* domain deregistrations and the number of names added or removed from the reserved lists since
* the previous refresh.
*/
public ImmutableList<UnblockableDomainChange> refreshStaleUnblockables() {
ImmutableList.Builder<UnblockableDomainChange> changes = new ImmutableList.Builder<>();
ImmutableList<BsaUnblockableDomain> batch;
Optional<BsaUnblockableDomain> lastRead = Optional.empty();
do {
batch = Queries.batchReadUnblockables(lastRead, transactionBatchSize);
if (!batch.isEmpty()) {
lastRead = Optional.of(batch.get(batch.size() - 1));
changes.addAll(recheckStaleDomainsBatch(batch));
}
} while (batch.size() == transactionBatchSize);
return changes.build();
}
ImmutableSet<UnblockableDomainChange> recheckStaleDomainsBatch(
ImmutableList<BsaUnblockableDomain> domains) {
ImmutableMap<String, BsaUnblockableDomain> nameToEntity =
domains.stream().collect(toImmutableMap(BsaUnblockableDomain::domainName, d -> d));
ImmutableSet<String> prevRegistered =
domains.stream()
.filter(d -> d.reason.equals(BsaUnblockableDomain.Reason.REGISTERED))
.map(BsaUnblockableDomain::domainName)
.collect(toImmutableSet());
ImmutableSet<String> currRegistered =
ImmutableSet.copyOf(
ForeignKeyUtils.load(Domain.class, nameToEntity.keySet(), now).keySet());
SetView<String> noLongerRegistered = Sets.difference(prevRegistered, currRegistered);
SetView<String> newlyRegistered = Sets.difference(currRegistered, prevRegistered);
ImmutableSet<String> prevReserved =
domains.stream()
.filter(d -> d.reason.equals(BsaUnblockableDomain.Reason.RESERVED))
.map(BsaUnblockableDomain::domainName)
.collect(toImmutableSet());
ImmutableSet<String> currReserved =
nameToEntity.keySet().stream()
.filter(domain -> isReservedDomain(domain, now))
.collect(toImmutableSet());
SetView<String> noLongerReserved = Sets.difference(prevReserved, currReserved);
ImmutableSet.Builder<UnblockableDomainChange> changes = new ImmutableSet.Builder<>();
// Newly registered: reserved -> registered
for (String domainName : newlyRegistered) {
BsaUnblockableDomain domain = nameToEntity.get(domainName);
UnblockableDomain unblockable =
UnblockableDomain.of(domain.label, domain.tld, Reason.valueOf(domain.reason.name()));
changes.add(UnblockableDomainChange.ofChanged(unblockable, Reason.REGISTERED));
}
// No longer registered: registered -> reserved/NONE
for (String domainName : noLongerRegistered) {
BsaUnblockableDomain domain = nameToEntity.get(domainName);
UnblockableDomain unblockable =
UnblockableDomain.of(domain.label, domain.tld, Reason.valueOf(domain.reason.name()));
changes.add(
currReserved.contains(domainName)
? UnblockableDomainChange.ofChanged(unblockable, Reason.RESERVED)
: UnblockableDomainChange.ofDeleted(unblockable));
}
// No longer reserved: reserved -> registered/None (the former duplicates with newly-registered)
for (String domainName : noLongerReserved) {
BsaUnblockableDomain domain = nameToEntity.get(domainName);
UnblockableDomain unblockable =
UnblockableDomain.of(domain.label, domain.tld, Reason.valueOf(domain.reason.name()));
if (!currRegistered.contains(domainName)) {
changes.add(UnblockableDomainChange.ofDeleted(unblockable));
}
}
return changes.build();
}
public ImmutableList<UnblockableDomainChange> getNewUnblockables() {
ImmutableSet<String> newCreated = getNewlyCreatedUnblockables(prevRefreshStartTime, now);
ImmutableSet<String> newReserved = getNewlyReservedUnblockables(now, transactionBatchSize);
SetView<String> reservedNotCreated = Sets.difference(newReserved, newCreated);
return Streams.concat(
newCreated.stream()
.map(name -> UnblockableDomain.of(name, Reason.REGISTERED))
.map(UnblockableDomainChange::ofNew),
reservedNotCreated.stream()
.map(name -> UnblockableDomain.of(name, Reason.RESERVED))
.map(UnblockableDomainChange::ofNew))
.collect(toImmutableList());
}
static ImmutableSet<String> getNewlyCreatedUnblockables(
DateTime prevRefreshStartTime, DateTime now) {
ImmutableSet<String> liveDomains = queryLivesDomains(prevRefreshStartTime, now);
return getUnblockedDomainNames(liveDomains);
}
static ImmutableSet<String> getNewlyReservedUnblockables(DateTime now, int batchSize) {
Stream<String> allReserved = getAllReservedNames(now);
return BatchedStreams.toBatches(allReserved, batchSize)
.map(DomainsRefresher::getUnblockedDomainNames)
.flatMap(ImmutableSet::stream)
.collect(toImmutableSet());
}
static ImmutableSet<String> getUnblockedDomainNames(ImmutableCollection<String> domainNames) {
Map<String, List<String>> labelToNames =
domainNames.stream().collect(groupingBy(BsaStringUtils::getLabelInDomain));
ImmutableSet<String> bsaLabels =
Queries.queryBsaLabelByLabels(ImmutableSet.copyOf(labelToNames.keySet()))
.map(BsaLabel::getLabel)
.collect(toImmutableSet());
return labelToNames.entrySet().stream()
.filter(entry -> !bsaLabels.contains(entry.getKey()))
.map(Entry::getValue)
.flatMap(List::stream)
.collect(toImmutableSet());
}
public void applyUnblockableChanges(ImmutableList<UnblockableDomainChange> changes) {
ImmutableMap<String, ImmutableSet<UnblockableDomainChange>> changesByType =
ImmutableMap.copyOf(
changes.stream()
.collect(
groupingBy(
change -> change.isDelete() ? "remove" : "change", toImmutableSet())));
tm().transact(
() -> {
if (changesByType.containsKey("remove")) {
tm().delete(
changesByType.get("remove").stream()
.map(c -> BsaUnblockableDomain.vKey(c.domainName()))
.collect(toImmutableSet()));
}
if (changesByType.containsKey("change")) {
tm().putAll(
changesByType.get("change").stream()
.map(UnblockableDomainChange::newValue)
.collect(toImmutableSet()));
}
});
}
}

View File

@@ -14,11 +14,19 @@
package google.registry.bsa.persistence;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Verify.verify;
import static google.registry.bsa.DownloadStage.CHECKSUMS_DO_NOT_MATCH;
import static google.registry.bsa.DownloadStage.MAKE_ORDER_AND_LABEL_DIFF;
import static google.registry.bsa.DownloadStage.NOP;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.auto.value.AutoValue;
import com.google.common.collect.ImmutableMap;
import google.registry.bsa.BlockList;
import google.registry.bsa.BlockListType;
import google.registry.bsa.DownloadStage;
import java.util.Optional;
import org.joda.time.DateTime;
/** Information needed when handling a download from BSA. */
@AutoValue
@@ -26,6 +34,8 @@ public abstract class DownloadSchedule {
abstract long jobId();
abstract DateTime jobCreationTime();
public abstract String jobName();
public abstract DownloadStage stage();
@@ -37,11 +47,55 @@ public abstract class DownloadSchedule {
* Returns true if download should be processed even if the checksums show that it has not changed
* from the previous one.
*/
abstract boolean alwaysDownload();
public abstract boolean alwaysDownload();
/** Updates the current job to the new stage. */
public void updateJobStage(DownloadStage stage) {
tm().transact(
() -> {
BsaDownload bsaDownload = tm().loadByKey(BsaDownload.vKey(jobId()));
verify(
stage.compareTo(bsaDownload.getStage()) > 0,
"Invalid new stage [%s]. Must move forward from [%s]",
bsaDownload.getStage(),
stage);
bsaDownload.setStage(stage);
tm().put(bsaDownload);
});
}
/**
* Updates the current job to the new stage and sets the checksums of the downloaded files.
*
* <p>This method may only be invoked during the {@code DOWNLOAD} stage, and the target stage must
* be one of {@code MAKE_DIFF}, {@code CHECK_FOR_STALE_UNBLOCKABLES}, {@code NOP}, or {@code
* CHECKSUMS_NOT_MATCH}.
*/
public DownloadSchedule updateJobStage(
DownloadStage stage, ImmutableMap<BlockListType, String> checksums) {
checkArgument(
stage.equals(MAKE_ORDER_AND_LABEL_DIFF)
|| stage.equals(NOP)
|| stage.equals(CHECKSUMS_DO_NOT_MATCH),
"Invalid stage [%s]",
stage);
return tm().transact(
() -> {
BsaDownload bsaDownload = tm().loadByKey(BsaDownload.vKey(jobId()));
verify(
bsaDownload.getStage().equals(DownloadStage.DOWNLOAD_BLOCK_LISTS),
"Invalid invocation. May only invoke during the DOWNLOAD stage.");
bsaDownload.setStage(stage);
bsaDownload.setChecksums(checksums);
tm().put(bsaDownload);
return of(bsaDownload);
});
}
static DownloadSchedule of(BsaDownload currentJob) {
return new AutoValue_DownloadSchedule(
currentJob.getJobId(),
currentJob.getCreationTime(),
currentJob.getJobName(),
currentJob.getStage(),
Optional.empty(),
@@ -52,6 +106,7 @@ public abstract class DownloadSchedule {
BsaDownload currentJob, CompletedJob latestCompleted, boolean alwaysDownload) {
return new AutoValue_DownloadSchedule(
currentJob.getJobId(),
currentJob.getCreationTime(),
currentJob.getJobName(),
currentJob.getStage(),
Optional.of(latestCompleted),
@@ -63,7 +118,7 @@ public abstract class DownloadSchedule {
public abstract static class CompletedJob {
public abstract String jobName();
public abstract ImmutableMap<BlockList, String> checksums();
public abstract ImmutableMap<BlockListType, String> checksums();
static CompletedJob of(BsaDownload completedJob) {
return new AutoValue_DownloadSchedule_CompletedJob(

View File

@@ -15,18 +15,22 @@
package google.registry.bsa.persistence;
import static com.google.common.base.Verify.verify;
import static google.registry.bsa.DownloadStage.CHECKSUMS_NOT_MATCH;
import static google.registry.bsa.DownloadStage.CHECKSUMS_DO_NOT_MATCH;
import static google.registry.bsa.DownloadStage.DONE;
import static google.registry.bsa.DownloadStage.NOP;
import static google.registry.bsa.persistence.RefreshScheduler.fetchMostRecentRefresh;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static org.joda.time.Duration.standardSeconds;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import google.registry.bsa.persistence.DownloadSchedule.CompletedJob;
import google.registry.config.RegistryConfig.Config;
import google.registry.util.Clock;
import java.util.Objects;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
import org.joda.time.Duration;
/**
@@ -61,7 +65,10 @@ public final class DownloadScheduler {
private final Clock clock;
@Inject
DownloadScheduler(Duration downloadInterval, Duration maxNopInterval, Clock clock) {
DownloadScheduler(
@Config("bsaDownloadInterval") Duration downloadInterval,
@Config("bsaMaxNopInterval") Duration maxNopInterval,
Clock clock) {
this.downloadInterval = downloadInterval;
this.maxNopInterval = maxNopInterval;
this.clock = clock;
@@ -71,26 +78,33 @@ public final class DownloadScheduler {
* Returns a {@link DownloadSchedule} instance that describes the work to be performed by an
* invocation of the download action, if applicable; or {@link Optional#empty} when there is
* nothing to do.
*
* <p>For an interrupted job, work will resume from the {@link DownloadSchedule#stage}.
*/
public Optional<DownloadSchedule> schedule() {
return tm().transact(
() -> {
ImmutableList<BsaDownload> recentJobs = loadRecentProcessedJobs();
if (recentJobs.isEmpty()) {
// No jobs initiated ever.
ImmutableList<BsaDownload> recentDownloads = fetchTwoMostRecentDownloads();
Optional<BsaDomainRefresh> mostRecentRefresh = fetchMostRecentRefresh();
if (mostRecentRefresh.isPresent() && !mostRecentRefresh.get().isDone()) {
// Ongoing refresh. Wait it out.
return Optional.empty();
}
if (recentDownloads.isEmpty()) {
// No downloads initiated ever.
return Optional.of(scheduleNewJob(Optional.empty()));
}
BsaDownload mostRecent = recentJobs.get(0);
BsaDownload mostRecent = recentDownloads.get(0);
if (mostRecent.getStage().equals(DONE)) {
return isTimeAgain(mostRecent, downloadInterval)
? Optional.of(scheduleNewJob(Optional.of(mostRecent)))
: Optional.empty();
} else if (recentJobs.size() == 1) {
} else if (recentDownloads.size() == 1) {
// First job ever, still in progress
return Optional.of(DownloadSchedule.of(recentJobs.get(0)));
return Optional.of(DownloadSchedule.of(recentDownloads.get(0)));
} else {
// Job in progress, with completed previous jobs.
BsaDownload prev = recentJobs.get(1);
BsaDownload prev = recentDownloads.get(1);
verify(prev.getStage().equals(DONE), "Unexpectedly found two ongoing jobs.");
return Optional.of(
DownloadSchedule.of(
@@ -101,6 +115,16 @@ public final class DownloadScheduler {
});
}
Optional<DateTime> latestCompletedJobTime() {
return tm().transact(
() -> {
return fetchTwoMostRecentDownloads().stream()
.filter(job -> Objects.equals(job.getStage(), DONE))
.map(BsaDownload::getCreationTime)
.findFirst();
});
}
private boolean isTimeAgain(BsaDownload mostRecent, Duration interval) {
return mostRecent.getCreationTime().plus(interval).minus(CRON_JITTER).isBefore(clock.nowUtc());
}
@@ -118,14 +142,25 @@ public final class DownloadScheduler {
.orElseGet(() -> DownloadSchedule.of(job));
}
/**
* Fetches up to two most recent downloads, ordered by time in descending order. The first one may
* be ongoing, and the second one (if exists) must be completed.
*
* <p>Jobs that do not download the data are ignored.
*/
@VisibleForTesting
ImmutableList<BsaDownload> loadRecentProcessedJobs() {
static ImmutableList<BsaDownload> fetchTwoMostRecentDownloads() {
return ImmutableList.copyOf(
tm().getEntityManager()
.createQuery(
"FROM BsaDownload WHERE stage NOT IN :nop_stages ORDER BY creationTime DESC")
.setParameter("nop_stages", ImmutableList.of(CHECKSUMS_NOT_MATCH, NOP))
"FROM BsaDownload WHERE stage NOT IN :nop_stages ORDER BY creationTime DESC",
BsaDownload.class)
.setParameter("nop_stages", ImmutableList.of(CHECKSUMS_DO_NOT_MATCH, NOP))
.setMaxResults(2)
.getResultList());
}
static Optional<BsaDownload> fetchMostRecentDownload() {
return fetchTwoMostRecentDownloads().stream().findFirst();
}
}

View File

@@ -0,0 +1,186 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.persistence;
import static com.google.common.base.Verify.verify;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Sets.difference;
import static google.registry.bsa.ReservedDomainsUtils.isReservedDomain;
import static google.registry.persistence.PersistenceModule.TransactionIsolationLevel.TRANSACTION_REPEATABLE_READ;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static java.util.stream.Collectors.groupingBy;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.flogger.FluentLogger;
import com.google.common.flogger.LazyArgs;
import google.registry.bsa.IdnChecker;
import google.registry.bsa.api.BlockLabel;
import google.registry.bsa.api.BlockLabel.LabelType;
import google.registry.bsa.api.UnblockableDomain;
import google.registry.bsa.api.UnblockableDomain.Reason;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import google.registry.model.tld.Tld;
import java.util.Map;
import java.util.stream.Stream;
import org.joda.time.DateTime;
/** Applies the BSA label diffs from the latest BSA download. */
public final class LabelDiffUpdates {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
private static final Joiner DOMAIN_JOINER = Joiner.on('.');
private LabelDiffUpdates() {}
/**
* Applies the label diffs to the database and collects matching domains that are in use
* (registered or reserved) for reporting.
*
* @return A collection of domains in use
*/
public static ImmutableList<UnblockableDomain> applyLabelDiff(
ImmutableList<BlockLabel> labels,
IdnChecker idnChecker,
DownloadSchedule schedule,
DateTime now) {
ImmutableList.Builder<UnblockableDomain> nonBlockedDomains = new ImmutableList.Builder<>();
ImmutableMap<LabelType, ImmutableList<BlockLabel>> labelsByType =
ImmutableMap.copyOf(
labels.stream().collect(groupingBy(BlockLabel::labelType, toImmutableList())));
tm().transact(
() -> {
for (Map.Entry<LabelType, ImmutableList<BlockLabel>> entry :
labelsByType.entrySet()) {
switch (entry.getKey()) {
case CREATE:
// With current Cloud SQL, label upsert throughput is about 200/second. If
// better performance is needed, consider bulk insert in native SQL.
tm().putAll(
entry.getValue().stream()
.filter(label -> isValidInAtLeastOneTld(label, idnChecker))
.map(
label ->
new BsaLabel(label.label(), schedule.jobCreationTime()))
.collect(toImmutableList()));
// May not find all unblockables due to race condition: DomainCreateFlow uses
// cached BsaLabels. Eventually will be consistent.
nonBlockedDomains.addAll(
tallyUnblockableDomainsForNewLabels(entry.getValue(), idnChecker, now));
break;
case DELETE:
ImmutableSet<String> deletedLabels =
entry.getValue().stream()
.filter(label -> isValidInAtLeastOneTld(label, idnChecker))
.map(BlockLabel::label)
.collect(toImmutableSet());
// Delete labels in DB. Also cascade-delete BsaUnblockableDomain.
int nDeleted = Queries.deleteBsaLabelByLabels(deletedLabels);
if (nDeleted != deletedLabels.size()) {
logger.atSevere().log(
"Only found %s entities among the %s labels: [%s]",
nDeleted, deletedLabels.size(), deletedLabels);
}
break;
case NEW_ORDER_ASSOCIATION:
ImmutableSet<String> affectedLabels =
entry.getValue().stream()
.filter(label -> isValidInAtLeastOneTld(label, idnChecker))
.map(BlockLabel::label)
.collect(toImmutableSet());
ImmutableSet<String> labelsInDb =
Queries.queryBsaLabelByLabels(affectedLabels)
.map(BsaLabel::getLabel)
.collect(toImmutableSet());
verify(
labelsInDb.size() == affectedLabels.size(),
"Missing labels in DB: %s",
LazyArgs.lazy(() -> difference(affectedLabels, labelsInDb)));
// Reuse registered and reserved names that are already computed.
Queries.queryBsaUnblockableDomainByLabels(affectedLabels)
.map(BsaUnblockableDomain::toUnblockableDomain)
.forEach(nonBlockedDomains::add);
for (BlockLabel label : entry.getValue()) {
getInvalidTldsForLabel(label, idnChecker)
.map(tld -> UnblockableDomain.of(label.label(), tld, Reason.INVALID))
.forEach(nonBlockedDomains::add);
}
break;
}
}
},
TRANSACTION_REPEATABLE_READ);
logger.atInfo().log("Processed %s of labels.", labels.size());
return nonBlockedDomains.build();
}
static ImmutableList<UnblockableDomain> tallyUnblockableDomainsForNewLabels(
ImmutableList<BlockLabel> labels, IdnChecker idnChecker, DateTime now) {
ImmutableList.Builder<UnblockableDomain> nonBlockedDomains = new ImmutableList.Builder<>();
for (BlockLabel label : labels) {
getInvalidTldsForLabel(label, idnChecker)
.map(tld -> UnblockableDomain.of(label.label(), tld, Reason.INVALID))
.forEach(nonBlockedDomains::add);
}
ImmutableSet<String> validDomainNames =
labels.stream()
.map(label -> validDomainNamesForLabel(label, idnChecker))
.flatMap(x -> x)
.collect(toImmutableSet());
ImmutableSet<String> registeredDomainNames =
ImmutableSet.copyOf(ForeignKeyUtils.load(Domain.class, validDomainNames, now).keySet());
for (String domain : registeredDomainNames) {
nonBlockedDomains.add(UnblockableDomain.of(domain, Reason.REGISTERED));
tm().put(BsaUnblockableDomain.of(domain, BsaUnblockableDomain.Reason.REGISTERED));
}
ImmutableSet<String> reservedDomainNames =
difference(validDomainNames, registeredDomainNames).stream()
.filter(domain -> isReservedDomain(domain, now))
.collect(toImmutableSet());
for (String domain : reservedDomainNames) {
nonBlockedDomains.add(UnblockableDomain.of(domain, Reason.RESERVED));
tm().put(BsaUnblockableDomain.of(domain, BsaUnblockableDomain.Reason.RESERVED));
}
return nonBlockedDomains.build();
}
static Stream<String> validDomainNamesForLabel(BlockLabel label, IdnChecker idnChecker) {
return getValidTldsForLabel(label, idnChecker)
.map(tld -> DOMAIN_JOINER.join(label.label(), tld));
}
static Stream<String> getInvalidTldsForLabel(BlockLabel label, IdnChecker idnChecker) {
return idnChecker.getForbiddingTlds(label.idnTables()).stream().map(Tld::getTldStr);
}
static Stream<String> getValidTldsForLabel(BlockLabel label, IdnChecker idnChecker) {
return idnChecker.getSupportingTlds(label.idnTables()).stream().map(Tld::getTldStr);
}
static boolean isValidInAtLeastOneTld(BlockLabel label, IdnChecker idnChecker) {
return getValidTldsForLabel(label, idnChecker).findAny().isPresent();
}
}

View File

@@ -0,0 +1,112 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.persistence;
import static com.google.common.base.Verify.verify;
import static google.registry.bsa.BsaStringUtils.DOMAIN_SPLITTER;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableCollection;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.ForeignKeyUtils;
import google.registry.model.domain.Domain;
import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.joda.time.DateTime;
/** Helpers for querying BSA JPA entities. */
class Queries {
private Queries() {}
private static Object detach(Object obj) {
tm().getEntityManager().detach(obj);
return obj;
}
static Stream<BsaUnblockableDomain> queryBsaUnblockableDomainByLabels(
ImmutableCollection<String> labels) {
return ((Stream<?>)
tm().getEntityManager()
.createQuery("FROM BsaUnblockableDomain WHERE label in (:labels)")
.setParameter("labels", labels)
.getResultStream())
.map(Queries::detach)
.map(BsaUnblockableDomain.class::cast);
}
static Stream<BsaLabel> queryBsaLabelByLabels(ImmutableCollection<String> labels) {
return ((Stream<?>)
tm().getEntityManager()
.createQuery("FROM BsaLabel where label in (:labels)")
.setParameter("labels", labels)
.getResultStream())
.map(Queries::detach)
.map(BsaLabel.class::cast);
}
static int deleteBsaLabelByLabels(ImmutableCollection<String> labels) {
return tm().getEntityManager()
.createQuery("DELETE FROM BsaLabel where label IN (:deleted_labels)")
.setParameter("deleted_labels", labels)
.executeUpdate();
}
static ImmutableList<BsaUnblockableDomain> batchReadUnblockables(
Optional<BsaUnblockableDomain> lastRead, int batchSize) {
return ImmutableList.copyOf(
tm().getEntityManager()
.createQuery(
"FROM BsaUnblockableDomain d WHERE d.label > :label OR (d.label = :label AND d.tld"
+ " > :tld) ORDER BY d.tld, d.label ")
.setParameter("label", lastRead.map(d -> d.label).orElse(""))
.setParameter("tld", lastRead.map(d -> d.tld).orElse(""))
.setMaxResults(batchSize)
.getResultList());
}
static ImmutableSet<String> queryUnblockablesByNames(ImmutableSet<String> domains) {
String labelTldParis =
domains.stream()
.map(
domain -> {
List<String> parts = DOMAIN_SPLITTER.splitToList(domain);
verify(parts.size() == 2, "Invalid domain name %s", domain);
return String.format("('%s','%s')", parts.get(0), parts.get(1));
})
.collect(Collectors.joining(","));
String sql =
String.format(
"SELECT CONCAT(d.label, '.', d.tld) FROM \"BsaUnblockableDomain\" d "
+ "WHERE (d.label, d.tld) IN (%s)",
labelTldParis);
return ImmutableSet.copyOf(tm().getEntityManager().createNativeQuery(sql).getResultList());
}
static ImmutableSet<String> queryLivesDomains(DateTime minCreationTime, DateTime now) {
ImmutableSet<String> candidates =
ImmutableSet.copyOf(
tm().getEntityManager()
.createQuery(
"SELECT domainName FROM Domain WHERE creationTime >= :time ", String.class)
.setParameter("time", CreateAutoTimestamp.create(minCreationTime))
.getResultList());
return ImmutableSet.copyOf(ForeignKeyUtils.load(Domain.class, candidates, now).keySet());
}
}

View File

@@ -0,0 +1,65 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.persistence;
import static com.google.common.base.Verify.verify;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.auto.value.AutoValue;
import com.google.errorprone.annotations.CanIgnoreReturnValue;
import google.registry.bsa.RefreshStage;
import org.joda.time.DateTime;
/** Information needed when handling a domain refresh. */
@AutoValue
public abstract class RefreshSchedule {
abstract long jobId();
abstract DateTime jobCreationTime();
public abstract String jobName();
public abstract RefreshStage stage();
/** The most recent job that ended in the {@code DONE} stage. */
public abstract DateTime prevRefreshTime();
/** Updates the current job to the new stage. */
@CanIgnoreReturnValue
public RefreshSchedule updateJobStage(RefreshStage stage) {
return tm().transact(
() -> {
BsaDomainRefresh bsaRefresh = tm().loadByKey(BsaDomainRefresh.vKey(jobId()));
verify(
stage.compareTo(bsaRefresh.getStage()) > 0,
"Invalid new stage [%s]. Must move forward from [%s]",
bsaRefresh.getStage(),
stage);
bsaRefresh.setStage(stage);
tm().put(bsaRefresh);
return of(bsaRefresh, prevRefreshTime());
});
}
static RefreshSchedule of(BsaDomainRefresh job, DateTime prevJobCreationTime) {
return new AutoValue_RefreshSchedule(
job.getJobId(),
job.getCreationTime(),
job.getJobName(),
job.getStage(),
prevJobCreationTime);
}
}

View File

@@ -0,0 +1,90 @@
// Copyright 2023 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.bsa.persistence;
import static google.registry.bsa.persistence.DownloadScheduler.fetchMostRecentDownload;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import java.util.Optional;
import javax.inject.Inject;
import org.joda.time.DateTime;
/** Assigns work for each cron invocation of domain refresh job. */
public class RefreshScheduler {
@Inject
RefreshScheduler() {}
public Optional<RefreshSchedule> schedule() {
return tm().transact(
() -> {
ImmutableList<BsaDomainRefresh> recentJobs = fetchMostRecentRefreshes();
Optional<BsaDownload> mostRecentDownload = fetchMostRecentDownload();
if (mostRecentDownload.isPresent() && !mostRecentDownload.get().isDone()) {
// Ongoing download exists. Must wait it out.
return Optional.empty();
}
if (recentJobs.size() > 1) {
BsaDomainRefresh mostRecent = recentJobs.get(0);
if (mostRecent.isDone()) {
return Optional.of(scheduleNewJob(mostRecent.getCreationTime()));
} else {
return Optional.of(
rescheduleOngoingJob(mostRecent, recentJobs.get(1).getCreationTime()));
}
}
if (recentJobs.size() == 1 && recentJobs.get(0).isDone()) {
return Optional.of(scheduleNewJob(recentJobs.get(0).getCreationTime()));
}
// No previously completed refreshes. Need start time of a completed download as
// lower bound of refresh checks.
if (!mostRecentDownload.isPresent()) {
return Optional.empty();
}
DateTime prevDownloadTime = mostRecentDownload.get().getCreationTime();
if (recentJobs.isEmpty()) {
return Optional.of(scheduleNewJob(prevDownloadTime));
} else {
return Optional.of(rescheduleOngoingJob(recentJobs.get(0), prevDownloadTime));
}
});
}
RefreshSchedule scheduleNewJob(DateTime prevRefreshTime) {
BsaDomainRefresh newJob = new BsaDomainRefresh();
tm().insert(newJob);
return RefreshSchedule.of(newJob, prevRefreshTime);
}
RefreshSchedule rescheduleOngoingJob(BsaDomainRefresh ongoingJob, DateTime prevJobStartTime) {
return RefreshSchedule.of(ongoingJob, prevJobStartTime);
}
@VisibleForTesting
static ImmutableList<BsaDomainRefresh> fetchMostRecentRefreshes() {
return ImmutableList.copyOf(
tm().getEntityManager()
.createQuery("FROM BsaDomainRefresh ORDER BY creationTime DESC", BsaDomainRefresh.class)
.setMaxResults(2)
.getResultList());
}
static Optional<BsaDomainRefresh> fetchMostRecentRefresh() {
return fetchMostRecentRefreshes().stream().findFirst();
}
}

View File

@@ -112,7 +112,6 @@ public class DelegatedCredentials extends GoogleCredentials {
* @param clock Used for setting token expiration times.
* @param tokenRefreshDelay The lifetime of each token. Should not exceed one hour according to
* GCP recommendations.
* @return
*/
static DelegatedCredentials createSelfSignedDelegatedCredential(
ServiceAccountSigner signer,
@@ -197,7 +196,9 @@ public class DelegatedCredentials extends GoogleCredentials {
String accessToken = validateString(responseData, "access_token", PARSE_ERROR_PREFIX);
int expiresInSeconds = validateInt32(responseData, "expires_in", PARSE_ERROR_PREFIX);
long expiresAtMilliseconds = clock.nowUtc().getMillis() + expiresInSeconds * 1000L;
return new AccessToken(accessToken, new Date(expiresAtMilliseconds));
@SuppressWarnings("JavaUtilDate")
AccessToken token = new AccessToken(accessToken, new Date(expiresAtMilliseconds));
return token;
}
String createAssertion(JsonFactory jsonFactory, long currentTime) throws IOException {

View File

@@ -33,9 +33,11 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import dagger.Module;
import dagger.Provides;
import google.registry.bsa.UploadBsaUnavailableDomainsAction;
import google.registry.dns.ReadDnsRefreshRequestsAction;
import google.registry.model.common.DnsRefreshRequest;
import google.registry.persistence.transaction.JpaTransactionManager;
import google.registry.util.RegistryEnvironment;
import google.registry.util.YamlUtils;
import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
@@ -251,7 +253,11 @@ public final class RegistryConfig {
return projectId + "-zonefiles";
}
/** @see RegistryConfig#getDatabaseRetention() */
/**
* Returns the length of time before commit logs should be deleted from the database.
*
* @see RegistryConfig#getDatabaseRetention()
*/
@Provides
@Config("databaseRetention")
public static Duration provideDatabaseRetention() {
@@ -269,6 +275,18 @@ public final class RegistryConfig {
return projectId + "-domain-lists";
}
/**
* The GCS bucket for exporting lists of unavailable names for the BSA.
*
* @see UploadBsaUnavailableDomainsAction
*/
@Provides
@Config("bsaUnavailableDomainsGcsBucket")
public static String provideBsaUnavailableNamesGcsBucket(
@Config("projectId") String projectId) {
return projectId + "-bsa-unavailable-domains";
}
/**
* Returns the Google Cloud Storage bucket for staging BRDA escrow deposits.
*
@@ -280,7 +298,11 @@ public final class RegistryConfig {
return projectId + "-icann-brda";
}
/** @see google.registry.rde.BrdaCopyAction */
/**
* Returns the day of the week on which BRDA deposits should be made.
*
* @see google.registry.rde.BrdaCopyAction
*/
@Provides
@Config("brdaDayOfWeek")
public static int provideBrdaDayOfWeek() {
@@ -1043,6 +1065,17 @@ public final class RegistryConfig {
return config.registryPolicy.whoisDisclaimer;
}
/**
* Message template for whois response when queried domain is blocked by BSA.
*
* @see google.registry.whois.WhoisResponse
*/
@Provides
@Config("domainBlockedByBsaTemplate")
public static String provideDomainBlockedByBsaTemplate(RegistryConfigSettings config) {
return config.registryPolicy.domainBlockedByBsaTemplate;
}
/**
* Maximum QPS for the Google Cloud Monitoring V3 (aka Stackdriver) API. The QPS limit can be
* adjusted by contacting Cloud Support.
@@ -1433,9 +1466,15 @@ public final class RegistryConfig {
}
@Provides
@Config("bsaLabelTxnBatchSize")
public static int provideBsaLabelTxnBatchSize(RegistryConfigSettings config) {
return config.bsa.bsaLabelTxnBatchSize;
@Config("bsaTxnBatchSize")
public static int provideBsaTxnBatchSize(RegistryConfigSettings config) {
return config.bsa.bsaTxnBatchSize;
}
@Provides
@Config("domainTxnMaxDuration")
public static Duration provideDomainTxnMaxDuration(RegistryConfigSettings config) {
return Duration.standardSeconds(config.bsa.domainTxnMaxDurationSeconds);
}
@Provides
@@ -1477,6 +1516,12 @@ public final class RegistryConfig {
return String.format("%s?%s", config.bsa.unblockableDomainsUrl, "action=remove");
}
@Provides
@Config("bsaUploadUnavailableDomainsUrl")
public static String provideBsaUploadUnavailableDomainsUrl(RegistryConfigSettings config) {
return config.bsa.uploadUnavailableDomainsUrl;
}
private static String formatComments(String text) {
return Splitter.on('\n').omitEmptyStrings().trimResults().splitToList(text).stream()
.map(s -> "# " + s)

View File

@@ -105,6 +105,7 @@ public class RegistryConfigSettings {
public String reservedTermsExportDisclaimer;
public String whoisRedactedEmailText;
public String whoisDisclaimer;
public String domainBlockedByBsaTemplate;
public String rdapTos;
public String rdapTosStaticUrl;
public String registryName;
@@ -271,11 +272,13 @@ public class RegistryConfigSettings {
public int bsaLockLeaseExpiryMinutes;
public int bsaDownloadIntervalMinutes;
public int bsaMaxNopIntervalHours;
public int bsaLabelTxnBatchSize;
public int bsaTxnBatchSize;
public int domainTxnMaxDurationSeconds;
public String authUrl;
public int authTokenExpirySeconds;
public Map<String, String> dataUrls;
public String orderStatusUrl;
public String unblockableDomainsUrl;
public String uploadUnavailableDomainsUrl;
}
}

View File

@@ -130,6 +130,12 @@ registryPolicy:
unlawful behavior. We reserve the right to restrict or deny your access to
the WHOIS database, and may modify these terms at any time.
# BSA blocked domain name template.
domainBlockedByBsaTemplate: |
Domain Name: %s
>>> This name is not available for registration.
>>> This name has been blocked by a GlobalBlock service.
# RDAP Terms of Service text displayed at the /rdap/help/tos endpoint.
rdapTos: >
By querying our Domain Database as part of the RDAP pilot program (RDAP
@@ -609,6 +615,22 @@ bulkPricingPackageMonitoring:
# Configurations for integration with Brand Safety Alliance (BSA) API
bsa:
# Algorithm for calculating block list checksums
bsaChecksumAlgorithm: SHA-256
# The time allotted to every BSA cron job.
bsaLockLeaseExpiryMinutes: 30
# Desired time between successive downloads.
bsaDownloadIntervalMinutes: 30
# Max time period during which downloads can be skipped because checksums have
# not changed from the previous one.
bsaMaxNopIntervalHours: 24
# A very lax upper bound of the time it takes to execute a transaction that
# mutates a domain. Please See `BsaRefreshAction` for use case.
domainTxnMaxDurationSeconds: 60
# Number of entities (labels and unblockable domains) to process in a single
# DB transaction.
bsaTxnBatchSize: 1000
# Http endpoint for acquiring Auth tokens.
authUrl: "https://"
# Auth token expiry.
@@ -621,3 +643,5 @@ bsa:
orderStatusUrl: "https://"
# Http endpoint for reporting changes in the set of unblockable domains.
unblockableDomainsUrl: "https://"
# API endpoint for uploading the list of unavailable domain names.
uploadUnavailableDomainsUrl: "https://"

View File

@@ -53,7 +53,7 @@ import javax.inject.Inject;
/**
* Action for fanning out cron tasks shared by TLD.
*
* <h3>Parameters Reference</h3>
* <h2>Parameters Reference</h2>
*
* <ul>
* <li>{@code endpoint} (Required) URL path of servlet to launch. This may contain pathargs.
@@ -68,7 +68,7 @@ import javax.inject.Inject;
* task.
* </ul>
*
* <h3>Patharg Reference</h3>
* <h2>Patharg Reference</h2>
*
* <p>The following values may be specified inside the "endpoint" param.
*

View File

@@ -14,7 +14,7 @@
package google.registry.dns;
import static google.registry.config.RegistryEnvironment.PRODUCTION;
import static google.registry.util.RegistryEnvironment.PRODUCTION;
import com.google.common.collect.ImmutableSet;
import com.google.monitoring.metrics.DistributionFitter;
@@ -24,7 +24,7 @@ import com.google.monitoring.metrics.FibonacciFitter;
import com.google.monitoring.metrics.IncrementableMetric;
import com.google.monitoring.metrics.LabelDescriptor;
import com.google.monitoring.metrics.MetricRegistryImpl;
import google.registry.config.RegistryEnvironment;
import google.registry.util.RegistryEnvironment;
import javax.inject.Inject;
import org.joda.time.Duration;
@@ -245,12 +245,13 @@ public class DnsMetrics {
if (batchSize > 0) {
normalizedProcessingTimePerCommitDist.record(
(double) processingDuration.getMillis() / batchSize,
tld, status.name(), dnsWriter);
processingDuration.getMillis() / (double) batchSize, tld, status.name(), dnsWriter);
normalizedProcessingTimePerItemDist.record(
(double) processingDuration.getMillis() / batchSize,
processingDuration.getMillis() / (double) batchSize,
batchSize,
tld, status.name(), dnsWriter);
tld,
status.name(),
dnsWriter);
}
totalBatchSizePerCommitDist.record(batchSize, tld, status.name(), dnsWriter);

View File

@@ -14,6 +14,7 @@
package google.registry.dns.writer;
import dagger.Binds;
import dagger.Module;
import dagger.Provides;
import dagger.multibindings.IntoMap;
@@ -23,14 +24,12 @@ import javax.inject.Named;
/** Dagger module that disables DNS updates. */
@Module
public final class VoidDnsWriterModule {
public abstract class VoidDnsWriterModule {
@Provides
@Binds
@IntoMap
@StringKey(VoidDnsWriter.NAME)
static DnsWriter provideWriter(VoidDnsWriter writer) {
return writer;
}
abstract DnsWriter provideWriter(VoidDnsWriter writer);
@Provides
@IntoSet

View File

@@ -363,8 +363,8 @@ public class CloudDnsWriter extends BaseDnsWriter {
* <p>This call should be used in conjunction with {@link #getResourceRecordsForDomains} in a
* get-and-set retry loop.
*
* <p>See {@link "<a href="https://cloud.google.com/dns/troubleshooting">Troubleshoot Cloud
* DNS</a>"} for a list of errors produced by the Google Cloud DNS API.
* <p>See <a href="https://cloud.google.com/dns/troubleshooting">Troubleshoot Cloud DNS</a> for a
* list of errors produced by the Google Cloud DNS API.
*
* @throws ZoneStateException if the operation could not be completely successfully because the
* records to delete do not exist, already exist or have been modified with different

View File

@@ -14,6 +14,7 @@
package google.registry.dns.writer.dnsupdate;
import dagger.Binds;
import dagger.Module;
import dagger.Provides;
import dagger.multibindings.IntoMap;
@@ -32,12 +33,10 @@ public abstract class DnsUpdateWriterModule {
return SocketFactory.getDefault();
}
@Provides
@Binds
@IntoMap
@StringKey(DnsUpdateWriter.NAME)
static DnsWriter provideWriter(DnsUpdateWriter writer) {
return writer;
}
abstract DnsWriter provideWriter(DnsUpdateWriter writer);
@Provides
@IntoSet

View File

@@ -138,4 +138,15 @@
</description>
<schedule>*/1 * * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/uploadBsaUnavailableNames]]></url>
<name>uploadBsaUnavailableNames</name>
<description>
This job uploads all unavailable domain names (those registered and
reserved) to the BSA.
</description>
<service>bsa</service>
<schedule>23 8,20 * * *</schedule>
</task>
</entries>

View File

@@ -13,12 +13,24 @@
<load-on-startup>1</load-on-startup>
</servlet>
<!-- Test action -->
<!-- Download action -->
<servlet-mapping>
<servlet-name>bsa-servlet</servlet-name>
<url-pattern>/_dr/task/bsaDownload</url-pattern>
</servlet-mapping>
<!-- Refresh action -->
<servlet-mapping>
<servlet-name>bsa-servlet</servlet-name>
<url-pattern>/_dr/task/bsaRefresh</url-pattern>
</servlet-mapping>
<!-- Upload unavailable domains to BSA action -->
<servlet-mapping>
<servlet-name>bsa-servlet</servlet-name>
<url-pattern>/_dr/task/uploadBsaUnavailableNames</url-pattern>
</servlet-mapping>
<!-- Security config -->
<security-constraint>
<web-resource-collection>

View File

@@ -273,4 +273,26 @@
</description>
<schedule>0 15 * * 1</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/bsaDownload]]></url>
<name>bsaDownload</name>
<service>bsa</service>
<description>
Downloads the BSA block list and processes the changes.
</description>
<!-- Runs every hour. -->
<schedule>0 * * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/uploadBsaUnavailableNames]]></url>
<name>uploadBsaUnavailableNames</name>
<description>
This job uploads all unavailable domain names (those registered and
reserved) to the BSA.
</description>
<service>bsa</service>
<schedule>23 8,20 * * *</schedule>
</task>
</entries>

View File

@@ -162,4 +162,26 @@
</description>
<schedule>0 15 * * 1</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/bsaDownload]]></url>
<name>bsaDownload</name>
<service>bsa</service>
<description>
Downloads the BSA block lists and processes the changes.
</description>
<!-- Runs every hour. -->
<schedule>0 * * * *</schedule>
</task>
<task>
<url><![CDATA[/_dr/task/uploadBsaUnavailableNames]]></url>
<name>uploadBsaUnavailableNames</name>
<description>
This job uploads all unavailable domain names (those registered and
reserved) to the BSA.
</description>
<service>bsa</service>
<schedule>23 8,20 * * *</schedule>
</task>
</entries>

View File

@@ -16,12 +16,14 @@ package google.registry.flows;
import static com.google.common.base.Strings.nullToEmpty;
import static com.google.common.net.HttpHeaders.ACCESS_CONTROL_ALLOW_ORIGIN;
import static google.registry.flows.domain.DomainFlowUtils.isBlockedByBsa;
import static google.registry.flows.domain.DomainFlowUtils.validateDomainName;
import static google.registry.flows.domain.DomainFlowUtils.validateDomainNameWithIdnTables;
import static google.registry.flows.domain.DomainFlowUtils.verifyNotInPredelegation;
import static google.registry.model.tld.label.ReservationType.getTypeOfHighestSeverity;
import static google.registry.model.tld.label.ReservedList.getReservationTypes;
import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.AVAILABLE;
import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.BSA_BLOCKED;
import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.REGISTERED;
import static google.registry.monitoring.whitebox.CheckApiMetric.Availability.RESERVED;
import static google.registry.monitoring.whitebox.CheckApiMetric.Status.INVALID_NAME;
@@ -30,6 +32,7 @@ import static google.registry.monitoring.whitebox.CheckApiMetric.Status.SUCCESS;
import static google.registry.monitoring.whitebox.CheckApiMetric.Status.UNKNOWN_ERROR;
import static google.registry.monitoring.whitebox.CheckApiMetric.Tier.PREMIUM;
import static google.registry.monitoring.whitebox.CheckApiMetric.Tier.STANDARD;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.pricing.PricingEngineProxy.isDomainPremium;
import static google.registry.util.DomainNameUtils.canonicalizeHostname;
import static org.json.simple.JSONValue.toJSONString;
@@ -55,7 +58,6 @@ import google.registry.request.Parameter;
import google.registry.request.RequestParameters;
import google.registry.request.Response;
import google.registry.request.auth.Auth;
import google.registry.util.Clock;
import java.util.Map;
import java.util.Optional;
import javax.inject.Inject;
@@ -79,7 +81,6 @@ public class CheckApiAction implements Runnable {
String domain;
@Inject Response response;
@Inject Clock clock;
@Inject CheckApiMetric.Builder metricBuilder;
@Inject CheckApiMetrics checkApiMetrics;
@@ -104,6 +105,20 @@ public class CheckApiAction implements Runnable {
private Map<String, Object> doCheck() {
String domainString;
InternetDomainName domainName;
try {
domainString = canonicalizeHostname(nullToEmpty(domain));
domainName = validateDomainName(domainString);
return tm().transact(() -> checkDomainName(domainName));
} catch (IllegalArgumentException | EppException e) {
metricBuilder.status(INVALID_NAME);
return fail("Must supply a valid domain name on an authoritative TLD");
}
}
private Map<String, Object> checkDomainName(InternetDomainName domainName) {
tm().assertInTransaction();
String domainString;
try {
domainString = canonicalizeHostname(nullToEmpty(domain));
domainName = validateDomainName(domainString);
@@ -115,7 +130,7 @@ public class CheckApiAction implements Runnable {
// Throws an EppException with a reasonable error message which will be sent back to caller.
validateDomainNameWithIdnTables(domainName);
DateTime now = clock.nowUtc();
DateTime now = tm().getTransactionTime();
Tld tld = Tld.get(domainName.parent().toString());
try {
verifyNotInPredelegation(tld, now);
@@ -126,13 +141,25 @@ public class CheckApiAction implements Runnable {
boolean isRegistered = checkExists(domainString, now);
Optional<String> reservedError = Optional.empty();
boolean isBsaBlocked = false;
boolean isReserved = false;
if (!isRegistered) {
reservedError = checkReserved(domainName);
isReserved = reservedError.isPresent();
}
Availability availability = isRegistered ? REGISTERED : (isReserved ? RESERVED : AVAILABLE);
String errorMsg = isRegistered ? "In use" : (isReserved ? reservedError.get() : null);
if (!isRegistered && !isReserved) {
isBsaBlocked = isBlockedByBsa(domainName.parts().get(0), tld, now);
}
Availability availability =
isRegistered
? REGISTERED
: (isReserved ? RESERVED : (isBsaBlocked ? BSA_BLOCKED : AVAILABLE));
String errorMsg =
isRegistered
? "In use"
: (isReserved
? reservedError.get()
: (isBsaBlocked ? "Blocked by a GlobalBlock service" : null));
ImmutableMap.Builder<String, Object> responseBuilder = new ImmutableMap.Builder<>();
metricBuilder.status(SUCCESS).availability(availability);

View File

@@ -14,15 +14,13 @@
package google.registry.flows;
import dagger.Binds;
import dagger.Module;
import dagger.Provides;
/** Dagger module for the server Trid provider. */
@Module
public class ServerTridProviderModule {
public abstract class ServerTridProviderModule {
@Provides
static ServerTridProvider provideServerTridProvider(ServerTridProviderImpl defaultProvider) {
return defaultProvider;
}
@Binds
abstract ServerTridProvider provideServerTridProvider(ServerTridProviderImpl defaultProvider);
}

View File

@@ -26,7 +26,6 @@ import com.google.common.net.InetAddresses;
import dagger.Module;
import dagger.Provides;
import google.registry.config.RegistryConfig.Config;
import google.registry.config.RegistryEnvironment;
import google.registry.flows.EppException.AuthenticationErrorException;
import google.registry.flows.certs.CertificateChecker;
import google.registry.flows.certs.CertificateChecker.InsecureCertificateException;
@@ -34,6 +33,7 @@ import google.registry.model.registrar.Registrar;
import google.registry.request.Header;
import google.registry.util.CidrAddressBlock;
import google.registry.util.ProxyHttpHeaders;
import google.registry.util.RegistryEnvironment;
import java.net.InetAddress;
import java.security.MessageDigest;
import java.util.Optional;

View File

@@ -42,9 +42,10 @@ import org.bouncycastle.openssl.jcajce.JcaMiscPEMGenerator;
import org.bouncycastle.util.io.pem.PemObjectGenerator;
import org.bouncycastle.util.io.pem.PemWriter;
import org.joda.time.DateTime;
import org.joda.time.DateTimeComparator;
import org.joda.time.Days;
/** An utility to check that a given certificate meets our requirements */
/** A utility to check that a given certificate meets our requirements */
public class CertificateChecker {
private final ImmutableSortedMap<DateTime, Integer> maxValidityLengthSchedule;
@@ -162,9 +163,9 @@ public class CertificateChecker {
// Check if currently in validity period
Date now = clock.nowUtc().toDate();
if (certificate.getNotAfter().before(now)) {
if (DateTimeComparator.getInstance().compare(certificate.getNotAfter(), now) < 0) {
violations.add(CertificateViolation.EXPIRED);
} else if (certificate.getNotBefore().after(now)) {
} else if (DateTimeComparator.getInstance().compare(certificate.getNotBefore(), now) > 0) {
violations.add(CertificateViolation.NOT_YET_VALID);
}
@@ -231,8 +232,8 @@ public class CertificateChecker {
X509Certificate certificate = getCertificate(certificateStr);
DateTime now = clock.nowUtc();
// expiration date is one day after lastValidDate
Date lastValidDate = certificate.getNotAfter();
if (lastValidDate.before(now.toDate())) {
DateTime lastValidDate = new DateTime(certificate.getNotAfter());
if (lastValidDate.isBefore(now)) {
return false;
}
/*
@@ -242,12 +243,11 @@ public class CertificateChecker {
* 2) client has received notification but the interval between now and
* lastExpiringNotificationSentDate is greater than expirationWarningIntervalDays.
*/
return !lastValidDate.after(now.plusDays(expirationWarningDays).toDate())
return !lastValidDate.isAfter(now.plusDays(expirationWarningDays))
&& (lastExpiringNotificationSentDate.equals(START_OF_TIME)
|| !lastExpiringNotificationSentDate
.plusDays(expirationWarningIntervalDays)
.toDate()
.after(now.toDate()));
.isAfter(now));
}
private String getViolationDisplayMessage(CertificateViolation certificateViolation) {

View File

@@ -18,6 +18,7 @@ import static com.google.common.base.Strings.emptyToNull;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static google.registry.bsa.persistence.BsaLabelUtils.getBlockedLabels;
import static google.registry.flows.FlowUtils.validateRegistrarIsLoggedIn;
import static google.registry.flows.ResourceFlowUtils.verifyTargetIdCount;
import static google.registry.flows.domain.DomainFlowUtils.checkAllowedAccessToTld;
@@ -34,6 +35,7 @@ import static google.registry.model.tld.Tld.TldState.START_DATE_SUNRISE;
import static google.registry.model.tld.label.ReservationType.getTypeOfHighestSeverity;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import com.google.common.collect.ImmutableCollection;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
@@ -83,9 +85,12 @@ import google.registry.model.tld.label.ReservationType;
import google.registry.persistence.VKey;
import google.registry.pricing.PricingEngineProxy;
import google.registry.util.Clock;
import java.util.Collection;
import java.util.HashSet;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Collectors;
import javax.inject.Inject;
import org.joda.time.DateTime;
@@ -177,6 +182,12 @@ public final class DomainCheckFlow implements TransactionalFlow {
.build());
ImmutableMap<String, VKey<Domain>> existingDomains =
ForeignKeyUtils.load(Domain.class, domainNames, now);
// Check block labels only when there are unregistered domains, since "In use" goes before
// "Blocked by BSA".
ImmutableSet<InternetDomainName> bsaBlockedDomainNames =
existingDomains.size() == parsedDomains.size()
? ImmutableSet.of()
: getBsaBlockedDomains(parsedDomains.values());
Optional<AllocationTokenExtension> allocationTokenExtension =
eppInput.getSingleExtension(AllocationTokenExtension.class);
Optional<AllocationTokenDomainCheckResults> tokenDomainCheckResults =
@@ -203,6 +214,7 @@ public final class DomainCheckFlow implements TransactionalFlow {
getMessageForCheck(
parsedDomains.get(domainName),
existingDomains,
bsaBlockedDomainNames,
domainCheckResults,
tldStates,
allocationToken);
@@ -234,6 +246,7 @@ public final class DomainCheckFlow implements TransactionalFlow {
private Optional<String> getMessageForCheck(
InternetDomainName domainName,
ImmutableMap<String, VKey<Domain>> existingDomains,
ImmutableSet<InternetDomainName> bsaBlockedDomains,
ImmutableMap<InternetDomainName, String> tokenCheckResults,
ImmutableMap<String, TldState> tldStates,
Optional<AllocationToken> allocationToken) {
@@ -251,7 +264,18 @@ public final class DomainCheckFlow implements TransactionalFlow {
}
}
}
return Optional.ofNullable(emptyToNull(tokenCheckResults.get(domainName)));
Optional<String> tokenResult =
Optional.ofNullable(emptyToNull(tokenCheckResults.get(domainName)));
if (tokenResult.isPresent()) {
return tokenResult;
}
if (bsaBlockedDomains.contains(domainName)) {
// TODO(weiminyu): extract to a constant for here and CheckApiAction.
// Excerpt from BSA's custom message. Max len 32 chars by EPP XML schema.
return Optional.of("Blocked by a GlobalBlock service");
} else {
return Optional.empty();
}
}
/** Handle the fee check extension. */
@@ -415,6 +439,21 @@ public final class DomainCheckFlow implements TransactionalFlow {
return availabilityCheckDomains;
}
static ImmutableSet<InternetDomainName> getBsaBlockedDomains(
ImmutableCollection<InternetDomainName> parsedDomains) {
Map<String, ImmutableList<InternetDomainName>> labelToDomainNames =
parsedDomains.stream()
.collect(
Collectors.groupingBy(
parsedDomain -> parsedDomain.parts().get(0), toImmutableList()));
ImmutableSet<String> blockedLabels =
getBlockedLabels(ImmutableList.copyOf(labelToDomainNames.keySet()));
labelToDomainNames.keySet().retainAll(blockedLabels);
return labelToDomainNames.values().stream()
.flatMap(Collection::stream)
.collect(toImmutableSet());
}
/** By server policy, fee check names must be listed in the availability check. */
static class OnlyCheckedNamesCanBeFeeCheckedException extends ParameterValuePolicyErrorException {
OnlyCheckedNamesCanBeFeeCheckedException() {

View File

@@ -264,16 +264,18 @@ public class DomainFlowUtils {
/**
* Verifies that the {@code domainLabel} is not blocked by any BSA block label for the given
* {@code tld} at the specified time.
*
* @throws DomainLabelBlockedByBsaException
*/
public static void verifyNotBlockedByBsa(String domainLabel, Tld tld, DateTime now)
throws DomainLabelBlockedByBsaException {
if (isEnrolledWithBsa(tld, now) && isLabelBlocked(domainLabel)) {
if (isBlockedByBsa(domainLabel, tld, now)) {
throw new DomainLabelBlockedByBsaException();
}
}
public static boolean isBlockedByBsa(String domainLabel, Tld tld, DateTime now) {
return isEnrolledWithBsa(tld, now) && isLabelBlocked(domainLabel);
}
/** Returns whether a given domain create request is for a valid anchor tenant. */
public static boolean isAnchorTenant(
InternetDomainName domainName,
@@ -515,7 +517,7 @@ public class DomainFlowUtils {
private static final ImmutableSet<ReservationType> RESERVED_TYPES =
ImmutableSet.of(RESERVED_FOR_SPECIFIC_USE, RESERVED_FOR_ANCHOR_TENANT, FULLY_BLOCKED);
static boolean isReserved(InternetDomainName domainName, boolean isSunrise) {
public static boolean isReserved(InternetDomainName domainName, boolean isSunrise) {
ImmutableSet<ReservationType> types = getReservationTypes(domainName);
return !Sets.intersection(types, RESERVED_TYPES).isEmpty()
|| !(isSunrise || intersection(TYPES_ALLOWED_FOR_CREATE_ONLY_IN_SUNRISE, types).isEmpty());

View File

@@ -104,9 +104,12 @@ public final class PgpHelper {
PGPPublicKey publicKey = lookupPublicKey(publics, query, want);
PGPPrivateKey privateKey;
try {
PGPSecretKey secret = verifyNotNull(privates.getSecretKey(publicKey.getKeyID()),
"Keyring missing private key associated with public key id: %x (query '%s')",
publicKey.getKeyID(), query);
PGPSecretKey secret =
verifyNotNull(
privates.getSecretKey(publicKey.getKeyID()),
"Keyring missing private key associated with public key id: %s (query %s)",
publicKey.getKeyID(),
query);
// We do not support putting a password on the private key so we're just going to
// put char[0] here.
privateKey = secret.extractPrivateKey(

View File

@@ -29,12 +29,12 @@ import com.google.common.collect.Iterators;
import com.google.common.flogger.FluentLogger;
import com.google.protobuf.Timestamp;
import google.registry.batch.CloudTasksUtils;
import google.registry.config.RegistryEnvironment;
import google.registry.request.Action;
import google.registry.request.Action.Service;
import google.registry.request.Parameter;
import google.registry.request.auth.Auth;
import google.registry.security.XsrfTokenManager;
import google.registry.util.RegistryEnvironment;
import java.time.Instant;
import java.util.Arrays;
import java.util.Iterator;

View File

@@ -349,7 +349,7 @@ public class EntityYamlUtils {
@Override
public TimedTransitionProperty<Money> deserialize(JsonParser jp, DeserializationContext context)
throws IOException {
SortedMap<String, LinkedHashMap> valueMap = jp.readValueAs(SortedMap.class);
SortedMap<String, LinkedHashMap<String, Object>> valueMap = jp.readValueAs(SortedMap.class);
return TimedTransitionProperty.fromValueMap(
valueMap.keySet().stream()
.collect(
@@ -359,7 +359,7 @@ public class EntityYamlUtils {
key ->
Money.of(
CurrencyUnit.of(valueMap.get(key).get("currency").toString()),
(double) valueMap.get(key).get("amount")))));
new BigDecimal(String.valueOf(valueMap.get(key).get("amount")))))));
}
}

View File

@@ -28,7 +28,6 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedMap;
import com.google.common.collect.Sets;
import com.google.common.collect.Streams;
import google.registry.config.RegistryEnvironment;
import google.registry.model.pricing.StaticPremiumListPricingEngine;
import google.registry.model.registrar.Registrar;
import google.registry.model.registrar.RegistrarAddress;
@@ -40,6 +39,7 @@ import google.registry.model.tld.label.PremiumList;
import google.registry.model.tld.label.PremiumListDao;
import google.registry.persistence.VKey;
import google.registry.util.CidrAddressBlock;
import google.registry.util.RegistryEnvironment;
import java.util.Collection;
import java.util.Optional;
import java.util.function.Function;

View File

@@ -57,6 +57,7 @@ public final class ResourceTransferUtils {
public static TransferResponse createTransferResponse(
EppResource eppResource, TransferData transferData) {
assertIsContactOrDomain(eppResource);
@SuppressWarnings("NonCanonicalType")
TransferResponse.Builder<? extends TransferResponse, ?> builder;
if (eppResource instanceof Contact) {
builder = new ContactTransferResponse.Builder().setContactId(eppResource.getForeignKey());

View File

@@ -120,7 +120,7 @@ public class DomainBase extends EppResource
* from (creationTime, deletionTime) there can only be one domain in the database with this name.
* However, there can be many domains with the same name and non-overlapping lifetimes.
*
* @invariant domainName == domainName.toLowerCase(Locale.ENGLISH)
* <p>Invariant: domainName == domainName.toLowerCase(Locale.ENGLISH)
*/
@Expose String domainName;

View File

@@ -147,7 +147,7 @@ public class Result extends ImmutableObject {
this.msgLang = "en"; // All of our messages are English.
}
/** @return true iff the response code is in the 1xxx category, representing success. */
/** Returns true iff the response code is in the 1xxx category, representing success. */
public boolean isSuccess() {
return code < 2000;
}

View File

@@ -122,6 +122,7 @@ public class HostBase extends EppResource {
@Deprecated
@Override
@SuppressWarnings("InlineMeSuggester")
public HostBase cloneProjectedAtTime(DateTime now) {
return this;
}

View File

@@ -22,6 +22,7 @@ import static com.google.common.base.Strings.nullToEmpty;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.ImmutableSortedSet.toImmutableSortedSet;
import static com.google.common.collect.Sets.immutableEnumSet;
import static com.google.common.collect.Streams.stream;
import static com.google.common.io.BaseEncoding.base64;
import static google.registry.config.RegistryConfig.getDefaultRegistrarWhoisServer;
import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
@@ -794,6 +795,24 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
}
}
// Making sure there's no registrar with the same ianaId already in the system
private static boolean isNotADuplicateIanaId(
Iterable<Registrar> registrars, Registrar newInstance) {
// Return early if newly build registrar is not type REAL or ianaId is
// reserved by ICANN - https://www.iana.org/assignments/registrar-ids/registrar-ids.xhtml
if (!Type.REAL.equals(newInstance.type)
|| ImmutableSet.of(1L, 8L).contains(newInstance.ianaIdentifier)) {
return true;
}
return stream(registrars)
.filter(registrar -> Type.REAL.equals(registrar.getType()))
.filter(registrar -> !Objects.equals(newInstance.registrarId, registrar.getRegistrarId()))
.noneMatch(
registrar ->
Objects.equals(newInstance.ianaIdentifier, registrar.getIanaIdentifier()));
}
public Builder setContactsRequireSyncing(boolean contactsRequireSyncing) {
getInstance().contactsRequireSyncing = contactsRequireSyncing;
return this;
@@ -873,7 +892,11 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
return this;
}
/** @throws IllegalArgumentException if provided passcode is not 5-digit numeric */
/**
* Set the phone passcode.
*
* @throws IllegalArgumentException if provided passcode is not 5-digit numeric
*/
public Builder setPhonePasscode(String phonePasscode) {
checkArgument(
phonePasscode == null || PHONE_PASSCODE_PATTERN.matcher(phonePasscode).matches(),
@@ -912,6 +935,15 @@ public class Registrar extends UpdateAutoTimestampEntity implements Buildable, J
"Supplied IANA ID is not valid for %s registrar type: %s",
getInstance().type, getInstance().ianaIdentifier));
// We do not allow creating Real registrars with IANA ID that's already in the system
// b/315007360 - for more details
checkArgument(
isNotADuplicateIanaId(loadAllCached(), getInstance()),
String.format(
"Rejected attempt to create a registrar with ianaId that's already in the system -"
+ " %s",
getInstance().ianaIdentifier));
// In order to grant access to real TLDs, the registrar must have a corresponding billing
// account ID for that TLD's billing currency.
ImmutableSet<String> nonBillableTlds =

View File

@@ -22,6 +22,7 @@ import static com.google.common.base.Strings.emptyToNull;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.collect.Maps.filterValues;
import static google.registry.model.CacheUtils.memoizeWithShortExpiration;
import static google.registry.model.tld.Tld.isEnrolledWithBsa;
import static google.registry.persistence.transaction.TransactionManagerFactory.tm;
import static google.registry.util.CollectionUtils.entriesToImmutableMap;
import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
@@ -38,6 +39,7 @@ import java.util.Optional;
import java.util.function.Supplier;
import java.util.stream.Stream;
import javax.persistence.EntityManager;
import org.joda.time.DateTime;
/** Utilities for finding and listing {@link Tld} entities. */
public final class Tlds {
@@ -151,4 +153,9 @@ public final class Tlds {
findTldForName(domainName).orElse(null),
"Domain name is not under a recognized TLD: %s", domainName.toString());
}
/** Returns true if at least one TLD is enrolled {@code now}. */
public static boolean hasActiveBsaEnrollment(DateTime now) {
return getTldEntitiesOfType(TldType.REAL).stream().anyMatch(tld -> isEnrolledWithBsa(tld, now));
}
}

View File

@@ -63,15 +63,13 @@ public final class ReservedList
extends BaseDomainLabelList<ReservationType, ReservedList.ReservedListEntry> {
/**
* Mapping from domain name to its reserved list info.
* Mapping from domain label to its reserved list info.
*
* <p>This field requires special treatment since we want to lazy load it. We have to remove it
* from the immutability contract so we can modify it after construction and we have to handle the
* database processing on our own so we can detach it after load.
*/
@Insignificant
@Transient
Map<String, ReservedListEntry> reservedListMap;
@Insignificant @Transient Map<String, ReservedListEntry> reservedListMap;
@Column(nullable = false)
boolean shouldPublish = true;
@@ -269,7 +267,7 @@ public final class ReservedList
}
/** Loads and returns the reserved lists with the given names, skipping those that don't exist. */
private static ImmutableSet<ReservedList> loadReservedLists(
public static ImmutableSet<ReservedList> loadReservedLists(
ImmutableSet<String> reservedListNames) {
return cache.getAll(reservedListNames).values().stream()
.filter(Optional::isPresent)

View File

@@ -19,9 +19,14 @@ import dagger.Component;
import dagger.Lazy;
import google.registry.config.CredentialModule;
import google.registry.config.RegistryConfig.ConfigModule;
import google.registry.keyring.KeyringModule;
import google.registry.keyring.secretmanager.SecretManagerKeyringModule;
import google.registry.module.bsa.BsaRequestComponent.BsaRequestComponentModule;
import google.registry.monitoring.whitebox.StackdriverModule;
import google.registry.persistence.PersistenceModule;
import google.registry.privileges.secretmanager.SecretManagerModule;
import google.registry.request.Modules.GsonModule;
import google.registry.request.Modules.UrlConnectionServiceModule;
import google.registry.request.Modules.UserServiceModule;
import google.registry.request.auth.AuthModule;
import google.registry.util.UtilsModule;
@@ -31,13 +36,18 @@ import javax.inject.Singleton;
@Component(
modules = {
AuthModule.class,
UtilsModule.class,
UserServiceModule.class,
GsonModule.class,
BsaRequestComponentModule.class,
ConfigModule.class,
StackdriverModule.class,
CredentialModule.class,
BsaRequestComponentModule.class
GsonModule.class,
PersistenceModule.class,
KeyringModule.class,
SecretManagerKeyringModule.class,
SecretManagerModule.class,
StackdriverModule.class,
UrlConnectionServiceModule.class,
UserServiceModule.class,
UtilsModule.class
})
interface BsaComponent {
BsaRequestHandler requestHandler();

View File

@@ -16,19 +16,23 @@ package google.registry.module.bsa;
import dagger.Module;
import dagger.Subcomponent;
import google.registry.bsa.PlaceholderAction;
import google.registry.bsa.BsaDownloadAction;
import google.registry.bsa.BsaRefreshAction;
import google.registry.bsa.UploadBsaUnavailableDomainsAction;
import google.registry.request.Modules.UrlConnectionServiceModule;
import google.registry.request.RequestComponentBuilder;
import google.registry.request.RequestModule;
import google.registry.request.RequestScope;
@RequestScope
@Subcomponent(
modules = {
RequestModule.class,
})
@Subcomponent(modules = {RequestModule.class, UrlConnectionServiceModule.class})
interface BsaRequestComponent {
PlaceholderAction bsaAction();
BsaDownloadAction bsaDownloadAction();
BsaRefreshAction bsaRefreshAction();
UploadBsaUnavailableDomainsAction uploadBsaUnavailableDomains();
@Subcomponent.Builder
abstract class Builder implements RequestComponentBuilder<BsaRequestComponent> {

View File

@@ -45,6 +45,7 @@ public abstract class CheckApiMetric {
public enum Availability {
RESERVED("reserved"),
REGISTERED("registered"),
BSA_BLOCKED("blocked"),
AVAILABLE("available");
private final String displayLabel;

View File

@@ -33,12 +33,12 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.google.common.flogger.StackSize;
import google.registry.config.RegistryEnvironment;
import google.registry.model.ImmutableObject;
import google.registry.persistence.JpaRetries;
import google.registry.persistence.PersistenceModule.TransactionIsolationLevel;
import google.registry.persistence.VKey;
import google.registry.util.Clock;
import google.registry.util.RegistryEnvironment;
import google.registry.util.Retrier;
import google.registry.util.SystemSleeper;
import java.io.Serializable;

View File

@@ -20,10 +20,10 @@ import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import com.google.appengine.api.utils.SystemProperty;
import com.google.appengine.api.utils.SystemProperty.Environment.Value;
import com.google.common.base.Suppliers;
import google.registry.config.RegistryEnvironment;
import google.registry.persistence.DaggerPersistenceComponent;
import google.registry.tools.RegistryToolEnvironment;
import google.registry.util.NonFinalForTesting;
import google.registry.util.RegistryEnvironment;
import java.util.function.Supplier;
/** Factory class to create {@link TransactionManager} instance. */

View File

@@ -14,8 +14,8 @@
package google.registry.pricing;
import dagger.Binds;
import dagger.Module;
import dagger.Provides;
import dagger.multibindings.IntoMap;
import dagger.multibindings.StringKey;
import google.registry.model.pricing.PremiumPricingEngine;
@@ -25,15 +25,14 @@ import google.registry.model.pricing.StaticPremiumListPricingEngine;
* Dagger module for injecting pricing engines.
*
* <p>To add a new pricing engine, create a new class that implements {@link PremiumPricingEngine},
* and add a module that provides an instance of {@link PremiumPricingEngine} with a unique
* {@link StringKey} annotation, and also <code>@Provides @IntoMap</code> annotations.
* and add a module that provides an instance of {@link PremiumPricingEngine} with a unique {@link
* StringKey} annotation, and also <code>@Provides @IntoMap</code> annotations.
*/
@Module
public class PricingModule {
public abstract class PricingModule {
@Provides @IntoMap
@Binds
@IntoMap
@StringKey(StaticPremiumListPricingEngine.NAME)
static PremiumPricingEngine provideStaticPremiumList(StaticPremiumListPricingEngine engine) {
return engine;
}
abstract PremiumPricingEngine provideStaticPremiumList(StaticPremiumListPricingEngine engine);
}

View File

@@ -195,7 +195,9 @@ final class RdapObjectClasses {
* <p>Not part of the spec, but seems convenient.
*/
private abstract static class RdapObjectBase extends ReplyPayloadBase {
@JsonableElement final ObjectClassName objectClassName;
@SuppressWarnings("unused")
@JsonableElement
final ObjectClassName objectClassName;
@JsonableElement abstract Optional<String> handle();
@JsonableElement abstract ImmutableList<PublicId> publicIds();

Some files were not shown because too many files have changed in this diff Show More