1
0
mirror of https://github.com/google/nomulus synced 2026-06-09 16:33:02 +00:00

Compare commits

...

35 Commits

Author SHA1 Message Date
Weimin Yu bd5299b2f2 Print more info during compatibility test (#440)
* Print more info during compatibility test

Prints a summary of the environments to be tested and their
corresponding version tags.
2020-01-10 15:52:27 -05:00
Shicong Huang 4eae10800d Add user types to support Array in Cloud SQL (#439)
* Add user types to support Array in Cloud SQL

* Remove duplicate null check

* Simplify class structure
2020-01-10 14:54:21 -05:00
Weimin Yu 533515817f Make Jacoco work with multiple test tasks (#437)
* Make Jacoco work with multiple test tasks

By default Jacoco only looks at execution data from
the 'test' task. This is a problem to 'core' which
has multiple test sets.
2020-01-09 16:32:33 -05:00
Weimin Yu d4b1048363 Upload Cloud Build schema-deploy config to GCS (#435)
* Upload Cloud Build schema-deploy config to GCS

Forgot to upload cloudbuild-schema-deploy.yaml to GCS.
2020-01-09 15:04:10 -05:00
Weimin Yu 2ffcfc4c5e Fix outdated comment (#433)
* Fix outdated comment

Removed reference to a testcontainer issues that may no longer
exist.
2020-01-08 16:02:04 -05:00
Weimin Yu b47e152ca5 Add script to run compatibility tests (#431)
A helper script to run server/schema compatibility tests from
a CI platform. It allows the caller to choose a system under
test (sql or nomulus) and to specify the deployed version
by environment (sandbox or production).
2020-01-08 11:34:09 -05:00
Michael Muller b523b3835f Implement changes suggested by testcontainer dev (#426)
* Implement changes suggested by testcontainer dev

See https://github.com/google/nomulus/issues/401
Specifically:
- Use getContainerIpAddress() instead of localhost to insulate us from
  off-machine docker usage.
- Remove shutdown hook to close the container, as testcontainers does this for
  us.
2020-01-08 10:41:36 -05:00
Ben McIlwain eb7d779bc8 Relax premium list existence check to allow Cloud SQL migration (#428)
* Relax premium list existence check to allow Cloud SQL migration

We need to be able to simultaneously update premium lists that already exist in
Datastore and create them in Cloud SQL (because they haven't been migrated over
yet). This temporarily relaxes the existence check for Cloud SQL so that
"updates" will work even when the list doesn't yet exist there.
2020-01-06 18:09:21 -05:00
Lai Jiang 8fb12d21a2 Use the correct certificate provider type (#427)
TESTED: tested in alpha. Previous I only  tested locally and missed this
typo.
2020-01-06 16:19:50 -05:00
sarahcaseybot 5d9bf64b96 Fix IcannReportingUploadAction to upload reports from the previous month (#425)
* Fix IcannReportingUploadAction to upload reports from the previous month

getFileName now sets the file name of the report to upload to use the month before cursor time.

IcannReportingUploadAction no longer uploads the MANIFEST.txt file since it is not required based on (https://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-07#page-9) and the previous implementation of this action did not upload it.

Deletes the ICANN_UPLOAD_MANIFEST cursor since it is no loner needed.

* Add the ICANN_UPLOAD_MANIFEST cursor back
2020-01-06 15:59:01 -05:00
Lai Jiang 76603812e3 Create kzip file for Kythe cross-referencing (#424)
This is set up per b/141716384.

TESTED=Tested on alpha and successfully uploaded the merged kzip file to
GCS.
2020-01-03 16:57:39 -05:00
Shicong Huang a28754dc5d Add dual read claims list (#413)
* Add dual read claims list

* Improve warning log and use longer duration for cache

* Extract the comparison logic to a method

* Move cache to DAO
2020-01-03 10:59:34 -05:00
Weimin Yu d6bf6f375e Fix flakiness in JodaMoneyConverterTest (#421)
* Fix flakiness in JodaMoneyConverterTest

JpdaMoneyConverterTest relies on Hibernate to deploy its schema.
This introduces an extra jdbc connection in the middle of a test
suite, and may break the connection count checks between tests made
by JpaTransactionManagerRule.

This change updated HibernateSchemaExporter to include Hibernate
proprietary mappings in META-INF/orm.xml.

This change also disabled Hibernate schema push for all tests,
and enabled sql statement logging.
2020-01-02 11:51:20 -05:00
Ben McIlwain 8f67c4b9cf Support premium list updating in Cloud SQL (#422)
* Support premium list updating in Cloud SQL

This also removes the requirement to specify --also_cloud_sql in nomulus premium
list tooling, instead always persisting to Cloud SQL. It removes a non-USD
premium label in the global test premium list (the Cloud SQL schema doesn't
support a mix of currency units in a single premium list). And it adds a method
to PremiumListDao to grab the most recent version of a given list.

* Merge branch 'master' into premium-lists-always-cloud-sql

* Revert test change

* Create new PremiumListUtils class and refactor out existing method

* Fix tests and update an existing premium price
2020-01-02 11:30:58 -05:00
gbrodman c17a5c489c Add unlock fields to RegistryLocks (#408)
* Add unlock fields to RegistryLocks

This will make it easier to reason around inter-connected registry lock
objects (like when we add dependent roids). It will make it easier to
answer the question of "Have all locks associated with this host/contact
roid been unlocked?", as well as the question of "Was the last lock
object associated with this domain unlocked?"

* Responses to CR

* Make the DAO API more specific

* whoops, undo rename
2019-12-30 14:34:06 -07:00
Lai Jiang dd0e3b7c24 Consolidate certificate supplier module (#410)
* Consolidate certificate supplier module

Both the proxy and the proxy needs certificate suppliers. The PR
consolidates the module that providings those bindings to a shared
module and switched the proxy to use that module. The prober currently
uses P12 file to store its certificates. I am debating keeping that
supplier ro converting them to PEM files for simplicity.

* Rename mode enum values to be more descriptive

* Update annotation names to be more descriptive
2019-12-23 13:09:47 -05:00
Weimin Yu 24d671c070 Separate JPA rules for unit and integration tests (#420)
* Separate JPA rules for unit and integration tests

Define two subclasses of JpaTransactionManagerRule, one for unit
tests and the other for integration tests. The difference is that
the former does not need nomulus schema and need not be included
in server/schema compatibility tests.

* Separate JPA rules for unit and integration tests

Define two subclasses of JpaTransactionManagerRule, one for unit
tests and the other for integration tests. The difference is that
the former does not need nomulus schema and need not be included
in server/schema compatibility tests.
2019-12-19 14:49:54 -05:00
Shicong Huang f76030f7f0 Add initial support to write/update reserved lists to Cloud SQL (#388)
* Add initial support to write reserved list to Cloud SQL

* Add support to update reserved list in Cloud SQL

* Fix wrong check when override is enabled in create command

* Add sql dependent tests to the suite

* Address comment

* Make invocation of super.execute more readable

* Add test to check upper case and non-puny code label

* Move ReservedListDao related classes to schema package

* Simplify a function name
2019-12-19 12:51:48 -05:00
Shicong Huang f301314d97 Move ClaimsListDao related classes to new package (#416) 2019-12-19 10:50:41 -05:00
gbrodman 25d43511f6 Add BCC capabilities to the Spec11 reports (#418)
* Add BCC capabilities to the Spec11 reports
2019-12-18 17:08:06 -05:00
Lai Jiang bfd61ef867 Fix another bug in the proxy (#419)
The promise should be set outside the try block because if we want
warning only, we still want the promise to be set even if the
clientCertificate.checkValidity() throws an error.
2019-12-18 16:24:23 -05:00
Weimin Yu ffe3eb1548 Remove the schema jar url override for tests (#415)
* Remove the schema jar url override for tests

Revert pull request 374: "Allow schema-loading from arbitrary url in tests".

Pull request 403 (f48e3933f5) is a more
general solution.

This reverts commit 68887d427f.
2019-12-17 11:24:20 -05:00
Weimin Yu 6aaf081489 Create a new app to hold GenerateSqlSchemaCommand (#409)
* Create a new app to hold GenerateSqlSchemaCommand

GenerateSqlSchemaCommand starts postgresql using testcontainer.
This makes junit etc a runtime dependency, allowing them to get
into release artifacts.

By moving this command to a separate tool, we can remove junit
etc as compile/runtime dependency.
2019-12-13 16:05:35 -05:00
Weimin Yu 1143b25391 Fix test on use of external schema jar (#412)
* Fix test on use of external schema jar

One test breaks if path to external schema is set and
test is run from the sqlIntegrationTest task.
2019-12-13 11:32:19 -05:00
Weimin Yu 65cf49f204 Fix sql script name conflict (#411)
* Fix sql script name conflict

There are two V11__ files due to concurrent merge. Renamed one
to V12__

Also removed a @NotNull annotation, which is the fist in the code base.
Most of the code base use @Nullable instead. If we do want to use
@NotNull, we may want to use the javax one instead.
2019-12-12 16:16:43 -05:00
Weimin Yu f48e3933f5 Run cross-release SQL integration tests (#403)
* Run cross-release SQL integration tests

Run SQL integration tests across arbitrary schema and server
releases.

Refer to integration/README.md in this change for more information.

TESTED=Cloud build changes tested with cloud-build-local
       Used the published jars to test sqlIntegration task locally.
2019-12-12 13:47:49 -05:00
Lai Jiang 9853f23d94 Fix null pointer excpetion bug (#407)
The factory method passes a null trustedCertificates instead of an empty
list.
2019-12-12 13:06:43 -05:00
Ben McIlwain db7fcf6c38 Add Cloud SQL premium list caches and compare prices with Datastore (#376)
* Add Cloud SQL premium list caches and compare prices with Datastore

Nothing will fail if the prices can't be loaded from Cloud SQL, or if the prices
are different. All that happens is that the error is logged. Then, once this is
running in production for awhile, we'll look at the logs and see if there will
be any pricing implications from switching over to the Cloud SQL version of the
premium lists.

* Add setMaxResults(1) per code review

* Add tests and reorder public functions

* Don't statically import caches

* Improve test pass rate

* Merge branch 'master' into dual-read-premium

* Add PremiumEntry mapping

* Allow update

* Revert column order

* Alphabetize PremiumEntry columns

* Don't bother trying to enforce order

* Private constructor
2019-12-11 16:20:19 -05:00
Weimin Yu 3aad8b6aa7 Use TextDiffSubject to compare multi-line text (#406)
* Use TextDiffSubject to compare  multi-line text

It illustrates differences better.

Moved TextDiffSubject.java to the common project for sharing.
2019-12-10 16:44:59 -05:00
Weimin Yu f7b243e390 Make devProject a project property (#405)
* Make devProject a project property

Properties set in rootProject's ext block are not overridden by
commandline flags.

* Make flyway commands runnable from FOSS repo

rootProject.devProject, defined in projects.gradle, cannot be
overridden by commandline flags. Added a flag check in :db's
project script.
2019-12-10 11:39:56 -05:00
Michael Muller e8745f7538 Update lockfiles (#402)
* Update lockfiles

Update lockfiles for the latest dependencies and metadata.
2019-12-10 09:28:44 -05:00
sarahcaseybot 2478a4a93b Add schema and DAO for cursors in cloudsql (#370)
* Add schema for Cursor

* Add CursorDao and CursorDaoTest

* Fix comment on getTld

* Change tld column to scope

* Fix cursorTime to be converted to DateTime internally and other small fixes

* Add a CursorType enum and a createGlobal constructor for Cursor

* Rename flyway file

* Use cursorType from common/Cursor.java and add null checks
2019-12-09 17:47:06 -05:00
Weimin Yu bba5aff4b6 Verify RegistryTool can instantiate (#400)
* Verify RegistryTool can instantiate

Add a task that instantiates all command classes in RegistryTool
with runtimeClasspath.

Also make sure that runtimeClasspath is a superset of
compileClasspath.
2019-12-06 12:08:16 -05:00
Michael Muller df7d272157 Adding junit back into the runtime classpath (#399)
* Adding junit back into the runtime classpath

Unfortunately, GenerateSqlSchemaCommand depends on junit via testcontainers.
We should really move GenerateSqlSchemaCommand out of nomulus tool (we only
use it during development) but this gets nomulus tool working for the time
being.

* Removed unnnecessary trace line.

* lockfiles generated after update_dependency.sh
2019-12-05 16:01:05 -05:00
Shicong Huang 988f78274e Check SQL dependency in super class for SqlIntegrationMembershipTest (#398)
In some cases, we define JpaTransactionManagerRule in a TestCase
class which is extended by the concrete test class. So, we need
to check if JpaTransactionManagerRule is also defined in the super
class.
2019-12-05 13:12:56 -05:00
157 changed files with 4745 additions and 872 deletions
+37 -9
View File
@@ -179,6 +179,16 @@ allprojects {
mavenCentral()
}
}
if (rootProject.enableCrossReferencing.toBoolean()) {
gradle.projectsEvaluated {
tasks.withType(JavaCompile) {
options.fork = true
options.forkOptions.executable =
"${project.rootDir}/kythe/extractors/javac-wrapper.sh"
}
}
}
}
task runPresubmits(type: Exec) {
@@ -190,24 +200,33 @@ subprojects {
// Skip no-op project
if (project.name == 'services') return
ext.createUberJar = { taskName, binaryName, mainClass ->
ext.createUberJar = {
taskName,
binaryName,
mainClass,
List<Configuration> configs = [project.configurations.runtimeClasspath],
List<SourceSetOutput> srcOutput = [project.sourceSets.main.output],
List<String> excludes = [] ->
project.tasks.create(
taskName, com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar) {
taskName, com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar) {
mergeServiceFiles()
baseName = binaryName
manifest {
attributes 'Main-Class': mainClass
if (mainClass != '') {
manifest {
attributes 'Main-Class': mainClass
}
}
zip64 = true
classifier = ''
archiveVersion = ''
configurations = [project.configurations.runtimeClasspath]
from project.sourceSets.main.output
configurations = configs
from srcOutput
// Excludes signature files that accompany some dependency jars, like
// bonuncycastle. If they are present, only classes from those signed jars are
// made available to the class loader.
// see https://discuss.gradle.org/t/signing-a-custom-gradle-plugin-thats-downloaded-by-the-build-system-from-github/1365
exclude "META-INF/*.SF", "META-INF/*.DSA", "META-INF/*.RSA"
exclude excludes
}
}
@@ -221,9 +240,18 @@ subprojects {
}
afterEvaluate {
if (rootProject.enableDependencyLocking.toBoolean()) {
// Lock application dependencies except for the gradle-license-report
// plugin. See dependency_lic.gradle for the reason why.
if (rootProject.enableDependencyLocking.toBoolean()
&& project.name != 'integration') {
// The ':integration' project runs server/schema integration tests using
// dynamically specified jars with no transitive dependency. Therefore
// dependency-locking does not make sense. Furthermore, during
// evaluation it resolves the 'testRuntime' configuration, making it
// immutable. Locking activation would trigger an invalid operation
// exception.
//
// For all other projects, due to problem with the gradle-license-report
// plugin, the dependencyLicenseReport configuration must opt out of
// dependency-locking. See dependency_lic.gradle for the reason why.
//
// To selectively activate dependency locking without hardcoding them
// in the 'configurations' block, the following code must run after
+14
View File
@@ -19,12 +19,21 @@ sourceSets {
runtimeClasspath += main.output
}
}
test {
java {
compileClasspath += testing.output
runtimeClasspath += testing.output
}
}
}
configurations {
testingCompile.extendsFrom compile
testingRuntime.extendsFrom runtime
testCompile.extendsFrom testingCompile
testRuntime.extendsFrom testingRuntime
// All testing util classes. Other projects may declare dependency as:
// testCompile project(path: 'common', configuration: 'testing')
testing
@@ -49,4 +58,9 @@ dependencies {
testingCompile deps['com.google.flogger:flogger']
testingRuntime deps['com.google.flogger:flogger-system-backend']
testingCompile deps['com.google.truth:truth']
testingCompile deps['io.github.java-diff-utils:java-diff-utils']
testCompile deps['junit:junit']
testCompile project(':third_party')
}
@@ -1,13 +1,22 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -1,13 +1,22 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -1,13 +1,23 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -1,13 +1,23 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger-system-backend:0.1
com.google.flogger:flogger:0.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -1,6 +1,7 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger:0.1
@@ -8,7 +9,14 @@ com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -1,6 +1,7 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger:0.1
@@ -8,7 +9,14 @@ com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -1,6 +1,7 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger-system-backend:0.1
@@ -9,7 +10,14 @@ com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -1,6 +1,7 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.google.auto.value:auto-value-annotations:1.6.3
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.3.2
com.google.flogger:flogger-system-backend:0.1
@@ -9,7 +10,14 @@ com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.3
com.google.truth:truth:1.0
com.googlecode.java-diff-utils:diffutils:1.3.0
io.github.java-diff-utils:java-diff-utils:4.0
javax.inject:javax.inject:1
joda-time:joda-time:2.9.2
junit:junit:4.12
org.checkerframework:checker-compat-qual:2.5.5
org.checkerframework:checker-qual:2.8.1
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.hamcrest:hamcrest-core:1.3
@@ -12,17 +12,17 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.testing;
package google.registry.testing.truth;
import static com.google.common.io.Resources.getResource;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.JUnitBackports.assertThrows;
import static google.registry.testing.TextDiffSubject.assertThat;
import static google.registry.testing.truth.TextDiffSubject.assertThat;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.io.Resources;
import google.registry.testing.TextDiffSubject.DiffFormat;
import google.registry.testing.truth.TextDiffSubject.DiffFormat;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import org.junit.Test;
@@ -33,15 +33,15 @@ import org.junit.runners.JUnit4;
@RunWith(JUnit4.class)
public class TextDiffSubjectTest {
private static final String RESOURCE_FOLDER = "google/registry/testing/truth/";
// Resources for input data.
private static final String ACTUAL_RESOURCE = "google/registry/testing/text-diff-actual.txt";
private static final String EXPECTED_RESOURCE = "google/registry/testing/text-diff-expected.txt";
private static final String ACTUAL_RESOURCE = RESOURCE_FOLDER + "text-diff-actual.txt";
private static final String EXPECTED_RESOURCE = RESOURCE_FOLDER + "text-diff-expected.txt";
// Resources for expected diff texts.
private static final String UNIFIED_DIFF_RESOURCE =
"google/registry/testing/text-unified-diff.txt";
private static final String UNIFIED_DIFF_RESOURCE = RESOURCE_FOLDER + "text-unified-diff.txt";
private static final String SIDE_BY_SIDE_DIFF_RESOURCE =
"google/registry/testing/text-sidebyside-diff.txt";
RESOURCE_FOLDER + "text-sidebyside-diff.txt";
@Test
public void unifiedDiff_equal() throws IOException {
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.testing;
package google.registry.testing.truth;
import static com.google.common.base.Preconditions.checkNotNull;
import static com.google.common.truth.Truth.assertAbout;
+168 -30
View File
@@ -12,8 +12,12 @@
// See the License for the specific language governing permissions and
// limitations under the License.
import java.lang.reflect.Constructor
import java.util.Optional
plugins {
id 'java-library'
id 'maven-publish'
}
// Path to code generated by ad hoc tasks in this project. A separate path is
@@ -86,8 +90,17 @@ sourceSets {
exclude '**/*.xjb'
}
}
nonprod {
java {
compileClasspath += main.output
runtimeClasspath += main.output
}
}
test {
java {
compileClasspath += nonprod.output
runtimeClasspath += nonprod.output
// Javadoc API is deprecated in Java 11 and removed in Java 12.
// TODO(jianglai): re-enable after migrating to the new Javadoc API
if ((JavaVersion.current().majorVersion as Integer) >= 11) {
@@ -109,6 +122,18 @@ configurations {
jaxb
soy
closureCompiler
devtool
nonprodCompile.extendsFrom compile
nonprodRuntime.extendsFrom runtime
testCompile.extendsFrom nonprodCompile
testRuntime.extendsFrom nonprodRuntime
// Published jars that are used for server/schema compatibility tests.
// See <a href="../integration/README.md">the integration project</a>
// for details.
nomulus_test
// Exclude non-canonical servlet-api jars. Our AppEngine deployment uses
// javax.servlet:servlet-api:2.5
@@ -139,9 +164,9 @@ dependencies {
// Custom-built objectify jar at commit ecd5165, included in Nomulus
// release.
implementation files(
compile files(
"${rootDir}/third_party/objectify/v4_1/objectify-4.1.3.jar")
testImplementation project(':third_party')
testRuntime project(':third_party')
testRuntime files(sourceSets.test.resources.srcDirs)
@@ -192,6 +217,7 @@ dependencies {
testCompile deps['com.thoughtworks.qdox:qdox']
compile deps['dnsjava:dnsjava']
testCompile deps['io.github.classgraph:classgraph']
testRuntime deps['io.github.java-diff-utils:java-diff-utils']
testCompile deps['javax.annotation:javax.annotation-api']
testCompile deps['javax.annotation:jsr250-api']
compile deps['javax.inject:javax.inject']
@@ -234,6 +260,7 @@ dependencies {
compile deps['org.testcontainers:postgresql']
testCompile deps['org.testcontainers:selenium']
testCompile deps['org.testcontainers:testcontainers']
testCompile deps['pl.pragmatists:JUnitParams']
compile deps['xerces:xmlParserAPIs']
compile deps['xpp3:xpp3']
// This dependency must come after javax.mail:mail as it would otherwise
@@ -660,10 +687,35 @@ task outcastTest(type: FilteringTest) {
maxParallelForks 5
}
// Whitebox test verifying that RegistryTool can be instantiated. Note the
// use of runtimeClasspath. This test emulates the logic in RegistryCli#run.
// A to-do is added there to refactor.
// TODO(weiminyu): Need a similar test for Registry server.
task registryToolIntegrationTest {
dependsOn compileJava
doLast {
def classLoader =
new URLClassLoader(sourceSets.main.runtimeClasspath.collect {
it.toURI().toURL()
} as URL[])
def commandClasses =
(classLoader.loadClass('google.registry.tools.RegistryTool')
.getDeclaredField('COMMAND_MAP').get(null) as Map).values()
commandClasses.each {
try {
Constructor<?> c = ((Class<?>) it).getDeclaredConstructor()
c.setAccessible(true)
c.newInstance()
} catch (Throwable e) {
throw new RuntimeException("Failed to instantiate ${it}:\n ${e}")
}
}
}
}
// Dedicated test suite for schema-dependent tests.
task sqlIntegrationTest(type: FilteringTest) {
systemProperties project.getProperties().subMap('sql_schema_resource_root')
excludeTestCases = false
tests = ['google/registry/schema/integration/SqlIntegrationTestSuite.*']
}
@@ -682,41 +734,59 @@ task findGoldenImages(type: JavaExec) {
args arguments
}
// To run the nomulus tool with these command line tokens:
Optional<List<String>> getToolArgsList() {
// If "-PtoolArgs=..." is present in the command line, use it to set the args,
// otherwise use the default flag, which is "--args" to set the args.
def toolArgs = rootProject.findProperty("toolArgs")
if (toolArgs != null) {
def delimiter = '|'
toolArgs += delimiter
def argsList = []
def currArg = ''
for (def i = 0; i < toolArgs.length(); i++) {
def currChar = toolArgs[i]
if (currChar != delimiter) {
currArg += currChar
} else if (i != 0 && toolArgs[i - 1] == '\\') {
currArg = currArg.substring(0, currArg.length() - 1) + currChar
} else {
argsList.add(currArg)
currArg = ''
}
}
return Optional.of(argsList)
}
return Optional.empty()
}
// To run the nomulus tools with these command line tokens:
// "--foo", "bar baz", "--qux=quz"
// gradle registryTool --args="--foo 'bar baz' --qux=quz"
// or:
// gradle registryTool --PtoolArgs="--foo|bar baz|--qux=quz"
// Note that the delimiting pipe can be backslash escaped if it is part of a
// parameter.
task registryTool(type: JavaExec) {
classpath = sourceSets.main.runtimeClasspath
main = 'google.registry.tools.RegistryTool'
ext.createToolTask = {
taskName,
mainClass,
sourceSet = sourceSets.main ->
project.tasks.create(taskName, JavaExec) {
classpath = sourceSet.runtimeClasspath
main = mainClass
// If "-PtoolArgs=..." is present in the command line, use it to set the args,
// otherwise use the default flag, which is "--args" to set the args.
doFirst {
def toolArgs = rootProject.findProperty("toolArgs")
if (toolArgs != null) {
def delimiter = '|'
toolArgs += delimiter
def argsList = []
def currArg = ''
for (def i = 0; i < toolArgs.length(); i++) {
def currChar = toolArgs[i]
if (currChar != delimiter) {
currArg += currChar
} else if (i != 0 && toolArgs[i - 1] == '\\') {
currArg = currArg.substring(0, currArg.length() - 1) + currChar
} else {
argsList.add(currArg)
currArg = ''
}
doFirst {
getToolArgsList().ifPresent {
args it
}
args = argsList
}
}
}
createToolTask('registryTool', 'google.registry.tools.RegistryTool')
createToolTask(
'devTool',
'google.registry.tools.DevTool',
sourceSets.nonprod)
task generateGoldenImages(type: FilteringTest) {
tests = ["**/webdriver/*"]
@@ -779,15 +849,82 @@ test {
// Don't run any tests from this task, all testing gets done in the
// FilteringTest tasks.
exclude "**"
}.dependsOn(fragileTest, outcastTest, standardTest)
}.dependsOn(fragileTest, outcastTest, standardTest, registryToolIntegrationTest)
createUberJar('nomulus', 'nomulus', 'google.registry.tools.RegistryTool')
project.nomulus.dependsOn project(':third_party').jar
// A jar with classes and resources from main sourceSet, excluding internal
// data. See comments on configurations.nomulus_test above for details.
task nomulusFossJar (type: Jar) {
archiveBaseName = 'nomulus'
archiveClassifier = 'public'
from (project.sourceSets.main.output) {
exclude 'google/registry/config/files/**'
}
from (project.sourceSets.main.output) {
include 'google/registry/config/files/default-config.yaml'
include 'google/registry/config/files/nomulus-config-unittest.yaml'
}
}
// An UberJar of registry test classes, resources and all dependencies.
// See comments on configurations.nomulus_test above for details.
createUberJar(
'testUberJar',
'nomulus-tests',
'',
[project.configurations.testRuntimeClasspath],
[project.sourceSets.test.output],
[
// Exclude SQL schema, which is a test dependency.
'sql/flyway/**',
// ShadowJar includes java source files when used on
// sourceSets.test.output.
'**/*.java'
])
tasks.testUberJar {
archiveClassifier = 'alldeps'
}
artifacts {
nomulus_test nomulusFossJar
nomulus_test testUberJar
}
publishing {
repositories {
maven {
url project.publish_repo
}
}
publications {
nomulusTestsPublication(MavenPublication) {
groupId 'google.registry'
artifactId 'nomulus_test'
version project.nomulus_version
artifact nomulusFossJar
artifact testUberJar
}
}
}
task buildToolImage(dependsOn: nomulus, type: Exec) {
commandLine 'docker', 'build', '-t', 'nomulus-tool', '.'
}
// Build the devtool jar.
createUberJar(
'devtool',
'devtool',
'google.registry.tools.DevTool',
[ project.configurations.nonprodRuntimeClasspath ],
[ project.sourceSets.nonprod.output, sourceSets.main.output ],
[ '**/*.java' ])
artifacts {
devtool devtool
}
task copyJsFilesForTestServer(dependsOn: assemble, type: Copy) {
// Unfortunately the test server relies on having some compiled JS/CSS
// in place, so copy it over here
@@ -802,5 +939,6 @@ task runTestServer(dependsOn: copyJsFilesForTestServer, type: JavaExec) {
classpath = sourceSets.test.runtimeClasspath
}
project.build.dependsOn devtool
project.build.dependsOn buildToolImage
project.build.dependsOn ':stage'
@@ -163,7 +163,6 @@ javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
@@ -193,7 +192,6 @@ org.dom4j:dom4j:2.1.1
org.easymock:easymock:3.0
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
@@ -205,7 +203,6 @@ org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
@@ -0,0 +1,3 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
@@ -0,0 +1,3 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
@@ -0,0 +1,24 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
com.github.kevinstern:software-and-algorithms:1.0
com.github.stephenc.jcip:jcip-annotations:1.0-1
com.google.auto:auto-common:0.10
com.google.code.findbugs:jFormatString:3.0.0
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotation:2.3.3
com.google.errorprone:error_prone_annotations:2.3.3
com.google.errorprone:error_prone_check_api:2.3.3
com.google.errorprone:error_prone_core:2.3.3
com.google.errorprone:error_prone_type_annotations:2.3.3
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:27.0.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc:j2objc-annotations:1.1
com.google.protobuf:protobuf-java:3.4.0
com.googlecode.java-diff-utils:diffutils:1.3.0
org.checkerframework:checker-qual:2.5.3
org.checkerframework:dataflow:2.5.3
org.checkerframework:javacutil:2.5.3
org.codehaus.mojo:animal-sniffer-annotations:1.17
org.pcollections:pcollections:2.1.2
@@ -0,0 +1,3 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
@@ -0,0 +1,234 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.9.10
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.google.api-client:google-api-client-appengine:1.29.0
com.google.api-client:google-api-client-jackson2:1.27.0
com.google.api-client:google-api-client-java6:1.27.0
com.google.api-client:google-api-client-servlet:1.29.0
com.google.api-client:google-api-client:1.29.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-bigtable-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:grpc-google-common-protos:1.12.0
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:proto-google-cloud-bigtable-v2:0.44.0
com.google.api.grpc:proto-google-cloud-datastore-v1:0.44.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:proto-google-common-protos:1.12.0
com.google.api.grpc:proto-google-iam-v1:0.12.0
com.google.api:api-common:1.7.0
com.google.api:gax-grpc:1.38.0
com.google.api:gax-httpjson:0.52.0
com.google.api:gax:1.38.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20181104-1.27.0
com.google.apis:google-api-services-clouddebugger:v2-rev20180801-1.27.0
com.google.apis:google-api-services-cloudkms:v1-rev12-1.22.0
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20181015-1.27.0
com.google.apis:google-api-services-dataflow:v1b3-rev20190607-1.27.0
com.google.apis:google-api-services-dns:v2beta1-rev6-1.22.0
com.google.apis:google-api-services-drive:v2-rev160-1.19.1
com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
com.google.cloud:google-cloud-core-grpc:1.61.0
com.google.cloud:google-cloud-core-http:1.55.0
com.google.cloud:google-cloud-core:1.61.0
com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.errorprone:error_prone_annotations:2.3.3
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.29.2
com.google.http-client:google-http-client-jackson2:1.30.1
com.google.http-client:google-http-client-jackson:1.20.0
com.google.http-client:google-http-client-protobuf:1.20.0
com.google.http-client:google-http-client:1.30.1
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.2
com.google.monitoring-client:metrics:1.0.6
com.google.monitoring-client:stackdriver:1.0.6
com.google.oauth-client:google-oauth-client-appengine:1.29.0
com.google.oauth-client:google-oauth-client-java6:1.28.0
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.29.0
com.google.oauth-client:google-oauth-client:1.29.2
com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-5
com.google.protobuf:protobuf-java-util:3.6.1
com.google.protobuf:protobuf-java:3.6.1
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:57.1
com.jcraft:jsch:0.1.55
com.kohlschutter.junixsocket:junixsocket-common:2.0.4
com.kohlschutter.junixsocket:junixsocket-native-common:2.0.4
com.squareup.okhttp:okhttp:2.5.0
com.squareup.okio:okio:1.13.0
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.2.0
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.1.2
io.grpc:grpc-all:1.17.1
io.grpc:grpc-alts:1.17.1
io.grpc:grpc-auth:1.17.1
io.grpc:grpc-context:1.19.0
io.grpc:grpc-core:1.17.1
io.grpc:grpc-grpclb:1.17.1
io.grpc:grpc-netty-shaded:1.17.1
io.grpc:grpc-netty:1.17.1
io.grpc:grpc-okhttp:1.17.1
io.grpc:grpc-protobuf-lite:1.17.1
io.grpc:grpc-protobuf-nano:1.17.1
io.grpc:grpc-protobuf:1.17.1
io.grpc:grpc-stub:1.17.1
io.grpc:grpc-testing:1.17.1
io.netty:netty-buffer:4.1.30.Final
io.netty:netty-codec-http2:4.1.30.Final
io.netty:netty-codec-http:4.1.30.Final
io.netty:netty-codec-socks:4.1.30.Final
io.netty:netty-codec:4.1.30.Final
io.netty:netty-common:4.1.30.Final
io.netty:netty-handler-proxy:4.1.30.Final
io.netty:netty-handler:4.1.30.Final
io.netty:netty-resolver:4.1.30.Final
io.netty:netty-tcnative-boringssl-static:2.0.17.Final
io.netty:netty-transport:4.1.30.Final
io.opencensus:opencensus-api:0.21.0
io.opencensus:opencensus-contrib-grpc-metrics:0.17.0
io.opencensus:opencensus-contrib-grpc-util:0.17.0
io.opencensus:opencensus-contrib-http-util:0.21.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
org.apache.beam:beam-runners-core-construction-java:2.16.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.16.0
org.apache.beam:beam-sdks-java-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.16.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.8.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.dom4j:dom4j:2.1.1
org.easymock:easymock:3.0
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -0,0 +1,232 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.9.10
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.google.api-client:google-api-client-appengine:1.29.0
com.google.api-client:google-api-client-jackson2:1.27.0
com.google.api-client:google-api-client-java6:1.27.0
com.google.api-client:google-api-client-servlet:1.29.0
com.google.api-client:google-api-client:1.29.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-bigtable-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:grpc-google-common-protos:1.12.0
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:proto-google-cloud-bigtable-v2:0.44.0
com.google.api.grpc:proto-google-cloud-datastore-v1:0.44.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:proto-google-common-protos:1.12.0
com.google.api.grpc:proto-google-iam-v1:0.12.0
com.google.api:api-common:1.7.0
com.google.api:gax-grpc:1.38.0
com.google.api:gax-httpjson:0.52.0
com.google.api:gax:1.38.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20181104-1.27.0
com.google.apis:google-api-services-clouddebugger:v2-rev20180801-1.27.0
com.google.apis:google-api-services-cloudkms:v1-rev12-1.22.0
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20181015-1.27.0
com.google.apis:google-api-services-dataflow:v1b3-rev20190607-1.27.0
com.google.apis:google-api-services-dns:v2beta1-rev6-1.22.0
com.google.apis:google-api-services-drive:v2-rev160-1.19.1
com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
com.google.cloud:google-cloud-core-grpc:1.61.0
com.google.cloud:google-cloud-core-http:1.55.0
com.google.cloud:google-cloud-core:1.61.0
com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.errorprone:error_prone_annotations:2.3.3
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.29.2
com.google.http-client:google-http-client-jackson2:1.30.1
com.google.http-client:google-http-client-jackson:1.20.0
com.google.http-client:google-http-client-protobuf:1.20.0
com.google.http-client:google-http-client:1.30.1
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.2
com.google.monitoring-client:metrics:1.0.6
com.google.monitoring-client:stackdriver:1.0.6
com.google.oauth-client:google-oauth-client-appengine:1.29.0
com.google.oauth-client:google-oauth-client-java6:1.28.0
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.29.0
com.google.oauth-client:google-oauth-client:1.29.2
com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-5
com.google.protobuf:protobuf-java-util:3.6.1
com.google.protobuf:protobuf-java:3.6.1
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:57.1
com.jcraft:jsch:0.1.55
com.kohlschutter.junixsocket:junixsocket-common:2.0.4
com.kohlschutter.junixsocket:junixsocket-native-common:2.0.4
com.squareup.okhttp:okhttp:2.5.0
com.squareup.okio:okio:1.13.0
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.2.0
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.1.2
io.grpc:grpc-all:1.17.1
io.grpc:grpc-alts:1.17.1
io.grpc:grpc-auth:1.17.1
io.grpc:grpc-context:1.19.0
io.grpc:grpc-core:1.17.1
io.grpc:grpc-netty-shaded:1.17.1
io.grpc:grpc-netty:1.17.1
io.grpc:grpc-okhttp:1.17.1
io.grpc:grpc-protobuf-lite:1.17.1
io.grpc:grpc-protobuf-nano:1.17.1
io.grpc:grpc-protobuf:1.17.1
io.grpc:grpc-stub:1.17.1
io.grpc:grpc-testing:1.17.1
io.netty:netty-buffer:4.1.30.Final
io.netty:netty-codec-http2:4.1.30.Final
io.netty:netty-codec-http:4.1.30.Final
io.netty:netty-codec-socks:4.1.30.Final
io.netty:netty-codec:4.1.30.Final
io.netty:netty-common:4.1.30.Final
io.netty:netty-handler-proxy:4.1.30.Final
io.netty:netty-handler:4.1.30.Final
io.netty:netty-resolver:4.1.30.Final
io.netty:netty-tcnative-boringssl-static:2.0.17.Final
io.netty:netty-transport:4.1.30.Final
io.opencensus:opencensus-api:0.21.0
io.opencensus:opencensus-contrib-grpc-metrics:0.17.0
io.opencensus:opencensus-contrib-grpc-util:0.17.0
io.opencensus:opencensus-contrib-http-util:0.21.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
org.apache.beam:beam-runners-core-construction-java:2.16.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.16.0
org.apache.beam:beam-sdks-java-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.16.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.8.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.dom4j:dom4j:2.1.1
org.easymock:easymock:3.0
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -0,0 +1,3 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
@@ -0,0 +1,235 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.9.10
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.google.api-client:google-api-client-appengine:1.29.0
com.google.api-client:google-api-client-jackson2:1.27.0
com.google.api-client:google-api-client-java6:1.27.0
com.google.api-client:google-api-client-servlet:1.29.0
com.google.api-client:google-api-client:1.29.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-bigtable-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:grpc-google-common-protos:1.12.0
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:proto-google-cloud-bigtable-v2:0.44.0
com.google.api.grpc:proto-google-cloud-datastore-v1:0.44.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:proto-google-common-protos:1.12.0
com.google.api.grpc:proto-google-iam-v1:0.12.0
com.google.api:api-common:1.7.0
com.google.api:gax-grpc:1.38.0
com.google.api:gax-httpjson:0.52.0
com.google.api:gax:1.38.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20181104-1.27.0
com.google.apis:google-api-services-clouddebugger:v2-rev20180801-1.27.0
com.google.apis:google-api-services-cloudkms:v1-rev12-1.22.0
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20181015-1.27.0
com.google.apis:google-api-services-dataflow:v1b3-rev20190607-1.27.0
com.google.apis:google-api-services-dns:v2beta1-rev6-1.22.0
com.google.apis:google-api-services-drive:v2-rev160-1.19.1
com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
com.google.cloud:google-cloud-core-grpc:1.61.0
com.google.cloud:google-cloud-core-http:1.55.0
com.google.cloud:google-cloud-core:1.61.0
com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.errorprone:error_prone_annotations:2.3.3
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.29.2
com.google.http-client:google-http-client-jackson2:1.30.1
com.google.http-client:google-http-client-jackson:1.20.0
com.google.http-client:google-http-client-protobuf:1.20.0
com.google.http-client:google-http-client:1.30.1
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.2
com.google.monitoring-client:metrics:1.0.6
com.google.monitoring-client:stackdriver:1.0.6
com.google.oauth-client:google-oauth-client-appengine:1.29.0
com.google.oauth-client:google-oauth-client-java6:1.28.0
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.29.0
com.google.oauth-client:google-oauth-client:1.29.2
com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-5
com.google.protobuf:protobuf-java-util:3.6.1
com.google.protobuf:protobuf-java:3.6.1
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:57.1
com.jcraft:jsch:0.1.55
com.kohlschutter.junixsocket:junixsocket-common:2.0.4
com.kohlschutter.junixsocket:junixsocket-native-common:2.0.4
com.squareup.okhttp:okhttp:2.5.0
com.squareup.okio:okio:1.13.0
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.2.0
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.1.2
io.grpc:grpc-all:1.17.1
io.grpc:grpc-alts:1.17.1
io.grpc:grpc-auth:1.17.1
io.grpc:grpc-context:1.19.0
io.grpc:grpc-core:1.17.1
io.grpc:grpc-grpclb:1.17.1
io.grpc:grpc-netty-shaded:1.17.1
io.grpc:grpc-netty:1.17.1
io.grpc:grpc-okhttp:1.17.1
io.grpc:grpc-protobuf-lite:1.17.1
io.grpc:grpc-protobuf-nano:1.17.1
io.grpc:grpc-protobuf:1.17.1
io.grpc:grpc-stub:1.17.1
io.grpc:grpc-testing:1.17.1
io.netty:netty-buffer:4.1.30.Final
io.netty:netty-codec-http2:4.1.30.Final
io.netty:netty-codec-http:4.1.30.Final
io.netty:netty-codec-socks:4.1.30.Final
io.netty:netty-codec:4.1.30.Final
io.netty:netty-common:4.1.30.Final
io.netty:netty-handler-proxy:4.1.30.Final
io.netty:netty-handler:4.1.30.Final
io.netty:netty-resolver:4.1.30.Final
io.netty:netty-tcnative-boringssl-static:2.0.17.Final
io.netty:netty-transport:4.1.30.Final
io.opencensus:opencensus-api:0.21.0
io.opencensus:opencensus-contrib-grpc-metrics:0.17.0
io.opencensus:opencensus-contrib-grpc-util:0.17.0
io.opencensus:opencensus-contrib-http-util:0.21.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
org.apache.beam:beam-runners-core-construction-java:2.16.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.16.0
org.apache.beam:beam-sdks-java-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.16.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.8.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.dom4j:dom4j:2.1.1
org.easymock:easymock:3.0
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.postgresql:postgresql:42.2.6
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -0,0 +1,235 @@
# This is a Gradle generated file for dependency locking.
# Manual edits can break the build and are not advised.
# This file is expected to be part of source control.
antlr:antlr:2.7.7
aopalliance:aopalliance:1.0
args4j:args4j:2.0.23
cglib:cglib-nodep:2.2
com.beust:jcommander:1.60
com.fasterxml.jackson.core:jackson-annotations:2.9.10
com.fasterxml.jackson.core:jackson-core:2.9.10
com.fasterxml.jackson.core:jackson-databind:2.9.10
com.fasterxml:classmate:1.3.4
com.google.api-client:google-api-client-appengine:1.29.0
com.google.api-client:google-api-client-jackson2:1.27.0
com.google.api-client:google-api-client-java6:1.27.0
com.google.api-client:google-api-client-servlet:1.29.0
com.google.api-client:google-api-client:1.29.2
com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-bigtable-v2:0.38.0
com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:grpc-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:grpc-google-common-protos:1.12.0
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.44.0
com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:0.38.0
com.google.api.grpc:proto-google-cloud-bigtable-v2:0.44.0
com.google.api.grpc:proto-google-cloud-datastore-v1:0.44.0
com.google.api.grpc:proto-google-cloud-pubsub-v1:1.43.0
com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.6.0
com.google.api.grpc:proto-google-cloud-spanner-v1:1.6.0
com.google.api.grpc:proto-google-common-protos:1.12.0
com.google.api.grpc:proto-google-iam-v1:0.12.0
com.google.api:api-common:1.7.0
com.google.api:gax-grpc:1.38.0
com.google.api:gax-httpjson:0.52.0
com.google.api:gax:1.38.0
com.google.apis:google-api-services-admin-directory:directory_v1-rev72-1.22.0
com.google.apis:google-api-services-appengine:v1-rev101-1.25.0
com.google.apis:google-api-services-bigquery:v2-rev20181104-1.27.0
com.google.apis:google-api-services-clouddebugger:v2-rev20180801-1.27.0
com.google.apis:google-api-services-cloudkms:v1-rev12-1.22.0
com.google.apis:google-api-services-cloudresourcemanager:v1-rev20181015-1.27.0
com.google.apis:google-api-services-dataflow:v1b3-rev20190607-1.27.0
com.google.apis:google-api-services-dns:v2beta1-rev6-1.22.0
com.google.apis:google-api-services-drive:v2-rev160-1.19.1
com.google.apis:google-api-services-groupssettings:v1-rev60-1.22.0
com.google.apis:google-api-services-monitoring:v3-rev426-1.23.0
com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0
com.google.apis:google-api-services-sheets:v4-rev483-1.22.0
com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0
com.google.appengine.tools:appengine-gcs-client:0.6
com.google.appengine.tools:appengine-mapreduce:0.9
com.google.appengine.tools:appengine-pipeline:0.2.13
com.google.appengine:appengine-api-1.0-sdk:1.9.48
com.google.appengine:appengine-remote-api:1.9.48
com.google.appengine:appengine-testing:1.9.58
com.google.auth:google-auth-library-credentials:0.16.1
com.google.auth:google-auth-library-oauth2-http:0.16.1
com.google.auto.value:auto-value-annotations:1.6.3
com.google.auto.value:auto-value:1.6.3
com.google.cloud.bigdataoss:gcsio:1.9.16
com.google.cloud.bigdataoss:util:1.9.16
com.google.cloud.bigtable:bigtable-client-core:1.8.0
com.google.cloud.datastore:datastore-v1-proto-client:1.6.0
com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha
com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha
com.google.cloud:google-cloud-bigtable:0.73.0-alpha
com.google.cloud:google-cloud-core-grpc:1.61.0
com.google.cloud:google-cloud-core-http:1.55.0
com.google.cloud:google-cloud-core:1.61.0
com.google.cloud:google-cloud-spanner:1.6.0
com.google.code.findbugs:jsr305:3.0.2
com.google.code.gson:gson:2.8.5
com.google.common.html.types:types:1.0.4
com.google.dagger:dagger:2.21
com.google.errorprone:error_prone_annotations:2.3.3
com.google.flogger:flogger-system-backend:0.3.1
com.google.flogger:flogger:0.3.1
com.google.flogger:google-extensions:0.3.1
com.google.guava:failureaccess:1.0.1
com.google.guava:guava:28.1-jre
com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava
com.google.gwt:gwt-user:2.8.2
com.google.http-client:google-http-client-appengine:1.29.2
com.google.http-client:google-http-client-jackson2:1.30.1
com.google.http-client:google-http-client-jackson:1.20.0
com.google.http-client:google-http-client-protobuf:1.20.0
com.google.http-client:google-http-client:1.30.1
com.google.inject.extensions:guice-multibindings:4.1.0
com.google.inject:guice:4.1.0
com.google.j2objc:j2objc-annotations:1.3
com.google.jsinterop:jsinterop-annotations:1.0.2
com.google.monitoring-client:metrics:1.0.6
com.google.monitoring-client:stackdriver:1.0.6
com.google.oauth-client:google-oauth-client-appengine:1.29.0
com.google.oauth-client:google-oauth-client-java6:1.28.0
com.google.oauth-client:google-oauth-client-jetty:1.28.0
com.google.oauth-client:google-oauth-client-servlet:1.29.0
com.google.oauth-client:google-oauth-client:1.29.2
com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-5
com.google.protobuf:protobuf-java-util:3.6.1
com.google.protobuf:protobuf-java:3.6.1
com.google.re2j:re2j:1.1
com.google.template:soy:2018-03-14
com.googlecode.charts4j:charts4j:1.3
com.googlecode.json-simple:json-simple:1.1.1
com.ibm.icu:icu4j:57.1
com.jcraft:jsch:0.1.55
com.kohlschutter.junixsocket:junixsocket-common:2.0.4
com.kohlschutter.junixsocket:junixsocket-native-common:2.0.4
com.squareup.okhttp:okhttp:2.5.0
com.squareup.okio:okio:1.13.0
com.sun.istack:istack-commons-runtime:3.0.7
com.sun.xml.fastinfoset:FastInfoset:1.2.15
com.thoughtworks.paranamer:paranamer:2.7
com.zaxxer:HikariCP:3.2.0
commons-codec:commons-codec:1.11
commons-logging:commons-logging:1.2
dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.1.2
io.grpc:grpc-all:1.17.1
io.grpc:grpc-alts:1.17.1
io.grpc:grpc-auth:1.17.1
io.grpc:grpc-context:1.19.0
io.grpc:grpc-core:1.17.1
io.grpc:grpc-grpclb:1.17.1
io.grpc:grpc-netty-shaded:1.17.1
io.grpc:grpc-netty:1.17.1
io.grpc:grpc-okhttp:1.17.1
io.grpc:grpc-protobuf-lite:1.17.1
io.grpc:grpc-protobuf-nano:1.17.1
io.grpc:grpc-protobuf:1.17.1
io.grpc:grpc-stub:1.17.1
io.grpc:grpc-testing:1.17.1
io.netty:netty-buffer:4.1.30.Final
io.netty:netty-codec-http2:4.1.30.Final
io.netty:netty-codec-http:4.1.30.Final
io.netty:netty-codec-socks:4.1.30.Final
io.netty:netty-codec:4.1.30.Final
io.netty:netty-common:4.1.30.Final
io.netty:netty-handler-proxy:4.1.30.Final
io.netty:netty-handler:4.1.30.Final
io.netty:netty-resolver:4.1.30.Final
io.netty:netty-tcnative-boringssl-static:2.0.17.Final
io.netty:netty-transport:4.1.30.Final
io.opencensus:opencensus-api:0.21.0
io.opencensus:opencensus-contrib-grpc-metrics:0.17.0
io.opencensus:opencensus-contrib-grpc-util:0.17.0
io.opencensus:opencensus-contrib-http-util:0.21.0
it.unimi.dsi:fastutil:6.5.16
javax.activation:activation:1.1
javax.activation:javax.activation-api:1.2.0
javax.annotation:javax.annotation-api:1.3.2
javax.annotation:jsr250-api:1.0
javax.inject:javax.inject:1
javax.jdo:jdo2-api:2.3-eb
javax.mail:mail:1.4
javax.persistence:javax.persistence-api:2.2
javax.servlet:servlet-api:2.5
javax.transaction:transaction-api:1.1
javax.validation:validation-api:1.0.0.GA
javax.xml.bind:jaxb-api:2.3.1
jline:jline:1.0
joda-time:joda-time:2.10.3
junit:junit:4.12
net.bytebuddy:byte-buddy:1.9.11
net.java.dev.jna:jna-platform:5.3.1
net.java.dev.jna:jna:5.3.1
org.apache.avro:avro:1.8.2
org.apache.beam:beam-model-job-management:2.16.0
org.apache.beam:beam-model-pipeline:2.16.0
org.apache.beam:beam-runners-core-construction-java:2.16.0
org.apache.beam:beam-runners-google-cloud-dataflow-java:2.16.0
org.apache.beam:beam-sdks-java-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.16.0
org.apache.beam:beam-sdks-java-extensions-protobuf:2.16.0
org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0
org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1
org.apache.beam:beam-vendor-grpc-1_21_0:0.1
org.apache.beam:beam-vendor-guava-26_0-jre:0.1
org.apache.commons:commons-compress:1.19
org.apache.commons:commons-lang3:3.5
org.apache.httpcomponents:httpclient:4.5.8
org.apache.httpcomponents:httpcore:4.4.11
org.bouncycastle:bcpg-jdk15on:1.61
org.bouncycastle:bcprov-jdk15on:1.61
org.checkerframework:checker-qual:2.8.1
org.codehaus.jackson:jackson-core-asl:1.9.13
org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.dom4j:dom4j:2.1.1
org.easymock:easymock:3.0
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
org.hamcrest:hamcrest-core:1.3
org.hibernate.common:hibernate-commons-annotations:5.1.0.Final
org.hibernate:hibernate-core:5.4.4.Final
org.hibernate:hibernate-hikaricp:5.4.4.Final
org.javassist:javassist:3.24.0-GA
org.jboss.logging:jboss-logging:3.3.2.Final
org.jboss.spec.javax.transaction:jboss-transaction-api_1.2_spec:1.1.1.Final
org.jboss:jandex:2.0.5.Final
org.jetbrains:annotations:17.0.0
org.joda:joda-money:1.0.1
org.json:json:20160810
org.jvnet.staxex:stax-ex:1.8
org.mockito:mockito-core:1.9.5
org.mortbay.jetty:jetty-util:6.1.26
org.mortbay.jetty:jetty:6.1.26
org.objenesis:objenesis:1.2
org.ow2.asm:asm-analysis:6.0
org.ow2.asm:asm-commons:6.0
org.ow2.asm:asm-tree:6.0
org.ow2.asm:asm-util:6.0
org.ow2.asm:asm:6.0
org.postgresql:postgresql:42.2.6
org.rnorth.duct-tape:duct-tape:1.0.8
org.rnorth.visible-assertions:visible-assertions:2.1.2
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.28
org.testcontainers:database-commons:1.12.1
org.testcontainers:jdbc:1.12.1
org.testcontainers:postgresql:1.12.1
org.testcontainers:testcontainers:1.12.1
org.threeten:threetenbp:1.3.3
org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -265,5 +265,6 @@ org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -263,5 +263,6 @@ org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -141,6 +141,7 @@ commons-logging:commons-logging:1.2
dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.52
io.github.java-diff-utils:java-diff-utils:4.0
io.grpc:grpc-all:1.17.1
io.grpc:grpc-alts:1.17.1
io.grpc:grpc-auth:1.17.1
@@ -226,6 +227,7 @@ org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.dom4j:dom4j:2.1.1
org.easymock:easymock:3.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
@@ -278,5 +280,6 @@ org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -141,6 +141,7 @@ commons-logging:commons-logging:1.2
dnsjava:dnsjava:2.1.7
io.dropwizard.metrics:metrics-core:3.1.2
io.github.classgraph:classgraph:4.8.52
io.github.java-diff-utils:java-diff-utils:4.0
io.grpc:grpc-all:1.17.1
io.grpc:grpc-alts:1.17.1
io.grpc:grpc-auth:1.17.1
@@ -226,6 +227,7 @@ org.codehaus.jackson:jackson-mapper-asl:1.9.13
org.codehaus.mojo:animal-sniffer-annotations:1.18
org.dom4j:dom4j:2.1.1
org.easymock:easymock:3.0
org.eclipse.jgit:org.eclipse.jgit:4.4.1.201607150455-r
org.flywaydb:flyway-core:5.2.4
org.glassfish.jaxb:jaxb-runtime:2.3.1
org.glassfish.jaxb:txw2:2.3.1
@@ -278,5 +280,6 @@ org.tukaani:xz:1.8
org.w3c.css:sac:1.3
org.xerial.snappy:snappy-java:1.1.4
org.yaml:snakeyaml:1.17
pl.pragmatists:JUnitParams:1.1.1
xerces:xmlParserAPIs:2.6.2
xpp3:xpp3:1.1.4c
@@ -917,6 +917,20 @@ public final class RegistryConfig {
return parseEmailAddress(config.misc.spec11OutgoingEmailAddress);
}
/**
* Returns the email addresses to which we will BCC Spec11 emails.
*
* @see google.registry.reporting.spec11.Spec11EmailUtils
*/
@Provides
@Config("spec11BccEmailAddresses")
public static ImmutableList<InternetAddress> provideSpec11BccEmailAddresses(
RegistryConfigSettings config) {
return config.misc.spec11BccEmailAddresses.stream()
.map(RegistryConfig::parseEmailAddress)
.collect(toImmutableList());
}
/**
* Returns the name of the registry, for use in spec 11 emails.
*
@@ -193,6 +193,7 @@ public class RegistryConfigSettings {
public String sheetExportId;
public String alertRecipientEmailAddress;
public String spec11OutgoingEmailAddress;
public List<String> spec11BccEmailAddresses;
public int asyncDeleteDelaySeconds;
public int transientFailureRetries;
}
@@ -389,6 +389,11 @@ misc:
# to be a deliverable email address to handle replies from registrars as well.
spec11OutgoingEmailAddress: abuse@example.com
# Addresses to which we will BCC all Spec11 email reports, in case one
# wishes to examine the output.
spec11BccEmailAddresses:
- abuse@example.com
# How long to delay processing of asynchronous deletions. This should always
# be longer than eppResourceCachingSeconds, to prevent deleted contacts or
# hosts from being used on domains.
@@ -32,7 +32,15 @@ public class CacheUtils {
* lists downloaded from the TMCH get updated in Datastore and the caches need to be refreshed.)
*/
public static <T> Supplier<T> memoizeWithShortExpiration(Supplier<T> original) {
Duration expiration = getSingletonCacheRefreshDuration();
return tryMemoizeWithExpiration(getSingletonCacheRefreshDuration(), original);
}
/**
* Memoize a supplier with the given expiration. If the expiration is zero(likely happens in a
* unit test), it returns the original supplier.
*/
public static <T> Supplier<T> tryMemoizeWithExpiration(
Duration expiration, Supplier<T> original) {
return expiration.isEqual(ZERO)
? original
: memoizeWithExpiration(original, expiration.getMillis(), MILLISECONDS);
@@ -87,7 +87,9 @@ public class Cursor extends ImmutableObject {
/** Cursor for tracking monthly uploads of ICANN activity reports. */
ICANN_UPLOAD_ACTIVITY(Registry.class),
/** Cursor for tracking monthly upload of MANIFEST.txt to ICANN. */
// TODO(sarahbot) Delete this cursor once all data in the database that refers to it is removed.
/** Cursor for tracking monthly uploads of MANIFEST.txt to ICANN. No longer used. */
@Deprecated
ICANN_UPLOAD_MANIFEST(EntityGroupRoot.class);
/** See the definition of scope on {@link #getScopeClass}. */
@@ -791,6 +791,12 @@ public class Registry extends ImmutableObject implements Buildable {
return this;
}
@VisibleForTesting
public Builder setPremiumListKey(@Nullable Key<PremiumList> premiumList) {
getInstance().premiumList = premiumList;
return this;
}
public Builder setRestoreBillingCost(Money amount) {
checkArgument(amount.isPositiveOrZero(), "restoreBillingCost cannot be negative");
getInstance().restoreBillingCost = amount;
@@ -49,7 +49,7 @@ public final class RegistryLockDao {
}
/** Returns all lock objects that this registrar has created. */
public static ImmutableList<RegistryLock> getByRegistrarId(String registrarId) {
public static ImmutableList<RegistryLock> getLockedDomainsByRegistrarId(String registrarId) {
return jpaTm()
.transact(
() ->
@@ -58,7 +58,9 @@ public final class RegistryLockDao {
.getEntityManager()
.createQuery(
"SELECT lock FROM RegistryLock lock WHERE"
+ " lock.registrarId = :registrarId",
+ " lock.registrarId = :registrarId "
+ "AND lock.lockCompletionTimestamp IS NOT NULL "
+ "AND lock.unlockCompletionTimestamp IS NULL",
RegistryLock.class)
.setParameter("registrarId", registrarId)
.getResultList()));
@@ -121,7 +121,7 @@ public abstract class BaseDomainLabelList<T extends Comparable<?>, R extends Dom
* (sans comment) and the comment (in that order). If the line was blank or empty, then this
* method returns an empty list.
*/
protected static List<String> splitOnComment(String line) {
public static List<String> splitOnComment(String line) {
String comment = "";
int index = line.indexOf('#');
if (index != -1) {
@@ -36,11 +36,13 @@ import com.google.common.cache.CacheLoader.InvalidCacheLoadException;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Streams;
import com.google.common.flogger.FluentLogger;
import com.googlecode.objectify.Key;
import google.registry.model.registry.Registry;
import google.registry.model.registry.label.DomainLabelMetrics.PremiumListCheckOutcome;
import google.registry.model.registry.label.PremiumList.PremiumListEntry;
import google.registry.model.registry.label.PremiumList.PremiumListRevision;
import google.registry.schema.tld.PremiumListDao;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
@@ -52,7 +54,9 @@ import org.joda.time.DateTime;
public final class PremiumListUtils {
/** The number of premium list entry entities that are created and deleted per batch. */
static final int TRANSACTION_BATCH_SIZE = 200;
private static final int TRANSACTION_BATCH_SIZE = 200;
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** Value type class used by {@link #checkStatus} to return the results of a premiumness check. */
@AutoValue
@@ -97,6 +101,19 @@ public final class PremiumListUtils {
listName,
checkResults.checkOutcome(),
DateTime.now(UTC).getMillis() - startTime.getMillis());
// Also load the value from Cloud SQL, compare the two results, and log if different.
try {
Optional<Money> priceFromSql = PremiumListDao.getPremiumPrice(label, registry);
if (!priceFromSql.equals(checkResults.premiumPrice())) {
logger.atWarning().log(
"Unequal prices for domain %s.%s from Datastore (%s) and Cloud SQL (%s).",
label, registry.getTldStr(), checkResults.premiumPrice(), priceFromSql);
}
} catch (Throwable t) {
logger.atSevere().withCause(t).log(
"Error loading price of domain %s.%s from Cloud SQL.", label, registry.getTldStr());
}
return checkResults.premiumPrice();
}
@@ -27,6 +27,10 @@ import static google.registry.util.DateTimeUtils.START_OF_TIME;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Supplier;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.MapDifference;
import com.google.common.collect.MapDifference.ValueDifference;
import com.google.common.collect.Maps;
import com.google.common.flogger.FluentLogger;
import com.google.common.util.concurrent.UncheckedExecutionException;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.EmbedMap;
@@ -40,6 +44,8 @@ import google.registry.model.annotations.NotBackedUp;
import google.registry.model.annotations.NotBackedUp.Reason;
import google.registry.model.annotations.VirtualEntity;
import google.registry.model.common.CrossTldSingleton;
import google.registry.schema.tmch.ClaimsList;
import google.registry.schema.tmch.ClaimsListDao;
import google.registry.util.CollectionUtils;
import google.registry.util.Concurrent;
import google.registry.util.Retrier;
@@ -71,6 +77,8 @@ import org.joda.time.DateTime;
@NotBackedUp(reason = Reason.EXTERNALLY_SOURCED)
public class ClaimsListShard extends ImmutableObject {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** The number of claims list entries to store per shard. */
private static final int SHARD_SIZE = 10000;
@@ -112,8 +120,7 @@ public class ClaimsListShard extends ImmutableObject {
Concurrent.transform(
shardKeys,
key ->
tm()
.transactNewReadOnly(
tm().transactNewReadOnly(
() -> {
ClaimsListShard claimsListShard = ofy().load().key(key).now();
checkState(
@@ -142,9 +149,52 @@ public class ClaimsListShard extends ImmutableObject {
}
}
}
return create(creationTime, ImmutableMap.copyOf(combinedLabelsToKeys));
ClaimsListShard datastoreList =
create(creationTime, ImmutableMap.copyOf(combinedLabelsToKeys));
// Also load the list from Cloud SQL, compare the two lists, and log if different.
try {
loadAndCompareCloudSqlList(datastoreList);
} catch (Throwable t) {
logger.atSevere().withCause(t).log("Error comparing reserved lists.");
}
return datastoreList;
};
private static final void loadAndCompareCloudSqlList(ClaimsListShard datastoreList) {
Optional<ClaimsList> maybeCloudSqlList = ClaimsListDao.getLatestRevision();
if (maybeCloudSqlList.isPresent()) {
ClaimsList cloudSqlList = maybeCloudSqlList.get();
MapDifference<String, String> diff =
Maps.difference(datastoreList.labelsToKeys, cloudSqlList.getLabelsToKeys());
if (!diff.areEqual()) {
if (diff.entriesDiffering().size() > 10) {
logger.atWarning().log(
String.format(
"Unequal claims lists detected, Cloud SQL list with revision id %d has %d"
+ " different records than the current Datastore list.",
cloudSqlList.getRevisionId(), diff.entriesDiffering().size()));
} else {
StringBuilder diffMessage = new StringBuilder("Unequal claims lists detected:\n");
diff.entriesDiffering().entrySet().stream()
.forEach(
entry -> {
String label = entry.getKey();
ValueDifference<String> valueDiff = entry.getValue();
diffMessage.append(
String.format(
"Domain label %s has key %s in Datastore and key %s in Cloud"
+ " SQL.\n",
label, valueDiff.leftValue(), valueDiff.rightValue()));
});
logger.atWarning().log(diffMessage.toString());
}
}
} else {
logger.atWarning().log("Claims list in Cloud SQL is empty.");
}
}
/**
* A cached supplier that fetches the claims list shards from Datastore and recombines them into a
* single {@link ClaimsListShard} object.
@@ -0,0 +1,127 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence;
import java.io.Serializable;
import java.sql.Array;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Types;
import java.util.Collection;
import java.util.Objects;
import org.hibernate.HibernateException;
import org.hibernate.engine.spi.SharedSessionContractImplementor;
import org.hibernate.usertype.UserType;
/** Generic Hibernate user type to store/retrieve Java collection as an array in Cloud SQL. */
public abstract class GenericCollectionUserType<T extends Collection> implements UserType {
abstract T getNewCollection();
abstract ArrayColumnType getColumnType();
enum ArrayColumnType {
STRING(Types.ARRAY, "text");
final int typeCode;
final String typeName;
ArrayColumnType(int typeCode, String typeName) {
this.typeCode = typeCode;
this.typeName = typeName;
}
int getTypeCode() {
return typeCode;
}
String getTypeName() {
return typeName;
}
String getTypeDdlName() {
return typeName + "[]";
}
}
@Override
public int[] sqlTypes() {
return new int[] {getColumnType().getTypeCode()};
}
@Override
public boolean equals(Object x, Object y) throws HibernateException {
return Objects.equals(x, y);
}
@Override
public int hashCode(Object x) throws HibernateException {
return x == null ? 0 : x.hashCode();
}
@Override
public Object nullSafeGet(
ResultSet rs, String[] names, SharedSessionContractImplementor session, Object owner)
throws HibernateException, SQLException {
if (rs.getArray(names[0]) != null) {
T result = getNewCollection();
for (Object element : (Object[]) rs.getArray(names[0]).getArray()) {
result.add(element);
}
return result;
}
return null;
}
@Override
public void nullSafeSet(
PreparedStatement st, Object value, int index, SharedSessionContractImplementor session)
throws HibernateException, SQLException {
if (value == null) {
st.setArray(index, null);
return;
}
T list = (T) value;
Array arr = st.getConnection().createArrayOf(getColumnType().getTypeName(), list.toArray());
st.setArray(index, arr);
}
// TODO(b/147489651): Investigate how to properly implement the below methods.
@Override
public Object deepCopy(Object value) throws HibernateException {
return value;
}
@Override
public boolean isMutable() {
return false;
}
@Override
public Serializable disassemble(Object value) throws HibernateException {
return (Serializable) value;
}
@Override
public Object assemble(Serializable cached, Object owner) throws HibernateException {
return cached;
}
@Override
public Object replace(Object original, Object target, Object owner) throws HibernateException {
return original;
}
}
@@ -32,6 +32,9 @@ import org.hibernate.tool.schema.TargetType;
/** Utility class to export DDL script for given {@link javax.persistence.Entity} classes. */
public class HibernateSchemaExporter {
// Hibernate proprietary mappings.
private static final String HIBERNATE_MAPPING_RESOURCES = "META-INF/orm.xml";
private final String jdbcUrl;
private final String username;
private final String password;
@@ -63,6 +66,7 @@ public class HibernateSchemaExporter {
try (StandardServiceRegistry registry =
new StandardServiceRegistryBuilder().applySettings(settings).build()) {
MetadataSources metadata = new MetadataSources(registry);
metadata.addResource(HIBERNATE_MAPPING_RESOURCES);
// Note that we need to also add all converters to the Hibernate context because
// the entity class may use the customized type.
@@ -13,6 +13,7 @@
// limitations under the License.
package google.registry.persistence;
import google.registry.persistence.GenericCollectionUserType.ArrayColumnType;
import java.sql.Types;
import org.hibernate.dialect.PostgreSQL95Dialect;
@@ -23,5 +24,8 @@ public class NomulusPostgreSQLDialect extends PostgreSQL95Dialect {
registerColumnType(Types.VARCHAR, "text");
registerColumnType(Types.TIMESTAMP_WITH_TIMEZONE, "timestamptz");
registerColumnType(Types.TIMESTAMP, "timestamptz");
for (ArrayColumnType arrayType : ArrayColumnType.values()) {
registerColumnType(arrayType.getTypeCode(), arrayType.getTypeDdlName());
}
}
}
@@ -0,0 +1,37 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence;
import com.google.common.collect.Lists;
import java.util.List;
/** Abstract Hibernate user type for storing/retrieving {@link List<String>}. */
public class StringListUserType extends GenericCollectionUserType<List<String>> {
@Override
List<String> getNewCollection() {
return Lists.newArrayList();
}
@Override
ArrayColumnType getColumnType() {
return ArrayColumnType.STRING;
}
@Override
public Class returnedClass() {
return List.class;
}
}
@@ -0,0 +1,37 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence;
import com.google.common.collect.Sets;
import java.util.Set;
/** Abstract Hibernate user type for storing/retrieving {@link Set<String>}. */
public class StringSetUserType extends GenericCollectionUserType<Set<String>> {
@Override
Set<String> getNewCollection() {
return Sets.newHashSet();
}
@Override
ArrayColumnType getColumnType() {
return ArrayColumnType.STRING;
}
@Override
public Class returnedClass() {
return Set.class;
}
}
@@ -20,7 +20,6 @@ import static com.google.common.net.MediaType.PLAIN_TEXT_UTF_8;
import static google.registry.model.common.Cursor.getCursorTimeOrStartOfTime;
import static google.registry.model.ofy.ObjectifyService.ofy;
import static google.registry.model.transaction.TransactionManagerFactory.tm;
import static google.registry.reporting.icann.IcannReportingModule.MANIFEST_FILE_NAME;
import static google.registry.reporting.icann.IcannReportingModule.PARAM_SUBDIR;
import static google.registry.request.Action.Method.POST;
import static javax.servlet.http.HttpServletResponse.SC_OK;
@@ -65,8 +64,9 @@ import org.joda.time.Duration;
* Action that uploads the monthly activity/transactions reports from GCS to ICANN via an HTTP PUT.
*
* <p>This should be run after {@link IcannReportingStagingAction}, which writes out the month's
* reports and a MANIFEST.txt file. This action reads the filenames from the MANIFEST.txt, and
* attempts to upload every file in the manifest to ICANN's endpoint.
* reports and a MANIFEST.txt file. This action checks each ICANN_UPLOAD_TX and
* ICANN_UPLOAD_ACTIVITY cursor and uploads the corresponding report if the cursor time is before
* now.
*
* <p>Parameters:
*
@@ -181,32 +181,22 @@ public final class IcannReportingUploadAction implements Runnable {
// Set cursor to first day of next month if the upload succeeded
if (success) {
Cursor newCursor;
if (cursorType.equals(CursorType.ICANN_UPLOAD_MANIFEST)) {
newCursor =
Cursor.createGlobal(
cursorType, cursorTime.withTimeAtStartOfDay().withDayOfMonth(1).plusMonths(1));
} else {
newCursor =
Cursor.create(
cursorType,
cursorTime.withTimeAtStartOfDay().withDayOfMonth(1).plusMonths(1),
Registry.get(tldStr));
}
Cursor newCursor =
Cursor.create(
cursorType,
cursorTime.withTimeAtStartOfDay().withDayOfMonth(1).plusMonths(1),
Registry.get(tldStr));
tm().transact(() -> ofy().save().entity(newCursor));
}
}
private String getFileName(CursorType cursorType, DateTime cursorTime, String tld) {
if (cursorType.equals(CursorType.ICANN_UPLOAD_MANIFEST)) {
return MANIFEST_FILE_NAME;
}
return String.format(
"%s%s%d%02d.csv",
tld,
(cursorType.equals(CursorType.ICANN_UPLOAD_ACTIVITY) ? "-activity-" : "-transactions-"),
cursorTime.year().get(),
cursorTime.monthOfYear().get());
cursorTime.withDayOfMonth(1).minusMonths(1).monthOfYear().get());
}
/** Returns a map of each cursor to the CursorType and tld. */
@@ -222,7 +212,6 @@ public final class IcannReportingUploadAction implements Runnable {
ImmutableSet.Builder<Key<Cursor>> keys = new ImmutableSet.Builder<>();
keys.addAll(activityKeyMap.keySet());
keys.addAll(transactionKeyMap.keySet());
keys.add(Cursor.createGlobalKey(CursorType.ICANN_UPLOAD_MANIFEST));
Map<Key<Cursor>, Cursor> cursorMap = ofy().load().keys(keys.build());
ImmutableMap.Builder<Cursor, CursorInfo> cursors = new ImmutableMap.Builder<>();
@@ -230,11 +219,6 @@ public final class IcannReportingUploadAction implements Runnable {
activityKeyMap, CursorType.ICANN_UPLOAD_ACTIVITY, cursorMap, cursors);
defaultNullCursorsToNextMonthAndAddToMap(
transactionKeyMap, CursorType.ICANN_UPLOAD_TX, cursorMap, cursors);
Cursor manifestCursor =
cursorMap.getOrDefault(
Cursor.createGlobalKey(CursorType.ICANN_UPLOAD_MANIFEST),
Cursor.createGlobal(CursorType.ICANN_UPLOAD_MANIFEST, clock.nowUtc().minusDays(1)));
cursors.put(manifestCursor, CursorInfo.create(CursorType.ICANN_UPLOAD_MANIFEST, null));
return cursors.build();
}
@@ -259,7 +243,14 @@ public final class IcannReportingUploadAction implements Runnable {
// report staged for upload.
Cursor cursor =
cursorMap.getOrDefault(
key, Cursor.create(type, clock.nowUtc().minusDays(1), registry));
key,
Cursor.create(
type,
clock.nowUtc().withDayOfMonth(1).withTimeAtStartOfDay().plusMonths(1),
registry));
if (!cursorMap.containsValue(cursor)) {
tm().transact(() -> ofy().save().entity(cursor));
}
cursors.put(cursor, CursorInfo.create(type, registry.getTldStr()));
});
}
@@ -58,6 +58,7 @@ public class Spec11EmailUtils {
private final SendEmailService emailService;
private final InternetAddress outgoingEmailAddress;
private final ImmutableList<InternetAddress> spec11BccEmailAddresses;
private final InternetAddress alertRecipientAddress;
private final ImmutableList<String> spec11WebResources;
private final String registryName;
@@ -67,10 +68,12 @@ public class Spec11EmailUtils {
SendEmailService emailService,
@Config("alertRecipientEmailAddress") InternetAddress alertRecipientAddress,
@Config("spec11OutgoingEmailAddress") InternetAddress spec11OutgoingEmailAddress,
@Config("spec11BccEmailAddresses") ImmutableList<InternetAddress> spec11BccEmailAddresses,
@Config("spec11WebResources") ImmutableList<String> spec11WebResources,
@Config("registryName") String registryName) {
this.emailService = emailService;
this.outgoingEmailAddress = spec11OutgoingEmailAddress;
this.spec11BccEmailAddresses = spec11BccEmailAddresses;
this.alertRecipientAddress = alertRecipientAddress;
this.spec11WebResources = spec11WebResources;
this.registryName = registryName;
@@ -125,11 +128,18 @@ public class Spec11EmailUtils {
private RegistrarThreatMatches filterOutNonPublishedMatches(
RegistrarThreatMatches registrarThreatMatches) {
ImmutableList<ThreatMatch> filteredMatches = registrarThreatMatches.threatMatches().stream()
.filter(threatMatch ->
ofy().load().type(DomainBase.class).filter("fullyQualifiedDomainName",
threatMatch.fullyQualifiedDomainName()).first().now().shouldPublishToDns()
).collect(toImmutableList());
ImmutableList<ThreatMatch> filteredMatches =
registrarThreatMatches.threatMatches().stream()
.filter(
threatMatch ->
ofy()
.load()
.type(DomainBase.class)
.filter("fullyQualifiedDomainName", threatMatch.fullyQualifiedDomainName())
.first()
.now()
.shouldPublishToDns())
.collect(toImmutableList());
return RegistrarThreatMatches.create(registrarThreatMatches.clientId(), filteredMatches);
}
@@ -146,7 +156,7 @@ public class Spec11EmailUtils {
.setContentType(MediaType.HTML_UTF_8)
.setFrom(outgoingEmailAddress)
.addRecipient(getEmailAddressForRegistrar(registrarThreatMatches.clientId()))
.setBcc(outgoingEmailAddress)
.setBccs(spec11BccEmailAddresses)
.build());
}
@@ -0,0 +1,116 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.cursor;
import static com.google.appengine.api.search.checkers.Preconditions.checkNotNull;
import google.registry.model.ImmutableObject;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.common.Cursor.CursorType;
import google.registry.schema.cursor.Cursor.CursorId;
import google.registry.util.DateTimeUtils;
import java.io.Serializable;
import java.time.ZonedDateTime;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.Id;
import javax.persistence.IdClass;
import javax.persistence.Table;
import org.joda.time.DateTime;
/**
* Shared entity for date cursors. This uses a compound primary key as defined in {@link CursorId}.
*/
@Entity
@Table
@IdClass(CursorId.class)
public class Cursor {
@Enumerated(EnumType.STRING)
@Column(nullable = false)
@Id
private CursorType type;
@Column @Id private String scope;
@Column(nullable = false)
private ZonedDateTime cursorTime;
@Column(nullable = false)
private UpdateAutoTimestamp lastUpdateTime = UpdateAutoTimestamp.create(null);
/** The scope of a global cursor. A global cursor is a cursor that is not specific to one tld. */
public static final String GLOBAL = "GLOBAL";
private Cursor(CursorType type, String scope, DateTime cursorTime) {
this.type = type;
this.scope = scope;
this.cursorTime = DateTimeUtils.toZonedDateTime(cursorTime);
}
// Hibernate requires a default constructor.
private Cursor() {}
/** Constructs a {@link Cursor} object. */
public static Cursor create(CursorType type, String scope, DateTime cursorTime) {
checkNotNull(
scope, "Scope cannot be null. To create a global cursor, use the createGlobal method");
return new Cursor(type, scope, cursorTime);
}
/** Constructs a {@link Cursor} object with a {@link GLOBAL} scope. */
public static Cursor createGlobal(CursorType type, DateTime cursorTime) {
return new Cursor(type, GLOBAL, cursorTime);
}
/** Returns the type of the cursor. */
public CursorType getType() {
return type;
}
/**
* Returns the scope of the cursor. The scope will typically be the tld the cursor is referring
* to. If the cursor is a global cursor, the scope will be {@link GLOBAL}.
*/
public String getScope() {
return scope;
}
/** Returns the time the cursor is set to. */
public DateTime getCursorTime() {
return DateTimeUtils.toJodaDateTime(cursorTime);
}
/** Returns the last time the cursor was updated. */
public DateTime getLastUpdateTime() {
return lastUpdateTime.getTimestamp();
}
static class CursorId extends ImmutableObject implements Serializable {
public CursorType type;
public String scope;
private CursorId() {}
public CursorId(CursorType type, String scope) {
this.type = type;
this.scope = scope;
}
}
}
@@ -0,0 +1,70 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.cursor;
import static com.google.appengine.api.search.checkers.Preconditions.checkNotNull;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import google.registry.model.common.Cursor.CursorType;
import google.registry.schema.cursor.Cursor.CursorId;
import java.util.List;
/** Data access object class for {@link Cursor}. */
public class CursorDao {
public static void save(Cursor cursor) {
jpaTm()
.transact(
() -> {
jpaTm().getEntityManager().merge(cursor);
});
}
public static Cursor load(CursorType type, String scope) {
checkNotNull(scope, "The scope of the cursor to load cannot be null");
checkNotNull(type, "The type of the cursor to load must be specified");
return jpaTm()
.transact(() -> jpaTm().getEntityManager().find(Cursor.class, new CursorId(type, scope)));
}
/** If no scope is given, use {@link Cursor.GLOBAL} as the scope. */
public static Cursor load(CursorType type) {
checkNotNull(type, "The type of the cursor to load must be specified");
return load(type, Cursor.GLOBAL);
}
public static List<Cursor> loadAll() {
return jpaTm()
.transact(
() ->
jpaTm()
.getEntityManager()
.createQuery("SELECT cursor FROM Cursor cursor", Cursor.class)
.getResultList());
}
public static List<Cursor> loadByType(CursorType type) {
checkNotNull(type, "The type of the cursors to load must be specified");
return jpaTm()
.transact(
() ->
jpaTm()
.getEntityManager()
.createQuery(
"SELECT cursor FROM Cursor cursor WHERE cursor.type = :type", Cursor.class)
.setParameter("type", type)
.getResultList());
}
}
@@ -21,13 +21,12 @@ import static google.registry.util.PreconditionsUtils.checkArgumentNotNull;
import google.registry.model.Buildable;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.ImmutableObject;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.util.DateTimeUtils;
import java.time.ZonedDateTime;
import java.util.Optional;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.EnumType;
import javax.persistence.Enumerated;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
@@ -70,12 +69,6 @@ import org.joda.time.DateTime;
})
public final class RegistryLock extends ImmutableObject implements Buildable {
/** Describes the action taken by the user. */
public enum Action {
LOCK,
UNLOCK
}
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(nullable = false)
@@ -99,28 +92,26 @@ public final class RegistryLock extends ImmutableObject implements Buildable {
/** The POC that performed the action, or null if it was a superuser. */
private String registrarPocId;
/**
* Lock action is immutable and describes whether the action performed was a lock or an unlock.
*/
@Enumerated(EnumType.STRING)
/** When the lock is first requested. */
@Column(nullable = false)
private Action action;
private CreateAutoTimestamp lockRequestTimestamp = CreateAutoTimestamp.create(null);
/** Creation timestamp is when the lock/unlock is first requested. */
@Column(nullable = false)
private CreateAutoTimestamp creationTimestamp = CreateAutoTimestamp.create(null);
/** When the unlock is first requested. */
private ZonedDateTime unlockRequestTimestamp;
/**
* Completion timestamp is when the user has verified the lock/unlock, when this object de facto
* becomes immutable. If this field is null, it means that the lock has not been verified yet (and
* thus not been put into effect).
* When the user has verified the lock. If this field is null, it means the lock has not been
* verified yet (and thus not been put into effect).
*/
private ZonedDateTime completionTimestamp;
private ZonedDateTime lockCompletionTimestamp;
/**
* The user must provide the random verification code in order to complete the lock and move the
* status from PENDING to COMPLETED.
* When the user has verified the unlock of this lock. If this field is null, it means the unlock
* action has not been verified yet (and has not been put into effect).
*/
private ZonedDateTime unlockCompletionTimestamp;
/** The user must provide the random verification code in order to complete the action. */
@Column(nullable = false)
private String verificationCode;
@@ -131,6 +122,9 @@ public final class RegistryLock extends ImmutableObject implements Buildable {
@Column(nullable = false)
private boolean isSuperuser;
/** Time that this entity was last updated. */
private UpdateAutoTimestamp lastUpdateTimestamp;
public String getRepoId() {
return repoId;
}
@@ -147,17 +141,25 @@ public final class RegistryLock extends ImmutableObject implements Buildable {
return registrarPocId;
}
public Action getAction() {
return action;
public DateTime getLockRequestTimestamp() {
return lockRequestTimestamp.getTimestamp();
}
public DateTime getCreationTimestamp() {
return creationTimestamp.getTimestamp();
/** Returns the unlock request timestamp or null if an unlock has not been requested yet. */
public Optional<DateTime> getUnlockRequestTimestamp() {
return Optional.ofNullable(unlockRequestTimestamp).map(DateTimeUtils::toJodaDateTime);
}
/** Returns the completion timestamp, or empty if this lock has not been completed yet. */
public Optional<DateTime> getCompletionTimestamp() {
return Optional.ofNullable(completionTimestamp).map(DateTimeUtils::toJodaDateTime);
public Optional<DateTime> getLockCompletionTimestamp() {
return Optional.ofNullable(lockCompletionTimestamp).map(DateTimeUtils::toJodaDateTime);
}
/**
* Returns the unlock completion timestamp, or empty if this unlock has not been completed yet.
*/
public Optional<DateTime> getUnlockCompletionTimestamp() {
return Optional.ofNullable(unlockCompletionTimestamp).map(DateTimeUtils::toJodaDateTime);
}
public String getVerificationCode() {
@@ -168,16 +170,16 @@ public final class RegistryLock extends ImmutableObject implements Buildable {
return isSuperuser;
}
public DateTime getLastUpdateTimestamp() {
return lastUpdateTimestamp.getTimestamp();
}
public Long getRevisionId() {
return revisionId;
}
public void setCompletionTimestamp(DateTime dateTime) {
this.completionTimestamp = toZonedDateTime(dateTime);
}
public boolean isVerified() {
return completionTimestamp != null;
public boolean isLocked() {
return lockCompletionTimestamp != null && unlockCompletionTimestamp == null;
}
@Override
@@ -198,8 +200,7 @@ public final class RegistryLock extends ImmutableObject implements Buildable {
checkArgumentNotNull(getInstance().repoId, "Repo ID cannot be null");
checkArgumentNotNull(getInstance().domainName, "Domain name cannot be null");
checkArgumentNotNull(getInstance().registrarId, "Registrar ID cannot be null");
checkArgumentNotNull(getInstance().action, "Action cannot be null");
checkArgumentNotNull(getInstance().verificationCode, "Verification codecannot be null");
checkArgumentNotNull(getInstance().verificationCode, "Verification code cannot be null");
checkArgument(
getInstance().registrarPocId != null || getInstance().isSuperuser,
"Registrar POC ID must be provided if superuser is false");
@@ -226,18 +227,18 @@ public final class RegistryLock extends ImmutableObject implements Buildable {
return this;
}
public Builder setAction(Action action) {
getInstance().action = action;
public Builder setUnlockRequestTimestamp(DateTime unlockRequestTimestamp) {
getInstance().unlockRequestTimestamp = toZonedDateTime(unlockRequestTimestamp);
return this;
}
public Builder setCreationTimestamp(CreateAutoTimestamp creationTimestamp) {
getInstance().creationTimestamp = creationTimestamp;
public Builder setLockCompletionTimestamp(DateTime lockCompletionTimestamp) {
getInstance().lockCompletionTimestamp = toZonedDateTime(lockCompletionTimestamp);
return this;
}
public Builder setCompletionTimestamp(DateTime lockTimestamp) {
getInstance().completionTimestamp = toZonedDateTime(lockTimestamp);
public Builder setUnlockCompletionTimestamp(DateTime unlockCompletionTimestamp) {
getInstance().unlockCompletionTimestamp = toZonedDateTime(unlockCompletionTimestamp);
return this;
}
@@ -0,0 +1,43 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.tld;
import java.io.Serializable;
import java.math.BigDecimal;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.Id;
/**
* Entity class for the premium price of an individual domain label.
*
* <p>These are not persisted directly, but rather, using {@link PremiumList#getLabelsToPrices()}.
*/
@Entity
public class PremiumEntry implements Serializable {
@Id
@Column(nullable = false)
Long revisionId;
@Column(nullable = false)
BigDecimal price;
@Id
@Column(nullable = false)
String domainLabel;
private PremiumEntry() {}
}
@@ -23,6 +23,7 @@ import com.google.common.hash.BloomFilter;
import google.registry.model.CreateAutoTimestamp;
import java.math.BigDecimal;
import java.util.Map;
import javax.annotation.Nullable;
import javax.persistence.CollectionTable;
import javax.persistence.Column;
import javax.persistence.ElementCollection;
@@ -34,6 +35,7 @@ import javax.persistence.Index;
import javax.persistence.JoinColumn;
import javax.persistence.MapKeyColumn;
import javax.persistence.Table;
import org.hibernate.LazyInitializationException;
import org.joda.money.CurrencyUnit;
import org.joda.time.DateTime;
@@ -97,10 +99,16 @@ public class PremiumList {
return name;
}
/** Returns the {@link CurrencyUnit} used for this list. */
public CurrencyUnit getCurrency() {
return currency;
}
/** Returns the ID of this revision, or throws if null. */
public Long getRevisionId() {
checkState(
revisionId != null, "revisionId is null because it is not persisted in the database");
revisionId != null,
"revisionId is null because this object has not yet been persisted to the DB");
return revisionId;
}
@@ -109,9 +117,17 @@ public class PremiumList {
return creationTimestamp.getTimestamp();
}
/** Returns a {@link Map} of domain labels to prices. */
/**
* Returns a {@link Map} of domain labels to prices.
*
* <p>Note that this is lazily loaded and thus will throw a {@link LazyInitializationException} if
* used outside the transaction in which the given entity was loaded. You generally should not be
* using this anyway as it's inefficient to load all of the PremiumEntry rows if you don't need
* them. To check prices, use {@link PremiumListDao#getPremiumPrice} instead.
*/
@Nullable
public ImmutableMap<String, BigDecimal> getLabelsToPrices() {
return ImmutableMap.copyOf(labelsToPrices);
return labelsToPrices == null ? null : ImmutableMap.copyOf(labelsToPrices);
}
/**
@@ -0,0 +1,106 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.tld;
import static google.registry.config.RegistryConfig.getDomainLabelListCacheDuration;
import static google.registry.config.RegistryConfig.getSingletonCachePersistDuration;
import static google.registry.config.RegistryConfig.getStaticPremiumListMaxCachedEntries;
import static google.registry.schema.tld.PremiumListDao.getPriceForLabel;
import static java.util.concurrent.TimeUnit.MILLISECONDS;
import com.google.auto.value.AutoValue;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import google.registry.util.NonFinalForTesting;
import java.math.BigDecimal;
import java.util.Optional;
import org.joda.time.Duration;
/** Caching utils for {@link PremiumList}s. */
class PremiumListCache {
/**
* In-memory cache for premium lists.
*
* <p>This is cached for a shorter duration because we need to periodically reload from the DB to
* check if a new revision has been published, and if so, then use that.
*/
@NonFinalForTesting
static LoadingCache<String, Optional<PremiumList>> cachePremiumLists =
createCachePremiumLists(getDomainLabelListCacheDuration());
@VisibleForTesting
static LoadingCache<String, Optional<PremiumList>> createCachePremiumLists(
Duration cachePersistDuration) {
return CacheBuilder.newBuilder()
.expireAfterWrite(cachePersistDuration.getMillis(), MILLISECONDS)
.build(
new CacheLoader<String, Optional<PremiumList>>() {
@Override
public Optional<PremiumList> load(String premiumListName) {
return PremiumListDao.getLatestRevision(premiumListName);
}
});
}
/**
* In-memory price cache for for a given premium list revision and domain label.
*
* <p>Note that premium list revision ids are globally unique, so this cache is specific to a
* given premium list. Premium list entries might not be present, as indicated by the Optional
* wrapper, and we want to cache that as well.
*
* <p>This is cached for a long duration (essentially indefinitely) because premium list revisions
* are immutable and cannot ever be changed once created, so the cache need not ever expire.
*
* <p>A maximum size is set here on the cache because it can potentially grow too big to fit in
* memory if there are a large number of distinct premium list entries being queried (both those
* that exist, as well as those that might exist according to the Bloom filter, must be cached).
* The entries judged least likely to be accessed again will be evicted first.
*/
@NonFinalForTesting
static LoadingCache<RevisionIdAndLabel, Optional<BigDecimal>> cachePremiumEntries =
createCachePremiumEntries(getSingletonCachePersistDuration());
@VisibleForTesting
static LoadingCache<RevisionIdAndLabel, Optional<BigDecimal>> createCachePremiumEntries(
Duration cachePersistDuration) {
return CacheBuilder.newBuilder()
.expireAfterWrite(cachePersistDuration.getMillis(), MILLISECONDS)
.maximumSize(getStaticPremiumListMaxCachedEntries())
.build(
new CacheLoader<RevisionIdAndLabel, Optional<BigDecimal>>() {
@Override
public Optional<BigDecimal> load(RevisionIdAndLabel revisionIdAndLabel) {
return getPriceForLabel(revisionIdAndLabel);
}
});
}
@AutoValue
abstract static class RevisionIdAndLabel {
abstract long revisionId();
abstract String label();
static RevisionIdAndLabel create(long revisionId, String label) {
return new AutoValue_PremiumListCache_RevisionIdAndLabel(revisionId, label);
}
}
private PremiumListCache() {}
}
@@ -17,9 +17,37 @@ package google.registry.schema.tld;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.cache.CacheLoader.InvalidCacheLoadException;
import com.google.common.util.concurrent.UncheckedExecutionException;
import google.registry.model.registry.Registry;
import google.registry.schema.tld.PremiumListCache.RevisionIdAndLabel;
import java.math.BigDecimal;
import java.util.Optional;
import java.util.concurrent.ExecutionException;
import org.joda.money.Money;
/** Data access object class for {@link PremiumList}. */
public class PremiumListDao {
/**
* Returns the premium price for the specified label and registry, or absent if the label is not
* premium.
*/
public static Optional<Money> getPremiumPrice(String label, Registry registry) {
// If the registry has no configured premium list, then no labels are premium.
if (registry.getPremiumList() == null) {
return Optional.empty();
}
String premiumListName = registry.getPremiumList().getName();
PremiumList premiumList =
getLatestRevisionCached(premiumListName)
.orElseThrow(
() ->
new IllegalStateException(
String.format("Could not load premium list '%s'", premiumListName)));
return getPremiumPriceFromList(label, premiumList);
}
/** Persist a new premium list to Cloud SQL. */
public static void saveNew(PremiumList premiumList) {
jpaTm()
@@ -27,18 +55,85 @@ public class PremiumListDao {
() -> {
checkArgument(
!checkExists(premiumList.getName()),
"A premium list of this name already exists: %s.",
"Premium list '%s' already exists",
premiumList.getName());
jpaTm().getEntityManager().persist(premiumList);
});
}
/** Persist a new revision of an existing premium list to Cloud SQL. */
public static void update(PremiumList premiumList) {
jpaTm()
.transact(
() -> {
// This check is currently disabled because, during the Cloud SQL migration, we need
// to be able to update premium lists in Datastore while simultaneously creating their
// first revision in Cloud SQL (i.e. if they haven't been migrated over yet).
// TODO(b/147246613): Reinstate this once all premium lists are migrated to Cloud SQL,
// and re-enable the test update_throwsWhenListDoesntExist().
// checkArgument(
// checkExists(premiumList.getName()),
// "Can't update non-existent premium list '%s'",
// premiumList.getName());
jpaTm().getEntityManager().persist(premiumList);
});
}
/**
* Returns the most recent revision of the PremiumList with the specified name, if it exists.
*
* <p>Note that this does not load <code>PremiumList.labelsToPrices</code>! If you need to check
* prices, use {@link #getPremiumPrice}.
*/
public static Optional<PremiumList> getLatestRevision(String premiumListName) {
return jpaTm()
.transact(
() ->
jpaTm()
.getEntityManager()
.createQuery(
"SELECT pl FROM PremiumList pl WHERE pl.name = :name ORDER BY"
+ " pl.revisionId DESC",
PremiumList.class)
.setParameter("name", premiumListName)
.setMaxResults(1)
.getResultStream()
.findFirst());
}
static Optional<BigDecimal> getPriceForLabel(RevisionIdAndLabel revisionIdAndLabel) {
return jpaTm()
.transact(
() ->
jpaTm()
.getEntityManager()
.createQuery(
"SELECT pe.price FROM PremiumEntry pe WHERE pe.revisionId = :revisionId"
+ " AND pe.domainLabel = :label",
BigDecimal.class)
.setParameter("revisionId", revisionIdAndLabel.revisionId())
.setParameter("label", revisionIdAndLabel.label())
.setMaxResults(1)
.getResultStream()
.findFirst());
}
/** Returns the most recent revision of the PremiumList with the specified name, from cache. */
static Optional<PremiumList> getLatestRevisionCached(String premiumListName) {
try {
return PremiumListCache.cachePremiumLists.get(premiumListName);
} catch (ExecutionException e) {
throw new UncheckedExecutionException(
"Could not retrieve premium list named " + premiumListName, e);
}
}
/**
* Returns whether the premium list of the given name exists.
*
* <p>This means that at least one premium list revision must exist for the given name.
*/
public static boolean checkExists(String premiumListName) {
static boolean checkExists(String premiumListName) {
return jpaTm()
.transact(
() ->
@@ -52,5 +147,24 @@ public class PremiumListDao {
> 0);
}
private static Optional<Money> getPremiumPriceFromList(String label, PremiumList premiumList) {
// Consult the bloom filter and immediately return if the label definitely isn't premium.
if (!premiumList.getBloomFilter().mightContain(label)) {
return Optional.empty();
}
RevisionIdAndLabel revisionIdAndLabel =
RevisionIdAndLabel.create(premiumList.getRevisionId(), label);
try {
Optional<BigDecimal> price = PremiumListCache.cachePremiumEntries.get(revisionIdAndLabel);
return price.map(p -> Money.of(premiumList.getCurrency(), p));
} catch (InvalidCacheLoadException | ExecutionException e) {
throw new RuntimeException(
String.format(
"Could not load premium entry %s for list %s",
revisionIdAndLabel, premiumList.getName()),
e);
}
}
private PremiumListDao() {}
}
@@ -0,0 +1,61 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.tld;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Iterables;
import com.google.common.collect.Maps;
import google.registry.model.registry.label.PremiumList.PremiumListEntry;
import java.math.BigDecimal;
import java.util.List;
import java.util.Map;
import org.joda.money.CurrencyUnit;
/** Static utility methods for {@link PremiumList}. */
public class PremiumListUtils {
public static PremiumList parseToPremiumList(String name, String inputData) {
List<String> inputDataPreProcessed =
Splitter.on('\n').omitEmptyStrings().splitToList(inputData);
ImmutableMap<String, PremiumListEntry> prices =
new google.registry.model.registry.label.PremiumList.Builder()
.setName(name)
.build()
.parse(inputDataPreProcessed);
ImmutableSet<CurrencyUnit> currencies =
prices.values().stream()
.map(e -> e.getValue().getCurrencyUnit())
.distinct()
.collect(toImmutableSet());
checkArgument(
currencies.size() == 1,
"The Cloud SQL schema requires exactly one currency, but got: %s",
ImmutableSortedSet.copyOf(currencies));
CurrencyUnit currency = Iterables.getOnlyElement(currencies);
Map<String, BigDecimal> priceAmounts =
Maps.transformValues(prices, ple -> ple.getValue().getAmount());
return google.registry.schema.tld.PremiumList.create(name, currency, priceAmounts);
}
private PremiumListUtils() {}
}
@@ -14,13 +14,20 @@
package google.registry.schema.tld;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.base.Preconditions.checkState;
import static com.google.common.collect.ImmutableList.sortedCopyOf;
import static com.google.common.collect.ImmutableList.toImmutableList;
import static google.registry.util.DomainNameUtils.canonicalizeDomainName;
import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.ImmutableObject;
import google.registry.model.registry.label.ReservationType;
import java.util.Map;
import java.util.stream.Stream;
import javax.annotation.Nullable;
import javax.persistence.CollectionTable;
import javax.persistence.Column;
@@ -114,6 +121,22 @@ public class ReservedList extends ImmutableObject {
/** Constructs a {@link ReservedList} object. */
public static ReservedList create(
String name, Boolean shouldPublish, Map<String, ReservedEntry> labelsToReservations) {
ImmutableList<String> invalidLabels =
labelsToReservations.entrySet().parallelStream()
.flatMap(
entry -> {
String label = entry.getKey();
if (label.equals(canonicalizeDomainName(label))) {
return Stream.empty();
} else {
return Stream.of(label);
}
})
.collect(toImmutableList());
checkArgument(
invalidLabels.isEmpty(),
"Label(s) [%s] must be in puny-coded, lower-case form",
Joiner.on(",").join(sortedCopyOf(invalidLabels)));
return new ReservedList(name, shouldPublish, labelsToReservations);
}
@@ -0,0 +1,47 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.tld;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
/** Data access object class for {@link ReservedList} */
public class ReservedListDao {
/** Persist a new reserved list to Cloud SQL. */
public static void save(ReservedList reservedList) {
jpaTm().transact(() -> jpaTm().getEntityManager().persist(reservedList));
}
/**
* Returns whether the reserved list of the given name exists.
*
* <p>This means that at least one reserved list revision must exist for the given name.
*/
public static boolean checkExists(String reservedListName) {
return jpaTm()
.transact(
() ->
jpaTm()
.getEntityManager()
.createQuery("SELECT 1 FROM ReservedList WHERE name = :name", Integer.class)
.setParameter("name", reservedListName)
.setMaxResults(1)
.getResultList()
.size()
> 0);
}
private ReservedListDao() {}
}
@@ -12,12 +12,16 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.tmch;
package google.registry.schema.tmch;
import static google.registry.config.RegistryConfig.getDomainLabelListCacheDuration;
import static google.registry.model.CacheUtils.tryMemoizeWithExpiration;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.base.Supplier;
import com.google.common.flogger.FluentLogger;
import google.registry.schema.tmch.ClaimsList;
import google.registry.util.NonFinalForTesting;
import java.util.Optional;
import javax.persistence.EntityManager;
/** Data access object for {@link ClaimsList}. */
@@ -25,6 +29,11 @@ public class ClaimsListDao {
private static final FluentLogger logger = FluentLogger.forEnclosingClass();
/** In-memory cache for claims list. */
@NonFinalForTesting
private static Supplier<Optional<ClaimsList>> cacheClaimsList =
tryMemoizeWithExpiration(getDomainLabelListCacheDuration(), ClaimsListDao::getLatestRevision);
private static void save(ClaimsList claimsList) {
jpaTm().transact(() -> jpaTm().getEntityManager().persist(claimsList));
}
@@ -48,10 +57,12 @@ public class ClaimsListDao {
}
/**
* Returns the current revision of the {@link ClaimsList} in Cloud SQL. Throws exception if there
* is no claims in the table.
* Returns the most recent revision of the {@link ClaimsList} in Cloud SQL, if it exists.
* TODO(shicong): Change this method to package level access after dual-read phase.
* ClaimsListShard uses this method to retrieve claims list in Cloud SQL for the comparison, and
* ClaimsListShard is not in this package.
*/
public static ClaimsList getCurrent() {
public static Optional<ClaimsList> getLatestRevision() {
return jpaTm()
.transact(
() -> {
@@ -64,9 +75,15 @@ public class ClaimsListDao {
+ " :revisionId",
ClaimsList.class)
.setParameter("revisionId", revisionId)
.getSingleResult();
.getResultStream()
.findFirst();
});
}
/** Returns the most recent revision of the {@link ClaimsList}, from cache. */
public static Optional<ClaimsList> getLatestRevisionCached() {
return cacheClaimsList.get();
}
private ClaimsListDao() {}
}
@@ -18,11 +18,11 @@ import static google.registry.request.Action.Method.POST;
import com.google.common.flogger.FluentLogger;
import google.registry.keyring.api.KeyModule.Key;
import google.registry.model.tmch.ClaimsListDao;
import google.registry.model.tmch.ClaimsListShard;
import google.registry.request.Action;
import google.registry.request.auth.Auth;
import google.registry.schema.tmch.ClaimsList;
import google.registry.schema.tmch.ClaimsListDao;
import java.io.IOException;
import java.security.SignatureException;
import java.util.List;
@@ -16,7 +16,6 @@ package google.registry.tools;
import static com.google.common.base.Strings.isNullOrEmpty;
import static google.registry.security.JsonHttp.JSON_SAFETY_PREFIX;
import static google.registry.tools.server.CreateOrUpdatePremiumListAction.ALSO_CLOUD_SQL_PARAM;
import static google.registry.tools.server.CreateOrUpdatePremiumListAction.INPUT_PARAM;
import static google.registry.tools.server.CreateOrUpdatePremiumListAction.NAME_PARAM;
import static google.registry.util.ListNamingUtils.convertFilePathToName;
@@ -58,12 +57,6 @@ abstract class CreateOrUpdatePremiumListCommand extends ConfirmingCommand
required = true)
Path inputFile;
@Parameter(
names = {"--also_cloud_sql"},
description =
"Persist premium list to Cloud SQL in addition to Datastore; defaults to false.")
boolean alsoCloudSql;
protected AppEngineConnection connection;
protected int inputLineCount;
@@ -97,7 +90,6 @@ abstract class CreateOrUpdatePremiumListCommand extends ConfirmingCommand
public String execute() throws Exception {
ImmutableMap.Builder<String, String> params = new ImmutableMap.Builder<>();
params.put(NAME_PARAM, name);
params.put(ALSO_CLOUD_SQL_PARAM, Boolean.toString(alsoCloudSql));
String inputFileContents = new String(Files.readAllBytes(inputFile), UTF_8);
String requestBody =
Joiner.on('&').withKeyValueSeparator("=").join(
@@ -14,9 +14,23 @@
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.registry.label.BaseDomainLabelList.splitOnComment;
import static google.registry.util.DomainNameUtils.canonicalizeDomainName;
import com.beust.jcommander.Parameter;
import com.google.common.base.Splitter;
import com.google.common.collect.HashMultiset;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Maps;
import com.google.common.collect.Multiset;
import com.google.common.flogger.FluentLogger;
import google.registry.model.registry.label.ReservationType;
import google.registry.schema.tld.ReservedList.ReservedEntry;
import google.registry.tools.params.PathParameter;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import javax.annotation.Nullable;
/**
@@ -25,6 +39,8 @@ import javax.annotation.Nullable;
*/
public abstract class CreateOrUpdateReservedListCommand extends MutatingCommand {
static final FluentLogger logger = FluentLogger.forEnclosingClass();
@Nullable
@Parameter(
names = {"-n", "--name"},
@@ -45,4 +61,81 @@ public abstract class CreateOrUpdateReservedListCommand extends MutatingCommand
"Whether the list is published to the concatenated list on Drive (defaults to true).",
arity = 1)
Boolean shouldPublish;
@Parameter(
names = {"--also_cloud_sql"},
description =
"Persist reserved list to Cloud SQL in addition to Datastore; defaults to false.")
boolean alsoCloudSql;
google.registry.schema.tld.ReservedList cloudSqlReservedList;
abstract void saveToCloudSql();
@Override
protected String execute() throws Exception {
// Save the list to Datastore and output its response.
String output = super.execute();
logger.atInfo().log(output);
String cloudSqlMessage;
if (alsoCloudSql) {
cloudSqlMessage =
String.format(
"Saved reserved list %s with %d entries",
name, cloudSqlReservedList.getLabelsToReservations().size());
try {
logger.atInfo().log("Saving reserved list to Cloud SQL for TLD %s", name);
saveToCloudSql();
logger.atInfo().log(cloudSqlMessage);
} catch (Throwable e) {
cloudSqlMessage =
"Unexpected error saving reserved list to Cloud SQL from nomulus tool command";
logger.atSevere().withCause(e).log(cloudSqlMessage);
}
} else {
cloudSqlMessage = "Persisting reserved list to Cloud SQL is not enabled";
}
return cloudSqlMessage;
}
/** Turns the list CSV data into a map of labels to {@link ReservedEntry}. */
static ImmutableMap<String, ReservedEntry> parseToReservationsByLabels(Iterable<String> lines) {
Map<String, ReservedEntry> labelsToEntries = Maps.newHashMap();
Multiset<String> duplicateLabels = HashMultiset.create();
for (String originalLine : lines) {
List<String> lineAndComment = splitOnComment(originalLine);
if (lineAndComment.isEmpty()) {
continue;
}
String line = lineAndComment.get(0);
String comment = lineAndComment.get(1);
List<String> parts = Splitter.on(',').trimResults().splitToList(line);
checkArgument(
parts.size() == 2 || parts.size() == 3,
"Could not parse line in reserved list: %s",
originalLine);
String label = parts.get(0);
checkArgument(
label.equals(canonicalizeDomainName(label)),
"Label '%s' must be in puny-coded, lower-case form",
label);
ReservationType reservationType = ReservationType.valueOf(parts.get(1));
ReservedEntry reservedEntry = ReservedEntry.create(reservationType, comment);
// Check if the label was already processed for this list (which is an error), and if so,
// accumulate it so that a list of all duplicates can be thrown.
if (labelsToEntries.containsKey(label)) {
duplicateLabels.add(label, duplicateLabels.contains(label) ? 1 : 2);
} else {
labelsToEntries.put(label, reservedEntry);
}
}
if (!duplicateLabels.isEmpty()) {
throw new IllegalStateException(
String.format(
"Reserved list cannot contain duplicate labels. Dupes (with counts) were: %s",
duplicateLabels));
}
return ImmutableMap.copyOf(labelsToEntries);
}
}
@@ -16,6 +16,7 @@ package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.registry.Registries.assertTldExists;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.util.ListNamingUtils.convertFilePathToName;
import static java.nio.charset.StandardCharsets.UTF_8;
import static org.joda.time.DateTimeZone.UTC;
@@ -26,6 +27,7 @@ import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import google.registry.model.registry.label.ReservedList;
import google.registry.schema.tld.ReservedListDao;
import java.nio.file.Files;
import java.util.List;
import org.joda.time.DateTime;
@@ -54,15 +56,35 @@ final class CreateReservedListCommand extends CreateOrUpdateReservedListCommand
validateListName(name);
}
DateTime now = DateTime.now(UTC);
List<String> allLines = Files.readAllLines(input, UTF_8);
boolean shouldPublish = this.shouldPublish == null || this.shouldPublish;
ReservedList reservedList =
new ReservedList.Builder()
.setName(name)
.setReservedListMapFromLines(Files.readAllLines(input, UTF_8))
.setShouldPublish(shouldPublish == null || shouldPublish)
.setReservedListMapFromLines(allLines)
.setShouldPublish(shouldPublish)
.setCreationTime(now)
.setLastUpdateTime(now)
.build();
stageEntityChange(null, reservedList);
if (alsoCloudSql) {
cloudSqlReservedList =
google.registry.schema.tld.ReservedList.create(
name, shouldPublish, parseToReservationsByLabels(allLines));
}
}
@Override
void saveToCloudSql() {
jpaTm()
.transact(
() -> {
checkArgument(
!ReservedListDao.checkExists(cloudSqlReservedList.getName()),
"A reserved list of this name already exists: %s.",
cloudSqlReservedList.getName());
ReservedListDao.save(cloudSqlReservedList);
});
}
private static void validateListName(String name) {
@@ -104,6 +104,8 @@ final class RegistryCli implements AutoCloseable, CommandRunner {
// Create all command instances. It would be preferrable to do this in the constructor, but
// JCommander mutates the command instances and doesn't reset them so we have to do it for every
// run.
// TODO(weiminyu): extract this into a standalone static method to simplify
// :core:registryToolIntegrationTest
try {
for (Map.Entry<String, ? extends Class<? extends Command>> entry : commands.entrySet()) {
Command command = entry.getValue().getDeclaredConstructor().newInstance();
@@ -60,7 +60,6 @@ public final class RegistryTool {
.put("generate_dns_report", GenerateDnsReportCommand.class)
.put("generate_escrow_deposit", GenerateEscrowDepositCommand.class)
.put("generate_lordn", GenerateLordnCommand.class)
.put("generate_sql_schema", GenerateSqlSchemaCommand.class)
.put("generate_zone_files", GenerateZoneFilesCommand.class)
.put("get_allocation_token", GetAllocationTokenCommand.class)
.put("get_claims_list", GetClaimsListCommand.class)
@@ -15,14 +15,17 @@
package google.registry.tools;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.util.ListNamingUtils.convertFilePathToName;
import static java.nio.charset.StandardCharsets.UTF_8;
import com.beust.jcommander.Parameters;
import com.google.common.base.Strings;
import google.registry.model.registry.label.ReservedList;
import google.registry.schema.tld.ReservedListDao;
import google.registry.util.SystemClock;
import java.nio.file.Files;
import java.util.List;
import java.util.Optional;
/** Command to safely update {@link ReservedList} on Datastore. */
@@ -32,18 +35,38 @@ final class UpdateReservedListCommand extends CreateOrUpdateReservedListCommand
@Override
protected void init() throws Exception {
name = Strings.isNullOrEmpty(name) ? convertFilePathToName(input) : name;
// TODO(shicong): Read existing entry from Cloud SQL
Optional<ReservedList> existing = ReservedList.get(name);
checkArgument(
existing.isPresent(), "Could not update reserved list %s because it doesn't exist.", name);
boolean shouldPublish =
this.shouldPublish == null ? existing.get().getShouldPublish() : this.shouldPublish;
List<String> allLines = Files.readAllLines(input, UTF_8);
ReservedList.Builder updated =
existing
.get()
.asBuilder()
.setReservedListMapFromLines(Files.readAllLines(input, UTF_8))
.setLastUpdateTime(new SystemClock().nowUtc());
if (shouldPublish != null) {
updated.setShouldPublish(shouldPublish);
}
.setReservedListMapFromLines(allLines)
.setLastUpdateTime(new SystemClock().nowUtc())
.setShouldPublish(shouldPublish);
stageEntityChange(existing.get(), updated.build());
if (alsoCloudSql) {
cloudSqlReservedList =
google.registry.schema.tld.ReservedList.create(
name, shouldPublish, parseToReservationsByLabels(allLines));
}
}
@Override
void saveToCloudSql() {
jpaTm()
.transact(
() -> {
checkArgument(
ReservedListDao.checkExists(cloudSqlReservedList.getName()),
"A reserved list of this name doesn't exist: %s.",
cloudSqlReservedList.getName());
ReservedListDao.save(cloudSqlReservedList);
});
}
}
@@ -21,9 +21,9 @@ import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.base.Joiner;
import com.google.common.io.Files;
import google.registry.model.tmch.ClaimsListDao;
import google.registry.model.tmch.ClaimsListShard;
import google.registry.schema.tmch.ClaimsList;
import google.registry.schema.tmch.ClaimsListDao;
import google.registry.tmch.ClaimsListParser;
import java.io.File;
import java.io.IOException;
@@ -14,26 +14,13 @@
package google.registry.tools.server;
import static com.google.common.base.Preconditions.checkArgument;
import static com.google.common.collect.ImmutableSet.toImmutableSet;
import static com.google.common.flogger.LazyArgs.lazy;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Iterables;
import com.google.common.collect.Maps;
import com.google.common.flogger.FluentLogger;
import google.registry.model.registry.label.PremiumList;
import google.registry.model.registry.label.PremiumList.PremiumListEntry;
import google.registry.request.JsonResponse;
import google.registry.request.Parameter;
import java.math.BigDecimal;
import java.util.List;
import java.util.Map;
import javax.inject.Inject;
import org.joda.money.CurrencyUnit;
/** Abstract base class for actions that update premium lists. */
public abstract class CreateOrUpdatePremiumListAction implements Runnable {
@@ -44,7 +31,6 @@ public abstract class CreateOrUpdatePremiumListAction implements Runnable {
public static final String NAME_PARAM = "name";
public static final String INPUT_PARAM = "inputData";
public static final String ALSO_CLOUD_SQL_PARAM = "alsoCloudSql";
@Inject JsonResponse response;
@@ -56,10 +42,6 @@ public abstract class CreateOrUpdatePremiumListAction implements Runnable {
@Parameter(INPUT_PARAM)
String inputData;
@Inject
@Parameter(ALSO_CLOUD_SQL_PARAM)
boolean alsoCloudSql;
@Override
public void run() {
try {
@@ -76,40 +58,15 @@ public abstract class CreateOrUpdatePremiumListAction implements Runnable {
return;
}
if (alsoCloudSql) {
try {
saveToCloudSql();
} catch (Throwable e) {
logger.atSevere().withCause(e).log(
"Unexpected error saving premium list to Cloud SQL from nomulus tool command");
response.setPayload(ImmutableMap.of("error", e.toString(), "status", "error"));
return;
}
try {
saveToCloudSql();
} catch (Throwable e) {
logger.atSevere().withCause(e).log(
"Unexpected error saving premium list to Cloud SQL from nomulus tool command");
response.setPayload(ImmutableMap.of("error", e.toString(), "status", "error"));
}
}
google.registry.schema.tld.PremiumList parseInputToPremiumList() {
List<String> inputDataPreProcessed =
Splitter.on('\n').omitEmptyStrings().splitToList(inputData);
ImmutableMap<String, PremiumListEntry> prices =
new PremiumList.Builder().setName(name).build().parse(inputDataPreProcessed);
ImmutableSet<CurrencyUnit> currencies =
prices.values().stream()
.map(e -> e.getValue().getCurrencyUnit())
.distinct()
.collect(toImmutableSet());
checkArgument(
currencies.size() == 1,
"The Cloud SQL schema requires exactly one currency, but got: %s",
ImmutableSortedSet.copyOf(currencies));
CurrencyUnit currency = Iterables.getOnlyElement(currencies);
Map<String, BigDecimal> priceAmounts =
Maps.transformValues(prices, ple -> ple.getValue().getAmount());
return google.registry.schema.tld.PremiumList.create(name, currency, priceAmounts);
}
/** Logs the premium list data at INFO, truncated if too long. */
void logInputData() {
logger.atInfo().log(
@@ -19,6 +19,7 @@ import static google.registry.model.registry.Registries.assertTldExists;
import static google.registry.model.registry.label.PremiumListUtils.doesPremiumListExist;
import static google.registry.model.registry.label.PremiumListUtils.savePremiumListAndEntries;
import static google.registry.request.Action.Method.POST;
import static google.registry.schema.tld.PremiumListUtils.parseToPremiumList;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableMap;
@@ -81,7 +82,7 @@ public class CreatePremiumListAction extends CreateOrUpdatePremiumListAction {
logger.atInfo().log("Saving premium list to Cloud SQL for TLD %s", name);
// TODO(mcilwain): Call logInputData() here once Datastore persistence is removed.
google.registry.schema.tld.PremiumList premiumList = parseInputToPremiumList();
google.registry.schema.tld.PremiumList premiumList = parseToPremiumList(name, inputData);
PremiumListDao.saveNew(premiumList);
String message =
@@ -60,12 +60,6 @@ public class ToolsServerModule {
return extractRequiredParameter(req, CreatePremiumListAction.INPUT_PARAM);
}
@Provides
@Parameter("alsoCloudSql")
static boolean provideAlsoCloudSql(HttpServletRequest req) {
return extractBooleanParameter(req, CreatePremiumListAction.ALSO_CLOUD_SQL_PARAM);
}
@Provides
@Parameter("premiumListName")
static String provideName(HttpServletRequest req) {
@@ -17,6 +17,7 @@ package google.registry.tools.server;
import static com.google.common.base.Preconditions.checkArgument;
import static google.registry.model.registry.label.PremiumListUtils.savePremiumListAndEntries;
import static google.registry.request.Action.Method.POST;
import static google.registry.schema.tld.PremiumListUtils.parseToPremiumList;
import com.google.common.base.Splitter;
import com.google.common.collect.ImmutableMap;
@@ -24,6 +25,7 @@ import com.google.common.flogger.FluentLogger;
import google.registry.model.registry.label.PremiumList;
import google.registry.request.Action;
import google.registry.request.auth.Auth;
import google.registry.schema.tld.PremiumListDao;
import java.util.List;
import java.util.Optional;
import javax.inject.Inject;
@@ -68,10 +70,17 @@ public class UpdatePremiumListAction extends CreateOrUpdatePremiumListAction {
response.setPayload(ImmutableMap.of("status", "success", "message", message));
}
// TODO(mcilwain): Implement this in a subsequent PR.
@Override
protected void saveToCloudSql() {
throw new UnsupportedOperationException(
"Updating of premium lists in Cloud SQL is not supported yet");
logger.atInfo().log("Updating premium list '%s' in Cloud SQL.", name);
// TODO(mcilwain): Add logInputData() call here once DB migration is complete.
google.registry.schema.tld.PremiumList premiumList = parseToPremiumList(name, inputData);
PremiumListDao.update(premiumList);
String message =
String.format(
"Updated premium list '%s' with %d entries.",
premiumList.getName(), premiumList.getLabelsToPrices().size());
logger.atInfo().log(message);
// TODO(mcilwain): Call response.setPayload() here once DB migration is complete.
}
}
@@ -149,11 +149,9 @@ public final class RegistryLockGetAction implements JsonGetAction {
}
private ImmutableList<ImmutableMap<String, ?>> getLockedDomains(String clientId) {
ImmutableList<RegistryLock> locks =
RegistryLockDao.getByRegistrarId(clientId).stream()
.filter(RegistryLock::isVerified)
.collect(toImmutableList());
return locks.stream().map(this::lockToMap).collect(toImmutableList());
return RegistryLockDao.getLockedDomainsByRegistrarId(clientId).stream()
.map(this::lockToMap)
.collect(toImmutableList());
}
private ImmutableMap<String, ?> lockToMap(RegistryLock lock) {
@@ -161,7 +159,7 @@ public final class RegistryLockGetAction implements JsonGetAction {
FULLY_QUALIFIED_DOMAIN_NAME_PARAM,
lock.getDomainName(),
LOCKED_TIME_PARAM,
lock.getCompletionTimestamp().map(DateTime::toString).orElse(""),
lock.getLockCompletionTimestamp().map(DateTime::toString).orElse(""),
LOCKED_BY_PARAM,
lock.isSuperuser() ? "admin" : lock.getRegistrarPocId());
}
@@ -22,8 +22,10 @@
<class>google.registry.model.domain.DomainBase</class>
<class>google.registry.schema.domain.RegistryLock</class>
<class>google.registry.schema.tmch.ClaimsList</class>
<class>google.registry.schema.cursor.Cursor</class>
<class>google.registry.model.transfer.BaseTransferObject</class>
<class>google.registry.schema.tld.PremiumList</class>
<class>google.registry.schema.tld.PremiumEntry</class>
<class>google.registry.schema.tld.ReservedList</class>
<class>google.registry.model.domain.secdns.DelegationSignerData</class>
<class>google.registry.model.domain.DesignatedContact</class>
@@ -0,0 +1,37 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools;
import com.google.common.collect.ImmutableMap;
/** Entry point of Nomulus development commands. */
public class DevTool {
/**
* Available commands.
*
* <p><b>Note:</b> If changing the command-line name of any commands below, remember to resolve
* any invocations in scripts (e.g. PDT, ICANN reporting).
*/
public static final ImmutableMap<String, Class<? extends Command>> COMMAND_MAP =
ImmutableMap.of("generate_sql_schema", GenerateSqlSchemaCommand.class);
public static void main(String[] args) throws Exception {
RegistryToolEnvironment.parseFromArgs(args).setup();
try (RegistryCli cli = new RegistryCli("devtool", COMMAND_MAP)) {
cli.run(args);
}
}
}
@@ -19,9 +19,9 @@ import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.testing.JUnitBackports.assertThrows;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.schema.domain.RegistryLock;
import google.registry.schema.domain.RegistryLock.Action;
import google.registry.testing.AppEngineRule;
import java.util.Optional;
import java.util.UUID;
@@ -37,8 +37,8 @@ public final class RegistryLockDaoTest {
@Rule public final AppEngineRule appEngine = AppEngineRule.builder().withDatastore().build();
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder().build();
public final JpaIntegrationTestRule jpaRule =
new JpaTestRules.Builder().buildIntegrationTestRule();
@Test
public void testSaveAndLoad_success() {
@@ -47,6 +47,7 @@ public final class RegistryLockDaoTest {
RegistryLock fromDatabase = RegistryLockDao.getByVerificationCode(lock.getVerificationCode());
assertThat(fromDatabase.getDomainName()).isEqualTo(lock.getDomainName());
assertThat(fromDatabase.getVerificationCode()).isEqualTo(lock.getVerificationCode());
assertThat(fromDatabase.getLastUpdateTimestamp()).isEqualTo(jpaRule.getTxnClock().nowUtc());
}
@Test
@@ -64,39 +65,63 @@ public final class RegistryLockDaoTest {
public void testSaveTwiceAndLoad_returnsLatest() {
RegistryLock lock = createLock();
jpaTm().transact(() -> RegistryLockDao.save(lock));
jpaTmRule.getTxnClock().advanceOneMilli();
jpaRule.getTxnClock().advanceOneMilli();
jpaTm()
.transact(
() -> {
RegistryLock updatedLock =
RegistryLockDao.getByVerificationCode(lock.getVerificationCode());
updatedLock.setCompletionTimestamp(jpaTmRule.getTxnClock().nowUtc());
RegistryLockDao.save(updatedLock);
RegistryLockDao.save(
updatedLock
.asBuilder()
.setLockCompletionTimestamp(jpaRule.getTxnClock().nowUtc())
.build());
});
jpaTm()
.transact(
() -> {
RegistryLock fromDatabase =
RegistryLockDao.getByVerificationCode(lock.getVerificationCode());
assertThat(fromDatabase.getCompletionTimestamp().get())
.isEqualTo(jpaTmRule.getTxnClock().nowUtc());
assertThat(fromDatabase.getLockCompletionTimestamp().get())
.isEqualTo(jpaRule.getTxnClock().nowUtc());
assertThat(fromDatabase.getLastUpdateTimestamp())
.isEqualTo(jpaRule.getTxnClock().nowUtc());
});
}
@Test
public void testSave_load_withUnlock() {
RegistryLock lock =
RegistryLockDao.save(
createLock()
.asBuilder()
.setLockCompletionTimestamp(jpaRule.getTxnClock().nowUtc())
.setUnlockRequestTimestamp(jpaRule.getTxnClock().nowUtc())
.setUnlockCompletionTimestamp(jpaRule.getTxnClock().nowUtc())
.build());
RegistryLockDao.save(lock);
RegistryLock fromDatabase = RegistryLockDao.getByVerificationCode(lock.getVerificationCode());
assertThat(fromDatabase.getUnlockRequestTimestamp())
.isEqualTo(Optional.of(jpaRule.getTxnClock().nowUtc()));
assertThat(fromDatabase.getUnlockCompletionTimestamp())
.isEqualTo(Optional.of(jpaRule.getTxnClock().nowUtc()));
assertThat(fromDatabase.isLocked()).isFalse();
}
@Test
public void testUpdateLock_usingSamePrimaryKey() {
RegistryLock lock = RegistryLockDao.save(createLock());
jpaTmRule.getTxnClock().advanceOneMilli();
jpaRule.getTxnClock().advanceOneMilli();
RegistryLock updatedLock =
lock.asBuilder().setCompletionTimestamp(jpaTmRule.getTxnClock().nowUtc()).build();
lock.asBuilder().setLockCompletionTimestamp(jpaRule.getTxnClock().nowUtc()).build();
jpaTm().transact(() -> RegistryLockDao.save(updatedLock));
jpaTm()
.transact(
() -> {
RegistryLock fromDatabase =
RegistryLockDao.getByVerificationCode(lock.getVerificationCode());
assertThat(fromDatabase.getCompletionTimestamp())
.isEqualTo(Optional.of(jpaTmRule.getTxnClock().nowUtc()));
assertThat(fromDatabase.getLockCompletionTimestamp())
.isEqualTo(Optional.of(jpaRule.getTxnClock().nowUtc()));
});
}
@@ -106,33 +131,48 @@ public final class RegistryLockDaoTest {
}
@Test
public void testLoad_byRegistrarId() {
RegistryLock lock = createLock();
RegistryLock secondLock = createLock().asBuilder().setDomainName("otherexample.test").build();
public void testLoad_lockedDomains_byRegistrarId() {
RegistryLock lock =
createLock().asBuilder().setLockCompletionTimestamp(jpaRule.getTxnClock().nowUtc()).build();
RegistryLock secondLock =
createLock()
.asBuilder()
.setDomainName("otherexample.test")
.setLockCompletionTimestamp(jpaRule.getTxnClock().nowUtc())
.build();
RegistryLock unlockedLock =
createLock()
.asBuilder()
.setDomainName("unlocked.test")
.setLockCompletionTimestamp(jpaRule.getTxnClock().nowUtc())
.setUnlockRequestTimestamp(jpaRule.getTxnClock().nowUtc())
.setUnlockCompletionTimestamp(jpaRule.getTxnClock().nowUtc())
.build();
RegistryLockDao.save(lock);
RegistryLockDao.save(secondLock);
RegistryLockDao.save(unlockedLock);
assertThat(
RegistryLockDao.getByRegistrarId("TheRegistrar").stream()
RegistryLockDao.getLockedDomainsByRegistrarId("TheRegistrar").stream()
.map(RegistryLock::getDomainName)
.collect(toImmutableSet()))
.containsExactly("example.test", "otherexample.test");
assertThat(RegistryLockDao.getByRegistrarId("nonexistent")).isEmpty();
assertThat(RegistryLockDao.getLockedDomainsByRegistrarId("nonexistent")).isEmpty();
}
@Test
public void testLoad_byRepoId() {
RegistryLock completedLock =
createLock().asBuilder().setCompletionTimestamp(jpaTmRule.getTxnClock().nowUtc()).build();
createLock().asBuilder().setLockCompletionTimestamp(jpaRule.getTxnClock().nowUtc()).build();
RegistryLockDao.save(completedLock);
jpaTmRule.getTxnClock().advanceOneMilli();
jpaRule.getTxnClock().advanceOneMilli();
RegistryLock inProgressLock = createLock();
RegistryLockDao.save(inProgressLock);
Optional<RegistryLock> mostRecent = RegistryLockDao.getMostRecentByRepoId("repoId");
assertThat(mostRecent.isPresent()).isTrue();
assertThat(mostRecent.get().isVerified()).isFalse();
assertThat(mostRecent.get().isLocked()).isFalse();
}
@Test
@@ -145,7 +185,6 @@ public final class RegistryLockDaoTest {
.setRepoId("repoId")
.setDomainName("example.test")
.setRegistrarId("TheRegistrar")
.setAction(Action.LOCK)
.setVerificationCode(UUID.randomUUID().toString())
.isSuperuser(true)
.build();
@@ -0,0 +1,109 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.transaction;
import static com.google.common.base.Preconditions.checkState;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
/**
* Holds specialized JUnit rules that start a test database server and provide {@link
* JpaTransactionManager} instances.
*/
public class JpaTestRules {
private static final String GOLDEN_SCHEMA_SQL_PATH = "sql/schema/nomulus.golden.sql";
/**
* Junit rule for integration tests with JPA framework, when the underlying database is populated
* with the Nomulus Cloud SQL schema.
*
* <p>Test classes that instantiate this class should be included in {@link
* google.registry.schema.integration.SqlIntegrationTestSuite}. This enforced by {@link
* google.registry.schema.integration.SqlIntegrationMembershipTest}.
*/
public static class JpaIntegrationTestRule extends JpaTransactionManagerRule {
private JpaIntegrationTestRule(
ImmutableList<Class> extraEntityClasses, ImmutableMap<String, String> userProperties) {
super(Optional.of(GOLDEN_SCHEMA_SQL_PATH), extraEntityClasses, userProperties);
}
}
/**
* Junit rule for unit tests with JPA framework, when the underlying database is populated by the
* optional init script.
*/
public static class JpaUnitTestRule extends JpaTransactionManagerRule {
private JpaUnitTestRule(
Optional<String> initScriptPath,
ImmutableList<Class> extraEntityClasses,
ImmutableMap<String, String> userProperties) {
super(initScriptPath, extraEntityClasses, userProperties);
}
}
/** Builder of test rules that provide {@link JpaTransactionManager}. */
public static class Builder {
private String initScript;
private List<Class> extraEntityClasses = new ArrayList<Class>();
private Map<String, String> userProperties = new HashMap<String, String>();
/**
* Sets the SQL script to be used to initialize the database. If not set,
* sql/schema/nomulus.golden.sql will be used.
*
* <p>The {@code initScript} is only accepted when building {@link JpaUnitTestRule}.
*/
public Builder withInitScript(String initScript) {
this.initScript = initScript;
return this;
}
/** Adds annotated class(es) to the known entities for the database. */
public Builder withEntityClass(Class... classes) {
this.extraEntityClasses.addAll(ImmutableSet.copyOf(classes));
return this;
}
/** Adds the specified property to those used to initialize the transaction manager. */
public Builder withProperty(String name, String value) {
this.userProperties.put(name, value);
return this;
}
/** Builds a {@link JpaIntegrationTestRule} instance. */
public JpaIntegrationTestRule buildIntegrationTestRule() {
checkState(initScript == null, "JpaNomulusIntegrationTestRule does not accept initScript");
return new JpaIntegrationTestRule(
ImmutableList.copyOf(extraEntityClasses), ImmutableMap.copyOf(userProperties));
}
/** Builds a {@link JpaUnitTestRule} instance. */
public JpaUnitTestRule buildUnitTestRule() {
return new JpaUnitTestRule(
Optional.ofNullable(initScript),
ImmutableList.copyOf(extraEntityClasses),
ImmutableMap.copyOf(userProperties));
}
}
}
@@ -19,6 +19,7 @@ import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.testing.JUnitBackports.assertThrows;
import static google.registry.testing.TestDataHelper.fileClassPath;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import google.registry.testing.FakeClock;
import java.math.BigInteger;
import javax.persistence.EntityManager;
@@ -32,10 +33,10 @@ import org.junit.runners.JUnit4;
@RunWith(JUnit4.class)
public class JpaTransactionManagerImplTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder()
.withInitScript(fileClassPath(getClass(), "test_schema.sql"))
.build();
.buildUnitTestRule();
@Test
public void inTransaction_returnsCorrespondingResult() {
@@ -53,7 +54,7 @@ public class JpaTransactionManagerImplTest {
@Test
public void getTransactionTime_throwsExceptionWhenNotInTransaction() {
FakeClock txnClock = jpaTmRule.getTxnClock();
FakeClock txnClock = jpaRule.getTxnClock();
txnClock.advanceOneMilli();
assertThrows(PersistenceException.class, () -> jpaTm().getTransactionTime());
jpaTm().transact(() -> assertThat(jpaTm().getTransactionTime()).isEqualTo(txnClock.nowUtc()));
@@ -18,13 +18,12 @@ import static com.google.common.truth.Truth.assertThat;
import static org.joda.time.DateTimeZone.UTC;
import static org.testcontainers.containers.PostgreSQLContainer.POSTGRESQL_PORT;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Charsets;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Maps;
import com.google.common.io.Resources;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.persistence.HibernateSchemaExporter;
import google.registry.persistence.NomulusPostgreSql;
import google.registry.persistence.PersistenceModule;
@@ -33,7 +32,6 @@ import google.registry.testing.FakeClock;
import java.io.File;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.sql.Connection;
@@ -41,10 +39,7 @@ import java.sql.Driver;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Properties;
import javax.persistence.EntityManagerFactory;
@@ -57,40 +52,24 @@ import org.testcontainers.containers.JdbcDatabaseContainer;
import org.testcontainers.containers.PostgreSQLContainer;
/**
* JUnit Rule to provision {@link JpaTransactionManagerImpl} backed by {@link PostgreSQLContainer}.
* Base class of JUnit Rules to provision {@link JpaTransactionManagerImpl} backed by {@link
* PostgreSQLContainer}. This class is not for direct use. Use specialized subclasses, {@link
* JpaIntegrationTestRule} or {@link JpaTestRules.JpaUnitTestRule} as befits the use case.
*
* <p>This rule also replaces the {@link JpaTransactionManagerImpl} provided by {@link
* TransactionManagerFactory} with the {@link JpaTransactionManagerImpl} generated by the rule
* itself, so that all SQL queries will be sent to the database instance created by {@link
* PostgreSQLContainer} to achieve test purpose.
*
* <p>The location of the Nomulus golden schema may be overridden with the {@code
* "sql_schema_resource_root} system property. This feature is needed by the server/schema
* compatibility tests, which need to use different versions of the schema off the classpath.
*
* <p>If defined, the value of the {@code "sql_schema_resource_root} should be an URL string that
* points to the jar or resource root directory. Here are some examples:
*
* <ul>
* <li>Absolute path to local directory: [file://]/path/to/resources
* <li>Absolute path to local jar: [file://]/path/to/schema.jar
* <li>URL to remote jar: https://host/path/to/schema.jar
* </ul>
*/
public class JpaTransactionManagerRule extends ExternalResource {
private static final String GOLDEN_SCHEMA_SQL_PATH = "sql/schema/nomulus.golden.sql";
abstract class JpaTransactionManagerRule extends ExternalResource {
private static final String DB_CLEANUP_SQL_PATH =
"google/registry/model/transaction/cleanup_database.sql";
private static final String MANAGEMENT_DB_NAME = "management";
private static final String POSTGRES_DB_NAME = "postgres";
// Name of the optional property that specifies the root path of the golden schema.
@VisibleForTesting
static final String GOLDEN_SCHEMA_RESOURCE_ROOT_PROP = "sql_schema_resource_root";
private final DateTime now = DateTime.now(UTC);
private final FakeClock clock = new FakeClock(now);
private final String initScriptPath;
private final Optional<String> initScriptPath;
private final ImmutableList<Class> extraEntityClasses;
private final ImmutableMap userProperties;
@@ -104,8 +83,8 @@ public class JpaTransactionManagerRule extends ExternalResource {
private EntityManagerFactory emf;
private JpaTransactionManager cachedTm;
private JpaTransactionManagerRule(
String initScriptPath,
protected JpaTransactionManagerRule(
Optional<String> initScriptPath,
ImmutableList<Class> extraEntityClasses,
ImmutableMap<String, String> userProperties) {
this.initScriptPath = initScriptPath;
@@ -118,14 +97,13 @@ public class JpaTransactionManagerRule extends ExternalResource {
new PostgreSQLContainer(NomulusPostgreSql.getDockerTag())
.withDatabaseName(MANAGEMENT_DB_NAME);
container.start();
Runtime.getRuntime().addShutdownHook(new Thread(() -> container.close()));
return container;
}
@Override
public void before() throws Exception {
executeSql(MANAGEMENT_DB_NAME, readSqlInClassPath(DB_CLEANUP_SQL_PATH));
executeSql(POSTGRES_DB_NAME, readInitialScript());
initScriptPath.ifPresent(path -> executeSql(POSTGRES_DB_NAME, readSqlInClassPath(path)));
if (!extraEntityClasses.isEmpty()) {
File tempSqlFile = File.createTempFile("tempSqlFile", ".sql");
tempSqlFile.deleteOnExit();
@@ -140,6 +118,9 @@ public class JpaTransactionManagerRule extends ExternalResource {
// If there are user properties, create a new properties object with these added.
ImmutableMap.Builder builder = properties.builder();
builder.putAll(userProperties);
// Forbid Hibernate push to stay consistent with flyway-based schema management.
builder.put(Environment.HBM2DDL_AUTO, "none");
builder.put(Environment.SHOW_SQL, "true");
properties = builder.build();
}
assertNormalActiveConnection();
@@ -196,38 +177,6 @@ public class JpaTransactionManagerRule extends ExternalResource {
}
}
@VisibleForTesting
Optional<String> getInitScriptUrlOverride() {
String schemaRootPath = System.getProperty(GOLDEN_SCHEMA_RESOURCE_ROOT_PROP, "").trim();
if (schemaRootPath.isEmpty() || !initScriptPath.equals(GOLDEN_SCHEMA_SQL_PATH)) {
return Optional.empty();
}
if (schemaRootPath.startsWith("/")) {
schemaRootPath = "file://" + schemaRootPath;
}
if (schemaRootPath.endsWith(".jar") && !schemaRootPath.startsWith("jar:")) {
schemaRootPath = "jar:" + schemaRootPath;
}
if (schemaRootPath.endsWith(".jar")) {
schemaRootPath += "!/" + GOLDEN_SCHEMA_SQL_PATH;
} else {
schemaRootPath += "/" + GOLDEN_SCHEMA_SQL_PATH;
}
return Optional.of(schemaRootPath);
}
private String readInitialScript() {
Optional<String> schemaUrlOverride = getInitScriptUrlOverride();
if (!schemaUrlOverride.isPresent()) {
return readSqlInClassPath(initScriptPath);
}
try {
return Resources.toString(new URL(schemaUrlOverride.get()), StandardCharsets.UTF_8);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
private void executeSql(String dbName, String sqlScript) {
try (Connection conn = createConnection(dbName);
Statement statement = conn.createStatement()) {
@@ -284,42 +233,4 @@ public class JpaTransactionManagerRule extends ExternalResource {
return clock;
}
/** Builder for {@link JpaTransactionManagerRule}. */
public static class Builder {
private String initScript;
private List<Class> extraEntityClasses = new ArrayList<Class>();
private Map<String, String> userProperties = new HashMap<String, String>();
/**
* Sets the SQL script to be used to initialize the database. If not set,
* sql/schema/nomulus.golden.sql will be used.
*/
public Builder withInitScript(String initScript) {
this.initScript = initScript;
return this;
}
/** Adds annotated class(es) to the known entities for the database. */
public Builder withEntityClass(Class... classes) {
this.extraEntityClasses.addAll(ImmutableSet.copyOf(classes));
return this;
}
/** Adds the specified property to those used to initialize the transaction manager. */
public Builder withProperty(String name, String value) {
this.userProperties.put(name, value);
return this;
}
/** Builds a {@link JpaTransactionManagerRule} instance. */
public JpaTransactionManagerRule build() {
if (initScript == null) {
initScript = GOLDEN_SCHEMA_SQL_PATH;
}
return new JpaTransactionManagerRule(
initScript,
ImmutableList.copyOf(extraEntityClasses),
ImmutableMap.copyOf(userProperties));
}
}
}
@@ -15,13 +15,12 @@
package google.registry.model.transaction;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.model.transaction.JpaTransactionManagerRule.GOLDEN_SCHEMA_RESOURCE_ROOT_PROP;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.testing.JUnitBackports.assertThrows;
import google.registry.model.ImmutableObject;
import google.registry.testing.SystemPropertyRule;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import google.registry.schema.tmch.ClaimsList;
import java.util.List;
import javax.persistence.Entity;
import javax.persistence.Id;
@@ -31,17 +30,15 @@ import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/** JUnit test for {@link JpaTransactionManagerRule} */
/** JUnit test for {@link JpaTransactionManagerRule}, with {@link JpaUnitTestRule} as proxy. */
@RunWith(JUnit4.class)
public class JpaTransactionManagerRuleTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
.withEntityClass(TestEntity.class)
.build();
@Rule public final SystemPropertyRule systemPropertyRule = new SystemPropertyRule();
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder()
.withEntityClass(ClaimsList.class, TestEntity.class)
.buildUnitTestRule();
@Test
public void verifiesRuleWorks() {
@@ -78,57 +75,6 @@ public class JpaTransactionManagerRuleTest {
assertThat(retrieved).isEqualTo(original);
}
@Test
public void testInitScriptUrl_noOverride() {
assertThat(jpaTmRule.getInitScriptUrlOverride()).isEmpty();
}
@Test
public void testInitScriptUrl_localDir_noProtocol() {
systemPropertyRule.setProperty(GOLDEN_SCHEMA_RESOURCE_ROOT_PROP, "/path/to/resources");
assertThat(jpaTmRule.getInitScriptUrlOverride())
.hasValue("file:///path/to/resources/sql/schema/nomulus.golden.sql");
}
@Test
public void testInitScriptUrl_localDir_hasProtocol() {
systemPropertyRule.setProperty(GOLDEN_SCHEMA_RESOURCE_ROOT_PROP, "file:///path/to/resources");
assertThat(jpaTmRule.getInitScriptUrlOverride())
.hasValue("file:///path/to/resources/sql/schema/nomulus.golden.sql");
}
@Test
public void testInitScriptUrl_localJar_noProtocol() {
systemPropertyRule.setProperty(
GOLDEN_SCHEMA_RESOURCE_ROOT_PROP, "/path/to/resources/schema.jar");
assertThat(jpaTmRule.getInitScriptUrlOverride())
.hasValue("jar:file:///path/to/resources/schema.jar!/sql/schema/nomulus.golden.sql");
}
@Test
public void testInitScriptUrl_localJar_hasPartialProtocol() {
systemPropertyRule.setProperty(
GOLDEN_SCHEMA_RESOURCE_ROOT_PROP, "file:///path/to/resources/schema.jar");
assertThat(jpaTmRule.getInitScriptUrlOverride())
.hasValue("jar:file:///path/to/resources/schema.jar!/sql/schema/nomulus.golden.sql");
}
@Test
public void testInitScriptUrl_localJar_hasFullProtocol() {
systemPropertyRule.setProperty(
GOLDEN_SCHEMA_RESOURCE_ROOT_PROP, "jar:file:///path/to/resources/schema.jar");
assertThat(jpaTmRule.getInitScriptUrlOverride())
.hasValue("jar:file:///path/to/resources/schema.jar!/sql/schema/nomulus.golden.sql");
}
@Test
public void testInitScriptUrl_remoteJar() {
systemPropertyRule.setProperty(
GOLDEN_SCHEMA_RESOURCE_ROOT_PROP, "http://host/path/to/resources/schema.jar");
assertThat(jpaTmRule.getInitScriptUrlOverride())
.hasValue("jar:http://host/path/to/resources/schema.jar!/sql/schema/nomulus.golden.sql");
}
@Entity(name = "TestEntity") // Specify name to avoid nested class naming issues.
static class TestEntity extends ImmutableObject {
@Id String key;
@@ -21,7 +21,8 @@ import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.collect.ImmutableSet;
import com.google.common.hash.BloomFilter;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import javax.persistence.Entity;
import javax.persistence.Id;
import org.junit.Rule;
@@ -34,10 +35,8 @@ import org.junit.runners.JUnit4;
public class BloomFilterConverterTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
.withEntityClass(TestEntity.class)
.build();
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
@Test
public void roundTripConversion_returnsSameBloomFilter() {
@@ -18,7 +18,8 @@ import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import google.registry.model.CreateAutoTimestamp;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import javax.persistence.Entity;
import javax.persistence.Id;
import org.joda.time.DateTime;
@@ -32,10 +33,8 @@ import org.junit.runners.JUnit4;
public class CreateAutoTimestampConverterTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
.withEntityClass(TestEntity.class)
.build();
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
@Test
public void testTypeConversion() {
@@ -57,7 +56,7 @@ public class CreateAutoTimestampConverterTest {
TestEntity result =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "autoinit"));
assertThat(result.cat.getTimestamp()).isEqualTo(jpaTmRule.getTxnClock().nowUtc());
assertThat(result.cat.getTimestamp()).isEqualTo(jpaRule.getTxnClock().nowUtc());
}
@Entity(name = "TestEntity") // Override entity name to avoid the nested class reference.
@@ -18,7 +18,8 @@ import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.testing.JUnitBackports.assertThrows;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.PersistenceException;
@@ -33,10 +34,8 @@ import org.junit.runners.JUnit4;
public class CurrencyUnitConverterTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
.withEntityClass(TestEntity.class)
.build();
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
@Test
public void roundTripConversion() {
@@ -18,7 +18,8 @@ import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import java.sql.Timestamp;
import javax.persistence.Entity;
import javax.persistence.Id;
@@ -34,8 +35,8 @@ import org.junit.runners.JUnit4;
public class DateTimeConverterTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder().withEntityClass(TestEntity.class).build();
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
private final DateTimeConverter converter = new DateTimeConverter();
@@ -18,7 +18,8 @@ import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.collect.ImmutableMap;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import java.math.BigDecimal;
import java.util.Arrays;
import java.util.List;
@@ -34,7 +35,6 @@ import javax.persistence.Id;
import javax.persistence.JoinColumn;
import javax.persistence.MapKeyColumn;
import javax.persistence.PostLoad;
import org.hibernate.cfg.Environment;
import org.joda.money.CurrencyUnit;
import org.joda.money.Money;
import org.junit.Rule;
@@ -63,11 +63,10 @@ import org.junit.runners.JUnit4;
@RunWith(JUnit4.class)
public class JodaMoneyConverterTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder()
.withEntityClass(TestEntity.class, ComplexTestEntity.class)
.withProperty(Environment.HBM2DDL_AUTO, "update")
.build();
.buildUnitTestRule();
@Test
public void roundTripConversion() {
@@ -81,7 +80,7 @@ public class JodaMoneyConverterTest {
jpaTm()
.getEntityManager()
.createNativeQuery(
"SELECT amount, currency FROM TestEntity WHERE name = 'id'")
"SELECT amount, currency FROM \"TestEntity\" WHERE name = 'id'")
.getResultList());
assertThat(result.size()).isEqualTo(1);
assertThat(Arrays.asList((Object[]) result.get(0)))
@@ -112,7 +111,7 @@ public class JodaMoneyConverterTest {
.getEntityManager()
.createNativeQuery(
"SELECT my_amount, my_currency, your_amount, your_currency FROM"
+ " ComplexTestEntity WHERE name = 'id'")
+ " \"ComplexTestEntity\" WHERE name = 'id'")
.getResultList());
assertThat(result.size()).isEqualTo(1);
assertThat(Arrays.asList((Object[]) result.get(0)))
@@ -126,7 +125,7 @@ public class JodaMoneyConverterTest {
jpaTm()
.getEntityManager()
.createNativeQuery(
"SELECT map_amount, map_currency FROM MoneyMap"
"SELECT map_amount, map_currency FROM \"MoneyMap\""
+ " WHERE entity_name = 'id' AND map_key = 'dos'")
.getResultList());
ComplexTestEntity persisted =
@@ -147,7 +146,9 @@ public class JodaMoneyConverterTest {
assertThat(persisted.moneyMap).containsExactlyEntriesIn(moneyMap);
}
@Entity(name = "TestEntity") // Override entity name to avoid the nested class reference.
// Override entity name to exclude outer-class name in table name. Not necessary if class is not
// inner class. The double quotes are added to conform to our schema generation convention.
@Entity(name = "\"TestEntity\"")
public static class TestEntity extends ImmutableObject {
@Id String name = "id";
@@ -161,7 +162,8 @@ public class JodaMoneyConverterTest {
}
}
@Entity(name = "ComplexTestEntity") // Override entity name to avoid the nested class reference.
// See comments on the annotation for TestEntity above for reason.
@Entity(name = "\"ComplexTestEntity\"")
// This entity is used to test column override for embedded fields and collections.
public static class ComplexTestEntity extends ImmutableObject {
@@ -0,0 +1,135 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.testing.JUnitBackports.assertThrows;
import com.google.common.collect.ImmutableList;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import java.util.List;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.NoResultException;
import org.hibernate.annotations.Type;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/** Unit tests for {@link StringListUserType}. */
@RunWith(JUnit4.class)
public class StringListUserTypeTest {
@Rule
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
@Test
public void roundTripConversion_returnsSameStringList() {
List<String> tlds = ImmutableList.of("app", "dev", "how");
TestEntity testEntity = new TestEntity(tlds);
jpaTm().transact(() -> jpaTm().getEntityManager().persist(testEntity));
TestEntity persisted =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
assertThat(persisted.tlds).containsExactly("app", "dev", "how");
}
@Test
public void testMerge_succeeds() {
List<String> tlds = ImmutableList.of("app", "dev", "how");
TestEntity testEntity = new TestEntity(tlds);
jpaTm().transact(() -> jpaTm().getEntityManager().persist(testEntity));
TestEntity persisted =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
persisted.tlds = ImmutableList.of("com", "gov");
jpaTm().transact(() -> jpaTm().getEntityManager().merge(persisted));
TestEntity updated =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
assertThat(updated.tlds).containsExactly("com", "gov");
}
@Test
public void testNullValue_writesAndReadsNullSuccessfully() {
TestEntity testEntity = new TestEntity(null);
jpaTm().transact(() -> jpaTm().getEntityManager().persist(testEntity));
TestEntity persisted =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
assertThat(persisted.tlds).isNull();
}
@Test
public void testEmptyCollection_writesAndReadsEmptyCollectionSuccessfully() {
TestEntity testEntity = new TestEntity(ImmutableList.of());
jpaTm().transact(() -> jpaTm().getEntityManager().persist(testEntity));
TestEntity persisted =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
assertThat(persisted.tlds).isEmpty();
}
@Test
public void testNativeQuery_succeeds() throws Exception {
executeNativeQuery("INSERT INTO \"TestEntity\" (name, tlds) VALUES ('id', '{app, dev}')");
assertThat(
getSingleResultFromNativeQuery("SELECT tlds[1] FROM \"TestEntity\" WHERE name = 'id'"))
.isEqualTo("app");
assertThat(
getSingleResultFromNativeQuery("SELECT tlds[2] FROM \"TestEntity\" WHERE name = 'id'"))
.isEqualTo("dev");
executeNativeQuery("UPDATE \"TestEntity\" SET tlds = '{com, gov}' WHERE name = 'id'");
assertThat(
getSingleResultFromNativeQuery("SELECT tlds[1] FROM \"TestEntity\" WHERE name = 'id'"))
.isEqualTo("com");
assertThat(
getSingleResultFromNativeQuery("SELECT tlds[2] FROM \"TestEntity\" WHERE name = 'id'"))
.isEqualTo("gov");
executeNativeQuery("DELETE FROM \"TestEntity\" WHERE name = 'id'");
assertThrows(
NoResultException.class,
() ->
getSingleResultFromNativeQuery("SELECT tlds[1] FROM \"TestEntity\" WHERE name = 'id'"));
}
private static Object getSingleResultFromNativeQuery(String sql) {
return jpaTm()
.transact(() -> jpaTm().getEntityManager().createNativeQuery(sql).getSingleResult());
}
private static Object executeNativeQuery(String sql) {
return jpaTm()
.transact(() -> jpaTm().getEntityManager().createNativeQuery(sql).executeUpdate());
}
@Entity(name = "TestEntity") // Override entity name to avoid the nested class reference.
private static class TestEntity extends ImmutableObject {
@Id String name = "id";
@Type(type = "google.registry.persistence.StringListUserType")
List<String> tlds;
private TestEntity() {}
private TestEntity(List<String> tlds) {
this.tlds = tlds;
}
}
}
@@ -0,0 +1,82 @@
// Copyright 2020 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.persistence;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.collect.ImmutableSet;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import java.util.Set;
import javax.persistence.Entity;
import javax.persistence.Id;
import org.hibernate.annotations.Type;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/** Unit tests for {@link StringSetUserType}. */
@RunWith(JUnit4.class)
public class StringSetUserTypeTest {
@Rule
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
@Test
public void roundTripConversion_returnsSameStringList() {
Set<String> tlds = ImmutableSet.of("app", "dev", "how");
TestEntity testEntity = new TestEntity(tlds);
jpaTm().transact(() -> jpaTm().getEntityManager().persist(testEntity));
TestEntity persisted =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
assertThat(persisted.tlds).containsExactly("app", "dev", "how");
}
@Test
public void testNullValue_writesAndReadsNullSuccessfully() {
TestEntity testEntity = new TestEntity(null);
jpaTm().transact(() -> jpaTm().getEntityManager().persist(testEntity));
TestEntity persisted =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
assertThat(persisted.tlds).isNull();
}
@Test
public void testEmptyCollection_writesAndReadsEmptyCollectionSuccessfully() {
TestEntity testEntity = new TestEntity(ImmutableSet.of());
jpaTm().transact(() -> jpaTm().getEntityManager().persist(testEntity));
TestEntity persisted =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "id"));
assertThat(persisted.tlds).isEmpty();
}
@Entity(name = "TestEntity") // Override entity name to avoid the nested class reference.
private static class TestEntity extends ImmutableObject {
@Id String name = "id";
@Type(type = "google.registry.persistence.StringSetUserType")
Set<String> tlds;
private TestEntity() {}
private TestEntity(Set<String> tlds) {
this.tlds = tlds;
}
}
}
@@ -18,7 +18,8 @@ import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import google.registry.model.ImmutableObject;
import google.registry.model.UpdateAutoTimestamp;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import javax.persistence.Entity;
import javax.persistence.Id;
import org.junit.Rule;
@@ -31,10 +32,8 @@ import org.junit.runners.JUnit4;
public class UpdateAutoTimestampConverterTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
.withEntityClass(TestEntity.class)
.build();
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
@Test
public void testTypeConversion() {
@@ -46,7 +45,7 @@ public class UpdateAutoTimestampConverterTest {
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "myinst"));
assertThat(result.name).isEqualTo("myinst");
assertThat(result.uat.getTimestamp()).isEqualTo(jpaTmRule.getTxnClock().nowUtc());
assertThat(result.uat.getTimestamp()).isEqualTo(jpaRule.getTxnClock().nowUtc());
}
@Test
@@ -58,7 +57,7 @@ public class UpdateAutoTimestampConverterTest {
TestEntity result1 =
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "myinst1"));
jpaTmRule.getTxnClock().advanceOneMilli();
jpaRule.getTxnClock().advanceOneMilli();
TestEntity ent2 = new TestEntity("myinst2", result1.uat);
@@ -68,7 +67,7 @@ public class UpdateAutoTimestampConverterTest {
jpaTm().transact(() -> jpaTm().getEntityManager().find(TestEntity.class, "myinst2"));
assertThat(result1.uat.getTimestamp()).isNotEqualTo(result2.uat.getTimestamp());
assertThat(result2.uat.getTimestamp()).isEqualTo(jpaTmRule.getTxnClock().nowUtc());
assertThat(result2.uat.getTimestamp()).isEqualTo(jpaRule.getTxnClock().nowUtc());
}
@Entity(name = "TestEntity") // Override entity name to avoid the nested class reference.
@@ -18,7 +18,8 @@ import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import google.registry.model.ImmutableObject;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaUnitTestRule;
import java.sql.Timestamp;
import java.time.Instant;
import java.time.ZonedDateTime;
@@ -34,10 +35,8 @@ import org.junit.runners.JUnit4;
public class ZonedDateTimeConverterTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder()
.withEntityClass(TestEntity.class)
.build();
public final JpaUnitTestRule jpaRule =
new JpaTestRules.Builder().withEntityClass(TestEntity.class).buildUnitTestRule();
private final ZonedDateTimeConverter converter = new ZonedDateTimeConverter();
@@ -64,9 +64,6 @@ public class IcannReportingUploadActionTest {
private static final byte[] PAYLOAD_SUCCESS = "test,csv\n13,37".getBytes(UTF_8);
private static final byte[] PAYLOAD_FAIL = "ahah,csv\n12,34".getBytes(UTF_8);
private static final byte[] MANIFEST_PAYLOAD =
"tld-transactions-200606.csv\ntld-activity-200606.csv\nfoo-transactions-200606.csv\nfoo-activity-200606.csv\n"
.getBytes(UTF_8);
private final IcannHttpReporter mockReporter = mock(IcannHttpReporter.class);
private final SendEmailService emailService = mock(SendEmailService.class);
private final FakeResponse response = new FakeResponse();
@@ -111,30 +108,23 @@ public class IcannReportingUploadActionTest {
gcsService,
new GcsFilename("basin/icann/monthly/2006-06", "foo-activity-200606.csv"),
PAYLOAD_SUCCESS);
writeGcsFile(
gcsService,
new GcsFilename("basin/icann/monthly/2006-06", "MANIFEST.txt"),
MANIFEST_PAYLOAD);
when(mockReporter.send(PAYLOAD_SUCCESS, "tld-transactions-200606.csv")).thenReturn(true);
when(mockReporter.send(PAYLOAD_SUCCESS, "foo-transactions-200606.csv")).thenReturn(true);
when(mockReporter.send(PAYLOAD_FAIL, "tld-activity-200606.csv")).thenReturn(false);
when(mockReporter.send(PAYLOAD_SUCCESS, "foo-activity-200606.csv")).thenReturn(true);
when(mockReporter.send(MANIFEST_PAYLOAD, "MANIFEST.txt")).thenReturn(true);
clock.setTo(DateTime.parse("2006-06-06T00:30:00Z"));
clock.setTo(DateTime.parse("2006-07-05T00:30:00Z"));
persistResource(
Cursor.create(
CursorType.ICANN_UPLOAD_ACTIVITY, DateTime.parse("2006-06-06TZ"), Registry.get("tld")));
CursorType.ICANN_UPLOAD_ACTIVITY, DateTime.parse("2006-07-01TZ"), Registry.get("tld")));
persistResource(
Cursor.create(
CursorType.ICANN_UPLOAD_TX, DateTime.parse("2006-06-06TZ"), Registry.get("tld")));
persistResource(
Cursor.createGlobal(CursorType.ICANN_UPLOAD_MANIFEST, DateTime.parse("2006-07-06TZ")));
CursorType.ICANN_UPLOAD_TX, DateTime.parse("2006-07-01TZ"), Registry.get("tld")));
persistResource(
Cursor.create(
CursorType.ICANN_UPLOAD_ACTIVITY, DateTime.parse("2006-06-06TZ"), Registry.get("foo")));
CursorType.ICANN_UPLOAD_ACTIVITY, DateTime.parse("2006-07-01TZ"), Registry.get("foo")));
persistResource(
Cursor.create(
CursorType.ICANN_UPLOAD_TX, DateTime.parse("2006-06-06TZ"), Registry.get("foo")));
CursorType.ICANN_UPLOAD_TX, DateTime.parse("2006-07-01TZ"), Registry.get("foo")));
loggerToIntercept.addHandler(logHandler);
}
@@ -176,7 +166,7 @@ public class IcannReportingUploadActionTest {
.load()
.key(Cursor.createKey(CursorType.ICANN_UPLOAD_ACTIVITY, Registry.get("tld")))
.now();
assertThat(cursor.getCursorTime()).isEqualTo(DateTime.parse("2006-07-01TZ"));
assertThat(cursor.getCursorTime()).isEqualTo(DateTime.parse("2006-08-01TZ"));
}
@Test
@@ -195,24 +185,6 @@ public class IcannReportingUploadActionTest {
new InternetAddress("sender@example.com")));
}
@Test
public void testSuccess_uploadManifest() throws Exception {
persistResource(
Cursor.createGlobal(CursorType.ICANN_UPLOAD_MANIFEST, DateTime.parse("2006-06-06TZ")));
IcannReportingUploadAction action = createAction();
action.run();
ofy().clearSessionCache();
Cursor cursor =
ofy().load().key(Cursor.createGlobalKey(CursorType.ICANN_UPLOAD_MANIFEST)).now();
assertThat(cursor.getCursorTime()).isEqualTo(DateTime.parse("2006-07-01TZ"));
verify(mockReporter).send(PAYLOAD_FAIL, "tld-activity-200606.csv");
verify(mockReporter).send(PAYLOAD_SUCCESS, "foo-activity-200606.csv");
verify(mockReporter).send(PAYLOAD_SUCCESS, "foo-transactions-200606.csv");
verify(mockReporter).send(PAYLOAD_SUCCESS, "tld-transactions-200606.csv");
verify(mockReporter).send(MANIFEST_PAYLOAD, "MANIFEST.txt");
verifyNoMoreInteractions(mockReporter);
}
@Test
public void testSuccess_withRetry() throws Exception {
IcannReportingUploadAction action = createAction();
@@ -262,7 +234,7 @@ public class IcannReportingUploadActionTest {
.load()
.key(Cursor.createKey(CursorType.ICANN_UPLOAD_ACTIVITY, Registry.get("tld")))
.now();
assertThat(cursor.getCursorTime()).isEqualTo(DateTime.parse("2006-06-06TZ"));
assertThat(cursor.getCursorTime()).isEqualTo(DateTime.parse("2006-07-01TZ"));
}
@Test
@@ -276,7 +248,7 @@ public class IcannReportingUploadActionTest {
.load()
.key(Cursor.createKey(CursorType.ICANN_UPLOAD_ACTIVITY, Registry.get("foo")))
.now();
assertThat(cursor.getCursorTime()).isEqualTo(DateTime.parse("2006-06-06TZ"));
assertThat(cursor.getCursorTime()).isEqualTo(DateTime.parse("2006-07-01TZ"));
verifyNoMoreInteractions(mockReporter);
}
@@ -323,8 +295,8 @@ public class IcannReportingUploadActionTest {
public void testWarning_fileNotStagedYet() throws Exception {
persistResource(
Cursor.create(
CursorType.ICANN_UPLOAD_ACTIVITY, DateTime.parse("2006-07-01TZ"), Registry.get("foo")));
clock.setTo(DateTime.parse("2006-07-01T00:30:00Z"));
CursorType.ICANN_UPLOAD_ACTIVITY, DateTime.parse("2006-08-01TZ"), Registry.get("foo")));
clock.setTo(DateTime.parse("2006-08-01T00:30:00Z"));
IcannReportingUploadAction action = createAction();
action.subdir = "icann/monthly/2006-07";
action.run();
@@ -349,39 +321,25 @@ public class IcannReportingUploadActionTest {
}
@Test
public void testSuccess_nullCursors() throws Exception {
public void testSuccess_nullCursorsInitiatedToFirstOfNextMonth() throws Exception {
createTlds("new");
writeGcsFile(
gcsService,
new GcsFilename("basin/icann/monthly/2006-06", "new-transactions-200606.csv"),
PAYLOAD_SUCCESS);
writeGcsFile(
gcsService,
new GcsFilename("basin/icann/monthly/2006-06", "new-activity-200606.csv"),
PAYLOAD_SUCCESS);
when(mockReporter.send(PAYLOAD_SUCCESS, "new-transactions-200606.csv")).thenReturn(true);
when(mockReporter.send(PAYLOAD_SUCCESS, "new-activity-200606.csv")).thenReturn(true);
IcannReportingUploadAction action = createAction();
action.run();
verify(mockReporter).send(PAYLOAD_SUCCESS, "foo-activity-200606.csv");
verify(mockReporter).send(PAYLOAD_FAIL, "tld-activity-200606.csv");
verify(mockReporter).send(PAYLOAD_SUCCESS, "new-activity-200606.csv");
verify(mockReporter).send(PAYLOAD_SUCCESS, "foo-transactions-200606.csv");
verify(mockReporter).send(PAYLOAD_SUCCESS, "tld-transactions-200606.csv");
verify(mockReporter).send(PAYLOAD_SUCCESS, "new-transactions-200606.csv");
verifyNoMoreInteractions(mockReporter);
verify(emailService)
.sendEmail(
EmailMessage.create(
"ICANN Monthly report upload summary: 5/6 succeeded",
"ICANN Monthly report upload summary: 3/4 succeeded",
"Report Filename - Upload status:\n"
+ "foo-activity-200606.csv - SUCCESS\n"
+ "new-activity-200606.csv - SUCCESS\n"
+ "tld-activity-200606.csv - FAILURE\n"
+ "foo-transactions-200606.csv - SUCCESS\n"
+ "new-transactions-200606.csv - SUCCESS\n"
+ "tld-transactions-200606.csv - SUCCESS",
new InternetAddress("recipient@example.com"),
new InternetAddress("sender@example.com")));
@@ -391,10 +349,10 @@ public class IcannReportingUploadActionTest {
.load()
.key(Cursor.createKey(CursorType.ICANN_UPLOAD_ACTIVITY, Registry.get("new")))
.now();
assertThat(newActivityCursor.getCursorTime()).isEqualTo(DateTime.parse("2006-07-01TZ"));
assertThat(newActivityCursor.getCursorTime()).isEqualTo(DateTime.parse("2006-08-01TZ"));
Cursor newTransactionCursor =
ofy().load().key(Cursor.createKey(CursorType.ICANN_UPLOAD_TX, Registry.get("new"))).now();
assertThat(newTransactionCursor.getCursorTime()).isEqualTo(DateTime.parse("2006-07-01TZ"));
assertThat(newTransactionCursor.getCursorTime()).isEqualTo(DateTime.parse("2006-08-01TZ"));
}
}
@@ -118,6 +118,8 @@ public class Spec11EmailUtilsTest {
emailService,
new InternetAddress("my-receiver@test.com"),
new InternetAddress("abuse@test.com"),
ImmutableList.of(
new InternetAddress("abuse@test.com"), new InternetAddress("bcc@test.com")),
FAKE_RESOURCES,
"Super Cool Registry");
@@ -142,7 +144,7 @@ public class Spec11EmailUtilsTest {
capturedContents.get(0),
"abuse@test.com",
"the.registrar@example.com",
Optional.of("abuse@test.com"),
ImmutableList.of("abuse@test.com", "bcc@test.com"),
"Super Cool Registry Monthly Threat Detector [2018-07-15]",
String.format(MONTHLY_EMAIL_FORMAT, "<tr><td>a.com</td><td>MALWARE</td></tr>"),
Optional.of(MediaType.HTML_UTF_8));
@@ -150,7 +152,7 @@ public class Spec11EmailUtilsTest {
capturedContents.get(1),
"abuse@test.com",
"new.registrar@example.com",
Optional.of("abuse@test.com"),
ImmutableList.of("abuse@test.com", "bcc@test.com"),
"Super Cool Registry Monthly Threat Detector [2018-07-15]",
String.format(
MONTHLY_EMAIL_FORMAT,
@@ -160,7 +162,7 @@ public class Spec11EmailUtilsTest {
capturedContents.get(2),
"abuse@test.com",
"my-receiver@test.com",
Optional.empty(),
ImmutableList.of(),
"Spec11 Pipeline Success 2018-07-15",
"Spec11 reporting completed successfully.",
Optional.empty());
@@ -180,7 +182,7 @@ public class Spec11EmailUtilsTest {
capturedMessages.get(0),
"abuse@test.com",
"the.registrar@example.com",
Optional.of("abuse@test.com"),
ImmutableList.of("abuse@test.com", "bcc@test.com"),
"Super Cool Registry Daily Threat Detector [2018-07-15]",
String.format(DAILY_EMAIL_FORMAT, "<tr><td>a.com</td><td>MALWARE</td></tr>"),
Optional.of(MediaType.HTML_UTF_8));
@@ -188,7 +190,7 @@ public class Spec11EmailUtilsTest {
capturedMessages.get(1),
"abuse@test.com",
"new.registrar@example.com",
Optional.of("abuse@test.com"),
ImmutableList.of("abuse@test.com", "bcc@test.com"),
"Super Cool Registry Daily Threat Detector [2018-07-15]",
String.format(
DAILY_EMAIL_FORMAT,
@@ -198,7 +200,7 @@ public class Spec11EmailUtilsTest {
capturedMessages.get(2),
"abuse@test.com",
"my-receiver@test.com",
Optional.empty(),
ImmutableList.of(),
"Spec11 Pipeline Success 2018-07-15",
"Spec11 reporting completed successfully.",
Optional.empty());
@@ -225,7 +227,7 @@ public class Spec11EmailUtilsTest {
capturedContents.get(0),
"abuse@test.com",
"new.registrar@example.com",
Optional.of("abuse@test.com"),
ImmutableList.of("abuse@test.com", "bcc@test.com"),
"Super Cool Registry Monthly Threat Detector [2018-07-15]",
String.format(MONTHLY_EMAIL_FORMAT, "<tr><td>c.com</td><td>MALWARE</td></tr>"),
Optional.of(MediaType.HTML_UTF_8));
@@ -233,7 +235,7 @@ public class Spec11EmailUtilsTest {
capturedContents.get(1),
"abuse@test.com",
"my-receiver@test.com",
Optional.empty(),
ImmutableList.of(),
"Spec11 Pipeline Success 2018-07-15",
"Spec11 reporting completed successfully.",
Optional.empty());
@@ -270,7 +272,7 @@ public class Spec11EmailUtilsTest {
capturedMessages.get(0),
"abuse@test.com",
"the.registrar@example.com",
Optional.of("abuse@test.com"),
ImmutableList.of("abuse@test.com", "bcc@test.com"),
"Super Cool Registry Monthly Threat Detector [2018-07-15]",
String.format(MONTHLY_EMAIL_FORMAT, "<tr><td>a.com</td><td>MALWARE</td></tr>"),
Optional.of(MediaType.HTML_UTF_8));
@@ -278,7 +280,7 @@ public class Spec11EmailUtilsTest {
capturedMessages.get(1),
"abuse@test.com",
"new.registrar@example.com",
Optional.of("abuse@test.com"),
ImmutableList.of("abuse@test.com", "bcc@test.com"),
"Super Cool Registry Monthly Threat Detector [2018-07-15]",
String.format(
MONTHLY_EMAIL_FORMAT,
@@ -288,7 +290,7 @@ public class Spec11EmailUtilsTest {
capturedMessages.get(2),
"abuse@test.com",
"my-receiver@test.com",
Optional.empty(),
ImmutableList.of(),
"Spec11 Emailing Failure 2018-07-15",
"Emailing Spec11 reports failed due to expected",
Optional.empty());
@@ -302,7 +304,7 @@ public class Spec11EmailUtilsTest {
contentCaptor.getValue(),
"abuse@test.com",
"my-receiver@test.com",
Optional.empty(),
ImmutableList.of(),
"Spec11 Pipeline Alert: 2018-07",
"Alert!",
Optional.empty());
@@ -351,7 +353,7 @@ public class Spec11EmailUtilsTest {
EmailMessage message,
String from,
String recipient,
Optional<String> replyTo,
ImmutableList<String> bccs,
String subject,
String body,
Optional<MediaType> contentType)
@@ -363,8 +365,8 @@ public class Spec11EmailUtilsTest {
.setSubject(subject)
.setBody(body);
if (replyTo.isPresent()) {
expectedContentBuilder.setBcc(new InternetAddress(replyTo.get()));
for (String bcc : bccs) {
expectedContentBuilder.addBcc(new InternetAddress(bcc));
}
contentType.ifPresent(expectedContentBuilder::setContentType);
assertThat(message).isEqualTo(expectedContentBuilder.build());
@@ -0,0 +1,133 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.cursor;
import static com.google.common.truth.Truth.assertThat;
import google.registry.model.common.Cursor.CursorType;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.testing.FakeClock;
import java.util.List;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/** Unit tests for {@link Cursor}. */
@RunWith(JUnit4.class)
public class CursorDaoTest {
private FakeClock fakeClock = new FakeClock();
@Rule
public final JpaIntegrationTestRule jpaRule =
new JpaTestRules.Builder().buildIntegrationTestRule();
@Test
public void save_worksSuccessfullyOnNewCursor() {
Cursor cursor = Cursor.create(CursorType.BRDA, "tld", fakeClock.nowUtc());
CursorDao.save(cursor);
Cursor returnedCursor = CursorDao.load(CursorType.BRDA, "tld");
assertThat(returnedCursor.getCursorTime()).isEqualTo(cursor.getCursorTime());
}
@Test
public void save_worksSuccessfullyOnExistingCursor() {
Cursor cursor = Cursor.create(CursorType.RDE_REPORT, "tld", fakeClock.nowUtc());
CursorDao.save(cursor);
Cursor cursor2 = Cursor.create(CursorType.RDE_REPORT, "tld", fakeClock.nowUtc().plusDays(3));
CursorDao.save(cursor2);
Cursor returnedCursor = CursorDao.load(CursorType.RDE_REPORT, "tld");
assertThat(returnedCursor.getCursorTime()).isEqualTo(cursor2.getCursorTime());
}
@Test
public void save_worksSuccessfullyOnNewGlobalCursor() {
Cursor cursor = Cursor.createGlobal(CursorType.RECURRING_BILLING, fakeClock.nowUtc());
CursorDao.save(cursor);
Cursor returnedCursor = CursorDao.load(CursorType.RECURRING_BILLING);
assertThat(returnedCursor.getCursorTime()).isEqualTo(cursor.getCursorTime());
}
@Test
public void save_worksSuccessfullyOnExistingGlobalCursor() {
Cursor cursor = Cursor.createGlobal(CursorType.RECURRING_BILLING, fakeClock.nowUtc());
CursorDao.save(cursor);
Cursor cursor2 =
Cursor.createGlobal(CursorType.RECURRING_BILLING, fakeClock.nowUtc().plusDays(3));
CursorDao.save(cursor2);
Cursor returnedCursor = CursorDao.load(CursorType.RECURRING_BILLING);
assertThat(returnedCursor.getCursorTime()).isEqualTo(cursor2.getCursorTime());
}
@Test
public void load_worksSuccessfully() {
Cursor cursor = Cursor.createGlobal(CursorType.RECURRING_BILLING, fakeClock.nowUtc());
Cursor cursor2 = Cursor.create(CursorType.RDE_REPORT, "tld", fakeClock.nowUtc());
Cursor cursor3 = Cursor.create(CursorType.RDE_REPORT, "foo", fakeClock.nowUtc());
Cursor cursor4 = Cursor.create(CursorType.BRDA, "foo", fakeClock.nowUtc());
CursorDao.save(cursor);
CursorDao.save(cursor2);
CursorDao.save(cursor3);
CursorDao.save(cursor4);
Cursor returnedCursor = CursorDao.load(CursorType.RDE_REPORT, "tld");
assertThat(returnedCursor.getCursorTime()).isEqualTo(cursor2.getCursorTime());
returnedCursor = CursorDao.load(CursorType.BRDA, "foo");
assertThat(returnedCursor.getCursorTime()).isEqualTo(cursor4.getCursorTime());
returnedCursor = CursorDao.load(CursorType.RECURRING_BILLING);
assertThat(returnedCursor.getCursorTime()).isEqualTo(cursor.getCursorTime());
}
@Test
public void loadAll_worksSuccessfully() {
Cursor cursor = Cursor.createGlobal(CursorType.RECURRING_BILLING, fakeClock.nowUtc());
Cursor cursor2 = Cursor.create(CursorType.RDE_REPORT, "tld", fakeClock.nowUtc());
Cursor cursor3 = Cursor.create(CursorType.RDE_REPORT, "foo", fakeClock.nowUtc());
Cursor cursor4 = Cursor.create(CursorType.BRDA, "foo", fakeClock.nowUtc());
CursorDao.save(cursor);
CursorDao.save(cursor2);
CursorDao.save(cursor3);
CursorDao.save(cursor4);
List<Cursor> returnedCursors = CursorDao.loadAll();
assertThat(returnedCursors.size()).isEqualTo(4);
}
@Test
public void loadAll_worksSuccessfullyEmptyTable() {
List<Cursor> returnedCursors = CursorDao.loadAll();
assertThat(returnedCursors.size()).isEqualTo(0);
}
@Test
public void loadByType_worksSuccessfully() {
Cursor cursor = Cursor.createGlobal(CursorType.RECURRING_BILLING, fakeClock.nowUtc());
Cursor cursor2 = Cursor.create(CursorType.RDE_REPORT, "tld", fakeClock.nowUtc());
Cursor cursor3 = Cursor.create(CursorType.RDE_REPORT, "foo", fakeClock.nowUtc());
Cursor cursor4 = Cursor.create(CursorType.BRDA, "foo", fakeClock.nowUtc());
CursorDao.save(cursor);
CursorDao.save(cursor2);
CursorDao.save(cursor3);
CursorDao.save(cursor4);
List<Cursor> returnedCursors = CursorDao.loadByType(CursorType.RDE_REPORT);
assertThat(returnedCursors.size()).isEqualTo(2);
}
@Test
public void loadByType_worksSuccessfullyNoneOfType() {
List<Cursor> returnedCursors = CursorDao.loadByType(CursorType.RDE_REPORT);
assertThat(returnedCursors.size()).isEqualTo(0);
}
}
@@ -19,7 +19,7 @@ import static com.google.common.truth.Truth.assertWithMessage;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Sets;
import com.google.common.collect.Sets.SetView;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import io.github.classgraph.ClassGraph;
import io.github.classgraph.ScanResult;
import java.lang.reflect.Field;
@@ -35,7 +35,7 @@ import org.junit.runners.Suite.SuiteClasses;
* system property as a comma-separated string.
*
* <p>A test is deemed dependent on the SQL schema iff it has a field with type {@link
* JpaTransactionManagerRule}.
* JpaIntegrationTestRule}.
*/
// TODO(weiminyu): consider generating a TestSuite class instead.
@RunWith(JUnit4.class)
@@ -71,8 +71,13 @@ public class SqlIntegrationMembershipTest {
}
private static boolean isSqlDependent(Class<?> testClass) {
return Stream.of(testClass.getDeclaredFields())
.map(Field::getType)
.anyMatch(JpaTransactionManagerRule.class::equals);
for (Class<?> clazz = testClass; clazz != null; clazz = clazz.getSuperclass()) {
if (Stream.of(clazz.getDeclaredFields())
.map(Field::getType)
.anyMatch(JpaIntegrationTestRule.class::equals)) {
return true;
}
}
return false;
}
}
@@ -15,17 +15,16 @@
package google.registry.schema.integration;
import google.registry.model.registry.RegistryLockDaoTest;
import google.registry.model.tmch.ClaimsListDaoTest;
import google.registry.model.transaction.JpaTransactionManagerImplTest;
import google.registry.model.transaction.JpaTransactionManagerRuleTest;
import google.registry.persistence.BloomFilterConverterTest;
import google.registry.persistence.CreateAutoTimestampConverterTest;
import google.registry.persistence.CurrencyUnitConverterTest;
import google.registry.persistence.DateTimeConverterTest;
import google.registry.persistence.JodaMoneyConverterTest;
import google.registry.persistence.UpdateAutoTimestampConverterTest;
import google.registry.persistence.ZonedDateTimeConverterTest;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.schema.cursor.CursorDaoTest;
import google.registry.schema.tld.PremiumListDaoTest;
import google.registry.schema.tld.PremiumListUtilsTest;
import google.registry.schema.tld.ReservedListDaoTest;
import google.registry.schema.tmch.ClaimsListDaoTest;
import google.registry.tools.CreateReservedListCommandTest;
import google.registry.tools.UpdateReservedListCommandTest;
import google.registry.tools.server.CreatePremiumListActionTest;
import google.registry.tools.server.UpdatePremiumListActionTest;
import google.registry.ui.server.registrar.RegistryLockGetActionTest;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
@@ -35,26 +34,23 @@ import org.junit.runners.Suite.SuiteClasses;
* Groups all tests that may depends on Cloud SQL schema. They will be run for server-schema
* compatibility check.
*
* <p>Schema dependency is approximated by the use of {@link
* google.registry.model.transaction.JpaTransactionManagerRule}.
* <p>Schema dependency is approximated by the use of {@link JpaIntegrationTestRule}.
*
* @see SqlIntegrationMembershipTest
*/
// TODO(weiminyu): refactor JpaTransactionManagerRule to eliminate false positives.
@RunWith(Suite.class)
@SuiteClasses({
BloomFilterConverterTest.class,
ClaimsListDaoTest.class,
CreateAutoTimestampConverterTest.class,
CurrencyUnitConverterTest.class,
DateTimeConverterTest.class,
JodaMoneyConverterTest.class,
JpaTransactionManagerImplTest.class,
JpaTransactionManagerRuleTest.class,
CreateReservedListCommandTest.class,
CursorDaoTest.class,
CreatePremiumListActionTest.class,
PremiumListDaoTest.class,
PremiumListUtilsTest.class,
RegistryLockDaoTest.class,
RegistryLockGetActionTest.class,
UpdateAutoTimestampConverterTest.class,
ZonedDateTimeConverterTest.class
ReservedListDaoTest.class,
UpdatePremiumListActionTest.class,
UpdateReservedListCommandTest.class
})
public class SqlIntegrationTestSuite {}
@@ -15,13 +15,27 @@
package google.registry.schema.tld;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.model.common.EntityGroupRoot.getCrossTldKey;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import static google.registry.testing.DatastoreHelper.createTld;
import static google.registry.testing.DatastoreHelper.newRegistry;
import static google.registry.testing.DatastoreHelper.persistResource;
import static google.registry.testing.JUnitBackports.assertThrows;
import static org.joda.money.CurrencyUnit.JPY;
import static org.joda.money.CurrencyUnit.USD;
import com.google.common.collect.ImmutableMap;
import google.registry.model.transaction.JpaTransactionManagerRule;
import com.googlecode.objectify.Key;
import google.registry.model.registry.Registry;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.testing.AppEngineRule;
import java.math.BigDecimal;
import java.util.Optional;
import org.joda.money.CurrencyUnit;
import org.joda.money.Money;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
@@ -32,8 +46,10 @@ import org.junit.runners.JUnit4;
public class PremiumListDaoTest {
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder().build();
public final JpaIntegrationTestRule jpaRule =
new JpaTestRules.Builder().buildIntegrationTestRule();
@Rule public final AppEngineRule appEngine = AppEngineRule.builder().withDatastore().build();
private static final ImmutableMap<String, BigDecimal> TEST_PRICES =
ImmutableMap.of(
@@ -46,40 +62,144 @@ public class PremiumListDaoTest {
@Test
public void saveNew_worksSuccessfully() {
PremiumList premiumList = PremiumList.create("testname", CurrencyUnit.USD, TEST_PRICES);
PremiumList premiumList = PremiumList.create("testname", USD, TEST_PRICES);
PremiumListDao.saveNew(premiumList);
jpaTm()
.transact(
() -> {
PremiumList persistedList =
jpaTm()
.getEntityManager()
.createQuery(
"SELECT pl FROM PremiumList pl WHERE pl.name = :name", PremiumList.class)
.setParameter("name", "testname")
.getSingleResult();
Optional<PremiumList> persistedListOpt = PremiumListDao.getLatestRevision("testname");
assertThat(persistedListOpt).isPresent();
PremiumList persistedList = persistedListOpt.get();
assertThat(persistedList.getLabelsToPrices()).containsExactlyEntriesIn(TEST_PRICES);
assertThat(persistedList.getCreationTimestamp())
.isEqualTo(jpaTmRule.getTxnClock().nowUtc());
.isEqualTo(jpaRule.getTxnClock().nowUtc());
});
}
@Test
public void update_worksSuccessfully() {
PremiumListDao.saveNew(PremiumList.create("testname", CurrencyUnit.USD, TEST_PRICES));
Optional<PremiumList> persistedList = PremiumListDao.getLatestRevision("testname");
assertThat(persistedList).isPresent();
long firstRevisionId = persistedList.get().getRevisionId();
PremiumListDao.update(
PremiumList.create(
"testname",
CurrencyUnit.USD,
ImmutableMap.of(
"update",
BigDecimal.valueOf(55343.12),
"new",
BigDecimal.valueOf(0.01),
"silver",
BigDecimal.valueOf(30.03))));
jpaTm()
.transact(
() -> {
Optional<PremiumList> updatedListOpt = PremiumListDao.getLatestRevision("testname");
assertThat(updatedListOpt).isPresent();
PremiumList updatedList = updatedListOpt.get();
assertThat(updatedList.getLabelsToPrices())
.containsExactlyEntriesIn(
ImmutableMap.of(
"update",
BigDecimal.valueOf(55343.12),
"new",
BigDecimal.valueOf(0.01),
"silver",
BigDecimal.valueOf(30.03)));
assertThat(updatedList.getCreationTimestamp())
.isEqualTo(jpaRule.getTxnClock().nowUtc());
assertThat(updatedList.getRevisionId()).isGreaterThan(firstRevisionId);
assertThat(updatedList.getCreationTimestamp())
.isEqualTo(jpaRule.getTxnClock().nowUtc());
});
}
@Test
public void saveNew_throwsWhenPremiumListAlreadyExists() {
PremiumListDao.saveNew(PremiumList.create("testlist", CurrencyUnit.USD, TEST_PRICES));
PremiumListDao.saveNew(PremiumList.create("testlist", USD, TEST_PRICES));
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() ->
PremiumListDao.saveNew(
PremiumList.create("testlist", CurrencyUnit.USD, TEST_PRICES)));
assertThat(thrown).hasMessageThat().contains("A premium list of this name already exists");
() -> PremiumListDao.saveNew(PremiumList.create("testlist", USD, TEST_PRICES)));
assertThat(thrown).hasMessageThat().isEqualTo("Premium list 'testlist' already exists");
}
// TODO(b/147246613): Un-ignore this.
@Test
@Ignore
public void update_throwsWhenListDoesntExist() {
IllegalArgumentException thrown =
assertThrows(
IllegalArgumentException.class,
() -> PremiumListDao.update(PremiumList.create("testlist", USD, TEST_PRICES)));
assertThat(thrown)
.hasMessageThat()
.isEqualTo("Can't update non-existent premium list 'testlist'");
}
@Test
public void checkExists_worksSuccessfully() {
assertThat(PremiumListDao.checkExists("testlist")).isFalse();
PremiumListDao.saveNew(PremiumList.create("testlist", CurrencyUnit.USD, TEST_PRICES));
PremiumListDao.saveNew(PremiumList.create("testlist", USD, TEST_PRICES));
assertThat(PremiumListDao.checkExists("testlist")).isTrue();
}
@Test
public void getLatestRevision_returnsEmptyForNonexistentList() {
assertThat(PremiumListDao.getLatestRevision("nonexistentlist")).isEmpty();
}
@Test
public void getLatestRevision_worksSuccessfully() {
PremiumListDao.saveNew(
PremiumList.create("list1", JPY, ImmutableMap.of("wrong", BigDecimal.valueOf(1000.50))));
PremiumListDao.update(PremiumList.create("list1", JPY, TEST_PRICES));
jpaTm()
.transact(
() -> {
Optional<PremiumList> persistedList = PremiumListDao.getLatestRevision("list1");
assertThat(persistedList).isPresent();
assertThat(persistedList.get().getName()).isEqualTo("list1");
assertThat(persistedList.get().getCurrency()).isEqualTo(JPY);
assertThat(persistedList.get().getLabelsToPrices())
.containsExactlyEntriesIn(TEST_PRICES);
});
}
@Test
public void getPremiumPrice_returnsNoneWhenNoPremiumListConfigured() {
persistResource(newRegistry("foobar", "FOOBAR").asBuilder().setPremiumList(null).build());
assertThat(PremiumListDao.getPremiumPrice("rich", Registry.get("foobar"))).isEmpty();
}
@Test
public void getPremiumPrice_worksSuccessfully() {
persistResource(
newRegistry("foobar", "FOOBAR")
.asBuilder()
.setPremiumListKey(
Key.create(
getCrossTldKey(),
google.registry.model.registry.label.PremiumList.class,
"premlist"))
.build());
PremiumListDao.saveNew(PremiumList.create("premlist", USD, TEST_PRICES));
assertThat(PremiumListDao.getPremiumPrice("silver", Registry.get("foobar")))
.hasValue(Money.of(USD, 10.23));
assertThat(PremiumListDao.getPremiumPrice("gold", Registry.get("foobar")))
.hasValue(Money.of(USD, 1305.47));
assertThat(PremiumListDao.getPremiumPrice("zirconium", Registry.get("foobar"))).isEmpty();
}
@Test
public void testGetPremiumPrice_throwsWhenPremiumListCantBeLoaded() {
createTld("tld");
IllegalStateException thrown =
assertThrows(
IllegalStateException.class,
() -> PremiumListDao.getPremiumPrice("foobar", Registry.get("tld")));
assertThat(thrown).hasMessageThat().isEqualTo("Could not load premium list 'tld'");
}
}
@@ -1,4 +1,4 @@
// Copyright 2017 The Nomulus Authors. All Rights Reserved.
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -12,43 +12,35 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.tools.server;
package google.registry.schema.tld;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth8.assertThat;
import static google.registry.schema.tld.PremiumListUtils.parseToPremiumList;
import static google.registry.testing.JUnitBackports.assertThrows;
import google.registry.schema.tld.PremiumList;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.testing.AppEngineRule;
import google.registry.testing.FakeJsonResponse;
import java.math.BigDecimal;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/** Unit tests for {@link CreateOrUpdatePremiumListAction}. */
/** Unit tests for {@link PremiumListUtils}. */
@RunWith(JUnit4.class)
public class CreateOrUpdatePremiumListActionTest {
public class PremiumListUtilsTest {
@Rule public final AppEngineRule appEngine = AppEngineRule.builder().withDatastore().build();
private CreatePremiumListAction action;
private FakeJsonResponse response;
@Before
public void init() {
action = new CreatePremiumListAction();
response = new FakeJsonResponse();
action.response = response;
action.name = "testlist";
}
@Rule
public final JpaIntegrationTestRule jpaRule =
new JpaTestRules.Builder().buildIntegrationTestRule();
@Test
public void parseInputToPremiumList_works() {
action.inputData = "foo,USD 99.50\n" + "bar,USD 30\n" + "baz,USD 10\n";
PremiumList premiumList = action.parseInputToPremiumList();
PremiumList premiumList =
parseToPremiumList("testlist", "foo,USD 99.50\n" + "bar,USD 30\n" + "baz,USD 10\n");
assertThat(premiumList.getName()).isEqualTo("testlist");
assertThat(premiumList.getLabelsToPrices())
.containsExactly("foo", twoDigits(99.50), "bar", twoDigits(30), "baz", twoDigits(10));
@@ -56,9 +48,12 @@ public class CreateOrUpdatePremiumListActionTest {
@Test
public void parseInputToPremiumList_throwsOnInconsistentCurrencies() {
action.inputData = "foo,USD 99.50\n" + "bar,USD 30\n" + "baz,JPY 990\n";
IllegalArgumentException thrown =
assertThrows(IllegalArgumentException.class, () -> action.parseInputToPremiumList());
assertThrows(
IllegalArgumentException.class,
() ->
parseToPremiumList(
"testlist", "foo,USD 99.50\n" + "bar,USD 30\n" + "baz,JPY 990\n"));
assertThat(thrown)
.hasMessageThat()
.isEqualTo("The Cloud SQL schema requires exactly one currency, but got: [JPY, USD]");
@@ -0,0 +1,70 @@
// Copyright 2019 The Nomulus Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.schema.tld;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.transaction.TransactionManagerFactory.jpaTm;
import com.google.common.collect.ImmutableMap;
import google.registry.model.registry.label.ReservationType;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.schema.tld.ReservedList.ReservedEntry;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/** Unit tests for {@link ReservedListDao}. */
@RunWith(JUnit4.class)
public class ReservedListDaoTest {
@Rule
public final JpaIntegrationTestRule jpaRule =
new JpaTestRules.Builder().buildIntegrationTestRule();
private static final ImmutableMap<String, ReservedEntry> TEST_RESERVATIONS =
ImmutableMap.of(
"food",
ReservedEntry.create(ReservationType.RESERVED_FOR_SPECIFIC_USE, null),
"music",
ReservedEntry.create(ReservationType.FULLY_BLOCKED, "fully blocked"));
@Test
public void save_worksSuccessfully() {
ReservedList reservedList = ReservedList.create("testname", false, TEST_RESERVATIONS);
ReservedListDao.save(reservedList);
jpaTm()
.transact(
() -> {
ReservedList persistedList =
jpaTm()
.getEntityManager()
.createQuery("FROM ReservedList WHERE name = :name", ReservedList.class)
.setParameter("name", "testname")
.getSingleResult();
assertThat(persistedList.getLabelsToReservations())
.containsExactlyEntriesIn(TEST_RESERVATIONS);
assertThat(persistedList.getCreationTimestamp())
.isEqualTo(jpaRule.getTxnClock().nowUtc());
});
}
@Test
public void checkExists_worksSuccessfully() {
assertThat(ReservedListDao.checkExists("testlist")).isFalse();
ReservedListDao.save(ReservedList.create("testlist", false, TEST_RESERVATIONS));
assertThat(ReservedListDao.checkExists("testlist")).isTrue();
}
}
@@ -15,6 +15,8 @@
package google.registry.schema.tld;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.model.registry.label.ReservationType.FULLY_BLOCKED;
import static google.registry.testing.JUnitBackports.assertThrows;
import com.google.common.collect.ImmutableMap;
import google.registry.model.registry.label.ReservationType;
@@ -50,4 +52,34 @@ public class ReservedListTest {
ReservedEntry.create(
ReservationType.RESERVED_FOR_ANCHOR_TENANT, "reserved for anchor tenant"));
}
@Test
public void create_throwsExceptionWhenLabelIsNotLowercase() {
Exception e =
assertThrows(
IllegalArgumentException.class,
() ->
ReservedList.create(
"UPPER.tld",
true,
ImmutableMap.of("UPPER", ReservedEntry.create(FULLY_BLOCKED, ""))));
assertThat(e)
.hasMessageThat()
.contains("Label(s) [UPPER] must be in puny-coded, lower-case form");
}
@Test
public void create_labelMustBePunyCoded() {
Exception e =
assertThrows(
IllegalArgumentException.class,
() ->
ReservedList.create(
"lower.みんな",
true,
ImmutableMap.of("みんな", ReservedEntry.create(FULLY_BLOCKED, ""))));
assertThat(e)
.hasMessageThat()
.contains("Label(s) [みんな] must be in puny-coded, lower-case form");
}
}
@@ -12,16 +12,14 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package google.registry.model.tmch;
package google.registry.schema.tmch;
import static com.google.common.truth.Truth.assertThat;
import static google.registry.testing.JUnitBackports.assertThrows;
import com.google.common.collect.ImmutableMap;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.schema.tmch.ClaimsList;
import google.registry.model.transaction.JpaTestRules;
import google.registry.model.transaction.JpaTestRules.JpaIntegrationTestRule;
import google.registry.testing.FakeClock;
import javax.persistence.NoResultException;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
@@ -34,18 +32,17 @@ public class ClaimsListDaoTest {
private FakeClock fakeClock = new FakeClock();
@Rule
public final JpaTransactionManagerRule jpaTmRule =
new JpaTransactionManagerRule.Builder().build();
public final JpaIntegrationTestRule jpaRule =
new JpaTestRules.Builder().buildIntegrationTestRule();
@Test
public void trySave_insertsClaimsListSuccessfully() {
ClaimsList claimsList =
ClaimsList.create(fakeClock.nowUtc(), ImmutableMap.of("label1", "key1", "label2", "key2"));
ClaimsListDao.trySave(claimsList);
ClaimsList insertedClaimsList = ClaimsListDao.getCurrent();
ClaimsList insertedClaimsList = ClaimsListDao.getLatestRevision().get();
assertClaimsListEquals(claimsList, insertedClaimsList);
assertThat(insertedClaimsList.getCreationTimestamp())
.isEqualTo(jpaTmRule.getTxnClock().nowUtc());
assertThat(insertedClaimsList.getCreationTimestamp()).isEqualTo(jpaRule.getTxnClock().nowUtc());
}
@Test
@@ -53,7 +50,7 @@ public class ClaimsListDaoTest {
ClaimsList claimsList =
ClaimsList.create(fakeClock.nowUtc(), ImmutableMap.of("label1", "key1", "label2", "key2"));
ClaimsListDao.trySave(claimsList);
ClaimsList insertedClaimsList = ClaimsListDao.getCurrent();
ClaimsList insertedClaimsList = ClaimsListDao.getLatestRevision().get();
assertClaimsListEquals(claimsList, insertedClaimsList);
// Save ClaimsList with existing revisionId should fail because revisionId is the primary key.
ClaimsListDao.trySave(insertedClaimsList);
@@ -63,14 +60,14 @@ public class ClaimsListDaoTest {
public void trySave_claimsListWithNoEntries() {
ClaimsList claimsList = ClaimsList.create(fakeClock.nowUtc(), ImmutableMap.of());
ClaimsListDao.trySave(claimsList);
ClaimsList insertedClaimsList = ClaimsListDao.getCurrent();
ClaimsList insertedClaimsList = ClaimsListDao.getLatestRevision().get();
assertClaimsListEquals(claimsList, insertedClaimsList);
assertThat(insertedClaimsList.getLabelsToKeys()).isEmpty();
}
@Test
public void getCurrent_throwsNoResultExceptionIfTableIsEmpty() {
assertThrows(NoResultException.class, ClaimsListDao::getCurrent);
public void getCurrent_returnsEmptyListIfTableIsEmpty() {
assertThat(ClaimsListDao.getLatestRevision().isPresent()).isFalse();
}
@Test
@@ -81,7 +78,7 @@ public class ClaimsListDaoTest {
ClaimsList.create(fakeClock.nowUtc(), ImmutableMap.of("label3", "key3", "label4", "key4"));
ClaimsListDao.trySave(oldClaimsList);
ClaimsListDao.trySave(newClaimsList);
assertClaimsListEquals(newClaimsList, ClaimsListDao.getCurrent());
assertClaimsListEquals(newClaimsList, ClaimsListDao.getLatestRevision().get());
}
private void assertClaimsListEquals(ClaimsList left, ClaimsList right) {
@@ -19,7 +19,7 @@ import com.beust.jcommander.Parameter;
import com.beust.jcommander.Parameters;
import com.google.common.collect.ImmutableList;
import com.google.common.net.HostAndPort;
import google.registry.model.transaction.JpaTransactionManagerRule;
import google.registry.model.transaction.JpaTestRules;
import google.registry.testing.AppEngineRule;
import google.registry.testing.UserInfo;
import google.registry.tools.params.HostAndPortParameter;
@@ -170,8 +170,8 @@ public final class RegistryTestServerMain {
.apply(runner, Description.EMPTY);
System.out.printf("%sLoading SQL fixtures and AppEngineRule...%s\n", BLUE, RESET);
new JpaTransactionManagerRule.Builder()
.build()
new JpaTestRules.Builder()
.buildIntegrationTestRule()
.apply(withAppEngine, Description.EMPTY)
.evaluate();
}

Some files were not shown because too many files have changed in this diff Show More